The why of integration tests

Running Magento 2 integration tests for your own extension is very valuable. Instead of only looking for unit tests or testing the entire output using functional tests or end-to-end tests, integration tests allow you to look at the parts where your extension is hooking into the rest of Magento.

Instead of seeing the HTML output, you would be looking at the Response object to see if it holds the right headers. Instead of seeing empty output due to unknown database issues, you can check to see if your repository is able to fetch data at all. Even better, all of this takes place in a controlled environment, with an empty database and data fixtures.

Setting up integration tests

The setup of integration tests is kind of cumbersome: You will need an empty database, you will need a clean Magento environment (where bad extensions that will not work with integration tests are removed) and you will need to follow a procedure to set things up.

This blog is not about this procedure of setting up integration tests. Instead, it hopefully helps you troubleshoot things when you bump into issues.

Database is simply not there yet

While starting the integration tests, your install-config-mysql.php file will be pointing to a database. If that database is not there, you can expect the setup:upgrade command to fail with message like Connection refused or No tables found or something similar.

What I usually do is connect to the same MySQL instance using a command like mysqladmin ping. In a CI/CD environment, such a command is definitely handy, for instance when both your tests and the database are run in Docker containers, but you want to make sure that the MySQL container is up first.

Database is empty while TESTS_CLEANUP is disabled

It might also be that the database is there but is empty, while the TESTS_CLEANUP flag in your phpunit.xml file is disabled. When the flag is enabled, Magento will run a setup on that empty database, but when the flag is disabled, it assumes that the setup has already been run.

I find this pretty annoying, I'm still hoping to provide some fix so that the flag can be disabled (for improved performance) but that the setup is simply run anyway when the database requires so.

SQLSTATE[42S02]: Base table or view not found

The most common issue that you might encounter is actually not due to your database setup, even though it tells you something is wrong with your database. For instance, a SQL error like SQLSTATE[42S02]: Base table or view not found: 1146 Table 'magento2.eav_entity_type' doesn't exist might be given.

A common reason why this is not working is because your module is not mentioning its dependencies properly. For instance, as soon as your module is active in either frontend or backend, chances are that your module needs to declare Magento_Store as a dependency in your etc/module.xml file. Similarly, if you're using dependencies from the catalog module, you should declare Magento_Catalog as a dependency.

Check your dependencies with Yireo ExtensionChecker

To help you track dependencies, I have developed an open source module Yireo ExtensionChecker that helps you with this. You can simply run the command bin/magento yireo_extensionchecker:scan Foo_Bar to see if the dependencies of Foo_Bar match up with its composer.json and module.xml file.

Not using proxies in your CLI commands

Another pitfall is that your module might contain a couple of CLI commands, with yet other dependencies. However, if those commands contain yet other dependencies and those dependencies require a working database, then it might be that the bin/magento command stops working. And because the integration tests require a command like bin/magento setup:upgrade to be working, the tests fail early again.

As a rule of thumb, it is best to create a di.xml file that creates a type for each command class, so that the original constructor arguments are replaced with proxy objects. If you are picky, you can try to do this only for those dependencies that require the database (like a repository), leaving out the others. Also, make sure not to use these dependencies in your configure() method (which will break the bin/magento command again) but only within the execute() method.

tasklist.exe not found

Perhaps you have bumped into the tasklist.exe not found error as well, like I did. It is not an error that is run before your integration tests, but actually after. But in a CI/CD environment, it might still break the tests. The error is generated by a method Magento\TestFramework\Helper\Memory::getRealMemoryUsage() that tries to sum up the consumed memory and report it to the bottom of your tests output.

Unfortunately, the flags of ps differ per platform. And depending on the environment where you're running these tests under, it might break things or not. Even stranger, when the UNIX command ps fails, the PHP code simply tries again with a Windows command tasklist.exe which is definitely not present in my Linux-based Docker containers. I usually patch the method by returning 42 or by letting it die() when the exception occurs. A bit strange, but it helped me to get going in various CI environments.

Summing up

Hopefully I documented a couple of ways of troubleshooting your integration testing setup that are useful to you. Feel free to comment on additional issues below.