Faster Magento 2 Integration Tests - a summary

Relevant blogs I wrote

In the past, I blogged on running integration tests already. To summarize these blogs first:

• A Magento 2 integration testing helper
• My ElasticSearch Docker setup for Magento 2
• A faster Docker for ElasticSearch
• Why Magento 2 integration tests might fail early?
• Faster Magento 2 Integration Tests
• Boost MySQL speed of Magento 2 integration tests

Information discussed in these blogs - plus some more tips & tricks - are found below.

Configure the Integration Test Framework properly

Set the constant TESTS_CLEANUP to enabled at first to initialize the Magento database. Disable it again after the first run.

Set the constant TESTS_MEM_USAGE_LIMIT to something high like 4096M.

Remove the Yandex\Allure\PhpUnit\AllurePhpUnit listener.

Note that setting the constant TESTS_PARALLEL_RUN to 1 in the phpunit.xml file does not enable parallel processing. It just prevents a cleanup if parallel processing is enabled. To enable parallel runs, you will need a tool like brianium/paratest or Robo parallel execution, which both lead into a lot of custom code.

Optimizing Magento

Make sure only those modules that are relevant are enabled. You can do this on the global level (app/etc/config.php). But you can also pass a disable-modules flag via the dev/tests/integration/etc/install-config-mysql.php. Because this gets pretty cumbersome, I created the Yireo Integration Testing Helper. Often I find myself disabling all modules, enabling all Magento core modules, disabling GraphQL and MSI and then only enabling the module that needs to be tested. You can also use composer replacements for this.

Run MySQL in tmpfs. This is also mentioned below. But to configure Magento for this, I use one Docker for development and one Docker for testing (including the tmpfs trick). I use the OOP syntax of the Yireo Integration Testing Helper to point to the right database.

Use Redis in the installation flags. Again with the Yireo Integration Testing Helper this becomes easy: $installConfig->addRedis().

Mount the folder dev/tests/integration/tmp/ in tmpfs as well.

Instead of the core its Integration Testing Framework, you might also want to try ReachDigitals version: github.com/ho-nl/magento2-ReachDigital_TestFramework

Optimizing the stack

Make sure to have plenty of RAM in your machine (for the tmpfs tricks) and add an SSD. I've got 32Gb of RAM which is awesome, but because of the other applications you're going to run (PHPStorm, the stack, browser, etc) having 16Gb of RAM is no luxury.

Run the latest version of MySQL and in tmpfs. With Docker this becomes pretty easy.

Tune the MySQL database correctly, even though it has less effect because MySQL is run in tmpfs anyway. My settings (for the tmpfs instance specifically) are here:

[mysqld]
innodb_buffer_pool_size=512M
innodb_buffer_pool_instances=1
innodb_read_io_threads=8
innodb_write_io_threads=8
innodb_lock_wait_timeout=50
innodb_doublewrite=0
innodb_flush_log_at_trx_commit=0
innodb_flush_log_at_timeout=360
innodb_flush_method=nosync

I'm no real expert here, this is just what sense to me. Further tuning might be possible.

Run the latest version of PHP and enable OPCache (with timestamp checking enabled). Possibly tune the OPCache cache but I personally always keep it up to 512Mb just in case. Obviously tune the memory limit. In the past I also tuned realpath_cache_size but nowadays this value seems already pretty good (4M).

Run ElasticSearch its data folder in tmpfs.

Tune the Java Xms and Xmx flags of ElasticSearch properly. I'm using 256m for both which fits both for my tests and my development environment. I'm doing this via Docker and made sure there is enough RAM assigned to that Docker instance.

ElasticSearch clustering is disabled via the flag discovery.type=single-node.

Hope you like it.