Migrating product-relations

VirtueMart offers the functionality of product-relations, but this relation was not yet migrated to Magento. With Vm2Mage 0.10.3 this has been now introduced by updating the extensions in both Joomla! as Magento. Some extra tuning was done to make sure mapping a child-product (so a related product) to the actual product was done by SKU and not ID - because Magento IDs don't match VirtueMart IDs.

A minor note needs to be given: If a product A is related to a product B, while that product B is not yet migrated, the product relation will fail. To migrate product relations properly, the migration needs to be run several times - the Vm2Mage architecture was actually designed to run the migration over-and-over again, so this doesn't pose a problem.

Fixes for migrating categories

The previous point of products relying on other products that might not yet have been migrated, actually also applies to categories: A category depends on a parent category, and if that parent category doesn't exist yet, that relation fails. With previous versions of Vm2Mage, the trick was just to run the migration at least twice. With 0.10.3, categories are sorted in a smarter way, so that parent-categories are migrated first. In all our test-cases, this resulted in all hierarchy-relations properly set right away from the start.

Another major improvement was made with multisite setups, or repeated migrations from different databases. When migrating a product, Vm2Mage is able to reuse the product-SKU to identiy the product in both VirtueMart as Magento. The same applies for a customer, by using its email address. Unforunately a category doesn't have such an unique property. So Vm2Mage keeps track of the VirtueMart ID and its corresponding Magento ID by using an extra mapping in a custom Vm2Mage database-table. This works fine.

However, in some cases, a VirtueMart category with ID 32 might be migrated from one Joomla! site, while afterwards a second VirtueMart category with the same ID 32 might be migrated from another Joomla! site to the same Magento instance. Because Magento is multisite-minded, this is a completely legal thing to do. However, the Vm2Mage mapping would cause the second category to use the mapping of the first. Vm2Mage 0.10.3 solves this by adding a hash based on the VirtueMart database.

Some work on migration speed

We did some tests ourselves to see how a migration could go as fast as possible. Because Vm2Mage is fully based on the Magento coding API, which again relies on a complex database-schema, for a single product to be inserted into Magento numerous queries might be needed - resulting in a longer waiting time. But there are tricks that improve this. The most important trick is to temporarily disable indexing, because otherwise every product-update would result in a partial reindexing which stresses the database too much. With indexing disabled, in some cases, the Vm2Mage migration takes a few hours less to be completed - which is quite impressive.

All tips and tricks are gathered in a performance document that helps you configure Magento and the hosting environment. One configuration we tried was to set the batch-size (so the number of products or customers to be migrated in a single AJAX-call) to extremely large (1000 items). We first bumped into an XML-RPC timeout, which is fixed in Vm2Mage 0.10.3 by setting the timeout to 3600 seconds. When the PHP memory_limit is then increased to a temporary limit of 2Gb (yes, that's a lot), migration was indeed improved because Magento was only initialized once. Of course, this assumes you have the hardware to make this tuning possible.