Chat on WhatsApp

Can I A/B test Hyvä against Luma before full cutover?

Yes, and for stores above $10M GMV I usually recommend it. Two patterns:

  1. Sub-domain split test. Hyvä on hyva.yourstore.com, Luma on www.yourstore.com, send 10-25% of traffic to the Hyvä sub-domain via a paid traffic source (Meta / Google Ads) for 2-4 weeks. Measure conversion, AOV, bounce, INP, error rate. If Hyvä outperforms (it usually does, +8-25%), do the full cutover. Cost: ~$2k, $5k of paid traffic for a clean signal.
  2. Geo split. Cut over one country / region first (typically the smallest by GMV), watch 4-6 weeks, then cut the others. Lower-risk pattern for multi-region brands. No extra licence cost.

Things to know:

  • A/B testing Luma vs Hyvä on the same URL via Optimizely / VWO doesn’t work cleanly, the two themes have completely different DOM + CSS so the A/B tool flickers visibly.
  • You need analytics consistent across both: same GA4 / Matomo setup, same conversion definitions, same UTM handling. I set this up as part of step 4 (Build).
  • The split-test “Hyvä loses” outcome is rare but real, usually means a specific Magento extension didn’t port cleanly and broke a critical flow. Diagnose, fix, re-test.
Was this helpful?