Skip to content

How to benchmark Magento 2 performance: a repeatable method

A Magento benchmark is only useful if someone else can repeat it and get a comparable result. This page sets out the method I use: which data to trust, which thresholds apply, how to run frontend and server tests, and a reporting template you can fill in for your own store.

How to benchmark Magento 2 performance: a repeatable method

What a benchmark is for

An earlier version of this page published speed ranges for several Magento frontends. Those ranges were not tied to a test anyone could repeat, so I removed them, along with the score bands in the FAQ. A number from one store says very little about another store with different extensions, images, tags and hosting.

A good benchmark answers a narrower question: did this change make these pages faster for these users, under these conditions? It gives you a baseline before work starts, a way to check each change, and evidence you can show to someone else. The method below is built for that. If you are choosing a frontend, read how the Magento frontend options differ first. If you are comparing two specific themes, the Hyvä vs Breeze comparison covers the products themselves.

Field data and lab data

Field data (CrUX)

The Chrome UX Report collects real user experiences. PageSpeed Insights shows them over the previous 28-day collection period, at page or origin level. A page or origin must have enough visitors to be included, and Google does not publish the exact threshold. Chrome on iOS, Android WebView and other Chromium browsers do not contribute data.

Lab data (Lighthouse)

Lighthouse loads a page in a simulated environment on a single device with fixed network conditions, so its values can differ from field data. In Lighthouse 10 the Performance score weights Total Blocking Time at 30%, LCP at 25%, CLS at 25%, FCP at 10% and Speed Index at 10%.

Field data tells you what users experienced. Lab data tells you why, and lets you test a fix before it ships. Because CrUX is a rolling 28-day window, a change you deploy today takes time to show up fully in field data. I use lab runs to confirm a change works and field data to confirm users felt it.

Google's documentation says much of the variation in a Lighthouse score is not caused by Lighthouse itself but by changing conditions: A/B tests or ads, network routing, different test devices and browser extensions. That is why the method below fixes as many conditions as possible.

The thresholds

These are the thresholds PageSpeed Insights uses, in line with the Web Vitals guidance. Google recommends judging them at the 75th percentile of page loads, segmented by mobile and desktop. The thresholds are the same for both device types.

MetricCore Web Vital?GoodNeeds improvementPoor
LCPYesup to 2500 msover 2500 ms, up to 4000 msover 4000 ms
INPYes, since 2024up to 200 msover 200 ms, up to 500 msover 500 ms
CLSYesup to 0.1over 0.1, up to 0.25over 0.25
FCPNoup to 1800 msover 1800 ms, up to 3000 msover 3000 ms
TTFBNo (experimental in PageSpeed Insights)up to 800 msover 800 ms, up to 1800 msover 1800 ms

A page passes the Core Web Vitals assessment only when LCP, INP and CLS all meet the good threshold at the 75th percentile. INP is not one of the five metrics in the Lighthouse 10 Performance score, so I treat Total Blocking Time in lab runs as a pointer and confirm INP in field data.

Frontend test plan

This is my method, not a measured result. Adjust it to your store, but write down every choice so the next run uses the same one.

  1. Pick the page types. Home, one category listing, one product page, search results and the cart. Use the same URLs every time, and avoid pages whose content changes daily.
  2. Fix the tool and profile. Record the Lighthouse or PageSpeed Insights version, the device profile (mobile or desktop), the throttling method and the test location. Do not change these between runs you plan to compare.
  3. Control the cache state. Warm the full page cache by requesting each URL first, then test. If you also test a cold cache, label those runs separately.
  4. Run several times. Run each URL at least three times in the same conditions and report the median, not the best run.
  5. List the third-party tags. Write down every tag manager container, analytics, chat, review and ad script that loaded. A new tag can move the result more than a code change.
  6. Record date and commit. Note the date, time, deployed commit and Magento version for each set of runs.
  7. Pull field data alongside. Copy the CrUX values for the same URLs, or the origin if the page has no data, with the date you read them.

Server test plan

Frontend scores hide server problems behind a single TTFB figure, so I test the server on its own as well. Run these tests on a staging copy that matches production, never on the live store during trading.

  1. Use known data. Adobe's Performance Toolkit generates test data with bin/magento setup:perf:generate-fixtures and a profile (small, medium, large or extra large) from setup/performance-toolkit/profiles. Adobe's guide says to reindex all indexers after the command runs. For a store with real data, use an anonymised copy instead.
  2. Separate cache hits from misses. Measure TTFB for each URL with the full page cache warm and with it missed. Magento's generated Varnish configuration sets an X-Magento-Cache-Debug header of HIT, MISS or UNCACHEABLE. The built-in full page cache only adds that header in developer mode, so on a production-mode copy compare repeat response times instead.
  3. Record the indexer and cron state. Indexers that are invalid or a cron that is not running change server timings. Record the output of the read-only checks below with every run.
  4. Load test separately. Response time for one request is not the same as response time under load. If you run a load test, record the tool, the number of simulated users, the duration and the scenario, and keep it apart from the single-request numbers.
bin/magento deploy:mode:show
bin/magento cache:status
bin/magento indexer:status
bin/magento indexer:show-mode
curl -sI https://staging.example.test/some-category.html | grep -i x-magento-cache-debug

Reporting template

Copy this table for each set of runs and fill in every row. A result with empty rows is not comparable with another one.

FieldWhat to recordYour value
Date and timeWhen the runs started, with time zoneFill in
Commit and versionsDeployed commit, Magento version, PHP version, theme and versionFill in
EnvironmentStaging or production, hosting plan, CDN, Varnish or built-in cacheFill in
ToolLighthouse or PageSpeed Insights version, device profile, throttling, test locationFill in
URLsThe exact URLs tested, one per page typeFill in
Cache stateWarm or cold, and the cache debug header result where availableFill in
RunsNumber of runs per URL and the median of each metricFill in
Lab metricsLCP, TBT, CLS, FCP, Speed Index and Performance score (median)Fill in
Field metricsCrUX LCP, INP, CLS at p75, page or origin level, and the date readFill in
ServerTTFB on cache hit and miss, indexer status, cron status, deploy modeFill in
Third-party tagsEvery external script that loadedFill in
Changes since last runWhat was deployed or configured between runsFill in

Publishing results

If you share a benchmark with a client, a team or the public, publish only runs you measured, with the filled-in template next to them. Do not round a range out of a handful of stores, and do not compare your numbers with a figure from another site unless you can repeat their setup. I hold my own work to the same rule, which is why this page has no table of results.

Measured or estimated? Label every figure in a report as one or the other. A measured figure has a date, a URL and a run behind it. Anything else is an estimate or a target and should say so.

Next steps

Once you have a baseline, work through the Magento performance checklist for server and configuration checks, and read why Magento is slow for the usual causes. Re-run the same tests after each change so you can see which one helped.

Frequently asked questions

Should I trust lab or field data?

Use both for different jobs. Field data from CrUX shows what real Chrome users experienced over the last 28 days. Lab data from Lighthouse runs on a single simulated device and network, which makes it good for testing a change before release.

Why does my Lighthouse score change between runs?

Google's documentation says much of the variation comes from changing conditions rather than from Lighthouse: A/B tests or ads, network routing, different devices and browser extensions. Fix as many conditions as you can, run each URL several times and compare medians.

How often should I re-test?

I re-run the lab tests after every change that touches the frontend, the cache or the server, and before and after each release. Because CrUX covers a rolling 28-day window, I check field data again once a full window has passed since the change.

Is TTFB a Core Web Vital?

No. The Core Web Vitals are LCP, INP and CLS. PageSpeed Insights reports TTFB as an experimental metric with a good threshold of 800 ms, and it is a useful server diagnostic.

Can anyone guarantee a Lighthouse score?

No. Scores vary between runs and depend on devices, networks, ads and third-party scripts that no developer fully controls. I agree measurable goals and report measured before and after results, but I do not promise a score.

Want a baseline for your own store, measured with this method? See my Magento performance optimization service, or send a short brief for a fixed written quote.

Benchmark my store
Kishan Savaliya Online 10 AM to 7 PM IST Chat on WhatsApp