What is Magento IndexNow?
Short answer
Magento IndexNow means connecting Magento to the IndexNow protocol, which Microsoft Bing and Yandex launched in 2021. Instead of waiting for a crawler to notice a change, the store tells the search engines which URLs changed. With my free mage2kishan/module-index-now extension, Magento sends the URL to api.indexnow.org when you save a product, category or CMS page (catalog_product_save_after, catalog_category_save_after, cms_page_save_after). Bing, Yandex, Seznam, Naver and Yep take part. Google does not.
- Generate the IndexNow API key 01
- Publish the key verification file at site root 02
- Configure trigger events and partner endpoints in admin 03
- URL change event fires, queue then POST the batch 04
- Bing and Yandex fetch, validate, and index 05
Most common mistake Pinging URLs that 404 or redirect
Five steps from key generation to live Bing pings
IndexNow is a short, public HTTP spec: a key, a key file that proves you own the domain, and one POST with a list of URLs. This is how that maps onto Magento, using my module as the example.
-
01
Generate the IndexNow API key
An IndexNow key is a string of 8 to 128 characters (letters, digits and dashes) that proves the pings come from the site owner. Bing’s IndexNow page will generate one for you, or any long hex string works. In my module you paste it into the
API Keyfield, config pathpanth_index_now/indexnow/api_key. It can be set per website or store view, so each brand or language store can have its own key. If you change the key, the verification file has to change with it, otherwise the engines reject your pings. -
02
Publish the key verification file at site root
The search engine needs to fetch a plain-text file on your domain that contains exactly the key. By default that is
https://yoursite.com/<key>.txt, but the protocol also lets you send akeyLocationpointing somewhere else. My module serves the key from a controller at/panth_indexnow/keyand sends that as thekeyLocation, so there is no file to upload topub/. After setting or changing the key I always runcurl -i https://yoursite.com/panth_indexnow/keyand check it returns the key with a 200. -
03
Configure trigger events and partner endpoints in admin
In admin open
Stores → Configuration → Panth Extensions → IndexNow. There are only two settings: the enable switch and the API key. The module is off by default, so a fresh install sends nothing until you switch it on and add a key. It posts toapi.indexnow.orgonly, because that shared endpoint passes the URLs on to every participating engine. You do not need to ping Bing and Yandex separately. -
04
URL change event fires, queue then POST the batch
When a product, category or CMS page is saved, an observer on
catalog_product_save_after,catalog_category_save_afterorcms_page_save_afterworks out the storefront URL and collects it. At the end of the request, after the admin has already got its response, the module sends one POST tohttps://api.indexnow.org/indexnowwith JSON{host, key, keyLocation, urlList: [...]}. URLs are split into batches of 10,000, the protocol limit. There is no queue table and no cron job, so saving a product in admin does not get slower. -
05
Bing and Yandex fetch, validate, and index
The endpoint answers
200(OK),202(received, key check still pending),403(key not valid),422(URLs do not belong to the host, or the key does not match) or429(too many requests). A 200 means the engine has the URLs. It does not promise a crawl or a ranking. When a submission fails, my module writes the HTTP status and response body tovar/log/system.log, and Bing Webmaster Tools shows the URLs it received under its IndexNow report.
Four scenarios where IndexNow is the obvious answer
IndexNow is free and quick to set up. These are the four cases where I turn it on for a store.
-
High-velocity content sites, blog, news, glossary
If you publish new URLs every week, such as blog posts, news or glossary pages like this one, the time before a page shows up in search matters. With IndexNow, Bing and the other engines hear about a new CMS page the moment it is saved, instead of finding it on their next crawl or sitemap read. Keep the XML sitemap as well. IndexNow tells engines about changes, the sitemap still lists everything.
-
E-commerce price/stock changes you want crawled fast
When a product comes back in stock, a price changes or a short sale starts, a normal re-crawl can arrive after the sale is over. A ping on
catalog_product_save_aftertells Bing straight away that the page changed, so the snippet has a better chance of showing the new price or stock. This helps most on stores that run short sales. Note that a price changed by an import or a scheduled update may not fire the admin save event. -
Site migrations and bulk URL changes
After a domain change, a new URL structure or a category tree rework, many canonical URLs change at once. Google still has to find them through the sitemap and redirects. For Bing and the other IndexNow engines you can submit the new URLs directly, up to 10,000 per request. My module only sends URLs on save, so for a full migration I post the list from a one-off script, or re-save the affected categories and pages.
-
Brand-mention indexing for new product / service pages
When a new product page ships or a new service landing page goes live tied to a PR push, paid-media campaign, or social burst, the URL needs to be indexable the moment the campaign starts, not 5 days later when organic discovery catches up. Ping IndexNow as part of the launch checklist (alongside GSC URL inspection for Google). Bing is increasingly important for AI-powered search (Bing Chat / Copilot pull from Bing’s index), so fast Bing indexing also feeds the AI-citation funnel.
Three IndexNow mistakes that get your key rate-limited
These are the three IndexNow mistakes I see most. Check for them before you switch it on.
-
Pinging URLs that 404 or redirect
Submit the final, canonical URL that returns
200. Sending URLs that 404 or redirect wastes the engine’s crawl on pages you do not want indexed. Disabled products and URLs that only exist as redirects are the usual culprits in Magento. If I bulk-submit an old archive, I first run acurl -Isweep over the list and drop anything that does not return200. -
Hammering the endpoint on bulk imports
A hand-written observer that sends one request per product save will fire thousands of requests during a big import, and the endpoint answers too many of them with
429 Too Many Requests. Collect the URLs and send them in one POST withurlList: [...](up to 10,000 per request). My module collects every URL saved in a request and sends them together at the end. Imports that write straight to the database skip the save events entirely, so they send nothing at all. -
Rotating the API key but forgetting the verification file
If you change the key in admin but the engine can only fetch a file with the old key, your submissions get rejected as an invalid key. My module serves whatever key is in config from
/panth_indexnow/key, so a change in admin is enough. If you followed a tutorial and uploaded<key>.txttopub/by hand, upload the new file too. Either way, check it with curl straight after the change.
Magento IndexNow, frequently asked questions
-
Does Google support IndexNow?
No, not in production as of 2026. Google announced in November 2021 that they would test the IndexNow protocol, then announced again in 2024 that testing was still ongoing, but as of this writing there is no Google IndexNow endpoint and Google does not honour pings to api.indexnow.org for crawl-priority. For Google, use the standard channels: an up-to-date XML sitemap submitted in Google Search Console, the URL Inspection tool for one-off urgent submissions, and (where available) the Indexing API for job-posting and live-stream URL types. IndexNow remains the right tool for Bing, Yandex, Seznam, Yep, and Naver, just not Google. -
Is IndexNow free? Is there a quota?
Yes, completely free, no cost, no paid tier, no API-key fees. The only quota is the rate limit, which sits at roughly 10,000 URLs per day per unverified key for the public endpoint. Verified sites (those with a Bing Webmaster Tools account linked to the same domain as the IndexNow key) get a higher effective ceiling. Bursts are rate-limited regardless of verification status, if you push 20,000 URLs in a single minute you will see 429 responses. That is why cron-deferred batching is the recommended pattern: queue the pings, batch them up to 10,000 per request, and POST one batch per cron tick. -
How fast does Bing actually crawl post-ping?
For established sites with healthy crawl budget, the typical IndexNow-to-crawl latency is hours, sometimes minutes for top-priority URLs (recent content from a domain Bing trusts). New domains and low-authority sites can wait 12-48 hours even with IndexNow, because the protocol guarantees crawl scheduling, not crawl-rate priority, Bing still applies its normal crawl-budget heuristics. The headline win is the comparison against organic discovery: without IndexNow, a brand-new URL waits a median of 4-14 days for Bing to find it via sitemap re-fetch or link-following. With IndexNow, the median drops to under 24 hours. -
What happens if I send a URL Bing already crawled recently?
No harm done, Bing returns 200 (accepted) and schedules the URL for re-evaluation. Whether it actually re-crawls depends on its own freshness heuristics: a CMS page that hasn’t changed since the last crawl might be skipped, while a product page with a price change in the meta will be re-fetched. There is no penalty for over-pinging a URL within reason, the rate limit is a per-key, per-day ceiling on the count of submitted URLs, not on duplicates. The mage2kishan module deduplicates within a 24-hour window by default to avoid wasting quota on no-op re-pings, but you can override that in admin. -
Can I share one IndexNow key across multiple sites?
Yes, the protocol explicitly supports this. The key is a credential proving you control at least one host. The verification file must be served at every host you ping for, and each ping sets thehostparameter explicitly so Bing knows which site the URLs belong to. This is convenient for agencies and multi-store Magento setups where one key serves five or six storefronts, rotate the key in one admin, re-publish the verification file via the controller route on every storefront, and you’re done. The mage2kishan module supports multi-store setups out of the box and writes the same key + per-store host into each ping. -
Does the mage2kishan/module-indexnow module handle key rotation cleanly?
Yes. Rotating the key from admin (Stores → Configuration → Panth → IndexNow → Regenerate Key) does three things atomically: it updates the stored key inpanth_indexnow/indexnow/api_key, it invalidates the old verification file served via the controller route (now 404), and it begins serving the new verification file at/<newkey>.txt. Subsequent pings include the new key in the JSON body and the newkeyLocationURL, so Bing re-validates within minutes. Old pings that were already queued at rotation time are re-keyed before being POSTed to avoid 422 mismatches. Verify withcurl https://yoursite.com/<newkey>.txtimmediately after rotation and checkvar/log/panth_indexnow.logfor the first post-rotation ping response code, you want a clean200.
Want IndexNow installed on your Magento store?
Send your storefront URL. I will install mage2kishan/module-index-now, set and verify the key, and check that the first submission shows up in Bing Webmaster Tools. Fixed price.