Skip to content
5.0 Upwork, 100+ reviews Adobe Certified since 2021 New: become your own Magento developer Learn more →
Hire me
Toggle Nav
My Cart
Magento glossary

What is Magento RabbitMQ?

Short answer Magento RabbitMQ is the AMQP message broker Magento can use as its queue backend. Bulk and async API operations, B2B shared catalog updates, MSI source item cleanup, product mass attribute updates and many third-party sync jobs go through it. It is required on Adobe Commerce (B2B and Cloud). On Magento Open Source it is optional, because queues fall back to MySQL when no broker is configured. Consumers are long-running PHP processes declared in etc/queue_consumer.xml, and I start them with bin/magento queue:consumers:start.

Need a queue audit? How it works

Written by Kishan Savaliya, Adobe Certified Magento & Hyvä developer, 10+ years on the platform

How it works
  1. Install RabbitMQ on the host or use a managed broker 01
  2. Wire credentials into env.php with setup:config:set 02
  3. Modules declare topics, exchanges and bindings in XML 03
  4. Consumers pick up messages one at a time 04
  5. Run consumers as long-lived processes under supervisor 05
Each step is explained below

Most common mistake Not running consumers at all

How it works

Five things RabbitMQ does inside a Magento install

You rarely notice the queue layer until something gets stuck. This is what happens between bin/magento setup:config:set and a worker draining a queue, in the order I check it when a store reports that bulk updates never finish.

  1. 01

    Install RabbitMQ on the host or use a managed broker

    You can install RabbitMQ from the OS packages (apt install rabbitmq-server on Debian or Ubuntu), run the official Docker image, or use a managed broker such as Amazon MQ or CloudAMQP. For production I lean towards managed, because I do not want to own broker upgrades and disk alarms on top of the store. Before picking a version, I check the Adobe system requirements page for the exact 2.4.x release, since the supported RabbitMQ version moves with each release line.

  2. 02

    Wire credentials into env.php with setup:config:set

    Run bin/magento setup:config:set --amqp-host=localhost --amqp-port=5672 --amqp-user=magento --amqp-password=… --amqp-virtualhost=/ once per environment. Magento writes an amqp block under queue in app/etc/env.php. From then on, every queue whose topology says connection="amqp" goes through RabbitMQ instead of the MySQL db connection. When one broker serves several projects, I give each Magento install its own vhost.

  3. 03

    Modules declare topics, exchanges and bindings in XML

    A module that publishes messages ships etc/communication.xml (topic name and message type), etc/queue_publisher.xml and etc/queue_topology.xml (exchange and bindings). When code publishes to a topic, Magento encodes the payload as JSON based on the type in communication.xml and sends it to the exchange. The exchanges, queues and bindings themselves are created during bin/magento setup:upgrade, so after adding a new topic I rerun it before testing anything.

  4. 04

    Consumers pick up messages one at a time

    Each consumer is declared in etc/queue_consumer.xml with a name, a queue, a handler and a connection. A few I see on almost every store: sales.rule.update.coupon.usage, async.operations.all, inventory.source.items.cleanup, product_action_attribute.update. The consumer takes a message, runs the handler and acks it when the handler finishes. If the handler throws, the message is rejected instead, and the error shows up in var/log/system.log or exception.log.

  5. 05

    Run consumers as long-lived processes under supervisor

    I start consumers with bin/magento queue:consumers:start <name> --single-thread --max-messages=10000. With --max-messages the process exits after that many messages and supervisord or systemd starts a fresh one, which keeps slow memory leaks in check. On smaller Open Source stores I skip supervisor and let the consumers_runner cron job start consumers instead, controlled by the cron_consumers_runner block in env.php.

When to use

Four situations where RabbitMQ is the right call

RabbitMQ is one more service to run, patch and monitor. These are the cases where I think it pays for itself.

  • All Adobe Commerce installs (required)

    On Adobe Commerce I treat RabbitMQ as a requirement. B2B features such as shared catalogs and company updates, plus bulk and async API operations, are built around message queues. On Adobe Commerce Cloud you add RabbitMQ in .magento/services.yaml and the platform provisions it; on a self-hosted Adobe Commerce install you set it up yourself and run setup:config:set with the AMQP options before going live.

  • Busy Open Source stores

    On Magento Open Source the MySQL queue (queue_message and queue_message_status tables) is fine for light use. On busier stores, with large admin mass updates or async API imports, I see those tables grow and consumers fall behind. Moving the heavy consumers to RabbitMQ takes that load off the database. A quick SELECT status, COUNT(*) FROM queue_message_status GROUP BY status tells me whether the database queue is keeping up.

  • Multi-source inventory / B2B catalogue deployments

    MSI publishes messages for jobs like source item cleanup after stock changes, and B2B shared catalogs publish permission and price updates. On a small catalog these run fine on the MySQL queue. With a large catalog and frequent stock imports, I move them to RabbitMQ so one big import does not leave consumers hours behind while the storefront shows stale stock.

  • Async web API and third-party sync

    Magento’s asynchronous web API (/rest/async/V1/... and /rest/async/bulk/V1/...) puts each request on a queue and returns a bulk UUID straight away; the async.operations.all consumer does the real work later. Many ERP and PIM connectors follow the same pattern. With RabbitMQ in place, a slow or offline third-party API means messages wait in the queue instead of slowing down the storefront.

Common mistakes

Three traps that take RabbitMQ from quietly humming to quietly broken

Every queue incident I've been paged into in the last three years collapses to one of these three root causes. Avoid them and the broker stays boring: which is the goal.

  • Not running consumers at all

    The single most common production incident I’m called in to fix: publishers are happily pushing messages to RabbitMQ, but no consumer process is running, so the queue fills indefinitely. Symptoms include orders stuck in “pending email”, stock reservations never released, and admin grids showing stale data. Always verify queue:consumers:list returns the expected workers and that rabbitmqctl list_queues shows depth trending to zero, not climbing.

  • Running consumers without a supervisor

    A bare php bin/magento queue:consumers:start … invocation dies on the first uncaught exception, memory leak or OOM. Without a supervisor (systemd, supervisord, Kubernetes Deployment) the process never restarts and the queue silently backs up. Always wrap each consumer in a unit file with Restart=always, set --max-messages=10000 so the process recycles before memory creeps up, and alert on consumer process count, not just queue depth.

  • Cron-runner conflict on the consumers_runner flag

    Magento ships a consumers_runner cron job that auto-starts every consumer. Leaving the cron flag on while also running supervisor-managed workers gives you duplicate consumers fighting over the same queue: half the messages process twice, half not at all. Either flip cron_consumers_runner -> cron_run -> false in env.php (use supervisor) or leave cron in charge and skip supervisor entirely. Pick one. Document which.

Related concepts

Five neighbour concepts most readers want to look at next. Click through for the full deep-dive.

FAQ

Magento RabbitMQ: frequently asked questions

  • RabbitMQ vs the MySQL queue: when must I use RabbitMQ?
    You must use RabbitMQ on Adobe Commerce (it is required by the framework) and on any Magento Open Source store pushing more than ~1,000 jobs per hour through the queue layer: typically anything over 500 orders per day, or any deployment using B2B, Multi-Source Inventory or async webhooks. The MySQL fallback (the db connection in queue.xml) is fine for development and for small Open Source stores below those thresholds, but it shares the main Magento database, locks on the queue_message table during contention, and does not scale horizontally. RabbitMQ is a dedicated broker with its own connection pool, persistent storage and clustering, so adding workers genuinely scales throughput.
  • How many consumers should I run, and how many workers per queue?
    Start with one worker process per consumer name and scale up only the hot queues. async.operations.all, product_action_attribute.update, sales.rule.update.coupon.usage and the MSI reservation consumers tend to be the busiest on a typical store; everything else can stay at one worker. Use rabbitmqctl list_queues name messages consumers to see depth and worker count side-by-side, then add workers to any queue where depth grows faster than it drains. The hard cap is CPU: each consumer is a full PHP process so 4 - 8 workers per CPU core is the practical ceiling. Most Open Source stores need fewer than 10 worker processes total; Adobe Commerce on a busy B2B catalogue can run 30+.
  • Why are my consumers dying silently after a few hours?
    Almost always PHP memory leak. Long-lived PHP processes accumulate memory across messages and eventually hit memory_limit, at which point PHP terminates the process with no useful log line. The fix is the --max-messages flag: bin/magento queue:consumers:start <name> --single-thread --max-messages=10000 tells the consumer to exit cleanly after N messages, after which supervisor / systemd starts a fresh worker. 10,000 is a safe default; tune lower if a specific handler is leaky. Also set memory_limit on the CLI php.ini high enough (1G is reasonable) so the worker survives the message batch before recycling. Pair this with alerting on consumer process count, not queue depth, so you find out the moment a worker dies and supervisor fails to restart it.
  • Can I cluster RabbitMQ for high availability, and what is the right sharding strategy?
    Yes, but think carefully about the strategy. RabbitMQ supports two HA modes: classic mirroring (deprecated in 3.9, removed in 4.0) and quorum queues (the current recommended option). Quorum queues use the Raft consensus protocol and survive losing one node in a three-node cluster without data loss. For Magento workloads, declare critical queues (async.operations.all, sales-rule recalc, MSI reservations) as quorum and leave low-priority queues classic for throughput. Adobe Commerce Cloud handles all of this for you on the managed broker. For self-hosted setups, CloudAMQP and AWS MQ both expose quorum-queue support with sensible defaults: managed almost always beats DIY clustering once you account for monitoring, backups and patching.
  • Does Hyvä affect queue or consumer usage at all?
    No. RabbitMQ is a backend concern: the queue layer runs entirely on the PHP side of Magento, before any HTML rendering happens. Switching the storefront theme from Luma to Hyvä changes how the browser receives and renders HTML, but does not touch publishers, consumers, queue declarations or the AMQP connection. A Hyvä migration neither speeds up nor slows down message throughput. The two systems are completely orthogonal, and both are worth tuning independently. If you are migrating to Hyvä on a high-volume store, audit the consumer setup at the same time: the front-of-house win loses some of its shine if checkout async-email is stuck behind a cold MySQL queue.
  • How do I monitor queue depth and consumer health in production?
    Three layers. Layer one is the RabbitMQ Management UI on port 15672 (enable with rabbitmq-plugins enable rabbitmq_management): gives a live dashboard of queue depth, message rate, consumer count and connection health. Layer two is CLI checks via rabbitmqctl list_queues name messages consumers messages_ready messages_unacknowledged, scriptable into your monitoring stack (Prometheus rabbitmq_exporter, Datadog agent, New Relic integration). Layer three is application-level: alert on Magento's own bin/magento queue:consumers:list output (consumer process count) and on queue depth growth rate: depth climbing without bound is the canonical signal that a consumer has died or a handler is throwing. Send all three layers to PagerDuty for prod.
Queue audit

Want a RabbitMQ + consumer audit on your Magento store?

Send your storefront URL: I will review queue topology, consumer supervisor setup, cron-runner config and message-rate metrics, then reply with a written remediation plan, fixed-price quote and earliest start date. 24-business-hour turnaround.

Get a queue audit Hire me directly
Kishan Savaliya Online 10 AM to 7 PM IST Chat on WhatsApp