A message broker is middleware that accepts messages from producers and delivers them to consumers. Producers and consumers do not call each other directly, which lets the system absorb bursts, retry work, and add consumers without changing the producer.
Message brokers are the transport layer behind many event-driven architectures. Common examples include Apache Kafka, RabbitMQ, Amazon SNS and SQS, Google Pub/Sub, Azure Service Bus, and NATS.
Broker models
Two models show up most often.
| Model | Shape | Best fit |
|---|---|---|
| Queue | One message is handled by one consumer in a worker pool | Background jobs, commands, task distribution |
| Topic or stream | One message can be observed by many subscribers | Domain events, analytics, integration, replay |
flowchart TB Producer["Producer"] --> Queue["Queue"] Queue --> WorkerA["Worker A"] Queue --> WorkerB["Worker B"] Publisher["Publisher"] --> Topic["Topic"] Topic --> Billing["Billing subscriber"] Topic --> Search["Search subscriber"] Topic --> Analytics["Analytics subscriber"]
A product can use both. Commands often belong in queues because one owner should do the work. Events often belong in topics or streams because several services may need to react.
Guarantees to understand
- Delivery. Most brokers provide at-least-once delivery, so consumers must handle duplicates.
- Durability. Messages can be stored on disk, replicated, or kept in memory depending on the broker and configuration.
- Ordering. Ordering is usually scoped to a partition, queue, or key. Global ordering is expensive and rarely worth it.
- Backpressure. Brokers buffer work when consumers are slow, but a growing backlog is a production signal, not free capacity.
- Acknowledgment. Consumers ack after successful handling. Missing or late acks cause redelivery.
Consumer groups
Many brokers let multiple consumers share a subscription. The broker assigns messages across the group so the service can scale horizontally. This improves throughput, but it also means any handler instance can see a message and duplicates remain possible after crashes or rebalances.
Choosing a broker
Choose based on workload rather than popularity.
- Use a queue when work should be claimed by one consumer.
- Use a stream when replay, ordered partitions, or event history matter.
- Use pub/sub when fan-out matters more than long retention.
- Prefer managed services when the team does not need broker internals as a core competency.
- Keep the payload contract explicit. Schema drift hurts more than broker choice.
Trade-offs
Brokers decouple runtime dependencies, but they add operational state. Teams need monitoring for lag, retries, poison messages, and dead-letter queues. They also move failure from a visible request path into asynchronous workflows, so tracing and ownership must be designed up front.