# Pub/Sub Explained: How Publish-Subscribe Messaging Works

URL: https://www.devopsness.com/blog/pub-sub-explained
Author: Kiril Urbonas
Published: 2026-08-15
Updated: 2026-10-31
Categories: DevOps
Tags: Pub Sub, Event-Driven, Architecture, Backend, Message Queues

Summary: A practical guide to publish-subscribe messaging, covering topics, fan-out, delivery guarantees, durable subscriptions, and the real systems that power it.

---
# Pub/Sub Explained: How Publish-Subscribe Messaging Works

Pub/sub (publish/subscribe) is a messaging pattern where publishers send events to a named topic and a broker delivers a copy to every subscriber. Neither side knows about the other, so one payment event can trigger a receipt email, a fraud check and a ledger write without the payment service calling any of them.

> [!VERIFIED]
> Apache Kafka 4.4 docs, Redis 7.0+ sharded pub/sub, Google Cloud Pub/Sub exactly-once delivery, and Amazon SNS to SQS fanout, against each vendor's official documentation. Checked October 2026.

## What pub/sub actually is

The core idea is decoupling. A **publisher** emits an event to a named destination (a topic or channel) and then forgets about it. It does not know who is listening, how many listeners exist, or whether anyone is listening at all. On the other side, **subscribers** register interest in a topic and receive matching events. They do not know who produced the event or how many other subscribers exist.

Sitting between them is the **broker**. The broker owns the topics, tracks subscriptions, and routes each published message to every interested subscriber. Because neither side holds a reference to the other, you can add, remove, or restart producers and consumers independently. That independence is the whole point.

Contrast a direct HTTP call, where A needs B's address, B must be up, and A blocks when B is slow. With pub/sub, A publishes and moves on.

## Pub/sub versus a point-to-point queue

A point-to-point queue delivers each message to exactly one consumer. If ten workers pull from the same queue, a given job goes to one of them. This is competing-consumer work distribution, ideal for spreading load.

Pub/sub does **fan-out**: each message goes to every subscriber. Ten subscribers means ten copies delivered.

```
Point-to-point queue           Publish/subscribe
--------------------           -----------------
                                          +--> Sub A (email)
Producer --> [ queue ] --> Worker         |
              (1 of N gets                Publisher --> (topic) --+--> Sub B (analytics)
               each message)              (fan-out to            |
                                           all subscribers)      +--> Sub C (audit log)
```

In practice systems blend both. A topic fans out to several subscriptions, and each subscription is itself a queue with competing consumers behind it. AWS models this explicitly with SNS (the topic, fan-out) feeding several SQS queues (each a load-balanced work pool). For a fuller tour of the queue side, see [message queue patterns](/blog/message-queue-patterns).

Here is the AWS version of that shape, wired with the CLI (the queue also needs an access policy that lets the topic send to it):

```bash
$ aws sns create-topic --name orders
$ aws sqs create-queue --queue-name orders-email
$ aws sns subscribe \
    --topic-arn arn:aws:sns:us-east-1:123456789012:orders \
    --protocol sqs \
    --notification-endpoint arn:aws:sqs:us-east-1:123456789012:orders-email
```

Repeat the last two commands per consumer, and each queue gets its own copy of every order event.

## Topics, subscriptions, and delivery

A **topic** (or channel, or subject) is the named routing key that publishers write to. A **subscription** is a subscriber's standing registration against a topic. The distinction matters because two subscriptions on the same topic each get their own independent copy of the stream, with their own acknowledgment state and backlog.

Delivery happens in one of two styles:

**Push:** the broker calls the subscriber, POSTing the message to an endpoint or invoking a handler. Low latency, but the subscriber must keep up or the broker throttles.

**Pull:** the subscriber asks the broker for messages when it is ready, then acknowledges them. This gives the consumer control over pacing and makes backpressure natural, at the cost of a polling loop.

Kafka is pull-based by design; consumers track their own offset. Google Pub/Sub offers both. MQTT and Redis lean toward push: Redis pushes each published message to every connected subscriber.

## Delivery guarantees and ordering

This is where good intentions meet distributed reality. Most brokers offer **at-least-once** delivery: a message is delivered one or more times, and duplicates are possible when an acknowledgment is lost and the broker retries. Your handlers should therefore be idempotent. Exactly-once exists in specific configurations (Kafka transactions, Google Pub/Sub's exactly-once subscriptions) but it comes with conditions and only holds within the broker's boundary, not across your side effects. Google's version, for example, works only on pull subscriptions, only when subscribers connect in the same region, and with significantly higher latency than a regular subscription.

Ordering is the other trap. A global order across a whole topic is expensive and usually not offered. What you typically get is per-partition or per-key ordering. Kafka writes events with the same key to the same partition and guarantees consumers read a partition in write order; Google Pub/Sub orders messages that share an ordering key. Picking that key well is its own topic, covered in [Kafka partition strategies](/blog/kafka-partition-strategies-scaling-consumers). If your logic depends on sequence, route related events to the same key so they land in order, and design everything else to tolerate reordering.

## Durable versus ephemeral subscriptions

A **durable** subscription survives the subscriber going offline. The broker retains messages (up to a retention window) and delivers the backlog when the subscriber returns. Kafka topics, Google Pub/Sub subscriptions, and NATS JetStream all persist this way.

An **ephemeral** subscription exists only while the subscriber is connected. Classic Redis pub/sub works like this, and Redis's own docs call it at-most-once: if a subscriber is disconnected or fails to handle a message, it is "forever lost." No storage, no replay. That makes Redis pub/sub very fast and perfectly fine for live presence indicators or cache invalidation, and completely wrong for anything you cannot afford to lose. Plain MQTT at QoS 0 has the same characteristic.

## Real systems you will meet

**Google Pub/Sub:** managed, durable, push or pull, ordering keys, strong for cloud-native event backbones.

**Kafka topics:** a partitioned, replayable log rather than a classic broker; consumers read by offset and can rewind. Best when you want durable history and high throughput.

**Redis pub/sub:** in-memory, ephemeral, at-most-once, no persistence. Sharded pub/sub (Redis 7.0 and later) scales it across a cluster; Redis Streams adds persistence and at-least-once delivery when you need it.

**NATS:** lightweight and fast, ephemeral by default with JetStream adding durability and replay.

**MQTT:** built for IoT and constrained networks, with QoS levels trading reliability for overhead.

**SNS + SQS:** the AWS fan-out pattern, where SNS pushes to subscribers and each SQS queue persists its copy for later processing.

## Benefits and pitfalls

**Loose coupling** lets teams ship producers and consumers independently, and **scalability** comes from fan-out plus competing consumers. **Extensibility** is the quiet win: adding a new consumer to an existing event stream needs zero changes to the publisher, so you bolt on a new audit log or search indexer without touching the code that emits the event.

The pitfalls are equally real. Pub/sub buys you **eventual consistency**, so a read right after a write may not reflect it yet, and your product and UI need to account for that. **Debugging** gets harder because there is no single call stack; a message crosses several independent consumers, and you need correlation IDs and good tracing to follow it. And **message loss** with ephemeral subscriptions bites teams who assumed the broker was storing things it never was. Always confirm the durability model before you rely on delivery.

## When to use it

Reach for pub/sub when one event needs to trigger several unrelated reactions, when producers and consumers scale or deploy independently, or when you want to add future consumers without disturbing today's code. This pattern is a building block of [event-driven architecture](/blog/event-driven-architecture-guide), so if you are already heading that direction, pub/sub is likely part of the answer.

Skip it when you need an immediate, synchronous answer, when strict global ordering is non-negotiable, or when a simple direct call or a single work queue does the job. Pub/sub adds a broker to operate and a mental model to maintain, and that cost is not always worth paying.

## Questions people ask

### What is the difference between pub/sub and a message queue?

A message queue delivers each message to exactly one consumer, so ten workers share the load. Pub/sub delivers each message to every subscriber, so ten subscribers each get a copy. Real systems combine them: a topic fans out to several subscriptions, and each subscription behaves like a queue with competing consumers behind it, which is exactly how Amazon SNS feeding several SQS queues works.

### Is Kafka a pub/sub system?

Yes, with a twist. Kafka topics fan out to every consumer group, which is pub/sub, while consumers inside one group split the partitions between them, which is queue-like. Unlike a classic broker, Kafka keeps events for a configured retention period after they are read, so consumers can rewind and replay. Ordering is guaranteed only within a partition, not across the whole topic.

## The call we'd make

Default to a durable broker unless you have a specific reason not to. The convenience of ephemeral Redis pub/sub is tempting, but the day you lose a message you cannot explain is the day you wish you had picked persistence. Start with durable subscriptions, make every handler idempotent, attach an ordering key where sequence matters, and thread a correlation ID through every message from day one. Do that and pub/sub becomes the flexible, forgiving backbone it is supposed to be rather than a source of mysteries.

## Sources

- [Apache Kafka 4.4 documentation: Introduction (topics, partitions, ordering)](https://kafka.apache.org/44/getting-started/introduction/)
- [Redis docs: Pub/Sub delivery semantics and sharded pub/sub](https://redis.io/docs/latest/develop/pubsub/)
- [Google Cloud Pub/Sub: Exactly-once delivery](https://docs.cloud.google.com/pubsub/docs/exactly-once-delivery)
- [Amazon SNS docs: Fanout to Amazon SQS queues](https://docs.aws.amazon.com/sns/latest/dg/sns-sqs-as-subscriber.html)

