Skip to main content
Event-Driven Architecture: The Complete Guide

Event-Driven Architecture: The Complete Guide

KU
· Founder & Chief Editor
• 8 min read•Updated •19 views

Synchronous calls couple your services together and fail together. Events let systems react instead of ask. This is the map: the patterns, the messaging tools, and the delivery guarantees that make it work.

Key takeaways

  • Event-driven architecture means services publish facts about what happened and other services react asynchronously, instead of calling each other directly.
  • It buys resilience, independent scaling and easy fan-out at the cost of eventual consistency.
  • Build it on at-least-once delivery with idempotent consumers, a dead-letter queue, and the transactional outbox pattern.

Event-driven architecture (EDA) means services publish events about what happened ("order placed") and other services react on their own schedule, instead of calling each other synchronously. Use it when work can be async, load is spiky, or one event must fan out to several systems. Build it on at-least-once delivery, idempotent consumers, a dead-letter queue and a transactional outbox.

Verified against

Apache Kafka 4.4 design docs, RabbitMQ 4.4 reliability guide, against the kafka.apache.org, rabbitmq.com and Amazon SQS developer guide pages. Checked October 2026.

Most systems start with services calling each other over HTTP. It works until one slow dependency stalls the whole request, a downstream outage takes everyone with it, and adding a consumer means editing the producer. Events flip the relationship: the producer that emits order.placed does not know who consumes it, and consumers process when they can. That decoupling in time and knowledge is the whole idea. You pay for it with eventual consistency and harder debugging.

Pick the messaging model first

  • Message queues move work from producers to competing consumers, so N workers share the load. The core shapes are in message queue patterns.
  • Publish/subscribe fans one event out to many independent subscribers, each with its own copy. How that works is in pub/sub explained.
  • The two dominant tools make opposite trade-offs: Kafka is a distributed, replayable log; RabbitMQ is a smart broker that routes and deletes on ack. The head-to-head is in Kafka vs RabbitMQ, and if you land on Kafka, partition strategy decides how far your consumers can scale.

Whichever you pick, the decision is less about the broker and more about the guarantees you build around it.

Delivery guarantees are the real design work

Kafka's design docs define the three options plainly: at-most-once ("messages may be lost but are never redelivered"), at-least-once ("never lost but may be redelivered", Kafka's default) and exactly-once. RabbitMQ's reliability guide says the same thing from the broker side: acknowledgements guarantee at-least-once delivery, and without them message loss is possible.

So you will get duplicates. Kafka's idempotent producer and transactions remove duplicates inside Kafka, but its own docs say exactly-once delivery to other systems "generally requires cooperation with such systems." Your database is one of those systems. The deep version of that argument is in exactly-once delivery. The practical answer is to make every consumer idempotent:

migrations/001_processed_events.sql
CREATE TABLE processed_events (
  event_id     uuid PRIMARY KEY,
  processed_at timestamptz NOT NULL DEFAULT now()
);

-- In the consumer, inside ONE transaction with the business write:
BEGIN;
INSERT INTO processed_events (event_id) VALUES ($1)
  ON CONFLICT (event_id) DO NOTHING;
-- if 0 rows were inserted, this is a duplicate: COMMIT and ack, do nothing else
UPDATE inventory SET reserved = reserved + $2 WHERE sku = $3;
COMMIT;

The dedupe row and the side effect commit together, so a redelivered event is a no-op. Ack the message only after COMMIT, which is exactly what RabbitMQ's guide advises: don't acknowledge until you have finished handling the message.

The transactional outbox, not dual writes

The classic bug: a service writes the order to its database, then publishes order.placed. If the process dies between the two, you have an order nobody hears about. Reverse the order and you announce orders that never committed. The fix is to write the event into the same database transaction and let a relay publish it:

migrations/002_outbox.sql
CREATE TABLE outbox (
  id           bigserial PRIMARY KEY,
  event_id     uuid NOT NULL UNIQUE,
  topic        text NOT NULL,
  payload      jsonb NOT NULL,
  published_at timestamptz
);

BEGIN;
INSERT INTO orders (id, customer_id, total) VALUES ($1, $2, $3);
INSERT INTO outbox (event_id, topic, payload)
  VALUES (gen_random_uuid(), 'order.placed', $4);
COMMIT;

-- Relay loop: claim a batch, publish, then mark it
SELECT id, event_id, topic, payload FROM outbox
  WHERE published_at IS NULL ORDER BY id LIMIT 100
  FOR UPDATE SKIP LOCKED;

The relay publishes each row, then sets published_at. If it crashes after publishing but before marking, the event goes out twice, which your idempotent consumers already handle. A change-data-capture tool reading the outbox table does the same job without polling.

Poison messages need a place to go

A message that always fails will block a partition or loop forever unless you cap retries. On Amazon SQS, a redrive policy moves a message to a dead-letter queue after maxReceiveCount receives:

Terminal
aws sqs create-queue --queue-name orders-dlq
aws sqs set-queue-attributes \
    --queue-url https://sqs.us-east-1.amazonaws.com/123456789012/orders \
    --attributes '{"RedrivePolicy":"{\"deadLetterTargetArn\":\"arn:aws:sqs:us-east-1:123456789012:orders-dlq\",\"maxReceiveCount\":\"5\"}"}'

The SQS docs warn about two details people miss. Set maxReceiveCount high enough to absorb transient failures (a value of 1 dead-letters on the first hiccup). And for standard queues, a message keeps its original enqueue timestamp in the DLQ, so give the DLQ a longer retention period than the source or messages expire before anyone looks. More on retry-with-backoff and redrive in dead-letter queues.

The failure modes you will actually hit

SymptomCauseFix
Customer charged twice, duplicate key value violates unique constraint in logsAt-least-once redelivery hit a non-idempotent consumerDedupe table keyed on event_id, committed with the side effect
Order saved, no downstream email or shipmentDual write: DB commit succeeded, publish failedTransactional outbox plus relay
One partition's consumer lag climbs foreverA poison message is retried in placeCap retries, route to a DLQ, alert on DLQ depth
CommitFailedException and constant rebalances in Kafka consumersProcessing a batch takes longer than max.poll.interval.msSmaller batches, or move slow work off the poll loop
Events applied out of orderRelated events landed on different partitions or queuesKey by entity ID so one entity's events share a partition

Monitor queue depth, consumer lag and DLQ depth from day one. They are the earliest signal that something downstream is failing, usually well before users notice.

Where this breaks

EDA is the wrong tool when the caller genuinely needs the answer now: a login check, a price quote, a payment authorization the user waits on. It is also overkill for a small system with one producer and one consumer, where a database table and a cron job do the same thing with less to operate. And eventual consistency leaks into the product: if the UI reads from a projection that lags by seconds, someone has to design the "your order is processing" state. If nobody owns that, keep it synchronous. The signals that you do need a broker are in when to use a message queue.

Questions people ask

What is the difference between event-driven and request-driven architecture?

In request-driven architecture, a service calls another and waits for a response, so both must be up and fast at the same moment. In event-driven architecture, a service publishes a fact about what happened and moves on; consumers process it later, independently. Requests give immediate answers and tight coupling. Events give decoupling, buffering and easy fan-out, but only eventual consistency.

Do I need Kafka for event-driven architecture?

No. A managed queue like Amazon SQS or a RabbitMQ broker covers most event-driven systems, including work queues and moderate pub/sub fan-out. Reach for Kafka when you need high throughput, long retention, or replay: rebuilding a projection or onboarding a new consumer from the beginning of history. If none of those apply, Kafka is mostly extra operational weight.

Is exactly-once delivery possible?

Inside Kafka, yes: the idempotent producer and transactions give exactly-once writes across partitions. Across systems, not on its own. Kafka's own design docs say exactly-once delivery to other destinations requires cooperation from those systems. In practice you accept at-least-once delivery and make the consumer idempotent with a dedupe key committed alongside its side effect.

The call we'd make

Reach for events when work can be async, load is spiky, or one action must fan out to several systems. Start with a single managed queue, idempotent workers, an outbox, and a DLQ from day one; move to Kafka only when throughput, retention or replay demand it. And don't distribute what doesn't need distributing: a synchronous call and a database are the right answer more often than architecture diagrams admit.

Sources

Explore topics:DevOps
React
Share this post
Cite this article
Plain text

Kiril Urbonas (2026). Event-Driven Architecture: The Complete Guide. DevOpsNess. Updated 2026-10-31. https://www.devopsness.com/blog/event-driven-architecture-guide

Markdown

[Event-Driven Architecture: The Complete Guide](https://www.devopsness.com/blog/event-driven-architecture-guide) (Kiril Urbonas, DevOpsNess, updated 2026-10-31)

KU

Kiril Urbonas

Founder & Chief Editor

605 articles
View all articles by Kiril Urbonas

You might have missed

Evergreen posts worth revisiting.