Skip to main content
The architectural choice is presented as binary; the practical answer is "depends on the workload." The patterns that earn their place and the failure modes we've hit.

Multi-Region — Active-Active vs Active-Passive, And What We Actually Run

KU
Kiril Urbonas
3 months ago 7 min read9 views

The architectural choice is presented as binary; the practical answer is "depends on the workload." The patterns that earn their place and the failure modes we've hit.

Key takeaways

The architectural choice is presented as binary; the practical answer is "depends on the workload." The patterns that earn their place and the failure modes we've hit.

Multi-Region — Active-Active vs Active-Passive, And What We Actually Run#

The textbook answer to "how do I survive a region outage?" is "go multi-region." The textbook answer to "active-active vs active-passive?" is usually a flowchart. The reality is messier — every component in your stack makes a different choice, and the architectural simplicity of one pattern doesn't help when your database can't follow it.

We run a multi-region production environment across two AWS regions. Some pieces are active-active; some are active-passive; some are single-region. This is the framework we use and the failures that shaped it.

The two patterns, briefly#

Active-Active. All regions serve traffic simultaneously. Each region has the full stack, capable of handling all traffic on its own. Traffic is routed by latency, by tenant, or by round-robin. Region failure = automatic failover, often invisible to users.

Active-Passive. One region serves all traffic. The other is a standby, kept in sync but not serving. Region failure = controlled failover (manual or automated) to the passive region.

Active-active is more resilient, more expensive, more complex. Active-passive is cheaper, simpler, but slower to recover.

Where active-active fits#

Components where active-active makes sense:

Stateless services (compute). Easy. Run identical service deployments in both regions; the load balancer routes traffic. No coordination needed. A region going down just means traffic shifts.

Read-heavy data with eventual consistency. Cached reads, search indexes, CDN. Replicate to both regions; clients read locally. Slight staleness OK.

Geo-distributed data. When users in one region's data primarily lives in that region (think: GDPR-required regional data residency), each region is largely independent and active-active is natural.

Stateless event consumers (with idempotency). Both regions consume from the same event source; idempotent processing means double-delivery is benign.

For these, active-active is straightforward and adds resilience.

Where active-active is hard (and we punt)#

The Postgres-shaped problem. Active-active relational databases are operationally painful:

  • Multi-master Postgres (BDR, Patroni multi-master) exists but conflict resolution is application-dependent. Two writes to the same row in different regions = which wins?
  • Aurora Global Database does cross-region replication but the secondary is read-only. Writes go to the primary region.
  • CockroachDB / Spanner / Yugabyte are designed for active-active globally but are different databases — moving to them is a re-architecture.

For most teams running standard Postgres / MySQL, the database is active-passive even if everything else is active-active. Writes go to the primary region; reads can be local; failover is manual or scripted.

We're in this camp. Postgres is active-passive (primary in us-east-1, replica in us-west-2). Application is active-active in front of it (both regions write to the same Postgres in us-east-1). Cross-region database latency adds ~70ms to write paths in us-west-2; we live with it.

What we run, component by component#

ComponentPatternNotes
Web tier (stateless)Active-activeIndependent in each region
API tier (stateless)Active-activeIndependent in each region
Postgres (primary DB)Active-passivePrimary in us-east-1; warm standby in us-west-2
Read replicasActive-activeOne per region; reads served locally
Redis (cache)Per-region activeEach region runs its own cache; no cross-region replication
S3 (storage)Active-activeCross-region replication; clients write to local
KafkaActive-passivePrimary cluster in one region; mirror in other
Search (Elasticsearch)Active-activeIndex in both regions; writes dual-routed

The pattern: stateless and eventually-consistent things go active-active. Strongly-consistent stateful things go active-passive. Failover for the active-passive components is the disaster recovery exercise.

The data consistency cost#

Active-active sounds great until you think about consistency:

  • Concurrent writes to the same data. Two users in two regions update the same row at the same time. Which wins?
  • Read-after-write. A user writes in us-east-1, then reads. If their read goes to us-west-2 and replication is delayed, they see stale data.
  • Time skew. Region clocks aren't perfectly synced. Sub-second consistency is hard to reason about.

Resolutions exist (CRDTs, vector clocks, last-write-wins, application-level conflict resolution), but each is operational complexity and possible user-visible surprise. We avoided the problem by keeping our write-path active-passive.

The failover exercise (the part that earns its keep)#

Active-passive is only valuable if failover actually works. We test ours quarterly.

The exercise:

  1. Simulate the primary region being unreachable (route 53 weight = 0).
  2. Promote the passive Postgres to primary.
  3. Update application config to point at the new primary.
  4. Verify the new primary is serving writes.
  5. Restore the original region.
  6. Re-promote, switch back.

Total exercise time: ~2 hours. Real downtime if we did this in anger: ~10 minutes if we move fast.

What we learn every time:

  • Some config still hard-codes the old primary's hostname. Found it twice in three exercises.
  • DNS TTL matters. We've got it down to 30 seconds.
  • Connection pools cache the wrong host. We added explicit pool invalidation.
  • Replication lag at failover means data loss; how much is OK?

The discipline of doing the exercise is what turns active-passive from "we have a standby" to "we can actually fail over." Active-passive without exercise is theater.

What goes wrong#

Cross-region latency. ~70ms between us-east-1 and us-west-2. Every cross-region call (web tier in us-west-2 writing to Postgres in us-east-1) adds that. It compounds with N calls per request. Architect around it: batch writes, use local caches.

Asymmetric failure modes. A region rarely fails completely. Often a single AZ degrades, or a specific service has issues. Multi-region helps with full-region outages; it doesn't help with partial degradation. Some of our worst incidents have been partial.

Cost. Multi-region roughly doubles infrastructure cost. The cost is justified for tier-1 user-facing services; less so for internal tools.

Operational complexity. Two of everything. Two sets of dashboards, two sets of alerts, two deployment targets. Operational surface area grows.

Replication-lag-triggered data loss. Active-passive failover with non-zero replication lag means some writes from before the failover are lost. We've held data loss under 5 seconds; getting to 0 requires synchronous replication, which costs latency on every write.

When multi-region isn't worth it#

Honest list:

  • Startups / small ops teams. Multi-region doubles your ops load. Until you have a team that can sustain it, single-region with strong backups is more resilient than poorly-operated multi-region.
  • Internal tools. A dashboard going down for an hour during a region outage isn't a business issue.
  • Workloads where DR can be slow. If "we can restore from backup in 4 hours" is acceptable, you don't need multi-region — you need good backups and tested restore procedures.

We have plenty of services that are single-region. The multi-region investment is reserved for the user-facing critical path.

Things that surprised us#

Active-active is mostly active-passive in disguise. Once you trace any active-active design, there's a stateful component somewhere that's actually active-passive. The "active-active" label often applies only to the stateless tier.

The bigger reliability win is intra-region. Most outages we've seen are AZ-level, not region-level. Multi-AZ deployments within a region catch most cases at a fraction of the multi-region cost. Multi-region is for the very tail.

Disaster recovery testing is harder than building DR. Standing up the standby region is one project. Running quarterly exercises that prove it works is forever.

Multi-region is one of those decisions where the architectural choice (active-active vs active-passive) is the easy part. The hard part is operating it: failover discipline, drift detection, the second of everything. Be honest about whether the resilience pays for the complexity. For us, on the user-facing critical path, it does. Elsewhere, it doesn't.

Explore topics:Cloud
React

Get the DevOps Troubleshooting Cheat Sheet

Subscribe and get our free one-page reference for the errors that eat an afternoon — CrashLoopBackOff, OOMKilled, Terraform state locks, and more — plus new guides as we publish them.

Share this post
KU

About Kiril Urbonas

DevOps Engineer

537 articles
View all articles by Kiril Urbonas

You might have missed

Evergreen posts worth revisiting.