Multi-Region Resilience: Failover, Data, and DNS
Design for region failure. Active/passive and active/active, data replication, and failover testing.
On this page
Single-region risk is high. Multi-region design improves availability and disaster recovery.
Patterns
- Active/passive: One primary region; failover to secondary on failure. RTO/RPO depend on replication and DNS.
- Active/active: Traffic in multiple regions; better latency and availability, harder data consistency.
Data
- Replication: Async for RPO tolerance; sync for strong consistency (with latency tradeoff).
- Conflict resolution: Last-write-wins or application-defined; test conflict scenarios.
DNS and Traffic
- Route53 / global load balancers: Health checks and failover or weighted routing.
- TTL: Lower TTL before failover to reduce propagation delay; balance with DNS load.
Best Practices
- Run failover drills regularly; automate where possible.
- Document RTO/RPO and runbooks; align with business.
- Monitor replication lag and health in both regions.
Multi-region adds cost and complexity; start with critical paths and expand.
Canary Releases: Gradual Rollout Strategy
We've run canary deploys on most services for two years. The mechanics are easy; the metrics that decide "promote or roll back" are where the design is.
DevOps Metrics and KPIs: Measuring Success
We track the four DORA metrics plus a handful of others. The trade-off between what's measurable and what's meaningful, and how we use the numbers.
More from Cloud
Explore more articles in this category
Azure OpenAI's Sweden Central Outage: A Health Check Postmortem
A slow database made a backend service look dead, and the health check that was supposed to protect it killed it faster. The lesson has nothing to do with AI.
GKE Pod Snapshots: Cold Starts Drop 89%, If Your Nodes Match
Google's benchmarks show a 70B model restoring in 37 seconds instead of minutes. The catch is a hash and a hardware match that silently refuses to restore when either is off.
AWS Lost a Region for Good: Multi-AZ Is Not Disaster Recovery
AWS says it cannot restore data held only in Bahrain (me-south-1) or in one UAE zone. Multi-AZ gave availability, not recovery, and only cross-region copies survived.
You might have missed
Evergreen posts worth revisiting.
