Three LLM Providers, One Cloud Region: The September 3 Outage

Three LLM Providers, One Cloud Region: The September 3 Outage#

On 3 September 2026, a regional failure in Microsoft Azure's East US infrastructure degraded ChatGPT and Codex, Claude, Grok, and Microsoft Copilot at the same time. Downdetector logged over 37,000 reports for ChatGPT, around 1,365 for Grok, and around 1,324 for Claude. Gemini took roughly 500 reports and stayed largely operational, because it runs on Google Cloud. OpenAI's status page listed 15 degraded components on ChatGPT and four on Codex, with elevated errors appearing around 10:58 UTC and services confirmed normal by early afternoon Eastern time.

If you built LLM failover in the last two years, you probably built it wrong, and this incident is the cleanest demonstration of why. The failure was not a model provider having a bad day. It was four nominally independent vendors turning out to be one dependency.

Multi-provider is not multi-cloud#

The standard resilience pattern for LLM applications is provider diversity: route to a primary, health-check it, fall back to a secondary from a different vendor. The reasoning is that two companies are unlikely to fail simultaneously, and for most failure modes that reasoning holds. Rate limits, model deprecations, capacity crunches and quality regressions are all vendor-specific.

Infrastructure failures are not. When your primary and your fallback both serve inference from the same cloud provider's region, the correlation coefficient between them is not the low number your architecture assumed. It approaches one for exactly the class of failure that takes everything down at once, which is the class you built the fallback for.

This is a familiar problem wearing new clothes. It is the same reasoning error as running your database replica in the same availability zone as the primary. What makes the LLM version harder is that the dependency is invisible. Your vendor's cloud footprint is not in their API documentation, it does not appear in their status page schema, and it can change without notice.

Why the failover config did not fire#

There is a second, more subtle failure in most implementations, and it showed up here. Health checks that test for the wrong thing do not trigger.

A typical fallback triggers on connection errors and 5xx responses. A regional degradation frequently produces neither. It produces slow responses, partial failures, intermittent timeouts, and elevated error rates that stay below whatever threshold you set. A client configured with a 60-second timeout against a provider that is answering in 45 seconds instead of 3 will sit there and wait, and your fallback will never engage, and your users will experience the outage in full while your dashboard shows the primary as healthy.

The fix is a latency budget rather than an error budget:

# Fail over on latency, not just on errors. A degraded region answers
# slowly and successfully, which an error-only health check never catches.
LATENCY_BUDGET_S = 8.0

async def complete(prompt: str) -> str:
    for provider in (PRIMARY, SECONDARY, TERTIARY):
        try:
            return await asyncio.wait_for(
                provider.complete(prompt),
                timeout=LATENCY_BUDGET_S,
            )
        except (asyncio.TimeoutError, ProviderError) as exc:
            log.warning("provider %s failed: %s", provider.name, exc)
            continue
    raise AllProvidersFailed()

The budget should be a number your product can actually tolerate, not a number derived from the provider's p99. If a response arriving in 30 seconds is useless to your user, then 30 seconds is a failure and your code should treat it as one.

What correlation looks like in an LLM stack#

Mapping the real dependency graph is unglamorous and takes an afternoon. The questions worth answering for each provider you depend on:

Which cloud and which region serves your inference, including for the specific model you call rather than the vendor's fleet overall. Whether your fallback provider answers that question differently from your primary. Whether your gateway, your vector database, your cache, and your observability pipeline are also in that region, because a fallback that cannot log or retrieve context is not a fallback. And whether your own application, which is doing the failing over, is in the affected region too.

Most teams discover at least one surprise. The common one is that the AI gateway sitting in front of both providers, bought specifically to provide routing resilience, is itself single-homed. We compared the routing options and their footprints in the AI gateway comparison.

The mechanics of doing the routing itself, including cost-based routing and model parity checks, are covered in our guide to multi-provider LLM routing and failover. This incident does not change any of those patterns. It changes which providers you should pick as your pair.

The decision, concretely#

• Running a primary and a fallback that both serve from Azure? You have one provider, not two. Pick a fallback on different infrastructure, which in practice today means a Google Cloud or AWS-hosted model, or a self-hosted one.
• Failing over on errors only? Add a latency budget. The most common outage shape is slow success, and error-only health checks are blind to it.
• Running an AI gateway for resilience? Verify its own regional footprint before you count it as redundancy. A single-homed router in front of two providers reduces your availability rather than improving it.
• Serving user-facing features that hard-fail without a model? Add a degraded mode. Cached responses, a smaller local model, or an honest "this feature is briefly unavailable" all beat a spinner. Semantic caching is the cheapest version of this and pays for itself on normal days too.

The call we'd make#

Pick your fallback provider by infrastructure, not by brand. The point of a second vendor is decorrelated failure, and two vendors on the same cloud region are correlated in precisely the scenario you are insuring against. Then test the failover by injecting latency rather than errors, because the 3 September shape was slow responses rather than refused connections, and a config that only understands refused connections will pass every drill and fail every real outage.