# Your AWS Bill Just Spiked: The First-Hour Runbook

URL: https://www.devopsness.com/blog/aws-bill-spike-first-hour-runbook
Author: Kiril Urbonas
Published: 2026-10-29
Updated: 2026-10-30
Categories: Cloud
Tags: AWS, Security, Cloud Cost, FinOps, Incident Response

Summary: A scary number on the AWS bill is an incident, not a finance question. What to check in the first ten minutes, how to find the line item with two CLI queries, and when to stop the bleeding before you understand it.

---
# Your AWS Bill Just Spiked: The First-Hour Runbook

A surprise number on the AWS bill is an incident, and it deserves the same discipline as an outage: confirm it's real, find the source, contain it, then fix the cause. The order matters because one common cause, leaked credentials running compute in regions you don't use, keeps getting more expensive every minute you spend reading dashboards.

Here's the runbook we use, built around the questions you need to answer in the first hour.

## Minute 0-10: is it real?

Not every scary number is a real charge. On July 17, 2026, a fault in AWS's estimated-billing computation sent customers estimates ranging from hundreds of thousands of dollars to several trillion. AWS said the displayed estimates did not reflect actual usage and charges, and recomputed the data within about a day.

So before you wake anyone:

1. Check the **AWS Health Dashboard** and the AWS status page for billing events.
2. Compare the **estimated bill** with **Cost Explorer's daily costs**. A forecast that jumped while daily actuals are flat is a display or forecast problem, not spend.
3. Check whether a **Savings Plan, Reserved Instance or annual subscription** just renewed. A large one-off line is often a purchase, not usage.

If daily actuals are up, it's real. Move on.

## Minute 10-25: find the line item

Cost Explorer data can lag by up to 24 hours, so look at the last two or three full days, grouped by service:

```bash
$ aws ce get-cost-and-usage \
    --time-period Start=2026-10-26,End=2026-10-29 \
    --granularity DAILY \
    --metrics UnblendedCost \
    --group-by Type=DIMENSION,Key=SERVICE \
    --query 'ResultsByTime[].Groups[?to_number(Metrics.UnblendedCost.Amount) > `50`].[Keys[0],Metrics.UnblendedCost.Amount]' \
    --output text
```

Once you know the service, drill into usage type, which tells you what kind of charge it is (data transfer, NAT processing, instance hours in a given region):

```bash
$ aws ce get-cost-and-usage \
    --time-period Start=2026-10-26,End=2026-10-29 \
    --granularity DAILY \
    --metrics UnblendedCost \
    --filter '{"Dimensions":{"Key":"SERVICE","Values":["Amazon Elastic Compute Cloud - Compute"]}}' \
    --group-by Type=DIMENSION,Key=USAGE_TYPE
```

Usage types are prefixed with a region code, so `USE1-BoxUsage:...` is us-east-1 and `APS3-...` is an Asia Pacific region you might never use. That prefix is the fastest way to spot compute you didn't launch.

## The usual suspects

| Pattern in the data | Likely cause | First action |
|---|---|---|
| EC2 or GPU instance hours in regions you don't use | Leaked access key, crypto-mining | Deactivate keys, then terminate |
| `NatGateway-Bytes` jumped | Traffic to S3 or a registry going through NAT | Add VPC gateway endpoints |
| `DataTransfer-Out-Bytes` jumped | Public egress, a scraper, or a misrouted backup | Find the top talker in VPC flow logs |
| CloudWatch Logs ingestion jumped | Debug logging left on, a log loop | Lower the log level, add retention |
| A new managed service you don't recognize | A teammate's experiment, or a compromise | Ask first, then check CloudTrail |
| Lambda or API Gateway requests exploded | Retry storm or abusive traffic | Throttle, check the caller |

NAT gateway data processing is the quiet one: it's charged per GB on top of transfer, and a single job pulling images or S3 objects through NAT can dominate a bill. We broke down that line in [NAT gateway costs](/blog/nat-gateway-costs-cutting-the-silent-line-item).

## Stop the bleeding before you fully understand it

If the evidence points to compute you didn't launch, contain first. Investigation can wait twenty minutes; the meter doesn't.

```bash
# Who launched it? CloudTrail shows the access key and identity
$ aws cloudtrail lookup-events --region ap-south-2 \
    --lookup-attributes AttributeKey=EventName,AttributeValue=RunInstances \
    --max-results 20

# Contain: deactivate the key (reversible), don't delete it yet
$ aws iam update-access-key --user-name ci-deployer \
    --access-key-id AKIAEXAMPLEKEYID --status Inactive
```

Then terminate the unknown resources in every affected region, rotate any other credentials that identity could reach, and open an AWS Support case. Document the timeline as you go; you'll need it for the support case and the postmortem.

## After the hour: make the next one shorter

Budgets and anomaly detection don't prevent a spike, but they shrink the time to notice it. AWS Budgets data updates up to three times a day, and Cost Anomaly Detection runs about three times a day on data that can lag up to 24 hours. Set both up, route alerts to a channel someone reads, and our [AWS cost anomaly detection](/blog/aws-cost-anomaly-detection) setup covers the thresholds that avoid alert fatigue. Disable regions you never use, and use an organization policy to deny launches outside your approved regions.

## Questions people ask

### Why did my AWS bill suddenly increase?

The most common causes are resources left running after testing, data transfer or NAT gateway processing from a new traffic path, logging volume, and, most expensive, leaked access keys used for crypto-mining in unused regions. Group Cost Explorer by service and then usage type for the last few days to find which one it is.

### What should I do if my AWS account is compromised and running up charges?

Deactivate the exposed access keys immediately, terminate unknown resources in every region, check CloudTrail for what the identity did, rotate related credentials, and open an AWS Support case. Contain first and investigate after, because unauthorized compute keeps billing while you investigate.

### How do I get alerted before an AWS bill gets out of control?

Create an AWS Budget with actual and forecasted alerts at 80% and 100%, and enable Cost Anomaly Detection with alerts to email or a chat channel. Both run a few times a day on data that can lag up to 24 hours, so they shorten detection to hours rather than a month.

## The decision, concretely

- **Estimate jumped but daily actuals are flat?** Check the Health Dashboard; it's likely a billing display or forecast issue.
- **Unfamiliar compute in an unused region?** Deactivate keys and terminate first, investigate second.
- **Data transfer or NAT line spiked?** Find the traffic path; gateway endpoints often fix it for free.
- **No budget or anomaly alerts today?** Set both up before closing the incident.

If NAT processing turns out to be the driver, the [NAT gateway cost calculator](/tools/nat-gateway-cost-calculator) shows what moving S3 and DynamoDB traffic onto gateway endpoints would save.

## The call we'd make

Write this runbook down before you need it, including who can deactivate keys at 3 a.m. Most bill spikes are boring once found, but the dangerous one, compromised credentials, rewards speed above everything else. Treat the bill like a production signal and it stops being a monthly surprise.

## Sources

- [AWS: Detecting unusual spend with AWS Cost Anomaly Detection](https://docs.aws.amazon.com/cost-management/latest/userguide/manage-ad.html)
- [AWS: Managing your costs with AWS Budgets](https://docs.aws.amazon.com/cost-management/latest/userguide/budgets-managing-costs.html)
- [AWS CLI reference: ce get-cost-and-usage](https://docs.aws.amazon.com/cli/latest/reference/ce/get-cost-and-usage.html)
- [The Stack: AWS admits bill error alarms failed to rouse engineering (July 2026)](https://www.thestack.technology/aws-admits-bill-error-alarms-failed-to-rouse-engineering/)