Skip to main content
Your AWS Bill Just Spiked: The First-Hour Runbook

Your AWS Bill Just Spiked: The First-Hour Runbook

KU
• 6 min read•1 view

A scary number on the AWS bill is an incident, not a finance question. What to check in the first ten minutes, how to find the line item with two CLI queries, and when to stop the bleeding before you understand it.

Key takeaways

  • Treat an AWS bill spike as an incident.
  • First rule out a billing display bug on the AWS Health Dashboard, as happened with inflated estimates on July 17, 2026.
  • Then group cost by service and usage type with Cost Explorer.
  • If the driver is unfamiliar compute in unused regions, deactivate access keys before investigating further.

A surprise number on the AWS bill is an incident, and it deserves the same discipline as an outage: confirm it's real, find the source, contain it, then fix the cause. The order matters because one common cause, leaked credentials running compute in regions you don't use, keeps getting more expensive every minute you spend reading dashboards.

Here's the runbook we use, built around the questions you need to answer in the first hour.

Minute 0-10: is it real?

Not every scary number is a real charge. On July 17, 2026, a fault in AWS's estimated-billing computation sent customers estimates ranging from hundreds of thousands of dollars to several trillion. AWS said the displayed estimates did not reflect actual usage and charges, and recomputed the data within about a day.

So before you wake anyone:

  1. Check the AWS Health Dashboard and the AWS status page for billing events.
  2. Compare the estimated bill with Cost Explorer's daily costs. A forecast that jumped while daily actuals are flat is a display or forecast problem, not spend.
  3. Check whether a Savings Plan, Reserved Instance or annual subscription just renewed. A large one-off line is often a purchase, not usage.

If daily actuals are up, it's real. Move on.

Minute 10-25: find the line item

Cost Explorer data can lag by up to 24 hours, so look at the last two or three full days, grouped by service:

bash.bash
$ aws ce get-cost-and-usage \
    --time-period Start=2026-10-26,End=2026-10-29 \
    --granularity DAILY \
    --metrics UnblendedCost \
    --group-by Type=DIMENSION,Key=SERVICE \
    --query 'ResultsByTime[].Groups[?to_number(Metrics.UnblendedCost.Amount) > `50`].[Keys[0],Metrics.UnblendedCost.Amount]' \
    --output text

Once you know the service, drill into usage type, which tells you what kind of charge it is (data transfer, NAT processing, instance hours in a given region):

bash.bash
$ aws ce get-cost-and-usage \
    --time-period Start=2026-10-26,End=2026-10-29 \
    --granularity DAILY \
    --metrics UnblendedCost \
    --filter '{"Dimensions":{"Key":"SERVICE","Values":["Amazon Elastic Compute Cloud - Compute"]}}' \
    --group-by Type=DIMENSION,Key=USAGE_TYPE

Usage types are prefixed with a region code, so USE1-BoxUsage:... is us-east-1 and APS3-... is an Asia Pacific region you might never use. That prefix is the fastest way to spot compute you didn't launch.

The usual suspects

Pattern in the dataLikely causeFirst action
EC2 or GPU instance hours in regions you don't useLeaked access key, crypto-miningDeactivate keys, then terminate
NatGateway-Bytes jumpedTraffic to S3 or a registry going through NATAdd VPC gateway endpoints
DataTransfer-Out-Bytes jumpedPublic egress, a scraper, or a misrouted backupFind the top talker in VPC flow logs
CloudWatch Logs ingestion jumpedDebug logging left on, a log loopLower the log level, add retention
A new managed service you don't recognizeA teammate's experiment, or a compromiseAsk first, then check CloudTrail
Lambda or API Gateway requests explodedRetry storm or abusive trafficThrottle, check the caller

NAT gateway data processing is the quiet one: it's charged per GB on top of transfer, and a single job pulling images or S3 objects through NAT can dominate a bill. We broke down that line in NAT gateway costs.

Stop the bleeding before you fully understand it

If the evidence points to compute you didn't launch, contain first. Investigation can wait twenty minutes; the meter doesn't.

bash.bash
# Who launched it? CloudTrail shows the access key and identity
$ aws cloudtrail lookup-events --region ap-south-2 \
    --lookup-attributes AttributeKey=EventName,AttributeValue=RunInstances \
    --max-results 20

# Contain: deactivate the key (reversible), don't delete it yet
$ aws iam update-access-key --user-name ci-deployer \
    --access-key-id AKIAEXAMPLEKEYID --status Inactive

Then terminate the unknown resources in every affected region, rotate any other credentials that identity could reach, and open an AWS Support case. Document the timeline as you go; you'll need it for the support case and the postmortem.

After the hour: make the next one shorter

Budgets and anomaly detection don't prevent a spike, but they shrink the time to notice it. AWS Budgets data updates up to three times a day, and Cost Anomaly Detection runs about three times a day on data that can lag up to 24 hours. Set both up, route alerts to a channel someone reads, and our AWS cost anomaly detection setup covers the thresholds that avoid alert fatigue. Disable regions you never use, and use an organization policy to deny launches outside your approved regions.

Questions people ask

Why did my AWS bill suddenly increase?

The most common causes are resources left running after testing, data transfer or NAT gateway processing from a new traffic path, logging volume, and, most expensive, leaked access keys used for crypto-mining in unused regions. Group Cost Explorer by service and then usage type for the last few days to find which one it is.

What should I do if my AWS account is compromised and running up charges?

Deactivate the exposed access keys immediately, terminate unknown resources in every region, check CloudTrail for what the identity did, rotate related credentials, and open an AWS Support case. Contain first and investigate after, because unauthorized compute keeps billing while you investigate.

How do I get alerted before an AWS bill gets out of control?

Create an AWS Budget with actual and forecasted alerts at 80% and 100%, and enable Cost Anomaly Detection with alerts to email or a chat channel. Both run a few times a day on data that can lag up to 24 hours, so they shorten detection to hours rather than a month.

The decision, concretely

  • Estimate jumped but daily actuals are flat? Check the Health Dashboard; it's likely a billing display or forecast issue.
  • Unfamiliar compute in an unused region? Deactivate keys and terminate first, investigate second.
  • Data transfer or NAT line spiked? Find the traffic path; gateway endpoints often fix it for free.
  • No budget or anomaly alerts today? Set both up before closing the incident.

If NAT processing turns out to be the driver, the NAT gateway cost calculator shows what moving S3 and DynamoDB traffic onto gateway endpoints would save.

The call we'd make

Write this runbook down before you need it, including who can deactivate keys at 3 a.m. Most bill spikes are boring once found, but the dangerous one, compromised credentials, rewards speed above everything else. Treat the bill like a production signal and it stops being a monthly surprise.

Sources

Explore topics:AWSSecurityCloud
React
Share this post
KU

Kiril Urbonas

AI Engineer

570 articles
View all articles by Kiril Urbonas

You might have missed

Evergreen posts worth revisiting.