Best Practices: Terraform State Isolation by Environment
One Terraform state file per environment sounds obvious until you watch a dev plan touch a prod resource. Here's how we actually isolate state and the mistakes we made getting there.
On this page
Terraform State Isolation by Environment
About two years ago I worked on a team where someone ran terraform apply against staging and accidentally destroyed an RDS instance in production. The state files for both environments were in the same S3 bucket, named identically except for a directory prefix. The engineer had cd'd into the wrong folder, typed apply, and answered yes to a destroy plan that referenced what they thought was staging.
Nothing apocalyptic happened — we restored from a snapshot in 40 minutes — but the close call drove a project to lock down environment isolation properly. This post is what stuck.
The cheap version that almost works
The pattern most teams start with: one S3 backend, one DynamoDB lock table, separate keys per environment.
terraform {
backend "s3" {
bucket = "my-tfstate"
key = "environments/dev/terraform.tfstate"
region = "us-east-1"
dynamodb_table = "my-tfstate-locks"
}
}
# environments/prod/main.tf — same bucket, different key
terraform {
backend "s3" {
bucket = "my-tfstate"
key = "environments/prod/terraform.tfstate"
region = "us-east-1"
dynamodb_table = "my-tfstate-locks"
}
}
This works in the sense that terraform apply in dev/ touches the dev state file and apply in prod/ touches the prod one. The directories are separate. The locks are separate. Most workflows are fine.
What it doesn't protect against: a human with credentials valid for both environments running the wrong command in the wrong directory. The accident I described above happened with this exact setup.
The version we actually run
Three changes to the cheap version, in order of importance:
1. Separate AWS accounts per environment
This is the one that did the most heavy lifting. Production resources live in a production AWS account. Staging in a staging account. Dev in a dev account.
The Terraform code provider blocks have explicit account_id allowlists:
provider "aws" {
region = "us-east-1"
allowed_account_ids = ["123456789012"] # production only
}
If anyone tries to apply this configuration with credentials for a different account, Terraform refuses before doing anything. This single check would have prevented the original incident: the engineer had no way to even authenticate to prod from the staging directory.
2. Separate state buckets per account
Each account hosts its own Terraform state bucket. Staging's bucket lives in the staging account, prod's in the prod account. The bucket policies allow access only from within the same account, full stop.
The directory structure looks like this:
infra/
├── modules/
│ ├── networking/
│ ├── eks/
│ └── rds/
├── environments/
│ ├── dev/
│ │ ├── main.tf
│ │ ├── backend.tf # points at dev account's bucket
│ │ └── provider.tf # allowed_account_ids = dev account
│ ├── staging/
│ │ ├── main.tf
│ │ ├── backend.tf
│ │ └── provider.tf
│ └── prod/
│ ├── main.tf
│ ├── backend.tf
│ └── provider.tf
The shared modules in modules/ are environment-agnostic. The environment-specific composition lives under each environments/{env}/.
3. Different IAM roles for each environment, assumed via SSO
Engineers don't have static AWS keys. They authenticate to our SSO provider, which lets them assume a role in the appropriate account. The role for dev allows broad access; the role for staging is more restricted; the role for prod is restricted further and limited to a small group.
# what an engineer types day-to-day
aws sso login
aws --profile prod s3 ls # this only works for the production team
cd infra/environments/prod
terraform plan # uses the assumed prod role
The role for prod requires MFA on assumption, with a 1-hour session lifetime. Plan-only operations are allowed broadly; apply operations require an additional explicit approval policy that Terraform Cloud (or in our case, a homemade equivalent) enforces.
What broke when we set this up
A few real issues during the migration:
Cross-environment data references stopped working. Some of our Terraform code had data "aws_ssm_parameter" lookups that crossed account boundaries — staging reading a value from a prod-account parameter store. When we split accounts, those lookups failed. The fix was to copy the values into each environment's parameter store, or use terraform_remote_state with explicit cross-account read permissions where unavoidable.
Karpenter / Crossplane-style "operators that create infrastructure" got complicated. We had a controller running in dev that was supposed to provision dev-account resources. Splitting accounts meant the controller needed cross-account IAM roles. Fixable, but not free.
Module-internal aws_caller_identity lookups assumed a single account. A few modules had data "aws_caller_identity" "current" and used the result to construct ARNs. When the modules were used in different accounts, the ARNs were correctly different — usually fine, occasionally not. We audited these and made the assumptions explicit (passing account IDs as variables instead of inferring them).
Common mistakes we still see
People hitting the team with these patterns, asking for help:
- "Can I use a single workspace for dev/staging/prod?" Workspaces (
terraform workspace) share a backend. If you're usingterraform workspace select prod, your prod state lives in the same place as your dev state. We don't allow this for environments. Workspaces are fine for ephemeral previews; they're not isolation. - "Can I just use directories like before but skip the accounts?" Sometimes, if you don't have access to AWS Organizations and can only have one account. In that case, IAM roles with strict policies per environment are the next-best-thing. It's strictly weaker than account separation; document it.
- "Do I really need a separate state bucket per account?" Yes. The bucket has to live somewhere; if it lives in a "shared" account that all engineers can access, you've recreated the problem. Bucket per account, with no cross-account read by default.
What we don't bother with
- Encrypting state at the state-file level (beyond S3's default encryption). Sensitive values in state are a problem we addressed by NOT putting sensitive values in state — passwords come from Vault at runtime, not from
random_passwordresources stored in state. - Versioning state files via additional tooling. S3 versioning is on. We've never needed point-in-time state recovery; the few times we've corrupted state, the fix was a
terraform importcycle, not a rewind.
Day-2 cost: not zero
Multi-account adds friction. Logging is split across accounts (we use a centralised CloudTrail aggregation in a shared "audit" account). Cost reports are per account (Cost Explorer with a payer account). Some monitoring tools need to be configured per account.
The friction is real. Worth it. Once a year there's an incident in a peer org where someone trashes prod from a dev terminal, and we get to say "that can't happen here." That's worth quite a lot of friction.
What I'd tell a team starting out
If you can use AWS Organizations and separate accounts: do it from day one. Refactoring later is genuinely painful (we did it; it took a quarter).
If you can't: at minimum, separate state buckets, separate IAM roles per environment, the allowed_account_ids provider check on every config, and a written rule that "apply" against prod requires a second human to be on a call. The rule is the lever; the tooling enforces it.
The only Terraform structure that reliably prevents the kind of incident that started this story is account-level isolation. Anything weaker works sometimes and fails sometimes.
Cite this article
Kiril Urbonas (2025). Best Practices: Terraform State Isolation by Environment. DevOpsNess. Updated 2026-11-20. https://www.devopsness.com/blog/best-practices-terraform-state-isolation-by-environment
[Best Practices: Terraform State Isolation by Environment](https://www.devopsness.com/blog/best-practices-terraform-state-isolation-by-environment) (Kiril Urbonas, DevOpsNess, updated 2026-11-20)
More from Infrastructure
Explore more articles in this category
Multi-Cloud Infrastructure: Managing Resources Across Providers
We run mostly on AWS but use GCP for specific workloads. The honest cost-benefit analysis of multi-cloud, plus the patterns that make it not awful.
Infrastructure as Code: Terraform vs Pulumi vs Ansible
Compare Terraform, Pulumi, and Ansible for Infrastructure as Code. Learn when to use each tool and how they complement each other in modern DevOps workflows.
How the TLS Handshake Works (Explained)
A developer-focused walkthrough of the TLS 1.3 handshake, certificate trust, forward secrecy, and how to debug the errors you actually hit.
You might have missed
Evergreen posts worth revisiting.