We had long-lived AWS keys sitting in a datacenter we don't own. IAM Roles Anywhere let us delete every one of them. Here's the real setup.
A security review flagged it: a batch job in our colo datacenter had an AKIA access key in an environment file, and that key had been valid for three years. It could read a production S3 bucket. Rotating it meant a coordinated change across four machines, so nobody had rotated it, ever. That's the trap with static keys outside AWS. They never expire on their own and rotating them is painful enough that they don't get rotated.
IAM Roles Anywhere is the fix. It lets a workload outside AWS exchange an X.509 certificate for temporary AWS credentials, the same short-lived credentials an EC2 instance gets from its instance profile. No more AKIA keys.
Three things connect: a trust anchor (your certificate authority, registered with AWS), a profile (which roles are reachable), and an IAM role with a trust policy that permits the Roles Anywhere service. Your workload already has a client certificate signed by that CA, or you issue it one.
First, register the CA as a trust anchor. You can use AWS Private CA or bring your own CA's certificate. We used our existing internal CA to avoid paying for Private CA.
aws rolesanywhere create-trust-anchor \
--name colo-internal-ca \
--source 'sourceType=CERTIFICATE_BUNDLE,sourceData={x509CertificateData="'"$(cat ca-cert.pem)"'"}' \
--enabled
Then the IAM role, whose trust policy is the important part. It trusts the Roles Anywhere service and, critically, constrains which certificates can assume it using a condition on the certificate subject.
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Principal": { "Service": "rolesanywhere.amazonaws.com" },
"Action": ["sts:AssumeRole", "sts:TagSession", "sts:SetSourceIdentity"],
"Condition": {
"StringEquals": {
"aws:PrincipalTag/x509Subject/CN": "batch-runner.colo.internal"
},
"ArnEquals": {
"aws:SourceArn": "arn:aws:rolesanywhere:us-east-1:111122223333:trust-anchor/abc-123"
}
}
}]
}
Without that StringEquals condition on the CN, any certificate your CA signs could assume the role. That's the mistake I see most often. The trust anchor says "I trust this CA," and the role's condition says "but only the specific cert I mean." You want both.
Then create a profile linking the role:
aws rolesanywhere create-profile \
--name batch-runner-profile \
--role-arns arn:aws:iam::111122223333:role/batch-runner \
--enabled \
--duration-seconds 3600
AWS ships a signing helper that speaks the credential-process protocol the AWS CLI and SDKs already understand. You point your ~/.aws/config at it and everything else just works.
[profile colo-batch]
credential_process = /usr/local/bin/aws_signing_helper credential-process \
--certificate /etc/pki/batch-runner.pem \
--private-key /etc/pki/batch-runner.key \
--trust-anchor-arn arn:aws:rolesanywhere:us-east-1:111122223333:trust-anchor/abc-123 \
--profile-arn arn:aws:rolesanywhere:us-east-1:111122223333:profile/def-456 \
--role-arn arn:aws:iam::111122223333:role/batch-runner
Now aws s3 ls --profile colo-batch returns credentials that live for one hour and refresh automatically. The private key never leaves the box, and there is no long-lived AWS secret anywhere in the config. If the machine is compromised, the attacker gets at most an hour of access after they lose the key, versus three years with the old scheme.
Certificate expiry becomes your new operational concern. The old static key never expired, which was the security problem, but it also meant nothing broke on a schedule. Now your client cert has a lifetime, and when it expires the workload loses AWS access instantly. We issue 90-day certs and renew them with a cron job that pulls a fresh cert from the internal CA at 60 days. Monitor cert expiry the same way you'd monitor a TLS cert on a load balancer, because a silent expiry at 2am is an outage.
Clock skew matters too. The signing helper signs a request with a timestamp, and if the workload's clock drifts more than a few minutes, STS rejects the assumption with an opaque error. Run NTP. We lost an hour to this before someone checked timedatectl and found the box was eight minutes fast.
If you have any workload outside AWS holding an AKIA key, and it's not behind OIDC federation already, move it to Roles Anywhere. The setup is an afternoon, the operational cost is monitoring cert expiry (which you should be doing anyway), and it eliminates an entire class of "the three-year-old key leaked" incidents. For workloads that can do OIDC, GitHub Actions and the like, prefer web identity federation instead, since there's no cert to manage. Roles Anywhere is specifically for the boxes that only have a certificate and a heartbeat.
Get the latest tutorials, guides, and insights on AI, DevOps, Cloud, and Infrastructure delivered directly to your inbox.
How we cut auth redirect latency to single-digit milliseconds and ran A/B tests without a flash of wrong content, using Vercel Edge Middleware.
Our node image shipped 240 CVEs, most from OS packages we never called. Moving to distroless dropped the count to single digits and cut image size by 70%.
Explore more articles in this category
We ran secrets three different ways across AWS, GCP, and Vault. External Secrets Operator gave us one Kubernetes-native workflow. Here's the setup and the gotchas.
Moving our fleet from x86 to Graviton promised 20% savings. We got 31%, but only after fixing native dependencies, a broken base image, and one nasty perf regression.
A p99 that jumped to 3.4 seconds during traffic ramps turned out to be cold starts. Here's how we measured them properly and cut the tail, with real init timings.