GCP Workload Identity Federation: Replacing Service Account Keys

Six weeks ago we had 34 GCP service account JSON keys scattered across CI systems, developer laptops, and one regrettable Slack DM. Today we have zero. Every workload — CI, on-prem agents, third-party SaaS — authenticates via Workload Identity Federation. Here's the migration plan, the friction we hit, and the policies that now keep this from regressing.

Why Static Keys Are the Problem

Three near-misses in the last year:

1. A CI runner image accidentally bundled a creds.json and was pushed to a public ECR mirror. Caught in 23 minutes by GCP's automated detection — but still.
2. A laptop with a long-lived key was reported missing during travel. Key had Editor on the dev project.
3. An ex-employee's key kept working for 11 months because no one rotated it. Discovered during a routine audit.

Static credentials don't expire and don't know who's using them. They're the worst of all worlds.

What Workload Identity Federation Actually Is

WIF lets external identities (GitHub OIDC tokens, AWS IAM roles, OIDC from Kubernetes, SAML, etc.) impersonate GCP service accounts without any long-lived credential.

The flow:

External Identity (GitHub OIDC token)
            │
            ▼
Workload Identity Pool (validates issuer + claims)
            │
            ▼
Workload Identity Provider (maps claims → attributes)
            │
            ▼
Service Account Impersonation (short-lived token, ≤1h)
            │
            ▼
GCP API call

The "credential" your workload sees is an identity token from its native trust system (GitHub, AWS, K8s). GCP validates that token and issues a short-lived access token in exchange.

The 6-Week Migration

Week 1: Inventory

# Find every service account key in every project
for proj in $(gcloud projects list --format="value(projectId)"); do
  gcloud iam service-accounts list --project=$proj --format=json \
    | jq -r '.[].email' \
    | while read sa; do
        gcloud iam service-accounts keys list --iam-account=$sa --project=$proj \
          --filter="keyType=USER_MANAGED" --format=json
      done
done > all-keys.json

We found 34 user-managed keys across 5 projects. For each, we mapped: who downloaded it, where it lives now, what it's used for.

Two keys turned out to be completely unused — leftover from migrations years ago. Delete them first; instant security win.

Week 2: Stand Up the Pool

resource "google_iam_workload_identity_pool" "github" {
  workload_identity_pool_id = "github-actions-pool"
  display_name              = "GitHub Actions"
  description               = "Identity pool for GitHub Actions OIDC"
}

resource "google_iam_workload_identity_pool_provider" "github" {
  workload_identity_pool_id          = google_iam_workload_identity_pool.github.workload_identity_pool_id
  workload_identity_pool_provider_id = "github"

  attribute_mapping = {
    "google.subject"            = "assertion.sub"
    "attribute.repository"      = "assertion.repository"
    "attribute.repository_owner" = "assertion.repository_owner"
    "attribute.workflow_ref"    = "assertion.workflow_ref"
  }

  attribute_condition = "assertion.repository_owner == 'kirilurbonas'"

  oidc {
    issuer_uri = "https://token.actions.githubusercontent.com"
  }
}

That attribute_condition is the most important line. Without it, any GitHub Actions workflow on the entire internet can authenticate to your pool. Restrict to your org.

Week 3: Bind Service Accounts to External Identities

For each service account that CI used:

resource "google_service_account_iam_binding" "ci_deployer" {
  service_account_id = google_service_account.ci_deployer.name
  role               = "roles/iam.workloadIdentityUser"

  members = [
    "principalSet://iam.googleapis.com/projects/${var.project_number}/locations/global/workloadIdentityPools/${google_iam_workload_identity_pool.github.workload_identity_pool_id}/attribute.repository/kirilurbonas/devopsness",
  ]
}

This binding says: any GitHub Actions workflow in kirilurbonas/devopsness can impersonate ci_deployer@. Not other repos in our org. Specific.

Week 4: Migrate Workflows in Waves

Old workflow:

- uses: google-github-actions/auth@v2
  with:
    credentials_json: ${{ secrets.GCP_SA_KEY }}

New workflow:

permissions:
  id-token: write   # required for GitHub OIDC
  contents: read

steps:
  - uses: google-github-actions/auth@v2
    with:
      workload_identity_provider: projects/123456789012/locations/global/workloadIdentityPools/github-actions-pool/providers/github
      service_account: ci_deployer@my-project.iam.gserviceaccount.com

No secrets. The OIDC token is generated automatically by GitHub for each job.

Week 5: Migrate On-Prem Workloads (the hard one)

We had a Jenkins instance running outside any cloud. WIF requires an OIDC issuer, which Jenkins doesn't natively provide.

Solution: stand up a small OIDC provider next to Jenkins (we used spiffe/spire) that issues short-lived JWTs to Jenkins jobs. Configure GCP WIF to trust that issuer. Jenkins jobs now get GCP credentials via WIF without ever holding a static key.

The setup took a week. Worth it: this Jenkins instance was the holder of two of the most powerful service account keys we had.

Week 6: Decommission

# For each migrated service account:
for sa in ci_deployer@... jenkins_deployer@... ; do
  gcloud iam service-accounts keys list --iam-account=$sa \
    --filter="keyType=USER_MANAGED" --format='value(name)' \
    | while read keyId; do
        gcloud iam service-accounts keys delete $keyId \
          --iam-account=$sa --quiet
      done
done

We waited two weeks after this before deleting the now-disabled service accounts entirely. Nothing broke.

The Org-Wide Policy We Now Enforce

# Organization Policy
constraint: constraints/iam.disableServiceAccountKeyCreation
listPolicy:
  allValues: DENY

This blocks creation of new user-managed service account keys org-wide. The single exception: a tagged emergency-only project where keys are allowed but require a 2-person approval to issue.

Within 6 weeks we went from "trust the team to do the right thing" to "the platform doesn't permit the wrong thing."

What Surprised Us

1. WIF Is Not Free in Latency

Each WIF token exchange adds ~150ms to a job's startup. For short jobs (< 30s), that's noticeable. We mitigated by reusing the access token within a single job (gcloud config set instead of re-authenticating per command).

2. Some SDKs Don't Auto-Refresh

Older client libraries fetched a token once and held it. After 1 hour, calls failed. We had to upgrade three Python services to google-cloud-* libraries that auto-refresh.

3. Audit Log Volume Tripled

Every WIF token exchange is logged. For active CI fleets this is a lot of events. We added a sink to BigQuery, now 90% of our audit log queries hit BQ instead of Cloud Logging — significantly cheaper.

4. Third-Party SaaS Tools Are the Last Mile

Two SaaS tools (a security scanner and a deploy automation tool) only support static keys. We:

• Filed feature requests with both.
• For one, we issue a short-lived key (gcloud iam service-accounts keys create --validity=72h) on a weekly schedule via Cloud Scheduler. Not perfect but bounded blast radius.
• For the other, replaced the tool. Cheaper than maintaining a perpetual key.

Best Practices We'd Recommend

1. Apply constraints/iam.disableServiceAccountKeyCreation at the organization level before you migrate. Otherwise old patterns creep back in.
2. Use attribute_condition on every WIF provider. Restrict to your repos/accounts.
3. Bind service accounts to specific repos, not the whole pool. Principle of least privilege.
4. Log every WIF token exchange to a SIEM. The audit trail is the value.
5. Test the rollback before deleting old keys. We kept disabled keys for 2 weeks before deletion.
6. Document the WIF provider configuration in your runbook. The first time CI breaks at 2am, the on-call engineer will need to re-grant the binding.

What's Still Worse Than Static Keys

• Initial complexity. WIF has more moving parts. The first hour of setup feels heavier than gcloud iam service-accounts keys create.
• Debugging mismapped attributes is annoying. Permission denied errors are generic; you have to enable detailed audit logs to see what claim didn't match.
• GCP-specific knowledge is required. The pool/provider/binding model has its own vocabulary.

These costs are paid once. The benefits compound forever.

Would We Migrate Again?

Without question. The single biggest security improvement we shipped this year — and the second-biggest reduction in toil. No more "rotate this key" Jira tickets. No more wondering where a credential file ended up. No more 11-month-old access from former employees.

If you have any service account keys in your org, start with the inventory query above. The number will be larger than you think; the migration will be smaller than you fear.