Twenty-three clusters, one app, and a folder of near-identical Application YAMLs that drifted constantly. ApplicationSets killed the copy-paste and the drift.
Argo CD ApplicationSets: Managing Many Clusters Without Copy-Paste
We had a platform-agent DaemonSet that needed to run on every cluster we operated: 23 of them across three regions, plus a rotating set of ephemeral preview clusters. Each cluster had its own Argo CD Application manifest checked into Git. When we bumped the agent image, someone had to touch 23 files. Inevitably two got missed, and we'd find a stale agent version three weeks later during an incident review. The fix was ApplicationSets.
An ApplicationSet is a controller that templates out Application resources from a generator. The cluster generator is the obvious starting point: it produces one Application per cluster registered in Argo CD (each cluster is a Secret labeled argocd.argoproj.io/secret-type: cluster).
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: platform-agent
namespace: argocd
spec:
goTemplate: true
generators:
- clusters:
selector:
matchLabels:
agent: enabled
template:
metadata:
name: 'agent-{{.name}}'
spec:
project: platform
source:
repoURL: https://github.com/acme/platform-charts
targetRevision: main
path: charts/agent
helm:
valueFiles:
- 'values-{{.metadata.labels.region}}.yaml'
destination:
server: '{{.server}}'
namespace: platform-system
syncPolicy:
automated:
prune: true
selfHeal: true
Now the set of managed clusters is driven by a label on the cluster secret. Add agent: enabled to a new cluster's secret and Argo generates the Application within seconds. Remove the label and, because we set prune: true at the ApplicationSet level too, the Application is deleted and its workloads cleaned up.
Notice values-{{.metadata.labels.region}}.yaml. Each cluster secret carries a region label (us-east, eu-west, ap-south), and the template picks the matching values file. That's how we handle the real-world truth that clusters are not identical: eu-west needs a lower memory limit, ap-south points at a regional mirror registry. One chart, per-region overrides, zero forked manifests.
The cluster generator alone stops being enough when you deploy several apps to several clusters and the combinations matter. The matrix generator takes the Cartesian product of two generators. We use it to deploy a list of infra addons (defined in a Git directory) across the same labeled clusters:
generators:
- matrix:
generators:
- git:
repoURL: https://github.com/acme/platform-charts
revision: main
directories:
- path: addons/*
- clusters:
selector:
matchLabels:
agent: enabled
The git generator yields one entry per directory under addons/ (say cert-manager, node-exporter, fluent-bit), the cluster generator yields the clusters, and the matrix produces one Application per pair. Adding a new addon is a new directory; adding a cluster is a label. Neither requires editing the ApplicationSet.
Two settings saved us from a bad afternoon. The first is preserveResourcesOnDeletion. Without it, deleting the ApplicationSet cascades to every generated Application and prunes real workloads. If you're refactoring the ApplicationSet itself, that's terrifying. Set it while you experiment:
spec:
syncPolicy:
preserveResourcesOnDeletion: true
The second is the ApplicationSet progressive sync (strategy: RollingSync), which rolls changes out cluster group by cluster group instead of hitting all 23 at once. We label clusters env: canary vs env: prod and sync canary first, wait for health, then prod. A bad image bump now blast-radiuses to one cluster, not the fleet.
strategy:
type: RollingSync
rollingSync:
steps:
- matchExpressions:
- key: env
operator: In
values: [canary]
- matchExpressions:
- key: env
operator: In
values: [prod]
Templating errors are opaque. A typo in a Go template field renders an Application with an empty path, and Argo happily tries to sync nothing, or worse, the repo root. Run argocd appset get platform-agent and check the generated Applications before trusting a sync. Also, the cluster generator only sees clusters Argo already knows about, so cluster registration has to happen first, usually via your Terraform or a cluster-bootstrap job.
If you manage more than a handful of clusters and find yourself copying Application YAML, switch to ApplicationSets now, starting with the cluster generator plus per-region values files. Add the matrix generator only when you have a genuine app-times-cluster grid. And turn on RollingSync before you point it at production, because "sync all 23 at once" is a great way to learn what a fleet-wide outage feels like.
