CI Pipeline Caching That Actually Pays Off

CI Pipeline Caching That Actually Pays Off#

Caching is the first optimization everyone reaches for when CI gets slow, and the one most often done wrong. A cache that misses on every run costs you upload/download time for zero benefit. A cache that restores stale artifacts costs you a flaky build that's worse than no cache at all. This is what we learned getting our pipeline cache from "configured" to "actually saving minutes."

Measure first: is the cache even hitting?#

Before tuning anything, instrument the hit rate. Most CI systems report cache restore status; surface it.

- name: Restore deps
  id: cache
  uses: actions/cache@v4
  with:
    path: ~/.npm
    key: npm-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
    restore-keys: |
      npm-${{ runner.os }}-

- name: Report cache status
  run: echo "cache-hit=${{ steps.cache.outputs.cache-hit }}"

We found our "cache" was hitting 11% of the time. The key included a timestamp someone added during debugging months earlier. Every run wrote a new entry and never matched. The cache was pure overhead.

The cache key is the whole game#

A cache key has one job: change exactly when the cached content should change, and not before. Two failure modes:

• Too volatile (timestamp, commit SHA, run number): never hits.
• Too stable (just the OS): restores stale content; you cache yesterday's dependencies.

The right key is a hash of the inputs that determine the output. For dependencies, that's the lockfile — not package.json, the lockfile, because that's what pins exact versions.

key: npm-${{ runner.os }}-${{ hashFiles('**/package-lock.json') }}

restore-keys: the partial-hit fallback#

When the exact key misses (lockfile changed), restore-keys lets you fall back to the most recent prefix match. You get last build's node_modules, then npm ci only reconciles the delta instead of downloading everything.

restore-keys: |
  npm-${{ runner.os }}-

This is the difference between a 90-second cold install and a 12-second warm one on a one-package bump. The exact-key write keeps future identical runs at a full hit; the prefix fallback keeps changed runs from going fully cold.

Scope the cache to what's actually reusable#

Don't cache build outputs keyed on source unless the build is deterministic. We cached a compiled bundle keyed on the lockfile, and it served a stale bundle because the source had changed but dependencies hadn't. Cache the inputs to an expensive step (downloaded packages, base layers), not the step's product, unless you key on the full input set.

Good caching candidates:

• Package manager download caches (~/.npm, ~/.cache/pip, ~/.cargo)
• Compiler caches keyed on source hashes (sccache, ccache)
• Docker layer caches via --cache-from

Bad candidates:

• node_modules itself across major version bumps (platform-specific binaries)
• Anything keyed loosely enough to serve stale results

Docker layer caching is its own discipline#

For image builds, ordering the Dockerfile so that rarely-changing layers come first is more impactful than any external cache. Copy the lockfile and install before copying source:

COPY package-lock.json package.json ./
RUN npm ci
COPY . .
RUN npm run build

Now a source-only change reuses the npm ci layer. Combined with registry-backed cache (--cache-from type=registry), cold runners still get warm layers.

What it actually saved#

After fixing the key, adding restore-keys, and reordering the Dockerfile:

• Dependency install: p50 dropped from 94s to 14s
• Cache hit rate: 11% → 86%
• Total pipeline p50: 7m20s → 4m05s

The cache upload/download overhead is real — roughly 8–15s per cache. It only pays off when the hit rate is high enough that the saved work exceeds that overhead. Below ~40% hit rate, we found several caches were net-negative and removed them.

The rule we landed on#

Cache the expensive, deterministic inputs. Key on the exact thing that invalidates them. Add a prefix fallback for partial reuse. Then measure the hit rate — a cache you don't measure is a cache you can't trust, and an untrusted cache is one more thing making your builds slow and flaky at the same time.