# Kubeflow Pipelines CVE-2026-77596: Tenants Can Rewrite Shared Pipelines. Upgrade to 2.18

URL: https://www.devopsness.com/blog/kubeflow-pipelines-cve-2026-77596-shared-pipeline-hijack
Author: Kiril Urbonas
Published: 2026-11-11
Updated: 2026-11-11
Categories: DevOps
Tags: Kubernetes, MLOps, Security, CVE, News

Summary: In multi-user Kubeflow Pipelines through 2.17.3, the HTTP upload endpoints skip authorization for shared pipelines. Any tenant can push a new version, and unpinned recurring runs in other namespaces will execute it. Checks, an audit script, the fix and the stopgap.

---
# Kubeflow Pipelines CVE-2026-77596: Tenants Can Rewrite Shared Pipelines. Upgrade to 2.18

Multi-user Kubeflow Pipelines through 2.17.3 lets any tenant upload a new version of a shared pipeline without a permission check. Recurring runs that follow the latest version can then execute the attacker's image and command in the victim's namespace, under its service account. Upgrade to KFP 2.18.0 and audit your shared pipeline versions.

> [!VERIFIED]
> KFP 2.17.3 (last affected) and 2.18.0 (fixed), against GitHub advisory GHSA-rjr3-43j3-4q8g, fix PR #14352, the KFP 2.18.0 release notes and manifests, the `kfp` 2.18.0 SDK client source, and the Kubeflow 26.10.0 release notes. Checked November 2026.

## What happened, in dates

- **September 9, 2026:** the fix merges into `master` as [PR #14352](https://github.com/kubeflow/pipelines/pull/14352), commit `62516d8`. The original patch came from the reporter, Vito Carolillo (credited as VitoCrl).
- **October 28:** KFP 2.18.0 ships, and the project publishes advisory GHSA-rjr3-43j3-4q8g: critical, CVSS 3.1 score 9.6 (`AV:N/AC:L/PR:L/UI:N/S:C/C:H/I:H/A:N`), CWE-862. GitHub's database entry adds CWE-639.
- **November 5:** NVD publishes CVE-2026-77596.
- **November 6:** Kubeflow 26.10.0, the full platform release, ships with Pipelines 2.18.0.
- **November 9:** GitHub reviews the entry into its global advisory database.

The advisory does not mention exploitation, and as of November 11, 2026 the CVE is not in CISA's KEV catalog.

## The attacker never touches your namespace

KFP stores "shared" pipelines with an empty namespace, so every tenant can see and run them. The two HTTP multipart endpoints, `POST /apis/v2beta1/pipelines/upload` and `POST /apis/v2beta1/pipelines/upload_version` (and their `v1beta1` twins), call a helper named `canUploadVersionedPipeline`. When the resolved namespace was empty, it returned early and never ran a `SubjectAccessReview`. An empty namespace is also what you get when the caller simply omits the `namespace` parameter, because `REQUIRE_NAMESPACE_FOR_PIPELINES` defaults to `false` and no shipped manifest sets it.

The gRPC create path already did the right thing: it maps an empty namespace to the KFP system namespace and authorizes the write there. The fix is a three-line change that copies that behavior into the upload helper. Shared writes now require `create` on `pipelines` in the `pipelines.kubeflow.org` API group in the system namespace, and the legacy `-` placeholder counts as shared too.

The damage comes from scheduling. A recurring run that references a pipeline without a pinned version resolves the newest version each time it fires. Push a version with a different image, command or args into a shared pipeline that some team schedules nightly, and their next run executes your code with their service account and mounted credentials.

Two limits from the advisory. The upload bypass was reproduced against a live API server, including with no identity header at all when the server was reachable without an authenticating proxy. The scheduled execution step was verified by reading the source, not by an end-to-end reproduction. The injected spec also has to accept the run's existing inputs.

## Am I affected?

You are exposed if all three hold: the API server image is 2.17.3 or older, `MULTIUSER` is `true`, and `REQUIRE_NAMESPACE_FOR_PIPELINES` is unset or `false`. Single-user installs skip upload authorization by design. In the standard manifests the deployment is `ml-pipeline` in `kubeflow`, and `MULTIUSER` comes from a generated `pipeline-api-server-config-*` ConfigMap rather than the pod spec.

```bash
# 1. API server image tag (2.17.3 or lower is affected)
$ kubectl -n kubeflow get deploy ml-pipeline \
    -o jsonpath='{.spec.template.spec.containers[?(@.name=="ml-pipeline-api-server")].image}{"\n"}'

# 2. MULTIUSER and REQUIRE_NAMESPACE_FOR_PIPELINES from the ConfigMaps the container loads
$ for cm in $(kubectl -n kubeflow get deploy ml-pipeline \
    -o jsonpath='{.spec.template.spec.containers[?(@.name=="ml-pipeline-api-server")].envFrom[*].configMapRef.name}'); do
    echo "== $cm"; kubectl -n kubeflow get cm "$cm" -o yaml | grep -E 'MULTIUSER|REQUIRE_NAMESPACE'
  done

# 3. Explicit env vars on the container, which override the ConfigMap
$ kubectl -n kubeflow set env deployment/ml-pipeline -c ml-pipeline-api-server --list \
    | grep -E 'MULTIUSER|REQUIRE_NAMESPACE'

# 4. Upload and authorization lines logged since the last restart
$ kubectl -n kubeflow logs deploy/ml-pipeline -c ml-pipeline-api-server \
    | grep -E 'Upload pipeline( version)? called|Authorizing request'
```

Per the advisory, a bypassed upload logs `Upload pipeline called` (`Upload pipeline version called` for versions, per the source) with no `Authorizing request` line after it. Logs only go back to the last restart unless you ship them, so the catalog is the stronger check.

## Audit the shared catalog

The API records no uploader, only `created_at`, so look for versions you cannot account for and schedules that would pick them up. Run it from a Kubeflow notebook with pipeline access, or pass `host` and `existing_token` to `kfp.Client`, with read access to each namespace you list.

```python
# audit_shared_pipelines.py   usage: python audit_shared_pipelines.py <days> <ns> [<ns> ...]
import sys
from datetime import datetime, timedelta, timezone
import kfp

client = kfp.Client()
since = datetime.now(timezone.utc) - timedelta(days=int(sys.argv[1]))

def pages(fn, field, **kw):
    token = ''
    while True:
        resp = fn(page_token=token, page_size=100, **kw)
        yield from getattr(resp, field) or []
        token = resp.next_page_token
        if not token:
            return

shared = {}
for p in pages(client.list_pipelines, 'pipelines'):  # no namespace = shared catalog
    shared[p.pipeline_id] = p.display_name
    for v in pages(client.list_pipeline_versions, 'pipeline_versions', pipeline_id=p.pipeline_id):
        flag = 'RECENT' if v.created_at and v.created_at >= since else ''
        print(f'{p.display_name}\t{v.display_name}\t{v.pipeline_version_id}\t{v.created_at}\t{flag}')

for ns in sys.argv[2:]:
    for r in pages(client.list_recurring_runs, 'recurring_runs', namespace=ns):
        ref = r.pipeline_version_reference
        if ref and ref.pipeline_id in shared and not ref.pipeline_version_id:
            print(f'UNPINNED\t{ns}\t{r.display_name}\t{r.status}\tfollows "{shared[ref.pipeline_id]}"')
```

Every `UNPINNED` line with status `ENABLED` is a schedule that would run whatever lands next in that shared pipeline. Diff each `RECENT` version's `pipeline_spec` (from `client.get_pipeline_version`) against the one before it, looking for changed images and commands.

## What to do now

**Upgrade.** For a standalone install, reapply the 2.18.0 manifests for the overlay you deployed, after reading the [2.18 upgrade guide](https://github.com/kubeflow/pipelines/blob/release-2.18/docs/operator-guides/upgrade-2.18.md). The guide warns that schema changes run before the API server serves requests, so large databases delay readiness.

```bash
$ export PIPELINE_VERSION=2.18.0
$ kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/cluster-scoped-resources?ref=$PIPELINE_VERSION"
$ kubectl wait --for condition=established --timeout=60s crd/applications.app.k8s.io
$ kubectl apply -k "github.com/kubeflow/pipelines/manifests/kustomize/env/platform-agnostic-multi-user?ref=$PIPELINE_VERSION"
```

On the full Kubeflow platform, 26.10.0 is the first release that carries Pipelines 2.18.0; 26.03.1 ships 2.16.1. Its release notes call out a required `Recreate` strategy patch on `ml-pipeline-ui` before applying.

**Stopgap if you cannot upgrade this week.** The advisory names one interim setting:

```bash
$ kubectl -n kubeflow set env deployment/ml-pipeline -c ml-pipeline-api-server \
    REQUIRE_NAMESPACE_FOR_PIPELINES=true
```

**Pin the schedules.** Recreate each `UNPINNED` recurring run with `create_recurring_run(..., pipeline_id=..., version_id=<reviewed id>)`, or disable it with `disable_recurring_run` until someone reviews the pipeline. The advisory confirms pinned runs and runs with an embedded spec are not substituted.

**Close the side door.** The unauthenticated variant needs direct access to the API server, whose `ml-pipeline` Service serves HTTP on port 8888. Make sure only the authenticating proxy and in-cluster components can reach it, the same network-policy discipline our [Kubernetes security checklist](/blog/kubernetes-security-best-practices) argues for.

## Where this breaks

`REQUIRE_NAMESPACE_FOR_PIPELINES=true` rejects pipeline creates, uploads, lists and name lookups that omit a namespace. The list call in the audit script fails once it is on, so run the audit first. Teams that browse or upload shared pipelines will get errors, so test in staging first.

Upgrading does not remove anything already uploaded. A malicious version that landed before the fix stays in the catalog and stays "latest" until someone deletes it or pins around it.

2.18.0 also changes who may write shared pipelines. Upload scripts must pass `namespace=` to `upload_pipeline` for tenant pipelines, since `Client(namespace=...)` alone does not set it and `None` now means a shared upload. Shared pipelines need a named maintainer with pipeline-write permission in the system namespace. The advisory warns against granting broad system-namespace access just to keep old scripts working.

## Questions people ask

### Does CVE-2026-77596 affect single-user Kubeflow Pipelines?

No. The flaw is a missing check on the multi-user path. In single-user mode the API server does not authorize uploads at all by design, so there is no tenant boundary to break. Keep the API server away from anyone you would not trust with the whole install.

### Is the full Kubeflow platform affected?

Yes, if it runs Pipelines 2.17.3 or older. The platform installs Pipelines in multi-user mode with profiles, which is exactly the vulnerable configuration. Kubeflow 26.10.0, released November 6, 2026, is the first platform release that ships Pipelines 2.18.0. Older releases need an upgrade, or the interim flag plus pinned schedules.

### How do I pin a recurring run to a pipeline version?

Pass a `version_id` when you create it, for example `client.create_recurring_run(experiment_id=..., job_name=..., cron_expression=..., pipeline_id=..., version_id=...)`. Per the advisory, a schedule created with only `pipeline_id` resolves the newest version each time it fires. Existing unpinned schedules need to be recreated with an explicit version id.

## Beyond Kubeflow: mutable pointers are the bug class

The attacker edited a shared recipe, and a scheduler pulled the newest version on its own. That is the same failure as deploying `image: app:latest` or a CI action referenced by a moving tag, which our piece on [software supply chain attacks](/blog/software-supply-chain-attacks) covers. Anything that runs unattended should reference an immutable id: a digest, a commit, a pipeline version id.

## The call we'd make

Upgrade multi-user installs to KFP 2.18.0, or Kubeflow 26.10.0, and run the catalog audit before and after. Pin every scheduled run that consumes a shared pipeline, whatever version you are on. The caveat: upgrading does not clean the catalog, so a version uploaded before the fix keeps running until someone removes it.

## Sources

- [Kubeflow Pipelines: GHSA-rjr3-43j3-4q8g, missing authorization for shared pipeline uploads](https://github.com/kubeflow/pipelines/security/advisories/GHSA-rjr3-43j3-4q8g)
- [kubeflow/pipelines PR #14352: fix(backend): authorize shared pipeline uploads](https://github.com/kubeflow/pipelines/pull/14352)
- [Kubeflow Pipelines: 2.18.0 release notes](https://github.com/kubeflow/pipelines/releases/tag/2.18.0)
- [Kubeflow: 26.10.0 platform release](https://github.com/kubeflow/community-distribution/releases/tag/26.10.0)