Security Posture improvements

[Klwilson2272]

cert-manager helm chart version v0.12.1
Does not allow setting security context fields.

Run as non-root. It appears the chart hardcodes the root user 0 in the daemonset.yaml

Also spec.secruityContext.privileged=false is desired, so that this avoids issues with security compliance tools.

[baurmatt]

Doesn't a CSI provider require root and privileged by design?

[wmgroot]

I deploy this csi-driver into my customers' Kubernetes clusters as part of my work, and this is regularly flagged as a high severity security issue. I'd like to get this addressed upstream rather than working with exceptions.

privileged: true is not required for a CSI driver to function. I believe the requirements of this CSIDriver are satisfied with use of hostPaths that are mounted into each pod that needs access to the certs being managed.

Here's the aws-ebs-csi-driver for reference, which does not set privileged: true.
https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/charts/aws-ebs-csi-driver/templates/_node.tpl

Docs for privileged.

kubectl explain pod.spec.containers.securityContext.privileged
KIND:       Pod
VERSION:    v1

FIELD: privileged <boolean>


DESCRIPTION:
    Run container in privileged mode. Processes in privileged containers are
    essentially equivalent to root on the host. Defaults to false. Note that
    this field cannot be set when spec.os.name is windows.

CSIDriver docs.
https://kubernetes-csi.github.io/docs/developing.html

I believe that this line can simply be removed without breaking any of the existing functionality of this CSIDriver, and will drastically improve the security surface area of the tool.
https://github.com/cert-manager/csi-driver/blob/main/deploy/charts/csi-driver/templates/daemonset.yaml#L86

[wallrj-cyberark]

Hi @Klwilson2272, @wmgroot, @baurmatt — and @yugstar who picked this up in #672. Apologies for the long reply; I wanted to test the privileged: false claim end-to-end before responding, because if it works there is no good reason for us to keep the hardcoded default. The short answer is that it does not work, and I want to share the evidence so we can have a more productive conversation about what would.

What I tested

csi-driver-spiffe v0.14.0 from the published chart, on a single-node kind cluster (k8s v1.36.1), with the official chart rendered via helm template and then patched manifest-by-manifest. Equivalent results apply to csi-driver itself — the driver code path is identical because both vendor github.com/cert-manager/csi-lib.

Results

Variant of the main driver container	API server admission	Driver pod	Workload /spiffe
Default (privileged: true, mountPropagation: Bidirectional)	✅ admits	Running	Certificate delivered
Flip privileged: false only	❌ spec.template.spec.containers[2].volumeMounts.mountPropagation: Forbidden: Bidirectional mount propagation is available only to privileged containers	n/a	n/a
privileged: false + capabilities.add: [SYS_ADMIN]	❌ Same admission error — the API server checks the privileged flag, not the underlying capability	n/a	n/a
privileged: false + SYS_ADMIN, and remove mountPropagation: Bidirectional	✅ admits	CrashLoop: error: failed to setup filesystem: mounting tmpfs: mount failed: exit status 32 ... mount: /csi-data-dir/inmemfs: permission denied	n/a
privileged: false + SYS_ADMIN+DAC_OVERRIDE+FOWNER, and drop readOnlyRootFilesystem: true, and remove Bidirectional	✅ admits	Running 3/3	Silently empty — pod reads READY=1/1, ls /spiffe returns nothing, the workload never receives a certificate

The last row is the dangerous one. Workloads start, liveness probes pass, the operator's dashboard is green — and then TLS handshakes fail in production because the certificate the workload expected at /spiffe/tls.crt is not there.

Why it fails silently

The driver:

1. mounts a tmpfs at /csi-data-dir/inmemfs/ inside its own mount namespace (this is why it needs CAP_SYS_ADMIN to start at all);
2. on NodePublishVolume, writes the certificate into a per-volume directory inside that tmpfs;
3. bind-mounts that directory onto the kubelet target at /var/lib/kubelet/pods/<pod-uid>/volumes/kubernetes.io~csi/spiffe/mount.

Step 3 is the load-bearing one. Both source and target are bind mounts that exist in the driver's mount namespace. For kubelet (in the host's namespace) to see them and propagate them into the workload pod, the pods-mount-dir volume in the daemonset has to be mounted with mountPropagation: Bidirectional — and Kubernetes only permits that on privileged containers. This is the Kubernetes mount propagation documentation, and it is enforced at admission, not just in policy:

Bidirectional mount propagation can be dangerous. It can damage the host operating system and therefore it is allowed only in privileged Containers.

Drop Bidirectional and the bind mount is invisible to kubelet, so the workload pod gets a bind mount of the empty target directory — exactly what we observe.

On the aws-ebs-csi-driver reference in this earlier comment

I followed the link to _node.tpl and confirmed that template contains no hardcoded privileged: line. However, that is because the field is templated out of values.yaml, where the default is:

containerSecurityContext:
  readOnlyRootFilesystem: true
  privileged: true

with the inline comment "Privileged containers always run as Unconfined, which means that they are not restricted by a seccomp profile." aws-ebs-csi-driver also uses mountPropagation: Bidirectional on its kubelet-dir mount, for the same reason ours does. So aws-ebs-csi-driver is not an existence proof of a CSI driver running unprivileged; it has the same architecture and the same default.

What this means for the security scanner finding

The scanner finding is real, but the framing — "the chart hardcodes privileged" — points away from the root cause. The chart hardcodes privileged: true because the driver requires mountPropagation: Bidirectional, and Kubernetes admission requires the container to be privileged in that case. Adding a privileged Helm value (as #672 / cert-manager/csi-driver-spiffe#538 propose) lets operators set it to false, but every working configuration in the table above still has privileged: true; the rendered manifest in any scanner-passing install would be functionally broken.

Two paths that would address the scanner alert:

1. Architectural change to the driver: redesign so that it no longer needs Bidirectional mount propagation — most likely by having the driver perform the underlying write differently (e.g. an emptyDir-backed staging path that kubelet itself bind-mounts, rather than a tmpfs mounted by the driver process). This is a non-trivial change to csi-lib and to the on-node lifecycle; I do not think it is doable as a Helm patch.
2. Document the requirement: add a short explanation in the chart README.md of why privileged: true is structural, with the Kubernetes mount-propagation reference. That gives operators something concrete to point their security review and scanner-exception process at.

The first is what actually fixes the scanner alert. The second at least closes the conversation in the meantime.

Pull request implications

I think #672 (and the companion cert-manager/csi-driver-spiffe#538) are reasonable as scoped to pod-level securityContext, the two sidecars, and the runAsUser / per-pod knobs — those are legitimate operator-customisation cases that match the precedent set by cert-manager/trust-manager#836. Exposing the main container's securityContext is the part that does not buy what it appears to buy: any operator who sets privileged: false to satisfy a scanner will silently break their workloads.

Reproduction (sketch)

# Default chart with v0.14.0
helm template csi-driver-spiffe jetstack/cert-manager-csi-driver-spiffe \
  --version v0.14.0 \
  --namespace cert-manager \
  ... > manifests-default.yaml

# The "best case" unprivileged variant
sed -i \
  -e 's/privileged: true/privileged: false/' \
  -e '/mountPropagation: "Bidirectional"/d' \
  -e 's/readOnlyRootFilesystem: true//' \
  manifests-default.yaml
# add capabilities.add: [SYS_ADMIN, DAC_OVERRIDE, FOWNER] to the main container

kubectl apply -f manifests-default.yaml
# Pod admits, daemonset runs.

# Then a workload requesting the volume:
kubectl run spiffe-workload --image=busybox:1.36 --restart=Never \
  --overrides='{"spec":{"volumes":[{"name":"spiffe","csi":{"driver":"spiffe.csi.cert-manager.io","readOnly":true}}],"containers":[{"name":"app","image":"busybox:1.36","command":["sleep","600"],"volumeMounts":[{"name":"spiffe","mountPath":"/spiffe","readOnly":true}]}]}}'

kubectl exec spiffe-workload -- ls -la /spiffe
# total 8
# dr--r-----    2 root root 4096 ...
# drwxr-xr-x    1 root root 4096 ...

Happy to share the full manifest diffs and driver-log captures if anyone wants to reproduce more exactly, and even happier to be shown wrong — if anyone has an existing CSI driver in production that genuinely runs with privileged: false and works for ephemeral volumes (not just block/external-attacher patterns), I would like to see how.