[Feature]: Support injecting POSIX rlimits (e.g. RLIMIT_MEMLOCK for RDMA) into the OCI spec via nvidia-container-runtime config

[liuxu623]

Problem

RDMA workloads (GPUDirect RDMA, NCCL over IB/RoCE, UCX) need to pin buffer pools that run to gigabytes. Kubernetes never sets RLIMIT_MEMLOCK, so every container inherits it from the container runtime daemon — with containerd under systemd that is the 8 MiB LimitMEMLOCK default. ibv_reg_mr then fails with ENOMEM and NCCL silently falls back to TCP sockets.

The standard advice — add the IPC_LOCK capability to the pod — only works for containers that run as root. For a non-root container, capabilities added via securityContext.capabilities.add land in the bounding set but never in the effective/permitted set (Kubernetes has no ambient-capability support), so CAP_IPC_LOCK is silently inert.

Measured on Kubernetes 1.36 / containerd 2.3.3 / toolkit v1.19.1 (via GPU Operator), same image, same node:

container user	CapEff	ulimit -l	mlock test
uid 1000 + IPC_LOCK added	0000000000000000 (cap lost)	8192 KB	64 MiB → ENOMEM
uid 0 + IPC_LOCK added	a80465fb (cap effective)	8192 KB	1 GiB → OK

Since GPU images increasingly run as non-root, there is currently no per-runtime way to give GPU containers a usable memlock limit:

• LimitMEMLOCK=infinity on the containerd systemd unit works, but is node-wide — every container on the node gets it, not just GPU workloads.
• containerd's per-runtime base_runtime_spec could scope it to the nvidia runtime, but it replaces the entire default OCI spec (fragile across containerd upgrades), and the toolkit option that could set it — RUNTIME_CONFIG_OVERRIDE — has been non-functional since the v1.17 refactor (see "Related regression" below).

Proposal

Let nvidia-container-runtime — which is already an OCI-spec-rewriting shim — optionally inject POSIX rlimits into the spec:

[nvidia-container-runtime]
rlimits = ["memlock=unlimited"]        # or e.g. "memlock=1073741824:unlimited"

Implementation sketch (happy to submit a PR, it is small and self-contained):

1. api/config/v1/runtime.go: add Rlimits []string to RuntimeConfig.
2. internal/modifier/rlimit.go (new): an oci.SpecModifier that parses the entries into specs.POSIXRlimit and merges them into spec.Process.Rlimits (replace an existing entry of the same type, append otherwise). unlimited maps to math.MaxUint64 (RLIM_INFINITY in runc).
3. internal/modifier/factory.go: register the modifier for all runtime modes.
4. cmd/nvidia-ctk-installer/toolkit/toolkit.go: a --nvidia-container-runtime.rlimits flag (env NVIDIA_CONTAINER_RUNTIME_RLIMITS) added to the existing optionalConfigValues map so the toolkit container can render it into the installed config.toml.

With (4), GPU Operator users can enable it with no operator changes via toolkit.env:

toolkit:
  env:
  - name: NVIDIA_CONTAINER_RUNTIME_RLIMITS
    value: memlock=unlimited

Design points worth agreeing on up front:

• Not gated on device requests: unlike the feature-gated modifiers, this should apply to every container handled by the runtime when configured. RDMA-only pods (e.g. rdma/… resource, no GPU) may not set NVIDIA_VISIBLE_DEVICES yet still need the limit.
• Merge semantics: an entry configured here overrides a same-type rlimit already present in the spec (the config expresses administrator intent).
• Scope: this only works where nvidia-container-runtime is the runtime (RuntimeClass / default runtime). The nvidia-container-runtime-hook-only path (docker --gpus) cannot alter rlimits — a prestart hook runs after the process rlimits are set. Documenting that limitation seems acceptable.

Prior art / why this belongs in the NVIDIA runtime

• Kubernetes has declined pod-level rlimits for a decade (rlimit support kubernetes/kubernetes#3595); a KEP discussion (KEP-5758) only started in the v1.36 cycle and has not landed.
• containerd maintainers explicitly rejected a dockerd-style daemon-wide default-ulimits option (Is there a way to set the default-ulimits? containerd/containerd#3150); the containerd answer is per-runtime base_runtime_spec, which replaces the entire default spec and is painful to maintain across containerd upgrades.
• dockerd's default-ulimits exists but is daemon-wide, not GPU-scoped.

nvidia-container-runtime is the one component in this stack that (a) already rewrites the OCI spec per container, (b) is naturally scoped to exactly the workloads that need pinned memory, and (c) is centrally configurable across a fleet via the GPU Operator.

Related regression: RUNTIME_CONFIG_OVERRIDE is parsed but never applied

While investigating alternatives we found that the containerd --runtime-config-override flag (added in #497, worked through v1.16.x where AddRuntime ended with runtimeSubtree.applyOverrides(configOverrides...)) lost its consumer in the v1.17 tools/container → cmd/nvidia-ctk-installer refactor:

• v1.19.1: the flag is absent from the installer tree entirely.
• v1.20.0: the flag and the runtimeConfigOverride() helper are defined again in cmd/nvidia-ctk-installer/container/runtime/containerd/containerd.go, but nothing calls the helper — Setup → Configure(cfg) → UpdateConfig(cfg) → AddRuntime(name, path, setAsDefault) carries no override anywhere, so setting RUNTIME_CONFIG_OVERRIDE is a silent no-op.

Happy to split that into a separate issue if preferred.