feat(inference): compare sticky least-loaded routing

[mikasenghaas]

Summary

• pin vllm-router to an experiment-only, checked-in Linux x86_64 wheel built from the exact head commit of feat(router): add sticky least-loaded session routing vllm-project/router#235, which adds sticky_least_loaded
• bump verifiers to current main (4cd87df5) for the latest bash-harness streaming and retry fixes
• add matched 200-step, four-node GLM-4.5-Air SWE configs using 1 trainer + 3 one-node inference replicas; model, training, evaluation, deployment, and routing-replay settings are identical, while job and sandbox identifiers keep the two arms isolated
• retain the single-trainer OOM-avoidance recipe: stateless sign-SGD, full CPU offload, 1,024-token LM-head chunking, activation checkpointing and offloading, and no optimizer checkpoint state
• isolate Triton, vLLM, and FlashInfer caches by user and SLURM job on node-local storage
• expand configured inference environment variables before starting in-process workers, preserving the concrete per-job paths from the launcher
• disable the optional fused allreduce/RMS compilation pass in both experiment arms to avoid concurrent FlashInfer JIT compilation during startup
• log both runs to the vllm-router-routing W&B project as consistent-hash and sticky-least-loaded
• release completed rollout sessions through the router /finish_session endpoint, automatically enabled for sticky least-loaded routing
• start both comparison arms from scratch; the experiment configs do not enable checkpoint resume

Session lifecycle

The pinned verifiers clients send each rollout trace UUID as X-Session-ID on every train and eval generation request. PrimeRL launches vLLM Router with --request-id-headers x-session-id, so router#235 can distinguish a new session from later turns in the same session.

This change additionally POSTs each completed trace ID to /finish_session. Cleanup is best-effort and bounded so router cleanup cannot discard a completed episode; cancelled or failed environment requests without returned trace IDs fall back to the router TTL.

Router pin

This experiment does not require an upstream or Prime router release. The repository vendors an 8.3 MB CPython abi3 Linux x86_64 wheel built directly from router#235 commit 6422afa, and uv installs it from deps/wheels. The wheel SHA-256 is 7c3047e87ff5a0a02f3dfadd6f8012ceb85aabd8f0c2a5fe25ba88ee61b19f83.

The pin temporarily moves from PrimeIntellect-ai/router v0.2.0 wheels to the upstream PR package version 0.1.15; this experiment uses the regular, non-P/D router path.

Validation

• uv lock --check
• full frozen uv sync --all-extras --all-packages --dry-run resolves the local wheel
• installed the wheel into an isolated target and verified the router CLI exposes sticky_least_loaded and --request-id-headers
• uv run ruff format ... and uv run ruff check ... on changed Python and test files
• uv run --no-sync pytest tests/unit/orchestrator/test_clients.py tests/unit/test_configs.py::test_sticky_router_auto_enables_session_release (12 passed)
• both corrected configs pass fresh-run uv run --no-sync rl @ <config> --dry-run
• resolved plans use 200 steps, one trainer node, three one-node inference replicas, the memory-safe trainer settings, their intended router policy, --request-id-headers x-session-id, and finish_sessions=false for consistent hash / true for sticky least-loaded
• verified concrete per-job cache expansion and staggered startup for both comparison arms

[mikasenghaas]

bump vf to latest main to grab the latest fix to bash harness

[mikasenghaas]

Addressed the review in caceead: bumped verifiers to current main (4cd87df5), including the latest bash-harness fixes. The lock now carries prime-runs 0.1.2, both configs dry-run successfully, and the focused suite passes (12 tests).