Add Modal execution and private CLaaS model serving

[kfallah]

CLaaS needs to execute the same upstream veRL job remotely and retain exact policy identity during inference. This PR adds optional Modal execution under exp/optimize/claas/backends/modal/ and private vLLM sampling/lifecycle under exp/runtime/claas/serving/. It stacks on #985 and delegates training to the published veRL TrainingWorker and FSDP engine.

Behavior

• Modal configuration, remote transport, and SDK app construction have separate modules. The trusted local operator chooses the worker image and existing Volume; neither is selected by remote job payloads. These use finite startup/runtime limits, an authorized cost ceiling, no automatic retries, and one single-use container. A missing Modal device environment selects GPU 0; explicit device bindings are preserved.
• Volume commit precedes the completion receipt. Native actor optimizer/scheduler/RNG state, teacher model state, and serving LoRA exports transfer under a byte ceiling and hash-bound manifest. Changed or incomplete files fail before publication.
• The shared veRL worker runs through exp.optimize.claas.backends.verl.worker. Modal owns execution and artifact transfer; veRL owns the optimizer and native checkpoints. Configuration remains SDK-free, and ordinary gateway imports do not load Modal or training dependencies.
• Serving contracts, decoders, vLLM sampling, launch configuration, and lifecycle have dedicated modules. The shared revision registry remains at the CLaaS runtime root. There are no compatibility modules for the previous import paths.
• Sampling retains original prompt/generated token IDs, aligned behavior log probabilities, and model/tokenizer/policy identity. Qwen3.5 XML, Hermes JSON, and text decoders reject incomplete tool syntax.
• Admission drains before sleep or LoRA replacement. Candidate evaluation runs while admission is closed; the active pointer is published before requests resume. The externally managed vLLM process stays alive across sleep/wake, while the training subprocess is rebuilt per batch. Resident training and overlapping updates are not implemented.

Validation

• The updated training/capture foundation includes current main and the bounded catalog helper extraction, with flagship 0.7.84 and native extension 0.3.75. The extension was rebuilt in this PR's environment.
• Whole-repository Ruff, format, and ty pass. The focused training, Modal, serving, catalog, import-boundary, and layout suite passes 149 tests, with one explicit CUDA integration skip.
• Tests exercise loopback HTTP sampling, decoding, drain/load/resume, bounded transfers, native optimizer-file tampering, cancellation, costs, and real Modal SDK app construction/local invocation without cloud calls. The renamed worker executes its module entrypoint in a regression test.
• CPU tests exercise upstream configuration, output preparation, objectives, and Qwen LoRA construction; they do not prove a CUDA optimizer update. The opt-in CUDA training/resume/export test, paid Modal execution, Qwen memory/throughput, and end-to-end vLLM adapter serving remain unrun. No deployment, provider call, or GPU allocation was performed. vLLM sleep releases model memory, not container billing.
• Every current-head CI check passes, Greptile reports an actual 5/5 on 4b17926d80095f0253a152352c289df4eac69b0e, and no review threads remain unresolved. An inherited replay-accounting finding was scoped out with exact equality to unchanged main/base code; no accounting behavior was changed. The earlier image-attestation advisory was rebutted using the operator/request authority boundary and dismissed by the security scan.

Base: codex/claas-05-training. Native checkpoint schema 2 and immutable artifact ownership come from #985. Do not merge.

[kfallah]

@greptileai review

[kfallah]

@greptileai review

[kfallah]

@greptileai review

[kfallah]

@greptileai review. The current head separates SDK-free Modal configuration, preserves explicit training lineage, and verifies downloaded checkpoints against the exact submitted job. Please review the latest diff.

[kfallah]

@greptileai review. Latest head fixes named-lineage checkpoint download and reloads LoRA after sleep or controller restart; focused runtime regression tests and full lint/type gates pass.

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge from a code-review perspective, with no outstanding findings at the reviewed head.

Summary

This PR adds optional Modal-backed CLaaS training and private vLLM serving while preserving checkpoint, model, tokenizer, and policy identity.

• Defines bounded, single-use Modal execution with explicit cost authorization and verified checkpoint transfer.
• Adds exact-token vLLM sampling, model-family completion decoders, and fail-closed evidence validation.
• Adds serving lifecycle controls for admission, draining, sleep/wake, candidate evaluation, and LoRA replacement.
• Introduces the optional claas-modal dependency group and associated tests.

Diagram

sequenceDiagram
    participant O as CLaaS Orchestrator
    participant M as Modal Worker
    participant V as Durable Volume
    participant L as vLLM Lifecycle
    participant S as Private vLLM Server

    O->>M: Submit one authorized TrainingJob
    M->>M: Run shared veRL worker
    M->>V: Commit native checkpoint and LoRA artifacts
    V-->>M: Commit complete
    M-->>O: Return completion receipt
    O->>V: Download manifest-listed files
    O->>O: Verify hashes, scope, lineage, and policy revision
    O->>L: Pause admission and drain requests
    L->>S: Load candidate LoRA
    O->>L: Evaluate candidate while admission is closed
    O->>O: Durably publish selected revision
    O->>L: Resume admission

Loading

_{Reviews (6) · Last reviewed commit: "Refresh Modal execution and serving on t..."}

[kfallah]

@greptileai review. Addressed incomplete Hermes tool syntax in 5f9a6ea with regressions. Please re-review current head.

[kfallah]

@greptileai review

[kfallah]

@greptileai Please review current head 66fe0e4. Modal now owns backends/modal/{backend,app,configuration}; runtime policy serving owns serving/{contracts,decoding,lifecycle,vllm,configuration}. Imports, sibling tests and package boundaries use the new paths without shims. Whole-repository Ruff, format and ty pass; 114 focused tests passed with one CUDA skip, then 31 affected tests passed after import cleanup. No paid execution or deployment occurred.

[kfallah]

@greptileai Review this stable head after carrying the verified veRL module entrypoint through the Modal stack. Whole-repository Ruff, format and ty pass; all 29 final worker/contract/Modal definition and configuration tests pass, including optional-import isolation. Package structure is frozen pending CI and review.

[kfallah]

@greptileai review

[superagent-security]

Superagent found 1 security concern(s).

[kfallah]

@greptileai review

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[kfallah]

The replay-repair concern is in unchanged base code, outside the current PR diff. At this head (4b17926d80095f0253a152352c289df4eac69b0e), all three relevant files are byte-identical to origin/main (e0a141f2a) and unchanged against this PR's training base codex/claas-05-training:

• exp/runtime/gateway/native/src/waterfall.rs: Git blob aed8c4cacc1bf198a54d55e2a37b45e1b3ef19a7.
• exp/runtime/gateway/native/src/replay_repair.rs: Git blob 719a2d23262301f0cfe503b9257300ecaf81a9e1.
• exp/runtime/gateway/native/src/relay.rs: Git blob 72b6abed8866578530258a15ab9f7d78b1e28a98.

git diff codex/claas-05-training...HEAD -- for these paths is empty. They arrived through the main/base refresh; this PR changes optional Modal execution and private CLaaS serving, with no replay-repair settlement changes or new call path to this logic. The known-usage concern warrants a separate base-code investigation, but it is not introduced or changed by the current PR. Please reassess the current diff and update the score accordingly. No accounting behavior or test was modified to bypass this finding.

@greptileai

[kfallah]

@greptileai review

[kfallah]

@greptileai review

[kfallah]

Superseded by the independent asynchronous learning stack #1018 through #1024. The replacement removes mandatory production-gateway coupling and separates finite burst training, resident full runs, and thin infrastructure hosting. This branch is preserved; no merge or branch deletion is performed.