Compose optional traffic learning above generic CLaaS

[kfallah]

Compose retained gateway traffic into the generic CLaaS learning interface without teaching the core about traffic mining or simulation. This optional workflow owns source selection, failure mining, scenario synthesis, world-model environments, and provider-backed evaluation.

Traffic sources live under exp/optimize/workflows/traffic_learning/sources/. Partition and evaluation files reject symlinks and stay within a 64 MB read/write bound. Durable held-out reservations exclude linked responses before selection; pending external requests block learning after restart. Preparation holds the application lock and recomputes the complete bounded provider plan before credentials or paid calls. Binary, scalar, and text feedback remain optional; private feedback stays outside policy-visible messages. Practice validates every input label and selects at most the latest 128 matching records; world receipts retain generic limit/error termination reasons.

This is a reusable workflow. It contains no committed benchmark-specific application, dataset, or scorer. Authored environments can call the generic learning API directly and do not need this package.

This incremental PR follows the generic-core PR and is one of four review units replacing the former large #991 diff. Rust gateway coordination and CLI composition follow separately. Traffic synthesis, practice, and evaluation share one owned blocking-call adapter; cancellation joins dispatched work before returning, including repeated cancellation and eventual provider failure. No merge is requested.

Validation: whole-repository Ruff, format, and ty checks pass. 208 focused traffic-workflow, simulation, generic lifecycle/evaluation, feedback, repository-layout, import-boundary, and dependency-surface tests pass. These exercise held-out isolation, source feedback provenance, retention, locked preparation, bounded provider reservations, fixture-driven complete learning cycles, teardown-failure evidence, rollback recovery, generation caps, and native-extension-absent SQLite readers. All added callables have docstrings and Python modules have sibling tests. The explicit optional backend loader avoids requiring the Modal SDK for local execution. All current-head CI checks pass, Greptile is 5/5, and every review thread is resolved. No paid provider calls, real GPU execution, or measured learning improvement is claimed.

[greptile-apps]

Confidence Score: 5/5

The reviewed traffic-learning changes appear safe to merge, with no outstanding blocking or non-blocking findings.

Summary

The PR composes retained gateway traffic into the generic CLaaS lifecycle while keeping traffic mining, simulation, provider budgeting, and synthetic evaluation outside the core.

• Adds durable source buffering, held-out reservations, stable partitioning, retention, and bounded state handling.
• Adds optional traffic-derived scenario synthesis, private source-feedback grounding, world-model practice, and provider-backed paired evaluation.
• Reuses one owned blocking-call adapter so cancellation waits for dispatched provider work.
• The three previous findings are resolved in the current code: partition and evaluation state reject links and oversized files, feedback selection retains at most the latest 128 validated matches, and nested world receipts preserve limit/error termination reasons.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Retained gateway traffic] --> B[Bounded source buffer]
    B --> C[Stable fit / held-out partition]
    C --> D[Fit source mining and synthesis]
    D --> E[Generic CLaaS practice lifecycle]
    C --> F[Frozen held-out evaluation]
    F --> G[World-model and judge evaluation]
    E --> H[Candidate policy]
    H --> G
    G --> I[Verified paired evaluation report]

Loading

_{Reviews (5) · Last reviewed commit: "Integrate current main and refreshed CLa..."}

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[kfallah]

Addressed the current review finding in the shared core: caller cancellation during environment cleanup now propagates even when execution already failed. The regression reproduces an execution error followed by cancellation of a pending close, and verifies both CancelledError propagation and the persisted partial receipt with both error types. The fix is carried from the core PR into this stack head; no training batch is returned after either failure.

[kfallah]

Addressed the latest review finding by integrating core commit 6cbce5b. An ordinary backend-close error now preserves any earlier training failure, while a new cancellation during close still propagates. Regression tests cover ValueError plus OSError and TimeoutError plus OSError: the original training exception is re-raised, failure_type retains the training type, cleanup_failure_type separately records OSError, and admission stays paused because backend cleanup is unproven. Whole Ruff/format/ty and 208 focused tests pass on this stack head.

[kfallah]

Superseded by the independent asynchronous learning stack #1018 through #1024. The replacement removes mandatory production-gateway coupling and separates finite burst training, resident full runs, and thin infrastructure hosting. This branch is preserved; no merge or branch deletion is performed.