Repository navigation
Conversation
Signed-off-by: wsj <279978307+wsj040303@users.noreply.github.com>
|
MLCommons CLA bot: |
|
@wsj040303 can you please sign the CLA.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #537 +/- ##
=======================================
Coverage ? 81.37%
=======================================
Files ? 157
Lines ? 22472
Branches ? 0
=======================================
Hits ? 18286
Misses ? 4186
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Signed-off-by: wsj <279978307+wsj040303@users.noreply.github.com>
|
The audit failure is caused by the older dependency pins, which upstream has already fixed in #542. I merged main at 93ec321 into this branch without changing the timing fix. Locally, the original dependency audit failed; after the merge, the audit passes with the existing #520 exception, 2,259 unit tests pass (6 skipped), and all pre-commit hooks pass. The branch now points to 238cfe5. The auto-review log separately reports HTTP 403 when requesting reviewers, and the CLA check remains outstanding; this dependency update does not resolve those checks. |
What does this PR do?
Use the receiver's
now_nsfor COMPLETE events, matching RECV_FIRST, RECV_NON_FIRST and ERROR events. A single socket drain can decode a QueryResult before the session handles its preceding chunks. Using its decode-timecompleted_atthen stamps completion before the first chunk and produces negative TPOT.The QueryResult timestamp and wire format are unchanged. The session continues reading the monotonic clock once per response. Completion now includes the same receiver-handling boundary that chunk events already use.
Type of change
Related issues
Closes #533.
Testing
Four batched-response regressions (one/multiple chunks, successful/failed terminal results) fail on the original source and pass after the fix. The existing metadata/turn test now checks the shared receiver timestamp.
pytest tests/unit/load_generator/test_async_session.py -q --no-cov: 65 passed.LC_ALL=C.UTF-8 pytest -m unit -q: 2259 passed, 6 skipped. The initial run's one failure expected an English shell error under the host's Chinese locale; the C-locale rerun passes.echo-serveralias. The alias was removed after testing. An inherited SOCKS proxy initially prevented the tokenizer subprocess from starting; the successful run removed proxy variables for local testing.pre-commit run --all-filespassed, including mypy (335 files), Ruff, formatting, license and generated-template checks.git diff --checkpassed.External serving and optional hardware/slow suites were not run. No model weights or GPU are required for this client event-timing fix.
Checklist
Dependency CI follow-up (2026-10-23)
Merged upstream main at 93ec321 (#542) to resolve the dependency audit failure. The original timing fix and its regression tests are unchanged. Locally: dependency audit passes with the existing #520 exception, 2,259 unit tests pass (6 skipped), and all pre-commit hooks pass. New head: 238cfe5. The CLA and reviewer-request permissions remain separate checks.