buckaroo_state_change requests are not superseded; rapid typing wedges the server

[paddymul]

Summary

The server processes incoming buckaroo_state_change messages strictly in arrival order with no supersede / debounce. Typing into the search box (one state-change per keystroke) therefore triggers N sequential full pipeline runs even though the client only needs the response to the last one — N-1 of them are dead work.

Empirically on the Boston restaurant data (883K rows, xorq backend):

Test	Total time	Per-keystroke
5 rapid state_changes typing "PIZZA"	48 s	~9.5 s each
Last state_change response time	~9.5 s	(after 5×9.5 = 47.5 s of serialized compute)

Server log confirms 5 sequential entries, each ~9s:

[tperf] state_change: total=9694.9ms
[tperf] state_change: total=9536.2ms
[tperf] state_change: total=9360.5ms
[tperf] state_change: total=8902.4ms
[tperf] state_change: total=8899.0ms

(pandas is faster per-state-change but has the same serialization problem: 5 keystrokes = 5 × ~1.5s = 7.4 s on the test driver.)

Reproduction

ws = await tornado.websocket.websocket_connect("ws://127.0.0.1:8700/ws/boston")
await ws.read_message()  # initial
for term in ("P", "PI", "PIZ", "PIZZ", "PIZZA"):
    ws.write_message(json.dumps({"type":"buckaroo_state_change",
        "new_state":{"quick_command_args":{"search":[term]},
            "post_processing":"","cleaning_method":"","df_display":"main",
            "show_commands":False,"sampled":False,"search_string":term}}))
# 5 frames each block ~9s on xorq — total 45-50s

Why it matters

The grid is unresponsive for the full duration. From the user's perspective, "search" looks like it doesn't work — they typed PIZZA and got stale "P"-filter results for nearly a minute. The rows-first spike (#787) makes this worse, not better — each state_change now produces two frames and the spike's 10ms inter-phase delay is meaningless when the actual compute is 9 seconds.

The scope-merged-SD cache from #785/#789 helps slightly (the raw + clean scope SDs are cached after the first state_change), but the filt-scope SD always misses because the filter changed.

Suggested fix

Pick the simplest cancel-previous strategy:

Drop pending in-flight state-changes when a new one arrives for the same session. Track a pending_state_change_token per session. The handler bumps the token on every incoming state_change; the in-flight compute checks the token after each phase boundary (_compute_processed_result, _get_summary_sd, _populate_sd_cache) and short-circuits if its token is stale.

Mechanically:

def _handle_buckaroo_state_change(self, new_state):
    session = ...
    token = session.state_change_token = session.state_change_token + 1

    # ... existing propagate work ...

    if session.state_change_token != token:
        log.info("state_change superseded; aborting compute session=%s", self.session_id)
        return
    # ... extract + broadcast ...

Coupled with: don't apply the new state to session.buckaroo_state until the response is being broadcast, so an aborted state-change doesn't leave the session in a partially-applied state.

Test plan

• Failing-test commit: WS test that fires 5 state_changes back-to-back, waits 15 s total, asserts the final response time is under 12 s. Should fail today on xorq backend (would be ~48 s total) and on pandas (~7 s sequential).
• Fix commit: token + abort check at phase boundaries.
• Regression coverage: existing test_state_change_* in test_load_expr.py continue to pass; add one new test for "5 state_changes resolve in the time of 1" with the xorq backend.

Related

• Hard upper bound on per-state-change compute is being addressed by other work (spike(rows-first): two-message state_change protocol behind env gate #787 spike, the per-column query loop in XorqStatPipeline). Supersede is orthogonal — it stops unnecessary work, doesn't speed up the necessary compute.
• Discovered during a cross-backend stress test (pandas / lazy-polars / xorq).

🤖 Generated with Claude Code