Skip to content

Fix flaky unit tests by replacing fixed sleeps with vi.waitFor#359

Merged
umair-ably merged 1 commit intomainfrom
fix-flaky-connection-lifecycle-subscribe-tests
Apr 20, 2026
Merged

Fix flaky unit tests by replacing fixed sleeps with vi.waitFor#359
umair-ably merged 1 commit intomainfrom
fix-flaky-connection-lifecycle-subscribe-tests

Conversation

@umair-ably
Copy link
Copy Markdown
Collaborator

Summary

Four unit test files used fixed setTimeout waits to synchronise with async work inside runCommand. When the work took longer than the hard-coded delay, the assertion ran against stale state and the test failed intermittently.

Replaced those fixed sleeps with vi.waitFor(() => expect(...)), which polls the condition and eliminates the race.

Files changed:

  • test/unit/commands/logs/connection-lifecycle/subscribe.test.ts — 5 tests scheduled setTimeout(..., 50) to fire the message callback, but channel.subscribe() wasn't always captured within 50 ms, so the callback check saw null and silently dropped the message. (This was the actual failure that triggered the investigation.)
  • test/unit/commands/interactive-terminal-behavior.test.ts — 2 tests slept 100 ms after rl.emit("line", ...) then asserted on async prompt/close state.
  • test/unit/commands/spaces/locations/subscribe.test.ts — 2 tests slept 50 ms after starting the command before invoking the captured subscribe handler.
  • test/unit/hooks/interactive-did-you-mean.test.ts — slept 30 ms after the hook returned before asserting on readline state restoration.

Test plan

  • pnpm exec eslint . — 0 errors (7 pre-existing warnings)
  • pnpm test:unit — 2292 passed, 0 failed
  • Targeted re-run of the four modified files — 42 passed

These tests used `setTimeout` (either scheduling a callback to fire
during a `runCommand`, or a fixed `await new Promise(setTimeout, N)`)
to wait for async work to complete before asserting. When the work
took longer than the timeout the assertion ran against stale state
and the test failed.

Switched to `vi.waitFor(() => expect(...))` which polls until the
condition holds, removing the race:

- logs/connection-lifecycle/subscribe: 5 tests fired the message
  callback at +50ms from test start, but the command's
  `channel.subscribe()` was not always captured by then.
- interactive-terminal-behavior: two tests slept 100ms after
  `rl.emit("line", ...)` then asserted on async prompt/close state.
- spaces/locations/subscribe: two tests slept 50ms after starting
  the command before invoking the captured subscribe handler.
- hooks/interactive-did-you-mean: slept 30ms after the hook
  returned before asserting on readline state restoration.
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 20, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cli-web-cli Ready Ready Preview, Comment Apr 20, 2026 7:13pm

Request Review

@claude-code-ably-assistant
Copy link
Copy Markdown

Walkthrough

Four unit test files used fixed setTimeout delays to synchronize with async work inside runCommand. When the async work exceeded the hard-coded delay, assertions ran against stale state, causing intermittent (flaky) failures. This PR replaces all fixed sleeps with vi.waitFor(() => expect(...)), which polls until the condition holds and removes the timing race entirely.

Changes

Area Files Summary
Tests test/unit/commands/logs/connection-lifecycle/subscribe.test.ts 5 tests: replaced setTimeout(..., 50) callbacks with vi.waitFor; the old delay wasn't enough to guarantee channel.subscribe() was captured before the message callback fired
Tests test/unit/commands/interactive-terminal-behavior.test.ts 2 tests: replaced 100 ms post-rl.emit sleeps with vi.waitFor assertions on async prompt/close state
Tests test/unit/commands/spaces/locations/subscribe.test.ts 2 tests: replaced 50 ms pre-handler-invocation sleep with vi.waitFor
Tests test/unit/hooks/interactive-did-you-mean.test.ts 1 test: replaced 30 ms post-hook sleep with vi.waitFor on readline state restoration

Review Notes

  • No production code changes — all 4 modified files are test files; zero risk to runtime behavior.
  • Net deletion (+127 / −132): the vi.waitFor form is marginally more verbose in a few spots but eliminates the arbitrary timeout values entirely.
  • No new dependenciesvi.waitFor is already available from Vitest.
  • Flakiness root cause confirmed: the logs/connection-lifecycle/subscribe tests were the reported failure; the other three files were found during the same investigation and fixed proactively.
  • No migration, deployment, or breaking-change considerations.

@umair-ably umair-ably requested a review from AndyTWF April 20, 2026 19:14
@umair-ably umair-ably merged commit 759fc85 into main Apr 20, 2026
13 checks passed
@umair-ably umair-ably deleted the fix-flaky-connection-lifecycle-subscribe-tests branch April 20, 2026 19:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants