Repository navigation
test(cloud): keep the Liquid template tests stable on a busy machine - #862
Merged
Merged
Conversation
The Liquid template tests read the real performance.now(), so on a busy machine they measured host load instead of the renderer. The default byte-cap test rendered 30,000 iterations against the real 1,000 ms budget and failed with render_timeout (#804). The captured-output and memory-limit tests fail the same way under heavier load, and several tests asserted wall-time upper bounds. Sixteen parallel runs of the file pinned to one CPU failed in every run. Every test in the file now runs on a fake clock. Time passes only where a test moves it, so only the limit under test can trip: - the default budget is proven exactly: 1,000 ms passes, 1,001 ms times out; - nested loops and where_exp expressions time out, which only checks inside the loop or expression can cause; - a slow final filter still times out; - 300 writes of 1,000 bytes render up to exactly the default byte cap. The byte-cap test and the where_exp range are smaller, so they cost little real time under load, and a missing where_exp check fails at once instead of scanning for about 25 seconds. Sixteen and thirty-two parallel runs pinned to one CPU now pass. A reintroduced quadratic output emitter (#759) is no longer caught. That is a timing property a deterministic test cannot observe: large renders made of many small writes would then fail with render_timeout, and no test would notice. No Fibel or cloud-dev skill change: the change is test-only, and runtime behavior and defaults are unchanged. Closes #804
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Outcome
The Liquid template tests in
packages/cloud/src/shared/template-rendering.test.tsno longer fail on a busy machine. Every test in the file now runs on a fakeperformance.now()clock, so only the limit under test can trip a render budget.where_expexpressions still time out, and only checks inside the loop or expression can cause that.Runtime behavior and defaults are unchanged.
Root cause
The tests read the real
performance.now(), so under host load they measured the machine instead of the renderer. The default byte-cap test rendered 30,000 iterations against the real 1,000 ms budget and failed withrender_timeout(#804). The captured-output and memory-limit tests fail the same way under heavier load, and several tests asserted wall-time upper bounds. Sixteen parallel runs of the file pinned to one CPU failed in every run.Trade-off: a reintroduced quadratic output emitter (#759) is no longer caught. That is a timing property a deterministic test cannot observe.
Docs
No Fibel or cloud-dev skill change: the change is test-only, and runtime behavior and defaults are unchanged.
Verification
bun test src/shared/template-rendering.test.tsinpackages/cloud: 28 pass.taskset -c 0) after the rebase ontomain: 16/16 pass (32 parallel runs also passed before the rebase).bun run checkafter the rebase.Closes #804