Skip to content

fix: clear the Changed bit when resetting api test flags, and shuffle in CI - #114

Merged
leet-c1 merged 1 commit into
mainfrom
fix/test-shuffle-isolation
Sep 3, 2026
Merged

fix: clear the Changed bit when resetting api test flags, and shuffle in CI#114
leet-c1 merged 1 commit into
mainfrom
fix/test-shuffle-isolation

Conversation

@leet-c1

@leet-c1 leet-c1 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

go test -shuffle failed on 4 of 5 seeds.

resetAPICmdFlags emptied --query/--header with pflag.SliceValue.Replace
but never cleared pflag's Changed bit. repeatableStringFlag rejects a flag
that was set while holding no values, so a reset flag read as "the user passed
an empty value"
and leaked

flag --query requires a non-empty value for every occurrence

into whichever api test happened to run next. #111 introduced the
sensitivity; the reset predates it.

The fix

resetCmdFlags already handled both traps — Set appending on a StringArray,
and the Changed bit — so resetAPICmdFlags delegates to it instead of
carrying a second implementation that drifted. Net −25/+11.

Delegating also covers the persistent flags cobra merges into a subcommand's
flag set, which fixes a second leak: one api test passes --dry-run through
rootCmd and nothing cleared it afterward.

Gated, so it stays fixed

CI now runs a shuffled pass. -shuffle=on rather than a fixed seed: the
permutation is derived from the test list, so adding or renaming any test
rerolls a fixed seed anyway, and a failure prints its seed to replay with.

Verified the gate detects the bug rather than decorating the workflow — with
the old reset restored, -shuffle=on fails 5 of 6 runs; with the fix, 0 of 6.
Seeds 1–50 and -race are clean. The diagnosis is complete rather than one
symptom of several: with only the old api_test.go restored, 6 of 7 seeds fail
and every failure carries the same --query message.

🤖 Generated with Claude Code

go test -shuffle failed on 4 of 5 seeds. resetAPICmdFlags emptied --query and
--header but left pflag's Changed bit set, and repeatableStringFlag rejects a
flag that was set while holding no values — so a reset flag read as "the user
passed an empty value" and leaked "requires a non-empty value" into whichever
api test ran next. The change in #111 introduced the sensitivity; the reset
predates it.

resetCmdFlags already handled both traps, so resetAPICmdFlags delegates to it
rather than keeping a second implementation that drifted. Delegating also
covers the persistent flags cobra merges in, which fixes a second leak: one
api test passed --dry-run through rootCmd and nothing cleared it.

CI runs a shuffled pass. -shuffle=on rather than a fixed seed: the permutation
is derived from the test list, so any added or renamed test rerolls a fixed
seed anyway, and a failure prints its seed to replay with.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@leet-c1
leet-c1 merged commit 01f86de into main Sep 3, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant