Skip to content

test(e2e): let a step redact its output with its own patterns - #912

Merged
wan9chi merged 1 commit into
mainfrom
test/e2e-step-redact
Oct 30, 2026
Merged

wan9chi merged 1 commit into
mainfrom
test/e2e-step-redact

Conversation

@wan9chi

@wan9chi wan9chi commented Oct 30, 2026

Copy link
Copy Markdown
Member

Motivation

Later PRs in this stack test clean checkouts, which remove several inputs at once. vt reports whichever mismatch its parallel validation finds first, so the step's cache miss: … line changes from run to run. Making vt compute every mismatch only to stabilize tests isn't worth it, so a step needs its own redaction.

Changes

A table step takes redact, a map from regex to replacement, applied in key order to that step's output after the global redactions. An invalid regex fails when the fixtures load. step_redact shows that a step's patterns touch only its own output.

Part of a stack that lets automatic tracking cache tasks which read their own outputs and list the folders holding them, so they also hit on a clean checkout.

🤖 Generated with Claude Code

@wan9chi
wan9chi added this pull request to stack #920 October 30, 2026 03:17
Some steps remove several inputs at once, and vt reports whichever
mismatch its parallel validation finds first, so the step's "cache miss"
line changes from run to run. A new `redact` step key maps regexes to
replacements; the harness applies them in key order to that step's
output after the global redactions, and an invalid regex fails when the
fixtures load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  -0.72%  [ -6.24% ..  +5.77%]  overhead   +59.55%
dynamic/access             change  -0.34%  [ -5.31% ..  +6.70%]  overhead   +24.21%
dynamic/access-relative    change  -3.18%  [-17.93% ..  +2.90%]  overhead   +71.18%
dynamic/access-contended   change  -1.20%  [ -5.34% ..  +6.61%]  overhead   +25.13%
static/launch              change  +0.49%  [ -5.05% ..  +6.76%]  overhead  +184.86%
static/access              change  +0.53%  [ -2.00% ..  +8.16%]  overhead +1457.79%
static/access-relative     change  -0.29%  [ -2.16% ..  +1.86%]  overhead +1964.59%
static/access-contended    change  -0.54%  [ -4.49% ..  +1.21%]  overhead +1722.36%

macos

dynamic/launch             change  +0.35%  [ -5.29% ..  +4.88%]  overhead  +240.74%
dynamic/access             change  -0.35%  [ -7.61% ..  +8.51%]  overhead    +2.79%
dynamic/access-relative    change  -0.10%  [ -7.96% ..  +4.28%]  overhead  +269.30%
dynamic/access-contended   change  +2.89%  [ -6.62% .. +35.49%]  overhead    +5.27%

windows

dynamic/launch             change  -0.86%  [ -4.18% ..  +2.62%]  overhead   +24.91%
dynamic/access             change  +0.00%  [ -0.55% ..  +0.73%]  overhead    +1.51%
dynamic/access-relative    change  +0.18%  [ -0.54% ..  +0.72%]  overhead    +1.45%
dynamic/access-contended   change  +0.17%  [ -2.53% ..  +1.59%]  overhead    +1.24%

@wan9chi
wan9chi force-pushed the test/e2e-step-redact branch from aeaa34c to 5ef290d Compare October 30, 2026 03:19
@wan9chi
wan9chi merged commit dc6fdaf into main Oct 30, 2026
20 checks passed
@wan9chi
wan9chi deleted the test/e2e-step-redact branch October 30, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant