Repository navigation
Perf regression: bun/hosted wall +110% (1169ae68, #472) #578
Description
[agent] Bench: bun/hosted and bun/rescan take about 2.1x as long since #472 (1169ae6, "Fix Bun/vlt bundled copies left unpatched"). Wall time is up 105–117% and CPU time 88–96%. The request count is unchanged (127).
Same-machine interleaved compare (socket-patch-bench, PR #485 suite)
| round | base | scenario | base wall | head wall | Δ wall [95% CI] | Δ CPU | Δ RSS | requests |
|---|---|---|---|---|---|---|---|---|
| daily, 15 pairs + 10 confirm | 6e7ef748 (main −24h) |
bun/hosted |
144.2 ms | 302.7 ms | +116.6% [+103.7, +129.7] | +96.2% | +0.0% | 127 |
| daily, 15 pairs + 10 confirm | 6e7ef748 |
bun/rescan |
132.3 ms | 277.8 ms | +113.2% [+101.8, +123.0] | +90.0% | +0.0% | 127 |
| confirm, 25 pairs + 20 | 6e7ef748 |
bun/hosted |
144.6 ms | 286.5 ms | +105.5% [+95.7, +112.7] | +87.5% | 127 | |
| confirm, 25 pairs + 20 | 6e7ef748 |
bun/rescan |
130.2 ms | 278.0 ms | +113.7% [+107.0, +117.5] | +89.9% | 127 | |
| bisect, 11 pairs | cbf1f748 (parent of #472) |
bun/hosted |
142.0 ms | 295.5 ms | +109.5% [+103.6, +117.5] | +94.8% | 127 |
An A/A check on the same runner (head against a copy of itself, 3 scenarios) found no regression, so the runner was not too noisy.
- head:
1169ae68(main), with the bench suite from Add ascanbenchmark suite and a CI performance gate #485 merged locally - culprit: 1169ae6 / Fix Bun/vlt bundled copies left unpatched (#469, #471) #472. It is the only commit in
6e7ef748..1169ae68that touches Bun code, and the parent→Fix Bun/vlt bundled copies left unpatched (#469, #471) #472 A/B reproduces the full slowdown.
Hot spot
Callgrind on bun/hosted (head) puts 62% of all instructions in socket_patch_core::vendor::bun_lock_text::is_bundled_entry, nearly all of it inside serde_json::from_str::<Value>.
rewrite_bun_lock (crates/socket-patch-core/src/patch/redirect/mod.rs, the for dep in &npm { for entry in &entries { loop) calls is_bundled_entry(entry) before it checks the spec:
if is_bundled_entry(entry)
&& (spec == target_spec
|| spec == url_spec
|| is_prior_hosted_bun_spec(&spec, &fname, &dep.artifact_url))That parses every entry's metadata object as JSON once per patched dependency: 60 patches × 3000 entries is 180k JSON parses, where the old loop did string compares. Proposed fix (not applied):
- swap the operands so the cheap spec match runs first and
is_bundled_entryonly runs on a matching entry; or - compute
bundledonce per entry before thedeploop.
The behavior stays the same either way. bundled_matches in vendor/bun_lock.rs has the same shape (it filters on is_bundled_entry first) on the vendor path.
Repro
export CARGO_PROFILE_PERF_INHERITS=release CARGO_PROFILE_PERF_LTO=thin CARGO_PROFILE_PERF_STRIP=none
git worktree add /tmp/base cbf1f748 && (cd /tmp/base && CARGO_TARGET_DIR=/tmp/tb cargo build --locked --profile perf -p socket-patch-cli)
# on a checkout of #485 merged with main:
cargo build --locked --profile perf -p socket-patch-cli -p socket-patch-bench
target/perf/socket-patch-bench compare --base /tmp/tb/perf/socket-patch --head target/perf/socket-patch -f '^bun/'Runner: 4 vCPU (nproc = 4), Intel(R) Xeon(R) Processor @ 2.10GHz, cloud sandbox.
Generated by Claude Code
Backlog review — 2026-10-08
Priority: P1 → P2. The Bun regression is measurable and worth fixing, but the supplied subsecond benchmark does not establish a P1 user-facing performance outage.
Activity
mikolalysenko commentedon Oct 2, 2026
[agent] Triaged as
priority:p1(Bun, npm-family). Confirmed onmain(6cd3754):rewrite_bun_lock(patch/redirect/mod.rs:3723) still callsis_bundled_entrybefore the spec compare, andvendor/bun_lock.rs:1048filters the same way. Related to #579 (same culprit, #472) but a different hot spot and fix, so not clustered. No open PR addresses it.Generated by Claude Code
mikolalysenko commentedon Oct 3, 2026
[agent] Bench: still regressed on
main045d7ec7(2026-10-03 run).2463257a(#277)bun/hosted2463257a(#277)bun/rescan1169ae68(#472)bun/hostedNone of the commits since #472 touched the Bun rewrite:
rewrite_bun_lockstill callsis_bundled_entry(entry)before it compares the spec. The A/A check on this runner was clean. The newbun-isolated/*scenario in #667, which uses Bun's isolated.bunstore, runs at about the same 320 ms, so it most likely has the same hot spot.Runner: 4 vCPU, Intel(R) Xeon(R) Processor @ 2.80GHz, cloud sandbox.
Generated by Claude Code
mikolalysenko commentedon Oct 4, 2026
[agent] Bench: still regressed on
main045d7ec7(2026-10-04 run). main hasn't moved since the 10-03 run, so there was no daily A/B.2463257a(#277)bun/hosted2463257a(#277)bun/rescanA/A on the same runner was clean (npm/hosted −1.0%, vlt/hosted +0.9%, poetry/hosted −0.4%). Runner: 4 vCPU Intel Xeon @ 2.10GHz.
For reference, the matching vlt regression (#579) was closed by a maintainer as an accepted cost of scanning previously skipped copies. If the same reasoning applies to Bun's bundled-entry check, this can be closed the same way. The bench steward won't close it, since a human triaged it.
Generated by Claude Code
mikolalysenko commentedon Oct 5, 2026
[agent] Bench: still regressed on
main045d7ec7(2026-10-05 run). main hasn't moved since 10-02, so there was no daily A/B.2463257a(#277)bun/hosted2463257a(#277)bun/rescanThe A/A check (npm/hosted, uv/hosted, poetry/hosted) was clean: +1.6%, +0.8% and +0.2%. Runner: 4 vCPU Intel Xeon @ 2.80GHz. The hot spot is unchanged (see earlier comments). No new data argues either for or against accepting this the way #579 was. That decision is still with a human.
Generated by Claude Code
mikolalysenko commentedon Oct 6, 2026
[agent] Bench: run 2026-10-06. The regression is still present on main
9c43dfc9.This run used a same-machine interleaved weekly A/B against
2463257a(#277) on a 4 vCPU Xeon @ 2.10GHz:bun/hostedbun/rescanThe daily A/B against
045d7ec7was flat: bun/hosted +0.8%, bun/rescan +5.4%. None of the 74 commits since the last run touched this regression. Requests are unchanged at 127.Generated by Claude Code
mikolalysenko commentedon Oct 7, 2026
[agent] Bench: run 2026-10-07. Still regressed on
main9c43dfc9. main hasn't moved since the 10-06 run, so there was no daily A/B.Weekly same-machine
compare(base2463257a→ head9c43dfc9, 4 vCPU Xeon @ 2.80GHz):bun/hostedbun/rescanA/A on this runner was clean. Nothing has changed since the last comment: the culprit is still #472, and accepting it the way #579 was is still a human call.
Generated by Claude Code
mikolalysenko commentedon Oct 7, 2026
Claim-ID: agent/fix-bun-open-issues. Draft fix PR: #1009 (it covers all open
pm:bunissues).Check bun spec match before the bundled-entry parse (#578)
Reconcile #371 with #578's bun.lock hot-loop budget
Tighten Bun user-tarball checks from #497 review (#497)
Reconcile #497 with #578's bundled-check budget
mikolalysenko commentedon Oct 8, 2026
[agent] Bench: run 2026-10-08. Still regressed on main
ea097142.Weekly same-machine interleaved
compare, base2463257a(#277) → headea097142, on a 4 vCPU Xeon @ 2.10GHz. A/A was clean.bun/hostedbun/rescanDaily (
9c43dfc9→ea097142): bun/hosted -2.8%, bun/rescan +1.9%. Nothing that landed today moved it. #1009 is the open fix.Generated by Claude Code
Merge main into agent/fix-bun-open-issues
mikolalysenko commentedon Oct 9, 2026
[agent] Bench: run 2026-10-09. bun is back at its pre-#472 speed on main
f3c6313a. #1051 (830749f9, "Stop re-parsing bun lock entries for every patch in scan") landed in this range.Same-machine interleaved
compareon a 4 vCPU Xeon @ 2.80GHz. A/A was clean.Daily,
ea097142→f3c6313a:bun/hostedbun/rescanbun-isolated/hostedbun-isolated/rescanWeekly, against
61cfb9b2, which predates #472 (1169ae68) and so serves as the pre-regression baseline:bun/hostedbun/rescanThat is run 1 of 2 within 5% of the pre-regression baseline. If tomorrow's run agrees, this issue can close. It stays open for now; #1009 is still open for the other bun issues.
Generated by Claude Code
mikolalysenko commentedon Oct 12, 2026
[agent] Bench: run 2026-10-12. bun has been back at or below its pre-#472 speed for 2 consecutive runs, so I'm closing this issue.
Same-machine interleaved
compare, base61cfb9b2(pre-#472) → head17793173(today'smain), 15 pairs, on a 4 vCPU Xeon @ 2.10GHz. A/A was clean.bun/hostedbun/rescanRun 1 was 2026-10-09: bun/hosted -8.1%, bun/rescan -26.3% vs the same base. Today's daily A/B (
f3c6313a→17793173) has bun/hosted at +1.0% and bun/rescan at +14.0%, and neither was flagged. The RSS growth is the npm-family hosted RSS increase already tracked in the ledger (#1058 and theea097142..ef48495crange). It is below the 30% gate.Generated by Claude Code