Skip to content

[test] Make sync()-driving tests hermetic instead of hitting the live registry - #104

Merged
kmturley merged 1 commit into
mainfrom
test/mock-registry-sync
Jul 31, 2026
Merged

[test] Make sync()-driving tests hermetic instead of hitting the live registry#104
kmturley merged 1 commit into
mainfrom
test/mock-registry-sync

Conversation

@kmturley

Copy link
Copy Markdown
Member

Summary

  • Nearly every sync()-driving test across Manager.test.ts, ManagerLocal.test.ts, Registry.test.ts, and RegistryLocal.test.ts synced against the real, live open-audio-stack-registry GitHub Pages site over the network. CI correctness was coupled to that site's uptime and current content - a transient outage or a content change unrelated to any code change could fail the suite, and it couldn't distinguish "my change broke this" from "the registry changed".
  • Adds mockRegistrySync() to tests/testUtils.ts, which stubs apiJson (the function sync() calls to fetch a registry) to resolve with a fixed local fixture instead. Applied to every sync() call across the four affected test files that only needs the existing single-version PLUGIN/PRESET/PROJECT fixtures.
  • Found and fixed a real bug in my first pass at the mock: mockResolvedValue(fixture) resolves the same object reference on every call, but ManagerLocal.install()/uninstall() mutate the PackageVersion object in place (pkgVersion.installed = true / delete pkgVersion.installed) rather than cloning it - a real network response is a fresh object graph from JSON.parse every time, so this never surfaced before. mockRegistrySync() now structuredClone()s the payload per call to match that real behavior, otherwise mutation state leaked between unrelated tests sharing the same imported fixture object.
  • One test (Project sync, install project, add new dependency, remove new dependency) is deliberately left hitting the live registry, with a comment explaining why: it adds a dependency on surge-synthesizer/surge@1.3.4, a second real published version beyond what the static fixtures cover, and installDependency() then downloads and sha256-verifies a real file for that version regardless of where the version JSON came from - mocking just the registry response there wouldn't remove the live-network dependency, only relocate it. It stays a genuine end-to-end integration test.
  • Registry.test.ts/RegistryLocal.test.ts's "Sync managers" tests previously only asserted toBeDefined() (passed regardless of what sync() actually did) - now that the data is deterministic, strengthened to toEqual(REGISTRY_PACKAGE_TYPES).
  • No specification.md changes - this is test infrastructure only, no observable behavior changed.

This is item 2 of the architectural review in review.md (Critical Blocker #3), and was sequenced before the ManagerLocal.install() transactional-rollback fix (item 3, next) specifically so that fix's tests can be trusted.

Test plan

  • npm run check (format, lint, build, test): 207/207 tests, 17/17 files pass.
  • Verified the mock-cloning fix by reproducing the failure first: without structuredClone(), 3 tests failed with stray installed: true leaking from an earlier test's mutation of the shared fixture object.
  • Confirmed every manager.sync()/registry.sync() call site across the four files is now either mocked or has an explicit comment explaining why it's intentionally still live.

🤖 Generated with Claude Code

… registry

Nearly every sync()-driving test synced against the real, live
open-audio-stack-registry over the network - CI correctness was
coupled to that site's uptime and current content, and the suite
couldn't tell "my change broke this" from "the registry changed" or
"GitHub had a blip". Add mockRegistrySync() (structuredClone()s the
payload per call, since PackageVersion objects are mutated in place
downstream - a real JSON response is a fresh object graph every time,
so resolving one shared fixture object repeatedly silently leaked
`installed` state between tests) and use it everywhere sync() only
needs the existing single-version fixtures.

One test is deliberately left hitting the live registry: adding a
dependency on surge-synthesizer/surge@1.3.4, a second real published
version beyond what the static fixtures cover, which install() then
downloads and sha256-verifies for real regardless of where the version
metadata came from - mocking just the registry JSON there wouldn't
remove the live dependency, only relocate it.

No specification.md changes - this is test infrastructure only, no
observable behavior changed.
// RFC 2606 guarantee to actually fail DNS resolution over a real network round trip) and one
// that responds normally, so this stays deterministic and network-free either way.
const apiJsonSpy = vi.spyOn(apiHelpers, 'apiJson').mockImplementation(async (url: string) => {
if (url.startsWith('https://example.invalid')) throw new Error('getaddrinfo ENOTFOUND example.invalid');
@kmturley
kmturley merged commit b286143 into main Jul 31, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants