[test] Make sync()-driving tests hermetic instead of hitting the live registry - #104
Merged
Conversation
… registry Nearly every sync()-driving test synced against the real, live open-audio-stack-registry over the network - CI correctness was coupled to that site's uptime and current content, and the suite couldn't tell "my change broke this" from "the registry changed" or "GitHub had a blip". Add mockRegistrySync() (structuredClone()s the payload per call, since PackageVersion objects are mutated in place downstream - a real JSON response is a fresh object graph every time, so resolving one shared fixture object repeatedly silently leaked `installed` state between tests) and use it everywhere sync() only needs the existing single-version fixtures. One test is deliberately left hitting the live registry: adding a dependency on surge-synthesizer/surge@1.3.4, a second real published version beyond what the static fixtures cover, which install() then downloads and sha256-verifies for real regardless of where the version metadata came from - mocking just the registry JSON there wouldn't remove the live dependency, only relocate it. No specification.md changes - this is test infrastructure only, no observable behavior changed.
| // RFC 2606 guarantee to actually fail DNS resolution over a real network round trip) and one | ||
| // that responds normally, so this stays deterministic and network-free either way. | ||
| const apiJsonSpy = vi.spyOn(apiHelpers, 'apiJson').mockImplementation(async (url: string) => { | ||
| if (url.startsWith('https://example.invalid')) throw new Error('getaddrinfo ENOTFOUND example.invalid'); |
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
sync()-driving test acrossManager.test.ts,ManagerLocal.test.ts,Registry.test.ts, andRegistryLocal.test.tssynced against the real, liveopen-audio-stack-registryGitHub Pages site over the network. CI correctness was coupled to that site's uptime and current content - a transient outage or a content change unrelated to any code change could fail the suite, and it couldn't distinguish "my change broke this" from "the registry changed".mockRegistrySync()totests/testUtils.ts, which stubsapiJson(the functionsync()calls to fetch a registry) to resolve with a fixed local fixture instead. Applied to everysync()call across the four affected test files that only needs the existing single-versionPLUGIN/PRESET/PROJECTfixtures.mockResolvedValue(fixture)resolves the same object reference on every call, butManagerLocal.install()/uninstall()mutate thePackageVersionobject in place (pkgVersion.installed = true/delete pkgVersion.installed) rather than cloning it - a real network response is a fresh object graph fromJSON.parseevery time, so this never surfaced before.mockRegistrySync()nowstructuredClone()s the payload per call to match that real behavior, otherwise mutation state leaked between unrelated tests sharing the same imported fixture object.Project sync, install project, add new dependency, remove new dependency) is deliberately left hitting the live registry, with a comment explaining why: it adds a dependency onsurge-synthesizer/surge@1.3.4, a second real published version beyond what the static fixtures cover, andinstallDependency()then downloads and sha256-verifies a real file for that version regardless of where the version JSON came from - mocking just the registry response there wouldn't remove the live-network dependency, only relocate it. It stays a genuine end-to-end integration test.Registry.test.ts/RegistryLocal.test.ts's "Sync managers" tests previously only assertedtoBeDefined()(passed regardless of whatsync()actually did) - now that the data is deterministic, strengthened totoEqual(REGISTRY_PACKAGE_TYPES).specification.mdchanges - this is test infrastructure only, no observable behavior changed.This is item 2 of the architectural review in
review.md(Critical Blocker #3), and was sequenced before theManagerLocal.install()transactional-rollback fix (item 3, next) specifically so that fix's tests can be trusted.Test plan
npm run check(format, lint, build, test): 207/207 tests, 17/17 files pass.structuredClone(), 3 tests failed with strayinstalled: trueleaking from an earlier test's mutation of the shared fixture object.manager.sync()/registry.sync()call site across the four files is now either mocked or has an explicit comment explaining why it's intentionally still live.🤖 Generated with Claude Code