Repository navigation
fix: reduce memory use when indexing Svelte projects - #526
Merged
Merged
Conversation
Svelte projects override resolveModuleNameLiterals without providing getModuleResolutionCache, so createProgram has no package.json cache. Each source file then rereads the nearest package.json and keeps its own parsed copy in packageJsonScope. Return the cache SvelteSupport already uses for module resolution. In Sourcegraph's client/web-sveltekit, 4,669 files shared 351 package.json objects instead of keeping 4,669 copies, and the live heap fell by about 60 MB. The index is byte-identical. Amp-Thread-ID: https://ampcode.com/threads/T-01a19cd4-1b61-72cc-ae15-02a5c02c5846
svelte2tsx emits high-resolution source maps, with a segment for most generated characters. SvelteSourceInfo decoded each map when the program was created and kept every decoded map until the project finished. In Sourcegraph's client/web-sveltekit (1,153 components), that kept 5.3M segments, about 540 MB, on the heap at once. Keep only the encoded map on each SvelteSourceInfo and build the TraceMap on first lookup. A new SourceInfo.releaseCaches drops the decoded map after the constructor has collected its ranges and after the file's document is indexed. Each map is decoded at most twice, which costs little compared to holding all of them. Amp-Thread-ID: https://ampcode.com/threads/T-01a19cd4-1b61-72cc-ae15-02a5c02c5846
eseliger
marked this pull request as ready for review
November 2, 2026 22:43
christoph-sg
approved these changes
Nov 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Some improvements found when investigating memory usage of indexing Sourcegraph's monorepo.
About 600 MB comes from two things the Svelte support keeps around without needing to:
SvelteSourceInfodecoded every map when the program was created and kept all of them until the project finished. Inclient/web-sveltekit, with 1,153 components, that was 5.3M segments, about 540 MB. EachSvelteSourceInfonow keeps only the encoded map and builds theTraceMapon first lookup. The newSourceInfo.releaseCaches()drops it after the constructor has collected its ranges and again after the file's document is written, so only the file being indexed holds a decoded map.resolveModuleNameLiteralswithoutgetModuleResolutionCache, socreateProgramhas no package.json cache. Every source file rereads the nearest package.json and keeps its own parsed copy inpackageJsonScope. The host now returns the cacheSvelteSupportalready uses, and 4,669 files share 351 copies instead of keeping 4,669, which saves about 60 MB.Test plan
Existing unit and snapshot tests pass unchanged; the Svelte snapshots exercise rebuilding the trace map after it has been released.
Measured on sourcegraph/sourcegraph, comparing builds of
mainand this branch:main(0.6.0)client/web-sveltekit(after GC)--max-old-space-sizeforindex --pnpm-workspacesindex.scipmain, for both the single project and the whole workspace