Repository navigation
perf: memoize url resolution in SchemaId - #6
Open
quentindemetz wants to merge 2 commits into
Open
quentindemetz wants to merge 2 commits into
quentindemetz wants to merge 2 commits into
Conversation
SchemaId is constructed for every sub-schema and reference, and each construction calls the legacy url.resolve / url.parse APIs, which are slow. Large OpenAPI documents create many SchemaIds with the same inputs, so caching these results avoids most of the work. On a codebase generating types for ~2,000 OpenAPI files, this cut url parsing from ~10.6s to ~3.2s of CPU time (-30% of total generation time), with byte-identical output.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SchemaId is constructed for every sub-schema and reference, and each construction calls the legacy
url.resolve/url.parseAPIs, which are slow. Large OpenAPI documents create many SchemaIds, so this PR avoids most of that work in two ways:url.resolveandurl.parseresults, since many SchemaIds share the same inputs.url.resolvefor simple references. Most calls resolve a fragment (#/components/...) or an absolute path (/properties/...) against a base with no scheme or host. For those, the result is just the base without its fragment followed by the fragment, or the path itself. The shortcut only applies when both strings contain letters, digits,-,_,.,~,/and#only, no.or..segments, and do not start with//. Everything else (percent-escaping, dot-segment removal, URLs with a host) still goes throughurl.resolve.On a codebase generating types for ~2,000 OpenAPI files:
url.resolvecalls take it, and every one returns the same string asurl.resolve.Output is byte-identical. New tests cover both the shortcut and the cases that must fall back to
url.resolve.