Skip to content

perf: memoize url resolution in SchemaId - #6

Open
quentindemetz wants to merge 2 commits into
anttiviljami:masterfrom
pennylane-hq:memoize-schema-id-url-parsing
Open

quentindemetz wants to merge 2 commits into
anttiviljami:masterfrom
pennylane-hq:memoize-schema-id-url-parsing

Conversation

@quentindemetz

@quentindemetz quentindemetz commented Oct 20, 2026 •

Copy link
Copy Markdown

SchemaId is constructed for every sub-schema and reference, and each construction calls the legacy url.resolve / url.parse APIs, which are slow. Large OpenAPI documents create many SchemaIds, so this PR avoids most of that work in two ways:

  1. Memoize url.resolve and url.parse results, since many SchemaIds share the same inputs.
  2. Skip url.resolve for simple references. Most calls resolve a fragment (#/components/...) or an absolute path (/properties/...) against a base with no scheme or host. For those, the result is just the base without its fragment followed by the fragment, or the path itself. The shortcut only applies when both strings contain letters, digits, -, _, ., ~, / and # only, no . or .. segments, and do not start with //. Everything else (percent-escaping, dot-segment removal, URLs with a host) still goes through url.resolve.

On a codebase generating types for ~2,000 OpenAPI files:

  • memoization cut url parsing from ~10.6 s to ~3.2 s of CPU time (−30% of total generation time);
  • the shortcut then cut total type generation from 8.1 s to 4.7 s of CPU time (−42%). 154,479 of the 154,504 distinct url.resolve calls take it, and every one returns the same string as url.resolve.

Output is byte-identical. New tests cover both the shortcut and the cases that must fall back to url.resolve.

SchemaId is constructed for every sub-schema and reference, and each construction calls the legacy url.resolve / url.parse APIs, which are slow. Large OpenAPI documents create many SchemaIds with the same inputs, so caching these results avoids most of the work. On a codebase generating types for ~2,000 OpenAPI files, this cut url parsing from ~10.6s to ~3.2s of CPU time (-30% of total generation time), with byte-identical output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant