Update v8 SNAPSHOT release, optimization - #129
Merged
Merged
Conversation
Adapter scans are ordered lazy sequences from the exclusive bound to the end of the endpoint. Since the physical layer was wired onto the routed engine (92c76be), classified-fetch-fn realized the WHOLE remainder with `vec` on every command, and only then did the reducer keep its physical chunk of 64. Every scan command therefore re-read the endpoint tail: walking an endpoint of degree n cost about n^2/128 realized values instead of n, and a 20-result first page realized the entire endpoint. The boundary now realizes exactly the descriptor's `:limit` (the reducer always sends its physical chunk size); descriptors without a limit keep realizing the complete scan. Released values, `more-physical?`, work counters, order and pages are unchanged by construction: the reducer already truncated to the same chunk. Atomic partial-output discard and retry semantics are unchanged because the chunk is still realized inside the try. Measured on the local million-server Datomic demo (recursive schema, warm second peer, single runs): count-resources for user-1 (24,024 results) realized 1,079,996 -> 96,188 scan values; a 30,000-limited super-user count 1,292,809 -> 114,337 (1,113 -> 730 ms); super-user first page of 20 realized 568 -> 128 values (15.7 -> 8.6 ms). On a synthetic DataScript endpoint with 8,000 direct subjects an exhaustive count realized 504,000 values before and at most 8,064 after. Regression test: routed-reads-realize-only-the-requested-chunk-test (fails 6 assertions under the previous behaviour). CI-equivalent battery 659 tests / 26,510 assertions green; DataScript CLJS 203 / 7,428 green; public-source-closure ledger regenerated.
…ization Realize only the requested chunk inside the classified read boundary
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.