Repository navigation
refactor(io): serve MATLAB subscripts in the base lazy array - #945
Draft
ehennestad wants to merge 1 commit into
Draft
ehennestad wants to merge 1 commit into
ehennestad wants to merge 1 commit into
Conversation
Each lazy array turned MATLAB subscripts into reads itself. The Zarr v3 lazy array re-implemented HDF5LazyArray's checks, expected-size rules and reordering, added its own choice between a bounding-box read and element reads, and reversed dimensions in several places. io.backend.base.LazyArray now implements load_mat_style: it checks the subscripts, reads each dimension's indices once, sorted, and returns the values in the order and shape the subscripts ask for, as a table for a compound dataset. A backend implements two reads in its own storage order: readSubscripts, which reads every combination of per-dimension indices, and readPoints, which reads single elements. A linear selection is read through its bounding box or element by element, as getBoundingBoxLimits allows. A backend whose storage order is the reverse of MatNWB's says so in isStorageOrderReversed, and the base class reverses subscripts and dimensions for it. readHyperslab serves start, count and stride through the same path. HDF5LazyArray implements the two reads with its existing union-of- hyperslabs and point selections; a linear selection that fills its bounding box is now read as one hyperslab in any rank. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ehennestad
added this pull request to stack #946
October 14, 2026 17:12
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Each lazy array turns MATLAB subscripts (
DataStubindexing,load_mat_style) into reads itself. The Zarr v3 lazy array in #853 re-implements the checks, expected-size rules and reordering ofHDF5LazyArray, adds its own choice between a bounding-box read and element reads, and reverses dimensions in several places. This is the second point of the design note on #853.io.backend.base.LazyArraynow implementsload_mat_style. It checks the subscripts, reads each dimension's indices once and sorted, and returns the values in the order and shape the subscripts ask for, as a table for a compound dataset. A backend implements two reads in its own storage order:readSubscriptsreads every combination of per-dimension indices, andreadPointsreads single elements. A linear selection is read through its bounding box or element by element, within limits the backend sets ingetBoundingBoxLimits. A backend that stores arrays in row-major order returns true fromisStorageOrderReversed, and the base class reverses subscripts and dimensions for it.readHyperslabserves start, count and stride through the same path.HDF5LazyArrayimplements the two reads with its existing selections: one union of hyperslabs for subscripts and one element selection for points. The subscript rules and results are unchanged. A linear selection that fills its bounding box is now read as one hyperslab in any rank; before, only a contiguous run in a one-dimensional dataset was.tests.unit.io.backend.base.LazyArraySelectionTestruns selections through an in-memory lazy array in both storage orders and compares them with MATLAB indexing of the same array.This conflicts with #923, which restructures
HDF5LazyArray/load_mat_style.m. That file's read code now lives inreadSubscripts, which is where #923's grouped reads belong; the reshaping it moves intoreshapeLoadedDatais now in the base class.🤖 Generated with Claude Code