Adopt pylsl 1.18.5 numpy pulls and string chunk pushes - #43
Merged
Merged
Conversation
Numeric inlet pulls now pass as_numpy=True alongside the pre-allocated fetch buffer. pylsl returns a view trimmed to the samples received plus float64 timestamps, so the manual slicing and the list->ndarray timestamp round-trip go away; the trim-copy stays because the buffer is reused. This also removes a silent dependency on ClockSync._offset being a np.float64: lsl2system() was being handed a Python list and only worked because numpy coerced it. String pulls stay on the list path deliberately -- as_numpy hands back raw undecoded bytes for string streams, while the list path decodes. Outlets push string streams in a single push_chunk instead of a push_sample loop. pylsl cannot unpack a 2-D array for string streams, so it gets a flat list, with one timestamp per sample to preserve the previous semantics. Fixed-width unicode and bytes dtypes now map to the "string" format instead of raising KeyError on outlet creation, and normalizing the width out of the format name keeps source_id stable as the longest value grows.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Picks up what the last two pylsl releases (v1.18.4, v1.18.5) make available. Requires
pylsl>=1.18.5.Inlet
Numeric pulls pass
as_numpy=Truealongside the existing pre-allocated fetch buffer. pylsl returns a view of that buffer trimmed to the samples received plus float64 timestamps, so the manualfetch_buffer[:n]slicing and thenp.array(timestamps)round-trip are gone. The trim-copy stays, because the buffer is reused by the next pull.This also removes a silent dependency on
ClockSync._offsetbeing anp.float64:lsl2system()was previously handed a Python list of timestamps and only worked because numpy coerced it.String pulls stay on the list path deliberately —
as_numpy=Truereturns raw undecoded bytes for string streams, whereas the list path gives decodedstr.Per-pull cost (float32,
min_samples=1, 1 s fetch buffer):Dropping the fetch buffer entirely (
as_numpywith nodest_obj) also works and is simpler, but pylsl then allocatesmax_samples x n_chper pull and concatenates the two-phase halves, so cost scales withlocal_buffer_dur * fs * n_chrather than with the data returned — 89 us at 512 ch x 200. Keeping the buffer asdest_objwas the faster option at every width.Outlet
String streams push in one
push_chunk(new in 1.18.5) instead of apush_sampleloop. pylsl can't unpack a 2-D array for string streams, so it gets a flat list, with one timestamp per sample to preserve the previous behaviour.New
canonical_channel_format()maps<U*,S*andobjectdtypes all to"string". Fixed-width unicode arrays previously raisedKeyError: '<U3'on outlet creation; normalizing the width out of the format name also keepssource_idstable as the longest value in a stream grows.Not adopted
Two-phase
min_samplespulls andtransp_sync_blockingwere already in use. 1.18.5's length-delimited string blobs mean values with embedded NULs now survive in both directions for free, and 1.18.4's non-UTF-8 metadata tolerance makes_parse_channel_metadatasafer with no change here.reset_uidis outlet-side and would defeat the inlet's uid-based reconnect detection.Test plan
pytest— 50 passed, including a new string round-trip throughOutletProcessorto a real inlet, and mock-inlet assertions thatas_numpyis set on the buffered path only.<Uvalues, float64 time axis).