Skip to content

Adopt pylsl 1.18.5 numpy pulls and string chunk pushes - #43

Merged
cboulay merged 1 commit into
devfrom
pylsl-1.18.5
Sep 21, 2026
Merged

cboulay merged 1 commit into
devfrom
pylsl-1.18.5

Conversation

@cboulay

@cboulay cboulay commented Sep 21, 2026

Copy link
Copy Markdown
Member

Picks up what the last two pylsl releases (v1.18.4, v1.18.5) make available. Requires pylsl>=1.18.5.

Inlet

Numeric pulls pass as_numpy=True alongside the existing pre-allocated fetch buffer. pylsl returns a view of that buffer trimmed to the samples received plus float64 timestamps, so the manual fetch_buffer[:n] slicing and the np.array(timestamps) round-trip are gone. The trim-copy stays, because the buffer is reused by the next pull.

This also removes a silent dependency on ClockSync._offset being a np.float64: lsl2system() was previously handed a Python list of timestamps and only worked because numpy coerced it.

String pulls stay on the list path deliberately — as_numpy=True returns raw undecoded bytes for string streams, whereas the list path gives decoded str.

Per-pull cost (float32, min_samples=1, 1 s fetch buffer):

stream before after
8 ch x 10 16 us 15 us
32 ch x 50 16 us 16 us
128 ch x 100 20 us 16 us
512 ch x 200 50 us 40 us

Dropping the fetch buffer entirely (as_numpy with no dest_obj) also works and is simpler, but pylsl then allocates max_samples x n_ch per pull and concatenates the two-phase halves, so cost scales with local_buffer_dur * fs * n_ch rather than with the data returned — 89 us at 512 ch x 200. Keeping the buffer as dest_obj was the faster option at every width.

Outlet

String streams push in one push_chunk (new in 1.18.5) instead of a push_sample loop. pylsl can't unpack a 2-D array for string streams, so it gets a flat list, with one timestamp per sample to preserve the previous behaviour.

New canonical_channel_format() maps <U*, S* and object dtypes all to "string". Fixed-width unicode arrays previously raised KeyError: '<U3' on outlet creation; normalizing the width out of the format name also keeps source_id stable as the longest value in a stream grows.

Not adopted

Two-phase min_samples pulls and transp_sync_blocking were already in use. 1.18.5's length-delimited string blobs mean values with embedded NULs now survive in both directions for free, and 1.18.4's non-UTF-8 metadata tolerance makes _parse_channel_metadata safer with no change here. reset_uid is outlet-side and would defeat the inlet's uid-based reconnect detection.

Test plan

  • pytest — 50 passed, including a new string round-trip through OutletProcessor to a real inlet, and mock-inlet assertions that as_numpy is set on the buffered path only.
  • Manual smoke test of the inlet string path end to end (decoded <U values, float64 time axis).

Numeric inlet pulls now pass as_numpy=True alongside the pre-allocated
fetch buffer. pylsl returns a view trimmed to the samples received plus
float64 timestamps, so the manual slicing and the list->ndarray timestamp
round-trip go away; the trim-copy stays because the buffer is reused.
This also removes a silent dependency on ClockSync._offset being a
np.float64: lsl2system() was being handed a Python list and only worked
because numpy coerced it.

String pulls stay on the list path deliberately -- as_numpy hands back
raw undecoded bytes for string streams, while the list path decodes.

Outlets push string streams in a single push_chunk instead of a
push_sample loop. pylsl cannot unpack a 2-D array for string streams, so
it gets a flat list, with one timestamp per sample to preserve the
previous semantics.

Fixed-width unicode and bytes dtypes now map to the "string" format
instead of raising KeyError on outlet creation, and normalizing the width
out of the format name keeps source_id stable as the longest value grows.
@cboulay
cboulay merged commit 36233ef into dev Sep 21, 2026
14 checks passed
@cboulay
cboulay deleted the pylsl-1.18.5 branch September 21, 2026 04:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant