Skip to content

Resolve the operating axis from chunk_dim, and deprecate the axis settings - #239

Merged
cboulay merged 2 commits into
devfrom
cboulay/chunk-dim-resolution
Sep 5, 2026
Merged

Resolve the operating axis from chunk_dim, and deprecate the axis settings#239
cboulay merged 2 commits into
devfrom
cboulay/chunk-dim-resolution

Conversation

@cboulay

@cboulay cboulay commented Sep 5, 2026

Copy link
Copy Markdown
Member

Follow-on to #235, which made Downsample's dimension non-configurable. This applies the same reasoning everywhere else.

The problem

Processors guessed positionally for the dimension their settings did not name — dims[0] for "the streaming axis", dims[-1] for "the channel axis". Both are positions rather than meanings, and AxisArray.chunk_dim is the producer's declaration of which dimension messages accumulate along.

The guess and the declaration disagree under transpose and downstream of any windowing stage. That meant a processor's state-reset logic and its arithmetic could pick different axes: baseproc's _message_hash already excluded chunk_dim="win" while _reset_state built filter state along dims[0].

Three resolution rules

One rule doesn't fit every case, so util/message.py grows three:

helper for rule
resolve_chunk_dim stages carrying state between messages (filter zi, running mean, sample buffer, previous-sample cache) chunk_dimSTREAMING_DIMSdims[0]
resolve_feature_dim stages whose axis is static (channels, coordinate components) position −1 or 0, skipping the chunk dim
resolve_transform_dim Spectrum innermost non-chunk LinearAxis, else the chunk dim

Downsample's private _resolve_axis collapses into the shared helper.

The third exists because Spectrum needs time on both a raw (time, ch) stream and a windowed (win, time, ch) one — win accumulates, but each window's spectrum is taken over time.

Bugs fixed along the way

  • Spectrum picked dims[0] on windowed input — FFT-ing across windows instead of within each one.
  • diff.py's documented axis=None default raised dim=None not present in object. __call__, __acall__ and _reset_state passed the raw setting to get_axis_idx; masked only because the diff() factory hardcoded "time".
  • The River scaler hardcoded axis_idx = 0 whenever axis was unset, silently transposing a (ch, time) stream.
  • Merge passed align_axis or "time", so every Merge hardcoded the guess.

Deprecating the axis settings (3.8 → removed in 4.0)

Where anything other than the chunk dimension cannot be meaningful, the setting can only be used to be wrong. 27 Settings classes get a __post_init__ hook — FilterBaseSettings covers ten filters, EWMASettings covers the scaler.

ez.Settings are frozen dataclasses and Transformer(*args, **kwargs) funnels through _unify_settings, so one hook catches all four construction paths: settings class, transformer, unit, and functional factory.

Three details worth reviewing:

  • FutureWarning, not DeprecationWarning. The latter is suppressed by default outside __main__, so pipelines — library code — would never see it. ezmsg core already uses FutureWarning for its own deprecations.
  • The stacklevel walks to the first frame whose module is outside ezmsg. A fixed level can't work (depth differs per construction path), and a filename test can't either: the dataclass __init__ is exec-generated, so co_filename is "<string>" — indistinguishable from python -c. Its f_globals name the defining module.
  • Behaviour is unchanged until removal. The setting is still honoured; test_a_configured_axis_is_still_honoured pins it.

suppress_axis_deprecation() covers stages that forward a setting internally. Without it, every filter-by-design would warn on every state reset — mid-stream, naming whatever drives the pipeline — because FilterByDesignTransformer rebuilds a child FilterSettings each time. Merge and FBCCA forward their own align_axis/time_dim, which users can't drop.

To migrate a pipeline: run its tests with -W error::FutureWarning; every remaining call site becomes a hard failure with a traceback.

The axis = "time" stages

adaptive_lattice_notch, adaptive_lnc, binned_aggregate, resample, wavelets, rollingscaler and filterbankdesign all carry state along their axis but defaulted to a literal "time" rather than a positional guess. Flipping them changes results wherever the chunk dim isn't "time" — and no setting exists to warn about, because the affected caller set nothing.

So resolve_configured_chunk_dim grows a legacy_default parameter that surfaces exactly that population, once, with the escape hatch of pinning axis="time" during the window. Silent on raw streams and on streams with no time dim at all.

Each of these used settings.axis at up to fifteen other sites, all of which break on a None default, so each caches its resolved axis in state at reset. Three needed more:

  • resamplepush_reference is an independent entry point that can precede any signal message, and __next__ has no message at all. It seeds from the reference path and overwrites from the authoritative signal path.
  • filterbankdesign_hash_message runs before _reset_state, so it resolves from the message directly.
  • binned_aggregate, resample — unit-level publish gates named a different dim than the one processed.

Behaviour changes to be aware of

  • FilterbankDesign after a Window now raises rather than designing along a 6.25 Hz win axis. A temporal filter along win is either wrong or very inefficient, so failing loudly is the intent. The legacy-default warning fires first.
  • Stages resolving to win instead of time downstream of a windowing stage — warned via legacy_default.
  • align/filterbank axis defaults changed "time"None (this is what made their or dims[0] dead code).

Deliberately excluded

FlattenSettings.preserve_axis stays configurable. Flatten holds no data between messages and already sets chunk_dim=None when the chunk dimension is folded into the merged axis, so preserving a non-chunk axis is coherent — closer to Slicer than to a filter. Its default now resolves via resolve_chunk_dim.

Testing

4285 unit + 51 integration tests pass; ruff clean. Two new test modules pin the three resolution rules, the deprecation contract, and the exact inventory of deprecated classes — so adding or dropping one is a deliberate edit, and the 4.0 removal has a checklist.

Two things reviewers should know:

  • The repo's own tests still pass axis= at 249 distinct call sites. Deliberate: during the window they're the regression tests for the "still honoured" guarantee, and the new modules cover the default path. They'll need migrating at 4.0.
  • The River scaler fix is untested hereriver is a lazy optional import and isn't installed, so test_scaler_looks_up_the_axis_index_rather_than_assuming_zero is the one skip.

Processors guessed positionally for the dimension their settings did not
name: dims[0] for "the streaming axis", dims[-1] for "the channel axis".
Both are positions rather than meanings, and AxisArray.chunk_dim is the
producer's declaration of which dimension messages accumulate along. The
guess and the declaration disagreed under transpose and downstream of any
windowing stage, so a processor's state-reset logic and its arithmetic
could each pick a different axis: baseproc's _message_hash already excluded
chunk_dim="win" while _reset_state built filter state along dims[0].

Three rules in util/message.py, because one does not fit every case:

* resolve_chunk_dim -- the accumulating dimension, for stages that carry
  state between messages (filter zi, a running mean, a sample buffer, a
  previous-sample cache). Carrying that along a static axis is not a
  smaller error but a different operation.
* resolve_feature_dim -- position -1 (or 0) skipping the chunk dim, for
  stages whose axis is static. dims[-1] can silently *be* the chunk dim on
  a (ch, time) stream, making AffineTransform matmul across time and
  Slicer discard samples.
* resolve_transform_dim -- the innermost non-chunk LinearAxis, else the
  chunk dim. Spectrum needs "time" both on a raw (time, ch) stream and on
  windowed (win, time, ch), where win accumulates but each window's
  spectrum is over time.

Downsample's private _resolve_axis collapses into the shared helper.

Fixes along the way:

* Spectrum picked dims[0] on windowed input, FFT-ing across windows rather
  than within each one.
* diff.py's documented axis=None default raised "dim=None not present in
  object": __call__, __acall__ and _reset_state passed the raw setting to
  get_axis_idx, masked only because the diff() factory hardcoded "time".
* The River scaler hardcoded axis_idx = 0 whenever axis was unset, which
  silently transposed a (ch, time) stream.
* Merge passed align_axis or "time", so every Merge hardcoded the guess.

Deprecate the axis settings (3.8, removed in 4.0)

Where anything other than the chunk dimension cannot be meaningful, the
setting can only be used to be wrong, so it is going away. 27 Settings
classes get a __post_init__ hook; FilterBaseSettings covers ten filters and
EWMASettings covers the scaler. ez.Settings are frozen dataclasses and
Transformer(*args, **kwargs) funnels through _unify_settings, so one hook
catches the settings class, the transformer, the unit and the factory.

* FutureWarning, not DeprecationWarning: the latter is suppressed by
  default outside __main__, so pipelines -- library code -- would never see
  it. ezmsg core already uses FutureWarning for its own deprecations.
* The stacklevel is computed by walking to the first frame whose module is
  outside ezmsg. A fixed level cannot work (the depth differs per
  construction path) and a filename test cannot either: the dataclass
  __init__ is exec-generated, so co_filename is "<string>", the same as
  `python -c`. Its f_globals name the defining module.
* Behaviour is unchanged until removal; the setting is still honoured.
* suppress_axis_deprecation() covers the stages that forward a setting
  internally. Without it every filter-by-design would warn on every state
  reset -- mid-stream, naming whatever drives the pipeline -- because
  FilterByDesignTransformer rebuilds a child FilterSettings each time.
  Merge and FBCCA forward their own align_axis/time_dim, settings the user
  cannot drop.

Run a pipeline's tests with -W error::FutureWarning to locate call sites.

Follow the same rule for the stages that hardcoded axis = "time"

adaptive_lattice_notch, adaptive_lnc, binned_aggregate, resample, wavelets,
rollingscaler and filterbankdesign all carry state along their axis but
defaulted to a literal "time" rather than to a positional guess. Flipping
them to follow chunk_dim changes results wherever the chunk dimension is
not "time" -- downstream of a windowing stage -- and no setting exists to
warn about, because the affected caller set nothing. resolve_configured_
chunk_dim grows a legacy_default parameter that surfaces exactly that
population once, with the escape hatch of pinning axis="time" for now.

Each of these used settings.axis at up to fifteen other sites, all of which
break on a None default, so each now caches its resolved axis in state at
reset. resample seeds it from push_reference, which is an independent entry
point that can precede any signal message, and overwrites from the
authoritative signal path; filterbankdesign resolves in _hash_message,
which runs before _reset_state; binned_aggregate and resample had unit-level
publish gates naming a different dim than the one that was processed.

FilterbankDesign after a Window now raises rather than designing along a
6.25 Hz win axis -- an improvement, since a temporal filter along win is
either wrong or very inefficient. The legacy-default warning fires first.

FlattenSettings.preserve_axis stays configurable: Flatten holds no data
between messages and already sets chunk_dim=None when the chunk dimension
is folded into the merged axis, so preserving a non-chunk axis is coherent.

Tests pin the three resolution rules, the deprecation contract, and the
exact inventory of deprecated classes, so the 4.0 removal has a checklist.
`isinstance(tuple[int, str], type)` is True on 3.10 and False from 3.11, so
a module-level generic alias -- align.py's `_AlignPair` -- got past
inspect.isclass and raised "issubclass() arg 1 must be a class". Only the
3.10 matrix job saw it; fail-fast cancelled the other twelve, which made it
look like a broad failure rather than one interpreter difference.

Verified against a real 3.10.15 environment, not just by inspection.
@cboulay
cboulay merged commit 2523824 into dev Sep 5, 2026
14 checks passed
@cboulay
cboulay deleted the cboulay/chunk-dim-resolution branch September 5, 2026 22:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant