Key the default state hash on the message layout

[cboulay]

Makes the default _hash_message key on the message layout instead of returning a constant, so a processor that doesn't override it resets when the stream actually changes.

Requires ezmsg 3.10.0b1 for CoordinateAxis.fingerprint and AxisArray.chunk_dim (ezmsg-org/ezmsg#265).

The problem

_hash_message returned 0, so a processor that didn't override it reset once and never again. That's correct for something operating elementwise, and wrong for anything caching state derived from the stream's shape — which the base class could never detect, because it never asked.

Worse is what overriding processors were missing. Per-channel state — a filter's zi, a scaler's running mean — is only valid for the channels it was built for, and shape alone cannot see a source that swaps channels without changing how many it sends. Reconfiguring a device at a fixed channel count leaves the new channel's first samples dominated by the old channel's filter history:

first 4 samples of the NEW channel 'armB-1':
  filter state carried from armA: [-5.919  -4.772  -0.5114  1.064]
  with a correct reset:           [ 0.     -0.0013 -0.0023 -0.0026]
  max |difference| = 11.12   vs new-data amplitude 0.024

~470x the new signal's amplitude, decaying over the filter's settling time, with nothing raised.

What the default folds in

The message key, its dims, the length of every dimension except the one it is a chunk along, the coordinate values on those dimensions (via fingerprint), and the gain and offset of any linear axis among them.

Two deliberate exclusions:

• The chunk dimension's length and offset, since both change with every message by definition. If it carries a CoordinateAxis (irregular event times) its values do too, so it is excluded entirely.
• offset only there. Elsewhere it locates the axis: a spectrum whose freq axis moves from 5-25 Hz to 70-90 Hz keeps the same gain and the same length and differs only in its offset.

Which dimension is the chunk dimension comes from AxisArray.chunk_dim when the producer declares it — the producer renamed the dims, so it is the only party that reliably knows. STREAMING_DIMS is the fallback for producers that don't, defaulting to ("time",).

That fallback is a guess, and documented as one. It is right for a raw (time, ch) signal and wrong downstream of a windowing stage, where the message is (win, time, ch) and win is what grows:

post-Window (win, time, ch)   0=first  1,2=win-count jitter  3=relabel  4=same  5=window-len change
  fallback ("time",):       resets at [0, 1, 2, 3]     thrashes, and misses [5]
  declared chunk_dim="win": resets at [0, 3, 5]        correct

A wrong answer is not a small error in either direction: name a stable dimension and the processor stops noticing real changes to it; name a growing one and it resets on every message.

Extending rather than replacing

_message_hash is exposed separately so an override can build on the default instead of rebuilding the hash and silently losing the axis coverage.

Auditing ezmsg-sigproc's 46 stateful processors showed both directions are needed. Some must add something the base cannot know — a dtype the state depends on, a value derived from the processor's own state. Five must narrow it: binned_aggregate and downsample would otherwise restart their BinSchedule and decimation phase (s_idx = 0) on a channel change they genuinely don't care about, shifting bin boundaries and sample phase.

Hence exclude_dims, include_key and extra. exclude_dims is additive to the chunk dimension, so naming one cannot accidentally un-exclude the other.

Cost

	per node	30-node graph	core @ 1 kHz
old default (return 0)	0.025 µs	0.75 µs	0.07%
typical hand-written override	0.165 µs	4.99 µs	0.50%
new default	0.414 µs	12.44 µs	1.24%

About 2.5x a typical override, +0.25 µs per node. The coordinate digest itself is near-free in the common case: fingerprint is computed once per axis object and cached on it, and rides the pickle across a process boundary, so consumers downstream of one hop read it rather than recompute it.

Constant factor

_message_hash runs on every message of every stream, so the third commit trims it: hoist dims / axes / data.shape out of the loop, build the exclusion as a tuple rather than a set (it holds one or two entries, where a linear scan beats the ~0.05 µs of building a set), and ask for only the attribute each branch uses. The loop previously fetched gain for every dimension before testing exclusion, so a coordinate axis paid a failed lookup it never used and then a second one for its fingerprint.

0.60 -> 0.53 µs, 1.14x, measured on a (time, ch, feature) message with a 256-channel ChannelMap axis. Hash values are unchanged, verified identical to the previous implementation across coordinate / linear / absent / non-AxisArray axes and every combination of the keyword arguments.

Three further candidates were measured and rejected:

candidate	result
isinstance dispatch instead of getattr	0.96x — two isinstance checks cost more than the getattrs they replace
list comprehension / genexp instead of the for-loop	0.65-0.99x at 2, 3 and 5 dimensions
memoising on the identity of the axes mapping	0.84x — see below

On the comprehension: the usual win is avoiding a LOAD_METHOD append per iteration, but this loop runs 2-5 times and the fixed setup does not amortise. Every variant improved steadily from 2 to 5 dimensions, so the break-even is somewhere past ~8 dimensions, which AxisArray messages do not reach. Binding axis once via the for axis in (get(dim),) idiom is the best of them at 0.98x. Hoisting parts.append is also a loss (0.94x) — creating the bound method costs more than the repeated lookup at these iteration counts.

On the memo: an earlier draft cached the result and short-circuited when message.axes was the same object as last time. That looked like a 3.3x win, but the benchmark shared one axes dict across its messages, which real streams do not:

source: consecutive messages share the mapping?   False
replace(msg, data=...)  preserves the mapping?    True
chain source->A->B->C, mapping same as previous message: 0/3 at every node

replace(msg, data=...) preserves the mapping along a single message's journey through the graph, but the memo compares successive messages at one node, and each message begins at a source that builds a fresh mapping. The identity check therefore never fires and the memo is a flat 16% loss (0.554 vs 0.476 µs). Dropped.

Per-axis identity does hold — a source rebuilds the dict but reuses the axis objects inside it — but a memo keyed that way still has to walk the dimensions, so it would only replace the fingerprint property call with a dict lookup, about 0.03 µs per coordinate axis. Not worth the state.

Compatibility

Non-AxisArray messages still hash to a constant, so producers and processors on other message types are unaffected — clockdriven's CounterProducer and friends keep their existing behaviour.

Every processor that currently overrides _hash_message is untouched; this only changes the default.

Testing

13 new tests covering chunk-size jitter, relabel at fixed channel count, channel-count / sample-rate / key changes, time offset advancing, a coordinate-valued streaming axis, declared vs fallback chunk dimension, a fallback naming an absent dimension, non-AxisArray messages, and each of the three escape hatches.

192 passed, 1 skipped (MLX not installed) against the published 3.10.0b1 wheel. Each commit is independently green — the pin bump alone is 178 passed.

Verified downstream: ezmsg-sigproc's full suite is 4171 passed with no regressions.

🤖 Generated with Claude Code