Skip to content

Let the base class decide when these transformers reset - #37

Merged
cboulay merged 1 commit into
devfrom
cboulay/baseproc-hashing
Sep 4, 2026
Merged

Let the base class decide when these transformers reset#37
cboulay merged 1 commit into
devfrom
cboulay/baseproc-hashing

Conversation

@cboulay

@cboulay cboulay commented Sep 4, 2026

Copy link
Copy Markdown
Member

ezmsg-baseproc 1.12.0's default _hash_message folds in the message key, the dims, the length of every dimension except the chunk dimension, the coordinate values on those dimensions, and the gain and offset of any linear axis among them. Two of the three overrides here were strict subsets of that.

What the deletions buy

The channel fingerprint. A source that renames or reorders channels without changing how many it sends — a device reconfigured mid-session, a montage swapped — was invisible to these hashes, so per-channel state carried onto channels it did not belong to:

  • RefractoryTransformer kept armA's elapsed counters, suppressing armB's events for the first refractory window.
  • ThresholdCrossingTransformer kept armA's adaptive z-scoring state, so armB's first crossings were judged against the wrong running mean.

Neither announces itself in the output.

refractory's override was return super()._hash_message(message) in any case. peak's hashed the key, the non-time shape and the gain — all of which the default already covers.

The deletions weren't decided by reading. A harness ran each override and the default over one stream and compared their reset points; an override was removed only where the default's set was a superset:

override                              its resets       default resets     verdict
peak.ThresholdCrossingTransformer     [0,5,6,7,8]      [0,3,5,6,7,8]      DELETABLE (default also resets at [3])
refractory.RefractoryTransformer      [0,3,5,6,7,8]    [0,3,5,6,7,8]      DELETABLE (identical)
kernel_activation.BinnedKernelAct.    [0,5,6,8,9]      [0,3,5,6,7,8]      KEEP -- default misses [9]

  3 = RELABEL at fixed channel count   9 = dtype change

BinnedKernelActivation keeps its override

For two things the base class has no way to know: the data dtype (via str(), since mlx arrays don't expose .kind) and the array backend, whose code paths differ between sparse, numpy and mlx. Those move to extra=.

It now also inherits the key and the fingerprint, which it previously ignored — activation, dense_carry and samples_since_update are per-channel running state and must not follow a stream switch. Its reset set goes from [0,5,6,8,9] to [0,3,5,6,7,8,9]: a strict superset of the default, plus the dtype.

Testing

166 passed, up from 155. 11 new tests, verified by restoring the old hashes: 3 of the 4 relabel tests fail without this change, including both behavioural ones.

The fourth was vacuous when I first wrote it — armA and armB fired at the same sample indices, so the carried refractory counters changed nothing and it passed either way. It now straddles the refractory window (armA on its last sample, armB on its sixth, against a 60-sample window) and fails on the old hash like the others.

Dependencies

ezmsg-baseproc>=1.12.0, and the direct ezmsg pin raised to 3.10.0b2 alongside it. The latter is not optional: baseproc needs that version for CoordinateAxis.fingerprint and AxisArray.chunk_dim, and uv only enables pre-releases for a package named with a pre-release marker in this file — leaving it to the transitive requirement fails resolution outright with requirements are unsatisfiable.

🤖 Generated with Claude Code

ezmsg-baseproc 1.12.0's default `_hash_message` folds in the message key, the
dims, the length of every dimension except the chunk dimension, the coordinate
*values* on those dimensions, and the gain and offset of any linear axis among
them. Two of the three overrides here were strict subsets of that and are gone;
the third keeps only what the default cannot see.

What the deletions buy is the channel fingerprint. A source that renames or
reorders channels without changing how many it sends -- a device reconfigured
mid-session, a montage swapped -- was invisible to these hashes, so per-channel
state carried onto channels it did not belong to:

* `RefractoryTransformer` kept armA's `elapsed` counters, suppressing armB's
  events for the first refractory window.
* `ThresholdCrossingTransformer` kept armA's adaptive z-scoring state, so
  armB's first crossings were judged against the wrong running mean.

Neither announces itself in the output. `refractory`'s override was
`return super()._hash_message(message)` in any case; `peak`'s hashed the key,
the non-time shape and the gain, all of which the default already covers.

`BinnedKernelActivation` still overrides, for two things the base class has no
way to know: the data dtype (via `str()`, since mlx arrays do not expose
`.kind`) and the array backend, whose code paths differ between sparse, numpy
and mlx. Those move to `extra=`, and it now inherits the key and the
fingerprint, which it previously ignored -- `activation`, `dense_carry` and
`samples_since_update` are per-channel running state and must not follow a
stream switch.

Requires ezmsg-baseproc 1.12.0. The direct `ezmsg` pin is raised to 3.10.0b2
alongside it: baseproc needs that version for `CoordinateAxis.fingerprint` and
`AxisArray.chunk_dim`, and uv only enables pre-releases for a package named
with a pre-release marker in *this* file, so leaving it to the transitive
requirement fails resolution outright.

11 new tests, verified by restoring the old hashes: 3 of the 4 relabel tests
fail without this change, including both behavioural ones. The fourth was
vacuous at first -- armA and armB fired at the same sample indices, so the
carried refractory counters changed nothing; it now straddles the refractory
window, and fails too.

166 passed, up from 155.
@cboulay
cboulay merged commit 226613f into dev Sep 4, 2026
4 checks passed
@cboulay
cboulay deleted the cboulay/baseproc-hashing branch September 4, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant