Skip to content

Drop the scaling from attrs once it has been applied - #28

Merged
cboulay merged 1 commit into
devfrom
fix/strip-spent-scaling-attrs
Aug 30, 2026
Merged

Drop the scaling from attrs once it has been applied#28
cboulay merged 1 commit into
devfrom
fix/strip-spent-scaling-attrs

Conversation

@cboulay

@cboulay cboulay commented Aug 29, 2026

Copy link
Copy Markdown
Member

gain and offset describe stored counts → values. Once the multiply has happened those counts are gone, and every later stage — a filter, a decimation, a re-reference — makes them less true still. Carrying them onward offers a downstream consumer a factor that looks authoritative and reconstructs nothing. Re-digitizing is a real need, but it happens after transformations these numbers never saw, so it takes its range and resolution from the caller instead.

Applying a scaling now consumes it: the output carries unit and no nwb_scaling_* keys. Attrs this module doesn't own are left alone.

reader   : {nwb_scaling_gain: 2.5e-07, nwb_scaling_offset: 0.0,
            nwb_scaling_unit: 'volts', nwb_scaling_applied: False,
            nwb_scaling_voltage: True}
converted: {unit: 'microvolts'}

Idempotency gets stronger, not weaker

It used to rest on the applied flag. Now there is simply nothing pending to apply twice — a second transformer in a chain returns the message untouched. Still never inferred from the dtype.

One consequence worth reviewing

The reader always reports applied=False (describe_stream_scaling never applies), so this transformer was the sole producer of applied=True. Stripping removes that state from the wire entirely — which would have silently killed target_unit retargeting of an already-converted message, a documented and tested feature.

It only ever needed the unit, though: "a prefix change on top of a known unit is well defined." So it now reads attrs["unit"] via a new _scaling_from_unit, and the feature survives with the same semantics. A non-voltage unit (a cursor stream in pixels) returns None there and passes through rather than being reinterpreted.

This is the judgment call in the PR. The alternative was to let chained retargeting go, since nothing produces applied=True any more. I kept it because your reason for stripping — stale gain/offset — doesn't argue against it, and dropping a documented feature silently seemed worse. Easy to remove if you'd rather.

Two knock-ons: scaling_fingerprint had to learn the same fallback, or a stream whose unit changed mid-run would keep a plan built for the old one; and a unit-carrying message from a non-NWB source is now retargetable, which matches the module's stated "graph whose source is not always a file" use case.

Also corrected

Two stale references found while in here, both pre-existing:

  • The module docstring said the slicer "applies a file's conversion on read by default". It never applies.
  • An error message advised reading with apply_conversion=False — not a parameter that exists.

Tests

TestSpentScalingIsDropped (only unit survives; unrelated attrs untouched; a second pass can't double-scale; a non-voltage stream still gets its file conversion) and TestRetargetOnUnitAlone (unit-only retarget; pixels left alone; a changed unit rebuilds the plan). Existing tests that asserted the restamped scaling now assert it's gone.

294 passed, 1 skipped, ruff clean. Verified on sub-CA001/ses-S006: convert → {unit: microvolts}, chained retarget to volts exact to 1e-6, and concat of two converted streams still merges.

gain and offset describe stored counts -> values. Once the multiply has
happened those counts are gone, and every later stage -- a filter, a
decimation, a re-reference -- makes them less true still. Carrying them
onward offers a downstream consumer a factor that looks authoritative and
reconstructs nothing. Re-digitizing is a real need, but it happens after
transformations these numbers never saw, so it takes its range and resolution
from the caller instead.

So applying a scaling now consumes it: the output carries unit and no
nwb_scaling_* keys. Attrs this module does not own are left alone.

Idempotency gets stronger rather than weaker. It used to rest on the applied
flag; now there is simply nothing pending to apply twice, and a second
transformer in a chain returns the message untouched.

target_unit keeps working on an already-converted message, which was the one
thing the applied flag was still buying. It only ever needed the unit -- a
prefix change on top of a known unit is well defined -- so it now reads that,
via _scaling_from_unit. A non-voltage unit (a cursor stream in pixels) returns
None there and passes through rather than being reinterpreted. scaling_
fingerprint had to learn the same fallback, or a stream whose unit changed
mid-run would have kept a plan built for the old one.

Note this is only reachable because the reader always reports applied=False:
describe_stream_scaling never applies, so this transformer was the sole
producer of applied=True and stripping it removes that state from the wire
entirely.

Also corrected two stale references while in here: the module docstring said
the slicer applies the conversion on read by default (it never applies), and
an error message advised reading with apply_conversion=False, which is not a
parameter that exists.
@cboulay
cboulay merged commit 5d9bb93 into dev Aug 30, 2026
14 checks passed
@cboulay
cboulay deleted the fix/strip-spent-scaling-attrs branch August 30, 2026 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant