Drop the scaling from attrs once it has been applied - #28
Merged
Conversation
gain and offset describe stored counts -> values. Once the multiply has happened those counts are gone, and every later stage -- a filter, a decimation, a re-reference -- makes them less true still. Carrying them onward offers a downstream consumer a factor that looks authoritative and reconstructs nothing. Re-digitizing is a real need, but it happens after transformations these numbers never saw, so it takes its range and resolution from the caller instead. So applying a scaling now consumes it: the output carries unit and no nwb_scaling_* keys. Attrs this module does not own are left alone. Idempotency gets stronger rather than weaker. It used to rest on the applied flag; now there is simply nothing pending to apply twice, and a second transformer in a chain returns the message untouched. target_unit keeps working on an already-converted message, which was the one thing the applied flag was still buying. It only ever needed the unit -- a prefix change on top of a known unit is well defined -- so it now reads that, via _scaling_from_unit. A non-voltage unit (a cursor stream in pixels) returns None there and passes through rather than being reinterpreted. scaling_ fingerprint had to learn the same fallback, or a stream whose unit changed mid-run would have kept a plan built for the old one. Note this is only reachable because the reader always reports applied=False: describe_stream_scaling never applies, so this transformer was the sole producer of applied=True and stripping it removes that state from the wire entirely. Also corrected two stale references while in here: the module docstring said the slicer applies the conversion on read by default (it never applies), and an error message advised reading with apply_conversion=False, which is not a parameter that exists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
gainandoffsetdescribe stored counts → values. Once the multiply has happened those counts are gone, and every later stage — a filter, a decimation, a re-reference — makes them less true still. Carrying them onward offers a downstream consumer a factor that looks authoritative and reconstructs nothing. Re-digitizing is a real need, but it happens after transformations these numbers never saw, so it takes its range and resolution from the caller instead.Applying a scaling now consumes it: the output carries
unitand nonwb_scaling_*keys. Attrs this module doesn't own are left alone.Idempotency gets stronger, not weaker
It used to rest on the
appliedflag. Now there is simply nothing pending to apply twice — a second transformer in a chain returns the message untouched. Still never inferred from the dtype.One consequence worth reviewing
The reader always reports
applied=False(describe_stream_scalingnever applies), so this transformer was the sole producer ofapplied=True. Stripping removes that state from the wire entirely — which would have silently killedtarget_unitretargeting of an already-converted message, a documented and tested feature.It only ever needed the unit, though: "a prefix change on top of a known unit is well defined." So it now reads
attrs["unit"]via a new_scaling_from_unit, and the feature survives with the same semantics. A non-voltage unit (a cursor stream inpixels) returnsNonethere and passes through rather than being reinterpreted.This is the judgment call in the PR. The alternative was to let chained retargeting go, since nothing produces
applied=Trueany more. I kept it because your reason for stripping — stalegain/offset— doesn't argue against it, and dropping a documented feature silently seemed worse. Easy to remove if you'd rather.Two knock-ons:
scaling_fingerprinthad to learn the same fallback, or a stream whose unit changed mid-run would keep a plan built for the old one; and aunit-carrying message from a non-NWB source is now retargetable, which matches the module's stated "graph whose source is not always a file" use case.Also corrected
Two stale references found while in here, both pre-existing:
apply_conversion=False— not a parameter that exists.Tests
TestSpentScalingIsDropped(onlyunitsurvives; unrelated attrs untouched; a second pass can't double-scale; a non-voltage stream still gets its file conversion) andTestRetargetOnUnitAlone(unit-only retarget;pixelsleft alone; a changed unit rebuilds the plan). Existing tests that asserted the restamped scaling now assert it's gone.294 passed, 1 skipped, ruff clean. Verified on
sub-CA001/ses-S006: convert →{unit: microvolts}, chained retarget to volts exact to 1e-6, and concat of two converted streams still merges.