Skip to content

v18: 23 of 33 decoded fields have no downstream consumer — inventory, and what should read them #845

Description

@vishk23

decodeWhoop5Historical emits 33 named fields per v18 record (Interpreter.swift). Seven physiological signals plus the row timestamp reach StreamStore and the analytics engine. The remaining 23 exist only as decoder output — visible in the frame inspector and whoop-decode, never persisted, never scored, never shown. The strap trims its history once NOOP acks the offload, so those values are gone permanently.

This isn't a criticism of the decode work — the decode is the hard part and it's done. It's a question about the other end of the pipe.

Precedent that this is worth doing

ad4cc1f4 ("Sleep: band sleep-state stream end-to-end") is exactly this pattern, in its own words:

The WHOOP band emits its own sleep-state in the history frames; it was decoded but dropped at extraction, starving a fully-built but dormant confirm-only sleep guard.

Wiring sleep_state (@81) through decode → store → the stager's re-onset guard cost one extraction line and one migration, and activated analytics already written. PR #738 is the follow-on using the same signal for a false-wake veto: ~30 min/night of spurious wake recovered at ~2 min/night reverse error across 12 banded nights. One already-decoded byte, two analytics wins. The question is whether any of the other 23 are the same shape.

Reaches analytics (7 signals + unix)

heart_rate@22 · rr[]@24+2i · gravity_x/y/z@45/49/53 · step_motion_counter@57 · activity_class@63 · skin_temp_raw@73 · sleep_state@81(hi-nibble)

Decoded, no downstream reader (23)

Field Off Type Decoder's own note
hist_version @9 u8 logged only
record_index @11 u32 "per-record counter"
rr_count @23 u8 only reader is the 4.0 path, unreachable for v18
cardiac_flags @33 u8 "raw byte near the HR fields"
hr_fixed_8_8 @36 u16 "higher-precision HR: bpm = value/256"
rr_packed @38 u16 "meaning not pinned"
cardiac_status @40 u8 "raw status-like byte near the HR fields"
dynamic_acceleration @41 f32 folded into a transient diag, never stored
step_cadence @59 u8 "cadence-like byte (raw)"
motion_wear_quality @63 u8 "0=still/good, 1, 2=poor contact" (same byte as activity_class)
temp_aux_1_raw @69 i16 "°C = value/10; tracks skin_temp (corr ~0.92)"
temp_aux_2_raw @71 i16 "°C = value/10; tracks skin_temp (corr ~0.97)"
status_word @75 u16 "packed status word; not deep-sleep"
status_word_1 @77 u16 "near-static sibling of @75 (low nibble = 1)"
status_word_2 @79 u16 "sibling of @75 (low nibble = 2)"
onwrist @81 bits 0-1 "on-wrist/validity flag"
wake_quality @81 bits 2-3 "quality code; observed nonzero only in wake"
aux_byte_82 @82 u8 "nonzero only while sleep_state = asleep"
spo2_candidate_82 @82 u8 gated 70-100 instrumentation candidate (#103)
optical_baseline_106 @106 u16 raw optical/ADC baseline
optical_amp_a @108 u8 paired channel
optical_amp_b @109 u8 paired channel
unknown_f32_113 @113 f32 "purpose unknown, carried raw"

Every one does land in ParsedFrame.parsed and is printed by whoop-decode — "dropped" means no persistence, no scoring, no UI; not invisible.

What I'm asking

1. What is known about the genuinely-unclear ones? These have no in-tree attribution beyond "raw byte near X", and I'd rather ask than re-derive:

  • cardiac_flags@33 / cardiac_status@40 — the names imply someone knew something. Decompile, capture, or guess? Across my own records @33 shows only 4 distinct values, so it looks like a small enum rather than a measurement — and the changelog notes it "did not decode consistently" on at least one firmware. Worth knowing before anyone reads meaning into the names.
  • status_word_1@77 / status_word_2@79 — noted as near-static siblings of @75 distinguished by low nibble 1/2. Is the low nibble a channel index (three instances of one status word for three subsystems)? That would make them interpretable rather than opaque.
  • aux_byte_82 — its "nonzero only while asleep" note predates the spo2_candidate_82 decode and reportedly came from an out-of-tree ~258k-record corpus. Does that corpus still exist?
  • wake_quality@81 bits 2-3 — "observed nonzero only in wake". Given @81's high nibble turned out genuinely useful, the other four bits of the same byte seem worth a look.
  • optical_baseline_106 / optical_amp_a@108 / optical_amp_b@109 — are these PPG signal-quality/AGC telemetry? A signal-quality gate would be immediately useful for deciding which HR/HRV epochs to trust. (Explicitly not proposing these as a BP or SpO2 substrate — that inference chain has already had to be retracted once on the v20 optical.)

2. Two that look like plain wiring gaps rather than research questions:

  • hr_fixed_8_8@36 gives HR at 1/256 bpm; NOOP persists the integer heart_rate@22. Any reason not to prefer the fixed-point field where present? (HRV is R-R-derived so untouched, but resting-HR and the HR curve would see it.)
  • temp_aux_1_raw@69 / temp_aux_2_raw@71 correlate with skin_temp_raw@73 at ~0.92/~0.97 and follow the same diurnal curve. Three thermal channels, one persisted. Are the aux channels a redundancy/validity check — which would help the recurring "skin temp shows nothing" reports? Worth noting the raw channels are offset, not redundant (a worn fixture reads ~24.7 / 26.5 / 30.6 °C across @69/@71/@73), so any fusion has to happen in deviation-space, not by averaging raw values.

3. One that blocks work already in flight. spo2_candidate_82 is decoded on both platforms and the docs state plainly that "nothing downstream reads it". But #103's promote-gate needs multi-device nightly correlation, and tools/linux-capture/validate_spo2_candidate.py consumes capture JSON — so the app itself cannot currently contribute the data needed to resolve its own open question, only the offline tooling can. Persisting the in-band value as instrumentation (no card, no score, guard test untouched) would let ordinary users produce the multi-device evidence #103 has been waiting on.

4. The framing question. dynamic_acceleration@41 already has an honest in-tree answer to "should this be consumed?" —

"the decoder has emitted this field all along with nothing consuming it, so there is no evidence on whether it beats the gravity-delta stillness the stager derives today"

— resolved by counting it into a strap-log diagnostic first, and paying for a migration only if real nights justify it. Should the rest of the table follow that pattern? A generic "count these N fields into the strap log for a release" would produce evidence on all of them for roughly the cost of one.

Happy to do the legwork on any of these. What I'd rather not do is re-derive facts that already exist in someone's capture corpus or notes.

Related: #761 (capability roadmap), #548 (empty-state labels), #103 (@82), #423 (raw IMU), #738 (the band-state precedent).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions