[Bug] Apple Health ECG CSV import rejects Apple's single-column sample layout

[HercoGC]

HealthLog version

1.37.19 (build d8cbe2596fb9)

Deployment type

Self-hosted (Docker), official arm64 image

What happened?

I found this while validating #775, which passed — thank you again for the very quick turnaround on that fix. During the otherwise successful Apple Health import, every ECG (EKG) CSV member discovered in the archive failed softly. The earlier successful import recorded the same all-member ECG failure outcome, so this appears to be a separate format-compatibility gap rather than a regression in the #775 repair.

I can see that HealthLog already has a dedicated Withings ECG sync, so for recordings made on a supported Withings or ScanWatch device, the Withings integration may well be the intended primary route. I would therefore treat this Apple archive path as a nice-to-have rather than a blocker. It would still be useful for portability, especially for Apple Watch recordings or ECGs written to HealthKit by another compatible device. HealthLog's v1.37.19 Apple Health documentation also describes the ECG archive reader as source-agnostic, which is why I thought this residual was worth reporting separately.

I checked only privacy-safe structure locally; no ECG filename, metadata value, waveform value, account identifier or archive content is included here. Every ECG member uses this general layout:

1. metadata key/value rows, including separate Lead,<lead name> and Unit,<unit> rows;
2. a blank separator; and
3. numeric waveform samples stored one per line in a single column.

None of the members contains a Lead,Voltage row. The v1.37.19 parser only changes into sample mode after that exact sentinel and then calls splitPair() for every non-empty line, so a single-column waveform row is rejected as malformed before samples can be imported.

The surrounding export.xml import still completes, which is good fail-soft behaviour, but no ECG member in this valid Apple export layout can be imported.

Expected behaviour

If Apple Health archive ECG import is intended to remain supported alongside the Withings integration, HealthLog should accept Apple's metadata-plus-single-column waveform layout as well as the currently supported paired-column layout, while keeping the existing resource limits and privacy protections. If a genuinely unsupported member is encountered, the terminal breakdown should retain a bounded, non-sensitive reason category.

Steps to reproduce

1. Export Apple Health data containing ECG CSV members in the metadata-plus-single-column waveform layout described above.
2. Import the archive through Settings → Export & Import.
3. Wait for the otherwise successful import to finish.
4. Observe that every discovered ECG member is counted as failed and no ECG is created from the archive.

Workaround used

Existing ECG records that arrive through the Withings integration or HealthLoger/native sync remain available. I have not modified or converted the private Apple archive.

Possible solution

• Detect the waveform section after the metadata separator when the declared lead and unit are already present.
• Accept a bounded single numeric sample per row in addition to the existing Lead,Voltage paired-row form.
• Convert according to the declared unit before applying the existing safe-integer and amplitude bounds.
• Add synthetic regression fixtures for both known layouts without patient labels or real waveform data.
• Keep per-member failures fail-soft so one malformed ECG does not discard supported export.xml records.

Relevant logs / screenshots

No private logs, screenshots, filenames or health values are included. The terminal import breakdown reported that every discovered ECG member failed while the main import completed.

Browser / OS

HealthLog web app on macOS 27 arm64 with Docker Desktop.

Pre-flight checks

• I searched this repository for ECG, electrocardiogram, Apple Health CSV, failed ECG and Lead,Voltage and found no matching issue.
• I redacted all personal data, identifiers, filenames and health values.

[MBombeck]

Confirmed, and once again your structural analysis is exactly right. parseAppleHealthEcgCsv only switches into sample mode after the literal Lead,Voltage pair, blank lines are skipped before that check ever matters, and every single-column waveform row then fails splitPair with the fixed "row is malformed" reason. So an archive in Apple's metadata-plus-single-column layout can never import, and the per-member fail-soft you observed is the parser working as designed around a layout it simply does not know.

The Apple archive path is intended to stay supported alongside the Withings route, precisely for the portability cases you name (Apple Watch recordings, third-party devices writing to HealthKit), so this gets fixed rather than documented away. The shape will follow your sketch: the parser treats the blank separator as the transition into the waveform section whenever the metadata already carries a lead and a unit, accepts one bounded numeric sample per row in that mode, converts according to the declared unit before the existing safe-integer and amplitude bounds apply, and keeps the paired Lead,Voltage form working unchanged. Both layouts get synthetic fixtures with no real waveform data, and a member that matches neither layout keeps a bounded, non-sensitive failure reason.

The fix rides the next release. I will report back here when it is live so you can re-run the archive. Refs #794

[MBombeck]

Fixed in v1.37.20. The importer only understood the paired Lead,Voltage layout; your archive uses Apple's single-column form, where the lead and unit are metadata rows, a blank line separates them from the waveform, and each following line is one bare reading. The parser now recognises that layout, converts the declared unit before validation (both microvolt and millivolt exports), and keeps the paired layout working exactly as before. Both layouts are covered by fixtures in the test suite. A re-run of your archive import should now bring the recordings in; a short confirmation once you have tried it would be appreciated. Refs #794