Skip to content

fix: real-time axis, non-UTF-8 file support, and OOM on large logs#12

Merged
kaklik merged 1 commit into
calibrationfrom
parser-fix
May 20, 2026
Merged

fix: real-time axis, non-UTF-8 file support, and OOM on large logs#12
kaklik merged 1 commit into
calibrationfrom
parser-fix

Conversation

@kaklik

@kaklik kaklik commented May 6, 2026

Copy link
Copy Markdown
Member
  • UnicodeDecodeError fix: log files are now opened with encoding="ascii", errors="replace", so non-ASCII bytes in file headers no longer crash the parser.

  • Real-time axis: the parser now reads the $TIME record present in AIRDOS04C fw 2.x logs and extracts the rtc_unix_offset field (= Unix timestamp − RTC counter). All timestamps in time_axis and telemetry are converted to Unix epoch. PlotCanvas detects the has_real_time metadata flag and switches to pg.DateAxisItem so graphs show actual UTC date/time instead of relative minutes. A SmartDateAxisItem subclass adds the calendar date to the first tick of each new day, so the date is always visible at midnight transitions (or at the first visible tick after opening a file). Logs without a $TIME record (old format) are unaffected.

  • OOM fix for large files: AirdosV2LogParser now runs a fast pre-scan pass over the file to find the highest channel index used in $E and $STOP records, then allocates hist and every current_hist copy only to that size with dtype=int32. For a file with max channel 7 647 and 11 001 records, the spectral_matrix shrinks from ~5.8 GB (65 536 ch × int64) to ~320 MB (7 648 ch × int32), making files that were previously killed by the OOM killer parseable in under a second.

@kaklik kaklik requested a review from roman-dvorak May 6, 2026 22:16
@kaklik kaklik merged commit f6e34e4 into calibration May 20, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant