Skip to content

Skipping MP4 fragments for packs that carry only parameter sets - #98

Merged
wberube merged 1 commit into
OpenIPC:masterfrom
erkin-io:fix/mp4-parameter-set-packs
Nov 17, 2026
Merged

wberube merged 1 commit into
OpenIPC:masterfrom
erkin-io:fix/mp4-parameter-set-packs

Conversation

@erkin-io

Copy link
Copy Markdown
Contributor

What was wrong

On HALs that deliver SPS, PPS (and VPS) in packs of their own, the fragmented MP4 output (/video.mp4 and the recorder) writes the previous frame again for each such pack, with its audio. Seen on a Ruision F8 board (FH8856V200), which delivers a key frame as separate SPS, PPS and slice packs: a 60 s /video.mp4 capture at 25 fps had 1560 video frames instead of 1500, and the repeated audio showed as gaps and jumps.

Why

send_mp4_to_client() (src/server.c) and record_ingest_stream() (src/record.c) loop over the packs of a stream. For each pack they feed its NAL units to the muxer and then always emit a fragment. A slice NAL (mp4_set_slice()) is what fills the fragment; SPS and PPS only update the stored parameter sets. For a pack with no slice, the fragment buffers still hold the previous frame (and the audio queued with it), which is then sent or written a second time.

What this changes

  • send_mp4_to_client(): note whether the pack had a slice and skip the fragment if it did not.
  • record_feed_nalu() returns whether it fed a slice; record_ingest_stream() skips the fragment for packs without one.

Packs that carry parameter sets together with a slice (the usual case on other HALs) are not affected.

How it was checked

  • Built with ./build.sh fhan-v2, no new warnings; ./test.sh -a passes.
  • On a Ruision F8 board, in the firmware that normally runs on the board, with this change in the same form: a 60 s /video.mp4 capture had 1500 frames, all 40 ms apart, and no audio gaps; recordings likewise. Before, 1560 frames in 60 s.

Effect on other platforms

Shared code. Only packs without a slice change behaviour (no fragment instead of a repeated one). If a HAL puts a slice and parameter sets in one pack, nothing changes.

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@qodo-free-for-open-source-projects

Copy link
Copy Markdown

PR Summary by Qodo

Skip MP4 fragments for packs without video slices

🐞 Bug fix 🕐 10-20 Minutes

Grey Divider

AI Description

• Prevent parameter-set-only packs from repeating the previous MP4 frame and its audio.
• Apply slice-based fragment gating to both HTTP streaming and recording.
• Preserve fragment output for packs containing both parameter sets and a slice.
Diagram

graph TD
    Pack["HAL video pack"] --> Feed["Feed NAL units"] --> Slice{"Slice fed?"}
    Slice -->|Yes| Emit["Emit MP4 fragment"] --> HTTP["HTTP client"]
    Emit --> File["Recording file"]
    Slice -->|No| Skip["Skip fragment"]
Loading
High-Level Assessment

Keep the per-pack checks in the HTTP and recording ingestion paths. The muxer accepts individual NAL units and has no pack boundary with which to decide whether a new fragment should be emitted; moving the check there would require a broader API change for a localized fix.

Files changed (2) +23 / -10

Bug fix (2) +23 / -10
record.cGate recorded fragments on a fed video slice +14/-7

Gate recorded fragments on a fed video slice

• The NAL-unit feeder now reports whether it fed a slice. Recording skips fragment emission for packs without one, while continuing to process their parameter sets.

src/record.c

server.cSuppress HTTP MP4 fragments for slice-free packs +9/-3

Suppress HTTP MP4 fragments for slice-free packs

• The streaming path tracks whether a pack fed a video slice and sends a fragment only when it did. Parameter sets are still passed to the muxer.

src/server.c

@qodo-free-for-open-source-projects

qodo-free-for-open-source-projects Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (0) 📎 Requirement gaps (0) 🎨 UX issues (0) 🔗 Cross-repo conflicts (0) 📜 Skill insights (0)

Grey Divider


Action required

1. Some HEVC frames vanish from MP4 output 🐞 Bug ≡ Correctness
Description
gotSlice is set only for NAL types 1, 5, and 19, although supported HALs also deliver HEVC slices
with types such as 0, 2, and 21. When one of those slices occupies its own pack, both the client and
recorder paths skip that pack, shortening the resulting video instead of limiting the change to
parameter-set-only packs.
Code

src/server.c[271]

+        if (!anyClient || !gotSlice) continue;
Evidence
The shared slice cases cover types 1, 5, and 19, while the T31 HAL defines and forwards other HEVC
slice types. The newly added gates now discard packs carrying those types, rather than emitting a
fragment for them.

src/server.c[259-271]
src/record.c[175-190]
src/record.c[365-372]
src/hal/inge/t31_venc.h[31-52]
src/hal/inge/t31_hal.c[685-696]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The new fragment gates skip packs containing valid HEVC slice types that the existing H.264-oriented slice cases do not recognize.
## Fix Focus Areas
- src/server.c[263-271]
- src/record.c[175-190]
- src/record.c[365-372]
## Recommended Fix
Classify coded slices according to `isH265` in both paths. Feed recognized HEVC slices to the muxer and mark their packs as containing a slice; distinguish key pictures from non-key pictures using the HEVC type.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

2. Offset video packs lose recorder fragments 🐞 Bug ≡ Correctness
Description
record_ingest_stream() passes NAL offsets and lengths to record_feed_nalu() after shifting the
data pointer and reducing the available length by pack->offset. On HALs that retain offsets
relative to the original buffer, a valid slice fails that validation, leaves gotSlice false, and
now causes the recorder to skip its fragment.
Code

src/record.c[372]

+        if (!gotSlice) continue;
Evidence
Hisi v2 and CVI pass through pack->offset as the NAL offset and the original pack length as the
NAL length. The recorder shortens its buffer first, so a nonzero offset makes the NAL fail the
bounds check; the added gotSlice condition turns that failure into a skipped fragment.

src/record.c[151-163]
src/record.c[357-372]
src/hal/hisi/v2_hal.c[815-823]
src/hal/plus/cvi_hal.c[1014-1024]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The recorder's new slice gate depends on validation that rejects valid HAL packs whose NAL offsets refer to the original buffer.
## Fix Focus Areas
- src/record.c[151-166]
- src/record.c[357-372]
## Recommended Fix
Validate and read each NAL against a consistent buffer origin. Normalize HAL-provided offsets and lengths before calling `record_feed_nalu()`, then use the normalized result to decide whether to emit a fragment.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Tip of the day
💡 Did you know, you can enable the Remediation agent and Qodo fixes findings in a dedicated fix PR

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

Grey Divider

Qodo Logo

@erkin-io

Copy link
Copy Markdown
Contributor Author

Reply to the automatic review. No change to the branch; both points are gaps that exist without this change too.

  1. H.265 slice types other than 1, 5 and 19 (T31): such slices never reach mp4_set_slice() today, in server.c or in record.c. Before this change a pack holding only such a slice still produced a fragment, but one that repeated the previous picture and its audio; with the change it produces none. The real fix is to classify H.265 slices by their own types in both paths, which is a separate change; I can send it if you want it.
  2. HiSilicon v2 and CVI packs with an offset: if record_feed_nalu() rejects these NAL units, it already did before this change, so no slice reached the muxer and every fragment repeated stale data; with the change those packs make no fragment. Either way the offset handling of the recorder for those HALs is its own question. I have no such board to check how the offsets are filled at run time.

🤖 Generated with Claude Code

@wberube
wberube merged commit 7f018a9 into OpenIPC:master Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants