Skip to content

fix(vision-mixer): enforce framerate on PGM/MV outputs in GL passthrough#549

Merged
srperens merged 2 commits into
mainfrom
fix/vision-mixer-mv-framerate-glpassthrough
May 6, 2026
Merged

fix(vision-mixer): enforce framerate on PGM/MV outputs in GL passthrough#549
srperens merged 2 commits into
mainfrom
fix/vision-mixer-mv-framerate-glpassthrough

Conversation

@srperens
Copy link
Copy Markdown
Collaborator

@srperens srperens commented May 6, 2026

Summary

  • The GPU pipeline path with gl_download=false (the default) had no capsfilter on either the PGM or multiview compositor output, so pgm_framerate and multiview_framerate properties were silently ignored
  • PGM often appeared correct because downstream blocks (e.g. SDI/EFP outputs) propagated framerate caps back through negotiation, but the multiview output had no such downstream constraint and ran at the input rate
  • Add a capsfilter with video/x-raw(memory:GLMemory) features in both gl_download=false branches so the configured framerate/resolution is enforced while keeping buffers in GL memory

Root cause

In vision_mixer/builder.rs, the GPU pipeline conditionally added a gldownload + capsfilter chain when gl_download=true. The gl_download=false branch went straight from compositor → tee → output queue with no caps constraint at all. With the default gl_download = false, the multiview_framerate property had no observable effect.

Test plan

  • cargo check clean
  • cargo build (debug) clean
  • cargo build --release clean
  • All 408 existing tests pass (cargo test), incl. pipeline_lifecycle_test and openapi_test
  • Pre-commit (fmt + clippy + secret check) passes
  • Verify on frank: build runs, MV WHEP stream actually delivers configured framerate (e.g. 24fps when multiview_framerate=24/1) — observable in chrome://webrtc-internals framesPerSecond field

🤖 Generated with Claude Code

Per Enstedt and others added 2 commits May 6, 2026 15:49
The GPU pipeline path with gl_download=false (the default) had no capsfilter
on either the PGM or multiview compositor output. As a result, the
pgm_framerate and multiview_framerate properties were silently ignored
whenever the GL-memory passthrough path was taken — the compositor produced
at the input rate instead of the configured rate.

In practice, PGM often appeared correct because downstream blocks (e.g. SDI/
EFP outputs with strict framerate caps) propagated their constraints back
through caps negotiation, but the multiview output had no such downstream
constraint and ran at the input rate.

Add a capsfilter with `video/x-raw(memory:GLMemory)` features in both
gl_download=false branches so the configured framerate/resolution is
enforced while keeping buffers in GL memory.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@srperens srperens force-pushed the fix/vision-mixer-mv-framerate-glpassthrough branch from 429a280 to e3d5ceb Compare May 6, 2026 13:49
@srperens srperens merged commit 50f9893 into main May 6, 2026
7 checks passed
@srperens srperens deleted the fix/vision-mixer-mv-framerate-glpassthrough branch May 6, 2026 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant