Add cinematic post-processing tools and fix teeth-whitening artifact - #423
Open
gonzabotmdp wants to merge 1 commit into
Open
Add cinematic post-processing tools and fix teeth-whitening artifact#423gonzabotmdp wants to merge 1 commit into
gonzabotmdp wants to merge 1 commit into
Conversation
- musetalk/utils/blending.py: attenuate the artificial bright-white teeth MuseTalk sometimes generates in the mouth region, without touching skin or lips (which already blend well). - scripts/cinematic_postprocess.py: depth-aware background blur (circular bokeh, not gaussian) with a full-subject sharp mask (MediaPipe Selfie Segmentation, so hair is kept sharp too, not just face skin), plus a light cinematic color grade and an Instagram-optimized export (AAC stereo 48kHz, H.264 with faststart). - scripts/gfpgan_postprocess.py, scripts/realesrgan_only_postprocess.py, scripts/codeformer_postprocess.py: alternative face/background restoration post-processing options, useful for comparison. - download_weights.sh: small fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a set of post-processing scripts for the generated talking-head videos, plus a small fix for an existing artifact.
scripts/cinematic_postprocess.py: applies a depth-aware background blur (circular bokeh via a disk kernel, not a plain gaussian) so the background looks like it was shot with a fast lens. The sharp/subject region follows a full-person segmentation mask (MediaPipe Selfie Segmentation) instead of only the face — this keeps hair sharp too, which a face-only mask does not. Also applies a light, generic cinematic color grade (contrast curve, saturation, vignette, subtle chromatic aberration, minimal film grain) and exports with settings recommended for Instagram (H.264 + faststart, AAC stereo 48kHz).scripts/gfpgan_postprocess.py,scripts/realesrgan_only_postprocess.py,scripts/codeformer_postprocess.py: alternative face/background restoration post-processing options (GFPGAN, Real-ESRGAN, CodeFormer), useful as points of comparison for anyone deciding which restoration approach fits their use case.musetalk/utils/blending.py: adds_correct_teeth_whiteness(), a small correction for an artifact where generated teeth can come out unnaturally bright/white. It only affects very bright, low-saturation pixels in the lower half of the face crop, so skin and lips (which already blend correctly) are untouched.download_weights.sh: minor fix.Test plan
scripts/cinematic_postprocess.pyend-to-end on generated MuseTalk output at 1080x1920, confirmed background blur follows the depth map, subject mask keeps hair/face fully sharp, and the exported file plays correctly with proper audio sync.🤖 Generated with Claude Code