Skip to content

Treat VS16 emoji-presentation sequences as double width - #32

Open
Junyi-99 wants to merge 1 commit into
doy:mainfrom
Junyi-99:fix/vs16-emoji-double-width
Open

Treat VS16 emoji-presentation sequences as double width#32
Junyi-99 wants to merge 1 commit into
doy:mainfrom
Junyi-99:fix/vs16-emoji-double-width

Conversation

@Junyi-99

Copy link
Copy Markdown

Problem

A text-presentation base codepoint followed by U+FE0F (variation selector-16) requests emoji presentation, which modern terminals and unicode-width's string-level width() render as two columns — e.g. ❤️ (U+2764 U+FE0F), ⚠️ (U+26A0 U+FE0F).

vt100 sets a cell's wide flag from the base char alone (c.width() > 1 in Cell::set) and appends the VS16 as a zero-width combining char without re-checking, so the sequence is stored as a single narrow cell. Every column after the emoji is then shifted by one, which shows up as on-screen residue when the line is redrawn or scrolled.

Repro:

let mut p = vt100::Parser::new(1, 20, 0);
p.process("A\u{2764}\u{FE0F}B".as_bytes());
// before: cell(0,1) "❤️" is_wide() == false, cell(0,2) == "B"  (shifted)
// after:  cell(0,1) is_wide(), cell(0,2) is_wide_continuation(), cell(0,3) == "B"

Fix

In Screen::text's combining-character (width == 0) branch: after appending the combiner, if the cell's full contents now measure two columns by UnicodeWidthStr::width while the cell is still flagged narrow, promote it to wide and insert a wide-continuation cell. If that continuation column already held the first half of an existing wide glyph, clear that glyph's continuation too — mirroring the existing width > 1 placement path so no continuation is left orphaned.

Keying on the string-level width keeps the parser in agreement with unicode-width (and the terminals driving it).

Tests

tests/emoji_width.rs covers the core promotion and the overlapping-wide-glyph (orphan-continuation) case. Existing suite stays green.

🤖 Generated with Claude Code

A text-presentation base codepoint followed by U+FE0F (variation
selector-16) requests emoji presentation, which terminals and
unicode-width's string-level width() render as two columns. The cell's
wide flag was computed from the base char alone (width 1) and the VS16
was appended as a zero-width combining char without re-checking, so the
sequence was stored as a single narrow cell, shifting every column after
it by one.

In Screen::text's combining-character branch, when appending the
combiner grows the cell's contents to double width by UnicodeWidthStr
measure while the cell is still flagged narrow, promote it to wide and
insert a wide-continuation cell. If that continuation column already
held the first half of an existing wide glyph, clear that glyph's
continuation too, mirroring the width > 1 placement path so nothing is
left orphaned.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 29, 2026 07:09

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Fixes a width-handling bug where a base character followed by U+FE0F (VS16) was stored as a single narrow cell, causing subsequent text to shift and leave on-screen residue. The fix detects when a combining character promotes a base to double-width emoji presentation and widens the cell, adding a wide continuation while cleaning up any glyph it clobbers.

Changes:

  • In screen.rs, after appending a zero-width char to the base cell, detect when the resulting string width exceeds 1 and the cell is still narrow; widen it, add a wide-continuation cell, and clear any orphaned continuation from a pre-existing wide glyph.
  • Expose Cell::set_wide as pub(crate) so Screen can promote a cell to wide.
  • Add regression tests in tests/emoji_width.rs covering basic VS16 promotion and promotion over an existing wide character.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
src/screen.rs Implements VS16 width promotion logic when a combining char widens the base cell.
src/cell.rs Makes set_wide crate-visible so screen.rs can flag the promoted cell as wide.
tests/emoji_width.rs Adds regression tests for VS16 emoji width handling, including the orphaned-continuation case.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/screen.rs
Comment on lines +789 to +790
let cont_col = base_col + 1;
let promote = cont_col < size.cols && {
Comment thread tests/emoji_width.rs
Comment on lines +57 to +61
assert_eq!(c[1].2, true, "col1 = ❤️ continuation");
assert!(
!c[2].2,
"col2 (clobbered 中's old continuation) must be cleared, not orphaned"
);
Comment thread src/screen.rs
Comment on lines +842 to +844
if pos.col == cont_col {
self.grid_mut().col_inc(1);
}
psmux added a commit to psmux/psmux that referenced this pull request Aug 4, 2026
…uests emoji presentation

Emoji presentation is a property of the sequence, not of any single
character. U+2733 is one column on its own, but U+2733 U+FE0F is two, in
real terminals and in tmux alike. Screen::text measured width one char at
a time with UnicodeWidthChar::width(), so the base settled the cell at one
column and the VS16 that followed was folded in afterwards as a zero width
combining mark. The grid ended up one column narrower than the child
believed, every column after the sequence drifted left by one, and the
skipped cell kept the previous frame content.

Reproduced before any code was read, straight through the parser:

  p.process("\u{2733}\u{FE0F}AB")   ->  B lands on col 2 where A belongs,
                                        cursor stops at col 1

tmux 3.4 parity oracle, cursor_x after writing each sequence:

  U+2733         1        U+2733 U+FE0F   2
  U+2733 U+FE0E  1        U+1F4DB         2
  space U+FE0F   2        U+1F4DB U+FE0F  2

The fix mirrors tmux screen_write_combine: when a zero width char lands on
a cell whose stored width is still 1 and either the char is VS16 or the
whole cell now measures two columns, the cell already in the grid is
promoted to wide, the following cell becomes its continuation and the
cursor steps over it. tmux ships variation-selector-always-wide on by
default, so promotion is the parity behaviour. An already wide cell is
never promoted again, so an emoji plus a redundant VS16 stays at two
columns instead of growing to four.

Because promotion is applied to the cell sitting in the grid rather than to
a pending grapheme, it does not require the base and the selector to arrive
in the same process() call. A PTY read splitting the pair is routine and is
covered across four arrangements.

Credit to Junyi Hou for the upstream approach in doy/vt100-rust#32, and to
kt81 for the report, the chunk boundary property and the port.

Tests
- crates/vt100-psmux/tests/issue533_vs16_width.rs, 14 tests: the reported
  drift, wide flag plus continuation, every chunk boundary including mid
  UTF-8, clobbered wide glyph continuation cleanup, last column safety,
  contents_formatted round trip, and regression guards for VS15, the bare
  base, plain wide emoji, keycap sequences and the #441 combining marks.
  9 of the 14 failed before this change and all 14 pass after.
- tests/test_issue533_vs16_width.ps1: pane level E2E over CLI, TCP and an
  attached Win32 TUI client. It probes the pre parse stream with
  PSMUX_PANE_RAW first, because without ConPTY passthrough conhost resolves
  the geometry itself and hands psmux an already composed row. On such a
  build the pane cannot discriminate this bug, so the column assertions are
  skipped with that stated rather than passed misleadingly.
- Full workspace cargo test green, zero regressions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants