Skip to content

fix(control): deliver send-key to the PTY; expose select-workspace#114

Open
tairea wants to merge 2 commits into
am-will:mainfrom
tairea:fix/send-key-keycode
Open

fix(control): deliver send-key to the PTY; expose select-workspace#114
tairea wants to merge 2 commits into
am-will:mainfrom
tairea:fix/send-key-keycode

Conversation

@tairea

@tairea tairea commented Jul 12, 2026

Copy link
Copy Markdown

Two defects made limux undrivable from the control socket. Together they broke the four-verb loop (send / send-key / read-screen / identify) that agent orchestration depends on. Both are fixed here.

I found these while porting disler/learning-cmux-with-agents to limux and discovering that no prompt in it could actually run.

1. send-key never reached the PTY

Terminal::send_key synthesized a ghostty key event with keycode: 0:

let press = translate_key_event(GHOSTTY_ACTION_PRESS,, keyval, 0, modifier);
//                                                                ^ no keycode

Ghostty resolves the physical key from the hardware keycode and does not fall back to the keyval, and translate_key_event always sets text: ptr::null(). So every key injected over the socket arrived as an unidentified key and was dropped.

The failure was silent — the caller still got OK:

$ limux send     --workspace $WS 'echo hello'
OK surface:22:terminal-0
$ limux send-key --workspace $WS enter
OK surface:22:terminal-0          # ← looks fine
$ limux read-screen --workspace $WS --lines 2echo hello                      # ← never ran

Text just accumulated at the prompt forever. No command sent over the control socket could execute, so no agent could be started or prompted. Reproduced on 0.1.19 and 0.1.21, focused and unfocused, across enter / Enter / Return / KP_Enter / cr / newline, and with a trailing \n/\r inside send.

The GTK key controller passes the real keycode (connect_key_pressed); only the socket path passed 0.

Fix: map the keyval back to a hardware keycode through the display's keymap — the inverse of the existing keyval_unicode_unshifted — so socket keys are encoded exactly like keys typed by a human.

2. Panes in a background workspace never realize

A pane only realizes its ghostty surface once its workspace is displayed. A split made in a workspace that isn't selected stays unrealized: surface-health reports healthy=false, and send_key returns false, which the bridge surfaces as the misleading -32602: unsupported key (the key was fine — the surface didn't exist).

The host already implements workspace.select, and limux-cli even calls it internally — it just wasn't reachable as a command. So a CLI-driven fleet could not bring its own workspace forward and could never start.

Fix: expose limux select-workspace --workspace <id|ref>.

Verification

End-to-end on Wayland / GTK4, with both patches applied:

$ limux send --workspace $WS 'echo FIXED-$((6*7))'; limux send-key --workspace $WS enter
$ limux read-screen --workspace $WS --lines 4echo FIXED-$((6*7))
FIXED-42                                  # ← it runs

Full loop: create a workspace → select-workspace → split a 2×2 → every pane healthy=true → drive each pane independently with send-key --surface → launch an interactive REPL in one pane, prompt it over the socket, and read the answer back off the screen:

>>> print('THE-ANSWER-IS', 6*7)
THE-ANSWER-IS 42

That is the agent-TUI pattern working: agent-team and CLI-driven fleets become possible.

Gate

  • cargo fmt --check — clean
  • cargo clippy --workspace --all-targets -- -D warnings — clean (exit 0)
  • cargo test --workspacehook_session_id_falls_back_to_transcript_stem fails, but it also fails on an unmodified checkout (and is the known failure called out in CLAUDE.md). Unrelated to this change.

Notes for review

  • I did not add a regression test: exercising send_key needs a live GTK display, and I couldn't see a headless harness in the tree. Happy to add one if you can point me at the right seam — I know the repo convention asks for it.
  • I built against the prebuilt libghostty.so from the 0.1.21 release tarball (dropped into ghostty/zig-out/lib/) rather than building the submodule, since I didn't have zig. The change itself doesn't depend on that; a clean-room build should use the real submodule.

Two defects made limux undrivable from the control socket. Both are fixed
here, and together they close the four-verb loop (send / send-key /
read-screen / identify) that agent orchestration depends on.

1. send-key never reached the PTY.

   Terminal::send_key synthesized a ghostty key event with `keycode: 0`.
   Ghostty resolves the *physical* key from the hardware keycode and does
   not fall back to the keyval, so every injected key arrived as an
   unidentified key and was dropped. The caller still got `OK`, so the
   failure was silent: `limux send "echo hi"` followed by
   `limux send-key enter` left the text sitting at the prompt forever.
   No command sent over the socket could ever execute.

   Map the keyval back to a real keycode through the display's keymap
   (the inverse of the existing keyval_unicode_unshifted) so socket keys
   are encoded exactly like keys typed by a human.

2. Panes in a background workspace never realized.

   A pane only realizes its ghostty surface once its workspace is
   displayed. Splits made in a workspace that isn't selected stay
   unrealized: surface-health reports healthy=false and send_key returns
   false, which the bridge reports as the misleading "unsupported key".
   The host has always implemented `workspace.select` — it simply was not
   reachable from the CLI, so a CLI-driven fleet could not bring its own
   workspace forward and could never start.

   Expose it as `limux select-workspace --workspace <id|ref>`.

Verified end to end on Wayland: create a workspace, select it, split a
2x2, launch an interactive REPL in one pane, prompt it over the socket and
read the answer back off the screen.

@bvolpato bvolpato left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocking: keycode mapping fixes Return and other physical keys, but printable send-key remains dropped. translate_key_event still leaves text null; Ghostty needs UTF-8 text for ordinary printable input. limux send-key a therefore still returns OK without writing a.

Keep mapped keycode and populate press text for printable keys. #112 covers missing text but not physical-key semantics; combined behavior is needed.

select-workspace wiring looks correct.

(Review assisted by gpt-5.6-sol)

Addresses @bvolpato's review. He is right, and the fix was incomplete.

A socket-injected key needs BOTH halves of the ghostty key event, and supplying
only one drops it silently:

  keycode only  ->  `limux send-key a` returns OK and writes nothing
  text only     ->  Enter, arrows, F-keys and ctrl-chords cannot be encoded

The original commit fixed the first half (keycode 0 meant ghostty saw an
unidentified physical key and dropped the event), which made Enter and arrows
work and closed the four-verb loop. But `translate_key_event` still left `text`
null, so ordinary printable input had nothing to write and vanished exactly as
before. It went unnoticed because orchestration uses `send` for text and
`send-key` only for Enter.

The codebase already says so, 550 lines above the bug, on the GTK key controller:

    // Send key events with the text field populated. Ghostty uses the
    // text field for actual character input and the keycode for bindings.

So: mirror the GTK path. Populate `press.text` from the existing `key_event_text`
helper, keeping the CString alive across `ghostty_surface_key`; leave the release
event textless, as the GTK controller does. The GTK path sources this text from
the input method -- a socket-injected key has no IM behind it, so derive it the
same way GTK's own fallback does.

`key_event_text` returns None for control characters, and that is load-bearing
rather than incidental: Enter and ctrl-chords must be *encoded* by ghostty from
key + mods, not written as literal bytes. Writing "\r" as text instead of letting
ghostty encode Return is how you break the kitty keyboard protocol.

This makes am-will#112 and this PR complementary rather than alternatives -- am-will#112 supplies
the text half, this supplies the physical-key half, and both are required.

Verified live on Wayland, all four classes at once, because fixing one by breaking
another is the obvious failure mode here:

  printable   `send-key e,c,h,o,space,h,i`  -> `echo hi` appears at the prompt
  Enter       `send-key enter`              -> it executes (once)
  arrows      `send-key up` then enter      -> history recalled; it executes twice
  ctrl-chord  `send-key '<ctrl>c'`          -> ^C interrupts a running `sleep 60`

Regression test pins the contract for all three key classes.
@tairea

tairea commented Jul 13, 2026

Copy link
Copy Markdown
Author

Thanks — you're right, and I've pushed a fix (02c40c3).

The printable gap reproduces exactly as you describe:

$ limux send-key --workspace $WS a
OK surface:1:terminal-0        # ...and the prompt line stays empty

The answer turned out to be written in this repo already, on the GTK key controller about 550 lines above the bug:

// Send key events with the text field populated. Ghostty uses the
// text field for actual character input and the keycode for bindings.

So a socket-injected key needs both halves, and supplying either alone drops it silently:

  • keycode onlysend-key a returns OK and writes nothing (this PR's original bug).
  • text only → Enter, arrows, F-keys and ctrl-chords can't be encoded at all.

The fix mirrors the GTK path: populate press.text from the existing key_event_text helper, keeping the CString alive across ghostty_surface_key, and leave the release event textless. The GTK path sources that text from the input method; a socket-injected key has no IM behind it, so it derives the same value GTK's own fallback would.

Worth noting for anyone touching this later: key_event_text returning None for control characters is load-bearing rather than incidental. Enter and ctrl-chords must be encoded by ghostty from key + mods, not written as literal bytes — writing \r as text instead of letting ghostty encode Return is how you'd break the kitty keyboard protocol.

It went unnoticed because agent orchestration uses send for text and send-key only for Enter, so the four-verb loop closed while printable send-key stayed broken. Good catch.

On #112

These are complementary, not alternatives@calebcarvalho's PR supplies the text half, this one supplies the physical-key semantics, and both are required. I've folded the text handling in here so this PR stands alone, but I'm equally happy to rebase onto #112 and drop the overlap if you'd rather land that first. Either order works; they just both need to land.

Verified live

All four key classes at once, on Arch/Wayland against an isolated XDG_DATA_HOME — since fixing one class by breaking another is the obvious failure mode here:

printable send-key e,c,h,o,space,h,iecho hi appears at the prompt
Enter send-key enter → it executes (once)
arrows send-key up then Enter → history recalled; hi executes a second time
ctrl-chord send-key '<ctrl>c'^C interrupts a running sleep 60

Regression test added pinning the contract for all three key classes (printable carries text; control keys withhold it; non-textual keys have none).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants