feat: attach renders to a live Julia session - #418
Draft
MichaelHatherly wants to merge 6 commits into
Draft
MichaelHatherly wants to merge 6 commits into
MichaelHatherly wants to merge 6 commits into
Conversation
Add attach mode: a notebook with `julia.attach: true` in its frontmatter renders inside a running interactive session instead of a spawned worker process. The session opts in with `QuartoNotebookWorker.serve!()`, which serves the worker protocol on a socket and registers itself in a registry directory keyed by project root. The host resolves the notebook to a registered session, connects instead of spawning, and never terminates or restarts the session it does not own; a dead connection re-attaches on the next render. Cells still evaluate in an isolated notebook module refreshed per render. What is shared is the process: packages stay loaded, compiled code stays warm, and cells reach session state explicitly through `Main`. `serve!` keeps `QuartoNotebookWorker` on the `LOAD_PATH` so fresh notebook modules resolve it after a render activates the notebook's own project. Verified end-to-end: first render attaches and sees session state, re-renders reuse the warm process with a fresh module, `close!` disconnects without killing the session, and a later render re-attaches.
Integration tests: a render attaches to a live session, sees its state through Main, re-renders warm with a fresh module, survives close! and re-attaches; a missing session surfaces a UserError pointing at QuartoNotebookWorker.serve!().
Attach used a `julia.attach: true` frontmatter flag, which travels with the shared `.qmd`. A notebook now attaches whenever a live session serves its root, so the opt-in is starting the session, not editing the file. `QUARTONOTEBOOKRUNNER_NO_ATTACH=1` forces a spawned worker. The serving session prints `attached render: <path>` per render so the implicit behavior is visible. The attach connection skips the manifest-in-sync check: the session's ambient project is the user's, and each render activates the notebook's own project. The convenient entrypoint moves to `QuartoTools.attach!()`, which loads the worker without pulling `QuartoNotebookRunner` into the notebook's project; the `QuartoNotebookRunner.attach!` wrapper is removed.
MichaelHatherly
changed the title
The `File` constructor called `init!`, and every render calls `init!` again through `refresh!`, so the first render of an attached notebook sent two `NotebookInitRequest`s and the session printed `attached render:` twice. A file is only ever constructed in `_create_file`, reached from `run!`, whose callback runs `evaluate!` before any cell executes, so the constructor's `init!` was always redundant. Drop it. The remaining `init!` runs under `file.lock` rather than `server.lock`.
Each attached notebook opens its own connection to the session, and a file keeps that connection open across renders. The accept loop served connections inline and sequentially, so a second notebook, or a second runner, blocked in the attach handshake until the first disconnected. Spawn a task per accepted connection so handshakes complete immediately. Renders mutate process-global state (working directory, active project, environment), so serialize dispatch on `_RENDER_LOCK`: connections are accepted concurrently, renders run one at a time.
The session announces renders with `printstyled(color = :cyan)`. Under forced color the line arrives wrapped in ANSI escapes, so `startswith(line, "attached render:")` failed in CI while passing locally. Strip the escapes before asserting on the text.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #418 +/- ##
==========================================
- Coverage 81.49% 80.57% -0.92%
==========================================
Files 73 73
Lines 2923 3069 +146
==========================================
+ Hits 2382 2473 +91
- Misses 541 596 +55 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Lets
quarto renderevaluate a notebook inside a live user REPL instead of a spawned worker process, so packages stay loaded and compiled code stays warm across renders. Cells still run in an isolated notebook module refreshed per render; what is shared is the process, reached throughMain.QuartoNotebookWorker.serve!()(orQuartoTools.attach!()).QUARTONOTEBOOKRUNNER_NO_ATTACH=1forces a spawned worker.attached render: <path>per render.The REPL-side entrypoint lives in QuartoTools, not here, so users don't pull
QuartoNotebookRunnerinto their notebook project: PumasAI/QuartoTools.jl#31.