Skip to content

Streaming agent chat with a live view of the agent graph - #5

Open
Jayanaka-98 wants to merge 1 commit into
mainfrom
agent-chat
Open

Streaming agent chat with a live view of the agent graph#5
Jayanaka-98 wants to merge 1 commit into
mainfrom
agent-chat

Conversation

@Jayanaka-98

Copy link
Copy Markdown
Collaborator

Adds a second tab that composes all four patterns into one conversational agent — real SSE streaming, with the agent's own architecture rendered beside the chat and lit up as it executes.

The agent is a graph, not a tool-loop

Perceive ──> Plan ──> Act ──> Synthesize
                       │
                       ├── Brainstorm   (Generate)
                       ├── Structure    (Extract)
                       ├── Research     (Invoke — tools/ReAct)
                       └── Mentors      (Route + Spawn)

Six properties, rather than tools bolted onto a loop:

The graph is the architecture Topology is data under root. agent_architecture() reads that same graph back, so the UI diagram is derived from the running system and cannot drift from it.
Behaviour lives on nodes Each capability implements its own can ... with CapabilityExec entry. Adding a capability adds a node, not a branch in a dispatch chain.
Typed contracts between stages Perceive returns Intent, Plan returns Plan — compiler-enforced objs, not free text the next stage re-parses.
The plan directs the route Small talk runs zero capabilities; a full pitch request runs four. Both verified.
Cognition ≠ transport Stages know nothing about HTTP. agent_chat_stream is a thin SSE adapter — swap it for a CLI and nothing changes.
The agent narrates itself Stages, capabilities and individual tool calls publish to one event bus, so the trace the UI draws is the execution path.

That last one is why capabilities run on a worker thread while the endpoint drains the bus: a tool call becomes a frame the moment it fires, not after the step finishes.

Streaming

def:pub ... -> Generator + report stream(), with the final answer streamed token by token via by llm(stream=True). Conversation history persists as Turn nodes in the graph. Real trace from a live run:

 0.0s  STAGE perceive   active
 1.3s  note  goal: build an app that helps students find study partners
 2.0s  note  selected: research
 2.4s   cap  research   active
 3.1s    TOOL search_github        call
 3.7s    TOOL describe_tech_stack  call
 5.2s    TOOL estimate_build_time  call
 9.2s   cap  research   done    github + stack + feasibility
 9.7s  [first token]  → 200 tokens streamed

sem granularity

Per the convention you asked for, prompts are split parameter- and field-wise: every by llm function has per-argument sem, every obj has per-field sem, and each Capability enum member documents itself — so the planner's prompt stays short and capability meaning lives in exactly one place (the same place Act routes on). 46 sem declarations in total.

Client-side care

Three things that are easy to get wrong and were hit here:

  • Stale reactive state after await. The transcript is accumulated in a local and assigned whole — reading turns back after an await returns the render-time snapshot and silently drops the user's own turn.
  • busy/draft reset in a finally. Without it, a stream that dies mid-flight leaves the composer permanently disabled and the turn unrecoverable. (This was the "can't type a new message" symptom.)
  • Frame reassembly. SSE payloads are JSON-encoded, so each frame is a JSON string holding the event — a double decode, with the closing data: {} frame skipped.

Verification

  • All 13 .jac files pass jac check
  • agent_architecture() returns the real graph (4 stages, 4 capabilities)
  • Full turn streams stage → plan → tool calls → tokens in order with live timing
  • Small talk path confirmed to run 0 capabilities
  • Client bundle builds (HTTP 200); the parsing algorithm was replayed against real captured bytes with adversarial 7-byte chunking — frames reassemble, 383 tokens, emoji intact
  • Docs site gains an Agent section; snippets still byte-exact, still self-contained, highlighter still round-trips

Known limitation

The event bus and run slot are module-level, so one server handles one conversation at a time. That keeps the example readable; a multi-tenant version would key both by session id. Called out in the file header and the PR-facing docs.

A second tab composes all four patterns into one conversational agent, with
real SSE streaming and the agent's own architecture rendered beside the chat.

The agent is a graph, not a tool-loop (agent.jac):

  Perceive -> Plan -> Act -> Synthesize
                       |
                       +-- Brainstorm (Generate)
                       +-- Structure  (Extract)
                       +-- Research   (Invoke)
                       +-- Mentors    (Route + Spawn)

- The graph is the architecture. Topology is data under `root`, and
  `agent_architecture()` reads that same graph back, so the UI diagram is
  derived from the running system and cannot drift from it.
- Behaviour lives on nodes: each capability implements its own
  `can ... with CapabilityExec entry`. Adding a capability adds a node
  rather than a branch in a dispatch chain.
- Typed contracts between stages: Perceive returns `Intent`, Plan returns
  `Plan`, both compiler-enforced instead of free text re-parsed downstream.
- The plan directs the route: small talk runs zero capabilities, a full
  pitch request runs four. Verified both paths.
- Cognition is separate from transport: stages know nothing about HTTP;
  `agent_chat_stream` is a thin SSE adapter.
- The agent narrates itself: stages, capabilities and individual tool calls
  publish to one event bus. Capabilities run on a worker thread while the
  endpoint drains that bus, so a tool call is visible the moment it fires
  rather than after the step completes.

Streaming uses `def:pub ... -> Generator` + `report stream()`, with the final
answer streamed token by token via `by llm(stream=True)`. Conversation history
persists as Turn nodes in the graph.

Per the sem-granularity convention, prompts are split parameter- and
field-wise: every `by llm` function has per-argument `sem`, every `obj` has
per-field `sem`, and each `Capability` enum member documents itself — so the
planner's own prompt stays short and capability meaning lives in one place.

Client (frontend/AgentChat.cl.jac):
- raw fetch + getReader SSE consumption, frame reassembly across arbitrary
  byte boundaries, double JSON decode (payloads are JSON-encoded on the wire)
- the transcript is accumulated in a local and assigned whole; reading `turns`
  back after an await returns the render-time snapshot and would drop the
  user's own turn
- `busy`/`draft` reset in a `finally`, so a stream that dies mid-flight cannot
  leave the composer permanently disabled

Also: jac.toml `[byllm]` unchanged; app.jac imports the streaming endpoints so
they register (a raw-fetch stream has no client stub, and without the
entry-module import the fetch 404s).

Verified: all 13 files pass `jac check`; architecture endpoint returns the real
graph; a full turn streams stage -> plan -> tool calls -> tokens in order with
live timing; small talk runs zero capabilities; the client bundle builds and
the parsing algorithm was replayed against real captured bytes with 7-byte
chunking (frames reassemble, 383 tokens, emoji intact).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant