Skip to content

Add the Consort live dashboard (apps/dashboard) + fix the DriftBanner false-alarm - #190

Merged
kevin-hartman merged 3 commits into
databricks-solutions:mainfrom
cathysnell:feat/add-dashboard
Sep 1, 2026
Merged

Add the Consort live dashboard (apps/dashboard) + fix the DriftBanner false-alarm#190
kevin-hartman merged 3 commits into
databricks-solutions:mainfrom
cathysnell:feat/add-dashboard

Conversation

@cathysnell

Copy link
Copy Markdown
Contributor

What this does

Two commits:

  1. feat(dashboard): vendor the Consort live dashboard as apps/dashboard. The Next.js live/replay observability board that watches a Consort run (or replays a recorded corpus), now living alongside the framework it observes. It complements — does not replace — the VS Code Consort extension: the extension is the in-editor viewer; this is the standalone web board.

    • Self-contained: its own package.json / lockfile / build / test; not wired into the root tsup build (consort isn't an npm-workspaces monorepo). The only root change is excluding apps/dashboard from the root tsconfig include (apps/**/*.ts) so npm run typecheck / CI doesn't pull the app's Next.js/React sources into the substrate program. Root vitest (tests/** only) and the explicit tsup entry list are unaffected. Verified: the root program resolves apps/mcp-server + dev-playground but zero apps/dashboard files.
  2. fix(dashboard): DriftBanner attributes to corpus pairing, not run health. On a live run the DriftBanner sometimes rendered a critical-red banner next to the orchestrator, so viewers read it as "the run is failing". It's only an observability signal — the dashboard can't cleanly pair the run to a recorded corpus — and says nothing about the run/build/deploy/telemetry. Per the report:

    • Distinguish benign vs genuine. New driftSeverity(): warning = a role the corpus never recorded (the RECORD_DIR likely points at a different run — worth flagging); info = a kit-version mismatch or a plain log-ahead tail (the normal live edge — quiet). The healthy/no-companion case still shows no banner.
    • Downgrade + reword. No more --status-critical: warning renders amber (role="alert"), info renders as a quiet muted note (role="note"), both attributing explicitly to pairing.

Verification

  • apps/dashboard: npx tsc --noEmit clean, 401 tests pass (npx vitest run).
  • Reviewed with Isaac Review (/review); its one finding against this change (a live-edge false-positive where severity was derived from the stricter report.healthy) is fixed in commit 2 with a regression test.

Known follow-ups (pre-existing in the dashboard, not introduced here — will be separate PRs)

  • lib/reducer.ts: a divergent FeatureSwitcher pin at the live edge shows 0 stories for the pinned past feature (no pinnedDivergent stories-derivation path).
  • app/AgentBubble.tsx: the "working {elapsed}" readout uses Date.now(), so on replay/scrub it shows an absurd elapsed value + a green "live" dot (the bubble never receives atLive).
  • app/usePolledState.ts: a non-2xx /api/state (500) is treated as a successful poll — no backoff, and "connected" is reported despite every response being an error.

cathysnell and others added 3 commits August 31, 2026 10:54
Adds the Next.js live/replay observability dashboard under apps/dashboard/ so
it lives alongside the framework it observes. It complements (does not replace)
the VS Code Consort extension: the extension is the in-editor viewer, this is
the standalone web board for a live run or a recorded replay corpus.

Self-contained: it keeps its own package.json / lockfile / build / test and is
NOT wired into the root tsup build (consort is not an npm-workspaces monorepo).
The only root-level change is excluding apps/dashboard from the root tsconfig
"include" (apps/**/*.ts) so `npm run typecheck` / CI does not pull the app's
Next.js/React sources into the substrate program. Root vitest (tests/** only)
and the explicit tsup entry list are already unaffected.

Co-authored-by: Isaac <no-reply@databricks.com>
On a live run the DriftBanner sometimes rendered a critical-red banner next to
the orchestrator, so viewers read it as "the orchestrator/run is failing". It is
only an observability signal: the dashboard cannot cleanly PAIR the run to a
recorded corpus. It says nothing about the run, the build, the deploy, or the
telemetry ingest.

Two changes, per the report:

1. Distinguish benign caveats from genuine drift. New driftSeverity() in
   correlate.ts classifies an unhealthy report:
     - warning: a role the corpus never recorded => the RECORD_DIR likely points
       at a DIFFERENT run. The one case worth flagging prominently.
     - info: a kit-version mismatch (expected-with-caveat) or a plain log-ahead
       tail (the normal live edge). A quiet note.
   The "no companion / healthy" case already produced no banner and is unchanged.
   Threaded through the correlation summary (new `severity` field on the type,
   set by both the live and replay sources).

2. Downgrade severity + reword. The banner no longer uses --status-critical.
   A warning renders amber (role="alert"); an info renders as a quiet muted note
   (role="note"). Both attribute explicitly to pairing and add "Turn drill-downs
   may be approximate; the run itself is unaffected."

Tests: render.test.tsx DriftBanner cases rewritten for info vs warning; driftSeverity
covered in correlate.test.ts. 400 pass, tsc clean.

Co-authored-by: Isaac <no-reply@databricks.com>
…istry

The vendored package-lock.json resolved 144 packages from the internal
npm-proxy.cloud.databricks.com, which fails the lockfile-public-registry guard and
would hang `npm install` off the Databricks network. Rewrite the resolved host to
registry.npmjs.org (integrity hashes unchanged). Root suite green (4373).
@kevin-hartman

Copy link
Copy Markdown
Collaborator

Reviewed end-to-end and pushed one fix to this branch before merge.

Fix (b41d8c74): the vendored apps/dashboard/package-lock.json had 144 resolved URLs pointing at the internal npm-proxy.cloud.databricks.com. That failed the repo's lockfile-public-registry guard and would hang npm install for anyone off the Databricks network. Rewrote the host to registry.npmjs.org — pure host swap, integrity hashes unchanged, still valid JSON.

Verification: full root suite green (4373 passed, 0 failed); the guard now passes.

Review notes: integration is minimal and clean (one tsconfig.json exclude line; root tsc --noEmit clean, zero apps/dashboard files pulled into the substrate program). The DriftBanner severity fix is correct — driftSeverity() returns ok when healthy, warning only on a role the corpus never recorded, info for a kit-version mismatch / live-edge tail — with regression tests. No secrets committed.

The three known dashboard follow-ups (reducer pinnedDivergent, AgentBubble Date.now() on replay, usePolledState treating 500 as success) are deferred to separate PRs as noted. Merging.

@kevin-hartman
kevin-hartman merged commit bfcbb22 into databricks-solutions:main Sep 1, 2026
@cathysnell
cathysnell deleted the feat/add-dashboard branch September 2, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants