Skip to content

ui.md

Latest commit

 

History

History
187 lines (130 loc) · 22.4 KB

ui.md

File metadata and controls

187 lines (130 loc) · 22.4 KB
Raw

Codexa UI

codexa ui boots a Streamlit dashboard. This doc walks the layout top-down: the search shell, the saved-chat sidebar, the search result rendering (retrieved passages + image augment), the faith / pin / exclude / follow-up surfaces, then the sidebar panel selector (Semantic Structure / Index / Chatbot Ops / Diagnostics).

← README · Decision records: ADR 0012 (Streamlit execution model), ADR 0013 (session persistence), ADR 0003 (the turn flow the panels render)

Table of contents

  1. Layout
  2. Saved chats / multi-turn sessions
  3. Retrieved passages — Image rendering
  4. Faith / grounding surfaces
  5. Search diagnostics / compare / insights
  6. Pin / exclude / follow-up suggestions
  7. Semantic Structure tab
  8. Index tab
  9. Chatbot Ops tab
  10. Diagnostics tab
  11. Internationalization

Layout

The dashboard has two panes:

  • Sidebar (left) — the language picker, any active warnings, saved chat threads grouped by date bucket with a free-text filter on top, and then a single-selection panel switcher. Four core panels appear in this order: 🧬 Semantic Structure (entity counts + co-occurrences for the documents your most-recent search returned), 📂 Index (run/force-reindex controls plus the folded-in Consistency report: manifest vs. data dirs), 🩺 Diagnostics (host + GPU + cache + Wikipedia tier reachability + OCR engine status), 🤖 Chatbot Ops (declared objective + counters + pilot batches — see Chatbot Ops tab). Only the selected panel executes on a rerun, so hidden diagnostics do not slow search interactions. Labels translate via the active locale (pt_BR renders them as 🧬 Estrutura Semântica → 📂 Index → 🩺 Diagnóstico → 🤖 Chatbot Ops; es / fr follow their catalogs).
  • Main (center) — the search input, the rendered RAG answer (when an LLM is configured), and the retrieved passages.

Saved chats / multi-turn sessions

Every search runs inside a session. The first question creates a fresh Session (id, name, created_at, updated_at, turns array); follow-ups append (user, assistant) turn pairs. Sessions live in {store.persist_dir}/sessions.{json,db} so they survive restarts.

In generation mode, a turn owns one immutable snapshot across worker retrieval and RAG. After retrieval and before row publication or answer generation, it parks a separate owner for render and persistence. Generation reads use pinned cfg; session/WAL paths use the original unresolved cfg, so operator data never moves inside a generation directory. The persisted assistant extra carries only the detached generation id and root-binding digest used for prior-pin snapshot matching, not a live ownership token or display text. Its path-free-but-not-secret persistence and export contract is documented in Privacy and Retention. Provider text remains buffered behind the grounded-emit completion gate rather than being painted token by token.

Two interchangeable backends (ADR 0013 records why): SessionStore (JSON, codexa.ui.sessions_json) and SessionStoreSqlite (codexa.ui.sessions_sqlite). The JSON backend rewrites the whole file on save (atomic via utils.io.atomic_write_text) and uses POSIX fcntl when available; on hosts without fcntl, even .json session paths route to a sibling .db automatically so cross-process saves stay serialized. The SQLite backend uses per-row UPSERTs with a process-wide schema-ready memo (_SCHEMA_READY LRU(1024) capped + AppContext-reset wired, CACHE-12).

Schema is versioned + flexible from day one: file envelope carries version, each row carries its own version, and every level has an extra: {} catch-all for forward-compat. The loader (codexa.ui.sessions_migrate) runs a migrator chain on read — currently v1 → v2 (single-shot rows → turns array). Future migrators append a single tuple to _MIGRATORS with no read-path edits.

Bounded prompt history: the dispatcher injects the last cfg.search.history_turns turns (default 6 = 3 user + 3 assistant) into the prompt's Recent conversation history: block. Older turns stay on disk regardless — only the prompt context is bounded so long threads don't blow the context window.

HistoryCompressor (codexa.search.history_compress, DDD-1 lift from the UI layer) summarises dropped prefix turns via the LLM when cfg.search.history_compress is "auto" / "always". The recap lands as a synthetic [Recap] turn at the head of the prompt-history block (PROMPT-5 fences the transcript + threads the locale directive). Saved-session reload paints the recap with a distinct 📋 avatar + [Recap] chip so the compression boundary stays visible (COH-12).

Operators can scribble an ephemeral per-turn note on the active assistant turn (BOT-22, Phase 94 COH-9): a text area under the answer captures into a per-turn session_state slot (_codexa_operator_note_draft_t{ti}) that survives turn-switching mid-thread. The persist path drains the singleton + every per-turn slot in one pass.

The sidebar groups sessions by date bucket (Today / Yesterday / Last 7 days / Older), supports free-text filtering (matches name + query content), and uses two-step delete confirms behind a ⋮ overflow popover so a misclick can't wipe history.

Retrieved passages

Each passage in the answer pane lives in a collapsible st.expander header — 📄 source.pdf · d=0.1234. Default expanded on the most recent assistant turn; older turns stay collapsed so long threads don't dominate the view. The header carries the source path + the cosine distance the retriever computed (lower = closer match).

Below each text block the panel renders POS-tagged chip buttons for the entities in the passage (extracted via NLTK + the existing metadata.entities toolkit). Clicking a chip stuffs the term into the search box and reruns immediately — turns each passage into a launchpad for follow-up exploration without typing. NLTK failures (resources missing on a fresh checkout) silently no-op so the rest of the passage still paints.

A render-time decoder fixes already-indexed text that came back as i�v�s�t�a�n�e — the classic UTF-16-LE bytes decoded as UTF-8 with errors='replace' pattern, where every other byte (the NUL half of each codeunit) became �. The recovery strips the replacement chars when the result is mostly printable and >15% of the chunk was replacement chars; otherwise the original (garbled) bytes stay visible so binary noise can't be silently mistaken for a "fixed" reading. Long-term the fix belongs upstream in the extractor (detect UTF-16 BOM / null-pair pattern at file open time) — this helper is the safety net for already-indexed corpora that pre-date that change.

Image rendering

Source-type-aware visual augment under each retrieved passage:

  • Standalone image (.png / .jpg / .jpeg / .tiff / .tif / .bmp / .gif / .webp) → inline thumbnail rendered eagerly under the OCR'd text (max 480 px wide).
  • Paged document (.pdf / .epub / .cbz / .fb2 / .xps / .oxps) → 📄 View page from … expander with a number-input page picker, opened at the page the matched chunk was located on rather than page 1. Pure on-demand: no rasterisation runs until the operator opens the expander. The page is rendered via pymupdf at cfg.search.passage_thumbnail_dpi (default 100). Bytes cache per (source, page, dpi) in a process-wide LRU (cap 64 entries) so re-clicks reuse the bitmap. Cache stats + reset surfaced via page_thumb_cache_stats() / clear_page_thumb_cache() (CACHE-1).
  • Archive with embedded media (.docx / .docm / .pptx / .xlsx / .odt / .odp / .odg / .ods) → 🖼 First embedded image expander pulling the first media entry straight from the container zip (word/media/, ppt/media/, xl/media/, or Pictures/ for the ODF family). Same LRU cache.
  • Anything else (plain text, code, …) stays text-only. EPUB is not in this group — pymupdf renders it natively, so it goes through the page picker above.

A path that moved between indexing and render falls back to a one-line caption ((image moved or removed: …)) so the rest of the passages list still paints. Same for unreadable image streams ((image render failed: …)), corrupt PDFs ((PDF metadata unreadable)), and DOCX files with no word/media/ content ((no embedded images)).

Faith / grounding surfaces

These are the UI face of the answer-path post-passes decided in ADR 0003 (decision 5: faithfulness is post-processing, always on). When cfg.llm.provider is configured, the answer pane runs the FAITH-* checks before painting:

  • Orphan-citation strip — [N] markers whose index doesn't resolve to a surviving passage get stripped before the answer reaches st.write AND before the answer cache stores it (FAITH-5). The citation_orphaned log event carries the dropped indices for audit. Saved-session reload re-runs the strip against the persisted t.results so legacy answers don't show stale chips (FAITH-7).
  • Grounded emit — provider and FLARE output stays buffered until orphan stripping and one full-answer grounding assessment finish. Mixed answers pass; when every scored sentence is unsupported, the pane receives the locale-matched no-answer phrase instead. A provider/FLARE exception discards its buffered prefix and shows only the failure banner. This completion gate trades progressive token paint and mid-generation Stop for visible/saved/cache parity.
  • Hallucination flags — the grounded-emit assessment records each sentence's score without appending display-only text to the answer. The separate citation panel surfaces ⚠️ captions, and persisted extra["hallucination_flags"] recreates that audit trail on reload (COH-8).
  • Internal vs external grounding split — when any result row carries is_external=True (Wikipedia synthetic rows) the panel auto-paints the internal-vs-external grounding caption (FAITH-8). Explicit cfg.search.faith.show_internal_external_split=True / False keeps the legacy force-on / opt-out override.
  • Regenerate-when-ungrounded — when cfg.search.regenerate_when_ungrounded.enabled is on, the dispatcher reruns generation once with a revision instruction for the flagged sentences (RAG-7). The post-regen score is re-evaluated against the passage pool and ungrounded_regenerate_outcome lands with (before_unsupported_n, after_unsupported_n, score_delta_mean) so operators can audit the lift (FAITH-9).
  • No-answer detection — programmatic is_no_answer check fires the BOT-25 escalation packet onto turn.extra["escalation"]; the chatbot ops dashboard counts escalation rate.

Search diagnostics / compare / insights

Three optional surfaces hang off the search render when an operator needs to understand why a result or answer behaved the way it did:

  • Retrieval debug — each retrieved passage has a collapsed 🔍 Retrieval debug expander (codexa.ui.panels.retrieval_debug) with source, dense / sparse / rerank scores, fusion rank, citation id, and external-source flags when those fields exist on the row. It is meant for per-passage triage, not for end-user presentation.
  • A/B answer compare — the 🔬 Compare A/B toggle runs a second answer path via search_ab_compare.py, paints the two halves side by side, and applies the same faithfulness checks to the comparison half. Escalations from the comparison side are tagged as compare-side diagnostics and do not overwrite the primary answer slot.
  • Passage insights — when cfg.search.insights.enabled: true, the panel renders 🧠 Passage insights (offline) below the answer using the retrieved passages only: LexRank-style extractive summary plus top keyphrases. It is LLM-free by default, cache-backed, and documented in more depth in docs/semantics.md.

Pin / exclude / follow-up suggestions

Per-passage operator controls under each retrieved row (Phase 86):

  • 📌 Pin — pins the source for the next turn's candidate pool (BOT-3). Pinned sources are prepended to fresh_results before the relevance horizon filter. Drops emit pin_dropped, paint a per-turn caption, persist on the assistant turn, and aggregate across the active chat thread (COH-4).
  • 🚫 Exclude — blacklists the source for the rest of the thread (BOT-23). filter_excluded_sources runs inside _merge_pins_from_prior_turn BEFORE the pin merge so an excluded source can't sneak back through pinning. Pin / exclude state persists on turn.extra["pinned_results"] / ["excluded_sources"] (WIRE-19 / WIRE-20).
  • 👍 / 👎 — per-passage thumbs feedback (BOT-21). Historical controls update the exact saved assistant turn and result; failed writes retry through the session WAL. State persists on turn.extra["passage_feedback"] for future analytics joins.

Follow-up suggestions (BOT-4) render under the answer when cfg.search.follow_ups.enabled is on. Two backends:

  • extractive — pull top RAKE keyphrases from the passages and template a per-locale stub (E sobre X? / ¿Y qué hay de X? / What about X?, COH-2).
  • llm — one cheap LLM call asking for k follow-up questions in the answer locale, with the same SEC-1 fence + locale directive contract as the other prompt builders (COH-6 + Phase 87 PROMPT-6).

Both backends pass the pool through _filter_followup_pool first: drops excluded sources, moves pinned sources to the head so the BOT-3 topical anchor survives the snippet budget (BOT-30).

Semantic Structure tab

Entity counts and co-occurrence pairs only render when a search is active. The panel reads a versioned search-scope record containing a detached source tuple plus generation id and root-binding digest. An absent search is distinct from an empty search: absent shows the "Run a search" prompt, while an empty source tuple is a valid completed search with no matching entity rows.

For a generational record, the panel reopens the exact selected generation under a shared read lease, validates the root-binding digest, and reads that generation's manifest and index info. It never falls back to CURRENT, so a later publish cannot mix the latest index with older search rows. A missing generation, identity mismatch, or expired or malformed record produces the actionable Search snapshot expired state: Run the search again. The generation-aware cache key contains manifest path/mtime, scope signature, generation id, and root digest. Flat-mode legacy source lists retain their in-place behavior.

The async worker publishes both non-empty and empty records as soon as retrieval finishes. The fragment performs that early publication inside the bound handle snapshot, before the answer completes, so the detached identity always matches the rows.

Index tab

The Index tab carries three operator-actionable controls:

  • Run Indexer Now button — spawns codexa-index in a detached subprocess (the process topology of ADR 0006). Auto-disables when another indexer holds the IndexerLock, and again when the corpus is already 100% indexed (no incremental work to do).
  • Force full reindex checkbox — sits next to the run button. When ticked, the spawned indexer gets --force-reindex so it rebuilds every chunk + embedding from scratch instead of using the manifest's incremental hashes.
  • Stale chroma lock cleanup — when the helper detects orphan chroma.sqlite3-shm / -wal files or named sharded sidecars such as pdf_a_0000000000.sqlite3-wal (typical after a kill -9 / power loss on a previous indexer run) AND no live indexer is holding the lock, the panel surfaces an expander with a Clear stale chroma lock button. The cleanup re-checks the IndexerLock at click time so a fresh indexer started between paint and click can't have its sidecars wiped out from under it.

After a UI launch, the run button stays disabled while the detached child starts. Success appears after the live child’s exact PID owns the IndexerLock, or after a fast no-op run exits cleanly before the next poll. An early failure keeps a bounded stderr tail on the panel so config/import/lock failures remain actionable across Streamlit reruns and desktop restarts.

The same cleanup is exposed on the CLI as codexa clear-stale-lock (--dry-run for a preview) — useful for headless deployments where the dashboard isn't running.

The Index tab also folds in the Consistency subsection (manifest vs. data dirs) — flags files present in the manifest but missing on disk, files on disk not yet indexed, and stat-mismatch rows that the next incremental pass will pick up.

Chatbot Ops tab

Operator-facing aggregation surface (codexa.ui.panels.chatbot_ops, consumes ChatbotOpsDashboardData.collect(cfg)):

  • Declared objective header — paints the operator's cfg.chatbot.objective block (statement + owner + review cadence + success metrics). When cfg.chatbot.objective.set_at (ISO timestamp) drifts past review_interval_days the header surfaces an ⏳ "review overdue" warning (BOT-29, Phase 94).
  • Counters block — sessions / turns / assistant-turns / grounded-answer rate / no-answer rate / hallucination rate / escalation rate / pin-drop counter / passage-feedback split. Each counter walks every saved session in one pass; the chatbot_ops_aggregated event covers triage.
  • Retrieval quality baseline block — reads the latest saved eval summary (metadata.eval_summary_file, then {store.persist_dir}/eval_baseline_summary.json, then the checkout's .codexa-quality/eval_baseline_summary.json CI artifact) and renders context_recall, ndcg_at_5, mrr_at_5, result_diversity, uncited_ungrounded_rate, eval no-answer rate, and context_precision only when at least one eval case declares an exhaustive closed_gold: true relevance set. Set indexer.post_index_eval.enabled: true and provide cases_path to refresh that summary after indexing; Codexa skips publication unless every expected source resolves inside data_dirs, so bundled repo-doc qrels cannot become KPIs for an unrelated corpus. Evaluation failure is logged but does not invalidate the published index. When a tuned overlay exists, the same hook persists its multi-metric baseline delta plus a keep / review recommendation and emits the corresponding operator action. Saved summaries from codexa eval baseline --save <path> / codexa eval docs-baseline --save <path> share this schema and include config + manifest fingerprints, so the panel labels the run fresh/stale/unknown and lists saved gate targets vs actuals. The automatic hook records targets too when indexer.post_index_eval.min_recall / min_precision / min_ndcg_at_5 are set — unlike the CLI gates these only describe the run and never drive an exit code. Live-index summaries also record the pinned published generation.
  • Pilot batches — PilotRunStore lists batches operators marked for offline eval. Status flips (pending → pass / fail) route through update(cfg=cfg) which BOT-28-rebases the persisted objective snapshot to the current cfg + emits pilot_objective_rebased when the snapshot diverged. The store uses tombstone overlay + lazy compaction (PERF-32, Phase 96) so UI status toggles are O(1) instead of O(n).
  • Success-metric summary — pairs each declared cfg.chatbot.objective.success_metrics line against the live counters via simple ≥ / ≤ parse so the dashboard can render retrieval@5 ≥ 0.7 ✅ 0.82 style status chips.

ChatbotObjective.from_cfg is memoised on cfg_fingerprint(cfg.chatbot) (PERF-33, Phase 88) so per-render call sites don't re-walk the nested dict.

Diagnostics tab

Host + GPU + cache + Wikipedia tier reachability + OCR engine status — the operator's first stop when something's wrong:

  • GPU / host identity — active embedder backend + device count, hostname / kernel / CPU cores / Python version.
  • RAM + disk — psutil snapshot + free space at store.persist_dir.
  • Embedding cache — entries / disk usage / hit rate + per-shard bar chart.
  • Per-process caches — cache stats from ingestion, Wikipedia, retrieval, metadata, and UI/session helpers render as grouped field/value tables via _gather_other_caches.
  • OCR engine — configured engine + PaddleOCR / Tesseract availability glyphs.
  • Wikipedia tiers — seed: ✔ ZIM: ✔ online: ✘ per docs/wikipedia.md.
  • Manifest fingerprint — IDX-9 surfaces the manifest's stamped model/dim so operators can spot a mismatch without grepping.
  • LLM provider probe — async warmup + reachability via app_probe (Phase 84 SCAL-13 carve from app.py). Probe cache is 30 s TTL keyed on (primary, host); the chip flips from Probing… → Ready / Unreachable without blocking the search panel.

Internationalization

Streamlit UI strings and the config-validator CLI banners flow through codexa.i18n._(). Other maintenance CLIs stay English by design so scripted output, package names, paths, and troubleshooting snippets remain stable. Catalogs live in codexa/locale/<code>/LC_MESSAGES/codexa.po (a tiny runtime PO loader — no babel/msgfmt step). The default ships en, es, fr, and pt_BR; add a language by dropping a new catalog and setting locale in config or the CODEXA_LOCALE envvar. Locale resolution is cfg.locale → CODEXA_LOCALE → LANG → en; a sub-tag like pt_BR falls back to its base (pt) when a sub-tag catalog is missing. Internal log event keys stay English regardless of locale.

Locale also flows into LLM prompt builders: _language_directive(current_locale()) + _language_header(...) prepend the prompt in HyDE, query_decomp, query_rewrite (PROMPT-10), history-summary (PROMPT-5), follow-ups (COH-6), and the answer template so a pt_BR session never gets an English sub-query / rewrite / recap / answer.

Privacy / Retention

Saved sessions survive restarts by default, and JSON exports include the full passage text needed to reproduce an answer. Operator retention, purge commands, and path-redaction limits are documented in docs/privacy.md.