Skip to content

Surface active experiment state during training (#59) - #78

Open
kunalgoesbyken wants to merge 2 commits into
evo-hq:mainfrom
kunalgoesbyken:fix/issue-59-metrics-and-active-attempt-state
Open

kunalgoesbyken wants to merge 2 commits into
evo-hq:mainfrom
kunalgoesbyken:fix/issue-59-metrics-and-active-attempt-state

Conversation

@kunalgoesbyken

@kunalgoesbyken kunalgoesbyken commented Jun 23, 2026 •

Copy link
Copy Markdown

Closes #59.
During long training runs (SFT, RL — often 1-2h) the dashboard and CLI showed nothing useful until the run finalized. The data existed on disk (attempt_state.json, diff.patch, metrics.jsonl) but evo didn't surface it.
Changes

  • cmd_show now includes active_attempt with phase, status, and diff text for active experiments — so evo show returns useful state instead of attempts: []
  • Dashboard _public_node enriches active nodes with active_attempt metadata (phase, status, started_at, updated_at, pid) from attempt_state.json; the drawer renders a summary line and meta row showing current phase
  • New evo metrics <exp_id> [--tail] CLI command that reads metrics.jsonl (checks attempt dir first, falls back to worktree root) with 2-second file-size polling for --tail
  • metrics.jsonl added to dashboard logs tab file listing — shows up alongside benchmark.log with live tail polling (already existed for .log/.out)

- cmd_show now includes active_attempt (phase, status, diff) from
  attempt_state.json and pre-benchmark diff.patch
- Dashboard enriches active nodes with active_attempt metadata and
  renders phase in the drawer
- evo metrics <exp_id> [--tail] cli command for live training metrics
- metrics.jsonl listed in dashboard logs tab with live tail

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

Open in Devin Review

Comment thread plugins/evo/src/evo/cli.py
Comment thread plugins/evo/src/evo/cli.py
alvaroborras added a commit to alvaroborras/evo that referenced this pull request Jun 27, 2026
…#1)

* fix: preserve incomplete trailing lines in evo metrics --tail, handle truncation

* Surface active experiment state during training (evo-hq#59)

- cmd_show now includes active_attempt (phase, status, diff) from
  attempt_state.json and pre-benchmark diff.patch
- Dashboard enriches active nodes with active_attempt metadata and
  renders phase in the drawer
- evo metrics <exp_id> [--tail] cli command for live training metrics
- metrics.jsonl listed in dashboard logs tab with live tail

* feat: live observability dashboard for active experiments (evo-hq#59)

Expand issue evo-hq#59 scope beyond PR evo-hq#78 to include:
- Live diff panel: _DiffRefresher daemon rewrites diff.patch every 5s during benchmark
- Metrics sparklines panel: tail metrics.jsonl with key filtering (--keys loss,lr)
- Samples preview panel: new cmd_samples --tail for last N samples
- Checkpoints panel: new cmd_checkpoints for saved model checkpoints
- HTML export: self-contained report via new cmd_export (--full for untruncated)
- Dashboard frontend: 5-panel Live tab auto-refreshing every 5s for active experiments
- CLI improvements: cmd_metrics --keys filtering, hardened per-attempt dirs with
  EVO_METRICS_PATH/EVO_SAMPLES_PATH env vars
- Tests: 40/40 passing (lifecycle, diff_refresher, export_html, active_attempt)
- Skill docs updated with live dashboard monitoring and env var I/O contract

---------

Co-authored-by: krockxz <71250874+krockxz@users.noreply.github.com>
Co-authored-by: Fernandez, Alvaro <alvaro.fernandez@oliverwyman.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Active experiments are observability-blind: no training logs, metrics, or sample outputs visible until evo run commits

1 participant