Repository navigation
Dashboard: fall back to trace data for committed experiments when result.tasks is missing - #66
Open
PredictiveManish wants to merge 3 commits into
Open
PredictiveManish wants to merge 3 commits into
PredictiveManish wants to merge 3 commits into
Conversation
Add validation in _cmd_run_check to ensure that when benchmarks write multiple per-task trace files, the result.json includes a tasks array containing those results. This catches a common bug where benchmarks write task_*.json files but omit the tasks array from result.json, causing empty Tasks panels in the dashboard for committed experiments. Fixes evo-hq#56
Remove the isActive && condition so that dashboard falls back to tracing directory for task data when benchmark.result.tasks is missing, regardless of experiment status. This ensures Tasks panel shows data from trace files even for committed experiments when benchmark failed to aggregate results. This complements the --check validation that prevents such benchmarks from passing validation in the first place. Fixes the UI aspect of evo-hq#56
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Complements #65
& Completely fixes #56
Problem
The Tasks panel currently builds its task map from
benchmark_result.tasks.When that field is missing, the dashboard falls back to per-task trace files only for active experiments:
As a result, committed experiments with valid
task_<id>.jsontraces but no aggregatedresult.tasksdisplay:even though task data is available through the traces endpoint.
Solution
Remove the
isActiverestriction from the trace fallback path.Before:
After:
The dashboard now uses trace data whenever
benchmark_result.tasksis unavailable, regardless of experiment status.Behavior
result.taskspresentresult.tasksmissing + traces availableNotes
This is a defensive UI fallback and complements the
evo run --checkvalidation that prevents new benchmarks from omittingresult.tasks. It does not change precedence: aggregated result data still takes priority when present.