Wrap incremental slicing results as ASV samples and restore the existing reducer

[CodyCBakerPhD]

Targets add_time_slicing_benchmark. With the incremental slicing benchmark returning its results the way the other tracking benchmarks do, most of the reduce_results rewrite on that branch is no longer needed.

AI Summary

Why

• track_cumulative_slice_times returned a plain dict. ASV 0.6.1 stores that in the result column with no samples. It also removes empty trailing columns from each row, so the row ends up with 5 entries instead of 12, and the existing reducer skips it. That was the only reason for the rewrite.
• All 31 other track_ benchmarks (network tracking) return dict(samples=..., number=None).
  • With the --record-samples flag that nwb_benchmarks run always passes, ASV then writes the dict to the samples column.
  • The existing reducer copies that value through as is, so it works with the list of cumulative times unchanged.

Changes

• track_incremental_slicing.py: track_cumulative_slice_times returns dict(samples={"cumulative_time_in_seconds": [...]}, number=None).
• setup/_reduce_results.py: back to the version on main, with one fix.
  • Previously, if one parameter set failed (its samples are null), the other parameter sets of that benchmark were dropped too, with a length-mismatch warning.
  • Now the parameter sets that succeeded are kept. That matters here: an ophys run hitting the 12-hour timeout would otherwise also discard the ecephys and icephys results.
  • The warning now fires only when the parameter, result and samples lists have different lengths.
• database/_models.py: parse_parameter_case no longer special-cases (). Only benchmarks with no parameters produce that, and the suite has none.
  • The guard in normalize_time_and_network_results stays. Without it, the database reader treats every dict result as network statistics and raises a KeyError on these results.
• Tests:
  • The reducer tests now run reduce_results itself on a raw results file regenerated with ASV 0.6.1. The file holds the wrapped incremental result (one parameter set fails), a network benchmark, a time_ benchmark with one parameter set not selected, and an unwrapped track_ benchmark.
  • A new test checks that track_cumulative_slice_times returns the wrapped form.

Net change against the base branch: +173 / −203. _reduce_results.py alone goes from +89/−34 against main to +17/−20.

How it was checked

• Unit tests: 40 pass locally; black and isort pass.
  • With main's reducer before the fix, 4 reducer tests fail.
  • With the base branch's unwrapped return, the new wrapping test fails.
• End to end:
  • HDF5PyNWBLocalIncrementalSliceBenchmark ran through asv run --python=same --record-samples on a synthetic local icephys file, with RUN_INCREMENTAL_SLICING_BENCHMARKS=icephys.
  • The DANDI API is blocked in the sandbox, so its two lookups were stubbed in a throwaway suite.
  • ASV wrote a 12-entry row. reduce_results kept the 7 cumulative times. nwb_benchmarks.database.Results loaded them as 7 increasing cumulative_time_in_seconds rows.

🤖 Generated with Claude Code

https://claude.ai/code/session_015K2S7gEkSTfhpnfbXVD5YG

---

Generated by Claude Code

[CodyCBakerPhD]

@oruebel So basically, if we make the output of the new proposed test match the style of the others, then we don't need to adjust the results parser at all

[oruebel]

So basically, if we make the output of the new proposed test match the style of the others, then we don't need to adjust the results parser at all

Thanks @CodyCBakerPhD . That makes sense. Thanks for fixing that.