Skip to content

Automated performance regression detection in ci pipeline#119

Open
clintjeff2 wants to merge 1 commit into
Utility-Protocol:mainfrom
clintjeff2:Automated-Performance-Regression-Detection-in-CI-Pipeline-#65
Open

Automated performance regression detection in ci pipeline#119
clintjeff2 wants to merge 1 commit into
Utility-Protocol:mainfrom
clintjeff2:Automated-Performance-Regression-Detection-in-CI-Pipeline-#65

Conversation

@clintjeff2

Copy link
Copy Markdown
Contributor

Motivation

  • Prevent unintended latency regressions on critical paths by enforcing a committed P99 baseline and an automated CI gate.
  • Provide a reproducible, small JSON snapshot format so benchmarks, observability exports, and load tests can feed the same gate.
  • Ensure the system meets the project's performance SLOs (hard SLO: 100ms P99, regression budget: 10%) before merges.

Description

  • Add a committed baseline of critical-path P99s in .github/performance/baseline.json describing meter_heartbeat_ingest, usage_settlement, oracle_rate_lookup, and dashboard_usage_summary.
  • Implement scripts/performance_regression_gate.py, a standalone Python CI gate that validates snapshot JSON, compares baseline vs current, applies a hard --max-p99-ms limit and a --regression-percent budget, prints pass/fail markers, and can emit a Markdown report.
  • Wire the gate into the test-and-lint job in .github/workflows/ci.yml to run after cargo test and upload a performance-regression-report artifact.
  • Add documentation docs/PERFORMANCE_REGRESSION_CI.md, update README.md to include the gate and local invocation, and add unit/CLI tests at tests/performance/test_performance_regression_gate.py.

Testing

  • Ran the Python unit tests with python3 -m unittest tests.performance.test_performance_regression_gate, which passed.
  • Executed the gate end-to-end with the committed baseline using python3 scripts/performance_regression_gate.py --baseline .github/performance/baseline.json --current .github/performance/baseline.json --max-p99-ms 100 --regression-percent 10 --report /tmp/perf.md, which succeeded and produced a Markdown report.
  • Verified repository checks: git diff --check passed, cargo fmt --all -- --check passed, cargo test passed, and cargo clippy --all-targets --all-features -- -D warnings passed.

Closes #65

@clintjeff2
clintjeff2 force-pushed the Automated-Performance-Regression-Detection-in-CI-Pipeline-#65 branch from d973aaa to dd6fe48 Compare July 19, 2026 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automated Performance Regression Detection in CI Pipeline

1 participant