[Bench] Detach op benchmarks from tests package - #1805
Merged
Ibuki-wind merged 3 commits intoJul 31, 2026
Merged
Conversation
Contributor
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Ibuki-wind
previously approved these changes
Jul 31, 2026
lcy-seso
force-pushed
the
refactor/bench-detach-tests-1803
branch
3 times, most recently
from
July 31, 2026 06:19
cb34e6e to
800915e
Compare
ManifestBenchmark annotated its workload as ShapeDtypeWorkload, but it never reads shape or dtype: BenchmarkBase only stores the workload, and roofline numbers come from op.eval_roofline(). Nothing in the tree reads `.shape` off a workload. The annotation still shaped the code. Workload classes under workloads/ define no `shape`, and the benchmark stage may not add one there, so a benchmark needing a manifest workload had to subclass it locally just to bolt on a `shape` property -- one that does not even satisfy the protocol, since it declares a mutable attribute and a read-only property is not assignable to it. Widen the annotation to what the class actually needs, drop the five dead `shape` properties in the GEMM and BMM benchmarks, and profile BmmWorkload directly now that its subclass carries nothing. Co-Authored-By: Ibuki 🍃 — a wind born from GPTs <Ibuki-wind@users.noreply.github.com>
lcy-seso
force-pushed
the
refactor/bench-detach-tests-1803
branch
from
July 31, 2026 09:05
800915e to
1ac6a3e
Compare
Ibuki-wind
approved these changes
Jul 31, 2026
yyttt6
pushed a commit
to yyttt6/TileOPs
that referenced
this pull request
Sep 4, 2026
## Summary Fixes tile-ai#1803. - Remove remaining production benchmark imports from `tests.*` in GEMM, BMM, and Mamba2 E2E benchmarks. - Move shared Mamba2 PyTorch reference code to `tileops.testing.mamba2_reference` for test/benchmark reuse without importing `tests.*`. - Add an AST boundary test preventing production benchmarks from importing the `tests` package. ## Validation - `rg` confirms no `tests.*` imports under `benchmarks/ops` or `benchmarks/kernels`. - `ruff check` on changed files. - `pytest -q benchmarks/tests/test_benchmark_base.py benchmarks/tests/test_benchmark_import_boundaries.py`. - `compileall` and `git diff --check`. Closes tile-ai#1803.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1803.
tests.*in GEMM, BMM, and Mamba2 E2E benchmarks.tileops.testing.mamba2_referencefor test/benchmark reuse without importingtests.*.testspackage.Validation
rgconfirms notests.*imports underbenchmarks/opsorbenchmarks/kernels.ruff checkon changed files.pytest -q benchmarks/tests/test_benchmark_base.py benchmarks/tests/test_benchmark_import_boundaries.py.compileallandgit diff --check.Closes #1803.