Skip to content

perf(persistence): delete replaced coverage and dead-code rows in one statement - #4359

Merged
swati510 merged 1 commit into
mainfrom
perf/bulk-delete-coverage-dead-code
Oct 22, 2026
Merged

swati510 merged 1 commit into
mainfrom
perf/bulk-delete-coverage-dead-code

Conversation

@RaghavChamadiya

Copy link
Copy Markdown
Member

What

The coverage and dead-code writers loaded every row they were about to replace and deleted it one ORM object at a time. They issue a single DELETE instead, the same shape the health findings and metrics writers use.

How

  • save_coverage_files: one DELETE on the repo's coverage_files rows, still before the inserts, so uq_coverage_files sees the same order.
  • save_dead_code_findings: one DELETE on the repo's open rows.
  • replace_dead_code_findings: reads only the (file, kind, symbol) keys of acted-on rows, then deletes the open rows. With a scope, the delete goes through the existing chunked _batch_delete_in, so a scope wider than SQLite's variable limit still works. Rows outside the scope and dismissed rows are left exactly as before.
  • coverage_map.save_test_coverage already used a single DELETE; nothing to change there.

Neither table has a foreign key, relationship cascade or ORM hook that the per-row delete was triggering, and no caller reads the loaded rows. The read-snapshot staleness hook fires for the bulk statement as it did for the flushed deletes.

Verification

Each copy re-saved its own stored rows with the old code and with this branch: coverage save, dead-code save, repo-wide replace, then a replace scoped to every other file with 10% of findings dismissed first. Row counts and a SHA-256 over every column except the id and the save-time stamp match between old and new on all three. Medians of 3 alternating runs:

Index Rows Step Before After
This repo 2,223 coverage files save_coverage_files 399 ms 275 ms
Superset 2,486 dead-code findings save_dead_code_findings 496 ms 276 ms
Superset scoped replace_dead_code_findings 159 ms 100 ms
MLflow 2,148 dead-code findings save_dead_code_findings 503 ms 208 ms
MLflow scoped replace_dead_code_findings 166 ms 90 ms

Repo-wide replace_dead_code_findings is within noise (Superset 326 vs 339 ms, MLflow 353 vs 283 ms); its time goes to the inserts.

New test test_scoped_replace_past_one_delete_chunk_keeps_exactly_the_right_rows locks the rows left after a scoped replace that spans more than one delete chunk. It passes on main and on this branch. Coverage, dead-code and persist-order suites: 128 passed.

… statement

save_coverage_files and save_dead_code_findings loaded every row they replaced and deleted it one object at a time. Both issue one DELETE now. replace_dead_code_findings reads only the acted-on rows' keys and deletes the open rows in scope through the shared chunked delete, so dismissals and rows outside the scope are untouched as before.
@swati510
swati510 merged commit 2c3b6e0 into main Oct 22, 2026
10 checks passed
@swati510
swati510 deleted the perf/bulk-delete-coverage-dead-code branch October 22, 2026 06:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants