Fix xdist database-is-locked flake in the mypy regression tests

[w-martin]

Fixes the intermittent sqlite3.OperationalError: database is locked in tests/integration/test_regression.py that has shown up on at least two recent PRs.

Root cause (confirmed, not assumed)

Reproduced locally and captured the trace. Every failure was identical:

mypy/build.py:1908: in create_metastore
mypy/metastore.py:200: in __init__          # SqliteMetadataStore shard connect loop
E   sqlite3.OperationalError: database is locked
mypy/metastore.py:174: OperationalError     # db.execute("PRAGMA journal_mode=WAL")

The chain:

1. mypy 2.x uses a sqlite cache by default. mypy/options.py:305 sets self.sqlite_cache = True, and defaults.SQLITE_NUM_SHARDS = 16, so a cache dir holds cache.0.db … cache.15.db rather than the old JSON metastore.
2. The main mypy process switches each shard to WAL on open. build.py:create_metastore passes set_journal_mode=not parallel_worker, and connect_db then runs PRAGMA journal_mode=WAL. Changing journal mode needs an exclusive lock and fails immediately instead of waiting on the sqlite busy timeout.
3. Two tests shared the default cache dir. test_should_not_catch_errors_without_plugin and test_should_accept_polarsframe_with_type_argument passed no --cache-dir, so both landed on .mypy_cache/. test_should_catch_errors_with_plugin already had its own dir from Fix slow mypy plugin regression test #65.
4. -n auto is in addopts, so those two tests can run in different xdist worker processes at the same time. mypy also spawns its own worker processes per run, which hold locks on those same shard files. Whichever test's main process reaches the WAL pragma second raises.

Two corrections to the original hypothesis, both verified:

• It is not contention against .mypy_cache/plugin_regression_test. That dir is a distinct sqlite file set and never appeared in a failure; the third test never failed once across the whole baseline.
• It is not contention with uv run inv lint. inv lint runs ruff/ty/bandit/complexipy/cargo — no mypy at all, and the pre-commit config has no mypy hook either. grep -rn "mypy_run\|from mypy.api" tests/ confirms this one file holds the only in-process mypy invocations in the suite, so the race is purely test-vs-test.

The failure is symmetric — either of the two sharing tests can lose the race. The reported flake was on test_should_accept_polarsframe_with_type_argument; in the baseline below it failed 4 times and the no-plugin test failed 3.

Fix

All three tests now go through a _run_mypy helper that injects --cache-dir .mypy_cache/<test method name>.

Why a derived path rather than more hand-written dedicated paths: adding two more fixed --cache-dir strings would fix today's flake but leaves the same trap for the next test added to this file — the failure mode is silent until it bites in CI. Deriving the path inside the single helper the tests call makes the correct behaviour the default and unittest's method-name uniqueness guarantees no two tests collide.

Why not tempfile.mkdtemp() per run: that defeats mypy's incremental cache and reintroduces exactly the problem #65 fixed — a full cold typeshed/stdlib check on every run. The derived path is stable across runs, so cache reuse is preserved while cross-test isolation is total. This is #65's reasoning generalised, not reversed.

Side effect: test_should_catch_errors_with_plugin moves from .mypy_cache/plugin_regression_test to .mypy_cache/test_should_catch_errors_with_plugin, costing one cold run. The explanatory comment #65 added moves into the helper docstring, expanded with the sqlite/WAL detail.

Stress-test evidence

Cold cache was produced by moving the repo-local .mypy_cache/ aside into a scratch dir between iterations — no deletes anywhere. Machine: macOS, 18 cores, so -n auto gives 18 workers.

Before the fix — 9 failures / 40 runs, all database is locked at metastore.py:174:

Scope	Cache	Runs	Failures
test_regression.py only, -n auto	cold	30	7 (4× polarsframe, 3× no-plugin)
full suite tests/, -n auto	cold	10	2 (both no-plugin)

After the fix — 0 failures / 65 runs:

Scope	Cache	Runs	Failures
test_regression.py only, -n auto	cold	30	0
test_regression.py only, -n auto	warm	20	0
full suite tests/, -n auto	cold	15	0

Checks

uv run inv all green: 301 passed, coverage TOTAL 969 stmts / 316 branches / 100%, coverage-threshold Success, Rust suite ok (172 + 13 tests). No new lint-ignore rules and no bandit skips.

https://claude.ai/code/session_013nRj6HvRbwkzAoj3QPbiSj