HYBIM-790: fix Windows test-suite slowness

[fercor-cisco]

Summary

Fixes HYBIM-790 — the test suite takes ~23–28 min on Windows × Python ≥ 3.11 vs ~3–6 min on 3.10 / Linux / macOS, for an identical test set on an identical OS image.

Root cause: the autouse set_validated_config fixture rebuilds the singleton GalileoPythonConfig on every test (deliberate per-test isolation for pytest-xdist on 3.14+). That build fires 3 already-mocked async validation requests (healthcheck / login / current_user) through galileo_core's async_run / EventLoopThreadPool. On Windows + CPython 3.11+ the underlying IOCP poll (GetQueuedCompletionStatus) blocks ~11× longer per call (212 ms vs 19 ms) — a CPython 3.10→3.11 behavioral change in the Windows ProactorEventLoop. The requests are fully respx-mocked (no real socket/DNS), so the cost is purely the event-loop machinery, not the HTTP.

This made per-test setup ~1335 ms (3.11) vs ~71 ms (3.10), and setup totals ≈ the whole pytest wall time.

Fix

Three logically-distinct, independently-revertible commits:

1. test: skip per-test async config validation on Windows-slow path — wrap only the per-test config build in a _fast_config_validation() context manager that stubs ApiClient.make_request / ApiClient.request with canned, await-free payloads. With no await, the async_run dispatch is trivial (~0.2 ms) and the slow IOCP poll never happens. Per-test reset/teardown is unchanged; test bodies still drive the real validation/connect code with their own mocks. This is the same patching pattern already idiomatic across the suite (test_metric.py, test_config.py, test_configuration.py).

2. test: use RFC 6761 fake.test host instead of dotless localtest — replace the dotless single-label host localtest with fake.test (RFC 6761 reserved .test TLD), avoiding the ~2.7 s Windows dotless-host getaddrinfo penalty if any path ever resolves it. fake.test (not console.test) keeps api_url == console_url, since galileo_core derives api_url = console_url.replace("console","api"). Host-only swap; env-var prefixes untouched.

3. test: apply fast config-validation fixture to galileo-adk via shared helper — galileo-adk had the same autouse GalileoPythonConfig.get(...) pattern and the same latent cost. Extract the config-build stub into a repo-level, non-shipped test_support.config helper and reuse it from adk's fixture. The helper lives at the repo root (not under tests/) so the sibling packages — which each have their own top-level tests package — can import it without a package-name collision; galileo-adk gets ".." added to its pytest pythonpath to resolve it. galileo-a2a is left as-is: it never builds GalileoPythonConfig, so it has no such cost.

Verification

Measured on Windows CI (probe branch, GH Actions run 27447060566, both jobs green):

metric	pre-fix 3.11	with fix 3.11	with fix 3.10
per-test setup avg (full suite)	1335 ms	9.01 ms (~148×)	9.19 ms
pytest full-suite wall time	1388 s	66 s (~21×)	50 s
result	2015 passed / 5 skipped	2015 passed / 5 skipped	2015 passed / 5 skipped

3.11 setup is now identical to 3.10 — the version gap is eliminated with zero regressions. Locally: full tests/ suite 2015 passed / 5 skipped, ruff/mypy clean, poetry check clean.

Notes

• Only production-relevant changes are shipped here — none of the investigation scaffolding (probe scripts, timing plugin, event-loop toggle, matrix trimming) from the probe branch is carried over. The CI matrix is unchanged.
• Sibling packages (now addressed in this PR): galileo-adk shared the same autouse GalileoPythonConfig.get(...) pattern and latent Windows cost — now fixed via the shared test_support.config helper (252 adk tests pass, no regression). galileo-a2a turned out not to share the pattern (it never builds GalileoPythonConfig), so it only received the fake.test host rename and needs no fixture fix. Note: these sub-package suites are not in the current CI matrix, so this is a local-dev / future-CI saving rather than a current CI-time reduction.
• Still out of scope: the underlying cause is galileo_core's per-construction synchronous validation over async_run; a production-side fix would help real Windows SDK users, not just CI.

See HYBIM-790-investigation.md attached to https://splunk.atlassian.net/browse/HYBIM-790 for the full investigation.

🤖 Generated with Claude Code

[fercor-cisco]

🤖 This review was auto-generated by Claude Code.

PR #22 Review — HYBIM-790: Windows test-suite slowness

Verdict: Safe to merge. Every load-bearing claim in the investigation was independently verified against the galileo_core source rather than trusted. The change is safe under both concurrent (pytest-xdist) and sequential execution.

What the PR does

Two mechanical, independently-revertible commits touching only test scaffolding (no src/ changes):

1. The fix — wraps the per-test config build in tests/conftest.py in a _fast_config_validation() context manager that stubs ApiClient.make_request/request with canned, await-free payloads, scoped to the GalileoPythonConfig.get() call only.
2. The rename — localtest → fake.test (RFC 6761) across the main suite + sibling packages.

Concurrency & sequencing safety — verified

• Cross-process (pytest-xdist): patch.object(ApiClient, ...) mutates the class object in the worker's own process memory. Workers are separate OS processes, so there is zero cross-worker interference.
• Sequential within a worker: The with _fast_config_validation(): block closes before the fixture's yield. So during the actual test body, make_request/request are the real methods — test bodies that drive real validation/connect with their own respx mocks are unaffected.
• No patch leakage: patch.object as a context manager restores the original attribute on exit even if .get() raises. No bleed into subsequent tests.
• No background-thread race: async_run blocks the calling thread until all 3 validation requests complete, so the with block can't exit while a stubbed coroutine is still in flight; the function lookup happens at coroutine-creation time on the main thread inside the block.

Correctness of the stub — verified against galileo_core source

Claim	Verification
make_request is @staticmethod async def(request_method, base_url, endpoint, **kw)	✓ matches stub (patched as staticmethod)
request is instance def(self, method, path, **kw), called positionally as request(RequestMethod.GET, path=...)	✓ stub (self, request_method, path=None, **kw) binds correctly
Endpoint matching: healthcheck="healthcheck", login="login/api_key", user="current_user"	✓ each routes to the right canned payload
Payload shapes: login→access_token, current_user→User.model_validate, healthcheck→ignored	✓ matches base_config parsing
fake.test keeps api_url == console_url	✓ console_url.replace("console","api") is a no-op (no console substring)

Residual safety check — verified

Grepped validated_api_client/.api_client in tests. The only real usages (test_datasets.py:439/460, test_api_client_integration.py) either use the config object purely as an identity reference in assert_called_once_with(client=...) (no network) or build their own ApiClient(...) with a dummy token. No test relies on the autouse fixture producing a real validated client.

Local verification (this branch, which has the SPLUNK_AO_ env rename the probe branch lacked)

• Config-sensitive subset (test_config, test_configuration, test_prompts_global, test_experiments): 143 passed, setup down to ~0.01s (fast path active).
• Full suite: 2015 passed, 5 skipped in ~14s — matches the claimed pass/skip counts exactly.
• No leftover localtest references anywhere in shippable code (rename is complete/consistent).

Minor notes (non-blocking)

• galileo-a2a / galileo-adk got only the host rename, not the fixture fix — they still pay the Windows cost. The PR body correctly flags this as out-of-scope follow-up. Not a safety issue.
• The _stub_request param name request_method (vs real method) would break if a caller passed method= as a keyword — but the only in-scope caller (current_user) passes it positionally, so it's fine.