OPCUA-3461 Add generator-selection invariant test - #387
Merged
Conversation
parasxos
requested review from
ben-farnham,
jimmat92,
jsouter,
luismiguensfernandez and
nknls
July 29, 2026 09:10
jimmat92
reviewed
Jul 30, 2026
There was a problem hiding this comment.
This also contains another important fix. Without chk() earlier scripts could fail and the job succeed.
You can separate this to another commit but not necessary in my opinion.
jimmat92
reviewed
Jul 30, 2026
There was a problem hiding this comment.
The tempfile.mkdtemp() dirs created on run_case() are not cleared. In test_no_unbound_delegate.py there is try/finally and shutil.rmtree(tmp, ignore_errors=True) which clears the generated dirs.
Worth to add this for consistency on run_case()
jimmat92
previously approved these changes
Jul 30, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A fourth hermetic invariants test,
.CI/test_generator_selection.py, guarding the CMake generator-selection contract ofgenerateCmake.pyat the argv level — plus wiring into both invariants lanes (alma9 + WS2025).Why
The OPCUA-3360 fix removed the dead hardcoded
Visual Studio 15 2017 Win64generator: with no env set, CMake now auto-detects the installed Visual Studio. But every Windows CI build job exportsQUASAR_CMAKE_GENERATOR=Ninja, and the BE-ICS w2025 provisioning (quasar-team/images) sets no generator env at all — so the no-env auto-detect path, the exact path a developer on a provisioned box hits out of the box, had zero CI coverage. A regression there would ship green and surface only on BE-ICS machines.The 6 cases
-G(auto-detect — the BE-ICS out-of-box path)QUASAR_CMAKE_GENERATOR=Ninja-G Ninja(the CI path)QUASAR_CMAKE_GENERATOR_PLATFORM=x64-G … -A x64_PLATFORMalone-G, no-A-G--builder Ninja-G NinjaEvery case additionally asserts the dead
Visual Studio 15 2017string never reappears.Hermetic in the style of the existing three invariants: the module graph is stubbed,
subprocessWithImprovedErrorscaptures the argv instead of executing — no CMake, no compiler, any host OS. Mutation-tested: re-introducing the old hardcode fails 4 of 6 cases, exit 1.Also
The WS2025 invariants step ran all tests as bare pwsh lines — a step fails only on the last command's exit code, so an early test failure could be masked by a later pass. Each test now gets an explicit exit-code check (same
chkpattern the SDK build step uses).