Skip to content

ci: add advisory macOS compile coverage - #802

Closed
luisleo526 wants to merge 4 commits into
mainfrom
ci/advisory-macos-compile
Closed

luisleo526 wants to merge 4 commits into
mainfrom
ci/advisory-macos-compile

Conversation

@luisleo526

@luisleo526 luisleo526 commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Adds one independent advisory macos-latest Debug job that builds the library and test binaries without running tests. It authorizes before checkout and selects an immutable repository commit. Eigen 3.4.0 is downloaded from its pinned upstream archive, verified by SHA-256 and installed before engine configuration. Cache steps restore Homebrew ccache downloads and compiler-cache objects; Eigen source downloads and installation are not cached. Caches save only after successful main pushes. Required merge/release aggregates and Linux release artifacts keep their existing scope.

This deliberately revises #532's Linux-only policy for compile coverage. WorkflowEventFleets.test_post_events_require_post_fleet_except_the_fixed_hosted_job, WorkflowEventFleets.test_pull_request_workflows_never_name_post_fleet and PreflightFailures.test_new_workflows_require_event_fleets_without_conditional_routes admit only the complete standalone compile contract while retaining generic event/runner refusals. PostLinuxBuildPolicy.test_post_refuses_a_retired_macos_job and PostLinuxBuildPolicy.test_release_prebuilt_refuses_retired_macos_steps remain enforced. The workflow loader rejects altered or missing compile workflows.

Validation: the original checker failed the new workflow's admission assertion as expected. The dependency repair and its exact policy mirror passed all 558 preflight tests across eight shards, five release-policy tests, workflow/runner/actionlint checks, 2,346 documentation anchors and documentation lint. Source hashes were unchanged before and after execution. The first hosted old/new runs both failed during dependency configuration and supplied no compiler witness. Actual hosted macOS compiler RED/GREEN evidence, dependency-install success, cache-hit timing and three green main runs remain unverified. The new job stays advisory unless maintainers change that policy after three real green main runs.

🤖 Generated with Claude Code

Pin the independent compile workflow and exercise input, authorization,
cache, build-only and gate-independence controls.
Admit only the pinned macOS compile workflow, leaving existing proof
aggregates and Linux release artifacts unchanged.

@graphify-labs graphify-labs Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify reviewed this change.

Worth a look — the grounded gate found no coupling regressions or blocking issues, but 1 advisory finding(s) below merit a look before merge.

Not checked: tests were not run; formal verification proved 0 of 3 changed function(s) (3 not verified).

Formal verification. PR-changed functions: 0/3 verified (0 proven, 0 may-equivalent, 0 distinguished) · 3 not verified (3 vacuous).

Not verified on this run: main (vacuous: never exercised), runner\_findings (vacuous: never exercised), workflow\_fleet\_findings (vacuous: never exercised).


Graphify review — findings

Adds a macOS compile workflow that builds the Debug library and test binaries on macos-latest for pull requests to main, pushes to main, and manual dispatch of an exact commit SHA. It runs an authorization step before checkout that admits only same-repo, write-access, or allowlisted-bot events, and a source-selection step that resolves the target to a full 40-hex commit and verifies it against the repo before building. Homebrew and ccache caches are saved only from successful main pushes, and the surrounding ci_preflight script, tests and docs are extended to admit and lint the new workflow.

Worth a look

  • PRs can bypass the workflow authorization gate — .github/workflows/macos-compile.yml:32 · Escalate · high
    • unverified hypothesis — survived the adversarial refute pass; needs human review

Review partial — this diff was larger than one review pass covers, so later files were not reviewed; some findings may be missing.

Analysis details — impact, health, verification

Impact & health

Graphify review

Impact — 1090 functions depend on the 993 functions this change touches.

Health — this change adds coupling hotspots:

  • worse: ci_workflow_findings() — 90 callers, 25 callees
  • worse: runner_findings() — 2 callers, 30 callees
  • worse: workflow_fleet_findings() — 1 callers, 6 callees

Verification — 1090 functions in the blast radius were not formally verified this run (proofs are advisory here).

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

Advisory (not blocking):

  • verification_scope: 1057 function(s) in the blast radius were not formally verified this run

Test selection

Test selection

1400 of 1400 test file(s) selected (100%) via static blast radius.

Escalated to a full run for safety (noncode-change, uncovered-code) — the selection is not trustworthy on its own (see below). CI should run the whole suite.

  • benchmarks/tests/test_aggregate.py — full-run-safety
  • benchmarks/tests/test_check_provenance.py — full-run-safety
  • benchmarks/tests/test_compare.py — full-run-safety
  • benchmarks/tests/test_public_names.py — full-run-safety
  • benchmarks/tests/test_run_all.py — full-run-safety
  • benchmarks/tests/test_run_strategy_feeds.py — full-run-safety
  • docker/pineforge_sink_finalize_test.py — full-run-safety
  • docker/pineforge_sink_replay_rq_test.py — full-run-safety
  • docker/pineforge_sink_replay_test.py — full-run-safety
  • docker/pineforge_sink_to_json_test.py — full-run-safety
  • docker/quota_columns_test.py — full-run-safety
  • docker/quota_convert_test.py — full-run-safety
  • docker/quota_finalize_test.py — full-run-safety
  • docker/replay_rq_engine_test.py — full-run-safety
  • docker/run_json_artifact_window_test.py — full-run-safety
  • docker/run_json_bin_test.py — full-run-safety
  • docker/run_json_bounded_transport_test.py — full-run-safety
  • docker/run_json_bulk_equity_fuzz_test.py — full-run-safety
  • docker/run_json_bulk_equity_test.py — full-run-safety
  • docker/run_json_codes_test.py — full-run-safety
  • docker/run_json_converter_metadata_test.py — full-run-safety
  • docker/run_json_converter_session_test.py — full-run-safety
  • docker/run_json_diagnostics_test.py — full-run-safety
  • docker/run_json_equity_scan_test.py — full-run-safety
  • docker/run_json_fused_test.py — full-run-safety
  • docker/run_json_harnesses_test.py — full-run-safety
  • docker/run_json_metrics_only_test.py — full-run-safety
  • docker/run_json_misc2_test.py — full-run-safety
  • docker/run_json_misc_test.py — full-run-safety
  • docker/run_json_pfbar_test.py — full-run-safety
  • docker/run_json_provenance_test.py — full-run-safety
  • docker/run_json_quota_supervisor_test.py — full-run-safety
  • docker/run_json_replay_test.py — full-run-safety
  • docker/run_json_selected_sink_test.py — full-run-safety
  • docker/run_json_sink_lexical_test.py — full-run-safety
  • docker/run_json_sink_summary_test.py — full-run-safety
  • docker/run_json_sink_test.py — full-run-safety
  • docker/run_json_sink_vectors_test.py — full-run-safety
  • docker/run_json_spool_work_test.py — full-run-safety
  • docker/run_json_stringio_test.py — full-run-safety
  • docker/run_json_symbol_feeds_test.py — full-run-safety
  • docker/run_json_syminfo_test.py — full-run-safety
  • docker/run_json_trade_scan_test.py — full-run-safety
  • docker/run_json_window_budget_test.py — full-run-safety
  • docker/run_json_wrapper_work_test.py — full-run-safety
  • include/pineforge/native_run_spec.hpp — full-run-safety
  • scripts/derive_corpus_feeds_self_test.py — full-run-safety
  • scripts/fingerprint_self_test.py — full-run-safety
  • scripts/run_corpus_self_test.py — full-run-safety
  • scripts/test_abi_receipt_skips.py — full-run-safety
  • … and 1350 more

non-code file(s) changed (.github/workflows/macos-compile.yml, docs/ci.md) → running the full suite for safety (a code graph can't see config/fixture/data deps)

changed code file(s) with no mapped test (.github/workflows/macos-compile.yml, docs/ci.md) — a coverage gap or a missing link — running the full suite rather than only the selected tests

Selection is safe under the controlled-regression assumption; always-run tests + a periodic full run are the backstops. Advisory — it never changes the check verdict.

Docs that may be stale (advisory)

Formal verification

Could not verify: Could not verify main.

The verifier did not have enough to check main, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: not verifiable: all 1 sampled inputs raised on both versions — the function never executed, so 'no divergence' would be vacuous (mostly NameError — names the real obstacle, not a sampling gap)

Could not verify: Could not verify runner\_findings.

The verifier did not have enough to check runner\_findings, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 2 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

Could not verify: Could not verify workflow\_fleet\_findings.

The verifier did not have enough to check workflow\_fleet\_findings, so it is saying so rather than guessing. No false assurance is the whole point.

Guarantee: No guarantee either way, this is an honest abstention, not a pass.

Note: Reason: non-vacuity: domain too small (only 2 distinct inputs exercised, need 3) — 'no divergence' would be near-vacuous

· 3 more finding(s) on lines outside this diff (see the check run).

Install checksum-verified upstream Eigen headers and CMake exports in a
job-owned prefix before configuring the engine. Pass the installed typed
Eigen3_DIR and keep the engine's disconnected dependency fallback.

Synchronize the workflow's exact policy mirror without changing admission
predicates, authorization, existing test expectations or engine sources.
@luisleo526

Copy link
Copy Markdown
Collaborator Author

Closing: the repository's CI admission accepts only the standard Ubuntu runners, so a macOS workflow cannot be admitted as written. The compile error this PR found (fdatasync on macOS) is fixed on main by #798.

🤖 Generated with Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant