Skip to content

Pin toolchain downloads by SHA-256 and GitHub Actions by commit - #3185

Closed
cramforce wants to merge 2 commits into
mainfrom
sec/supply-chain-pins
Closed

cramforce wants to merge 2 commits into
mainfrom
sec/supply-chain-pins

Conversation

@cramforce

Copy link
Copy Markdown
Contributor

What changes

LLVM archives were downloaded without a checksum in CI, in the release jobs and in the LLVM distribution workflow. Node and Zig for the Sandbox image were checked against a checksum list fetched from the same origin as the archive. Actions were referenced by tag, and ci.yml and release.yml had no top-level permissions while the repository's default token can write.

  • One pin file. scripts/toolchain.sha256 holds the SHA-256 of every toolchain archive that CI, the release jobs and the Sandbox image download: LLVM 22.1.8 for Linux x64, Linux arm64 and Windows, the LLVM sources, Node 24.22.1, Zig 0.16.0 for both Linux architectures, and the key that signs the LLVM apt packages. scripts/verify-toolchain.sh <file> [<name>] checks a file against it and fails on another digest or on a file with no pin. It is POSIX sh because the Sandbox image downloads Node before it has Node, and it runs in Git Bash on the Windows runners.
  • Every download is checked before it is unpacked: .github/actions/llvm-linux, the Windows LLVM step of ci.yml, the Linux and Windows LLVM steps of release.yml, both archives in llvm-dist.yml, scripts/sandbox-bootstrap.sh, scripts/install-zig.sh and Dockerfile.sandbox. A version bump without a new line in the pin file fails with pins no SHA-256 for <file>.
  • The Linux LLVM cache key is the hash of the pin file, so a cache never outlives the digest its archive was checked against.
  • Actions by commit. All 141 uses: lines name a 40-digit commit with the version in a comment (actions/checkout@11d5960… # v4.4.0); each commit is what the tag pointed to today. .github/dependabot.yml proposes updates once a month as one pull request. If you would rather move the pins by hand, drop that file; nothing else depends on it.
  • Permissions. ci.yml and release.yml get top-level permissions: contents: read.
  • A test. tests/harness/ci-supply-chain.test.ts fails on a uses: that is not a commit with a version comment, on a workflow without top-level permissions, on a malformed pin file, and on a file that downloads from LLVM's releases, nodejs.org, ziglang.org or the apt key URL without running the verifier.
  • .dockerignore excluded scripts/install-zig.sh, which Dockerfile.sandbox already copied; it now lets through the three scripts the image build copies.

Where the digests come from

Each was computed from a download in a Sandbox and compared with a second source: GitHub's release-asset digests for the four LLVM files, Node's SHASUMS256.txt, and Zig's download index. All agree.

Permissions: what each job uses

  • ci.yml: no job writes to the repository, comments, or requests an OIDC token. Checkout needs contents: read; caches and artifacts use the runner's own token. validation_scope and lanes already declared theirs.
  • release.yml: check-release, cli-profile, publish (id-token: write, actions: read), npm-available and github-release (contents: write, actions: read) already declare what they use and are unchanged. The two jobs without a block, build-native-packages and build-mobile-runtime-packs, check out, build and upload artifacts, and now get contents: read instead of the repository default.
  • The other four workflows already had top-level permissions.

Checked

In a Sandbox (Ubuntu x86-64, 8 vCPU, Xeon 2.90 GHz):

  • verify-toolchain.sh accepts all eight downloaded files, rejects a wrong file under a pinned name and a file with no pin.
  • scripts/sandbox-bootstrap.sh runs to the end with the committed pins; with a wrong digest for the apt key or for Node it stops at that file before using it; with a Node version that has no pin it stops. scripts/install-zig.sh installs with the committed pin and stops on a wrong one.
  • ci-supply-chain, ci-stack-scope, ci-lanes, sandbox-config and node-pin tests: 89 pass. pnpm lint, pnpm format:check. Every workflow and action parses.
  • This branch merges without conflict with Balance the CI shards' side queue from measured costs and split checker identity #3119 and Run one Effect shard on pull requests that change the compiler or runtime #3121.
  • No compiler or runtime source changes, so no executable can change.

Not exercised

  • Release steps: release.yml's Linux and Windows LLVM download steps and its new top-level permissions only run in a release. The Windows steps are the same lines as ci.yml's, which this pull request's own run exercises; the Linux arm64 archive's digest was checked against a download but no arm64 job runs here. Nothing else in the release changes.
  • llvm-dist.yml (manual) and the docker build of Dockerfile.sandbox (no Docker in the Sandboxes).
  • actionlint was not available in the Sandbox; the workflows were parsed with js-yaml and this run is the check.

Not pinned by digest

Homebrew's LLVM on the macOS runners; container images referenced by tag (manylinux for the libnode packs, node:24-bookworm-slim, alpine:3.22, amazonlinux:2023); the Android NDK through sdkmanager; the tsc-ts checkout used as profile training input; and the profiled LLVM distribution from this repository's own release, which is checked against a .sha256 from the same release.

One-time cost

The cache keys of the Linux LLVM, the LLVM helpers and the runtime packs include the files this changes, so the first run after it lands downloads LLVM again (now verified) and rebuilds the helper and pack caches once.

🤖 Generated with Claude Code

LLVM archives were downloaded without a checksum in CI, in the release jobs
and in the LLVM distribution workflow; Node and Zig for the Sandbox image
were checked against a checksum list fetched from the same origin as the
archive.

- scripts/toolchain.sha256 holds the SHA-256 of every toolchain archive that
  CI, the release jobs and the Sandbox image download (LLVM for Linux x64 and
  arm64 and for Windows, the LLVM sources, Node, Zig, the LLVM apt key), and
  scripts/verify-toolchain.sh checks a file against it. Every download runs
  the check before the file is unpacked; a different digest or a file with no
  pin fails.
- The Linux LLVM cache key is the hash of the pin file, so a cache never
  outlives the digest its archive was checked against.
- Every action is referenced by commit with its version in a comment, and
  Dependabot proposes the updates once a month.
- ci.yml and release.yml get top-level permissions (contents: read); the
  jobs that publish, read other runs' artifacts or write the GitHub release
  already declare what they need.
- tests/harness/ci-supply-chain.test.ts enforces the three rules.
- .dockerignore lets the Sandbox image build see the three scripts its
  Dockerfile copies.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
scriptc Ready Ready Preview, v0 Nov 7, 2026 6:07am UTC

@socket-security

socket-security Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatedgithub/​pnpm/​action-setup@​f40ffcd9367d9f12939873eb1018b921a783ffaa ⏵ b906affcce14559ad1aafd4ab0e942779e9f58b198100100100100

View full report

Comment thread .github/workflows/ci.yml Resolved
The seed contract test found the upload and download steps by their tag
(actions/upload-artifact@v4). The actions are pinned to commits now, so the
lookup returned undefined and the test threw. Select them by name, as the test
plan's test already does.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@cramforce

Copy link
Copy Markdown
Contributor Author

The four red macOS jobs on bd83d90 fail on two corpus programs that this change does not touch, and both fail the same way without it:

  • 6125-constants-module.cjs (test (macOS, plain, darwin 2/3), test (macOS, san, darwin 2/3), test (macOS arm64 LLVM artifacts, no clang, 3/5)): the same test fails in the macOS jobs of four unrelated pull requests run today (runs 42346273120, 42345721897, 42345284565, 42345203114).
  • 8830-process-report-island/main.ts (test (macOS arm64 LLVM artifacts, no clang, 5/5)): fails in main's last nightly macOS run (run 42180209633, test (macOS, plain, 4/10)).

Every other check passed, including the jobs this change affects: the Linux and Windows LLVM downloads with the digest check, every job that uses a pinned action, and the bootstrap seed contract test fixed in bd83d90. The aggregate test is red only through those macOS jobs.

@cramforce

Copy link
Copy Markdown
Contributor Author

Superseded by train security-1, #3430. The train carries this branch at bd83d90, merged with current main: main's retry settings are kept inside this branch's download-then-check steps, and the eleven uses: lines main gained by tag since this branch was cut (among them the ICU download cache action) are pinned to the same commits. The train also extends ci-supply-chain.test.ts to one digest check per download line and to the ICU download cache. Validation, including the negative tests of the pins, is in #3430's description.

🤖 Generated with Claude Code

@cramforce cramforce closed this Nov 8, 2026
yowainwright pushed a commit to yowainwright/scriptc that referenced this pull request Nov 9, 2026
Toolchain downloads are checked against pinned SHA-256 digests and actions
are referenced by commit (vercel-labs#3185).

Conflicts, resolved by reading both sides:

- .github/actions/native-packages/action.yml: the helper cache key from the
  train (with the evaluator's inputs), main's comment, and the branch's
  commit pin on the cache step below it.
- .github/workflows/ci.yml and release.yml, the Windows LLVM steps: the
  branch's layout stays (one step downloads and checks the archive, the next
  unpacks it), so main's download line inside the unpack step is gone; the
  download keeps main's five attempts thirty seconds apart.
- .github/workflows/llvm-dist.yml and release.yml, the Linux LLVM steps: the
  branch's check after each download, with main's retry settings.

Main gained eleven `uses:` lines by tag after the branch was cut (the ICU
download cache action, two cache steps of the native packages action, the
generative workflow, one job of ci.yml and one step of release.yml). They
are pinned here to the same commits as the rest; each of the seven pinned
actions was checked against GitHub: its version tag names that commit.

Co-Authored-By: Claude Code <noreply@anthropic.com>

This branch was successfully deployed

1 active deployment
Preview — bd83d904 Deployed Nov 7, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant