diff --git a/.github/workflows/commitlint.yml b/.github/workflows/commitlint.yml index 3a23197a7..abef85faa 100644 --- a/.github/workflows/commitlint.yml +++ b/.github/workflows/commitlint.yml @@ -2,9 +2,12 @@ # SPDX-License-Identifier: MPL-2.0 # ───────────────────────────────────────────────────────────────────────────── -# Commitlint — Conventional Commits advisory check on PR titles +# Commitlint — Conventional Commits REQUIRED check on PR titles # -# Phase R1a of `docs/architecture/release-automation-plan.md`. +# Phase R1b of `docs/architecture/release-automation-plan.md` +# (promoted from the R1a advisory gate on 2026-06-10 after ≥1 month of +# observation; the trailing 80 first-parent merges on `main` were 100% +# conformant at promotion time). # # Why this exists: # @@ -19,21 +22,22 @@ # This workflow surfaces non-conforming titles at PR-open time so the # author can fix them before merge. # -# Why advisory (not enforcing): +# Why required (now enforcing): # -# Phase R1a is the OBSERVATION period. We're at ~100% adherence over -# the last 3 days (24/24 PRs) but only 83.3% over the last 30 days -# (75/90 PRs). The 15 non-conforming PRs use project-internal prefixes -# like `security:`, `bench:`, `shmem:`, `stream-stress:`, -# `cross-tool-benchmark:`, `gitignore:` (see -# `docs/architecture/release-automation-baseline.md` §4). Some of those -# may be re-tagged as `chore(security):`, `bench:` (if we choose to -# accept it as a real type), etc. Phase R1a gathers the data; Phase -# R1b (≥1 month later) makes the gate mandatory using the refined type -# list informed by the observation. +# Phase R1a (2026-04-25 → 2026-06-10) was the OBSERVATION period. At +# R1a start adherence was ~100% over the last 3 days (24/24 PRs) but +# only 83.3% over the last 30 days (75/90 PRs); the non-conforming PRs +# used project-internal prefixes like `security:`, `bench:`, `shmem:`. +# Over the observation window the project converged on the 11 standard +# Conventional Commits types (the `security:` carve-out was migrated to +# `fix(security):` / `chore(security):` — see the R1b CC-type +# convergence deviation row). At promotion the trailing 80 first-parent +# merges on `main` were 100% conformant, so the gate is now mandatory +# with negligible false-positive risk. # -# In advisory mode this workflow ALWAYS exits 0. Non-conforming titles -# produce a sticky PR comment, never a merge block. +# In required mode this workflow exits non-zero on a non-conforming +# title AND posts the sticky PR comment, so the author both sees the +# guidance and is blocked from merging until the title conforms. # # Sticky-comment design: # @@ -47,7 +51,6 @@ # # What this does NOT do: # -# • Block merge. Always exits 0. R1b will flip a single line to enforce. # • Validate individual commit messages on the feature branch. UFFS # uses squash-merge exclusively, so the PR title (which becomes the # squash subject) is the only commit message that lands on `main`. @@ -62,7 +65,7 @@ # (the spec that defines the type list and breaking-change syntax) # • plan §2.8 (current adherence baseline) and §3 (R1a → R1b transition) -name: "📝 Commitlint (advisory)" +name: "📝 Commitlint (required)" on: pull_request: @@ -132,7 +135,7 @@ jobs: echo "| --- | --- |" echo "| Title | \`${TITLE}\` |" echo "| Pattern | \`${PATTERN}\` |" - echo "| Mode | **advisory** (Phase R1a) — never blocks merge |" + echo "| Mode | **required** (Phase R1b) — blocks merge until the title conforms |" echo "" } >> "$GITHUB_STEP_SUMMARY" @@ -200,9 +203,9 @@ jobs: --- - **🟡 This check is ADVISORY during release-automation Phase R1a** — non-conforming titles do **not** block merge. Phase R1b (scheduled after ≥1 month of observation) will make this a required gate. Until then, please update the title when convenient; this comment will auto-delete on the next workflow run once the title conforms. + **🔴 This check is REQUIRED (release-automation Phase R1b)** — a non-conforming title **blocks merge**. Edit the PR title to match the pattern above; this check re-runs on every title edit and turns green (and this comment auto-deletes) once the title conforms. - *(Workflow: \`.github/workflows/commitlint.yml\` · Plan: [\`docs/architecture/release-automation-plan.md\`](https://github.com/${GITHUB_REPOSITORY}/blob/main/docs/architecture/release-automation-plan.md) Phase R1a)* + *(Workflow: \`.github/workflows/commitlint.yml\` · Plan: [\`docs/architecture/release-automation-plan.md\`](https://github.com/${GITHUB_REPOSITORY}/blob/main/docs/architecture/release-automation-plan.md) Phase R1b)* EOF ) @@ -229,7 +232,9 @@ jobs: --body "${BODY}" fi - # ADVISORY MODE: exit 0 even on non-conformance. - # Phase R1b will replace this single line with `exit 1` to - # promote the check from advisory to required. - exit 0 + # REQUIRED MODE (Phase R1b): exit non-zero on non-conformance + # so the check fails and (once added to branch protection's + # required-status-checks for `main`) blocks merge until the + # author fixes the title. The sticky comment above tells them + # how. Promoted from the R1a advisory `exit 0` on 2026-06-10. + exit 1 diff --git a/.github/workflows/release-plz.yml b/.github/workflows/release-plz.yml index 97c82a479..58d5640c6 100644 --- a/.github/workflows/release-plz.yml +++ b/.github/workflows/release-plz.yml @@ -428,13 +428,16 @@ jobs: # dormancy switch, NOT a lint-suppression hack — the job still never # runs until the maintainer deliberately sets the variable. # - # Enabling this in R8 requires: + # The OIDC auth + publish steps below are now wired (R8 bootstrap + # complete: uffs-time + uffs-text published manually with a token on + # 2026-06-10). Activating this automated path requires only: # 1. Configure crates.io crate-level trusted publishers (web UI) + # — Workflow filename `release-plz.yml`, Environment + # `crates.io-publish` (must match `environment:` below exactly) # 2. Create the `crates.io-publish` environment with required - # reviewers (manual approval gate for the dress rehearsal) + # reviewers (manual approval gate for each published version) # 3. Set repo variable `ENABLE_CRATES_IO_PUBLISH = true` # (Settings → Secrets and variables → Actions → Variables) - # 4. Uncomment the publish step below # # OIDC replaces the legacy `CARGO_REGISTRY_TOKEN` secret entirely — # no long-lived token is ever stored once trusted publishing is on. @@ -476,10 +479,38 @@ jobs: echo "OIDC token endpoint: $ACTIONS_ID_TOKEN_REQUEST_URL" echo "OIDC request token available: ${{ secrets.ACTIONS_ID_TOKEN_REQUEST_TOKEN != '' }}" - # Placeholder for R8 — actual publish steps will go here - # - name: Publish to crates.io (R8) - # env: - # # No CARGO_REGISTRY_TOKEN needed — OIDC handles auth - # run: | - # cargo publish -p uffs-time --dry-run # Dry-run first - # # Actual publish gated by manual approval in R8 + # Mint a short-lived crates.io token via OIDC trusted publishing. + # No long-lived CARGO_REGISTRY_TOKEN secret is ever stored — the + # token is exchanged at runtime against the trusted-publisher + # registrations on crates.io (one per publishable crate, bound to + # this workflow filename + the `crates.io-publish` environment). + - name: Authenticate with crates.io (OIDC) + id: auth + uses: rust-lang/crates-io-auth-action@bbd81622f20ce9e2dd9622e3218b975523e45bbe # v1.0.4 + + # Publish the publishable crates, dependency-ordered. The set is + # derived from `cargo metadata` (publish != []) so it adapts as + # more crates flip to publishable — today this is exactly + # uffs-time + uffs-text (both dependency-free leaves, any order). + # + # `cargo publish` is a no-op-error if the version already exists, + # so in steady state release-plz bumps the version first; this job + # then publishes the freshly bumped version. `--locked` enforces + # the committed Cargo.lock. `--no-verify` is intentionally NOT + # used — we want the pre-upload build to catch packaging errors. + - name: Publish to crates.io (OIDC, dependency-ordered) + env: + CARGO_REGISTRY_TOKEN: ${{ steps.auth.outputs.token }} + run: | + set -euo pipefail + mapfile -t crates < <( + cargo metadata --no-deps --format-version 1 \ + | jq -r '.packages[] | select(.publish != []) | .name' \ + | sort + ) + echo "Publishable crates: ${crates[*]}" + for crate in "${crates[@]}"; do + echo "::group::cargo publish -p ${crate}" + cargo publish -p "${crate}" --locked + echo "::endgroup::" + done diff --git a/docs/architecture/release-automation-plan.md b/docs/architecture/release-automation-plan.md index 99eee14e9..fd32137e0 100644 --- a/docs/architecture/release-automation-plan.md +++ b/docs/architecture/release-automation-plan.md @@ -1326,7 +1326,7 @@ green). **Steps**: 1. **Create a GitHub Environment** in the repo settings: - - Name: `crates-io-production` + - Name: `crates.io-publish` - Protection rule: required reviewers = 1 (maintainer list) - Deployment branch rule: only `main` branch - Environment secrets: **none** (trusted publishing doesn't @@ -1341,7 +1341,7 @@ green). - Repository name: `UltraFastFileSearch` - Workflow filename: `release-plz.yml` (relative to `.github/workflows/`) - - Environment: `crates-io-production` + - Environment: `crates.io-publish` - This binds the crate name to the exact GHA workflow that is authorized to publish it. 3. **Add the publish-eligible job to `release-plz.yml`** (but @@ -1352,7 +1352,7 @@ green). needs: release if: false # Gate flipped in Phase R9 runs-on: ubuntu-latest - environment: crates-io-production + environment: crates.io-publish permissions: contents: read id-token: write # MANDATORY for OIDC @@ -1395,7 +1395,7 @@ green). - Trusted-publisher registration done for all 13 reserved crate names. -- `crates-io-production` GitHub Environment exists with correct +- `crates.io-publish` GitHub Environment exists with correct protection rules. - OIDC dry-run via manual `workflow_dispatch` succeeds end-to-end (token minted, cargo auth succeeds, `--dry-run` publish @@ -1569,7 +1569,7 @@ on: Recommend Option A for simplicity. The workflow job from R7/R8 then becomes a dry-run integration test, not the publishing path. -4. Lock the publishing workflow behind the `crates-io-production` +4. Lock the publishing workflow behind the `crates.io-publish` GitHub Environment with required reviewers. Every published version needs a human click. The review should be "yes, the release notes are accurate and the version bump is @@ -1819,7 +1819,7 @@ crates.io Trusted Publisher form (as of 2026): | Repository owner | `skyllc-ai` | GitHub org or user | | Repository name | `UltraFastFileSearch` | Exact case | | Workflow filename | `release-plz.yml` | Relative to `.github/workflows/` | -| Environment name | `crates-io-production` | Must match the GHA `environment:` value exactly | +| Environment name | `crates.io-publish` | Must match the GHA `environment:` value in `release-plz.yml` exactly | The workflow must request `id-token: write` in its permissions block to mint the OIDC token. The OIDC audience crates.io expects @@ -1854,7 +1854,7 @@ and at that point the OIDC publish job fires (from R9 onward). - [ ] All 13 crate names reserved on crates.io under the project account - [ ] Trusted-publisher registrations complete for all 13 names -- [ ] `crates-io-production` GitHub Environment exists +- [ ] `crates.io-publish` GitHub Environment exists - [ ] `release-plz.yml` publish job has `if: true` - [ ] `release-plz.toml` has `publish = true` at workspace level - [ ] First-release communication drafted (blog post, release notes, @@ -1930,7 +1930,7 @@ are cheap, and owning `uffs` on crates.io prevents confusion. | 5 | **Commit convention violations** by contributors who bypass commitlint | Medium | Low → High over time (version accuracy degrades) | Advisory phase (R1a) gathers data; mandatory gate (R1b) blocks violators | | 6 | **crates.io rate limits** hit during full-workspace publish in R9 | Low | Medium (partial publish, DAG disrupted) | Release-plz handles rate limit backoff; publish order (§5.2) respects DAG so partial publish fails closed, not open | | 7 | **OIDC misconfiguration** in R7 blocks publishing in R9 | Medium | Medium (blocks release) | R7 dry-run validation catches this; rollback is trivial (reset `if: false`, investigate offline) | -| 8 | **Premature publishing** — someone flips `publish = true` before R9 gates are satisfied | Low | Very high (irreversible — cannot delete crates.io versions) | `if: false` gate + `crates-io-production` env required-reviewer + `publish = false` at workspace level + per-crate `publish = false` — **four independent locks**, all must be explicitly defeated | +| 8 | **Premature publishing** — someone flips `publish = true` before R9 gates are satisfied | Low | Very high (irreversible — cannot delete crates.io versions) | `if: false` gate + `crates.io-publish` env required-reviewer + `publish = false` at workspace level + per-crate `publish = false` — **four independent locks**, all must be explicitly defeated | | 9 | **SemVer regression** — patch release accidentally breaks API, violating SemVer contract | Medium | High (ecosystem trust damage) | Pre-R9 gate: `cargo-semver-checks` in PR Fast CI for ≥1 release cycle | | 10 | **Orphaned meta-crate** (`uffs`) reserved but never maintained | Low | Low (name squatting for one's own project) | Reserve as stub 0.0.0 with clear pointer to canonical crates; yank if decision is "no meta crate" | | 11 | **Release-plz tooling breakage** (upstream bug, API change, action deprecation) | Low per cycle, High over 2 years | Medium (blocks releases until fixed) | Action SHA pinning; fall back to manual `cargo publish` via runbook (§5.7); monitor release-plz changelog | @@ -2048,15 +2048,15 @@ Single source of truth for phase progress. Mirror the format of |---|---|---|---|---|---|---| | R0 | Baseline & cleanup (remove dead cargo-dist + release-plz metadata, add lockfile-drift patch per decision 1, new baseline doc) | 🟢 | `873a668c4` | 2026-04-25 | [#64](https://github.com/skyllc-ai/UltraFastFileSearch/pull/64) | Final landed PR shape: 6 files, +1258 / −160 LOC. Includes: lockfile patch; **promotion of `build/update_all_versions.rs` into version control** via `.gitignore` carve-out (1073-line script was previously gitignored despite 4 callsites depending on it); drive-by deletion of stale `crates/uffs-mft/Cargo.toml.bak` (v0.4.106 auto-commit artifact); baseline metrics in `release-automation-baseline.md`. | | R1a | Conventional commits (advisory) | 🟢 | `966f09c5f` | 2026-04-25 | [#65](https://github.com/skyllc-ai/UltraFastFileSearch/pull/65) | Final landed PR shape: 1 file added (commitlint workflow), 224 LOC. Workflow self-validated by running on its own opening PR (3-second pass). Sticky-comment mechanism via `gh api PATCH/DELETE` confirmed working. CONTRIBUTING.md "Commit message conventions" section already landed pre-R0 (lines 150-187). | -| R1b | Conventional commits (mandatory gate) | ⬜ | | | | After ≥1 month of advisory observation | +| R1b | Conventional commits (mandatory gate) | 🟢 | `HEAD` | 2026-06-10 | (this PR) | Promoted from R1a advisory after ~1.5 months observation (R1a landed 2026-04-25). Pre-flip verification: trailing **80 first-parent merges on `main` were 100% conformant** against the 11-standard-type pattern (the `security:`/`bench:`/`shmem:` prefixes from the R1a baseline converged out — `security` migrated to `fix(security):`/`chore(security):`, see the R1b CC-type convergence deviation row). `commitlint.yml` now `exit 1` (was `exit 0`) on non-conforming titles + posts the sticky comment. **Enforced**: `PR title — Conventional Commits` added to the `main-protection` ruleset's `required_status_checks` (ruleset id `11889528`, `strict` policy on) alongside `PR Fast CI / required`, so a non-conforming title now blocks merge. Caveat: the job is skipped on fork PRs (no write token for the sticky comment) — the normal UFFS flow (internal + Dependabot branches) is unaffected since those satisfy `head.repo.full_name == github.repository`; true external forks are guided via the PR template instead. | | R2 | `git-cliff` + `cliff.toml` (local validation) | 🟢 | `d49a778d6` | 2026-04-25 | [#66](https://github.com/skyllc-ai/UltraFastFileSearch/pull/66) | Final landed PR shape: 3 files (1 new, 2 modified), +495 / −3 LOC. `cliff.toml` template iterated against full history until output matches Keep-a-Changelog spacing; type → section mapping mirrors `commitlint.yml` regex (11 types). Validation captured in `release-automation-baseline.md` §8. Two iteration issues caught + fixed during template tuning (extra blank line after `## [version]`, duplicate `(#NN)` PR links). | | R3 | release-plz shadow mode | 🟢 | `1b0aa55b7` | 2026-04-25 | [#67](https://github.com/skyllc-ai/UltraFastFileSearch/pull/67) | Final landed PR shape: 2 files (1 new workflow, 1 new release-plz.toml) + ~370 LOC. Workflow runs `release-plz update` (local-only by design) on every `push: main` and posts the proposed diff to the workflow summary. Three layers of dormancy: `publish = false` in config, missing `CARGO_REGISTRY_TOKEN`, read-only workflow permissions. **Post-merge observation** revealed shadow output stayed empty across ≥12 days because `release-plz update` failed silently inside the workflow on `cargo package`'s "dependency `uffs-X` does not specify a version" error — fixed in R3.5 below by adding `version = ` requirements to internal `[workspace.dependencies]` entries. | | R3.5 | Internal-dep `version = ` requirements + polars git-pin version annotation | 🟢 | `cccf4f111` | 2026-05-07 | [#145](https://github.com/skyllc-ai/UltraFastFileSearch/pull/145) | Bundled into the R6 PR (see §8.1 deviations log first row). Adds `version = "0.5.90"` to all 8 internal workspace.dependencies, to the 2 direct path-deps in `uffs-cli/Cargo.toml`, and `version = "0.53.0"` to the polars git dep. Updates `just polars` to keep the polars version pin in lockstep with the resolved git rev. Without these, every `cargo package` invocation (release-plz `update` and any future `release-pr`) fails with "dependency `` does not specify a version". Verified locally: `release-plz update --config release-plz.toml` now lists all 12 publishable crates without error. | | R4 | release-plz active (release PR mode) | 🟢 | `HEAD` | 2026-06-09 | (this PR) | Push trigger re-enabled after Polars 0.54.4 resolved the chrono conflict. Workflow now auto-triggers on every push to main, analyzes conventional commits, and opens release PRs when feat/fix/perf/security changes land. Added workflow_dispatch bridge to release.yml to handle GitHub's anti-loop policy (GITHUB_TOKEN-created tags don't trigger downstream workflows). First release through new flow will satisfy exit criterion. | | R5 | Retire bespoke tooling (incl. `scripts/ci/ci-pipeline.rs` thin wrapper per its `REMOVE-AFTER: v0.5.73` marker) | 🟢 | `HEAD` | 2026-06-09 | (this PR) | Re-applied now that Polars 0.54.4 on crates.io resolves the publishability blocker. Deleted: `build/update_all_versions.rs`, `scripts/ci/ci-pipeline.rs`, `.github/workflows/auto-tag-release.yml`, version-bump functions from `version.rs`, `STEP_VERSION_INCREMENT` from workflow. Updated: `justfile` recipes, `.gitignore` restored to blanket `build/`. Version bumps now handled entirely by release-plz when conventional commits land on `main`. | | R6 | crates.io metadata audit + dry-run CI | 🟢 | `cccf4f111` | 2026-05-07 | [#145](https://github.com/skyllc-ai/UltraFastFileSearch/pull/145) | Adds: `[package.metadata.docs.rs]` to all 12 publishable crates with appropriate `targets`/`default-target` per crate's platform surface; explicit `publish = false` to `crates/uffs-diag/Cargo.toml`; per-package `release = false` blocks for the 3 internal CI tools in `release-plz.toml`; `.github/workflows/crates-io-dry-run.yml` (advisory weekly + workflow_dispatch); `docs/publishing.md` DORMANT runbook. R6 step 6 (crate name reservations on crates.io) is intentionally **deferred** — those happen from a throwaway external workspace per plan §R6 step 6, not from this repo. | -| R7 | OIDC trusted publisher (dormant) | 🟡 | `HEAD` | 2026-06-09 | (this PR) | Scaffolding complete. Added `crates-io-publish` job with OIDC token permissions, environment protection, and placeholder steps. Originally gated `if: false`; replaced with a repo-variable gate (`if: vars.ENABLE_CRATES_IO_PUBLISH == 'true'`) after actionlint flagged the constant-false expression. Dormant until after the R8 bootstrap: trusted publishers can only be configured on crates.io for crates that already exist, so the first publish of each crate uses a maintainer `CARGO_REGISTRY_TOKEN`; then configure trusted publishers and set the repo variable (see `R8 prep` deviation row, 2026-06-10). | -| R8 | First publish dress rehearsal (`uffs-time` only) | ⬜ | | | | **External state change** — one crate goes live on crates.io | +| R7 | OIDC trusted publisher (dormant, now wired) | 🟡 | `HEAD` | 2026-06-10 | (this PR) | Scaffolding complete **and publish steps now wired** (post-R8-bootstrap). Added `crates-io-publish` job with OIDC token permissions and environment protection. Originally gated `if: false`; replaced with a repo-variable gate (`if: vars.ENABLE_CRATES_IO_PUBLISH == 'true'`) after actionlint flagged the constant-false expression. The placeholder publish step is replaced by a real `rust-lang/crates-io-auth-action@v1.0.4` (pinned SHA `bbd81622…`) token mint + a dependency-ordered `cargo publish` loop over `cargo metadata` publishable crates (today: `uffs-time`, `uffs-text`). Still DORMANT: job only runs once the maintainer (1) registers trusted publishers on crates.io with environment `crates.io-publish`, (2) creates that GitHub Environment with required reviewers, and (3) sets `ENABLE_CRATES_IO_PUBLISH = true`. Flips to 🟢 on first OIDC-driven publish (R9). | +| R8 | First publish dress rehearsal (bootstrap publish) | 🟢 | `d496d03b7` | 2026-06-10 | (manual) | **External state change DONE** — `uffs-time` **and** `uffs-text` v0.5.120 published to crates.io via maintainer `CARGO_REGISTRY_TOKEN` (manual `cargo publish`, both dependency-free leaves, order interchangeable). Verified: `cargo search` shows 0.5.120 for both; consumed `uffs-time` from a throwaway `cargo new` project (clean build). Bootstrap uses a token (NOT OIDC) because crates.io trusted publishers can only be registered on crates that already exist — see `R8 prep` deviation row + the `R8 OIDC wire-up` row below. Token to be revoked after R9 trusted-publisher registration. | | R9 | Live publishing (full workspace) | ⬜ | | | | **DEFERRED** — explicit maintainer decision, separate plan | Legend: ⬜ pending · 🟡 in progress · 🟢 complete · 🔴 blocked · ⏸️ paused @@ -2084,6 +2084,7 @@ Mirror the format of | R6 → R8 publishability resolution (Path A) | 2026-05-08 | Resolution of the prior `R6 → R8 publishability` deviation row. Probed option (b) of the original row's resolution column ("aligning chrono with crates.io polars expectations") and found it infeasible. Probe details: dropped the polars `git/rev` pin in `crates/uffs-polars/Cargo.toml` and switched to `polars = "=0.53.0"` from crates.io. Workspace `chrono` pinned to `=0.4.41` to satisfy crates.io polars-arrow 0.53.0's `<=0.4.41` constraint. `cargo update` succeeded. But `cargo build --workspace` then hard-failed in two independent places: (1) `polars-arrow-0.53.0/src/bitmap/bitmask.rs:2` — `use std::simd::{LaneCount, SupportedLaneCount, …}` against current nightly (`nightly-2026-05-08`) reports "no `LaneCount` in `simd`" because the upstream `std::simd` API has moved post-0.53.0-release; (2) `polars-ops-0.53.0/src/chunked_array/strings/case.rs:79` — `use core::unicode::{Case_Ignorable, Cased}` reports "no `Cased` in `unicode`" and "function `Case_Ignorable` is private" against the same nightly. Both code paths are gated by `polars/nightly`, but `polars/nightly` is also pulled transitively through `polars-stream`/`polars-lazy`/`polars-expr`/`polars-plan` even when the top-level `nightly` feature is disabled in our config. Conclusion: the in-workspace polars `git/rev` pin (`1e9a63b9...`) was NOT opportunistic. It carries upstream nightly-API patches that the published 0.53.0 release lacks, and dropping it breaks the build. Path B-i abandoned. | The git-rev / published-version skew is fundamental: the same crate version (`0.53.0`) ships TWO different sets of source contents. The git rev's `polars-arrow` declares `chrono ^0.4.42`; the published `polars-arrow 0.53.0` declares `chrono <=0.4.41` — no chrono version satisfies both. The git rev is necessary for the workspace to build on current nightly, so it cannot be dropped. An older nightly pin would also break unrelated workspace deps (Tokio/`std::simd`/`tracing` API drift over the same window), so a Path C ("regress nightly") was rejected without probing. | Executed option (a) of the original row. Added `publish = false` to the 8 polars-tainted crates' Cargo.toml: `uffs-polars`, `uffs-mft`, `uffs-format`, `uffs-core`, `uffs-daemon`, `uffs-client`, `uffs-mcp`, `uffs-cli` (the user's table called out 6; the actual chain is 8 because `uffs-client` inherits polars via `uffs-format → uffs-mft → uffs-polars`, and `uffs-mcp` via `uffs-client → …`). Replaced the corresponding `[[package]]` blocks in `release-plz.toml` from `changelog_path = "CHANGELOG.md"` to `release = false` so release-plz skips them entirely (no version bump computation, no `cargo package` step). The 4 polars-free crates (`uffs-time`, `uffs-text`, `uffs-security`, `uffs-broker`) remain release-eligible with their original `changelog_path` entries. Retired the `just polars` recipe (PR-internal — replaced with a deprecation stub in `just/test.just` that points users at `cargo update -p polars` plus a chrono-pin compat checklist) since bumping the rev now risks pulling in MORE nightly-API drift faster than upstream polars publishes patches. Removed the `just polars` line from `just/help.just`. | R6 PARTIALLY RESOLVED — the publishability invariant is now scoped to 4 of 12 crates. R7 (OIDC scaffolding) unaffected — the dormancy gate doesn't care which crates are publishable. R8 dress rehearsal still feasible on its originally-chosen leaf target (`uffs-time` is polars-free). R9 (full publish) DEFERRED until polars upstream publishes a release containing the nightly-API patches our `git/rev` carries (track via `crates-io-dry-run.yml` weekly). When that release ships: (1) flip the 8 × `publish = false` to unset (or remove the line), (2) flip the 8 × `release = false` back to `changelog_path = "CHANGELOG.md"` in `release-plz.toml`, (3) drop the `git/rev` pin in `uffs-polars/Cargo.toml` in favor of the new published version, (4) re-evaluate the workspace `chrono = "=0.4.41"` pin (likely loosen if polars-arrow relaxes its upper bound), (5) restore the `just polars` recipe (or replace with a `just bump-polars` that takes a version arg). `crates-io-dry-run.yml`'s ADVISORY mode comments at lines 19-39 and 245-251 reference the original deviation by old framing — those will be refreshed in the same future PR that re-enables the polars subtree. | | R6 → R8 publishability resolution (crates.io polars 0.54) | 2026-06-09 | Final resolution of the `R6 → R8 publishability` / `…resolution (Path A)` rows. Polars published `0.54.4` to crates.io (edition 2024, 2026-06-04) carrying the nightly-`std::simd` / `core::unicode` patches that the abandoned Path B-i probe found missing in published `0.53.0`, and its `polars-arrow` now accepts modern `chrono` (workspace resolves `chrono 0.4.45`). Dropped the `git`+`rev` pin in `crates/uffs-polars/Cargo.toml` → plain `polars = "0.54.4"` from crates.io. Handled the 0.54 breaking changes: feature `new_streaming` → `streaming`; `LazyFrame::with_new_streaming` → `with_streaming`; `&ChunkedArray` no longer implements `IntoIterator` (replaced `.into_iter()` with `.iter()` at ~25 call sites in `uffs-core`/`uffs-diag`). Retired the track-upstream-main machinery: `update_polars_git()` + `STEP_UPDATE_POLARS` (`scripts/ci-pipeline`), the `just polars` HEAD-bump recipe (now a crates.io SemVer bump), and the `pola-rs/polars` `deny.toml` `allow-git` entry (zero git sources remain). `cargo check --workspace --all-targets` + `cargo clippy --workspace --all-targets` both clean. | The original blocker was the git-rev / published-version skew on `0.53.0` (git `polars-arrow` wanted `chrono ^0.4.42`, registry wanted `<=0.4.41`; and registry `0.53.0` was un-buildable on current nightly). A published `0.54.x` that both builds on nightly and accepts modern chrono removes the need for any git source, collapsing the divergence to zero. | `uffs-polars` now packages as a plain registry dependency, so `cargo package -p uffs-polars` is no longer blocked by the chrono clash. **Unblocks R5/R6/R8**: release-plz reactivation + the crates.io dry-run / `uffs-time` dress rehearsal can proceed without the polars publishability caveat. No release-workflow flips made in this change — gate-lift recorded here for the next release-automation pass. | | R5 re-application (post-Polars) | 2026-06-09 | R5 originally landed via PR #153 on 2026-05-08 but was rolled back same-day when the chrono conflict in `cargo package --workspace` forced release-plz deferral to `workflow_dispatch`-only (PR #157). With Polars 0.54.4 now on crates.io (see prior row), the publishability blocker is resolved and R5 can safely re-land. | The Polars git-rev / crates.io skew was the only remaining blocker preventing `cargo package` from succeeding. Once Polars 0.54.4 arrived with nightly-compatible code and modern chrono support, the path to R5 was cleared. | Re-applied R5: deleted `build/update_all_versions.rs`, `.github/workflows/auto-tag-release.yml`, `scripts/ci/ci-pipeline.rs`, version-bump functions from `version.rs`, and `STEP_VERSION_INCREMENT`. Updated justfile recipes and `.gitignore`. Dashboard R5 flips from 🔴 ROLLBACK to 🟢. | R5 now 🟢. Bespoke version tooling is retired; release-plz owns all version bumps. This is the final state — no further rollback expected. | +| R8 OIDC wire-up + env-name doc fix | 2026-06-10 | After the R8 bootstrap published `uffs-time` + `uffs-text` v0.5.120 by hand (token), the dormant `crates-io-publish` job in `release-plz.yml` still carried a commented-out placeholder publish step, and §R7/§5.6 of this plan named the OIDC environment `crates-io-production` while the live workflow uses `environment: crates.io-publish` — a mismatch that would have caused trusted-publisher registration to silently fail (crates.io requires the registered environment name to match the workflow's `environment:` value exactly). | (1) Bootstrap publish had to come first (chicken-and-egg: trusted publishers can only be registered on crates that already exist), so the publish step was deliberately left dormant through R7 and only wired after R8. (2) The `crates-io-production` name was an early-draft placeholder in the plan that was never reconciled against the workflow when the job was scaffolded in R7. | Replaced the placeholder with a real publish path: `rust-lang/crates-io-auth-action@bbd81622…` (v1.0.4, SHA-pinned) mints a short-lived OIDC token, then a `cargo metadata`-derived, dependency-ordered `cargo publish --locked` loop publishes the publishable set (today `uffs-time`, `uffs-text`). Job stays dormant behind the `ENABLE_CRATES_IO_PUBLISH` repo-variable gate + the `crates.io-publish` environment (required reviewers). Renamed all `crates-io-production` → `crates.io-publish` across §R7, §5.6, §5.7 checklist, §R9, and the risk table so the doc matches the workflow exactly. | Unblocks the R9 cutover: the maintainer now only needs to register trusted publishers (env `crates.io-publish`), create the GitHub Environment with reviewers, set the repo variable, and revoke the bootstrap token — no further workflow edits required. R7 advances to "wired"; R8 marked 🟢 (bootstrap done). | | R8 prep — publishable set narrowed (12 → 2) + credential path | 2026-06-10 | Pre-publish audit for R8 found the "12/13 publishable crates" figure used throughout this plan and `release-automation-baseline.md` (R0/R3.5/R6 entries) is no longer the live state. `cargo metadata --no-deps` (authoritative, resolves workspace inheritance) shows exactly **2** members with `publish = ANY`: `uffs-time` and `uffs-text`. All 17 others resolve to `publish = []` (false) — `uffs-broker`, `uffs-broker-protocol`, and `uffs-security` were flipped back to `publish.workspace = true` (= false) during the publishability deep-dive (name-squat reservations / internal-only; see `docs/refactor/crates-io-publishability-deep-dive.md` §7.3–7.5), and the root-manifest comment lagged behind (corrected in PR [#385](https://github.com/skyllc-ai/UltraFastFileSearch/pull/385)). | The strategic publish scrub happened incrementally across the deep-dive without a single "recount" pass; historical plan/baseline entries are point-in-time records and stay unedited per append-only discipline — this row is the correction of record. | **Publish sequence for R8/R9 (dependency-ordered):** both publishable crates are dependency-free leaves with zero internal deps, so the bootstrap is simply (1) `cargo publish -p uffs-time`, (2) `cargo publish -p uffs-text` (order interchangeable); dry-runs verified green for both, names available on crates.io. No other crate can join the set without first flipping its never-publish internal deps (`uffs-polars`/`uffs-security`/`uffs-format` → blocking `uffs-mft` → blocking `uffs-client`/`uffs-mcp`/`uffs-cli`) — a deliberate architecture decision, not a gap. **Credential path:** crates.io Trusted Publishing is configured per-crate in the crate's settings page and therefore requires the crate to already exist — no pending-publisher flow exists — so the **first** publish of each crate must use a maintainer `CARGO_REGISTRY_TOKEN` (manual `cargo publish` or repo secret); after bootstrap, configure the trusted publisher for both crates and flip the dormant R7 OIDC job (`vars.ENABLE_CRATES_IO_PUBLISH`), then revoke the token. | R8's dress-rehearsal target (`uffs-time` only) is unchanged and now fully unblocked. R9 "full workspace" should be re-read as "the publishable set" (currently 2 crates). | ## 9. Cross-references diff --git a/docs/publishing.md b/docs/publishing.md index 2b46c4923..56f745364 100644 --- a/docs/publishing.md +++ b/docs/publishing.md @@ -7,14 +7,22 @@ UFFS Publishing Runbook # UFFS Publishing Runbook -> **STATUS**: **DORMANT** — publishing is not yet live. +> **STATUS**: **BOOTSTRAP DONE — automated publishing still DORMANT.** > -> Do not execute any of the steps in this runbook until the **R9 go-live -> decision** has been recorded in +> R8 bootstrap complete (2026-06-10): `uffs-time` and `uffs-text` +> v0.5.120 were published to crates.io **once, by hand**, using a +> maintainer `CARGO_REGISTRY_TOKEN` (the chicken-and-egg below). +> Automated, ongoing publishing via the `release-plz.yml` OIDC job is +> still **dormant** and must not be activated until the **R9 go-live +> decision** is recorded in > [`docs/architecture/release-automation-plan.md` §8 status dashboard][dashboard]. -> Until then this document is a **forward-looking specification** of how -> UFFS will eventually ship to crates.io, captured here so the steps can -> be reviewed, audited, and refined while still safe to do so. +> +> **Chicken-and-egg (why the bootstrap is manual):** crates.io Trusted +> Publishing (OIDC) is configured per-crate on the crate's settings +> page, which only exists *after* the crate is published. There is no +> pending-publisher flow. So the **first** publish of each crate must +> use a token; OIDC is wired up afterwards and every subsequent publish +> is tokenless. > > [dashboard]: architecture/release-automation-plan.md#8-status-dashboard @@ -25,20 +33,24 @@ in the release-PR review step. Release-plz opens a release PR; a maintainer reviews the changelog, confirms the version bump, merges, and at that point the OIDC publish job fires (from R9 onward). -The four-layer dormancy stack that protects us until R9 is live: - -1. **`publish = false`** at the workspace level in `release-plz.toml` - — release-plz never invokes `cargo publish` while this is `false`. -2. **`publish = false`** per-package in selected `Cargo.toml`s - (`crates/uffs-diag`, `scripts/ci-pipeline`, `scripts/ci/gen-hooks`, - `scripts/ci/gen-workflow`) — even a manual `cargo publish` from a - developer machine refuses these. -3. **`if: false`** on the OIDC publish job (added in R7) — the - workflow step never runs even when triggered. -4. **No `CARGO_REGISTRY_TOKEN`** secret in the repository — even a - misconfigured workflow has no credential to authenticate with. - -All four must be defeated independently to ship a crate. There is no +The dormancy stack that keeps *automated* publishing off until R9: + +1. **`ENABLE_CRATES_IO_PUBLISH` repo variable unset** — the + `crates-io-publish` OIDC job in `release-plz.yml` is gated on + `if: vars.ENABLE_CRATES_IO_PUBLISH == 'true'`, so it never runs + until a maintainer deliberately sets the variable. (Replaces the + former `if: false`, which actionlint rejected as a constant + condition.) +2. **No trusted-publisher registration** on crates.io for the + `crates.io-publish` environment — even if the job ran, the OIDC + token mint would fail with no registered publisher to match. +3. **No `crates.io-publish` GitHub Environment** with reviewers — the + `environment:` reference would not resolve to an approval gate. +4. **No long-lived `CARGO_REGISTRY_TOKEN`** secret in the repository + — the bootstrap token lived only on the maintainer's machine and + is revoked after R9 registration; CI never stores it. + +All must be defeated independently to ship a crate via CI. There is no single accidental flip that can leak. ## Phase status as of this document's last update @@ -46,14 +58,14 @@ single accidental flip that can leak. | Phase | What it adds | Status | |---|---|---| | R3 | Shadow-mode `release-plz update` workflow | ✅ landed | -| R3.5 | `version = ` requirements on internal & polars deps (this PR) | 🟡 in progress | -| R4 | Active-mode release-PR generator | ⬜ pending | -| R5 | Retire bespoke `build/update_all_versions.rs` tooling | ⬜ pending | -| R6 | Per-crate metadata + dry-run CI workflow (this PR's R6 work) | 🟡 in progress | -| R6 step 6 | Crate-name reservations on crates.io | ⬜ deferred | -| R7 | OIDC trusted-publishing scaffolding (`if: false` gated) | ⬜ pending | -| R8 | Dress rehearsal — publish `uffs-time` (foundation crate) | ⬜ pending | -| R9 | Live publishing for the full publishable set | ⬜ pending | +| R3.5 | `version = ` requirements on internal & polars deps | ✅ landed | +| R4 | Active-mode release-PR generator | ✅ landed | +| R5 | Retire bespoke `build/update_all_versions.rs` tooling | ✅ landed | +| R6 | Per-crate metadata + dry-run CI workflow | ✅ landed | +| R6 step 6 | Crate-name reservations on crates.io | ✅ via bootstrap (2 crates) | +| R7 | OIDC trusted-publishing scaffolding (repo-variable gated, wired) | 🟡 wired, dormant | +| R8 | Bootstrap publish — `uffs-time` + `uffs-text` v0.5.120 | ✅ done (token, manual) | +| R9 | Live OIDC publishing for the publishable set | ⬜ pending | See the canonical dashboard at [`docs/architecture/release-automation-plan.md` §8][dashboard]. @@ -62,22 +74,22 @@ See the canonical dashboard at These must all be ✅ before the R9 go-live PR opens: -- [ ] All publishable crate names reserved on crates.io under the - project owner's account (R6 step 6, deferred). -- [ ] **Known-blocker resolution**: the `uffs-polars` git pin's - published-form `polars = "0.53.0"` resolves cleanly against - the workspace `chrono` pin OR `uffs-polars` is converted to - `publish = false`. (Tracked in - [release-automation-plan.md §6.1 risk #13][r6-known-blockers] - and the `crates-io-dry-run.yml` workflow header comment.) +- [x] Publishable crate names exist on crates.io under the project + owner's account (`uffs-time`, `uffs-text` — published in the + 2026-06-10 bootstrap; R6 step 6 satisfied for the 2-crate set). +- [x] **Known-blocker resolution**: `uffs-polars` now uses plain + `polars = "0.54.4"` from crates.io (git pin dropped), so the + chrono clash is gone. Moot for the publishable set anyway — + `uffs-polars` resolves to `publish = false`. (History: + [release-automation-plan.md §6.1 risk #13][r6-known-blockers].) - [ ] Trusted-publisher (OIDC) registrations complete for every - publishable crate name (R7). -- [ ] `crates-io-production` GitHub Environment exists with the + publishable crate name (`uffs-time`, `uffs-text`) with + environment `crates.io-publish`. +- [ ] `crates.io-publish` GitHub Environment exists with the required-reviewer rule active. -- [ ] `release-plz.yml` publish job has `if: true` (currently - `if: false` per the four-layer dormancy stack above). -- [ ] `release-plz.toml` has `publish = true` at the workspace - level (currently `publish = false`). +- [ ] Repo variable `ENABLE_CRATES_IO_PUBLISH = true` set (this is + the live gate; replaces the former `if: false`). +- [ ] Bootstrap `CARGO_REGISTRY_TOKEN` revoked once OIDC verified. - [ ] First-release communication drafted (blog post, release notes, social-media announcement). @@ -99,7 +111,8 @@ These must all be ✅ before the R9 go-live PR opens: 1 CHECKSUMS, 13 SBOMs — see `release.yml` for the asset manifest). - [ ] Publish job succeeds for all eligible crates (check Actions - run logs; expect 12 successful per-crate publish steps). + run logs; expect one successful publish step per publishable + crate — currently 2: `uffs-time`, `uffs-text`). - [ ] Each published crate appears on crates.io within 60 sec (`cargo search uffs-time`, `uffs-text`, etc.). - [ ] docs.rs builds succeed for all published crates within @@ -144,24 +157,21 @@ release MUST ship, fall back to manual `cargo publish` in dependency order. **Use this only as an emergency lever — the standard path is release-plz.** -The publish order is the topological sort of the internal-dep DAG. -For UFFS as of v0.5.90: +The publishable set as of v0.5.120 is exactly **two** dependency-free +leaves (`cargo metadata --no-deps` is authoritative; all other members +resolve to `publish = false`): ``` 1. uffs-time (zero internal deps) 2. uffs-text (zero internal deps) -3. uffs-security (zero internal deps) -4. uffs-polars (zero internal deps; external git pin) -5. uffs-mft (deps: polars, text, security) -6. uffs-format (deps: time, mft) -7. uffs-core (deps: polars, text, time, mft, format, security) -8. uffs-client (deps: security, format) -9. uffs-broker (deps: security) -10. uffs-mcp (deps: client) -11. uffs-daemon (deps: security, mft, core, client) -12. uffs-cli (deps: client, format, time) ``` +Order is interchangeable — neither depends on the other. No other crate +can join the set without first flipping its never-publish internal deps +(`uffs-polars`/`uffs-security`/`uffs-format` → blocking `uffs-mft` → +blocking `uffs-client`/`uffs-mcp`/`uffs-cli`), a deliberate architecture +decision (see `docs/refactor/crates-io-publishability-deep-dive.md`). + Per-crate command: ```bash @@ -173,20 +183,36 @@ the index has time to update — otherwise the next crate's `cargo publish` may fail with "no matching package named `` found". -## Trusted publishing (OIDC) configuration — to be filled in during R7 - -This section will document: - -- Each crate's trusted-publisher form-field values on crates.io - (repository, workflow filename, environment name). -- The `crates-io-production` GitHub Environment's required-reviewer - list. -- Rotation procedure if the OIDC trust breaks (e.g., the workflow - filename changes or the repository is renamed). -- Revocation procedure if a maintainer leaves or a credential - is suspected compromised. - -Until R7 lands, the section is intentionally empty. +## Trusted publishing (OIDC) configuration + +The `crates-io-publish` job in `release-plz.yml` mints a short-lived +crates.io token via `rust-lang/crates-io-auth-action` and runs a +dependency-ordered `cargo publish` loop. To activate it (R9), register +a trusted publisher on crates.io for **each** publishable crate with +these exact form-field values: + +| Field | Value | +|---|---| +| Repository owner | `skyllc-ai` | +| Repository name | `UltraFastFileSearch` | +| Workflow filename | `release-plz.yml` | +| Environment | `crates.io-publish` | + +**The environment name MUST be `crates.io-publish`** — it has to match +the `environment:` value in `release-plz.yml` exactly, or the OIDC +token mint fails. (An earlier draft of the plan said +`crates-io-production`; that was a doc bug, corrected 2026-06-10.) + +Then create the `crates.io-publish` GitHub Environment (Settings → +Environments) with a required-reviewer rule, and set the repo variable +`ENABLE_CRATES_IO_PUBLISH = true`. + +- **Rotation**: if the workflow filename changes or the repo is + renamed, ALL trusted-publisher registrations break and must be + re-registered with the new values. +- **Revocation**: revoke the bootstrap `CARGO_REGISTRY_TOKEN` (crates.io + → Account Settings → API Tokens) once OIDC is verified working; after + that, no long-lived credential exists anywhere. ## References @@ -200,7 +226,8 @@ Until R7 lands, the section is intentionally empty. shared between `release-plz update` and `git cliff` developer iteration. - [`.github/workflows/release-plz.yml`](../.github/workflows/release-plz.yml) - — shadow-mode release-PR generator. + — active release-PR generator + tag/release bridge + the dormant + `crates-io-publish` OIDC job. - [`.github/workflows/crates-io-dry-run.yml`](../.github/workflows/crates-io-dry-run.yml) — weekly metadata-drift detection job (R6 step 4). - crates.io documentation: