ci: gate chart releases on version bumps and image tags, and alert on drift

[Andrew Grathwohl (agrathwohl)]

Linear: FIRE-197

Chart releases lagged image releases, and a published chart version could change after release. The release bot in socket-nginx-firewall already opens the image bump PR here (PR 51 for 2.9.10). This PR adds gates and a drift alert around it.

• .github/workflows/chart-checks.yaml (new, every PR to main) fails when helm/ changed but version in helm/Chart.yaml did not, when image.tag in helm/values.yaml is set and differs from appVersion, when docker manifest inspect cannot resolve the image at appVersion, and when a pin the firewall release bot rewrites differs from helm/Chart.yaml. The bot updates the chart version and image pins in helm/values.yaml, helm/README.md, cloudformation/firewall-eks.yaml, cloudformation/values/dns-override.values.yaml and cloudformation/README.md, and exits without opening its bump PR when one is missing or does not match. The check uses the bot's patterns. A GitHub release can exist without its Docker Hub tag: v2.0.9 has one, and socketdev/socket-registry-firewall:2.0.9 returns 404.
• .github/workflows/release.yaml exits with an error when the chart tarball already exists on gh-pages instead of overwriting it, checks the image before publishing, and triggers on helm/**. Chart 0.9.0 was published twice with different contents after a helm/README.md change fired the release without a version bump.
• .github/workflows/chart-drift.yaml (new, daily at 14:00 UTC) posts to Slack through SLACK_HELM_DRIFT_WEBHOOK_URL in the alerts environment and fails the run when the newest stable image tag on Docker Hub differs from the published chart's appVersion and was pushed more than 72 hours ago.
• image.tag stays pinned in helm/values.yaml because the release bot rewrites it on every bump. The new check keeps it equal to appVersion.
• Deleted helm/.github/workflows/release.yaml. GitHub only reads workflows from the repo root, so it never ran.
• Chart 0.11.8 to 0.11.9, with the ChartVersion default in cloudformation/firewall-eks.yaml and cloudformation/README.md to match. The release bot requires them to equal the chart version.

Testing

• Ran each chart-checks.yaml step in a scratch repo. It passes on this branch and fails on a helm/ change without a bump, on a mismatched image.tag, and on appVersion 2.0.9. A change outside helm/ skips the version check.
• The pin check passes on this branch and fails on the PR 48 tree, on the ChartVersion default in cloudformation/firewall-eks.yaml and the chart version line in cloudformation/README.md. The release bot's updater gives the same result on both trees.
• docker manifest inspect resolves socketdev/socket-registry-firewall:2.9.10 and fails for 2.0.9.
• Ran the drift script against stubbed curl responses. In sync and drift under 72 hours exit 0. Drift past 72 hours fails, posts when the webhook is set, and reports the missing secret when it is not. 2.9.10 sorts above 2.9.9.
• The publish guard copies a new tarball and exits 1 when it already exists.
• actionlint passes on chart-checks.yaml. Not run: zizmor.

After merge

• Create a Slack incoming webhook and set it as SLACK_HELM_DRIFT_WEBHOOK_URL in the alerts environment.
• A repo admin makes the Chart checks job a required status check on main with "Require branches to be up to date before merging". The version check compares against the base at the PR's last push, so two PRs can both bump to the same version. With the up-to-date rule, the second PR has to rebase, the check runs again, and it fails until the version is bumped.
• Chart checks fails PR 48 until it moves the ChartVersion default in cloudformation/firewall-eks.yaml and cloudformation/README.md to its chart version.

Out of scope: OCI registry push (CE-119) and moving off the socketdev-demo download URL (CE-238).

---

Note

Low Risk
Changes are limited to GitHub Actions, Helm metadata, and CloudFormation default strings; release behavior is stricter but does not alter runtime firewall code.

Overview
Adds CI gates and monitoring so Helm chart releases stay immutable, aligned with Docker images, and don’t publish missing tags.

PR checks (chart-checks.yaml): any helm/ change must bump Chart.yaml version; non-empty values.yaml image.tag must match appVersion; docker manifest inspect must resolve the image at appVersion.

Release workflow (release.yaml): triggers on all helm/** changes; verifies the image exists before publish; refuses to overwrite an already-published chart tarball on gh-pages.

Drift detection (chart-drift.yaml): daily job compares newest semver Docker Hub tag to the published chart’s appVersion and fails (and Slack-alerts via alerts env) when the image has been newer for more than 72 hours.

Removes the unused helm/.github/workflows/release.yaml. Bumps chart 0.11.8 → 0.11.9 and syncs CloudFormation ChartVersion defaults; Chart.yaml comments document the image.tag / appVersion contract enforced by CI.

^{Reviewed by Cursor Bugbot for commit eb943f7. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issue.}

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit eb943f7. Configure here.}

[Andre Coetzee (Andre153)]

Approving. The three workflows do what the description says. I reproduced the version-bump, tag==appVersion and manifest steps, the already-published guard, and the drift job's jq against live Docker Hub and index.yaml data. actionlint and zizmor are clean on the new files; the two hits in release.yaml predate this PR.

Two things the gates leave open, both inline: the version check is stale once main moves, so same-number bumps from two PRs can both land; and CloudFormation ChartVersion is not covered although the release bot refuses to run when it drifts (#48 trips this today).

Bugbot's ordering finding is wrong; evidence in that thread. An autofix that flips the sign makes the drift job alert on every run.

Also confirmed: every published tarball through 0.11.8 contains .github/workflows/release.yaml (there is no .helmignore), so the 0.11.9 bump is a real content change.