refactor: consolidate Scaleway buckets/IAM, merge AWS state-access roots, rename domains by function - #51
Merged
Conversation
added 7 commits
August 1, 2026 01:32
…et-with-identity modules scaleway-machine-identity: IAM application + a map of policies (each supporting one or more project/org-scoped rule blocks) + a rotating API key - replaces four copy-pasted app/policy/key blocks across the repo with one module, keyed by policy-purpose slug so a call site with more than one scaleway_iam_policy (github-ci's cluster-management + backup-management) can be represented without merging policy objects. scaleway-bucket-with-identity: packages a Scaleway object bucket + its SSE config + one scaleway-machine-identity instance into a single unit, so a domain that owns several buckets can loop over a map instead of hand-writing bucket+identity blocks per bucket. Both include a throwaway examples/basic/ validated via init+validate before being wired into any real root's state.
…e, drop Infisical
github-ci (01-iam/bootstrap/scaleway) now goes through
module "ci_identity" (two policies: cluster_management,
backup_management), replacing the hand-written app/policy/key
resources. Also drops infisical_secret_folder.ci and the two
infisical_secret resources - Infisical is retired repo-wide, the key
is distributed by hand as already documented. 01-iam/bootstrap/infisical
(the Infisical CI trust anchor) is removed entirely, no longer used by
anything.
external-dns moves from 04-dns/scaleway to 01-iam/workload/scaleway -
it owns no DNS zone/record resource, only a workload identity, and
isn't a CI trust anchor, so it doesn't belong under 01-iam/bootstrap/
or as its own numbered domain. Wrapped with
module "identities" { for_each = var.identities } so a future
non-bucket workload identity is a map entry, not new resources.
workload_access_key/workload_secret_key outputs stay pinned to
external-dns specifically since 05-secrets/openbao/managed reads them
by that name; generic access_keys/secret_keys map outputs cover future
identities.
All state moves verified via a clean terraform plan (zero resource
recreation, same live credentials before/after).
…ncryption (KMS)
03-backup/scaleway conflated two unrelated concerns: Scaleway object
buckets + their workload identities, and an AWS KMS key + IAM user for
OpenBao's own auto-unseal. Splits them:
- 03-storage/scaleway: the backup + velero buckets, now provisioned
via a single `module "buckets" { for_each = var.buckets }` block
(modules/scaleway-bucket-with-identity) instead of hand-written
duplicate bucket/identity blocks per bucket - a future tool bucket
is a map entry, not new resources. Root-level output names
(workload_access_key, velero_bucket_name, etc.) unchanged, so
05-secrets/openbao/managed's terraform_remote_state reads need no
update.
- 02-encryption/aws: the AWS KMS key + IAM user, moved via a genuine
cross-backend terraform state mv (new backend key
openbao-unseal/aws) rather than a directory rename. The tfvars
filename is deliberately kept identical
(03-backup-dev-bucket.tfvars) to keep terraform.workspace unchanged
- local.unseal_name derives the live KMS alias + IAM user name from
it, so renaming the workspace would have renamed/recreated them.
All moves verified: 0 resource recreation on the bucket/identity side
(only two description-string updates), and the KMS resources kept
their exact same AWS resource IDs across the cross-backend move.
…ndation/aws 00-remote_state + 01-iam/bootstrap/aws + 01-iam/ci-managed/aws-state-access were three roots for what was fundamentally one concern: the AWS substrate every other root depends on. Merges them into 00-foundation/aws: - The state bucket moves in via a pure directory rename (zero state impact - workspace_key_prefix/tfvars filename untouched). - The GitHub OIDC provider moves in via a real cross-backend state mv (same module/address in both roots, a straight 1:1 move). - One new role, terraform-state-access (via terraform-aws-modules/iam, matching the OIDC-provider module already in use), replaces the old github-actions-terraform + tf-state-access pair. Scoped to exactly S3 list/get/put/delete on the state bucket - no IAM-management capability at all, unlike the system it replaces: the old bootstrap/aws + ci-managed/aws-state-access together built an entire "CI can mint further IAM roles" mechanism (a permissions boundary + a policy letting the CI role create/attach other roles under a managed path) whose only actual consumer was minting the one role that did state R/W. Once that role's job narrows to exactly "read/write this bucket," there's no IAM-management capability left to guard against escalating, so the whole guardrail system goes with it. Both old roles destroyed only after confirming (via a live GitHub Actions dry run) that scaleway.yml successfully assumes the new role under vars.AWS_TERRAFORM_ROLE_ARN. iam_terraform-backend-role.yml deleted - nothing left for it to apply. 01-iam/bootstrap/aws and 01-iam/ci-managed/aws-state-access are empty after this (the former is recreated with different, narrower content in the next commit).
…tion New role openbao-unseal-ci, OIDC-trusted (repo:IntegratedDynamic/ infrastructure:* only), with exactly two policies: full CRUD (create/read/update/destroy) on the KMS key+alias and IAM user+access- key that 02-encryption/aws manages - nothing broader - plus the same state-bucket policy 00-foundation/aws's terraform-state-access role uses, read via a data.terraform_remote_state lookup (not a hardcoded ARN), so this role can also read/write the bucket for its own backend. Closes the gap 02-encryption/aws's own comments used to flag: it required broad IAM/KMS rights the S3-state-only terraform-state-access role doesn't have, so it was admin-applied only. Now there's a second, narrowly-scoped role that can carry that capability without widening terraform-state-access itself - this domain is meant to run through CI/CD like everything else, not stay admin-only. Verified with a clean "No changes" plan after switching the state-bucket policy reference from a hardcoded ARN variable to the terraform_remote_state lookup.
Pure git mv (zero state impact - workspace_key_prefix and tfvars filenames untouched, same decoupling convention used for every other directory move in this refactor). Frees up low numbers for domains that are actually part of the early bootstrap chain (00-foundation, 01-iam, 02-encryption, 03-storage) - 10-cluster sits downstream of all of them.
Adds a spec-style README.md to every remaining domain (00-foundation, 01-iam, 02-encryption, 05-secrets) - what the domain is for, its contract, what deliberately doesn't belong there, independent of which provider implements it today. 02-encryption's is deliberately concrete rather than provider-agnostic: its shape is directly driven by which vendors OpenBao's auto-unseal plugins support, not a generic architectural choice. Updates mise.toml/CLAUDE.md's lock task path lists, CLAUDE.md's Architecture section (tree diagram, dependency spine, per-root docs) to match the new domain layout, terraform-lock.yml's matrix (was pointed at two now-deleted directories - would have failed the next matching PR; also widened to cover roots it never included), and dangling path references left in comments/READMEs after the various moves (05-secrets/openbao/managed's comments and README, the state bucket reference in the root README).
nbrieussel
force-pushed
the
refactor/consolidate-buckets-iam
branch
from
July 31, 2026 23:34
c524050 to
a93be57
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
modules/scaleway-machine-identity(IAM app + map of policies + rotating API key) andmodules/scaleway-bucket-with-identity(bucket + scoped identity, wraps the former), eliminating four copy-pasted IAM blocks and the hand-written backup/velero bucket duplication.terraform state mv— verified with a cleanplanat every step, no resource recreation, same live credentials before/after.01-iam/bootstrap/infisicaland all Infisical usage from01-iam/bootstrap/scaleway(retired, no longer used anywhere in this repo).00-remote_state+01-iam/bootstrap/aws+01-iam/ci-managed/aws-state-accessinto a single00-foundation/awsroot: one CI role (terraform-state-access, viaterraform-aws-modules/iam) scoped to exactly S3 read/write on the state bucket, replacing a much larger "CI can mint further IAM roles" permissions-boundary system that had exactly one consumer. New role applied and verified; old roles destroyed and removed only after confirming the cutover.01-iam/bootstrap/awsback, narrowly scoped this time: a CI role (openbao-unseal-ci) that can only manage the KMS key/alias + IAM user/access-key under/openbao/, plus the shared state-bucket policy (viaterraform_remote_state, not a hardcoded ARN) — closing the gap that used to force02-encryption/awsto be admin-applied only.00-remote_state→00-foundation/aws04-dns/scaleway→01-iam/workload/scaleway(it only ever held a workload identity, no DNS resource)03-backup/scaleway→03-storage/scaleway(buckets) +06-openbao-unseal/aws→02-encryption/aws(KMS/IAM, extracted via a genuine cross-backendstate mv, workspace name deliberately kept identical to avoid renaming/recreating the live KMS alias + IAM user)02-cluster→10-cluster(moved up to free room for the domains above)README.mdto every domain (00-foundationthrough10-cluster): what the domain is for, its contract, what deliberately doesn't belong there — independent of which provider implements it today.Verification
terraform planshowing zero unexpected diff (only intentional description/comment updates) before applying.00-remote_state/aws's bucket, OIDC provider, and new role all confirmed live and functional.terraform_remote_statefrom the dependent root.state mv/state rm/importthis session.10-cluster/scalewayapply (cluster stop + relaunch) twice against this branch. Bootstrap converges cleanly both times — OpenBao, ESO, cert-manager, ArgoCD app-of-apps all reachHealthy. All transient errors observed during bootstrap (Gateway API CRD race, ClusterSecretStore chicken-and-egg, node memory pressure on oneDEV1-Mnode) are expected first-boot ordering/scheduling churn, reproduced identically on both runs, and traced to root cause via pod/controller logs — none attributable to this refactor.Test plan
terraform validate/planclean on every touched rootAWS_TERRAFORM_ROLE_ARNis the only AWS role variable GitHub Actions needs going forward (oldAWS_GITHUB_ACTIONS_ROLE_ARN/AWS_TF_STATE_ROLE_ARNcan be removed from repo variables once merged)