Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -37,3 +37,5 @@ override.tf.json
terraform.rc

.specify

*.conf
25 changes: 25 additions & 0 deletions 04-vpn/wireguard/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

120 changes: 120 additions & 0 deletions 04-vpn/wireguard/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
# 04-vpn/wireguard — WireGuard peer keys + client configs for the OpenBao tunnel

## Connect

Requires `wireguard-tools` (`brew install wireguard-tools` on macOS) — the
official WireGuard.app GUI has known DNS resolution issues with its
sandboxed NetworkExtension that `wg-quick` doesn't have; use the CLI.

```bash
mise run vpn-generate # once, and again any time your config needs refreshing
mise run vpn-up # your own tunnel — add a peer name if you're not "nicolas"
mise run vpn-down # when you're done
```

`vpn-generate` writes a ready-to-use `wg-quick` config per peer to
`generated/` (gitignored, `0600`) — `vpn-up`/`vpn-down` just point
`wg-quick` at the right one, since its own "bare name" lookup only finds
configs already living in its own config directory, not ones Terraform
generates here.

Adding a new peer: add an entry to `var.peers` (both `variables.tf`'s
default and `env/04-network-wireguard.tfvars`), `mise run vpn-generate`,
then do the manual hand-offs below for just that new entry.

## Manual hand-offs — hardcoded cross-repo coupling to track

Nothing here talks to the gitops repo or to OpenBao directly. Every row is
a human copying a `terraform output` value somewhere else — same "two
systems kept in sync by convention, not automation" pattern as
`05-secrets/openbao/managed`'s `secrets_sync_github` vs. the gitops repo's
`apps/secrets-sync/values.yaml`. Accepted for now; each row is a spot that
silently breaks if only one side changes.

| Value | From | To | Why manual |
|---|---|---|---|
| `server_public_key`, `peer_public_keys` (not secret) | this root's outputs | gitops `services/platform/wireguard/config/values-scaleway.yaml` | no cross-repo automation exists anywhere in this setup |
| `server_private_key` (sensitive) | this root's output | `05-secrets/openbao/managed`'s `wireguard_server_private_key` (`local.auto.tfvars`) → `kv/apps/wireguard/server-key` → gitops `wireguard-init`'s ExternalSecret | same |
| `peer_private_keys["ci-github-actions"]` (sensitive) | this root's output | `05-secrets/openbao/managed`'s `wireguard_ci_private_key` → merged into the *existing* `kv/apps/secrets-sync/github/infrastructure-scaleway` object (not its own KV path — nothing reads that) → secrets-sync → GitHub Actions secret | same; direction is OpenBao→GitHub via in-cluster ESO, so CI never needs the tunnel to fetch its own tunnel key |
| every other peer's private key | this root's output | that peer's own machine only (`generated/<name>.conf`) | never touches OpenBao or git, by design |
| NodePort `30820` | gitops `services/platform/wireguard/config/values.yaml` `server.nodePort` | infra `10-cluster/scaleway/main.tf`'s security group rule, **and** this root's `wg_endpoint` port | two independent files, must match exactly |

## Why this exists

`05-secrets/openbao/{bootstrap,managed}`'s `vault` provider needs a network
path to OpenBao's API — this root exists so that path can be a self-hosted
tunnel instead of the public gateway route (`openbao.scalepack.fr`), which
stays public only for a separate, legitimate reason: human OIDC/UI login
(gitops repo's `services/platform/openbao/config`), untouched by this
domain. No third-party control plane (Tailscale evaluated, ruled out) —
just `wg`, terminated by a small server workload in the gitops repo
(`services/platform/wireguard`).

**End-to-end validated live 2026-08-09**: a peer handshakes and reaches
OpenBao's `/v1/sys/health` through the full chain (tunnel → gitops's
`proxy-openbao` sidecar → OpenBao's Service).

## Why exposure is a NodePort, not the shared Gateway

Scaleway's cloud-controller-manager silently drops any non-TCP `Service`
port when building its LoadBalancer (`loadbalancers.go`:
`if port.Protocol != v1.ProtocolTCP { skip }`) — a `UDPRoute` on the shared
Gateway's LB was never reachable no matter how correctly Envoy Gateway
itself was configured; confirmed live, zero datagrams ever arrived despite
every Gateway API resource reporting `Accepted`/`Programmed`.

Exposed instead as a `NodePort` directly on a Kapsule node's public IP — a
genuinely new entry point, hardened accordingly: `externalTrafficPolicy:
Local` + an explicit least-privilege `NetworkPolicy` (gitops repo's
`services/platform/wireguard/config`), plus a dedicated
`scaleway_instance_security_group` replacing Scaleway's auto-managed
default one, which ships with zero inbound rules and would otherwise block
this (and everything else) at the instance level regardless of any
Kubernetes-side config (`10-cluster/scaleway/main.tf`).

A raw node IP isn't stable — it changes on every reschedule, including
this cluster's own daily destroy/rebuild. Fixed via `external-dns`, not a
custom script: its `service` source has first-class support for exactly
`NodePort` + `externalTrafficPolicy: Local` (resolves the ExternalIP of
whichever node currently has a live pod), so `wg.scalepack.fr` — the
`wg_endpoint` default — stays correct on its own, the same mechanism every
other `*.scalepack.fr` hostname in this cluster already relies on. See
that chart's `templates/service.yaml` annotation.

## Why the tunnel doesn't route to OpenBao's ClusterIP directly

An earlier design routed/NAT'd traffic to OpenBao's `ClusterIP` at the
kernel level (`net.ipv4.ip_forward` + `iptables` MASQUERADE) — a dead end:
Kapsule's kubelet doesn't allowlist that sysctl, and Scaleway's
`kubelet_args` API refuses to widen the allowlist for this cluster's k8s
version at all. Even where available, that's a cluster-wide relaxation for
one workload's benefit — worth avoiding regardless.

The gitops chart instead terminates the tunnel and proxies to OpenBao at
the application layer (a `socat` sidecar resolving
`openbao.openbao.svc.cluster.local` via ordinary in-cluster DNS) — no
kernel routing, no sysctls, no dependency on OpenBao's `ClusterIP` being
stable across a cluster rebuild.

## Why its own domain, and why it's temporary

Identities normally live in `01-iam/`, secrets in `05-secrets/`. Neither
fits a WireGuard peer well: it isn't a cloud-provider IAM identity, and a
keypair's public half isn't a secret at all. Placeholder domain of its
own, explicitly temporary — expect this to fold into `01-iam/workload/` (or
be replaced entirely) once the shape of "what kind of thing is a WireGuard
peer" is clearer.

## What it creates

- `wireguard_asymmetric_key.server` — the tunnel server's own keypair.
- `wireguard_asymmetric_key.peer` (`for_each = var.peers`) — one keypair
per peer.
- `data.wireguard_config_document.peer` — renders each peer's full
`wg-quick` config (their own key, the server's public key, `AllowedIPs`,
`wg_endpoint`).
- `local_sensitive_file.peer_conf` — writes that config to `generated/`.

Providers: [`OJFord/wireguard`](https://registry.terraform.io/providers/OJFord/wireguard)
(local key generation + config rendering, no credentials, no API calls) and
`hashicorp/local` (writes the file).
8 changes: 8 additions & 0 deletions 04-vpn/wireguard/env/04-network-wireguard.tfvars
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# Matches variables.tf's defaults — spelled out explicitly rather than left
# implicit, same convention as every other root's env/ file.
peers = {
"nicolas" = { address = "10.100.0.2/32" }
"ci-github-actions" = { address = "10.100.0.3/32" }
}
server_address = "10.100.0.1/24"
wg_endpoint = "wg.scalepack.fr:30820"
42 changes: 42 additions & 0 deletions 04-vpn/wireguard/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# The WireGuard server's own keypair — one per cluster, not per peer. Private
# key hands off to 05-secrets/openbao/managed (kv/apps/wireguard/server-key),
# which ESO then materializes into the gitops repo's
# services/platform/wireguard Deployment. Public key is non-secret — copy it
# into that same chart's values (it's the identity peers dial, not a secret).
resource "wireguard_asymmetric_key" "server" {}

# One keypair per peer in var.peers. Public keys are non-secret — routed
# through 05-secrets/openbao/managed into the gitops repo's peer allowlist.
# Private keys: ci-github-actions hands off to 05-secrets/openbao/managed
# the same way the server key does (see README.md); every other entry is a
# human peer's own credential — see outputs.tf's peer_confs for why that
# never gets written to disk by this root itself.
resource "wireguard_asymmetric_key" "peer" {
for_each = var.peers
}

# Renders a ready-to-import wg-quick config per peer — the whole point of
# minting keys in Terraform instead of by hand with `wg genkey`.
# ci-github-actions gets one too for consistency, but nothing consumes its
# file: CI only ever needs the raw private key (see
# 05-secrets/openbao/managed's wireguard_ci_private_key), never a config file.
data "wireguard_config_document" "peer" {
for_each = var.peers

private_key = wireguard_asymmetric_key.peer[each.key].private_key
addresses = [each.value.address]
# Split DNS: the tunnel server's own `dns` sidecar (gitops repo's
# services/platform/wireguard/config) answers *.scalepack.fr with its own
# tunnel address while forwarding everything else upstream — so every
# OIDC-gated app (Grafana, ArgoCD, ...) works over the tunnel using its
# real hostname, scoped to exactly this interface's lifetime (nothing to
# revert when the tunnel goes down, unlike an /etc/hosts edit).
dns = [split("/", var.server_address)[0]]

peer {
public_key = wireguard_asymmetric_key.server.public_key
allowed_ips = ["${split("/", var.server_address)[0]}/32"]
endpoint = var.wg_endpoint
persistent_keepalive = 25
}
}
39 changes: 39 additions & 0 deletions 04-vpn/wireguard/outputs.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
output "server_public_key" {
description = "WireGuard server public key — non-secret, copy into gitops services/platform/wireguard's values."
value = wireguard_asymmetric_key.server.public_key
}

output "server_private_key" {
description = "WireGuard server private key — sensitive. Read directly by 05-secrets/openbao/managed via terraform_remote_state (kv/apps/wireguard/server-key). Never committed here."
value = wireguard_asymmetric_key.server.private_key
sensitive = true
}

output "peer_public_keys" {
description = "peer name -> public key. Non-secret. Read (with peer_addresses) by 05-secrets/openbao/managed via terraform_remote_state into kv/apps/wireguard/peers, which the gitops repo's services/platform/wireguard/init syncs down — nothing hardcodes these."
value = { for name, key in wireguard_asymmetric_key.peer : name => key.public_key }
}

output "peer_addresses" {
description = "peer name -> overlay tunnel address (var.peers[name].address, echoed back as its own output for the same reason as peer_public_keys above)."
value = { for name, cfg in var.peers : name => cfg.address }
}

output "peer_private_keys" {
description = "peer name -> private key. Sensitive. ci-github-actions is read directly by 05-secrets/openbao/managed via terraform_remote_state; every other entry stays local — see peer_confs — never committed, never round-tripped through OpenBao."
value = { for name, key in wireguard_asymmetric_key.peer : name => key.private_key }
sensitive = true
}

# peer name -> full rendered wg-quick config. Deliberately NOT written to
# disk by this root (no local_sensitive_file/for_each here) — that would
# mean anyone running `terraform apply` on this root ends up with every
# peer's private key in plaintext on their own machine, not just their
# own. Extracting and writing just one peer's file is `mise run
# vpn-generate`'s job (mise.toml), scoped to a single name via
# `terraform output -json peer_confs | jq -r '.["<name>"]'`.
output "peer_confs" {
description = "peer name -> full rendered wg-quick config (sensitive). Never written to disk in bulk — see mise.toml's vpn-generate task for how one peer's own file gets extracted."
value = { for name, doc in data.wireguard_config_document.peer : name => doc.conf }
sensitive = true
}
41 changes: 41 additions & 0 deletions 04-vpn/wireguard/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# One keypair per identity that needs to reach OpenBao's ClusterIP over the
# tunnel instead of its public gateway route — a human dev machine or the CI
# service identity. `address` is that peer's own /32 on the tunnel's overlay
# subnet (var.server_address) — this map is the single source of truth for
# peer overlay addressing; the gitops repo's services/platform/wireguard
# chart copies it in as a hand-synced value (see README.md's hand-off table).
variable "peers" {
description = "Peer name -> its own overlay tunnel address. One keypair minted per entry."
type = map(object({
address = string
}))
default = {
"nicolas" = { address = "10.100.0.2/32" }
"ci-github-actions" = { address = "10.100.0.3/32" }
}
}

# The tunnel server's own overlay address (CIDR) — must match the gitops
# repo's services/platform/wireguard/config values.yaml `server.address`.
variable "server_address" {
description = "WireGuard server's own overlay address (CIDR, e.g. 10.100.0.1/24)."
type = string
default = "10.100.0.1/24"
}

# Where a peer actually dials to reach the tunnel server. A hostname, not a
# raw node IP: Scaleway LBs don't do UDP passthrough (see README.md), so
# this has to be a Kapsule node's public IP directly — which isn't stable
# on its own (changes whenever the server pod reschedules, including this
# cluster's own daily destroy/rebuild). external-dns keeps wg.scalepack.fr
# pointed at whichever node currently hosts the pod (gitops repo's
# services/platform/wireguard/config/templates/service.yaml's
# external-dns.alpha.kubernetes.io/hostname annotation — same mechanism
# every other *.scalepack.fr hostname already uses, no custom script).
# Port must match that chart's values.yaml `server.nodePort` exactly (same
# coupling noted in this infra repo's 10-cluster/scaleway/main.tf).
variable "wg_endpoint" {
description = "Where peers dial the tunnel server: <hostname>:<NodePort>."
type = string
default = "wg.scalepack.fr:30820"
}
20 changes: 20 additions & 0 deletions 04-vpn/wireguard/version.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
terraform {
backend "s3" {
bucket = "id-terraform-state20260612164136440800000001"
region = "eu-west-3"
workspace_key_prefix = "network/wireguard"
key = "terraform.tfstate"
encrypt = true
use_lockfile = true
}

required_providers {
wireguard = {
source = "OJFord/wireguard"
version = "~> 0.4"
}
}
}

# Pure local keypair generation — no API, no credentials to configure.
provider "wireguard" {}
17 changes: 10 additions & 7 deletions 05-secrets/openbao/bootstrap/version.tf
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
terraform {
backend "s3" {
bucket = "id-terraform-state20260612164136440800000001"
region = "eu-west-3"
workspace_key_prefix = "secrets/bootstrap/openbao"
key = "terraform.tfstate"
encrypt = true
use_lockfile = true
bucket = "id-terraform-state20260612164136440800000001"
region = "eu-west-3"
workspace_key_prefix = "secrets/bootstrap/openbao"
key = "terraform.tfstate"
encrypt = true
use_lockfile = true
}

required_providers {
Expand All @@ -21,7 +21,10 @@ terraform {
# session doesn't populate the env var this hashicorp/vault provider expects.
# Only the token stays in the environment — see README.
provider "vault" {
# Same hostname as the public route, resolved through the WireGuard
# tunnel via split-DNS while it's up — see
# 05-secrets/openbao/managed/version.tf's comment for the full rationale.
address = "https://openbao.scalepack.fr/"
# address = "http://127.0.0.1:8200/"
# address = "http://127.0.0.1:8200/" # kubectl port-forward, independent of the tunnel
# token = var.root_token
}
25 changes: 25 additions & 0 deletions 05-secrets/openbao/managed/.terraform.lock.hcl

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading