Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions services/platform/gateway/config/values-scaleway.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ acmeEmail: nicolas.brieussel@gmail.com
# namespace (same Scaleway DNS credentials external-dns already uses).
scalewayCredentialsSecretName: scaleway-dns-credentials


# PreSync hook (templates/cert-restore-*.yaml): restores the ACME account key
# + wildcardCertSecretName above from the latest Velero backup BEFORE this
# chart's own ClusterIssuers/Gateway get created on a fresh cluster — see
Expand Down
16 changes: 14 additions & 2 deletions services/platform/secrets-sync/config/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -72,15 +72,27 @@ targets:
secrets:
- SCW_ACCESS_KEY
- SCW_SECRET_KEY
- WG_CI_PRIVATE_KEY
# This same round-trip (OpenBao -> ESO -> GitHub env secret) is also
# meant to close the CI bootstrap loop for the infra repo's
# 05-secrets/<service>/managed roots: each root's AppRole role_id/
# secret_id should get merged into its KV object and pushed through
# here too, so CI can run `terraform apply` on those roots without a
# human re-supplying credentials each time. Not yet wired for
# 05-secrets/openbao/managed specifically (only SCW_ACCESS_KEY/
# SCW_SECRET_KEY are merged today) — see infra repo's
# 05-secrets/openbao/README.md for the full pattern.
# SCW_SECRET_KEY/WG_CI_PRIVATE_KEY are merged today) — see infra
# repo's 05-secrets/openbao/README.md for the full pattern.
#
# WG_CI_PRIVATE_KEY: CI's own WireGuard peer key (infra repo's
# 04-vpn/wireguard's peer_private_keys["ci-github-actions"]
# output) — brings up the tunnel to OpenBao's ClusterIP before
# `terraform plan/apply` on 05-secrets/openbao/{bootstrap,managed}.
# Merged into THIS same kv/apps/secrets-sync/github/infrastructure-scaleway
# object (not a separate kv/apps/wireguard/* one — nothing but this
# target's own pipeline would ever read that): supplied straight into
# 05-secrets/openbao/managed's local.auto.tfvars from
# 04-vpn/wireguard's output, same as every other
# externally-issued credential on that root.

# Visibilité des secrets d'organisation créés : "all" ou "private".
orgSecretVisibility: private
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Wave 5: needs wireguard-init's wave-3 Secret (the tunnel server's private
# key) to exist first. No Gateway dependency (NodePort, not routed through
# it — see templates/service.yaml) — wave 5 kept anyway for a comfortable
# margin past wireguard-init, not because it's required here.
name: wireguard-config
namespace: wireguard
chartPath: services/platform/wireguard/config
valueFile: values-scaleway.yaml
syncWave: "5"
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
name: wireguard-init
namespace: wireguard
chartPath: services/platform/wireguard/init
valueFile: values-scaleway.yaml
syncWave: "3"
5 changes: 5 additions & 0 deletions services/platform/wireguard/config/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: v2
name: wireguard-config
description: Self-hosted WireGuard tunnel server, fronted by the shared Envoy Gateway, giving Terraform's vault provider a non-public path to OpenBao's ClusterIP
type: application
version: 0.1.0
27 changes: 27 additions & 0 deletions services/platform/wireguard/config/templates/configmap.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Non-secret half of wg0.conf ([Interface] only). The private key comes
# from the Secret services/platform/wireguard/init materializes; [Peer]
# blocks come from that same init chart's wireguard-peers Secret — both
# combined by an initContainer at pod start (see deployment.yaml), never
# by Helm/ArgoCD, since ArgoCD's rendering has no access to a Secret's
# live content.
apiVersion: v1
kind: ConfigMap
metadata:
name: wireguard-conf-template
namespace: {{ .Values.namespace }}
data:
wg0.conf.tmpl: |
[Interface]
Address = {{ .Values.server.address }}
ListenPort = {{ .Values.server.port }}
PrivateKey = __PRIVATE_KEY__
# No PostUp/PostDown NAT+forwarding rules on purpose (see deployment.yaml's
# proxy-gateway container) — this pod isn't a router for the wider cluster, it terminates the
# tunnel and re-originates plain outbound connections via userspace proxies.
# That needs no net.ipv4.ip_forward/src_valid_mark: tried the kernel-routing
# approach first, but Kapsule's kubelet doesn't allowlist either sysctl and
# Scaleway's own kubelet_args API rejects widening that allowlist for this
# k8s version anyway — and even where it's technically available, an
# unsafe-sysctls allowlist is a cluster-wide relaxation for one workload's
# benefit, worth avoiding on its own merits, not just because it was
# blocked here.
158 changes: 158 additions & 0 deletions services/platform/wireguard/config/templates/deployment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,158 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: wireguard
namespace: {{ .Values.namespace }}
spec:
replicas: 1
strategy:
# A second replica during a rollout would mean two pods both claiming
# the same peer set with independent state — recreate rather than
# risk that, a short tunnel blip during a rollout is fine.
type: Recreate
selector:
matchLabels:
app.kubernetes.io/name: wireguard
template:
metadata:
labels:
app.kubernetes.io/name: wireguard
spec:
serviceAccountName: wireguard
initContainers:
# Combines the non-secret template (configmap.yaml) with the private
# key from services/platform/wireguard/init's Secret into the actual
# wg0.conf the server container reads — this has to happen at pod
# start, not at Helm render time, since ArgoCD never sees the
# Secret's live content.
- name: render-wg-conf
image: alpine:3.22
command: ["sh", "-c"]
args:
- |
set -eu
apk add --no-cache jq
PRIVATE_KEY=$(cat /secrets/{{ .Values.server.privateKeySecretKey }})
sed "s|__PRIVATE_KEY__|$PRIVATE_KEY|" /template/wg0.conf.tmpl > /config/wg_confs/wg0.conf
# One file per peer (the wireguard-peers Secret mounted below),
# each containing that peer's {publicKey, allowedIPs} as JSON —
# see infra repo's 05-secrets/openbao/managed/main.tf for how
# that Secret's content is shaped.
for f in /peers/*; do
name=$(basename "$f")
pubkey=$(jq -r '.publicKey' "$f")
allowedips=$(jq -r '.allowedIPs' "$f")
printf "\n# %s\n[Peer]\nPublicKey = %s\nAllowedIPs = %s\n" "$name" "$pubkey" "$allowedips" >> /config/wg_confs/wg0.conf
done
chmod 600 /config/wg_confs/wg0.conf
volumeMounts:
- name: conf-template
mountPath: /template
readOnly: true
- name: server-key
mountPath: /secrets
readOnly: true
- name: peers
mountPath: /peers
readOnly: true
- name: wg-confs
mountPath: /config/wg_confs
containers:
- name: wireguard
image: "{{ .Values.server.image.repository }}:{{ .Values.server.image.tag }}"
securityContext:
capabilities:
add: ["NET_ADMIN"]
env:
- name: PUID
value: "1000"
- name: PGID
value: "1000"
# Custom wg_confs/wg0.conf is supplied directly (mounted below)
# — this disables the image's own key/peer auto-generation,
# since Terraform (infra repo's 04-vpn/wireguard) already
# owns that.
ports:
- name: wireguard
containerPort: {{ .Values.server.port }}
protocol: UDP
volumeMounts:
- name: wg-confs
mountPath: /config/wg_confs
# One sidecar per proxyTargets entry — each terminates the tunnel
# connection and re-originates a plain outbound TCP connection to
# its own target. Userspace proxying, not kernel routing, so no
# ip_forward/src_valid_mark, no iptables NAT, no capabilities
# beyond the wireguard container's own NET_ADMIN above (needed to
# create wg0 at all, unrelated to this). Each listens on every
# interface in this pod's netns, including wg0 once the tunnel is
# up — a peer with the right AllowedIPs dials this pod's tunnel
# address on that target's port directly.
{{- range $name, $proxyTarget := .Values.proxyTargets }}
- name: proxy-{{ $name }}
image: alpine:3.22
command: ["sh", "-c"]
args:
- |
set -eu
apk add --no-cache socat
exec socat TCP-LISTEN:{{ $proxyTarget.port }},fork,reuseaddr TCP:{{ $proxyTarget.target }}
{{- end }}
# Raw TCP passthrough to Envoy Gateway's own real Service — SNI/Host
# travel untouched inside the TLS ClientHello, so Envoy does its
# normal SNI-based routing to whichever app (Grafana, ArgoCD, Dex,
# OpenBao's UI, ...) exactly as it does for public traffic. No
# per-app sidecar, no TLS termination here, and it covers every
# app behind the Gateway automatically — this is what makes OIDC
# logins (state cookie tied to the app's one real origin) work
# over the tunnel: from every app's point of view this traffic IS
# indistinguishable from the public path, just arriving over a
# different network. Service name is resolved at startup (RBAC in
# rbac.yaml) rather than hardcoded — Envoy Gateway generates it
# per-Gateway-object and it isn't guaranteed stable across a
# recreate.
- name: proxy-gateway
image: alpine/kubectl:1.35.3
command: ["sh", "-c"]
args:
- |
set -eu
apk add --no-cache socat
SVC=$(kubectl get svc -n envoy-gateway-system -l gateway.envoyproxy.io/owning-gateway-name=scaleway-gateway -o jsonpath='{.items[0].metadata.name}')
TARGET="${SVC}.envoy-gateway-system.svc.cluster.local"
echo "proxying to Envoy Gateway Service: ${TARGET}"
socat TCP-LISTEN:443,fork,reuseaddr TCP:${TARGET}:443 &
socat TCP-LISTEN:80,fork,reuseaddr TCP:${TARGET}:80 &
wait
# Split DNS: answers *.scalepack.fr with this pod's own tunnel
# address (server.address, stripped of its CIDR suffix) and
# forwards everything else upstream. Peers get this pushed via
# their wg-quick config's DNS setting (infra repo's
# 04-vpn/wireguard/main.tf) — while the tunnel is up, every
# *.scalepack.fr hostname resolves to the tunnel automatically, no
# /etc/hosts editing, and it reverts the moment the tunnel goes
# down (scoped to the interface's own DNS config, not a standing
# file edit).
- name: dns
image: alpine:3.22
command: ["sh", "-c"]
args:
- |
set -eu
apk add --no-cache dnsmasq
exec dnsmasq --no-daemon --no-resolv \
--address=/{{ .Values.dnsZone }}/{{ (splitList "/" .Values.server.address) | first }} \
--server=1.1.1.1 --server=8.8.8.8 \
--port=53
volumes:
- name: conf-template
configMap:
name: wireguard-conf-template
- name: server-key
secret:
secretName: {{ .Values.server.privateKeySecretName }}
- name: peers
secret:
secretName: {{ .Values.server.peersSecretName }}
- name: wg-confs
emptyDir: {}
24 changes: 24 additions & 0 deletions services/platform/wireguard/config/templates/networkpolicy.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Explicit least-privilege ingress for the one pod in this cluster reachable
# from the raw internet via a node's public IP (see service.yaml) — a
# default Kubernetes stance of "whatever the pod spec exposes is reachable"
# isn't good enough for a pod sitting on that entry point. This is the
# enforced contract: exactly UDP/{{ .Values.server.port }}, from anywhere (peers roam, can't be
# source-IP-scoped), nothing else — so a future change that adds another
# container port to this pod doesn't silently become internet-reachable.
# Enforced by Cilium (this cluster's CNI), which programs NodePort-DNAT'd
# traffic through the same policy engine as any other ingress.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: wireguard
namespace: {{ .Values.namespace }}
spec:
podSelector:
matchLabels:
app.kubernetes.io/name: wireguard
policyTypes:
- Ingress
ingress:
- ports:
- protocol: UDP
port: {{ .Values.server.port }}
38 changes: 38 additions & 0 deletions services/platform/wireguard/config/templates/rbac.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Dedicated ServiceAccount (not "default") so the RBAC grant below is scoped
# to exactly this pod, nothing else in the wireguard namespace.
apiVersion: v1
kind: ServiceAccount
metadata:
name: wireguard
namespace: {{ .Values.namespace }}
---
# get/list on Services in envoy-gateway-system only — lets the
# proxy-gateway sidecar (deployment.yaml) resolve Envoy Gateway's real
# Service name at startup instead of hardcoding it. That name is
# auto-generated per Gateway object (envoy-gateway-scaleway-gateway-<hash>)
# and not guaranteed stable across a Gateway recreate — the same class of
# problem wg_endpoint had, solved the same way: discover it live instead of
# tracking one more hardcoded reference.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: wireguard-read-gateway-svc
namespace: envoy-gateway-system
rules:
- apiGroups: [""]
resources: ["services"]
verbs: ["get", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: wireguard-read-gateway-svc
namespace: envoy-gateway-system
subjects:
- kind: ServiceAccount
name: wireguard
namespace: {{ .Values.namespace }}
roleRef:
kind: Role
name: wireguard-read-gateway-svc
apiGroup: rbac.authorization.k8s.io
46 changes: 46 additions & 0 deletions services/platform/wireguard/config/templates/service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# NodePort, not routed through the shared Gateway — confirmed live 2026-08-09
# that Scaleway's cloud-controller-manager silently drops any non-TCP
# Service port when building the LB (scaleway-cloud-controller-manager's
# loadbalancers.go: `if port.Protocol != v1.ProtocolTCP { skip }`), so a
# UDPRoute on the Gateway's LB was never reachable — zero datagrams ever
# arrived at Envoy despite every Gateway API resource reporting Accepted/
# Programmed. Kapsule nodes already carry individual public IPs, so this
# exposes the tunnel directly on those instead — a genuinely new entry
# point, hardened accordingly:
# - externalTrafficPolicy: Local — only node(s) actually running this
# pod accept traffic on this port at all; every other node in the pool
# stays fully closed on it. Also preserves the real client source IP
# (no extra SNAT hop) and avoids Kubernetes' cross-node NodePort
# redirect entirely.
# - templates/networkpolicy.yaml is the explicit least-privilege
# complement: ingress to this pod is enforced at exactly UDP/{{ .Values.server.port }},
# nothing else, regardless of what the pod spec happens to expose.
# - WireGuard's own protocol does the rest: an unauthenticated packet is
# silently dropped, no response, no banner — a port scan can't tell
# this apart from a closed port.
apiVersion: v1
kind: Service
metadata:
name: {{ .Values.server.serviceName }}
namespace: {{ .Values.namespace }}
annotations:
# external-dns's `service` source has first-class NodePort support: for
# externalTrafficPolicy: Local (above), it resolves the ExternalIP of
# whichever node(s) currently have a live pod backing this Service
# (source/service.go's nodesExternalTrafficPolicyTypeLocal) and keeps
# this record pointed there — no custom script, this is exactly the
# same external-dns already running for every other *.scalepack.fr
# hostname (services/platform/external-dns). Peers dial this hostname,
# never a raw node IP that changes on every reschedule/cluster rebuild.
external-dns.alpha.kubernetes.io/hostname: {{ .Values.server.hostname }}
spec:
type: NodePort
externalTrafficPolicy: Local
selector:
app.kubernetes.io/name: wireguard
ports:
- name: wireguard
port: {{ .Values.server.port }}
targetPort: {{ .Values.server.port }}
nodePort: {{ .Values.server.nodePort }}
protocol: UDP
6 changes: 6 additions & 0 deletions services/platform/wireguard/config/values-scaleway.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# No overrides needed — the peer list used to be hardcoded here, replaced
# by services/platform/wireguard/init's wireguard-peers Secret (synced from
# OpenBao, itself sourced from the infra repo's 04-vpn/wireguard state).
# Kept as an empty file so this chart follows the same
# applications/scaleway -> values-scaleway.yaml convention as every other
# chart in this repo, in case a real override shows up later.
Loading
Loading