Skip to content

fix(client, caddytls): port #75 client-side fixes onto v0.7.0 - #84

Closed
sosyz wants to merge 6 commits into
mainfrom
fix/client-caddytls
Closed

sosyz wants to merge 6 commits into
mainfrom
fix/client-caddytls

Conversation

@sosyz

@sosyz sosyz commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Ports the client-side fixes from #75 onto the v0.7.0 layout (update actions, [[Certificate]], pkg/client/updateactions/file), with the maintainer's review applied. Scope: pkg/client (not the tencentcloud/kubernetes actions), pkg/config/client*.go, pkg/logging, and exec/caddytls.

  • logging: the logger and debug flag are atomic. This fixes the data race when the Caddy plugin swaps the logger on config reload. SetLogger(nil) is ignored.
  • client constructors: MakeCertDXHttpClient and its options now return an error instead of calling logging.Fatal on a bad mTLS bundle. The gRPC client loads its credentials on the first Stream, so a load failure becomes a stream error that gets retried. The client no longer calls os.Exit, so a bad bundle cannot kill the Caddy process.
  • HTTP poller:
    • One HTTP client (and transport) per server is reused across polls. Before, every attempt built a new transport. Idle connections expire after 90s.
    • The cached client is rebuilt when its mTLS bundle changes on disk: the bundle is hashed on every attempt. So a renewed or replaced client certificate still takes effect on the next poll round, as it did when every round rebuilt the client.
    • The poll interval is still RenewTimeLeft/4. It is clamped only in bad cases: <= 0 becomes 1 minute, and a sub-second value is floored at 1s (never beyond RenewTimeLeft itself).
    • When no server can be reached, the poller retries on a 15s→60s backoff. A well-formed server error (resp.Err) keeps the normal interval, so a permanent refusal is not hammered.
  • SDS: Stream refuses to start when two watched certs share a name (the dispatch key). A duplicate would starve one cert of updates.
  • file action:
    • The temp cert/key are fsynced before the rename, and the directories after it. Both are best effort, so mounts that answer fsync with ENOSYS/EINVAL still get the certificate. The file action keeps its own writer and does not depend on a pkg/utils helper.
    • reloadCommand runs under the action's context with a 5 minute timeout.
  • client config validation (adapted to [[Certificate]] + update actions):
    • Duplicate domain sets are rejected.
    • Duplicate names are rejected in gRPC mode.
    • Two file actions that would write the same <savePath>/<name>.pem are rejected, whether they are in different certificates or the same one.
  • caddytls:
    • authMethod is validated in the Caddyfile adapter.
    • Provision checks the transport config, including that the mTLS/gRPC bundle exists.
    • Native-JSON configs get the adapter's string defaults.
    • A token server with no token logs an INSECURE warning.
    • The keypair is parsed once per renewal instead of on every handshake, and the real error is returned, tagged with the cert id.
    • Stop has a nil guard.
  • docs: docs/client.md covers the reload timeout, fsync, poll/backoff behaviour and the new validation errors. docs/caddytls.md covers the authMethod/pem load-time checks.

Review feedback from #75 addressed

  • handler.go reload timeout, "Give it 5 minutes for now": reloadCommandTimeout = 5 * time.Minute, now in pkg/client/updateactions/file/file.go. docs/client.md says 5 minutes.
  • caddytls.go SNI check, "Is it a configuration error … or is it the case that attacker can reach?": it is a configuration error, so the check and certPackCovers are removed. Analysis (Caddy v2.11.4 / certmagic v0.25.4):
    • Caddy picks the certmagic config for a handshake with TLS.getAutomationPolicyForName(SNI). That is the first automation policy whose subjects match the SNI (wildcards included), or a policy with no subjects. get_certificate certdx <id> in a site's tls {} block becomes a manager on that site's policy, whose subjects are the site's hostnames. So the manager is only asked about names the user routed to it.
    • An attacker controls the SNI, so they can reach the manager with an unrelated name in two cases: a catch-all policy (a site with no hostname, e.g. :443 { tls { get_certificate certdx x } }), or a wildcard site. Either way they get our certificate back. It is public anyway (CT logs), no key material leaves, and their TLS client rejects it because it does not cover the name. That is no different from serving a default certificate. With a precise site address, the check could never fail.
    • The one behavioural difference is several managers in one policy (get_certificate repeated in one tls {} block): certmagic uses the first manager that returns a certificate, so without the check the first cert pack always wins. That is also a configuration error; the fix is one site block per cert pack.
    • certmagic does not cache manager-provided certificates, so the check would run on every handshake. Given the above, it is not worth that cost.
  • certdx.go sharedCerts usage pool, "Just start from empty cache": removed. There is no usage pool and no cross-reload state. Each CertDXCaddyDaemon builds an empty per-instance map in Provision, and GetCertificate returns no certificate material available yet until the first fetch lands. The docs/caddytls.md paragraph about keeping certificates across reloads is not added.

Breaking changes

These configs loaded on v0.7.0 and now fail validation. In each case the config was already broken at runtime.

certdx_client TOML

  • Two [[Certificate]] entries with the same domain set fail with certificate <b> duplicates the domain set of certificate <a>. Domains are compared case-insensitively, in any order, ignoring a trailing dot. Before, the second entry silently replaced the first in the daemon's watch map, and the first one's update actions never ran. Fix: merge the entries into one [[Certificate]] with all of their update actions.
  • In mode = "grpc", two certificates with the same name fail with duplicate certificate name: <name>. Before, they collided in the SDS dispatch map. HTTP mode still allows a repeated name.
  • Two type = "file" actions that resolve to the same <savePath>/<name>.pem (same savePath and certificate name) fail with certificate <a>: file update action writes <path>, which certificate <b> also writes. This applies whether they are in different certificates or the same one.

Caddy plugin

  • A Caddyfile authMethod other than token/mtls (e.g. mTLS, none) now fails caddy adapt. Before, it was silently accepted and no auth was sent.

  • At config load (Provision), the plugin now rejects:

    • an HTTP server with an unknown authMethod;
    • an mTLS or gRPC pem that does not exist (before, the process exited at the first request);
    • a missing main server;
    • an unknown mode.

    All of these used to fail later, at Start or at request time.

  • A native-JSON config (not written through the Caddyfile adapter) that leaves mode, reconnect_interval or authMethod empty now gets http / 10m / token. Before, the first two were load errors, and an empty authMethod sent no credentials. retry_count is not defaulted, because 0 (a single attempt) is valid.

Library API (pkg/client)

  • CertDXHttpClientOption is now func(*CertDXHttpClient) error.
  • MakeCertDXHttpClient now returns (*CertDXHttpClient, error).
  • logging.SetLogger(nil) is now a no-op.

Test results

  • gofmt -l pkg exec test: clean.
  • go vet ./pkg/... and go test -race ./pkg/... from the root: all packages pass.
  • exec/caddytls: go vet ./... and go test -race ./... pass. GOWORK=off go build ./... also passes, so the plugin still builds against the published pkg.para.party/certdx v0.7.0 and uses no new root-module API.
  • exec/client: go vet ./... and go test -race ./... pass (no test files).
  • go build ./... in ., exec/caddytls, exec/client, exec/server, exec/tools, test/e2e: all OK.
  • e2e (cd test/e2e && go test -tags e2e -count=1 ./..., with xcaddy building the plugin from this tree): ok pkg.para.party/certdx/test/e2e 269.1s (rerun after the mTLS-reload fix), including the caddytls HTTP-token and gRPC-mTLS scenarios.

Split from #75.

🤖 Generated with Claude Code

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-29T08:28:33.880326Z 5d07545 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5d0754594f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pkg/client/daemon.go Outdated
Comment on lines +147 to +148
if c, ok := r.httpClients[server]; ok {
return c, nil

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reload mTLS credentials after the PEM changes

When an HTTP client uses mTLS, the cached client permanently retains the tls.Config loaded by WithCertDXServerInfo during its first construction. Previously each polling round rebuilt the client and reread the PEM, so replacing a compromised or renewed client bundle took effect automatically; now every cache hit continues presenting the old certificate and trusting the old CA until the daemon or Caddy process restarts. Invalidate/rebuild this client when the bundle changes, or load credentials dynamically while still reusing the transport.

Useful? React with 👍 / 👎.

sosyz and others added 6 commits September 29, 2026 17:51
The Caddy plugin swaps the package logger on every config reload while
other goroutines are logging, which is a data race on a plain variable.
Store the logger and debug flag atomically, and ignore a nil logger in
SetLogger so in-flight logging never panics.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ller

- MakeCertDXHttpClient and its options return an error instead of calling
  logging.Fatal on an unreadable mTLS bundle, and the gRPC client loads
  its credentials lazily on the first Stream. A bad bundle is now a
  retryable fetch/stream error rather than os.Exit, which also stops the
  Caddy plugin from killing the whole Caddy process.
- The daemon builds one HTTP client per server and reuses it, instead of
  a fresh http.Transport (and idle connection pool) per poll attempt.
  Transports expire idle connections after 90s.
- The poll interval is RenewTimeLeft/4 as before, but a zero or negative
  RenewTimeLeft falls back to one minute and a sub-second one is floored
  at one second (never beyond RenewTimeLeft itself), so a bad server
  value cannot spin the poller.
- A round in which no server answered retries on a 15s..60s doubling
  backoff instead of waiting a whole success interval. A well-formed
  server error (resp.Err) keeps the success interval, so a permanent
  refusal is not hammered.
- The SDS stream refuses to start when two watched certs share a name,
  which would otherwise collide in the dispatch map and starve one of
  them of updates.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- The file action fsyncs the temp cert/key before renaming them into
  place and the parent directories afterwards, so a crash cannot leave a
  zero-length cert or key behind. Both syncs are best effort: mounts that
  answer fsync with ENOSYS/EINVAL still get the certificate.
- reloadCommand runs under the action's context with a 5 minute timeout,
  so a hung command is killed instead of holding back every later
  delivery for that certificate and hanging daemon shutdown.
- docs/client.md describes the reload timeout, the fsync and the poll
  interval / retry backoff behaviour.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Two certificates over the same domain set: the daemon keys its
  watchers on the domain set, so the second silently replaced the first
  and the first one's update actions never ran.
- In gRPC mode, two certificates with the same name: the name is the SDS
  resource name, so they collided in the stream's dispatch map. HTTP mode
  still allows a repeated name as long as the files differ.
- Two file actions writing the same <savePath>/<name>.pem, across
  certificates or within one: they raced each other and ran the reload
  command twice.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- authMethod is checked while adapting the Caddyfile, so a typo such as
  "mTLS" fails `caddy adapt` instead of silently sending unauthenticated
  requests. expectArg1 errors now name the directive, not its argument.
- Provision validates the transport config the way the standalone client
  does: a missing main server, an unknown mode or auth method, or a
  missing mTLS bundle now fails the config load (which Caddy can roll
  back) instead of surfacing at request time — before the client stopped
  calling os.Exit, a missing bundle took the whole Caddy process down.
- A native-JSON config that omits mode, reconnect_interval or authMethod
  gets the same defaults as the Caddyfile adapter, and a token server
  without a token logs an INSECURE warning.
- The keypair is parsed once per renewal instead of on every handshake
  (certmagic does not cache manager-provided certificates), and
  GetCertificate returns the real reason there is no certificate, tagged
  with the cert id. Every config load, reload included, starts from an
  empty cache.
- Stop is safe when Provision never built the daemon.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Caching one HTTP client per server kept the tls.Config loaded on first
use for the life of the daemon, so a renewed or replaced client bundle
no longer took effect: before the cache, every poll round re-read it.
Fingerprint the bundle on each call and rebuild the client (closing the
old transport's idle connections) when it changed. The transport is
still reused while the bundle stays the same.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants