Skip to content

feat(gateway): verify Cloudflare Access edge tokens - #4866

Open
ericcurtin wants to merge 4 commits into
NVIDIA:mainfrom
ericcurtin:feat/4813-edge-jwt-auth/ericcurtin
Open

ericcurtin wants to merge 4 commits into
NVIDIA:mainfrom
ericcurtin:feat/4813-edge-jwt-auth/ericcurtin

Conversation

@ericcurtin

Copy link
Copy Markdown
Contributor

Summary

Verify the Cloudflare Access token on every gRPC request, so bypassing the proxy no longer grants admin.

Related Issue

Closes #4813

Changes

  • Add [openshell.gateway.edge_auth] and an EdgeAuthenticator.
  • Reuse JwksCache for key fetch, rotation, and validation.
  • Share RBAC policy construction between OIDC and edge.
  • Refuse edge_auth with OIDC or allow_unauthenticated_users.
  • Docs.

Without admin_role and user_role, every admitted caller is a platform admin (as with mTLS). Startup warns.

Release Notes

Gateways behind Cloudflare Access can verify the Access token with [openshell.gateway.edge_auth] instead of allow_unauthenticated_users.

Testing

  • Checks appropriate to the affected code and behavior pass
  • Unit tests added/updated (if applicable)
  • E2E tests added/updated (if applicable)

Ran cargo fmt, cargo clippy -D warnings, cargo test for openshell-server and openshell-core, markdownlint, and fern check.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
@copy-pr-bot

copy-pr-bot Bot commented Oct 23, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@ericcurtin

Copy link
Copy Markdown
Contributor Author

@mrunalp @johntmyers PTAL, and /ok to test 1f7556b266831c0f2d94c08939ebd7c2a739f513 if it looks good. Thank you!

@mrunalp mrunalp left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The structure looks right: JwksCache is reused through TokenKind, the gateway refuses to start with edge auth alongside OIDC or allow_unauthenticated_users, and a present-but-invalid token stops the chain. I also checked the WebSocket tunnel path. The SDK interceptor puts cf-access-jwt-assertion on every inner gRPC call, and /_ws_tunnel hands those calls back to MultiplexService, so they go through the same auth chain.

A few things to settle before approving:

1. OIDC users with an empty sub are now rejected.
Before this PR, an OIDC token with "sub": "" was accepted: sub only had to be present. With this PR, the empty-subject check sits in the shared validate_token, so OIDC deployments also start returning Unauthenticated: empty subject. It's sensible hardening, but the PR description doesn't mention it and it's outside the issue's scope. Could you either note it in the release notes or limit the check to TokenKind::CloudflareAccess? (crates/openshell-server/src/auth/oidc.rs:1003-1011)

2. The default setup still makes every caller an admin.
Before this PR, every caller behind Access was one shared admin principal. With this PR and no roles configured, each caller gets their own identity, but every one is still a platform admin and workspace membership isn't checked. That includes the config #4813 suggests, which sets only roles_claim. The issue's first acceptance criterion says the subject "is checked against workspace membership", which only happens once both admin_role and user_role are set. This matches how OIDC behaves without roles, and there's a startup warning (crates/openshell-server/src/cli.rs:761) plus docs. We should still decide whether this mode is an acceptable default or should need an explicit opt-in.

3. Automation using Access service tokens has no path in edge mode.
Before this PR, automation using Cloudflare Access service tokens got through as the local-dev principal. With edge_auth on, those tokens are rejected for having an empty sub. OIDC and allow_unauthenticated_users can't be combined with edge auth, and the documented Helm setup disables TLS, which rules out mTLS. Operators who switch would lose that automation entirely. It's documented and opt-in, so it isn't a regression. Should service tokens be mapped via common_name here, or in a follow-up?

4. The main security promise has no direct test.
A request with no edge token is rejected rather than treated as local-dev only because of two things together: validate_config forbids allow_unauthenticated_users, and the chain is non-empty. No router-level test builds a ServerState with edge_auth and sends a request without a token. rejected_edge_credential_emits_classified_authentication_failure uses a mock rejection, and the authenticator test stops at NotApplicable. Since this is the issue's second acceptance criterion, a test of that whole path would protect it from future refactors. The E2E box is also unchecked.

5. Nit: only the first Cookie header is read.
HTTP/2 lets clients split cookies across several cookie fields, and edge_credential reads only the first (crates/openshell-server/src/auth/edge.rs:37). The CLI and SDK send the header, so they're unaffected; a split-cookie client would get NotApplicable and then Unauthenticated.

Out of scope, for context: browser-facing sandbox service routing (service_routing.rs) doesn't check the gateway user at all, and this PR doesn't change that. The docs correctly say gRPC callers that bypass the proxy are rejected. The issue's "every request" wording is broader than what ships.

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Reject empty subjects for edge tokens only, read split Cookie headers, and test the router rejects tokenless requests.

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
@ericcurtin

Copy link
Copy Markdown
Contributor Author

@mrunalp thanks. Merged main and:

  1. Empty sub is rejected for edge tokens only. OIDC is unchanged and pinned by a test.

  2. Added a router test: no token and bad token are rejected, a valid token is admitted.

  3. All Cookie headers are read.

  4. Kept as is (same as OIDC without roles, with a startup warning and docs). Happy to add an opt-in if you prefer.

  5. Service tokens in a follow-up.

PTAL, and /ok to test ac090df9a10b3603b22e55f5762b7fc9bf6271dd if it looks good. Thank you!

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
@ericcurtin

Copy link
Copy Markdown
Contributor Author

Merged main and fixed the conflicts. @mrunalp PTAL, and /ok to test 23ecae03500ade5a0d22af85696c94173193d6e2 if it looks good. Thank you!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(gateway): validate Cloudflare Access edge JWTs instead of trusting the proxy

2 participants