[PRD}: Keyless read-only MCP access via a default MCP token

[larsgeorge-db]

Problem Statement

As an operator who wants Genie (and similar native MCP connectors) to fetch governed glossary/concept definitions from Ontos mid-answer, I can't: Genie carries one credential in the Authorization header, clears the Databricks app-gate, but is then rejected by Ontos's inner X-API-Key check. Today my only options are a token-injection shim or a proxy — both are extra moving parts to build, secure, and keep in sync with the MCP tool surface.

Background: the two gates

Ontos's MCP endpoint (/api/mcp) sits behind two independent gates:

1. App proxy (Databricks Apps OAuth2) — external, mandatory for any app endpoint. On success it injects X-Forwarded-Access-Token / X-Forwarded-Email / X-Forwarded-User. The regular HTTP API already reads these (common/authorization.py).
2. In-app MCP token (X-API-Key) — guards only /api/mcp; governs authentication, tool authorization (scopes drive tools/list filtering + tools/call enforcement via MCPHandler._has_scope), audit identity (created_by), and lifecycle (expires_at, is_active).

Native connectors like Genie clear gate 1 but present no X-API-Key, so gate 2 rejects (401 / JSON-RPC -32001). Verified end-to-end.

Established during exploration:

• MCP today never reads the forwarded identity — it's purely X-API-Key-driven — but the headers are present on every proxied request.
• The MCP token is a tool-authz gate, not a data-access gate: global_search deliberately bypasses per-user ACLs and delegates access control to token scopes.
• _has_scope is pure/data-driven, so any resolved principal's scopes flow through existing tool gating with zero tool changes.

Solution

When an MCP request arrives that has already cleared the app-gate (a real, forwarded Databricks user) but carries no X-API-Key, Ontos resolves it to an admin-designated default MCP token instead of rejecting it. That default token's scopes bound exactly what the keyless caller can do; the request is audited under the caller's forwarded email. Admins designate and scope the default token from the existing MCP-token settings UI, with a clear warning that its scopes apply to any authenticated app user. Removing/deactivating the default token turns the capability off. This lets Genie phone Ontos for governed definitions with no shim and no token dance, while keeping coarse revocation and full per-user audit attribution.

Confirmed decisions:

• Keyless caller resolves to a designated default MCP token (a real mcp_tokens row), re-stamped with the caller's forwarded email for audit.
• Default token = ordinary token row; admin sets its scopes freely (no code-enforced restriction), UI warns.
• Switch = presence: keyless works iff an active, non-expired token flagged as the keyless default exists. No separate settings flag; default off = no such row.
• Principal-resolution logic extracted onto MCPTokensManager (deep, unit-testable); routes stay thin.

User Stories

1. As a Genie user, I want Ontos glossary/concept lookups to resolve mid-answer, so that I get governed definitions without configuring an Ontos API key.
2. As an Ontos admin, I want to designate one existing MCP token as the "keyless default", so that keyless app-gate callers inherit exactly its scopes.
3. As an Ontos admin, I want to choose the default token's scopes freely in the settings UI, so that I control the keyless blast radius.
4. As an Ontos admin, I want a clear warning that the default token's scopes apply to any authenticated app user with no key, so that I don't over-grant by accident.
5. As an Ontos admin, I want keyless access disabled by default, so that upstream merges and fresh deploys never silently open this path.
6. As an Ontos admin, I want to turn keyless access off instantly by deactivating/deleting the default token, so that I retain a coarse revocation lever.
7. As an Ontos admin, I want the default token to honor expires_at, so that keyless access lapses unless deliberately re-blessed.
8. As a security reviewer, I want every keyless call attributed to the caller's forwarded email in the audit log, so that "who did what" is preserved despite a shared token identity.
9. As a security reviewer, I want keyless callers to be denied tools outside the default token's scopes (e.g. SPARQL, writes), so that least privilege holds.
10. As a security reviewer, I want a keyless request with no forwarded identity to be rejected and logged as anonymous, so that unauthenticated access is impossible.
11. As an existing keyed integration, I want my X-API-Key behavior completely unchanged, so that this feature doesn't regress me.
12. As an Ontos admin, I want at most one active keyless-default token at a time, so that the keyless capability is unambiguous.
13. As an Ontos admin, I want designating a new default to clear the previous one, so that switching defaults is a single action.
14. As an Ontos admin, I want the token list to show which token is the keyless default, so that I can see the current state at a glance.
15. As a developer, I want principal resolution in a pure, testable manager method, so that the security-relevant branches are covered by unit tests.
16. As a Genie user, I want write/mutating and SPARQL tools refused on the keyless path (assuming a read-only default), so that keyless access can't change governed state.

Implementation Decisions

• Schema: add is_keyless_default: bool (default False, indexed) to the mcp_tokens model, via an Alembic migration following the repo's short-revision convention (rebase onto development, descend from the live head — re-verify, don't hardcode).
• Repository: get_keyless_default(db) (active, non-expired, flagged row) and set_keyless_default(db, token_id) (clears any other flag then sets target) — single active default enforced in code.
• Manager (deep module): MCPTokensManager.resolve_keyless_default(email) -> Optional[MCPTokenInfo] — fetch default, update_last_used, return MCPTokenInfo built from the row but with created_by = email and a distinguishing name. Scopes taken verbatim from the row. Thread the default flag through create/designate paths.
• Routes: replace validate_api_key(...) with a thin resolve_mcp_principal(db, x_api_key, request): key present → validate_token (unchanged); key absent → read forwarded email (header-trust only, no OBO/SDK call) and resolve the default if one is active, else None. Swap the three identical call sites (SSE GET, POST handler, DELETE). Existing if not token_info: <reject/audit> blocks unchanged; keyless failures still log anonymous.
• No changes to MCPHandler, _get_username, _has_scope, session handling, or any tool — attribution and scoping flow through the existing MCPTokenInfo.
• API/admin routes: add is_keyless_default to token info responses; add an admin-only, audited action to designate an existing token as the keyless default (mirrors existing CREATE/REVOKE audit).
• Frontend: token type + create/designate calls; a "keyless default" badge/toggle in the MCP-tokens settings; an inline warning that the default token's scopes apply to any authenticated app user (call out global_search's ACL bypass); en-locale strings as source of truth.

Testing Decisions

Good tests here assert external behavior — the resolved principal and the endpoint's authz/audit outcomes — not internal call sequencing. Prior art: existing MCP token manager/route tests and the audit-logging assertions already in the suite.

• Principal resolution (unit, on the manager): key present → validates as today, default ignored; no key + active default + email → principal with created_by == email and the row's scopes; no key + no default → None; no key + inactive/expired default → None; no key + default active but no forwarded email → None.
• Single-default invariant (unit/repo): set_keyless_default clears any prior default; at most one active default exists.
• MCP route integration: with mock forwarded headers and a semantic:read+search:read default — tools/list filtered to scope; search_glossary_terms succeeds; execute_sparql_query → -32002; no email + no key → -32001 and audit username anonymous; keyless success audit username == forwarded email.

Out of Scope

• Per-caller rate limiting on the keyless path (noted as a follow-up).
• Code-enforced scope restrictions on the default token (admin choice is deliberate; UI warns instead).
• Reading/validating the OBO access token or calling the Databricks SDK on the MCP path (header-trust only, matching the regular path's trust level).
• Row-level data ACLs inside tools (global_search bypass is pre-existing behavior, unchanged).
• Multi-default or per-connector default tokens.

Further Notes

• Tradeoff accepted: keyless path loses per-token expiry/revocation granularity (revocation is coarse — one default row), but attribution is preserved via created_by = X-Forwarded-Email. Recommend enabling only where "any app user can read glossary terms" is acceptable policy.
• The default token's scopes are the entire keyless blast radius: semantic:read reaches find_entities_by_concept (lists linked data products/contracts); search:read reaches global_search (whole-index, no per-user ACL). PR should recommend read-only scopes (no *:write, sparql:query, or *).
• Chosen over the proxy/shim route because it reuses existing token machinery (UI-managed scopes, is_active revocation, expires_at) and confines the change to one resolver + three call sites.

[mvkonchits-db]

Solid design. The token-row mechanism is a real improvement over synthesizing scopes in code, since you get expires_at and is_active back for free on the keyless path. Two things I'd flag before it's built.

1. Least privilege is advisory, not enforced.

The PRD deliberately leaves the default token's scopes to admin choice, with a UI warning instead of a code check. That means an admin can flag a default token carrying sparql:query, *:write, or *, and now any authenticated app user inherits it with no key and no per-integration revocation lever.

The warning helps, but warnings get clicked through. Since the whole point is "read-only glossary lookups for Genie," consider a code-enforced ceiling: reject designating a token as keyless-default if it carries anything beyond a safelist (say semantic:read, search:read). Keeps the exact mechanism, just refuses to arm the dangerous configuration. Cheap insurance against a fat-finger that widens the blast radius for every app user at once.

Worth spelling out in the UI too: search:read reaches global_search, which bypasses per-user ACLs and scans the whole index. So a "read-only" default isn't "any user can read glossary terms," it's "any user can search the entire governed index regardless of their own entitlements." Fine for some policies, surprising for others.

2. The audit-attribution benefit depends on Genie's connection config, and that path is undocumented.

Stories 8 and 10 lean on X-Forwarded-Email being the human who asked the Genie question. Whether that holds depends on how the Genie to Ontos MCP connection is authenticated:

• OAuth U2M Per-User -> forwards the end user. Audit story works as written.
• Bearer / OAuth M2M (shared SP) -> forwards one shared service identity for every user. Then created_by logs the same principal on every keyless call, which is the shared-token problem this design is trying to avoid, except now with no X-API-Key to scope or revoke per integration either.

Genie's built-in connectors use end-user OBO. But Ontos is a custom external MCP registered via a UC HTTP connection, and I couldn't find docs confirming which auth type Genie's external-MCP path uses by default when the target is a Databricks App. So the audit guarantee is currently a config precondition, not a property of the design.

I'd suggest documenting U2M Per-User as a requirement for the audit story to hold, and verifying empirically before relying on it (echo the X-Forwarded-* headers on a real Genie to Ontos call and confirm X-Forwarded-Email is the human).

Neither of these blocks the approach. They just move it from "trust the admin and the connector" to "enforce the floor and verify the identity."

---

Drafted with Claude Code, reviewed and posted by me.