feat(config): add standalone and O11y deployment authentication

[pradystar]

Summary

Add configuration detection for standalone and O11y deployments. This establishes the configuration and authentication foundation needed for the SDK's OTLP transition while preserving the existing standalone authentication flow.

Why

The SDK must select the correct endpoints and authentication mechanism without requiring users to set an explicit deployment-mode option. Standalone customers continue using their existing configuration, while O11y customers configure a realm and Splunk access token and do not perform the standalone JWT exchange.

What changed

• Detect standalone or O11y deployment from the configured environment variables and reject ambiguous or incomplete configuration with actionable errors.
• Add deployment-specific configuration models that derive CRUD, console, and OTLP endpoints from a realm or standalone console/API URL.
• Add an O11y API client that authenticates CRUD requests with X-SF-Token and consistently applies the /v2/ao prefix to synchronous, asynchronous, and streaming requests.
• Prefer SPLUNK_AO_SF_API_TOKEN for CRUD when provided and otherwise use SPLUNK_AO_SF_TOKEN.
• Bypass standalone JWT creation and refresh for O11y while retaining the existing standalone validation and client behavior.
• Export the new configuration exceptions from the public package API.
• Declare httpx as a direct dependency because it is imported by the SDK.
• Add unit coverage for deployment detection, endpoint derivation, validation failures, token selection, path prefixing, and standalone compatibility.

Configuration

Deployment	Required	Optional
O11y	SPLUNK_AO_REALM, SPLUNK_AO_SF_TOKEN	SPLUNK_AO_SF_API_TOKEN
Standalone	SPLUNK_AO_API_KEY, SPLUNK_AO_CONSOLE_URL	SPLUNK_AO_API_URL

Testing

HOME=/private/tmp/splunk-ao-pr1-test-home poetry run pytest -q — 125 passed; Ruff lint and format checks, mypy, and git diff --check also passed.

[fercor-cisco]

🤖 This review was generated by the Astra agent (claude-opus-4-8). It may contain mistakes.

Verdict: needs_discussion — CRUD auth path is solid and well-tested, but the OTLP export path—the PR's stated purpose—is not wired for O11y and would crash; scope needs confirmation.

General Comments

• 🟠 major (question): The PR summary frames this as "the configuration and authentication foundation needed for the SDK's OTLP transition," and adds O11yConfig.otlp_endpoint, but the actual trace-export path is not wired for O11y and will crash for any O11y user who exports spans.

SplunkAOOTLPExporter.__init__ (src/splunk_ao/otel.py:119-145) unconditionally does:

• api_key = config.api_key.get_secret_value() if config.api_key else None then if not api_key: raise ValueError("API key is required."). For O11y, set_jwt_token nulls the tokens and no api_key is ever set, so config.api_key is None → this raises.
• Builds the endpoint from config.api_url (which for O11y is https://api.{realm}.../v2/ao) + otel/traces and sends a Splunk-AO-API-Key header — neither matches the realm-derived ingest endpoint (https://ingest.{realm}.../v2/trace/otlp) or the X-SF-Token scheme this PR introduces.

Meanwhile O11yConfig.otlp_endpoint (the correct ingest URL) is defined but never consumed anywhere in src/. Is trace export for O11y intended as a follow-up PR? If so, that's fine, but please confirm — as it stands the SDK's core tracing function is broken for O11y even though the config/detection succeeds. If it is a follow-up, a tracking ticket reference in the code or PR would help.

Follow-ups

Suggested follow-up work that could be tracked as Shortcut stories:

• src/splunk_ao/deployment.py:104-136: StandaloneConfig and both otlp_endpoint properties (O11y and standalone) are unused in src/ — only exercised by tests. Once the OTLP export path is wired up they'll have a consumer; until then they're dead code. Additionally, StandaloneConfig.otlp_endpoint derives the api host via .replace("://console.", ...).replace("://app.", ...), which silently does nothing for custom console domains (e.g. https://galileo.mycorp.com) and diverges from galileo-core's own set_api_url, which replaces console→api anywhere in the string. Worth reconciling these two derivations when the exporter starts using them.

[fercor-cisco]

🟡 minor (design): The O11y auth guard passes when either SPLUNK_AO_SF_TOKEN or SPLUNK_AO_SF_API_TOKEN is present, and resolve_deployment() likewise treats SPLUNK_AO_SF_API_TOKEN alone as O11y — but O11yConfig.from_env() (deployment.py:63-78) requires both SPLUNK_AO_REALM and SPLUNK_AO_SF_TOKEN. This creates two configs that clear the auth guard and deployment detection but then fail during construction:

1. SPLUNK_AO_SF_API_TOKEN only (no realm, no SF_TOKEN) → guard returns None, detection returns O11Y, then _bridge_env_vars → O11yConfig.from_env() raises MissingConfigurationError.
2. SPLUNK_AO_REALM + SPLUNK_AO_SF_API_TOKEN but no SPLUNK_AO_SF_TOKEN → a user who only has an API token (sufficient for CRUD, since crud_token prefers the API token) still cannot construct the config because the ingest SF_TOKEN is mandated.

The resulting error is actionable (it names the missing vars), so this isn't a silent failure, but the three layers disagree on what constitutes a valid O11y config. Consider aligning them — e.g. require SPLUNK_AO_REALM in the auth guard, and decide whether SF_TOKEN should truly be mandatory when a CRUD-only API token is supplied.

🤖 Generated by the Astra agent

[pradystar]

Good catch. Changed configuration layers so SPLUNK_AO_REALM plus either token is valid. SPLUNK_AO_SF_API_TOKEN alone with a realm now supports CRUD only use, OTLP exporter construction separately requires SPLUNK_AO_SF_TOKEN and raises an actionable error if it is absent. An API token
without a realm still fails because the realm is required to derive the
CRUD endpoint. Added additional tests.

[pradystar]

🤖 This review was generated by the Astra agent (claude-opus-4-8). It may contain mistakes.

Verdict: needs_discussion — CRUD auth path is solid and well-tested, but the OTLP export path—the PR's stated purpose—is not wired for O11y and would crash; scope needs confirmation.

General Comments

• 🟠 major (question): The PR summary frames this as "the configuration and authentication foundation needed for the SDK's OTLP transition," and adds O11yConfig.otlp_endpoint, but the actual trace-export path is not wired for O11y and will crash for any O11y user who exports spans.

SplunkAOOTLPExporter.__init__ (src/splunk_ao/otel.py:119-145) unconditionally does:

• api_key = config.api_key.get_secret_value() if config.api_key else None then if not api_key: raise ValueError("API key is required."). For O11y, set_jwt_token nulls the tokens and no api_key is ever set, so config.api_key is None → this raises.
• Builds the endpoint from config.api_url (which for O11y is https://api.{realm}.../v2/ao) + otel/traces and sends a Splunk-AO-API-Key header — neither matches the realm-derived ingest endpoint (https://ingest.{realm}.../v2/trace/otlp) or the X-SF-Token scheme this PR introduces.

Meanwhile O11yConfig.otlp_endpoint (the correct ingest URL) is defined but never consumed anywhere in src/. Is trace export for O11y intended as a follow-up PR? If so, that's fine, but please confirm — as it stands the SDK's core tracing function is broken for O11y even though the config/detection succeeds. If it is a follow-up, a tracking ticket reference in the code or PR would help.

Follow-ups

Suggested follow-up work that could be tracked as Shortcut stories:

• src/splunk_ao/deployment.py:104-136: StandaloneConfig and both otlp_endpoint properties (O11y and standalone) are unused in src/ — only exercised by tests. Once the OTLP export path is wired up they'll have a consumer; until then they're dead code. Additionally, StandaloneConfig.otlp_endpoint derives the api host via .replace("://console.", ...).replace("://app.", ...), which silently does nothing for custom console domains (e.g. https://galileo.mycorp.com) and diverges from galileo-core's own set_api_url, which replaces console→api anywhere in the string. Worth reconciling these two derivations when the exporter starts using them.

This is expected follow up work. This PR is limited to deployment detection and CRUD authentication it does not complete O11y trace export. The existing SplunkAOOTLPExporter/SplunkAOSpanProcessor path will be made deployment aware in https://splunk.atlassian.net/browse/HYBIM-887, using the standalone/O11y exporter factories. That change will use the realm-derived ingest endpoint and X-SF-Token for O11y. The direct logger egress path is tracked separately in https://splunk.atlassian.net/browse/HYBIM-890.

[fercor-cisco]

Does PR #80 implement Option 3 (Auto-Detection with Strict Validation)?

Verdict: Mostly implemented, with two intentional deviations and one piece of dead code.

The core of Option 3 — auto-detection via presence of o11y-specific vs. standalone-specific variables, with ambiguity and missing-config rejection — is implemented in src/splunk_ao/deployment.py:resolve_deployment() and wired into SplunkAOConfig.get().

Rule-by-rule mapping

Spec rule	Implemented?	Where
has_o11y_config = REALM ∨ SF_TOKEN ∨ SF_API_TOKEN	✅	_O11Y_ENV_VARS, deployment.py:11,28
has_standalone_config = API_KEY ∨ CONSOLE_URL	⚠️ broader — also includes API_URL	_STANDALONE_ENV_VARS, deployment.py:12
both present → ambiguous error	✅	deployment.py:31-36; fires in get() before construction (test test_o11y_config.py:247)
o11y → require REALM	✅	deployment.py:59-60, config.py:201-202
o11y → require SF_TOKEN	❌ deviation — accepts SF_API_TOKEN alone	deployment.py:61-62, config.py:203-204
o11y → optionally SF_API_TOKEN	✅	crud_token prefers it, deployment.py:89-95
standalone → require API_KEY	⚠️ enforced in StandaloneConfig only (unused); runtime accepts jwt/sso/user-pass too	config.py:208-246
standalone → require CONSOLE_URL	❌ not enforced at runtime — console_url has a default	config.py:73, constants:8
neither → missing-config error	✅	deployment.py:42-46

The three things worth attention

1. o11y no longer strictly requires SPLUNK_AO_SF_TOKEN (deviation from spec).
The spec says require SPLUNK_AO_SF_TOKEN / optionally SF_API_TOKEN. The PR instead requires at least one of the two (deployment.py:61, config.py:203), and accepts SF_API_TOKEN alone. This appears intentional — it matches commit beb6d4c "allow only SPLUNK_AO_SF_API_TOKEN for CRUD use", and the code correctly distinguishes CRUD (either token) from OTLP ingest (require_ingest_token() still hard-requires SF_TOKEN, deployment.py:97-104). A deliberate refinement, not a bug — but it does not match the literal spec text.

2. Standalone detection is broader than spec; standalone validation is looser than spec.

• Detection adds SPLUNK_AO_API_URL as a standalone discriminator (deployment.py:12). This makes REALM + SF_TOKEN + API_URL correctly ambiguous (test test_deployment.py:64) — a robustness improvement beyond the spec's literal has_standalone_config definition.
• The spec's require SPLUNK_AO_CONSOLE_URL is not enforced on the real path: console_url defaults to DEFAULT_CONSOLE_URL (config.py:73), and _check_auth_config returns OK on api_key alone (config.py:208). It also accepts jwt/sso/username-password, which the spec's standalone section doesn't mention (these are pre-existing galileo-core auth methods being preserved).

3. StandaloneConfig is effectively dead code.
O11yConfig is used throughout config.py, but StandaloneConfig is referenced only in tests/test_deployment.py — never in src/. So its strict require API_KEY and CONSOLE_URL validation (deployment.py:138-144) and its otlp_endpoint derivation are tested but never exercised by the actual SDK config flow. If the intent was for standalone validation to mirror the spec's strictness, that wiring is missing.

Bottom line

The PR faithfully implements the auto-detection + ambiguity + missing-config heart of Option 3, and is more robust than the spec on ambiguity detection. It diverges on two "require" clauses: o11y no longer mandates SF_TOKEN (intentional, CRUD-vs-ingest split), and standalone doesn't enforce CONSOLE_URL at runtime (defaulted, and the strict StandaloneConfig validator is unused). Whether those two count as gaps depends on whether the SF_API_TOKEN relaxation and the standalone leniency were sanctioned decisions — the commit history suggests the first was.

_{🤖 Analysis generated with Claude Code}

[shuningc]

This method is tested but never called by the OTLP export path.

[pradystar]

#80 (comment)

[pradystar]

Does PR #80 implement Option 3 (Auto-Detection with Strict Validation)?

Verdict: Mostly implemented, with two intentional deviations and one piece of dead code.

The core of Option 3 — auto-detection via presence of o11y-specific vs. standalone-specific variables, with ambiguity and missing-config rejection — is implemented in src/splunk_ao/deployment.py:resolve_deployment() and wired into SplunkAOConfig.get().

Rule-by-rule mapping

Spec rule Implemented? Where
has_o11y_config = REALM ∨ SF_TOKEN ∨ SF_API_TOKEN ✅ _O11Y_ENV_VARS, deployment.py:11,28
has_standalone_config = API_KEY ∨ CONSOLE_URL ⚠️ broader — also includes API_URL _STANDALONE_ENV_VARS, deployment.py:12
both present → ambiguous error ✅ deployment.py:31-36; fires in get() before construction (test test_o11y_config.py:247)
o11y → require REALM ✅ deployment.py:59-60, config.py:201-202
o11y → require SF_TOKEN ❌ deviation — accepts SF_API_TOKEN alone deployment.py:61-62, config.py:203-204
o11y → optionally SF_API_TOKEN ✅ crud_token prefers it, deployment.py:89-95
standalone → require API_KEY ⚠️ enforced in StandaloneConfig only (unused); runtime accepts jwt/sso/user-pass too config.py:208-246
standalone → require CONSOLE_URL ❌ not enforced at runtime — console_url has a default config.py:73, constants:8
neither → missing-config error ✅ deployment.py:42-46

The three things worth attention

1. o11y no longer strictly requires SPLUNK_AO_SF_TOKEN (deviation from spec). The spec says require SPLUNK_AO_SF_TOKEN / optionally SF_API_TOKEN. The PR instead requires at least one of the two (deployment.py:61, config.py:203), and accepts SF_API_TOKEN alone. This appears intentional — it matches commit beb6d4c "allow only SPLUNK_AO_SF_API_TOKEN for CRUD use", and the code correctly distinguishes CRUD (either token) from OTLP ingest (require_ingest_token() still hard-requires SF_TOKEN, deployment.py:97-104). A deliberate refinement, not a bug — but it does not match the literal spec text.

2. Standalone detection is broader than spec; standalone validation is looser than spec.

• Detection adds SPLUNK_AO_API_URL as a standalone discriminator (deployment.py:12). This makes REALM + SF_TOKEN + API_URL correctly ambiguous (test test_deployment.py:64) — a robustness improvement beyond the spec's literal has_standalone_config definition.
• The spec's require SPLUNK_AO_CONSOLE_URL is not enforced on the real path: console_url defaults to DEFAULT_CONSOLE_URL (config.py:73), and _check_auth_config returns OK on api_key alone (config.py:208). It also accepts jwt/sso/username-password, which the spec's standalone section doesn't mention (these are pre-existing galileo-core auth methods being preserved).

3. StandaloneConfig is effectively dead code. O11yConfig is used throughout config.py, but StandaloneConfig is referenced only in tests/test_deployment.py — never in src/. So its strict require API_KEY and CONSOLE_URL validation (deployment.py:138-144) and its otlp_endpoint derivation are tested but never exercised by the actual SDK config flow. If the intent was for standalone validation to mirror the spec's strictness, that wiring is missing.

Bottom line

The PR faithfully implements the auto-detection + ambiguity + missing-config heart of Option 3, and is more robust than the spec on ambiguity detection. It diverges on two "require" clauses: o11y no longer mandates SF_TOKEN (intentional, CRUD-vs-ingest split), and standalone doesn't enforce CONSOLE_URL at runtime (defaulted, and the strict StandaloneConfig validator is unused). Whether those two count as gaps depends on whether the SF_API_TOKEN relaxation and the standalone leniency were sanctioned decisions — the commit history suggests the first was.

🤖 Analysis generated with Claude Code

1. This decision is deliberate, based on feat(config): add standalone and O11y deployment authentication #80 (comment) this decision allows users to configure SF_API_TOKEN alone for CRUD operations, however, ingest path will require SPLUNK_AO_SF_TOKEN.
2. DEFAULT_CONSOLE_URL(also jwt/sso/username-password,) is pre existing, I had not taken that into consideration so kept pre existing defaults for standalone.
3. Correct that StandaloneConfig is not yet consumed by the current SDK execution path in this configuration foundation PR. It represents the standalone OTLP egress configuration rather than replacing the existing standalone CRUD/JWT/SSO configuration flow. The standalone exporter PR consumes this type, and the subsequent native/user- wired OTel and logger-egress integrations construct it through StandaloneConfig.from_env(), which is where the strict API-key and console validation becomes active. Wiring it unconditionally into SplunkAOConfig.get() would incorrectly reject supported standalone CRUD authentication methods that do not use an API key.

[fercor-cisco]

Follow-up summary (post-merge)

Consolidating the outcome of the review discussion so the follow-up scope is clear before merge.

Tracked in Jira

• HYBIM-887 — Deployment-aware OTLP trace export. Makes O11y tracing functional (it is not wired in this PR). Scope:
  • Use the realm-derived ingest endpoint (https://ingest.{realm}.observability.splunkcloud.com/v2/trace/otlp) instead of api_url + otel/traces.
  • Send X-SF-Token instead of Splunk-AO-API-Key.
  • Drop the api_key requirement (null for O11y) and use require_ingest_token(), which enforces SPLUNK_AO_SF_TOKEN.
  • Consumes the code introduced here that is currently tested-but-unwired: O11yConfig.otlp_endpoint, O11yConfig.require_ingest_token(), StandaloneConfig, and StandaloneConfig.otlp_endpoint — via the from_env() factories.
• HYBIM-890 — Direct logger egress path. The log-export path made deployment-aware for O11y; tracked separately from the exporter.

Decisions made in review (no further action)

• O11y OTLP export crash / missing wiring — accepted as follow-up (HYBIM-887 / HYBIM-890).
• Tested-but-unwired code (require_ingest_token, StandaloneConfig, otlp_endpoint properties) — accepted; consumer lands in HYBIM-887.
• StandaloneConfig.otlp_endpoint host derivation (console./app. → api. prefix replacement) — kept as-is by design. It is intentionally more precise than galileo-core's substring set_api_url (which would rewrite any host merely containing "console"). No reconciliation needed when HYBIM-887 wires it up.

Scope of this PR

Deployment detection + CRUD authentication only. Standalone behavior is unchanged; O11y CRUD is complete. O11y trace/log export is intentionally deferred to the tickets above.

[fercor-cisco]

See comment above about follow-ups.