[Enhancement]: Support Oracle Cloud Infrastructure (OCI) Generative AI as a provider

[fede-kamel]

Affected area

Rust core runtime, Python binding, Node.js binding, Framework integrations, Observability or exporters, Documentation or examples

Problem or opportunity

NeMo Relay ships provider codecs for OpenAI Chat, OpenAI Responses, and Anthropic Messages, so middleware (PII redaction, guardrails, policy) and observability subscribers can reason over those providers' payloads through the normalized annotated shapes. There is currently no coverage for Oracle Cloud Infrastructure (OCI) Generative AI, whose chat API uses its own request envelope (ChatDetails with servingMode and chatRequest) and its own wire formats (GENERIC, COHERE, and COHEREV2).

OCI Generative AI serves managed on-demand models (Meta Llama, Cohere Command, xAI Grok, Google Gemini, OpenAI) and dedicated AI cluster endpoints, including imported open-weights models such as NVIDIA Nemotron 3 — so agents calling those models today are invisible to Relay's request intercepts and get no normalized response annotations.

Proposed enhancement

Add OCI Generative AI as a supported provider surface, delivered as a four-part stacked series (plan of record in the comments below, each layer independently green):

1. feat(codec): add OCI GenAI typed variants and response codec (1/4) #554 — typed ApiSpecific request/response variants and an LlmResponseCodec decoding GENERIC, COHERE, and COHEREV2 REST responses (content, tool calls, finish reasons, usage, unmodeled-field preservation).
2. Request-side LlmCodec with encode(decode(x), x) == x identity and merge-not-replace encoding, so intercepts (redaction, policy) can safely edit OCI requests.
3. ProviderSurface::OCIGenAI registration and detection, an OCIGenAIStreamingCodec (SSE for all formats), and provider awareness in guardrails, the PII-redaction overlay, and adaptive components.
4. Python and Node.js binding exposure (pyo3 + napi classes, .pyi/codecs.py), with binding tests.

A supported-integrations guide for observing LangChain agents backed by langchain-oci follows separately if it has genuine value-add beyond examples. Follow-ups under discussion with the maintainers: OCI model pricing entries for cost accounting, and Switchyard routing support.

Runtime contract and binding impact

The Rust core is the source of truth: codecs live in crates/core/src/codec with typed variants in crates/types, and accept the documented camelCase REST wire format only (alternate renderings from OCI tooling are the caller's responsibility). Python and Node.js get codec classes mirroring the existing built-in codec exposure (layer 4); Go and the raw C FFI are unaffected (experimental, source-first). Managed-execution semantics are unchanged — the codecs plug into the existing middleware and event pipeline, and responses annotate LLMEnd events exactly like the other providers.

Alternatives considered

• Python-level integration only (Add OCI Generative AI provider codecs and LangChain integration guide #549): closed per review — the runtime contract requires provider behavior in the Rust core with binding parity.
• One monolithic PR (feat(codec): add OCI Generative AI provider codec in Rust core #552): closed in favor of this reviewable stacked split; the series recombines byte-identically to the validated original.
• Routing GENERIC through the existing OpenAI codec: rejected — the wire is OpenAI-inspired but not compatible (flat tool calls without the nested function object, ChatResult envelope, uppercase enums, different usage keys); the OpenAI codec hard-errors on it.
• Native Cohere V1/V2 codecs instead of an OCI codec: rejected — Oracle's CohereChatResponse wire schema (camelCase, chatResponse envelope, no generation_id/response_id) is not Cohere's own API; a native Cohere codec would decode none of it.

Acceptance criteria

• Relay decodes live OCI GENERIC, COHERE, and COHEREV2 chat responses into normalized annotations (message, tool calls, finish reason, usage), with unmodeled fields preserved in extra.
• Request decode/encode satisfies encode(decode(original), original) == original with unmodeled-field preservation, verified byte-identically against live captures.
• OCI is auto-detected as a provider surface and streaming responses are handled for all formats.
• Python and Node.js expose the codec with tests; docs cover the provider in the supported list.
• Every layer passes the full workspace gate (build, tests, clippy, fmt) independently, and behavior is validated against the live service (all model families, tool calls, finish-reason and usage variants).

[fede-kamel]

Plan updated after review in #549: the OCI codec will be implemented in the Rust core (crates/types/src/codec) with exhaustive tests and Python/Node binding exposure, pinned by the wire-format fixtures validated live in #549. The LangChain guide is deferred unless it has genuine value-add beyond examples.

[fede-kamel]

Series plan of record (supersedes the single-PR approach in #552)

The OCI Generative AI provider lands as a four-part stacked series, foundation-ordered, each layer independently green (full workspace build + tests + clippy + fmt on every branch), with the combined content byte-identical to the end-to-end-validated original from #552 (encoded payload posted verbatim to the signed OCI REST endpoint, answered by a dedicated AI cluster serving an imported NVIDIA Nemotron 3 model).

#	PR	Content	Size
1	#554 (open, ready for review)	Typed ApiSpecific OCI variants + LlmResponseCodec (GENERIC + COHERE ChatResult decode, usage/finish-reason normalization)	+562
2	feat/oci-codec-2-request-codec — opens when #554 merges	LlmCodec decode + merge-not-replace encode with encode(decode(x), x) == x identity and unmodeled-field preservation	+1,414
3	feat/oci-codec-3-resolver-surfaces	Provider-surface registration + detection, OCIGenAIStreamingCodec (descriptor requires it at this layer), guardrails / PII-redaction overlay / adaptive support, parity cases	+1,060
4	feat/oci-codec-4-bindings — closes this issue	PyOCIGenAIChatCodec (pyo3), napi class, .pyi/codecs.py exposure, binding tests	+202

Each later branch is a pure superset of the previous, so review comments on layer N apply cleanly to N+1..4 via rebase.

[willkill07]

@fede-kamel this issue doesn't match the prescribed template format for enhancements. Could you please update accordingly?

[fede-kamel]

Reformatted to the enhancement template — thanks for the pointer @willkill07. The issue now carries the template's sections (affected area, problem, proposal, runtime contract and binding impact, alternatives considered, acceptance criteria), the [Enhancement]: title prefix, and the Enhancement issue type. Content is updated to the current plan of record: the response codec in #554 covers GENERIC, COHERE, and COHEREV2 (the last now confirmed against the live service), and the alternatives section records the approaches reviewed and closed in #549/#552 plus why a native Cohere codec or the OpenAI codec can't decode Oracle's wire formats.