[Epic]: Upgrade Switchyard to production-ready native and library integration

[bbednarski9]

Outcome

Deliver a production-ready Switchyard integration for NeMo Relay that:

• Works through native Relay runtime and plugin-host lifecycles, including Hermes, LangChain Deep Agents, and future native harnesses, without requiring the CLI wrapper as the integration boundary.
• Can be discovered, configured, and executed by the published Python package.
• Is available as a supported, published Rust crate or clearly defined crate split.
• Supports an in-process Switchyard DecisionProvider without requiring a separately managed sidecar.
• Retains the HTTP Decision API as a supported compatibility and deployment mode.

Motivation

The current experimental nemo-relay-switchyard integration is source-only, CLI feature-gated, and uses the Switchyard Decision API over HTTP. This is sufficient for the existing sidecar workflow, but it does not support native Relay harness integrations or provide a stable consumer-facing package.

The Python wheel also cannot currently configure the plugin: the native extension neither depends on nor registers nemo-relay-switchyard, so ComponentSpec(kind="switchyard", ...) is treated as an unknown component. Publishing the Rust crate alone does not solve this; Python support also requires native-extension wiring, component registration, configuration coverage, and runtime documentation.

As Switchyard evolves toward a library-first architecture, Relay should be able to use its decision and accumulation components in-process while preserving service-based deployment compatibility.

Scope

Included:

• Define the native host/plugin contract for configuration, lifecycle, capabilities, and error reporting.
• Register and activate the plugin through native and worker/plugin-host lifecycles.
• Support Hermes, LangChain Deep Agents, and future native harnesses without using nemo-relay-cli as the integration boundary.
• Preserve request and stream intercept behavior, target binding, retries, fallbacks, optimization accounting, and observability semantics.
• Wire the Switchyard component into the Python native extension and expose and test its configuration path.
• Define the public API, ownership boundary, feature flags, compatibility policy, dependency strategy, documentation, and release CI for a publishable Rust crate or supported crate split.
• Keep the integration optional so users do not incur Switchyard dependencies unless they enable it.
• Add an in-process Switchyard DecisionProvider behind a Relay-owned abstraction.
• Use Switchyard-owned ATOF accumulation, snapshots, decision logic, and provider-protocol translation where appropriate.
• Preserve a service/HTTP adapter during migration.
• Define snapshot freshness, lifecycle, concurrency, and failure semantics for embedded accumulators and decision providers.
• Keep the versioned request/decision contract compatible across service and library modes.
• Add supported examples that do not depend on internal services, credentials, or untracked development worktrees.

Current limitations to address:

• The crate has publish = false and must be built from a Relay source checkout.
• The CLI feature is the supported activation path; native harness integrations are not covered.
• Decision API routing and ATOF-backed profiles require a reachable Switchyard service.
• switchyard-translation is library-backed, but decision logic and ATOF accumulation remain service-bound.
• Existing examples are experimental compatibility and end-to-end workflows rather than a production deployment contract.

Non-goals

• Removing the HTTP Decision API or sidecar deployment mode during the migration.
• Making Switchyard dependencies mandatory for Relay users who do not enable the integration.
• Treating Rust crate publication as a substitute for Python native-extension integration.
• Depending on internal-only services, credentials, or development worktrees in supported examples.

Child work

1. Document and implement the native host/plugin lifecycle contract.
2. Add native-harness registration and integration tests.
3. Wire the Switchyard component into the Python native extension, expose the configuration path, and add package-level tests.
4. Define and stabilize the publishable crate boundary and feature/dependency policy.
5. Add an in-process Switchyard DecisionProvider behind a Relay-owned abstraction.
6. Add service and in-process conformance tests against the same request/decision fixtures.
7. Add snapshot freshness, embedded accumulator lifecycle, concurrency, and failure-mode tests.
8. Update user-facing documentation and examples once the supported deployment model is established.

Related work:

• Relay Switchyard plugin: feat(plugin)!: Add Switchyard Decision API routing plugin #369
• Switchyard integration branch

Implementation should be split into independently reviewable pull requests.

Completion criteria

• Hermes and LangChain Deep Agents can activate and use Switchyard routing through native integration paths without the CLI wrapper.
• The published Python package can discover, configure, and execute the Switchyard component through its native/plugin API.
• A supported published Rust crate or package can be consumed without a Relay source checkout.
• Relay can run against an in-process Switchyard DecisionProvider with no mandatory Switchyard sidecar.
• The HTTP Decision API remains available as a compatibility and deployment mode during migration.
• Service and library modes produce equivalent routing, translation, retry and fallback, optimization, and observability results for the same fixtures.
• ATOF-backed routing has documented snapshot freshness, lifecycle, concurrency, and failure behavior.
• Supported examples require no internal-only services, credentials, or development worktrees.
• Production readiness is documented with versioning, compatibility, performance, security, and operational guidance.