feat(gateway): add InvocationTarget routing to /v1/chat/completions by ashum9 · Pull Request #1647 · mofa-org/mofa

ashum9 · 2026-04-20T05:01:53Z

Summary

This PR replaces the hard-coded mock response ("Hello from MoFA gateway!") in the /v1/chat/completions endpoint with a robust, kernel-level InvocationTarget routing layer.

By introducing the InvocationRouter, the gateway now resolves requests through the AgentRegistry first, providing a seamless "Agentic OS" experience where registered agents can overshadow or complement local model inference.

Architecture

The request flow is now explicitly decoupled from the HTTP handler:

POST /v1/chat/completions { "model": "..." }
          │
          ▼
   InvocationRouter::resolve() 
          │
    ┌─────┴────────────────────────┐
    │ Is model a registered Agent? │
    └─────┬──────────────────┬─────┘
          │ yes              │ no (fallback)
          ▼                  ▼
  InvocationTarget::Agent   InvocationTarget::LocalInference
          │                  │
          ▼                  ▼
  [Registry Metadata]      InferenceOrchestrator

Replace the placeholder 'Hello from MoFA gateway!' mock response with real InvocationTarget-based dispatch through the AgentRegistry. ## What changed ### mofa-kernel: new InvocationTarget type - Add `InvocationTarget` enum to `mofa-kernel/src/gateway/types.rs` with three variants: Agent, LocalInference, Proxy - `#[non_exhaustive]` for forward compatibility - `Display` and `label()` helpers for logs and metrics - Re-export from `mofa-kernel::gateway` ### mofa-gateway: InvocationRouter - New `handlers/invocation.rs`: `InvocationRouter` struct wraps `AgentRegistry` + `InferenceBridge` and provides: - `resolve(model) -> InvocationTarget`: checks registry first, falls back to LocalInference if no agent matches - `dispatch(target, request) -> ChatCompletionResponse`: executes the resolved target - Agent dispatch: returns agent metadata (description/name) confirming the routing decision; full agent invocation is a follow-up task - LocalInference dispatch: delegates to existing InferenceBridge - Proxy dispatch: returns Internal error (handled by proxy/ module) - Wildcard arm for future non_exhaustive variants ### mofa-gateway: openai handler - Remove mock handler from `handlers/openai.rs` - Wire `InvocationRouter via `axum::Extension` - Resolve and dispatch on every request with structured tracing logs ### mofa-gateway: GatewayServer - Inject `InvocationRouter` Extension in both socketio and non-socketio build paths in `server.rs` - OpenAI router is now always added (no longer optional); uses `OrchestratorConfig::default()` when no explicit config is provided ## Tests (5 new, all pass) - `resolve_known_agent_returns_agent_target` - `resolve_unknown_model_returns_local_inference` - `dispatch_local_inference_returns_valid_response` - `dispatch_agent_returns_valid_response` - `dispatch_proxy_returns_error` Fixes: #TODO (open issue for mock /v1/chat/completions)

ashum9 · 2026-04-20T05:02:16Z

@lijingrs @yangrudan kindly review this PR :)

Copilot

Pull request overview

Adds a kernel-level InvocationTarget type and wires the gateway’s OpenAI /v1/chat/completions endpoint through a new InvocationRouter that resolves requests via the AgentRegistry first, then falls back to local inference.

Changes:

Introduce InvocationTarget (Agent / LocalInference / Proxy) in mofa-kernel and re-export it from the gateway module.
Add InvocationRouter in mofa-gateway to resolve + dispatch chat-completion requests based on registry contents.
Update the gateway server + OpenAI handler to inject and use InvocationRouter instead of a hard-coded mock response.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
crates/mofa-kernel/src/gateway/types.rs	Adds the `InvocationTarget` routing enum plus helpers (`label`, `Display`).
crates/mofa-kernel/src/gateway/mod.rs	Re-exports `InvocationTarget` and documents it in the module-level table.
crates/mofa-gateway/src/server.rs	Always mounts OpenAI routes and injects `InvocationRouter` (built from registry + inference bridge).
crates/mofa-gateway/src/handlers/openai.rs	Routes `/v1/chat/completions` through `InvocationRouter` and updates tests.
crates/mofa-gateway/src/handlers/mod.rs	Exposes the new `invocation` module and `InvocationRouter` export.
crates/mofa-gateway/src/handlers/invocation.rs	Implements resolve/dispatch logic and adds unit tests for routing/dispatch.

Copilot · 2026-04-20T05:07:14Z

+        // Always add OpenAI router with InvocationTarget routing.
+        // InvocationRouter checks the registry first; if no agent matches
+        // it falls back to InferenceBridge → InferenceOrchestrator.
+        let bridge_config = self
+            .orchestrator_config
+            .clone()
+            .unwrap_or_default();
+        let bridge = Arc::new(InferenceBridge::new(bridge_config));
+        let invocation_router = Arc::new(InvocationRouter::new(
+            self.registry.clone(),
+            bridge,
+        ));
+        router = router
+            .merge(openai_router())
+            .layer(axum::Extension(invocation_router));


Same concern as the Socket.IO branch: this always initializes an InferenceBridge (and mounts /v1/chat/completions) even when orchestrator_config is None, which changes startup behavior and may do unnecessary hardware detection. Consider gating LocalInference behind explicit configuration and returning GatewayError::NotConfigured("inference") when inference isn’t enabled.

Copilot · 2026-04-20T05:07:14Z

+//! that route requests through [`InvocationTarget`] dispatch:
+//!
+//! 1. **Resolve** — check the [`AgentRegistry`] for a matching agent.
+//! 2. **Dispatch** — route to the registered agent OR fall back to the
+//!    [`InferenceOrchestrator`] when no agent is found.


The module docs use intra-doc links like [InvocationTarget], [AgentRegistry], and [InferenceOrchestrator] but those names aren’t in scope in this module. This can produce broken intra-doc link warnings. Prefer fully-qualified paths (e.g. [mofa_kernel::gateway::InvocationTarget]) or import the types solely for doc-link resolution.

Copilot · 2026-04-20T05:07:15Z

+    #[tokio::test]
+    async fn test_chat_completions_unknown_model_falls_back_to_inference() {
+        let router_ext = make_router_ext();
+        // No agent registered — should fall through to LocalInference
+        let result = chat_completions(
+            router_ext,
+            HeaderMap::new(),
+            Json(make_request("gpt-4")),
+        )
+        .await;
+        assert!(result.is_ok());


This test only asserts result.is_ok() and would still pass if the handler returned a malformed OpenAI response (or routed to the wrong target). Since the handler now does resolution + dispatch, it would be more robust to also assert key response fields (e.g. object, id prefix, choices[0].message.role) and add a second test case where an agent is registered and the response reflects the agent route.

Copilot · 2026-04-20T05:07:15Z

+//!
+//! [`InvocationRouter`] resolves an incoming OpenAI-compatible request to an
+//! [`InvocationTarget`] by consulting the [`AgentRegistry`] first, then
+//! falling back to the local [`InferenceOrchestrator`] when no registered


The top-level docs reference [InferenceOrchestrator], but that type isn’t in scope in this module, so the intra-doc link may be broken. Consider using a fully-qualified path (e.g. mofa_foundation::inference::InferenceOrchestrator) or importing it for documentation purposes.

Suggested change

//! falling back to the local [`InferenceOrchestrator`] when no registered

//! falling back to the local

//! [`mofa_foundation::inference::InferenceOrchestrator`] when no registered

Copilot · 2026-04-20T05:07:15Z

+            InvocationTarget::Proxy { ref url } => {
+                // Proxy dispatch is handled by `crates/mofa-gateway/src/proxy/`.
+                Err(GatewayError::Internal(format!(
+                    "Proxy dispatch to '{}' is not supported through InvocationRouter; \
+                     use the dedicated proxy module instead.",
+                    url
+                )))


InvocationTarget::Proxy currently returns GatewayError::Internal(...). Since GatewayError::Internal maps to HTTP 500, this will look like a server bug rather than an unsupported/configuration state. Consider returning GatewayError::NotConfigured("proxy") (501) or GatewayError::InvalidRequest(...) (400) so clients get an accurate status code if this variant becomes reachable via routing config.

Suggested change

InvocationTarget::Proxy { ref url } => {

// Proxy dispatch is handled by `crates/mofa-gateway/src/proxy/`.

Err(GatewayError::Internal(format!(

"Proxy dispatch to '{}' is not supported through InvocationRouter; \

use the dedicated proxy module instead.",

url

)))

InvocationTarget::Proxy { .. } => {

// Proxy dispatch is handled by `crates/mofa-gateway/src/proxy/`.

Err(GatewayError::NotConfigured("proxy"))

Copilot · 2026-04-20T05:07:15Z

+                Ok(ChatCompletionResponse {
+                    id: format!("chatcmpl-{}", uuid::Uuid::new_v4()),
+                    object_type: "chat.completion".to_string(),
+                    created: std::time::SystemTime::now()
+                        .duration_since(std::time::UNIX_EPOCH)


The Agent dispatch path manually constructs ChatCompletionResponse (id/object/created/usage). Similar response-building logic already exists in inference_bridge (e.g., convert_to_openai_response) and elsewhere; duplicating it increases the risk of response-shape drift over time. Consider extracting a shared helper (e.g., build_chat_completion_response(model, content, prompt_tokens, completion_tokens)) and reusing it here and in the bridge.

Copilot · 2026-04-20T05:07:16Z

+            InvocationTarget::Agent { agent_id } =>
+                write!(f, "Agent({})", agent_id),
+            InvocationTarget::LocalInference { model } =>
+                write!(f, "LocalInference({})", model),
+            InvocationTarget::Proxy { url } =>
+                write!(f, "Proxy({})", url),


InvocationTarget::Proxy is formatted with the full url in Display. Since InvocationTarget is used in logs/metrics, this risks leaking sensitive URL components (e.g., embedded credentials, internal hostnames). Consider redacting the URL (log only scheme/host, or a hash), or make Display omit variant payloads and use explicit structured fields when needed.

Suggested change

InvocationTarget::Agent { agent_id } =>

write!(f, "Agent({})", agent_id),

InvocationTarget::LocalInference { model } =>

write!(f, "LocalInference({})", model),

InvocationTarget::Proxy { url } =>

write!(f, "Proxy({})", url),

InvocationTarget::Agent { .. } =>

write!(f, "Agent"),

InvocationTarget::LocalInference { .. } =>

write!(f, "LocalInference"),

InvocationTarget::Proxy { .. } =>

write!(f, "Proxy"),

Copilot · 2026-04-20T05:07:16Z

+            // Always add OpenAI router with InvocationTarget routing.
+            // InvocationRouter checks the registry first; if no agent matches
+            // it falls back to InferenceBridge → InferenceOrchestrator.
+            let bridge_config = self
+                .orchestrator_config
+                .clone()
+                .unwrap_or_default();
+            let bridge = Arc::new(InferenceBridge::new(bridge_config));
+            let invocation_router = Arc::new(InvocationRouter::new(
+                self.registry.clone(),
+                bridge,
+            ));
+            router = router
+                .merge(openai_router())
+                .layer(axum::Extension(invocation_router));


GatewayServer::new(...) previously did not mount the OpenAI routes unless orchestrator_config was provided. This change always constructs an InferenceBridge with OrchestratorConfig::default(), which triggers hardware detection in InferenceOrchestrator::new and changes the exposed API surface (the endpoint now exists even when inference wasn’t configured). If the intent is “agents-only unless inference is enabled”, consider keeping the router mounted but making InvocationRouter hold an Option<InferenceBridge> and returning GatewayError::NotConfigured("inference") for LocalInference when orchestrator_config is None, to avoid surprising startup cost/behavior.

Suggested change

// Always add OpenAI router with InvocationTarget routing.

// InvocationRouter checks the registry first; if no agent matches

// it falls back to InferenceBridge → InferenceOrchestrator.

let bridge_config = self

.orchestrator_config

.clone()

.unwrap_or_default();

let bridge = Arc::new(InferenceBridge::new(bridge_config));

let invocation_router = Arc::new(InvocationRouter::new(

self.registry.clone(),

bridge,

));

router = router

.merge(openai_router())

.layer(axum::Extension(invocation_router));

// Only add OpenAI routing when inference is explicitly configured.

// This preserves the previous agents-only behavior when no

// orchestrator configuration is provided and avoids unexpected

// inference initialization side effects during startup.

if let Some(bridge_config) = self.orchestrator_config.clone() {

let bridge = Arc::new(InferenceBridge::new(bridge_config));

let invocation_router = Arc::new(InvocationRouter::new(

self.registry.clone(),

bridge,

));

router = router

.merge(openai_router())

.layer(axum::Extension(invocation_router));

}

ashum9 · 2026-04-20T17:48:49Z

@BH3GEI kindly review this PR sir.

Copilot AI review requested due to automatic review settings April 20, 2026 05:01

Copilot started reviewing on behalf of ashum9 April 20, 2026 05:02 View session

Copilot AI reviewed Apr 20, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat(gateway): add InvocationTarget routing to /v1/chat/completions - #1647

feat(gateway): add InvocationTarget routing to /v1/chat/completions#1647
ashum9 wants to merge 1 commit into
mofa-org:mainfrom
ashum9:ashum9/feat/gateway-invocation-target-routing

ashum9 commented Apr 20, 2026

Uh oh!

ashum9 commented Apr 20, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 20, 2026

Uh oh!

Copilot AI Apr 20, 2026

Uh oh!

Copilot AI Apr 20, 2026

Uh oh!

Copilot AI Apr 20, 2026

Uh oh!

Copilot AI Apr 20, 2026

Uh oh!

Copilot AI Apr 20, 2026

Uh oh!

Copilot AI Apr 20, 2026

Uh oh!

Copilot AI Apr 20, 2026

Uh oh!

ashum9 commented Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	//! falling back to the local [`InferenceOrchestrator`] when no registered
	//! falling back to the local
	//! [`mofa_foundation::inference::InferenceOrchestrator`] when no registered

Uh oh!

Conversation

ashum9 commented Apr 20, 2026

Summary

Architecture

Uh oh!

ashum9 commented Apr 20, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 20, 2026

Choose a reason for hiding this comment

Uh oh!

ashum9 commented Apr 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants