Run route code unchanged against a shared gateway, and follow LiteLLM's Router

.github/workflows/local-gateway-build.yml

@@ -14,7 +14,7 @@ jobs:
       fail-fast: false
       matrix:
         include:
-          - os: macos-14
+          - os: macos-15
             target: darwin-arm64
           - os: ubuntu-24.04
             target: linux-amd64

CHANGELOG.md

@@ -2,6 +2,22 @@
 
 All notable changes to this project are listed here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and versions follow [Semantic Versioning](https://semver.org/).
 
+## Unreleased
+
+### Changed
+
+- `route()` and LiteLLM's `Router` no longer raise when `AGENTROUTER_BASE_URL` is set. They set up nothing on this computer; calls send the route's name to that gateway, whose route of the same name applies, and the SDK warns once. A `Router` sends the group name, such as `chat`. Code tested against the local gateway now runs unchanged against a shared one.
+- `local_gateway.export_config()` names each LiteLLM `Router` route after its group, `chat` instead of `router-1a2b3c4d:chat`, so a gateway made from it answers to the names code calls. Two routes that would get the same name raise `BadRequestError`.
+
+### Fixed
+
+- LiteLLM's `Router` shares a group's traffic by `weight`, `rpm` or `tpm` in `litellm_params`, as LiteLLM's default `simple-shuffle` does, instead of equally. A deployment without the setting, beside one with it, gets no traffic, as in LiteLLM. Weights that are not whole numbers keep their ratio: `0.01` and `100` become 1 and 10000.
+- LiteLLM's `Router` warns about what it does not use instead of dropping it without a word: a `routing_strategy` other than `simple-shuffle` (LiteLLM's usage-based strategies hold deployments to `rpm` and `tpm`), and `litellm_params` such as `api_base` and `timeout`.
+- A call with `api_base=` warns once per address that it goes there in place of the gateway, so routes, fallbacks and limits apply only if that address is an Agent Router gateway that has them.
+- A LiteLLM `Router` chooses its route name at each call, so one made before `AGENTROUTER_BASE_URL` was set sends the group name to that gateway, and one made with it set sets up its route on this computer once it is unset. Before, it kept the name it was made with and the gateway answered `NotFoundError`.
+- When a gateway set with `AGENTROUTER_BASE_URL` answers 403 in plain text, the `PermissionDeniedError` message names `AGENTROUTER_API_KEY` as a possible cause, which an exported gateway needs set to the same value on its server. It says "possible" because the gateway gives the same answer for other access rules, such as an address allowlist. Before, it carried only the gateway's own refusal text. The message no longer starts with the provider's name, since the gateway refused it, not the provider.
+- A call to a model without a known price, such as a route name on a gateway you set, reports `response_cost` as `None` and warns nothing, as LiteLLM does. Before, every such call warned "No price is known" and reported $0. `completion_cost()` called directly still warns and returns 0.
+
 ## [0.1.0a11] - 2026-11-03
 
 ### Added

README.md

@@ -96,7 +96,7 @@ Here a call to `chat` goes to `openai/gpt-5.4`. If it cannot be reached or answe
 
 - A route can also limit requests: `limit=ar.Limit(60, per="minute")` lets 60 requests a minute through and answers the rest with a `RateLimitError`.
 - Routes are saved in `~/.agentrouter/routes.json`, so every program on this computer can use them.
-- Routes work with the local gateway only. With `AGENTROUTER_BASE_URL` set, set up fallbacks on that gateway instead.
+- Routes defined in code set up the local gateway. With `AGENTROUTER_BASE_URL` set, the same code runs, and that gateway's route of the same name applies; `local_gateway.export_config()` gives it this computer's routes.
 - `Router` from `theagentrouter.compat.litellm`, with LiteLLM's `fallbacks` and `num_retries`, is turned into routes.
 
 Details: [Routes](docs/usage.md#routes-fallbacks-retries-and-names-of-your-own).

apps/litellm-assistant/tests/test_litellm_behaviour.py

@@ -456,6 +456,13 @@ def test_success_callback_runs_after_a_stream(base):
     assert [type(r).__name__ for r in seen if type(r).__name__ == "ModelResponse"] == ["ModelResponse"]
 
 
+def test_router_does_not_hold_a_deployment_to_rpm_by_default():
+    # LiteLLM's default simple-shuffle uses rpm as a traffic share; only its usage-based strategies limit by it.
+    router = litellm.Router(model_list=[{"model_name": "chat", "litellm_params": {"model": OPENAI, "rpm": 1}}])
+    replies = [router.completion(model="chat", messages=HELLO, mock_response="ok") for _ in range(3)]
+    assert [r.choices[0].message.content for r in replies] == ["ok"] * 3
+
+
 def test_mock_response():
     resp = litellm.completion(model=OPENAI, messages=HELLO, mock_response="Offline reply")
     assert resp.choices[0].message.content == "Offline reply"

docs/coming-from-litellm.md

@@ -32,7 +32,8 @@ A full example: [`examples/from_litellm/change_the_import.py`](../examples/from_
 | `stream=True` | Same chunks, with `usage` on the last one |
 | `tools`, `tool_choice`, `parallel_tool_calls` | OpenAI-shaped inputs, translated for Anthropic too; `parallel_tool_calls=False` disables parallel Anthropic tool calls |
 | `response_format`, `temperature`, `max_tokens` and the other OpenAI parameters | Passed through for OpenAI Chat Completions. Unsupported Anthropic options raise `UnsupportedParamsError` ([the full list](usage.md#parameters)) |
-| `api_base`, `timeout`, `extra_headers`, `extra_body` | Same |
+| `api_base` | Same: the call goes to that address in place of the gateway, so routes, fallbacks and limits apply only if it is an Agent Router gateway that has them. Warns once per address. Remove an `api_base` that points at a provider to call through the gateway |
+| `timeout`, `extra_headers`, `extra_body` | Same |
 | `num_retries`, `max_retries` | Bounded request retries before response/stream output; a partially consumed completion stream is never replayed. Default is zero. Route retries remain gateway settings |
 | Module `api_base`, `api_key`, `request_timeout`, `num_retries` | Defaults for completion, embeddings and Responses calls; per-call values win. `modify_params=True` warns because message rewriting is unsupported |
 | `api_key` | Used with `api_base`, or as the key of a gateway you set. With the local gateway it is ignored with a warning: the gateway reads provider keys from the environment ([Models and keys](usage.md#models-and-keys)) |
@@ -48,7 +49,7 @@ A full example: [`examples/from_litellm/change_the_import.py`](../examples/from_
 | `drop_params`, `additional_drop_params` | Same, on the call or on the module, off by default: with it, Claude drops the parameters it does not take (`seed`, `presence_penalty` and others, and on newer models `temperature` and `top_p`) instead of raising, and a forced tool call becomes `auto` on the models that refuse one ([the full list](usage.md#parameters)). `n=1` is always fine |
 | `set_verbose`, `suppress_debug_info` | Accepted |
 | Response types under `litellm.types.utils` (`ModelResponse`, `Choices`, `Message`, `Usage` and others) | Same names |
-| `Router` | For `model_list`, `fallbacks` and `num_retries`: the groups become routes in the local gateway, which does the retries and fallbacks. Each route is named after its group and its definition (`router-1a2b3c4d:chat`), so programs with a group of the same name keep their own routes, and a group named after a model does not take over that model in other programs. Call the Router's methods with the group name. A new definition of a group removes the route an earlier one left, and a program still using that route adds it back on its next call. Any change restarts the shared gateway on the next call. Deployments with one `model_name` share its traffic; keys come from the environment. Other `Router` options warn and are ignored. See [Routes](usage.md#routes-fallbacks-retries-and-names-of-your-own) |
+| `Router` | For `model_list`, `fallbacks` and `num_retries`: the groups become routes in the local gateway, which does the retries and fallbacks. Each route is named after its group and its definition (`router-1a2b3c4d:chat`), so programs with a group of the same name keep their own routes, and a group named after a model does not take over that model in other programs. Call the Router's methods with the group name. A new definition of a group removes the route an earlier one left, and a program still using that route adds it back on its next call. Any change restarts the shared gateway on the next call. Deployments with one `model_name` share its traffic as LiteLLM's default `simple-shuffle` shares it: by `weight`, `rpm` or `tpm` in `litellm_params`, the first that any deployment sets, and equally when none does. Keys come from the environment. Other `Router` options warn and are ignored, and so do other `litellm_params`, such as `api_base` (see below). With `AGENTROUTER_BASE_URL` set, nothing is set up on this computer: calls send the group name, `chat`, and that gateway's route of that name applies. `export_config()` names each group's route `chat` too. See [Routes](usage.md#routes-fallbacks-retries-and-names-of-your-own) |
 
 ## What is missing
 
@@ -73,6 +74,14 @@ These `completion` arguments work in LiteLLM but do nothing in theagentrouter. C
 | `custom_llm_provider` | Name the provider in the model, for example `openai/gpt-5.4-mini` |
 | `api_version` | Set `AZURE_API_VERSION` where the gateway starts |
 
+`Router` warns in the same way, once for each `Router`, for what it does not use:
+
+| `Router` setting | Instead |
+|---|---|
+| `routing_strategy` other than `simple-shuffle` | The gateway shares a group's traffic as `simple-shuffle` does. LiteLLM's usage-based strategies also hold each deployment to its `rpm` and `tpm`; with the default strategy LiteLLM does not, and neither does the SDK. For a request limit, define a route with `limit=Limit(...)`; it counts requests per route, not tokens |
+| `api_base`, `api_version` in `litellm_params` | Set the provider's address where the gateway starts, such as `AZURE_API_BASE`. `os.environ/AZURE_API_BASE` and `os.environ/AZURE_API_VERSION` do not warn |
+| Other `litellm_params`, such as `timeout` | Not used |
+
 A value that asks for nothing does not warn. Other unknown completion arguments warn instead of disappearing silently.
 
 ## Other differences

docs/local-gateway.md

@@ -22,7 +22,7 @@ The macOS arm64 wheel is about 80 MB compressed (the program itself is 281 MB).
 
 `routing.resolve`, first match wins:
 
-1. `api_base` on the call: that endpoint, in the provider's own format (LiteLLM's argument).
+1. `api_base` on the call: that endpoint, in the provider's own format (LiteLLM's argument). It skips the gateway, so it warns once per address.
 2. A gateway set with `AGENTROUTER_BASE_URL` (or `config.json`).
 3. A route name: the local gateway's route for it. The running gateway is used only if it has this computer's current routes (`routes_fingerprint` in `state.json`); otherwise it restarts. With `AGENTROUTER_AUTOSTART=0`, a running gateway that already has the route is still used.
 4. Anything else: the running local gateway, if it has a rule for this name and the name was not a route when it started (`local_gateway.serves`). Otherwise, if the package is installed and `AGENTROUTER_AUTOSTART` is not `0`, the gateway is started with this model, or restarted with a rule for it, as LiteLLM sends any name to the provider. With no package or no autostart, an error says to set `AGENTROUTER_BASE_URL`. An `api_key=` here cannot reach the provider (the gateway holds the keys), so it warns once.