st-orcarouter

A Streamlit connection for OrcaRouter — an OpenAI-compatible LLM gateway with adaptive routing across 150+ models behind a single API key.

st-orcarouter plugs OrcaRouter into Streamlit's st.connection mechanism, so you get cached completions, token-by-token streaming for st.write_stream, and a model list for st.selectbox without writing the client wiring yourself.

I'm an engineer on the OrcaRouter team.

Install

pip install st-orcarouter

Quickstart

1. Get an API key at https://www.orcarouter.ai/console.

2. Add it to .streamlit/secrets.toml:

   [connections.orcarouter]
   api_key = "sk-orca-xxxx"

3. Use it in your app:

   import streamlit as st
   from st_orcarouter import OrcaRouterConnection
   
   conn = st.connection("orcarouter", type=OrcaRouterConnection)
   
   # One-shot, cached completion
   st.write(conn.chat("Tell me a joke", ttl=3600))
   
   # Streaming straight into the chat UI
   st.write_stream(conn.stream("Write a short story"))
   
   # Populate a model picker with the models your key can use
   model = st.selectbox("Model", conn.list_models())

4. Run it:

   streamlit run your_app.py

A complete chatbot is in examples/streamlit_app.py.

API

st.connection("name", type=OrcaRouterConnection) returns an OrcaRouterConnection with:

Member	Description
chat(messages, *, model="orcarouter/auto", ttl=None, **params) -> str	Non-streaming completion, cached with st.cache_data. messages may be a prompt string or a list of chat messages.
stream(messages, *, model="orcarouter/auto", **params) -> Iterator[str]	Streaming completion that yields content deltas; pass it to st.write_stream.
list_models(*, ttl=3600, chat_only=True) -> list[str]	Model ids your key can use (via GET /v1/models), cached. chat_only filters out image / audio / embedding models.
client	The underlying authenticated openai.OpenAI client for anything not wrapped here (embeddings, tool calls, etc.).

Extra keyword arguments (temperature, reasoning_effort, extra_body, ...) are forwarded to the OpenAI chat-completions call unchanged.

Models and routing

• orcarouter/auto is OrcaRouter's adaptive router — it picks an upstream per request based on your console routing strategy. See https://www.orcarouter.ai/console/routing.
• Use fully qualified ids for a specific model, e.g. openai/gpt-5, anthropic/claude-opus-4.7.
• Browse the full catalog at https://www.orcarouter.ai/models.

Configuration

The API key is resolved in this order:

1. st.connection(..., api_key="sk-orca-...")
2. the [connections.<name>] section of .streamlit/secrets.toml
3. the ORCAROUTER_API_KEY environment variable

You can override the base URL with base_url= or a base_url secret (defaults to https://api.orcarouter.ai/v1).

Development

pip install -e ".[dev]"
pytest                              # offline unit tests, no API key needed
python scripts/orcarouter-live-test.py   # live end-to-end check (needs a real key)

License

Apache-2.0.