pin-relay

One upstream model, many advertised names.

pin-relay puts a local, OpenAI-compatible and Ollama-compatible and WebSocket-speaking front end in front of a hosted inference API. Production callers keep sending the model names they already send — muse-local:latest, GLM-4-32B, llama3:8b — and the relay resolves every one of them to the real upstream model, injecting the API key server-side.

Zero dependencies. Node >= 24. Stdlib only, including a hand-rolled RFC 6455 WebSocket server.

Why

pin-clientd and the rest of the stack advertise and serve models by name. A hosted provider names them its own way (gpuai/glm-5.3-flash). Changing every caller to match the vendor is how a stack acquires a vendor. This keeps the names ours.

Run

export GPUAI_API_KEY=...            # never printed, never logged
cp pin-relay.config.example.json pin-relay.config.json
node bin/pin-relay.mjs --config pin-relay.config.json

curl -s localhost:11435/health | jq
curl -s localhost:11435/v1/models | jq '.data[].id'

curl -N localhost:11435/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{"model":"GLM-4-32B","messages":[{"role":"user","content":"Hello"}],"stream":true}'

No key is needed in any client request — the relay holds it.

Surfaces

Route	Dialect	Notes
GET /health	—	upstream reachability, alias table, deadlines, counters
GET /v1/models, GET /v1/models/:id	OpenAI	advertises aliases only
POST /v1/chat/completions	OpenAI	JSON or SSE
GET /api/tags, /api/ps, /api/version, POST /api/show	Ollama
POST /api/chat	Ollama	JSON or NDJSON stream
WS /ws	JSON envelope	{type, id, payload}
POST /api/generate, /v1/embeddings	—	501 with a reason. Not silently faked.

WebSocket envelope

-> { "type":"chat", "id":"a1", "payload": { "model":"GLM-4-32B", "messages":[...] } }
<- { "type":"chunk", "id":"a1", "delta": { "reasoning":"..." } }
<- { "type":"chunk", "id":"a1", "delta": { "content":"OK" } }
<- { "type":"done",  "id":"a1", "usage": {...}, "telemetry": {...} }

-> { "type":"models" }          -> { "type":"cancel","id":"a1" }   -> { "type":"ping" }

Control frames (ping, cancel) are answered immediately and never queue behind inference output.

Config

Aliases are data. Repoint the upstream without touching code:

{
  "upstream": { "base_url": "https://api.gpu.ai/v1", "key_env": "GPUAI_API_KEY",
                "model": "gpuai/glm-5.3-flash" },
  "aliases": ["GLM-4-32B", "muse-local:latest", "llama3:8b"]
}

An alias may be an object to override per name:

{ "name": "big", "upstream_model": "gpuai/glm-5.3", "max_tokens": 16000 }

Env overrides: RELAY_PORT, RELAY_HOST, RELAY_UPSTREAM_BASE_URL, RELAY_UPSTREAM_MODEL, RELAY_KEY_ENV, RELAY_ALIASES (comma-separated), RELAY_LOG_LEVEL, RELAY_CONFIG.

Alias matching is case-insensitive. Real callers in this stack send both GLM-4-32B and GLM-4-32b; a compatibility shim that 404s on a capitalization is not compatible. Two aliases differing only in case are rejected at startup.

Behaviour worth knowing

These are measured facts about the upstream, not guesses.

Thinking models answer late. glm-5.3-flash spent 37 of 40 completion tokens in reasoning_content to answer "OK" — 13 reasoning deltas, then a single content delta. Consequences the relay handles for you:

• max_tokens is the total budget. Omit it and the relay applies defaults.max_tokens (8192), because the upstream's own default is small enough to return HTTP 200 with empty content.
• Set it explicitly and it is honoured exactly — never raised, never removed. Below reasoning_floor you get a loud warning, not a rewrite.
• A response with no content is counted and logged with the finish_reason and usage, because a silent empty answer is indistinguishable from a bug.
• Reasoning is never merged into content. Ollama clients get it as message.thinking; WebSocket clients get delta.reasoning.

Stream deadlines are inactivity-only. inactivity_ms resets on every arriving chunk — reasoning deltas included, since they are the only sign of life for the first several seconds. There is no wall-clock cap on a generation. first_byte_ms separately covers connect + TLS + headers.

Failures name themselves. No empty catch, no silent skip. HTTP status, upstream body, and the failing alias are all reported; transport failures say that DNS, TLS, proxy refusal and a client disconnect are the candidate causes rather than picking one. If a stream dies after headers are sent, the relay emits an error frame in-band and then [DONE] — ending quietly would look like a complete, empty answer.

Secrets never reach a log. Redaction runs over every serialized log line in the single writer, so a key cannot leak via a nested field. Query strings are treated as a log surface (WebSocket upgrades can't set headers).

Tests

npm test        # node --test test/

Tests run against a stub upstream that reproduces the real frame shapes, including the terminal usage frame with an empty choices array and a delta carrying "reasoning_content": null alongside real content. The stub is deliberately dumber than production — a double more capable than the real dependency proves round trips that cannot happen.

Status

Written 2026-09-12 against probed behaviour of api.gpu.ai. Not yet exercised against a live pin-clientd. Verification is the operator's gate.

---

© Interchained LLC. BUSL-1.1 — see LICENSE.