Skip to content

feat: add google-vertex-openai provider for Vertex AI partner models#716

Open
bluet wants to merge 2 commits intoanomalyco:devfrom
bluet:feat/google-vertex-openai
Open

feat: add google-vertex-openai provider for Vertex AI partner models#716
bluet wants to merge 2 commits intoanomalyco:devfrom
bluet:feat/google-vertex-openai

Conversation

@bluet
Copy link

@bluet bluet commented Jan 26, 2026

Add Vertex AI Partner Models with [provider] Overrides

Adds partner/open models served via Google Vertex AI's OpenAI-compatible endpoint, consolidated under the existing google-vertex provider using per-model [provider] npm overrides (same pattern as Azure's Claude models).

Models Added/Updated

Model ID Context Output Cost In/Out Modalities
GLM-4.7 (updated) zai-org/glm-4.7-maas 200K 128K $0.60/$2.20 text, pdf
DeepSeek V3.1 deepseek-ai/deepseek-v3.1-maas 163K 32K $0.60/$1.70 text, pdf
Llama 4 Maverick meta/llama-4-maverick-17b-128e-instruct-maas 524K 8K $0.35/$1.15 text, image
Llama 3.3 70B meta/llama-3.3-70b-instruct-maas 128K 8K $0.72/$0.72 text
Qwen3 235B qwen/qwen3-235b-a22b-instruct-2507-maas 262K 16K $0.22/$0.88 text

Approach

Per maintainer feedback ("Ideally we don't make 3 different providers for vertex"), all models live under providers/google-vertex/models/ in vendor-namespaced subdirectories. Each uses:

[provider]
npm = "@ai-sdk/openai-compatible"

This follows the established pattern (e.g., Azure's claude-haiku-4-5.toml uses [provider] npm = "@ai-sdk/anthropic" to override the provider's default npm package).

Verification

GLM-4.7 Corrections from Original

Fixed specs that differed from official docs:

  • context: 204,800 → 200,000
  • output: 131,072 → 128,000
  • release_date: 2025-12-22 → 2026-01-06
  • Added pdf input modality
  • Added structured_output = true

Related

  • Depends on opencode PR #10303 for runtime support of [provider] overrides with @ai-sdk/openai-compatible

@rekram1-node
Copy link
Contributor

I think im going to consolidate all the google vertex stuff under a single provider and override npm and api fields using [provider] section per model as needed.

This will require additional tweaks in opencode tho

@bluet
Copy link
Author

bluet commented Jan 30, 2026

@rekram1-node anything I can do/help with to accelerate this?

I've also created a related PR in opencode anomalyco/opencode#10742

@opencode-agent
Copy link
Contributor

User bluet does not have write permissions

github run

@bluet
Copy link
Author

bluet commented Jan 30, 2026

@rekram1-node I've closed my opencode PR #10742 in favor of the existing PR #10303 by @leehack, which takes the same consolidation approach you mentioned (using protocol: "openapi" within the existing google-vertex provider).

For this models.dev PR, what would you like me to do?

Options:

  1. Close this PR - if you're planning to handle the consolidation yourself
  2. Update this PR - to use the protocol: "openapi" field format that matches PR #10303's approach (move models back under google-vertex with appropriate overrides)
  3. Keep as-is temporarily - until the opencode PR #10303 is merged and we know the exact model schema

Let me know what works best for you!

Add DeepSeek V3.1, Llama 4 Maverick, Llama 3.3 70B, and Qwen3 235B as
partner models under google-vertex provider. Update GLM-4.7 with
corrected specs from official Google Cloud docs.

Each partner model uses [provider] npm override to @ai-sdk/openai-compatible
since these models are served via Google's OpenAI-compatible endpoint,
while staying consolidated under the google-vertex provider per
maintainer feedback.

All specs (context windows, output limits, pricing, modalities)
verified against official Google Cloud documentation:
- cloud.google.com/vertex-ai/generative-ai/pricing
- cloud.google.com/vertex-ai/generative-ai/docs/maas/*

Changes:
- Update zai-org/glm-4.7-maas: fix context=200K, output=128K, add pdf
  modality, correct release_date, add structured_output, add [provider]
- Add deepseek-ai/deepseek-v3.1-maas ($0.60/$1.70, 163K context)
- Add meta/llama-4-maverick-17b-128e-instruct-maas (vision, 524K ctx)
- Add meta/llama-3.3-70b-instruct-maas ($0.72/$0.72, 128K context)
- Add qwen/qwen3-235b-a22b-instruct-2507-maas ($0.22/$0.88, 262K ctx)
@bluet bluet force-pushed the feat/google-vertex-openai branch from adeba4e to 4c36121 Compare February 14, 2026 22:35
@bluet
Copy link
Author

bluet commented Feb 14, 2026

@rekram1-node Restructured per your feedback — all models are now consolidated under the existing google-vertex provider instead of creating a separate google-vertex-openai provider.

Each partner model uses a [provider] override to specify npm = "@ai-sdk/openai-compatible" (same pattern as Azure's Claude models), while staying under providers/google-vertex/models/ in vendor-namespaced subdirectories.

Also corrected all specs against official Google Cloud docs — the original values had some discrepancies (context windows, output limits, pricing).

Ready for review when you have time!

@rekram1-node
Copy link
Contributor

ill look over tmr, sorry for delays been crazy

@rekram1-node
Copy link
Contributor

So I see what you're doing but there is a pretty critical bug here, none of these models have a base url...

You need:

[provider]
npm = ....
api = ... // THIS ONE

And the tricky thing here is we havent actually introduced this pattern yet and its why google vertex anthropic was originally separate provider

What api should look like is also tied to the env vars, a lot of the apis would look something like this?

https://${ENDPOINT}/v1/projects/${PROJECT_ID}/locations/${REGION}/endpoints/openapi

But instead of ${PROJECT_ID} it should be consistent w/ the vars stated in the provider.toml so for example:

GOOGLE_VERTEX_PROJECT

And then in opencode it should be handling the replacing of these values.

Add the api field with env-var template URL to all partner models so
opencode's loadBaseURL() can resolve the OpenAI-compatible endpoint.

Uses GOOGLE_VERTEX_PROJECT (not GOOGLE_CLOUD_PROJECT) because
googleVertexVars() resolves it through the full fallback chain
(GOOGLE_VERTEX_PROJECT → options.project → GOOGLE_CLOUD_PROJECT →
GCP_PROJECT → GCLOUD_PROJECT).
@bluet
Copy link
Author

bluet commented Feb 16, 2026

Good catch! Added the api field to all partner models:

[provider]
npm = "@ai-sdk/openai-compatible"
api = "https://${GOOGLE_VERTEX_ENDPOINT}/v1/projects/${GOOGLE_VERTEX_PROJECT}/locations/${GOOGLE_VERTEX_LOCATION}/endpoints/openapi"

Note: I used ${GOOGLE_VERTEX_PROJECT} rather than ${GOOGLE_CLOUD_PROJECT} because the merged loadBaseURL() in opencode PR #10303 resolves ${GOOGLE_VERTEX_PROJECT} through the full fallback chain in googleVertexVars() (GOOGLE_VERTEX_PROJECToptions.projectGOOGLE_CLOUD_PROJECTGCP_PROJECTGCLOUD_PROJECT), whereas ${GOOGLE_CLOUD_PROJECT} would only resolve via direct Env.get() and miss users who set GOOGLE_VERTEX_PROJECT instead.

bun validate passes. Also congrats on merging opencode PR #10303!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants