feat: add google-vertex-openai provider for Vertex AI partner models#716
feat: add google-vertex-openai provider for Vertex AI partner models#716bluet wants to merge 2 commits intoanomalyco:devfrom
Conversation
|
I think im going to consolidate all the google vertex stuff under a single provider and override npm and api fields using [provider] section per model as needed. This will require additional tweaks in opencode tho |
|
@rekram1-node anything I can do/help with to accelerate this? I've also created a related PR in opencode anomalyco/opencode#10742 |
|
User bluet does not have write permissions |
|
@rekram1-node I've closed my opencode PR #10742 in favor of the existing PR #10303 by @leehack, which takes the same consolidation approach you mentioned (using For this models.dev PR, what would you like me to do? Options:
Let me know what works best for you! |
Add DeepSeek V3.1, Llama 4 Maverick, Llama 3.3 70B, and Qwen3 235B as partner models under google-vertex provider. Update GLM-4.7 with corrected specs from official Google Cloud docs. Each partner model uses [provider] npm override to @ai-sdk/openai-compatible since these models are served via Google's OpenAI-compatible endpoint, while staying consolidated under the google-vertex provider per maintainer feedback. All specs (context windows, output limits, pricing, modalities) verified against official Google Cloud documentation: - cloud.google.com/vertex-ai/generative-ai/pricing - cloud.google.com/vertex-ai/generative-ai/docs/maas/* Changes: - Update zai-org/glm-4.7-maas: fix context=200K, output=128K, add pdf modality, correct release_date, add structured_output, add [provider] - Add deepseek-ai/deepseek-v3.1-maas ($0.60/$1.70, 163K context) - Add meta/llama-4-maverick-17b-128e-instruct-maas (vision, 524K ctx) - Add meta/llama-3.3-70b-instruct-maas ($0.72/$0.72, 128K context) - Add qwen/qwen3-235b-a22b-instruct-2507-maas ($0.22/$0.88, 262K ctx)
adeba4e to
4c36121
Compare
|
@rekram1-node Restructured per your feedback — all models are now consolidated under the existing Each partner model uses a Also corrected all specs against official Google Cloud docs — the original values had some discrepancies (context windows, output limits, pricing). Ready for review when you have time! |
|
ill look over tmr, sorry for delays been crazy |
|
So I see what you're doing but there is a pretty critical bug here, none of these models have a base url... You need: [provider] And the tricky thing here is we havent actually introduced this pattern yet and its why google vertex anthropic was originally separate provider What api should look like is also tied to the env vars, a lot of the apis would look something like this?
But instead of ${PROJECT_ID} it should be consistent w/ the vars stated in the provider.toml so for example:
And then in opencode it should be handling the replacing of these values. |
Add the api field with env-var template URL to all partner models so opencode's loadBaseURL() can resolve the OpenAI-compatible endpoint. Uses GOOGLE_VERTEX_PROJECT (not GOOGLE_CLOUD_PROJECT) because googleVertexVars() resolves it through the full fallback chain (GOOGLE_VERTEX_PROJECT → options.project → GOOGLE_CLOUD_PROJECT → GCP_PROJECT → GCLOUD_PROJECT).
|
Good catch! Added the [provider]
npm = "@ai-sdk/openai-compatible"
api = "https://${GOOGLE_VERTEX_ENDPOINT}/v1/projects/${GOOGLE_VERTEX_PROJECT}/locations/${GOOGLE_VERTEX_LOCATION}/endpoints/openapi"Note: I used
|
Add Vertex AI Partner Models with
[provider]OverridesAdds partner/open models served via Google Vertex AI's OpenAI-compatible endpoint, consolidated under the existing
google-vertexprovider using per-model[provider]npm overrides (same pattern as Azure's Claude models).Models Added/Updated
zai-org/glm-4.7-maasdeepseek-ai/deepseek-v3.1-maasmeta/llama-4-maverick-17b-128e-instruct-maasmeta/llama-3.3-70b-instruct-maasqwen/qwen3-235b-a22b-instruct-2507-maasApproach
Per maintainer feedback ("Ideally we don't make 3 different providers for vertex"), all models live under
providers/google-vertex/models/in vendor-namespaced subdirectories. Each uses:This follows the established pattern (e.g., Azure's
claude-haiku-4-5.tomluses[provider] npm = "@ai-sdk/anthropic"to override the provider's default npm package).Verification
bun validatepasses (exit code 0)GLM-4.7 Corrections from Original
Fixed specs that differed from official docs:
context: 204,800 → 200,000output: 131,072 → 128,000release_date: 2025-12-22 → 2026-01-06pdfinput modalitystructured_output = trueRelated
[provider]overrides with@ai-sdk/openai-compatible