Skip to content

Add China API models.dev pricing lookup - #2376

Open
joeVenner wants to merge 8 commits into
steipete:mainfrom
joeVenner:codex/modelsdev-china-api-models
Open

Add China API models.dev pricing lookup#2376
joeVenner wants to merge 8 commits into
steipete:mainfrom
joeVenner:codex/modelsdev-china-api-models

Conversation

@joeVenner

@joeVenner joeVenner commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add provider-aware models.dev pricing lookup for China API provider families.
  • Route Alibaba coding plans to alibaba-coding-plan/alibaba-coding-plan-cn, Alibaba token plans to alibaba-token-plan/alibaba-token-plan-cn, z.ai to zai/zai-coding-plan, and DeepSeek to deepseek.
  • Wire those provider-scoped lookups into the Pi session runtime cost path and expose the supported providers through codexbar cost.

Proof

  • Previous-context check: PR Improve China Kimi/GLM routes: Coding Plan 5h, Kimi Monthly, clearer labels #2351 covers Kimi/GLM quota routing, not these models.dev provider pricing mappings. PR feat(providers): add Qwen Cloud individual token-plan provider #2361 adds Qwen Cloud token-plan usage, not Qwen model pricing lookup.
  • Absence check against current base returned no matches for the IDs used here:
    git grep -n -E "glm-5\.1|glm-5v-turbo|qwen3\.7-max|deepseek-v4-pro" origin/main -- Sources Tests docs
  • Source catalog check from https://models.dev/api.json showed these live provider keys: alibaba, alibaba-cn, alibaba-coding-plan, alibaba-coding-plan-cn, alibaba-token-plan, alibaba-token-plan-cn, deepseek, zai, zai-coding-plan, zhipuai, zhipuai-coding-plan.
  • Live catalog spot checks showed:
    • alibaba-coding-plan: qwen3.7-max / input 2.5 / output 7.5 / cache_read 0.5 / cache_write 3.125
    • alibaba-token-plan: qwen3.7-plus / input 0 / output 0 / cache_read 0 / cache_write 0
    • zai: glm-5v-turbo / input 1.2 / output 4 / cache_read 0.24 / cache_write 0
    • zhipuai: glm-5v-turbo / input 5 / output 22 / cache_read 1.2 / cache_write 0, intentionally not used for z.ai routing
    • deepseek: deepseek-v4-pro / input 0.435 / output 0.87 / cache_read 0.003625 / context 1000000
  • Runtime regression test uses a cached models.dev fixture with deliberately divergent fallback entries, then verifies PiSessionCostScanner.loadDailyReport computes provider-scoped costUSD for Alibaba coding plan, Alibaba token plan, z.ai, and DeepSeek Pi JSONL rows.
  • Built CLI proof used an isolated temp home with a synthetic .pi/agent/sessions/2026-07-28T10-00-00-000Z_china-proof.jsonl containing only assistant rows. No real account files, Keychain, or browser cookies were read. Command shape:
    HOME=[temp] CFFIXED_USER_HOME=[temp] XDG_CONFIG_HOME=[temp]/.config XDG_CACHE_HOME=[temp]/.cache .build/debug/CodexBarCLI cost --provider <provider> --format json --pretty --refresh
  • Built CLI output excerpts:
    { "provider" : "alibaba", "source" : "local", "sessionTokens" : 150000, "sessionCostUSD" : 0.625,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "qwen3.7-max", "totalTokens" : 150000, "cost" : 0.625 }] }] }
    { "provider" : "alibabatokenplan", "source" : "local", "sessionTokens" : 150000, "sessionCostUSD" : 0,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "qwen3.7-plus", "totalTokens" : 150000, "cost" : 0 }] }] }
    { "provider" : "zai", "source" : "local", "sessionTokens" : 180000, "sessionCostUSD" : 0.3224,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "glm-5v-turbo", "totalTokens" : 180000, "cost" : 0.3224 }] }] }
    { "provider" : "deepseek", "source" : "local", "sessionTokens" : 160000, "sessionCostUSD" : 0.08703625,
      "daily" : [{ "date" : "2026-07-28", "modelBreakdowns" : [{ "modelName" : "deepseek-v4-pro", "totalTokens" : 160000, "cost" : 0.08703625 }] }] }
    The Alibaba token-plan zero is expected from the current public models.dev catalog for alibaba-token-plan/qwen3.7-plus.

Verification

  • CLANG_MODULE_CACHE_PATH=/private/tmp/codexbar-clang-cache swift test --disable-sandbox --scratch-path /Users/mosaab/Documents/Projects/CodexBar/.build --filter PiSessionCostScannerTests: 19 tests passed.
  • CLANG_MODULE_CACHE_PATH=/private/tmp/codexbar-clang-cache swift test --disable-sandbox --scratch-path /Users/mosaab/Documents/Projects/CodexBar/.build --filter 'provider lookup resolves current China API models': 1 test passed.
  • git diff --check: clean.
  • make check completed generated parser hash, package/docs/locales checks, SwiftFormat lint mode, and SwiftLint strict with 0 violations, then exited on the sandbox-only macOS plist cache write permission error after lint shutdown.

No live provider probes, browser-cookie imports, or Keychain reads were run.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ac458da2f6

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift Outdated
Comment thread Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift Outdated
@joeVenner

Copy link
Copy Markdown
Contributor Author

@clawsweeper review

@clawsweeper

clawsweeper Bot commented Jul 27, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. labels Jul 27, 2026
@clawsweeper

clawsweeper Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codex review: needs changes before merge. Reviewed August 9, 2026, 6:38 PM ET / 22:38 UTC.

ClawSweeper review

What this changes

Adds models.dev pricing lookup and Pi-session cost reporting for Alibaba plans, z.ai, and DeepSeek through CodexBar’s cost command.

Merge readiness

Blocked by patch quality or review findings - 4 items remain

Keep open. The useful cost-reporting capability is not on current main, but the rebased PR still prices China-region configurations through fixed catalog priority, so configured China users can receive incorrect estimates.

Priority: P2
Reviewed head: a3b98f87f0469904172f4719682d6b1cc9af33bb

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The CLI proof is concrete, but the unresolved regional-routing defect makes the current patch unsafe to merge.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (terminal): The PR body provides redacted after-fix terminal output from an isolated synthetic Pi-session setup for all four provider paths.
Patch quality 🦪 silver shellfish (2/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The PR body provides redacted after-fix terminal output from an isolated synthetic Pi-session setup for all four provider paths.
Evidence reviewed 5 items Current main lacks this capability: Current main limits Pi-session scanning to Codex and Claude, so the PR’s central provider-cost support is not already implemented.
Persisted regional settings: Alibaba Coding Plan and z.ai both persist region choices, including China mainland and BigModel CN, independently of the provider ID.
Unresolved fixed-priority lookup: The proposed provider-ID list puts the international Alibaba catalog before its China catalog and has no selected-region input; the equivalent z.ai mapping likewise cannot select the BigModel CN catalog.
Findings 1 actionable finding [P1] Select the pricing catalog from the configured region
Security None None.

How this fits together

CodexBar’s cost command reads local Pi session JSONL usage and applies cached models.dev rates to produce daily and session cost estimates. Provider descriptors expose those estimates in the CLI and app.

flowchart LR
A[Pi session JSONL] --> B[Cost scanner]
C[Provider settings] --> B
D[models.dev catalog cache] --> E[Pricing lookup]
B --> E
E --> F[Daily cost report]
F --> G[Cost CLI and app]
Loading

Before merge

  • Select the pricing catalog from the configured region (P1) - This lookup receives only UsageProvider, so Alibaba always checks the international catalog before -cn, and z.ai has no path to select BigModel CN pricing. Both providers persist those endpoint choices and overlapping model IDs can have different rates; thread the selected region/catalog through this path and test divergent regional fixtures.
  • Resolve merge risk (P1) - Merging as written can show China-mainland Alibaba or BigModel CN z.ai users international-catalog rates when model IDs overlap.
  • Complete next step (P2) - A narrow mechanical repair can preserve the PR’s provider-cost feature while honoring existing regional configuration.
  • Improve patch quality - Pass persisted provider region into catalog selection and add divergent regional pricing assertions.

Findings

  • [P1] Select the pricing catalog from the configured region — Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift:797-800
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production versus test delta production +203/-34, tests +316/-85 The PR adds a substantial provider-cost path with more regression coverage than production code.
Affected files 11 files changed The change spans shared cost scanning, four provider descriptors, generated metadata, and tests.

Merge-risk options

Maintainer options:

  1. Route pricing by configured region (recommended)
    Pass the selected Alibaba or z.ai region into catalog selection and add divergent-rate regression fixtures before merge.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Thread persisted provider region into models.dev catalog selection and add divergent international/China rate tests through the cost path.

Technical review

Best possible solution:

Thread the already-persisted provider region into pricing-catalog selection and prove divergent international/China catalog rates through the normal cost path before merging.

Do we have a high-confidence way to reproduce the issue?

Yes. Set Alibaba to China mainland or z.ai to BigModel CN and use fixtures with overlapping model IDs but divergent regional prices; the current resolver has no region input to select the matching catalog.

Is this the best way to solve the issue?

No. Provider-specific lookup is the right seam, but fixed catalog order is not sufficient where existing provider settings choose a different regional endpoint and price catalog.

Full review comments:

  • [P1] Select the pricing catalog from the configured region — Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift:797-800
    This lookup receives only UsageProvider, so Alibaba always checks the international catalog before -cn, and z.ai has no path to select BigModel CN pricing. Both providers persist those endpoint choices and overlapping model IDs can have different rates; thread the selected region/catalog through this path and test divergent regional fixtures.
    Confidence: 0.98

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against a0d77c37f9cc.

Labels

Label justifications:

  • P2: The feature is valuable but its remaining defect is limited to cost estimates for selected provider regions.
  • merge-risk: 🚨 compatibility: Existing China-region configurations can receive materially incorrect estimated prices after upgrade.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🐚 platinum hermit and patch quality is 🦪 silver shellfish.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body provides redacted after-fix terminal output from an isolated synthetic Pi-session setup for all four provider paths.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body provides redacted after-fix terminal output from an isolated synthetic Pi-session setup for all four provider paths.

Evidence

Acceptance criteria:

  • [P1] swift test --filter 'provider lookup resolves current China API models'.
  • [P1] swift test --filter PiSessionCostScannerTests.
  • [P1] make test.
  • [P1] make check.

What I checked:

Likely related people:

  • steipete: The owner reviewed this branch’s shared cost-scanner surface and specified the required landing coordination with the parallel pricing PRs. (role: owner reviewer and likely decision owner; confidence: high; commits: 0e46d1940c35; files: Sources/CodexBarCore/PiSessionCostScanner.swift, Sources/CodexBarCore/Vendored/CostUsage/CostUsagePricing.swift)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (28 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-07T18:59:27.663Z sha 740a86b :: needs changes before merge. :: [P1] Honor the selected pricing region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T05:04:22.787Z sha 740a86b :: needs changes before merge. :: [P1] Honor the configured pricing region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T11:26:13.611Z sha 740a86b :: found issues before merge. :: [P1] Route pricing by the configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T13:48:39.921Z sha 740a86b :: found issues before merge. :: [P1] Honor the configured pricing region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T16:55:47.392Z sha 740a86b :: needs changes before merge. :: [P1] Route pricing by the configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-08T22:59:22.842Z sha 740a86b :: needs changes before merge. :: [P1] Select pricing catalog from configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-09T05:44:46.604Z sha 740a86b :: found issues before merge. :: [P1] Route price catalogs by configured region | [P2] Update the provider architecture gatekeeper
  • reviewed 2026-08-09T14:32:38.447Z sha 5ce6ad6 :: needs changes before merge. :: [P1] Route price catalogs by configured region | [P2] Update the provider architecture gatekeeper

@joeVenner

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 28, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Command router queued. I will update this comment with the next step.

@joeVenner

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Jul 28, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event repository_dispatch).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Jul 28, 2026
@joeVenner
joeVenner force-pushed the codex/modelsdev-china-api-models branch from 68c556c to bccd82b Compare July 31, 2026 17:24
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels Jul 31, 2026
@steipete

steipete commented Aug 5, 2026

Copy link
Copy Markdown
Owner

Thanks for the models.dev pricing series! The China API provider mappings (Alibaba, z.ai, DeepSeek) here haven't landed on main (the scanner still only maps anthropic/openai), so this is still valuable — but the branch has been merge-dirty since July 21, and #2374/#2376/#2377 all touch the same files (PiSessionCostScanner.swift, CostUsagePricing.swift, CLICostCommand.swift), so they'll conflict with each other as well as with main. Could you rebase within the next two weeks, ideally picking one PR to land first and stacking the other two on top afterwards?

@clawsweeper clawsweeper Bot added status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. and removed status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. merge-risk: 🚨 auth-provider 🚨 Merging this PR could break OAuth, tokens, provider routing, model choice, or credentials. labels Aug 6, 2026
…DevCostUSD out of enum to satisfy lint limits
The hash commits were dropped during the rebase. Regenerate the final hash now that all China pricing changes are applied.
ClawSweeper flagged that .alibaba and .alibabatokenplan fell back to the direct 'alibaba' catalog when their dedicated plan catalogs lacked a model. Plan billing can differ from direct API billing, so return no estimate instead. Add regression coverage for missing plan-catalog rows.
@joeVenner
joeVenner force-pushed the codex/modelsdev-china-api-models branch from 740a86b to 5ce6ad6 Compare August 9, 2026 14:28
@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. labels Aug 9, 2026
@joeVenner

Copy link
Copy Markdown
Contributor Author

@steipete — this PR has been rebased onto latest main and is ready for review. Thanks!

After rebasing the China API models.dev pricing work, ProviderArchitectureGatekeeperTests failed because new provider-specific branches (Alibaba, Zai, DeepSeek) appeared in shared cost code and existing allowlist anchors shifted. Add '// Provider-specific by design:' markers at each provider-owned dispatch point, update the gatekeeper's hardcoded supportsTokenSnapshot set and shifted suppressed references, and remove obsolete CostUsageFetcher/PiSessionCostScanner allowlist entries whose anchors no longer match.

Closes provider-architecture gatekeeper failures on the China API rebase.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants