Copilot: surface AI credit usage for token-based-billing seats - #2647
Copilot: surface AI credit usage for token-based-billing seats#2647KSEGIT wants to merge 10 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: aaeb632f41
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| else { return nil } | ||
| return CopilotDeviceFlow.makeRequestURL( | ||
| host: CopilotUsageFetcher.apiHost(enterpriseHost: enterpriseHost), | ||
| path: "/orgs/\(encoded)/settings/billing/ai_credit/usage") |
There was a problem hiding this comment.
For the organization AI-credit flow, GitHub's REST docs list this endpoint as GET /organizations/{org}/settings/billing/ai_credit/usage, not /orgs/{org}/... (checked https://docs.github.com/en/rest/billing/usage#get-billing-ai-credit-usage-report-for-an-organization). With the current path, authorized org admins receive a 404 and fetchCreditsUsed returns nil, so the new organization credits row never appears.
Useful? React with 👍 / 👎.
| return 0 | ||
| } | ||
|
|
||
| let creditItems = report.usageItems.filter { $0.unitType == "ai-credits" } |
There was a problem hiding this comment.
For organization AI-credit reports, GitHub's example response uses unitType: "credits", while "ai-credits" is shown for user-level reports (checked https://docs.github.com/en/rest/billing/usage#get-billing-ai-credit-usage-report-for-an-organization). After the path is corrected, real org responses will fall into the “none matched” branch here and return nil, so organization usage still will not render unless this accepts the org unit type.
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
rating: 🦪 silver shellfish
|
Codex review: needs real behavior proof before merge. Reviewed August 9, 2026, 6:02 PM ET / 22:02 UTC. ClawSweeper reviewWhat this changesThe PR adds per-Copilot-account AI-credit entitlements and renders token-billed seat consumption as a menu-card text row or progress bar. Merge readiness⛔ Blocked until stronger real behavior proof is added - 4 items remain Keep this PR open: it adds the remaining menu-card presentation beyond current main’s raw Copilot credit row, but its claimed offline cached-row repair still cannot turn a text-only row into a bar after an entitlement is entered. Priority: P2 Review scores
Verification
How this fits togetherCodexBar fetches Copilot usage into a provider snapshot and renders its detail rows in the macOS menu card. A saved seat-credit entitlement is combined with observed credit usage to display an accurate ratio without inventing a quota. flowchart LR
A[Copilot usage response] --> B[Usage snapshot]
C[Saved seat entitlement] --> D[Credit row builder]
B --> D
D --> E[Cached provider snapshot]
E --> F[Menu-card credit row]
Before merge
Findings
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Retain structured raw credit usage for every seat-credit row, then rebuild text-only and progress rows immediately when the entitlement changes; attach redacted current-head proof of that offline transition. Do we have a high-confidence way to reproduce the issue? Yes, from source: fetch and cache a token-billed credit row without an entitlement, enter one while refresh fails, and the current guard leaves the text-only row unchanged. Is this the best way to solve the issue? No. The proposed immediate-update path lacks a structured numerator for text-only rows, so it cannot create the promised cached bar without a successful refresh. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 6f27b777afba. LabelsLabel justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (24 earlier review cycles; latest 8 shown)
|
|
Thanks for this, and for the excellent original report in #2593 — you found both the |
clawsweeper
Bot
added
rating: 🧂 unranked krab
|
Re The upstream data is real, not inferredLive from a Copilot Business seat (redacted), via the endpoint CodexBar already calls: "quota_snapshots": {
"premium_interactions": { "unlimited": true, "entitlement": 0, "remaining": 0,
"percent_remaining": 100.0, "credits_used": 31,
"token_based_billing": true, "overage_permitted": true },
"chat": { "unlimited": true, "entitlement": 0, "credits_used": 0 },
"completions": { "unlimited": true, "entitlement": 0, "credits_used": 0 }
},
"organization_login_list": ["<ORG>"],
"copilot_plan": "business", "token_based_billing": true,
"quota_reset_date": "2026-09-01"Note And the org endpoint: { "usageItems": [
{ "product": "Copilot", "sku": "Copilot AI Credits", "model": "Code Review model",
"unitType": "ai-credits", "pricePerUnit": 0.01, "grossQuantity": 31.13 },
{ "product": "Copilot", "sku": "Copilot Cloud Agent", "model": "Coding Agent model",
"unitType": "ai-credits", "pricePerUnit": 0.01, "grossQuantity": 49.97 } ] }31.13 + 49.97 = 81.10, which matches the "81 / 6,000 AI credits" numerator on the org billing page exactly. That agreement is the strongest evidence I have that the org lane reads the right thing. What the card actually rendersAsserted on exact strings through the real
The org tests drive What I can't evidence yetNo screenshot. Producing one means building this branch, packaging it, and running it against a real Copilot Business account — which per AGENTS.md is exactly the kind of validation that can raise Keychain prompts, so I didn't do it unasked. Happy to if you want it. Two of three surfaces are untouched. I've added a table to the PR description rather than leave it implied: the menu card shows credits; the menu bar icon, the widget, and So today a Business token-billing user still gets an empty icon and has to open the menu. I can wire the icon up in this PR — publishing credits as a Since the PR was opened
|
clawsweeper
Bot
added
rating: 🦪 silver shellfish
bfe422a to
b7406fb
Compare
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
clawsweeper
Bot
added
merge-risk: 🚨 auth-provider
|
After-fix runtime proof — head Menu card (live)
Org lane trace (live)
The best-effort degradation works exactly as designed: warning logged, card unaffected, no fabricated org row, seat lane intact. A finding from the live run: the 404 is a scope issue, not a path issueWith a The Copilot device flow requests only |
|
@clawsweeper re-review |
clawsweeper
Bot
added
proof: sufficient
|
@steipete Hi there, I think it is ready to merge :) Let me know if you want me to do anything more :) |
clawsweeper
Bot
added
rating: 🦐 gold shrimp
Remove the opt-in organization billing lane pending an approved authorization model for org-wide billing reads with device-flow tokens. Rewrite the cached seat credit row immediately when the entitlement changes so a stale denominator or bar never survives a failed refresh.
|
@clawsweeper re-review Addressed the 2026-08-09 14:17 UTC review on head 14599df:
|
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
clawsweeper
Bot
added
merge-risk: 🚨 compatibility
2 participants
Closes #2593.
The problem
On a Copilot Business seat with
token_based_billing: true,GET /copilot_internal/userreports every quota snapshot asunlimited: true, entitlement: 0, remaining: 0, percent_remaining: 100. The#1258guards correctly drop those so no misleading "0% used" bar appears — but the result is a Copilot card showing only the plan label and no usage at all.Real consumption is available, in two places CodexBar didn't read:
credits_used, already present on each quota snapshot in the response CodexBar already fetches (landed in Decode Copilot credits_used for token-billed seats (#2593) #2613).GET /organizations/{org}/settings/billing/ai_credit/usage— org-wide, per-model (this PR).GitHub publishes no credit entitlement anywhere
This is the finding that shaped the design. I probed all 8 documented billing endpoints plus
budgets,cost-centers,included_credits,ai_credit/entitlement,copilot/metricsandusage/summary. None expose the included-credit ceiling — the "6,000" that the org billing page displays.discountQuantityreveals only what included credits absorbed, so the ceiling is observable only once exceeded.It is derivable as seats × per-seat allowance, but the per-seat figure is currently a promotional 3,000 against a standard 1,900 — a hardcoded table would silently produce a wrong bar when the promo ends. So the denominator is user-entered and never inferred. A lane with no entitlement renders a text row, never a bar with a fabricated ceiling.
If you know of an endpoint I missed, that would simplify this considerably.
What this adds
Seat credit bar (builds on #2613). Extends the shared provider-detail row contract with an optional row id and progress ratio (
ProviderDetailSection.Row), so the existing "Credits used" row becomes a "31 / 3000" bar when a seat entitlement is set, and stays plain text otherwise. Created only when it carries real signal (token-billed / unlimited quota / credits > 0 / entitlement set), so metered accounts reportingcredits_used: 0don't gain a permanent empty row. No guard line inCopilotUsageFetcher.swiftis modified — the#1258guard block is byte-identical to main.Org lane (opt-in, off by default).
GET /organizations/{org}/settings/billing/ai_credit/usage(the documented path), org read fromorganization_login_list.first, summingusageItems[].grossQuantityfiltered to credit unit types —"credits"per the organization-report docs,"ai-credits"per the user-level docs and live org responses; both are accepted, unrelated unit types are rejected (a response with no credit items returnsnil, never a fabricated 0). Strictly best-effort, shaped like the existingaddBudgetWindowsIfNeeded: any failure returns the snapshot untouched. The device flow still requests onlyread:user, so rejection is the expected common case, not an error — it logs a warning rather than failing silently.On my account the org endpoint returns 31.13 + 49.97 = 81.10, matching the billing page's numerator exactly.
Per-account entitlements. Seat and org allowances are stored on the selected
ProviderTokenAccount(following the z.ai per-account fields precedent), resolved incopilotSettingsSnapshotwith the existing global UserDefaults values as fallback — so existing single-account setups keep working unchanged, and two Copilot accounts with different allowances each get the right denominator. The settings fields write to the selected account when one exists.Which surfaces this covers
Worth being explicit, since it's a menu bar app:
codexbar usageCLICredits are detail rows, not a
RateWindow, because a lane with no user-entered entitlement has no percentage to show.MenuBarLayout,IconRemainingResolverand the widget all build fromprimary/secondary/tertiary/extraRateWindows, so they see nothing.The consequence: a Business token-billing user still gets an empty menu bar icon and must open the menu to see credits. I'd rather state that than have you find it.
Making the icon work needs a decision I didn't want to make unilaterally: publishing credits as a
NamedRateWindowwhen an entitlement exists would light up the icon, widget and the existing "menu bar secondary metric" picker for free, but Copilot's extra windows are gated behindcopilotBudgetExtrasEnabledin the card renderer, so it risks either double-rendering the row or coupling credits to an unrelated setting. Happy to do it in this PR if you'll say which shape you want.Testing
make check— 0 violations across 1,811 files.make test— full suite green exceptMiniMaxMenuCardBillingTests, which fails identically on a cleanorigin/maincheckout on this machine (locale-dependent number grouping, "1.234" vs "1,234"); unrelated to this PR.ProviderHTTPTransportStubor the existingURLProtocolstub. No live network, no Keychain access, per AGENTS.md.Regression coverage worth calling out: a test asserts the
#1258behaviour still holds — a Business token-billing payload yields credits andprimary == nil, secondary == nil. New per-account tests cover: account override wins over the global fallback, fallback when the account has no value, two accounts producing different snapshot entitlements, and a Codable round-trip through the config store (including legacy JSON without the new keys).Known limitations
L("AI credits"),L("Org credits"),L("credits used")have noLocalizable.stringsentries.Scripts/check-app-locales.mjsenforces all 23 catalogs together, so a partial addition failsmake check— left out deliberately rather than done badly. Happy to add all 23 here if you'd prefer.organization_login_list.firstwins when a user belongs to several orgs. The row title names the org so it isn't ambiguous, but there's no picker.resetsAtis available and a monthly credit budget is a good fit for it (cf. Show ahead/behind pace for all providers and in the menu bar #807, Add Projections to Copilot Tracker #2166).Commits
9 focused commits, each with its own tests, left unsquashed so they can be reviewed in sequence. Squash on merge if you'd rather.
🤖 Generated with Claude Code