Skip to content

Fix entitlement chunk collision and active-lease truncation - #4

Merged
yummybomb merged 5 commits into
mainfrom
hypeship/cls-entitlement-chunks-leases-all
Sep 4, 2026
Merged

Fix entitlement chunk collision and active-lease truncation#4
yummybomb merged 5 commits into
mainfrom
hypeship/cls-entitlement-chunks-leases-all

Conversation

@yummybomb

@yummybomb yummybomb commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

What this fixes

The exporter's numbers didn't match what the NVIDIA licensing portal shows. Three reasons:

1. Entitlements bought in separate orders overwrote each other.
nvidia_cls_entitlement_total_quantity was labeled only by feature, product, version, and license type. If we bought the same product twice (say 72 seats, then 128 more), both chunks had identical labels and one silently replaced the other. The org total came out short.

Fix: entitlement metrics now also carry ems_entitlement_id and ems_product_key_id, so each chunk is its own series. Sum across series to get the org total.

2. In-use and unassigned seat counts were fetched but thrown away.
The API already returns them per entitlement feature. They're now exported as nvidia_cls_entitlement_in_use_quantity and nvidia_cls_entitlement_unassigned_quantity, same labels as the total.

3. Active lease counts capped at 100.
The /leases endpoint truncates the client list for large orgs. We saw lease counts flatline at exactly 100. The portal UI uses /leases/all instead, which returns every client. For big payloads it sends them gzip+base64 encoded in a compressedClients field, which the client now decodes.

Things to know

  • Dashboards and alerts. Adding labels changes the series identity of nvidia_cls_entitlement_total_quantity. Anything reading it as one series needs to sum across the new labels. Grouping by product or feature still works as before. The license-server metrics (nvidia_cls_license_server_feature_*) are unchanged.
  • Not tested against the live API. The /leases/all path and the emsEntitlementId / emsProductKeyId / compressedClients field names come from the licensing portal's frontend bundle, which calls the same API. I didn't have a production key to hit it directly. If it fails on deploy, you'll see nvidia_cls_up=0 with stale data, and reverting restores the old behavior.
  • Lease counts may go up. Once the 100 cap is gone, alerts that compare active leases against capacity can fire where they previously couldn't. That's the alerts working correctly.

Testing

Unit tests cover the compressed-clients decode path, the per-chunk entitlement extraction, the /leases/all HTTP contract, and the Prometheus and OTEL output. go test ./... passes.


Note

Medium Risk
Entitlement metric label changes break existing dashboards and recording rules; the new leases API and compressed payload path were not verified against live CLS and can fail scrapes (nvidia_cls_up=0).

Overview
Fixes three gaps between CLS portal data and exported metrics.

Entitlement series now include ems_entitlement_id and ems_product_key_id on nvidia_cls_entitlement_total_quantity (and related series), so separate purchases of the same product no longer collide in Prometheus. Org-wide totals require summing across those labels.

New gauges nvidia_cls_entitlement_in_use_quantity and nvidia_cls_entitlement_unassigned_quantity expose quantities the client already parsed but previously dropped; Prometheus and OTEL paths both emit them with the same label set.

Active leases are loaded from .../leases/all instead of .../leases, with decoding of optional compressedClients (base64 + gzip JSON). That addresses truncated client lists that capped active-lease metrics for large orgs.

README documents the entitlement labeling and new metrics. Unit tests cover compressed lease responses, the /leases/all HTTP contract, per-chunk entitlement extraction, and collector/OTEL observation mapping.

Reviewed by Cursor Bugbot for commit 33dfc89. Bugbot is set up for automated code reviews on this repo. Configure here.

- Label entitlement metrics with ems_entitlement_id / ems_product_key_id
  so same-product entitlements purchased in separate chunks emit distinct
  series instead of overwriting each other.
- Export entitlement in-use and unassigned quantities, which were fetched
  but dropped.
- Fetch active leases from /leases/all (with compressedClients support)
  instead of /leases, which truncates the client list for large orgs.
@yummybomb
yummybomb marked this pull request as ready for review August 31, 2026 20:47
@yummybomb
yummybomb marked this pull request as draft August 31, 2026 21:42
@yummybomb
yummybomb marked this pull request as ready for review September 1, 2026 14:00
@yummybomb
yummybomb merged commit 70a8e46 into main Sep 4, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants