Skip to content

feat(LAM-1978): auto-capture git state into trace metadata - #311

Open
laminar-coding-agent[bot] wants to merge 2 commits into
mainfrom
feat/lam-1978-git-metadata
Open

feat(LAM-1978): auto-capture git state into trace metadata#311
laminar-coding-agent[bot] wants to merge 2 commits into
mainfrom
feat/lam-1978-git-metadata

Conversation

@laminar-coding-agent

@laminar-coding-agent laminar-coding-agent Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Why

To support the agent-versions feature we want to know which git commit an agent was running off of (LAM-1978). Braintrust does this for their evals; this brings the same capability to Laminar with broader coverage (all traces, not just evals) and a CI env-var fallback Braintrust lacks.

What

  • New lmnr.sdk.git_metadata.collect_git_metadata(): best-effort collection of git.commit, git.branch, git.dirty via git subprocesses (1.5s timeout each, never raises, lru_cached per process). Falls back to CI env vars (GITHUB_SHA, VERCEL_GIT_COMMIT_SHA, CI_COMMIT_SHA, CIRCLE_SHA1, RENDER_GIT_COMMIT, RAILWAY_GIT_COMMIT_SHA + branch counterparts) when git is unavailable — e.g. gitless CI checkouts.
  • Laminar.initialize() merges the git keys into global trace metadata at the lowest precedence: both LMNR_TRACE_METADATA and the explicit metadata= argument override them. They land on every span as lmnr.association.properties.metadata.git.* and flow into traces.metadata — filterable in the traces-table UI and queryable in the SQL editor with zero backend changes.
  • Eval runs get the same keys stamped into their run-level metadata (_with_git_metadata, composed with _with_debugger_session_metadata), so the evaluation entity itself records the code state.
  • Opt-out: disable_git_metadata=True init param or LMNR_DISABLE_GIT_METADATA env var.
  • git.dirty counts tracked-file changes only; git.branch is omitted on detached HEAD.
  • Deliberately excludes author name/email, commit message, and diffs (PII surface Braintrust later had to gate behind org-level redaction settings).

Cross-language parity with the lmnr-ts PR (same key names, CI env list, precedence, truthy set).

Testing

  • tests/test_git_metadata.py: 15 new tests covering collection in a temp repo (clean/dirty/untracked/detached HEAD), CI env fallback, git-beats-env precedence, both opt-outs, span-attribute stamping via initialize(), precedence of user/env metadata over git keys, and the eval metadata wrapper.
  • Full suite: failure set identical to clean main (34 pre-existing failures, 0 new; 890 passed).
  • tests/conftest.py sets LMNR_DISABLE_GIT_METADATA=true session-wide so the SDK repo's own git state doesn't leak into exact-metadata assertions.
  • End-to-end with a toy project: a scratch git repo (my-agent-v2 branch, dirty tree) running a plain @observe agent against the local app-server produced this in Postgres traces.metadata:
    {"git.dirty": true, "git.branch": "my-agent-v2", "git.commit": "22b41ab266b5d534e167f238dae933829ba4fc06"}
    
    and the same values are queryable in ClickHouse via simpleJSONExtractString(trace_metadata, 'git.commit') — the exact path the traces-table metadata filter uses.

🤖 Generated with Claude Code


Note

Low Risk
Additive metadata collection is best-effort, timeout-bounded, and fully opt-out; it does not change auth, export protocols, or existing metadata precedence beyond inserting git keys at the bottom.

Overview
Adds best-effort git metadata (git.commit, git.branch, git.dirty) so traces and eval runs record which code revision produced them, without backend changes.

Laminar.initialize() collects git state once per process (short-timeout git subprocesses, with CI env fallbacks when .git is missing) and merges it into global trace metadata at the lowest precedence—LMNR_TRACE_METADATA and the metadata= argument override git keys. Keys flow to spans as lmnr.association.properties.metadata.git.* and into traces.metadata. Opt out via disable_git_metadata=True or LMNR_DISABLE_GIT_METADATA.

Evaluations stamp the same keys on run-level metadata through _with_git_metadata, composed with the existing debugger-session helper at client.evals.init. Process-wide set_git_metadata_disabled ensures the init opt-out applies to eval metadata even when collection was cached earlier.

Tests and hygiene: tests/test_git_metadata.py covers collection edge cases, precedence, span stamping, and eval wrapping; session conftest disables git collection by default so the SDK repo’s git state does not break other tests.

Reviewed by Cursor Bugbot for commit c36bbd1. Bugbot is set up for automated code reviews on this repo. Configure here.

Collect git.commit / git.branch / git.dirty once at Laminar.initialize()
(best-effort subprocess, 1.5s timeout, CI env-var fallback) and merge into
global trace metadata at the lowest precedence, so LMNR_TRACE_METADATA and
the explicit metadata= argument both override. The keys land on every span
as lmnr.association.properties.metadata.git.* and flow into traces.metadata
with no backend changes. Eval runs additionally get the same keys stamped
into their run metadata via _with_git_metadata.

Opt out with disable_git_metadata=True or LMNR_DISABLE_GIT_METADATA.

Cross-language parity surface with lmnr-ts src/git-metadata.ts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d1fca5b. Configure here.

Comment thread src/lmnr/sdk/evaluations.py
Bugbot on #311: Laminar.initialize(disable_git_metadata=True) suppressed
git keys in global trace metadata, but _with_git_metadata still collected
them into evaluation run metadata unless the env var was also set.

Record the init-time opt-out process-wide (set_git_metadata_disabled) and
check it inside collect_git_metadata() on every call — outside the
lru_cache, so an opt-out recorded after a prior collection still applies.
The cache moves to _collect_git_metadata_cached with a
reset_git_metadata_cache() test helper.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@laminar-coding-agent

Copy link
Copy Markdown
Contributor Author

Valid catch — fixed in c36bbd1. Laminar.initialize(disable_git_metadata=True) now records the opt-out process-wide (set_git_metadata_disabled), and collect_git_metadata() checks it on every call, outside the lru_cache — so eval run metadata honors the init param too, and an opt-out recorded after a prior collection still applies. Added regression tests test_initialize_opt_out_also_disables_eval_git_metadata and test_opt_out_applies_even_after_prior_collection.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant