LAM-2168: record OpenAI Agents span errors as exception events - #328
Open
laminar-coding-agent[bot] wants to merge 1 commit into
Open
LAM-2168: record OpenAI Agents span errors as exception events#328laminar-coding-agent[bot] wants to merge 1 commit into
laminar-coding-agent[bot] wants to merge 1 commit into
Conversation
Agent SDK span errors were set via `set_status(StatusCode.ERROR)` only, which Laminar never renders: app-server's `from_otel_span` does not read `otel_span.status`, and `prepare_span_for_recording` derives `status = "error" solely from the presence of an event named `exception`. A tool call that raised was therefore recorded as `status=success`. `apply_span_error` now also emits an `exception` event with the `exception.type` / `exception.message` attributes the trace view's error card renders. Also fixes the field extraction: `SpanError` is a TypedDict, so `getattr(error, "message")` always returned None and the message fell back to `str(error)` (the whole dict repr). Both the dict and object shapes are now handled, with `data` carrying the specifics. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Errors on OpenAI Agents SDK spans never surfaced in Laminar. A tool call that raised was recorded as
status=success.apply_span_errorreported failure withset_status(Status(StatusCode.ERROR, ...))— but nothing downstream reads that:app-server/src/traces/spans.rs::from_otel_spannever readsotel_span.status; the OTLP status field is discarded at ingestion.app-server/src/traces/utils.rs::prepare_span_for_recordingsetsstatus = "error"only when the span carries an event namedexception.So
set_status(ERROR)is silently a no-op end-to-end. Verified with a minimal two-way probe against staging, no Agents SDK involved:spans.statusspan.set_status(Status(StatusCode.ERROR, "…"))successspan.record_exception(RuntimeError("…"))errorFound while instrumenting the Toolathlon benchmark harness (LAM-2168). The Agents SDK populated the error correctly —
{'message': 'Error running tool (non-fatal)', 'data': {'tool_name': 'lookup_paper', 'error': '…'}}— and Laminar still showed the tool span green.Fix
apply_span_errornow emits anexceptionevent alongside the status. The trace view's error card (frontend/components/traces/error-card.tsx) readsexception.typeas the headline andexception.messageas the body, so both are set:exception.type<- theSpanErrorlabel ("Error running tool (non-fatal)","Max turns exceeded")exception.message<-dataas JSON, falling back to the label when there is no dataSecond bug fixed in passing:
SpanErroris aTypedDict, so it is a plain dict at runtime andgetattr(error, "message", None)always returnedNone— the status description fell back tostr(error), i.e. the raw dict repr. Both the dict and object shapes are now handled.Scope
This is a general app-server contract, not an Agents-specific quirk — any instrumentor or user calling
set_status(ERROR)gets no error status. Fixing it properly means honoringotel_span.status.codeinfrom_otel_span; that is a separate app-server PR. This one unblocks the Agents path without waiting on it.Tests
New
tests/test_instrumentations/test_openai_agents/test_span_error.py— 3 cases (dict-shapedSpanError, object-shaped, no error), asserting against real exported spans via thespan_exporterfixture.🤖 Generated with Claude Code
Note
Low Risk
Localized change to OpenAI Agents error reporting plus docs and unit tests; no auth, data migration, or broad API surface impact.
Overview
OpenAI Agents SDK failures (e.g. non-fatal tool errors) were exported with Laminar success status because ingestion ignores OTel
set_status(ERROR)and only marks spans error when anexceptionevent is present.apply_span_errornow readsSpanErroras either a runtime dict (TypedDict) or an object, sets OTel error status as before, and adds anexceptionevent withexception.type(short label) andexception.message(JSONdataor the label) so the trace error card and app-server agree.CLAUDE.mddocuments this Laminar-wide contract for the OpenAI Agents instrumentor. New tests intest_span_error.pyassert exported spans for dict/object errors and no event when there is no error.Reviewed by Cursor Bugbot for commit a7face9. Bugbot is set up for automated code reviews on this repo. Configure here.