Clean up error recording#282
Open
lmolkova wants to merge 3 commits into
Open
Conversation
`Error.type` becomes a plain string (the `error.type` attribute value) instead of an exception class. `GenAIInvocation.fail` derives it via a new `fq_exception_type` helper that module-qualifies non-builtin exceptions, so `error.type` matches the `exception.type` recorded on the exception event. For OpenAI streaming, `response.failed` / `response.incomplete` / `response.error` now record the actual event type or error code instead of a generic RuntimeError.
lmolkova
force-pushed
the
genai-error-type-string
branch
from
July 18, 2026 04:23
94ca61c to
93e2a0a
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates GenAI error recording to (a) use a canonical fully-qualified exception type string for error.type and (b) propagate OpenAI Responses API failure codes into error.type instead of reporting a generic error type. These changes span the shared opentelemetry-util-genai error model plus OpenAI instrumentation wrappers/tests.
Changes:
- Add canonical exception-type formatting (
fq_exception_type) and switchError.typeto a string-based representation (supporting exception types and non-exception error codes). - Update util invocation failure handling to derive
Errorconsistently from exceptions. - Update OpenAI Responses stream/non-stream error handling to record response error codes and treat
response.incompleteas a finish reason (not an error), with corresponding test updates.
Reviewed changes
Copilot reviewed 22 out of 23 changed files in this pull request and generated 25 comments.
Show a summary per file
| File | Description |
|---|---|
| uv.lock | Bumps wrapt lower bound in lock to align with updated dependency floor. |
| util/opentelemetry-util-genai/src/opentelemetry/util/genai/utils.py | Adds helper to compute canonical fully-qualified exception type names. |
| util/opentelemetry-util-genai/src/opentelemetry/util/genai/types.py | Changes Error model (type: str) and adds from_exception constructor. |
| util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py | Uses new Error.from_exception and applies error.type as a string attribute. |
| util/opentelemetry-util-genai/pyproject.toml | Raises wrapt lower bound to >= 1.17.0. |
| util/opentelemetry-util-genai/.changelog/+error-type-string.changed | Adds changelog fragment for util change (but filename/content issues). |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_structured_outputs.py | Updates expected error.type for 404 structured output tests. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_async_structured_outputs.py | Async counterpart of structured output 404 expectations. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_responses.py | Updates Responses API error-type expectations and adds response-error handling coverage. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_async_responses.py | Async Responses API error-type expectation updates. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_response_wrappers.py | Adds coverage for response failed/incomplete/error events producing correct Error. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_response_extractors.py | Adds tests for get_response_error() behavior across response statuses. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_embeddings.py | Updates expected error.type for embeddings error spans/metrics. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_async_embeddings.py | Async embeddings error-type expectation update. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_chat_completions.py | Updates expected error.type for chat errors in spans/metrics. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/test_async_chat_completions.py | Async chat completion error-type expectation updates. |
| instrumentation/opentelemetry-instrumentation-genai-openai/tests/requirements.oldest.txt | Adds editable util-genai install (conflicts with oldest-env intent). |
| instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/response_wrappers.py | Records response error codes and treats response.incomplete as non-error completion. |
| instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/response_extractors.py | Adds get_response_error() to map failed responses to Error(code, message). |
| instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/patch.py | Passes caught exceptions directly to invocation.fail(...). |
| instrumentation/opentelemetry-instrumentation-genai-openai/src/opentelemetry/instrumentation/genai/openai/patch_responses.py | Fails invocations when responses return an error payload (even without raising). |
| instrumentation/opentelemetry-instrumentation-genai-openai/.changelog/+error-type-string.fixed | Adds OpenAI changelog fragment (but filename issues). |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_llm_call.py | Updates expected OpenAI error.type in LangChain tests. |
lmolkova
force-pushed
the
genai-error-type-string
branch
from
July 18, 2026 05:20
ef5bb93 to
1ef02b0
Compare
- Update util tests to construct Error with a string type (new API) - Bridge oldest openai/langchain envs to workspace util-genai (needs 1.1b0) - Rename changelog fragments to PR number, fix typo Assisted-by: Claude Opus 4.8 (1M context)
lmolkova
force-pushed
the
genai-error-type-string
branch
from
July 18, 2026 05:26
1ef02b0 to
25fdea4
Compare
Pull request dashboard statusStatus last refreshed: 2026-07-23 09:31:04 UTC.
This automated status or its linked feedback items may be incorrect. If something looks wrong, please report it with the result you expected. |
rads-1996
reviewed
Jul 22, 2026
| self._self_invocation.request_model = model | ||
|
|
||
| if event_type == "response.completed": | ||
| if event_type in {"response.completed", "response.incomplete"}: |
Contributor
There was a problem hiding this comment.
Why has response.incomplete been moved here?
Member
Author
There was a problem hiding this comment.
because incomplete response is not a failure, it's expressed in finish reason - similarly to chat calls - max_length finish reason is reported as success
rads-1996
approved these changes
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Use canonical exception name when exception is thrown, use proper error codes for openai responses when error is returned