From 04aa30165d3704c028027f3dadcbc7ff0fbebd2e Mon Sep 17 00:00:00 2001 From: Olzhas Nurpeisov Date: Wed, 26 Aug 2026 09:52:49 +0000 Subject: [PATCH] Fix duplicate LLM spans from litellm's API bridge re-entrancy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit LiteLLM re-enters its own public API, and both entry points are instrumented. `completion()` for a model whose `model_info["mode"] == "responses"` (or a `responses/`-prefixed model, or gpt-5.4+ with tools and `reasoning_effort`) routes through `completion_extras/litellm_responses_transformation`, which does `from litellm import responses` and calls it. The reverse bridge sends `responses()` for a chat-only provider back through `completion()`. The result was that one user-visible call emitted two LLM spans with identical messages AND identical usage, so a trace's token and cost rollups were exactly doubled. On the async path the duplicate was not even nested: `wrap_completion` returns a coroutine, so by the time the bridge fires our span is no longer the active OTel span and the second span lands as a sibling of the caller's parent — reading as two independent LLM calls rather than an obvious nesting bug. Guard both wrappers on `is_in_litellm_context()`: whichever entry point the user called owns the span, the bridged inner call passes straight through. `wrap_responses` also now enters `in_litellm_context()` around `wrapped()` (`wrap_completion` always did), without which the guard never trips in the responses -> completion direction. Nothing is lost by deduplicating — the surviving span still carries the full `gen_ai.input.messages` / `gen_ai.output.messages`, transformed back into the entry point's own shape. Co-Authored-By: Claude Opus 5 --- CLAUDE.md | 3 + .../litellm/wrappers/__init__.py | 37 +++++-- ...via_responses_bridge_creates_one_span.yaml | 93 +++++++++++++++++ ...sponses_bridge_creates_one_span_async.yaml | 93 +++++++++++++++++ ...ia_completion_bridge_creates_one_span.yaml | 58 +++++++++++ tests/test_litellm.py | 99 +++++++++++++++++++ 6 files changed, 375 insertions(+), 8 deletions(-) create mode 100644 tests/cassettes/test_litellm/test_litellm_completion_via_responses_bridge_creates_one_span.yaml create mode 100644 tests/cassettes/test_litellm/test_litellm_completion_via_responses_bridge_creates_one_span_async.yaml create mode 100644 tests/cassettes/test_litellm/test_litellm_responses_via_completion_bridge_creates_one_span.yaml diff --git a/CLAUDE.md b/CLAUDE.md index a3682fe0..a3a158b3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -242,6 +242,9 @@ LMNR_BASE_URL # API base URL (default: https://api.lmnr.ai) - **`_translate_openinference` must NOT trust `openinference.span.kind=TOOL` on a span that carries LLM signals (LAM-1784 follow-up).** litellm's `langfuse_otel` callback runs through arize's `_utils.set_attributes`, which forces `openinference.span.kind=TOOL` on ANY completion that merely passes `tools=[...]` (see `litellm/integrations/arize/_utils.py`: `if (optional_tools or metadata_tools) and span_kind != TOOL: span_kind = TOOL`). That's a genuine LLM call (`litellm_request`, `gen_ai.request.model=...`), not a tool execution — so a `litellm_request` span that came AFTER a real `@observe(as_type="tool")` span (or any tool-calling completion) was leaking `lmnr.span.type=TOOL`. The first plain LLM call (no `tools`) was unaffected because litellm leaves its kind `LLM`. Originally `is_llm` only checked `openinference.span.kind==LLM` OR indexed `llm.{input,output}_messages.*` keys, so the leak was masked ONLY when those indexed message keys happened to be present. Fix: `is_llm` now ALSO trips on any LLM signal — `llm.model_name`, `llm.token_count.{prompt,completion,total}`, or the indexed message keys — and `SPAN_TYPE=TOOL` is set ONLY when none of those are present. Genuine tool spans (TOOL kind, no model/tokens/messages, just `input.value`/`output.value`) still type TOOL. Pinned by `test_translator_does_not_mistype_tool_forced_llm_call_as_tool` + `test_translator_keeps_genuine_tool_span_as_tool`. - **Laminar's own litellm wrapper MUST activate its `litellm.completion` / `litellm.responses` span as the current OTel span around `wrapped()` (`with Laminar.use_span(span):`) — not just create it.** LAM-1784: litellm's `langfuse_otel` (and any base-`OpenTelemetry`-derived) success callback runs synchronously inside `wrapped()` and resolves its parent via `OpenTelemetry._get_span_context`, whose Priority 3 is `trace.get_current_span()`. If our span isn't the active OTel span, the callback latches onto whatever ambient span is current — a user's `@observe`/langfuse root — and, because `_handle_success` then sees a non-None `parent_span` with `USE_OTEL_LITELLM_REQUEST_SPAN` unset, takes its ELSE branch: it creates NO `litellm_request`/`raw_gen_ai_request` span and instead folds its hybrid openinference (`llm.*`, `openinference.span.kind=LLM`) + `langfuse.*` attributes straight onto that root. The bridge translator (or app-server's `gen_ai.*`-keyed `span_type()` heuristic) then mis-marks the root as `LLM`, and no separate litellm span ever arrives — both reported symptoms. Activating our span makes the callback parent onto IT, so litellm's attrs land on the LLM span where they belong and the root stays clean. The wrapper was already doing this on the rollout-mode and async-coroutine paths; the plain sync path was the gap. Side effect: our `litellm.completion` span now ALSO carries litellm's redundant `llm.*`/`langfuse.*`/`openinference.*` attrs on top of our own `gen_ai.*` — harmless (our `lmnr.span.type=LLM` and `gen_ai.*` already won). Regression-pinned by `tests/test_litellm.py::test_litellm_completion_activates_span_for_otel_callbacks` (a bare litellm `CustomLogger` captures `trace.get_current_span()` at success time and asserts it equals our span id; `mock_response`, no network/VCR). app-server does NOT parse openinference/`langfuse.*` attrs, so without the bridge connected the polluted root is not server-classified LLM — but the pollution is still wrong and the fix cleans it at the source. +- **`wrap_completion` and `wrap_responses` MUST bail out to a bare `wrapped(*args, **kwargs)` when `is_in_litellm_context()` — litellm re-enters its OWN public API and both entry points are instrumented.** `completion()` for a model whose `model_info["mode"] == "responses"` (or a `responses/`-prefixed model, or gpt-5.4+ with tools + `reasoning_effort`) routes through `completion_extras/litellm_responses_transformation`, which does `from litellm import responses` and calls it; the reverse bridge sends `responses()` for a chat-only provider (e.g. `gemini/*`) back through `completion()`. Without the guard ONE user call emits TWO LLM spans with identical messages AND identical usage, so the trace's token and cost rollups are exactly doubled. On the async path the duplicate is not even nested: the wrapper returns a coroutine, so by the time the bridge fires our span is no longer the active OTel span and the second span lands as a SIBLING of the caller's parent — which is why it reads as two independent LLM calls rather than an obvious nesting bug. Whichever entry point the user called owns the span; the inner one passes through and loses nothing (the surviving span still carries full `gen_ai.input.messages` / `gen_ai.output.messages`, transformed back to the entry point's own shape). `wrap_responses` also has to ENTER `in_litellm_context()` around `wrapped()` (`wrap_completion` always did) or the guard never trips in the responses→completion direction. Pinned by `test_litellm_completion_via_responses_bridge_creates_one_span{,_async}` and `test_litellm_responses_via_completion_bridge_creates_one_span`. +- **`mock_response` cannot exercise the responses bridge** — `litellm.main.completion` returns from its `if mock_response or mock_tool_calls or mock_timeout:` branch ~25 lines BEFORE `responses_api_bridge_check` runs. Bridge tests need a VCR cassette. The cheapest trigger is the `responses/` model prefix WITH an explicit provider (`model="openai/responses/gpt-4.1-nano"`); a bare `responses/gpt-4.1-nano` raises `BadRequestError: LLM Provider NOT provided` because provider resolution happens first. VCR matches on method+URI only, so a bridge cassette can be copied verbatim from an existing `POST https://api.openai.com/v1/responses` recording. + - `Instruments.PYDANTIC_AI` is **auto-enabled by default** when `pydantic-ai-slim` (or `pydantic-ai`) is installed. When auto-enabled, the overlapping raw-provider instrumentors — OPENAI, ANTHROPIC, GOOGLE_GENAI, GROQ, MISTRAL, COHERE, BEDROCK — are auto-removed from the default set so the same model call isn't traced twice (pydantic_ai emits its own GenAI spans at the model abstraction layer). The exact set lives in `_PYDANTIC_AI_PROVIDER_CONFLICTS` in `src/lmnr/opentelemetry_lib/tracing/instruments.py`. - To opt out of the auto-enable, pass `disabled_instruments={Instruments.PYDANTIC_AI}` to `Laminar.initialize`. To keep both pydantic_ai and the raw SDK instrumentors active (accepting duplicate spans), pass an explicit `instruments` set that includes both. - Installation: `pip install lmnr pydantic-ai-slim>=1.0` — there is no `[pydantic-ai]` extra in `pyproject.toml`. diff --git a/src/lmnr/opentelemetry_lib/opentelemetry/instrumentation/litellm/wrappers/__init__.py b/src/lmnr/opentelemetry_lib/opentelemetry/instrumentation/litellm/wrappers/__init__.py index 7af0c6f4..a2178b53 100644 --- a/src/lmnr/opentelemetry_lib/opentelemetry/instrumentation/litellm/wrappers/__init__.py +++ b/src/lmnr/opentelemetry_lib/opentelemetry/instrumentation/litellm/wrappers/__init__.py @@ -9,6 +9,7 @@ ) from lmnr.opentelemetry_lib.tracing.context import ( in_litellm_context, + is_in_litellm_context, _in_litellm_context, ) from lmnr.sdk.log import get_default_logger @@ -67,6 +68,16 @@ def wrap_completion( kwargs = {} if args is None: args = [] + # LiteLLM bridges between its own two public entry points. A `responses()` + # call for a model that only speaks chat completions is re-entered through + # `completion()` (`completion_extras/litellm_responses_transformation`), and + # the reverse bridge sends `completion()` back through `responses()`. Both + # entry points are instrumented, so without this guard one user-visible call + # yields two LLM spans carrying identical messages and identical usage — + # double-counting tokens and cost. The outermost wrapper owns the span; the + # inner one passes the call straight through. + if is_in_litellm_context(): + return wrapped(*args, **kwargs) span = Laminar.start_span( name=to_wrap["span_name"], span_type=to_wrap["span_type"], @@ -232,6 +243,11 @@ def wrap_responses( kwargs = {} if args is None: args = [] + # See `wrap_completion`: litellm re-enters `responses()` from `completion()` + # for models that only speak the Responses API. Whichever entry point the + # user called owns the span; the bridged inner call passes through. + if is_in_litellm_context(): + return wrapped(*args, **kwargs) span = Laminar.start_span( name=to_wrap["span_name"], span_type=to_wrap["span_type"], @@ -264,18 +280,23 @@ def wrap_responses( # If in rollout mode, delegate to rollout wrapper if rollout_wrapper: with Laminar.use_span(span): - result = rollout_wrapper.wrap_responses( - wrapped, - args, - kwargs, - is_streaming=kwargs.get("stream", False), - ) + with in_litellm_context(): + result = rollout_wrapper.wrap_responses( + wrapped, + args, + kwargs, + is_streaming=kwargs.get("stream", False), + ) else: # See `wrap_completion`: activate our span so litellm's # `langfuse_otel` callback parents its attributes onto the - # `litellm.responses` span instead of the user's `@observe` root. + # `litellm.responses` span instead of the user's `@observe` root, + # and mark the litellm context so a bridged inner `completion()` + # call — plus the raw provider instrumentors underneath it — know + # this call is already traced. with Laminar.use_span(span): - result = wrapped(*args, **kwargs) + with in_litellm_context(): + result = wrapped(*args, **kwargs) # Handle case where async methods call sync methods internally and return a coroutine if iscoroutine(result): diff --git a/tests/cassettes/test_litellm/test_litellm_completion_via_responses_bridge_creates_one_span.yaml b/tests/cassettes/test_litellm/test_litellm_completion_via_responses_bridge_creates_one_span.yaml new file mode 100644 index 00000000..79146e42 --- /dev/null +++ b/tests/cassettes/test_litellm/test_litellm_completion_via_responses_bridge_creates_one_span.yaml @@ -0,0 +1,93 @@ +interactions: +- request: + body: '{"model":"gpt-4.1-nano","input":[{"content":"Be very crisp in your response.","role":"system"},{"content":"What + is the capital of France?","role":"user"}]}' + headers: + Accept: + - '*/*' + Accept-Encoding: + - gzip, deflate, zstd + Connection: + - keep-alive + Content-Length: + - '155' + Content-Type: + - application/json + Host: + - api.openai.com + User-Agent: + - litellm/1.81.0 + method: POST + uri: https://api.openai.com/v1/responses + response: + body: + string: !!binary | + H4sIAAAAAAAAA3xUwXKjMAy95ysYn5uOE0gT8hW9d3YYYUTqrbG8tpxpppN/38EEAtt0b6AnPUvv + yf5aZZnQjThmwmNwlTwUdY1yj7BRMq93Ur6Ue4mylqD2xWFT5uV2h5si3+9qWUPbFOKpp6D6Nyoe + acgGHOLKIzA2FfTYZv9SSrnb5tuEBQaOoa9R1DmDjM1QVIP6OHmKtu+rBRNwCGtjtD2JY/a1yrIs + Ew4u6Pv6Bs9oyKEXqyy7DgePlA+PRu+pr7TRmBRoPf6JaNWlcmjB8EUcM/ksE6btSFY1yKBNmFdq + G9hHxZrsIt7BZ0WRXeSK6QO/g0xkKgVmSddRg6af6eR4XTxv1hYsrbdyu1vLYr0Z5U7E4pi9JSUG + PSYnu3D6j5GYq2RkfZDwUhcllGory32emBMLXxwmHgwBTngHfnIsgYoso703NW9sQTuqgp88VacE + sJYYRiXffi1AQyfnqX6AJKJjJl7B6/AsJuh6+5qyhSeTOoAQdGCwPCT3iSlJOPBgDJqlOezjsIHO + Y0Cr8MGSOI9nTTFU4/5XyYvJV+epc1wpUO9YfeDlR8xjr6ImO8/wCIHsYvmxbcnzLKn3J3Yd+JF7 + ugsBWuRLpZueuNW42PyA/qwVVqzHu9RCNIMzIjB5nCvA2Dn0wDGFN7fhbw7cOmvJd3D/nzmf8gbJ + bx2f0dcUdJJSdNjo2N3v8GDCO2k1uBaZxATcF0EwuWq2HnIKunmPPloFN2FFowPUZnxwYlrzaQBt + F7d2u3v6Hp89BdOYycDmXigXo/77GOSP4o9oJ/N/YmZiMLN+D5OCMSzN7pChAYae/rq6/gUAAP// + AwBZwZbm/gUAAA== + headers: + CF-RAY: + - 9c1773efaa76940c-LHR + Connection: + - keep-alive + Content-Encoding: + - gzip + Content-Type: + - application/json + Date: + - Wed, 21 Jan 2026 14:20:33 GMT + Server: + - cloudflare + Set-Cookie: + - __cf_bm=q4mdigQAA0Wd4QQdsGk08T8EqLvxjRGlXuH40Byz21I-1769005233-1.0.1.1-VghArTCk4TVMFIM8YsoKghYOe2z8VdwAl0DfPGj1fEllKCTmeo.p5J7h.xZzNS9CmjW5T8.7Tmvcax.XYjVt.5koj83qagPMh4osDSHY1mQ; + path=/; expires=Wed, 21-Jan-26 14:50:33 GMT; domain=.api.openai.com; HttpOnly; + Secure; SameSite=None + - _cfuvid=Bhb9mODzqL5tu_cm.Nd5X4ROFg7_gMC7RhsjJwuNJU8-1769005233129-0.0.1.1-604800000; + path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None + Strict-Transport-Security: + - max-age=31536000; includeSubDomains; preload + Transfer-Encoding: + - chunked + X-Content-Type-Options: + - nosniff + alt-svc: + - h3=":443"; ma=86400 + cf-cache-status: + - DYNAMIC + openai-organization: + - user-xzaeeoqlanzncr8pomspsknu + openai-processing-ms: + - '402' + openai-project: + - proj_aT4OgTR5NJ9iNjg4xWc82hiE + openai-version: + - '2020-10-01' + x-envoy-upstream-service-time: + - '405' + x-ratelimit-limit-requests: + - '30000' + x-ratelimit-limit-tokens: + - '150000000' + x-ratelimit-remaining-requests: + - '29999' + x-ratelimit-remaining-tokens: + - '149999955' + x-ratelimit-reset-requests: + - 2ms + x-ratelimit-reset-tokens: + - 0s + x-request-id: + - req_693fa2662a9e47b9bb23b6997a1442e7 + status: + code: 200 + message: OK +version: 1 diff --git a/tests/cassettes/test_litellm/test_litellm_completion_via_responses_bridge_creates_one_span_async.yaml b/tests/cassettes/test_litellm/test_litellm_completion_via_responses_bridge_creates_one_span_async.yaml new file mode 100644 index 00000000..79146e42 --- /dev/null +++ b/tests/cassettes/test_litellm/test_litellm_completion_via_responses_bridge_creates_one_span_async.yaml @@ -0,0 +1,93 @@ +interactions: +- request: + body: '{"model":"gpt-4.1-nano","input":[{"content":"Be very crisp in your response.","role":"system"},{"content":"What + is the capital of France?","role":"user"}]}' + headers: + Accept: + - '*/*' + Accept-Encoding: + - gzip, deflate, zstd + Connection: + - keep-alive + Content-Length: + - '155' + Content-Type: + - application/json + Host: + - api.openai.com + User-Agent: + - litellm/1.81.0 + method: POST + uri: https://api.openai.com/v1/responses + response: + body: + string: !!binary | + H4sIAAAAAAAAA3xUwXKjMAy95ysYn5uOE0gT8hW9d3YYYUTqrbG8tpxpppN/38EEAtt0b6AnPUvv + yf5aZZnQjThmwmNwlTwUdY1yj7BRMq93Ur6Ue4mylqD2xWFT5uV2h5si3+9qWUPbFOKpp6D6Nyoe + acgGHOLKIzA2FfTYZv9SSrnb5tuEBQaOoa9R1DmDjM1QVIP6OHmKtu+rBRNwCGtjtD2JY/a1yrIs + Ew4u6Pv6Bs9oyKEXqyy7DgePlA+PRu+pr7TRmBRoPf6JaNWlcmjB8EUcM/ksE6btSFY1yKBNmFdq + G9hHxZrsIt7BZ0WRXeSK6QO/g0xkKgVmSddRg6af6eR4XTxv1hYsrbdyu1vLYr0Z5U7E4pi9JSUG + PSYnu3D6j5GYq2RkfZDwUhcllGory32emBMLXxwmHgwBTngHfnIsgYoso703NW9sQTuqgp88VacE + sJYYRiXffi1AQyfnqX6AJKJjJl7B6/AsJuh6+5qyhSeTOoAQdGCwPCT3iSlJOPBgDJqlOezjsIHO + Y0Cr8MGSOI9nTTFU4/5XyYvJV+epc1wpUO9YfeDlR8xjr6ImO8/wCIHsYvmxbcnzLKn3J3Yd+JF7 + ugsBWuRLpZueuNW42PyA/qwVVqzHu9RCNIMzIjB5nCvA2Dn0wDGFN7fhbw7cOmvJd3D/nzmf8gbJ + bx2f0dcUdJJSdNjo2N3v8GDCO2k1uBaZxATcF0EwuWq2HnIKunmPPloFN2FFowPUZnxwYlrzaQBt + F7d2u3v6Hp89BdOYycDmXigXo/77GOSP4o9oJ/N/YmZiMLN+D5OCMSzN7pChAYae/rq6/gUAAP// + AwBZwZbm/gUAAA== + headers: + CF-RAY: + - 9c1773efaa76940c-LHR + Connection: + - keep-alive + Content-Encoding: + - gzip + Content-Type: + - application/json + Date: + - Wed, 21 Jan 2026 14:20:33 GMT + Server: + - cloudflare + Set-Cookie: + - __cf_bm=q4mdigQAA0Wd4QQdsGk08T8EqLvxjRGlXuH40Byz21I-1769005233-1.0.1.1-VghArTCk4TVMFIM8YsoKghYOe2z8VdwAl0DfPGj1fEllKCTmeo.p5J7h.xZzNS9CmjW5T8.7Tmvcax.XYjVt.5koj83qagPMh4osDSHY1mQ; + path=/; expires=Wed, 21-Jan-26 14:50:33 GMT; domain=.api.openai.com; HttpOnly; + Secure; SameSite=None + - _cfuvid=Bhb9mODzqL5tu_cm.Nd5X4ROFg7_gMC7RhsjJwuNJU8-1769005233129-0.0.1.1-604800000; + path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None + Strict-Transport-Security: + - max-age=31536000; includeSubDomains; preload + Transfer-Encoding: + - chunked + X-Content-Type-Options: + - nosniff + alt-svc: + - h3=":443"; ma=86400 + cf-cache-status: + - DYNAMIC + openai-organization: + - user-xzaeeoqlanzncr8pomspsknu + openai-processing-ms: + - '402' + openai-project: + - proj_aT4OgTR5NJ9iNjg4xWc82hiE + openai-version: + - '2020-10-01' + x-envoy-upstream-service-time: + - '405' + x-ratelimit-limit-requests: + - '30000' + x-ratelimit-limit-tokens: + - '150000000' + x-ratelimit-remaining-requests: + - '29999' + x-ratelimit-remaining-tokens: + - '149999955' + x-ratelimit-reset-requests: + - 2ms + x-ratelimit-reset-tokens: + - 0s + x-request-id: + - req_693fa2662a9e47b9bb23b6997a1442e7 + status: + code: 200 + message: OK +version: 1 diff --git a/tests/cassettes/test_litellm/test_litellm_responses_via_completion_bridge_creates_one_span.yaml b/tests/cassettes/test_litellm/test_litellm_responses_via_completion_bridge_creates_one_span.yaml new file mode 100644 index 00000000..baaf135d --- /dev/null +++ b/tests/cassettes/test_litellm/test_litellm_responses_via_completion_bridge_creates_one_span.yaml @@ -0,0 +1,58 @@ +interactions: +- request: + body: '{"contents":[{"role":"user","parts":[{"text":"What is the capital of France?"}]}],"generationConfig":{}}' + headers: + Accept: + - '*/*' + Accept-Encoding: + - gzip, deflate, zstd + Connection: + - keep-alive + Content-Length: + - '104' + Content-Type: + - application/json + Host: + - generativelanguage.googleapis.com + User-Agent: + - litellm/1.81.0 + method: POST + uri: https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash-lite:generateContent + response: + body: + string: !!binary | + H4sIAAAAAAAC/22QS2uEMBSF9/6KkKWMw7Slduiu9AGFPqS1Qx/M4mKuYzAmkmSKrcx/b4zVcWhd + xHDPybmcrw0IoRlIxhlYNPScfLgJIa0/O01Ji9I6YRi5YQ3a7r39107uzmKx6R7RtECSQc0tCKJy + cqNBZki4IWGYgOYmDOd08nI33tez/T6tBHZhlWIoBvtuMNCcS26KJwSjZGd7Th8TOqpcMmzceBEM + C3w03RrY4D1acM1h7EdrrarapqpEeam2vvmyz5pw+le2ypU8UI7i2Z9Qc+VWcjHFNyHrGoLg9suj + u35N6YSCPVg6UPD/dfDLo0e0Qm14z2KDlaMTHc9Po1yAKSKXjj6VajS1kgZvWWdcnjWf8AYnD+9R + WbKXZBHffccXhga74AdkK4q4JQIAAA== + headers: + Alt-Svc: + - h3=":443"; ma=2592000,h3-29=":443"; ma=2592000 + Content-Encoding: + - gzip + Content-Type: + - application/json; charset=UTF-8 + Date: + - Tue, 20 Jan 2026 17:35:47 GMT + Server: + - scaffolding on HTTPServer2 + Server-Timing: + - gfet4t7; dur=273 + Transfer-Encoding: + - chunked + Vary: + - Origin + - X-Origin + - Referer + X-Content-Type-Options: + - nosniff + X-Frame-Options: + - SAMEORIGIN + X-XSS-Protection: + - '0' + status: + code: 200 + message: OK +version: 1 diff --git a/tests/test_litellm.py b/tests/test_litellm.py index 0d05362e..fc0e4c39 100644 --- a/tests/test_litellm.py +++ b/tests/test_litellm.py @@ -1521,6 +1521,105 @@ async def test_litellm_completion_streaming_with_tool_call_async( check_span_has_basic_attributes(spans[0]) +@pytest.mark.vcr(record_mode="once") +def test_litellm_completion_via_responses_bridge_creates_one_span( + span_exporter: InMemorySpanExporter, +): + """A model that only speaks the Responses API makes litellm re-enter its own + public API: `completion()` transforms the request and calls `responses()`. + Both entry points are instrumented, so without a re-entrancy guard a single + user-visible call produced two LLM spans carrying identical messages and + identical usage — doubling the token and cost totals on the trace. + """ + if "OPENAI_API_KEY" not in os.environ: + os.environ["OPENAI_API_KEY"] = "test-key" + + response = litellm.completion( + model="openai/responses/gpt-4.1-nano", + messages=[ + {"content": "Be very crisp in your response.", "role": "system"}, + {"content": "What is the capital of France?", "role": "user"}, + ], + ) + + spans = span_exporter.get_finished_spans() + # The surviving span is the entry point the caller actually invoked, and it + # carries the usage exactly once. + assert [span.name for span in spans] == ["litellm.completion"] + assert spans[0].attributes["gen_ai.usage.input_tokens"] == 25 + assert spans[0].attributes["gen_ai.usage.output_tokens"] == 3 + # Deduplicating must not cost us any detail: the kept span still has the + # full request and response, transformed back to the completions shape. + assert spans[0].attributes["gen_ai.response.model"] == "gpt-4.1-nano" + assert json.loads(spans[0].attributes["gen_ai.input.messages"]) == [ + {"content": "Be very crisp in your response.", "role": "system"}, + {"content": "What is the capital of France?", "role": "user"}, + ] + assert json.loads(spans[0].attributes["gen_ai.output.messages"]) == [ + { + "content": "Paris.", + "role": "assistant", + "tool_calls": None, + "function_call": None, + } + ] + assert response.choices[0].message.content == "Paris." + check_span_has_basic_attributes(spans[0]) + + +@pytest.mark.asyncio +@pytest.mark.vcr(record_mode="once") +async def test_litellm_completion_via_responses_bridge_creates_one_span_async( + span_exporter: InMemorySpanExporter, +): + """Async twin of the above, and the path that actually bit us in production: + the wrapper returns a coroutine, so the nested `responses()` call runs while + the wrapper only holds the litellm context flag — our span is no longer the + active OTel span. The duplicate therefore landed as a *sibling* of the + caller's parent span rather than as a child of `litellm.completion`. + """ + if "OPENAI_API_KEY" not in os.environ: + os.environ["OPENAI_API_KEY"] = "test-key" + + response = await litellm.acompletion( + model="openai/responses/gpt-4.1-nano", + messages=[ + {"content": "Be very crisp in your response.", "role": "system"}, + {"content": "What is the capital of France?", "role": "user"}, + ], + ) + + spans = span_exporter.get_finished_spans() + assert [span.name for span in spans] == ["litellm.completion"] + assert spans[0].attributes["gen_ai.usage.input_tokens"] == 25 + assert spans[0].attributes["gen_ai.usage.output_tokens"] == 3 + assert response.choices[0].message.content + check_span_has_basic_attributes(spans[0]) + + +@pytest.mark.vcr(record_mode="once") +def test_litellm_responses_via_completion_bridge_creates_one_span( + span_exporter: InMemorySpanExporter, +): + """The bridge runs in both directions: a `responses()` call for a provider + that only speaks chat completions is re-entered through `completion()`. Same + guard, opposite nesting — here `litellm.responses` is the span that survives. + """ + if "GEMINI_API_KEY" not in os.environ: + os.environ["GEMINI_API_KEY"] = "test-key" + + litellm.responses( + model="gemini/gemini-2.5-flash-lite", + input=[{"content": "What is the capital of France?", "role": "user"}], + ) + + spans = span_exporter.get_finished_spans() + assert [span.name for span in spans] == ["litellm.responses"] + assert spans[0].attributes["gen_ai.usage.input_tokens"] == 8 + assert spans[0].attributes["gen_ai.usage.output_tokens"] == 8 + check_span_has_basic_attributes(spans[0]) + + @pytest.mark.vcr(record_mode="once") def test_litellm_responses_basic(span_exporter: InMemorySpanExporter): if "OPENAI_API_KEY" not in os.environ: