LAM-2172: Fix duplicate LLM spans from litellm's API bridge re-entrancy

[laminar-coding-agent]

Why

LiteLLM re-enters its own public API, and Laminar instruments both entry points.

• litellm.completion() for a model whose model_info["mode"] == "responses" (or a responses/-prefixed model, or gpt-5.4+ with tools + reasoning_effort) routes through litellm/completion_extras/litellm_responses_transformation/handler.py, which does from litellm import responses and calls it.
• The reverse bridge sends litellm.responses() for a chat-only provider (e.g. gemini/*) back through completion().

So one user-visible call emitted two LLM spans carrying identical messages and identical usage — token and cost rollups on the trace were exactly doubled.

On the async path the duplicate isn't even nested: wrap_completion returns a coroutine, so by the time the bridge fires our span is no longer the active OTel span, and the second span lands as a sibling of the caller's parent. That's why it reads as two independent LLM calls rather than an obvious nesting bug.

Real trace that surfaced this (azure/gpt-5.6-sol, APEX trajectory collection): 585dee37-d9ec-e106-9b8a-faf3922bda04 — litellm.completion and litellm.responses as siblings, both 23387 in / 74 out, costed $0.119155 and $0.095028 independently.

What changed

src/lmnr/opentelemetry_lib/opentelemetry/instrumentation/litellm/wrappers/__init__.py:

• wrap_completion and wrap_responses both bail out to a bare wrapped(*args, **kwargs) when is_in_litellm_context(). Whichever entry point the user actually called owns the span; the bridged inner call passes straight through.
• wrap_responses now also enters in_litellm_context() around wrapped() — wrap_completion always did. Without this the guard never trips in the responses → completion direction (and the raw provider instrumentors underneath don't know the call is already traced).

Deduplicating loses nothing: the surviving span still carries full gen_ai.input.messages / gen_ai.output.messages, transformed back into the entry point's own shape. The new sync test asserts that explicitly.

Tests

Three new VCR-backed tests in tests/test_litellm.py, one per direction plus the async variant:

• test_litellm_completion_via_responses_bridge_creates_one_span
• test_litellm_completion_via_responses_bridge_creates_one_span_async (the path that actually bit us)
• test_litellm_responses_via_completion_bridge_creates_one_span

Each asserts exactly one span survives and that its usage is counted once.

Two gotchas worth knowing (also written into CLAUDE.md): mock_response cannot exercise the bridge — litellm.main.completion returns from its if mock_response or ... branch ~25 lines before responses_api_bridge_check runs, hence the cassettes. And the responses/ prefix needs an explicit provider (openai/responses/gpt-4.1-nano); a bare responses/gpt-4.1-nano raises BadRequestError: LLM Provider NOT provided because provider resolution happens first.

Evidence

• tests/test_litellm.py: 34 passed (31 before + 3 new).
• Full suite: 38 failed / 1044 passed, against a stashed-clean baseline of 38 failed / 1041 passed — identical failure set, +3 = exactly the new tests. Pre-existing failures live in test_anthropic/test_thinking.py, test_openai/traces/test_azure.py, test_langchain.py (langgraph), test_observe_concurrency.py, test_tracing*.py, plus 3 bedrock ERRORs.
• Reverse direction confirmed empirically: stashing only src/ and re-running gives 2 spans of 8/8 tokens; with the fix, 1.

Full end-to-end runs — two APEX agent tasks re-run against the editable SDK, each a same-task/same-model before/after, with the harness's own usage.call_log as ground truth:

	task / model	steps (real LLM calls)	LLM spans	trace tokens	trace cost
before	task_699ea5ca / gpt-5.6-terra	10	20 (10 completion + 10 responses)	587,530 / 4,704	$0.298273
after	task_699ea5ca / gpt-5.6-terra	9	9 (completion only)	266,117 / 3,303	$0.156983
before	task_0dab6565 / claude-sonnet-5	4	4 (completion only)	209,448 / 2,518	$0.202708
after	task_0dab6565 / claude-sonnet-5	4	4 (completion only)	209,478 / 2,312	$0.200686

Before, the trace rollup was exactly 2x the truth (the harness reported 293,765 / 2,352 and $0.149136 for that run). After, the trace rollup matches the harness's totals exactly, and the nine spans' per-call (input, output) pairs reproduce the nine call_log entries in order — 23206/78, 23337/103, 24309/181, 24791/389, 31165/303, 33628/1162, 34853/515, 35228/435, 35600/137 — all parented to the apex root, no siblings.

The Claude run is the control: anthropic/* never hits the bridge, and it is unchanged before and after, so the guard doesn't suppress legitimate spans.

Post-fix traces: ae3a6803-763c-612c-d697-93e9b7085309 (gpt), 45fe4d74-f45a-1e1c-f7d4-2fe9cf6d50cf (claude).

Memory

CLAUDE.md gains two bullets in the litellm section: the bridge re-entrancy invariant (including the sibling-not-child async detail and the requirement that wrap_responses enter the context), and the mock_response / model-prefix testing gotchas.

🤖 Generated with Claude Code