openai-v2: StreamWrapper.headers fails for with_raw_response streaming even after #4184 fix

[adityamehra]

Description

The fix in open-telemetry/opentelemetry-python-contrib#4184 added __getattr__ to StreamWrapper to proxy unknown attributes to self.stream. However, accessing .headers still fails when using with_raw_response.create(stream=True) with LiteLLM (or any client that accesses raw_response.headers directly on the stream wrapper).

This is a follow-up to open-telemetry/opentelemetry-python-contrib#4032 and open-telemetry/opentelemetry-python-contrib#4113.

Note: Originally filed as open-telemetry/opentelemetry-python-contrib#4606 — moving here as the openai-v2 instrumentation now lives in this repo.

Root Cause

__getattr__ proxies to self.stream, which is an AsyncStream (the result of calling result.parse()). However, AsyncStream does not have a .headers attribute — headers live on the original LegacyAPIResponse that was discarded before wrapping.

The call chain in patch.py:

result = await wrapped(*args, **kwargs)        # LegacyAPIResponse (has .headers)
parsed_result = result.parse()                  # AsyncStream (no .headers)
return StreamWrapper(parsed_result, ...)        # self.stream = AsyncStream

When a caller then does:

headers = dict(raw_response.headers)            # StreamWrapper.__getattr__("headers")
                                                # → getattr(AsyncStream, "headers")
                                                # → AttributeError  ← still crashes!

Steps to Reproduce

import asyncio
from unittest.mock import AsyncMock, MagicMock, patch

import httpx
from openai import AsyncAzureOpenAI
from opentelemetry.instrumentation.openai_v2 import OpenAIInstrumentor
from opentelemetry.sdk.trace import TracerProvider

SSE_CHUNKS = [
    b'data: {"id":"chatcmpl-123","object":"chat.completion.chunk","model":"gpt-4o","choices":[{"index":0,"delta":{"role":"assistant","content":"Hello"},"finish_reason":null}]}\n\n',
    b'data: {"id":"chatcmpl-123","object":"chat.completion.chunk","model":"gpt-4o","choices":[{"index":0,"delta":{"content":"!"},"finish_reason":"stop"}]}\n\n',
    b"data: [DONE]\n\n",
]

def make_mock_response():
    async def aiter_bytes(_=None):
        for chunk in SSE_CHUNKS:
            yield chunk
    mock_request = MagicMock(spec=httpx.Request)
    mock_request.headers = httpx.Headers({"X-Stainless-Raw-Response": "true"})
    mock_response = MagicMock(spec=httpx.Response)
    mock_response.status_code = 200
    mock_response.headers = httpx.Headers({"content-type": "text/event-stream", "x-request-id": "abc"})
    mock_response.aiter_bytes = aiter_bytes
    mock_response.aclose = AsyncMock()
    mock_response.request = mock_request
    mock_response.http_version = "HTTP/1.1"
    mock_response.elapsed = MagicMock()
    return mock_response

# Verbatim copy of LiteLLM's make_azure_openai_chat_completion_request
async def make_azure_openai_chat_completion_request(azure_client, data, timeout):
    raw_response = await azure_client.chat.completions.with_raw_response.create(
        **data, timeout=timeout
    )
    headers = dict(raw_response.headers)   # <-- crashes here
    response = raw_response.parse()
    return headers, response

async def reproducer():
    OpenAIInstrumentor().instrument(tracer_provider=TracerProvider())
    client = AsyncAzureOpenAI(
        api_key="test", azure_endpoint="https://test.openai.azure.com", api_version="2024-02-15-preview"
    )
    with patch.object(client._client, "send", new_callable=AsyncMock, return_value=make_mock_response()):
        headers, response = await make_azure_openai_chat_completion_request(
            client, {"model": "gpt-4o", "messages": [{"role": "user", "content": "hi"}], "stream": True}, 60.0
        )
    print("headers:", headers)

asyncio.run(reproducer())

Actual Behavior

AttributeError: 'StreamWrapper' object has no attribute 'headers'

Observed in production when OpenAI v2 instrumentation was active and LiteLLM's Azure provider called with_raw_response.create(stream=True).

Expected Behavior

raw_response.headers should return the HTTP response headers from the original LegacyAPIResponse, same as when uninstrumented.
raw_response.parse() should return the StreamWrapper itself for iteration.

Suggested Fix

Capture LegacyAPIResponse.headers before calling .parse(), and store them directly on the wrapper:

# Before result.parse():
raw_headers = getattr(result, "headers", None)
parsed_result = result.parse()
if is_streaming(kwargs):
    return ChatStreamWrapper(parsed_result, ..., raw_headers=raw_headers)

# In ChatStreamWrapper / _ChatStreamMixin:
self._self_raw_headers = raw_headers

@property
def headers(self):
    return self._self_raw_headers

def parse(self):
    return self

Environment

• opentelemetry-instrumentation-openai-v2: latest main
• openai: 1.82.0
• litellm: 1.x (Azure provider)
• Python: 3.11

References

• Original report: openai-v2: StreamWrapper.headers fails for with_raw_response streaming even after #4184 fix opentelemetry-python-contrib#4606
• Related issues: openai-v2: stream wrapper not able to wrap streaming with_raw_response opentelemetry-python-contrib#4032, openai-v2: StreamWrapper missing .headers attribute when using with_raw_response streaming opentelemetry-python-contrib#4113
• Insufficient fix: Fix StreamWrapper missing attribute proxying for with_raw_response streaming opentelemetry-python-contrib#4184
• SDOT fix (complete, with tests): fix(openai-v2): Ports the fix from upstream opentelemetry-python-contrib PR #4184 (fixes upstream issue #4113) signalfx/splunk-otel-python-contrib#331
• Upstream fix (in progress): https://github.com/adityamehra/opentelemetry-python-contrib/tree/fix/stream-wrapper-headers-with-raw-response

[eternalcuriouslearner]

Howdy @adityamehra thanks for posting detailed writeup. Few questions:

1. Are you hitting this with the Responses API instrumentation too, or only chat completions?

2. Heads up that the two paths expose headers differently today. Chat (in fix(openai): expose headers on streaming with_raw_response wrapper #147) exposes them directly as raw_response.headers, to match the uninstrumented LegacyAPIResponse. Responses exposes them via a proxy — raw_response.response.headers:

with client.responses.stream(model="gpt-4o", input="hi") as stream:
    headers = stream.response.headers
    request_id = headers.get("x-request-id")
    for event in stream:
        ...

Which shape does your integration actually call? From the repro it looks like dict(raw_response.headers) (direct), but I want to confirm so we expose the right thing.

3. Given that — do you want chat to keep the direct .headers (as fix(openai): expose headers on streaming with_raw_response wrapper #147 does), or would a .response proxy like responses work for you too?

4. Should we standardize and expose headers across chat and responses the same way? (Anthropic messages already returns native .headers, since its raw response isn't wrapped, so it's likely out of scope.)

5. Could you also try reviewing fix(openai): expose headers on streaming with_raw_response wrapper #147 against your use case and confirm it unblocks you?

Thanks in advance.

[lmolkova]

Our wrappers should be transparent. User should be able to add and remove instrumentation without any behavior changes, so we can't standardize stream APIs. Is it possible to just proxy all methods, vars, and properties to the underlying stream without listing them in wrappers? Otherwise we'll have to play whack-a-mole game with underlying APIs forever

[YuxiangJiangCT]

@lmolkova the proxy's already there — SyncStreamWrapper/AsyncStreamWrapper are wrapt.ObjectProxy subclasses, and the parsed Stream/AsyncStream keeps self.response, so wrapper.response.headers works today. The only gap is the top-level .headers: uninstrumented with_raw_response returns a LegacyAPIResponse exposing .headers directly (what LiteLLM reads here), whereas the stream exposes it at .response.headers. My PR just stashes those headers pre-parse() and re-exposes a top-level .headers alias. So it's really a shape question — keep the alias for drop-in parity, or point callers at the already-working .response.headers? Happy either way.