feat(langchain): add ChatAnthropic support, including tool-calling, to langchain instrumentation#188
Conversation
There was a problem hiding this comment.
Pull request overview
Adds ChatAnthropic coverage to the LangChain GenAI instrumentation so Anthropic chat invocations emit spans and model/request parameters are captured consistently with existing OpenAI/Bedrock handling.
Changes:
- Extend the
on_chat_model_startallowlist and parameter key extraction to supportChatAnthropic(model/stop/max_tokens fallbacks). - Add a
ChatAnthropictest fixture, dependency, and a VCR-backed test/cassette to validate span creation and model attribution.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/callback_handler.py | Adds ChatAnthropic to supported chat models and introduces Anthropic-specific request param fallbacks. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_llm_call.py | Adds a VCR-backed test asserting Anthropic invocation produces a span and captures the request model. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/conftest.py | Adds a ChatAnthropic fixture for use in tests. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/requirements.txt | Adds langchain-anthropic to the test dependencies. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/cassettes/test_chat_anthropic_claude_sonnet_llm_call.yaml | Adds the recorded cassette for the new Anthropic test. |
Pull request dashboard statusStatus last refreshed: 2026-07-22 06:19:59 UTC.
This automated status or its linked feedback items may be incorrect. If something looks wrong, report it with the result you expected. |
|
question(non-blocking): Can you please check if we need to add tool support for anthropic? |
|
@eternalcuriouslearner Good question — the existing on_llm_end handler already captures tool calls via chat_generation.message.tool_calls, which reads from LangChain's unified message format and is provider-agnostic. So tool calls through ChatAnthropic should flow through that path without additional changes. That said, I haven't tested a tool-calling scenario with ChatAnthropic specifically — happy to add a test for it if you'd like that coverage before merging. |
That would be great. Please add a test with vcr cassettes if possible. I just want to ensure we are supporting the current workflows this instrumentation supports for bedrock and openai. |
|
@eternalcuriouslearner Added the VCR tool-calling test for ChatAnthropic. While validating it, found a real bug: the finish_reason extraction was looking for Bedrock's camelCase stopReason key, but ChatAnthropic's response_metadata uses snake_case stop_reason — so tool-call responses were falling through to unknown instead of the correct reason. Fixed the key lookup and confirmed the fix with the new test. All checks are green now. |
|
@bhumikadangayach Some of these changes have been addressed in PR - #129. Do you mind rebasing, I believe the tests still do apply and probably the changes for stop sequences. Thanks. |
@rads-1996, @wrisa can you please take a look into this pr when you have sometime. Thanks a lot in advance. |
7544771 to
bc6b94c
Compare
|
Rebased onto main — thanks for the heads up on #129. The allowlist change was fully superseded by the generic model key lookup there, so I dropped it. Kept the stop_sequences/max_tokens None-check fixes and the Anthropic tool-calling test/fixture, since those weren't covered by #129. All local tests pass (156/156) against the rebased code. |
|
howdy @bhumikadangayach I am looking into this sorry for the delay. Will be getting back on this. |
rads-1996
left a comment
rads-1996
left a comment
There was a problem hiding this comment.
LGTM. Nit: Perhaps the changelog can be rephrased, since the callback handler mostly handles None conditions.
|
🙌🙌🙌🙌🙌🙌🙌
- Blog: http://salaboy.com <http://salaboy.wordpress.com>
- Github user: http://github.com/salaboy
- Twitter: http://twitter.com/salaboy
- Mauricio "Salaboy" Salatino -
…On Fri, 10 Jul 2026 at 16:35, Radhika Gupta ***@***.***> wrote:
***@***.**** approved this pull request.
LGTM. Nit: Perhaps the changelog can be rephrased, since the callback
handler mostly handles None conditions.
—
Reply to this email directly, view it on GitHub
<#188?email_source=notifications&email_token=AACCMXQYMGKR2ZS6KJ22PJ35EEELPA5CNFSNUABKM5UWIORPF5TWS5BNNB2WEL2QOVWGYUTFOF2WK43UKJSXM2LFO4XTINRXGI4DIMBYG4Y2M4TFMFZW63VHNVSW45DJN5XKKZLWMVXHJLDGN5XXIZLSL5RWY2LDNM#pullrequestreview-4672840871>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AACCMXSXYKO4L3WQPSQPYXT5EEELPAVCNFSNUABGKJSXA33TNF2G64TZHMYTEMZXGAYDENJVHE5US43TOVSTWNBXG44TQMRVGQZDNILWAI>
.
You are receiving this because you were mentioned.Message ID:
<open-telemetry/opentelemetry-python-genai/pull/188/review/4672840871@
github.com>
|
03e78f5 to
3961919
Compare
|
Thanks for the review, @rads-1996! Good point on the changelog — will tighten the wording to reflect that this is mostly correcting existing None-handling rather than adding new behavior. And @salaboy — glad this finally landed properly, appreciate you flagging the original gap back in the contrib repo! |
3b20249 to
5e468fd
Compare
|
Hi @eternalcuriouslearner — I think I've addressed both of your requested changes (added the VCR coverage in cfaa251/19b2587 and added the AI-generated header to both cassettes). All 66 checks are green now. Let me know if there's anything else needed! |
40e3c5d to
2e7a0c7
Compare
f9a2ef0 to
03f38a7
Compare
…with tool_use convention
Adds a VCR-backed test exercising the else branch in the stop/stop_sequences
fallback (params.get('stop') is None, params.get('stop_sequences') set),
which was previously untested. Cassette is AI-synthesized pending re-record
against the live API.
Assisted-by: Claude Sonnet 5
…ENTS.md The test_chat_anthropic_claude_sonnet_tool_call.yaml cassette was missing the required disclosure header for synthetic cassettes.
The langchain-conformance tox env was missing langchain-anthropic, causing a ModuleNotFoundError once conftest.py imports ChatAnthropic unconditionally (surfaced after rebasing onto upstream/main).
Upstream's rebase added body to the VCR match_on config, and the existing tool_call cassette was a hand-approximated recording that no longer matched the actual payload langchain-anthropic produces (max_tokens, temperature, and tool schema field order had drifted). Rebuilt the cassette body from the real payload captured via ChatAnthropic._get_request_payload() using the exact fixture params. Also removes a redundant inline import in the stop_sequences test flagged by ruff's PLC0415 (import already exists at module level).
The oldest tox env was missing langchain-anthropic entirely, causing the same ModuleNotFoundError seen earlier in the conformance env. Pinned to 0.3.0, matching the era of the other oldest-supported provider pins (langchain-openai==0.2.0, langchain-google-genai==2.0.0) and requiring langchain-core>=0.3.0, compatible with this package's own langchain>=0.3.21 floor.
…stop_sequences
Per eternalcuriouslearner's suggestion: when stop_sequences is passed
at ChatAnthropic construction time rather than via bind() or invoke(),
it only appears in serialized['kwargs'], never in invocation_params.
Confirmed via debug prints (removed before commit) that the current
two-level fallback (params.get('stop') / params.get('stop_sequences'))
silently misses this case. Added a third fallback checking
serialized.get('kwargs', {}).get('stop_sequences'), with a new VCR
test covering the constructor-time path specifically.
…ty, fix typecheck Explicitly pins max_tokens=1024 in the constructor-fallback test so the request body is identical across langchain-anthropic versions (the library's own default changed between the oldest-pinned 0.3.0 and latest, causing a VCR body-match failure on the oldest env). Also fixes a pyright reportUnknownVariableType/reportUnknownMemberType error by giving serialized_kwargs an explicit dict[str, Any] annotation.
03f38a7 to
216acf7
Compare
Co-authored-by: Liudmila Molkova <neskazu@gmail.com>
github-project-automation
Bot
moved this to Done
in GenAI Semantic Conventions and Instrumentation libraries
Right at the very top, as its own line before the opening sentence—that's the GitHub convention that triggers auto-closing the issue on merge. Here's the final version:
Fixes #139
This PR adds support for ChatAnthropic in the LangChain instrumentation, including tool-calling support and a small finish_reason bug fix I found while working on it.
What changed
Added Anthropic-specific parameter fallbacks:
model for the model name (Anthropic uses "model" instead of "model_name/model_id")
stop_sequences for stop parameters
max_tokens
Extended stop_sequences fallback support to cover the following:
the .bind() path
constructor-time values from serialized["kwargs"]
Added VCR-backed tests for:
a regular ChatAnthropic request
a tool-calling request
Fixed finish_reason extraction. It was only checking Bedrock's stopReason (camelCase), while Anthropic returns stop_reason (snake case). Because of this, Anthropic tool-call responses were ending up with "unknown" as the finish reason. I found this while writing the tool-calling test and verified the fix.
Added langchain-anthropic to the test dependencies (latest and oldest supported versions), updated the tool-call cassette to match the actual request payload, and included the required AI-generated-content header for the synthetic cassettes.
Notes
While rebasing onto main, I dropped the allowlist changes from my original approach because they were already covered by #129's generic model-key lookup. The Anthropic-specific fixes, None handling, and the new test coverage were still missing, so those are included in this PR.
This work is based on the original prototype in opentelemetry-python-contrib#4496 by @salaboy, before the repository migration.