feat: Support AgentScope GenAI instrumentation#247
Open
123liuziming wants to merge 1 commit into
Open
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new opentelemetry-instrumentation-genai-agentscope package to this monorepo, aiming to instrument AgentScope v1 (wrappers/patching) and v2 (middleware) for GenAI spans/metrics, plus VCR-backed tests and a Weaver live-check conformance scenario. This fits the repo’s goal of providing GenAI client/framework instrumentations built on opentelemetry-util-genai.
Changes:
- Introduces a new AgentScope instrumentation package (instrumentor + v1 wrappers + v2 middleware + converters/utilities).
- Adds unit tests, VCR cassettes, and a v2 orchestration conformance scenario wired into tox.
- Updates the root tox matrix to include AgentScope envs (oldest/latest/conformance + lint).
Reviewed changes
Copilot reviewed 54 out of 55 changed files in this pull request and generated 18 comments.
Show a summary per file
| File | Description |
|---|---|
| tox.ini | Adds AgentScope test/lint/conformance envs and special-case deps install. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/test_v2_instrumentation.py | v2 middleware/instrumentor behavior + error/stream/tool tests + VCR E2E. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/test_utils.py | Unit tests for message/block conversion helpers. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/test_span_content.py | Content-capture behavior tests across layers (VCR-backed). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/test_multimodal.py | Multimodal block/message conversion tests (image/audio/video/etc). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/test_model.py | v1 model span tests (sync/async/streaming + content-capture) (VCR-backed). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/test_instrumentor.py | Instrumentor lifecycle + patching behavior tests. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/test_conformance.py | Conformance runner for AgentScope scenarios (v2-only). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/test_call_chain_reentrancy.py | Proxy-chain reentrancy tests (avoid duplicate spans). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/test_agent.py | v1 agent/tool tests (VCR-backed). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/requirements.oldest.txt | Oldest-deps test requirements (pins stable OTel stack). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/requirements.latest.txt | Latest-deps test requirements (pins stable OTel stack). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/requirements.conformance.txt | Adds opentelemetry-test-utils (weaver helper) via git URL for conformance. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/conftest.py | Package test config, fixtures, VCR config, and version-based test collection. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/conformance/orchestration.py | v2 orchestration conformance scenario (invoke_agent/chat/execute_tool). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/conformance/init.py | Conformance package marker. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/TestSpanContentCapture.test_span_content_with_span_only.yaml | VCR cassette for SPAN_ONLY content capture. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/TestSpanContentCapture.test_span_content_with_span_and_event.yaml | VCR cassette for SPAN_AND_EVENT content capture. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/TestSpanContentCapture.test_span_content_disabled.yaml | VCR cassette for NO_CONTENT content capture. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/TestAgentError.test_agent_with_invalid_model.yaml | VCR cassette for invalid-key/error scenario. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/TestAgentBasic.test_agent_with_tool.yaml | VCR cassette for tool call flow. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/TestAgentBasic.test_agent_simple_call.yaml | VCR cassette for basic agent call. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/TestAgentBasic.test_agent_multiple_turns.yaml | VCR cassette for multi-turn agent flow. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/TestAgentAttributes.test_agent_span_attributes.yaml | VCR cassette for attribute assertions. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_v2_agent_streaming_e2e.yaml | VCR cassette for v2 streaming agent E2E. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_v2_agent_non_streaming_e2e.yaml | VCR cassette for v2 non-streaming agent E2E. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_v2_agent_concurrent_e2e.yaml | VCR cassette for v2 concurrent agent calls. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_model_call_with_parameters.yaml | VCR cassette for model call with parameters. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_model_call_with_messages.yaml | VCR cassette for multi-message model call. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_model_call_with_content_capture.yaml | VCR cassette for content-capture model call. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_model_call_streaming.yaml | VCR cassette for streaming model call. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_model_call_no_content_capture.yaml | VCR cassette for no-content model call. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_model_call_basic.yaml | VCR cassette for basic model call. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/test_model_call_async.yaml | VCR cassette for async model call. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/cassettes/orchestration_conformance.yaml | VCR cassette for conformance orchestration scenario. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/_test_helpers.py | Shared test helpers for span searching + removed-telemetry assertions. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/init.py | Tests package marker. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/src/opentelemetry/instrumentation/genai/agentscope/version.py | New package version module. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/src/opentelemetry/instrumentation/genai/agentscope/patch.py | v1 tool-call patch wrapper. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/src/opentelemetry/instrumentation/genai/agentscope/package.py | Dependency-range detection based on installed AgentScope major version. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/src/opentelemetry/instrumentation/genai/agentscope/_wrapper.py | v1 model/agent/embedding wrappers incl. streaming handling + reentrancy guard. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/src/opentelemetry/instrumentation/genai/agentscope/_v2_middleware.py | v2 middleware instrumentation for reply/model_call/acting hooks. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/src/opentelemetry/instrumentation/genai/agentscope/init.py | Instrumentor implementation for v1/v2 + patching/unwrap logic. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/src/opentelemetry/instrumentation/genai/agentscope/utils.py | Message/tool/model conversion and invocation setup helpers. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/README.rst | Package README (install/usage). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/pyproject.toml | New package build metadata, deps, entry point, towncrier config. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/LICENSE | Apache-2.0 license file. |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/.gitignore | Package-local ignore patterns (examples env files). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/.changelog/+migrate-agentscope.added | Towncrier fragment (currently misnamed). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/.changelog/+agentscope-v2-chat-attributes.fixed | Towncrier fragment (currently misnamed). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/.changelog/+agentscope-conformance-tests.added | Towncrier fragment (currently misnamed). |
| instrumentation/opentelemetry-instrumentation-genai-agentscope/.changelog/.gitignore | Keeps .gitignore tracked in .changelog/. |
Pull request dashboard statusStatus last refreshed: 2026-07-21 05:29:18 UTC.
This automated status or its linked feedback items may be incorrect. If something looks wrong, report it with the result you expected. |
123liuziming
changed the title
123liuziming
changed the title
123liuziming
changed the title
lmolkova
reviewed
Jul 14, 2026
lmolkova
left a comment
lmolkova
left a comment
Member
There was a problem hiding this comment.
LGTM, just some minor questions and concerns around build and release
| f"expected at least one execute_tool span, saw {operations}" | ||
| ) | ||
| assert agent_names, ( | ||
| "invoke_agent span must carry gen_ai.agent.name; " |
Member
There was a problem hiding this comment.
it might make sense to add it to the list of expected attributes for everyone in
- I don't think we have any frameworks that don't have agent name.
Author
There was a problem hiding this comment.
Should I modify the genai_span_validation.rego in this PR or should I raise a new PR for this?
Contributor
|
@lmolkova do you think we should separate this into two PRs like say:
This might basically reduce the cognitive load on reviewers. Wdyt? |
Member
|
@eternalcuriouslearner it's not a super-strong opinion, but I'd prefer to have minimal viable instrumentation in one PR.
|
Author
|
Thanks for your review, I'll resolve these comments as soon as possible. |
|
|
lmolkova
requested changes
Jul 21, 2026
lmolkova
left a comment
lmolkova
left a comment
Member
There was a problem hiding this comment.
it seems agentscope comes with native otel instrumentation (but in a broken way since it depends on the exporter) - do we know if instrumentation is good? does it follow conventions?
Since there is an existing instrumentation, we should not invent another one. Was there an attempt to contribute directly to https://github.com/agentscope-ai/agentscope if there any gaps?
123liuziming
force-pushed
the
feat/agentscope
branch
from
July 21, 2026 02:31
3436922 to
61bcd78
Compare
Add the opentelemetry-instrumentation-genai-agentscope package with v1/v2
support, chat/tool/agent spans, streaming, content capture, and a Weaver
conformance scenario.
Rebased onto current main and addressed review feedback:
- Follow the established dependency/versioning pattern in pyproject.toml
(>= x, <N bounds; packaging/wrapt come transitively from
opentelemetry-instrumentation).
- tests/requirements.{oldest,latest}.txt follow the shared pattern
(oldest floors via UV_RESOLUTION=lowest-direct; no per-package OTel pins).
- Drop the conformance special-casing: agentscope resolves cleanly against
dev-requirements-conformance.txt like the other packages, so remove the
extra requirements.conformance.txt, the --no-deps weaver install, and the
conformance-!agentscope exclusion.
- Conformance scenario uses expected_spans={op: count}; drop redundant
operation-count assertions.
- Call wrapt.wrap_function_wrapper positionally and assert against
wrapt.FunctionWrapper (wrapt 2.x no longer subclasses ObjectProxy).
- Version 1.1b0.dev; list the package in the release workflows.
Change-Id: Ifdb8abbdc3db5c905799f74daadd6e79da30c74d
Co-developed-by: Claude <noreply@anthropic.com>
Assisted-by: Claude Opus 4.8
123liuziming
force-pushed
the
feat/agentscope
branch
from
July 21, 2026 02:37
61bcd78 to
69707e8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add AgentScope GenAI instrumentation with support for AgentScope v1 and v2, including model calls, agent invocation, tool execution, content capture, and conformance coverage.
This change adds a new
opentelemetry-instrumentation-genai-agentscopepackage and wires it into the workspace, tox matrix, changelog, and lockfile. It also adds Weaver live-check conformance coverage for AgentScope v2 agent orchestration coveringinvoke_agent,chat, andexecute_tool.Fixes #185
Type of change
How has this been tested?
ruff check instrumentation/opentelemetry-instrumentation-genai-agentscope/src/opentelemetry/instrumentation/genai/agentscope/_v2_middleware.py instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/conformance/orchestration.pyruff format --check instrumentation/opentelemetry-instrumentation-genai-agentscope/src/opentelemetry/instrumentation/genai/agentscope/_v2_middleware.py instrumentation/opentelemetry-instrumentation-genai-agentscope/tests/conformance/orchestration.pyuvx --with tox-uv tox -e py312-test-instrumentation-genai-agentscope-conformance -- -raThe conformance test passed locally with Weaver live-check enabled: