diff --git a/docs/platforms/python/tracing/instrumentation/automatic-instrumentation.mdx b/docs/platforms/python/tracing/instrumentation/automatic-instrumentation.mdx index fc44a217d9a721..e079f89270ef34 100644 --- a/docs/platforms/python/tracing/instrumentation/automatic-instrumentation.mdx +++ b/docs/platforms/python/tracing/instrumentation/automatic-instrumentation.mdx @@ -17,7 +17,7 @@ supported: description: "Learn what instrumentation automatically captures transactions." --- -Many integrations for popular frameworks automatically capture transactions. If you already have any of the following frameworks set up for Sentry error reporting, you will start to see traces immediately: +Many integrations for popular frameworks automatically capture transactions (or service spans in stream mode). If you already have any of the following frameworks set up for Sentry error reporting, you will start to see traces immediately: - All WSGI-based web frameworks (Django, Flask, Pyramid, Falcon, Bottle) - Celery @@ -26,11 +26,13 @@ Many integrations for popular frameworks automatically capture transactions. If See the full [list of available integrations](/platforms/python/integrations/). -Spans are instrumented for the following operations within a transaction: +Spans are instrumented for the following operations within a transaction/service span: - Database queries that use SQLAlchemy or the Django ORM - HTTP requests made with HTTPX, requests, the stdlib, AIOHTTP, or pyreqwest - Spawned subprocesses - Redis operations -Spans are only created within an existing transaction. If you're not using any of the supported frameworks, you'll need to create transactions manually. +In transaction mode, spans are only created within an existing transaction. If you're not using any of the supported frameworks, you'll need to create transactions manually. + +Stream mode removes this limitation. Since there are no transactions, any span started without a parent is automatically promoted to a service span (the equivalent of a transaction). You can also force any span to become a service span when starting it by removing its parent. See Custom Instrumentation to learn more. diff --git a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module.mdx b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module.mdx index 566e9c354f64b0..8b307cdb12ab8b 100644 --- a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module.mdx +++ b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/ai-agents-module.mdx @@ -10,6 +10,13 @@ With Sentry AI Agent Monitoringset up tracing. Once this is done, the Python SDK will automatically instrument AI agents created with supported libraries. If that doesn't fit your use case, you can use custom instrumentation described below. + + +This page covers both transaction mode (default) and stream mode. See New Spans to learn more. + + + + ## Automatic Instrumentation The Python SDK supports automatic instrumentation for some AI libraries. We recommend adding their integrations to your Sentry configuration to automatically capture spans for AI agents. @@ -27,7 +34,8 @@ The Python SDK supports automatic instrumentation for some AI libraries. We reco For your AI agents data to show up in the [AI Agents Dashboards](https://sentry.io/orgredirect/organizations/:orgslug/dashboards/?filter=onlyPrebuilt&query=agents&sort=mostPopular), at least one of the AI spans needs to be created and have well-defined names and data attributes. See below. -The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create these spans. +In transaction mode, you can also use the [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator to create these spans, using its `template` parameter. +In stream mode, these spans need to be created directly with `sentry_sdk.traces.start_span()`, as shown below. ## Span Hierarchy @@ -55,7 +63,7 @@ This span represents a request to an LLM model or service that generates a respo #### Example AI Request span -```python +```python {tabTitle:Transaction Mode (Default)} import json import sentry_sdk @@ -78,11 +86,41 @@ with sentry_sdk.start_span(op="gen_ai.chat", name="chat o3-mini") as span: span.set_data("gen_ai.usage.output_tokens", result.usage.completion_tokens) ``` +```python {tabTitle:Stream Mode} +import json +import sentry_sdk + +messages = [{"role": "user", "parts": [{"type": "text", "content": "Tell me a joke"}]}] + +with sentry_sdk.traces.start_span( + name="chat o3-mini", + attributes={"sentry.op": "gen_ai.chat"}, +) as span: + span.set_attributes({ + "gen_ai.operation.name": "chat", + "gen_ai.request.model": "o3-mini", + "gen_ai.provider.name": "openai", + "gen_ai.input.messages": json.dumps(messages), + }) + + result = client.chat.completions.create(model="o3-mini", messages=messages) + + span.set_attributes({ + "gen_ai.response.model": result.model, + "gen_ai.output.messages": json.dumps([ + {"role": "assistant", "parts": [{"type": "text", "content": result.choices[0].message.content}]} + ]), + "gen_ai.response.finish_reasons": json.dumps([result.choices[0].finish_reason]), + "gen_ai.usage.input_tokens": result.usage.prompt_tokens, + "gen_ai.usage.output_tokens": result.usage.completion_tokens, + }) +``` + #### Thinking / reasoning messages Models with extended thinking (such as Anthropic's `thinking` blocks, Gemini's `thought`, or DeepSeek's `reasoning_content`) produce internal reasoning that isn't part of the user-visible reply. Represent this content as a `reasoning` part inside the assistant message, alongside the user-facing `text` part. Sentry surfaces reasoning parts separately and filters them out of the user-facing Conversations view, so don't fold thinking into a `text` part. -```python +```python {tabTitle:Transaction Mode (Default)} import json import sentry_sdk @@ -111,6 +149,45 @@ with sentry_sdk.start_span(op="gen_ai.chat", name="chat o3-mini") as span: span.set_data("gen_ai.usage.output_tokens.reasoning", result.usage.completion_tokens_details.reasoning_tokens) ``` +```python {tabTitle:Stream Mode} +import json +import sentry_sdk + +messages = [{"role": "user", "parts": [{"type": "text", "content": "What is 6 * 7?"}]}] + +with sentry_sdk.traces.start_span( + name="chat o3-mini", + attributes={"sentry.op": "gen_ai.chat"}, +) as span: + span.set_attributes({ + "gen_ai.operation.name": "chat", + "gen_ai.request.model": "o3-mini", + "gen_ai.provider.name": "openai", + "gen_ai.input.messages": json.dumps(messages), + }) + + result = client.chat.completions.create(model="o3-mini", messages=messages) + + span.set_attributes({ + "gen_ai.response.model": result.model, + "gen_ai.output.messages": json.dumps([ + { + "role": "assistant", + "parts": [ + {"type": "reasoning", "content": "6 times 7 is 42."}, + {"type": "text", "content": "The answer is 42."}, + ], + } + ]), + "gen_ai.usage.output_tokens": result.usage.completion_tokens, + }) + # If the provider reports reasoning tokens, record them as a subset of output tokens + span.set_attribute( + "gen_ai.usage.output_tokens.reasoning", + result.usage.completion_tokens_details.reasoning_tokens, + ) +``` + When previous thinking is fed back into a multi-turn request, include the same `reasoning` parts in the assistant messages within `gen_ai.input.messages`. ### Invoke Agent Span @@ -129,7 +206,7 @@ For a complete guide on naming agents across all supported frameworks, see [Nami #### Example of an Invoke Agent Span: -```python +```python {tabTitle:Transaction Mode (Default)} import json import sentry_sdk @@ -147,6 +224,31 @@ with sentry_sdk.start_span(op="gen_ai.invoke_agent", name="invoke_agent Weather span.set_data("gen_ai.usage.output_tokens", final_output.usage.output_tokens) ``` +```python {tabTitle:Stream Mode} +import json +import sentry_sdk + +with sentry_sdk.traces.start_span( + name="invoke_agent Weather Agent", + attributes={"sentry.op": "gen_ai.invoke_agent"}, +) as span: + span.set_attributes({ + "gen_ai.operation.name": "invoke_agent", + "gen_ai.request.model": "o3-mini", + "gen_ai.agent.name": "Weather Agent", + }) + + final_output = my_agent.run() + + span.set_attributes({ + "gen_ai.output.messages": json.dumps([ + {"role": "assistant", "parts": [{"type": "text", "content": str(final_output)}]} + ]), + "gen_ai.usage.input_tokens": final_output.usage.input_tokens, + "gen_ai.usage.output_tokens": final_output.usage.output_tokens, + }) +``` + ### Execute Tool Span This span represents the execution of a tool or function that was requested by an AI model, including the input arguments and resulting output. @@ -157,7 +259,7 @@ This span represents the execution of a tool or function that was requested by a #### Example Execute Tool span -```python +```python {tabTitle:Transaction Mode (Default)} import json import sentry_sdk @@ -171,6 +273,25 @@ with sentry_sdk.start_span(op="gen_ai.execute_tool", name="execute_tool get_weat span.set_data("gen_ai.tool.call.result", json.dumps(result)) ``` +```python {tabTitle:Stream Mode} +import json +import sentry_sdk + +with sentry_sdk.traces.start_span( + name="execute_tool get_weather", + attributes={"sentry.op": "gen_ai.execute_tool"}, +) as span: + span.set_attributes({ + "gen_ai.operation.name": "execute_tool", + "gen_ai.tool.name": "get_weather", + "gen_ai.tool.call.arguments": json.dumps({"location": "Paris"}), + }) + + result = get_weather(location="Paris") + + span.set_attribute("gen_ai.tool.call.result", json.dumps(result)) +``` + ### Handoff Span This span marks the transition of control from one agent to another, typically when the current agent determines another agent is better suited to handle the task. @@ -181,7 +302,7 @@ This span marks the transition of control from one agent to another, typically w #### Example of a Handoff Span -```python +```python {tabTitle:Transaction Mode (Default)} import sentry_sdk with sentry_sdk.start_span(op="gen_ai.handoff", name="handoff from Weather Agent to Travel Agent"): @@ -192,10 +313,28 @@ with sentry_sdk.start_span(op="gen_ai.invoke_agent", name="invoke_agent Travel A pass ``` +```python {tabTitle:Stream Mode} +import sentry_sdk + +with sentry_sdk.traces.start_span( + name="handoff from Weather Agent to Travel Agent", + attributes={"sentry.op": "gen_ai.handoff"}, +): + pass # Handoff span just marks the transition + +with sentry_sdk.traces.start_span( + name="invoke_agent Travel Agent", + attributes={"sentry.op": "gen_ai.invoke_agent"}, +): + # Run the target agent here + pass +``` + ## Tracking Conversations - Tracking Conversations has **beta** stability. Configuration options and behavior may change. + Tracking Conversations has **beta** stability. Configuration options and + behavior may change. For AI applications that involve multi-turn conversations, you can use `sentry_sdk.ai.set_conversation_id()` to associate all AI spans from the same conversation. This enables you to track and analyze complete conversation flows within Sentry. diff --git a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/caches-module.mdx b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/caches-module.mdx index 2ae6ef48f87b88..05a707a3b1327a 100644 --- a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/caches-module.mdx +++ b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/caches-module.mdx @@ -3,15 +3,22 @@ title: Instrument Caches sidebar_order: 1000 description: "Learn how to manually instrument your code to use Sentry's Caches module. " --- + A cache can be used to speed up data retrieval, thereby improving application performance. Because instead of getting data from a potentially slow data layer, your application will be getting data from memory (in a best case scenario). Caching can speed up read-heavy workloads for applications like Q&A portals, gaming, media sharing, and social networking. + + +This page covers both transaction mode (default) and stream mode. See New Spans to learn more. + + + Sentry offers a [cache-monitoring dashboard](https://sentry.io/orgredirect/organizations/:orgslug/dashboards/) that can be auto-instrumented for popular Python caching setups (like Django and Redis). If you're using a custom caching solution that doesn't have auto instrumentation, you can manually instrument it and use Sentry to get a look into how your caching solution is performing by following the setup instructions below. To make it possible for Sentry to give you an overview of your cache performance, you'll need to create two spans - one indicating that something is being put into the cache, and a second one indicating that something is being fetched from the cache. -Make sure that there's a transaction running when you create the spans. If you're using a web framework those transactions will be created for you automatically. See Tracing for more information. +In transaction mode, make sure that there's a transaction running when you create the spans. If you're using a web framework those transactions will be created for you automatically. See Tracing for more information. For detailed information about which data can be set, see the [Cache Module Developer Specification](https://develop.sentry.dev/sdk/performance/modules/caches/). @@ -21,16 +28,16 @@ If you're using anything other than Dja ### Add Span When Putting Data Into the Cache -If the cache you’re using isn’t supported by auto instrumentation mentioned above, you can use the custom instrumentation instructions below to emit cache spans: +If the cache you're using isn't supported by auto instrumentation mentioned above, you can use the custom instrumentation instructions below to emit cache spans: 1. Set the cache value with whatever cache library you happen to be using. -2. Wrap the part of your application that uses the cached value with `with sentry_sdk.start_span(...)` +2. Wrap the part of your application that uses the cached value with `with sentry_sdk.start_span(...)` (transaction mode) or `with sentry_sdk.traces.start_span(...)` (stream mode). 3. Set `op` to `cache.put`. 4. Set `cache.item_size` to an integer representing the size of the cached item. (The steps described above are documented in the snippet.) -```python +```python {tabTitle:Transaction Mode (Default)} import my_caching import sentry_sdk @@ -52,20 +59,43 @@ with sentry_sdk.start_span(op="cache.put") as span: span.set_data("cache.item_size", len(value)) # Warning: if value is very big this could use lots of memory ``` +```python {tabTitle:Stream Mode} +import my_caching +import sentry_sdk + +key = "myCacheKey123" +value = "The value I want to cache." + +with sentry_sdk.traces.start_span(attributes={"sentry.op": "cache.put"}) as span: + # Set a key in your caching using your custom caching solution + my_caching.set(key, value) + + # Describe the cache server you are accessing + span.set_attributes({ + "network.peer.address": "cache.example.com/supercache", + "network.peer.port": 9000, + + # Add the key(s) you want to set + "cache.key": [key], + + # Add the size of the value you stored in the cache + "cache.item_size": len(value), # Warning: if value is very big this could use lots of memory + }) +``` ### Add Span When Retrieving Data From the Cache -If the cache you’re using isn’t supported by auto instrumentation mentioned above, you can use the custom instrumentation instructions below to emit cache spans: +If the cache you're using isn't supported by auto instrumentation mentioned above, you can use the custom instrumentation instructions below to emit cache spans: 1. Fetch the cached value from whatever cache library you happen to be using. -2. Wrap the part of your application that uses the cached value with `with sentry_sdk.start_span(...)` +2. Wrap the part of your application that uses the cached value with `with sentry_sdk.start_span(...)` (transaction mode) or `with sentry_sdk.traces.start_span(...)` (stream mode). 3. Set `op` to `cache.get`. 4. Set `cache.hit` to a boolean value representing whether the value was successfully fetched from the cache or not. 5. Set `cache.item_size` to an integer representing the size of the cached item. (The steps described above are documented in the snippet.) -```python +```python {tabTitle:Transaction Mode (Default)} import my_caching import sentry_sdk @@ -94,4 +124,35 @@ with sentry_sdk.start_span(op="cache.get") as span: span.set_data("cache.hit", False) ``` +```python {tabTitle:Stream Mode} +import my_caching +import sentry_sdk + +key = "myCacheKey123" +value = None + +with sentry_sdk.traces.start_span(attributes={"sentry.op": "cache.get"}) as span: + # Get a key from your caching solution + value = my_caching.get(key) + + # Describe the cache server you are accessing + span.set_attributes({ + "network.peer.address": "cache.example.com/supercache", + "network.peer.port": 9000, + + # Add the key(s) you just retrieved from the cache + "cache.key": [key], + }) + + if value is not None: + # If you retrieved a value, the cache was hit + span.set_attribute("cache.hit", True) + + # Optionally also add the size of the value you retrieved + span.set_attribute("cache.item_size", len(value)) + else: + # If you could not retrieve a value, it was a miss + span.set_attribute("cache.hit", False) +``` + You should now have the right spans in place. Head over to the [Cache dashboard](https://sentry.io/orgredirect/organizations/:orgslug/dashboards/) to see how your cache is performing. diff --git a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/mcp-module.mdx b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/mcp-module.mdx index 74b8af8cac9412..5c951a785c2cb1 100644 --- a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/mcp-module.mdx +++ b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/mcp-module.mdx @@ -8,6 +8,12 @@ With Sentry's [MCP monitoring](/ai/monitoring/mcp/), you can track and debug MCP As a prerequisite to setting up MCP monitoring with Python, you'll need to first set up tracing. Once this is done, the Python SDK will automatically instrument MCP servers created with supported libraries. If that doesn't fit your use case, you can use custom instrumentation described below. + + +This page covers both transaction mode (default) and stream mode. See New Spans to learn more. + + + ## Automatic Instrumentation The Python SDK supports automatic instrumentation for MCP servers. We recommend adding the MCP integration to your Sentry configuration to automatically capture spans for MCP operations. @@ -18,7 +24,8 @@ The Python SDK supports automatic instrumentation for MCP servers. We recommend For your MCP data to show up in Sentry, spans must be created with well-defined names and data attributes. See below for the different types of MCP operations you can instrument. -The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create these spans. +In transaction mode, you can also use the [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator to create these spans, using its `template` parameter. +In stream mode, these spans need to be created directly with `sentry_sdk.traces.start_span()`, as shown below. ## Spans @@ -28,7 +35,7 @@ The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instr #### Example Tool Execution Span: -```python +```python {tabTitle:Transaction Mode (Default)} import sentry_sdk import json @@ -72,13 +79,59 @@ with sentry_sdk.start_span( raise ``` +```python {tabTitle:Stream Mode} +import sentry_sdk +import json + +sentry_sdk.init(...) + +# Example tool execution +tool_name = "get_weather" +tool_arguments = {"city": "San Francisco"} + +with sentry_sdk.traces.start_span( + name=f"tools/call {tool_name}", + attributes={"sentry.op": "mcp.server"}, +) as span: + # Set MCP-specific attributes + span.set_attributes({ + "mcp.tool.name": tool_name, + "mcp.method.name": "tools/call", + + # Set request metadata + "mcp.request.id": "req_123abc", + "mcp.session.id": "session_xyz789", + "mcp.transport": "stdio", # or "http", "sse" for HTTP/WebSocket/SSE + "network.transport": "pipe", # or "tcp" for HTTP/SSE + }) + + # Set tool arguments (optional, if send_default_pii=True) + for key, value in tool_arguments.items(): + span.set_attribute(f"mcp.request.argument.{key}", value) + + # Execute the tool + try: + result = execute_tool(tool_name, tool_arguments) + + # Set result data + span.set_attribute("mcp.tool.result.content", json.dumps(result)) + span.set_attribute("mcp.tool.result.is_error", False) + + # Set result content count if applicable + if isinstance(result, (list, dict)): + span.set_attribute("mcp.tool.result.content_count", len(result)) + except Exception as e: + span.set_attribute("mcp.tool.result.is_error", True) + raise +``` + ### Prompt Retrieval Span #### Example Prompt Retrieval Span: -```python +```python {tabTitle:Transaction Mode (Default)} import sentry_sdk import json @@ -123,13 +176,60 @@ with sentry_sdk.start_span( ) ``` +```python {tabTitle:Stream Mode} +import sentry_sdk +import json + +sentry_sdk.init(...) + +# Example prompt retrieval +prompt_name = "code_review" +prompt_arguments = {"language": "python"} + +with sentry_sdk.traces.start_span( + name=f"prompts/get {prompt_name}", + attributes={"sentry.op": "mcp.server"}, +) as span: + # Set MCP-specific attributes + span.set_attributes({ + "mcp.prompt.name": prompt_name, + "mcp.method.name": "prompts/get", + + # Set request metadata + "mcp.request.id": "req_456def", + "mcp.session.id": "session_xyz789", + "mcp.transport": "http", + "network.transport": "tcp", + }) + + # Set prompt arguments (optional, if send_default_pii=True) + for key, value in prompt_arguments.items(): + span.set_attribute(f"mcp.request.argument.{key}", value) + + # Retrieve the prompt + prompt_result = get_prompt(prompt_name, prompt_arguments) + + # Set result data + messages = prompt_result.get("messages", []) + span.set_attribute("mcp.prompt.result.message_count", len(messages)) + + # For single-message prompts, set role and content + if len(messages) == 1: + span.set_attribute("mcp.prompt.result.message_role", messages[0].get("role")) + # Content is PII, only set if send_default_pii=True + span.set_attribute( + "mcp.prompt.result.message_content", + json.dumps(messages[0].get("content")) + ) +``` + ### Resource Read Span #### Example Resource Read Span: -```python +```python {tabTitle:Transaction Mode (Default)} import sentry_sdk sentry_sdk.init(...) @@ -155,6 +255,34 @@ with sentry_sdk.start_span( resource_data = read_resource(resource_uri) ``` +```python {tabTitle:Stream Mode} +import sentry_sdk + +sentry_sdk.init(...) + +# Example resource access +resource_uri = "file:///path/to/resource.txt" + +with sentry_sdk.traces.start_span( + name=f"resources/read {resource_uri}", + attributes={"sentry.op": "mcp.server"}, +) as span: + # Set MCP-specific attributes + span.set_attributes({ + "mcp.resource.uri": resource_uri, + "mcp.method.name": "resources/read", + + # Set request metadata + "mcp.request.id": "req_789ghi", + "mcp.session.id": "session_xyz789", + "mcp.transport": "http", + "network.transport": "tcp", + }) + + # Access the resource + resource_data = read_resource(resource_uri) +``` + ## Common Span Attributes diff --git a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/queues-module.mdx b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/queues-module.mdx index 5741c398a766b8..4f5212baa3ff6e 100644 --- a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/queues-module.mdx +++ b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/queues-module.mdx @@ -3,26 +3,34 @@ title: Instrument Queues sidebar_order: 3000 description: "Learn how to manually instrument your code to use Sentry's Queues module. " --- + Sentry comes with a [queue-monitoring dashboard](https://sentry.io/orgredirect/organizations/:orgslug/dashboards/) that can be auto-instrumented for popular Python queue setups (like Celery). In case yours isn't supported, you can still instrument custom spans and transactions around your queue producers and consumers to ensure that you have performance data about your messaging queues. + + +This page covers both transaction mode (default) and stream mode. See New Spans to learn more. + + + ## Producer Instrumentation -To start capturing performance metrics, use the `sentry_sdk.start_span()` function to wrap your queue producer events. Your span `op` must be set to `queue.publish`. Include the following span data to enrich your producer spans with queue metrics: +To start capturing performance metrics, use the `start_span()` function to wrap your queue producer events. Your span `op` must be set to `queue.publish`. Include the following span data to enrich your producer spans with queue metrics: -| Data Attribute | Type | Description | -|:--|:--|:--| -| `messaging.message.id ` | string | The message identifier | -| `messaging.destination.name` | string | The queue or topic name | -| `messaging.message.body.size` | int | Size of the message body in bytes | +| Data Attribute | Type | Description | +| :---------------------------- | :----- | :-------------------------------- | +| `messaging.message.id ` | string | The message identifier | +| `messaging.destination.name` | string | The queue or topic name | +| `messaging.message.body.size` | int | Size of the message body in bytes | -Your `queue.publish` span must exist inside a transaction in order to be recognized as a producer span. If you are using a supported web framework, the transaction is created by the integration. If you use plain Python, you can start a new one using `sentry_sdk.start_transaction()`. +In transaction mode, your `queue.publish` span must exist inside a transaction in order to be recognized as a producer span. If you are using a supported web framework, the transaction is created by the integration. If you use plain Python, you can start a new one using `sentry_sdk.start_transaction()`. -You must also include trace headers (`sentry-trace` and `baggage`) in your message so that your consumers can continue your trace once your message is picked up. +In stream mode, there's no separate transaction to depend on. If your `queue.process` span has no parent, it's automatically promoted to a service span. If you'd rather group it under an explicit service span, start one with `sentry_sdk.traces.start_span(parent_span=None)` first. +You must also include trace headers (`sentry-trace` and `baggage`) in your message so that your consumers can continue your trace once your message is picked up. -```python +```python {tabTitle:Transaction Mode (Default)} from datetime import datetime, timezone import sentry_sdk @@ -68,25 +76,74 @@ with sentry_sdk.start_transaction( ) ``` +```python {tabTitle:Stream Mode} +from datetime import datetime, timezone + +import sentry_sdk +import my_custom_queue + +# Initialize Sentry +sentry_sdk.init(...) + +connection = my_custom_queue.connect() + +# The message you want to send to the queue +queue = "messages" +message = "Hello World!" +message_id = "abc123" + +# Create the service span +# If you are using a web framework, the framework integration +# will create this for you and you can omit this. +with sentry_sdk.traces.start_span( + name="queue_producer_transaction", + attributes={"sentry.op": "function"}, + parent_span=None, +): + # Create the span + with sentry_sdk.traces.start_span( + name="queue_producer", + attributes={"sentry.op": "queue.publish"}, + ) as span: + # Set span data + span.set_attributes({ + "messaging.message.id": message_id, + "messaging.destination.name": queue, + "messaging.message.body.size": len(message.encode("utf-8")), + }) + + # Publish the message to the queue (including trace information and current time stamp) + now = int(datetime.now(timezone.utc).timestamp()) + connection.publish( + queue=queue, + body=message, + timestamp=now, + headers={ + "sentry-trace": sentry_sdk.get_traceparent(), + "baggage": sentry_sdk.get_baggage(), + }, + ) +``` ## Consumer Instrumentation -To start capturing performance metrics, use the `sentry_sdk.start_span()` function to wrap your queue consumers. Your span `op` must be set to `queue.process`. Include the following span data to enrich your consumer spans with queue metrics: +To start capturing performance metrics, use the `start_span()` function to wrap your queue consumers. Your span `op` must be set to `queue.process`. Include the following span data to enrich your consumer spans with queue metrics: -| Data Attribute | Type | Description | -|:--|:--|:--| -| `messaging.message.id ` | string | The message identifier | -| `messaging.destination.name` | string | The queue or topic name | -| `messaging.message.body.size` | number | Size of the message body in bytes | -| `messaging.message.retry.count ` | number | The number of times a message was attempted to be processed | +| Data Attribute | Type | Description | +| :----------------------------------- | :----- | :------------------------------------------------------------------ | +| `messaging.message.id ` | string | The message identifier | +| `messaging.destination.name` | string | The queue or topic name | +| `messaging.message.body.size` | number | Size of the message body in bytes | +| `messaging.message.retry.count ` | number | The number of times a message was attempted to be processed | | `messaging.message.receive.latency ` | number | The time in milliseconds that a message awaited processing in queue | -Your `queue.process` span must exist inside a transaction in order to be recognized as a consumer span. If you are using a supported web framework, the transaction is created by the integration. If you use plain Python, you can start a new one using `sentry_sdk.start_transaction()`. +In transaction mode, your `queue.process` span must exist inside a transaction in order to be recognized as a consumer span. If you are using a supported web framework, the transaction is created by the integration. If you use plain Python, you can start a new one using `sentry_sdk.start_transaction()`. -Use `sentry_sdk.continue_trace()` to connect your consumer spans to their associated producer spans, and `transaction.set_status()` to mark the trace of your message as success or failed. +In stream mode, there's no separate transaction to depend on. If your `queue.process` span has no parent, it's automatically promoted to a service span. If you'd rather group it under an explicit service span, start one with `sentry_sdk.traces.start_span(parent_span=None)` first. +Use `continue_trace()` to connect your consumer spans to their associated producer spans. To mark the trace of your message as success or failed, use `transaction.set_status()` in transaction mode, or set `.status = "ok"/"error"` directly on the service span in stream mode. -```python +```python {tabTitle:Transaction Mode (Default)} from datetime import datetime, timezone import sentry_sdk @@ -134,3 +191,54 @@ with sentry_sdk.start_transaction(transaction): # In case of an error set the status to "internal_error" transaction.set_status("internal_error") ``` + +```python {tabTitle:Stream Mode} +from datetime import datetime, timezone + +import sentry_sdk +import my_custom_queue + +# Initialize Sentry +sentry_sdk.init(...) + +connection = my_custom_queue.connect() + +# Pick up message from queues +queue = "messages" +message = connection.consume(queue=queue) + +# Calculate latency (optional, but valuable) +now = datetime.now(timezone.utc) +message_time = datetime.fromtimestamp(message["timestamp"], timezone.utc) +latency = now - message_time + +# Continue the trace started in the producer +# If you are using a web framework, the framework integration +# will create this for you and you can omit this. +sentry_sdk.traces.continue_trace(message["headers"]) +with sentry_sdk.traces.start_span( + name="queue_consumer_transaction", + attributes={"sentry.op": "function"}, + parent_span=None, +) as service_span: + # Create the span + with sentry_sdk.traces.start_span( + name="queue_consumer", + attributes={"sentry.op": "queue.process"}, + ) as span: + # Set span data + span.set_attributes({ + "messaging.message.id": message["message_id"], + "messaging.destination.name": queue, + "messaging.message.body.size": message["body"], + "messaging.message.receive.latency": latency, + "messaging.message.retry.count": 0, + }) + + try: + # Process the message + process_message(message) + except Exception: + # In case of an error set the status to "error" + service_span.status = "error" +``` diff --git a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/requests-module.mdx b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/requests-module.mdx index d25a4355f9f808..0d5514b4233ff3 100644 --- a/docs/platforms/python/tracing/instrumentation/custom-instrumentation/requests-module.mdx +++ b/docs/platforms/python/tracing/instrumentation/custom-instrumentation/requests-module.mdx @@ -3,8 +3,15 @@ title: Instrument HTTP Requests sidebar_order: 2000 description: "Learn how to manually instrument your code to use Sentry's Requests module." --- + As a prerequisite to setting up [Requests](/product/dashboards/sentry-dashboards/outbound-api-requests/), you’ll need to first set up tracing. Once this is done, the Python SDK will automatically instrument outgoing HTTP requests made via `HTTPConnection` and show the data in the [requests-monitoring dashboard](https://sentry.io/orgredirect/organizations/:orgslug/dashboards/). If that doesn't fit your use case, you can set up using custom instrumentation described below. + + +This page covers both transaction mode (default) and stream mode. See New Spans to learn more. + + + ## Custom Instrumentation For detailed information about which data can be set, see the [Requests Module developer specifications](https://develop.sentry.dev/sdk/performance/modules/requests/). @@ -15,9 +22,10 @@ NOTE: Refer to [HTTP Span Data Conventions](https://develop.sentry.dev/sdk/perfo Here is an example of an instrumented function that makes HTTP requests: -```python +```python {tabTitle:Transaction Mode (Default)} from urllib.parse import urlparse import requests +import sentry_sdk def make_request(method, url): span = sentry_sdk.start_span( @@ -35,10 +43,39 @@ def make_request(method, url): response = requests.request(method=method, url=url) span.set_data("http.response.status_code", response.status_code) - span.set_data("http.response_content_length", response.headers["content-length"]) + span.set_data("http.response_content_length", response.headers.get("content-length")) span.finish() return response ``` + +```python {tabTitle:Stream Mode} +from urllib.parse import urlparse +import requests +import sentry_sdk + +def make_request(method, url): + span = sentry_sdk.traces.start_span( + name="%s %s" % (method, url), + attributes={"sentry.op": "http.client"}, + ) + parsed_url = urlparse(url) + span.set_attributes({ + "http.request.method": method, + "url": url, + "server.address": parsed_url.hostname, + "server.port": parsed_url.port, + }) + response = requests.request(method=method, url=url) + + span.set_attribute("http.response.status_code", response.status_code) + + content_length = response.headers.get("content-length") + if content_length is not None: + span.set_attribute("http.response_content_length", content_length) + + span.finish() + return response +``` diff --git a/docs/platforms/python/tracing/instrumentation/index.mdx b/docs/platforms/python/tracing/instrumentation/index.mdx index be7475828ecfc3..cbc92494f3ae8f 100644 --- a/docs/platforms/python/tracing/instrumentation/index.mdx +++ b/docs/platforms/python/tracing/instrumentation/index.mdx @@ -14,8 +14,8 @@ There are two ways that instrumentation is applied to your application: ## Automatic Instrumentation -Many integrations for popular frameworks automatically capture transactions that can be sent to Sentry. Read more about automatic instrumentation [here](/platforms/python/tracing/instrumentation/automatic-instrumentation/). +Many integrations for popular frameworks automatically capture transactions and spans that can be sent to Sentry. Read more about automatic instrumentation [here](/platforms/python/tracing/instrumentation/automatic-instrumentation/). ## Custom Instrumentation -To add custom performance data to your application, you need to add custom instrumentation in the form of [spans](/concepts/key-terms/tracing/distributed-tracing/#traces-transactions-and-spans). Spans are a way to measure the time it takes for a specific action to occur. For example, you can create a span to measure the time it takes for a function to execute. Learn more about span lifecycles [here](/platforms/python/tracing/span-lifecycle/). +To add custom performance data to your application, you need to add custom instrumentation in the form of [spans](/concepts/key-terms/tracing/distributed-tracing/#traces-transactions-and-spans). Spans are a way to measure the time it takes for a specific action to occur. For example, you can create a span to measure the time it takes for a function to execute. Learn more about span lifecycles [here](/platforms/python/tracing/span-lifecycle/). diff --git a/includes/tracing/ai-agents-module/ai-client-span.mdx b/includes/tracing/ai-agents-module/ai-client-span.mdx index f9f47eb95c3054..36d0af9d815df9 100644 --- a/includes/tracing/ai-agents-module/ai-client-span.mdx +++ b/includes/tracing/ai-agents-module/ai-client-span.mdx @@ -1,8 +1,10 @@ -The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span. + +In transaction mode, the [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span using its `template` parameter. The stream-mode decorator doesn't have a `template` option. + -- The span `op` MUST be `"gen_ai.{gen_ai.operation.name}"`. (e.g. `"gen_ai.chat"`) +- The span `op` (transaction mode) or the span's `sentry.op` attribute (stream mode) MUST be `"gen_ai.{gen_ai.operation.name}"`. (e.g. `"gen_ai.chat"`) - The span `name` SHOULD be `"{gen_ai.operation.name} {gen_ai.request.model}"`. (e.g. `"chat o3-mini"`) - The `gen_ai.request.model` attribute MUST be the requested model. (e.g. `"o3-mini"`) - The `gen_ai.response.model` attribute MUST be the concrete model that responded. (e.g. `"gpt-4o-2024-08-06"`) @@ -12,9 +14,9 @@ The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instr ### Request Attributes -| Data Attribute | Type | Requirement Level | Description | Example | -| :--------------------------------- | :----- | :---------------- | :------------------------------------------------------------------------------------------------------------ | :-------------------------------------------------------------------- | -| `gen_ai.input.messages` | string | optional | List of message objects sent to the LLM. **[0]**, **[1]** | `'[{"role": "user", "parts": [{"type": "text", "content": "..."}]}]'` | +| Data Attribute | Type | Requirement Level | Description | Example | +| :--------------------------------- | :----- | :---------------- | :------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------- | +| `gen_ai.input.messages` | string | optional | List of message objects sent to the LLM. **[0]**, **[1]** | `'[{"role": "user", "parts": [{"type": "text", "content": "..."}]}]'` | | `gen_ai.tool.definitions` | string | optional | List of objects describing the available tools. **[0]** | `'[{"name": "random_number", "description": "..."}]'` | | `gen_ai.system_instructions` | string | optional | The system instructions passed to the model. | `"You are a helpful assistant."` | | `gen_ai.request.frequency_penalty` | float | optional | Model configuration parameter. | `0.5` | @@ -29,17 +31,17 @@ The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instr ### Response Attributes -| Data Attribute | Type | Requirement Level | Description | Example | -| :-------------------------------- | :------ | :---------------- | :------------------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------- | -| `gen_ai.response.model` | string | required | The concrete model that responded (may differ from `gen_ai.request.model`). | `"gpt-4o-2024-08-06"` | -| `gen_ai.output.messages` | string | optional | Stringified array of message objects representing the model's output. **[0]**, **[1]** | `'[{"role": "assistant", "parts": [{"type": "text", "content": "..."}]}]'` | -| `gen_ai.response.finish_reasons` | string | optional | Stringified array of reasons the model stopped generating. **[0]** | `'["stop"]'` | -| `gen_ai.response.id` | string | optional | Unique identifier for the completion. | `"chatcmpl-abc123"` | -| `gen_ai.response.streaming` | boolean | optional | Whether the response was streamed. | `true` | -| `gen_ai.response.time_to_first_token` | double | optional | Seconds until first response chunk in streaming. | `0.5` | -| `gen_ai.response.tokens_per_second` | double | optional | Output tokens per second throughput. | `50.0` | -| `gen_ai.response.text` | string | optional | **Deprecated.** Use `gen_ai.output.messages` instead. The text representation of the model's responses. | `"The weather in Paris is rainy"` | -| `gen_ai.response.tool_calls` | string | optional | **Deprecated.** Use `gen_ai.output.messages` instead. The tool calls in the model's response. **[0]** | `'[{"name": "random_number", "type": "function_call", "arguments": "..."}]'` | +| Data Attribute | Type | Requirement Level | Description | Example | +| :------------------------------------ | :------ | :---------------- | :------------------------------------------------------------------------------------------------------ | :--------------------------------------------------------------------------- | +| `gen_ai.response.model` | string | required | The concrete model that responded (may differ from `gen_ai.request.model`). | `"gpt-4o-2024-08-06"` | +| `gen_ai.output.messages` | string | optional | Stringified array of message objects representing the model's output. **[0]**, **[1]** | `'[{"role": "assistant", "parts": [{"type": "text", "content": "..."}]}]'` | +| `gen_ai.response.finish_reasons` | string | optional | Stringified array of reasons the model stopped generating. **[0]** | `'["stop"]'` | +| `gen_ai.response.id` | string | optional | Unique identifier for the completion. | `"chatcmpl-abc123"` | +| `gen_ai.response.streaming` | boolean | optional | Whether the response was streamed. | `true` | +| `gen_ai.response.time_to_first_token` | double | optional | Seconds until first response chunk in streaming. | `0.5` | +| `gen_ai.response.tokens_per_second` | double | optional | Output tokens per second throughput. | `50.0` | +| `gen_ai.response.text` | string | optional | **Deprecated.** Use `gen_ai.output.messages` instead. The text representation of the model's responses. | `"The weather in Paris is rainy"` | +| `gen_ai.response.tool_calls` | string | optional | **Deprecated.** Use `gen_ai.output.messages` instead. The tool calls in the model's response. **[0]** | `'[{"name": "random_number", "type": "function_call", "arguments": "..."}]'` | ### Token Usage diff --git a/includes/tracing/ai-agents-module/execute-tool-span.mdx b/includes/tracing/ai-agents-module/execute-tool-span.mdx index d39f2ebc372cbd..a91cc02ca4270e 100644 --- a/includes/tracing/ai-agents-module/execute-tool-span.mdx +++ b/includes/tracing/ai-agents-module/execute-tool-span.mdx @@ -1,10 +1,12 @@ Describes a tool execution. -The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span. + +In transaction mode, the [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span using its `template` parameter. The stream-mode decorator doesn't have a `template` option. + -- The span `op` MUST be `"gen_ai.execute_tool"`. +- The span `op` (transaction mode) or the span's `sentry.op` attribute (stream mode) MUST be `"gen_ai.execute_tool"`. - The span `name` SHOULD be `"execute_tool {gen_ai.tool.name}"`. (e.g. `"execute_tool query_database"`) - The `gen_ai.tool.name` attribute SHOULD be set to the name of the tool. (e.g. `"query_database"`) - All [Common Span Attributes](#common-span-attributes) SHOULD be set (all `required` common attributes MUST be set). diff --git a/includes/tracing/ai-agents-module/handoff-span.mdx b/includes/tracing/ai-agents-module/handoff-span.mdx index b39be4999e1373..2cc275a842dbac 100644 --- a/includes/tracing/ai-agents-module/handoff-span.mdx +++ b/includes/tracing/ai-agents-module/handoff-span.mdx @@ -1,5 +1,5 @@ A span that describes the handoff from one agent to another. -- The spans `op` MUST be `"gen_ai.handoff"`. +- The spans `op` (transaction mode) or the span's `sentry.op` attribute (stream mode) MUST be `"gen_ai.handoff"`. - The spans `name` SHOULD be `"handoff from {from_agent} to {to_agent}"`. - All [Common Span Attributes](#common-span-attributes) SHOULD be set. diff --git a/includes/tracing/ai-agents-module/invoke-agent-span.mdx b/includes/tracing/ai-agents-module/invoke-agent-span.mdx index 4e0729bf969889..c7e97a6714e4c2 100644 --- a/includes/tracing/ai-agents-module/invoke-agent-span.mdx +++ b/includes/tracing/ai-agents-module/invoke-agent-span.mdx @@ -1,10 +1,12 @@ Describes AI agent invocation. -The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span. + +In transaction mode, the [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span using its `template` parameter. The stream-mode decorator doesn't have a `template` option. + -- The span `op` MUST be `"gen_ai.invoke_agent"`. +- The span `op` (transaction mode) or the span's `sentry.op` attribute (stream mode) MUST be `"gen_ai.invoke_agent"`. - The span `name` SHOULD be `"invoke_agent {gen_ai.agent.name}"`. - The `gen_ai.operation.name` attribute MUST be `"invoke_agent"`. - The `gen_ai.agent.name` attribute SHOULD be set to the agent's name. (e.g. `"Weather Agent"`) @@ -20,7 +22,7 @@ Additional attributes on the span: | `gen_ai.input.messages` | string | optional | List of message objects given to the agent. **[0]**, **[1]** | `'[{"role": "user", "parts": [{"type": "text", "content": "..."}]}]'` | | `gen_ai.tool.definitions` | string | optional | List of objects describing the available tools. **[0]** | `'[{"name": "random_number", "description": "..."}]'` | | `gen_ai.system_instructions` | string | optional | The system instructions passed to the model. | `"You are a helpful assistant."` | -| `gen_ai.pipeline.name` | string | optional | The name of the AI workflow or pipeline the agent belongs to. | `"weather-pipeline"` | +| `gen_ai.pipeline.name` | string | optional | The name of the AI workflow or pipeline the agent belongs to. | `"weather-pipeline"` | | `gen_ai.request.messages` | string | optional | **Deprecated.** Use `gen_ai.input.messages` instead. List of message objects given to the agent. **[0]** | `'[{"role": "system", "content": "..."}]'` | | `gen_ai.request.available_tools` | string | optional | **Deprecated.** Use `gen_ai.tool.definitions` instead. List of objects describing the available tools. **[0]** | `'[{"name": "random_number", "description": "..."}]'` | diff --git a/includes/tracing/mcp-module/prompt-retrieval-span.mdx b/includes/tracing/mcp-module/prompt-retrieval-span.mdx index 5c68cec65f6973..faccf4775c574c 100644 --- a/includes/tracing/mcp-module/prompt-retrieval-span.mdx +++ b/includes/tracing/mcp-module/prompt-retrieval-span.mdx @@ -1,8 +1,6 @@ Describes MCP prompt retrieval. -The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span. - -- The span's `op` MUST be `"mcp.server"`. +- The span `op` (transaction mode) or the span's `sentry.op` attribute (stream mode) MUST be `"mcp.server"`. - The span `name` SHOULD be `"prompts/get {mcp.prompt.name}"`. - The `mcp.prompt.name` attribute MUST be set to the prompt's name. (e.g. `"code_review"`) - The `mcp.method.name` attribute SHOULD be set to `"prompts/get"`. diff --git a/includes/tracing/mcp-module/resource-read-span.mdx b/includes/tracing/mcp-module/resource-read-span.mdx index 83d25e859345ad..f670977d0635de 100644 --- a/includes/tracing/mcp-module/resource-read-span.mdx +++ b/includes/tracing/mcp-module/resource-read-span.mdx @@ -1,8 +1,6 @@ Describes MCP resource access. -The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span. - -- The span's `op` MUST be `"mcp.server"`. +- The span `op` (transaction mode) or the span's `sentry.op` attribute (stream mode) MUST be `"mcp.server"`. - The span `name` SHOULD be `"resources/read {mcp.resource.uri}"`. - The `mcp.resource.uri` attribute MUST be set to the resource's URI. (e.g. `"file:///path/to/resource"`) - The `mcp.method.name` attribute SHOULD be set to `"resources/read"`. diff --git a/includes/tracing/mcp-module/tool-execution-span.mdx b/includes/tracing/mcp-module/tool-execution-span.mdx index 6af1a75028215a..de435384e6a3c4 100644 --- a/includes/tracing/mcp-module/tool-execution-span.mdx +++ b/includes/tracing/mcp-module/tool-execution-span.mdx @@ -1,8 +1,6 @@ Describes MCP tool execution. -The [@sentry_sdk.trace()](/platforms/python/tracing/instrumentation/custom-instrumentation/#span-templates) decorator can also be used to create this span. - -- The span's `op` MUST be `"mcp.server"`. +- The span `op` (transaction mode) or the span's `sentry.op` attribute (stream mode) MUST be `"mcp.server"`. - The span `name` SHOULD be `"tools/call {mcp.tool.name}"`. - The `mcp.tool.name` attribute MUST be set to the tool's name. (e.g. `"get_weather"`) - The `mcp.method.name` attribute SHOULD be set to `"tools/call"`.