feat(datadog-aws): add per-service interceptor crates for SQS, SNS, and EventBridge by Dogbu-cyber · Pull Request #189 · DataDog/dd-trace-rs

Dogbu-cyber · 2026-03-17T20:39:06Z

PR Stack: #194 (workspace setup) -> #189 (aws-sdk injection)

What does this PR do?

Adds trace context injection for the AWS SDK for Rust via three focused interceptor crates. Each crate pulls in only the AWS SDK dependency it needs -- users add only what they use.

Ref: #190

Crate architecture

instrumentation/
├── datadog-aws-core/       # shared -- ServiceHandler trait, generic AwsInterceptor,
│                           #            attribute_keys, limits
├── datadog-aws-core-test-utils/       # common test helpers (not published)
├── datadog-aws-sqs/        # SqsInterceptor
├── datadog-aws-sns/        # SnsInterceptor
└── datadog-aws-eventbridge/ # EventBridgeInterceptor

Usage

// Only pull in the services you use
use datadog_aws_sqs::SqsInterceptor;
use datadog_aws_sns::SnsInterceptor;
use datadog_aws_eventbridge::EventBridgeInterceptor;

let sqs = aws_sdk_sqs::Client::from_conf(
    config.to_builder().interceptor(SqsInterceptor::new()).build()
);

Supported operations

Service	Inject operations	Tag-only operations	Injection point
SQS	`SendMessage`, `SendMessageBatch`	`ReceiveMessage`, `DeleteMessage`, `DeleteMessageBatch`	`_datadog` MessageAttribute (String, JSON)
SNS	`Publish`, `PublishBatch`	`GetTopicAttributes`, `ListSubscriptionsByTopic`, `Subscribe`, `CreateTopic`, etc.	`_datadog` MessageAttribute (Binary) -- avoids subscription filter policy interference
EventBridge	`PutEvents`	`PutRule`, `PutTargets`, `DescribeRule`, etc.	`_datadog` key in event `detail` JSON

Each interceptor also creates a client span (sqs.request, sns.request, eventbridge.request) with standard Datadog tags: aws.service, aws.operation, aws.region, aws.partition, resource.name, operation.name, etc.

Design notes

datadog-aws-core exposes a ServiceHandler trait; each service crate implements it and wraps the generic AwsInterceptor in a concrete named type (SqsInterceptor etc.)
Injection never fails an AWS call -- errors are swallowed and logged at debug level
SQS/SNS respect the 10-attribute cap but overwrite a stale _datadog attribute if present
EventBridge skips injection if the resulting detail would exceed 1 MB
Integration test infrastructure (mock_aws, init_test_tracer, sdk_config, span_attrs) lives in datadog-aws-core-test-utils

Trace Example

ygree · 2026-03-18T02:09:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 302fbb5267

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

joeyzhao2018

LGTM

Dogbu-cyber · 2026-03-30T19:21:53Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80424aa3a6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-30T19:28:50Z

+        // EventBridge entries have a 1 MB detail size limit.
+        if new_detail.len() > ONE_MB {
+            continue;


Validate full PutEvents entry size before mutating Detail

The size guard only checks new_detail.len(), but EventBridge enforces the 1 MB limit on the entire PutEventsRequestEntry payload, not just the Detail field. If an entry is already near the limit because of Source, DetailType, Resources, etc., adding _datadog can push the total entry size over quota and make PutEvents fail, even though tracing is intended to be non-intrusive. This should use the documented entry-size calculation (or equivalent full-entry check) before writing the mutated detail.

Useful? React with 👍 / 👎.

TL;DR This is a valid concern. I'm thinking about dropping this check because it's not correct or useful.

The limit is documented as a service-side contract. AWS’s current EventBridge API reference says PutEvents total entry size must be less than 1 MB, and the user guide says the whole request must stay under 1,048,576 bytes, with a single entry allowed to use that full budget if it is the only one. Invalid entries are reported back by the service through PutEventsResultEntry.ErrorCode / ErrorMessage, which is consistent with backend enforcement rather than SDK preflight. Sources: PutEvents API, PutEvents user guide, PutEventsResultEntry.

The latest generated Rust SDK docs still say 256KB

I think we should keep the check for now, just to provide an upper limit. If the payload exceeds the limit for any reason, we can skip the injection path altogether, as such a request will be rejected by the backend anyway.

> **PR Stack:** **#194 (workspace setup)** -> #213 (lambda root span) -> #189 (aws-sdk injection) -> #190 (lambda inferred spans) # What does this PR do? Establishes the `instrumentation/` Cargo workspace that houses Datadog AWS instrumentation crates. This is the foundation PR -- #213, #189, and #190 add the implementations on top. ## Workspace structure ``` instrumentation/ ├── Cargo.toml # shared workspace (resolver = "2", MSRV 1.91.1) ├── datadog-aws-core/ # stub -- internal interceptor machinery (added in #189) ├── datadog-aws-sqs/ # stub -- SqsInterceptor (added in #189) ├── datadog-aws-sns/ # stub -- SnsInterceptor (added in #189) ├── datadog-aws-eventbridge/ # stub -- EventBridgeInterceptor (added in #189) └── datadog-aws-lambda/ # distributed tracing for Rust Lambda functions (added in #213, #190) ```

…tracing" This reverts commit 3347bdc.

Move out shared test_helpers and integration_test_helpers (mock server, tracer setup, SDK config, and utilities). This way the common dev-only test parts can stay in an internal crate and never be published.

ygree · 2026-05-05T04:48:09Z

+
+        let detail = entry.detail.as_deref().unwrap_or("{}");
+        let mut detail_map: serde_json::Map<String, serde_json::Value> =
+            match serde_json::from_str(detail) {


This requires full detail deserialization to an expensive memory representation. There must be a lighter-weight way to achieve adding the _datadog data field.

Fixed in 9f56055

Replace full PutEvents detail deserialization with a top-level streaming JSON rewrite using serde_json RawValue and Serializer. This changes EventBridge injection to replace an existing top-level _datadog field or append one when missing, while copying other fields through without building the full nested payload in memory. It also skips invalid, non-object, and oversize detail payloads without mutating the original entry. Add tests for replace, append, nested _datadog, invalid/non-object detail, and per-entry size-limit behavior.

ygree · 2026-05-07T17:57:26Z

        let trace_ctx = serde_json::Value::Object(entry_ctx);

        let detail = entry.detail.as_deref().unwrap_or("{}");
-        let mut detail_map: serde_json::Map<String, serde_json::Value> =


It used to deserialize entire detail. Now it uses a visitor to scan only top level and replace/inject missing context recreating a new detail content.

datadog-official · 2026-05-20T18:35:16Z

Tests

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 1 Pipeline job failed

Check Pull Request CI Status | ensure-ci-success

🔄 Retry job. This looks flaky and may succeed on retry.
Some checks are still running, but retrying is not allowed anymore due to the timeout.

ℹ️ Info

No other issues found (see more)

🧪 All tests passed
❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 70857ca | Docs | Datadog PR Page | Give us feedback!}

Dogbu-cyber added the enhancement New feature or request label Mar 17, 2026

Dogbu-cyber mentioned this pull request Mar 17, 2026

feat(datadog-aws-lambda): add trigger extraction and inferred spans #190

Merged

Dogbu-cyber marked this pull request as ready for review March 17, 2026 22:01

Dogbu-cyber requested a review from a team as a code owner March 17, 2026 22:01

joeyzhao2018 reviewed Mar 17, 2026

View reviewed changes

Comment thread integrations/aws/aws-sdk-rust/README.md Outdated

chatgpt-codex-connector Bot reviewed Mar 18, 2026

View reviewed changes

Comment thread integrations/aws/aws-sdk-rust/Cargo.toml Outdated

Comment thread integrations/aws/aws-sdk-rust/src/services/sns.rs Outdated

Comment thread integrations/aws/aws-sdk-rust/src/services/sqs.rs Outdated

Dogbu-cyber changed the base branch from main to david.ogbureke/ci-integration-msrv March 22, 2026 00:54

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from ee01b82 to 612838b Compare March 22, 2026 00:55

Dogbu-cyber force-pushed the david.ogbureke/ci-integration-msrv branch 2 times, most recently from 141c5dd to 1184437 Compare March 22, 2026 20:52

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 612838b to 4f7442a Compare March 23, 2026 14:54

Dogbu-cyber mentioned this pull request Mar 23, 2026

chore(instrumentation): set up instrumentation workspace #194

Merged

Dogbu-cyber changed the title ~~AWS SDK Integration~~ feat(datadog-aws-sdk): add trace context injection for SQS, SNS, and EventBridge Mar 23, 2026

joeyzhao2018 approved these changes Mar 27, 2026

View reviewed changes

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 3fa3937 to 80424aa Compare March 30, 2026 19:20

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 80424aa to 80fedd0 Compare March 30, 2026 19:26

chatgpt-codex-connector Bot reviewed Mar 30, 2026

View reviewed changes

Dogbu-cyber force-pushed the david.ogbureke/ci-integration-msrv branch 2 times, most recently from 17f2c7a to 78a6876 Compare March 30, 2026 20:30

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 80fedd0 to 18c7914 Compare March 30, 2026 20:48

Dogbu-cyber force-pushed the david.ogbureke/ci-integration-msrv branch from 78a6876 to bf4ea81 Compare April 2, 2026 14:54

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch 2 times, most recently from 35b2e4f to 7d1fbbe Compare April 2, 2026 15:32

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch from 29211f0 to d999596 Compare April 13, 2026 12:51

ygree reviewed Apr 13, 2026

View reviewed changes

Comment thread instrumentation/aws/datadog-aws/tests/integration.rs Outdated

ygree reviewed Apr 14, 2026

View reviewed changes

Comment thread instrumentation/aws/datadog-aws/src/interceptor.rs Outdated

Dogbu-cyber force-pushed the david.ogbureke/aws-sdk-rust branch 2 times, most recently from 7e64f6f to f1e120e Compare April 14, 2026 18:06

ygree reviewed Apr 22, 2026

View reviewed changes

Comment thread instrumentation/datadog-aws-core/Cargo.toml Outdated

ygree reviewed Apr 22, 2026

View reviewed changes

Comment thread instrumentation/Cargo.toml

Dogbu-cyber force-pushed the david.ogbureke/lambda-root-invocation branch from c2f0ee8 to 2cd84c3 Compare April 23, 2026 15:27

ygree changed the base branch from david.ogbureke/lambda-root-invocation to main April 27, 2026 22:24

ygree force-pushed the david.ogbureke/aws-sdk-rust branch from 30948ab to 0655a34 Compare April 27, 2026 23:49

ygree changed the base branch from main to david.ogbureke/lambda-root-invocation April 27, 2026 23:54

Revert "feat(datadog-aws-lambda): add root invocation span with OTel …

8c4901a

…tracing" This reverts commit 3347bdc.

ygree changed the base branch from david.ogbureke/lambda-root-invocation to main April 28, 2026 18:55

ygree force-pushed the david.ogbureke/aws-sdk-rust branch 2 times, most recently from b6799f2 to 5c5ff0f Compare April 29, 2026 03:57

ygree added 2 commits April 28, 2026 21:07

Fix Cargo.toml. Generate Cargo.lock. Format code.

acb1ccb

Extract datadog-aws-core-test-utils

458d4d5

Move out shared test_helpers and integration_test_helpers (mock server, tracer setup, SDK config, and utilities). This way the common dev-only test parts can stay in an internal crate and never be published.

ygree force-pushed the david.ogbureke/aws-sdk-rust branch 3 times, most recently from 9a80990 to 458d4d5 Compare April 29, 2026 04:35

Merge branch 'main' into david.ogbureke/aws-sdk-rust

f391600

ygree reviewed May 5, 2026

View reviewed changes

ygree force-pushed the david.ogbureke/aws-sdk-rust branch from 6cf73a4 to 9f56055 Compare May 5, 2026 17:26

ygree added 2 commits May 5, 2026 15:54

chore(instrumentation): use per-crate versions in aws workspace

06aec15

upgrade deps

e79aaff

ygree mentioned this pull request May 6, 2026

feat(datadog-aws-lambda): add trigger extraction and inferred spans #219

Open

ygree reviewed May 7, 2026

View reviewed changes

ygree added 3 commits May 12, 2026 11:30

Merge branch 'main' into david.ogbureke/aws-sdk-rust

adf235b

Merge branch 'main' into david.ogbureke/aws-sdk-rust

4a858d0

Merge branch 'main' into david.ogbureke/aws-sdk-rust

d5822e5

ygree added 2 commits May 22, 2026 10:07

Merge branch 'main' into david.ogbureke/aws-sdk-rust

560e6ad

Updated Cargo.lock

70857ca

Conversation

Dogbu-cyber commented Mar 17, 2026 • edited by ygree Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Crate architecture

Usage

Supported operations

Design notes

Trace Example

Uh oh!

Uh oh!

ygree commented Mar 18, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

joeyzhao2018 left a comment

Choose a reason for hiding this comment

Uh oh!

Dogbu-cyber commented Mar 30, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

ygree May 4, 2026

Choose a reason for hiding this comment

Uh oh!

ygree May 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ygree May 5, 2026

Choose a reason for hiding this comment

Uh oh!

ygree May 5, 2026

Choose a reason for hiding this comment

Uh oh!

ygree May 7, 2026

Choose a reason for hiding this comment

Uh oh!

datadog-official Bot commented May 20, 2026 • edited by datadog-datadog-prod-us1 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Dogbu-cyber commented Mar 17, 2026 •

edited by ygree

Loading

datadog-official Bot commented May 20, 2026 •

edited by datadog-datadog-prod-us1 Bot

Loading