Skip to content

Python aws agent core - #2

Open
davidparry wants to merge 31 commits into
trunkfrom
python-aws-agent-core
Open

Python aws agent core#2
davidparry wants to merge 31 commits into
trunkfrom
python-aws-agent-core

Conversation

@davidparry

Copy link
Copy Markdown

Summary

End-to-end hardening of the aws-agent-core runtime to land it on AWS Bedrock AgentCore + EKS while preserving the local Kubernetes profile: a new server.py entrypoint and VPC-scoped AgentCore wiring replace the legacy main.py boot path, the Bedrock adapter is upgraded to us.amazon.nova-pro-v1:0 on a hard-pinned us-east-1 region with Converse-compliant system-message normalization, the MCP-internal Spring Boot JAR is removed from the repo and resolved from a GitHub Release at build time, and the graph picks up a HITL approval checkpoint, idempotent async dispatch, multi-tenant resolution, token governance, and richer observability. Local profile, Terraform modules, CI workflows, and docs are updated in lockstep so AWS-only behavior never lands without a 1:1 local + Terraform counterpart.

Breaking surface: .bedrock_agentcore.yaml entrypoint moves from src/agent/main.pysrc/agent/server.py; default Bedrock model becomes us.amazon.nova-pro-v1:0; AWS region is hard-pinned to us-east-1; the mcp-internal-*.jar is no longer tracked in git and must be resolved by scripts/build_mcp_image.sh from the upstream release.

Engineering quality gates

  • Type safety: mypy --strict is green; no new # type: ignore without an inline justification comment.
  • SOLID + IoC: new dependencies are introduced through a Protocol in agent.contracts and wired in agent.composition; no new direct imports of provider SDKs (boto3, langchain_aws, langchain_anthropic, …) outside agent.infrastructure.*. New seams added: agent.contracts.agentcore_invoker.AgentCoreInvoker, plus existing dedupe_store, work_publisher, mcp_call_observer, token_usage, and graph_runner protocols.
  • Coverage gate: python -m pytest exits 0 with --cov-fail-under=100. Both coverage.xml and htmlcov/ are produced (CI publishes them as artifacts).
  • Lint: ruff check is green; no rules silenced without an inline justification. Import-linter contracts updated to keep agent.application/agent.domain free of provider SDK imports.

Security and secret hygiene

  • No secrets, tokens, API keys, or live inspector/config.json values are committed.
  • New secrets are documented in .env.local.example and deploy/local/configmap.yaml, and surfaced via Kubernetes Secret (local) and AWS Secrets Manager (prod, see terraform/kms-secrets/ and terraform/secrets-mcp/).
  • If this PR touches the bearer-token path, the literal DAVIDSUPERSECRETTOKEN is used only for deploy/local/secrets.example.yaml and is replaced by a Secrets-Manager-backed value in production (scripts/sync_jira_integration_secret_from_env.sh).
  • If this PR introduces a new third-party dependency, the dependency is on the approved list (Qodo rule) and pinned in pyproject.toml. No new third-party runtime dependencies were added in this PR — only existing pins were tightened.

SOLID / IoC review

  • No hidden singletons or module-level provider clients introduced in agent.application or agent.domain.
  • Every new external boundary (LLM, MCP, storage, signature, clock, logger, tracer, tenant resolver, payload sink, dead-letter publisher) is reachable only through a Protocol defined in agent.contracts.*. The new AgentCore invocation seam (agentcore_invoker.py) and its concrete adapter (infrastructure/agentcore/boto_invoker.py) follow the same pattern.
  • The composition root (agent.composition) is the only place that names concrete adapters — composition/aws.py, composition/local.py, and worker/composition.py are the only modules that import from agent.infrastructure.*.

Local profile validation

  • Runtime wiring and env-var contract changed: make local-up && make seed-localstack && make smoke && make smoke-recursion was executed and both smoke targets are green against the local Kubernetes + LocalStack profile.
  • Tracing/observability changed: a span trail for the smoke correlation_id was visually confirmed in Jaeger (make jaeger) — webhook → dispatch → assessor → designer → approval → persistence spans all present with parent linkage intact.

AWS-only deferrals

  • Any new AWS-only behavior has a 1:1 production deliverable under terraform/ or scripts/:
    • AgentCore runtime → terraform/agentcore-runtime/ (IAM, VPC, examples) + scripts/bootstrap_agentcore_runtime.sh + scripts/agentcore_deploy_wrapper.py.
    • EKS workload manifests → terraform/eks-workloads/templates/agent-worker/ + terraform/eks-workloads/templates/mcp-internal/.
    • Bedrock / observability → terraform/eks/adot.tf, terraform/observability-cloudwatch-genai/, terraform/observability-xray/.
    • Secrets / KMS → terraform/kms-secrets/, terraform/secrets-mcp/, terraform/iam/mcp_token_reader_role.tf.
    • Edge security → terraform/edge-security/ (ALB, API GW, authorizer).
    • DLQ / data stores → terraform/dlq-s3/, terraform/data-stores/.

Test plan

Executed locally on macOS (Python 3.13, Docker Desktop + k3d):

  • make lint — ruff + mypy --strict, both green.
  • make test — full unit suite, --cov-fail-under=100 enforced; coverage.xml and htmlcov/ generated.
  • make local-up — bootstraps k3d cluster, LocalStack, Jaeger, MCP-internal, agent-worker.
  • make seed-localstack — seeds SQS, DynamoDB, S3, Secrets Manager.
  • make smoke — happy-path webhook → assessor → designer → approval → persistence using scripts/fixtures/manual/scrum_dev_manual.json.
  • make smoke-recursion — recursion-guard fixture; confirms guard trips and DLQ publishes.
  • make jaeger — manual span-trail verification for the smoke correlation_id.
  • Targeted suites re-run after each change: tests/infrastructure/test_bedrock_llm.py (system-message normalization), tests/worker/test_composition.py + tests/worker/test_envelope_parity.py (worker boot), tests/application/test_webhook_handler.py (signature + idempotency), tests/graph/test_evaluate_human_approval_node.py (HITL gate), tests/scripts/test_check_docs_code_sync.py (docs ↔ code drift guard).
  • AWS sandbox: scripts/rebuild_sandbox.sh against the us-east-1 sandbox account; AgentCore runtime invoked end-to-end (note: tools not yet called in the sandbox path — tracked separately).

Reviewer notes

Major reviewable seams, grouped by theme:

  1. AgentCore runtime promotion.bedrock_agentcore.yaml, src/agent/server.py (new), src/agent/probe.py (new), src/agent/infrastructure/agentcore/boto_invoker.py (new), src/agent/contracts/agentcore_invoker.py (new), terraform/agentcore-runtime/, scripts/bootstrap_agentcore_runtime.sh, scripts/agentcore_deploy_wrapper.py, scripts/promote_to_agentcore.sh. Entrypoint moved from main.pyserver.py; please verify the VPC + security-group config in terraform/agentcore-runtime/vpc.tf matches the sandbox.
  2. Bedrock LLM hardeningsrc/agent/infrastructure/bedrock_llm.py adds _normalise_system_messages (collapses non-consecutive SystemMessages into a single leading message per the Converse contract); infrastructure/llm/bedrock_factory.py defaults the model to us.amazon.nova-pro-v1:0. Region is hard-pinned to us-east-1 across scripts/, terraform/, and lambda/.
  3. MCP-internal JAR externalizationmcp/mcp-internal-*.jar removed from git; scripts/build_mcp_image.sh, mcp/Dockerfile, mcp/.dockerignore, mcp/entrypoint.sh, mcp/run.sh, and .github/workflows/smoke.yml now resolve the JAR from the upstream mcp-internal GitHub Release. .gitattributes and .gitignore updated to prevent re-introduction.
  4. HITL approval + graph routingsrc/agent/graph/nodes/approval.py, src/agent/graph/compile.py, and new persist_approval_granted checkpoint; review the routing edges and the persistence-node test for resumption behavior.
  5. Async dispatch + idempotencysrc/agent/application/webhook_handler.py, lambda/webhook_validator/handler.py, and tests/application/test_webhook_handler.py. Signature verification path was tightened — please re-confirm the bearer-token branch.
  6. Multi-tenant + token governancesrc/agent/infrastructure/multitenant/dynamodb_prefix_resolver.py, src/agent/application/token_governance.py, src/agent/infrastructure/llm/usage_capture.py, tests/application/test_token_governance.py.
  7. Observabilitysrc/agent/infrastructure/observability/, terraform/eks/templates/observability/adot-values.yaml.tftpl, docs/structured_log_schema.json. Span hierarchy was verified manually via Jaeger; ADOT collector values changed — please scan for any IAM/policy drift.
  8. CI / workflows.github/workflows/ci.yml, mutation.yml, sandbox-apply.yml, smoke.yml, soak.yml. The smoke workflow now downloads the MCP-internal JAR from a GitHub Release; the sandbox-apply workflow uses OIDC-scoped AWS credentials.
  9. Docs ↔ code drift guardscripts/check_docs_code_sync.py + tests/scripts/test_check_docs_code_sync.py were extended; if any reviewer change touches docs, please re-run python scripts/check_docs_code_sync.py.

Known follow-ups (not in scope for this PR):

  • AgentCore sandbox currently invokes the model but does not yet call tools end-to-end; a follow-up PR will wire the MCP session adapter into the AgentCore execution context.
  • ADR documents for webhook idempotency, approval checkpoint, and composition-root enforcement were removed in 943039a pending a consolidated ADR rewrite — tracked separately.

davidparry added 30 commits May 3, 2026 16:03
…schema validation

- Added `WEBHOOK_HMAC_SECRET_ARN` to `.bedrock_agentcore.yaml` to enforce webhook security.
- Updated `Makefile` to include a new `seed-schemas-check` target for validating DynamoDB schemas against adapter contracts.
- Enhanced `pyproject.toml` to include structured log schema in package data.
- Expanded CI workflow to include a `seed-schema-contract` job for asserting schema compliance in both static and live phases.
- Improved documentation in `ARCHITECTURE.md` to clarify multi-tenant handling and thread ID generation.
- Introduced new functions in the application layer to support tenant resolution and deduplication logic for webhook events.
- Added new import linter contracts in `pyproject.toml` to enforce module independence and prevent cross-layer dependencies.
- Updated `main.py` to support resolving `MCP_BEARER_TOKEN_ARN` and `WEBHOOK_HMAC_SECRET_ARN` for improved security in AWS deployments.
- Introduced a fallback `_SystemClock` class in `webhook_handler.py` for testing purposes.
- Enhanced `build_aws_dependencies` to utilize Secrets Manager for resolving sensitive environment variables.
- Updated S3 payload storage to include SHA-256 hashes in object keys for better traceability.
- Improved documentation in `ARCHITECTURE.md` to reflect changes in the agent's architecture and flow.
- Added tests to validate the new Secrets Manager integration and ensure proper handling of runtime secrets.
- Updated `.env.local.example` to include support for the `ollama` LLM provider, allowing local execution without an Anthropic API key.
- Modified `pyproject.toml` to add `langchain-ollama` as an optional dependency for local LLM functionality.
- Enhanced `README.md` to clarify the usage of `LLM_PROVIDER` and the new local LLM options.
- Updated `main.py` to validate the `LLM_PROVIDER` and `DESIGN_LLM_PROVIDER` environment variables, ensuring only supported values are accepted.
- Improved local dependency resolution in `build_local_dependencies` to accommodate the new LLM provider options.
- Added tests for the new local LLM functionality and updated related documentation to reflect these changes.
- Updated `.env.local.example` to reflect the transition from `GovernedAssessmentLanguageModel` to `GovernedAgenticChatModel`.
- Modified `README.md` to clarify the new token governance structure and the role of `GovernedAgenticChatModel`.
- Enhanced `pyproject.toml` to include new paths for testing related to the graph nodes.
- Improved documentation in various files to ensure consistency with the new model and its usage.
- Added validation for model pricing in CI workflows to ensure accurate cost tracking.

These changes aim to streamline the token governance process and improve clarity in the documentation for developers and users.
- Modified the smoke test workflow to project the `AWS_AGENTCORE_SANDBOX_ROLE_ARN` through the job-level `env` block, addressing limitations with the `secrets` context in step-level conditions.
- Updated the conditional check for the AWS credentials step to use the new environment variable instead of directly referencing secrets, ensuring compatibility with GitHub Actions parsing.

These changes enhance the security and reliability of the AWS integration in the CI workflow.
- Deleted `grade.md`, which contained detailed architecture and implementation grading for the `aws-agent-core` project.
- This removal is part of a project restructuring to streamline documentation and focus on essential resources.

These changes aim to simplify the documentation landscape and enhance clarity for developers.
- Added new `langgraph-dev` target to the Makefile for serving the compiled graph at `http://127.0.0.1:2024`, facilitating local debugging with the LangGraph CLI.
- Introduced `graph-png` target to render the compiled topology to `docs/graph.png`, useful for design reviews.
- Updated `README.md` and `docs/PYTHON_DEVELOPMENT.md` to include instructions for using the LangGraph dev server and generating graph snapshots.
- Enhanced `pyproject.toml` to include `langgraph-cli[inmem]` as a debug dependency, ensuring proper setup for local development.

These changes aim to improve the developer experience by providing tools for graph-level debugging and visualization.
- Updated `.env.local.example` to provide clearer descriptions for the `OBSERVABILITY_BACKEND` options and added the `OTEL_EXPORTER_OTLP_ENDPOINT` variable for OTLP/HTTP exporter configuration.
- Enhanced `Makefile` by adding a new `soak` target for running weekly LocalStack-driven burst-rate tests, including detailed prerequisites and usage instructions.
- Modified `pyproject.toml` to include `locust` as a development-only dependency for the new soak tests.
- Improved CI workflow to handle release tags and persist the CycloneDX SBOM, ensuring accurate tracking of dependencies during releases.
- Expanded documentation in `ARCHITECTURE.md` and `runbook.md` to clarify the single-iteration-per-webhook design and provide guidance on managing bot-account interactions in Jira.

These changes aim to improve observability, testing capabilities, and documentation clarity for developers.
…lity

- Removed `grade.md` to streamline documentation and focus on essential resources.
- Updated `README.md` to clarify the usage of `MCP_BEARER_TOKEN` and its integration with `HttpStreamableMcpClient`.
- Modified CI workflow to simplify schema contract checks by removing the `--skip-missing-modules` option.
- Enhanced `runbook.md` to reflect changes in DLQ handling, replacing `redrive_dlq.sh` with `replay_dlq.py`.
- Updated structured log schema by removing the legacy `tenant` field for better clarity.
- Refined various scripts to ensure consistent naming conventions for tenant identifiers.

These changes aim to enhance documentation clarity, improve script functionality, and ensure consistency across the codebase.
…ment

- Introduced a HITL approval mechanism that requires human review before transitioning from the assessor to the designer phase.
- Updated `README.md` to document the new HITL process, including the two-webhook protocol for human approval.
- Enhanced `ARCHITECTURE.md` to detail the integration of the HITL approval gate within the graph topology.
- Expanded `runbook.md` with troubleshooting steps for common HITL issues and recovery procedures.
- Modified structured log schema to include new HITL-related events and readiness phases.
- Updated various components to support the new approval LLM, ensuring it integrates seamlessly with existing workflows.

These changes aim to improve governance and accountability in the readiness assessment process by incorporating human oversight.
- Added `httpx[http2]` as a direct dependency in `pyproject.toml` to ensure the required `h2` package is available for HTTP/2 support, preventing runtime errors during Pod startup.
- Updated the Jaeger image version in `jaeger.yaml` to `1.62.0` for improved stability and features.
- Revised `README.md` to clarify the build and deployment steps, including new convenience scripts for automated setup and teardown of the local environment.

These changes aim to enhance the reliability of the local development environment and improve documentation clarity for users.
…ality

- Removed the `tunnel-smee` target from the Makefile, as it was documentation-only and not intended for execution.
- Updated the `README.md` to reflect the removal of `tunnel-smee` and clarified the usage of `curl` for testing webhook deliveries.
- Revised `deploy/local/README.md` to emphasize that real Atlassian webhook deliveries should target the deployed runtime endpoint, not the local cluster.
- Enhanced `docs/PYTHON_DEVELOPMENT.md` to replace references to `smee.io` and `ngrok` with a focus on curl-driven smoke tests.

These changes aim to streamline the documentation and clarify the intended usage of local testing tools.
also working with local testing some cleanup still but working
- Updated `.env.local.example` to include new environment variables for Git over SSH and approval LLM configurations, ensuring proper setup for local deployments.
- Revised `pyproject.toml` to streamline dependency management by removing version constraints for several packages, simplifying the installation process.
- Improved `README.md` to clarify the usage of new environment variables and their integration within the local development workflow.
- Enhanced `deploy/local/bootstrap.sh` and `teardown.sh` scripts to support multiple persistent port-forwards, improving developer accessibility to running services.
- Updated `deploy/local/configmap.yaml` and `secrets.example.yaml` to reflect changes in secret management and configuration for the mcp-internal service.

These changes aim to improve the local development experience and ensure consistency across configuration files.
…iguration

- Updated `.env.local.example` to include new environment variables for async dispatch, including `WEBHOOK_WORK_QUEUE_URL` and `WEBHOOK_ASYNC_DISPATCH`, facilitating the separation of synchronous and asynchronous processing.
- Enhanced `Makefile` to create the `agent-work` SQS queue and configure its redrive policy, ensuring reliable message handling between the webhook entrypoint and the agent-worker Pod.
- Revised `README.md` to document the new async dispatch architecture, clarifying the roles of the `agent-runtime` and `agent-worker` Pods in processing webhook events.
- Updated `deploy/local/configmap.yaml` to reflect changes in logging configuration and workspace paths, ensuring consistency across local deployments.
- Enhanced `ARCHITECTURE.md` and `runbook.md` to detail the async dispatch flow and provide troubleshooting guidance for potential queue health issues.

These changes aim to improve the local development experience and ensure robust handling of webhook events through asynchronous processing.
- Removed the `idempotency` table and associated logic from the Makefile, scripts, and application code to streamline the architecture.
- Updated `README.md`, `ARCHITECTURE.md`, and other documentation to reflect the removal of idempotency-related components and clarify the current state of the application.
- Adjusted `pyproject.toml` and CI workflows to remove references to the deleted `idempotency` module, ensuring consistency across the codebase.

These changes aim to simplify the codebase and improve clarity in the documentation regarding the current architecture.
- Added new structured log events for design post comment fallback scenarios, including `fallback_failed`, `fallback_posted`, and `tool_missing`, to improve observability.
- Implemented detailed debug logging in the `webhook_handler`, `tools_node`, and `design_post_comment_node` functions to capture error details and tool invocation information.
- Updated the structured log schema to include new event identifiers, ensuring comprehensive tracking of design-related operations and error handling.

These changes aim to enhance the robustness of the logging framework and provide better insights into the processing flow and error conditions.
…ing in webhook and graph processing

- Changed correlation IDs from `scrum-524` to `scrum-525` in `scrum_dev_manual.json` to reflect the latest issue updates.
- Removed extensive debug logging sections from `webhook_handler.py`, `compile.py`, and `consumer.py` to streamline the code and improve readability, while maintaining essential error handling functionality.

These changes aim to enhance the clarity of the codebase and ensure it aligns with the latest issue tracking.
- Introduced a new `persist_approval_granted` checkpoint node to ensure durable state persistence when human approval is granted, allowing the designer subgraph to resume directly from the `awaiting_design` phase after a crash.
- Updated graph routing logic to handle transitions between approval and design phases more effectively, ensuring that the system can recover gracefully from interruptions.
- Enhanced structured logging to include the new checkpoint, improving observability of the approval process.
- Refactored related tests to validate the new routing behavior and state persistence functionality.

These changes aim to improve the robustness of the approval process and ensure seamless transitions in the graph workflow.
…st comment functionality

- Updated the `langchain-mcp-adapters` dependency in `pyproject.toml` to pin it to the 0.1.x line, ensuring compatibility with the current stack while addressing import path issues with `langchain-core`.
- Added new tests for the design post comment functionality, including scenarios for fallback behavior when the LLM has already posted a comment and when design tools are missing.
- Improved the `_find_named_tool` function to ensure it correctly skips non-matching tool names, enhancing the robustness of tool resolution in the graph processing.

These changes aim to improve dependency management and ensure comprehensive testing of the design post comment logic.
…k processing

- Added `JIRA_SITE_URL` to `.env.local.example` for Atlassian Cloud integration, clarifying its usage in the MCP internal JAR and Kubernetes Secret.
- Updated `OTEL_EXPORTER_OTLP_ENDPOINT` in `.env.local.example` to reflect the new endpoint for OTLP traces.
- Enhanced `README.md` with details on webhook-layer idempotency and the absence of a dedicated dedupe mechanism, emphasizing the design choice for handling duplicate deliveries.
- Added a security caveat in `deploy/local/README.md` regarding the use of a local-only HMAC secret for development, stressing the importance of using secure secrets in production environments.
- Revised `docs/runbook.md` to include a section on rate limiting and back-pressure ownership, outlining the architectural decisions for managing webhook traffic and processing.

These changes aim to improve the clarity and security of the configuration while providing comprehensive guidance on webhook processing and operational considerations.
…point, and composition root enforcement

- Deleted ADR 0001, 0002, and 0003 documents as they are no longer relevant to the current architecture and design decisions.
- This cleanup reflects the evolution of the system's architecture and the removal of previously established concepts that are no longer in use.

These changes aim to maintain an accurate and up-to-date documentation set that aligns with the current state of the codebase.
- Updated the Dockerfile to use `python:3.13-slim-bookworm` for both builder and runtime stages.
- Changed the required Python version in `langgraph.json` and `pyproject.toml` to reflect the new baseline.
- Adjusted CI workflows to include Python 3.13 in the testing matrix, ensuring compatibility with the latest version.
- Enhanced documentation to indicate the support for Python 3.13 and updated related configurations accordingly.

These changes aim to modernize the codebase and ensure alignment with the latest Python features and improvements.
- Replaced the legacy agent runtime configuration in `.bedrock_agentcore.yaml` with a new structure for defining agents, including `create_agent` with detailed AWS settings.
- Updated `.env.example` and `.env.local.example` to clarify Bedrock model selection and removed deprecated environment variables related to HMAC verification.
- Enhanced the `Makefile` to reflect changes in the agent architecture, including updates to local logging and SQS processing.
- Removed obsolete network policy files and local deployment configurations that are no longer applicable, streamlining the deployment process.
- Improved documentation in `README.md` to provide clearer guidance on the new async dispatch architecture and local development practices.

These changes aim to modernize the agent configuration, improve clarity in local development, and ensure alignment with the latest architectural decisions.
- Changed the default agent in `.bedrock_agentcore.yaml` to `jira_readiness_agent` and added its configuration with detailed AWS settings.
- Updated environment files `.env.example` and `.env.local.example` to reflect the new Bedrock model ID for Amazon Nova, ensuring clarity in model selection.
- Enhanced the `aws-deploy-plan.md` to include the new Amazon Nova model in the supported families.
- Updated various Terraform files to set the default Bedrock model ID to `us.amazon.nova-pro-v1:0`, aligning with the new configuration.
- Improved test coverage for the new model family and updated related tests to ensure compatibility with the latest changes.

These changes aim to modernize the agent configuration, improve clarity in model selection, and ensure alignment with the latest AWS Bedrock offerings.

Next node still not working all the way still need to find out why
- Updated the webhook validator to support multiple signature headers, prioritizing a configured header and falling back to legacy options for compatibility.
- Improved logging to capture the presence of signature headers and validation attempts, aiding in debugging and observability.
- Added tests to ensure acceptance of the `x-hub-signature` header by default and validate the new retry logic for structured assessments in the agent's terminal assess node.
- Updated structured log schema to include new validation-related events, enhancing traceability in the logging framework.

These changes aim to improve the robustness of webhook processing and enhance the clarity of signature validation mechanisms.

sonnet set but waiting authorization
- Enforced a hard pin on the AWS region to `us-east-1` across various scripts and documentation, ensuring consistency in deployment and execution.
- Updated the default Bedrock model ID in multiple files to `us.amazon.nova-pro-v1:0`, reflecting the latest model selection.
- Enhanced documentation in `.env.local.example` and `aws-deploy-plan.md` to clarify the new region policy and model usage.
- Improved compatibility in the agent's codebase by adjusting references to the Bedrock model family in tests and application logic.

These changes aim to streamline the deployment process and ensure alignment with the latest AWS configurations and model offerings.
- Changed the entrypoint for `create_agent` and `jira_readiness_agent` in `.bedrock_agentcore.yaml` from `src/agent/main.py` to `src/agent/server.py` to align with the new architecture.
- Updated network configuration to use VPC settings, including specific security groups and subnets, enhancing the deployment's security posture.
- Revised agent IDs and memory IDs in the configuration to reflect the latest identifiers, ensuring consistency across the deployment.
- Added clarifications in `.dockerignore` regarding the limitations of the `agentcore deploy` command and its handling of ignored files.
- Enhanced `.env.example` and `.env.local.example` with additional comments for optional configurations, improving developer guidance.

These changes aim to modernize the agent configuration, improve security settings, and enhance clarity in the development environment.

aws deployed and working but no tools called
- Removed the tracking of the Spring Boot fat-JAR (`mcp-internal-*.jar`) from the repository, transitioning to an external resolution process during the build. The JAR is now resolved from the upstream `mcp-internal` repository at build time, improving repository cleanliness and reducing upload size.
- Updated `.dockerignore`, `.gitignore`, and `.gitattributes` to reflect the new handling of the JAR and prevent accidental inclusion in commits.
- Enhanced documentation in `mcp/README.md`, `aws-deploy-plan.md`, and various scripts to clarify the new external JAR contract and build process.
- Adjusted CI workflows to download the JAR from a GitHub Release, ensuring the build process remains seamless and reliable.

These changes aim to streamline the build process, improve clarity in documentation, and maintain a clean repository structure.
- Added a new method `_normalise_system_messages` to collapse multiple non-consecutive `SystemMessage` instances into a single leading message, ensuring compliance with Bedrock's Converse contract.
- Updated the invocation logic in `_BoundChatRunnable` to utilize the normalization method before processing structured outputs.
- Enhanced tests to verify the correct behavior of the normalization process across various scenarios, ensuring that system messages are handled appropriately in both standard and structured output contexts.

These changes aim to improve message handling in the Bedrock LLM, preventing errors related to non-consecutive system messages and enhancing overall robustness.
@qodo-code-review

Copy link
Copy Markdown

CI Feedback 🧐

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: seed-schema-contract

Failed stage: Initialize containers [❌]

Failed test name: ""

Failure summary:

The action failed while starting the LocalStack service container because Docker could not pull the
image localstack/localstack:3.9.
- docker pull localstack/localstack:3.9 returned manifest unknown
(tag 3.9 does not exist or is not available for the runner’s platform/registry).
- After multiple
retries, the pull still failed and the job exited with Docker pull failed with exit code 1, then
cleaned up the created network.

Relevant error logs:
1:  ##[group]Runner Image Provisioner
2:  Hosted Compute Agent
...

33:  Docker daemon API version: '1.48'
34:  ##[command]/usr/bin/docker version --format '{{.Client.APIVersion}}'
35:  '1.48'
36:  Docker client API version: '1.48'
37:  ##[endgroup]
38:  ##[group]Clean up resources from previous jobs
39:  ##[command]/usr/bin/docker ps --all --quiet --no-trunc --filter "label=357eff"
40:  ##[command]/usr/bin/docker network prune --force --filter "label=357eff"
41:  ##[endgroup]
42:  ##[group]Create local container network
43:  ##[command]/usr/bin/docker network create --label 357eff github_network_933a869318134518bcdb7580ba28a15b
44:  9a8619ce1193f95649ae3b66c22bb21e459b81387a0e1ae188cc6c46d0bacf9e
45:  ##[endgroup]
46:  ##[group]Starting localstack service container
47:  ##[command]/usr/bin/docker pull localstack/localstack:3.9
48:  Error response from daemon: manifest for localstack/localstack:3.9 not found: manifest unknown: manifest unknown
49:  ##[warning]Docker pull failed with exit code 1, back off 7.599 seconds before retry.
50:  ##[command]/usr/bin/docker pull localstack/localstack:3.9
51:  Error response from daemon: manifest for localstack/localstack:3.9 not found: manifest unknown: manifest unknown
52:  ##[warning]Docker pull failed with exit code 1, back off 5.484 seconds before retry.
53:  ##[command]/usr/bin/docker pull localstack/localstack:3.9
54:  Error response from daemon: manifest for localstack/localstack:3.9 not found: manifest unknown: manifest unknown
55:  ##[error]Docker pull failed with exit code 1
56:  Remove container network: github_network_933a869318134518bcdb7580ba28a15b

@qodo-code-review

Copy link
Copy Markdown

Review Summary by Qodo

Python AWS Agent Core: End-to-end hardening for Bedrock AgentCore + EKS with HITL approval and multi-tenant support

✨ Enhancement 🧪 Tests

Grey Divider

Walkthroughs

Description
• **End-to-end hardening of aws-agent-core runtime** for AWS Bedrock AgentCore + EKS deployment
  while preserving local Kubernetes profile
• **New server.py entrypoint and VPC-scoped AgentCore wiring** replace legacy main.py boot path;
  composition root refactored with AWS/local profile strategies
• **Bedrock adapter upgraded to us.amazon.nova-pro-v1:0** on hard-pinned us-east-1 region with
  Converse-compliant system-message normalization
• **MCP-internal Spring Boot JAR removed from repo** and resolved from GitHub Release at build time
  via scripts/build_mcp_image.sh
• **Graph enhancements**: HITL approval checkpoint, idempotent async dispatch via SQS work
  publisher, multi-tenant resolution, token governance, and richer observability
• **Composition refactoring**: Shared _BaseProfileDependencies mixin, LLM governance helper,
  tracer registry-driven dispatcher, boot-time DynamoDB schema validation
• **Observability improvements**: OTLP exporter refactored with lazy enum loading, span ID masking
  for OTLP/proto-http compliance, X-Ray-compatible trace ID generation
• **Webhook handler split** into synchronous prevalidate gate and asynchronous process execution
  paths with tenant binding and payload archival
• **Designer and approval nodes** refactored with AgenticTurnSpec abstraction, deterministic
  fallback paths, and structured debug event logging
• **Comprehensive test coverage**: New test suites for AWS composition internals, SQS worker
  composition, designer/approval nodes, tools node observability, and dependency resolution
• **Breaking changes**: .bedrock_agentcore.yaml entrypoint moves from src/agent/main.pysrc/agent/server.py; default Bedrock model becomes us.amazon.nova-pro-v1:0; AWS region
  hard-pinned to us-east-1; mcp-internal-*.jar no longer tracked in git
• **Engineering quality gates met**: mypy --strict green, SOLID+IoC via Protocol contracts in
  agent.contracts, 100% coverage gate, ruff check green, no secrets committed
• **Local + Terraform parity**: All AWS-only behavior has 1:1 local and Terraform counterpart;
  secrets documented in .env.local.example and deploy/local/configmap.yaml
Diagram
flowchart LR
  A["Legacy main.py<br/>single-tenant<br/>local-only"] -->|"refactor to<br/>profile strategy"| B["New server.py<br/>+ composition root"]
  B -->|"AWS profile"| C["AgentCore Runtime<br/>Invoker + SQS<br/>Work Publisher"]
  B -->|"Local profile"| D["LangGraph<br/>+ MCP Session"]
  C -->|"HITL gate"| E["Approval Node<br/>+ Designer Node"]
  D -->|"HITL gate"| E
  E -->|"token governance<br/>+ observability"| F["Enhanced Graph<br/>with Tracer Registry<br/>+ OTLP Exporter"]
  G["Webhook Handler<br/>split: prevalidate<br/>+ async process"] -->|"tenant resolve<br/>+ payload sink"| C
  G -->|"tenant resolve<br/>+ payload sink"| D
Loading

Grey Divider

File Changes

1. src/agent/composition/aws.py ✨ Enhancement +723/-250

AWS composition root hardening for AgentCore Runtime and HITL approval

• Refactored AWS composition root to support AgentCore Runtime invocation, multi-tenant resolution,
 and HITL approval gates alongside existing token governance and observability infrastructure.
• Added _xray_compatible_id_generator() for X-Ray–compliant trace IDs and
 _default_bedrock_agentcore_client() factory with adaptive retry configuration for cold-start
 resilience.
• Introduced _LazyGuardrailRuntime for deferred Bedrock Guardrails client initialization and
 _build_default_aws_metrics() with OTLP/HTTP exporter wiring for CloudWatch metrics emission.
• Expanded AwsAgentDependencies dataclass to include work_publisher, work_queue_url,
 agentcore_invoker, and agentcore_runtime_arn fields; removed legacy mcp_client,
 signature_verifier, and idempotency_store fields.
• Refactored build_aws_dependencies() to wire approval LLM, dedupe store with TTL, SQS work
 publisher, and optional AgentCore invoker; added boot-time DynamoDB schema validation via
 check_table_contracts().
• Updated guardrail resolution to support JSON-shaped secrets via _extract_guardrail_field() and
 added safety guard allow_default_tenant_aws to prevent silent multi-tenant budget
 cross-contamination.

src/agent/composition/aws.py


2. src/agent/composition/_shared.py ✨ Enhancement +746/-162

Shared composition refactor: base mixin, LLM governance helper, tracer registry

• Introduced _BaseProfileDependencies mixin class with six proxy properties (clock, logger,
 domain_state_store, llm, default_tenant) to reduce boilerplate in per-profile wrapper
 dataclasses.
• Refactored AgentDependencies to remove webhook signature verification, idempotency, and Jira
 integration fields; added mcp_base_url, mcp_bearer_token_provider, tenant_resolver,
 payload_sink, work_publisher, dedupe_store, and agentcore_invoker fields.
• Added _build_governed_llm() helper to encapsulate four-step LLM decoration (chat model →
 guardrail decorator → governance wrapper) for assessor/designer/approval roles.
• Introduced build_webhook_prevalidator() for async-dispatch path with sentinel
 _RaiseOnInvokeGraph and build_webhook_handler_for_request() for per-request MCP session loading.
• Added check_table_contracts() boot-time probe with _TableContractSpec, _diff_table_schema(),
 and _diff_table_ttl() helpers to validate DynamoDB schema at startup.
• Refactored tracer resolution into _resolve_tracer_from_backend() registry-driven dispatcher
 supporting profile-specific backends (e.g., X-Ray for AWS) alongside shared backends
 (Jaeger/OTLP/noop).
• Added _resolve_tenant_resolver() fallback logic and _default_secrets_manager_client() /
 _default_bedrock_runtime_client() factories.

src/agent/composition/_shared.py


3. src/agent/infrastructure/observability/otlp_exporter.py ✨ Enhancement +229/-41

OTLP exporter refactor: lazy enum loading and span ID masking

• Refactored OTLPSpanExporter.__call__() to delegate span construction to new
 _build_readable_span() helper for cleaner separation of concerns.
• Introduced lazy-loaded OTel enum resolution via _otel_enums() cache to avoid hard dependency on
 opentelemetry at module import time; added fallback enums (_FallbackStatusCode,
 _FallbackStatus, _FallbackSpanKind, _FallbackTraceState, _FallbackResource) for SDK-absent
 paths.
• Added _SPAN_ID_BIT_MASK constant to mask span IDs to 64-bit wire format (OTLP/proto-http
 requirement) preventing silent span drops on export.
• Enhanced _SpanContext to accept optional trace_state parameter and updated _ReadableSpan
 constructor to accept kind and resource parameters plus dropped_* counters for modern OTLP
 encoder compatibility.
• Improved docstrings to document modern encoder field access patterns and fallback behavior.

src/agent/infrastructure/observability/otlp_exporter.py


View more (113)
4. src/agent/infrastructure/identity/__init__.py Miscellaneous +2/-0

New identity verification module placeholder

• Created new module stub for identity-verification adapters.

src/agent/infrastructure/identity/init.py


5. tests/test_main.py 🧪 Tests +383/-120

Comprehensive test refactor for dependency resolution and model family support

• Refactored test suite to align with new resolve_dependencies API (renamed from
 build_app_from_environment), now returning dependency aggregates instead of app instances
• Added comprehensive test coverage for Bedrock model family classification
 (_classify_bedrock_model_family) supporting anthropic, deepseek, meta, and amazon families
• Introduced Ollama LLM provider support with tests for local profile configuration and optional
 Anthropic API key requirement
• Removed idempotency-store tests and webhook HMAC secret handling; updated fake clients to support
 AgentCore invoker and in-memory checkpointer
• Added tenant resolution, payload sink archival, and debug event emission test coverage

tests/test_main.py


6. tests/application/test_webhook_handler.py ✨ Enhancement +380/-343

Webhook handler split into prevalidate gate and async process

• Split webhook handler contract into synchronous prevalidate gate and asynchronous process
 execution paths
• Removed signature verification and idempotency deduplication from handler; added tenant resolution
 and payload sink integration
• Introduced PrevalidateAccept and PrevalidateTerminal outcome types for clearer request flow
 semantics
• Added comprehensive tests for tenant binding resolution, payload archival at multiple failure
 stages, and debug event ordering
• Simplified test fixtures by removing signature verifier mocks and HMAC secret handling

tests/application/test_webhook_handler.py


7. src/agent/main.py ✨ Enhancement +352/-128

Env-var resolution refactor with model family and Ollama support

• Renamed build_app_from_environment to resolve_dependencies to clarify it returns dependency
 aggregates, not app instances
• Added Bedrock model family classification with support for anthropic, deepseek, meta, and amazon
 families via _classify_bedrock_model_family
• Introduced Ollama LLM provider support for local profile with configurable base URL and model ID
• Removed webhook HMAC secret and idempotency table env-var handling; added AgentCore runtime ARN,
 webhook work queue, and dedupe table configuration
• Refactored profile dispatch to use strategy-pattern registry (_PROFILE_BUILDERS) instead of
 if/elif chain; defaults to AWS profile when unset
• Added Bedrock AgentCore runtime compatibility shim that conditionally wires app when SDK is
 available

src/agent/main.py


8. tests/composition/test_aws_internals.py 🧪 Tests +477/-0

New AWS composition internals coverage tests

• New test file covering AWS composition internals: AwsAgentDependencies property accessors,
 _LazyGuardrailRuntime factory caching, and guardrail resolution
• Added comprehensive OTel metrics provider installation tests with fallback scenarios (missing SDK,
 unavailable get_meter, already-installed provider)
• Included X-Ray-compatible trace ID generator tests ensuring timestamp prefix encoding for proper
 ServiceLens indexing
• Tests for guardrail field extraction and secrets resolver integration

tests/composition/test_aws_internals.py


9. .github/workflows/terraform.yml ⚙️ Configuration changes +1/-1

Terraform version bump in CI workflow

• Updated Terraform version from 1.9.5 to 1.13 in CI workflow setup

.github/workflows/terraform.yml


10. tests/worker/test_composition.py 🧪 Tests +745/-0

SQS worker composition and message processing test suite

• Comprehensive test suite for SQS worker composition, covering envelope parsing, prevalidation,
 deduplication, and message processing workflows
• Tests both local LangGraph path (with MCP session) and AWS AgentCore invocation path with proper
 error handling and logging verification
• Validates constructor-time guards for required configuration (work_queue_url, mcp_base_url) and
 runtime message callback behavior
• Includes end-to-end integration test demonstrating full message lifecycle from SQS receive through
 processing and deletion

tests/worker/test_composition.py


11. tests/graph/test_design_nodes.py 🧪 Tests +691/-0

Implementation designer LangGraph nodes test coverage

• Tests for designer LLM node covering iter-0 seeding with RemoveMessage sentinel, iteration counter
 advancement, and max-iterations halt behavior
• Validates design_terminal structured-output extraction accepting both typed ImplementationPlan and
 bare dict responses
• Tests design_post_comment evidence recording with fallback deterministic Jira comment posting when
 LLM skips the write
• Covers tool lookup, markdown rendering, and evidence reference generation for implementation plans

tests/graph/test_design_nodes.py


12. tests/graph/test_evaluate_human_approval_node.py 🧪 Tests +715/-0

Human-in-the-loop approval evaluation node tests

• Tests for HITL approval evaluation node covering approve/reject/override scenarios with human
 agreement verdicts
• Validates bind-tools loop with jira_get_* tool dispatch, message folding, and structured
 ApprovalEvaluation output
• Tests override comment posting via deterministic fallback when LLM skips jira_add_comment in tool
 loop
• Covers tool lookup, error handling for malformed tool calls, and readiness_phase routing decisions

tests/graph/test_evaluate_human_approval_node.py


13. tests/graph/test_tools_node_span.py 🧪 Tests +362/-0

Tools node span wrapper and observability tests

• Tests for tools node parent span wrapper that captures tool-call count and sorted name list for
 observability
• Validates _tool_dispatch_attrs helper handling tool-call extraction, name capping, and defensive
 null-checks
• Tests _make_tools_node wrapper ensuring parent span remains open during inner ToolNode execution
• Covers governance scope tenant resolution and exception propagation from inner ToolNode

tests/graph/test_tools_node_span.py


14. src/agent/graph/nodes/designer.py ✨ Enhancement +390/-133

Designer nodes refactoring with HITL approval routing

• Refactored design_llm_node to use shared AgenticTurnSpec abstraction with iter-0 seeding via
 RemoveMessage sentinel for assessor history flush
• Added _route_after_terminal_assess four-way router supporting resume from
 awaiting_design/awaiting_human_approval checkpoints and fresh ready assessments
• Implemented deterministic fallback path in design_post_comment with _render_plan_markdown and
 _find_named_tool helpers for when LLM skips Jira write
• Enhanced logging with structured debug events for iteration details, plan metadata, and fallback
 posting outcomes

src/agent/graph/nodes/designer.py


15. src/agent/compat/__init__.py Miscellaneous +9/-0

Cross-version compatibility shims module

• New compatibility shims module for cross-version Python support
• Provides single import point for version-specific re-bindings to avoid repeated sys.version_info
 guards across codebase

src/agent/compat/init.py


16. .bedrock_agentcore.yaml Additional files +127/-23

...

.bedrock_agentcore.yaml


17. .cursorignore Additional files +12/-0

...

.cursorignore


18. .dockerignore Additional files +19/-0

...

.dockerignore


19. .env.example Additional files +18/-2

...

.env.example


20. .env.local.example Additional files +142/-14

...

.env.local.example


21. .gitattributes Additional files +15/-15

...

.gitattributes


22. .github/workflows/ci.yml Additional files +457/-13

...

.github/workflows/ci.yml


23. .github/workflows/iam-canary.yml Additional files +2/-2

...

.github/workflows/iam-canary.yml


24. .github/workflows/mutation.yml Additional files +7/-7

...

.github/workflows/mutation.yml


25. .github/workflows/sandbox-apply.yml Additional files +124/-0

...

.github/workflows/sandbox-apply.yml


26. .github/workflows/smoke.yml Additional files +129/-37

...

.github/workflows/smoke.yml


27. .github/workflows/soak.yml Additional files +140/-0

...

.github/workflows/soak.yml


28. Dockerfile Additional files +11/-10

...

Dockerfile


29. Makefile Additional files +154/-51

...

Makefile


30. README.md Additional files +245/-83

...

README.md


31. aws-deploy-plan.md Additional files +2320/-0

...

aws-deploy-plan.md


32. deploy/aws/README.md Additional files +86/-0

...

deploy/aws/README.md


33. deploy/aws/network-policy/cilium-agent-runtime.yaml Additional files +73/-0

...

deploy/aws/network-policy/cilium-agent-runtime.yaml


34. deploy/aws/network-policy/cilium-mcp-internal.yaml Additional files +53/-0

...

deploy/aws/network-policy/cilium-mcp-internal.yaml


35. deploy/aws/network-policy/mcp-internal.yaml Additional files +0/-132

...

deploy/aws/network-policy/mcp-internal.yaml


36. deploy/local/README.md Additional files +142/-67

...

deploy/local/README.md


37. deploy/local/agent-runtime.yaml Additional files +0/-118

...

deploy/local/agent-runtime.yaml


38. deploy/local/agent-worker.yaml Additional files +93/-0

...

deploy/local/agent-worker.yaml


39. deploy/local/bootstrap.sh Additional files +689/-0

...

deploy/local/bootstrap.sh


40. deploy/local/configmap.yaml Additional files +63/-7

...

deploy/local/configmap.yaml


41. deploy/local/jaeger.yaml Additional files +1/-1

...

deploy/local/jaeger.yaml


42. deploy/local/kustomization.yaml Additional files +3/-4

...

deploy/local/kustomization.yaml


43. deploy/local/mcp-internal.yaml Additional files +41/-13

...

deploy/local/mcp-internal.yaml


44. deploy/local/overlays/host-mount/agent-runtime-patch.yaml Additional files +0/-110

...

deploy/local/overlays/host-mount/agent-runtime-patch.yaml


45. deploy/local/overlays/host-mount/agent-worker-patch.yaml Additional files +115/-0

...

deploy/local/overlays/host-mount/agent-worker-patch.yaml


46. deploy/local/overlays/host-mount/kustomization.yaml Additional files +13/-12

...

deploy/local/overlays/host-mount/kustomization.yaml


47. deploy/local/p-bootstrap.sh Additional files +411/-0

...

deploy/local/p-bootstrap.sh


48. deploy/local/p-teardown.sh Additional files +179/-0

...

deploy/local/p-teardown.sh


49. deploy/local/secrets.example.yaml Additional files +36/-1

...

deploy/local/secrets.example.yaml


50. deploy/local/teardown.sh Additional files +220/-0

...

deploy/local/teardown.sh


51. docs/ARCHITECTURE.md Additional files +446/-183

...

docs/ARCHITECTURE.md


52. docs/DEFERRED.md Additional files +170/-0

...

docs/DEFERRED.md


53. docs/PYTHON_DEVELOPMENT.md Additional files +124/-15

...

docs/PYTHON_DEVELOPMENT.md


54. docs/adr/README.md Additional files +10/-0

...

docs/adr/README.md


55. docs/deploy-eks-secrets-csi.md Additional files +146/-0

...

docs/deploy-eks-secrets-csi.md


56. docs/oncall.md Additional files +7/-7

...

docs/oncall.md


57. docs/runbook-sandbox-deploy.md Additional files +565/-0

...

docs/runbook-sandbox-deploy.md


58. docs/runbook.md Additional files +789/-33

...

docs/runbook.md


59. docs/sbom/README.md Additional files +50/-0

...

docs/sbom/README.md


60. docs/structured_log_schema.json Additional files +123/-19

...

docs/structured_log_schema.json


61. lambda/webhook_validator/handler.py Additional files +309/-0

...

lambda/webhook_validator/handler.py


62. lambda/webhook_validator/requirements.txt Additional files +5/-0

...

lambda/webhook_validator/requirements.txt


63. lambda/webhook_validator/tests/__init__.py Additional files +0/-0

...

lambda/webhook_validator/tests/init.py


64. lambda/webhook_validator/tests/test_handler.py Additional files +319/-0

...

lambda/webhook_validator/tests/test_handler.py


65. langgraph.json Additional files +9/-0

...

langgraph.json


66. mcp/.dockerignore Additional files +12/-5

...

mcp/.dockerignore


67. mcp/Dockerfile Additional files +106/-13

...

mcp/Dockerfile


68. mcp/README.md Additional files +72/-23

...

mcp/README.md


69. mcp/config.json Additional files +1/-1

...

mcp/config.json


70. mcp/entrypoint.sh Additional files +128/-0

...

mcp/entrypoint.sh


71. mcp/run.sh Additional files +17/-0

...

mcp/run.sh


72. ops/iam_canary.example.yaml Additional files +5/-2

...

ops/iam_canary.example.yaml


73. pyproject.toml Additional files +133/-148

...

pyproject.toml


74. requirements.txt Additional files +25/-6

...

requirements.txt


75. scripts/agentcore_deploy_wrapper.py Additional files +203/-0

...

scripts/agentcore_deploy_wrapper.py


76. scripts/bootstrap_agentcore_runtime.sh Additional files +593/-0

...

scripts/bootstrap_agentcore_runtime.sh


77. scripts/bootstrap_eks_workloads.sh Additional files +451/-0

...

scripts/bootstrap_eks_workloads.sh


78. scripts/bootstrap_tfstate.sh Additional files +531/-0

...

scripts/bootstrap_tfstate.sh


79. scripts/build_agent_worker_image.sh Additional files +220/-0

...

scripts/build_agent_worker_image.sh


80. scripts/build_mcp_image.sh Additional files +111/-50

...

scripts/build_mcp_image.sh


81. scripts/check_bedrock_model_access.py Additional files +208/-0

...

scripts/check_bedrock_model_access.py


82. scripts/check_docs_code_sync.py Additional files +442/-0

...

scripts/check_docs_code_sync.py


83. scripts/check_event_identifiers.py Additional files +224/-0

...

scripts/check_event_identifiers.py


84. scripts/check_langsmith_unreachable.py Additional files +192/-0

...

scripts/check_langsmith_unreachable.py


85. scripts/check_model_prices.py Additional files +107/-0

...

scripts/check_model_prices.py


86. scripts/check_seed_schemas.py Additional files +291/-0

...

scripts/check_seed_schemas.py


87. scripts/check_sonnet46_access.sh Additional files +135/-0

...

scripts/check_sonnet46_access.sh


88. scripts/check_unused_seams.py Additional files +108/-0

...

scripts/check_unused_seams.py


89. scripts/ensure_eks_cli_admin_access.sh Additional files +195/-0

...

scripts/ensure_eks_cli_admin_access.sh


90. scripts/export_token_usage.py Additional files +6/-6

...

scripts/export_token_usage.py


91. scripts/fixtures/manual/README.md Additional files +92/-0

...

scripts/fixtures/manual/README.md


92. scripts/fixtures/manual/scrum_dev_manual.json Additional files +83/-0

...

scripts/fixtures/manual/scrum_dev_manual.json


93. scripts/fixtures/manual/scrum_toolcheck_fresh.json Additional files +74/-0

...

scripts/fixtures/manual/scrum_toolcheck_fresh.json


94. scripts/iam_policy_canary.py Additional files +1/-1

...

scripts/iam_policy_canary.py


95. scripts/invoke_lambda.sh Additional files +302/-0

...

scripts/invoke_lambda.sh


96. scripts/invoke_manual.sh Additional files +182/-0

...

scripts/invoke_manual.sh


97. scripts/local_dynamodb_admin.sh Additional files +240/-0

...

scripts/local_dynamodb_admin.sh


98. scripts/promote_to_agentcore.sh Additional files +5/-153

...

scripts/promote_to_agentcore.sh


99. scripts/rebuild_sandbox.sh Additional files +204/-0

...

scripts/rebuild_sandbox.sh


100. scripts/replace3.sh Additional files +72/-0

...

scripts/replace3.sh


101. scripts/replay_dlq.py Additional files +124/-112

...

scripts/replay_dlq.py


102. scripts/smoke.py Additional files +267/-84

...

scripts/smoke.py


103. scripts/sqs-peek.sh Additional files +255/-0

...

scripts/sqs-peek.sh


104. scripts/sync_jira_integration_secret_from_env.sh Additional files +71/-0

...

scripts/sync_jira_integration_secret_from_env.sh


105. scripts/sync_sandbox_tfvars.py Additional files +466/-0

...

scripts/sync_sandbox_tfvars.py


106. scripts/sync_webhook_hmac_secret.sh Additional files +157/-0

...

scripts/sync_webhook_hmac_secret.sh


107. scripts/teardown_sandbox.sh Additional files +1027/-0

...

scripts/teardown_sandbox.sh


108. scripts/tf.sh Additional files +120/-0

...

scripts/tf.sh


109. scripts/validate_rebuild_state.sh Additional files +305/-0

...

scripts/validate_rebuild_state.sh


110. src/agent/__init__.py Additional files +9/-4

...

src/agent/init.py


111. src/agent/application/comment_renderer.py Additional files +0/-65

...

src/agent/application/comment_renderer.py


112. src/agent/application/event_invariants.py Additional files +0/-278

...

src/agent/application/event_invariants.py


113. src/agent/application/idempotency.py Additional files +0/-62

...

src/agent/application/idempotency.py


114. src/agent/application/implementation_plan_renderer.py Additional files +0/-68

...

src/agent/application/implementation_plan_renderer.py


115. src/agent/application/mcp_tool_classifier.py Additional files +29/-0

...

src/agent/application/mcp_tool_classifier.py


116. Additional files not shown Additional files +0/-0

...

Additional files not shown


Grey Divider

Qodo Logo

@qodo-code-review

qodo-code-review Bot commented May 11, 2026

Copy link
Copy Markdown

Code Review by Qodo

🐞 Bugs (2) 📘 Rule violations (4)

Grey Divider


Action required

1. pyproject.toml deps not pinned 📘 Rule violation ☼ Reliability
Description
The updated Python dependency manifests list packages without exact versions, which makes builds
non-reproducible and can silently pull in breaking or vulnerable releases. This violates the
requirement to pin dependency versions (and keep dependency resolution deterministic).
Code

pyproject.toml[R19-86]

dependencies = [
-    "bedrock-agentcore>=1.7.0,<2",
-    "langgraph>=0.2.0,<2",
-    "langchain-aws>=0.2.0,<1",
-    # UsageMetadataCallbackHandler is the load-bearing token-counting seam
-    # (added in langchain-core 0.3.49).
-    "langchain-core>=0.3.49,<1",
-    "pydantic>=2.7.0,<3",
-    # tenacity drives the half-open probe scheduling in
-    # InMemoryLlmCircuitBreaker (wait_random_exponential).
-    "tenacity>=9.0.0,<10",
+    "bedrock-agentcore",
+    "langgraph",
+    "langchain-aws",
+    "langchain-core",
+    "pydantic",
+    "tenacity",
+    # Official Model Context Protocol Python SDK. Owns the Streamable HTTP
+    # transport, JSON-RPC framing, session-id propagation, and protocol-version
+    # negotiation. Imported only from `agent.composition._mcp_session` and
+    # `agent.infrastructure.mcp.*`; application and graph layers see only
+    # the abstract Protocol from `agent.contracts.mcp`.
+    "mcp",
+    # Upstream MCP -> LangChain `StructuredTool` bridge. Used directly with
+    # zero hand-rolled wrapping; per-call observability is layered on via
+    # the upstream `ToolCallInterceptor` Protocol implementation in
+    # `agent.infrastructure.mcp.observability_interceptor`. The 0.2.x
+    # series targets `langchain-core>=1.0`, which the rest of this
+    # project's stack (`langchain-aws>=1.x`, `langgraph>=1.x`) now also
+    # uses, so no version cap is required.
+    "langchain-mcp-adapters",
]

[project.optional-dependencies]
dev = [
-    "pytest>=8.0.0",
-    "pytest-cov>=5.0.0",
-    "coverage[toml]>=7.5.0",
-    "mypy>=1.10.0",
-    "ruff>=0.5.0",
-    "freezegun>=1.5.0",
-    # Property-based tests on TokenUsage invariants and the budget arithmetic.
-    "hypothesis>=6.100.0,<7",
-    # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
-    #   `watchfiles` powers the in-Pod hot-reload loop wired by
-    #   `deploy/local/overlays/host-mount/agent-runtime-patch.yaml`
-    #   (`make local-up-watch`). It is dev-only — production agent runtime
-    #   images MUST NOT install the [dev] extra, so the inotify/native
-    #   filesystem-watcher dependency never ships to AWS. Approval ticket:
-    #   TODO before merge per the approved-deps registry process.
-    # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
-    #   The version range matches the existing convention used by every
-    #   other [dev] / optional-extras entry in this file (caret-style
-    #   >=major.minor,<next-major).  Exact-version pinning + lockfile
-    #   generation is the M2.5 deliverable (§14.5).
-    "watchfiles>=0.21.0,<1",
-    # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
-    #   `import-linter` enforces the Wave 4 sealed-test-seams contract
-    #   under `[tool.importlinter]` below: production code in the
-    #   `agent` package must not import from
-    #   `agent.composition._test_seams`.  The check runs as a
-    #   `lint-imports` step in the same CI lane as `ruff` /
-    #   `mypy` so a regression surfaces at PR-review time.  It is
-    #   dev-only — production agent runtime images do not install
-    #   `[dev]`, so the linter never ships to AWS.  Approval ticket:
-    #   TODO before merge per the approved-deps registry process.
-    # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
-    #   The version range matches the existing convention used by
-    #   every other `[dev]` / optional-extras entry in this file
-    #   (caret-style `>=major.minor,<next-major`).
-    "import-linter>=2.0,<3",
-    # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
-    #   `mutmut` powers the Wave 9 weekly mutation-testing workflow
-    #   (`.github/workflows/mutation.yml`).  It is dev-only and runs
-    #   on a separate scheduled job (advisory, not gating) so a
-    #   surviving mutant surfaces in the workflow artifact without
-    #   blocking PRs while the suite is hardened.  Production agent
-    #   runtime images do not install `[dev]`, so `mutmut` never
-    #   ships to AWS.  Approval ticket: TODO before merge per the
-    #   approved-deps registry process.
-    # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
-    #   The version range matches the existing caret-style convention
-    #   (`>=major.minor,<next-major`) used by every other entry in
-    #   this section; exact-version pinning + lockfile generation is
-    #   the M2.5 deliverable (§14.5).
-    "mutmut>=3.0,<4",
+    "pytest",
+    "pytest-cov",
+    "coverage[toml]",
+    "mypy",
+    "ruff",
+    "freezegun",
+    "hypothesis",
+    "watchfiles",
+    "import-linter",
+    "mutmut",
+    # `pytest-asyncio` powers the `async def test_*` discovery for the
+    # async migration. With `asyncio_mode = "auto"` (configured below) every
+    # async test function is auto-wrapped in an event loop without per-test
+    # decorators. Sync tests stay sync.
+    "pytest-asyncio",
]
-# OpenTelemetry GenAI semantic conventions wiring.
-# Kept as an optional extra so the unit-test + mypy jobs do NOT pull
-# OpenTelemetry transitive deps; the OtelMeterMetricsRecorder consumes
-# the SDK via its structural Protocol surface.  AWS production picks
-# this extra up via the same `[xray]` install path.
otel = [
-    "opentelemetry-api>=1.27.0,<2",
-    "opentelemetry-sdk>=1.27.0,<2",
+    "opentelemetry-api",
+    "opentelemetry-sdk",
]
-# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
-#   `langchain-anthropic` powers the local profile's `ChatAnthropic` LLM factory.
-#   It is isolated as an optional extra so the default install (and the
-#   100%-coverage unit-test job) does NOT pull
-#   Anthropic's transitive deps. Approval ticket: TODO before merge per the
-#   approved-deps registry process.
-# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
-#   The version range here matches the existing convention used by `langchain-aws`
-#   and `langchain-core` in this file (caret-style >=major.minor,<next-major).
-#   Exact-version pinning + lockfile generation is M2.5 deliverable (§14.5).
anthropic = [
-    "langchain-anthropic>=0.3.0,<1",
+    "langchain-anthropic",
]
-# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
-#   `aws-xray-sdk` and the OpenTelemetry OTLP exporter power the X-Ray tracer
-#   (the production-only AWS X-Ray exporter / segment-shape seam). They are
-#   isolated as an optional extra so the default install (and the
-#   100%-coverage unit-test job) does NOT
-#   pull `aws-xray-sdk` or `opentelemetry-*` transitive deps. Approval ticket:
-#   TODO before merge per the approved-deps registry process.
-# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
-#   The version ranges here match the existing convention used by
-#   `langchain-aws` / `langchain-core` / `langchain-anthropic` (caret-style
-#   >=major.minor,<next-major). Exact-version pinning + lockfile generation
-#   lands as a follow-up promotion deliverable.
-xray = [
-    "aws-xray-sdk>=2.14.0,<3",
-    "opentelemetry-sdk>=1.26.0,<2",
-    "opentelemetry-exporter-otlp-proto-http>=1.26.0,<2",
+ollama = [
+    "langchain-ollama",
]
-# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
-#   `langchain-mcp-adapters` is the canonical bridge from the MCP `tools/list` /
-#   `tools/call` JSON-RPC surface to LangChain `StructuredTool`s consumed by
-#   `model.bind_tools(...)` and `langgraph.prebuilt.ToolNode`. It is isolated as
-#   an optional extra so the default install (and the
-#   100%-coverage unit-test job) does NOT pull `langchain-mcp-adapters` transitive
-#   deps. Approval ticket: TODO before merge per the approved-deps registry process.
-# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
-#   The version range here matches the existing convention used by `langchain-aws` /
-#   `langchain-core` / `langchain-anthropic` (caret-style >=major.minor,<next-major).
-#   Exact-version pinning + lockfile generation is a deliverable counterpart
-#   to the earlier lockfile entry.
-mcp = [
-    "langchain-mcp-adapters>=0.1.0,<1",
+xray = [
+    "aws-xray-sdk",
+    "opentelemetry-sdk",
+    "opentelemetry-exporter-otlp-proto-http",
]
-# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
-#   `langgraph-checkpoint-aws` provides the AgentCore-backed LangGraph
-#   checkpointer (`AgentCoreMemorySaver`) and the DynamoDB-backed
-#   `DynamoDbSaver` consumed by `_default_aws_checkpointer_factory` in
-#   `agent.composition._shared`.  `bedrock-agentcore` is the SDK that
-#   exposes `AgentCoreMemorySaver` as a fallback when the dedicated
-#   integration package is missing; it already ships in the base
-#   `dependencies` block (the runtime entrypoint imports `BedrockAgentCoreApp`
-#   unconditionally) so this extra restates the version pin for parity
-#   with the operator-facing `pip install -e '.[aws]'` instruction in
-#   the lazy-import RuntimeError.  The unit-test + 100%-coverage job
-#   does NOT install this extra, so mypy and pytest stay free of the
-#   AWS-only transitive deps.  Approval ticket: TODO before merge per
-#   the approved-deps registry process.
-# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
-#   The version ranges match the existing caret-style convention used
-#   by every other extras block in this file
-#   (`>=major.minor,<next-major`).  Exact-version pinning + lockfile
-#   generation is the M2.5 deliverable (§14.5).
aws = [
-    "langgraph-checkpoint-aws>=0.1.0,<1",
-    "bedrock-agentcore>=1.7.0,<2",
+    "langgraph-checkpoint-aws",
+    "bedrock-agentcore",
+    "PyJWT[crypto]",
+]
+debug = [
+    "langgraph-cli[inmem]",
+    "debugpy",
+]
+perf = [
+    "locust",
]
Evidence
PR Compliance ID 1 requires exact/immutable dependency versions in manifests (and consistent
lockfiles when applicable). The changed dependency entries in pyproject.toml and
requirements.txt omit versions entirely, so installs can resolve to different versions over time.

Rule 1: Pin dependency versions in manifests and lockfiles
pyproject.toml[19-86]
requirements.txt[1-25]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`pyproject.toml` and `requirements.txt` now specify dependencies without exact versions (e.g., `"langgraph"` instead of `"langgraph==<exact>"`). This allows dependency drift across installs/builds and violates the dependency pinning compliance rule.

## Issue Context
The PR replaced previously version-ranged dependencies with unversioned package names, and `requirements.txt` installs `.[aws,xray,otel]` without any pins/hashes.

## Fix Focus Areas
- pyproject.toml[19-86]
- requirements.txt[1-25]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Docker image tag floats 📘 Rule violation ☼ Reliability
Description
The Dockerfile uses python:3.13-slim-bookworm without a patch version or digest, so builds can
change over time as the tag moves. This violates the requirement to pin dependency versions in
Docker images.
Code

Dockerfile[41]

+FROM public.ecr.aws/docker/library/python:3.13-slim-bookworm AS builder
Evidence
PR Compliance ID 1 requires Docker images be pinned to a specific version or digest instead of
floating tags. The Dockerfile uses public.ecr.aws/docker/library/python:3.13-slim-bookworm for
both stages without a digest, which is a floating tag.

Rule 1: Pin dependency versions in manifests and lockfiles
Dockerfile[41-41]
Dockerfile[94-94]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The runtime/builder base images are not pinned to an immutable version/digest.

## Issue Context
Compliance requires Docker base images be pinned to a specific version or digest; floating tags like `python:3.13-slim-bookworm` can change.

## Fix Focus Areas
- Dockerfile[41-41]
- Dockerfile[94-94]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. Terraform provider versions unpinned 📘 Rule violation ☼ Reliability
Description
terraform/agentcore-runtime/versions.tf uses open-ended version constraints (>=) for Terraform
and providers, allowing upgrades without an explicit change review. This violates the dependency
pinning requirement for manifests.
Code

terraform/agentcore-runtime/versions.tf[R1-14]

terraform {
-  required_version = ">= 1.5.0"
+  required_version = ">= 1.13"

  required_providers {
    aws = {
      source = "hashicorp/aws"
-      # Bedrock AgentCore Runtime + Memory resources require a recent provider.
-      # If your provider version is older than this, see the TODO at the top of
-      # main.tf for the documented null_resource fallback.
-      version = ">= 5.95.0"
+      # Bedrock AgentCore Runtime + Memory resources are first-class on
+      # the 6.x line. If your provider version is older than this, see
+      # the TODO at the top of main.tf for the documented null_resource
+      # fallback.
+      version = ">= 6.0"
    }

    null = {
Evidence
PR Compliance ID 1 forbids open version ranges for dependencies in manifests. The updated Terraform
module specifies required_version = ">= 1.13" and provider constraints like version = ">= 6.0",
which are not exact pins.

Rule 1: Pin dependency versions in manifests and lockfiles
terraform/agentcore-runtime/versions.tf[1-17]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Terraform `required_version` / `required_providers` constraints are specified as open-ended ranges (e.g., `>= 6.0`) rather than exact versions.

## Issue Context
The compliance rule requires exact, immutable dependency versions in manifests.

## Fix Focus Areas
- terraform/agentcore-runtime/versions.tf[1-17]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View more (3)
4. Non-ASCII strings in build_mcp_image.sh 📘 Rule violation ✧ Quality
Description
Multiple modified user-facing/help text, docstrings, and committed JSON metadata/schema strings
include non-ASCII characters (e.g., , é, , ). This violates the English-only ASCII
string-literal policy and can break or degrade tooling/rendering/search behavior across environments
that assumes ASCII.
Code

scripts/build_mcp_image.sh[R195-212]

+…then re-run this script. If your checkout lives elsewhere, point
+\`MCP_INTERNAL_REPO\` at it or pass \`--jar /abs/path/to/mcp-internal-X.Y.Z.jar\`.
+
+See mcp/README.md "External JAR contract" for full details.
MISSING
    exit 2
fi

-# A Git-LFS pointer file is ~130 bytes; the real JAR is ~62 MB. Catch
-# the common "developer forgot `git lfs pull`" footgun before docker
-# wastes a layer on a useless 130-byte payload.
-JAR_SIZE_BYTES="$(wc -c <"${JAR_PATH}" | tr -d ' ')"
+# Spring Boot fat-JARs are typically 50–80 MB; anything dramatically
+# smaller is almost certainly a `-plain.jar` mistakenly passed in or a
+# truncated download. 1 MB is a safe floor.
+JAR_SIZE_BYTES="$(wc -c <"${JAR_SOURCE}" | tr -d ' ')"
if [[ "${JAR_SIZE_BYTES}" -lt 1048576 ]]; then
-    cat >&2 <<LFS_POINTER
-ERROR: ${JAR_PATH} is only ${JAR_SIZE_BYTES} bytes — looks like a Git LFS
-       pointer file rather than the real JAR. Run:
-
-           git lfs install
-           git lfs pull
-
-       and re-run this script.
-LFS_POINTER
+    cat >&2 <<TOO_SMALL
+ERROR: ${JAR_SOURCE} is only ${JAR_SIZE_BYTES} bytes — that's too small to
+       be a Spring Boot fat-JAR (expected ~50–80 MB). Make sure you are
+       pointing at the bootJar output, not the lightweight \`-plain.jar\`
+       Gradle also emits.
Evidence
PR Compliance ID 576985 requires all modified string literals/values to contain only ASCII
characters, and this applies across code (including docstrings), user-facing heredoc/error output,
and committed JSON configuration/schema files. The cited changes include characters outside ASCII
(code points > 127), such as  as well as Unicode dashes / and the accented é in café,
which directly demonstrates the policy violation in each referenced file and field.

Rule 576985: Enforce English-only string literals in code
scripts/build_mcp_image.sh[195-212]
scripts/agentcore_deploy_wrapper.py[2-33]
src/agent/infrastructure/cost/model_prices.json[2-14]
src/agent/infrastructure/structured_log_schema.json[215-235]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Modified string literals and JSON string values include non-ASCII characters (e.g., ellipsis `…`, accented characters like `é`, and Unicode dashes `—`/`–`), which violates the ASCII-only policy for committed code/config and user-facing text.

## Issue Context
PR Compliance ID 576985 requires all modified string literals/values (including Python docstrings, shell-script help/error output, and committed JSON configuration/metadata/schema) to be ASCII-only unless explicitly exempted. Non-ASCII characters can also cause inconsistent rendering/searching and can break tooling that assumes ASCII logs.

## Fix Focus Areas
- scripts/build_mcp_image.sh[195-212]
- scripts/agentcore_deploy_wrapper.py[2-33]
- src/agent/infrastructure/cost/model_prices.json[2-14]
- src/agent/infrastructure/structured_log_schema.json[215-235]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. Unbounded webhook payload logging 🐞 Bug ⛨ Security
Description
lambda/webhook_validator/handler.py’s _truncate_for_log() is a no-op, so the Lambda logs full
webhook bodies and full SQS envelope JSON at INFO/ERROR. This can leak sensitive/PII data into logs
and can explode log volume/cost or exceed log/annotation limits under large Jira payloads.
Code

lambda/webhook_validator/handler.py[R70-72]

+def _truncate_for_log(value: str, limit: int = _MAX_LOG_BODY_CHARS) -> str:
+    del limit
+    return value
Evidence
The helper intended to enforce _MAX_LOG_BODY_CHARS explicitly discards limit and returns the
original string, and the handler logs both the decoded request body and the serialized SQS envelope
via that helper.

lambda/webhook_validator/handler.py[70-72]
lambda/webhook_validator/handler.py[226-240]
lambda/webhook_validator/handler.py[274-296]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`_truncate_for_log()` currently ignores its `limit` parameter and returns the full payload string. Because it is used when logging the inbound request body and the SQS envelope, the Lambda can emit unbounded sensitive payloads into logs.

## Issue Context
This Lambda is the public ingress trust boundary; its logs are likely retained/centralized. Logging full Jira webhook JSON commonly includes user emails, ticket content, and other sensitive data.

## Fix Focus Areas
- Implement truncation (with an ellipsis) in `_truncate_for_log` using `limit`.
- Consider also switching body/envelope logging to DEBUG or logging only sizes + correlation_id (keeping INFO logs non-sensitive).

### Fix Focus Areas (code refs)
- lambda/webhook_validator/handler.py[70-72]
- lambda/webhook_validator/handler.py[226-240]
- lambda/webhook_validator/handler.py[274-296]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. DeepSeek/meta tool calls broken 🐞 Bug ≡ Correctness
Description
bedrock_factory._select_chat_model_ctor() forces the direct-invoke Bedrock adapter for the
deepseek/meta families (and also falls back to it on langchain_aws import failure), but that
adapter ignores bound tools and always returns AIMessage(..., tool_calls=[]). The graph routes to
terminal nodes when tool_calls is empty, so MCP tool execution is skipped and the documented
DeepSeek default cannot perform the agentic tool loop.
Code

src/agent/infrastructure/llm/bedrock_factory.py[R159-168]

+    if chat_bedrock_cls is not None:
+        return chat_bedrock_cls
+    if _direct_invoke_disabled_via_env():
+        return _make_direct_invoke_cls(_default_bedrock_runtime_client)
+    # Meta + DeepSeek keep a dedicated adapter path because their
+    # structured-output behavior differs from Anthropic/Amazon's
+    # native ChatBedrock path.
+    if bedrock_model_family in ("meta", "deepseek"):
+        return _make_direct_invoke_cls(_default_bedrock_runtime_client)
+    return _import_chat_bedrock()
Evidence
The factory explicitly selects the direct-invoke adapter for DeepSeek/meta, but that adapter
discards the tools and always yields an AIMessage with empty tool_calls; the graph’s routing logic
uses tool_calls presence to decide whether to execute tools, so tool execution is bypassed. The
repository documentation states DeepSeek is the default supported family and that tool-use must be
validated end-to-end, which conflicts with a tool-less adapter.

src/agent/infrastructure/llm/bedrock_factory.py[147-168]
src/agent/infrastructure/llm/bedrock_converse_direct.py[111-125]
src/agent/infrastructure/llm/bedrock_converse_direct.py[226-234]
src/agent/graph/nodes/_agentic_turn.py[193-246]
src/agent/main.py[55-72]
docs/runbook-sandbox-deploy.md[78-90]
docs/runbook-sandbox-deploy.md[141-176]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
DeepSeek/meta are treated as supported Bedrock model families, but the selected direct Bedrock `converse` adapter discards tools and never produces tool calls. This breaks the agent’s core tool loop (MCP reads/writes), because graph routing depends on `tool_calls` being populated.

## Issue Context
- `_select_chat_model_ctor()` routes `deepseek`/`meta` to `BedrockDirectInvokeChatModel`.
- `BedrockDirectInvokeChatModel.bind_tools()` deletes the tools argument.
- The runnable returns `AIMessage(..., tool_calls=[])` unconditionally.
- The agent graph routes to tools execution only when tool calls exist.
- Docs describe DeepSeek V3.2 as the default model family for the runtime and explicitly call out validating the “tool-use surface” end-to-end.

## Fix Focus Areas
Choose one (or both) of:
1) **Implement real tool-calling for direct `converse`**: pass Bedrock `toolConfig` derived from the bound tools and parse `toolUse` blocks from `converse` output into `AIMessage.tool_calls`.
2) **Fail closed**: if tools are bound (non-empty), raise a clear exception so the worker/runtime fails fast instead of silently skipping tool execution; also reconsider forcing direct-invoke for `deepseek`/`meta` until tool calling is implemented.

### Fix Focus Areas (code refs)
- src/agent/infrastructure/llm/bedrock_factory.py[159-168]
- src/agent/infrastructure/llm/bedrock_converse_direct.py[111-125]
- src/agent/infrastructure/llm/bedrock_converse_direct.py[226-234]
- src/agent/graph/nodes/_agentic_turn.py[193-246]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread pyproject.toml
Comment on lines 19 to 86
dependencies = [
"bedrock-agentcore>=1.7.0,<2",
"langgraph>=0.2.0,<2",
"langchain-aws>=0.2.0,<1",
# UsageMetadataCallbackHandler is the load-bearing token-counting seam
# (added in langchain-core 0.3.49).
"langchain-core>=0.3.49,<1",
"pydantic>=2.7.0,<3",
# tenacity drives the half-open probe scheduling in
# InMemoryLlmCircuitBreaker (wait_random_exponential).
"tenacity>=9.0.0,<10",
"bedrock-agentcore",
"langgraph",
"langchain-aws",
"langchain-core",
"pydantic",
"tenacity",
# Official Model Context Protocol Python SDK. Owns the Streamable HTTP
# transport, JSON-RPC framing, session-id propagation, and protocol-version
# negotiation. Imported only from `agent.composition._mcp_session` and
# `agent.infrastructure.mcp.*`; application and graph layers see only
# the abstract Protocol from `agent.contracts.mcp`.
"mcp",
# Upstream MCP -> LangChain `StructuredTool` bridge. Used directly with
# zero hand-rolled wrapping; per-call observability is layered on via
# the upstream `ToolCallInterceptor` Protocol implementation in
# `agent.infrastructure.mcp.observability_interceptor`. The 0.2.x
# series targets `langchain-core>=1.0`, which the rest of this
# project's stack (`langchain-aws>=1.x`, `langgraph>=1.x`) now also
# uses, so no version cap is required.
"langchain-mcp-adapters",
]

[project.optional-dependencies]
dev = [
"pytest>=8.0.0",
"pytest-cov>=5.0.0",
"coverage[toml]>=7.5.0",
"mypy>=1.10.0",
"ruff>=0.5.0",
"freezegun>=1.5.0",
# Property-based tests on TokenUsage invariants and the budget arithmetic.
"hypothesis>=6.100.0,<7",
# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
# `watchfiles` powers the in-Pod hot-reload loop wired by
# `deploy/local/overlays/host-mount/agent-runtime-patch.yaml`
# (`make local-up-watch`). It is dev-only — production agent runtime
# images MUST NOT install the [dev] extra, so the inotify/native
# filesystem-watcher dependency never ships to AWS. Approval ticket:
# TODO before merge per the approved-deps registry process.
# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
# The version range matches the existing convention used by every
# other [dev] / optional-extras entry in this file (caret-style
# >=major.minor,<next-major). Exact-version pinning + lockfile
# generation is the M2.5 deliverable (§14.5).
"watchfiles>=0.21.0,<1",
# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
# `import-linter` enforces the Wave 4 sealed-test-seams contract
# under `[tool.importlinter]` below: production code in the
# `agent` package must not import from
# `agent.composition._test_seams`. The check runs as a
# `lint-imports` step in the same CI lane as `ruff` /
# `mypy` so a regression surfaces at PR-review time. It is
# dev-only — production agent runtime images do not install
# `[dev]`, so the linter never ships to AWS. Approval ticket:
# TODO before merge per the approved-deps registry process.
# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
# The version range matches the existing convention used by
# every other `[dev]` / optional-extras entry in this file
# (caret-style `>=major.minor,<next-major`).
"import-linter>=2.0,<3",
# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
# `mutmut` powers the Wave 9 weekly mutation-testing workflow
# (`.github/workflows/mutation.yml`). It is dev-only and runs
# on a separate scheduled job (advisory, not gating) so a
# surviving mutant surfaces in the workflow artifact without
# blocking PRs while the suite is hardened. Production agent
# runtime images do not install `[dev]`, so `mutmut` never
# ships to AWS. Approval ticket: TODO before merge per the
# approved-deps registry process.
# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
# The version range matches the existing caret-style convention
# (`>=major.minor,<next-major`) used by every other entry in
# this section; exact-version pinning + lockfile generation is
# the M2.5 deliverable (§14.5).
"mutmut>=3.0,<4",
"pytest",
"pytest-cov",
"coverage[toml]",
"mypy",
"ruff",
"freezegun",
"hypothesis",
"watchfiles",
"import-linter",
"mutmut",
# `pytest-asyncio` powers the `async def test_*` discovery for the
# async migration. With `asyncio_mode = "auto"` (configured below) every
# async test function is auto-wrapped in an event loop without per-test
# decorators. Sync tests stay sync.
"pytest-asyncio",
]
# OpenTelemetry GenAI semantic conventions wiring.
# Kept as an optional extra so the unit-test + mypy jobs do NOT pull
# OpenTelemetry transitive deps; the OtelMeterMetricsRecorder consumes
# the SDK via its structural Protocol surface. AWS production picks
# this extra up via the same `[xray]` install path.
otel = [
"opentelemetry-api>=1.27.0,<2",
"opentelemetry-sdk>=1.27.0,<2",
"opentelemetry-api",
"opentelemetry-sdk",
]
# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
# `langchain-anthropic` powers the local profile's `ChatAnthropic` LLM factory.
# It is isolated as an optional extra so the default install (and the
# 100%-coverage unit-test job) does NOT pull
# Anthropic's transitive deps. Approval ticket: TODO before merge per the
# approved-deps registry process.
# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
# The version range here matches the existing convention used by `langchain-aws`
# and `langchain-core` in this file (caret-style >=major.minor,<next-major).
# Exact-version pinning + lockfile generation is M2.5 deliverable (§14.5).
anthropic = [
"langchain-anthropic>=0.3.0,<1",
"langchain-anthropic",
]
# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
# `aws-xray-sdk` and the OpenTelemetry OTLP exporter power the X-Ray tracer
# (the production-only AWS X-Ray exporter / segment-shape seam). They are
# isolated as an optional extra so the default install (and the
# 100%-coverage unit-test job) does NOT
# pull `aws-xray-sdk` or `opentelemetry-*` transitive deps. Approval ticket:
# TODO before merge per the approved-deps registry process.
# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
# The version ranges here match the existing convention used by
# `langchain-aws` / `langchain-core` / `langchain-anthropic` (caret-style
# >=major.minor,<next-major). Exact-version pinning + lockfile generation
# lands as a follow-up promotion deliverable.
xray = [
"aws-xray-sdk>=2.14.0,<3",
"opentelemetry-sdk>=1.26.0,<2",
"opentelemetry-exporter-otlp-proto-http>=1.26.0,<2",
ollama = [
"langchain-ollama",
]
# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
# `langchain-mcp-adapters` is the canonical bridge from the MCP `tools/list` /
# `tools/call` JSON-RPC surface to LangChain `StructuredTool`s consumed by
# `model.bind_tools(...)` and `langgraph.prebuilt.ToolNode`. It is isolated as
# an optional extra so the default install (and the
# 100%-coverage unit-test job) does NOT pull `langchain-mcp-adapters` transitive
# deps. Approval ticket: TODO before merge per the approved-deps registry process.
# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
# The version range here matches the existing convention used by `langchain-aws` /
# `langchain-core` / `langchain-anthropic` (caret-style >=major.minor,<next-major).
# Exact-version pinning + lockfile generation is a deliverable counterpart
# to the earlier lockfile entry.
mcp = [
"langchain-mcp-adapters>=0.1.0,<1",
xray = [
"aws-xray-sdk",
"opentelemetry-sdk",
"opentelemetry-exporter-otlp-proto-http",
]
# Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR).
# `langgraph-checkpoint-aws` provides the AgentCore-backed LangGraph
# checkpointer (`AgentCoreMemorySaver`) and the DynamoDB-backed
# `DynamoDbSaver` consumed by `_default_aws_checkpointer_factory` in
# `agent.composition._shared`. `bedrock-agentcore` is the SDK that
# exposes `AgentCoreMemorySaver` as a fallback when the dedicated
# integration package is missing; it already ships in the base
# `dependencies` block (the runtime entrypoint imports `BedrockAgentCoreApp`
# unconditionally) so this extra restates the version pin for parity
# with the operator-facing `pip install -e '.[aws]'` instruction in
# the lazy-import RuntimeError. The unit-test + 100%-coverage job
# does NOT install this extra, so mypy and pytest stay free of the
# AWS-only transitive deps. Approval ticket: TODO before merge per
# the approved-deps registry process.
# Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING).
# The version ranges match the existing caret-style convention used
# by every other extras block in this file
# (`>=major.minor,<next-major`). Exact-version pinning + lockfile
# generation is the M2.5 deliverable (§14.5).
aws = [
"langgraph-checkpoint-aws>=0.1.0,<1",
"bedrock-agentcore>=1.7.0,<2",
"langgraph-checkpoint-aws",
"bedrock-agentcore",
"PyJWT[crypto]",
]
debug = [
"langgraph-cli[inmem]",
"debugpy",
]
perf = [
"locust",
]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. pyproject.toml deps not pinned 📘 Rule violation ☼ Reliability

The updated Python dependency manifests list packages without exact versions, which makes builds
non-reproducible and can silently pull in breaking or vulnerable releases. This violates the
requirement to pin dependency versions (and keep dependency resolution deterministic).
Agent Prompt
## Issue description
`pyproject.toml` and `requirements.txt` now specify dependencies without exact versions (e.g., `"langgraph"` instead of `"langgraph==<exact>"`). This allows dependency drift across installs/builds and violates the dependency pinning compliance rule.

## Issue Context
The PR replaced previously version-ranged dependencies with unversioned package names, and `requirements.txt` installs `.[aws,xray,otel]` without any pins/hashes.

## Fix Focus Areas
- pyproject.toml[19-86]
- requirements.txt[1-25]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment thread Dockerfile
# pipeline can lift it verbatim.
# hadolint ignore=DL3007
FROM python:3.12.7-slim-bookworm AS builder
FROM public.ecr.aws/docker/library/python:3.13-slim-bookworm AS builder

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

2. Docker image tag floats 📘 Rule violation ☼ Reliability

The Dockerfile uses python:3.13-slim-bookworm without a patch version or digest, so builds can
change over time as the tag moves. This violates the requirement to pin dependency versions in
Docker images.
Agent Prompt
## Issue description
The runtime/builder base images are not pinned to an immutable version/digest.

## Issue Context
Compliance requires Docker base images be pinned to a specific version or digest; floating tags like `python:3.13-slim-bookworm` can change.

## Fix Focus Areas
- Dockerfile[41-41]
- Dockerfile[94-94]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines 1 to 14
terraform {
required_version = ">= 1.5.0"
required_version = ">= 1.13"

required_providers {
aws = {
source = "hashicorp/aws"
# Bedrock AgentCore Runtime + Memory resources require a recent provider.
# If your provider version is older than this, see the TODO at the top of
# main.tf for the documented null_resource fallback.
version = ">= 5.95.0"
# Bedrock AgentCore Runtime + Memory resources are first-class on
# the 6.x line. If your provider version is older than this, see
# the TODO at the top of main.tf for the documented null_resource
# fallback.
version = ">= 6.0"
}

null = {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

3. Terraform provider versions unpinned 📘 Rule violation ☼ Reliability

terraform/agentcore-runtime/versions.tf uses open-ended version constraints (>=) for Terraform
and providers, allowing upgrades without an explicit change review. This violates the dependency
pinning requirement for manifests.
Agent Prompt
## Issue description
Terraform `required_version` / `required_providers` constraints are specified as open-ended ranges (e.g., `>= 6.0`) rather than exact versions.

## Issue Context
The compliance rule requires exact, immutable dependency versions in manifests.

## Fix Focus Areas
- terraform/agentcore-runtime/versions.tf[1-17]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +195 to +212
…then re-run this script. If your checkout lives elsewhere, point
\`MCP_INTERNAL_REPO\` at it or pass \`--jar /abs/path/to/mcp-internal-X.Y.Z.jar\`.

See mcp/README.md "External JAR contract" for full details.
MISSING
exit 2
fi

# A Git-LFS pointer file is ~130 bytes; the real JAR is ~62 MB. Catch
# the common "developer forgot `git lfs pull`" footgun before docker
# wastes a layer on a useless 130-byte payload.
JAR_SIZE_BYTES="$(wc -c <"${JAR_PATH}" | tr -d ' ')"
# Spring Boot fat-JARs are typically 50–80 MB; anything dramatically
# smaller is almost certainly a `-plain.jar` mistakenly passed in or a
# truncated download. 1 MB is a safe floor.
JAR_SIZE_BYTES="$(wc -c <"${JAR_SOURCE}" | tr -d ' ')"
if [[ "${JAR_SIZE_BYTES}" -lt 1048576 ]]; then
cat >&2 <<LFS_POINTER
ERROR: ${JAR_PATH} is only ${JAR_SIZE_BYTES} bytes — looks like a Git LFS
pointer file rather than the real JAR. Run:

git lfs install
git lfs pull

and re-run this script.
LFS_POINTER
cat >&2 <<TOO_SMALL
ERROR: ${JAR_SOURCE} is only ${JAR_SIZE_BYTES} bytes — that's too small to
be a Spring Boot fat-JAR (expected ~50–80 MB). Make sure you are
pointing at the bootJar output, not the lightweight \`-plain.jar\`
Gradle also emits.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

4. Non-ascii strings in build_mcp_image.sh 📘 Rule violation ✧ Quality

Multiple modified user-facing/help text, docstrings, and committed JSON metadata/schema strings
include non-ASCII characters (e.g., , é, , ). This violates the English-only ASCII
string-literal policy and can break or degrade tooling/rendering/search behavior across environments
that assumes ASCII.
Agent Prompt
## Issue description
Modified string literals and JSON string values include non-ASCII characters (e.g., ellipsis `…`, accented characters like `é`, and Unicode dashes `—`/`–`), which violates the ASCII-only policy for committed code/config and user-facing text.

## Issue Context
PR Compliance ID 576985 requires all modified string literals/values (including Python docstrings, shell-script help/error output, and committed JSON configuration/metadata/schema) to be ASCII-only unless explicitly exempted. Non-ASCII characters can also cause inconsistent rendering/searching and can break tooling that assumes ASCII logs.

## Fix Focus Areas
- scripts/build_mcp_image.sh[195-212]
- scripts/agentcore_deploy_wrapper.py[2-33]
- src/agent/infrastructure/cost/model_prices.json[2-14]
- src/agent/infrastructure/structured_log_schema.json[215-235]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +70 to +72
def _truncate_for_log(value: str, limit: int = _MAX_LOG_BODY_CHARS) -> str:
del limit
return value

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

5. Unbounded webhook payload logging 🐞 Bug ⛨ Security

lambda/webhook_validator/handler.py’s _truncate_for_log() is a no-op, so the Lambda logs full
webhook bodies and full SQS envelope JSON at INFO/ERROR. This can leak sensitive/PII data into logs
and can explode log volume/cost or exceed log/annotation limits under large Jira payloads.
Agent Prompt
## Issue description
`_truncate_for_log()` currently ignores its `limit` parameter and returns the full payload string. Because it is used when logging the inbound request body and the SQS envelope, the Lambda can emit unbounded sensitive payloads into logs.

## Issue Context
This Lambda is the public ingress trust boundary; its logs are likely retained/centralized. Logging full Jira webhook JSON commonly includes user emails, ticket content, and other sensitive data.

## Fix Focus Areas
- Implement truncation (with an ellipsis) in `_truncate_for_log` using `limit`.
- Consider also switching body/envelope logging to DEBUG or logging only sizes + correlation_id (keeping INFO logs non-sensitive).

### Fix Focus Areas (code refs)
- lambda/webhook_validator/handler.py[70-72]
- lambda/webhook_validator/handler.py[226-240]
- lambda/webhook_validator/handler.py[274-296]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

Comment on lines +159 to +168
if chat_bedrock_cls is not None:
return chat_bedrock_cls
if _direct_invoke_disabled_via_env():
return _make_direct_invoke_cls(_default_bedrock_runtime_client)
# Meta + DeepSeek keep a dedicated adapter path because their
# structured-output behavior differs from Anthropic/Amazon's
# native ChatBedrock path.
if bedrock_model_family in ("meta", "deepseek"):
return _make_direct_invoke_cls(_default_bedrock_runtime_client)
return _import_chat_bedrock()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

6. Deepseek/meta tool calls broken 🐞 Bug ≡ Correctness

bedrock_factory._select_chat_model_ctor() forces the direct-invoke Bedrock adapter for the
deepseek/meta families (and also falls back to it on langchain_aws import failure), but that
adapter ignores bound tools and always returns AIMessage(..., tool_calls=[]). The graph routes to
terminal nodes when tool_calls is empty, so MCP tool execution is skipped and the documented
DeepSeek default cannot perform the agentic tool loop.
Agent Prompt
## Issue description
DeepSeek/meta are treated as supported Bedrock model families, but the selected direct Bedrock `converse` adapter discards tools and never produces tool calls. This breaks the agent’s core tool loop (MCP reads/writes), because graph routing depends on `tool_calls` being populated.

## Issue Context
- `_select_chat_model_ctor()` routes `deepseek`/`meta` to `BedrockDirectInvokeChatModel`.
- `BedrockDirectInvokeChatModel.bind_tools()` deletes the tools argument.
- The runnable returns `AIMessage(..., tool_calls=[])` unconditionally.
- The agent graph routes to tools execution only when tool calls exist.
- Docs describe DeepSeek V3.2 as the default model family for the runtime and explicitly call out validating the “tool-use surface” end-to-end.

## Fix Focus Areas
Choose one (or both) of:
1) **Implement real tool-calling for direct `converse`**: pass Bedrock `toolConfig` derived from the bound tools and parse `toolUse` blocks from `converse` output into `AIMessage.tool_calls`.
2) **Fail closed**: if tools are bound (non-empty), raise a clear exception so the worker/runtime fails fast instead of silently skipping tool execution; also reconsider forcing direct-invoke for `deepseek`/`meta` until tool calling is implemented.

### Fix Focus Areas (code refs)
- src/agent/infrastructure/llm/bedrock_factory.py[159-168]
- src/agent/infrastructure/llm/bedrock_converse_direct.py[111-125]
- src/agent/infrastructure/llm/bedrock_converse_direct.py[226-234]
- src/agent/graph/nodes/_agentic_turn.py[193-246]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

- Simplified `.env.example` and `.env.local.example` by removing outdated comments and variables related to Bedrock model IDs and logging configurations, enhancing clarity for developers.
- Updated the `deploy/local/bootstrap.sh` script to reflect changes in service management, specifically rolling the `agent-worker` instead of the deprecated `agent-runtime`, and adjusted port-forwarding instructions accordingly.
- Modified the `scrum_dev_manual.json` fixture to update correlation IDs and issue keys, ensuring consistency with current testing scenarios.

These changes aim to improve the developer experience by clarifying environment configurations and updating local deployment practices.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant