Python aws agent core - #2
Conversation
…schema validation - Added `WEBHOOK_HMAC_SECRET_ARN` to `.bedrock_agentcore.yaml` to enforce webhook security. - Updated `Makefile` to include a new `seed-schemas-check` target for validating DynamoDB schemas against adapter contracts. - Enhanced `pyproject.toml` to include structured log schema in package data. - Expanded CI workflow to include a `seed-schema-contract` job for asserting schema compliance in both static and live phases. - Improved documentation in `ARCHITECTURE.md` to clarify multi-tenant handling and thread ID generation. - Introduced new functions in the application layer to support tenant resolution and deduplication logic for webhook events.
- Added new import linter contracts in `pyproject.toml` to enforce module independence and prevent cross-layer dependencies. - Updated `main.py` to support resolving `MCP_BEARER_TOKEN_ARN` and `WEBHOOK_HMAC_SECRET_ARN` for improved security in AWS deployments. - Introduced a fallback `_SystemClock` class in `webhook_handler.py` for testing purposes. - Enhanced `build_aws_dependencies` to utilize Secrets Manager for resolving sensitive environment variables. - Updated S3 payload storage to include SHA-256 hashes in object keys for better traceability. - Improved documentation in `ARCHITECTURE.md` to reflect changes in the agent's architecture and flow. - Added tests to validate the new Secrets Manager integration and ensure proper handling of runtime secrets.
- Updated `.env.local.example` to include support for the `ollama` LLM provider, allowing local execution without an Anthropic API key. - Modified `pyproject.toml` to add `langchain-ollama` as an optional dependency for local LLM functionality. - Enhanced `README.md` to clarify the usage of `LLM_PROVIDER` and the new local LLM options. - Updated `main.py` to validate the `LLM_PROVIDER` and `DESIGN_LLM_PROVIDER` environment variables, ensuring only supported values are accepted. - Improved local dependency resolution in `build_local_dependencies` to accommodate the new LLM provider options. - Added tests for the new local LLM functionality and updated related documentation to reflect these changes.
- Updated `.env.local.example` to reflect the transition from `GovernedAssessmentLanguageModel` to `GovernedAgenticChatModel`. - Modified `README.md` to clarify the new token governance structure and the role of `GovernedAgenticChatModel`. - Enhanced `pyproject.toml` to include new paths for testing related to the graph nodes. - Improved documentation in various files to ensure consistency with the new model and its usage. - Added validation for model pricing in CI workflows to ensure accurate cost tracking. These changes aim to streamline the token governance process and improve clarity in the documentation for developers and users.
- Modified the smoke test workflow to project the `AWS_AGENTCORE_SANDBOX_ROLE_ARN` through the job-level `env` block, addressing limitations with the `secrets` context in step-level conditions. - Updated the conditional check for the AWS credentials step to use the new environment variable instead of directly referencing secrets, ensuring compatibility with GitHub Actions parsing. These changes enhance the security and reliability of the AWS integration in the CI workflow.
- Deleted `grade.md`, which contained detailed architecture and implementation grading for the `aws-agent-core` project. - This removal is part of a project restructuring to streamline documentation and focus on essential resources. These changes aim to simplify the documentation landscape and enhance clarity for developers.
- Added new `langgraph-dev` target to the Makefile for serving the compiled graph at `http://127.0.0.1:2024`, facilitating local debugging with the LangGraph CLI. - Introduced `graph-png` target to render the compiled topology to `docs/graph.png`, useful for design reviews. - Updated `README.md` and `docs/PYTHON_DEVELOPMENT.md` to include instructions for using the LangGraph dev server and generating graph snapshots. - Enhanced `pyproject.toml` to include `langgraph-cli[inmem]` as a debug dependency, ensuring proper setup for local development. These changes aim to improve the developer experience by providing tools for graph-level debugging and visualization.
- Updated `.env.local.example` to provide clearer descriptions for the `OBSERVABILITY_BACKEND` options and added the `OTEL_EXPORTER_OTLP_ENDPOINT` variable for OTLP/HTTP exporter configuration. - Enhanced `Makefile` by adding a new `soak` target for running weekly LocalStack-driven burst-rate tests, including detailed prerequisites and usage instructions. - Modified `pyproject.toml` to include `locust` as a development-only dependency for the new soak tests. - Improved CI workflow to handle release tags and persist the CycloneDX SBOM, ensuring accurate tracking of dependencies during releases. - Expanded documentation in `ARCHITECTURE.md` and `runbook.md` to clarify the single-iteration-per-webhook design and provide guidance on managing bot-account interactions in Jira. These changes aim to improve observability, testing capabilities, and documentation clarity for developers.
…lity - Removed `grade.md` to streamline documentation and focus on essential resources. - Updated `README.md` to clarify the usage of `MCP_BEARER_TOKEN` and its integration with `HttpStreamableMcpClient`. - Modified CI workflow to simplify schema contract checks by removing the `--skip-missing-modules` option. - Enhanced `runbook.md` to reflect changes in DLQ handling, replacing `redrive_dlq.sh` with `replay_dlq.py`. - Updated structured log schema by removing the legacy `tenant` field for better clarity. - Refined various scripts to ensure consistent naming conventions for tenant identifiers. These changes aim to enhance documentation clarity, improve script functionality, and ensure consistency across the codebase.
…ment - Introduced a HITL approval mechanism that requires human review before transitioning from the assessor to the designer phase. - Updated `README.md` to document the new HITL process, including the two-webhook protocol for human approval. - Enhanced `ARCHITECTURE.md` to detail the integration of the HITL approval gate within the graph topology. - Expanded `runbook.md` with troubleshooting steps for common HITL issues and recovery procedures. - Modified structured log schema to include new HITL-related events and readiness phases. - Updated various components to support the new approval LLM, ensuring it integrates seamlessly with existing workflows. These changes aim to improve governance and accountability in the readiness assessment process by incorporating human oversight.
- Added `httpx[http2]` as a direct dependency in `pyproject.toml` to ensure the required `h2` package is available for HTTP/2 support, preventing runtime errors during Pod startup. - Updated the Jaeger image version in `jaeger.yaml` to `1.62.0` for improved stability and features. - Revised `README.md` to clarify the build and deployment steps, including new convenience scripts for automated setup and teardown of the local environment. These changes aim to enhance the reliability of the local development environment and improve documentation clarity for users.
…ality - Removed the `tunnel-smee` target from the Makefile, as it was documentation-only and not intended for execution. - Updated the `README.md` to reflect the removal of `tunnel-smee` and clarified the usage of `curl` for testing webhook deliveries. - Revised `deploy/local/README.md` to emphasize that real Atlassian webhook deliveries should target the deployed runtime endpoint, not the local cluster. - Enhanced `docs/PYTHON_DEVELOPMENT.md` to replace references to `smee.io` and `ngrok` with a focus on curl-driven smoke tests. These changes aim to streamline the documentation and clarify the intended usage of local testing tools.
also working with local testing some cleanup still but working
- Updated `.env.local.example` to include new environment variables for Git over SSH and approval LLM configurations, ensuring proper setup for local deployments. - Revised `pyproject.toml` to streamline dependency management by removing version constraints for several packages, simplifying the installation process. - Improved `README.md` to clarify the usage of new environment variables and their integration within the local development workflow. - Enhanced `deploy/local/bootstrap.sh` and `teardown.sh` scripts to support multiple persistent port-forwards, improving developer accessibility to running services. - Updated `deploy/local/configmap.yaml` and `secrets.example.yaml` to reflect changes in secret management and configuration for the mcp-internal service. These changes aim to improve the local development experience and ensure consistency across configuration files.
…iguration - Updated `.env.local.example` to include new environment variables for async dispatch, including `WEBHOOK_WORK_QUEUE_URL` and `WEBHOOK_ASYNC_DISPATCH`, facilitating the separation of synchronous and asynchronous processing. - Enhanced `Makefile` to create the `agent-work` SQS queue and configure its redrive policy, ensuring reliable message handling between the webhook entrypoint and the agent-worker Pod. - Revised `README.md` to document the new async dispatch architecture, clarifying the roles of the `agent-runtime` and `agent-worker` Pods in processing webhook events. - Updated `deploy/local/configmap.yaml` to reflect changes in logging configuration and workspace paths, ensuring consistency across local deployments. - Enhanced `ARCHITECTURE.md` and `runbook.md` to detail the async dispatch flow and provide troubleshooting guidance for potential queue health issues. These changes aim to improve the local development experience and ensure robust handling of webhook events through asynchronous processing.
- Removed the `idempotency` table and associated logic from the Makefile, scripts, and application code to streamline the architecture. - Updated `README.md`, `ARCHITECTURE.md`, and other documentation to reflect the removal of idempotency-related components and clarify the current state of the application. - Adjusted `pyproject.toml` and CI workflows to remove references to the deleted `idempotency` module, ensuring consistency across the codebase. These changes aim to simplify the codebase and improve clarity in the documentation regarding the current architecture.
- Added new structured log events for design post comment fallback scenarios, including `fallback_failed`, `fallback_posted`, and `tool_missing`, to improve observability. - Implemented detailed debug logging in the `webhook_handler`, `tools_node`, and `design_post_comment_node` functions to capture error details and tool invocation information. - Updated the structured log schema to include new event identifiers, ensuring comprehensive tracking of design-related operations and error handling. These changes aim to enhance the robustness of the logging framework and provide better insights into the processing flow and error conditions.
…ing in webhook and graph processing - Changed correlation IDs from `scrum-524` to `scrum-525` in `scrum_dev_manual.json` to reflect the latest issue updates. - Removed extensive debug logging sections from `webhook_handler.py`, `compile.py`, and `consumer.py` to streamline the code and improve readability, while maintaining essential error handling functionality. These changes aim to enhance the clarity of the codebase and ensure it aligns with the latest issue tracking.
- Introduced a new `persist_approval_granted` checkpoint node to ensure durable state persistence when human approval is granted, allowing the designer subgraph to resume directly from the `awaiting_design` phase after a crash. - Updated graph routing logic to handle transitions between approval and design phases more effectively, ensuring that the system can recover gracefully from interruptions. - Enhanced structured logging to include the new checkpoint, improving observability of the approval process. - Refactored related tests to validate the new routing behavior and state persistence functionality. These changes aim to improve the robustness of the approval process and ensure seamless transitions in the graph workflow.
…st comment functionality - Updated the `langchain-mcp-adapters` dependency in `pyproject.toml` to pin it to the 0.1.x line, ensuring compatibility with the current stack while addressing import path issues with `langchain-core`. - Added new tests for the design post comment functionality, including scenarios for fallback behavior when the LLM has already posted a comment and when design tools are missing. - Improved the `_find_named_tool` function to ensure it correctly skips non-matching tool names, enhancing the robustness of tool resolution in the graph processing. These changes aim to improve dependency management and ensure comprehensive testing of the design post comment logic.
…k processing - Added `JIRA_SITE_URL` to `.env.local.example` for Atlassian Cloud integration, clarifying its usage in the MCP internal JAR and Kubernetes Secret. - Updated `OTEL_EXPORTER_OTLP_ENDPOINT` in `.env.local.example` to reflect the new endpoint for OTLP traces. - Enhanced `README.md` with details on webhook-layer idempotency and the absence of a dedicated dedupe mechanism, emphasizing the design choice for handling duplicate deliveries. - Added a security caveat in `deploy/local/README.md` regarding the use of a local-only HMAC secret for development, stressing the importance of using secure secrets in production environments. - Revised `docs/runbook.md` to include a section on rate limiting and back-pressure ownership, outlining the architectural decisions for managing webhook traffic and processing. These changes aim to improve the clarity and security of the configuration while providing comprehensive guidance on webhook processing and operational considerations.
…point, and composition root enforcement - Deleted ADR 0001, 0002, and 0003 documents as they are no longer relevant to the current architecture and design decisions. - This cleanup reflects the evolution of the system's architecture and the removal of previously established concepts that are no longer in use. These changes aim to maintain an accurate and up-to-date documentation set that aligns with the current state of the codebase.
- Updated the Dockerfile to use `python:3.13-slim-bookworm` for both builder and runtime stages. - Changed the required Python version in `langgraph.json` and `pyproject.toml` to reflect the new baseline. - Adjusted CI workflows to include Python 3.13 in the testing matrix, ensuring compatibility with the latest version. - Enhanced documentation to indicate the support for Python 3.13 and updated related configurations accordingly. These changes aim to modernize the codebase and ensure alignment with the latest Python features and improvements.
- Replaced the legacy agent runtime configuration in `.bedrock_agentcore.yaml` with a new structure for defining agents, including `create_agent` with detailed AWS settings. - Updated `.env.example` and `.env.local.example` to clarify Bedrock model selection and removed deprecated environment variables related to HMAC verification. - Enhanced the `Makefile` to reflect changes in the agent architecture, including updates to local logging and SQS processing. - Removed obsolete network policy files and local deployment configurations that are no longer applicable, streamlining the deployment process. - Improved documentation in `README.md` to provide clearer guidance on the new async dispatch architecture and local development practices. These changes aim to modernize the agent configuration, improve clarity in local development, and ensure alignment with the latest architectural decisions.
- Changed the default agent in `.bedrock_agentcore.yaml` to `jira_readiness_agent` and added its configuration with detailed AWS settings. - Updated environment files `.env.example` and `.env.local.example` to reflect the new Bedrock model ID for Amazon Nova, ensuring clarity in model selection. - Enhanced the `aws-deploy-plan.md` to include the new Amazon Nova model in the supported families. - Updated various Terraform files to set the default Bedrock model ID to `us.amazon.nova-pro-v1:0`, aligning with the new configuration. - Improved test coverage for the new model family and updated related tests to ensure compatibility with the latest changes. These changes aim to modernize the agent configuration, improve clarity in model selection, and ensure alignment with the latest AWS Bedrock offerings. Next node still not working all the way still need to find out why
- Updated the webhook validator to support multiple signature headers, prioritizing a configured header and falling back to legacy options for compatibility. - Improved logging to capture the presence of signature headers and validation attempts, aiding in debugging and observability. - Added tests to ensure acceptance of the `x-hub-signature` header by default and validate the new retry logic for structured assessments in the agent's terminal assess node. - Updated structured log schema to include new validation-related events, enhancing traceability in the logging framework. These changes aim to improve the robustness of webhook processing and enhance the clarity of signature validation mechanisms. sonnet set but waiting authorization
- Enforced a hard pin on the AWS region to `us-east-1` across various scripts and documentation, ensuring consistency in deployment and execution. - Updated the default Bedrock model ID in multiple files to `us.amazon.nova-pro-v1:0`, reflecting the latest model selection. - Enhanced documentation in `.env.local.example` and `aws-deploy-plan.md` to clarify the new region policy and model usage. - Improved compatibility in the agent's codebase by adjusting references to the Bedrock model family in tests and application logic. These changes aim to streamline the deployment process and ensure alignment with the latest AWS configurations and model offerings.
- Changed the entrypoint for `create_agent` and `jira_readiness_agent` in `.bedrock_agentcore.yaml` from `src/agent/main.py` to `src/agent/server.py` to align with the new architecture. - Updated network configuration to use VPC settings, including specific security groups and subnets, enhancing the deployment's security posture. - Revised agent IDs and memory IDs in the configuration to reflect the latest identifiers, ensuring consistency across the deployment. - Added clarifications in `.dockerignore` regarding the limitations of the `agentcore deploy` command and its handling of ignored files. - Enhanced `.env.example` and `.env.local.example` with additional comments for optional configurations, improving developer guidance. These changes aim to modernize the agent configuration, improve security settings, and enhance clarity in the development environment. aws deployed and working but no tools called
- Removed the tracking of the Spring Boot fat-JAR (`mcp-internal-*.jar`) from the repository, transitioning to an external resolution process during the build. The JAR is now resolved from the upstream `mcp-internal` repository at build time, improving repository cleanliness and reducing upload size. - Updated `.dockerignore`, `.gitignore`, and `.gitattributes` to reflect the new handling of the JAR and prevent accidental inclusion in commits. - Enhanced documentation in `mcp/README.md`, `aws-deploy-plan.md`, and various scripts to clarify the new external JAR contract and build process. - Adjusted CI workflows to download the JAR from a GitHub Release, ensuring the build process remains seamless and reliable. These changes aim to streamline the build process, improve clarity in documentation, and maintain a clean repository structure.
- Added a new method `_normalise_system_messages` to collapse multiple non-consecutive `SystemMessage` instances into a single leading message, ensuring compliance with Bedrock's Converse contract. - Updated the invocation logic in `_BoundChatRunnable` to utilize the normalization method before processing structured outputs. - Enhanced tests to verify the correct behavior of the normalization process across various scenarios, ensuring that system messages are handled appropriately in both standard and structured output contexts. These changes aim to improve message handling in the Bedrock LLM, preventing errors related to non-consecutive system messages and enhancing overall robustness.
CI Feedback 🧐A test triggered by this PR failed. Here is an AI-generated analysis of the failure:
|
Review Summary by QodoPython AWS Agent Core: End-to-end hardening for Bedrock AgentCore + EKS with HITL approval and multi-tenant support
WalkthroughsDescription• **End-to-end hardening of aws-agent-core runtime** for AWS Bedrock AgentCore + EKS deployment while preserving local Kubernetes profile • **New server.py entrypoint and VPC-scoped AgentCore wiring** replace legacy main.py boot path; composition root refactored with AWS/local profile strategies • **Bedrock adapter upgraded to us.amazon.nova-pro-v1:0** on hard-pinned us-east-1 region with Converse-compliant system-message normalization • **MCP-internal Spring Boot JAR removed from repo** and resolved from GitHub Release at build time via scripts/build_mcp_image.sh • **Graph enhancements**: HITL approval checkpoint, idempotent async dispatch via SQS work publisher, multi-tenant resolution, token governance, and richer observability • **Composition refactoring**: Shared _BaseProfileDependencies mixin, LLM governance helper, tracer registry-driven dispatcher, boot-time DynamoDB schema validation • **Observability improvements**: OTLP exporter refactored with lazy enum loading, span ID masking for OTLP/proto-http compliance, X-Ray-compatible trace ID generation • **Webhook handler split** into synchronous prevalidate gate and asynchronous process execution paths with tenant binding and payload archival • **Designer and approval nodes** refactored with AgenticTurnSpec abstraction, deterministic fallback paths, and structured debug event logging • **Comprehensive test coverage**: New test suites for AWS composition internals, SQS worker composition, designer/approval nodes, tools node observability, and dependency resolution • **Breaking changes**: .bedrock_agentcore.yaml entrypoint moves from src/agent/main.py → src/agent/server.py; default Bedrock model becomes us.amazon.nova-pro-v1:0; AWS region hard-pinned to us-east-1; mcp-internal-*.jar no longer tracked in git • **Engineering quality gates met**: mypy --strict green, SOLID+IoC via Protocol contracts in agent.contracts, 100% coverage gate, ruff check green, no secrets committed • **Local + Terraform parity**: All AWS-only behavior has 1:1 local and Terraform counterpart; secrets documented in .env.local.example and deploy/local/configmap.yaml Diagramflowchart LR
A["Legacy main.py<br/>single-tenant<br/>local-only"] -->|"refactor to<br/>profile strategy"| B["New server.py<br/>+ composition root"]
B -->|"AWS profile"| C["AgentCore Runtime<br/>Invoker + SQS<br/>Work Publisher"]
B -->|"Local profile"| D["LangGraph<br/>+ MCP Session"]
C -->|"HITL gate"| E["Approval Node<br/>+ Designer Node"]
D -->|"HITL gate"| E
E -->|"token governance<br/>+ observability"| F["Enhanced Graph<br/>with Tracer Registry<br/>+ OTLP Exporter"]
G["Webhook Handler<br/>split: prevalidate<br/>+ async process"] -->|"tenant resolve<br/>+ payload sink"| C
G -->|"tenant resolve<br/>+ payload sink"| D
File Changes1. src/agent/composition/aws.py
|
Code Review by Qodo
1. pyproject.toml deps not pinned
|
| dependencies = [ | ||
| "bedrock-agentcore>=1.7.0,<2", | ||
| "langgraph>=0.2.0,<2", | ||
| "langchain-aws>=0.2.0,<1", | ||
| # UsageMetadataCallbackHandler is the load-bearing token-counting seam | ||
| # (added in langchain-core 0.3.49). | ||
| "langchain-core>=0.3.49,<1", | ||
| "pydantic>=2.7.0,<3", | ||
| # tenacity drives the half-open probe scheduling in | ||
| # InMemoryLlmCircuitBreaker (wait_random_exponential). | ||
| "tenacity>=9.0.0,<10", | ||
| "bedrock-agentcore", | ||
| "langgraph", | ||
| "langchain-aws", | ||
| "langchain-core", | ||
| "pydantic", | ||
| "tenacity", | ||
| # Official Model Context Protocol Python SDK. Owns the Streamable HTTP | ||
| # transport, JSON-RPC framing, session-id propagation, and protocol-version | ||
| # negotiation. Imported only from `agent.composition._mcp_session` and | ||
| # `agent.infrastructure.mcp.*`; application and graph layers see only | ||
| # the abstract Protocol from `agent.contracts.mcp`. | ||
| "mcp", | ||
| # Upstream MCP -> LangChain `StructuredTool` bridge. Used directly with | ||
| # zero hand-rolled wrapping; per-call observability is layered on via | ||
| # the upstream `ToolCallInterceptor` Protocol implementation in | ||
| # `agent.infrastructure.mcp.observability_interceptor`. The 0.2.x | ||
| # series targets `langchain-core>=1.0`, which the rest of this | ||
| # project's stack (`langchain-aws>=1.x`, `langgraph>=1.x`) now also | ||
| # uses, so no version cap is required. | ||
| "langchain-mcp-adapters", | ||
| ] | ||
|
|
||
| [project.optional-dependencies] | ||
| dev = [ | ||
| "pytest>=8.0.0", | ||
| "pytest-cov>=5.0.0", | ||
| "coverage[toml]>=7.5.0", | ||
| "mypy>=1.10.0", | ||
| "ruff>=0.5.0", | ||
| "freezegun>=1.5.0", | ||
| # Property-based tests on TokenUsage invariants and the budget arithmetic. | ||
| "hypothesis>=6.100.0,<7", | ||
| # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR). | ||
| # `watchfiles` powers the in-Pod hot-reload loop wired by | ||
| # `deploy/local/overlays/host-mount/agent-runtime-patch.yaml` | ||
| # (`make local-up-watch`). It is dev-only — production agent runtime | ||
| # images MUST NOT install the [dev] extra, so the inotify/native | ||
| # filesystem-watcher dependency never ships to AWS. Approval ticket: | ||
| # TODO before merge per the approved-deps registry process. | ||
| # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING). | ||
| # The version range matches the existing convention used by every | ||
| # other [dev] / optional-extras entry in this file (caret-style | ||
| # >=major.minor,<next-major). Exact-version pinning + lockfile | ||
| # generation is the M2.5 deliverable (§14.5). | ||
| "watchfiles>=0.21.0,<1", | ||
| # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR). | ||
| # `import-linter` enforces the Wave 4 sealed-test-seams contract | ||
| # under `[tool.importlinter]` below: production code in the | ||
| # `agent` package must not import from | ||
| # `agent.composition._test_seams`. The check runs as a | ||
| # `lint-imports` step in the same CI lane as `ruff` / | ||
| # `mypy` so a regression surfaces at PR-review time. It is | ||
| # dev-only — production agent runtime images do not install | ||
| # `[dev]`, so the linter never ships to AWS. Approval ticket: | ||
| # TODO before merge per the approved-deps registry process. | ||
| # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING). | ||
| # The version range matches the existing convention used by | ||
| # every other `[dev]` / optional-extras entry in this file | ||
| # (caret-style `>=major.minor,<next-major`). | ||
| "import-linter>=2.0,<3", | ||
| # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR). | ||
| # `mutmut` powers the Wave 9 weekly mutation-testing workflow | ||
| # (`.github/workflows/mutation.yml`). It is dev-only and runs | ||
| # on a separate scheduled job (advisory, not gating) so a | ||
| # surviving mutant surfaces in the workflow artifact without | ||
| # blocking PRs while the suite is hardened. Production agent | ||
| # runtime images do not install `[dev]`, so `mutmut` never | ||
| # ships to AWS. Approval ticket: TODO before merge per the | ||
| # approved-deps registry process. | ||
| # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING). | ||
| # The version range matches the existing caret-style convention | ||
| # (`>=major.minor,<next-major`) used by every other entry in | ||
| # this section; exact-version pinning + lockfile generation is | ||
| # the M2.5 deliverable (§14.5). | ||
| "mutmut>=3.0,<4", | ||
| "pytest", | ||
| "pytest-cov", | ||
| "coverage[toml]", | ||
| "mypy", | ||
| "ruff", | ||
| "freezegun", | ||
| "hypothesis", | ||
| "watchfiles", | ||
| "import-linter", | ||
| "mutmut", | ||
| # `pytest-asyncio` powers the `async def test_*` discovery for the | ||
| # async migration. With `asyncio_mode = "auto"` (configured below) every | ||
| # async test function is auto-wrapped in an event loop without per-test | ||
| # decorators. Sync tests stay sync. | ||
| "pytest-asyncio", | ||
| ] | ||
| # OpenTelemetry GenAI semantic conventions wiring. | ||
| # Kept as an optional extra so the unit-test + mypy jobs do NOT pull | ||
| # OpenTelemetry transitive deps; the OtelMeterMetricsRecorder consumes | ||
| # the SDK via its structural Protocol surface. AWS production picks | ||
| # this extra up via the same `[xray]` install path. | ||
| otel = [ | ||
| "opentelemetry-api>=1.27.0,<2", | ||
| "opentelemetry-sdk>=1.27.0,<2", | ||
| "opentelemetry-api", | ||
| "opentelemetry-sdk", | ||
| ] | ||
| # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR). | ||
| # `langchain-anthropic` powers the local profile's `ChatAnthropic` LLM factory. | ||
| # It is isolated as an optional extra so the default install (and the | ||
| # 100%-coverage unit-test job) does NOT pull | ||
| # Anthropic's transitive deps. Approval ticket: TODO before merge per the | ||
| # approved-deps registry process. | ||
| # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING). | ||
| # The version range here matches the existing convention used by `langchain-aws` | ||
| # and `langchain-core` in this file (caret-style >=major.minor,<next-major). | ||
| # Exact-version pinning + lockfile generation is M2.5 deliverable (§14.5). | ||
| anthropic = [ | ||
| "langchain-anthropic>=0.3.0,<1", | ||
| "langchain-anthropic", | ||
| ] | ||
| # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR). | ||
| # `aws-xray-sdk` and the OpenTelemetry OTLP exporter power the X-Ray tracer | ||
| # (the production-only AWS X-Ray exporter / segment-shape seam). They are | ||
| # isolated as an optional extra so the default install (and the | ||
| # 100%-coverage unit-test job) does NOT | ||
| # pull `aws-xray-sdk` or `opentelemetry-*` transitive deps. Approval ticket: | ||
| # TODO before merge per the approved-deps registry process. | ||
| # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING). | ||
| # The version ranges here match the existing convention used by | ||
| # `langchain-aws` / `langchain-core` / `langchain-anthropic` (caret-style | ||
| # >=major.minor,<next-major). Exact-version pinning + lockfile generation | ||
| # lands as a follow-up promotion deliverable. | ||
| xray = [ | ||
| "aws-xray-sdk>=2.14.0,<3", | ||
| "opentelemetry-sdk>=1.26.0,<2", | ||
| "opentelemetry-exporter-otlp-proto-http>=1.26.0,<2", | ||
| ollama = [ | ||
| "langchain-ollama", | ||
| ] | ||
| # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR). | ||
| # `langchain-mcp-adapters` is the canonical bridge from the MCP `tools/list` / | ||
| # `tools/call` JSON-RPC surface to LangChain `StructuredTool`s consumed by | ||
| # `model.bind_tools(...)` and `langgraph.prebuilt.ToolNode`. It is isolated as | ||
| # an optional extra so the default install (and the | ||
| # 100%-coverage unit-test job) does NOT pull `langchain-mcp-adapters` transitive | ||
| # deps. Approval ticket: TODO before merge per the approved-deps registry process. | ||
| # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING). | ||
| # The version range here matches the existing convention used by `langchain-aws` / | ||
| # `langchain-core` / `langchain-anthropic` (caret-style >=major.minor,<next-major). | ||
| # Exact-version pinning + lockfile generation is a deliverable counterpart | ||
| # to the earlier lockfile entry. | ||
| mcp = [ | ||
| "langchain-mcp-adapters>=0.1.0,<1", | ||
| xray = [ | ||
| "aws-xray-sdk", | ||
| "opentelemetry-sdk", | ||
| "opentelemetry-exporter-otlp-proto-http", | ||
| ] | ||
| # Following Qodo rule: Only add dependencies from the approved third-party registry (ERROR). | ||
| # `langgraph-checkpoint-aws` provides the AgentCore-backed LangGraph | ||
| # checkpointer (`AgentCoreMemorySaver`) and the DynamoDB-backed | ||
| # `DynamoDbSaver` consumed by `_default_aws_checkpointer_factory` in | ||
| # `agent.composition._shared`. `bedrock-agentcore` is the SDK that | ||
| # exposes `AgentCoreMemorySaver` as a fallback when the dedicated | ||
| # integration package is missing; it already ships in the base | ||
| # `dependencies` block (the runtime entrypoint imports `BedrockAgentCoreApp` | ||
| # unconditionally) so this extra restates the version pin for parity | ||
| # with the operator-facing `pip install -e '.[aws]'` instruction in | ||
| # the lazy-import RuntimeError. The unit-test + 100%-coverage job | ||
| # does NOT install this extra, so mypy and pytest stay free of the | ||
| # AWS-only transitive deps. Approval ticket: TODO before merge per | ||
| # the approved-deps registry process. | ||
| # Following Qodo rule: Pin dependency versions in manifests and lockfiles (WARNING). | ||
| # The version ranges match the existing caret-style convention used | ||
| # by every other extras block in this file | ||
| # (`>=major.minor,<next-major`). Exact-version pinning + lockfile | ||
| # generation is the M2.5 deliverable (§14.5). | ||
| aws = [ | ||
| "langgraph-checkpoint-aws>=0.1.0,<1", | ||
| "bedrock-agentcore>=1.7.0,<2", | ||
| "langgraph-checkpoint-aws", | ||
| "bedrock-agentcore", | ||
| "PyJWT[crypto]", | ||
| ] | ||
| debug = [ | ||
| "langgraph-cli[inmem]", | ||
| "debugpy", | ||
| ] | ||
| perf = [ | ||
| "locust", | ||
| ] |
There was a problem hiding this comment.
1. pyproject.toml deps not pinned 📘 Rule violation ☼ Reliability
The updated Python dependency manifests list packages without exact versions, which makes builds non-reproducible and can silently pull in breaking or vulnerable releases. This violates the requirement to pin dependency versions (and keep dependency resolution deterministic).
Agent Prompt
## Issue description
`pyproject.toml` and `requirements.txt` now specify dependencies without exact versions (e.g., `"langgraph"` instead of `"langgraph==<exact>"`). This allows dependency drift across installs/builds and violates the dependency pinning compliance rule.
## Issue Context
The PR replaced previously version-ranged dependencies with unversioned package names, and `requirements.txt` installs `.[aws,xray,otel]` without any pins/hashes.
## Fix Focus Areas
- pyproject.toml[19-86]
- requirements.txt[1-25]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| # pipeline can lift it verbatim. | ||
| # hadolint ignore=DL3007 | ||
| FROM python:3.12.7-slim-bookworm AS builder | ||
| FROM public.ecr.aws/docker/library/python:3.13-slim-bookworm AS builder |
There was a problem hiding this comment.
2. Docker image tag floats 📘 Rule violation ☼ Reliability
The Dockerfile uses python:3.13-slim-bookworm without a patch version or digest, so builds can change over time as the tag moves. This violates the requirement to pin dependency versions in Docker images.
Agent Prompt
## Issue description
The runtime/builder base images are not pinned to an immutable version/digest.
## Issue Context
Compliance requires Docker base images be pinned to a specific version or digest; floating tags like `python:3.13-slim-bookworm` can change.
## Fix Focus Areas
- Dockerfile[41-41]
- Dockerfile[94-94]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| terraform { | ||
| required_version = ">= 1.5.0" | ||
| required_version = ">= 1.13" | ||
|
|
||
| required_providers { | ||
| aws = { | ||
| source = "hashicorp/aws" | ||
| # Bedrock AgentCore Runtime + Memory resources require a recent provider. | ||
| # If your provider version is older than this, see the TODO at the top of | ||
| # main.tf for the documented null_resource fallback. | ||
| version = ">= 5.95.0" | ||
| # Bedrock AgentCore Runtime + Memory resources are first-class on | ||
| # the 6.x line. If your provider version is older than this, see | ||
| # the TODO at the top of main.tf for the documented null_resource | ||
| # fallback. | ||
| version = ">= 6.0" | ||
| } | ||
|
|
||
| null = { |
There was a problem hiding this comment.
3. Terraform provider versions unpinned 📘 Rule violation ☼ Reliability
terraform/agentcore-runtime/versions.tf uses open-ended version constraints (>=) for Terraform and providers, allowing upgrades without an explicit change review. This violates the dependency pinning requirement for manifests.
Agent Prompt
## Issue description
Terraform `required_version` / `required_providers` constraints are specified as open-ended ranges (e.g., `>= 6.0`) rather than exact versions.
## Issue Context
The compliance rule requires exact, immutable dependency versions in manifests.
## Fix Focus Areas
- terraform/agentcore-runtime/versions.tf[1-17]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| …then re-run this script. If your checkout lives elsewhere, point | ||
| \`MCP_INTERNAL_REPO\` at it or pass \`--jar /abs/path/to/mcp-internal-X.Y.Z.jar\`. | ||
|
|
||
| See mcp/README.md "External JAR contract" for full details. | ||
| MISSING | ||
| exit 2 | ||
| fi | ||
|
|
||
| # A Git-LFS pointer file is ~130 bytes; the real JAR is ~62 MB. Catch | ||
| # the common "developer forgot `git lfs pull`" footgun before docker | ||
| # wastes a layer on a useless 130-byte payload. | ||
| JAR_SIZE_BYTES="$(wc -c <"${JAR_PATH}" | tr -d ' ')" | ||
| # Spring Boot fat-JARs are typically 50–80 MB; anything dramatically | ||
| # smaller is almost certainly a `-plain.jar` mistakenly passed in or a | ||
| # truncated download. 1 MB is a safe floor. | ||
| JAR_SIZE_BYTES="$(wc -c <"${JAR_SOURCE}" | tr -d ' ')" | ||
| if [[ "${JAR_SIZE_BYTES}" -lt 1048576 ]]; then | ||
| cat >&2 <<LFS_POINTER | ||
| ERROR: ${JAR_PATH} is only ${JAR_SIZE_BYTES} bytes — looks like a Git LFS | ||
| pointer file rather than the real JAR. Run: | ||
|
|
||
| git lfs install | ||
| git lfs pull | ||
|
|
||
| and re-run this script. | ||
| LFS_POINTER | ||
| cat >&2 <<TOO_SMALL | ||
| ERROR: ${JAR_SOURCE} is only ${JAR_SIZE_BYTES} bytes — that's too small to | ||
| be a Spring Boot fat-JAR (expected ~50–80 MB). Make sure you are | ||
| pointing at the bootJar output, not the lightweight \`-plain.jar\` | ||
| Gradle also emits. |
There was a problem hiding this comment.
4. Non-ascii strings in build_mcp_image.sh 📘 Rule violation ✧ Quality
Multiple modified user-facing/help text, docstrings, and committed JSON metadata/schema strings include non-ASCII characters (e.g., …, é, —, –). This violates the English-only ASCII string-literal policy and can break or degrade tooling/rendering/search behavior across environments that assumes ASCII.
Agent Prompt
## Issue description
Modified string literals and JSON string values include non-ASCII characters (e.g., ellipsis `…`, accented characters like `é`, and Unicode dashes `—`/`–`), which violates the ASCII-only policy for committed code/config and user-facing text.
## Issue Context
PR Compliance ID 576985 requires all modified string literals/values (including Python docstrings, shell-script help/error output, and committed JSON configuration/metadata/schema) to be ASCII-only unless explicitly exempted. Non-ASCII characters can also cause inconsistent rendering/searching and can break tooling that assumes ASCII logs.
## Fix Focus Areas
- scripts/build_mcp_image.sh[195-212]
- scripts/agentcore_deploy_wrapper.py[2-33]
- src/agent/infrastructure/cost/model_prices.json[2-14]
- src/agent/infrastructure/structured_log_schema.json[215-235]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| def _truncate_for_log(value: str, limit: int = _MAX_LOG_BODY_CHARS) -> str: | ||
| del limit | ||
| return value |
There was a problem hiding this comment.
5. Unbounded webhook payload logging 🐞 Bug ⛨ Security
lambda/webhook_validator/handler.py’s _truncate_for_log() is a no-op, so the Lambda logs full webhook bodies and full SQS envelope JSON at INFO/ERROR. This can leak sensitive/PII data into logs and can explode log volume/cost or exceed log/annotation limits under large Jira payloads.
Agent Prompt
## Issue description
`_truncate_for_log()` currently ignores its `limit` parameter and returns the full payload string. Because it is used when logging the inbound request body and the SQS envelope, the Lambda can emit unbounded sensitive payloads into logs.
## Issue Context
This Lambda is the public ingress trust boundary; its logs are likely retained/centralized. Logging full Jira webhook JSON commonly includes user emails, ticket content, and other sensitive data.
## Fix Focus Areas
- Implement truncation (with an ellipsis) in `_truncate_for_log` using `limit`.
- Consider also switching body/envelope logging to DEBUG or logging only sizes + correlation_id (keeping INFO logs non-sensitive).
### Fix Focus Areas (code refs)
- lambda/webhook_validator/handler.py[70-72]
- lambda/webhook_validator/handler.py[226-240]
- lambda/webhook_validator/handler.py[274-296]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| if chat_bedrock_cls is not None: | ||
| return chat_bedrock_cls | ||
| if _direct_invoke_disabled_via_env(): | ||
| return _make_direct_invoke_cls(_default_bedrock_runtime_client) | ||
| # Meta + DeepSeek keep a dedicated adapter path because their | ||
| # structured-output behavior differs from Anthropic/Amazon's | ||
| # native ChatBedrock path. | ||
| if bedrock_model_family in ("meta", "deepseek"): | ||
| return _make_direct_invoke_cls(_default_bedrock_runtime_client) | ||
| return _import_chat_bedrock() |
There was a problem hiding this comment.
6. Deepseek/meta tool calls broken 🐞 Bug ≡ Correctness
bedrock_factory._select_chat_model_ctor() forces the direct-invoke Bedrock adapter for the deepseek/meta families (and also falls back to it on langchain_aws import failure), but that adapter ignores bound tools and always returns AIMessage(..., tool_calls=[]). The graph routes to terminal nodes when tool_calls is empty, so MCP tool execution is skipped and the documented DeepSeek default cannot perform the agentic tool loop.
Agent Prompt
## Issue description
DeepSeek/meta are treated as supported Bedrock model families, but the selected direct Bedrock `converse` adapter discards tools and never produces tool calls. This breaks the agent’s core tool loop (MCP reads/writes), because graph routing depends on `tool_calls` being populated.
## Issue Context
- `_select_chat_model_ctor()` routes `deepseek`/`meta` to `BedrockDirectInvokeChatModel`.
- `BedrockDirectInvokeChatModel.bind_tools()` deletes the tools argument.
- The runnable returns `AIMessage(..., tool_calls=[])` unconditionally.
- The agent graph routes to tools execution only when tool calls exist.
- Docs describe DeepSeek V3.2 as the default model family for the runtime and explicitly call out validating the “tool-use surface” end-to-end.
## Fix Focus Areas
Choose one (or both) of:
1) **Implement real tool-calling for direct `converse`**: pass Bedrock `toolConfig` derived from the bound tools and parse `toolUse` blocks from `converse` output into `AIMessage.tool_calls`.
2) **Fail closed**: if tools are bound (non-empty), raise a clear exception so the worker/runtime fails fast instead of silently skipping tool execution; also reconsider forcing direct-invoke for `deepseek`/`meta` until tool calling is implemented.
### Fix Focus Areas (code refs)
- src/agent/infrastructure/llm/bedrock_factory.py[159-168]
- src/agent/infrastructure/llm/bedrock_converse_direct.py[111-125]
- src/agent/infrastructure/llm/bedrock_converse_direct.py[226-234]
- src/agent/graph/nodes/_agentic_turn.py[193-246]
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
- Simplified `.env.example` and `.env.local.example` by removing outdated comments and variables related to Bedrock model IDs and logging configurations, enhancing clarity for developers. - Updated the `deploy/local/bootstrap.sh` script to reflect changes in service management, specifically rolling the `agent-worker` instead of the deprecated `agent-runtime`, and adjusted port-forwarding instructions accordingly. - Modified the `scrum_dev_manual.json` fixture to update correlation IDs and issue keys, ensuring consistency with current testing scenarios. These changes aim to improve the developer experience by clarifying environment configurations and updating local deployment practices.
Summary
End-to-end hardening of the
aws-agent-coreruntime to land it on AWS Bedrock AgentCore + EKS while preserving the local Kubernetes profile: a newserver.pyentrypoint and VPC-scoped AgentCore wiring replace the legacymain.pyboot path, the Bedrock adapter is upgraded tous.amazon.nova-pro-v1:0on a hard-pinnedus-east-1region with Converse-compliant system-message normalization, the MCP-internal Spring Boot JAR is removed from the repo and resolved from a GitHub Release at build time, and the graph picks up a HITL approval checkpoint, idempotent async dispatch, multi-tenant resolution, token governance, and richer observability. Local profile, Terraform modules, CI workflows, and docs are updated in lockstep so AWS-only behavior never lands without a 1:1 local + Terraform counterpart.Engineering quality gates
mypy --strictis green; no new# type: ignorewithout an inline justification comment.Protocolinagent.contractsand wired inagent.composition; no new direct imports of provider SDKs (boto3,langchain_aws,langchain_anthropic, …) outsideagent.infrastructure.*. New seams added:agent.contracts.agentcore_invoker.AgentCoreInvoker, plus existingdedupe_store,work_publisher,mcp_call_observer,token_usage, andgraph_runnerprotocols.python -m pytestexits 0 with--cov-fail-under=100. Bothcoverage.xmlandhtmlcov/are produced (CI publishes them as artifacts).ruff checkis green; no rules silenced without an inline justification. Import-linter contracts updated to keepagent.application/agent.domainfree of provider SDK imports.Security and secret hygiene
inspector/config.jsonvalues are committed..env.local.exampleanddeploy/local/configmap.yaml, and surfaced via KubernetesSecret(local) and AWS Secrets Manager (prod, seeterraform/kms-secrets/andterraform/secrets-mcp/).DAVIDSUPERSECRETTOKENis used only fordeploy/local/secrets.example.yamland is replaced by a Secrets-Manager-backed value in production (scripts/sync_jira_integration_secret_from_env.sh).pyproject.toml. No new third-party runtime dependencies were added in this PR — only existing pins were tightened.SOLID / IoC review
agent.applicationoragent.domain.Protocoldefined inagent.contracts.*. The new AgentCore invocation seam (agentcore_invoker.py) and its concrete adapter (infrastructure/agentcore/boto_invoker.py) follow the same pattern.agent.composition) is the only place that names concrete adapters —composition/aws.py,composition/local.py, andworker/composition.pyare the only modules that import fromagent.infrastructure.*.Local profile validation
make local-up && make seed-localstack && make smoke && make smoke-recursionwas executed and both smoke targets are green against the local Kubernetes + LocalStack profile.correlation_idwas visually confirmed in Jaeger (make jaeger) — webhook → dispatch → assessor → designer → approval → persistence spans all present with parent linkage intact.AWS-only deferrals
terraform/orscripts/:terraform/agentcore-runtime/(IAM, VPC, examples) +scripts/bootstrap_agentcore_runtime.sh+scripts/agentcore_deploy_wrapper.py.terraform/eks-workloads/templates/agent-worker/+terraform/eks-workloads/templates/mcp-internal/.terraform/eks/adot.tf,terraform/observability-cloudwatch-genai/,terraform/observability-xray/.terraform/kms-secrets/,terraform/secrets-mcp/,terraform/iam/mcp_token_reader_role.tf.terraform/edge-security/(ALB, API GW, authorizer).terraform/dlq-s3/,terraform/data-stores/.Test plan
Executed locally on macOS (Python 3.13, Docker Desktop + k3d):
make lint— ruff +mypy --strict, both green.make test— full unit suite,--cov-fail-under=100enforced;coverage.xmlandhtmlcov/generated.make local-up— bootstraps k3d cluster, LocalStack, Jaeger, MCP-internal, agent-worker.make seed-localstack— seeds SQS, DynamoDB, S3, Secrets Manager.make smoke— happy-path webhook → assessor → designer → approval → persistence usingscripts/fixtures/manual/scrum_dev_manual.json.make smoke-recursion— recursion-guard fixture; confirms guard trips and DLQ publishes.make jaeger— manual span-trail verification for the smokecorrelation_id.tests/infrastructure/test_bedrock_llm.py(system-message normalization),tests/worker/test_composition.py+tests/worker/test_envelope_parity.py(worker boot),tests/application/test_webhook_handler.py(signature + idempotency),tests/graph/test_evaluate_human_approval_node.py(HITL gate),tests/scripts/test_check_docs_code_sync.py(docs ↔ code drift guard).scripts/rebuild_sandbox.shagainst theus-east-1sandbox account; AgentCore runtime invoked end-to-end (note: tools not yet called in the sandbox path — tracked separately).Reviewer notes
Major reviewable seams, grouped by theme:
.bedrock_agentcore.yaml,src/agent/server.py(new),src/agent/probe.py(new),src/agent/infrastructure/agentcore/boto_invoker.py(new),src/agent/contracts/agentcore_invoker.py(new),terraform/agentcore-runtime/,scripts/bootstrap_agentcore_runtime.sh,scripts/agentcore_deploy_wrapper.py,scripts/promote_to_agentcore.sh. Entrypoint moved frommain.py→server.py; please verify the VPC + security-group config interraform/agentcore-runtime/vpc.tfmatches the sandbox.src/agent/infrastructure/bedrock_llm.pyadds_normalise_system_messages(collapses non-consecutiveSystemMessages into a single leading message per the Converse contract);infrastructure/llm/bedrock_factory.pydefaults the model tous.amazon.nova-pro-v1:0. Region is hard-pinned tous-east-1acrossscripts/,terraform/, andlambda/.mcp/mcp-internal-*.jarremoved from git;scripts/build_mcp_image.sh,mcp/Dockerfile,mcp/.dockerignore,mcp/entrypoint.sh,mcp/run.sh, and.github/workflows/smoke.ymlnow resolve the JAR from the upstreammcp-internalGitHub Release..gitattributesand.gitignoreupdated to prevent re-introduction.src/agent/graph/nodes/approval.py,src/agent/graph/compile.py, and newpersist_approval_grantedcheckpoint; review the routing edges and the persistence-node test for resumption behavior.src/agent/application/webhook_handler.py,lambda/webhook_validator/handler.py, andtests/application/test_webhook_handler.py. Signature verification path was tightened — please re-confirm the bearer-token branch.src/agent/infrastructure/multitenant/dynamodb_prefix_resolver.py,src/agent/application/token_governance.py,src/agent/infrastructure/llm/usage_capture.py,tests/application/test_token_governance.py.src/agent/infrastructure/observability/,terraform/eks/templates/observability/adot-values.yaml.tftpl,docs/structured_log_schema.json. Span hierarchy was verified manually via Jaeger; ADOT collector values changed — please scan for any IAM/policy drift..github/workflows/ci.yml,mutation.yml,sandbox-apply.yml,smoke.yml,soak.yml. The smoke workflow now downloads the MCP-internal JAR from a GitHub Release; the sandbox-apply workflow uses OIDC-scoped AWS credentials.scripts/check_docs_code_sync.py+tests/scripts/test_check_docs_code_sync.pywere extended; if any reviewer change touches docs, please re-runpython scripts/check_docs_code_sync.py.Known follow-ups (not in scope for this PR):
943039apending a consolidated ADR rewrite — tracked separately.