Repository navigation
test(tasks): add bounded CB/CAN evidence fixtures for #567 - #797
solloek369-arch wants to merge 2 commits into
Conversation
Signed-off-by: Loek <solloek369@gmail.com>
dinakarjs
left a comment
There was a problem hiding this comment.
Independent bounded review of 1206ad2 under Linux/Python 3.12.14 with the repository's hash-locked dev environment: tests/unit/task_evidence567 gives 360 passed. I independently replaced the acknowledged notification's params with the requested listen params; the focused suite gives exactly 1 failed, 359 passed, exposing requested-but-not-accepted as intended. Source restored byte for byte afterward. An earlier mutation invocation resolved the venv symlink incorrectly and could not import pytest; it is not the mutation result cited here.
The code and README consistently keep role authority/population completeness as independently stipulated synthetic premises, distinguish exact-ack exclusion from whole-request exclusion, and preserve operand binding as a separate prerequisite from joint context coverage. No additional source blocker found in this slice. This does not implement runtime callback authentication, replay ordering or terminal-effect verification.
Current hosted CI fails at Security scan before the unit matrix executes; I have not labeled those skipped tests green or diagnosed the scan from a step name alone. Retain normal current-head CI and maintainer approval gates. Reviewed with OpenAI Codex.
|
CI follow-through: I read failed Ubuntu/Python 3.11 job 119756976070. Security scan reports certifi 2026.7.22 / PYSEC-2026-4320, fixed version 2026.10.19, from the inherited dev lock. This is the dependency update already submitted in #795, rather than a failure of the new CB/CAN fixtures. The hosted tests remain unexecuted in that run; my 360-test local result is separate. Please refresh current-head CI after the dependency repair lands. No scan or approval gate was bypassed. Prepared with Codex. |
carloshvp
left a comment
There was a problem hiding this comment.
Reviewed exact head 1206ad22949c4e31d041fb82cda2ac5a3545c1f5 in an isolated checkout.
Approving the bounded, test-only CB/CAN fixture slice. The source keeps the acknowledged task list separate from the requested list, requires independently pinned role authority, and preserves authentication, operand binding, joint applicability and accounting as separate prerequisites.
Independent validation: the complete task_evidence567 selection passes 360 tests; repository Ruff passes. In a separate process I substituted the requested listen parameters for the acknowledgement parameters. Exactly the requested-but-not-accepted regression fails (1 failed, 359 passed), while its positive twin stays valid. The checkout was unchanged by this trial.
The synthetic authority and fixed-population premises remain explicit; these fixtures do not establish production callback authentication, replay ordering or terminal effects. The failed hosted Ubuntu job reports the inherited certifi 2026.7.22 audit finding before the unit matrix executes; #795 carries the separate lock repair. Hosted cancelled jobs remain unvalidated. All PR commits carry DCO sign-off.
Reviewed with OpenAI Codex.
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
There was a problem hiding this comment.
Reviewed the CB/CAN fixture slice on 1206ad2,a few points needs to be addressed:
- CB-06 does not inspect the actual subscriptions/listen taskIds. A valid client + valid permission for T still returns ESTABLISHED even when the listen request asked for a different task or none. Missing negative test.
- CAN-01/action is hard-coded to "tasks/cancel" and never compared to the real request.method. A tasks/update request with an action witness still saying "tasks/cancel" makes the named CAN-01/action relation ESTABLISHED (composite correctly fails via policy).
- CB-05 never checks the listen request’s own taskIds or jsonrpc == "2.0". listen.taskIds=[] + ack containing T, or missing/"1.0" jsonrpc, still yields ESTABLISHED.
- dict(evidence.bindings) silently collapses duplicate operand names. No cardinality/uniqueness check on the five CAN-01 operands.
ruff import grouping + blank-line rules also fail on test_callback_cancellation.py.
What
Smallest acceptance counterexample: a
subscriptions/listenrequest asks for task T, but its separately authenticated, exactly bound completenotifications/subscriptions/acknowledgednotification accepts no tasks. Reading the requested list instead of the acknowledged list would incorrectly establish acceptance. This test-only continuation of #701 distinguishes those worlds; it does not implement runtime callback authentication or Tasks.Expected and measured behavior
The positive acknowledgement includes T:
ESTABLISHED. The complete negative acknowledgement excludes T:VERIFIED_CONTRADICTIONonly for acceptance in that acknowledgement, not for the whole request or any other subscription. Missing or inapplicable acknowledgement evidence isNOT_ESTABLISHED.In
test_callback_cancellation.py, the exact controls are:test_cb05_exact_native_ack_only[positive-ESTABLISHED]test_cb05_exact_native_ack_only[requested-not-accepted-VERIFIED_CONTRADICTION]Both pass. Weakening
subscription_acceptanceto taketaskIdsfromrequest["params"]["notifications"]instead of the separate acknowledgement makes the negative assertion fail: actualESTABLISHED, expectedVERIFIED_CONTRADICTION. The positive twin remains green; the full focused mutant run gives 1 failed, 359 passed. Byte-exact restoration returns 360 passed. This was reproduced on both cold-review Python versions below. The consequence is a load-bearing distinction between requesting a subscription and evidence that this acknowledgement accepted it—not a production defect claim.Four additive test/documentation files
All are under
tests/unit/task_evidence567/:callback_cancellation.py: conditional CB/CAN relation helpers, reusing existing signed-witness verification, Boundary/Cedar evaluation and accounting.test_callback_cancellation.py: 98 relation/integration cases with established positive twins and decisive negative controls.test_callback_accounting.py: 150 completeness/accounting cases.CB_CAN_README.md: approved profile, trust premises, evidence boundaries and limitations.No existing file is changed. The original #701 README, eight Python files and A–G runner remain byte-identical. Its open production callback-authentication/authority work is not contradicted by these synthetic fixtures.
Why
Imran accepted the matrix and completeness contract and authorized fixture code, with two decisions:
subscriptions/listenfirst, against ext-tasks at7b8e2bd; CB-05–07 are included, webhook waits.The approved matrix is covered as separate relations:
Why 150 accounting cases
There are fourteen fixed obligations: seven per captured delivery and seven per captured cancellation request (five CAN-01 operands, CAN-02 and composite authority). The 150 cases comprise 140 per-boundary parameter cases plus ten population/scope controls. They reuse #701's count-before-eligibility mechanism while testing this slice's source triggers: omission, wrong occurrence/boundary, duplicates, malformed attempts and valid-plus-malformed duplicates stay visible. An explicit
NOT_ESTABLISHEDcan be accounted without becoming established; an empty or supporting-only population is outside the bounded contract, not a vacuous pass. The frozen source population is a fixture premise, not proof of complete real traffic capture.Security impact
None to production code or security boundaries. Synthetic deterministic test keys and their role authority are explicit fixture premises, not real credentials. No dependencies, production files or wire schemas change; existing #701 behavior and mutation-runner interface remain unchanged.
Compatibility/limitations: no live callback authentication, Tasks runtime/storage, real delivery, cancellation effect, terminal-result verification, external effects, replay/out-of-order semantics, webhook profile or final wire representation is implemented or established. Evidence validity is scoped to the selected fixture context; this introduces no new per-delivery reauthorization requirement. This contribution does not close #567.
Test plan
PYTHONPATH=src python -m pytest tests/unit/task_evidence567 -q— 360 passed on Python 3.11, 3.12 and 3.13.tests/unit/task_evidence567/), mypy (src/cmcp_runtime/) and configured Bandit (src/) — passed.subscription_acceptanceto read task IDs from thesubscriptions/listenrequest instead of the separately boundnotifications/subscriptions/acknowledgednotification. The testtest_cb05_exact_native_ack_only[requested-not-accepted-VERIFIED_CONTRADICTION]then fails (actualESTABLISHED, expectedVERIFIED_CONTRADICTION). The positive twin remains green. Full focused run: 1 failed, 359 passed. Restoring the check returns 360 passed.certifi/pip-audit issue is addressed in PR #795.DCO sign-off
Developer Certificate of Origin (https://developercertificate.org).
Signed-off-by: Loek solloek369@gmail.com
Commit:
1206ad22949c4e31d041fb82cda2ac5a3545c1f5.