Skip to content

test(conformance): add POLICY-TEE-001 through POLICY-TEE-005 - #780

Draft
kingztech2019 wants to merge 2 commits into
agentrust-io:mainfrom
kingztech2019:test/policy-tee-conformance
Draft

kingztech2019 wants to merge 2 commits into
agentrust-io:mainfrom
kingztech2019:test/policy-tee-conformance

Conversation

@kingztech2019

Copy link
Copy Markdown

What

Executable acceptance evidence for the five bounded cases on #124, plus the Group 7 table in the conformance suite README and a STATUS.md correction. Draft, per your note that a draft is enough to start review.

Why

Closes the conformance half of #124. Software-only by design, the same choice #552 made: SoftwareOnlyProvider presents the same collector-side shape as SEV-SNP and TDX, so the round trip exercises the contract rather than the silicon. Provider hardware requirements are recorded in the README rather than asserted as passing.

Following the 24 September scoping:

  • POLICY-TEE-005 tests both constructions against the same policy mutation, with unchanged-policy controls, and asserts each verification check rejects its own mismatch. Their digests are not asserted equal.
  • POLICY-TEE-002 separates startup binding from session-claim evidence. make_measurement_bound_nonce and make_audit_bound_nonce share a layout and both occupy report_data[32:64], so one report carries one of them. A session claim commits the audit root there, so startup wiring alone does not establish both bindings in a session claim.
  • No sixth case. Host-bypass prevention is not demonstrated here, and a documentation citation is not a passing conformance case, so that criterion and the TPM limitations stay open and are recorded in the suite README instead.

The TPM rows pair the two independent rejections #709 made available, and the bound provider set is asserted against report_binding.py as sev-snp, azure-cvm-sev-snp, tdx.

Security impact

Touches TEE boundaries as test-only coverage; no runtime code changes. The suite pins properties that are currently load-bearing but untested at conformance level, notably that the measurement check is opt-in so a claim is never asked to vouch for itself, and that an unusable expected digest fails closed rather than skipping the check.

Test plan

  • pytest passes: 27 new cases, 85 in tests/conformance/
  • ruff check passes
  • mypy passes (config scopes it to src/cmcp_runtime and src/cmcp_verify; tests fall outside it, consistent with the existing conformance suite)
  • Manual test performed, below

Mutation-checked rather than assumed, since a case that cannot fail is not evidence:

Mutation Result
Swap the halves of make_measurement_bound_nonce 5 cases fail across POLICY-TEE-002/003/004
Short-circuit refresh_measurement_binding on an unchanged digest exactly test_an_unchanged_policy_still_re_attests fails
Make policy_digest constant 5 fail, including both POLICY-TEE-005 movement cases

Source was reverted after each and the suite returns to 27 passing.

One unrelated note: tests/unit/test_startup.py fails on my machine before any change, because a TPM device is present without tpm2-tools, so the provider auto-detects and then cannot read PCRs. Not touched by this branch and not expected in CI.

DCO sign-off

Executable acceptance evidence for the five bounded cases on agentrust-io#124. Software-only
by design, the same choice agentrust-io#552 made: SoftwareOnlyProvider presents the same
collector-side shape as SEV-SNP and TDX, so the round trip exercises the
contract rather than the silicon.

POLICY-TEE-001 pins that the measurement commits code, policy and config and
keeps them separable, so a verifier learns which input moved.

POLICY-TEE-002 covers the startup binding into report_data[32:64] and,
separately, what a session claim does not show. make_measurement_bound_nonce and
make_audit_bound_nonce share a layout and both occupy the second slot, so one
report carries one of them. A session claim commits the audit root there, which
means startup wiring alone does not establish both bindings in a session claim.
The TPM rows record that a negative result means "not handled by this
mechanism", and pair the two independent rejections agentrust-io#709 made available.

POLICY-TEE-003 covers re-attestation on reload, including the control that an
unchanged policy still re-signs, since report_data carries no history.

POLICY-TEE-004 covers verifier-side rejection and that the check is opt-in, so a
claim is never asked to vouch for itself.

POLICY-TEE-005 tests the measurement's policy component and the bundle's
canonical hash against the same mutation, with unchanged-policy controls, and
asserts each rejects its own mismatch. Their digests are deliberately not
asserted equal: they are separate constructions computed at different startup
steps and agentrust-io#552 kept them apart on purpose.

No sixth case. Host-bypass prevention is not demonstrated here, and a
documentation citation is not a passing conformance case, so that criterion and
the TPM limitations stay open and are recorded in the suite README instead.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: oluwajuwon omotayo <ginuxtechacademy@gmail.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for this, @kingztech2019. Closing it for now, and it is not about the change itself.

This repository asks first-time contributors to be vouched by a maintainer before opening a pull request. That is because agent-written contributions are easy to produce and expensive to review, and we would rather talk to you first than review something neither of us can explain.

To get vouched: open an issue saying what you want to change and why, in your own words. A maintainer will reply, and add you with /vouch. After that, reopen this pull request or open a new one.

See CONTRIBUTING.md for the detail.

@github-actions github-actions Bot closed this Oct 17, 2026
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@imran-siddique

Copy link
Copy Markdown
Member

/vouch @kingztech2019

github-actions Bot added a commit that referenced this pull request Oct 19, 2026
Vouched by imran-siddique in #780.
@imran-siddique

Copy link
Copy Markdown
Member

@kingztech2019 sorry, the first-time contributor gate closed this automatically. I have vouched for you and reopened it; CI will run now and we will review it against #124.

@imran-siddique imran-siddique left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kingztech2019 ran it here on 0a58538: 27 pass. A few things before this comes out of draft.

The POLICY-TEE-004 cases and test_the_measurement_check_rejects_its_own_mismatch build report_data from a fresh SigningKey(), but _claim() signs with another key it creates itself. So every one of those claims also fails public_key_binding, test_a_matching_measurement_verifies included. Have _claim() take the key, so the matching case is clean apart from the hardware fields.

test_neither_is_asserted_to_equal_the_other asserts the two digests differ. That pins the same incidental property from the other side. Drop it and keep the docstring note.

test_the_bundle_hash_check_rejects_its_own_mismatch never calls the verifier. Verify a claim against the pre-mutation approved hash and assert policy_bundle.hash is in unverified_fields; I checked that it lands there.

Small one: "\xee".encode().hex() * 0 or "ee" * 32 is "ee" * 32.

Copy link
Copy Markdown
Contributor

A concrete DCO-signed author-adoption companion is available: 5c4c95af3d4d0502899e1fc24e87181954d99cd1, branch fix/780-isolated-tee-controls, based on reviewed head 0a58538629b1e46b609128b566115268b727a798.

Uses a matching signing key for isolated measurement controls, calls the real verifier for the bundle-hash mismatch, removes the incidental digest-inequality assertion and simplifies the digest fixture.

Earlier exact-commit validation: 26 tests pass; restoring the mismatched-key helper causes five failures; Ruff passes. These results come from the recorded correction run; I verified the published commit/sign-off/files today, not a new runtime rerun.

These are isolated software acceptance controls, not TPM/TEE hardware validation.

For adoption, fetch https://github.com/dinakarjs/cmcp.git branch fix/780-isolated-tee-controls and apply the missing correction against the current head. Please run final-head checks and obtain independent review after incorporation. Original PR ownership is retained; this handoff is not approval of my own correction.

Reference: Imran's current-head review and tests/conformance/test_policy_tee_conformance.py are the primary acceptance sources. Prepared with Codex.

…e hash

Three corrections from review feedback on agentrust-io#780.

The claim fixture built report_data from a throwaway key while the claim was
signed with another, so the two halves of the nonce committed different keys. No
assertion noticed, because the cases only read measurement_binding, which lives
in the second half. The fixture now signs with the key it commits, and a new
case pins both halves so a substitution cannot pass unseen again. Reintroducing
the old mismatch now fails that case; before it failed nothing.

The POLICY-TEE-005 bundle-hash case compared two hashes and called that a
rejection. It now goes through verify_trace_claim the way the measurement case
does, and asserts public_key_binding and policy_bundle.hash on a claim carrying
the mutated bundle, with a control on the approved one so the rejection is not
vacuous.

Dropped the assertion that the two policy digests differ. Their inequality is
incidental: the required property is that both track the same mutation and
neither moves without one, which the surrounding cases already pin. Asserting
inequality is the mirror of asserting equality and over-specifies just as much.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: oluwajuwon omotayo <ginuxtechacademy@gmail.com>
@kingztech2019

Copy link
Copy Markdown
Author

Thanks for the review. Two of these were right and are fixed in 1826379.

The fixture did sign each claim with one key while building report_data from another, so the two halves of the nonce committed different keys. Fixed at the source, and I added a case asserting both halves so a substitution cannot pass unnoticed again.

The POLICY-TEE-005 bundle-hash case was comparing two hashes rather than exercising a check. It now goes through verify_trace_claim the way the measurement case does, with a control on the approved bundle so the rejection is not vacuous. That one mattered, since the acceptance criterion asks for each check to reject its own mismatch and that case was not exercising a check at all.

I also dropped the digest-inequality assertion. Agreed that it was incidental: asserting the two differ is the mirror of asserting they match, and the required property is already covered by the movement and unchanged-policy cases.

One note for the record, since it shaped the fix. Restoring the mismatched key against the branch head did not produce five failures here, it produced none. Every case asserted only on measurement_binding, which lives in the second half of the nonce and is untouched by a first-half substitution, so the defect was real but invisible to the suite. That is why the new case pins both halves rather than only correcting the fixture. With it, reintroducing the mismatch fails exactly one test.

I implemented these directly rather than adopting the branch, to keep authorship and the DCO sign-off on this PR straightforward. Appreciate the observations.

@carloshvp carloshvp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving the software conformance coverage at exact head 182637983c1d6e043c5e6c0a38f11b1f89733d80. The key-binding fixture is corrected, the suite now proves the first-half substitution is observable, and the bundle mismatch actually exercises verify_trace_claim. The incidental inequality assertion has been removed.

Independent Python 3.12 validation in a fresh hash-locked environment installed editable from this checkout: all 28 POLICY-TEE cases pass; changed-file Ruff and diff whitespace checks pass. Independent controls confirm that substituting a different nonce/signing key fails the new first-half test, suppressing measurement-binding rejection fails three negative tests, and suppressing policy-bundle mismatch rejection fails its dedicated case.

These tests exercise software contracts and simulated report_data. They do not validate hardware quotes, host-bypass prevention, or TPM reload/NV-certify continuity. The README and STATUS retain those open limits and keep startup measurement binding distinct from session audit-root binding. The draft's hosted checks and the remaining #124 hardware/host acceptance still need their normal completion.

Nonblocking cleanup: the old "\xee".encode().hex() * 0 or "ee" * 32 fixture can be simplified to "ee" * 32 as previously suggested. All commits are DCO-signed.

Reviewed with OpenAI Codex.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants