Skip to content

chore(conformance): run against 0.2.0-alpha.12 - #1336

Merged
DaleSeo merged 2 commits into
modelcontextprotocol:mainfrom
SergeevDmitry:chore/conformance-json-schema-preservation
Oct 10, 2026
Merged

DaleSeo merged 2 commits into
modelcontextprotocol:mainfrom
SergeevDmitry:chore/conformance-json-schema-preservation

Conversation

@SergeevDmitry

Copy link
Copy Markdown
Contributor

Bumps the conformance suite from 0.2.0-alpha.10 to 0.2.0-alpha.12 and adds the one client scenario the conformance client does not handle yet.

Motivation and Context

0.2.0-alpha.11 (2026-08-07) added the json-schema-2020-12-preservation client scenario (modelcontextprotocol/conformance#335). It is in the all suite for both 2025-11-25 and 2026-07-28. conformance-client has no arm for it and exits with Unsupported conformance scenario, so on alpha.11 or alpha.12 both client suites fail it:

[json-schema-2020-12-client-tool-found    ] FAILURE Client called tools/list and the mock server advertised 'json_schema_2020_12_tool'
[json-schema-2020-12-client-echo-completed] FAILURE Client called tools/call 'json_schema_echo' with a 'schema' object argument

The new arm does what the scenario asks:

  1. connect with conformance_lifecycle();
  2. list tools;
  3. send the inputSchema of json_schema_2020_12_tool back through json_schema_echo as { "schema": ... }.

rmcp keeps every keyword the scenario checks: $schema, $defs, additionalProperties, allOf/anyOf, if/then/else, and $anchor.

How Has This Been Tested?

I ran every step of conformance.yml locally with npx @modelcontextprotocol/conformance@0.2.0-alpha.12, on macOS arm64 with stable Rust 1.99.

Step main This PR
server, 2025-11-25 83/83 83/83
server, 2026-07-28 153/153 153/153
server, the 10 task extension scenarios against expected-failures-extensions.yaml exit 0 exit 0
client, 2025-11-25 208 passed, 2 failed 217/217
client, 2026-07-28 398 passed, 2 failed 407/407
client, extensions suite against expected-failures-extensions.yaml not run exit 0 (only the listed auth scenarios fail)

The server binary does not change in this PR, so its two columns are the same run.

  • json-schema-2020-12-preservation alone: 9/9 on both spec versions.
  • cargo clippy -p mcp-conformance --all-targets -- -D warnings is clean.
  • cargo test -p mcp-conformance passes.
  • expected-failures-extensions.yaml needs no change.

Breaking Changes

None. Only the conformance client and CI change.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

The arm returns an error if json_schema_2020_12_tool is not listed, so a change to the scenario shows up as a failed run instead of a silent pass.

AI assistance: I used Claude Code to help write this change.

0.2.0-alpha.11 added the json-schema-2020-12-preservation client
scenario. The conformance client does not handle it and exits with
"Unsupported conformance scenario", so both client suites fail on it
once CI moves past alpha.10.

Handle the scenario as it asks: list tools, then send the inputSchema of
json_schema_2020_12_tool back through json_schema_echo. rmcp keeps every
keyword the scenario checks. Bump CONFORMANCE_VERSION to alpha.12.
@SergeevDmitry
SergeevDmitry requested a review from a team as a code owner October 10, 2026 09:18
@github-actions github-actions Bot added T-CI Changes to CI/CD workflows and configuration T-config Configuration file changes labels Oct 10, 2026
DaleSeo
DaleSeo previously approved these changes Oct 10, 2026

@DaleSeo DaleSeo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @SergeevDmitry! Left a minor comment.

Comment thread conformance/src/bin/client.rs Outdated
The json_schema_echo result was dropped, so a tool-level error still let
the client exit 0. Check is_error so local runs fail loudly.

@DaleSeo DaleSeo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the quick fix, LGTM.

@DaleSeo
DaleSeo merged commit c1a8eea into modelcontextprotocol:main Oct 10, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

T-CI Changes to CI/CD workflows and configuration T-config Configuration file changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants