CONFORMANCE.md

Conformance

How pluginpack verifies that emitted output matches each target app's real plugin format.

Why this is hard

Agent Plugins is the only format with a referenceable upstream JSON Schema. Every other target's source of truth is something other than a stable schema URL:

Target	Canonical source of truth	Referenceable schema?
claude	claude plugin validate CLI + plugins-reference docs	No. The $schema URL the manifest declares (https://anthropic.com/claude-code/marketplace.schema.json) returns 404.
cursor	Glean-authored schemas in gleanwork/cursor-plugins/schemas/	No upstream. The schema $id (https://cursor.com/schemas/cursor-plugin/...) 500s; no Cursor-published schema found.
antigravity	Antigravity CLI plugin docs (plugin.json, optional mcp_config.json)	No. Defined by product docs and observed CLI layout, not a published schema.
copilot	github/copilot-plugins — a Claude-marketplace-derived format	Structural. Copilot shares the Claude marketplace base but extends entries (skills[], mcpServers as a path), which claude plugin validate rejects — so conformance is asserted structurally against the official format.
codex	OpenAI plugin packaging docs: an Agent Plugins package + .agents/plugins/marketplace.json	Package: yes (Agent Plugins 1.0 schemas, vendored). Marketplace: no published schema; asserted structurally against the docs (retrieved 2026-10-02).
agent-plugins	Agent Plugins Specification 1.0.0	Yes. schemas/1.0.0/plugin.schema.json and mcp.schema.json, vendored in tests/fixtures/agent-plugins/ with provenance in SOURCE.md. The spec text wins where they disagree.

Oracles the harness uses

Conformance tests live in tests/conformance.test.ts and run the real built CLI against a temp fixture via bintastic.

• claude — runs claude plugin validate --strict against the emitted marketplace and plugin directories when the claude CLI is on PATH (skipped otherwise). CI installs the CLI before running the suite. This is Anthropic's own validator, the same check their submission pipeline runs, so it is an external oracle that tracks upstream automatically. A structural golden assertion also pins the exact emitted shape.
• cursor — validates the emitted marketplace.json and plugin.json against the vendored schemas in tests/fixtures/cursor/ (provenance in tests/fixtures/cursor/SOURCE.md). Caveat: those schemas are authored by Glean, not Cursor, so this check is partly self-referential. The external signal is empirical — gleanwork/cursor-plugins is live in Cursor's marketplace and loads. The published schema's additionalProperties: false is also stricter than Cursor's runtime: the marketplace version field is tolerated in practice, so the test allows that one key explicitly while still rejecting any other unexpected field.
• copilot — asserted structurally against the official github/copilot-plugins format: the marketplace is written to both .claude-plugin/marketplace.json and .github/plugin/marketplace.json (the test checks the copies are identical) and entries carry skills[], version, and mcpServers. claude plugin validate is deliberately not used here: Copilot derives from the Claude marketplace but extends entries (e.g. mcpServers as a .mcp.json path), which Claude's stricter schema rejects. Because it shares the Claude marketplace path, the claude and copilot targets need separate output roots. An agent-plugins target with clients.copilot also writes .github/plugin/marketplace.json, so it needs an output root separate from the copilot target's.
• antigravity — covered structurally by the cross-target build test in tests/core.test.ts (required plugin.json fields present; optional mcp_config.json written when MCP servers are present). Antigravity CLI does not expose a published schema to validate against.
• agent-plugins — the emitted plugin.json and mcp.json validate against the vendored Agent Plugins 1.0 schemas (draft 2020-12, via ajv's 2020 build), and pluginpack validate must pass on the same output. The shipped validator (src/agent-plugins.ts) re-implements the closed-schema checks without ajv, plus the semantic rules a JSON Schema can't express: skill names per the Agent Skills specification and matching their directory, and MCP server rules from spec §7.2 and §9 (src/mcp.ts). With clients, the same package oracle applies. The Codex marketplace uses the codex checks below, and the Copilot marketplace and com.github.copilot/agents/ layout are asserted structurally against the Copilot CLI plugin reference (retrieved 2026-10-20), which publishes no schema. Client facts and their citations live in src/package-clients.ts.
• codex — the package half uses the Agent Plugins oracle above. The marketplace half is asserted structurally in tests/conformance.test.ts and tests/core.test.ts against the OpenAI packaging docs (re-verified 2026-10-02): a repo-scoped .agents/plugins/marketplace.json ({ name, interface, plugins }, no owner field). The default package is a root plugin.json with OpenAI settings under extensions.com.openai. format: "legacy" instead emits .codex-plugin/plugin.json. In Codex rust-v0.146.0's legacy parser, every recognized field has a serde default; an omitted or blank name falls back to the plugin directory basename. version, description, skills, hooks, and mcpServers are optional. Pluginpack keeps a stricter rule: an explicit name must match the marketplace entry. validate detects either layout per plugin. Every marketplace entry must carry policy.installation, policy.authentication, and category; pluginpack has no way to infer these, so the base entry stays guess-free. Build warns and validateOutput errors if an author never supplies them via the per-plugin entry passthrough (or clients.codex.entry for a package client). Build also warns about skill frontmatter that package validation rejects. An entry's source is a bare string only for local plugins (the only shape pluginpack itself ever emits); a url/git-subdir/npm source added via entry is a structured object with an inner source discriminator (e.g. { source: "git-subdir", url, path, ref }), validated by shape rather than requiring a local directory to exist. No published JSON Schema exists. Among the native targets, Codex shares no marketplace path, but an agent-plugins target with clients.codex writes the same .agents/plugins/marketplace.json, so the two need separate output roots.

MCP dialect facts

src/mcp.ts renders one authored MCP config into each client's dialect. Each dialect rule carries a citation (getMcpDialectCitations), and the citation test in tests/install-snippet.test.ts fails if a rule has no documentation URL or date. All were verified on 2026-10-21:

Dialect	Rule	Source
claude	${CLAUDE_PLUGIN_ROOT} and ${CLAUDE_PLUGIN_DATA} expand in a stdio server's command, args, and env	https://code.claude.com/docs/en/plugins-reference#environment-variables
claude	no type means stdio; Streamable HTTP is http (streamable-http is an alias)	https://code.claude.com/docs/en/mcp
claude	cwd is not a field Claude Code reads for a plugin stdio server, so a server with cwd is a build error	https://code.claude.com/docs/en/plugins-reference#environment-variables
cursor	${CURSOR_PLUGIN_ROOT} and ${CLAUDE_PLUGIN_ROOT} expand in command, args, env, and cwd; ${PLUGIN_ROOT} and ${PLUGIN_DATA} don't	https://cursor.com/docs/reference/plugins#mcp-servers
cursor	no plugin-data variable is documented, so a data variable is a build error	https://cursor.com/docs/reference/plugins#mcp-servers
cursor	Cursor plugins can infer the transport from command or url, so type is dropped	https://cursor.com/docs/reference/plugins#mcp-servers
copilot	${PLUGIN_ROOT}, with ${CLAUDE_PLUGIN_ROOT} as an alias	https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
copilot	${PLUGIN_DATA} is documented only for Agent Plugins packages, so legacy-format data variables stay as authored	https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
copilot	remote servers may be labelled http, sse, or streamable-http	https://docs.github.com/en/copilot/reference/copilot-cli-reference/cli-plugin-reference
agent-plugins	closed server variants; one-token command; args, env, cwd expand ${PLUGIN_ROOT} and ${PLUGIN_DATA}	https://agent-plugins.org/specification (§7.2.1, §9)
codex	the agent-plugins rules, plus per-server auth in extensions["com.openai"].auth	https://developers.openai.com/codex/plugins/build

Claude Code ignores cwd

Claude Code's plugins reference lists the fields where ${...} resolves for an MCP stdio server as command, args, and env; its MCP page lists the same three. Neither documents cwd. In a check with Claude Code 2.1.284, a plugin server with "cwd": "${CLAUDE_PLUGIN_ROOT}/scripts" connected, but its process ran in the session's working directory, not scripts/. pluginpack 0.11 copied cwd to claude output as authored, so a server that relied on it started in the wrong directory. The claude dialect now fails the build for any server with cwd, and the error says to resolve paths in args with ${CLAUDE_PLUGIN_ROOT} or to use a claude-only mcp/config.json override.

Open point: Copilot and the legacy .mcp.json

The copilot target emits a legacy plugin: a manifest plus .mcp.json. The cited Copilot CLI reference documents placeholder expansion for MCP servers in two other places, and neither covers the legacy shared .mcp.json:

• Agent Plugins packages (mcp.json): ${PLUGIN_ROOT} and ${PLUGIN_DATA} expand in args, env values, and cwd, not in command.
• MCP servers in a plugin-shipped agent's own frontmatter: ${PLUGIN_ROOT} (with the ${CLAUDE_PLUGIN_ROOT} and ${COPILOT_PLUGIN_ROOT} aliases) expands in command and args, and the reference says this substitution applies only there.

So the copilot rows above are the closest documented behavior, not a documented rule for .mcp.json. pluginpack writes a ./ command or cwd as ${PLUGIN_ROOT}/... for copilot. The docs neither confirm nor rule out that Copilot CLI expands it there. Copilot CLI 1.0.102 loads such a plugin and lists the command as written; whether it expands the variable at launch is unverified. pluginpack keeps this rewriting until Copilot documents the legacy format or a launch check shows it is wrong.

Conflict: Codex per-server extensions

The Agent Plugins 1.0 specification (§7.2.1) closes each MCP server variant: an unknown field makes the server entry invalid, and the client skips it. The vendored tests/fixtures/agent-plugins/mcp.schema.json agrees (additionalProperties: false). OpenAI's plugin packaging docs (verified 2026-10-21) put per-server MCP auth in the same file, under extensions["com.openai"].auth on each server. A server written that way is not valid Agent Plugins 1.0.

pluginpack resolves the conflict per target:

• codex writes Codex-only output, so it passes a per-server extensions object through. That object may hold only com.openai, whose value must be an object. validate accepts the same shape. A server with this object does not validate against the vendored schema, so the schema oracle covers only codex servers without it.
• agent-plugins keeps the strict 1.0 shape, with or without clients.codex. Its package is meant for every conforming client, and Copilot CLI skips an invalid server entry. The error says this and names the workaround.

Workaround for a project that builds both targets: keep the portable config in shared/<plugin>/mcp/config.json. Set overrides: "overrides/codex/<plugin>" on the codex plugin, and put a full copy of config.json with extensions["com.openai"] in overrides/codex/<plugin>/mcp/config.json. An override replaces the whole file; it is not merged.

Command files

A ./ or ${<root variable>}/ command names a file inside the plugin. If the artifact has no such file, the build prints a warning on every target. The build does not fail, because a file such as a compiled binary can be added after the build. validate reports an error for an Agent Plugins package (codex, agent-plugins) whose command file is missing on disk.

Update-check hook facts

The generated update-check hook (updateCheck config, claude/cursor targets) relies on two upstream hook behaviors that have no referenceable schema. Both were verified against product docs on 2026-07-22:

• Claude Code — a SessionStart hook may print JSON whose top-level systemMessage field is shown directly to the user (separate from model context), and plugin hooks are auto-discovered from hooks/hooks.json with ${CLAUDE_PLUGIN_ROOT} substituted in commands. Source: https://code.claude.com/docs/en/hooks.
• Cursor — the sessionStart hook supports only additional_context (injected into the agent's context); there is no user-visible message field, so the cursor nudge asks the agent to relay it. Plugin hooks live in a hooks.json ({ "version": 1, "hooks": { "sessionStart": [...] } }) referenced from the plugin manifest, with commands relative to the plugin root. Source: https://cursor.com/docs/agent/hooks.

The emitted hooks/hooks.json shapes are covered by the conformance suite (the cursor manifest's hooks key validates against the vendored plugin schema, and claude plugin validate --strict exercises the claude hooks file when the CLI is present).

Every target's validateOutput shares one validateHooksShape (src/targets/validation-shared.ts) for the parts of a hooks file that are target-agnostic: each event's entries must be an array, no command string may be empty, and any command referencing the generated update-check script (scripts/pluginpack-update-check.sh) must ship that script.

Install-snippet facts

pluginpack install-info (and the buildInstallSnippet library function) prints the real command or URL a user needs to add a pluginpack-built marketplace. Each target's snippet and citation live on its own PluginTargetDefinition.installSnippet in src/targets/<name>.ts, verified directly against product docs:

Target	Snippet	Source	Verified
claude	claude plugin marketplace add <repo> then claude plugin install <name>@<marketplace>	https://code.claude.com/docs/en/plugins/cli-reference	2026-10-21
codex	codex plugin marketplace add <repo>	https://learn.chatgpt.com/codex/developer-commands	2026-10-21
copilot	copilot plugin marketplace add <repo> then copilot plugin install <name>@<marketplace>	https://docs.github.com/en/copilot/how-tos/copilot-cli/customize-copilot/plugins-finding-installing	2026-10-21
antigravity	agy plugin install https://github.com/<owner>/<repo>	https://antigravity.google/docs/plugins?tab=cli	2026-10-21
cursor	Prints the repo URL, pasted into Dashboard → Plugins & MCPs → Team Marketplaces → "Import from Repo."	https://cursor.com/docs/plugins	2026-10-21

The 2026-10-21 refresh corrected both install snippets. Claude's shell commands were checked against the current command reference and Claude Code 2.1.284's claude plugin --help and claude plugin marketplace --help. The old plugins-reference#cli-commands-reference URL now redirects to the manifest reference, so the citation points to the current command reference. The interactive /plugin marketplace add <repo> and /plugin install <name>@<marketplace> forms remain documented. The latter opens the plugin details for review and scope selection rather than installing immediately; the snippet's note explains this.

Antigravity's current CLI docs show the exact command agy plugin install https://github.com/<owner>/<repo> and the local form agy plugin install </path/to/local/plugin>. install-info uses the configured repository URL. Its note retains clone-then-local-install with the generated pluginPath, because the docs do not specify a GitHub URL subdirectory selector.

The legacy Codex citation was also corrected on 2026-10-21 against the tagged OpenAI source described above. The native parser does not require an explicit name; pluginpack's marketplace-matching rule does. Only these three corrected citations advance their verifiedAt dates in this follow-up.

agent-plugins is the one target with userConfigurable: false. The specification defines no marketplace or install command, and installation is left to each client (for example, VS Code installs a plugin directly from a Git repository URL: https://code.visualstudio.com/docs/agent-customization/agent-plugins, verified 2026-10-02).

Evidence refresh cadence

Re-verify every citation and vendored schema at least every 90 days. The Evidence freshness workflow runs tests/evidence-freshness.test.ts every Monday and on demand. It checks every literal verifiedAt in src/ and the Verified-at date in each vendored schema directory's SOURCE.md. It fails when a date is more than 90 days old, is in the future, or is invalid. The date checks do not run in npm test, because their result depends on the day rather than the change. Run them locally with env PLUGINPACK_CHECK_FRESHNESS=1 npx vitest run tests/evidence-freshness.test.ts. The tests do not fetch the network.

To refresh a citation, open its documentationUrl, check the claim against the current product docs, and run the affected target's conformance tests. Update verifiedAt only after that check. Correct the implementation or record a format conflict here if the upstream behavior changed.

To refresh a vendored schema, follow the fetch commands in its SOURCE.md, diff the fetched bytes against the pinned copy, and run npx vitest run tests/conformance.test.ts and the freshness command above. Record the checked commit and update Verified-at: YYYY-MM-DD in SOURCE.md, even when the bytes are unchanged. Do not advance dates just to pass CI.

Refreshing vendored schemas

The Cursor and Agent Plugins schemas are pinned copies. To update them, re-fetch from the source recorded in tests/fixtures/<format>/SOURCE.md, then re-run the suite. Do not hand-edit — they are an oracle. Published Agent Plugins schema identifiers are never reassigned (spec §10.1), so a new specification version means a new vendored directory, not an edit.

Why vendor instead of fetch at runtime?

Even where a canonical URL existed, tests should not fetch it at runtime: network flakiness makes tests non-deterministic, and a moving upstream would break builds unpredictably. Pinning a copy with recorded provenance — plus, for Claude, shelling out to the installed official validator — keeps the checks deterministic and hermetic.