docs(agent-plugins): attribute the 8000-byte skill cap to its real owner - #1511
Conversation
The 8000-byte SKILL.md ratchet was widely believed to be a requirement of the Agent Plugins standard. Verified against primary sources on 2026-08-21: it is not. Neither the Agent Plugins spec (1.0.0 and 1.1.0) nor the Agent Skills spec it defers to imposes any body size limit -- Agent Skills constrains only frontmatter and explicitly frames its "< 5000 tokens" / "under 500 lines" numbers as recommendations. The real 8000 is Codex's MAX_SKILL_PROMPT_BYTES (codex-rs/ext/skills/src/ render.rs:19), which applies only on the Agent Plugins path. Record a second, previously unmodeled bound alongside it: Claude Code auto-compaction re-attaches each invoked skill keeping only its first 5,000 tokens, within a 25,000-token combined budget, on every path regardless of manifest. At roughly 20KB that is looser than the existing ratchet, which therefore subsumes it -- so the ratchet stays at 8000 and no separate compaction gate is added. Both truncations keep the start of the file, which makes body ordering load-bearing: what must survive belongs above what may be cut. Also correct two stale upstream claims this verification falsified -- a comment asserting the constant was absent from the Codex source, and a doc citing the since-removed core-skills crate for it -- and capture the generalizable method as a new learning: settle externally-attributed constraints at the spec text and the implementing source, and treat negative claims about upstream as perishable. No behavior change; no test assertions or versions touched. Claude-Session: https://claude.ai/code/session_013DVEGTARyLzqxxzvzoVXEA
PR SummaryLow Risk Overview Records a second, previously unmodeled bound that does apply on the shipping (schema-less) path: Claude Code auto-compaction keeps the first 5,000 tokens per skill inside a 25,000-token combined session budget. The per-skill half is only approximated by the byte ratchet; the aggregate is unguarded and no per-file test can express it. Adds a provenance table in Reviewed by Cursor Bugbot for commit a823da2. Bugbot is set up for automated code reviews on this repo. Configure here. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1e57deb90f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Separate the two halves of the Claude Code compaction bound. The per-skill 5,000-token cap is subsumed by the 8000-byte ratchet; the combined 25,000-token cap is not, and no per-file check can express it. At ~4 bytes/token an at-cap body is ~2,000 tokens, so ~12 fully compliant skills exhaust the combined budget and the oldest are then dropped entirely. tests/codex-skill-prompt-budget.test.ts sizes each SKILL.md independently, so a green run there is not evidence the aggregate holds. Documented as a known, deliberate gap rather than a covered case. Corrected at every site this PR introduced the claim: docs/specs/agent-plugins.md (including the re-verify trigger), the test header, size-driven-skill-restructure.md, verify-externally-attributed-constraints-at-the-source.md, and the CONCEPTS.md Host prompt budget entry. Claude-Session: https://claude.ai/code/session_013DVEGTARyLzqxxzvzoVXEA
The 25,000-token aggregate is a property of a session's invocation sequence, not of the repository, so no repo-side check can establish it with certainty. A guard would only ever cover a declared chain. Bodies are now small enough that a chained workflow keeps real headroom, and the start-preserving truncation is handled by ordering rather than size, so longer-horizon chaining is left to the host. Recorded so the gap is not reopened as backlog. Claude-Session: https://claude.ai/code/session_013DVEGTARyLzqxxzvzoVXEA
Stating that the unguarded aggregate is accepted rather than backlog is implicit; the factual note that nothing guards it is what the review asked for and is enough on its own. Claude-Session: https://claude.ai/code/session_013DVEGTARyLzqxxzvzoVXEA
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a08ed61e41
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
State the per-skill compaction relationship as an approximation, not a guarantee. A byte count never proves a token count: bytes per token is content-dependent, so an 8000-byte body is ~2000 tokens at ordinary prose density but the ratchet cannot prove a 5,000-token bound. Breaching 5,000 tokens inside 8000 bytes would take ~1.6 bytes per token, which Markdown prose does not reach, so the margin is wide -- but it is a margin, and a token-dense body erodes it. Took the review's documentation branch rather than adding a token-count guard. Corrected at all four sites carrying the claim: the spec section, the test header, size-driven-skill-restructure.md, and the convention doc. Claude-Session: https://claude.ai/code/session_013DVEGTARyLzqxxzvzoVXEA
1 participant
Summary
The 8,000-byte
SKILL.mdratchet is now attributed to the host that actually imposes it, and a second host bound we had never modeled is recorded beside it. The ratchet value does not change and no test assertion changes — what changes is whether the number can be reasoned about instead of obeyed.A belief had settled that 8,000 bytes is a requirement of the Agent Plugins standard. It is not. Checked against primary sources on 2026-08-21:
MAX_SKILL_PROMPT_BYTES,codex-rs/ext/skills/src/render.rs:19, applying only on the Agent Plugins path.The bound we had missed
Claude Code auto-compaction re-attaches each invoked skill keeping only its first 5,000 tokens, within a 25,000-token combined budget filled from the most recently invoked — so older skills drop entirely. This applies on every Claude Code path regardless of manifest, which means it applies to us where the Codex bound does not (our root manifest stays schema-less by design).
That bound has two halves, and the ratchet fully covers neither — Codex caught both gaps in review (threads resolved), fixed in d7f24ee and a823da2:
tests/codex-skill-prompt-budget.test.tssizes eachSKILL.mdindependently.Measured while fixing: at ~4 bytes/token an at-cap body is ~2,000 tokens, so ~12 fully compliant skills exhaust the combined budget. A 7-skill
lfgchain already sits at ~14,500 tokens (58%); all 33 skills total ~55,300 (2.2x over). Real headroom, not a theoretical concern.No gate was added for either. A chain guard is only expressible for a declared chain and would still say nothing about arbitrary sessions; a token guard would need a pinned tokenizer this repo doesn't ship, trading an honest approximation for a fragile one that looks authoritative. The docs and the test header now state both limits plainly instead of implying coverage.
The operational half matters more than either number: both truncations keep the start of the file, silently. A skill can pass every mechanical gate, sit comfortably under the line, and still lose its stop conditions because routing detail was placed above the boundary rule. No test catches that.
Corrections
Two claims this verification falsified, both instances of the same failure mode:
tests/real-plugin-conversion.test.tsasserted the body constant "is not in the Codex source." True when written; upstream has since added it.docs/solutions/integrations/agent-plugins-schema-is-a-host-routing-switch.mdcitedcodex-rs/core-skills/src/{lib,injection}.rsfor the constant. That crate no longer exists andbuild_skill_injectionsis gone.Negative claims about someone else's codebase rot silently, where a positive claim breaks loudly when a path moves. Both now carry a
file:lineand a verification date, and that lesson is the new learning indocs/solutions/conventions/.Why three different 8,000s keep regenerating this
Codex
MAX_SKILL_PROMPT_BYTES(body truncation), CodexDEFAULT_SKILL_METADATA_CHAR_BUDGET(skills listing budget — same file, same number), and Claude Code's historical listing fallback. Any contributor who greps one of them "confirms" the folklore, which is why all three are now enumerated with their distinct scopes.Validation
bun run test— 3,539 pass, 0 fail.bun run release:validate— in sync. Every commit here is docs- and comment-only: no assertion, threshold, orOVER_BUDGETmembership is touched.Related
Related: #1412, #1411. Upstream: openai/codex#37027 introduced the bound; openai/codex#37463 is open and proposes widening it to all skill sources, which is recorded as a re-verify trigger.
Security Disclosure
No security-relevant changes. Documentation and code comments only; no shell/exec, path or input handling, converter output, credential handling, permissions, or dependency changes.
Agent Disclosure
Claude Code · claude-opus-5[1m]