[claude-code-user-docs-review] 🔍 Claude Code User Documentation Review - 2026-10-10

[github-actions[bot]]

Executive Summary

Reviewed gh-aw docs as a Claude Code user who does not use Copilot. The CLAUDE_CODE_OAUTH_TOKEN trap remains resolved (documented, not silent) for a 9th consecutive run. No critical blockers remain, but Claude Code users still hit real friction: gh aw init scaffolds a custom agent file and MCP wiring only for Copilot, example/smoke-test coverage for Claude trails Copilot by ~1.9x (especially enterprise-auth and ARM variants), and Claude's full setup is link-only rather than inline. Key finding: core engine parity (conformance tests, dynamic workflows, fallback-models) is solid — the gap is concentrated in advanced-config depth and first-run scaffolding, not basic functionality.

Severity Findings

Critical / Major / Minor (click to expand)

Critical Blockers: None this run.

Major Obstacles:

1. gh aw init scaffolding parity gap — only --engine copilot auto-generates a custom agent file and MCP wiring; other engines get a prose pointer with no template (docs/src/content/docs/setup/cli.md:107, :127-132, :131).
2. Example/smoke-test parity gap — Claude: 62 engine declarations vs Copilot: 117 (~1.9x) across .github/workflows/*.md. Copilot has dedicated Azure/OIDC auth-variant tests (smoke-copilot-aoai-entra.md, smoke-copilot-aoai-apikey.md) and an ARM runner test (smoke-copilot-arm.md) with no Claude analog.
3. Claude setup requires a doc-hop — docs/src/content/docs/setup/quick-start.mdx:153 defers "complete setup" entirely to an external /gh-aw/engines/claude/ page, while Copilot's two auth paths are fully inline in the same document (quick-start.mdx:131-141).
4. Web tool workaround asymmetry — Copilot's web-search/web-fetch restriction comes with an inline BYOK proxy workaround (engine.copilot-sdk: true, docs/src/content/docs/reference/tools.md:130); Claude's restriction (tools.md:128) is one sentence with no workaround guidance.

Minor Confusion:

1. tools.timeout default stated only for Claude/Codex (60s); Copilot/Gemini/Pi defaults unstated (reference/tools.md:239).
2. "agy (experimental)" engine named in cli.md:244 with no definition anywhere in the reviewed docs.
3. architecture.mdx is 833 lines but only ~5 lines reference specific engines — per-engine sandbox/isolation differences (e.g. does Claude Code run the same sandbox as Copilot?) are unaddressed.

Engine & Tool Matrix

Engine	Setup (inline?)	Examples (count)	Auth	Score
Copilot	Full inline, 2 auth paths (quick-start.mdx:131-141)	117 declarations; 10+ smoke-auth variants, ARM, SDK sub-agents	Org billing or COPILOT_GITHUB_TOKEN	Best documented
Claude	Short inline + link-only depth (quick-start.mdx:143-153)	62 declarations; conformance + dynamic-workflow present, no OIDC/ARM/model-tier variant	ANTHROPIC_API_KEY or Anthropic WIF; claude login token explicitly rejected	Functional, thinner depth
Codex	Short inline, no dedicated engine page linked	77 declarations	OPENAI_API_KEY/CODEX_API_KEY	Comparable to Claude
Custom	Generic MCP pointer only, no scaffold	0 standalone examples (only nested in shared/genaiscript.md)	N/A — bring your own	Thinnest

Tool classification (from reference/tools.md): most tools (edit, github, bash, cache-memory, playwright, mcp-servers, etc. — 11 of ~16 reviewed) are engine-generic. Engine-differentiated tools are web-fetch/web-search (Copilot rejects outright with a proxy workaround; Claude/Codex support natively with lighter docs) and tools.timeout (default only specified for Claude/Codex). Two entries (drive-memory, preview-gated) and one ambiguous (agy) round out the remainder.

Parity observation (engine-example-counter): Base-level conformance is solid — every engine, including Claude, has its own engine-conformance-<id>.md smoke test, and dynamic-workflow coverage is at parity (smoke-claude-dynamic.md vs smoke-copilot-dynamic-workflow.md). The gap is concentrated at the advanced-config layer: enterprise/OIDC auth, ARM runners, and SDK sub-agent orchestration examples exist for Copilot but not Claude, so a Claude Code user searching for an edge-case config example is statistically more likely to land on a Copilot example.

Auth Gaps

• quick-start.mdx:153 — Claude's full setup is entirely link-only (/gh-aw/engines/claude/), unlike Copilot's fully inline PAT-creation steps in the same file.
• cli.md:132 — MCP server registration for Claude/Codex/Gemini/Pi is generic and link-only ("register gh aw mcp-server in your own MCP host configuration"); no Claude-specific config snippet (e.g. no sample .mcp.json or claude mcp add invocation) appears in any of the six reviewed files.
• cli.md:131 — "Claude Code subagents" is named but never shown; zero inline example of what such a subagent file looks like.
• quick-start.mdx:151 / cli.md:232 — CLAUDE_CODE_OAUTH_TOKEN from claude login is explicitly rejected (now documented, resolving the prior critical finding), but no file explains what a Claude Code user coming from local claude CLI usage should do differently beyond "create an API key or use WIF."
• tools.md:130 vs :128 — Copilot's web-tool restriction gets a workaround; Claude's does not.

Recommended Actions

Priority 1:

• Add a worked inline example of a Claude Code subagent/MCP config file (even a minimal .mcp.json snippet) directly in cli.md near line 131, closing the biggest "I have no idea what to write" gap for Claude Code users running gh aw init.

Priority 2:

• Add at least one enterprise-auth (OIDC/WIF) smoke-test example for Claude mirroring smoke-copilot-aoai-entra.md, and inline the Claude "complete setup" steps in quick-start.mdx instead of deferring entirely to the engines page.

Priority 3:

• State tools.timeout defaults for all engines, not just Claude/Codex, and define "agy" on first mention in cli.md.

Generated by 📝 Claude Code User Documentation Review · claude · agent · 268.1 AIC · ⊞ 5.2K · ◷

• expires on Oct 11, 2026, 4:49 AM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-10-11T12:49:09.630Z.

Closed by Workflow