Summary
The run commands documented in the workflow-security-audit and
license-compliance-audit eval READMEs omit --cli "claude -p", so the suites
execute but never score.
Background
Without --cli, the runner has no model to drive and prose or judge cases come
back MANUAL instead of PASS/FAIL. Anyone following these READMEs sees a
run that looks successful while grading nothing. Sibling READMEs such as
release-archive-sweep already include the flag, so this is drift, not a
convention.
Where to look
tools/skill-evals/evals/workflow-security-audit/README.md lines 11-24
tools/skill-evals/evals/license-compliance-audit/README.md lines 14-22
tools/skill-evals/evals/release-archive-sweep/README.md - a README that has
it right; copy the shape.
Acceptance criteria
Estimated effort
~30 minutes for someone new to the codebase.
Summary
The run commands documented in the
workflow-security-auditandlicense-compliance-auditeval READMEs omit--cli "claude -p", so the suitesexecute but never score.
Background
Without
--cli, the runner has no model to drive and prose or judge cases comeback
MANUALinstead ofPASS/FAIL. Anyone following these READMEs sees arun that looks successful while grading nothing. Sibling READMEs such as
release-archive-sweepalready include the flag, so this is drift, not aconvention.
Where to look
tools/skill-evals/evals/workflow-security-audit/README.mdlines 11-24tools/skill-evals/evals/license-compliance-audit/README.mdlines 14-22tools/skill-evals/evals/release-archive-sweep/README.md- a README that hasit right; copy the shape.
Acceptance criteria
--cli "claude -p".MANUAL) after the change.Estimated effort
~30 minutes for someone new to the codebase.