Skip to content

Add --cli to two eval READMEs so their suites actually grade #943

Description

@justinmclean

Summary

The run commands documented in the workflow-security-audit and
license-compliance-audit eval READMEs omit --cli "claude -p", so the suites
execute but never score.

Background

Without --cli, the runner has no model to drive and prose or judge cases come
back MANUAL instead of PASS/FAIL. Anyone following these READMEs sees a
run that looks successful while grading nothing. Sibling READMEs such as
release-archive-sweep already include the flag, so this is drift, not a
convention.

Where to look

  • tools/skill-evals/evals/workflow-security-audit/README.md lines 11-24
  • tools/skill-evals/evals/license-compliance-audit/README.md lines 14-22
  • tools/skill-evals/evals/release-archive-sweep/README.md - a README that has
    it right; copy the shape.

Acceptance criteria

  • Both READMEs' run commands include --cli "claude -p".
  • The invocation form matches the working sibling READMEs.
  • The PR description shows one suite grading (not MANUAL) after the change.

Estimated effort

~30 minutes for someone new to the codebase.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions