Skip to content

Fix three robustness defects in skill-and-tool-validator #941

Description

@justinmclean

Summary

Three small, independent defects in the validator and its test suite: a
docstring unit error, a test that aborts instead of skipping, and an unguarded
directory read.

Background

All three were raised in review, agreed as non-blocking, and merged without
being actioned. None changes behaviour on a healthy checkout; each degrades
badly in a specific environment.

Where to look

  • .../skill_and_tool_validator/__init__.py line 2488 - docstring says the
    content is shorter than _MIN_LICENSE_FILE_SIZE characters, but line 2498
    compares path.stat().st_size, which is bytes. Equivalent for ASCII; a
    file with a BOM or multi-byte content measures differently.
  • tools/skill-and-tool-validator/tests/test_validator.py line 1686 -
    test_real_repo_tool_python_files_all_have_headers calls find_repo_root()
    with no guard. In a shallow or out-of-tree run it aborts the session rather
    than skipping. Add a skipif.
  • .../__init__.py line 3285 - validate_eval_coverage calls .iterdir() with
    no try/except. A restrictive CI runner gets an unhandled PermissionError
    instead of a graceful violation. collect_tool_python_files shows the pattern.

Acceptance criteria

  • The docstring states bytes.
  • The repo-wide header test skips rather than errors, with a test proving it.
  • validate_eval_coverage handles OSError as collect_tool_python_files does.
  • uv run --directory tools/skill-and-tool-validator --group dev pytest passes.

Estimated effort

~2 hours for someone new to the codebase.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions