[ericdelorefice]

• fixes Validate crashes or drops checks when a check reports more than one check-error #1866

---

Problem

Resource.validate() removes a misconfigured check with del checks[index] inside for index, check in enumerate(checks). It does this once per check-error the check yields, while still enumerating the list. So:

• a check that yields two check-errors and is the last check makes validate() raise IndexError: list assignment index out of range (the error reported in Exception generating Task Error occurs when validating a resource applying custum checks with wrong column's name related to these custom checks (using schema_sync) #1361),
• if another check follows it, that check is removed too and its errors never reach the report,
• the check that moves into a removed check's slot never has its validate_start() run, so its own check-error is not reported.

#1389 by @shashigharti fixed the #1361 crash on the v4 branch by collecting the indexes first; the v5 loop still deletes inside the loop.

Reproduction

from frictionless import Checklist, Resource

source = [["row", "name"], [1, "Alex"], [2, "John"]]
report = Resource(source).validate(Checklist.from_descriptor({"checks": [
    {"type": "deviated-value", "fieldName": "name", "average": "avg"},
    {"type": "forbidden-value", "fieldName": "row", "values": [2]},
]}))
print(report.flatten(["type", "note"]))
# 5.21.0: the two deviated-value check-errors only; the forbidden-value error for row 3 is missing
# without the forbidden-value check: IndexError

Reproduced with frictionless 5.21.0 from PyPI and on main (c08fa29), using built-in checks and inline data; nothing is mocked.

Change

Collect the indexes of checks that yield a check-error, and filter the list once the loop is done (the same approach as #1389). A failed check is still removed and still gets no validate_row() or validate_end() calls.

Intended behaviour change: validate() returns a report instead of raising in the first case. The report now carries every check-error and the errors of checks that follow a failed one. Every check's validate_start() now runs; before, the check after a removed one was skipped.

Testing

• Added five tests in frictionless/resource/__spec__/test_validate.py. Three fail on main and pass with the fix (the IndexError, the dropped next check, the unreported second check-error). Two controls pass on both: one check-error followed by a valid check, and a custom check that confirms a failed check gets no row or end calls.
• pytest frictionless: 2047 passed, 292 skipped with the fix; 2042 passed, 292 skipped on main (Python 3.10; mysql, postgresql, spss and visidata extras not installed here).
• ruff check frictionless passes, and pyright reports 0 errors on both changed files.