Skip to content

fix(contributor-check): retry transient GitHub API failures within a bound - #49

Merged
imran-siddique merged 1 commit into
agentrust-io:mainfrom
Mayur021:fix/bounded-retry-in-api
Oct 2, 2026
Merged

imran-siddique merged 1 commit into
agentrust-io:mainfrom
Mayur021:fix/bounded-retry-in-api

Conversation

@Mayur021

@Mayur021 Mayur021 commented Oct 2, 2026

Copy link
Copy Markdown
Member

Closes item 1 of #27.

What was wrong

_api already looped three times, but only a 403 ever continued the loop. A 5xx raised on the first attempt, and URLError was not caught at all, which is the case that matters: urlopen(req, timeout=15) raises URLError on a socket timeout, a DNS failure, a reset or a TLS abort, so that one propagated straight out of the loop. URLError was not even imported.

_count_maintainer_merged alone walks up to ten PR-detail calls for a single candidate, paced half a second apart, and the run makes more calls besides. One blip anywhere in that walk ended the whole check.

Since #46 that no longer mislabels anyone, which was the needs-review:UNKNOWN symptom reported in #27. It now surfaces as CheckExecutionError and the action exits non-zero, correctly. But a single transient network blip still fails a contributor check that would have succeeded on a retry, and that is item 1.

What this does

Extends the existing loop to cover the transient cases, with full-jitter exponential backoff capped at 60s, and keeps everything else reporting as itself:

Response Behaviour
404 return None on the first attempt, never retried
403 + Retry-After honoured, clamped to 5-60s, as before
403 after the bound raises, and now says it was a rate limit
5xx backs off, then raises as that status (was: raised immediately)
URLError backs off, then raises as a network error (was: uncaught)
any other status raises on the first attempt, unchanged

The attempt bound stays at three. 404 moved above the 403 branch so the "an answer, not a failure" case is the first thing the handler decides; the codes are distinct so that is a readability change, not a behavioural one.

On your constraint that a genuine 404 or a rate-limit answer still reports as itself: 404 never enters the retry path, and each exhausted path now prints a line naming its own cause before re-raising. That last part is load-bearing rather than cosmetic. _run_check in contributor_check_action.py takes stderr.strip().splitlines()[-1] as the CheckExecutionError detail, so the last stderr line is what a maintainer reads. Upstream's 403 path raised silently after its third attempt, which meant an exhausted rate limit reached the maintainer as a bare exited 1, with nothing to say whether to wait or re-run.

Vendored-copy integrity

Taking your go-ahead for a declared second modification. contributor_check.py moves out of the checksum table in vendor/agt/README.md and out of the sha256sum -c list in the workflow. The other three files stay hash-pinned and the integrity job still fails if they move.

README.md now records both modifications with a behaviour table, and the "Updating" section no longer reads as a sync point against upstream. Per your note there is no upstream offer to AGT: that copy is owned here now.

Tests

24 pass, 8 new in tests/test_api_retry.py. Both outcomes of each transient path, the 404 short-circuit, the attempt bound on every exhausted path, and the backoff cap.

They were written against a mutated source rather than just run green, because a retry test that still passes with the retry removed is worth nothing. Eight mutations, each caught by at least one test:

Mutation Tests failing
5xx retry disabled 2
URLError retry disabled 2
404 early return removed 1
retry bound widened 3
backoff cap removed 1
rate-limit cause line removed 1
5xx cause line removed 1
network cause line removed 1

Four of those tests were weak on the first pass and the sweep is what showed it. The three cause-line tests asserted a substring that the per-attempt "retrying in Xs..." messages also contain, so they passed with the final line deleted. The backoff test only evaluated attempts 0-2, where the raw curve is 1-4s and a 60s cap cannot bind either way.

One note for the record. Widening the range on the loop changes no observable behaviour, because the decision to retry is governed by last = _RETRY_MAX_ATTEMPTS - 1 and every branch at attempt == last raises, so the extra iterations are unreachable. That mutant survives by being equivalent, not by being uncovered. The bound that the tests do pin is last.

Items 2 and 3 of #27 are untouched here.

…bound

`_api` already looped three times, but only a 403 continued the loop. A
5xx raised on the first attempt, and `URLError` was not caught at all, so
the `timeout=15` socket timeout, a DNS failure, a reset or a TLS abort
propagated straight out of the loop. `URLError` was not even imported.

`_count_maintainer_merged` alone walks up to ten PR-detail calls for one
candidate, paced half a second apart, and the run makes more calls
besides. One blip anywhere in that walk ended the whole check. Since agentrust-io#46
that no longer mislabels anyone, which was the `needs-review:UNKNOWN`
symptom in agentrust-io#27; it fails the action instead. But a transient blip still
ends a check a retry would have passed.

The existing loop now covers the transient cases, with full-jitter
exponential backoff capped at 60s, and the bound stays at three attempts:

- 404 returns None on the first attempt and is never retried. It is an
  answer, not a failure. Moved above the 403 branch for readability; the
  codes are distinct, so behaviour is unchanged.
- 403 with Retry-After is honoured, clamped to 5-60s, as before. If it
  outlasts the bound it raises, and now says it was a rate limit.
- 5xx and URLError back off, then raise as themselves.
- Every other status raises on the first attempt, unchanged.

Each exhausted path prints a line naming its own cause before re-raising.
`_run_check` in contributor_check_action.py takes the last stderr line as
the CheckExecutionError detail, and upstream's 403 path raised silently,
so an exhausted rate limit used to reach a maintainer as a bare `exited
1` with nothing to say whether to wait or re-run.

This is a second declared local modification to the vendored copy, so
contributor_check.py moves out of the checksum table and the three
remaining vendored files stay hash-pinned in CI. vendor/agt/README.md
records both modifications and drops the upstream sync framing: AGT is
not being invested in and this copy is owned here.

24 tests pass. The eight added cover both outcomes of each transient
path, the 404 short-circuit, the attempt bound on every exhausted path,
and the backoff cap. Each was checked against a mutated source to confirm
it fails when the behaviour it names is removed.

Refs: agentrust-io#27
Signed-off-by: Mayur021 <mayur.agnihotri0021@gmail.com>
@Mayur021

Mayur021 commented Oct 2, 2026

Copy link
Copy Markdown
Member Author

Flagging the two red checks, because neither comes from this branch.

Policy: Awaiting maintainer review is the gate doing its job.

Contributor Reputation Check / check is the one worth a look. It runs on pull_request_target, and its checkout step logs 79cf0e4, so it executed the copy on main rather than anything in this PR. The error is contributor_check.py exited 1: urllib.error.HTTPError: HTTP Error 422.

It also looks like the first time that path has run on a real pull request since vendoring. The run on #46 predates the vendored copy landing on main, and the job's if: skips the maintainer account, so #36, #44 and #48 never exercised it.

What I cannot tell you is which call returned the 422, because the error carries no endpoint. I ran the same profile check locally against the same username and target repo and every call came back 200, so it does not reproduce with a user token, which points at the Actions identity rather than the query shape.

I have kept it out of this PR rather than widening the scope. The one-line improvement is that _api should name the path it was calling when it raises, which would have turned that log line into a diagnosis. Happy to open a separate issue, or to fold it into item 2 if you would rather keep everything on #27.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants