add input delay option#169
Conversation
There was a problem hiding this comment.
Pull request overview
Adds an optional delay to pace URL submissions so the action is less likely to trigger rate-limiting (e.g., Cloudflare 429s) when archiving multiple URLs.
Changes:
- Parse a new
delayBetweenRequestsinput inInput. - Add a delay between processing each URL in the runner loop.
- Document the new input in the README.
Show a summary per file
| File | Description |
|---|---|
src/runner.ts |
Adds an inter-URL delay after each archive attempt. |
src/input.ts |
Reads the new delayBetweenRequests input from environment. |
README.md |
Documents the new input and its intended use. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (2)
README.md:80
- Minor formatting: there are two spaces after the period in “web requests. Use…”. Consider using a single space for consistency.
Time, in milliseconds, to wait between web requests. Use this to work with rate-limited hosting.
src/input.ts:31
Number(delayInput)can produceNaN(and accepts negative values). Because runner treats falsy values as “no delay”, invalid input would silently disable the feature. Consider validating that the parsed value is a finite, non-negative integer (and throwing a clear error when it isn’t).
const delayInput = this.getInput('delayBetweenRequests', { trimWhitespace: true });
this.delayBetweenRequests = delayInput ? Number(delayInput) : null;
- Files reviewed: 3/3 changed files
- Comments generated: 5
| const delayInput = this.getInput('delayBetweenRequests', { trimWhitespace: true }); | ||
| this.delayBetweenRequests = delayInput ? Number(delayInput) : null; | ||
|
|
There was a problem hiding this comment.
This adds a new GitHub Action input, but action.yml currently does not declare delayBetweenRequests. Undeclared inputs are treated as unexpected by the runner and typically won’t be passed through, so this option may be unusable in real workflows until it’s added to the action metadata.
This issue also appears on line 29 of the same file.
| if (input.delayBetweenRequests) { | ||
| await Promise.delay(input.delayBetweenRequests); | ||
| } |
There was a problem hiding this comment.
New behavior (delayBetweenRequests parsing + runner waiting between URLs) isn’t covered by tests. There are existing input/runner tests (Vitest), so please add coverage for: (1) valid delay parsing, (2) invalid delay rejecting with a helpful error, and (3) runner actually awaiting the delay (fake timers).
|
|
||
| ### `delayBetweenRequests` | ||
|
|
||
| Time, in milliseconds, to wait between web requests. Use this to work with rate-limited hosting. |
There was a problem hiding this comment.
Docs say “wait between web requests”, but the implementation delays only between top-level URL captures (and WayBack.save itself performs multiple requests/polls). Consider clarifying wording to avoid implying every internal request is delayed.
This issue also appears on line 80 of the same file.
| Time, in milliseconds, to wait between web requests. Use this to work with rate-limited hosting. | |
| Time, in milliseconds, to wait between capturing each input URL. Use this to work with rate-limited hosting. |
| if (input.delayBetweenRequests) { | ||
| await Promise.delay(input.delayBetweenRequests); | ||
| } |
There was a problem hiding this comment.
Promise.delay is not a standard API and this repo doesn't appear to include Bluebird (no existing Promise.delay usage/dependency). If a user sets delayBetweenRequests, this will throw at runtime. Use a real delay implementation (e.g., import { setTimeout as delay } from 'node:timers/promises' and await delay(ms)), or reuse a shared helper (similar to WayBack.sleep).
| readonly ifNotArchivedWithin = this.getInput('ifNotArchivedWithin', { | ||
| trimWhitespace: true, | ||
| }); | ||
| readonly delayBetweenRequests; |
There was a problem hiding this comment.
delayBetweenRequests is declared without a type, which becomes an implicit any under the repo's strict TS config (extends @tsconfig/strictest). Add an explicit type (e.g., number | null) so the project continues to typecheck cleanly.
| readonly delayBetweenRequests; | |
| readonly delayBetweenRequests: number | null; |
it tries to crawl my site a bit too fast and I end up with a bunch of HTTP 429s from cloudflare, adding something to space requests out a little bit to avoid that