Enhanced retry to wait returned rate limit if present#179
Enhanced retry to wait returned rate limit if present#179m4t7w wants to merge 4 commits intoGewoonJaap:mainfrom
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces dynamic rate limit handling for the Gemini API by parsing quota reset times from error messages. It adds a utility to extract reset durations and updates the client to wait for these periods or trigger fallbacks for long resets. Feedback recommends using a named constant for the reset threshold and adding a radix to parseInt calls for improved code quality.
…and specified radix on parseInt
| if (dynamicDelay !== null && attempt < retryDelays.length) { | ||
| console.log(`Got ${response.status} for ${currentModel}, waiting for quota reset: ${dynamicDelay}ms`); | ||
| await new Promise((resolve) => setTimeout(resolve, dynamicDelay)); | ||
| continue; |
There was a problem hiding this comment.
Bug: The retry loop's final iteration incorrectly skips the API-provided dynamic delay because the loop condition attempt <= retryDelays.length is inconsistent with the internal check attempt < retryDelays.length.
Severity: MEDIUM
Suggested Fix
Align the loop's boundary with the internal checks. Change the for loop condition from attempt <= retryDelays.length to attempt < retryDelays.length. This ensures the loop does not execute an extra iteration where the delay logic would be skipped.
Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.
Location: src/gemini-client.ts#L633-L636
Potential issue: In `src/gemini-client.ts`, the retry loop is defined to run from
`attempt = 0` to `retryDelays.length` inclusive. However, the conditional logic inside
the loop, such as `if (dynamicDelay !== null && attempt < retryDelays.length)`, uses a
strict less-than comparison. On the final iteration, when `attempt` is equal to
`retryDelays.length`, this condition becomes false. As a result, the code skips the
`await` for the `dynamicDelay` returned by the API, immediately breaks from the loop,
and proceeds to model fallback or error throwing. This defeats the purpose of honoring
the API's rate limit reset time on the last retry attempt.
|
Thanks for the PR can you please addresses the issues from the Gemini bot? |
hi, the last 2 commits resolve both gemini and the first sentry bot suggestions. Last sentry bot message is wrong, |
Changes
NOTE: implementation uses a regex on error message because gemini api doesn't return a rate-limit header