fix: add timeout to FreeIPA http.Client to prevent server-side hangs - #2177
Open
SebTardif wants to merge 1 commit into
Open
fix: add timeout to FreeIPA http.Client to prevent server-side hangs#2177SebTardif wants to merge 1 commit into
SebTardif wants to merge 1 commit into
Conversation
The IPA IAM http.Client has no Timeout configured. If the FreeIPA server becomes unresponsive (accepts TCP connection but never responds), every goroutine servicing an S3 request that requires authentication hangs indefinitely in ipa.client.Do(). The existing retry logic (requestRetries = 3) and isRetryable() already handle net.Error timeouts, but without a Timeout on the client, timeout errors never fire for silently unresponsive servers. Add a 30-second timeout, consistent with how other http.Client instances in the codebase (s3log/webhook.go, s3event/webhook.go) are configured with explicit timeouts. Signed-off-by: Sebastien Tardif <sebtardif@ncf.ca>
Collaborator
|
This won't automatically run in continuous integration without approval. A member of the Versity organization must allow it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The IPA IAM
http.Clientatauth/iam_ipa.go:79has noTimeoutconfigured. If the FreeIPA server becomes unresponsive (accepts the TCP connection but never sends a response), every goroutine servicing an S3 request that requires IPA authentication hangs indefinitely inipa.client.Do().Since
login()is called before everyrpc()call (line 298), and each authenticated S3 request flows throughrpc(), an unresponsive FreeIPA server can exhaust all available goroutines over time.The existing retry logic (
requestRetries = 3) andisRetryable()already handlenet.Errortimeouts correctly (line 388), but without aTimeouton the client, timeout errors never fire for silently unresponsive servers. The retries only help with connection resets and EOF.Fix
Add
Timeout: 30 * time.Secondto thehttp.Client, consistent with how other clients in the codebase are configured:s3log/webhook.go:46:Timeout: 3 * time.Seconds3event/webhook.go:68:Timeout: 3 * time.Second30 seconds is appropriate for IPA RPC calls (which involve vault access and crypto operations) vs 1-3 seconds for fire-and-forget webhooks.
Origin
Introduced in #1005 (
ee315276, 2025-01-09). PR #1377 later added retry logic for IPA requests but did not add a client timeout.Related