Skip to content

refactor(commands): replace hand-rolled fetch-with-timeout blocks with helpers - #166

Merged
Rushaway merged 3 commits into
masterfrom
refactor/152-shared-fetch-helper
Sep 6, 2026
Merged

refactor(commands): replace hand-rolled fetch-with-timeout blocks with helpers#166
Rushaway merged 3 commits into
masterfrom
refactor/152-shared-fetch-helper

Conversation

@Rushaway

@Rushaway Rushaway commented Sep 5, 2026

Copy link
Copy Markdown
Member

Summary

  • Five near-identical blocks in Commands.py (WolframAlpha, UrbanDictionary, OpenWeather, WUnderground ×2) each opened their own ClientSession and wrapped both the request and the body read in asyncio.wait_for. aiohttp provides that natively via ClientTimeout, so the wait_for wrappers were redundant scaffolding repeated five times.
  • Added Utils.FetchJson / Utils.FetchText, which own the session and the timeout budget, and switched all five sites over.

Notes on scope and behavior

  • URLInfo.get_url_data deliberately keeps its own session. It needs response headers plus a bounded 64 KB read, so forcing it into the shared helper would mean a third helper shape used by exactly one caller. Its redundant wait_for pair is replaced with a ClientTimeout though, since that's the same underlying point.
  • FlareSolverr is no longer part of this pattern — the issue listed it, but fix(flaresolverr): reuses a pooled session instead of opening one per call #144 already moved it to a pooled session with its own timeout. Outdated by the time I got here.
  • The helpers let exceptions propagate rather than swallowing them, so the two WUnderground sites keep their existing try/except and their user-facing "Failed to retrieve data" message. Verified a timeout still surfaces as asyncio.TimeoutError — the exact type asyncio.wait_for raised — so those handlers behave identically.
  • Dropped the if not resp: return N guards, which were dead code: aiohttp.ClientResponse defines neither __bool__ nor __len__, so it's always truthy. Each site keeps its reachable if not data code.

Closes #152

Test plan

Ran the helpers against a live local aiohttp server:

  • FetchJson returns parsed JSON.
  • FetchJson(params=...) forwards query params correctly.
  • FetchText returns the body text.
  • A slow endpoint with timeout=0.5 raises asyncio.TimeoutError (same exception type as before, so existing handlers still catch it).
  • Removed the now-unused aiohttp import from Commands.py and asyncio from URLInfo.py (ruff F401 would otherwise fail CI).

🤖 Generated with Claude Code

…h helpers

Five near-identical blocks in Commands.py (WolframAlpha, UrbanDictionary,
OpenWeather and WUnderground twice) each opened their own ClientSession and
wrapped both the request and the body read in asyncio.wait_for. aiohttp
already provides that natively via ClientTimeout, so the wait_for wrappers
were redundant scaffolding repeated five times.

Added Utils.FetchJson/FetchText, which own the session and the timeout
budget, and switched all five call sites to them. URLInfo.get_url_data
needs response headers plus a bounded 64K read, so it does not fit the same
shape and keeps its own session -- but its redundant wait_for pair is
replaced with a ClientTimeout too.

The helpers let exceptions propagate rather than swallowing them, so the
two WUnderground call sites keep their existing try/except and their
user-facing failure message. A timeout still surfaces as
asyncio.TimeoutError, exactly as asyncio.wait_for did.

Also drops the `if not resp` guards, which were dead: aiohttp's
ClientResponse defines neither __bool__ nor __len__, so it is always truthy
and those branches could never run.

Bumps VERSION to 1.8.19.

Closes #152

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Dolly132 <109222243+Dolly132@users.noreply.github.com>
@Rushaway
Rushaway requested a review from Dolly132 September 5, 2026 13:17

@Rushaway Rushaway left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self-review: the thing I was most careful about is that the helper does NOT swallow exceptions -- if it did, WUnderground's try/except and its user-facing error message would silently stop firing. Verified against a live server that a timeout still raises asyncio.TimeoutError, the same type wait_for raised, so those handlers are unaffected. Also note I left URLInfo out of the shared helper on purpose (it needs headers + a capped read) and that FlareSolverr had already dropped out of this pattern in #144.

The previous commit used Utils.FetchJson/FetchText in Commands.py without
importing Utils, so all five converted commands would have raised NameError
at runtime. Caught by CI's ruff check (F821) -- my local ast.parse check
only validated syntax, not name resolution.

Also applies ruff format, which CI runs as `ruff format . --diff` in the
same step and which was failing on the wunderground call site.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Dolly132 <109222243+Dolly132@users.noreply.github.com>
Dolly132 added a commit to Dolly132/torchlight that referenced this pull request Sep 6, 2026

@Dolly132 Dolly132 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested, works fine as intended.

@Rushaway
Rushaway merged commit b97a263 into master Sep 6, 2026
2 checks passed
@Rushaway
Rushaway deleted the refactor/152-shared-fetch-helper branch September 6, 2026 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Fetch-with-timeout boilerplate is hand-rolled 7+ times across Commands.py/URLInfo.py/FlareSolverr.py

2 participants