Skip to content

refactor(streaming): Use aiohttp instead of curl and use Flaresolverr to bypass bot checks - #124

Open
Dolly132 wants to merge 25 commits into
srcdslab:masterfrom
Dolly132:dolly-fix2
Open

refactor(streaming): Use aiohttp instead of curl and use Flaresolverr to bypass bot checks#124
Dolly132 wants to merge 25 commits into
srcdslab:masterfrom
Dolly132:dolly-fix2

Conversation

@Dolly132

@Dolly132 Dolly132 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

this PR focuses on changing the logic used in FFmpegAudioPlayer.
Previously, it was using a curl process to retrieves the bytes of a desired url, no matter whether it was local or from the internet.
however, this PR will not require any process other than ffmpeg itself to play local sounds, and a aiohttp for extern sounds.

Another change:
Recently, myinstants website added a cloudflare bot protection, so genuinely, before this PR, the torchlight bot could not resolve the bot protection, so it ended up not playing any sound from myinstants.
A fix was introduced to use FlareSolverr from https://github.com/Flaresolverr/Flaresolverr .
Added the required Host and Port in config for *FlareSolverr`.

Changelog:

  • Removed the --no-deps flag on PIP dependencies on production and dev environments. pip will not install any dependency required from the desired one to install.
  • Added aiohttp-socks and curl-cffi dependencies.
  • Added FlareSolverr.py module to handle everything related to FlareSolverr.
  • Local files are played internally from ffmpeg process alone now.
  • Got rid of curl process and used aiohttp client sessions instead.
  • Added better handling for youtube videos, (static duration instead of counting it while streaming it)
  • Some other minor changes related to the goal of this PR.

Note:
This is a huge PR, i tried my best to test. however there could be errors!

This PR also combines #120

**Summary**
Well, YouTube is so strict sometimes, so it blocks buffer from actually being sent to the ffmpeg process in time, and we can't control that, however, a 15 seconds timeout was set, youtube usually takes more than 5 seconds to send buffers in time.

**Why aiohttp?**
I had to get rid of curl because handling it is harder, aiohttp is way easier and more reliable for requests like YouTube and MyInstants.

- Solves srcdslab#119
- Now local files don't need a curl process to be made, it can be done inside ffmpeg itself
- Set the duration of the youtube video before playing it instead of calculating it during the stream.
@Dolly132
Dolly132 marked this pull request as draft September 3, 2026 13:41
@Dolly132
Dolly132 marked this pull request as ready for review September 3, 2026 14:21
@Rushaway

Rushaway commented Sep 3, 2026

Copy link
Copy Markdown
Member

Code review

Bugs

1. self.seconds is stuck at 0 for YouTube clips — FFmpegAudioPlayer.py:273
YouTubeSearch calls audio_clip.SetDuration(float(info['duration'])) then Play(). SetDuration sets self.seconds = duration and self.duration_set = True, but PlayURI() then runs self.seconds = 0.0 synchronously and wipes it. Since duration_set stays True, _read_stream skips self.seconds += seconds, so self.seconds remains 0.0 for the whole clip. Consequences: the if self.seconds > 0 termination/progress branch in _updater is dead (a stalled YouTube stream with no EOF is never stopped), and the Update callback reports unbounded wall-clock elapsed to anti-spam / advertiser progress.

2. curl_cffi worker thread deadlocks on early stop — FFmpegAudioPlayer.py:153
In the needs_cf_bypass path, sync_fetch_curl_cffi runs in the default executor and blocks on run_coroutine_threadsafe(queue.put(chunk), loop).result() with queue maxsize=10. When Stop() cancels the consumer while it is awaiting queue.get(), the queue fills and the worker thread blocks forever on future.result(). The cancelled coroutine then hits finally: await fetch_future, which never completes. Repeated !mi plays that are stopped mid-stream exhaust the executor and wedge the bot.

3. aiohttp.ClientSession is never closed — FFmpegAudioPlayer.py:196
On the plain HTTP(S) path (no proxy / http proxy), req_session = aiohttp.ClientSession(...) is assigned to self.session but only closed when connector is not None (socks). Stop() never touches it. A new FFmpegAudioPlayer is created per clip, so every YouTube play leaks a session + connector → "Unclosed client session" warnings and gradual fd/socket exhaustion.

4. Race on self.ffmpeg_processFFmpegAudioPlayer.py:163
Stop() sets self.ffmpeg_process = None. If that runs while the cf-bypass consumer loop dereferences self.ffmpeg_process.returncode / .stdin, the stream task raises AttributeError: 'NoneType'.

Dependencies / packaging

5. Lockfiles contradict pyproject — requirements.txt:65, requirements-dev.txt
Both pin aiohttp-socks==0.8.0, but pyproject.toml requires aiohttp-socks>=0.8.4. The lockfiles were hand-edited instead of regenerated with uv pip compile.

6. --no-deps removed from pip wheelDockerfile:36
The new deps pull transitive packages not present in requirements.txt (python-socks for aiohttp-socks; cffi/pycparser for curl_cffi), and certifi was removed even though requests (geoip2 / gtts / MyInstants) still needs it. Dropping --no-deps makes pip re-resolve those at build time → unpinned, non-reproducible images. Please re-run uv pip compile to regenerate a complete pinned set and restore --no-deps.

Docs

7. Invalid FlareSolverr example — README.md:144
"Host": "127.0.0.0" is not a loopback address (should be 127.0.0.1, as in config/config.json), and the snippet is not valid JSON (missing colon after "FlareSolverr").

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants