fix: run sync client guardrails when an event loop is already running - #131
Open
Linxiushen wants to merge 1 commit into
Open
Linxiushen wants to merge 1 commit into
Linxiushen wants to merge 1 commit into
Conversation
GuardrailsOpenAI._run_stage_guardrails() and its GuardrailsAzureOpenAI counterpart bridged into the async guardrail runtime with asyncio.get_event_loop() + loop.run_until_complete(). Inside a running event loop (an async request handler, a notebook cell, a pytest-asyncio test) get_event_loop() returns the running loop and run_until_complete() raises "RuntimeError: This event loop is already running", so the first call that reaches a configured guardrail stage crashes. The plain openai.OpenAI client these classes are documented as drop-in replacements for does not have this problem. Add a module-level _run_coroutine_blocking() helper: when a loop is already running in the calling thread, run the coroutine with asyncio.run on a single worker thread (with the caller's contextvars copied), which is the pattern resources/chat and resources/responses already use; otherwise keep the existing get_event_loop / new_event_loop path unchanged. Both sync clients use the helper. Adds two tests that call the sync clients from inside a running loop, covering the return-value path and a tripwire raised across the thread boundary.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
GuardrailsOpenAI._run_stage_guardrails()and itsGuardrailsAzureOpenAIcounterpart bridge the synchronous API into the async guardrail runtime with:Inside a running event loop (an async FastAPI/Starlette handler, a notebook cell, a pytest-asyncio test, any coroutine that calls a sync helper)
get_event_loop()returns the running loop andrun_until_complete()raises:So the first
chat.completions.create(...)/responses.create(...)that reaches a configured guardrail stage crashes. The plainopenai.OpenAIclient these classes are documented as drop-in replacements for (README, docs/quickstart.md) does not have this problem: it just blocks on the HTTP call. Reproduced with the quickstart configuration plus one Keyword Filter guardrail, called from insideasyncio.run(main()); the same call againstopenai.OpenAIonly raises the expected connection error.Fix
Add a module-level
_run_coroutine_blocking()helper:asyncio.runon a single worker thread, with the caller'scontextvarscopied. This is the sameThreadPoolExecutor(max_workers=1)+copy_context()patternresources/chatandresources/responsesalready use;get_event_loop()/new_event_loop()path exactly as it was.Both sync clients call the helper instead of the inline loop handling. The change is purely additive: the path that worked before is untouched, so the existing
test_run_stage_guardrails_creates_event_loopstill exercises real code. I deliberately did not switch the no-loop path toasyncio.run, which would clear the thread's current loop on return and change behaviour for existing callers.Verification
tests/unit/test_client_sync.py, written like the existing_run_stage_guardrailstests:GuardrailsOpenAIreturning a result from inside a running loop, andGuardrailsAzureOpenAIsurfacing a tripwire across the thread boundary. Both fail onmainwith theRuntimeErrorabove and pass with the change.tests/unit/test_client_sync.py,test_client_async.py,test_resources_chat.py,test_resources_responses.py,test_streaming.py,test_runtime.py: 82 passed.tests/unitbefore/after: the set of failing tests is identical (all failures arepresidio/spacy-dependent PII tests that my offline environment stubs out; unrelated).suppress_tripwire, clean pass-through and empty-stage short circuit give byte-identical results and exception types with and without a running loop.ruff format --check/ruff checkclean;mypy src testsreports the same 8 pre-existing errors before and after, none inclient.py. Not run:make pyright(needs node, not available offline).This fix was developed with AI assistance (Claude); the change and the tests were reviewed and verified locally before submission.