[FIX] Preserve configured LLM in non-batch fallback extractors - #535
Open
gillespied wants to merge 1 commit into
Open
[FIX] Preserve configured LLM in non-batch fallback extractors#535gillespied wants to merge 1 commit into
gillespied wants to merge 1 commit into
Conversation
BatchLLMPropositionExtractorSync and BatchTopicExtractorSync accept an
explicit llm and store it as self.llm. When a node set falls below
Bedrock's minimum batch size (100 records), _run_non_batch_extractor
falls back to LLMPropositionExtractor / TopicExtractor respectively,
but constructs them without passing llm=self.llm.
Both target classes default to GraphRAGConfig.extraction_llm when no
llm is given, which resolves to DEFAULT_EXTRACTION_MODEL
('us.anthropic.claude-sonnet-4-6'). Outside a region where that us.*
inference profile resolves, this raises:
ValidationException: The provided model identifier is invalid.
Even where it doesn't error, it silently uses a different
model/max_tokens/temperature than configured.
This is a different failure mode from awslabs#344 (LLMCache losing
region_name when BedrockConverse is pickled to a spawned worker) and
isn't fixed by awslabs#347's workaround of using global.* inference
profiles, since here the configured LLM is never passed to the
fallback extractor at all - the fallback ignores it outright.
Fix: pass llm=self.llm through to both fallback extractors.
Added regression tests for both classes asserting the fallback
extractor is constructed with the configured llm.
noel-improv
approved these changes
Sep 10, 2026
noel-improv
left a comment
Collaborator
There was a problem hiding this comment.
@gillespied thanks for your contribution so far these looks good. Both extractors take LLMCacheType, so self.llm is the right thing to pass here.
mykola-pereyma
approved these changes
Sep 10, 2026
mykola-pereyma
left a comment
Collaborator
There was a problem hiding this comment.
Thank you for the contribution! Clean, well-scoped fix — passing llm=self.llm into both non-batch fallbacks correctly preserves the configured inference profile, and the per-class regression tests pin the behavior nicely. Verified these are the only two fallback sinks (no non-sync siblings share the omission). LGTM.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
_run_non_batch_extractorinBatchLLMPropositionExtractorSyncandBatchTopicExtractorSyncbuilds its fallback extractor without passing the configuredllm, so it silently defaults toGraphRAGConfig.extraction_llm(us.anthropic.claude-sonnet-4-6) instead of whatever LLM was explicitly configured.Fixes #534.
Changes
batch_llm_proposition_extractor_sync.py: passllm=self.llmto the fallbackLLMPropositionExtractorbatch_topic_extractor_sync.py: passllm=self.llmto the fallbackTopicExtractorllmProblem
Related issue: #534
When a node set falls below Bedrock's minimum batch size (100 records), both batch extractors fall back to a non-batch extractor. That fallback ignores
self.llmentirely, so it usesGraphRAGConfig.extraction_llm(DEFAULT_EXTRACTION_MODEL = 'us.anthropic.claude-sonnet-4-6') regardless of what LLM was configured. Outside a region where thatus.*profile resolves, this raisesValidationException: The provided model identifier is invalid.We hit this ineu-west-2with an explicitly configuredeu.anthropic.claude-sonnet-4-6LLM.This is a different mechanism from #344/#347 (pickled
BedrockConverselosingregion_namein a spawned worker) — here the configured LLM is never passed to the fallback at all, so theglobal.*inference profile workaround for #344 doesn't apply.Testing
llm=<the configured llm>pytest) — rantests/unit/indexing/extract/test_batch_llm_proposition_extractor_sync.pyandtest_batch_topic_extractor_sync.py(11 passed), and the fulltests/unitsuite (2137 passed; one pre-existing, unrelated error intest_integ_dependency_compatibility.pycaused by nopipbinary in my venv, reproduced identically onmainwithout this change)Manually reproduced against a real
eu-west-2account: aBatchConfig-configuredeu.anthropic.claude-sonnet-4-6extraction, where oneProcessPoolExecutorworker's chunk count landed under 100 and hit this exact fallback path, raising theValidationExceptionquoted in #534.Checklist
llm=kwarg passthrough only