fix(server): classify extra_models_dir models as embedding or reranking instead of always classifying as LLM - #3333
Open
anditherobot wants to merge 8 commits into
Conversation
added 3 commits
August 24, 2026 19:20
…ys LLM These files always showed up as LLMs before. The name now decides whether a model is an embedding, reranking, or chat model, and the filename takes precedence over the folder it sits in. Only one mode label is applied, since a model deploys in exactly one mode. Also fixes a second bug found while testing: the default chat label was set before the name was read, and chat is checked ahead of every other mode, so the name check alone would have changed nothing. Fixes lemonade-sdk#1667
The old tests only checked the naming logic on its own. These build real files in a temp folder and run discovery against them, covering root files, split variants, folder models, the folder and filename conflict, and mmproj. This is the test that caught the label ordering bug. discover_extra_models() was made public so it can be called directly.
The docs said imported files are always LLMs, which is no longer true. This updates them and writes down the limits of matching on names, along with the lemonade pull --label workaround for files the name gets wrong.
github-actions
Bot
added
bug
added 5 commits
August 25, 2026 05:02
Removes comments that restated the code and shortens the rest.
The section ran longer than every other one in the spec. Same rules, said once. The workaround now points at user_models.json, which is the path that actually works for a local file.
Two claims did not match the code. Matching is not per-file for a folder kept as one model: that folder takes one mode from its primary file, and its other files are ignored, which the spec already says elsewhere about splitting. And user_models.json does not correct a wrong label, since the entry it adds is a separate user.* model that sits beside the extra.* one. Renaming the file is the fix that actually changes what discovery reads.
BGE is an acronym for BAAI General Embedding, so names such as bge-small-en-v1.5 do not contain the existing embed substring and were imported as chat models. Check the bge- family after rerank so bge-reranker-v2-m3 remains a reranker. Families that declare neither mode, such as gte- and e5-, retain the documented chat fallback.
The function was made public so the test could reach it; a friend seam keeps it off the ModelManager API, matching RoutingHelperTestHook. Adds the case the spec now describes: a folder whose files do not all belong to a named variant stays one model, and its primary file decides the mode for the whole folder.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1667.
Reported bug
Every GGUF in
extra_models_dirwas registered as a chat LLM in all three discovery layouts. Embedding and reranking models were therefore started in the wrong mode, and their endpoints did not work.New behavior
The name now decides the mode, and the filename takes precedence over the folder containing it.
nomic-embed-text-v2.ggufembedbge-small-en-v1.5.ggufbge-reranker-v2.ggufreranktakes precedence overbge-Qwen3-8B-Instruct.ggufembed-models/bge-reranker-v2.ggufreranker-models/nomic-embed-text.ggufmy-reranker-model/model.ggufembed-rerank-model.ggufreranktakes precedencegemma-vision/model.ggufplusmmproj-*mmprojattaches visionOnly one mode label is applied because a model runs in exactly one mode.
illegal_deployment_labels()rejects conflicting deployment modes that would advertise an endpoint the process was not started to serve.Additional bug
While testing this behavior, I found that
init_extra_model_info()applied the defaultchatlabel before reading the filename. Becausefind_deployment_mode()checkschatbefore the other modes, the model remained an LLM regardless of its name.The default is now applied only after label inference at all three discovery call sites.
Limitations
Detection depends on the model name. BGE models are recognized through the
bge-family prefix, while Nomic embedding models are recognized throughembedin names such asnomic-embed-text-v1.5.Names that declare neither mode, such as
all-MiniLM-*,gte-*, ande5-*, still default to chat. A chat model whose name containsembedorrerankcan also be classified incorrectly. Renaming the file is the direct fix because discovery reads only the name.Registering the same file in
user_models.jsonwith an explicitlabelsarray creates a separate, correctly labeleduser.*model. It does not change the automatically discoveredextra.*model.Tests
Windows, Visual Studio 2022, Release:
ctest -L cpp-ci -C Release: 53/53 passedRelated change
The
/pullpreview now uses the same classifier as model registration.AI-assisted contribution
Used AI tools for research and implementation guidance. Reviewed to make sure only the changes needed were made.