Cognee with vLLM and HuggingFace embedding model

[LStromann]

=moved your question here <@402399659697373187>

Hi everyone. I have recently test Cognee with vLLM and HuggingFace embedding model, here is my .env file:

LLM_PROVIDER=custom
LLM_MODEL=hosted_vllm/gemma-3-12b
LLM_ENDPOINT=https://vllm_endpoint/v1
LLM_API_KEY=.

EMBEDDING_PROVIDER=custom
EMBEDDING_MODEL=intfloat/multilingual-e5-base
EMBEDDING_ENDPOINT=http://embedding_endpoint/v1
EMBEDDING_DIMENSIONS=768

Here is the error:
Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=gemma-3-12b\n Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers/n\n\n

Is this the source code problem or it is my own .env setup problem?

This discussion was automatically pulled from Discord.

[LStromann]

i think what <@714696027176960101> shared earlier here is a good example

LLM_PROVIDER="openai" LLM_MODEL="openai/<model name>" #must start with openai/ LLM_ENDPOINT="https://vllm-host/v1" #must end with /v1 LLM_API_KEY="<key>"

let us know if it works! 🙂

[LStromann]

Hi, thank you for your support.

I have tried your method, and unfortunately, it is still return the same old problem

This is the response of my /v1/models API Endpoint

{
"object": "list",
"data": [
{
"id": "gemma-3-12b",
"object": "model",
"created": 1764237193,
"owned_by": "vllm",
"root": "/data/hf/gemma-3-12b-it",
"parent": null,
"max_model_len": 32768,
"permission": [
{
"id": "modelperm-ba4a02a5c9e9453d91ccc60fc319786b",
"object": "model_permission",
"created": 1764237193,
"allow_create_engine": false,
"allow_sampling": true,
"allow_logprobs": true,
"allow_search_indices": false,
"allow_view": true,
"allow_fine_tuning": false,
"organization": "*",
"group": null,
"is_blocking": false
}
]
}
]
}

Am i missing something?

Other way that i have tried:

• provider: custom, model: <model_name> ---> error
• provider: custom, model: hosted_llm/<model_name> ---> error
• provider: custom, model: openai/<model_name> ---> error

[LStromann]

Best to check what the proper model name is on the LiteLLM side. When LLM_PROVIDER is set to "custom" we just forward requests with the configuration provided to LiteLLM

You can check their docs here: https://docs.litellm.ai/docs/providers/vllm

[LStromann]

it might be something like LLM_MODEL=hosted_vllm/google/gemma-3-12b-it

[LStromann]

it still not working after all. maybe i will switch to use litellm directly, thank you all for your time

[LStromann]

I have re-checked my .env, so after using provider as "custom" and model as "hosted_vllm/<model_name>", its finally works !