Skip to content

Litellm fallback ollama error #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ruslanmv merged 1 commit into from
Aug 10, 2026
Merged

Litellm fallback ollama error #57

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 39 additions & 2 deletions .env.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,15 +95,52 @@ GITPILOT_OLLAMA_MODEL=llama3
# =============================================================================
# OllaBridge / OllaBridge Cloud Configuration
# =============================================================================
# OllaBridge base URL (default: http://localhost:8000)
# OLLABRIDGE_BASE_URL=http://localhost:8000
# OllaBridge gateway base URL (default: http://localhost:11435)
# Must NOT be GitPilot's own address (http://localhost:8000) — that is this
# server, not an OllaBridge gateway.
# OLLABRIDGE_BASE_URL=http://localhost:11435

# Model to use with OllaBridge (default: qwen2.5:1.5b)
# GITPILOT_OLLABRIDGE_MODEL=qwen2.5:1.5b

# Optional: API key for authenticated endpoints
# OLLABRIDGE_API_KEY=

# =============================================================================
# Open WebUI Configuration
# =============================================================================
# Root URL of your Open WebUI instance (default: http://localhost:3000).
# Paste the root — GitPilot appends the OpenAI-compatible API path itself.
# OPENWEBUI_BASE_URL=http://localhost:3000

# Optional: only if your instance requires authentication.
# Create one under Settings -> Account -> API keys.
# OPENWEBUI_API_KEY=

# =============================================================================
# Custom OpenAI-compatible Endpoint
# =============================================================================
# Any gateway that speaks the OpenAI chat-completions API: self-hosted
# inference, a corporate model gateway, a proxy in front of several vendors.
# A full /chat/completions path works too — GitPilot trims it back to the root.
# GITPILOT_CUSTOM_BASE_URL=https://inference.example.com/v1
# GITPILOT_CUSTOM_API_KEY=
# GITPILOT_CUSTOM_MODEL=

# Extra request headers (attribution, routing) are configured in
# VS Code: GitPilot Settings -> AI Providers -> Custom endpoint.

# =============================================================================
# MCP Server Registry (Optional)
# =============================================================================
# Where "MCP Servers -> Search" looks for servers to attach. Any registry
# serving a JSON catalogue works; point this at your own to publish an
# internal catalogue of approved servers.
# GITPILOT_MATRIXHUB_URL=https://api.matrixhub.io

# Bearer token, if your registry requires one.
# GITPILOT_MATRIXHUB_TOKEN=

# =============================================================================
# Lite Mode (Optional — for small LLMs under 7B parameters)
# =============================================================================
Expand Down
126 changes: 126 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,132 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Runtime-aware onboarding** — `/api/status` now reports `workspace.runtime` (`cloud`/`local`); in a cloud workspace the empty state guides you to connect GitHub and pick a repository, while a local install offers Folder / Local Git paths with GitHub optional.
- **Account-first auth across split deployments** — portable `X-GitPilot-Session` token (cross-origin Vercel↔HF), a dedicated email-verification screen, a Settings → Account tab (update name, change password, delete account), and "GitHub not linked" now shows a calm Connect prompt instead of a repo-fetch error.

### Added
- **`make install-cli` and `make check-cli`** — point the `gitpilot` command at
the current checkout, and report which GitPilot it actually runs. `make
install` syncs `.venv` (which `make run` uses) but never touched PATH, so on
any machine that had once run `pip install gitcopilot`, `gitpilot serve` kept
starting a released wheel from `~/.local/lib/python3.11/site-packages/`
— a different version, different dependencies, none of your changes, and no
warning. The only visible difference was a version number inside a banner
(`v0.2.7` against `v0.2.8`), so a fix that was definitely in the tree could
appear not to work at all. `make install` now ends by naming the mismatch;
fixing it stays opt-in, because installing a command outside the project is
something to ask for rather than have done to you.

### Fixed
- **Reasoning models could not build an agent at all.** Every query from the
web app failed with `2 validation errors for Agent` — `Agent.llm` is a
validated pydantic field typed `str | BaseLLM`, and the reasoning wrapper was
a plain class that is neither. Composition over subclassing was a deliberate
choice (CrewAI's LLM class changes between versions) and was safe right up
until that field started validating; after which deepseek-r1, QwQ and every
other reasoning model failed at agent construction, before a single token was
generated. The wrapper now subclasses CrewAI's `BaseLLM` — an ABC whose one
abstract method, `call`, is the method the wrapper existed to intercept — and
is built lazily so importing it still does not drag CrewAI into every
process. Verified end to end against Ollama 0.12.9 and deepseek-r1: agent
builds, crew runs, no `<think>` leakage. Non-reasoning models are returned
unwrapped exactly as before. This module had no tests, which is how it
shipped; it has 17 now.
- **GitPilot's own log lines never reached the console.**
`uvicorn.run(log_level="info")` configures uvicorn's three loggers and
nothing else; GitPilot's records propagated to a root logger with no handler,
so Python fell back to `logging.lastResort` at WARNING. Every route decision,
provider call and timing was written and discarded, while uvicorn's own INFO
lines made logging look like it was working. GitPilot now logs at INFO by
default, with `gitpilot serve --log-level` / `GITPILOT_LOG_LEVEL` (the env
var also carries the choice into a `--reload` child). The root logger is left
alone — GitPilot is importable as a library and does not own the process's
logging.
- **Agent runs were silent on exactly the path that needed a trace.** The Lite
crews — the ones a small local model uses — were built with `verbose=False`,
so a multi-agent run produced no console output at all. They narrate now;
`GITPILOT_AGENT_VERBOSE=0` restores the quiet behaviour.
- **The workspace scan hid the project from the model.** The VS Code context
builder walked the tree depth-first and stopped at 300 entries, so the budget
went to whichever directory `readdir` returned first. Measured on this
repository: `extensions/` took 171 entries and `docs/` another 65, leaving
`pyproject.toml` and every file under `gitpilot/` — the application itself —
absent from the context. A model handed that listing cannot tell it is seeing
a fraction of a repository, so "explain this project's architecture" came back
describing something else entirely, confidently. The walk is breadth-first
now, files rank ahead of directories at each level, and the ignore list gained
the cache and build directories it was missing (`__pycache__`,
`.pytest_cache`, `.ruff_cache`, `.mypy_cache`, `.tox`, `target`, `vendor`,
`out` and others — the first two alone were taking 14 of the 300 slots). Same
repository after the change: `gitpilot/` 118 entries, `tests/` 61,
`pyproject.toml` and `README.md` present, no cache entries at all.
- **`Fallback to LiteLLM is not available` on a working Ollama.** Planning
goes through CrewAI, and CrewAI routes a call natively only when the
provider is in its `SUPPORTED_NATIVE_PROVIDERS` list — everything else
goes to its optional LiteLLM fallback, which raises outright when LiteLLM
is not installed. On CrewAI 1.6 that list is `openai, anthropic, claude,
azure, azure_openai, google, gemini, bedrock, aws`; Ollama joined it
around 1.10. So `POST /api/chat/plan` returned 500 on a configured,
reachable Ollama, and neither obvious spelling helped:
`provider="ollama"` names a provider CrewAI will not route, and
`model="ollama/llama3:8b"` is validated against CrewAI's model constants,
which no locally served model satisfies.

**Ollama, OllaBridge, Open WebUI and custom endpoints** were all affected
— the last three spelled it `model="openai/<model>"`, which fails the same
validation. All four now ask for `provider="openai"` explicitly and pass
the endpoint's own base URL and model id through untouched. This needs no
LiteLLM and works on old and new CrewAI alike.
- **Advice that pointed at the provider you were already using.** The agent
runtime hint told whoever hit this to "switch to Ollama, OllaBridge, Open
WebUI, OpenAI or a custom endpoint" — always the provider they were on. It
now reads the active provider and, for the endpoints that need no LiteLLM,
reports that the installed CrewAI is too old and gives the upgrade command.
- **The thinking animation stopped and restarted for a single question.** A
session with no GitHub repository has nothing for the multi-agent planner to
plan against, so the SSE stream closes at once and the client falls back to
`/api/chat/send`. That handover is deliberate — but it was announced as
`status_change("done")`, and "done" is what the UI reads to stop the spinner.
A request that had not started was reported finished, the animation cleared,
and the extension raised it again for the batch call. `agent_done` alone
terminates the stream now; the status belongs to work that happened.
- **Reasoning models on the direct path.** deepseek-r1 and QwQ think before
answering. Ollama (checked on 0.12.9 and 0.32.6) puts that in a sibling
`reasoning` field, while llama.cpp, vLLM and LM Studio inline it as
`<think>` tags. `build_llm()` has always stripped the tags, but the direct
provider path does not go through CrewAI and handled neither: inlined
reasoning was shown as the answer, and a model that spent its whole budget
thinking — leaving `content` empty with the substance in `reasoning` — was
reported as "returned an empty response", blaming a provider that had
answered. Tags are now stripped, `reasoning` is read when `content` is
empty, and a reply that is *only* reasoning is shown rather than discarded.
- **No way to tell which chat pipeline ran.** The agent path prints CrewAI's
full verbose trace to the server console and the direct path prints nothing,
because it has no agents — so comparing the web app (which plans against a
repo) with VS Code (which often has only a folder) looked like logs being
suppressed. Both branches now announce themselves, with model and elapsed
time. The extension's Output channel gained the client half: why a stream
was abandoned (unreachable / HTTP status / empty), the event tally, and the
size and duration of the batch call that followed.
- **VS Code timed out on requests the backend was completing.** Chat, plan and
execute took the deadline meant for an ordinary HTTP request — 20s — while
a local Ollama answering with repository context routinely needs 20-60s.
The web app allowed five minutes for the identical call, so the same
backend appeared to work there and fail in VS Code
(`POST /api/chat/send took 21.26s (status=200)` against
`timed out after 20000ms`). A timeout also carries no HTTP status, so it
escaped the non-retryable-status guard and the default two retries fired:
three runs of a call the server was still executing, queued behind each
other, each slower than the last. `/api/chat/send` appends to the session
before returning, so duplicates that landed wrote the exchange to history
twice. These calls now take a five-minute deadline matching the web app,
are never retried, and report a timeout in terms of the model rather than
the elapsed milliseconds. New setting `gitpilot.llmTimeoutSeconds`
(default 300, minimum 30) for machines that need longer.
- **VS Code discarded the server's explanation.** `ErrorTranslator` chose its
text from the HTTP status alone, so a 503 whose body named the provider,
the missing package and the command surfaced as "circuit breaker active" —
a breaker that was never involved. The server's `detail` now wins for
500/502/503/504, and the API client attaches it so there is something to
win with.

### Changed — `make run` now starts the MCP Context Forge stack by default

**Heads-up for upgraders.** Until this release, `make run` started only the
Expand Down
101 changes: 99 additions & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ DOCKER_COMPOSE := $(shell if command -v docker > /dev/null && docker compose ver
vercel vercel-build vercel-deploy \
build-container run-container stop-container logs-container clean-container publish-container \
extension-install extension-compile extension-package extension-publish publish-extension \
extension-test extension-dev extension-uninstall \
mcp mcp-down mcp-logs gateway gateway-down gateway-logs gateway-register \
install-mcp run-mcp run-all run-all-local stop-mcp logs-mcp sync-mcp uninstall-mcp \
fix-line-endings install-mcp-workflows register-mcp-servers \
Expand All @@ -42,6 +43,8 @@ help:
@echo " make install Install runtime deps + frontend + MCP stack"
@echo " make install-dev Install developer/test tooling"
@echo " make install-full Install runtime + dev/docs tooling + MCP stack"
@echo " make install-cli Point the 'gitpilot' command at this checkout"
@echo " make check-cli Report which GitPilot the 'gitpilot' command runs"
@echo " make uv-install Create/refresh Python env with runtime deps only"
@echo " make uv-install-dev Add developer/test tooling via uv"
@echo " make uv-install-docs Add documentation tooling via uv"
Expand Down Expand Up @@ -77,6 +80,9 @@ help:
@echo " make extension-install Install extension npm dependencies"
@echo " make extension-compile Compile TypeScript to JavaScript"
@echo " make extension-package Package extension into .vsix file"
@echo " make extension-test Run the extension test suites"
@echo " make extension-dev Package + install into your local VS Code"
@echo " make extension-uninstall Remove the locally installed extension"
@echo " make extension-publish Publish extension to VS Code Marketplace"
@echo " make publish-extension Alias for extension-publish"
@echo ""
Expand Down Expand Up @@ -113,6 +119,23 @@ install: uv-install frontend-install install-mcp install-matrixlab-soft
@echo " Run 'make startup' for the full GitPilot + MatrixLab + URL-fixup flow."
@echo " No Docker? Use 'make run-bare' to start GitPilot without MCP."
@echo " Optional: 'make install-dev' for test/lint/build tooling."
@bash scripts/check-cli-version.sh

## Point the `gitpilot` command at this checkout.
##
## `make install` prepares .venv and stops there — `make run` uses it, and
## installing a command outside the project is something to ask for rather
## than have done to you. On a machine that ever ran `pip install gitcopilot`,
## `gitpilot serve` keeps running that released wheel until this is run, which
## is why a fix that is definitely in the tree can appear not to work.
.PHONY: install-cli
install-cli:
@bash scripts/install-cli.sh

## Report which GitPilot the `gitpilot` command actually runs.
.PHONY: check-cli
check-cli:
@bash scripts/check-cli-version.sh

## Soft variant of install-matrixlab — warns and skips on docker-missing /
## daemon-down / port-held instead of aborting the parent installer. Wired
Expand Down Expand Up @@ -618,10 +641,54 @@ extension-package: extension-compile
@echo "✅ Extension packaged successfully!"
@echo ""
@echo "📁 VSIX file:"
@ls -lh $(EXTENSION_DIR)/*.vsix 2>/dev/null || echo " (no .vsix found)"
@ls -lh $$(ls -t $(EXTENSION_DIR)/*.vsix 2>/dev/null | head -1) 2>/dev/null || echo " (no .vsix found)"
@echo ""
@echo "Install locally with:"
@echo " code --install-extension $(EXTENSION_DIR)/gitpilot-vscode-*.vsix"
@echo " code --install-extension $$(ls -t $(EXTENSION_DIR)/*.vsix | head -1) --force"
@echo ""
@echo " --force is required: the version does not change between dev builds,"
@echo " so VS Code otherwise refuses to reinstall. Or just: make extension-dev"

## Run the extension's test suites (webview + panel host)
extension-test: extension-compile
@echo "🧪 Running VS Code extension tests..."
@cd $(EXTENSION_DIR) && node test/run.js

## Package and install the extension into your local VS Code, then reload
extension-dev: extension-package
@echo ""
@echo "🔎 Built from: $$(git rev-parse --short HEAD 2>/dev/null || echo 'unknown') $$(git diff --quiet 2>/dev/null && echo '(clean)' || echo '(with uncommitted changes)')"
@echo " If a change you expected is missing, check you have pulled it."
@echo ""
@$(MAKE) --no-print-directory version-check
@echo "💻 Installing into VS Code..."
@if command -v code >/dev/null 2>&1; then \
code --install-extension $$(ls -t $(EXTENSION_DIR)/*.vsix | head -1) --force && \
echo "" && \
echo "✅ Installed. Reload VS Code to pick it up:" && \
echo " Ctrl/Cmd+Shift+P → 'Developer: Reload Window'" && \
echo "" && \
echo "Then try it:" && \
echo " Ctrl/Cmd+Shift+P → 'GitPilot: Settings'"; \
else \
echo "⚠️ The 'code' command is not on your PATH."; \
echo ""; \
echo " VS Code → Ctrl/Cmd+Shift+P → Shell Command: Install code command in PATH"; \
echo ""; \
echo " Or install the .vsix by hand:"; \
echo " Extensions view → ... menu → Install from VSIX..."; \
echo " $$(ls -t $(EXTENSION_DIR)/*.vsix | head -1)"; \
exit 1; \
fi

## Remove the locally installed extension
extension-uninstall:
@if command -v code >/dev/null 2>&1; then \
code --uninstall-extension ruslanmv.gitpilot-vscode || true; \
echo "✅ Uninstalled. Reload VS Code to finish."; \
else \
echo "⚠️ The 'code' command is not on your PATH — remove it from the Extensions view."; \
fi

## Publish extension to VS Code Marketplace
extension-publish: extension-compile
Expand Down Expand Up @@ -954,3 +1021,33 @@ fix-matrixlab-url:
.PHONY: diagnose-matrixlab
diagnose-matrixlab:
@bash scripts/diagnose-matrixlab.sh


## Report the extension and backend versions, and flag a mismatch.
##
## `make extension-dev` builds the extension only. The Python backend is
## installed separately, and `gitpilot` is a console script that imports from
## site-packages rather than the directory you are standing in — so a fresh
## checkout and a stale install look identical until a feature misbehaves for
## reasons nothing on screen explains.
.PHONY: version-check
version-check:
@ext_ver=$$(grep -m1 '"version"' $(EXTENSION_DIR)/package.json | sed 's/.*"version": *"\([^"]*\)".*/\1/'); \
repo_ver=$$(grep -m1 '^version' pyproject.toml | sed 's/.*"\([^"]*\)".*/\1/'); \
inst_ver=$$(python3 -c "from importlib.metadata import version; print(version('gitcopilot'))" 2>/dev/null); \
echo "🧩 Versions"; \
echo " extension (built) $$ext_ver"; \
echo " backend (repo) $$repo_ver"; \
echo " backend (installed) $${inst_ver:-not installed}"; \
if [ -n "$$inst_ver" ] && [ "$$inst_ver" != "$$repo_ver" ]; then \
echo ""; \
echo " ⚠️ The installed backend is $$inst_ver but this checkout is $$repo_ver."; \
echo " Nothing here touches Python. Reinstall the backend:"; \
echo " pip install -e . --no-deps"; \
echo " Then check that 'gitpilot serve' prints v$$repo_ver."; \
elif [ -z "$$inst_ver" ]; then \
echo ""; \
echo " ⚠️ No installed gitcopilot found. The extension will have no backend."; \
echo " pip install -e . --no-deps"; \
fi
@echo ""
Loading
Loading