feat(llm): use Lemonade's auto_evict policy instead of working around eviction (unlocked by v2026.39.1)

[kovtcharov]

Lemonade v2026.39.1 adds VRAM-aware auto-eviction (auto_evict, auto_evict_threshold_pct) and — separately — stops erasing the llama.cpp prompt cache when a model goes soft-idle. GAIA currently has no way to express eviction policy, so it works around eviction instead.

Why this matters here. Model eviction is a recurring source of GAIA bugs, not a hypothetical. RAGSDK scopes its embedder unload specifically so a global /unload can't evict the chat model (#1544); the non-streaming chat path had to regain a reload check after the embedder warm-up evicted the model (#1030); and ModelSlotBroker (src/gaia/daemon/broker.py) exists to serialise sidecar loads so they don't race-evict the slot. Each of those is GAIA compensating for eviction it cannot see or control.

Two concrete wins worth scoping:

• Let GAIA set an eviction policy rather than defending against the default, and pin the behaviour in tests/integration/test_lemonade_model_residency.py — which already exists to stop this being re-litigated from assumption.
• Prompt-cache survival across soft-idle should cut the ~100s cold-reload cost that test documents for agent switching.

Needs measurement before design: confirm on AMD GPU hardware what the thresholds actually do under GAIA's one-LLM-plus-embedder residency pattern.

Proposed from the Lemonade v2026.39.1 release scan; maintainer triage decides scope. cc @kovtcharov-amd

[github-actions]

In scope, and the plumbing is already in the right shape — every model load in GAIA funnels through one place, so an eviction-policy setting only has to be threaded once rather than sprinkled across the agents, RAG and sidecar paths.

Three things worth pinning down before design:

• Version gate first. GAIA installs Lemonade 11.8.1 today and still supports servers as far back as 10.2.0, so a policy field can't just be sent unconditionally — older servers would either ignore it or reject the load. That needs an explicit version check with a clear error, not a quiet drop.
• Don't retire the existing guards on the strength of the new knob. Auto-eviction triggers on a memory threshold, so under pressure it could evict the embedder that [Bug][P0]: RAG embedder load issues a global /unload that evicts the chat model → 100s+ cold reloads & eval timeout #1544 specifically protected. Keep the current guards until a test proves otherwise.
• The prompt-cache win is measurable right now — it needs no GAIA change, so measuring the agent-switch cost on AMD GPU hardware before and after would tell us whether the ~100s figure actually moves.

Tagging @kovtcharov-amd for the scope call on whether this rides a Lemonade version bump.

🔍 Technical details

Single chokepoint. LemonadeClient.load_model (src/gaia/llm/lemonade_client.py:3841) is where every load reaches Lemonade — UI preload, per-request pre-flight, RAG/code-index embedder warm-up, VLM client — and where the broker lease is already taken (#2248). auto_evict / auto_evict_threshold_pct belong alongside ctx_size and llamacpp_args there and in _load_model_leased.

Version floor. LEMONADE_VERSION = "11.8.1" and LEMONADE_MIN_VERSION = "10.2.0" (src/gaia/version.py:12,17). Note the issue cites v2026.39.1 — a different scheme from the semver GAIA pins; worth confirming which release maps to which before writing the gate. Per-profile floors already exist via INIT_PROFILES[...]["min_lemonade_version"], which is the natural precedent. CLAUDE.md's no-silent-fallbacks rule means a too-old server must raise, naming the version and the upgrade path.

Eviction-guard interaction. The guards the issue lists are src/gaia/rag/sdk.py:465-514 (scoped embedder unload, #1544) and src/gaia/daemon/broker.py (ModelSlotBroker). tests/integration/test_lemonade_model_residency.py:114 records that co-residency holds without the pinned flag, which is exactly the assumption a VRAM-threshold evictor could invalidate — the missing case is "LLM + embedder resident, memory pressure crosses the threshold, embedder survives." That test file is the right home for it.