agent-server: OH_PERSISTENCE_DIR doesn't fully isolate state — LLM/agent profile stores ignore it

[neubig]

Summary

Starting agent-server with OH_PERSISTENCE_DIR=/tmp/foo doesn't actually isolate state — the materialized <persistence_dir>/settings.json is silently seeded with values pulled from ~/.openhands/profiles/, which OH_PERSISTENCE_DIR has no influence over.

The result: callers (e.g. agent-canvas CI / dev / test rigs) can't get a known-empty agent-server, and stale LLM endpoints from the host user's profiles leak into supposedly-isolated instances.

Repro

rm -rf /tmp/fresh-state && mkdir -p /tmp/fresh-state

env -i \
  PATH="$HOME/.local/bin:/usr/local/bin:/usr/bin:/bin" HOME="$HOME" \
  PYTHONUTF8=1 \
  OH_PERSISTENCE_DIR=/tmp/fresh-state \
  OH_SESSION_API_KEYS_0="$(openssl rand -hex 32)" \
  uvx --from openhands-agent-server==1.28.1 \
      --with openhands-sdk==1.28.1 \
      --with openhands-tools==1.28.1 \
      --with openhands-workspace==1.28.1 \
      agent-server --host 127.0.0.1 --port 8050 &

sleep 8
curl -sS -H "X-Session-API-Key: $OH_SESSION_API_KEYS_0" \
  http://127.0.0.1:8050/api/settings \
| jq '.agent_settings.llm | {model, base_url, api_key}'

Expected

A brand-new OH_PERSISTENCE_DIR should yield default values:

{ "model": "gpt-5.5", "base_url": null, "api_key": null }

Actual

agent_settings.llm is populated from whichever LLM profile happens to live in ~/.openhands/profiles/ — values that have nothing to do with the configured persistence dir:

{
  "model": "openai/Qwen3.6-35B-A3B-GGUF",
  "base_url": "https://statusquo-amd-lemonade.ngrok-free.dev/v1",
  "api_key": null
}

/tmp/fresh-state/settings.json is then written with those leaked values.

Root cause

LLMProfileStore and AgentProfileStore default their base_dir to Path.home() / ".openhands" / ...:

• openhands-sdk/openhands/sdk/llm/llm_profile_store.py:26

  _DEFAULT_PROFILE_DIR: Final[Path] = Path.home() / ".openhands" / "profiles"

• openhands-sdk/openhands/sdk/profiles/agent_profile_store.py:28

  _DEFAULT_PROFILE_DIR: Final[Path] = Path.home() / ".openhands" / "agent-profiles"

The stores accept a base_dir override, but all 21 construction sites inside openhands-agent-server use the no-arg form LLMProfileStore() / AgentProfileStore(). So they never honor OH_PERSISTENCE_DIR:

$ grep -rn 'LLMProfileStore()\|AgentProfileStore()' openhands-agent-server --include='*.py' | wc -l
21

Compare to FileSettingsStore / FileSecretsStore / FileWorkspacesStore, which go through _get_persistence_dir() (which does read OH_PERSISTENCE_DIR) — openhands-agent-server/openhands/agent_server/persistence/store.py:693.

Suggested fix

Introduce profile-store accessors next to get_settings_store() / get_secrets_store() in persistence/store.py:

_llm_profile_store: LLMProfileStore | None = None
_agent_profile_store: AgentProfileStore | None = None

def get_llm_profile_store(config: Config | None = None) -> LLMProfileStore:
    global _llm_profile_store
    if _llm_profile_store is None:
        with _store_lock:
            if _llm_profile_store is None:
                _llm_profile_store = LLMProfileStore(
                    base_dir=_get_persistence_dir(config) / "profiles",
                )
    return _llm_profile_store

def get_agent_profile_store(config: Config | None = None) -> AgentProfileStore:
    # symmetrical ...

Then replace the 21 default-constructed call sites with the accessor. Extend reset_stores() to clear both.

Add a regression test that asserts:

• with OH_PERSISTENCE_DIR=$TMP set, LLMProfileStore.base_dir == $TMP/profiles
• agent_settings.llm.model after server start matches the default, not whatever lives in ~/.openhands/profiles/.

Context

Discovered while testing agent-canvas PR OpenHands/agent-canvas#1389. For the onboarding-flow QA we needed a known-empty agent-server; the leak hid the actual onboarding state-machine bug we were trying to test.

A duplicate Linear ticket was filed at AGE-1894 before realizing code bugs should be filed on GitHub — cancelling that.

This issue was filed by an AI agent (OpenHands) on behalf of @gneubig.

[simonrosenberg]

Cross-link (2026-06-21) — this is a regression in the AgentProfile epic's own Phase-2 substrate (epic #3713). The leaking AgentProfileStore is the store introduced by #3716 (PR #3775), and the no-arg construction sites are in the agent-server wiring from #3719/#3720. Two points worth recording:

1. The latent leak SHIPPED in v1.29.0. The leaking default-constructed call sites are in the released agent-server; only the fix (PR fix(agent_server): honor OH_PERSISTENCE_DIR in profile stores (#3815) #3816) is post-v1.29.0 and unreleased. A v1.29.1 patch is the way to get the fix to canvas's dev/CI/QA rigs (this is what canvas Add instance id input to run-eval dispatch #1389 needed — a known-empty agent-server).

2. PR fix(agent_server): honor OH_PERSISTENCE_DIR in profile stores (#3815) #3816 needs a code push, not just a human merge. It is MERGEABLE/BLOCKED, but the pre-commit check is red for real: ruff-format reformatted a file and pyright reports 2 errors in the new tests/agent_server/test_profile_store_persistence_dir.py (generator return-type annotations at ~L26/L33). The Validate PR description gate is also failing (the usual HUMAN/AGENT-marker requirement). Fix the type annotations + run ruff format/pre-commit run -a, push, satisfy the description gate, then merge.

Coordination: the model-router work (SDK #3744 MetaProfileStore) reads ~/.openhands/meta-profiles/ directly with the same home-dir-default pattern — it will reintroduce this exact leak unless it adopts the get_*_store() accessor from this PR. Land #3816 first.