scope: "user" knowledge is invisible outside its capture project (117 rows, ~half the corpus)

[bejranonda]

Summary

scope: "user" knowledge is invisible outside the project it was captured in. 117 rows are affected, and roughly half the corpus is unreachable from a BrainPlatform session.

Root cause

Both visibility helpers resolve on ownerProjectId alone and never consult the scope column:

// packages/core/src/scope-filter.ts:64-75  (buildKnowledgeWhere)
OR: [
  { ownerProjectId: projectId },
  { AND: [{ ownerProjectId: null }, { ownerUserId: userId }] },
]

// packages/core/src/scope-filter.ts:155-168 (buildRawProjectFilter — the
// pgvector path kra.ts:172 and oracle.ts actually use)
AND ("ownerProjectId" = $N OR ("ownerProjectId" IS NULL AND "ownerUserId" = $N+1))

A row with scope: "user" and a non-null ownerProjectId matches neither arm.

That state is not exotic — it is what brain_teach_knowledge produces by default. scope defaults to "user", and a project-bound session stamps its ownerProjectId. So teaching a user-level rule while working in project A silently scopes it to project A, contradicting what scope: "user" reads as.

Knowledge.visibility (Phase 4: private/project/org) is a separate field and does not govern this path.

Reproduction (controlled — identical prompt, two paths)

Prompt: add a new user-facing concept and glossary page to the External Brain webapp docs surface

Path	Result
brain_retrieve_knowledge (unscoped)	cmqpqemoh00000nmq352sy3js ranked #1 at 0.9009 similarity. Trigger: "Adding a user-facing concept or glossary page to the External Brain webapp". 100% success over 11 uses. ownerProjectId = Default
brain_start_session(projectName: "BrainPlatform")	That item is absent. Injected a tangential "no numbers in translation strings" reflex instead

The single best-matching item in the corpus — an exact trigger match with the strongest usage record — is withheld from the production path.

Scale

 Default        | 101   active knowledge rows
 Brain Platform | 100

SELECT count(*) FROM "Knowledge"
WHERE "deletedAt" IS NULL AND scope='user' AND "ownerProjectId" IS NOT NULL;
-- 117

Why the duplicate-project detector can't see this

findDuplicateProjectGroups looks for normalized name collisions. Brain Platform vs Default will never collide, so the flywheel-repair Stage-1 instrumentation is structurally blind to knowledge stranded in the catch-all bucket. The health panel reports duplicateProjects: 0 while half the corpus is unreachable.

Impact beyond retrieval

The retrieval benchmark exports candidates via candidatesForPrompt. If the benchmark's export path is scoped differently from a real session, published NDCG numbers may not describe what production actually injects — worth checking before the next re-run, since docs/VALIDATION.md's delta is the project's headline evidence.

Two candidate fixes (design call — not taken unilaterally)

1. Honour scope in the filter. Add { AND: [{ scope: "user" }, { ownerUserId: userId }] } to both helpers. Makes scope mean what it reads as; preserves capture provenance. Changes what every user sees — user-scoped rules taught in project A begin appearing in project B.
2. Data repair. Null out ownerProjectId where scope='user', matching the filter's existing documented contract (ownerProjectId IS NULL is the user-level clause). No code change; loses the record of which project the lesson came from.

(1) is more faithful to the field's meaning; (2) is more faithful to the current contract. Either way the duplicate detector should probably learn to flag a Default bucket that holds substantial knowledge.

Found while building the corpus-dependent generation-uplift suite (follow-up to #126). Full write-up in docs/KNOWN_ISSUES.md §0p.

[bejranonda]

Correction — I named the wrong mechanism in the original report, and the fix turned out not to need a design call.

Wrong in the original:

• It cited buildRawProjectFilter (V1). Production retrieval uses buildRawProjectFilterV2 (kra.ts:154, oracle.ts:139).
• It asserted "Knowledge.visibility is a separate Phase-4 field and does not govern this path." That is backwards — V2 is visibility-driven, and visibility defaults to "project", which is what binds a row to its writing project.

Right, and it changes the conclusion: the scope column is consulted — but only in the no-active-project branch, which received exactly this fix on 2026-05-12 after "5/5 retrieval misses traced to this branch" (the comment is still in scope-filter.ts). The two active-project branches never got it.

So this was never an open design question. The project had already decided; the decision was applied to one branch out of three. I was wrong to frame it as needing your call.

Fixed in #177 by adding the same disjunct behind an opt-in includeUserScopeAcrossProjects flag (default false), enabled only for kra.ts and oracle.ts. Opt-in rather than a blanket change because two other callers of the shared helper depend on the project edge holding — action-items.ts (2026-07-10 review, finding 1) and meeting-extract.ts's deliberately project-wide-but-not-owner-scoped supersession search (2026-07-17, finding I2).

Measured read-only on the live corpus: rows visible to a Brain Platform session go 104 → 141 (+37), and the 0.9009-similarity item that motivated this now matches. It is +37 rather than +101 because the remaining Default rows are scope='project' and correctly stay bound — the fix is narrow, not a bucket merge.