feat(core): suite-2 uplift read (+40pp) + scope-filter blindness, capture gap, Redis drift

[bejranonda]

0. Suite-2 generation-uplift read — +40pp, and where injection pays

Pre-registration committed at f62895b before any arm ran. Five
corpus-dependent tasks; treatment arm injected verbatim from the live KRA path
rather than hand-written — so this measures retrieval and generation together.

	Convention applied (n=5)
Control	3/5 (60%)
Treatment (live-KRA)	5/5 (100%)
Paired difference	+40pp, 0 regressions

The aggregate is the less interesting half. What separates a flip from a tie is
whether the convention is locally arbitrary:

#	Convention	C	T
1	force-dynamic on env-reading server component	❌	✅	flip
2	x-forwarded-host instead of req.url	✅	✅	tie
3	mount-gate window.location	✅	✅	tie
4	@brain/core/format-relative subpath	❌	✅	flip
5	no baked numbers in i18n strings	✅	✅	tie

• Flips are unguessable. Control emitted no static-rendering opt-out of any
  kind (task 1) — the v0.14.0 bug verbatim — and hand-rolled its own
  formatRelative (task 4), exactly the divergence v0.15.0 consolidated away.
• Ties are good craft. Control reached for x-forwarded-host unprompted,
  mount-gated its window read, and used {count} placeholders without being told.

So injection pays where the convention is arbitrary and ties where it is craft —
which predicts where capture effort returns most. Suite 1's four ties had the same
cause; two independent suites now agree on the mechanism.

Caveat that matters most: the five rules were verified retrievable from a
project-scoped session first, to control for #174 below. Roughly half the corpus
is invisible to such a session today, so an uncurated run would score lower.
This measures the Brain with its known retrieval defect controlled for.

---

What

Three findings from building the second generation-uplift suite. That suite takes
its treatment-arm injection from the live KRA path rather than a hand-written
block — the change you make when you want to measure the product instead of the
mechanism. The first probe returned a null, and chasing it surfaced more than the
benchmark itself would have.

1. Capture gap — partially closed

Probed three documented repo rules. Two were absent from the Brain entirely —
force-dynamic (GUIDELINES §10) and the package-boundary rule — despite each
carrying a class-of-bug entry here and real debugging cost. The Oracle said so
outright: "isn't captured in your current knowledge base."

Proven to be capture, not retrieval: taught force-dynamic, re-ran the
identical prompt, and it came back ranked first. KRA ranked it correctly the
moment it existed. Nine rules backfilled from GUIDELINES §9/§10 and AGENTS.md.

This inverts the diagnosis I opened with. I'd recommended work on retrieval
quality (NDCG 0.3075) — but retrieval can only rank what capture put there.

2. scope: "user" is invisible outside its capture project — open, #174

buildKnowledgeWhere and buildRawProjectFilter (the pgvector path kra.ts and
oracle.ts actually use) resolve visibility as ownerProjectId = $activeProject OR (ownerProjectId IS NULL AND ownerUserId = $user). The scope column is
never consulted. A row with scope: "user" and a non-null ownerProjectId
matches neither arm — and that is exactly what brain_teach_knowledge produces by
default in a project-bound session.

Controlled reproduction, identical prompt:

Path	Result
unscoped brain_retrieve_knowledge	best-matching item #1 at 0.9009, exact trigger match, 100% success / 11 uses
brain_start_session(projectName:"BrainPlatform")	absent — injected a tangential i18n rule instead

Scale: 101 active rows in Default vs 100 in Brain Platform; 117 rows
repo-wide are scope='user' with a non-null project.

Deliberately not fixed here. Both candidate fixes change what every user sees;
that's a design call, written up in #174 rather than taken unilaterally.

3. Redis config drift — doc fixed

.env.example claimed "Production docker-compose provides redis://redis:6379".
It does not — compose only passes ${REDIS_URL:-} through. Consequences on the
reference instance: a healthy deploy-redis-1 has run for weeks with nothing
connected; rate-limit state lives in an in-process Map that resets on every
deploy, silently clearing daily caps like RATE_LIMIT_MEETING_EXTRACT_PER_DAY;
and the get-then-set race fixed in #173 was therefore a live production
bypass, not a dev-only concern.

Test plan

• scripts/check-doc-refs.sh passes locally (DOC_REF_CEILING=172).
• Capture-gap claim verified both ways — Oracle stated the rule was absent,
  and the before/after on an identical prompt moved it from absent to rank 1.
• Scope-filter claim verified three ways: read both helpers, confirmed
  kra.ts:172 uses the raw one, and reproduced the miss with a controlled
  scoped-vs-unscoped comparison.
• Row counts from read-only psql against prod (SELECT only, permitted
  autonomously).
• .env.example claim checked against deploy/docker-compose.yml:147-149.
• Relying on CI for typecheck / test / build — no code changes in this PR
  (docs + .env.example only), but CI is still the gate.
• Operator action: set REDIS_URL="redis://redis:6379" in the live .env
  and redeploy. .env is gitignored — no PR can do this.
• Reviewer: decide the scope: "user" knowledge is invisible outside its capture project (117 rows, ~half the corpus) #174 fix direction (honour scope in the filter vs.
  null out ownerProjectId on user-scope rows).

🤖 Generated with Claude Code

https://claude.ai/code/session_01Rqj5y9fT3XPUqnqKyustp9

[coderabbitai]

📝 Walkthrough

Walkthrough

Updated Redis deployment guidance and added a Known Issues entry documenting corpus capture gaps, user-scope visibility behavior, measurements, backfill notes, and corrected Compose configuration details.

Changes

Documentation updates

Layer / File(s)	Summary
Redis deployment guidance .env.example	Explains empty REDIS_URL behavior, Docker Compose pass-through, rate-limit resets, and the recommended server Redis URL.
Corpus visibility investigation docs/KNOWN_ISSUES.md	Documents corpus capture and scope: "user" visibility findings, measured impact, backfill notes, and corrected Redis documentation.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Possibly related issues

• Issue 174 — Documents the same scope: "user" visibility defect and root cause.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title accurately summarizes the main changes: suite-2 uplift findings, scope-filter blindness, capture gap, and Redis docs drift.
Description check	✅ Passed	The description covers the change, rationale, test plan, and gate expectations, though the template's explicit Gates section is not formatted separately.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/scope-filter-diagnosis-and-redis-drift

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/KNOWN_ISSUES.md`:
- Line 403: Update the issue heading and wording in the documented entry to
state that the visibility blind spot affects project-scoped retrieval or
project-scoped sessions, not user-scope retrieval generally. Preserve the
existing evidence and distinction that unscoped retrieval can return these rows,
while keeping the open design-call status unchanged.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d73c8c27-316d-409b-b311-27e11d63cd08

📥 Commits

Reviewing files that changed from the base of the PR and between d97c01b and bda82ee.

📒 Files selected for processing (2)

• .env.example
• docs/KNOWN_ISSUES.md

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Scope the heading to project-scoped retrieval.

The current wording implies that scope: "user" knowledge is generally invisible outside its capture project, but this entry also says unscoped retrieval returns the row, and the later resolved note documents scope-aware behavior when no project is supplied. Please state explicitly that the blind spot affects project-scoped sessions, so readers do not infer that all user-scope retrieval is broken.

Proposed wording

-| **`scope: "user"` knowledge is invisible outside the project it was captured in — 117 rows affected.**
+| **Project-scoped sessions can hide `scope: "user"` knowledge captured in another project — 117 rows affected.**

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

| **`scope: "user"` knowledge is invisible outside the project it was captured in — 117 rows affected.** Both `buildKnowledgeWhere` and `buildRawProjectFilter` (the pgvector path that `kra.ts` and `oracle.ts` actually use) resolve visibility as `ownerProjectId = $activeProject OR (ownerProjectId IS NULL AND ownerUserId = $user)`. **The `scope` column is never consulted.** A row with `scope: "user"` and a non-null `ownerProjectId` therefore matches neither arm and is filtered out — even though `scope: "user"` is `brain_teach_knowledge`'s *default* and reads as "follows the user". (`Knowledge.visibility` is a separate Phase-4 field and does not govern this path.) **Measured impact, same prompt both arms:** unscoped retrieval ranked `cmqpqemoh…` — trigger *"Adding a user-facing concept or glossary page to the External Brain webapp"*, 100% success over 11 uses — **first, at 0.9009 similarity**. The project-scoped session path did not return it at all, injecting a tangential i18n rule instead. **Scale:** 101 active items in `Default` vs 100 in `Brain Platform`; 117 rows repo-wide are `scope='user'` with a non-null project. Roughly half the corpus is invisible to a `BrainPlatform` session. **Not fixed here** — the fix is a design call with real blast radius (honour `scope` in the filter, vs. null out `ownerProjectId` on user-scope rows as a data repair) and it changes what every user sees. Note the duplicate-project detector cannot surface this: it looks for *normalized name collisions*, and `Brain Platform` vs `Default` will never collide. | `packages/core/src/scope-filter.ts:64-75,155-168`; `kra.ts:172` | open — needs a design call |

| **Project-scoped sessions can hide `scope: "user"` knowledge captured in another project — 117 rows affected.** Both `buildKnowledgeWhere` and `buildRawProjectFilter` (the pgvector path that `kra.ts` and `oracle.ts` actually use) resolve visibility as `ownerProjectId = $activeProject OR (ownerProjectId IS NULL AND ownerUserId = $user)`. **The `scope` column is never consulted.** A row with `scope: "user"` and a non-null `ownerProjectId` therefore matches neither arm and is filtered out — even though `scope: "user"` is `brain_teach_knowledge`'s *default* and reads as "follows the user". (`Knowledge.visibility` is a separate Phase-4 field and does not govern this path.) **Measured impact, same prompt both arms:** unscoped retrieval ranked `cmqpqemoh…` — trigger *"Adding a user-facing concept or glossary page to the External Brain webapp"*, 100% success over 11 uses — **first, at 0.9009 similarity**. The project-scoped session path did not return it at all, injecting a tangential i18n rule instead. **Scale:** 101 active items in `Default` vs 100 in `Brain Platform`; 117 rows repo-wide are `scope='user'` with a non-null project. Roughly half the corpus is invisible to a `BrainPlatform` session. **Not fixed here** — the fix is a design call with real blast radius (honour `scope` in the filter, vs. null out `ownerProjectId` on user-scope rows as a data repair) and it changes what every user sees. Note the duplicate-project detector cannot surface this: it looks for *normalized name collisions*, and `Brain Platform` vs `Default` will never collide. | `packages/core/src/scope-filter.ts:64-75,155-168`; `kra.ts:172` | open — needs a design call |

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/KNOWN_ISSUES.md` at line 403, Update the issue heading and wording in
the documented entry to state that the visibility blind spot affects
project-scoped retrieval or project-scoped sessions, not user-scope retrieval
generally. Preserve the existing evidence and distinction that unscoped
retrieval can return these rows, while keeping the open design-call status
unchanged.