feat(core): suite-2 uplift read (+40pp) + scope-filter blindness, capture gap, Redis drift

.env.example

@@ -248,8 +248,16 @@ SENTRY_DSN=""
 SENTRY_TRACES_SAMPLE_RATE="0"        # 0..1 — leave at 0 until you've sized the volume
 
 # --- Redis (Wave 2 — multi-replica rate-limit state) -------------------------
-# Dev / single-host deploys can leave empty; the proxy falls back to an
-# in-process Map. Production docker-compose provides `redis://redis:6379`.
+# You MUST set this yourself for a server deploy — compose does NOT provide it.
+# deploy/docker-compose.yml only passes the value through (`REDIS_URL:
+# ${REDIS_URL:-}`), so leaving it empty here means the `redis` service runs but
+# nothing connects to it, and rate-limit state lives in an in-process Map.
+#
+# In-process state is correct on a single replica but resets on every deploy or
+# `reload.sh web` — which silently clears daily caps such as
+# RATE_LIMIT_MEETING_EXTRACT_PER_DAY. Set the line below on any server deploy:
+#   REDIS_URL="redis://redis:6379"
+# Local dev can leave it empty.
 REDIS_URL=""
 
 # --- Email (optional — falls back to manual-link UX when disabled) ----------

docs/KNOWN_ISSUES.md

@@ -384,6 +384,27 @@ compose allowlist for the `qwen3-coder` default, or override
 
 ---
 
+## 0p. Corpus capture gap + scope-filter blindness (2026-07-28)
+
+Found while building the second generation-uplift suite (#126 follow-up). That
+suite's treatment arm takes its injected block from the **live KRA path** rather
+than a hand-written file — the change you make when you want to measure the
+product instead of the mechanism. The first probe returned a null, and chasing
+it produced two defects and one measurement.
+
+**Method (reproducible).** Ask a well-formed technical question whose answer is a
+documented repo rule, via both `brain_start_session(projectName:
+"BrainPlatform")` (the production path) and `brain_retrieve_knowledge`
+(unscoped), then compare what comes back.
+
+| Issue | Where | Status |
+|---|---|---|
+| ~~**Hard-won repo rules were never taught to the Brain.**~~ **Partially closed (2026-07-28).** A probe of three documented rules found two absent from the corpus entirely: the `force-dynamic` rule (§10) and the package-boundary rule — each with a class-of-bug entry here and real debugging cost behind it. The Oracle said so outright: *"isn't captured in your current knowledge base."* Only the `#418` mount-gate rule was present. **This is a capture gap, not a retrieval gap** — proven by teaching `force-dynamic` and re-running the *identical* prompt, which then ranked it **first**. KRA ranked it correctly the moment it existed. Nine rules were backfilled from GUIDELINES §9/§10 and AGENTS.md. **The rest of GUIDELINES/KNOWN_ISSUES has not been swept** — the docs remain the de facto memory for pre-Stage-1 lessons. | `docs/GUIDELINES.md`, the Brain corpus | partially done |
+| **`scope: "user"` knowledge is invisible outside the project it was captured in — 117 rows affected.** Both `buildKnowledgeWhere` and `buildRawProjectFilter` (the pgvector path that `kra.ts` and `oracle.ts` actually use) resolve visibility as `ownerProjectId = $activeProject OR (ownerProjectId IS NULL AND ownerUserId = $user)`. **The `scope` column is never consulted.** A row with `scope: "user"` and a non-null `ownerProjectId` therefore matches neither arm and is filtered out — even though `scope: "user"` is `brain_teach_knowledge`'s *default* and reads as "follows the user". (`Knowledge.visibility` is a separate Phase-4 field and does not govern this path.) **Measured impact, same prompt both arms:** unscoped retrieval ranked `cmqpqemoh…` — trigger *"Adding a user-facing concept or glossary page to the External Brain webapp"*, 100% success over 11 uses — **first, at 0.9009 similarity**. The project-scoped session path did not return it at all, injecting a tangential i18n rule instead. **Scale:** 101 active items in `Default` vs 100 in `Brain Platform`; 117 rows repo-wide are `scope='user'` with a non-null project. Roughly half the corpus is invisible to a `BrainPlatform` session. **Not fixed here** — the fix is a design call with real blast radius (honour `scope` in the filter, vs. null out `ownerProjectId` on user-scope rows as a data repair) and it changes what every user sees. Note the duplicate-project detector cannot surface this: it looks for *normalized name collisions*, and `Brain Platform` vs `Default` will never collide. | `packages/core/src/scope-filter.ts:64-75,155-168`; `kra.ts:172` | open — needs a design call |
+| ~~**`.env.example` claimed compose provides Redis; it does not.**~~ **Doc fixed (2026-07-28).** The comment read *"Production docker-compose provides `redis://redis:6379`"*, but `deploy/docker-compose.yml` only passes the value through (`REDIS_URL: ${REDIS_URL:-}`). Consequence on the reference instance: a healthy `deploy-redis-1` container has been up for weeks with **nothing connected to it**, and rate-limit state lives in an in-process Map. Correct on a single replica, but it resets on every deploy / `reload.sh web`, silently clearing daily caps such as `RATE_LIMIT_MEETING_EXTRACT_PER_DAY`. It also means the pre-2026-07-28 get-then-set race (§0o) was a **live production** bypass, not a dev-only concern. **The operator must still set `REDIS_URL` in the live `.env`** — a gitignored file no PR can reach. | `.env.example`, live `.env` | doc fixed; operator action pending |
+
+---
+
 ## 0. MVP-complete open items (2026-04-29, operator action required)
 
 These are not blocking pilot but must be resolved before a second contributor joins or the platform is advertised publicly.

docs/VALIDATION.md

@@ -19,6 +19,13 @@
   regressions). Positive, but n=6 is small and the task suite under-tests
   "well-known" utilities — see `packages/core/generation-uplift/RESULTS.md`
   for the full honest read, including where injection made no difference.
+- ✅ **Suite 2 — corpus-dependent tasks, live-KRA injection (2026-07-28,
+  below)**: +40pp (control 3/5, treatment 5/5, n=5, 0 regressions). More
+  importantly it identifies *where* injection pays: the two flips were
+  **locally arbitrary** conventions (a workspace subpath; a build-pipeline
+  quirk), and all three ties were conventions that coincide with general good
+  practice the base model already had. See
+  `packages/core/generation-uplift/suite-2/RESULTS.md`.
 
 ## First published run — retrieval NDCG@5 (2026-07-06)
 
@@ -87,6 +94,46 @@ before any run) and `RESULTS.md` (the read). Summary:
   including a grading-mechanics substitution (no local vitest available;
   `harness/grade.ts` re-implements the same assertions with `node:assert`).
 
+## Second published run — suite 2, corpus-dependent + live KRA (2026-07-28)
+
+Suite 1 measured the injection *mechanism*: generic utility tasks with a
+hand-written injected block. Suite 2 changes both variables — tasks whose correct
+answer is only derivable from BrainPlatform convention, and a treatment block
+taken **verbatim from the live KRA path** rather than authored. That makes it a
+test of retrieval and generation together. Pre-registration committed before any
+run (`packages/core/generation-uplift/suite-2/README.md`).
+
+| | Convention applied (n=5) |
+|---|---|
+| Control | 3/5 (60%) |
+| **Treatment (live-KRA)** | **5/5 (100%)** |
+| Paired difference | **+40pp, 0 regressions** |
+
+- **The aggregate is not the interesting part.** The two flips were conventions
+  that are *locally arbitrary* and cannot be derived from expertise: the
+  `@brain/core/format-relative` subpath (control hand-rolled its own formatter —
+  exactly the divergence v0.15.0 consolidated away), and `force-dynamic` on an
+  env-reading server component (control emitted **no** static-rendering opt-out of
+  any kind — the v0.14.0 bug verbatim). All three ties were conventions that
+  coincide with general good practice: the control arm reached for
+  `x-forwarded-host` unprompted, mount-gated its `window` read, and used `{count}`
+  placeholders without being told.
+- **So injection pays off where the convention is arbitrary, and ties where it is
+  good craft.** That predicts where capture effort has the highest return — local
+  arbitrariness (package paths, build-pipeline quirks, project decisions), not
+  general engineering practice. Suite 1's ties had the same cause, so two
+  independent suites now agree on the mechanism.
+- **Reading it honestly:** n=5; grading is static assertion over emitted source
+  (weaker than suite 1's executable tests, pre-registered as such); isolation was
+  instruction-enforced and every control pass was checked for contamination (none
+  found — no repo identifiers in any control output). **Most important caveat:**
+  the five rules were verified retrievable from a project-scoped session *before*
+  the run, to control for the scope-filter defect in `KNOWN_ISSUES §0p` / #174.
+  Roughly half the corpus is invisible to a project-scoped session until that is
+  resolved, so **a run against the uncurated corpus would score lower.** This
+  number describes the Brain with its known retrieval defect controlled for, not
+  the Brain as a user experiences it today.
+
 ### The earlier attempt, and why the label changed
 
 A previous pair of scripts ran against the dev seed corpus

packages/core/generation-uplift/suite-2/README.md

@@ -0,0 +1,127 @@
+# Generation-uplift suite 2 — corpus-dependent, live-KRA injection
+
+Follow-up to suite 1 (`../README.md`, issue #126). Suite 1 answered *does knowing
+a non-obvious convention up front change output?* using generic utility tasks and
+a **hand-written** injected block. It found +33.3pp (control 4/6 → treatment 6/6),
+but four of six tasks tied because the base model already knew the edge case from
+training — the suite was corpus-*independent* by design, to avoid testing the
+Brain with knowledge the same session had taught it.
+
+This suite changes the two things that made suite 1 a mechanism test rather than a
+product test.
+
+## Pre-registration
+
+**Written and committed before any run. This file's first commit is the
+pre-registration timestamp. Do not edit the task list, the injection procedure, or
+the metric after runs start** — per the honesty guardrails in
+`docs/VALIDATION.md §2` and the precedent set by suite 1.
+
+### What changes from suite 1
+
+| | Suite 1 | Suite 2 |
+|---|---|---|
+| Task knowledge | corpus-independent (generic utilities) | **corpus-dependent** — only derivable from BrainPlatform convention |
+| Treatment block | hand-written `injected-knowledge.md` | **live KRA output**, captured verbatim |
+| What it measures | the injection mechanism | **retrieval + generation together** (the product) |
+| Grading | runtime tests (`node:assert`) | **static assertions over emitted source** (see caveat) |
+
+### Design
+
+Five tasks. Each asks for a small, self-contained file of the kind this repo
+actually contains. Each has exactly one **non-obvious BrainPlatform convention**
+that a competent TypeScript/Next.js developer would plausibly get wrong, because
+the idiomatic general answer differs from this repo's required answer.
+
+Only the task prompt is shown. Two measures keep the control arm from discovering
+the convention by other means:
+
+1. **Prompts are framed generically** — they describe the artifact wanted
+   ("a Next.js server component that reads an env var and renders it") and never
+   name this repo, its files, or its conventions. A control agent has no cue that
+   a project-specific rule exists, so no reason to go looking for one.
+2. **File reading is forbidden in the prompt**, and each arm writes its single
+   output file to a scratch directory outside the repo.
+
+**Enforcement caveat, stated up front.** Measure 2 is an *instruction*, not a
+sandbox: the agent harness available here runs with a working directory inside
+this repository, so a determined agent could read `docs/GUIDELINES.md` and find
+the answer. Suite 1 described its arms as running with "no access to this repo";
+that was also instruction-enforced, and it mattered less there because suite 1's
+tasks were generic utilities whose answers are not written down here. For a
+corpus-*dependent* suite the risk is real and asymmetric: it would inflate the
+**control** arm and therefore **understate** uplift. Any control-arm pass is
+consequently checked against its transcript for evidence of file reads, and a
+task whose control arm read repo files is reported as **void**, not as a tie.
+
+### Procedure
+
+For each task, run the same agent twice:
+
+- **Control arm:** task prompt only.
+- **Treatment arm:** task prompt + the knowledge block returned by
+  `brain_start_session({ prompt: <the task's retrieval query>, projectName:
+  "BrainPlatform" })`, pasted verbatim under a "Relevant knowledge from your
+  Brain" heading.
+
+The treatment block is **not authored** — whatever KRA returns is what the agent
+gets, including irrelevant items. If KRA returns nothing useful for a task, that
+task is expected to tie, and that tie is a real result about retrieval, not a
+flaw in the harness.
+
+Same model both arms. Same task prompt text both arms; the injected block is the
+single deliberate variable. Seed/temperature are not controllable through the
+available agent harness — a known limitation, recorded rather than hidden.
+
+### Confound control (why these five tasks)
+
+`KNOWN_ISSUES §0p` documents that `scope: "user"` knowledge is invisible outside
+its capture project (117 rows; issue #174). Every rule below was **taught into the
+canonical `Brain Platform` project on 2026-07-28 and verified retrievable** from a
+`projectName: "BrainPlatform"` session before this file was written. Without that
+control, a null result could not distinguish "the knowledge didn't help" from
+"the knowledge wasn't visible" — which is exactly the ambiguity that makes an
+uncontrolled live-KRA benchmark unpublishable.
+
+### Task list (pre-registered)
+
+| # | Task | Required convention | The plausible wrong answer |
+|---|---|---|---|
+| 1 | `app/status/page.tsx` — server component rendering `process.env.BRAIN_PUBLIC_HOSTNAME` | `export const dynamic = "force-dynamic"` | omit it; Docker build bakes the empty value |
+| 2 | route handler redirecting `/old` → `/new` absolutely | path-only `Location`, or `x-forwarded-host` + `x-forwarded-proto` | `new URL("/new", req.url)` → emits `0.0.0.0:3000` |
+| 3 | client component displaying the current hostname | SSR-safe default, real read in `useEffect` | read `window.location` in render / `useMemo` / `useState(init)` → React #418 |
+| 4 | session card rendering a relative timestamp | import `formatRelative` from `@brain/core/format-relative` + hydration-safe wrapper | hand-roll a "N days ago" helper |
+| 5 | i18n dictionary entry for a retrieved/cited count | format-string substitution or conditional trailing key | bake a sample count (`"0 items retrieved · 2 cited"`) into the string |
+
+### Metric (pre-registered)
+
+Per-task binary pass/fail on the convention assertion, control vs treatment,
+n=5 per arm. Report the paired difference and the full pass/fail matrix,
+including which tasks tied and why. At n=5 no confidence interval is meaningful;
+this is reported as a small-n indicative read, consistent with suite 1 and with
+the retrieval benchmark's own "n is small" caveat.
+
+**Pre-committed reporting rule:** the result is published either way, including a
+null or a negative. A tie is reported as a tie, and the retrieved block that
+produced it is included verbatim in `RESULTS.md` so a reader can judge whether
+retrieval or generation was at fault.
+
+### Honesty caveats (read before trusting any number)
+
+1. **Grading is static, not runtime.** These conventions govern code *shape*
+   (a directive is present, a read sits inside an effect, an import comes from a
+   specific subpath) — there is no runtime behaviour to assert without booting
+   Next.js and a Docker build. Assertions are mechanical regex/AST checks over the
+   emitted file, so there is still no human or LLM judgement in the loop, but this
+   is a **weaker instrument** than suite 1's executable tests. Stated here rather
+   than discovered later.
+2. **The author of the tasks also authored the backfilled knowledge** (2026-07-28,
+   same session). Mitigated by taking the rules verbatim from pre-existing repo
+   docs (`GUIDELINES §9/§10`, `AGENTS.md`) that long predate this suite — the
+   conventions are not invented for the benchmark — but the *selection* of which
+   five to test is mine, and selection is a bias surface.
+3. **n=5.** Indicative, not powered.
+4. **Live KRA output is not reproducible over time.** The corpus changes; decay
+   and usage counts move rankings. Each run's retrieved block is committed to
+   `RESULTS.md` so the read is auditable even though it is not re-runnable to the
+   same input.