diff --git a/ARCHITECTURE_DECISIONS.md b/ARCHITECTURE_DECISIONS.md index 71c0a76..273acfc 100644 --- a/ARCHITECTURE_DECISIONS.md +++ b/ARCHITECTURE_DECISIONS.md @@ -74,6 +74,14 @@ This ledger records repository-level decisions. Feature-level decisions should m - **Decision:** Define strict query and result contracts in `@founderos/knowledge-schema` and execute them as a pure operation in `@founderos/knowledge-engine` over a caller-supplied candidate set. Apply exact-match filters and declarative context constraints by intersection, reject invalid or duplicate candidates, sort returned objects by ID, and copy source metadata into each result's provenance record. - **Consequences:** Query behavior is auditable, deterministic, and independently testable against the Priority 1 corpus. Callers remain responsible for supplying candidates, context constraints do not confer authorization, project matching is limited to documented object fields, and semantic relevance or ranking requires a future architecture decision. +## ADR-0010: Separate candidate provision from deterministic knowledge access + +- **Status:** Accepted +- **Date:** 2026-07-28 +- **Context:** Milestone 05 requires callers to assemble candidate arrays directly. Future filesystems, databases, and external providers need a stable access boundary, but Milestone 06 excludes durable persistence, external integrations, and retrieval intelligence. +- **Decision:** Define versioned candidate-source batches and asynchronous repository interfaces in `@founderos/knowledge-schema`. Implement a validated in-memory candidate source and immutable repository snapshot in `@founderos/knowledge-engine`. Candidate sources provide objects and source provenance; repositories revalidate, reject duplicate identities, sort observable results, and supply candidates to the existing query filter through a repository-backed application service. +- **Consequences:** Query execution no longer needs to know how candidates were obtained, and future providers can implement the same asynchronous contract. The in-memory repository is rebuilt from its sources, carries no durability or update semantics, and deliberately performs no ranking, semantic selection, or authorization. + ## ADR template ```markdown diff --git a/CHANGELOG.md b/CHANGELOG.md index 36f7e7e..8001e58 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -20,3 +20,5 @@ All notable changes to FounderOS will be documented here. - Path-contained migration CLI and deterministic `migration-report.json` generation. - Strict KnowledgeOS query and result contracts with consumer context, exact filters, and source provenance. - Deterministic in-memory query execution and Priority 1 corpus evaluation fixtures. +- Versioned candidate-source and Knowledge Repository contracts. +- Validated in-memory candidate provider, deterministic repository access, and repository-backed query execution. diff --git a/DOCUMENTATION_INDEX.md b/DOCUMENTATION_INDEX.md index 95c41d4..0fc4751 100644 --- a/DOCUMENTATION_INDEX.md +++ b/DOCUMENTATION_INDEX.md @@ -72,6 +72,16 @@ The documents below are the official FounderOS v1.0 bootstrap specification, org - [Milestone 05 Acceptance Criteria v1.0](./docs/milestones/milestone-05/FounderOS_Milestone_05_Acceptance_Criteria_v1.0.md) - [Milestone 05 Codex Execution Prompt v1.0](./docs/milestones/milestone-05/FounderOS_Milestone_05_Codex_Execution_Prompt_v1.0.md) +### Milestone 06 — Knowledge Repository and Candidate Source Foundation + +- [Knowledge Repository and Candidate Source Foundation Specification v1.0](./docs/milestones/milestone-06/FounderOS_Milestone_06_Knowledge_Repository_and_Candidate_Source_Foundation_Specification_v1.0.md) +- [Knowledge Repository Contract v1.0](./docs/milestones/milestone-06/FounderOS_Knowledge_Repository_Contract_v1.0.md) +- [Candidate Source Contract v1.0](./docs/milestones/milestone-06/FounderOS_Candidate_Source_Contract_v1.0.md) +- [Knowledge Repository Architecture v1.0](./docs/milestones/milestone-06/FounderOS_Knowledge_Repository_Architecture_v1.0.md) +- [Milestone 06 Acceptance Criteria v1.0](./docs/milestones/milestone-06/FounderOS_Milestone_06_Acceptance_Criteria_v1.0.md) +- [Milestone 06 Verification Checklist v1.0](./docs/milestones/milestone-06/FounderOS_Milestone_06_Verification_Checklist_v1.0.md) +- [Milestone 06 Codex Execution Prompt v1.0](./docs/milestones/milestone-06/FounderOS_Milestone_06_Codex_Execution_Prompt_v1.0.md) + ## Repository governance - [Architecture decisions](./ARCHITECTURE_DECISIONS.md) diff --git a/README.md b/README.md index 1e78cbb..a59d1cb 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ FounderOS is an AI-native operating system for founder decision-making, organizational memory, and governed AI-assisted execution. This repository is a documentation-first TypeScript monorepo. -The repository currently provides the governed KnowledgeOS schema, ingestion, migration, and deterministic query foundations. It does **not** implement persistence, semantic retrieval, Hermes, an agent runtime, MCP connectors, or a user interface. +The repository currently provides the governed KnowledgeOS schema, ingestion, migration, repository access, and deterministic query foundations. It does **not** implement persistence, semantic retrieval, Hermes, an agent runtime, MCP connectors, or a user interface. ## Architecture at a glance @@ -20,12 +20,12 @@ The official specifications are indexed in [DOCUMENTATION_INDEX.md](./DOCUMENTAT ## Implemented foundations -- [`@founderos/knowledge-schema`](./packages/knowledge-schema/README.md) provides strict runtime schemas and inferred TypeScript contracts for KnowledgeOS metadata, relationships, the seven official knowledge object categories, queries, and query results. -- [`@founderos/knowledge-engine`](./services/knowledge-engine/README.md) provides read-only ingestion, manifest-controlled Priority 1 corpus migration, and deterministic in-memory filtering with preserved source provenance. +- [`@founderos/knowledge-schema`](./packages/knowledge-schema/README.md) provides strict runtime schemas and inferred TypeScript contracts for KnowledgeOS metadata, objects, migration, queries, candidate sources, repositories, and results. +- [`@founderos/knowledge-engine`](./services/knowledge-engine/README.md) provides read-only ingestion, manifest-controlled Priority 1 corpus migration, validated in-memory repository access, and deterministic filtering with preserved source provenance. - [`specs/knowledge-templates`](./specs/knowledge-templates) provides valid Markdown templates for all seven KnowledgeOS object types. - [`knowledge/migration-manifest.yaml`](./knowledge/migration-manifest.yaml) binds the eight canonical FounderOS Priority 1 documents to reviewed object identities, logical destinations, metadata, and source hashes. -Vault watching, persistence, semantic retrieval, embeddings, ranking, graph storage, agent behavior, connectors, and interfaces remain unimplemented. +Vault watching, durable persistence, semantic retrieval, embeddings, ranking, graph storage, agent behavior, connectors, and interfaces remain unimplemented. ## Repository layout diff --git a/docs/milestones/milestone-06/FounderOS_Candidate_Source_Contract_v1.0.md b/docs/milestones/milestone-06/FounderOS_Candidate_Source_Contract_v1.0.md new file mode 100644 index 0000000..fb8e462 --- /dev/null +++ b/docs/milestones/milestone-06/FounderOS_Candidate_Source_Contract_v1.0.md @@ -0,0 +1,32 @@ +# FounderOS Candidate Source Contract v1.0 + +## Purpose + +Define how knowledge sources provide candidates to KnowledgeOS. + +## Future Sources + +- File system +- Database +- External APIs +- MCP connectors + +## Contract + +A candidate source provides: + +- Source identity +- Available objects +- Provenance metadata + +## Rules + +Candidate sources must not: + +- Modify knowledge objects +- Bypass validation +- Remove provenance + +## Principle + +Sources provide candidates; repositories manage access. diff --git a/docs/milestones/milestone-06/FounderOS_Knowledge_Repository_Architecture_v1.0.md b/docs/milestones/milestone-06/FounderOS_Knowledge_Repository_Architecture_v1.0.md new file mode 100644 index 0000000..66fd35e --- /dev/null +++ b/docs/milestones/milestone-06/FounderOS_Knowledge_Repository_Architecture_v1.0.md @@ -0,0 +1,29 @@ +# FounderOS Knowledge Repository Architecture v1.0 + +## Architecture + + Candidate Sources + + ↓ + + Repository Layer + + ↓ + + Query Engine + + ↓ + + Knowledge Results + +## Responsibilities + +Candidate Source: Provides validated candidates. + +Repository: Provides access abstraction and lookup. + +Query Engine: Provides filtering and result generation. + +## Principle + +Separate knowledge access from knowledge intelligence. diff --git a/docs/milestones/milestone-06/FounderOS_Knowledge_Repository_Contract_v1.0.md b/docs/milestones/milestone-06/FounderOS_Knowledge_Repository_Contract_v1.0.md new file mode 100644 index 0000000..d5d70da --- /dev/null +++ b/docs/milestones/milestone-06/FounderOS_Knowledge_Repository_Contract_v1.0.md @@ -0,0 +1,38 @@ +# FounderOS Knowledge Repository Contract v1.0 + +## Purpose + +Define the abstraction for accessing Knowledge Objects. + +## Responsibilities + +The repository provides: + +- Knowledge object retrieval +- Object lookup +- Candidate discovery +- Deterministic access + +## Contract + +Example: + +``` typescript +interface KnowledgeRepository { + find(query): KnowledgeObject[]; + getById(id): KnowledgeObject | null; +} +``` + +## Requirements + +Repository implementations must: + +- Preserve identity +- Preserve provenance +- Return validated objects +- Maintain deterministic behavior + +## Principle + +Repository provides access, not intelligence. diff --git a/docs/milestones/milestone-06/FounderOS_Milestone_06_Acceptance_Criteria_v1.0.md b/docs/milestones/milestone-06/FounderOS_Milestone_06_Acceptance_Criteria_v1.0.md new file mode 100644 index 0000000..6ebc9d2 --- /dev/null +++ b/docs/milestones/milestone-06/FounderOS_Milestone_06_Acceptance_Criteria_v1.0.md @@ -0,0 +1,31 @@ +# FounderOS Milestone 06 Acceptance Criteria v1.0 + +## Functional Criteria + +- [ ] Repository contract implemented. +- [ ] Candidate source contract implemented. +- [ ] Query engine consumes repository candidates. +- [ ] Provenance preserved. +- [ ] Deterministic behavior maintained. + +## Quality Criteria + +- [ ] Milestone 05 tests remain passing. +- [ ] Repository tests added. +- [ ] Package boundaries preserved. + +## Non Goals + +Not included: + +- Database +- Embeddings +- Vector search +- Ranking +- Agents +- MCP + +## Definition of Done + +KnowledgeOS has a stable access boundary between sources and query +execution. diff --git a/docs/milestones/milestone-06/FounderOS_Milestone_06_Codex_Execution_Prompt_v1.0.md b/docs/milestones/milestone-06/FounderOS_Milestone_06_Codex_Execution_Prompt_v1.0.md new file mode 100644 index 0000000..afb887f --- /dev/null +++ b/docs/milestones/milestone-06/FounderOS_Milestone_06_Codex_Execution_Prompt_v1.0.md @@ -0,0 +1,72 @@ +You are the lead engineer implementing FounderOS Milestone 06 --- +Knowledge Repository and Candidate Source Foundation. + +Before making changes, read: + +- README.md +- AGENTS.md +- CONTRIBUTING.md +- ARCHITECTURE_DECISIONS.md +- Repository audit +- Milestone 04 documents +- Milestone 05 documents +- Milestone 06 documents + +Review: + +- packages/knowledge-schema/ +- services/knowledge-engine/ + +Understand existing contracts, query flow, tests, and package +boundaries. + +Objective: + +Move KnowledgeOS from caller-supplied candidates to repository-supplied +candidates. + +Implement: + +1. Knowledge Repository Contract +2. Candidate Source Contract +3. Repository-backed query flow +4. Tests for retrieval, provenance, determinism, and regression + +Do not implement: + +- Database persistence +- Vector database +- Embeddings +- Semantic search +- Ranking +- Knowledge graph +- Agents +- Hermes +- MCP +- UI + +Follow: + +- Documentation first +- Architecture before code +- Preserve package boundaries +- Add tests +- Avoid unnecessary dependencies + +Run: + +pnpm format:check pnpm lint pnpm build pnpm typecheck pnpm test + +Final report: + +1. Status GO or NOT READY +2. Summary +3. Changed files +4. Tests +5. Verification +6. Architecture impact +7. Limitations +8. Next milestone recommendation + +Build a stable knowledge access foundation before adding intelligence +layers. diff --git a/docs/milestones/milestone-06/FounderOS_Milestone_06_Knowledge_Repository_and_Candidate_Source_Foundation_Specification_v1.0.md b/docs/milestones/milestone-06/FounderOS_Milestone_06_Knowledge_Repository_and_Candidate_Source_Foundation_Specification_v1.0.md new file mode 100644 index 0000000..1bef2bd --- /dev/null +++ b/docs/milestones/milestone-06/FounderOS_Milestone_06_Knowledge_Repository_and_Candidate_Source_Foundation_Specification_v1.0.md @@ -0,0 +1,44 @@ +# FounderOS Milestone 06 Knowledge Repository and Candidate Source Foundation Specification v1.0 + +## Purpose + +Define the repository abstraction layer that provides KnowledgeOS query +capabilities with managed knowledge sources. + +## Objective + +Move from Milestone 05 caller-supplied candidates to a controlled +Knowledge Repository foundation. + +Current: + +Caller -\> Query Engine -\> Knowledge Objects + +Target: + +Knowledge Repository -\> Candidate Source -\> Query Engine -\> Knowledge +Results + +## Scope + +Included: + +- Knowledge repository contract +- Candidate source contract +- Repository query integration +- Deterministic evaluation + +Excluded: + +- Database persistence +- Vector databases +- Embeddings +- Semantic search +- Ranking systems +- Knowledge graph +- Agents +- MCP integrations + +## Principle + +Separate knowledge access from knowledge intelligence. diff --git a/docs/milestones/milestone-06/FounderOS_Milestone_06_Verification_Checklist_v1.0.md b/docs/milestones/milestone-06/FounderOS_Milestone_06_Verification_Checklist_v1.0.md new file mode 100644 index 0000000..c73af7b --- /dev/null +++ b/docs/milestones/milestone-06/FounderOS_Milestone_06_Verification_Checklist_v1.0.md @@ -0,0 +1,29 @@ +# FounderOS Milestone 06 Verification Checklist v1.0 + +## Architecture + +- [ ] Repository boundary exists. +- [ ] Query engine remains storage-independent. +- [ ] Candidate sources are replaceable. + +## Functional + +- [ ] Objects retrieved through repository. +- [ ] Query filtering works. +- [ ] Provenance preserved. + +## Regression + +- [ ] Migration tests pass. +- [ ] Query tests pass. +- [ ] Repository tests pass. + +## Commands + +``` bash +pnpm format:check +pnpm lint +pnpm build +pnpm typecheck +pnpm test +``` diff --git a/packages/knowledge-schema/README.md b/packages/knowledge-schema/README.md index 137f2fe..108bca5 100644 --- a/packages/knowledge-schema/README.md +++ b/packages/knowledge-schema/README.md @@ -24,6 +24,8 @@ Milestone 04 adds strict migration-manifest contracts for object identity, objec Milestone 05 adds strict, versioned query and result contracts. Queries carry identity, consumer context, optional context constraints, and exact-match filters for object type, project, lifecycle status, tags, source, domain, and category. Results carry validated objects, matching source provenance, candidate and match counts, and the sorted set of applied constraints. The schema package defines these boundaries but does not execute queries. +Milestone 06 adds candidate-source and repository access contracts. A candidate batch binds a validated source descriptor and its provenance to schema-valid Knowledge Objects. The repository interface supports deterministic candidate listing, identity lookup, multi-identity finding, and source inspection. Provider execution and storage behavior remain outside this package. + ## Usage ```typescript @@ -46,4 +48,14 @@ const query = KnowledgeQuerySchema.parse(queryInput); const result = KnowledgeQueryResultSchema.parse(resultInput); ``` +```typescript +import { + KnowledgeCandidateBatchSchema, + type KnowledgeCandidateSource, + type KnowledgeRepository, +} from "@founderos/knowledge-schema"; + +const batch = KnowledgeCandidateBatchSchema.parse(candidateBatchInput); +``` + All schemas reject unknown fields so contract changes remain explicit and versioned. diff --git a/packages/knowledge-schema/package.json b/packages/knowledge-schema/package.json index a6dffce..6c54e20 100644 --- a/packages/knowledge-schema/package.json +++ b/packages/knowledge-schema/package.json @@ -2,7 +2,7 @@ "name": "@founderos/knowledge-schema", "version": "0.1.0", "private": true, - "description": "KnowledgeOS object, migration, and query contracts", + "description": "KnowledgeOS object, migration, query, and repository contracts", "type": "module", "files": [ "dist" diff --git a/packages/knowledge-schema/src/index.ts b/packages/knowledge-schema/src/index.ts index 13c4a03..790fcf1 100644 --- a/packages/knowledge-schema/src/index.ts +++ b/packages/knowledge-schema/src/index.ts @@ -6,3 +6,4 @@ export * from "./parse.js"; export * from "./primitives.js"; export * from "./query.js"; export * from "./query-result.js"; +export * from "./repository.js"; diff --git a/packages/knowledge-schema/src/parse.ts b/packages/knowledge-schema/src/parse.ts index 62f5ef5..9a5d69c 100644 --- a/packages/knowledge-schema/src/parse.ts +++ b/packages/knowledge-schema/src/parse.ts @@ -2,6 +2,14 @@ import { KnowledgeMetadataSchema, type KnowledgeMetadata } from "./metadata.js"; import { KnowledgeObjectSchema, type KnowledgeObject } from "./objects.js"; import { KnowledgeQuerySchema, type KnowledgeQuery } from "./query.js"; import { KnowledgeQueryResultSchema, type KnowledgeQueryResult } from "./query-result.js"; +import { + KnowledgeCandidateBatchSchema, + KnowledgeCandidateSourceDescriptorSchema, + KnowledgeRepositoryFindRequestSchema, + type KnowledgeCandidateBatch, + type KnowledgeCandidateSourceDescriptor, + type KnowledgeRepositoryFindRequest, +} from "./repository.js"; export function parseKnowledgeMetadata(input: unknown): KnowledgeMetadata { return KnowledgeMetadataSchema.parse(input); @@ -34,3 +42,31 @@ export function parseKnowledgeQueryResult(input: unknown): KnowledgeQueryResult export function safeParseKnowledgeQueryResult(input: unknown) { return KnowledgeQueryResultSchema.safeParse(input); } + +export function parseKnowledgeCandidateSourceDescriptor( + input: unknown, +): KnowledgeCandidateSourceDescriptor { + return KnowledgeCandidateSourceDescriptorSchema.parse(input); +} + +export function safeParseKnowledgeCandidateSourceDescriptor(input: unknown) { + return KnowledgeCandidateSourceDescriptorSchema.safeParse(input); +} + +export function parseKnowledgeCandidateBatch(input: unknown): KnowledgeCandidateBatch { + return KnowledgeCandidateBatchSchema.parse(input); +} + +export function safeParseKnowledgeCandidateBatch(input: unknown) { + return KnowledgeCandidateBatchSchema.safeParse(input); +} + +export function parseKnowledgeRepositoryFindRequest( + input: unknown, +): KnowledgeRepositoryFindRequest { + return KnowledgeRepositoryFindRequestSchema.parse(input); +} + +export function safeParseKnowledgeRepositoryFindRequest(input: unknown) { + return KnowledgeRepositoryFindRequestSchema.safeParse(input); +} diff --git a/packages/knowledge-schema/src/repository.ts b/packages/knowledge-schema/src/repository.ts new file mode 100644 index 0000000..5c94c92 --- /dev/null +++ b/packages/knowledge-schema/src/repository.ts @@ -0,0 +1,70 @@ +import { z } from "zod"; + +import { SourceMetadataSchema } from "./metadata.js"; +import { KnowledgeObjectSchema, type KnowledgeObject } from "./objects.js"; +import { IdentifierSchema, NonEmptyStringSchema } from "./primitives.js"; + +export const KnowledgeCandidateSourceDescriptorSchema = z + .object({ + schemaVersion: z.literal("1.0"), + sourceId: IdentifierSchema, + sourceType: NonEmptyStringSchema, + provenance: SourceMetadataSchema, + }) + .strict(); + +export const KnowledgeCandidateBatchSchema = z + .object({ + schemaVersion: z.literal("1.0"), + source: KnowledgeCandidateSourceDescriptorSchema, + candidates: z.array(KnowledgeObjectSchema), + }) + .strict() + .superRefine((batch, context) => { + const indexesById = new Map(); + + batch.candidates.forEach((candidate, index) => { + indexesById.set(candidate.metadata.id, [ + ...(indexesById.get(candidate.metadata.id) ?? []), + index, + ]); + }); + + for (const [id, indexes] of indexesById) { + if (indexes.length > 1) { + for (const index of indexes) { + context.addIssue({ + code: "custom", + message: `Knowledge object ID ${id} is duplicated by the candidate source`, + path: ["candidates", index, "metadata", "id"], + }); + } + } + } + }); + +export const KnowledgeRepositoryFindRequestSchema = z + .object({ + ids: z + .array(IdentifierSchema) + .min(1) + .refine((ids) => new Set(ids).size === ids.length, "Knowledge object IDs must be unique"), + }) + .strict(); + +export type KnowledgeCandidateSourceDescriptor = z.infer< + typeof KnowledgeCandidateSourceDescriptorSchema +>; +export type KnowledgeCandidateBatch = z.infer; +export type KnowledgeRepositoryFindRequest = z.infer; + +export interface KnowledgeCandidateSource { + loadCandidates(): Promise; +} + +export interface KnowledgeRepository { + find(request: KnowledgeRepositoryFindRequest): Promise; + getById(id: string): Promise; + getCandidates(): Promise; + getSources(): Promise; +} diff --git a/packages/knowledge-schema/tests/repository.test.ts b/packages/knowledge-schema/tests/repository.test.ts new file mode 100644 index 0000000..a3778c3 --- /dev/null +++ b/packages/knowledge-schema/tests/repository.test.ts @@ -0,0 +1,104 @@ +import { describe, expect, it } from "vitest"; + +import { + KnowledgeCandidateBatchSchema, + KnowledgeCandidateSourceDescriptorSchema, + KnowledgeRepositoryFindRequestSchema, + parseKnowledgeCandidateBatch, + parseKnowledgeCandidateSourceDescriptor, + safeParseKnowledgeCandidateBatch, +} from "../src/index.js"; +import { createMetadata } from "./fixtures.js"; + +function descriptor() { + return { + schemaVersion: "1.0", + sourceId: "priority-one-memory", + sourceType: "in_memory", + provenance: { + sourceType: "migration_report", + sourceReference: "knowledge/migration-manifest.yaml", + originalCreator: "FounderOS", + }, + }; +} + +function candidate(id = "knowledge-001") { + return { + metadata: { ...createMetadata("knowledge"), id }, + content: "Validated FounderOS knowledge.", + }; +} + +describe("KnowledgeCandidateSourceDescriptorSchema", () => { + it("parses source identity, type, and provenance", () => { + expect(parseKnowledgeCandidateSourceDescriptor(descriptor())).toEqual(descriptor()); + }); + + it.each([ + ["missing identity", { ...descriptor(), sourceId: "" }], + ["missing provenance", { ...descriptor(), provenance: undefined }], + ["unknown field", { ...descriptor(), connectionString: "private" }], + ])("rejects %s", (_label, input) => { + expect(KnowledgeCandidateSourceDescriptorSchema.safeParse(input).success).toBe(false); + }); +}); + +describe("KnowledgeCandidateBatchSchema", () => { + it("returns validated candidates with object provenance intact", () => { + const batch = parseKnowledgeCandidateBatch({ + schemaVersion: "1.0", + source: descriptor(), + candidates: [candidate()], + }); + + expect(batch.candidates[0]?.metadata.source).toEqual(candidate().metadata.source); + }); + + it("supports an empty candidate source", () => { + expect( + KnowledgeCandidateBatchSchema.parse({ + schemaVersion: "1.0", + source: descriptor(), + candidates: [], + }).candidates, + ).toEqual([]); + }); + + it("rejects invalid candidates and duplicate identities", () => { + expect( + safeParseKnowledgeCandidateBatch({ + schemaVersion: "1.0", + source: descriptor(), + candidates: [{ metadata: {} }], + }).success, + ).toBe(false); + + const duplicate = { + schemaVersion: "1.0", + source: descriptor(), + candidates: [candidate(), candidate()], + }; + const result = safeParseKnowledgeCandidateBatch(duplicate); + + expect(result.success).toBe(false); + if (!result.success) { + expect(result.error.issues.map((issue) => issue.path.join("."))).toEqual([ + "candidates.0.metadata.id", + "candidates.1.metadata.id", + ]); + } + }); +}); + +describe("KnowledgeRepositoryFindRequestSchema", () => { + it("accepts unique identities and rejects empty or duplicate requests", () => { + expect(KnowledgeRepositoryFindRequestSchema.parse({ ids: ["one", "two"] })).toEqual({ + ids: ["one", "two"], + }); + expect(KnowledgeRepositoryFindRequestSchema.safeParse({ ids: [] }).success).toBe(false); + expect(KnowledgeRepositoryFindRequestSchema.safeParse({ ids: ["one", "one"] }).success).toBe( + false, + ); + }); +}); diff --git a/services/knowledge-engine/README.md b/services/knowledge-engine/README.md index 4cd7c3c..74cf299 100644 --- a/services/knowledge-engine/README.md +++ b/services/knowledge-engine/README.md @@ -6,16 +6,30 @@ Milestone 04 adds manifest-controlled corpus execution. It loads a strict YAML m Milestone 05 adds a pure, storage-free query boundary over an explicitly supplied set of validated Knowledge Objects. It validates the query and every candidate, rejects duplicate object IDs, applies exact filters and context constraints by intersection, and sorts results by object ID. Multi-value filters match any allowed value; tag filters explicitly support `all` or `any`. Project filters match an object's domain, a project object's ID or name, or a decision object's `relatedProjectIds`. Every result includes the object's unchanged source metadata as provenance. +Milestone 06 makes repository-backed querying the primary access flow. Candidate sources emit versioned batches; the in-memory repository revalidates them, rejects duplicate source or object identities, builds a deterministic snapshot, and returns independent validated copies. It provides identity lookup and candidate discovery without persistence or search intelligence. The Milestone 05 candidate-array function remains available as the compatibility filtering core. + ```typescript import { queryKnowledgeObjects } from "@founderos/knowledge-engine"; const result = queryKnowledgeObjects(query, candidateObjects); ``` +```typescript +import { + InMemoryKnowledgeCandidateSource, + InMemoryKnowledgeRepository, + queryKnowledgeRepository, +} from "@founderos/knowledge-engine"; + +const source = new InMemoryKnowledgeCandidateSource(sourceDescriptor, candidateObjects); +const repository = await InMemoryKnowledgeRepository.create([source]); +const result = await queryKnowledgeRepository(query, repository); +``` + From the repository root: ```bash pnpm knowledge:migrate ``` -Directory ingestion is recursive, Markdown-only, stable in path order, and does not follow symbolic links. Querying remains deterministic exact filtering—not full-text search, semantic retrieval, ranking, or authorization. The implementation remains read-only and does not watch a vault or implement persistence, embeddings, graph storage, Hermes, agents, or MCP integrations. +Directory ingestion is recursive, Markdown-only, stable in path order, and does not follow symbolic links. Repository access is an immutable in-memory snapshot, not durable storage. Querying remains deterministic exact filtering—not full-text search, semantic retrieval, ranking, or authorization. The implementation remains read-only and does not watch a vault or implement persistence, embeddings, graph storage, Hermes, agents, or MCP integrations. diff --git a/services/knowledge-engine/package.json b/services/knowledge-engine/package.json index 8e364ee..404c317 100644 --- a/services/knowledge-engine/package.json +++ b/services/knowledge-engine/package.json @@ -2,7 +2,7 @@ "name": "@founderos/knowledge-engine", "version": "0.1.0", "private": true, - "description": "KnowledgeOS ingestion, migration, and query foundation", + "description": "KnowledgeOS ingestion, migration, repository, and query foundation", "type": "module", "files": [ "dist" diff --git a/services/knowledge-engine/src/application/query-knowledge-repository.ts b/services/knowledge-engine/src/application/query-knowledge-repository.ts new file mode 100644 index 0000000..17e84dc --- /dev/null +++ b/services/knowledge-engine/src/application/query-knowledge-repository.ts @@ -0,0 +1,11 @@ +import type { KnowledgeQueryResult, KnowledgeRepository } from "@founderos/knowledge-schema"; + +import { queryKnowledgeObjects } from "./query-knowledge.js"; + +export async function queryKnowledgeRepository( + queryInput: unknown, + repository: KnowledgeRepository, +): Promise { + const candidates = await repository.getCandidates(); + return queryKnowledgeObjects(queryInput, candidates); +} diff --git a/services/knowledge-engine/src/domain/knowledge-query.ts b/services/knowledge-engine/src/domain/knowledge-query.ts index 698492a..61691bf 100644 --- a/services/knowledge-engine/src/domain/knowledge-query.ts +++ b/services/knowledge-engine/src/domain/knowledge-query.ts @@ -4,3 +4,10 @@ export class DuplicateKnowledgeObjectIdError extends Error { this.name = "DuplicateKnowledgeObjectIdError"; } } + +export class DuplicateKnowledgeCandidateSourceIdError extends Error { + public constructor(id: string) { + super(`Duplicate knowledge candidate source ID: ${id}`); + this.name = "DuplicateKnowledgeCandidateSourceIdError"; + } +} diff --git a/services/knowledge-engine/src/index.ts b/services/knowledge-engine/src/index.ts index 26624bd..79ae73f 100644 --- a/services/knowledge-engine/src/index.ts +++ b/services/knowledge-engine/src/index.ts @@ -3,11 +3,14 @@ export * from "./application/ingest-markdown-directory.js"; export * from "./application/execute-knowledge-migration.js"; export * from "./application/normalize-frontmatter.js"; export * from "./application/query-knowledge.js"; +export * from "./application/query-knowledge-repository.js"; export * from "./application/run-migration-command.js"; export * from "./domain/frontmatter.js"; export * from "./domain/knowledge-query.js"; export * from "./domain/safe-path.js"; export * from "./infrastructure/load-migration-manifest.js"; +export * from "./infrastructure/in-memory-candidate-source.js"; +export * from "./infrastructure/in-memory-knowledge-repository.js"; export * from "./infrastructure/parse-markdown.js"; export * from "./infrastructure/safe-path.js"; export * from "./interfaces/ingestion-report.js"; diff --git a/services/knowledge-engine/src/infrastructure/in-memory-candidate-source.ts b/services/knowledge-engine/src/infrastructure/in-memory-candidate-source.ts new file mode 100644 index 0000000..7d3600d --- /dev/null +++ b/services/knowledge-engine/src/infrastructure/in-memory-candidate-source.ts @@ -0,0 +1,21 @@ +import { + KnowledgeCandidateBatchSchema, + type KnowledgeCandidateBatch, + type KnowledgeCandidateSource, +} from "@founderos/knowledge-schema"; + +export class InMemoryKnowledgeCandidateSource implements KnowledgeCandidateSource { + readonly #batch: KnowledgeCandidateBatch; + + public constructor(sourceInput: unknown, candidateInputs: readonly unknown[]) { + this.#batch = KnowledgeCandidateBatchSchema.parse({ + schemaVersion: "1.0", + source: sourceInput, + candidates: candidateInputs, + }); + } + + public async loadCandidates(): Promise { + return KnowledgeCandidateBatchSchema.parse(this.#batch); + } +} diff --git a/services/knowledge-engine/src/infrastructure/in-memory-knowledge-repository.ts b/services/knowledge-engine/src/infrastructure/in-memory-knowledge-repository.ts new file mode 100644 index 0000000..539546b --- /dev/null +++ b/services/knowledge-engine/src/infrastructure/in-memory-knowledge-repository.ts @@ -0,0 +1,102 @@ +import { + IdentifierSchema, + KnowledgeCandidateBatchSchema, + KnowledgeCandidateSourceDescriptorSchema, + KnowledgeObjectSchema, + KnowledgeRepositoryFindRequestSchema, + type KnowledgeCandidateBatch, + type KnowledgeCandidateSource, + type KnowledgeCandidateSourceDescriptor, + type KnowledgeObject, + type KnowledgeRepository, + type KnowledgeRepositoryFindRequest, +} from "@founderos/knowledge-schema"; + +import { + DuplicateKnowledgeCandidateSourceIdError, + DuplicateKnowledgeObjectIdError, +} from "../domain/knowledge-query.js"; + +function compareStrings(left: string, right: string): number { + return left < right ? -1 : left > right ? 1 : 0; +} + +function copyObject(object: KnowledgeObject): KnowledgeObject { + return KnowledgeObjectSchema.parse(object); +} + +function copyDescriptor( + descriptor: KnowledgeCandidateSourceDescriptor, +): KnowledgeCandidateSourceDescriptor { + return KnowledgeCandidateSourceDescriptorSchema.parse(descriptor); +} + +export class InMemoryKnowledgeRepository implements KnowledgeRepository { + readonly #objects: KnowledgeObject[]; + readonly #objectsById: Map; + readonly #sources: KnowledgeCandidateSourceDescriptor[]; + + private constructor(objects: KnowledgeObject[], sources: KnowledgeCandidateSourceDescriptor[]) { + this.#objects = objects; + this.#objectsById = new Map(objects.map((object) => [object.metadata.id, object])); + this.#sources = sources; + } + + public static async create( + candidateSources: readonly KnowledgeCandidateSource[] = [], + ): Promise { + const batches: KnowledgeCandidateBatch[] = []; + + for (const candidateSource of candidateSources) { + batches.push(KnowledgeCandidateBatchSchema.parse(await candidateSource.loadCandidates())); + } + + batches.sort((left, right) => compareStrings(left.source.sourceId, right.source.sourceId)); + const sourceIds = new Set(); + const objectIds = new Set(); + const objects: KnowledgeObject[] = []; + + for (const batch of batches) { + if (sourceIds.has(batch.source.sourceId)) { + throw new DuplicateKnowledgeCandidateSourceIdError(batch.source.sourceId); + } + sourceIds.add(batch.source.sourceId); + + for (const candidate of batch.candidates) { + if (objectIds.has(candidate.metadata.id)) { + throw new DuplicateKnowledgeObjectIdError(candidate.metadata.id); + } + objectIds.add(candidate.metadata.id); + objects.push(candidate); + } + } + + objects.sort((left, right) => compareStrings(left.metadata.id, right.metadata.id)); + return new InMemoryKnowledgeRepository( + objects.map(copyObject), + batches.map((batch) => copyDescriptor(batch.source)), + ); + } + + public async find( + requestInput: KnowledgeRepositoryFindRequest, + ): Promise { + const request = KnowledgeRepositoryFindRequestSchema.parse(requestInput); + const requestedIds = new Set(request.ids); + return this.#objects.filter((object) => requestedIds.has(object.metadata.id)).map(copyObject); + } + + public async getById(idInput: string): Promise { + const id = IdentifierSchema.parse(idInput); + const object = this.#objectsById.get(id); + return object === undefined ? null : copyObject(object); + } + + public async getCandidates(): Promise { + return this.#objects.map(copyObject); + } + + public async getSources(): Promise { + return this.#sources.map(copyDescriptor); + } +} diff --git a/services/knowledge-engine/tests/knowledge-repository.test.ts b/services/knowledge-engine/tests/knowledge-repository.test.ts new file mode 100644 index 0000000..b6be7cc --- /dev/null +++ b/services/knowledge-engine/tests/knowledge-repository.test.ts @@ -0,0 +1,169 @@ +import type { KnowledgeCandidateSource, KnowledgeObject } from "@founderos/knowledge-schema"; +import { describe, expect, it } from "vitest"; + +import { + DuplicateKnowledgeCandidateSourceIdError, + DuplicateKnowledgeObjectIdError, + InMemoryKnowledgeCandidateSource, + InMemoryKnowledgeRepository, +} from "../src/index.js"; + +function descriptor(sourceId: string) { + return { + schemaVersion: "1.0", + sourceId, + sourceType: "in_memory", + provenance: { + sourceType: "migration_report", + sourceReference: `reports/${sourceId}.json`, + originalCreator: "FounderOS", + }, + }; +} + +function knowledgeObject(id: string, tag = "architecture"): KnowledgeObject { + return { + metadata: { + id, + title: id, + objectType: "knowledge", + domain: "FounderOS", + source: { + sourceType: "official_specification", + sourceReference: `docs/${id}.md`, + originalCreator: "FounderOS", + }, + createdAt: "2026-07-28", + updatedAt: "2026-07-28", + status: "active", + confidence: "high", + importance: "high", + tags: [tag], + relationships: [], + }, + content: `${id} content`, + }; +} + +describe("InMemoryKnowledgeCandidateSource", () => { + it("validates source identity and every candidate", () => { + expect(() => new InMemoryKnowledgeCandidateSource({ sourceId: "" }, [])).toThrow(); + expect( + () => + new InMemoryKnowledgeCandidateSource(descriptor("invalid-candidates"), [{ metadata: {} }]), + ).toThrow(); + }); + + it("preserves source and object provenance without exposing mutable state", async () => { + const input = knowledgeObject("object-one"); + const source = new InMemoryKnowledgeCandidateSource(descriptor("source-one"), [input]); + input.metadata.title = "Mutated after construction"; + + const first = await source.loadCandidates(); + first.candidates[0]!.metadata.title = "Mutated returned copy"; + const second = await source.loadCandidates(); + + expect(second.source.provenance).toEqual(descriptor("source-one").provenance); + expect(second.candidates[0]?.metadata).toMatchObject({ + title: "object-one", + source: { + originalCreator: "FounderOS", + sourceReference: "docs/object-one.md", + sourceType: "official_specification", + }, + }); + }); +}); + +describe("InMemoryKnowledgeRepository", () => { + it("retrieves validated objects deterministically and looks up identity", async () => { + const source = new InMemoryKnowledgeCandidateSource(descriptor("source-one"), [ + knowledgeObject("z-object"), + knowledgeObject("a-object"), + ]); + const repository = await InMemoryKnowledgeRepository.create([source]); + + expect((await repository.getCandidates()).map((object) => object.metadata.id)).toEqual([ + "a-object", + "z-object", + ]); + expect((await repository.getById("z-object"))?.metadata.id).toBe("z-object"); + expect(await repository.getById("missing-object")).toBeNull(); + expect( + (await repository.find({ ids: ["z-object", "missing-object", "a-object"] })).map( + (object) => object.metadata.id, + ), + ).toEqual(["a-object", "z-object"]); + }); + + it("supports an empty repository", async () => { + const repository = await InMemoryKnowledgeRepository.create(); + + expect(await repository.getCandidates()).toEqual([]); + expect(await repository.getSources()).toEqual([]); + expect(await repository.getById("missing-object")).toBeNull(); + expect(await repository.find({ ids: ["missing-object"] })).toEqual([]); + }); + + it("returns independent object copies", async () => { + const source = new InMemoryKnowledgeCandidateSource(descriptor("source-one"), [ + knowledgeObject("object-one"), + ]); + const repository = await InMemoryKnowledgeRepository.create([source]); + const first = await repository.getById("object-one"); + first!.metadata.title = "Changed by caller"; + + expect((await repository.getById("object-one"))?.metadata.title).toBe("object-one"); + }); + + it("exposes candidate source identity and provenance in stable order", async () => { + const repository = await InMemoryKnowledgeRepository.create([ + new InMemoryKnowledgeCandidateSource(descriptor("z-source"), []), + new InMemoryKnowledgeCandidateSource(descriptor("a-source"), []), + ]); + + const sources = await repository.getSources(); + expect(sources.map((source) => source.sourceId)).toEqual(["a-source", "z-source"]); + expect(sources[0]?.provenance.sourceReference).toBe("reports/a-source.json"); + }); + + it("rejects duplicate object identities across candidate sources", async () => { + const first = new InMemoryKnowledgeCandidateSource(descriptor("first-source"), [ + knowledgeObject("duplicate-object"), + ]); + const second = new InMemoryKnowledgeCandidateSource(descriptor("second-source"), [ + knowledgeObject("duplicate-object"), + ]); + + await expect(InMemoryKnowledgeRepository.create([first, second])).rejects.toThrow( + DuplicateKnowledgeObjectIdError, + ); + }); + + it("rejects duplicate candidate source identities", async () => { + const first = new InMemoryKnowledgeCandidateSource(descriptor("duplicate-source"), [ + knowledgeObject("first-object"), + ]); + const second = new InMemoryKnowledgeCandidateSource(descriptor("duplicate-source"), [ + knowledgeObject("second-object"), + ]); + + await expect(InMemoryKnowledgeRepository.create([first, second])).rejects.toThrow( + DuplicateKnowledgeCandidateSourceIdError, + ); + }); + + it("revalidates candidate source output at the repository boundary", async () => { + const invalidSource = { + async loadCandidates() { + return { + schemaVersion: "1.0", + source: descriptor("invalid-source"), + candidates: [{ metadata: {} }], + }; + }, + } as unknown as KnowledgeCandidateSource; + + await expect(InMemoryKnowledgeRepository.create([invalidSource])).rejects.toThrow(); + }); +}); diff --git a/services/knowledge-engine/tests/repository-query.test.ts b/services/knowledge-engine/tests/repository-query.test.ts new file mode 100644 index 0000000..822ee0b --- /dev/null +++ b/services/knowledge-engine/tests/repository-query.test.ts @@ -0,0 +1,89 @@ +import { resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +import type { KnowledgeObject } from "@founderos/knowledge-schema"; +import { beforeAll, describe, expect, it } from "vitest"; + +import { + executeKnowledgeMigration, + InMemoryKnowledgeCandidateSource, + InMemoryKnowledgeRepository, + queryKnowledgeObjects, + queryKnowledgeRepository, + serializeKnowledgeQueryResult, +} from "../src/index.js"; +import { PRIORITY_ONE_QUERY_EVALUATIONS } from "./fixtures/query-evaluations.js"; + +const REPOSITORY_ROOT = resolve(fileURLToPath(new URL("../../../", import.meta.url))); +let priorityOneObjects: KnowledgeObject[] = []; + +function source(objects: readonly KnowledgeObject[]) { + return new InMemoryKnowledgeCandidateSource( + { + schemaVersion: "1.0", + sourceId: "founderos-priority-one", + sourceType: "in_memory", + provenance: { + sourceType: "migration_manifest", + sourceReference: "knowledge/migration-manifest.yaml", + originalCreator: "FounderOS", + }, + }, + objects, + ); +} + +beforeAll(async () => { + const report = await executeKnowledgeMigration({ + manifestPath: "knowledge/migration-manifest.yaml", + rootPath: REPOSITORY_ROOT, + }); + + expect(report.status).toBe("accepted"); + priorityOneObjects = report.documents.flatMap((document) => + document.status === "accepted" ? [document.object] : [], + ); +}); + +describe("repository-backed knowledge queries", () => { + for (const fixture of PRIORITY_ONE_QUERY_EVALUATIONS) { + it(`preserves the Milestone 05 ${fixture.name} evaluation`, async () => { + const repository = await InMemoryKnowledgeRepository.create([source(priorityOneObjects)]); + const result = await queryKnowledgeRepository(fixture.query, repository); + + expect(result.objects.map((object) => object.metadata.id)).toEqual(fixture.expectedObjectIds); + expect(result.provenance).toEqual( + result.objects.map((object) => ({ + objectId: object.metadata.id, + source: object.metadata.source, + })), + ); + }); + } + + it("is byte-identical to the Milestone 05 candidate-array flow", async () => { + const query = PRIORITY_ONE_QUERY_EVALUATIONS[0]!.query; + const repository = await InMemoryKnowledgeRepository.create([ + source([...priorityOneObjects].reverse()), + ]); + + const repositoryResult = await queryKnowledgeRepository(query, repository); + const milestoneFiveResult = queryKnowledgeObjects(query, priorityOneObjects); + + expect(serializeKnowledgeQueryResult(repositoryResult)).toBe( + serializeKnowledgeQueryResult(milestoneFiveResult), + ); + }); + + it("returns a valid result from an empty repository", async () => { + const repository = await InMemoryKnowledgeRepository.create(); + const result = await queryKnowledgeRepository( + PRIORITY_ONE_QUERY_EVALUATIONS[0]!.query, + repository, + ); + + expect(result.objects).toEqual([]); + expect(result.provenance).toEqual([]); + expect(result.evaluation).toMatchObject({ candidateCount: 0, matchedCount: 0 }); + }); +});