Serverless Second Brain

⚠️ Work in progress — Phases 1–2 deployed and functional, Phases 3–4 not yet started.

Serverless backend for a personal knowledge graph on AWS. DynamoDB, Lambda, Bedrock, API Gateway, CloudFront, and Cognito.

From the essay: From Prototype to Production: A Serverless Second Brain on AWS

Architecture

flowchart TB
  accTitle: Serverless second brain architecture
  accDescr: Three layers — Interface, Compute, Memory — with AWS services per block and two access doors for humans and AI agents

  User((User))
  Agent((AI Agent))

  subgraph Interface
    CF[CloudFront + S3]
    APIGW[API Gateway REST]
    GW[AgentCore Gateway]
  end

  subgraph Compute
    LCapture[Lambda Capture]
    LSearch[Lambda Search]
    LGraph[Lambda Graph]
    ACR[AgentCore Runtime]
    EB[EventBridge Scheduler]
    LSurface[Lambda Surfacing]
  end

  subgraph Memory
    DDB[(DynamoDB)]
    S3C[S3 Content]
  end

  subgraph AI
    BR[Bedrock Claude]
    BRE[Bedrock Embeddings]
  end

  User -- "browser" --> CF
  User -- "REST API" --> APIGW
  Agent -- "MCP" --> GW

  APIGW -- "/capture" --> LCapture
  APIGW -- "/search" --> LSearch
  APIGW -- "/graph" --> LGraph

  LCapture -- "async" --> LEnrich[Lambda Enrich]

  GW -- "MCP tools" --> LCapture
  GW -- "MCP tools" --> LSearch
  GW -- "MCP tools" --> LGraph

  ACR -- "reasons" --> BR
  EB -- "daily cron" --> LSurface

  LCapture -- "writes" --> DDB
  LCapture -- "stores MDX" --> S3C
  LCapture -- "classifies" --> BR
  LCapture -- "embeddings" --> BRE

  LSearch -- "queries" --> DDB
  LSearch -- "query embedding" --> BRE
  LGraph -- "reads edges" --> DDB
  LSurface -- "reads" --> DDB
  LSurface -- "notifies" --> SNS[SNS]

Loading

Two doors, same brain

Door	Protocol	Auth	Use case
Human	REST API (API Gateway)	Cognito JWT for writes, open reads	SPA, curl, integrations
Agent	MCP (AgentCore Gateway)	OAuth + semantic discovery	AI agents, Copilot, Claude

Both doors call the same Lambda functions — the difference is protocol and auth.

Cost

Scales to zero. No minimum costs beyond S3 storage.

Load	Monthly cost
Idle (0 req/day)	~$0.51
Moderate (100 req/day)	~$2.44
High (1,000 req/day)	~$11.21

Real cost data: docs/benchmarks/results.md

API

Method	Endpoint	Description	Auth
POST	/capture	Ingest text → Bedrock classifies → DynamoDB	Cognito JWT
GET	/search?q=	Hybrid keyword + semantic search	None
GET	/graph	Full knowledge graph (nodes + edges)	None
GET	/nodes/{id}	Single node with edges and related nodes	None
GET	/health	Service health check	None

Full API spec: .kiro/steering/api-spec.md

MCP tools (agent door)

Tool	Lambda	Description
read_node	Graph	Read a single node by slug
list_nodes	Graph	List nodes with filters
search	Search	Semantic + keyword search
add_node	Capture	Create a new knowledge node
connect_nodes	Capture	Create an edge between nodes
flag_stale	Surfacing	Mark a node for review

Full MCP spec: .kiro/steering/mcp-tools.md

Phased delivery

Each phase is independently deployable.

Phase	Components	Status
1 — Capture	Terraform foundation, DynamoDB, S3, Capture Lambda, API Gateway, Cognito auth	✅ Deployed
2 — Read	Search Lambda, Graph Lambda, CloudFront + S3 frontend	✅ Deployed
3 — Agent	AgentCore Gateway + Runtime, MCP tools, reasoning agent	🔲 Not started
4 — Proactive	EventBridge scheduler, Surfacing Lambda, SNS notifications	🔲 Not started

Architecture Decision Records

Every significant technical decision is documented in docs/decisions/:

ADR	Decision	Status
001	Lambda packaging (zip) with no web framework	Accepted
002	Write safety — 6 controls for MCP agent mutations	Accepted
003	Cognito authentication and visibility model	Accepted
004	DynamoDB single-table design with 2 GSIs	Accepted
005	Hybrid keyword + semantic search	Accepted
006	Step Functions Express for capture pipeline	Superseded
007	AgentCore Gateway over self-hosted MCP server	Accepted
008	In-memory embedding scan (temporary)	Accepted
009	Spec-Driven Development approach	Accepted
010	Bedrock token optimization — recent slugs	Accepted
011	CloudFront + S3 over Vercel/Amplify	Accepted
012	GitHub Actions OIDC over static credentials	Accepted
013	Vite SPA over Next.js	Accepted
014	Async enrichment — split capture into sync + async	Accepted

ADR-006 superseded: capture now uses a monolithic Lambda handler via API Gateway Lambda proxy integration. Step Functions infrastructure and step handlers have been fully removed.

Prerequisites

• AWS account with Bedrock model access enabled:
  • us.anthropic.claude-sonnet-4-20250514-v1:0
  • amazon.titan-embed-text-v2:0
• Terraform >= 1.5
• Node.js 22.x
• GitHub repo with OIDC configured for AWS

Project structure

infra/
  bootstrap/              → One-time Terraform state backend
  modules/                → Reusable Terraform modules
  environments/{dev,prod} → Environment roots

src/
  functions/{capture,enrich,search,graph,connect,flag,surfacing}/
  shared/                 → Types, clients, HTTP helpers, math, key prefixes

frontend/                 → React SPA (Vite + TailwindCSS)

scripts/                  → Migration, benchmarks, utilities
docs/decisions/           → ADRs
docs/benchmarks/          → Performance and cost data
.kiro/steering/           → SDD specifications
.github/workflows/        → CI/CD (GitHub Actions OIDC)

Specifications

All contracts defined before code — Spec-Driven Development:

• architecture.md — layers, services, cost constraints, phases
• dynamodb-schema.md — single-table design, item types, GSIs
• api-spec.md — REST endpoints, schemas, error codes
• mcp-tools.md — MCP tool definitions, write safety rules
• event-schemas.md — EventBridge, SNS contracts
• terraform-conventions.md — naming, modules, state, security
• code-conventions.md — TypeScript, Lambda patterns, commits

Development

# Bootstrap (one-time)
cd infra/bootstrap && terraform init && terraform apply

# Deploy
cd infra/environments/dev && terraform init && terraform apply

# Lambda development
cd src/functions/capture && npm install && npm run build

License

MIT