docs: Decide the initial chunking and retrieval strategy

[Williamzhibo]

What this changes

Documents the recommended RAG chunking and retrieval strategies based on our sample Google Docs cases: 500-token chunks with ~10% token overlap, and hybrid keyword/vector search.

Why

Closes #208

This spike evaluates chunking and retrieval strategies against representative Google Docs content so the decisions can be carried into the downstream chunking and indexing work.

• Chunking: Recommend 500-token chunks with ~10% token overlap based on the sample cases tested.
• Retrieval: Recommend hybrid keyword/vector search to support both specific name/identifier matches within our organization and semantically similar but distinct events.

The evaluation is limited to the currently available Google Docs corpus and should not be assumed to generalize to future content sources such as GitHub.

Zone

services/documentation-system

How to verify

Review services/documentation-system/docs/ARCHITECTURE.md and confirm that it documents:

• The 500-token chunk size and ~10% overlap recommendation.
• The reasoning and sample document cases used to evaluate the chunking strategy.
• The hybrid keyword/vector search recommendation and its reasoning.
• The limitations of the evaluation and its scope to the current Google Docs corpus.

Checklist

• Branched off staging and targeting staging (or this is a deliberate staging → main promotion).
• Ran the service's test suite locally — the full one, including Postgres if the service has a database.
• uv run ruff check . and uv run ruff format --check . clean — CI gates both on every Python job.
• Read and followed the service's docs/CONTRIBUTING.md pre-push checklist.
• Docs updated in this PR where the change makes them wrong — ARCHITECTURE.md for a new trade-off.

Deployment notes

No deployment changes. This PR documents architecture decisions only.

Anything you're unsure about

The exact tokenization behavior depends on the embedding model selected in #173. The 500-token recommendation is based on the spike's sample cases and should be revisited if the selected embedding model introduces relevant tokenization or input-size constraints.

[angelayzheng]

LGTM (as we discussed in meeting today)!