ContextMesh

ContextMesh is an enterprise private-knowledge retrieval and context-orchestration system. It combines document ingestion, tenant-aware hybrid retrieval, evidence-preserving context assembly, and streaming RAG chat in one Spring Boot and Vue application.

The working local application is available at http://localhost:9527 after startup. The backend listens on http://localhost:8081/api/v1.

What It Does

• Uploads Word, PDF, TXT, and other documents with multipart resume support.
• Processes parsing, chunking, embedding, and indexing asynchronously through Kafka.
• Stores durable application and chat data in MySQL, short-lived state in Redis, and files in MinIO.
• Runs independent authorized BM25 and KNN recall, deterministic RRF fusion, Qwen3 Cross-Encoder reranking, parent-context restoration, and Dynamic Top-K selection.
• Applies the same owner, organization, and public-document permission boundary to both recall branches.
• Streams cited answers to the Vue 3 chat interface and preserves persistent chat history.

Architecture

Browser (Vue 3)
  |  REST / WebSocket
Spring Boot API
  |-- MySQL: users, organizations, documents, durable chat history
  |-- Redis: sessions, short-lived context, upload bitmap
  |-- MinIO: source files and multipart objects
  |-- Kafka: parse -> chunk -> embed -> index pipeline
  `-- Elasticsearch
        |-- authorized BM25 recall ----\
        |-- authorized KNN recall -----+-> RRF -> Qwen3 rerank
        `-- child chunks + parent text /            |
                                      parent restore -> Dynamic Top-K -> LLM

See docs/contextmesh-architecture.md for component contracts, fallbacks, permission handling, and measurement methodology.

Measured Results

Retrieval results use a frozen bounded public subset of BEIR SciFact/NFCorpus: 48 queries, 12 per category, 3,931 parent documents, and 7,212 child chunks. This is an offline project benchmark, not a full-corpus BEIR leaderboard or production evaluation.

Retrieval metric	LEGACY	FULL	Change
Recall@5	0.4910	0.5480	+5.703 pp
Recall@10	0.5153	0.5555	+4.027 pp
MRR@10	0.7360	0.8318	+9.580 pp
NDCG@10	0.6351	0.7125	+7.739 pp
Average latency	2413.195 ms	2600.790 ms	+187.596 ms
P95 latency	2462.854 ms	2643.513 ms	+180.659 ms

For the 12-query cross-section category, parent restoration increased Recall@10 from 0.2937 to 0.3543 and evidence retention from 61.290% to 87.097%. Dynamic Top-K selected 5.75 fragments on average instead of 20 reranked candidates, retained 86.667% of measured evidence, and reduced retrieval P95 from 3037.491 ms to 2643.513 ms. Its fixed Top-10 token baseline was not captured, so no token-reduction percentage is claimed.

The upload benchmark uses a 1 GiB payload split into 205 chunks of 5 MiB. Across five measured runs per setting, concurrency 4 reduced median upload time from 50102.329 ms to 13258.377 ms (73.537%, 3.779x) and nearest-rank P95 from 52060.452 ms to 13489.564 ms (74.089%, 3.859x), with zero retries or failed runs. Merge, Kafka ETL, parsing, and embedding remain outside the upload timing scope.

Sanitized evidence is in benchmarks/retrieval/reports and benchmarks/upload/reports. The rerank provider did not return token usage; a report value of zero means unavailable, not zero consumption.

Stack

• Backend: Java 17, Spring Boot 3.4.2, Spring Security, JPA, WebFlux, WebSocket
• Frontend: Vue 3, TypeScript, Vite, Naive UI, Pinia
• Data: MySQL 8, Redis 7, Elasticsearch 8.10, Kafka, MinIO
• Document and AI: Apache Tika/LiteParse, configurable LLM and embedding providers, SiliconFlow Qwen3 reranker

Local Setup on D Drive

The provided Windows/WSL workflow keeps repository files and caches under D:\PaiSmart. PaiSmartLocal, paismart-*, and the MySQL database name PaiSmart are compatibility-only runtime identifiers.

Prerequisites: Windows 11 with WSL2, Docker in the project-local WSL distribution, Java 17, Maven, Node.js 18.20+, and pnpm 8.7+.

1. Copy .env.example to .env and set local credentials and provider keys. Never commit .env.
2. Start the complete local runtime:

powershell -ExecutionPolicy Bypass -File .\scripts\start-local.ps1

3. Open http://localhost:9527/#/chat or http://localhost:9527/#/chat-history.
4. Stop the application and dependency containers while preserving Docker volumes:

powershell -ExecutionPolicy Bypass -File .\scripts\stop-local.ps1

For normal backend development, run the application from the IDE and trigger hot reload after Java changes with:

mvn -q -DskipTests compile

Configuration

Primary configuration sources are root .env, src/main/resources/application.yml, src/main/resources/application-dev.yml, and frontend/.env*.

Retrieval controls include:

SILICONFLOW_API_URL=https://api.siliconflow.cn/v1
SILICONFLOW_API_KEY=
SILICONFLOW_RERANK_MODEL=Qwen/Qwen3-Reranker-0.6B
CONTEXTMESH_RERANK_MAX_CANDIDATES=20
CONTEXTMESH_RERANK_TOP_N=10
CONTEXTMESH_MAX_LIVE_RERANK_REQUESTS=60

Production deployments must use real secrets, trusted Elasticsearch certificates, restricted allowed origins, and disabled administrator bootstrap.

Verification

# Backend compile and tests
mvn -q -DskipTests compile
mvn test

# Retrieval benchmark tests
.\.local\python\Scripts\python.exe -m unittest discover benchmarks\retrieval -p 'test_*.py' -v

# Frontend
Set-Location frontend
pnpm typecheck
pnpm exec eslint <modified-file>

Raw benchmark queries, captures, vectors, caches, credentials, and generated files remain under ignored .runtime paths. Only aggregate, query-text-free reports are tracked.

Documentation

• Architecture
• Project overview
• Evidence-backed resume bullets
• Retrieval benchmark reproduction

License and Attribution

The repository is distributed under the Apache License 2.0.