Services
| Service | Language | Role |
|---|---|---|
knowledge-api | Python (FastAPI, uv) | Hybrid retrieval, served over an HTTP API and an MCP stdio entrypoint |
ingest-workers | Go | Corpus scanner + embedding workers, coordinated over NATS JetStream |
render-farm | Go | Render + diff workers behind a job API; drives real browsers, stores screenshots in MinIO |
telemetry-ingest | Go | HMAC-signed usage-event ingest that feeds usage-weighted retrieval |
gate | TypeScript | Six-check quality gate run in CI, including axe-core accessibility and an LLM judge |
Alongside the services, packages/ holds the assembly CLI (generation), a shared Anthropic model client, and a Next.js review UI for approving visual baselines.
Data layer
Everything meets in one shared Postgres:
- pgvector stores embeddings under an HNSW index (
vector_cosine_ops) for approximate nearest-neighbor search. - Postgres full-text search covers the same chunks for keyword recall.
- Migrations live in
db/migrations, applied with dbmate (make db-migrate).
NATS JetStream queues ingest and render work; MinIO stores render-farm artifacts (screenshots, diffs).
The ingest path
corpus files ── manifest.yaml ──▶ scanner ──▶ chunks ──▶ embed workers ──▶ Postgres (pgvector + FTS)The manifest maps corpus paths to doc types (component_doc, component_source, story, token). The scanner walks the corpus, chunks files, and tags every chunk with source_path, component_name, doc_type, and chunk_index. Embedding workers pick chunks off the queue and write vectors back.
The query path
MCP / HTTP ──▶ vector search + full-text search ──▶ RRF fusion ──▶ ranked chunksBoth searches run over the same chunk store, and reciprocal-rank fusion merges the rankings. Results carry an rrf_score and fts_mode alongside the chunk metadata. Optionally, a usage-weighted boost nudges chunks that real agents actually use.
The feedback path
Agent queries become telemetry events (HMAC-signed), telemetry becomes ranking signal, and /analytics/misses surfaces searches the corpus failed to answer — a direct to-do list for corpus gaps. See Telemetry.