RAG and debugger
Hybrid search, reranking, guardrails, and the playground debug panel.
ChatAI answers with retrieval-augmented generation: embed the question, retrieve chunks, optionally rerank, then stream an answer with citations.
Pipeline (v0.5+)
- Retrieve — hybrid fusion of pgvector similarity and Postgres
tsvectorkeyword search (RRF), when hybrid is enabled - Expand — optional query expansion for short/ambiguous questions
- Rerank — LLM listwise rerank; Cohere rerank when
COHERE_API_KEYis set - Generate — streamed chat with source markers
- Guardrails — optional require-context, citation verify, refuse on low confidence (one regenerate attempt when verifying)
Chunking modes:
| Mode | Use when |
|---|---|
standard | Default fixed-size chunks |
parent_child | Retrieve small children, expand to parent context (reprocess to activate) |
Outcomes
Public chat metadata can include outcomes such as answered / low confidence / refused depending on retrieval quality and guardrails. Empty sources usually means retrieval found nothing useful (missing knowledge, failed ingest, or overly strict mode).
Playground debugger
Owners get a RAG debug panel on the playground (not exposed to widget visitors). Use it to inspect:
- Retrieved chunks and ranks
- Hybrid vector vs keyword contributions
- Rerank / guardrail decisions
- Timing and model metadata when present
Eval sampling
Per-assistant evalSampleRate (0–1) can enqueue async quality scoring for a fraction of production answers. Offline eval sets and the Analytics AI Quality section cover regressions. Restart pnpm dev after changing packages/evals so the worker picks up new code.