ChatAI Docs

Providers

Swap LLM and embedding providers via env and per-assistant overrides.

ChatAI resolves chat and embeddings through a small registry in @chatai/ai. Instance defaults come from environment variables; assistants may override provider/model when keys exist on the instance.

Chat providers

IdTypical env
openaiAI_API_KEY, optional AI_BASE_URL / AI_MODEL
openai-compatibleAI_API_KEY, AI_BASE_URL
anthropicANTHROPIC_API_KEY
googleGOOGLE_GENERATIVE_AI_API_KEY
openrouterOPENROUTER_API_KEY
azureAZURE_OPENAI_API_KEY, AZURE_OPENAI_RESOURCE
ollamaOLLAMA_BASE_URL (key optional / placeholder)
groqGROQ_API_KEY

Set AI_PROVIDER to one of the ids above. Missing credentials for the selected provider → 503.

Embedding providers

IdTypical env
openaiAI_API_KEY
openai-compatibleAI_API_KEY, AI_BASE_URL
cohereCOHERE_API_KEY
voyageVOYAGE_API_KEY

EMBEDDING_MODEL defaults to text-embedding-3-small. Optional COHERE_API_KEY also enables Cohere rerank before LLM listwise fallback.

Dimension constraint

EMBEDDING_DIMENSIONS (default 1536) must match the Postgres column chunks.embedding (vector(1536)).

  • Per-assistant embedding overrides that change native width are rejected
  • Local models like nomic-embed-text (768-d) are not compatible without a schema change — see Ollama

Ollama hybrid

Recommended pattern for local chat:

AI_PROVIDER=ollama
AI_MODEL=llama3.2
OLLAMA_BASE_URL=http://ollama:11434/v1   # or http://127.0.0.1:11434/v1 on the host
# keep OpenAI (or another 1536-d) embeddings
EMBEDDING_DIMENSIONS=1536

React

Use the ChatWidget React package in your app.

REST API

Bearer-authenticated /api/v1 routes, dual-auth chat, rate limits, and OpenAPI.