AI Systems Overview
Seven distinct LLM call types, one embedding pipeline, one orchestration graph — all funneled through two service modules and watched by the observability layer.
Every LLM touchpoint in the platform
| # | Call | Function | Model | Temp | Output | Triggered by |
|---|---|---|---|---|---|---|
| 1 | Agent chat | generate_agent_response | OPENAI_MODEL (default gpt-4o-mini) | 0.3 | free text | POST /agents/{id}/chat |
| 2 | Email analysis | analyze_email | 〃 | 0.2 | strict JSON | POST .../email/analyze + workflow stage 1 |
| 3 | Reply drafting | run_reply_agent | 〃 | 0.3 | free text | workflow stage 3 |
| 4 | Quality review | run_reviewer_agent | 〃 | 0 | strict JSON | workflow stage 4 |
| 5 | Retrieval evaluation | evaluate_retrieved_context | 〃 | 0 | strict JSON | RAG ask (evaluate node) |
| 6 | RAG answering | generate_rag_answer | 〃 | 0.2 | grounded text | RAG ask (answer node) |
| 7 | Embeddings | create_embedding | OPENAI_EMBEDDING_MODEL (default text-embedding-3-small) | — | 1536-dim vector | document upload + every search/ask |
Design themes
One small model, many roles
Everything runs on gpt-4o-mini by default — the platform gets its quality not from a bigger model but from decomposition: extraction, drafting, and judging are separate calls with separate prompts and temperatures (Design Decisions).
Deterministic where it matters
Judging calls (reviewer, retrieval evaluator) run at temperature 0; extraction at 0.2; user-facing generation at 0.3. The temperature ladder is a deliberate contract, not an accident.
Structure by prompt, validated by parse
Wherever the platform needs machine-readable output it demands "Return ONLY valid JSON" and validates with json.loads — failures raise, get logged as failed tool calls, and return 500 rather than silently passing garbage downstream.
Watched by default
LLM stages in the workflow log model_name, estimated_tokens, estimated_cost, and latency_ms onto their ToolCall rows (Observability); the reviewer's quality_score aggregates into a per-agent average.
Section pages
Deep dives live in their own sections: RAG · Workflows · Agents · Prompts.
What the AI layer does not have (yet)
Streaming responses, function calling / structured outputs, conversation memory (each chat call sees only the current message — history is stored but not replayed into context), retries, model routing, and fine-tuning are all absent by design at this stage — see the Roadmap.
Message rows preserve full chat history per agent, but generate_agent_response receives only the current user message — there is no context window assembly. An agent will not remember what you said one message ago. This is the single most impactful known limitation for chat UX.