Request Lifecycle
This page traces POST /agents/{id}/chat — the canonical request — through every layer, then shows how the other endpoint families differ.
Chat request, end to end
The exact code path
- Route match & validation. FastAPI matches
POST /agents/{agent_id}/chat; the body must satisfyschemas.ChatRequest(a single requiredmessage: str) or FastAPI returns422before the handler runs. - Session injection.
db: Session = Depends(get_db)opens a session fromSessionLocal;get_db()guaranteesdb.close()in itsfinallyblock (Database Layer). - Agent lookup. Missing agent →
404 Agent not found. This guard opens nearly every endpoint in the codebase. - Persist the user message (added to the session; committed later).
- LLM call inside
try/except.generate_agent_responsecomposes the runtime system prompt from the agent's storedsystem_prompt,language, andtone(Prompt Catalog) and calls OpenAI at temperature 0.3. On any exception the handler substitutes the fallback text and capturesstatus="failed"+error_message— the request still succeeds. - Persist the assistant message and the
AgentRunwith wall-clocklatency_msmeasured around the whole handler. - Single
db.commit()flushes user message, assistant message, and run atomically, then theChatResponseis returned.
A failed LLM call still produces a well-formed conversation turn and a recorded failed run. The client sees the fallback text; operators see the real error in GET /agents/{id}/runs. Contrast this with the tool/workflow endpoints below, which return 500 on failure. Rationale in Error Handling.
How other endpoint families differ
| Family | Extra steps vs. chat | Failure mode |
|---|---|---|
CRUD reads (GET /agents, /tasks, /runs, …) | None — fetch, serialize via response_model | 404 if the parent agent is missing |
Email analyze (POST .../email/analyze) | Tool gate (email_analyzer required, else 403) → LLM analysis → optional task creation → 1–2 ToolCall logs | 500 + failed ToolCall logged |
Multi-agent workflow (POST .../workflows/email) | Creates a WorkflowRun first (status="running"), then logs a WorkflowStep + ToolCall per stage, finalizes run status/quality | 500; run marked failed; error step logged |
Document upload (POST .../documents/upload) | File-type/encoding guards (400) → chunk → embed each chunk → write SQLite + Chroma | 400 for bad input; embedding errors propagate as 500 |
RAG ask (POST .../documents/ask) | Tool gate (document_search) → LangGraph graph (retrieve → evaluate → answer/refuse) → ToolCall log | 500 + failed ToolCall logged |
Full traces for the workflow and RAG pipelines are in Data Flow.
Latency anatomy
For a typical chat request, nearly all time is the OpenAI round trip:
| Segment | Typical share |
|---|---|
| FastAPI routing + Pydantic validation | ~1 ms |
| SQLite queries (lookup + 3 inserts + commit) | a few ms |
| OpenAI chat completion | 500–3000 ms |
This is why latency_ms on AgentRun/ToolCall is effectively an LLM latency metric — and why the multi-agent workflow (3 sequential LLM calls) takes several seconds. Performance discussion: Design Decisions.
Related pages
- Data Flow — the multi-step pipelines
- Routers: Agents — the handler code
- Observability — what gets recorded for every request