System Overview
Three tiers, six backend layers, two storage engines, one external dependency (OpenAI).
Component diagram
The six backend layers
| # | Layer | Files | Responsibility |
|---|---|---|---|
| 1 | Application entry | main.py | Create tables, run migrations, mount routers, health endpoints |
| 2 | Routers | routers/agents.py, routers/documents.py | HTTP endpoints, tool gating, orchestration, observability logging |
| 3 | Schemas | schemas.py | Pydantic request validation and response serialization |
| 4 | Services | services/*.py | All LLM calls, prompts, chunking, embeddings, vector search, cost estimation |
| 5 | Models | models.py | 9 SQLAlchemy tables and their relationships |
| 6 | Database layer | database.py | Engine, SessionLocal, get_db() dependency, startup migrations |
Requests only ever flow downward (router → service → model → DB); services never import routers, and models never import services. Details per layer: Backend.
Division of storage
| Concern | Engine | Why |
|---|---|---|
| Agents, messages, runs, tasks, tool calls, documents metadata, workflow traces | SQLite | Relational integrity, cascade deletes, trivially queryable for dashboards |
| Document chunk embeddings for semantic search | Chroma | Purpose-built ANN search with metadata filtering (agent_id scoping) |
The two stores are linked by vector_id: each document_chunks row in SQLite records the ID of its vector in Chroma (agent_{aid}_document_{did}_chunk_{i}), so the relational DB remains the source of truth for what was indexed. See Database Schema and RAG Ingestion.
Frontend ↔ backend contract
Two patterns, chosen per direction of data flow:
- Reads — direct server-side fetch. Server components (
dashboard/page.tsx,agents/page.tsx) fetch the FastAPI endpoints directly withcache: "no-store"andexport const dynamic = "force-dynamic", so every page load reflects live data. The browser never sees the backend URL. - Writes — route-handler proxy. The create-agent form posts to
/api/agents(same origin); the handler forwards to${AGENTOPS_API_BASE_URL}/agents/and relays status + body. This avoids CORS entirely and keeps backend topology server-side. See API Proxy.
External dependencies
| Dependency | Used for | Failure behavior |
|---|---|---|
| OpenAI Chat Completions | Chat, email analysis, reply drafting, review, RAG answering, retrieval evaluation | Chat degrades to a fallback message; other endpoints return 500 with a logged failed ToolCall — see Error Handling |
| OpenAI Embeddings | Document ingestion and query embedding | Upload/search/ask return 500; failed ToolCall logged |
There is deliberately no Redis, no message queue, no background worker, no WebSocket layer — every operation is a synchronous request/response. Long-running LLM pipelines (the email workflow makes 3 LLM calls) simply hold the HTTP request open. The trade-offs and the migration path are discussed in Design Decisions and the Roadmap.
Deployment shape
Everything runs locally today: two dev servers (uvicorn :8001, next :3000) plus file-based storage. The documentation site is the only deployed artifact (Vercel). See Deployment.
Related pages
- Request Lifecycle — one request traced end to end
- Data Flow — the two big pipelines
- Design Decisions — why it is built this way