Design Decisions#

Every significant choice in the codebase, the reasoning behind it, and what it costs. Read this before proposing structural changes.

Backend#

FastAPI over Flask/Django#

Decision: FastAPI with Pydantic schemas and dependency-injected DB sessions.

• Why: first-class request/response validation (schemas.py gives typed contracts for free), automatic OpenAPI docs at /docs, and Depends(get_db) makes session lifecycle explicit and testable.
• Cost: endpoints are a mix of async def and def (only the upload endpoint truly awaits); the LLM calls inside are synchronous SDK calls that block the worker — fine at this scale, a scaling constraint later.

SQLite over Postgres#

Decision: sqlite:///./agentops.db with check_same_thread=False.

• Why: zero setup for a single-node dev platform; the whole database is one inspectable file; create_all + ad-hoc ALTER TABLE is enough migration machinery (Migrations).
• Cost: single-writer concurrency, no server-side types (costs are stored as strings — see below), and a migration to Postgres will eventually be needed. The SQLAlchemy layer keeps that migration mostly a URL change plus a real migration tool (Alembic is the roadmap item).

Ad-hoc startup migrations over Alembic#

Decision: run_startup_migrations() in database.py checks PRAGMA table_info(tool_calls) and issues ALTER TABLE ... ADD COLUMN for missing columns.

• Why: exactly one schema evolution has happened (adding cost columns to tool_calls); a full Alembic setup was more machinery than the problem needed.
• Cost: no downgrade path, no version history, and every new column needs a hand-written entry. The contract: when you add a column to an existing model, add a matching entry there (guide).

Costs stored as strings#

Decision: ToolCall.estimated_cost and WorkflowRun.quality_score are VARCHAR, parsed defensively with safe_float() at dashboard time.

• Why: cost is a formatted estimate (f"{cost:.6f}"), not money that gets summed transactionally; storing the display string was the shortest path.
• Cost: aggregation requires parsing (the dashboard does), and SQL-level SUM() is impossible. A numeric column is the obvious improvement — noted in Known Issues.

enabled_tools as a comma-separated string#

Decision: the tool allow-list is a single Text column ("email_analyzer,task_creator"), parsed by agent_has_tool().

• Why: SQLite has no array type; a join table for a 4-value flag set is heavy; the string round-trips trivially to the frontend checkbox UI.
• Cost: no referential integrity — a typo silently disables a tool. The canonical tool names are documented in Tools.

Observability as first-class rows, not logs#

Decision: every automated step writes ToolCall / AgentRun / WorkflowRun / WorkflowStep rows with latency, model, token and cost estimates — the "observability contract."

• Why: the dashboard, run history, and workflow traces are product features, not ops tooling; SQL rows make them queryable by the same API.
• Cost: write amplification on every request (the email workflow writes ~9 rows), and estimates are approximations (chars/4), not billing truth. See Observability and Cost Monitoring.

AI systems#

gpt-4o-mini as the default model#

Why: the workloads (triage, extraction, drafting, judging) are high-volume and structured; a small model at ~$0.15/$0.60 per 1M tokens keeps the always-on cost trivial, and the model is overridable via OPENAI_MODEL without code changes.

Temperature ladder by task type#

JSON-by-prompt over function calling#

Decision: structured outputs (analysis, evaluation, review) are requested via "Return ONLY valid JSON" prompts and parsed with json.loads, raising ValueError on mismatch.

• Why: zero SDK coupling, prompts remain plain and portable, and the failure mode (raise + failed ToolCall) is explicit.
• Cost: occasional parse failures surface as 500s instead of being auto-retried. OpenAI structured outputs / function calling is the natural upgrade (Roadmap).

LangGraph only where routing exists#

Decision: the corrective RAG pipeline is a LangGraph StateGraph (it has a conditional edge); the email workflow is plain sequential Python in the router (it does not).

• Why: graphs earn their complexity when control flow branches. RAG genuinely routes (answer vs. refuse); the email pipeline is a straight line whose "orchestration" is really persistence choreography.
• Cost: two orchestration styles to learn. The comparison is instructive — see Workflows vs. Corrective RAG.

Chroma over pgvector/FAISS#

Why: persistent local storage with metadata filtering in two lines of code — the where={"agent_id": ...} filter is the tenant-isolation mechanism. No server to run, matching the SQLite philosophy.

Frontend#

Server components for reads, BFF proxy for writes#

Decision: dashboard/agents pages fetch the backend directly server-side (cache: "no-store", force-dynamic); the create form posts to app/api/agents/route.ts, which forwards to FastAPI.

• Why: no CORS anywhere, backend URL stays server-side, pages are always fresh, and the proxy normalizes FastAPI error shapes (detail string vs. validation array) into one UI-friendly message (API Proxy).
• Cost: each new browser-initiated mutation needs a matching route handler — the documented convention for extending the app (guide).

No client state library#

Decision: the only client state is useState in CreateAgentForm; everything else is server-rendered per request.

• Why: with no realtime updates and page-level reads, a store would manage nothing. Re-render-on-navigate is the state strategy.

Performance posture#

Latency is dominated by OpenAI round trips (see Request Lifecycle); everything local is single-digit milliseconds. Known hot spots, accepted for now:

• Sequential embedding on upload — one OpenAI call per chunk; a large file means a slow upload. Batch embedding is the fix when it matters.
• Dashboard aggregates in Python — loads all rows for an agent and counts in memory; fine for thousands of rows, SQL aggregation later.
• Three sequential LLM calls per email workflow — inherent to the design; the reviewer needs the reply, the reply needs the analysis.

Related pages#

• System Overview · Security
• Roadmap — where the accepted costs get paid down