Lesson 09b: Documents Router — Deep Dive#

"Five endpoints: one writes, four read (or reason)."

backend/app/routers/documents.py (290 lines) is the HTTP surface for everything RAG-related. It's smaller and more uniform than agents.py, but it's where three separate services — document_service, openai_service, and rag_graph_service — all meet.

The Import Line Worth a Second Look#

from services.document_service import (
    split_text_into_chunks,
    add_chunk_to_vector_store,
    search_agent_documents,
)
from services.openai_service import generate_rag_answer, evaluate_retrieved_context
from services.rag_graph_service import generate_rag_answer, run_corrective_rag_graph

Look closely: generate_rag_answer is imported twice — once from openai_service, then immediately reassigned by importing a name of the same spelling from rag_graph_service. This isn't a bug in practice, because rag_graph_service.py itself does from services.openai_service import (evaluate_retrieved_context, generate_rag_answer) — so rag_graph_service.generate_rag_answer and openai_service.generate_rag_answer are the exact same function object, just re-exported. The second import silently shadows the first with an identical value. It still reads as a mistake to a reviewer, and generate_rag_answer is never actually called directly from this router anyway (only run_corrective_rag_graph is) — the import is unused after the shadowing. This is a small, real example of why import hygiene matters even when it happens not to break anything.

Local Copies of Shared Helpers#

def agent_has_tool(agent: models.Agent, tool_name: str) -> bool: ...
def log_tool_call(db, agent_id, tool_name, ...): ...

Both functions are verbatim duplicates of the same-named functions in routers/agents.py — with one difference: this file's log_tool_call doesn't accept agent_run_id, model_name, estimated_tokens, or estimated_cost parameters, because nothing in this router computes cost estimates. Two independent copies of the same logic, silently able to drift apart over time, is the concrete case behind the "Utility Layer" gap discussed in Lesson 10 and Lesson 27.

POST /{agent_id}/documents/upload — the Only Write Endpoint#

if not file.filename.endswith(".txt"):
    raise HTTPException(400, "Only .txt files are supported in this step")
 
file_bytes = await file.read()
try:
    text = file_bytes.decode("utf-8")
except UnicodeDecodeError:
    raise HTTPException(400, "File must be UTF-8 encoded text")
 
chunks = split_text_into_chunks(text)
if not chunks:
    raise HTTPException(400, "File is empty or could not be processed")
 
document = models.Document(..., status="indexed", chunks_count=len(chunks))
db.add(document)
db.commit()
db.refresh(document)
 
for index, chunk in enumerate(chunks):
    vector_id = f"agent_{agent.id}_document_{document.id}_chunk_{index}"
    db.add(models.DocumentChunk(document_id=document.id, ..., vector_id=vector_id))
    add_chunk_to_vector_store(vector_id=vector_id, chunk_text=chunk, metadata={...})
 
db.commit()
return document

Validation is entirely extension- and encoding-based:

• Filename suffix only — file.filename.endswith(".txt"). Nothing inspects the actual bytes or Content-Type header, so a file named notes.txt containing binary data would pass this check and only fail (or half-fail) at the .decode("utf-8") step.
• No size limit anywhere — a multi-hundred-megabyte .txt file would be read entirely into memory (await file.read()), chunked, and then embedded chunk-by-chunk with one OpenAI API call per chunk, synchronously, inside this one HTTP request. A 500KB document at the default 800-character chunk size is roughly 625 chunks — 625 sequential embedding calls before the response is returned. See Lesson 24: Performance and Lesson 18: Background Tasks.

Two commits happen: the first persists the Document row alone (so document.id exists for building vector_id strings), the second persists all the DocumentChunk rows together with their Chroma writes already done. If add_chunk_to_vector_store throws partway through the loop (e.g., an OpenAI rate limit on chunk 300 of 625), the exception propagates uncaught out of this endpoint — unlike almost every other write endpoint in this codebase, there is no try/except around the upload loop. The Document row from the first commit stays in the database with chunks_count claiming more chunks than actually made it into Chroma. This is a genuine consistency gap between SQL and the vector store.

GET /{agent_id}/documents and GET /{agent_id}/chunks#

Plain list endpoints, same "fetch agent or 404, then filter and order" pattern as everywhere else. chunks is ordered ascending by id (reading order), unlike most other lists in this codebase which order descending (newest first).

POST /{agent_id}/documents/search — Raw Retrieval#

Gated by agent_has_tool(agent, "document_search"). Thin wrapper: calls document_service.search_agent_documents() directly, logs one ToolCall, and returns the raw ranked chunk list — no LLM-generated answer, just retrieval. This is the endpoint you'd use to debug retrieval quality in isolation from generation quality.

POST /{agent_id}/documents/ask — Corrective RAG#

Also gated by agent_has_tool(agent, "document_search") — note this is the same tool flag as plain search; there is no separate "rag_ask" or "corrective_rag" tool name, so any agent allowed to search documents is also allowed to ask questions against them. Calls rag_graph_service.run_corrective_rag_graph() and logs the whole interaction under tool_name="langgraph_corrective_rag". This is the only place in the entire backend where a ToolCall's tool_output records a structured {answer, sources, retrieval_evaluation} payload rather than a single string or a list.

Search vs. Ask, Side by Side#

Common Mistakes#

• Uploading a large file and expecting a fast response — the upload endpoint is fully synchronous; response time scales linearly with chunk count.
• Assuming a partially-failed upload leaves consistent state — the Document row and its chunks_count can end up ahead of what actually landed in Chroma if an embedding call fails mid-loop.
• Assuming .txt extension checking is a security control — it isn't; it's a content-type convenience check, not a validation of what's inside the file.

Exercises#

1. Wrap the chunk-upload loop in a try/except, and on failure, roll the Document.status back to "failed" (a new status value) and set chunks_count to the number of chunks that actually succeeded, instead of leaving it silently wrong.
2. Replace the duplicated agent_has_tool/log_tool_call in this file with imports from a shared module, and confirm both routers still pass the same manual test cases.
3. Add a MAX_FILE_SIZE check to the upload endpoint and return 413 Payload Too Large for files over the limit, before reading the full body into memory.

Key Takeaways#

🎯 search and ask are gated by the same document_search tool flag, despite very different cost and behavior. 🎯 The upload endpoint has no size limit and no failure recovery — a partial failure leaves SQL and Chroma out of sync. 🎯 Duplicated agent_has_tool/log_tool_call logic between this file and agents.py is a maintenance risk, not just style noise.

Next Steps#

👉 Lesson 13: AI Agents →

Module 4 · API & Business Logic