Lesson 18: Background Tasks#

Verifying "Not Implemented" Directly From the Code#

grep -r "BackgroundTasks" backend/app returns nothing. grep -r "celery\|redis\|rq" backend/pyproject.toml returns nothing. This is not an oversight to hide — it's an accurate description of the current system, and it's worth confirming from the source rather than assuming.

Where It Would Matter Most: Document Upload#

The clearest candidate in this codebase is POST /agents/{id}/documents/upload (Lesson 09b). Walk through what currently happens, in order, inside a single HTTP request-response cycle:

1. Read the full file into memory.
2. Decode and chunk it.
3. Insert a Document row, commit (so document.id exists).
4. For every chunk: call OpenAI's embeddings API, then write to ChromaDB, then stage a DocumentChunk row — serially, one chunk at a time.
5. Commit all chunk rows.
6. Return the Document to the client.

The HTTP client is blocked, waiting, for the entire duration of step 4 — which is N sequential network round-trips for a document with N chunks. A 500KB file at 800 characters/chunk is roughly 625 chunks, meaning the client's request doesn't return until 625 OpenAI calls have completed one after another.

What the Fix Would Look Like#

FastAPI's built-in BackgroundTasks is the smallest change that would help:

from fastapi import BackgroundTasks
 
@router.post("/{agent_id}/documents/upload", response_model=schemas.DocumentResponse)
async def upload_document(
    agent_id: int,
    background_tasks: BackgroundTasks,
    file: UploadFile = File(...),
    db: Session = Depends(get_db),
):
    # ... validate, create Document row with status="processing", commit, get document.id ...
    background_tasks.add_task(process_chunks_and_embed, document.id, text)
    return document  # returns immediately, status is "processing" not "indexed"

The endpoint would return as soon as the Document row exists, with status="processing" rather than "indexed". The actual chunking/embedding/storing work would run after the response is sent, in the same process. The client would then need to poll GET /{agent_id}/documents (already implemented) to see status flip to "indexed" once background work finishes.

What FastAPI's BackgroundTasks does not give you, which matters for evaluating whether it's sufficient here: it runs in the same process and worker, with no retry, no persistence if the server crashes mid-task, and no visibility into progress beyond whatever you build yourself into the status column. For a small single-instance deployment (which is what this backend currently is — no worker pool, no queue), that's an acceptable trade. At larger scale, the next step up is a real task queue (Celery + Redis, or an equivalent), which adds a durable job store and retry semantics BackgroundTasks doesn't have.

The Multi-Agent Email Workflow Has the Same Property#

POST /{agent_id}/workflows/email (Lesson 09a) runs four sequential LLM calls (analysis, reply, review, plus task creation) inside one request. It's a smaller version of the same synchronous-chain pattern as document upload, just with LLM calls instead of embedding calls. The same BackgroundTasks treatment would apply, with the same trade-off: the client would need a way to poll GET /{agent_id}/workflow-runs (already implemented) for the run's status to change from "running" to "success" or "failed".

Why This Wasn't Built This Way#

Nothing in the codebase or its history says why explicitly, but the pattern is consistent with everything else about this backend: it's a synchronous, single-file-per-endpoint FastAPI app with no task queue infrastructure (no Redis, no Celery, no broker) provisioned anywhere in pyproject.toml. Adding background processing without a durable queue would only be a partial fix — an in-process BackgroundTasks job is lost if the server restarts mid-task, which is a real risk for a 625-chunk upload that takes long enough to embed.

Common Mistakes#

• Assuming a long-running upload has failed and retrying it — with no background processing and no progress indicator beyond the final response, a large upload can look "stuck" when it's simply still working through its chunk loop.
• Assuming adding BackgroundTasks alone solves durability — it doesn't survive process restarts; that requires an actual queue.

Exercises#

1. Add a "processing" value to Document.status, implement the BackgroundTasks version of the upload endpoint sketched above, and update GET /{agent_id}/documents behavior (already works unmodified) to demonstrate a client polling for completion.
2. Estimate, using estimate_openai_cost-style back-of-envelope math, how long a 2MB .txt upload would take today, assuming each embedding call takes ~200ms. What does that mean for the HTTP client's timeout settings?
3. Compare BackgroundTasks and a Celery+Redis setup on three axes: durability across restarts, ability to retry a failed step, and operational complexity to run. Which is the right choice for this codebase's actual current scale?

Key Takeaways#

🎯 No background task infrastructure exists in this codebase today — confirmed by absence, not inferred. 🎯 Document upload and the multi-agent email workflow are the two endpoints that would benefit most. 🎯 FastAPI's BackgroundTasks is the smallest fix, but doesn't survive a server restart mid-task — a real queue is the next step up if that matters.

Next Steps#

👉 Lesson 19: Prompt Engineering & Tool Calling →

Module 5 · Advanced AI Engineering