Service — document_service.py#

Chunk text, embed it, store it, search it: the four functions that make documents retrievable, plus the module-level Chroma client that persists them.

Purpose#

backend/app/services/document_service.py owns all vector-store mechanics. Nothing else in the codebase touches Chroma or the embeddings API.

Module setup#

OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
EMBEDDING_MODEL = os.getenv("OPENAI_EMBEDDING_MODEL", "text-embedding-3-small")
 
openai_client = OpenAI(api_key=OPENAI_API_KEY)
chroma_client = chromadb.PersistentClient(path="./chroma_db")     # working-dir relative!
documents_collection = chroma_client.get_or_create_collection(name="agent_documents")

One persistent client, one collection (agent_documents) shared by all agents — isolation happens at query time via metadata filtering, not via separate collections.

Public API#

split_text_into_chunks(text, chunk_size=800, overlap=100) -> list[str]#

Character-window chunker: strip the text, then slide a window of 800 chars advancing by 700 (i.e., 100 chars of overlap), stripping and skipping empty chunks. The if start <= 0: start = end guard prevents an infinite loop if overlap >= chunk_size.

|-------- chunk 0 (0–800) --------|
                       |-------- chunk 1 (700–1500) --------|
                                              |---- chunk 2 (1400–2200) ----|

Why character-based? No tokenizer dependency, predictable sizes, and the 100-char overlap preserves sentences that straddle boundaries well enough for .txt knowledge bases. Semantic/token-aware chunking is a possible upgrade (RAG Ingestion).

create_embedding(text) -> list[float]#

Guards on the API key, then one openai_client.embeddings.create call with EMBEDDING_MODEL. text-embedding-3-small returns 1536-dimensional vectors. Used for both document chunks (ingest) and queries (search) — the same model must embed both sides for distances to be meaningful.

add_chunk_to_vector_store(vector_id, chunk_text, metadata) -> None#

Embeds the chunk and writes one record to the collection:

documents_collection.add(
    ids=[vector_id],                # "agent_{a}_document_{d}_chunk_{i}"
    documents=[chunk_text],
    embeddings=[embedding],
    metadatas=[metadata],           # {agent_id, document_id, filename, chunk_index}
)

The deterministic ID means re-uploading the same document ID would overwrite rather than duplicate vectors (Chroma upserts on identical IDs are add-errors in some versions — in practice each upload creates a new document_id, so collisions don't occur).

search_agent_documents(agent_id, query, top_k=3) -> list[dict]#

The retrieval function:

results = documents_collection.query(
    query_embeddings=[create_embedding(query)],
    n_results=top_k,
    where={"agent_id": agent_id},          # tenant isolation
)

Zips Chroma's parallel arrays (documents, metadatas, distances) into a list of dicts: {chunk_text, score, document_id, filename, chunk_index}.

Data flow#

Edge cases#

• Empty/whitespace text → [] from the chunker; the upload endpoint turns that into 400.
• Query with no matching vectors → empty lists from Chroma → []; the RAG graph's evaluate node short-circuits to refusal.
• Working directory sensitivity: ./chroma_db (like ./agentops.db) resolves against the CWD — starting the server elsewhere silently creates a second, empty store (Troubleshooting).
• No deletion API — vectors are never removed; see the cascade warning in ORM Models.

Performance notes#

Ingestion cost is linear in chunks: one embedding round trip each, sequential — the dominant cost of upload. Query-side cost is one embedding + one ANN search (fast at this scale). Batching embeddings.create (it accepts lists) is the first optimization when uploads grow.

Security notes#

Chunk text is sent to OpenAI for embedding — uploaded documents leave the machine. The where={"agent_id": ...} filter is the only thing separating agents' knowledge bases; keep it on every new query path. See Security.

Future improvements#

• Batch embeddings; retry with backoff on rate limits.
• Configurable chunk size/overlap via environment.
• Cosine distance space + score normalization for human-friendly relevance.
• Vector deletion on document/agent delete.

Related files#

• RAG Ingestion · Retrieval
• Routers: Documents — the calling endpoints
• Service: RAG Graph — the primary consumer of search