Service — rag_graph_service.py#

A four-node LangGraph StateGraph that retrieves, judges the retrieval, and then either answers from context or refuses — compiled once at import and reused for every question.

Purpose#

backend/app/services/rag_graph_service.py implements the query side of RAG as an explicit graph: retrieval and generation are separated by an evaluation gate with a conditional edge. It is the only LangGraph usage in the codebase (rationale: Design Decisions).

The state#

class RAGGraphState(TypedDict):
    agent_id: int
    agent_name: str
    question: str
    top_k: int
    language: str
    tone: str
 
    retrieved_chunks: list[dict[str, Any]]
    retrieval_evaluation: dict[str, Any] | None
    answer: str
    sources: list[dict[str, Any]]
 
    route: str

The first block is request input; the second is accumulated by nodes; route carries the evaluator's decision to the conditional edge. Nodes return partial dicts that LangGraph merges into the state — the idiomatic pattern.

The graph#

retrieve_node#

Calls search_agent_documents (embed query → Chroma query filtered to this agent) and returns both the full retrieved_chunks and a slimmed sources list (filename, document_id, chunk_index, score) for the response.

evaluate_node#

Two cases:

• No chunks retrieved → short-circuits with a synthetic evaluation (has_answer: False, confidence: 0, reason: "No chunks were retrieved.") and route="refuse" — no LLM call spent.
• Chunks exist → calls evaluate_retrieved_context (temperature 0) and sets route = "answer" if evaluation["has_answer"] else "refuse".

answer_node / refusal_node#

answer_node calls generate_rag_answer with the agent's name/language/tone. refusal_node returns the fixed string "I could not find this information in the uploaded documents." — the same string the answer prompt mandates, so refusals look identical regardless of which path produced them.

Assembly#

graph.add_edge(START, "retrieve_node")
graph.add_edge("retrieve_node", "evaluate_node")
graph.add_conditional_edges("evaluate_node", route_after_evaluation,
                            {"answer_node": "answer_node", "refusal_node": "refusal_node"})
graph.add_edge("answer_node", END)
graph.add_edge("refusal_node", END)
corrective_rag_graph = build_corrective_rag_graph()   # compiled once, module-level

Public API#

run_corrective_rag_graph(agent_id, agent_name, question, top_k, language="English", tone="Professional") -> dict#

Builds the initial state (empty accumulators), invokes the compiled graph synchronously, and returns:

{
  "question": ...,
  "answer": final_state.get("answer"),
  "sources": final_state.get("sources", []),
  "retrieval_evaluation": final_state.get("retrieval_evaluation"),
  "retrieved_chunks": final_state.get("retrieved_chunks", []),
}

The documents router serializes everything except retrieved_chunks into the API response (chunks would bloat it; they are logged in the ToolCall output instead — partially, via answer/sources/evaluation).

Cost profile#

The design spends more on answerable questions and less on unanswerable ones — the opposite of naive RAG, which pays full price to hallucinate.

Edge cases & error handling#

• Any node exception (OpenAI failure, evaluator JSON parse error) propagates out of invoke(); the router logs a failed langgraph_corrective_rag ToolCall and returns 500.
• The evaluator can be wrong in both directions: a false no refuses an answerable question (safe, annoying); a false yes lets generation proceed — but the answer prompt's own context-only rule is the second line of defense.
• top_k is caller-controlled (default 3); large values inflate evaluator input tokens.

Future improvements#

• A retry edge: on refuse, reformulate the query and re-retrieve once before giving up (true "corrective" RAG in the CRAG-paper sense).
• Persist per-node latencies as WorkflowStep-style traces (the schema already exists).
• Async nodes to unblock the worker during LLM calls.

Related files#

• Corrective RAG — the concept-level walkthrough
• Service: Documents — retrieval internals
• Evaluation — how the evaluator verdict is used