Skip to content
4 min read · 704 words

Service — openai_service.py

Every direct OpenAI chat-completion call in the platform funnels through this module's four functions. If you want to change how the platform talks to the LLM, you change it here.

Purpose

backend/app/services/openai_service.py owns the OpenAI client, the model selection, and four prompt-bearing functions. Routers and other services call these; nothing else in the codebase constructs chat completions (the multi-agent service has its own two calls for reply/review — see its page).

Module setup

python
load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
OPENAI_MODEL = os.getenv("OPENAI_MODEL", "gpt-4o-mini")

Every function begins with the same guard — raising ValueError("OPENAI_API_KEY is missing...") when the key is absent — so failures are explicit rather than cryptic SDK errors. How each caller handles that: Error Handling.

Public API

generate_agent_response(system_prompt, user_message, agent_name, language="English", tone="Professional") -> str

Powers POST /agents/{id}/chat. Composes a runtime system prompt around the agent's stored instructions:

You are {agent_name}.
Agent instructions: {system_prompt}
Language: {language}
Tone: {tone}
Rules: Answer clearly · Be helpful and practical ·
       If you do not know the answer, say you do not know · Do not invent facts.

Temperature 0.3. Returns the raw completion text. Full prompt text: Prompt Catalog.

analyze_email(email_text) -> dict

Powers email analysis (both single-agent and the workflow's Analysis Agent). A strict-JSON system prompt demands:

json
{
  "summary": "...", "intent": "...",
  "priority": "low | medium | high",
  "deadline": "... or null",
  "action_items": ["..."],
  "suggested_reply": "..."
}

Temperature 0.2. The response is parsed with json.loads; a parse failure raises ValueError("OpenAI did not return valid JSON: ...") — no retry, by design (Design Decisions).

generate_rag_answer(question, contexts, agent_name, language, tone) -> str

Powers the RAG answer node. Formats each retrieved chunk as a numbered source block (filename, chunk index, content), then instructs the model to answer only from that context, refusing with the exact string "I could not find this information in the uploaded documents." when the context lacks the answer. Temperature 0.2.

The system_prompt/user_prompt assignments are indented inside the for loop that formats sources. The behavior is correct for non-empty contexts (each iteration rebuilds the prompt; the last iteration's version — containing the full accumulated context_text — is used), but with an empty contexts list the variables are never assigned and the function raises NameError. In practice the corrective RAG graph routes empty retrievals to the refusal node so this path is unreachable today — but it is a real landmine for new callers. Tracked in Known Issues.

evaluate_retrieved_context(question, contexts) -> dict

The corrective-RAG evaluator (temperature 0). Asks the model to judge — without answering — whether the retrieved context can answer the question, returning strict JSON that the function normalizes defensively:

python
return {
    "has_answer": bool(result.get("has_answer")),
    "confidence": float(result.get("confidence", 0)),
    "reason": result.get("reason", ""),
}

Used by evaluate_node in the RAG graph to route between answering and refusing (Evaluation).

Inputs / outputs summary

FunctionInputOutputTempJSON-strict
generate_agent_responseprompt parts + user messagefree text0.3no
analyze_emailraw email textanalysis dict0.2yes
generate_rag_answerquestion + context chunksgrounded text0.2no
evaluate_retrieved_contextquestion + context chunksverdict dict0yes

Performance & cost notes

Each call is one synchronous HTTPS round trip (0.5–3 s typical) that blocks the worker thread. Token/cost figures logged alongside these calls come from monitoring_service, not from the API response — they are estimates.

Security notes

The API key never leaves this layer; prompts include user-controlled content (email text, questions, document chunks) — see the prompt-injection discussion in Security.

Future improvements

  • Fix the loop-indentation quirk in generate_rag_answer (and add an explicit empty-context guard).
  • Use OpenAI structured outputs for the JSON functions; add bounded retries on parse failure.
  • Capture actual token usage from response.usage instead of estimating.
PreviousDocuments RouterNextMulti-Agent Email