Skip to content
2 min read · 483 words

Service — monitoring_service.py

41 lines of deliberate approximation: estimate tokens at 4 characters each, price them at gpt-4o-mini rates, and feed the result into every costed ToolCall.

Purpose

backend/app/services/monitoring_service.py supplies the numbers behind the observability dashboard's token and cost figures. It performs no I/O and no LLM calls — pure arithmetic, instant and free.

The code

python
def estimate_tokens(text: str | None) -> int:
    """1 token ≈ 4 characters in English. Not exact — for the monitoring dashboard."""
    if not text:
        return 0
    return max(1, len(text) // 4)
 
 
def estimate_openai_cost(input_text, output_text, model_name="gpt-4o-mini") -> dict:
    input_tokens = estimate_tokens(input_text)
    output_tokens = estimate_tokens(output_text)
 
    input_cost_per_1m = 0.15      # $ per 1M input tokens (gpt-4o-mini, approximate)
    output_cost_per_1m = 0.60     # $ per 1M output tokens
 
    input_cost = (input_tokens / 1_000_000) * input_cost_per_1m
    output_cost = (output_tokens / 1_000_000) * output_cost_per_1m
 
    return {
        "model_name": model_name,
        "estimated_tokens": input_tokens + output_tokens,
        "estimated_cost": f"{(input_cost + output_cost):.6f}",   # string!
    }

Who calls it

Only the multi-agent workflow handler today — once per LLM stage (analysis, reply, review), passing the stage's input/output text and os.getenv("OPENAI_MODEL", "gpt-4o-mini") as the label:

python
analysis_monitoring = estimate_openai_cost(
    input_text=workflow_data.email_text,
    output_text=json.dumps(analysis),
    model_name=os.getenv("OPENAI_MODEL", "gpt-4o-mini"),
)
log_tool_call(..., model_name=..., estimated_tokens=..., estimated_cost=...)

The chat, single-agent analyze, and document endpoints log tool calls without cost data — a known inconsistency (Known Issues).

Accuracy — read this before trusting the dashboard

AspectReality
Token countchars ÷ 4 heuristic; real tokenization varies ±30%+, more for code/non-English
Prompt overheadsystem prompts and JSON scaffolding are not counted — only the payload text passed in
Pricinghardcoded gpt-4o-mini rates; passing a different model_name changes the label, not the math
Currency of pricesfrozen at time of writing; OpenAI price changes are not reflected

estimated_total_cost on the dashboard answers "roughly how much activity is this agent generating?" — not "what will OpenAI charge me?". For real numbers, read response.usage from the API (future improvement) or the OpenAI usage dashboard.

Why estimate instead of reading response.usage?

The OpenAI SDK does return exact token counts. The estimate approach was chosen because cost logging happens in the router, which never sees the raw OpenAI response object (services return parsed text/dicts). Threading usage data through every service return type is the correct fix and is on the Roadmap; the estimator keeps observability cheap until then.

Edge cases

  • None/empty text → 0 tokens; non-empty text → at least 1 (max(1, ...)).
  • Output is a string with 6 decimal places — downstream summation uses safe_float (Design Decisions).

Future improvements

  • Real usage from response.usage; per-model price table; numeric storage.
  • Apply cost logging uniformly to chat / analyze / RAG tool calls.
PreviousDocument ServiceNextObservability