Lesson 02: Backend Architecture#

"Good architecture is not about perfection, it's about making the right trade-offs."

📚 Table of Contents#

1. Introduction
2. Layered Architecture
3. Separation of Concerns
4. The Three-Layer Pattern
5. Component Responsibilities
6. Data Flow
7. Design Patterns Used
8. Why This Architecture
9. Alternatives Considered
10. Scalability Considerations
11. Key Takeaways
12. Interview Questions
13. Exercises
14. Next Steps

Introduction#

In Lesson 01, you learned what VeyraOps AI is. In this lesson, you'll learn how it's structured.

Architecture is the foundation of maintainable software. A good architecture makes your codebase:

• Easy to understand — New developers can navigate quickly
• Easy to modify — Changes don't break unrelated code
• Easy to test — Components can be tested in isolation
• Easy to scale — Performance bottlenecks are isolated

VeyraOps AI follows a layered architecture pattern. This lesson will teach you:

• What layered architecture means
• Why we separate concerns
• How data flows through the system
• What design patterns we use
• Why these decisions were made

By the end, you'll understand the architectural principles that make this backend maintainable and scalable.

Layered Architecture#

What is Layered Architecture?#

Layered architecture organizes code into horizontal layers where each layer has a specific responsibility.

The Three Main Layers#

1. Presentation Layer (API/Routers)#

• Location: app/routers/
• Responsibility: Handle HTTP requests and responses
• What it does:
  • Receives HTTP requests
  • Validates input using Pydantic schemas
  • Calls business logic in services
  • Returns HTTP responses

Key Files:

• routers/agents.py — Agent-related endpoints
• routers/documents.py — Document/RAG endpoints

2. Business Logic Layer (Services)#

• Location: app/services/
• Responsibility: Implement core functionality
• What it does:
  • Orchestrates LLM calls
  • Implements workflows
  • Processes data
  • Contains no HTTP-specific code

Key Files:

• services/openai_service.py — LLM interactions
• services/multi_agent_email_service.py — Workflow orchestration
• services/document_service.py — RAG operations
• services/monitoring_service.py — Cost tracking

3. Data Access Layer (Models & Database)#

• Location: app/models.py, app/database.py
• Responsibility: Manage data persistence
• What it does:
  • Defines database schema
  • Handles CRUD operations
  • Manages relationships
  • Provides database sessions

Key Files:

• database.py — SQLAlchemy engine and session management
• models.py — ORM models (Agent, Message, Task, etc.)
• schemas.py — Pydantic validation models

Separation of Concerns#

The Core Principle#

Each layer should only know about the layer directly below it.

# ✅ GOOD: Router calls Service, Service calls Database
@router.post("/agents/{agent_id}/chat")
def chat_with_agent(agent_id: int, db: Session = Depends(get_db)):
    agent = db.query(Agent).filter(Agent.id == agent_id).first()  # Data layer
    response = generate_agent_response(agent.system_prompt, message)  # Business layer
    return {"response": response}  # Presentation layer

# ❌ BAD: Service layer knows about HTTP
def generate_agent_response(request: Request):  # Service shouldn't know about Request
    return Response(content=result)  # Service shouldn't return Response

Why This Matters#

1. Testability#

# Easy to test services without HTTP
def test_generate_response():
    result = generate_agent_response("You are helpful", "Hello")
    assert "Hi" in result  # No need for HTTP client

2. Reusability#

# Same service can be used by:
# - REST API
# - GraphQL API
# - CLI tool
# - Background job

3. Maintainability#

# Change database from SQLite to PostgreSQL?
# Only modify database.py, everything else stays the same

The Three-Layer Pattern#

Let's trace a real request through all three layers:

Example: Chat with Agent#

Step 1: Presentation Layer (Router)#

# File: routers/agents.py
@router.post("/{agent_id}/chat", response_model=schemas.ChatResponse)
def chat_with_agent(
    agent_id: int,
    chat_data: schemas.ChatRequest,  # Pydantic validates input
    db: Session = Depends(get_db)    # Dependency injection
):
    start_time = time.time()
    
    # Step 1: Get agent from database
    agent = db.query(models.Agent).filter(models.Agent.id == agent_id).first()
    if agent is None:
        raise HTTPException(status_code=404, detail="Agent not found")
    
    # Step 2: Call business logic
    assistant_text = generate_agent_response(
        system_prompt=agent.system_prompt,
        user_message=chat_data.message,
        agent_name=agent.name
    )
    
    # Step 3: Save to database
    user_message = models.Message(agent_id=agent.id, role="user", content=chat_data.message)
    assistant_message = models.Message(agent_id=agent.id, role="assistant", content=assistant_text)
    db.add(user_message)
    db.add(assistant_message)
    
    # Step 4: Return response
    return {
        "agent_id": agent.id,
        "user_message": chat_data.message,
        "assistant_response": assistant_text
    }

What the Router Does:

• ✅ Validates input (Pydantic)
• ✅ Handles HTTP concerns (status codes, exceptions)
• ✅ Orchestrates calls to business logic
• ✅ Manages database transactions
• ❌ Does NOT contain LLM logic
• ❌ Does NOT know about OpenAI API

Step 2: Business Logic Layer (Service)#

# File: services/openai_service.py
def generate_agent_response(
    system_prompt: str,
    user_message: str,
    agent_name: str,
    language: str = "English",
    tone: str = "Professional"
) -> str:
    """
    Generate a response using OpenAI.
    Pure business logic - no HTTP, no database.
    """
    if not os.getenv("OPENAI_API_KEY"):
        raise ValueError("OPENAI_API_KEY is missing")
    
    final_system_prompt = f"""
You are {agent_name}.
 
Agent instructions:
{system_prompt}
 
Language: {language}
Tone: {tone}
"""
    
    response = client.chat.completions.create(
        model=OPENAI_MODEL,
        messages=[
            {"role": "system", "content": final_system_prompt},
            {"role": "user", "content": user_message}
        ],
        temperature=0.3
    )
    
    return response.choices[0].message.content

What the Service Does:

• ✅ Implements business logic (prompt construction)
• ✅ Calls external APIs (OpenAI)
• ✅ Returns pure data (string)
• ❌ Does NOT know about HTTP requests
• ❌ Does NOT touch the database directly

Step 3: Data Access Layer (Models)#

# File: models.py
class Message(Base):
    __tablename__ = "messages"
    
    id = Column(Integer, primary_key=True, index=True)
    agent_id = Column(Integer, ForeignKey("agents.id"), nullable=False)
    role = Column(String(20), nullable=False)  # "user" or "assistant"
    content = Column(Text, nullable=False)
    created_at = Column(DateTime(timezone=True), server_default=func.now())
    
    agent = relationship("Agent", back_populates="messages")

What the Model Does:

• ✅ Defines database schema
• ✅ Manages relationships (foreign keys)
• ✅ Provides SQLAlchemy interface
• ❌ Does NOT contain business logic
• ❌ Does NOT know about HTTP

Component Responsibilities#

Routers (app/routers/)#

Purpose: HTTP request handling

Responsibilities:

• Define API endpoints
• Validate request data (Pydantic schemas)
• Handle authentication (future)
• Manage HTTP errors (HTTPException)
• Coordinate database transactions
• Call service functions
• Return formatted responses

Should NOT:

• Contain business logic
• Make OpenAI API calls directly
• Process data (that's for services)

Services (app/services/)#

Purpose: Business logic implementation

Responsibilities:

• Implement core functionality
• Call external APIs (OpenAI, etc.)
• Process and transform data
• Orchestrate workflows
• Calculate costs and metrics

Should NOT:

• Know about HTTP requests/responses
• Access database directly (pass db from router)
• Handle HTTP exceptions

Models (app/models.py)#

Purpose: Database schema definition

Responsibilities:

• Define tables and columns
• Specify relationships and foreign keys
• Set default values
• Configure cascades

Should NOT:

• Contain business logic
• Know about HTTP or services
• Perform calculations

Schemas (app/schemas.py)#

Purpose: Request/response validation

Responsibilities:

• Define API request formats
• Define API response formats
• Validate data types
• Set default values
• Configure serialization

Should NOT:

• Define database structure (that's models)
• Contain business logic
• Access the database

Data Flow#

Complete Request Flow#

Data Transformation at Each Layer#

# Layer 1: HTTP Request (JSON)
{
  "message": "Hello"
}
 
# Layer 2: Pydantic Schema (validated)
ChatRequest(message="Hello")
 
# Layer 3: Python function call
generate_agent_response(
    system_prompt="You are helpful",
    user_message="Hello",
    agent_name="Assistant"
)
 
# Layer 4: External API call (OpenAI)
client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[...]
)
 
# Layer 5: External API response
ChatCompletion(choices=[...])
 
# Layer 6: Extract content
"Hi! How can I help?"
 
# Layer 7: Save to database
Message(role="assistant", content="Hi! How can I help?")
 
# Layer 8: Return HTTP response (JSON)
{
  "assistant_response": "Hi! How can I help?",
  "run_id": 42,
  "latency_ms": 850
}

Design Patterns Used#

1. Dependency Injection#

def chat_with_agent(
    agent_id: int,
    chat_data: schemas.ChatRequest,
    db: Session = Depends(get_db)  # ← Dependency Injection
):
    # db is automatically provided by FastAPI

Benefits:

• Easy to test (mock the database)
• Centralized session management
• Automatic cleanup (db.close())

Learn more: Lesson 10: Dependency Injection

2. Repository Pattern (Implicit)#

# Instead of raw SQL everywhere:
agent = db.query(Agent).filter(Agent.id == agent_id).first()
 
# We use SQLAlchemy ORM as our repository

Benefits:

• Database-agnostic code
• Type-safe queries
• Relationship handling

3. Service Layer Pattern#

# Routers don't contain business logic
# They delegate to services:
response = generate_agent_response(...)  # Service function

Benefits:

• Business logic is reusable
• Easier to test
• Clear separation

4. DTO (Data Transfer Object)#

# Pydantic schemas are DTOs
class ChatRequest(BaseModel):
    message: str
 
class ChatResponse(BaseModel):
    agent_id: int
    assistant_response: str
    run_id: int
    latency_ms: int

Benefits:

• Explicit API contracts
• Automatic validation
• Type safety

Why This Architecture#

Advantage 1: Maintainability#

When you need to change something, you know exactly where to look:

Advantage 2: Testability#

Each layer can be tested independently:

# Test services without HTTP
def test_generate_response():
    result = generate_agent_response("You are helpful", "Hello")
    assert isinstance(result, str)
 
# Test routers with HTTP client
def test_chat_endpoint(client):
    response = client.post("/agents/1/chat", json={"message": "Hello"})
    assert response.status_code == 200

Advantage 3: Scalability#

Each layer can scale independently:

# Scale routers (horizontal)
Deploy multiple FastAPI instances behind load balancer
 
# Scale services (vertical)
Move expensive operations to background workers
 
# Scale database (vertical → horizontal)
SQLite → PostgreSQL → Sharded PostgreSQL

Advantage 4: Team Collaboration#

Different developers can work on different layers without conflicts:

# Developer A: Works on new API endpoint
# File: routers/agents.py
 
# Developer B: Works on new AI feature
# File: services/openai_service.py
 
# Developer C: Adds database table
# File: models.py
 
# No merge conflicts!

Alternatives Considered#

Alternative 1: MVC (Model-View-Controller)#

Why Not?

• VeyraOps AI is an API, not a traditional web app
• No "views" (HTML templates)
• FastAPI is not designed for MVC

When to Use MVC?

• Building traditional web applications
• Server-side rendered pages
• Django/Flask with templates

Alternative 2: Hexagonal Architecture (Ports & Adapters)#

Why Not?

• More complex than needed
• Overkill for this project size
• Adds abstractions without clear benefit

When to Use Hexagonal?

• Large enterprise systems
• Multiple data sources
• Complex domain models

Alternative 3: Microservices#

Why Not?

• Operational overhead too high
• Network latency between services
• Deployment complexity
• Overkill for current scale

When to Use Microservices?

• Large teams (100+ developers)
• Independent scaling requirements
• Organizational boundaries

Scalability Considerations#

Current Architecture Strengths#

✅ Easy to understand — New developers onboard quickly
✅ Fast development — No over-engineering
✅ Sufficient for 100-1000 users — SQLite handles this
✅ Low operational cost — Single deployment

Future Scaling Path#

Phase 1: Current (0-1,000 users)#

FastAPI + SQLite + ChromaDB
Single server deployment

Phase 2: Database Upgrade (1,000-10,000 users)#

FastAPI + PostgreSQL + Weaviate
Read replicas for database

Phase 3: Horizontal Scaling (10,000-100,000 users)#

Multiple FastAPI instances + Load Balancer
PostgreSQL with connection pooling
Redis for caching
Background workers for async tasks

Phase 4: Service Splitting (100,000+ users)#

API Gateway
↓
├── Agent Service
├── Document Service
├── Workflow Service
└── Monitoring Service
 
Each service has its own database

Key Takeaways#

🎯 VeyraOps AI uses layered architecture with three main layers: Presentation (Routers), Business Logic (Services), and Data Access (Models).

🎯 Separation of concerns means each layer has a single responsibility and doesn't know about implementation details of other layers.

🎯 Routers handle HTTP, Services implement business logic, Models define database schema.

🎯 Design patterns used include Dependency Injection, Service Layer, Repository (via ORM), and DTOs (via Pydantic).

🎯 This architecture is maintainable because changes are localized to specific layers.

🎯 Alternative architectures like microservices or hexagonal were considered but rejected as over-engineering.

🎯 The system can scale by upgrading the database, adding caching, horizontal scaling, and eventually splitting into microservices.

Interview Questions#

Junior Level#

Q1: What are the three layers in VeyraOps AI architecture?
A: Presentation (Routers), Business Logic (Services), Data Access (Models/Database).

Q2: What is the responsibility of the router layer?
A: Handle HTTP requests, validate input, coordinate calls to services, return HTTP responses.

Q3: Why shouldn't services know about HTTP?
A: To make them reusable in different contexts (CLI, background jobs, different APIs).

Mid-Level#

Q4: Explain separation of concerns with an example from VeyraOps AI.
A: Routers handle HTTP, services implement LLM logic, models define database. For example, generate_agent_response() in services doesn't know it's being called from an HTTP endpoint.

Q5: What design pattern does FastAPI's Depends(get_db) implement?
A: Dependency Injection pattern.

Q6: How would you test a service function?
A: Call it directly with parameters, without needing HTTP client or database, by mocking external dependencies.

Senior Level#

Q7: Compare layered architecture vs microservices for VeyraOps AI. When would you choose microservices?
A: Layered (monolithic) is simpler and sufficient for current scale. Microservices would be chosen when: (1) teams need independent deployment, (2) services scale differently, (3) organizational boundaries exist, (4) handling 100K+ users.

Q8: How does this architecture support the transition from SQLite to PostgreSQL?
A: Using SQLAlchemy ORM abstracts the database. Only database.py needs changes (connection string). All queries in routers remain unchanged.

Q9: Design a caching strategy for this architecture. Which layer should implement it?
A: Caching should be in the service layer (or a dedicated caching service). For example, cache OpenAI responses in openai_service.py with Redis. Routers remain unchanged, benefiting from transparent caching.

Exercises#

Exercise 1: Layer Identification#

Task: Take this code snippet and identify which layer it belongs to and why:

def analyze_email(email_text: str) -> dict:
    response = client.chat.completions.create(
        model=OPENAI_MODEL,
        messages=[
            {"role": "system", "content": "Analyze this email..."},
            {"role": "user", "content": email_text}
        ]
    )
    return json.loads(response.choices[0].message.content)

Answer: Business Logic Layer (Service). It calls an external API, processes data, returns pure data, doesn't know about HTTP.

Exercise 2: Refactoring#

Task: This code violates separation of concerns. Refactor it:

@router.post("/agents/{agent_id}/chat")
def chat(agent_id: int, message: str):
    # BAD: Business logic in router
    response = openai.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": message}]
    )
    return {"response": response.choices[0].message.content}

Solution: Move OpenAI call to services/openai_service.py.

Exercise 3: Architecture Diagram#

Task: Draw the architecture diagram from memory, showing all layers and their relationships.

Exercise 4: Design Decision#

Task: A teammate suggests moving all database queries into a separate repositories/ folder. Analyze pros and cons.

Exercise 5: Scaling Planning#

Task: The system now has 50,000 users. Chat response time is 5 seconds. Diagnose bottlenecks and propose solutions.

Next Steps#

You now understand how VeyraOps AI is architected and why these decisions were made.

In the next lesson, you'll follow a complete request lifecycle from HTTP request to database to AI to response.

Ready to trace the data flow?

👉 Lesson 03: Request Lifecycle →

Related Lessons#

• Lesson 10: Dependency Injection — Deep dive into FastAPI's DI
• Lesson 24: Performance — Real, verified performance costs in the current architecture
• Lesson 27: Configuration & Environment Variables — Code organization and configuration patterns

← Previous: Project Overview | ↑ Back to Index | Next: Request Lifecycle →

Module 1 · Foundation

Last Updated: 2026-07-17