Citrus LLM Evaluation Platform - Development Guide

Version: 2.4.0
Last Updated: March 2026

Complete development guide for contributing to and extending the Citrus LLM Evaluation Platform.

Development Environment Setup
Project Structure
Backend Development
Frontend Development
Testing
Code Style & Standards
Adding New Features
Debugging
Contributing Guidelines

Development Environment Setup

Prerequisites

Tool	Version	Purpose
Python	3.9+	Backend runtime
Node.js	18+	Frontend runtime
MongoDB	6.0+	Database
Docker	20.10+	Vault container
Git	2.30+	Version control

Initial Setup

# 1. Clone the repository
git clone <repository-url>
cd Citrust

# 2. Create Python virtual environment
python -m venv venv

# Windows
venv\Scripts\activate

# macOS/Linux
source venv/bin/activate

# 3. Install backend dependencies
pip install -r app/requirements.txt

# 4. Install SpaCy model
python -m spacy download en_core_web_lg

# 5. Setup frontend
cd citrus_frontend
npm install
cd ..

# 6. Copy environment template
cp .env.example .env
# Edit .env with your API keys

Start Development Services

# Terminal 1: Start Vault
docker network create citrus-network
docker-compose -f docker-compose.vault.yml up -d

# Terminal 2: Start Backend
venv\Scripts\activate  # or source venv/bin/activate
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Terminal 3: Start Frontend
cd citrus_frontend
npm run dev

Access Points

Service	URL
Frontend	http://localhost:5173
Backend API	http://localhost:8000
API Docs	http://localhost:8000/docs
Vault UI	http://localhost:8200

Project Structure

Repository Layout

Citrust/
├── app/                          # Backend (FastAPI)
│   ├── __init__.py
│   ├── config.py                 # Configuration
│   ├── main.py                   # Application entry
│   ├── requirements.txt          # Python dependencies
│   ├── core/                     # Core infrastructure
│   │   ├── database.py           # MongoDB client
│   │   ├── vault_client.py       # Vault integration
│   │   ├── pii_redaction.py      # PII detection
│   │   ├── tracing.py            # Request tracing
│   │   ├── trace_storage.py      # Trace persistence
│   │   └── model_wrappers.py     # LLM abstractions
│   ├── models/                   # Pydantic schemas
│   │   ├── schemas.py            # General schemas
│   │   ├── user_schemas.py       # Auth schemas
│   │   ├── evaluation_schemas.py # Evaluation schemas
│   │   ├── trace_schemas.py      # Trace schemas
│   │   └── state.py              # LangGraph state
│   ├── routers/                  # API endpoints
│   │   ├── auth.py               # Authentication
│   │   ├── evaluations.py        # Chat/evaluations
│   │   └── traces.py             # Analytics
│   ├── services/                 # Business logic
│   │   ├── graph.py              # LangGraph workflow
│   │   ├── model_client.py       # LLM client
│   │   ├── evaluation_runner.py  # Evaluation orchestration
│   │   └── vaultgemma_evaluator.py
│   └── tests/                    # Backend tests
│
├── citrus_frontend/              # Frontend (React)
│   ├── src/
│   │   ├── main.tsx              # Entry point
│   │   ├── App.tsx               # Root component
│   │   ├── api*.ts               # API clients
│   │   ├── context/              # React contexts
│   │   ├── components/           # UI components
│   │   │   ├── ui/               # Base components
│   │   │   └── *.tsx             # Feature components
│   │   ├── pages/                # Page components
│   │   ├── lib/                  # Utilities
│   │   └── __tests__/            # Frontend tests
│   ├── package.json
│   ├── vite.config.ts
│   ├── tailwind.config.js
│   └── tsconfig.json
│
├── docs/                         # Documentation
│   ├── FEATURES.md
│   ├── TECH_STACK.md
│   ├── ARCHITECTURE.md
│   ├── API_REFERENCE.md
│   ├── PRIVACY_SECURITY.md
│   └── DEVELOPMENT.md            # This file
│
├── .env.example                  # Environment template
├── docker-compose.vault.yml      # Vault configuration
├── DOCUMENTATION.md              # Setup guide
├── README.md                     # Project overview
└── SETUP.md                      # Installation guide

Backend Development

Adding a New Endpoint

Create/Update Schema (app/models/)

# app/models/schemas.py
from pydantic import BaseModel

class NewFeatureRequest(BaseModel):
    field1: str
    field2: int = 10

class NewFeatureResponse(BaseModel):
    result: str
    processed_at: datetime

Create Service Logic (app/services/)

# app/services/new_feature_service.py
from app.models.schemas import NewFeatureRequest, NewFeatureResponse

class NewFeatureService:
    async def process(self, request: NewFeatureRequest) -> NewFeatureResponse:
        # Business logic here
        result = f"Processed: {request.field1}"
        return NewFeatureResponse(
            result=result,
            processed_at=datetime.now(timezone.utc)
        )

Create Router (app/routers/)

# app/routers/new_feature.py
from fastapi import APIRouter, Depends
from app.models.schemas import NewFeatureRequest, NewFeatureResponse
from app.services.new_feature_service import NewFeatureService

router = APIRouter(prefix="/api/v1/new-feature", tags=["New Feature"])

@router.post("/", response_model=NewFeatureResponse)
async def create_new_feature(request: NewFeatureRequest):
    service = NewFeatureService()
    return await service.process(request)

Register Router (app/main.py)

from app.routers import new_feature

# In main.py
app.include_router(new_feature.router)

Database Operations

# app/core/database.py

class MongoDB:
    # ... existing code ...
    
    @property
    def new_collection(self):
        return self.database["new_collection"]

# Usage in service
from app.core.database import mongodb

async def save_item(item: dict):
    result = await mongodb.new_collection.insert_one(item)
    return str(result.inserted_id)

async def get_item(item_id: str):
    return await mongodb.new_collection.find_one({"_id": ObjectId(item_id)})

Adding New LLM Models

Update Config (app/config.py)

class Settings(BaseSettings):
    # Add new model API key
    new_model_api_key: str = os.getenv("NEW_MODEL_API_KEY", "")

Create Model Wrapper (app/core/model_wrappers.py)

class NewModelClient:
    def __init__(self, api_key: str, model_name: str):
        self.api_key = api_key
        self.model_name = model_name
    
    async def generate(self, prompt: str, **kwargs) -> str:
        # Implementation
        pass
    
    async def astream(self, prompt: str, **kwargs):
        # Streaming implementation
        pass

Register in Factory

def get_model_client(model_name: str):
    if model_name.startswith("new-model"):
        return NewModelClient(settings.new_model_api_key, model_name)
    # ... existing models

Frontend Development

Creating a New Page

Create Page Component (src/pages/)

// src/pages/NewFeaturePage.tsx
import { useState, useEffect } from 'react';
import { MetricCard } from '../components/MetricCard';

export function NewFeaturePage() {
  const [data, setData] = useState(null);
  const [loading, setLoading] = useState(true);

  useEffect(() => {
    fetchData();
  }, []);

  const fetchData = async () => {
    try {
      const response = await fetch('/api/v1/new-feature');
      const result = await response.json();
      setData(result);
    } catch (error) {
      console.error('Failed to fetch:', error);
    } finally {
      setLoading(false);
    }
  };

  if (loading) return <div>Loading...</div>;

  return (
    <div className="p-6">
      <h1 className="text-2xl font-bold mb-6">New Feature</h1>
      {/* Page content */}
    </div>
  );
}

Add Route (src/App.tsx)

import { NewFeaturePage } from './pages/NewFeaturePage';

// In Routes
<Route path="/new-feature" element={
  <ProtectedRoute>
    <NewFeaturePage />
  </ProtectedRoute>
} />

Add Navigation (src/components/Sidebar.tsx)

const navItems = [
  // ... existing items
  { path: '/new-feature', label: 'New Feature', icon: Star },
];

Creating a New Component

// src/components/FeatureCard.tsx
import { Card, CardHeader, CardContent } from './ui/card';

interface FeatureCardProps {
  title: string;
  description: string;
  onClick?: () => void;
}

export function FeatureCard({ title, description, onClick }: FeatureCardProps) {
  return (
    <Card 
      className="glass-panel cursor-pointer hover:border-primary/50 transition-colors"
      onClick={onClick}
    >
      <CardHeader>
        <h3 className="text-lg font-semibold">{title}</h3>
      </CardHeader>
      <CardContent>
        <p className="text-gray-400">{description}</p>
      </CardContent>
    </Card>
  );
}

Adding API Functions

// src/api_new_feature.ts
const API_BASE = '/api/v1/new-feature';

export interface NewFeatureData {
  id: string;
  name: string;
  value: number;
}

export const newFeatureApi = {
  getAll: async (): Promise<NewFeatureData[]> => {
    const response = await fetch(API_BASE);
    if (!response.ok) throw new Error('Failed to fetch');
    return response.json();
  },

  getById: async (id: string): Promise<NewFeatureData> => {
    const response = await fetch(`${API_BASE}/${id}`);
    if (!response.ok) throw new Error('Failed to fetch');
    return response.json();
  },

  create: async (data: Partial<NewFeatureData>): Promise<NewFeatureData> => {
    const response = await fetch(API_BASE, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify(data),
    });
    if (!response.ok) throw new Error('Failed to create');
    return response.json();
  },
};

Using Context

// src/context/NewFeatureContext.tsx
import { createContext, useContext, useState, ReactNode } from 'react';

interface NewFeatureContextType {
  selectedItem: string | null;
  setSelectedItem: (item: string | null) => void;
}

const NewFeatureContext = createContext<NewFeatureContextType | null>(null);

export function NewFeatureProvider({ children }: { children: ReactNode }) {
  const [selectedItem, setSelectedItem] = useState<string | null>(null);

  return (
    <NewFeatureContext.Provider value={{ selectedItem, setSelectedItem }}>
      {children}
    </NewFeatureContext.Provider>
  );
}

export function useNewFeature() {
  const context = useContext(NewFeatureContext);
  if (!context) {
    throw new Error('useNewFeature must be used within NewFeatureProvider');
  }
  return context;
}

Testing

Backend Tests

# app/tests/test_evaluations.py
import pytest
from httpx import AsyncClient
from app.main import app

@pytest.fixture
async def client():
    async with AsyncClient(app=app, base_url="http://test") as client:
        yield client

@pytest.mark.asyncio
async def test_health_check(client):
    response = await client.get("/health")
    assert response.status_code == 200
    assert response.json()["status"] == "healthy"

@pytest.mark.asyncio
async def test_dual_responses(client):
    response = await client.post("/api/v1/evaluations/dual-responses", json={
        "user_message": "Test message",
        "chat_history": [],
        "session_id": "test-session"
    })
    assert response.status_code == 200

Run backend tests:

pytest app/tests/ -v
pytest app/tests/ -v --cov=app --cov-report=html

Frontend Tests

// src/components/__tests__/MetricCard.test.tsx
import { render, screen } from '@testing-library/react';
import { MetricCard } from '../MetricCard';

describe('MetricCard', () => {
  it('renders title and value', () => {
    render(<MetricCard title="Test Metric" value={42} />);
    
    expect(screen.getByText('Test Metric')).toBeInTheDocument();
    expect(screen.getByText('42')).toBeInTheDocument();
  });

  it('formats large numbers', () => {
    render(<MetricCard title="Large Number" value={1234567} />);
    
    expect(screen.getByText('1,234,567')).toBeInTheDocument();
  });
});

Run frontend tests:

cd citrus_frontend
npm run test           # Run tests
npm run test:ui        # Run with UI
npm run test -- --coverage  # With coverage

Code Style & Standards

Python (Backend)

Tools:

black - Code formatting
flake8 - Linting
mypy - Type checking

# Format code
black app/

# Lint
flake8 app/

# Type check
mypy app/

Configuration (.flake8):

[flake8]
max-line-length = 100
exclude = venv,__pycache__
ignore = E203, W503

Style Guide:

# Good
async def get_user_by_id(user_id: str) -> Optional[User]:
    """Fetch a user by their ID."""
    return await mongodb.users.find_one({"_id": ObjectId(user_id)})

# Avoid
async def getUser(id):
    return await mongodb.users.find_one({"_id": ObjectId(id)})

TypeScript (Frontend)

Tools:

eslint - Linting
prettier - Formatting (via ESLint)

cd citrus_frontend
npm run lint
npm run lint -- --fix  # Auto-fix

Style Guide:

// Good
interface UserProfile {
  id: string;
  name: string;
  email: string;
}

function UserCard({ user }: { user: UserProfile }) {
  return (
    <div className="p-4">
      <h2>{user.name}</h2>
      <p>{user.email}</p>
    </div>
  );
}

// Avoid
function userCard(props: any) {
  return <div><h2>{props.user.name}</h2></div>
}

Git Commit Messages

<type>(<scope>): <subject>

<body>

<footer>

Types:

feat: New feature
fix: Bug fix
docs: Documentation
style: Formatting
refactor: Code restructuring
test: Adding tests
chore: Maintenance

Examples:

feat(auth): add OTP-based authentication

Implement email-based OTP authentication with JWT tokens.
- Add /api/auth/login endpoint
- Add /api/auth/verify-otp endpoint
- Configure SMTP settings

Closes #123

Adding New Features

Feature Checklist

Example: Adding a New Metric

Backend Schema

class NewMetricResponse(BaseModel):
    metric_name: str
    value: float
    unit: str
    timestamp: datetime

Backend Service

async def calculate_new_metric() -> NewMetricResponse:
    # Calculate metric
    value = await aggregate_data()
    return NewMetricResponse(
        metric_name="New Metric",
        value=value,
        unit="requests/sec",
        timestamp=datetime.now(timezone.utc)
    )

Backend Endpoint

@router.get("/new-metric", response_model=NewMetricResponse)
async def get_new_metric():
    """Get the new metric value."""
    return await calculate_new_metric()

Frontend API

export const getNewMetric = async () => {
  const response = await fetch('/api/v1/new-metric');
  return response.json();
};

Frontend Component

function NewMetricDisplay() {
  const [metric, setMetric] = useState(null);
  
  useEffect(() => {
    getNewMetric().then(setMetric);
  }, []);
  
  return <MetricCard title={metric?.metric_name} value={metric?.value} />;
}

Debugging

Backend Debugging

Enable Debug Logging:

import logging
logging.basicConfig(level=logging.DEBUG)

VS Code Launch Config:

{
  "name": "Debug Backend",
  "type": "python",
  "request": "launch",
  "module": "uvicorn",
  "args": ["app.main:app", "--reload", "--port", "8000"],
  "jinja": true
}

Common Issues:

Issue	Solution
Import errors	Ensure virtual env activated
MongoDB connection	Check MONGODB_URL
Vault errors	Verify Vault is running
API key errors	Check .env configuration

Frontend Debugging

React DevTools:

Install React DevTools browser extension
Use Components tab for component tree
Use Profiler tab for performance

Console Logging:

useEffect(() => {
  console.log('Data changed:', data);
}, [data]);

Network Debugging:

Open DevTools Network tab
Filter by XHR/Fetch
Check request/response payloads

Contributing Guidelines

Pull Request Process

Fork & Branch

git checkout -b feature/your-feature-name

Make Changes

Follow code style guidelines
Add tests for new features
Update documentation

Test Locally

# Backend
pytest app/tests/ -v

# Frontend
cd citrus_frontend && npm run test

Commit

git add .
git commit -m "feat(scope): description"

Push & Create PR

git push origin feature/your-feature-name
# Create PR on GitHub

PR Requirements

Code Review

Reviewers check for:

Correctness
Test coverage
Security implications
Performance impact
Code readability
Documentation

Resources

Documentation

Internal Docs

FEATURES.md - Feature documentation
TECH_STACK.md - Technology stack
ARCHITECTURE.md - System architecture
API_REFERENCE.md - API documentation
PRIVACY_SECURITY.md - Security guide

Development Guide for Citrus LLM Evaluation Platform v2.4.0

FilesExpand file tree

DEVELOPMENT.md

Latest commit

History