Skip to content

feat(ai-service): add OpenTelemetry tracing around LLM calls#365

Open
ugoocreates-pixel wants to merge 1 commit into
ChainForgee:mainfrom
ugoocreates-pixel:feat/otel-tracing
Open

feat(ai-service): add OpenTelemetry tracing around LLM calls#365
ugoocreates-pixel wants to merge 1 commit into
ChainForgee:mainfrom
ugoocreates-pixel:feat/otel-tracing

Conversation

@ugoocreates-pixel

@ugoocreates-pixel ugoocreates-pixel commented Jul 16, 2026

Copy link
Copy Markdown

Closes #273

This PR introduces distributed tracing to the humanitarian verification service by wrapping LLM calls in OpenTelemetry spans. This will allow for bottleneck tracking, cost attribution, and distributed trace visualization in Jaeger.

Key Changes

  1. OpenTelemetry Setup (app/ai-service/tracing/otel_setup.py):

    • Implemented a unified OpenTelemetry initialization setup.
    • Wires the service to an OTLP/HTTP exporter if the OTEL_EXPORTER_OTLP_ENDPOINT environment variable is defined.
    • Automatically configures an InMemorySpanExporter when in-memory/test environments (APP_ENV=test) or fallback modes are active.
    • Exposes clean methods to access the tracer and fetch captured spans for verification.
  2. Lifespan Initialization (app/ai-service/main.py):

    • Hooked OpenTelemetry initialization (setup_tracing()) into the FastAPI startup lifecycle.
  3. Span Wrapping around LLM Calls (app/ai-service/services/humanitarian_verification.py):

    • Wrapped _call_provider, _call_openai, and _call_groq in canonical OTel trace spans.
    • Populated the spans with attributes containing the LLM model and prompt_variant.
    • Propagated the prompt_variant parameter seamlessly from verify_claim down through _call_provider to the provider-specific calls.
  4. Integration Testing (app/ai-service/tests/test_otel_spans.py):

    • Added integration unit tests verifying that exactly two spans (the parent provider span and the nested provider implementation span) are emitted per request with correct model and prompt variant attributes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generate canonical trace spans around LLM calls (OpenTelemetry)

1 participant