Add load-shedding behavior under pressure for AI service (#621)

[gidadoabdullateef5]

Closes #621

---

Summary

Implements load-shedding for the AI service when the system is under pressure, addressing #621.

When overloaded, the service now returns HTTP 503 with the standardized ErrorEnvelope (code: SERVICE_OVERLOADED) and emits a dedicated requests_shed_total Prometheus counter so shed requests are observable independently of general traffic metrics.

Load shedding triggers on three conditions:

Condition	Reason label	Where checked
RAM/VRAM above threshold (default 90%)	memory	Request middleware (all /v1/* routes)
Celery queue at capacity (default ≥ 100)	queue_full	Middleware + create_task()
Redis/broker unreachable	broker_unavailable	Middleware + create_task()
All configured LLM providers circuit-open	provider_down	Middleware on humanitarian verify route

Infrastructure routes (/health, /ai/metrics, docs, legacy redirects) remain exempt so load balancers and scrapers stay healthy during overload.

Changes

• New services/load_shedder.py — central shed evaluation, response builder, queue depth probe
• New LoadShedError exception + global handler returning 503 envelope with Retry-After: 30
• metrics.py — requests_shed_total{reason,method,endpoint} counter + celery_queue_depth gauge
• config.py — load_shed_memory_threshold_percent, load_shed_max_celery_queue_depth
• main.py — middleware delegates to load shedder (replaces raw {"detail": ...} 503)
• tasks.py — ensure_queue_capacity() before Celery enqueue
• humanitarian_verification.py — all_providers_unavailable() for provider-down detection
• Tests — test_load_shedder.py + envelope/throttle coverage updates

Acceptance criteria

• Returns 503 with clear error envelope (SERVICE_OVERLOADED, reason in details)
• Emits metric for shed requests (requests_shed_total)

Test plan

• pytest tests/test_load_shedder.py tests/test_error_envelope.py::TestServiceOverloaded — 14 passed
• Verify /health returns 200 even when memory pressure is simulated
• Verify /v1/ai/inference returns 503 + envelope when queue depth exceeds threshold
• Verify /ai/metrics exposes requests_shed_total after a shed event
• Confirm Retry-After: 30 header is present on 503 responses

Monitoring

Suggested alert:

rate(requests_shed_total[5m]) > 0

[vercel]

@gidadoabdullateef5 is attempting to deploy a commit to the Cedarich's projects Team on Vercel.

A member of the Team first needs to authorize it.

[Cedarich]

@gidadoabdullateef5 fix workflow

[drips-wave]

@gidadoabdullateef5 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

[gidadoabdullateef5]

Hi Maintainer, kindly review the workflow

[Cedarich]

@gidadoabdullateef5 fix CI

[gidadoabdullateef5]

Hello Maintainer, the CI has been now fixed alongside with the conflict, please check again