Goddard — HIL Test Intelligence Knowledge Base

An event-driven system that makes HIL (Hardware-in-the-Loop) test runs intelligent. A thin pytest plugin sends test events to a FastAPI server that classifies failures via Claude API and builds a compounding wiki-style knowledge base. Each nightly run benefits from everything learned in previous runs.

Architecture

HIL Pytest Plugin ──── HTTP events ────> FastAPI Server
  (thin client)                            │
                                           ├── Ingestion (persist events)
                                           ├── Agent Reasoning (Claude API)
                                           │     classify failure
                                           │     update knowledge base
                                           │     persist classification
                                           └── Read-Only API
                                                 │
                                           PostgreSQL
                                             ├── hil_events (immutable audit trail)
                                             ├── wiki_pages + revisions
                                             ├── wiki_cross_references
                                             └── classifications

Four components:

Component	What it does	Location
Pytest Plugin	Sends fire-and-forget HTTP events from HIL rigs. No LLM, no slowdown.	plugin/
FastAPI Server	Ingests events, classifies failures async via Claude API, builds knowledge base.	server/
Knowledge Base	Wiki-style pages in Postgres with revision tracking and cross-references.	Postgres tables
Web Viewer	Read-only UI for browsing the knowledge base.	Not yet built

Project Structure

goddard/
├── server/                          # FastAPI server
│   ├── pyproject.toml
│   ├── alembic.ini
│   ├── migrations/                  # Alembic async migrations
│   ├── src/goddard_server/
│   │   ├── main.py                  # FastAPI app + lifespan
│   │   ├── config.py                # Settings (env vars with GODDARD_ prefix)
│   │   ├── db.py                    # SQLAlchemy async engine + session
│   │   ├── dependencies.py          # FastAPI Depends (DB session)
│   │   ├── models/                  # SQLAlchemy models (5 tables)
│   │   ├── schemas/                 # Pydantic request/response schemas
│   │   ├── routers/                 # API routes (events, sessions, wiki)
│   │   └── services/                # Business logic
│   │       ├── event_service.py     # Event persistence
│   │       ├── wiki_service.py      # Wiki CRUD + revisions
│   │       ├── classification_service.py  # Claude API classification
│   │       └── reasoning_service.py # Orchestration pipeline
│   └── tests/                       # 21 tests (pytest + SQLite)
├── plugin/                          # Pytest plugin (separate package)
│   ├── pyproject.toml
│   ├── src/pytest_goddard_hil/
│   │   └── plugin.py               # Hooks + fire-and-forget HTTP
│   └── tests/                       # 5 tests
└── docs/
    └── superpowers/
        ├── specs/                   # Design spec
        └── plans/                   # Implementation plan

API Endpoints

Method	Path	Description
POST	/hil/events	Ingest a plugin event (session_start, test_result, etc.)
GET	/hil/sessions/{session_id}	Session triage report with events + classifications
GET	/hil/wiki/pages	List wiki pages (optional ?category= filter)
GET	/hil/wiki/pages/{slug}	Page detail with full content
GET	/hil/wiki/pages/{slug}/revisions	Revision history for a page
GET	/health	Health check

Classification Types

When a test fails or errors, the reasoning pipeline classifies it as:

Type	Description
infra_failure	Hardware/infrastructure (hung COM ports, broken fixtures, network)
flaky_test	Timing-sensitive, fails intermittently without code changes
regression	Real functional regression from a code/config change
known_issue	Matches a previously documented failure pattern

Wiki Page Categories

The knowledge base organizes pages by category:

Category	Purpose	Example slug
test_profile	One page per test case — history, flakiness, linked failures	test_profile/tests-test_ecu-py--test_wakeup
failure_pattern	Reusable patterns across tests/rigs	failure_pattern/com_port_hung_on_rig3
rig_profile	Per-rig health, hardware quirks, recurring issues	rig_profile/rig_03
fix_recipe	Known solutions — the runbook layer	fix_recipe/restart_com_port_service
session_summary	Triage report per nightly run	session_summary/2026-04-07_rig03_nightly
trend_analysis	Cross-session insights	trend_analysis/ecu_wakeup_degradation_q2

Phased Rollout

• Phase 1 (current): Events + classification + knowledge base + read-only API. No feedback to pytest. Validate accuracy.
• Phase 2: Plugin polls /advisories/{rig_id} before sessions. Skip/retry directives with auto-expiry.
• Phase 3: Mid-session advisory polling, dynamic timeouts, "stop session" signals.

Tests

# Server tests (21 tests, uses SQLite — no Postgres needed)
cd server && uv run pytest -v

# Plugin tests (5 tests)
cd plugin && uv run pytest -v

Full Spec

See docs/superpowers/specs/2026-04-07-hil-knowledge-base-design.md for the complete design document.