AI for the receiving side of government forms.
A first wave of AI products has made it remarkably easy for businesses and individuals to fill out forms. The other side of that same workflow has not kept up. Local, state, and federal governments still receive most of those forms on paper, route them between desks, and process them by hand.
The result is long backlogs, inconsistent decisions, and a lot of human time spent on tasks a machine can prepare in seconds.
DocQFlow is a working prototype of what AI can look like on the government side of the counter.
DocQFlow picks one real-world form, the San Francisco Form 3-8 building permit, and runs every incoming PDF through six stages:
- Classify the document so unknown forms get bounced before a reviewer ever sees them.
- Gate-check the file (size, page count, AcroForm presence) so broken inputs never reach the pipeline.
- Store the original PDF in object storage with a content hash so duplicates collapse into one record.
- Extract all 87 fields from the AcroForm.
- Validate those fields against a gazetteer and the form's own internal rules.
- Reason about the result with an LLM and produce a list of human-readable issues.
A reviewer opens the Inbox, sees one row per permit with its current state and the issues already flagged, and either confirms or overrides. They never have to read a clean permit cover to cover again. The system records every decision so the next iteration can learn from it.
The Inbox UI in the gif above is the reviewer experience. The two terminal recordings below show the system from the outside, with each one exercising a different slice of the pipeline.
Every change to the extract, validate, and reason stages is scored against a labelled corpus. Same script, same corpus, same labels, so the diff in metrics tells you exactly what got better or worse.
Full details and the metric definitions live in docs/eval.md.
With the API running locally, you can post a permit PDF and get back a classification with per-class probabilities.
A sample permit is checked in at samples/sample-permit.pdf so the command in the gif is reproducible end-to-end.
| Layer | Stack |
|---|---|
| API | FastAPI, Python 3.12, async by default |
| Pipeline LLM | LiteLLM with a cloud-fast profile (OpenAI gpt-4o-mini) |
| Frontend | React, Vite, TypeScript |
| Data | Supabase Postgres, Supabase Auth (magic link plus allowlist) |
| Blobs | Google Cloud Storage |
| Tracking | MLflow for classifier training runs |
| Deploy | Google Cloud Run with a no-traffic canary tag before promote |
The deeper technical pieces all live in docs/:
- docs/api.md: endpoint reference, auth, CORS
- docs/eval.md: eval harness, metrics, how to score a new profile
- docs/llm-profiles.md: LLM profile definitions and credentials
- docs/model-training.md: classifier training for Stages 1 to 3
- docs/permit-generation.md: synthetic training data generator
- docs/docker-registry.md: pushing the image to Artifact Registry
- docs/setup/gcp.md and docs/setup/supabase.md: infra setup walkthroughs
uv sync
cp .env.example .env # then fill in MLFLOW_TRACKING_URI and any other vars you need
uv run uvicorn src.server:app --reload --port 8080Open http://localhost:8080/docs for the interactive API docs. For Docker, training, or the eval harness, follow the links in the section above.
DocQFlow is a school project. There is one environment (dev), one form (SF Form 3-8), and no production deployment planned. Training data is generated by scripts/generate_permits.py from publicly available San Francisco Data Portal records, so the system only ever sees information that is already in the public domain.
The point of the project is to make the AI-for-government argument concrete. Pick a real form, build the receiving-side pipeline end to end, and show that the resulting reviewer experience is one a real agency could actually use.
See CONTRIBUTING.md for development setup and code style. Recent changes are recorded in CHANGELOG.md.