Variant in, corrective edit out.
A variant-driven, multi-modality, uncertainty-aware CRISPR guide & edit design framework — across SpCas9 nuclease, base editors, and prime editors, with population-aware off-target nomination and a public benchmark.
Warning
AlleleForge is a research tool. It is not a medical device and does not provide medical advice. It produces ranked, explicitly uncertain design hypotheses. Every off-target nomination it makes is computational and must be experimentally validated before any wet-lab or therapeutic use. See Scope & responsible use.
Most monogenic disease is, in effect, a copy-paste error at the allele level. The job of a genome editor is to forge the corrective edit. Today that job is fragmented across a dozen single-purpose tools — one to pick a guide, another to predict efficiency, a third to enumerate prime-editing extensions, a fourth to scan for off-targets — none of which speak the same language and few of which agree on what "uncertain" means.
AlleleForge unifies the journey behind one typed interface: you supply a variant, it returns a ranked, safety-annotated menu of candidate edits spanning every applicable modality, each carrying a calibrated uncertainty interval, a predicted edit outcome, and a population- and haplotype-aware off-target report.
For prime editing in particular, no existing open-source tool combines all four of:
| Axis | PRIDICT2.0 | PrimeDesign / PrimeVar | CRISPRme | AlleleForge |
|---|---|---|---|---|
| Therapeutic variant front-end | ✗ | ✓ | ✗ | ✓ |
| ML efficiency with calibrated uncertainty | ✓ | ✗ | ✗ | ✓ |
| Outcome / byproduct prediction | partial | ✗ | ✗ | ✓ |
| Population-aware off-target | ✗ | ✗ | ✓ | ✓ |
AlleleForge's contribution is to wrap the best existing models (PRIDICT2.0, BE-Hive, BE-DICT, inDelphi, Cas-OFFinder, …) behind a unified, typed, uncertainty-honest interface and add value at the seams.
- Variant-first. The canonical journey starts from what is broken, not from a guide.
- Honest uncertainty. Every numeric prediction ships with a calibrated interval. No scorer returns a bare float — including
P(intended), the probability the edit produces the allele you asked for, which is the number a reader is most likely to act on. Where a chemistry's outcome predictor makes no such prediction (SpCas9 nuclease), the figure is labelled derived from the outcome distribution rather than given a band it does not have. - Population-aware, and explicit when it cannot be. Reference-only off-target analysis is a known
safety gap (the Casgevy / BCL11A
rs114518452case is the canonical cautionary tale), so population and haplotype variation is a first-class search pass rather than an add-on. It is not on by default, because AlleleForge vendors no gnomAD data: supply a frequency source (--gnomad,--haplotypes,--patient-vcf) and the scan is population-aware; supply none and it is reference-only — and every surface says so out loud, because an empty ancestry breakdown means not measured, not clean. - Wrap, don't rebuild. Integrate proven tools; add new ML only at genuine coverage gaps.
- Reproducible to the byte. Pinned environments, versioned datasets, deterministic seeds, content-hashed checkpoints.
- Three audiences, one core. The library is the source of truth; CLI and web are thin shells over it.
- Typed and tested.
mypy --strict,ruff, and Hypothesis property tests on all core logic. - Cite everything. Every dataset and model in the registries carries a literature citation and a version, and both travel into a result's provenance. A user-supplied input (your own gnomAD slice, haplotype panel or patient VCF) has no literature to cite, so it is pinned by content hash instead — recorded, not attributed.
The principles above are realized by a handful of concrete, non-obvious engineering tradeoffs. Each was chosen to maximize reproducibility, honest uncertainty, and population-aware safety — in that order; the rationale and the code that enforces it:
| Decision | Why — and where it lives |
|---|---|
Weight-free stubs are the CI default; real weights are opt-in (real_weights marker) |
The full gate (lint, type, test, docs, examples, reproduce) runs with no GPU, network, or torch, so any contributor reproduces it byte-for-byte. The consent/license/checksum flow is still exercised in CI with an injected downloader; only the tensor load / forward pass is gated. See SPEC_V2.md R1. |
| An unverifiable artifact is refused, never fetched | A null checkpoint/dataset hash blocks the download by design — you cannot silently load an unpinned weight or dataset. The pin is a content hash, never a mutable tag. (model_zoo/loader.py, R0/R1.) |
The linear PAM pass is the default at every size; the FM-index is opt-in (FM_INDEX_AUTO_ENGAGES) |
It used to auto-engage past 1 Mb, on the grounds that the index build "only amortizes at contig scale". Measured, that is backwards: the default path was 2.7x slower at 1 Mb and 4.6x slower at 8 Mb, and it does not amortize across guides either. The path stays, exact and parity-pinned, reached by asking (use_fm_index=True, or a supplied genome_index=). The result is byte-identical either way. (offtarget/engine.py.) |
The k-mer seed prefilter never engages on its own (SEED_PREFILTER_AUTO_ENGAGES) |
Honest micro-benchmark finding, twice revised. It was calibrated at ~2–4× when selective (k ≥ 5), re-measured neutral, and is now a net cost at every budget: it prunes one native evaluate_anchor call, which is cheaper than its own O(n) pass costs to decide. It stays reachable as scan_sequence(seed=True) and stays parity-tested, because the proven-superset property is exact. (offtarget/_search.py, R2/R411.) |
| Every native kernel keeps a parity-tested pure-Python fallback; the library never requires the crate | prefer_native selects Rust when built; CI runs the off-target engine on both paths. Trades raw speed-when-unbuilt for "installs and passes anywhere; native is a pure bonus." (SPEC_V2.md R2.) |
| Off-target nomination is an OR of two thresholds (CFD ≥ 0.20 or MIT ≥ 0.10), and both scores are recorded per site | Two complementary specificity models catch different failure shapes; recording both (OffTargetSite.mit_score) keeps a MIT-nominated, low-CFD site auditable rather than mysteriously retained. (offtarget/scoring.py.) |
| Ancestry risk is the worst-affected population, never the average; "carrying" means at/above the MAF threshold | Averaging hides risk concentrated in one ancestry — the BCL11A cautionary tale. The carrying threshold is applied identically on the population and haplotype paths, so a trace, sub-threshold frequency cannot inflate the per-ancestry burden. (types/offtarget.py.) |
| The cross-run off-target cache is safety-gated to reference-only, default-scorer searches | A wrong off-target report is a missed danger, so a possibly-stale entry is never served once population / haplotype / patient augmentation is present (the key cannot fully capture that external data). (offtarget/cache.py, R4.) |
| Intervals are recalibrated by split-conformal; probabilities by isotonic regression | Different calibration targets need different tools — a finite-sample coverage guarantee for regression intervals, monotone probability calibration for classification — with empirical_coverage / ECE flagging when each is needed. (scoring/uncertainty.py, R5.) |
| The default backbone is non-commercial, and the license gate enforces it | Nucleotide Transformer v2 (500M) is CC-BY-NC-SA-4.0 — loadable for research, refused for commercial use at load time. Real weights are never vendored. (model_zoo/cards/, R1.) |
| Results, splits, and caches are content-addressed | A published benchmark number cannot be silently edited (each result carries a signature); a split pins both its dataset-content hash and its own membership hash, re-verified on read (SplitIntegrityError on drift). (benchmark/.) |
A ReferenceGenome is thread-safe to share; cohort workers still get their own |
pyfaidx has a shared file position, so concurrent reads on one handle race. The web server's sync handlers run in a threadpool over one shared reference, so each read is guarded by a per-instance lock (covering the read, not the CPU-bound design that follows). The cohort path instead hands each worker its own handle via a reference_factory — safe and parallel I/O. (genome/reference.py.) |
AlleleForge is strictly layered: lower layers know nothing about higher ones. The Designer is the only component that sees the whole pipeline; every domain service is independently testable and usable.
flowchart TB
subgraph I["Interfaces"]
PY["Python library"]
CLI["aforge CLI"]
WEB["Web UI (FastAPI + Next.js)"]
end
subgraph O["Orchestration"]
DES["Designer: variant → routing → candidates → score → outcome → off-target → rank → report"]
end
subgraph D["Domain services"]
VR["Variant resolver<br/>(HGVS, ClinVar)"]
EN["Guide enumerators<br/>(cas9, base, prime)"]
SC["Scoring<br/>(efficiency, outcome, uncertainty)"]
OT["Off-target engine<br/>(population / haplotype)"]
end
subgraph F["Foundations"]
GA["Genome access<br/>(FASTA, FM-index)"]
DR["Data registry<br/>(DVC, gnomAD, ClinVar)"]
MZ["Model zoo<br/>(ckpt hashing)"]
CT["Core types and schemas"]
end
RUST["Rust / PyO3 — aforge_native: BWT off-target search · k-mer hashing · haplotype walking · bulged alignment"]
I --> O --> D --> F
OT -.calls.-> RUST
EN -.calls.-> RUST
sequenceDiagram
autonumber
actor U as User
participant R as Resolver
participant Rt as Router
participant E as Enumerators
participant S as Scorers
participant X as Off-target engine
participant K as Ranker
U->>R: ClinVar / rsID / HGVS / VCF / coords
R->>Rt: normalized Variant and consequence
Rt->>E: eligible chemistries (nuclease / base / prime)
E->>S: candidate guides and pegRNAs
Note over S: efficiency and outcome (calibrated Prediction)
E->>X: spacers and nicks
Note over X: reference, then population, then haplotype, then patient VCF
S->>K: scored candidates
X->>K: ancestry-stratified off-target reports
K-->>U: RankedMenu (Pareto front, provenance, disclaimer)
AlleleForge is built in ordered phases (see SPEC.md, the authoritative build contract). Phases
0–5 establish the spine before any modality or ML code.
| Phase | Component | Status |
|---|---|---|
| 0 | Repo bootstrap, CI, packaging, Rust toolchain | ✅ done |
| 1 | Core domain types & schemas (types/) |
✅ done |
| 2 | Genome access & indexing (genome/) |
✅ done |
| 3 | Data registry & population datasets (data/) |
✅ done |
| 4 | Variant resolver (variant/) |
✅ done |
| 5 | Off-target engine — population & haplotype aware (offtarget/) |
✅ done |
| 6 | Scoring foundations: model zoo, embeddings, uncertainty (scoring/, model_zoo/) |
✅ done |
| 7 | Chemistry: SpCas9 nuclease (enumerate/, scoring/, design/) |
✅ done |
| 8 | Chemistry: base editing — ABE / CBE (enumerate/, scoring/, design/) |
✅ done |
| 9 | Chemistry: prime editing — the flagship (enumerate/, scoring/, design/) |
✅ done |
| 10 | Designer: routing, candidate menu, ranking (design/) |
✅ done |
| 11 | Reporting & oligo output (report/) |
✅ done |
| 12 | CLI (aforge) (cli/) |
✅ done |
| 13 | Web UI & API (web/) |
✅ done |
| 14 | CRISPR-Bench: tasks, frozen splits, metrics, runner, leaderboard (benchmark/) |
✅ done |
| 15 | Docs, runnable examples, release engineering (docs/, examples/) |
✅ done |
All fifteen v0.1.0 phases are complete. Post-v0.1.0 work to "bake" the release toward v1.0 is tracked
in SPEC_V2.md:
| Track | Scope | Status |
|---|---|---|
| R0 | Release hardening: pin real artifact hashes; supply-chain; reproducibility audit | ◐ in progress |
| R1 | Real-weights model integration through the consent-gated model zoo | ◐ in progress |
| R2 | Native bwt/kmer/haplotype kernels wired onto the off-target hot paths |
◐ in progress |
| R3 | External-tool adapters (Cas-OFFinder, VEP, HGVS) behind the registry | ◐ in progress |
| R4 | Scale: whole-genome on-disk FM-index (SA-IS), cohort throughput, cross-run caches | ◐ in progress |
| R5 | Validation, calibration study (ECE on real data), methods preprint | ◐ in progress |
| R6 | v1.0 release criteria | ☐ not started — measured by python scripts/release_readiness.py, whose criteria are named after these phases but are narrower: it reports R2's v1.0 condition as met while the R2 phase above is still in progress |
Landed since v0.1.0. R0 — Dependabot across pip/cargo/actions, a CI pip-audit+cargo audit
job, a CycloneDX SBOM on release, and a scripts/reproduce.py reproducibility audit gated in CI.
R1 — the consent/license/checksum resolution wired for the backbone and every trained scorer through
a shared WeightGate, plus a backbone ONNX export path (export_onnx, dynamic batch/sequence
axes, opset 17) for portable inference (the trained forward pass and the export both stay
real_weights-gated), and each menu's provenance.models now records the card-backed
ModelCheckpoint of every model invoked (deduped, scoped to the eligible chemistries, rendered in
the report footer, and captured by the reproducibility golden). R2 — all three spec
kernels (bwt/kmer/haplotype) are now on their hot paths: a true-linear SA-IS
FM-index build, a native k-mer seed kernel, FM-index seed-and-extend wired into the engine's
reference scan (opt-in, byte-identical to the linear scan), and a native
haplotype-walk kernel that materializes each common haplotype's alternative sequence (~4x, pinned
byte-for-byte to the Python fallback). R3 — the three external-tool adapters are now real behind
recorded-fixture tests: Cas-OFFinder (input-deck builder + legacy/bulge output parser +
injectable-runner cross-check), VEP (region-endpoint predictor with an injectable fetcher, MANE
selection, and (variant, assembly, transcript) caching), and HGVS (HgvsLibraryProjector over
the real hgvs/UTA/SeqRepo stack) — with live network/binary calls factored behind injection points
(live_integration-marked, opt-in, never run in CI). R4 — cohort-scale batch design
(design.design_many) streams a whole VCF/iterable through design with bounded memory (each
menu summarized then released; O(1) with on_result), a resumable JSONL run manifest (a re-run
skips recorded items), per-item failure isolation, and an optional thread-parallel path — fed by the
cyvcf2 fast path (variant.iter_vcf) that streams a VCF into the cohort (one record per concrete
ALT, multi-allelic split, non-PASS/symbolic dropped; injectable reader, CI-tested without htslib);
and content-addressed cross-run caches (alleleforge.cache) that memoize embeddings
(CachedEmbedder.persistent) and the reference off-target scan (OffTargetCache via
search(..., cache=...), safety-gated to the default-scorer reference-only case) to disk so a value
computed in one run is reused by the next; and a persistent, memory-mapped whole-genome FM-index
(genome.GenomeIndex) — driven by R2's native SA-IS so the on-disk build scales to whole
chromosomes, consumed by the engine via search(..., genome_index=...), built once and reused across
runs (parity-tested vs the per-call build; scale-tested on a downsampled chromosome in CI). R5 — the
calibration & generalization machinery: scoring.ConformalCalibrator recalibrates predictive
intervals to a target coverage with the finite-sample split-conformal guarantee (the regression
analog of isotonic), benchmark.generalization_gap quantifies the cross-cell-type generalization
gap (in-context vs held-out cell type, oriented so positive = worse), and
scripts/calibration_study.py regenerates the per-task-ECE + gap + recalibration report from
CRISPR-Bench (the real-data numbers fill in with R1); and the methods-preprint draft
(docs/paper/preprint.md) turns the outline into a full manuscript —
abstract, methods, benchmark design, the weight-free end-to-end results (reference-bias reproduction +
the split-conformal coverage table), and reproducibility — with the accuracy-vs-published numbers
marked [pending R1]; and reproducible SVG figures (alleleforge.viz, a dependency-free hand-rolled
renderer) for the reference-bias reproduction, conformal coverage, per-task ECE, and the generalization
gap — committed under docs/assets/figures/, embedded above and in the preprint, regenerated byte-for-byte
by make figures. The one remaining R0 item is pinning the real
artifact hashes, which requires freezing the published upstream artifacts; the consent gate already
refuses any null-hash fetch by design.
AlleleForge targets Python ≥ 3.11. The core install is deliberately light; heavy scientific, ML, and web stacks live in optional dependency groups so the base package installs fast and CI stays reliable.
# Core library (light: pydantic types, config, model-card parsing — no torch/numpy)
pip install alleleforge # once published to PyPI
# From source: the same extras CI installs, kept in one place
git clone https://github.com/clay-good/alleleforge
cd alleleforge
make install # pip install -e ".[dev,docs,core,cli,web,genome-light]"The from-source line used to be spelled out here as
pip install -e ".[core,genome,variant,cli,ml,dev]", which could not succeed: variant
is hgvs, hgvs requires psycopg2, and psycopg2 publishes Windows wheels only, so
everywhere else pip builds it from source and stops at Error: pg_config executable not found. No CI job installs variant, so nothing noticed. make install is the set the
gate actually runs against — which is what CONTRIBUTING.md already told contributors to
use, for exactly this reason.
| Group | Pulls in | Needed for |
|---|---|---|
core |
polars, pyarrow, numpy | tabular I/O |
genome |
pyfaidx, pysam, cyvcf2, mappy, pyliftover | reference access, indexing (Phase 2) |
variant |
hgvs | c./p. HGVS resolution (Phase 4). Needs PostgreSQL client headers: hgvs requires psycopg2, which has no wheel outside Windows — install libpq-dev (Debian) or libpq (Homebrew) first. Coordinates and genomic g. need none of this and work on every install. |
cli |
typer | the aforge command-line interface (Phase 12) |
web |
fastapi, uvicorn, httpx | the web API + served frontend (Phase 13) |
ml |
torch, transformers, lightning, scikit-learn | real embedding backbones (Phase 6+); the uncertainty core needs none of these |
cas9-rs3 |
lightgbm, sglearn | the real trained Rule Set 3 SpCas9-efficiency model (no torch); see below |
docs |
mkdocs-material, mkdocstrings | documentation site |
dev |
ruff, mypy, pytest, hypothesis, maturin | development |
The performance kernels live in a PyO3 crate (aforge_native) built with
maturin. All three spec kernels are implemented, plus a fourth
(align) that profiling identified as the scan's remaining hot spot — each behind a correct
pure-Python fallback and a byte-identical parity test, and each wired into its hot path:
| Kernel | What it does | Hot path | Parity test | Speedup |
|---|---|---|---|---|
bwt |
FM-index build/count/locate/pam_sites |
reference scan (PAM seed-and-extend) | test_native.py |
genome-scale |
kmer |
exact length-k seed positions |
seed prefilter, now opt-in (scan_sequence(seed=True)) |
test_kmer.py |
~5–7x lookup; scan-level a net cost, so it no longer runs by default — the prefilter's own O(n) pass exceeds what it saves now that the anchor scan is C-level. scripts/native_speedup.py prints the pair; test_the_seed_prefilter_is_opt_in.py carries the numbers. |
haplotype |
apply a haplotype's variant set to a window | haplotype walk (stage 3 materialization) | test_haplotype_kernel.py |
~4x |
align |
best single-base removal within a mismatch budget | the scan's innermost alignment (two calls per PAM anchor) | test_native_align_parity.py |
~43% off a whole scan |
FMIndex.build(prefer_native=True) transparently uses the Rust index when the crate is present; the
k-mer, haplotype and alignment dispatchers do the same. AlleleForge imports and runs cleanly without the crate
(pure-Python mode); build it for the genome-scale path:
pip install maturin
make native # build the wheel, install it, and run the suite against itmake native is what CI's rust job runs, so a local build is checked the same way. It
builds a wheel and installs that, rather than maturin develop, which installs a build
of your working tree into whichever virtualenv is active — convenient until the tree moves
under it, and shared by every checkout using that environment.
alleleforge._native.NATIVE_AVAILABLE reports whether the compiled extension is present, and
alleleforge.genome.native_fm_available() whether the FM-index kernels specifically are built. The
native suffix array is built by SA-IS (sais.rs — Nong–Zhang–Chan induced sorting, O(n)) rather
than the direct sort's O(n² log n), which collapses on the long poly-A / poly-N runs and tandem
repeats real genomes are full of; the unique sentinel keeps the suffix array unique so the result
stays byte-identical to the direct sort — pinned directly (the exposed fm_suffix_array vs the
ground-truth direct sort, over textbook-pathological and fuzz inputs) and end-to-end (FM-index
count/locate parity over low-complexity and random-long inputs).
That linear-time build is what makes the whole-genome index practical:
genome.GenomeIndex.build_genome(reference) persists one content-addressed FM-index per contig (both
strands) to disk and memory-maps it, so a genome index is built once, survives across runs
(a re-run maps the cache instead of rebuilding), and never pins itself in RAM. The off-target engine
takes it directly — search(spacer, pam, reference=ref, genome_index=gi) — anchoring PAMs through the
persistent index instead of rebuilding one per call, with hits identical to the per-call path
(parity-tested) and the memory-mapped query path validated at scale on a downsampled chromosome in CI.
The end-to-end design pipeline is live:
alleleforge.design.design()resolves a variant, routes it to every eligible chemistry, enumerates and scores candidates, runs population-aware off-target, and returns a ranked, explained menu (see the designer section), and reporting & oligo output renders it to cloning-ready oligos, HTML, PDF, JSON, and TSV. The whole pipeline is driven from theaforgeCLI and the web API + browser UI, and the same scorers are graded by CRISPR-Bench. All fifteen build phases are complete; three runnable example notebooks execute in CI, and the release pipeline (PyPI · multi-arch Docker · GitHub Release) is wired and tag-triggered. The snippets below show the lower-level building blocks the designer composes.
from alleleforge.types import DNASequence, Prediction, UncertaintyMethod
seq = DNASequence("ACGTRYN") # validates IUPAC alphabet
print(seq.reverse_complement()) # ambiguity-aware: R↔Y, N↔N → "NRYACGT"
# Every numeric prediction carries a calibrated interval, never a bare float.
p = Prediction(value=0.72, interval=(0.61, 0.83), method=UncertaintyMethod.ENSEMBLE,
in_distribution=True)
print(p.interval_level) # 0.80 by default
print(p.calibrated) # False — the flag is unforgeable
# `calibrated=True` is a guarantee, not a self-report: a direct construction
# asserting it is coerced to False. Only a fitted calibrator can certify it,
# through the single authorized path, and never for an out-of-distribution input.
calibrated = Prediction.calibrated_by(value=0.72, interval=(0.61, 0.83),
method=UncertaintyMethod.CONFORMAL)
print(calibrated.calibrated) # TrueResolve a variant — every input form normalizes to one canonical, left-aligned record:
from alleleforge.variant import resolve, RawTarget
from alleleforge.types import DNASequence
# A raw target sequence with a marked edit — no reference file needed.
rv = resolve(RawTarget(sequence=DNASequence("ACGTAACGTACGT"), position=4, ref="A", alt="G"))
print(rv.variant) # target:4:A>G
print(rv.working_interval) # 0-based half-open analysis window around it
# With a reference genome, indels are left-aligned and the asserted ref is
# validated against the build (a mismatch is a hard error — likely wrong build):
# resolve("chr2:g.5226001del", reference=hg38, dbsnp=dbsnp_db)
# resolve("VCV000012345", clinvar=clinvar_db) # ClinVar accession → VariantInspect the data registry — every external dataset is versioned and license-aware:
from alleleforge.data import DEFAULT_REGISTRY
print(DEFAULT_REGISTRY.names) # ('1000g', 'clinvar', 'dbsnp', 'encode', ...)
clinvar = DEFAULT_REGISTRY.get("clinvar")
print(clinvar.version, clinvar.license) # 2024-05 public-domain (NCBI)
# Non-redistributable sources are never vendored; downloads are consent-gated
# and checksum-verified. See docs/data.md for the full provenance table.The same journey from the aforge CLI (pip install "alleleforge[cli]"):
# Variant → ranked, safety-annotated menu, rendered as an interactive HTML report.
# `--gnomad` is what makes the off-target scan population-aware; `--populations` only
# names the ancestries to stratify by, so without a sites file the scan is
# reference-only and the ancestry breakdown comes back empty (the command says so).
aforge design 'chr11:5227002:A>T' --reference-fasta hg38.fa \
--intent correct --gnomad gnomad.sites.tsv.gz --populations afr,eur,eas \
--cell-context HEK293T --format html --out report.html
# Coordinates, because that is what this surface can resolve on its own. A ClinVar
# accession or an rsID needs a lookup database, and neither the CLI nor the web API has
# a way to be given one — the refusal says so and names this form.
# Standalone population/haplotype-aware off-target for a spacer. Every engine knob is
# tunable: the bulge budget, the CFD/MIT reporting thresholds, and the carrying MAF.
# Pass `--on-target` so the guide's own locus is not counted against its specificity.
aforge offtarget GACGGAGGCTAAGCGTCGCAA --reference-fasta hg38.fa --pam NGG --json \
--gnomad gnomad.sites.tsv.gz --haplotypes panel.tsv.gz --patient-vcf sample.vcf.gz \
--populations afr,eur,eas --on-target 'chr2:28-48(+)' \
--dna-bulges 1 --rna-bulges 1 --cfd-threshold 0.20 --mit-threshold 0.10 --maf 0.001
# Normalize any input form and show its class (debugging aid)
aforge resolve 'chr2:100:A>G' --jsonPhases 2–4 implement everything from an input to a validated, annotated variant with its genomic context — the foundation every modality plugs into.
flowchart LR
subgraph IN["Accepted inputs"]
A1["ClinVar accession"]
A2["dbSNP rsID"]
A3["HGVS g./c./p."]
A4["VCF record"]
A5["raw coordinates"]
A6["raw target seq"]
end
R["resolve()"]
subgraph NORM["Normalize"]
N1["left-align + trim<br/>(bcftools-norm)"]
N2["validate ref vs build<br/>(hard error on mismatch)"]
end
OUT["ResolvedVariant<br/>variant · working interval ·<br/>consequence · T2T recommendation"]
A1 & A2 & A3 & A4 & A5 & A6 --> R --> NORM --> OUT
R -. ClinVar/dbSNP/HGVS lookups .- DATA["Data registry<br/>(versioned, license-aware)"]
NORM -. fetch + flag ambiguous loci .- GEN["Genome access<br/>(FASTA, FM-index, liftover)"]
Coordinate convention cheat-sheet. Internals are uniformly 0-based half-open; only I/O boundaries are 1-based. Every parser converts on read.
| Surface | System | Converted by |
|---|---|---|
AlleleForge internals (GenomicInterval, Variant.pos) |
0-based half-open | — (canonical) |
| ClinVar / gnomAD / dbSNP VCF | 1-based | pos − 1 on read |
| GENCODE GTF | 1-based inclusive | [start − 1, end) on read |
| ENCODE bedGraph | 0-based half-open | unchanged |
HGVS (g.) |
1-based | hgvs_adapter on read |
Human-readable reports (HTML/PDF/TSV locus) |
0-based half-open | — (stated in the report's own provenance block; in the TSV, in the leading # note lines) |
JSON export (locus) |
0-based half-open | — (the report's own coordinate_system field, so a machine consumer need not read prose) |
Dataset provenance (pinned, versioned, citation-stamped — full table in docs/data.md):
| Dataset | Version | License | Role |
|---|---|---|---|
| ClinVar | 2024-05 | Public domain | accession → variant + significance |
| gnomAD | v4.1 | CC0-1.0 | per-population allele frequencies |
| 1000 Genomes | phase 3 high-cov | Public (IGSR) | phased common haplotypes |
| HGDP | gnomAD v3.1 | CC0-1.0 | ancestry breadth |
| dbSNP | b156 | Public domain | rsID ↔ locus |
| GENCODE | v47 | Open | gene models / transcripts |
| ENCODE | 2024 | Open | chromatin tracks |
AlleleForge's safety core, and its clearest point of novelty: off-target nomination that is
reference-, population-, and haplotype-aware for every chemistry, behind one search() call that
returns an ancestry-stratified report. Reference-only off-target analysis has a known blind spot —
a minor allele can create a de novo PAM the reference never shows — and because allele frequencies
differ by ancestry, that blind spot concentrates risk in under-represented populations.
flowchart TB
SP["spacer + PAM"] --> S1
subgraph ENG["search() — five stages"]
direction TB
S1["1 · Reference scan<br/>PAM-anchored · ≤4 mismatch · ≤1 DNA + ≤1 RNA bulge · both strands<br/>FM-index seed-and-extend available opt-in"]
S2["2 · Population augmentation<br/>gnomAD alt-allele re-scan → de-novo PAMs / strengthened seed sites"]
S3["3 · Haplotype walk<br/>common 1000G / HGDP haplotypes (variant combinations)<br/>native haplotype kernel materializes each alt sequence (~4x)"]
S4["4 · Patient VCF (optional)<br/>personalize to one genome"]
S5["5 · Score · threshold · de-dup · stratify"]
S1 --> S5
S2 --> S5
S3 --> S5
S4 --> S5
end
S5 --> R["OffTargetReport<br/>ancestry-stratified · every site tagged<br/>reference / population / patient + causal allele + freq"]
Every site records where it came from — the reference, a population variant (which allele, which
populations, at what frequency), or a patient's VCF — so a nomination can be audited, not trusted
blindly. The report's worst-case is computed against the worst-affected ancestry, never the
average, and it rolls every site into one aggregate genome-wide specificity score
(specificity_score(), see the cheat-sheet below).
A population is annotated as carrying a site only at or above the MAF safety threshold — applied
identically on the population-variant and haplotype paths, so the per-ancestry stratification can never
attribute a site's burden to a population that merely shows a trace, sub-threshold frequency. The
populations and ancestries provenance on each site are the same carrying set, by construction.
Note
k-mer seed acceleration (R2). The scan carries an optional, proven-equivalent k-mer
seed-and-extend prefilter (native Rust kernel + pure-Python fallback): by the pigeonhole bound, any
in-budget alignment shares an exact length-k seed with the spacer, so anchors whose window contains
no seed can be skipped without ever dropping a hit (an exhaustive randomized test pins seeded ≡
brute-force). It auto-engages only when the seed is selective (k ≥ 5, i.e. high-stringency / low
edit-budget scans) and is a transparent no-op at the default ≤4-mismatch+bulge budget, where the FM-index
remains the genome-scale path.
Its scan-level payoff is currently ~1x, and that is worth stating plainly. The prefilter once measured
~2–4x on a high-stringency scan; since then the per-anchor work it prunes got roughly 50x cheaper (see the
off-target scan entries in CHANGELOG.md), so its own O(n) cost — building seed
positions and the covered-index prefix sum — now cancels the saving. Re-measured across six
mismatch/bulge configurations with repeats: 0.94–1.12x, i.e. neutral within noise, with hit sets
identical in every case. The kernel's own lookup is still ~5–7x native-over-Python; what changed is
what there was left to prune. The prefilter stays because it is exact and free, not because it is
currently fast. See SPEC_V2.md R2 and
scripts/native_speedup.py.
Note
Every kernel's speedup is re-measurable, not just quoted.
scripts/native_speedup.py times all six functions the crate exposes
plus the contig fold, and a test fails if the crate gains one the script does not cover. On this
machine: bulged alignment ~10x native, per-anchor evaluation ~9x native, and the fold to the
index alphabet ~13x (clean 2 Mb contig) to ~19x (one non-ACGTN base) after moving from a
per-base loop to str.translate. Wall-clock is hardware-dependent — run the script rather than
trusting these numbers.
Note
FM-index seed-and-extend on the reference scan (R2, landed). Stage 1 now anchors PAMs through a
content-addressed FM-index (search(..., use_fm_index=...)): each concrete PAM is located in the
index (the PAM is the seed) and only those anchors are extended by the shared alignment, replacing
the linear O(n) PAM pass. It returns byte-identical hits to the brute-force scan — pinned by a
randomized parity test at both the scan_sequence and search levels.
It is opt-in, and was not always: it engaged itself past 1 Mb until that threshold was measured
rather than reasoned about. On this machine the default path was 2.7x slower at 1 Mb, 2.7x at 2 Mb
and 4.6x at 8 Mb — diverging with size rather than converging, and no better across five guides
sharing one contig. scripts/native_speedup.py had been printing SLOWER for the pair all along.
Ask for it with use_fm_index=True or a prebuilt genome_index=; the hits are identical either way,
so the only thing at stake is time.
The canonical cautionary tale is the BCL11A enhancer variant rs114518452 (Cancellieri & Pinello,
Nat Genet 2023). AlleleForge reproduces it as an integration test: a reference-only scan returns
zero sites, while the population-aware scan nominates the high-CFD off-target the minor allele
creates — ancestry-stratified, with its African-ancestry-enriched frequency recorded.
from alleleforge.offtarget import search
from alleleforge.types.guide import PAM
report = search(spacer, PAM(pattern="NGG"), reference=hg38, gnomad=gnomad_db)
for site in report.sites:
print(site.origin, round(site.score, 2), site.causal_allele, site.populations)
worst = report.worst_ancestry() # ('afr', 1.0) — flagged, not averaged away
spec = report.specificity_score() # aggregate genome-wide specificity in (0,1], 1.0 = clean
Every figure in this README is regenerated byte-for-byte from the weight-free,
deterministic pipeline by python scripts/figures.py — committed SVGs, no plotting
dependency (the same hand-rolled-renderer discipline as the PDF report).
| Score | Source | Status in AlleleForge |
|---|---|---|
| MIT / Hsu | Hsu et al., Nat Biotechnol 2013 | Exact — published 20-position weight table |
| CFD | Doench et al., Nat Biotechnol 2016 | Exact — the published Doench 2016 matrix is the default (vendored, cross-verified byte-for-byte against CRISPOR and CRISPRitz); a transparent seed-tolerance approximation stays available via CfdScorer(approximate=True) |
| CFD-Cas12a | analog | Seed at the PAM-proximal 5' end, TTTV PAM |
Those score one site. The report also rolls every site into one aggregate genome-wide specificity
score — report.specificity_score(), the CFD-scale analog of the Hsu 2013 / MIT guide score
100/(100+Σ), i.e. 1/(1 + Σ site scores) ∈ (0, 1], 1.0 for a guide with no off-targets and
decreasing as the total burden grows. It is the single number every design tool headlines, and unlike the
worst-case it distinguishes two guides with the same worst site but a different number of off-targets.
It surfaces on every output surface that summarizes off-target: the HTML/PDF report and the
CandidateReport.offtarget_specificity export field, the standalone aforge offtarget command and its
POST /api/offtarget web equivalent (both alongside the site count and worst-case score), and the cohort
batch summary (best_specificity), so triage can rank by total burden, not just the single worst site.
Specificity and the worst-case are both frequency-blind: a 0.1%-MAF population hit and a universal
reference hit of the same raw score are identical in them. When any site's presence in a genome is
probabilistic, the same surfaces also carry expected_burden — each site's score weighted by the
probability a genome actually carries it — which weights those two a thousandfold apart. It appears only
when it says something the other two cannot: with reference sites alone it is just the unweighted score sum.
All three site scores sit behind one swappable OffTargetScorer protocol, so a Phase 6 ML scorer drops in
without touching the engine. Reporting thresholds default to CFD ≥ 0.20 or MIT ≥ 0.10 — an OR, so
a site can be nominated on its MIT score even when its CFD is sub-threshold. So that a nomination stays
auditable, every site records both: the primary score (under score_method) and the companion
mit_score (OffTargetSite.mit_score, None when MIT is undefined — a bulged or non-20-nt alignment).
The MIT score that retained a low-CFD site is therefore visible in the serialized report, never silently
dropped.
The genome-scale search is the FM-index seed-and-extend path (native Rust
bwtkernel when built, a correct pure-Python FM-index otherwise — byte-identical, pinned by parity tests; CI never blocks on the native build). It is wired into the engine's reference scan, opt-in (seeFM_INDEX_AUTO_ENGAGES).
AlleleForge is independent of external tools but integrates them at the seams, each behind a
swappable interface so its absence degrades gracefully and its presence adds a cross-check or a
richer annotation. Every adapter is tested against recorded fixtures; only the live network/binary
call is opt-in (live_integration-marked, never run in CI).
| Adapter | Role | Pure (CI-tested) | Live (opt-in) |
|---|---|---|---|
| Cas-OFFinder | off-target cross-check vs. the native engine | input-deck builder, legacy/bulge output parser, disagreements() |
the binary subprocess (injectable runner) |
| VEP (Ensembl REST) | molecular consequence, stated in the menu rationale (routing itself is by variant class and intent, not consequence) | parse_vep_response (MANE/canonical selection, SO term → impact), (variant, assembly, transcript) cache |
the region-endpoint GET (injectable fetcher, consent-gated — see below) |
HGVS (hgvs/UTA/SeqRepo) |
c./p. ⇄ g. projection |
dependency-free g. parser; import-guarded HgvsLibraryProjector |
AssemblyMapper.c_to_g against UTA |
Disagreements are surfaced as flags, never hidden: a Cas-OFFinder locus the native engine misses (or vice versa) is reported, not silently dropped.
Two kinds of consent, and they are not interchangeable. Downloading an artifact and disclosing your data are different acts, so AlleleForge asks for them separately:
| Governs | How to permit it | |
|---|---|---|
| Artifact download | fetching a dataset, checkpoint or reference genome into your cache | consent=True at the call, or allow_network = true in your config / ALLELEFORGE_ALLOW_NETWORK=1 for the whole process |
| Variant disclosure | sending a variant to the Ensembl VEP REST API | VepRestPredictor(consent=True) — asked separately, and never satisfied by allow_network |
A VEP lookup sends the chromosome, position and both alleles off your machine, and AlleleForge accepts
patient VCFs as input. Agreeing to download a reference genome is not agreeing to that. Injecting your
own fetcher needs no consent flag — you supplied the transport and know where it goes, which is also
how CI replays recorded responses with no network at all. With neither form of consent, nothing is
downloaded and nothing is sent.
Before any chemistry-specific predictor, AlleleForge establishes the reusable ML substrate: a
license-gated model zoo, a swappable embedding backbone, and the calibrated-uncertainty
machinery that realizes the honest-uncertainty principle. The whole substrate is pure stdlib in its
core path — no numpy or torch — so it runs in CI on a weight-free stub embedder; real 500M-parameter
backbones are gated behind the real_weights marker.
flowchart LR
SEQ["DNA sequence"] --> EMB["SequenceEmbedder<br/>(NT v2 · Caduceus · Evo 2 · Stub)"]
EMB --> CACHE["embedding cache<br/>(by sequence hash)"]
EMB --> OOD["OODDetector<br/>distance vs training reference"]
CACHE --> MODEL["scorer / ensemble"]
MODEL --> U{"uncertainty"}
U -->|N=5 default| ENS["deep ensemble<br/>mean ± z·σ (disagreement)"]
U -->|fallback| EV["evidential<br/>aleatoric + epistemic"]
U -->|if quantiles| QT["quantile interval"]
ENS & EV & QT --> CAL["isotonic calibration<br/>(reduces ECE)"]
OOD --> CAL
CAL --> PRED["Prediction[float]<br/>value · 80% interval · method ·<br/>in_distribution · calibrated"]
No bare floats. Every scorer returns a Prediction, never a number; ensure_prediction is the
runtime guard at the orchestration seam. No undocumented models. Every checkpoint loads through the
model zoo, which refuses a missing card, a license that forbids the use, or an unverifiable hash, and
surfaces a ModelCheckpoint into result provenance.
Consent-gated real weights (R1). Every trained model — the sequence backbone and the
per-chemistry adapters (cas9 efficiency/outcome, base-edit outcome, prime efficiency) — resolves its
weights through one shared gate, model_zoo.loader.WeightGate, not a bare from_pretrained:
resolve_weights() runs the license gate (the default Nucleotide Transformer v2 is CC-BY-NC-SA
and the trained adapters are research-only — all refused for commercial use), requires explicit
consent before any download, checksum-verifies a pinned artifact, and records the resolved
ModelCheckpoint for provenance (e.g. EnsembleEfficiencyScorer.backbone_checkpoint()). The whole
consent/license/checksum flow is exercised in CI with an injected downloader — no network, no torch;
only the tensor load / forward pass itself stays behind the real_weights marker. Every model ships a
bundled, license-gated card. Each menu's provenance.models records the card-backed ModelCheckpoint
of every model invoked — deduped, scoped to the chemistries that ran, and rendered in the HTML/PDF
report footer — so a result names the exact models that produced it. The checkpoint carries the card's
known_failure_modes alongside its name, version, license, and citation, so the provenance is
self-contained for safety audit: a consumer can check a design against what each model is documented
to get wrong without re-opening the cards. See SPEC_V2.md R1.
| Method | Role | Interval |
|---|---|---|
| Deep ensemble (N=5) | default | mean ± z·σ from member disagreement — widens on OOD |
| Evidential (NIG) | single-model fallback | splits aleatoric (data) vs epistemic (model) variance |
| Quantile | when the model emits quantiles | read off the (1±level)/2 quantiles |
| Isotonic calibration | post-hoc, recalibrates probabilities | PAV fit; expected_calibration_error quantifies the gain |
| Conformal recalibration | post-hoc, recalibrates intervals | split-conformal width scale to a target coverage (finite-sample guarantee); empirical_coverage flags when it's needed |
from alleleforge.scoring import DeepEnsemble, ensemble_prediction, OODDetector, StubEmbedder
ens = DeepEnsemble([m1, m2, m3, m4, m5]) # five members
emb = StubEmbedder().embed(["GACCATGCAACCTTGAACGT"])[0] # NT v2 in production
ood = OODDetector(training_reference) # embedding-space density
pred = ensemble_prediction(ens.predict(features), in_distribution=ood.is_in_distribution(emb))
print(pred.value, pred.interval, pred.method, pred.in_distribution) # honest by constructionThe most mature chemistry, and the right one to prove the full vertical slice end to end. From a
resolved variant, design_cas9 enumerates guides, scores efficiency and outcome with calibrated
uncertainty, runs the population-aware off-target engine, and returns ranked candidates.
flowchart LR
V["ResolvedVariant<br/>+ intent"] --> EN["enumerate_cas9<br/>PAM-anchored · strand-aware ·<br/>cut 3 bp 5' of PAM · actionable window"]
EN --> EF["efficiency<br/>RS3 baseline / deep ensemble<br/>(80% interval + OOD)"]
EN --> OUT["outcome<br/>microhomology / MMEJ +<br/>1-bp insertion spectrum"]
EN --> OT["off-target<br/>(Phase 5 engine,<br/>ancestry-stratified)"]
EF & OUT & OT --> C["DesignCandidate[]<br/>ranked: efficiency then safety"]
EN -.precise intent.-> HDR["HDR donor template<br/>+ PAM-blocking silent mutation"]
HDR --> C
Defaults & decisions. Primary PAM NGG; NG (SpCas9-NG) and NRN/NYN (SpRY) are emitted only
when no NGG guide is actionable and opted in. Cut site 3 bp 5' of the PAM. The actionable window
is tight around the edit for precise intents (HDR efficiency falls off with cut-to-edit distance) and
the whole working interval for a knock-out, which marks frameshift outcomes as intended.
Note
A precise nuclease candidate is a pair, and is shipped as one. A double-strand break alone
corrects nothing — NHEJ repairs it into indels — so a candidate offered for a correction carries the
HDR donor that makes the edit, complete with the PAM-blocking silent mutation that stops the
repaired allele being re-cut. It is labeled the whole way down: hdr-donor:recut-blocked /
:recut-not-blocked / :none and outcome-is-nhej-spectrum on the candidate (that last one because
the attached distribution is the byproduct spectrum, not the correction — such a candidate scores
0 on cleanliness, which is the honest number rather than an invented HDR rate); the donor named on
the reagent line; and the donor emitted as an orderable single-stranded template
(kind="hdr-donor-ssodn") beside the sgRNA duplex, with ordering hazards promoted to the top —
longer than a vendor synthesizes as one oligo, or a product still cuttable by its own guide. A donor
whose homology arm would reach an assembly-gap N is refused, not built: its bases are written
into the genome permanently. An arm that merely runs past a contig end is shortened to the sequence
the reference actually provides.
| Axis | Default (CI, weight-free) | Trained alternative (model zoo) |
|---|---|---|
| Efficiency | RS3-style feature heuristic + backbone deep ensemble | Rule Set 3 (real, wired — cas9-rs3); fine-tuned NT v2 ensemble (ml) |
| Outcome | microhomology/MMEJ + 1-bp insertion model | inDelphi (default) · Lindel · X-CRISP + agreement |
| Off-target | Phase 5 engine (pure-Python fallback) | Phase 5 engine (Rust FM-index) |
Note
Heuristic vs. trained, stated honestly. The weight-free defaults that run in CI are heuristic
baselines (method=HEURISTIC, calibrated=False) — transparent feature models, not the published
networks. The first real trained model is now wired: TrainedRuleSet3Scorer
resolves the published Rule Set 3 LightGBM model through the consent-gated, checksum-verified model
zoo (as a version-independent text booster) and reproduces upstream rs3.predict_seq bit-for-bit
(parity-tested). It is opt-in (pip install "alleleforge[cas9-rs3]", no torch) and gated behind the
real_weights marker so CI stays weight-free. The trained prime/base-editing networks remain
heuristic baselines for now; see specs/model-integration.md for the
roadmap.
Every efficiency score carries an 80% interval and an OOD flag; every outcome is a normalized distribution over indel alleles; every candidate carries an ancestry-stratified off-target report — so a ranked menu is honest about what it does and does not know.
Base editors install a single transition (ABE: A·T→G·C; CBE: C·G→T·A) without a double-strand break, within a narrow activity window. The hard part is the window outcome: of the editable bases in the window, which get edited — and what bystanders ride along. AlleleForge enumerates every sgRNA placing the target base in-window per editor, predicts the window-allele distribution, and ranks by the probability of the exact intended allele while minimizing bystander burden.
flowchart LR
V["ResolvedVariant<br/>(transition SNV)"] --> EL{"editor eligible?<br/>ABE: A·T→G·C<br/>CBE: C·G→T·A"}
EL --> EN["enumerate_base_edits<br/>target base in window 4–8 ·<br/>strand-aware · bystanders flagged"]
EN --> WO["window outcome<br/>per-position p(edit) × motif →<br/>2ᵏ allele distribution"]
WO --> M["p_intended_exact<br/>+ bystander_burden"]
EN --> OT["off-target<br/>(Phase 5, ancestry-stratified)"]
M & OT --> C["DesignCandidate[]<br/>ranked: clean-edit then bystander<br/>cleanest = recommended"]
Declarative editor registry. ABE8e, CBE4max, and evoCDA1 ship as data; adding an editor (deaminase, chemistry, window, PAM, motif preference) is a one-descriptor change, not code.
| Editor | Deaminase | Edit | Window | Motif preference |
|---|---|---|---|---|
| ABE8e | TadA-8e | A→G | 4–8 | none (broad) |
| CBE4max | APOBEC1 | C→T | 4–8 | TC (prefers 5′ T) |
| evoCDA1 | evoCDA1 | C→T | 2–10 | none (broad window) |
Every candidate carries the tradeoff explicitly — the bystander-present:N / clean flag, the full
window-allele distribution, an ancestry-stratified off-target report, and a calibrated
bystander_burden (the expected number of bystander edits, with an 80% interval) persisted as a
structured field on the candidate. The burden the ranking minimizes is therefore exportable, not just
printable: it rides through the JSON/TSV/Parquet exports (a bystander_burden column), the HTML/PDF
reports, and the cohort batch summary (best_bystander_burden), alongside the p_intended_exact it is
ranked against. The recommendation is the cleanest editor/guide combination, not just the first one found.
Prime editing is the chemistry where AlleleForge contributes the most. PRIDICT2.0 is SOTA for efficiency but has no variant front-end and no off-target module; PrimeDesign/PrimeVar give ClinVar-to-pegRNA but only rule-based scoring and reference-only off-target; CRISPRme does population off-target but designs no pegRNAs. AlleleForge stitches all four axes together and fills the seams.
flowchart LR
V["ResolvedVariant + intent"] --> EN["enumerate_prime"]
EN --> G["pegRNA geometry:<br/>nick · PBS 8-17 · RTT 7-34 (edit + >=5 homology) ·<br/>tevopreQ1 epegRNA · PE3/PE3b nick"]
G --> EF["efficiency<br/>PRIDICT2.0-style + ePRIDICT<br/>(80% interval, OOD flag)"]
G --> OUT["outcome<br/>intended vs. byproduct<br/>(scaffold / partial RTT / indel)"]
G --> OT["off-target on BOTH nicks<br/>pegRNA nick + ngRNA nick<br/>merged, ancestry-stratified"]
EF --> C["DesignCandidate[] (ranked)"]
OUT --> C
OT --> C
| Axis | PRIDICT2.0 | PrimeDesign / PrimeVar | CRISPRme | AlleleForge |
|---|---|---|---|---|
| Therapeutic variant front-end | no | yes | no | yes |
| ML efficiency + calibrated uncertainty | yes | no | no | yes |
| Outcome / byproduct prediction | partial | no | no | yes |
| Population-aware off-target | no | no | yes | yes |
Honest by construction. PRIDICT2.0 is trained on HEK293T/K562; any other cell context flags the
efficiency prediction out-of-distribution and raises an ood flag rather than hiding it. The
off-target engine runs on the pegRNA nick and the ngRNA nick, merging into one ancestry-stratified
report. The PE3b nicking guide is preferred when a seed-disrupting ngRNA exists (it nicks only the
edited strand, suppressing indels). Every PE3 candidate states where its second nick is — a signed
nick-distance:+62nt flag and PE3 (+62 nt nick) on the reagent line — because that distance is the one
parameter PE3 design turns on, and two opposite-strand nicks close together are a staggered double-strand
break, the outcome prime editing is chosen to avoid. A nick inside a conservative floor is annotated
close-nick. It is shown, not scored: turning nick distance into a ranking term needs a byproduct
model calibrated against real PE3 data, which this project does not have, and a fabricated weight would
make the composite look better informed than it is. Edit-class scope: the enumeration templates a variable-length
RTT, so the whole small-edit repertoire is designed — substitutions, MNVs, short insertions, short
deletions, and delins. The RT template reads 5' homology + the desired allele + 3' homology, so a
deleted span costs no template length (a 44 bp deletion is as cheap to write as a 1 bp one) while an
inserted one pays for every base. Two budgets bind, and route() mirrors both so it never advertises
what enumeration cannot produce: the replaced reference span must fit PRIME_MAX_EDIT (44 bp), and the
allele the RTT must write must fit PRIME_MAX_TEMPLATED_EDIT (29 bp = the RTT ceiling minus the
minimum 3' homology). Because the reagent is enumerated against the genome the patient actually carries,
a protospacer that spans a length-changing edit is placed on the reference footprint its bases come
from — wider for a deletion, narrower for an insertion — rather than a locus of convenience. See the
canonical journey end to end in examples/01_clinvar_to_design.ipynb.
Note
Default vs. real PRIDICT2.0. The built-in PridictScorer is a transparent heuristic
geometry baseline (method=HEURISTIC) — it is not the trained network. The real PRIDICT2.0 is now
wired as an opt-in, sequence-level engine: PridictEngineAdapter
shells out to a local PRIDICT2 checkout (Mathis et al. 2024, MIT), runs its attention-RNN ensemble +
DeepCas9 pipeline, and returns ranked pegRNA designs each carrying a calibrated efficiency. Because
PRIDICT2 ships as a Git repo (not a PyPI package) and pins TensorFlow 2.13 + PyTorch 2.0.1 (which
conflict with AlleleForge's own deps), the adapter invokes PRIDICT2 in its own environment — point
it there with ALLELEFORGE_PRIDICT2_REPO / ALLELEFORGE_PRIDICT2_PYTHON. It is gated behind the
real_weights marker and parity-tested against captured PRIDICT2 output, so CI stays weight-free.
What that means for a menu: PRIDICT2 designs and scores its own pegRNAs and exposes no
"score this externally-supplied pegRNA" entry point, so the engine is a parallel path, not a scorer
the designer can call. A design() menu's prime efficiency is therefore the heuristic baseline today,
whatever weights are available — design() accepts a prime_efficiency_scorer override, but no trained
per-pegRNA prime scorer ships to pass it. Closing that needs the per-pegRNA parity scorer tracked as
(P2) in specs/pridict2-integration.md.
The keystone that realizes the variant-first promise end to end. design() takes any input form, decides
which chemistries can biologically make the edit, generates and scores candidates from each, ranks them on
one footing, and returns an explained RankedMenu with a Pareto front and full provenance.
flowchart LR
V["variant input<br/>(any form)"] --> R["resolve()"]
R --> RT["route()<br/>transparent rules:<br/>variant class + intent"]
RT --> ABE["base ABE/CBE<br/>(transition SNV)"]
RT --> PE["prime<br/>(precise small edit)"]
RT --> NUC["nuclease<br/>(disruption intent)"]
ABE & PE & NUC --> RANK["rank_candidates()<br/>weighted sum + Pareto front"]
RANK --> M["RankedMenu<br/>ordered · rationale ·<br/>Pareto front · provenance"]
Routing is transparent and inspectable. Each rule is a chemistry paired with a one-line biological
rationale and a pure (resolved, intent) predicate. Adding or relaxing a rule is a one-line data change,
and route() explains every verdict — kept and dropped.
| Chemistry | Eligible when | Biological reason |
|---|---|---|
| Base editing (ABE) | transition SNV, required change A:T→G:C |
one in-window transition, no double-strand break — the cleanest fix |
| Base editing (CBE) | transition SNV, required change G:C→A:T |
same, complementary transition |
| Prime editing | any precise small edit (SNV, MNV, insertion, deletion, delins), non-disruptive intent, replaced span ≤ 44 bp and templated allele ≤ 29 bp | writes the edit from a variable-length RTT template with no break; the two bounds are exactly what the RT template can carry, so routing never advertises an edit enumeration cannot produce |
| SpCas9 nuclease | disruption (knock-out) intent; or a precise edit no break-free chemistry can reach | a break repaired by NHEJ yields frameshifting indels — the knock-out route. For a precise edit it needs an HDR donor and is strictly worse (inefficient, S/G2-restricted, NHEJ indels as the majority product), so it is the last resort: offered only when neither base nor prime editing can reach the edit, e.g. a deletion longer than any RT template can write back. Such a candidate carries its donor and is flagged outcome-is-nhej-spectrum. |
Ranking puts every chemistry on one footing. Candidates are projected onto four shared, higher-is-better objectives and ordered by a transparent weighted sum, with the Pareto front always exposed for users who weight differently.
| Objective | Definition | Default weight |
|---|---|---|
| Efficiency | uncertainty-discounted on-target efficiency (point estimate in-distribution, lower interval bound out-of-distribution) | 0.35 |
| Cleanliness | probability mass on the intended allele | 0.30 |
| Safety | 1 − off-target score of the worst-affected ancestry |
0.30 |
| Simplicity | reagent simplicity (single sgRNA > pegRNA + nick + motif) | 0.05 |
The efficiency term is uncertainty-aware: an out-of-distribution prediction is ranked on its lower interval bound, so a confident-looking OOD candidate cannot outrank an otherwise-equal in-distribution one, and each candidate's interval and OOD status appear in its score breakdown. The safety term uses the worst-affected ancestry, never the average, so a guide safe on average but dangerous in one population is correctly down-ranked. The designer degrades gracefully: an unavailable model, a failing enumeration, or a chemistry that finds nothing is recorded with its reason in the menu rationale while the rest of the menu still returns.
The menu leads with what is known about the target, not with chemistry. When the variant came from ClinVar, or an effect predictor annotated it, the rationale states so before anything else — because a menu is only meaningful once the reader knows what is being edited:
Predicted effect: missense variant (moderate impact) in HBB, p.Glu7Val on ENST00000335295
ClinVar: pathogenic (criteria provided, multiple submitters)
The review status is carried alongside the class deliberately: Pathogenic, no assertion criteria provided and Pathogenic, reviewed by expert panel are the same class and very different evidence. When the intent and what is known pull in different directions — correcting a variant ClinVar calls benign, correcting a variant of uncertain significance, correcting a variant with only modifier impact, or installing a pathogenic allele (a disease model, not a therapy) — the menu says so plainly.
These annotate; they never refuse. Correcting a benign variant can be entirely right — a research control, a reclassification the database has not caught up with — and a variant with no predicted protein consequence can still be a splice or regulatory target. The job is to make sure you are not doing it by accident. A design whose intent and evidence agree gets no caution at all, so the caution means something when it appears.
from alleleforge.design import design, eligible_chemistries
from alleleforge.types.edit import EditIntent
# Which chemistries can even make this edit?
print(eligible_chemistries(resolved, EditIntent.CORRECT)) # [BASE_CBE, PRIME]
# One call: resolve → route → enumerate → score → off-target → rank.
menu = design("VCV000012345", reference=hg38, clinvar=clinvar_db,
intent=EditIntent.CORRECT, populations=["afr", "eur", "eas"])
best = menu.best
print(best.chemistry, best.rationale) # includes the score breakdown
print(menu.pareto_front) # trade-off-optimal candidates
print(menu.provenance.seed) # reproducible to the byte
print([m.name for m in menu.provenance.models])
# every model invoked. By default these are the transparent baselines — e.g.
# ['be-dict-baseline', 'pridict2-baseline', 'prime-outcome-baseline'] — and the
# `-baseline` suffix is load-bearing: it is how the artifact says the number did
# not come from the published trained model. The trained ones (`be-dict`,
# `pridict2`/`deepprime`, `rule-set-3`, `lindel`) are opt-in and appear here only
# when you ask for them.design_many is the cohort multiplier over design, built so a whole VCF is no different from three
rows: it streams the input (bounded memory — each menu is summarized then released), is
resumable (a JSONL run manifest a re-run skips past), and isolates per-item failures (an
unresolvable variant is recorded, not fatal). variant.iter_vcf is the cyvcf2 fast path that
produces the lazy stream straight from a VCF.
from alleleforge.design import design_many
from alleleforge.variant import iter_vcf
report = design_many(
iter_vcf("cohort.vcf.gz"), # streams a VCF: one record per concrete ALT, multi-allelic split,
# symbolic/spanning alleles skipped, non-PASS dropped by default
reference=hg38, intent=EditIntent.INSTALL,
manifest_path="run.jsonl", # resume point: a re-run skips items already recorded
output_dir="menus/", # durable per-sample menu JSON (survives the run)
on_result=print, # stream results → O(1) memory in cohort size
)
print(report.succeeded, report.failed, report.skipped)iter_vcf also accepts any iterable duck-typed to the cyvcf2 Variant shape (a region query, a
generator, a test list), so the whole pipeline is testable without the native htslib dependency; a
path open names the genome extra in a clear error when cyvcf2 is absent.
The same cohort run is one command from the aforge CLI —
the batch subcommand auto-detects a VCF (cyvcf2 fast path) vs a one-variant-per-line list:
# Whole-VCF cohort → resumable run, durable per-sample menus, a per-item TSV summary
aforge batch cohort.vcf.gz --reference-fasta hg38.fa --intent correct \
--gnomad gnomad.sites.tsv.gz --populations afr,eur,eas \
--manifest run.jsonl --output-dir menus/ --summary-tsv summary.tsv --max-workers 8
# Summary columns: best_chemistry · best_efficiency · best_bystander_burden · worst_offtarget · best_specificity · n_candidates
# `worst_offtarget` is empty when the search was skipped or an item produced no candidates — that is "not measured", not "clean".…and over HTTP from the web API: POST /api/batch takes a JSON
variant list and returns the same per-item summaries with provenance — cohort design reaches all three
audiences (library, CLI, web) over one core.
| Guarantee | How |
|---|---|
| Bounded memory | input consumed lazily; only the per-item menu is held, then released (on_result ⇒ O(1)) |
| Resumable | JSONL run manifest with a provenance header; a re-run skips recorded item_ids |
| Failure-isolated | a per-variant error is captured in the manifest; the cohort continues |
| Parallel (safe) | max_workers + a reference_factory (a pyfaidx handle is not thread-safe to share) |
| VCF fast path | iter_vcf(path) streams a VCF (cyvcf2), splitting multi-allelic rows and dropping non-PASS/symbolic calls — injectable, so CI-tested without htslib |
| Auditable | CohortRunReport carries run counts + provenance (version, seed, build, intent, and the content-hashed datasets its items read) |
| Placed | every row carries the resolved variant beside the item_id that was typed — an accession names no locus, and left-alignment moves a coordinate |
A cohort recomputes the same embeddings and the same reference scans constantly. alleleforge.cache
is the cross-run memo: a sharded, atomically-written (temp-then-rename) disk key/value store under
the cache dir, keyed by the SHA-256 of the inputs that determine the result, so a value computed in
one run is reused by the next.
| Cache | Key | How to use | Safety |
|---|---|---|---|
| Embeddings | sequence hash, scoped per backbone identity | CachedEmbedder.persistent(embedder) |
content-addressed; two backbones never collide |
| Off-target | spacer · PAM · budget · thresholds · reference (build + contig lengths) · regions | search(..., cache=OffTargetCache()) |
only the default-scorer, reference-only case is cached — gnomAD/haplotype/patient or a custom scorer bypasses it, so a danger scan is never served stale |
A wrong off-target report is a missed danger, so the off-target cache refuses to key anything it cannot fully capture: a changed budget/PAM/threshold/reference is a new key, and any population/haplotype/patient augmentation skips the cache entirely.
A ranked menu is only useful if a bench scientist can order it and a pipeline can
parse it. Phase 11 turns a RankedMenu into the artifacts users actually consume —
cloning-ready oligos, a structured report model, machine-readable exports, an
interactive HTML page, and a static print-ready PDF — every render leading
with the research-use disclaimer and ending with full provenance. The whole phase
is dependency-free: no plotting library, no PDF toolchain, nothing for CI to
flake on.
flowchart LR
M["RankedMenu"] --> B["build_report()"]
B --> R["DesignReport<br/>disclaimer · candidates · provenance"]
R --> OL["oligos_for()<br/>annealed duplexes, round-trip-checked"]
R --> J["JSON / TSV / Parquet<br/>(machine-readable)"]
R --> H["render_html()<br/>inlined SVG charts, ancestry tables"]
R --> P["render_pdf()<br/>print-ready, pure-Python"]
Cloning oligos round-trip by construction. oligos_for(candidate) dispatches
by chemistry; the cardinal invariant — enforced on build and re-checked by
reconstruct() — is that the oligos rebuild the intended spacer / RTT / PBS. A
design whose oligos do not reconstruct is a cloning error caught before synthesis.
| Chemistry | Oligos emitted | Default scheme |
|---|---|---|
| SpCas9 sgRNA | one duplex (vector 5' overhangs + U6 G) |
lentiGuide BsmBI |
| Base-editor sgRNA | one duplex (standard sgRNA) | lentiGuide BsmBI |
| pegRNA | spacer duplex + 3' extension (RTT + PBS + epegRNA motif) + ngRNA duplex | pegRNA GG BsaI |
The vector is yours to name, because the hazard screen follows it. Every insert is
screened for a copy of its scheme's Type IIS recognition site — the classic Golden-Gate
failure, where the enzyme that assembles the construct also cuts it and the clone
silently dies. That screen is only as right as the vector. pX330 / pSpCas9(BB), the
other standard sgRNA protocol, cuts with BbsI; its overhangs are the same
CACC/AAAC, so the oligos are correct to order either way, and a spacer carrying
GAAGAC was reported clean against lentiGuide's CGTCTC. Name your vector with
--vector-scheme (aforge design), the vector_scheme request field (POST /api/design), or scheme= (build_report): lentiguide-bsmbi (default),
px330-bbsi, pegrna-gg-bsai. An sgRNA vector has no 3'-extension overhangs and so
cannot receive a pegRNA — naming one leaves pegRNA candidates on the pegRNA acceptor
rather than failing the report, and every candidate's block names the scheme it used.
Honest rendering. The report is a fixed light document — it declares the colour scheme it was drawn for and paints its own background, because it is the artifact a collaborator is sent and is opened on a machine whose theme the author never sees. HTML charts are inlined SVG, drawn by AlleleForge's own dependency-free renderer — so no Python plotting dependency is needed and the page makes no network request at all. They were interactive Plotly figures pulled from a CDN, which left every report issuing a third-party request when it was opened; "no sequence data leaves the page" was true and beside the point, since the request itself is the disclosure. Off-target tables are ancestry-stratified, surfacing the worst-affected population per candidate. The PDF is a small self-contained writer (no weasyprint / reportlab) for a clean leave-behind.
Every number says what it is conditional on. A report a collaborator receives has to be readable on its own, so each render carries the settings its numbers depend on:
- The off-target search.
2 nominated site(s), specificity 0.82means one thing at a 0.20 CFD cut-off and another at 0.05, so the mismatch budget, the DNA/RNA bulge budgets and the CFD/MIT reporting thresholds are printed beside the count — plus the scorer and the effective weight matrix. - Model limitations. A Model limitations section names, per model, what its card says it is not for and how it fails. This is not boilerplate: the default Cas9 efficiency scorer's card states that trusting its point estimate as a trained activity prediction is out of scope, "the heads are an unfitted pseudo-random scaffold". A model documenting nothing produces no line and no heading — an empty Model limitations heading would read as a claim that there are none.
- The datasets, not just the models. The provenance footer names the datasets and tools a run consumed alongside its model checkpoints. "Population-aware" is a claim about data; a footer that names only the code cannot support it.
- What an HDR donor quietly adds. A donor marked re-cut blocked is blocked because it carries a second, unrequested base change in the guide's PAM or seed, written into the genome permanently. The order states its position, its base change and the check to run — confirm it is synonymous in your reading frame — rather than filing it in a note nobody opens.
from alleleforge.report import build_report, render_html, render_pdf, report_to_tsv
report = build_report(menu, variant="chr11:5226778:T>A", intent="correct")
open("report.html", "w").write(render_html(report)) # interactive, self-contained
open("report.pdf", "wb").write(render_pdf(report)) # static, print-ready
open("menu.tsv", "w").write(report_to_tsv(report)) # one row per candidate
report.best.oligos.reconstruct() # ('spacer', 'rtt', 'pbs')A thin, reproducible, config-driven Typer shell over the library — no
business logic of its own. Every command resolves its inputs, calls the same functions the Python API
exposes, and can emit machine-readable JSON. Install with pip install "alleleforge[cli]".
flowchart LR
CFG["--config run.toml<br/>+ CLI flags + --seed"] --> CMD
CMD["aforge subcommand"] --> RES["resolve"]
CMD --> DES["design"]
CMD --> BAT["batch (cohort)"]
CMD --> OT["offtarget"]
CMD --> DAT["data list/show"]
DES --> R["library: resolve → design → report"]
BAT --> MANY["library: iter_vcf → design_many"]
MANY --> SUM["per-item summary (TSV/JSON)<br/>+ JSONL manifest · menus/"]
R --> OUT["JSON · TSV · HTML · PDF<br/>+ .provenance.json sidecar"]
| Command | Purpose |
|---|---|
aforge resolve <input> |
Normalize any input form; show the canonical variant + class. --clinvar / --dbsnp name the release a VCV… accession or an rs… rsID is looked up in (supplied by you; never downloaded), and the release is pinned by content hash under resolved_from. --vep adds the predicted molecular consequence (opt-in: it sends the variant to Ensembl's public VEP API). |
aforge design <input> |
Variant → ranked, multi-chemistry menu rendered to JSON/TSV/Parquet/HTML/PDF (--format; TSV and Parquet are one table in two encodings, same columns in the same order, each carrying the disclaimer, reference build and coordinate convention). --clinvar / --dbsnp accept an accession or an rsID as the variant, carrying ClinVar's classification into the menu rationale — the reason to type an accession rather than the coordinates it stands for; --allow-ng / --allow-spry offer the SpCas9-NG and SpRY PAM-flexible fallbacks when no NGG guide is actionable; --trained-efficiency / --trained-outcome / --trained-base-outcome / --trained-prime swap in the consent-gated trained models; --vep annotates the menu with the variant's predicted consequence (opt-in: it sends the variant to Ensembl's public VEP API). |
aforge lift <locus>… --chain <file> --from <build> --to <build> |
Lift loci to another assembly, in the same locus form --region accepts. An unmappable locus prints UNMAPPED and exits non-zero rather than being dropped. |
aforge batch <vcf|list> |
Cohort design over a VCF (cyvcf2 fast path) or variant list — streaming, resumable, failure-isolated. --clinvar / --dbsnp apply to every item, so a cohort can be a list of accessions. --vep annotates each item's consequence (opt-in: it sends every variant to Ensembl's public VEP API); --cache and --genome-index let a cohort reuse the reference scan its items share. |
aforge offtarget <spacer> |
Standalone population/haplotype-aware off-target search. `--scorer cfd |
Important
The three safety inputs are opt-in files, and the scan is reference-only without them.
--populations names the ancestries to stratify by; it carries no alleles. The data comes from
--gnomad (population allele frequencies), --haplotypes (a phased common-haplotype panel) and
--patient-vcf (this genome's own variants) — all three available on design, batch and offtarget.
Asking for ancestries with none of them supplied prints a warning saying the scan was reference-only and
the empty ancestry breakdown means not measured, not clean — and the report itself names them as
requested but not examined, so the statement survives into the HTML, the PDF and the TSV rather than
living only in the terminal the run happened in. Scope a scan with --region chrom:start-end (repeatable) or --regions-bed panel.bed — over a real reference that is usually what
makes a run practical. The open-chromatin efficiency adjustment takes --encode-tracks track.bedgraph --chromatin-track <name> (both required together, and the name is checked against the file — a track
the bedGraph does not contain is refused with the list of names it does, rather than declining every
chemistry into an empty menu). The bedGraph is pinned in provenance by content hash like every other
user-supplied source, because the accessibility signal moves the efficiency number and the track name
alone cannot tell two files apart. Separately, --cell-context <line> is what raises the
out-of-distribution flag on a prime efficiency prediction — the one vertical that consumes it. SpCas9
nuclease and base editing do not take a cell context; when one is supplied and they run, the rationale
names them and says their in-distribution flag describes the guide context alone, so an unqualified "in
distribution" beside a nuclease candidate is not mistaken for a claim about the cell line. Every design() capability is reachable from
the CLI. Four of its parameters are not passed by any command, and none of them is a capability: one is
the HGVS adapter for c./p. inputs, which needs a projector from the hgvs library (not a dependency,
and no file a flag could name — genomic g. needs no adapter and already works everywhere), one is an
injection point with no trained model to select, one is the positional variant argument and one is a
test-only provenance hook. That list lives with its reasons in
tests/test_shells_expose_the_library.py, which fails if a
fifth appears or if one of the four becomes reachable and the reason is left behind. It went from six to
four when --clinvar and --dbsnp were added: the excuse for those two said the lookups were
Protocols with no shipped implementation, and ClinVarDB/DbSnpDB had shipped all along. On the web API a client-supplied filesystem path would be a server-side file-read primitive, so
the file-backed inputs are configured by the operator, exactly as the reference already is:
ALLELEFORGE_GNOMAD_TSV, ALLELEFORGE_HAPLOTYPES and ALLELEFORGE_ENCODE_TRACKS (or the matching
create_app(...) arguments) make the population-aware search, the haplotype-aware pass and the chromatin
adjustment available over HTTP. ALLELEFORGE_VEP is operator-configured for a different reason — enabling
it means this deployment discloses its clients' variants to an external VEP server — and a request then
opts in per call with annotate_consequence. ALLELEFORGE_TRAINED_MODELS splits the same way and for the
same kind of reason — the Rule Set 3, Lindel, BE-DICT and DeepPrime weights are a consent-gated download
onto the operator's disk — so the operator lists which are offered and a request picks one per call with
trained_efficiency / trained_outcome / trained_base_outcome / trained_prime, the same names
aforge design uses as flags. Until then every menu the API returned was scored by the transparent
baseline, and nothing on it said so. GET /api/health reports which of them this deployment loaded, and names
the reason when a configured one could not be read — a client cannot supply them, so it has to be able to
see them. --patient-vcf remains absent for a different reason: a personal genotype is the caller's
data, not the operator's, so server-side configuration is the wrong shape for it and an upload path is a
separate decision.
Everything that is data rather than a path — region scoping, cell context, the render cap, the
on-target locus, the specificity scorer — is available over HTTP, and every request model forbids
unknown fields: a parameter this server does not support is a 422 naming it, never a 200
describing a run the client did not ask for.
| aforge verify <result.json \| result.provenance.json> | Check a result's provenance is complete — it names every model and dataset used (checked against the result: a menu names a model for each chemistry it ranks, and a scoring matrix a candidate names must appear in datasets) and carries seed, version and config — and, with --cache-dir, re-hash each pinned checkpoint and dataset found there against the recorded hash. Exits non-zero on incomplete provenance or a hash mismatch: provenance as a checkable contract, not a record. Takes either shape design produces — the result JSON, or the bare .provenance.json sidecar written beside it, which for --format tsv, parquet, html and pdf is the only machine-readable provenance a run leaves behind. Without --cache-dir no bytes are re-hashed, and the command says so: completeness and artifact integrity are two different claims and only one of them is free. |
| aforge data list / show <name> | Inspect the dataset registry (versions, licenses, provenance). |
| aforge bench list / run | List and run CRISPR-Bench tasks against frozen splits. |
| aforge bench compare <a.json> <b.json> | Ask whether two results are the same scientific result — the question the reproducibility digest exists for. It covers task, split identity, dataset, metrics and model, and excludes timestamps, package versions and local settings, so two labs on two platforms agree iff the science does. Each result's digest is re-derived from its own body first, and a mismatch names the fields that differ. |
| aforge bench leaderboard <result.json…> | Aggregate signed results into the model-card-gated leaderboard (Markdown/HTML). |
Global options sit before the subcommand (--seed, --reference, --cache-dir, --verbose,
--version); every command takes --json. Exit codes are distinct and scriptable: 0 success,
2 usage/input error, 3 missing data (e.g. reference FASTA not found), 4 an unavailable model or
feature. A run is reproducible from its echoed --seed + config (byte-identical modulo the UTC
timestamp), and a <output>.provenance.json sidecar is written next to every file output.
# Reproducible design from a config file; CLI flags override the file
aforge --seed 20240501 design 'chr2:71:A>C' \
--reference-fasta hg38.fa --config run.toml \
--chemistry prime --weights 0.5,0.2,0.2,0.1 --format html --out report.html
# → wrote report.html and report.html.provenance.jsonThe accessible front door for users who will not touch a terminal: a FastAPI backend that exposes the
library over HTTP, and a dependency-free served single-page frontend that drives the variant-first
journey in the browser — with a single-variant tab, a cohort (batch) tab that posts a variant
list to /api/batch and renders the per-item summary table, and a check a spacer tab that posts to
/api/offtarget for the guide someone already holds. Every endpoint the API exposes is reachable from
the page or carries a written reason
(test_the_page_reaches_every_endpoint_or_says_why.py). The app is a thin async layer with no
business logic of its own — it validates each request with a pydantic model, calls the same functions the
Python API and CLI use, and returns a Phase 1 / Phase 11 schema-validated response, with OpenAPI
auto-generated at /docs.
flowchart LR
B["Browser SPA<br/>(served, no Node build)<br/>single · cohort tabs"] -->|POST /api/design · /api/batch| API
CURL["curl / httpx / any client"] -->|JSON| API
subgraph API["FastAPI app (local)"]
EP["resolve · design · batch · offtarget<br/>data · bench · health · jobs"]
JQ["in-process async job queue<br/>(thread worker + progress)"]
EP --> LIB
JQ --> LIB
end
LIB["library: resolve → design → report"] --> OUT["JSON · HTML · PDF<br/>(Phase 1 / Phase 11 schemas)"]
Important
Local, private, no egress. All compute is local and user-controlled. The app makes no outbound network call and transmits no sequence data externally — a guarantee enforced by a test that fails if any socket connects during a design request. The served frontend says so prominently and loads no third-party scripts.
| Method & path | Purpose |
|---|---|
GET /api/health |
Liveness, disclaimer, and which data sources this deployment loaded (reference, population sites, haplotype panel, accessibility track names) — plus why a configured one failed to load |
POST /api/resolve |
Normalize any input form to a canonical variant |
POST /api/design |
Variant → ranked menu; ?format=json|html|pdf|tsv|parquet — the same set aforge design --format offers, so a pipeline gets the flat table over HTTP too |
POST /api/jobs/design → GET /api/jobs/{job_id} |
Async job submit + status/progress/result |
POST /api/jobs/batch → GET /api/jobs/{job_id} |
The same, for a whole cohort — the operation that actually takes minutes, and the one the async path did not cover |
POST /api/batch |
Cohort design over a variant list; per-item summaries + provenance, failures isolated |
POST /api/offtarget |
Standalone population-aware off-target search — full report plus the aggregate summary (site count, worst-case, specificity) |
GET /api/data · /api/data/{name} |
Inspect the dataset registry |
GET /api/bench |
List the CRISPR-Bench tasks, datasets, and primary metrics |
GET / |
The served single-page frontend |
# One-command local deploy (reference FASTA mounted at ./data/reference.fa)
docker compose up --build # → http://localhost:8000 · /docs for OpenAPI
# Or run directly
pip install "alleleforge[web]"
ALLELEFORGE_REFERENCE_FASTA=hg38.fa uvicorn alleleforge.web.api.app:app --port 8000
# Cohort design over HTTP: post a variant list, get per-item summaries + provenance
curl -s localhost:8000/api/batch -H 'content-type: application/json' \
-d '{"variants": ["chr2:71:A>C", "chr11:5227002:A>T"], "intent": "correct"}'The async job worker is in-process (the default deployment is single-user and local), so no broker or
separate worker container is needed; a multi-user deployment can swap in a real broker behind the same
JobManager interface. The served vanilla-JS frontend (single-variant, cohort and spacer tabs) ships inside the
wheel and is exercised end to end by the API tests; a production Next.js + JBrowse 2 frontend can replace
it behind the same API unchanged.
The sister deliverable and a field-level contribution in its own right: a common yardstick for guide- and
edit-design models — versioned datasets, frozen content-hashed splits, a fixed five-task contract, a
metric battery where calibration is required on every task, a runner that turns any Scorer into a
signed result, and a model-card-gated leaderboard. It is valuable independently of the rest of
AlleleForge, and the same scorers the designer uses are graded by it.
flowchart LR
DS["datasets/<br/>provenance-stamped,<br/>content-hashed"] --> SP
SP["splits/<br/>frozen · cross-context<br/>hash-verified on read"] --> RUN
SC["any Scorer<br/>(returns a calibrated<br/>Prediction)"] --> RUN
RUN["runner<br/>metrics + ECE"] --> RES["signed, provenance-<br/>stamped result"]
RES --> LB["leaderboard<br/>(model-card gated)"]
The five tasks — every chemistry AlleleForge designs for, plus off-target. Each reports its accuracy metric and Expected Calibration Error, because a model that is accurate but overconfident is dangerous for edit design:
| Task | Kind | Source corpus | Primary metric | + required |
|---|---|---|---|---|
cas9-efficiency |
regression | Rule Set 3, DeepHF/DeepSpCas9 | Spearman | Pearson, ECE |
cas9-outcome |
distribution | FORECasT, inDelphi, Lindel | KL ↓ | top-1, ECE |
be-outcome |
distribution | BE-Hive, BE-DICT | KL ↓ | top-1, ECE |
pe-efficiency |
regression | PRIDICT2 Library-Diverse | Spearman | Pearson, ECE |
offtarget-classification |
classification | GUIDE-seq / CHANGE-seq | AUROC | AUPRC, ECE |
Frozen, content-hashed, cross-context splits. A split is immutable once published. Each split file pins
its fold membership and two hashes — one over the dataset content it was cut from, one over its own
membership — and load_split() re-verifies both on read, raising SplitIntegrityError on any drift.
Changing the data, or the split, means minting a new version; you never edit a published one. Test folds
hold out a whole cell context, so the benchmark measures generalization, not memorization — the known
weak spot of guide models, made a headline feature instead of a footnote. benchmark.generalization_gap
turns that into a number: a model's primary metric on an in-context fold vs the held-out cell type,
oriented so a positive gap means worse generalization (R5; reported in the calibration study).
Honest by construction. Results are content-addressed (signature) so a published number cannot be
silently edited, and the leaderboard refuses any submission lacking a model card (name, license, citation) or
carrying a bad signature — aforge bench leaderboard *.json aggregates signed results into the board
(Markdown/HTML), enforcing both gates on read. The board shows ECE and an OOD column — the share of
the scored test fold the model itself declared out-of-distribution — so a model that stood behind every
prediction is not on the same row as one that disclaimed 87% of them and scored the same. An unmeasurable
share reads n/a, never 0%: silence and a clean bill are different claims. Neither column enters the
ranking; trading coverage against accuracy needs an exchange rate this project does not have. The regression-task ECE is interval-coverage calibration
(|empirical coverage − nominal|), and because that is only well-defined against a single nominal level it
is computed per interval_level and count-weighted — a scorer that mixes interval levels in one batch is
scored correctly, never pooled against one prediction's level. The shipped datasets are small synthetic fixtures so the
whole benchmark runs in CI with no downloads; the real corpora are fetched at runtime through the same
consent-gated registry as the population data. See
src/alleleforge/benchmark/README.md.
aforge bench list # the five tasks, datasets, and metrics
aforge bench run cas9-efficiency # score the reference baseline on the frozen split
aforge bench run pe-efficiency --out result.json # signed, provenance-stamped result JSON
aforge bench leaderboard *.json --format html --out board.html # model-card-gated boardfrom alleleforge.benchmark import build_baseline, get_task, load_split, run_benchmark
task = get_task("offtarget-classification")
split, dataset = load_split(task.name) # hash-verified on read
result = run_benchmark(build_baseline(task, split, dataset), task, split=split, dataset=dataset)
print(result.primary_metric, round(result.primary_value, 3), "ece", round(result.metrics["ece"], 3))
assert result.verify_signature()Note
The benchmark lives at alleleforge.benchmark (an installed subpackage) rather than the spec's sketched
top-level benchmark/ tree, so it ships in the wheel, is reachable from aforge bench, and is held to the
same mypy --strict / ruff / coverage gates as the rest of the library.
Calibration & generalization, at a glance. Every task reports its ECE, and the cross-cell-type gap is a first-class number. The figures below are computed on the weight-free splits — they verify the machinery (the metric battery, the split mechanics, the generalization-gap computation), not model quality, which awaits the real-weights integration (R1). Split-conformal recalibration restores interval coverage to its nominal target with a finite-sample guarantee.
Three notebooks walk the journey end to end. Each is self-contained — it builds a small synthetic
locus and runs against the weight-free stub models — so they execute in CI on every push (pytest --nbmake examples/) and reproduce without downloading a genome or model weights. Point them at a real
hg38 reference, a gnomAD database, and trained weights via the model zoo, and the call shapes are identical.
| Notebook | What it demonstrates |
|---|---|
01_clinvar_to_design |
The canonical journey: a variant → ranked prime-editing design across all four axes. |
02_population_offtarget |
The reference-bias case (rs114518452): a reference-only scan is blind to a population allele that creates a de-novo PAM; the population-aware engine nominates it and reports it ancestry-stratified. |
03_batch_vcf |
Cohort-scale design: resolve a batch of variants, design each, and reduce to one auditable summary with provenance. |
04_indel_prime_correction |
Correcting a small deletion (ΔF508-shaped): the variable-length RT template writes the missing bases back, read apart into homology + restored allele + homology. |
Full docs (concept guides, deployment, CLI reference, CRISPR-Bench, a
methods-preprint outline) build with mkdocs build --strict in CI.
The release pipeline is wired and tag-triggered (.github/workflows/release.yml) —
it stays inert until v0.1.0 is tagged, then it:
| Target | Mechanism |
|---|---|
| PyPI | python -m build → pypa/gh-action-pypi-publish via OIDC Trusted Publishing (no stored token) |
| Docker | multi-arch (linux/amd64 + linux/arm64) image pushed to GHCR with buildx |
| GitHub Release | sdist + wheel + CycloneDX SBOM attached, notes auto-generated |
| SBOM | cyclonedx-py over the resolved dependency closure, attached to the release |
| Zenodo DOI | minted on the tagged release (.zenodo.json) |
| conda | bioconda-style recipe (conda/meta.yaml) |
First public release is v0.1.0 (three chemistries end to end with the benchmark); v1.0.0 is reserved
for after external validation and the methods preprint. CITATION.cff ships for citation.
Every default is overridable; these are the spec-mandated starting points.
| Topic | Default | Notes |
|---|---|---|
| Reference / coordinates | hg38, 0-based half-open | T2T-CHM13 auto-recommended for ambiguous loci, and every candidate designed there carries the ambiguous-region:<kind> caveat; mm39 for mouse |
| Strand | always explicit | no implicit "default strand"; spacers stored 5'→3' |
| SpCas9 PAM | NGG (primary), NAG low-stringency |
NG / SpRY opt-in when no NGG is actionable |
| Off-target search | ≤ 4 mismatches, ≤ 1 DNA + ≤ 1 RNA bulge | report CFD ≥ 0.20 or MIT ≥ 0.10 |
| Population inclusion | MAF ≥ 0.001, all populations | de-novo PAM & seed-mismatch changes always evaluated |
| Base-editing window | protospacer positions 4–8 | ABE8e (A→G), CBE4max / evoCDA1 (C→T); bystanders always reported |
| Prime editing | PE5max + epegRNA (tevopreQ1) | PBS 8–17 nt, RTT 7–34 nt; PE3b nicking guide when seed-disrupting; nick-to-nick distance shown on every PE3 candidate, close-nick below 30 nt |
| Uncertainty | 80% predictive interval | deep ensemble (N=5) + isotonic calibration |
| Seed | 20240501 |
threaded through every stochastic step, recorded in provenance |
alleleforge/
├── pyproject.toml # hatchling build, deps, ruff/mypy/pytest config
├── SPEC.md # the authoritative, phase-by-phase build contract
├── rust/ # PyO3 crate: aforge_native (BWT, k-mer, haplotype)
├── src/alleleforge/
│ ├── config.py # typed Settings (pydantic-settings), defaults, paths
│ ├── cache.py # R4: content-addressed cross-run disk cache (embeddings · off-target)
│ ├── _native.py # optional Rust bridge
│ ├── types/ # Phase 1: core domain vocabulary
│ ├── genome/ # Phase 2: reference access, FM-index, liftover
│ ├── data/ # Phase 3: registry, ClinVar, gnomAD, 1000G/HGDP, dbSNP, annotations
│ ├── variant/ # Phase 4: resolver, HGVS adapter, consequence
│ ├── offtarget/ # Phase 5: population/haplotype-aware off-target
│ ├── model_zoo/ # Phase 6: license-gated model cards + checkpoints
│ ├── scoring/ # Phase 6: embeddings, uncertainty, Scorer (this release)
│ ├── enumerate/ # Phases 7–9: SpCas9 guide · base-editor window · pegRNA enumeration
│ ├── design/ # Phases 7–10: nuclease · base · prime verticals + designer (routing · ranking) + cohort (R4 batch)
│ ├── report/ # Phase 11: oligos · report builder · JSON/TSV/Parquet · HTML · PDF
│ ├── cli/ # Phase 12: the aforge Typer CLI (resolve · design · batch · offtarget · data · bench)
│ ├── web/ # Phase 13: FastAPI api/ + served frontend/ (variant-first journey)
│ ├── benchmark/ # Phase 14: CRISPR-Bench — tasks · datasets · frozen splits · runner · leaderboard · calibration (R5)
│ ├── viz/ # R5: dependency-free SVG figure renderer (reference bias · coverage · ECE · gap)
│ └── ...
├── Makefile # local mirror of the CI gate (make ci · reproduce · figures · native)
├── tests/ # mirrors src/; pytest + hypothesis
├── examples/ # Phase 15: runnable notebooks (executed in CI via nbmake)
├── scripts/ # schema export · benchmark-fixture generator · reproduce (R0) · native_speedup · calibration_study · figures (R5)
├── conda/ # Phase 15: bioconda-style recipe
├── docs/ # mkdocs-material site (concepts · deployment · reference · paper · assets/figures)
└── .github/ # workflows: ci.yml (lint·type·test·docs·examples·rust·security·reproduce) · release.yml · dependabot.yml
make install # editable install with the extras `make ci` needs
make ci # the whole gate: lint · type · test · docs · examples · reproduce
make native # build the Rust crate and run the suite against itmake ci is the local mirror of the blocking CI jobs, and
tests/test_gate_mirrors_ci.py compares the two by
command, not by job name — so what you run here is what the pipeline runs. Prefer it to
typing the individual tools: this block used to spell them out, and drifted, running ruff
over three paths where CI ran four and telling contributors to maturin develop, which
installs a build of the working tree into whatever virtualenv is active.
Individual targets, when you want one: make lint, make type, make test, make docs,
make examples, make reproduce, make figures. make help lists them.
The library is fully typed and ships a PEP 561 py.typed marker, so mypy/pyright see its types
when you depend on it. A Makefile mirrors the gate so make ci reproduces it locally
(make lint type test docs reproduce; make figures for the docs figures, make native for the crate).
CI (GitHub Actions) runs lint, type-check (mypy --strict), tests (Python 3.11 + 3.12 on Linux & macOS),
a strict docs build, notebook execution, the Rust crate (cargo fmt · clippy · maturin build plus a
native↔Python FM-index parity run), a supply-chain audit (pip-audit + cargo audit), and a
reproducibility audit (scripts/reproduce.py re-derives the canonical run from config + seed and
diffs it against a committed golden) on every push and PR. See .github/workflows/ci.yml;
releases are cut on v* tags by .github/workflows/release.yml and emit
a CycloneDX SBOM. Dependabot tracks pip, cargo, and github-actions. The native
Rust crate builds locally with maturin (cd rust && maturin develop); the library runs in pure-Python
mode without it.
Contributions are welcome — please read CONTRIBUTING.md and the
Contributor Covenant 2.1 code of conduct. To report a security
issue, follow SECURITY.md — open a private advisory, not an issue.
tests/test_acceptance.py encodes the v0.1.0 release contract — the
specification's "definition of done" — as six end-to-end tests that run on every push:
| Release criterion | Proven by |
|---|---|
| A ClinVar accession flows end to end to a complete, provenance-stamped menu | test_clinvar_accession_to_complete_menu |
| The unified entry point reaches every chemistry (base · prime · nuclease) | test_every_chemistry_reachable_through_one_entry_point |
| A run is reproducible from config + seed (identical serialized menu) | test_run_is_reproducible_from_seed |
The reference-bias / rs114518452 off-target case is reproduced |
test_reference_bias_case_reproduced |
| Prime editing unifies all four axes | test_prime_unifies_all_four_axes |
| CRISPR-Bench publishes the Cas9-/PE-efficiency + off-target tasks with calibration & a leaderboard | test_crispr_bench_publishes_required_tasks |
- Research use only. AlleleForge produces hypotheses and rankings, not medical advice or clinical decisions. Every generated report repeats this.
- Off-target predictions require experimental validation. Computational nomination narrows the search; it does not replace GUIDE-seq / CHANGE-seq / amplicon confirmation.
- No telemetry, no phone-home. All computation runs locally or on user-controlled infrastructure. User sequences are never transmitted externally.
- Honest uncertainty over false confidence. Where models are out of distribution (e.g., prime-editing efficiency outside PRIDICT's HEK293T / K562 training context), AlleleForge flags it rather than hiding it.
- Dual-use awareness. This is a design and safety-analysis tool for legitimate therapeutic and basic research. It contains no wet-lab protocols or synthesis instructions.
AlleleForge is released under the MIT License — all code, schemas, benchmark, and any first-party model weights. It is fully open source and free to use, modify, and redistribute.
Each wrapped third-party tool or model retains its own upstream license, recorded in its model/tool card; the registry refuses to bundle any component whose license is incompatible with redistribution and fetches it at runtime with the user's consent instead.
If you use AlleleForge, please cite it via CITATION.cff. A Zenodo DOI is minted on the first
tagged release. The methods are written up in the draft preprint at
docs/paper/preprint.md (the posted version, with the real-data validation
numbers, follows the v1.0 release).