Add variance-stratified residual-Mahalanobis FDR flavor

[katosh]

Summary

Opt-in FDR calibration mode FDRSettings(mode="variance_stratified") that residualises log(1 + D²) against a smooth surface in (log_mean, log_var) fit on the permutation null, then applies kompot's existing 1-D local FDR to the standardised residual Z. Intended for low-replication designs (≈2 biological replicates per condition), where the raw Mahalanobis is dominated by per-gene variance rather than condition-specific signal.

• Default behaviour is unchanged (mode="raw").
• Raw *_mahalanobis, *_mahalanobis_local_fdr, and *_is_de columns are preserved.
• New columns added alongside:
  *_residual_mahalanobis, *_residual_z, *_residual_local_fdr, *_residual_is_de
  (plus _pvalue / _tail_fdr / _log_mean / _log_var under store_additional_stats=True).
• Public helpers in kompot.residualization for post-hoc use on any existing kompot output.

Background

When GPSettings.use_empirical_variance=False (kompot's default — chosen so per-cell sampling noise is not conflated with epistemic posterior uncertainty), D²_g scales monotonically with per-gene expression variance σ²_g. The gene-shuffled null inherits that scaling, so the 1-D local FDR is genome-wide calibrated but not gene-specific. With ~2 replicates per condition, any 2-vs-2 partition — including the real one — produces mean differences roughly proportional to σ²_g, so high-variance genes rank at the top under any partition.

Method

1. Per gene: m_g = log1p(mean_g), v_g = log1p(var_g) from the DE expression layer.
2. Fit null-trend surface φ̂(m, v) = E[log(1 + D²_null) | m, v] via OLS on a 7-term tensor polynomial over the gene-shuffled null draws.
3. Homoscedastic null residual scale σ̂_null.
4. R_g = log(1 + D²_g) − φ̂(m_g, v_g), Z_g = R_g / σ̂_null.
5. Apply kompot's existing 1-D local FDR to max(0, Z_g). Below-null genes get local_fdr ≈ 1.

Tal1 chimera validation

Post-hoc on /export/chimera_tal1_kompot_ps01_*:

	raw top-20	residual top-20
Hematopoietic (Hbb-bh1, Itga2b, Lyl1, Stab1/2, Tfec, Ackr1, …)	5	11
Imprinted (Cdkn1c, Phlda2, Grb10, Impact, Mcts2, Nnat, …)	12	0
Trend R² on Tal1 null	—	0.88
σ̂_null on Tal1 null	—	0.48

Residual top-20 (in order): Hbb-bh1, Hba-x, Hbb-y, Stab2, Itga2b, Stab1, Tfec, Aqp8, Rplp0, Rpl41, Hba-a1, Slc22a18, Ackr1, Ushbp1, Erdr1, Rps18, Adora2a, Clec14a, Lyl1, Hba-a2. Matches the known Tal1 hematopoietic phenotype; the raw list was dominated by high-variance imprinted / X-linked genes.

Design doc

Full derivation, evidence table, and limitations section (also destined for the methods supplement) live in the design note reports/kompot_notebooks_private_2026-04-17_142500_residual-mahalanobis-flavor.md of the companion kompot_notebooks_private workspace.

What this does and does not do

Does

• Remove the strictly gene-level monotone dependence on (mean, variance) from the statistic before FDR calibration.
• Rank genes by how unusual their Mahalanobis is conditional on their (m, v).
• Reuse kompot's existing GP fit / Mahalanobis / null-gene machinery; no GP changes.

Does not

• Remove manifold / cell-type structure. Genes that vary strongly between cell types will still have large Mahalanobis under any cell partition that splits cell-type mixtures.
• Fix the underlying experimental-design limitation of two biological replicates per condition. With sample fully confounded with condition, no statistic can separate condition-level biology from embryo-to-embryo variability in principle; the residual flavor correctly refuses to rank variability genes first but does not convert a 4-embryo design into a well-replicated one.

Settings

FDRSettings(
    mode="variance_stratified",               # opt-in; 'raw' is default
    null_trend_features=("log_mean", "log_var"),
    null_trend_model="poly3",                  # 7-term tensor polynomial (default)
    # or "poly3_mean_only" for a diagnostic mean-only model
)

Test plan

• 19 new tests in tests/test_residualization.py (feature extraction, null-trend fit, residualisation math, end-to-end through kompot.de, settings validation) — all pass.
• Full FDR test suite (test_fdr_utils, test_fdr_edge_cases, test_fdr_integration, test_model_settings): 75 / 75 pass.
• Full DE test suite (test_differential, test_differential_expression_core, test_differential_expression_comprehensive, test_de_comprehensive_params, test_field_tracking, test_anndata_tracking): 74 pass, 4 pre-existing skips.
• Tal1 top-20 flip validated post-hoc (see table above).
• Upstream CI on PR branch.

Files

• kompot/residualization.py — pure-numpy module with compute_gene_features, fit_null_trend, residualize_mahalanobis, residual_local_fdr
• kompot/settings.py — FDRSettings knobs: mode, null_trend_features, null_trend_model
• kompot/anndata/_de_helpers.py — _compute_residual_fdr helper; extended _compute_fdr, _store_de_results, _build_field_mapping, _check_overwrites
• kompot/anndata/differential_expression.py — plumbs settings through de() and legacy compute_differential_expression()
• kompot/anndata/utils/field_tracking.py — residual field-name keys
• docs/source/variance_stratified_fdr.rst — user-facing guide
• docs/source/index.rst, README.md, CHANGELOG.md
• tests/test_residualization.py