Repository navigation
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LKxc9GKLwnyeV2jaZBxH9N
|
|
||
|
|
||
| @jax.jit | ||
| def _truncated_normal_icdf( |
There was a problem hiding this comment.
shall we add this as a staticmethod in LeftTruncatedDistribution so that this is not a random aux function?
juanitorduz
left a comment
There was a problem hiding this comment.
Thanks for the fix. The two-sided Normal case is correct now. Requesting changes for the items below.
Tests
The icdf check against osp.truncnorm.ppf and the (15, 16) case are the parts that matter. The sample + support assertion can go:
sampleisicdf(uniform(tiny, 1)), and the new branch clips its output to[low, high], so the support check can only fail if the branch isn't taken. The icdf check catches that first.- 256 draws never get near q = 0 or 1, so it adds no coverage of the extremes.
- A clip-only fix (
jnp.clip(ppf, low, high)) would pass it, with every draw at -15. Only the icdf check rejects that.
Keep (15, 16): it's the only case that goes through the reflected branch (sign = -1), which covers the sign and the -inf/0 limit in log_tail_at, and loc + sign * scale * z.
The jnp.float32 casts only matter under JAX_ENABLE_X64=1, which CI doesn't use for this test. Keep or drop, either is fine.
Suggested version:
@pytest.mark.parametrize("low,high", [(-16, -15), (15, 16)])
def test_truncated_normal_icdf_in_tail(low, high):
d = dist.TruncatedNormal(low=jnp.float32(low), high=jnp.float32(high))
q = np.array([0.1, 0.5, 0.9], dtype=np.float32)
assert_allclose(d.icdf(q), osp.truncnorm.ppf(q, low, high), rtol=0, atol=2e-5)If you want a second assertion, check gradients rather than support. The fix changes reparameterized gradients in the tail (on master, d sample / d loc is 1), and a NaN leaking through the jnp.where(..., -100.0) placeholder is the regression nothing else would catch. Gradients are correct today (they match central finite differences), so this one is optional.
Code
- One-sided truncation has the same bug, with ±inf draws. On this branch,
TruncatedNormal(low=15)returns+inffor every draw andTruncatedNormal(high=-15)returns-inf.supportaccepts these, sincegreater_than_eq(15)allowsinf. The new helper already handles infinite bounds: calling_truncated_normal_icdfwith(low=15, high=inf, sign=-1)and(low=-inf, high=-15, sign=1)matches SciPy to ~1e-6. Could you wire it intoLeftTruncatedDistribution.icdfandRightTruncatedDistribution.icdfhere and add the one-sided cases to the test? If you'd rather keep this PR to two-sided, please open an issue for it. - Drop
@jax.jiton_truncated_normal_icdf. No other helper innumpyro/distributionsis jitted. Under an outerjitit does nothing, and in eager mode it adds a compile cache per input shape. - Not blocking:
LogisticandLaplacebases fail the same way far in the tail.TruncatedDistribution(Logistic(), low=-110, high=-105)puts every draw at -87.34, andLaplacegives-inf. Fine to leave out of this PR.
What I checked locally
- The new test fails on master (max abs diff 2.2).
- f32: icdf vs SciPy for q from
tinyto1 - 2**-24, max error ≤ 3.2e-6 for [-16, -15], [15, 16], [-13.5, -12.5], [12.5, 13.5], [-40, -39],loc=3, scale=2, and an explicit-inflower bound. Master is off by 2 to 27. - x64: [-40, -39] and [39, 40] within 1.2e-13.
jit+vmapover a batch mixing tail and non-tail intervals stays in bounds.- Reparameterized gradients match central finite differences for
loc,scaleandhigh. - Speed (jitted f32, 1e6 draws, single run): non-tail 2.0 → 2.6 ms, tail 2.5 → 7.1 ms. The
lax.condkeeps the common case cheap. ruff check,ruff format --checkandty checkpass;pytest test/test_distributions.py -k "runcated or Truncated": 1359 passed.
Assisted-by: OpenAI Codex
|
That's a good point. I changed those. |
Benchmark reportthis PR - run time: 1 slower, 1 faster
- compile time: 2 slower, 0 fasterSignificant changes (4) ──────── run time ─────── ────── compile time ──────
benchmark baseline this PR Δ baseline this PR Δ
────────────────────────────────────────────────────────────────────────────────────────
+ student_t_log_prob 1.2 ms 1.2 ms -5.0% 54.4 ms 45.0 ms -17.3%
- predictive_forward_sampling 524.6 ms 499.2 ms -4.8% 134.3 ms 195.4 ms +45.5%
- hmc_logistic_regression 753.9 ms 814.6 ms +8.0% 1.79 s 2.08 s +16.1%
- nuts_dense_mass_funnel 938.5 ms 967.8 ms +3.1% 1.49 s 1.87 s +25.7%Red is slower, green is faster; a row is coloured by the worse of its two columns. A delta in parentheses cleared the threshold on a measurement below the resolution floor, so it is shown without being called a change. † marks a benchmark that could not be compared — see below. Full results
|
| baseline | this PR | |
|---|---|---|
| ref | master |
fix-truncated-normal-tail |
| commit | fc7691c7 |
bb4e2c95 |
| numpyro | 0.22.0 | 0.22.0 |
| jax | 0.12.1 | 0.12.1 |
| backend | cpu | cpu |
| python | 3.14.4 | 3.14.4 |
Runner: Linux-7.0.0-1019-azure-x86_64-with-glibc2.43, 4 CPUs.
Produced by this benchmark run.
TruncatedNormal(low=-16, high=-15)can draw-12.95with float32. Callsample(key, (256,))and checksupportfor each draw. The patch computes and inverts the log CDF when the tail probability underflows. The test compares quantiles to SciPy and checks that draws lie between the bounds.🤖 Generated with Claude Code
https://claude.ai/code/session_01LKxc9GKLwnyeV2jaZBxH9N