Skip to content

Nested Archimedean copulas#367

Open
thisiscam wants to merge 26 commits into
lrnv:mainfrom
thisiscam:nested-archimedean
Open

Nested Archimedean copulas#367
thisiscam wants to merge 26 commits into
lrnv:mainfrom
thisiscam:nested-archimedean

Conversation

@thisiscam
Copy link
Copy Markdown
Contributor

Adds support for hierarchical Archimedean copulas of arbitrary tree depth with per-subtree generator choices. Density via Faà di Bruno over the tree (Bell-polynomial recursion + Cauchy product per node); closed-form CDF.

Reuses upstream ϕ⁽ᵏ⁾, taylor(), and the existing generator interface throughout — no parallel derivative machinery. Tests check the tree-density against an independent nested-ForwardDiff reference and against frozen CSVs from a JAX implementation.

Three coordination questions before un-drafting:

  1. src/nested/FrankTaylorGenerator.jl is 4 lines adding ϕ / ϕ⁻¹ of FrankGenerator on ::Taylor1 (works around log1mexp not having a Taylor1 rule). Happy to split as a tiny main-targeted PR first and rebase.
  2. src/CensoredLikelihood.jl currently has our per-variable censored likelihood. Once Missing logpdf(::Copulas.DistortedDist{Copulas.ArchimedeanDistortion{Copulas.ClaytonGenerator{Float64}, Float64, 1}, Normal{Float64}}, ::Float64) #345 lands, the natural form is the decomposition via condition / subsetdims. Prefer (a) wait for Missing logpdf(::Copulas.DistortedDist{Copulas.ArchimedeanDistortion{Copulas.ClaytonGenerator{Float64}, Float64, 1}, Normal{Float64}}, ::Float64) #345 and rebase onto it, (b) drop censoring from this PR and follow up separately, or (c) keep here for now and remove on rebase?
  3. condition / subsetdims aren't specialised for NestedArchimedeanCopula yet — the generic AD-on-_cdf fallback works (we provide a closed-form _cdf) but is slow. Want a tree-walking specialisation in this PR or as a follow-up?

Reference: Yang & Li, "Archimedean Copula Inference via Taylor-Mode AD," arXiv:2605.23134 (Algorithm 1 = the nested-AC density).

@lrnv
Copy link
Copy Markdown
Owner

lrnv commented May 29, 2026

  1. I understand why you want to do that, but our interface should already cover the case fully through ϕ, ϕ⁻¹, ϕ⁽¹⁾, ϕ⁻¹⁽¹⁾, ϕ⁽ᵏ⁾ and ϕ⁽ᵏ⁾⁻¹. Do you really need ϕ⁻¹⁽ᵏ⁾ for k > 1 ?
    (a) You do need ϕ⁻¹⁽ᵏ⁾ for k > 1: Then maybe we should provide a generic interface so that it is available to all generators, and can be overwritten, as we did for others.
    (b) You do not need it: Then please do not use Taylor1 directly and pass through our generics: this will make the Frank one go through without a specific fix. This should avoid other issues as the ones you might have with very-dependent gumbels, or other things like that.

  2. The other PR landed alreayd on master, but i still prefer (b) make it a separate PR, as this clearly is a separate feature

  3. If possible, yes, it would be better to support the whole interface. Also, the fit() function ? At least maximum likelihood should be possible if the tree and generators are provided, since we have a density.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 29, 2026

Codecov Report

❌ Patch coverage is 90.81886% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 79.21%. Comparing base (73649fe) to head (9f06bd0).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
src/nested/NestedConditioning.jl 67.50% 26 Missing ⚠️
src/NestedArchimedeanCopula.jl 93.54% 6 Missing ⚠️
src/nested/NestedArchimedeanFitting.jl 93.42% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #367      +/-   ##
==========================================
+ Coverage   78.28%   79.21%   +0.93%     
==========================================
  Files          84       88       +4     
  Lines        5106     5508     +402     
==========================================
+ Hits         3997     4363     +366     
- Misses       1109     1145      +36     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@thisiscam
Copy link
Copy Markdown
Contributor Author

  1. I understand why you want to do that, but our interface should already cover the case fully through ϕ, ϕ⁻¹, ϕ⁽¹⁾, ϕ⁻¹⁽¹⁾, ϕ⁽ᵏ⁾ and ϕ⁽ᵏ⁾⁻¹. Do you really need ϕ⁻¹⁽ᵏ⁾ for k > 1 ?
    (a) You do need ϕ⁻¹⁽ᵏ⁾ for k > 1: Then maybe we should provide a generic interface so that it is available to all generators, and can be overwritten, as we did for others.

Yes, we'd need $\textrm{taylor1}(\phi_{\text{outer}}^{-1} \circ \phi_{\text{inner}}, d) $.
A more robust way to fix this, perhaps, is to allow registering custom ops in TaylorSeries.jl, since the op that misses treatment is log1mexp.
Let me know what you think?

thisiscam and others added 10 commits May 29, 2026 21:24
Add the density kernels for nested (hierarchical) Archimedean copulas in
src/nested/. The nested-copula density is the mixed partial of the nested
CDF; differentiating the composition of generators is Faà di Bruno's
formula, carried here by partial Bell polynomials expressed through
truncated Taylor series:

  * _composition_taylor — Taylor coefficients of ϕ⁻¹_outer ∘ ϕ_inner,
  * _faa_di_bruno_coeffs — partial-Bell-polynomial step per child block,
  * _cauchy_product — convolution of sibling contributions,
  * _phi_deriv / _assemble_density — contraction against ϕ⁽ᵏ⁾ of the
    outer generator,
  * _process_node / _leaf_log_jacobian / _nested_logpdf — the recursion,
    with censored leaves shifting the argument but not the differentiation
    order (the per-variable survival likelihood).

Follows the algorithm of Yang & Li, arXiv:2605.23134 (2026).

Built only on the public Generator interface (ϕ, ϕ⁻¹, ϕ⁽ᵏ⁾, ϕ⁻¹⁽¹⁾); no
parallel generator system. Generic in the value type with a Float64
default; BigFloat / Double64 coordinates flow precision through the whole
recursion for adversarial high-d / deep-tail inputs.

FrankTaylorGenerator.jl adds Taylor1-compatible ϕ / ϕ⁻¹ methods for the
Frank generator (its default log1mexp form has no Taylor1 method), using
the equivalent expm1/log1p closed forms; the scalar ϕ path is untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add the public NestedArchimedeanCopula{d,TG} <: Copula{d} and wire it into
the module (include + export). Follows the algorithm of Yang & Li
(arXiv:2605.23134).

  * Unified constructor NestedArchimedeanCopula(G; leaves=, children=):
    children may be ArchimedeanCopula OR NestedArchimedeanCopula, nesting
    to arbitrary depth; children are auto-placed on consecutive free dim
    blocks or pinned with `child => dims`. A purely flat declaration
    (leaves only) returns the native ArchimedeanCopula fast path. The legacy
    positional form NestedArchimedeanCopula(G, children) still works.
  * Distributions._logpdf — uncensored nested density (element-typed).
  * logpdf(C, u; censored=δ) — optional per-variable right-censoring mask:
    the mixed partial of the nested CDF over observed dims only (the
    survival likelihood); default (omitted) is the plain density.

No nesting-validity check: the caller supplies a valid parameter
combination.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add test/NestedArchimedeanCopula.jl (wired into runtests.jl) covering:

  * flat declaration dispatches to the native ArchimedeanCopula with a
    bit-for-bit identical logpdf;
  * uncensored density vs an INDEPENDENT reference — the nested CDF
    assembled from the generators and mixed-partial-differentiated by
    nested ForwardDiff (no shared code path with the Faà di Bruno
    recursion), same-family and heterogeneous trees;
  * uncensored density vs an EXTERNAL acopula reference (log-likelihoods
    committed under test/data/nested/), Clayton/Gumbel/Frank, d = 10, 20;
  * censored likelihood (logpdf with a mask) vs the ForwardDiff reference,
    incl. the bivariate Clayton closed form and the omitted-mask ==
    plain-density default;
  * arbitrary-depth nesting;
  * constructor validation (bad dim tiling) and a fit/smoke usage.

Adds DelimitedFiles to the test target (with its [compat] entry, so Aqua's
dependency-compat check stays green) for reading the reference CSVs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add docs/src/manual/nested.md (linked in docs/make.jl) covering the
definition, building trees (auto/pinned dims, root leaves, mixed families,
arbitrary depth), the Float64/BigFloat precision story, and the optional
per-variable censored / survival likelihood via the `censored` keyword of
`logpdf`. Follows the algorithm of Yang & Li (arXiv:2605.23134).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move the per-variable right-censored survival likelihood to the SklarDist
level, where the margins and data live (a Copula{d} stays the pure
dependence on [0,1]^d). The copula-scale mixed partial remains the internal
kernel that SklarDist calls, and is exposed as the lower-level
`censored_logpdf`.

  * logpdf(S::SklarDist, x; censored = δ, T = Float64): survival
    log-likelihood = Σ_{i observed} log f_i(x_i) + log[ mixed partial of
    the copula CDF over the OBSERVED coords at u = (F_i(x_i)) ]. Omitted /
    all-false mask reduces to the plain joint density; all-true reduces to
    log cdf(S, x). Does NOT route through Distributions.censored (that maps
    the censoring atom to the copula boundary u=1 and returns -Inf).
  * censored_logpdf(C::Copula, u, δ; T = Float64): copula-scale mixed
    partial over observed coords, GENERAL over Copula{d}. Implemented for
    flat ArchimedeanCopula (via ϕ⁽ᵏ⁾) and NestedArchimedeanCopula (via the
    Faà di Bruno recursion); any copula supplying _censored_copula_logpdf
    gains the SklarDist survival likelihood for free.
  * NestedArchimedeanCopula loses its logpdf(...; censored) kwarg method
    (kept pure); gains an analytic Distributions._cdf (ϕ_root ∘ Σ ϕ⁻¹) so
    cdf no longer falls back to slow numerical integration.

Default working precision is Float64 (matching the uncensored path);
BigFloat is opt-in via T= for adversarial high-d / deep-tail inputs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Tests (test/NestedArchimedeanCopula.jl, 45 pass):
  * censored_logpdf (copula scale): bivariate Clayton closed form; FLAT
    ArchimedeanCopula via ϕ⁽ᵏ⁾ (general non-nested path); nested across
    sectors vs ForwardDiff reference; all-observed == density; all-censored
    == log cdf.
  * SklarDist survival likelihood: bivariate Clayton closed form on the data
    scale; omitted mask == plain joint density; FINITE where the
    Distributions.censored route returns -Inf; all-censored == log cdf(S, x);
    nested factor == observed-margin densities + copula censored_logpdf.

Docs (docs/src/manual/nested.md): the survival likelihood section now uses
logpdf(SklarDist(C, margins), x; censored = δ), with the math, the
"don't use Distributions.censored" warning, and the note that it is general
over copulas; censored_logpdf documented as the copula-scale building block.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the local `_phi_deriv` (generic Taylor-based ϕ⁽ᵏ⁾) with calls
to upstream `ϕ⁽ᵏ⁾(G, k, t)` in `_assemble_density` and in the
censored-likelihood helper. Rewrite `_composition_taylor` as a thin
wrapper around upstream `taylor(f, x₀, d)`. No parallel derivative
machinery; closed-form per-family ϕ⁽ᵏ⁾ overrides now win dispatch
automatically.

`FrankTaylorGenerator.jl` retained (still needed by
`_composition_taylor` for nested-Frank trees, since `log1mexp` has no
Taylor1 rule); docstring updated to reflect the narrower role.

Tests: same 24/26 pass as before the refactor (the 2 unchanged errors
are a pre-existing `rng` test-harness scoping issue, identical on
upstream/main).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@thisiscam thisiscam force-pushed the nested-archimedean branch from 3cf02b1 to 845ab0c Compare May 30, 2026 01:26
…poke CensoredLikelihood

Make NestedArchimedeanCopula a first-class citizen of the conditioning/
subsetting framework so per-variable censoring is an emergent capability of
the standard API rather than a bespoke function.

New src/nested/NestedConditioning.jl:
- SubsetCopula(::NestedArchimedeanCopula, dims): prune the generator tree to
  the observed marginal (drop 0-observed subtrees, collapse 1-observed
  subtrees to a bare leaf by single-coordinate marginalisation-invariance,
  keep >=2), re-index to 1:p; collapse to a flat ArchimedeanCopula when no
  sub-nesting survives. Its existing _logpdf gives the observed-marginal
  density via the Faa-di-Bruno walk.
- NestedDistortion + DistortionFromCop(::NestedArchimedeanCopula, js, ujs, i):
  closed-form conditional marginal whose cdf is the mixed partial over the
  conditioned set / observed-marginal density, routed through the O(d^2) tree
  walk (not ForwardDiff). General p, so it also accelerates the per-coordinate
  distortions the generic ConditionalCopula builds.

Per-variable censoring is now the "gist recipe"
  logpdf(subsetdims(X,O), x_O) + logcdf(condition(X,O,x_O), x_C).

- Delete src/CensoredLikelihood.jl (bespoke censored_logpdf /
  logpdf(::SklarDist; censored=)); keep the internal _censored_copula_logpdf
  nested kernel as the numerator behind the distortion.
- Fix _cdf(::NestedArchimedeanCopula) to flow Real/ForwardDiff.Dual eltypes
  (was forcing Float64, which crashed the generic multi-censored fallback).
- Retarget the censoring tests to the gist recipe against the same
  independent references; fix the pre-existing rng-scoping bug. 50/50 pass;
  the JAX-CSV bit-identity regression is untouched.

The multi-censored-dim conditional CDF (|unobserved|>=2) stays on upstream's
generic ForwardDiff fallback (correct, slower) — ConditionalCopula stores the
inner copula as a field, not a type param, so it cannot dispatch on the inner
type; a fast multi-dim path is a documented follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…walk

The single-censored conditional and the observed-marginal density already used
our nested kernel, but the MULTI-censored-dim conditional CDF (|censored| >= 2)
went through SklarDist(ConditionalCopula, …), whose _cdf calls the generic
_partial_cdf — i.e. upstream's nested ForwardDiff over the closed-form CDF. That
nests one ForwardDiff.derivative per observed coordinate: Dual-of-Dual type
explosion at compile time, O(2^k) at run time, infeasible in high dimension.

Add a _partial_cdf(C::NestedArchimedeanCopula, is, js, uᵢₛ, uⱼₛ) specialisation
that returns exp(_censored_copula_logpdf(C, assembled_u, cens, T)), the order-|js|
mixed partial via our polynomial Faà di Bruno tree walk. ConditionalCopula stores
its inner copula in an abstract field but _cdf calls _partial_cdf(CC.C, …), which
dispatches on the runtime type (concretely nested), so the override is selected
without ConditionalCopula carrying the inner type. Net: the gist recipe routes
through our algorithm for ANY number of censored coordinates — zero ForwardDiff.

- Tests 50/50 -> 58/58: tighten the multi-censored gist tolerance and add a
  direct-kernel-equality + CDF-contract (monotone, [0,1]) testset. JAX-CSV
  bit-identity regression untouched and green; full ConditionalDistribution.jl
  suite green (non-nested conditioning/sampling unaffected).
- Docs/comments updated: multi-censored now uses the tree walk for any censoring
  count; precision caveat (Float64 high-order fast-tail -> BigFloat) noted.

Caveat (documented): end-to-end BigFloat *through* condition() is not yet enabled
— upstream's ConditionalCopula/DistortionFromCop store Float64 fields; BigFloat
reaches the kernel via direct calls or the single-censored path.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@lrnv
Copy link
Copy Markdown
Owner

lrnv commented May 30, 2026

So you do need ϕ⁻¹⁽ᵏ⁾, k > 1: the higher derivatives of the inverse of the generator. The right thing to do would then be to add the taylor1() generic version of this function to the list of potentially-overwrittable functions in Generator.jl, and then maybe try to fill out its value for the few generators where it is known ? Also in the docs on the generator page there's a mention of the list of potentially-overwritable functions, where it hsoul dbe added.

Then, you code should not use the taylor1() function but rely on the new ϕ⁻¹⁽ᵏ⁾, function, so that if a generator has a faster version (or a version that handles better extreme cases maybe) it'll be used directly.


Regarding log1mexp, the right thing to do long-term is to propose a package extension to TaylorSeries.jl that handles LogExpFunctions.jl. I already mentioned it there a while ago but never got the time to do it.

Adding the two methods you proposed is fine as a temporary measure.

I feel like those methods belong to the Frank generator file and not really their own file, maybe you could move them there ?


Can you please add some examples of nested archimedean copulas to the GenericTests.jl file ?


Comment thread src/nested/FrankTaylorGenerator.jl Outdated
@@ -0,0 +1,33 @@
# =============================================================================
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The content of this file should be moved to the file of the frank generator

function _composition_taylor(outer::Generator, inner::Generator, t₀::T, d::Int) where {T}
coefs = taylor(x -> ϕ⁻¹(outer, ϕ(inner, x)), t₀, d)
return T[coefs[k+1] for k in 1:d]
end
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, the derivatives of ϕ⁻¹(outer, ϕ(inner, x)) should IMHO be obtained by manually running the faa-di-bruno formula leveraging the methods we have for the derivatives of these two functions : computing these derivatives is sometimes challenging in term of precision, and the direct taylor() version does not use the potentially overwritten version of those.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I deliberatedly chose the current version because 1) it's simple and clear on paper 2) NAC density most likely requires BigFloat in most cases, due to the convolution-of-the-taylor-polynomials step requiring a wide range of precision.
That said, a more robust version we had in our preprint and also the JAX implementation was based on implicit differentiation (in Appendix A.4), which avoids the high-order derivative of the inverse.

Do we want that implemented?

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, its better to use the higher-order derivatives of the inverses as it allows implementers of generators to specify them if they want. Can you leverage the implementation of FdB/Bell that you already have ?

Copy link
Copy Markdown
Contributor Author

@thisiscam thisiscam Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1. I tried FdB method and found it is broken because of catastrophic cancellation at the FdB summation. On a Frank/Frank example, the derivative of the nesting becomes completely incorrect, even after feeding in highly-accurate high-order derivatives using BigFloat-512.
2. The direct jet and the implicit solver are both fine and equally accurate. Max rel.
error of the edge-link coefficients vs a 2048-bit reference:

  precision | direct    implicit   fdb
  Float64   | 2.3e-12   2.9e-12    1.2e+00
  64-bit    | 9.4e-16   4.2e-16    2.0e-04
  96-bit    | 2.1e-25   7.7e-25    1.3e-13

This is on Frank/Frank. From my experience with the JAX package, certain nesting $\phi_1^{-1}\circ\phi_2$ has mathematical cancellations so that the direct method will also fail numerically while the implicit method will stay accurate since it completely avoids calculating the high-order derivative of the inverse.

3. IMHO the productive fix for this class of numerical issue is to let the user override the composition's Taylor calculation directly (composition_taylor). Then, per generator-pair, they can pick the naive taylor method, the implicit method, or a taylor1 over a hand-simplified composition ----- maximal flexibility to retain precision. I have just updated this PR to include this hook --- and we have all three methods: direct, implicit, FdB implemented as user-specifiable options.

Repro (self-contained, plain Copulas.jl): https://gist.github.com/thisiscam/8fa9a4ea2b2dffa474cd4ea131efdc21

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done here.

Thats looks great. In the documentation where you explain the problem and the possibility to overwrite the default choice, can you add a proper example while you make the argument, starting with the case of someone just trying to compute density of a NAC and obtaining 1) good results for e.g. clayton/clayton, then 2) switching to frank and obtaining bad results (show diff with bigfloats) and 3) using the hook to fix it, and 4) even proposing to implement it yourself through the third option.

That would help clarify things for a user.

EDIT: The faa-di-bruno version should be removed then since its performing so poor.

Comment thread src/nested/NestedConditioning.jl Outdated
_kid_dims(ch::NestedArchimedeanCopula) = ch.dims

function SubsetCopula(C::NestedArchimedeanCopula{d, TG}, dims::NTuple{p, Int}) where {d, TG, p}
# `subsetdims` already short-circuits p==1 (Uniform) and p==d (C unchanged),
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subsetdims indeed short-circuits p==1 (Uniform) and dims == 1:d (unchanged) but not every p==d (reordering)

thisiscam and others added 5 commits May 31, 2026 23:56
SubsetCopula(::NestedArchimedeanCopula, dims) relabelled the pruned marginal
by SORTED surviving dims, ignoring the requested order. For a non-exchangeable
nested structure this returned the wrong marginal under a reordered subset
(e.g. subsetdims(C, (3,1)) gave the (1,3) copula). Relabel by the requested
`dims` order instead. (Regression test lands with the nested test-suite update.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Relocate the Taylor1-specialised ϕ/ϕ⁻¹ methods for FrankGenerator (needed by the
direct edge-composition path because Frank's θ>0 scalar forms use log1mexp, which
has no Taylor1 method) from the standalone src/nested/FrankTaylorGenerator.jl into
src/Generator/FrankGenerator.jl, and drop the now-unused include. No behaviour
change. (A future TaylorSeries⊕LogExpFunctions extension giving log1mexp(::Taylor1)
would retire these; the implicit edge-composition method already avoids them.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ethod)

The per-edge Taylor expansion of ϕ⁻¹_outer ∘ ϕ_inner now goes through an
overloadable hook composition_taylor(outer, inner, t₀, d), defaulting to
_composition_taylor_direct (the current jet-over-composition, byte-identical).
Ships a second building block, _composition_taylor_implicit (the implicit-
equation method of Yang & Li App. A.4 / acopula): it solves ϕ_outer(h)=ϕ_inner
order-by-order using only forward derivatives ϕ⁽ᵏ⁾ and a scalar ϕ⁻¹_outer, never
differentiating the inverse and never composing generators on a single jet.

Users select a method or register a closed form per generator pair the same way
they override ϕ⁽ᵏ⁾ — by dispatch, no keyword. Ships the Clayton/Clayton closed
form as the worked example (more accurate than the direct jet: 4.9e-77 vs 8.1e-77
rel-err vs a 512-bit reference). Default path is bit-identical; implicit matches
direct to ~1e-15 (Float64) / ~1e-30 (BigFloat) across families and on full nested
densities. Docs section shows how to switch methods and add custom compositions.

Also carries the nested test-suite update (edge-composition tests + the
subsetdims-reordering regression test): 58 -> 92 assertions, all passing; the
JAX-CSV bit-identity regression is untouched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- fit(CopulaModel, C0, U) / fit(C0, U): MLE of the generator parameters holding the tree structure fixed (unconstrained reparam + LBFGS); coef/coefnames/dof read from the fitted tree so AIC/BIC are correct.

- Distributions._rand! via inverse-Rosenblatt; _logpdf promotes the working type so ForwardDiff generator params flow through Float64 data.

- Two nested Clayton entries added to the GenericTests bestiary; fit-recovery / subsetdims-order / rand / coef tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- ϕ⁻¹⁽ᵏ⁾(G, k, t): overridable kth derivative of the inverse generator (generic Taylor-jet default; exact Clayton closed form).

- _composition_taylor_fdb: a third selectable edge composition behind the composition_taylor hook, assembling ϕ⁻¹_outer∘ϕ_inner from ϕ⁻¹⁽ᵏ⁾(outer) and ϕ⁽ᵏ⁾(inner). Opt-in only; the default remains the direct jet.

- Note: catastrophically ill-conditioned for fast-tail generators (Frank/Joe) at high d — kept as a selectable option, not the default. See PR discussion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
function _composition_taylor(outer::Generator, inner::Generator, t₀::T, d::Int) where {T}
coefs = taylor(x -> ϕ⁻¹(outer, ϕ(inner, x)), t₀, d)
return T[coefs[k+1] for k in 1:d]
end
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done here.

Thats looks great. In the documentation where you explain the problem and the possibility to overwrite the default choice, can you add a proper example while you make the argument, starting with the case of someone just trying to compute density of a NAC and obtaining 1) good results for e.g. clayton/clayton, then 2) switching to frank and obtaining bad results (show diff with bigfloats) and 3) using the hook to fix it, and 4) even proposing to implement it yourself through the third option.

That would help clarify things for a user.

EDIT: The faa-di-bruno version should be removed then since its performing so poor.

Comment thread src/nested/NestedArchimedeanDensity.jl Outdated
k < d && (gpow = _poly_mul_inc(gpow, g, d)) # ĝᵏ⁺¹, truncated to order d
end
return h
end
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this shoudl be removed then, thanks for trying out

Comment thread src/nested/NestedArchimedeanDensity.jl Outdated
h[k] = (θ_in_pow / θ_out) * binom * base^(r - k)
end
return h # [h₁,…,h_d], length d, no constant
end
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this function should move to the ClaytonGenerator file, with correct comments on top explaining what it should compute

# dispatches on a TEMPLATE INSTANCE `C0::NestedArchimedeanCopula` that carries the
# full tree — neither `fit(NestedArchimedeanCopula, U)` (a bare type) nor
# `fit(typeof(C0), U)` can rebuild the tree, since the type parameters
# `{d, root-TG}` encode only the dimension and the root generator family.
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand why you did this, otherwise MLE is not possible. I kinda agree with you, but this shoul dbe in the docs too : maybe you could add an example of fitting a NAC on some dataset ? inside the example parts of the documentation

thisiscam and others added 3 commits June 1, 2026 16:33
fit() optimises through a parametrisation `α -> NestedArchimedeanCopula` decoupled from the generator objects: default (per-generator, unchanged); enforce_nesting=true (same-family θ-ordered trees parametrise each child's θ as a non-negative increment over its parent, so every optimiser step is a valid nesting); or a fully custom reparam=(α->C), init=α₀ (share parameters, fit on another scale, encode any constraint) — ForwardDiff differentiates straight through it.

dof now reads the free-parameter count (nparams) so AIC/BIC stay correct even when a custom reparam shares parameters.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ride to its generator file

Remove _composition_taylor_fdb + _poly_mul_inc and the ϕ⁻¹⁽ᵏ⁾ interface (generic + Clayton closed form): the FdB route is catastrophically ill-conditioned (per-PR discussion) and was the only consumer of ϕ⁻¹⁽ᵏ⁾. The direct (default) and implicit methods remain behind the composition_taylor hook.

Move the closed-form Clayton/Clayton composition_taylor override into Generator/ClaytonGenerator.jl, with a header comment explaining what it computes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a Fitting section (MLE on a fixed tree + the three parametrisation modes: default / enforce_nesting / custom reparam) and rewrite the edge-composition section around the three selectable methods (direct default / implicit / closed-form per pair). Method choice is about availability and speed; BigFloat is the precision fix for deep-tail / high-d inputs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
thisiscam and others added 6 commits June 1, 2026 16:41
…ied reparam

The decoupled parametrisation already lets the caller encode any constraint via `reparam`; a built-in enforce_nesting hard-coded one constraint and re-introduced a per-family whitelist into fit(). Remove it and its helpers; the docs and a test now demonstrate nesting as a user-written reparam (child θ = parent θ + softplus(δ)).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A custom parametrisation no longer needs a dummy template copula: `fit(CopulaModel, reparam, init, U)` takes the map and its initial point directly and infers the dimension from `reparam(init)`. Distinguished from the template form `fit(CopulaModel, C0, U)` by arity; `fit(C0, U)` remains a quick shim. (No untyped `fit(reparam, init, U)` shim — that would be type piracy on Distributions.fit.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ote, update fit examples

The underbrace texttt labels broke MathJax (subscripts + thin-spaces inside texttt); use plain text labels and give the code mapping in prose. Remove the Distributions.censored warning. Update the fit examples to the template-free fit(CopulaModel, reparam, init, U) form.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename _composition_taylor_direct/_implicit to composition_taylor_direct/composition_taylor_implicit (users select them via the composition_taylor hook), and give the hook + both methods docstrings — including the DIY-with-taylor recipe.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…upports p==d via lrnv#369)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@thisiscam thisiscam marked this pull request as ready for review June 2, 2026 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants