splinediff's s search seeds don't scale with signal length or noise

[pavelkomarov]

Another warning made visible by #206 scoping optimize()'s silencing to the call itself. Notebook 4's sweep produces a steady stream of:

scipy/interpolate/_fitpack_repro.py:993: RuntimeWarning: error. a theoretically impossible result
was found during the iteration process for finding a smoothing spline with fp = s.
probably causes : s too small.

scipy is right: s really is too small, and the search space is why.

The problem

splinediff's s is an absolute residual budget — sum_t (x[t] - spline[t])**2 <= s — so the natural value scales with both signal length and noise variance, roughly N * sigma**2. But the search space is a fixed list:

{'degree': {3, 4, 5}, 's': [0.2, 0.5, 0.75, 0.9, 1, 10], 'num_iterations': [1, 5, 10]}
bounds: {'s': (0.01, 10000.0)}

Sweeping s on a fine geometric grid and taking the best objective value:

case	natural N*sigma**2	best s (objective)	best s (true RMSE)
lorenz, noise scale 1	4.0	3.49	3.49
lorenz, noise scale 4	64.0	57.90	57.90
sine, noise scale 1	4.0	3.49	3.49
sine, noise scale 4	64.0	57.90	57.90

The optimum tracks N*sigma**2 closely (~0.87x), and the objective picks the same s as true RMSE does, so the surrogate is working fine. The seeds are the problem: they happen to bracket the optimum at N=400, sigma=0.1 — exactly the default config — and are entirely below it at higher noise. At noise scale 4 every seed is <= 10 while the optimum is ~58, so the search starts wholly inside the regime scipy is complaining about.

Options

1. Widen the seeds geometrically, e.g. [0.5, 2, 8, 32, 128]. One line, but still wrong for a different N.
2. Derive the seeds from the data — multiples of N * sigma_hat**2, with sigma_hat from something like mad(diff(x)). Correct for any N and noise level, and keeps s a genuine hyperparameter rather than pinning it. Needs optimize() to build a data-dependent search space instead of reading the static dict, which is new plumbing but has precedent in how tvgamma is derived from cutoff frequency.
3. Leave it if Nelder-Mead reliably climbs out from the seeds, in which case this is only noise in the logs.

Option 2 looks right. Setting s inside splinediff itself would fix the warning but foreclose optimizing over it, so that's not the way.

Unresolved: whether Nelder-Mead actually reaches ~58 from seeds capped at 10, which decides how much of this is cosmetic versus splinediff being under-served at high noise in the benchmark.

---

🤖 Written by Claude Code on behalf of @pavelkomarov