Releases: anindex/polystep
Releases · anindex/polystep
Release list
v0.9.0
Added
SparseDeltaEvaluator: a full-space candidate replacesparticle_dimscalars, so its
output differs from the base in at most that many columns. Propagates the delta through
elementwise layers and goes dense oneLinearpast the perturbed one, instead of a
densebmmoverNweight sets. Automatic fornn.SequentialofLinearand
elementwise layers, with a per-chunk fallback.FactoredSubspace: perturbs each 2D parameter bydW = A @ BwithBfixed, so
x (W + A B)^T = x W^T + (x B^T) A^Tand no candidate weight is built. Not the
default: the fixedBconfines perturbations torankinput directions.step(..., objective_token=...)marks which objective a step measures, so cost rows
cannot be reused across minibatches.release_evaluator(),resync_from_model(),
MinCostGreedySolverandTopKMeanSolverare public.MANIFEST.in, so the sdist ships a runnable test suite. CI runs it from the tarball.
Fixed
api.train()never calledregister_evaluator, the only assignment of
_cost_evaluator,_fused_inputs,_factored_evaluatorand_sparse_delta_evaluator,
so every fast evaluation path was unreachable through the training API.- Cost-row reuse under
adaptive_probeswas per particle, but a candidate is the whole
configuration with one row replaced, so a stagnant particle's row was still measured
against every other particle's position. Reuse is now all or nothing. multifidelity_screenaveraged direction contrast across particles, which carry
independent rotations, so the kept set was chosen at random. Ranked per particle now.- Tied weights raised on both evaluation paths.
batch_unflattenemits the canonical key
only;unflattenstill carries aliases forload_state_dict. restore_bestkeyed on the OT cost when no callback was registered, restoring the step
with the cheapest probe cloud rather than the best weights. It also snapshots to CPU.- Chunk size ignored the weight set a non-fused subspace builds per candidate and the vmap
activations. A 120K-param step peaked at 1661 MB in full space and 10565 MB in subspace
mode; now 946 MB and 1461 MB. AdaptiveSubspaceand per-layer rotation replaced the projection while the coordinates
were non-zero, moving the weights with nothing evaluated behind them.CMAAdaptiveSubspacenever rotated or absorbed, left its hyperparameters at0.0so
the covariance update was inert, normalized the evolution paths bysigmainstead of
step_radiussoC_diagdecayed to its floor, carried state through basis rotations
untransformed, and accumulated at bf16 where1 + c_1*xrounds back to1.- Anderson acceleration subtracted
dXinstead ofdX + dR, soanderson_depth > 0
converged slower than no acceleration. num_particles = 1froze the optimizer: balanced OT with one row forces a uniform plan.
The default now selectsSoftmaxSolverand warns.scale_costdivided before recentering, so adding a constant to every cost changed the
plan.SinkhornSolveralso reportedent_reg_costin the wrong frame; the plan and
duals were unaffected. The greedy and top-k solvers had the same reporting bug.- Softmax paths recentred by the global minimum, so a row above it underflowed to
-inf
and returned NaN; the fused kernel did not recenter at all. sanitize_costmapped-infto the worst finite value instead of the best, and used an
absolute penalty that entered thescale_cost='mean'reduction.state_dictcaptured onlySolverState, so resume was not exact foramortize_steps,
adaptive_probes,trust_regionorHybridSubspace. Format is 3; format 2 loads with
a warning.- A stagnation absorb never cleared
stagnation_count, redrawing the basis every step on
a plateau.resync_from_model()left momentum and rotation state on the old anchor. - Probe losses accumulated in FP32 unconditionally, so a float64 objective whose costs
differ below FP32 resolution produced a constant cost matrix. - Block-wise biased rotation raised on bf16 under
mixed_precision=True. A zero descent
direction produced a singular rotation collapsing+e0and-e0. - Smaller: the in-place evaluator dropped unknown parameter keys silently; the
trust-region ratio scored a Newton step that was never applied; displacement history
mixed bases; randomized PCA drew from the global RNG; rank transitions dropped
HybridSubspacesettings;data_dependent_inithad the wrong sign; a float64 model
with a subspace raised; a frozen model gave an opaque error;pyproject.tomldeclared
torch>=2.4where 0.2.0 announcedtorch>=2.8.
Changed
adaptive_probesandadaptive_num_probedefault on underblock_strategy='monolithic'.use_csais refused with a warning: the OT step is a deterministic descent direction, so
CSA reads agreement as the normal state andsigmagrows without bound.multifidelity_screencuts the forward budget instead of only reweighting, and runs only
whenscreen_fidelity/num_probe + screen_keep_ratio < 1. Needs a cheap closure from
screen_closure_from();api.trainsupplies one.- Block-wise warns for options it ignores;
groupedwarns when combined with a subspace.
LinearSubspaceamplifies rather than dilutes the perturbation, as its docstring said. _step_blockwiseclaimed blockwise cuts OT cost fromO(N^2)toO(N^2/L). The forward
count is identical;Lsolves overP/Lrows replace one overP.- Removed
get_sobol_rotation_matrices,HybridSubspace.reconstruct_baseand
reconstruct_batch_delta: no callers, no coverage.
Performance
thin_qrpins basis construction to one thread:HybridSubspacesetup for the MNIST
example drops from 27.0 s to 97.8 ms.apply_biased_rotationuses modified Gram-Schmidt
instead oftorch.linalg.qr, 7-33x faster at these sizes. Compiled kernels take radius
and epsilon as 0-d tensors, so a scheduled radius no longer recompiles every step.- Example 06 on an RTX 5090: 1024 s to 845 s, shift-recovery +9.3 pp to +10.7 pp.
Tests, examples and docs
- 967 tests in 262 s with an intermittent timeout failure, now 941 in 8 s with none.
tests/test_subspace_contract.pyreplaces per-class copies of the properties every
subspace shares. Removed tests that could not fail. Suite verified by mutation: eight
semantic changes to the source are all caught. tests/test_nondiff_data.pyguarded nine tests on thepysatimport but not on the
experiments/tree they import from, so they failed rather than skipped in the sdist
when pysat happened to be installed.- Examples pin CPU threads, check optional imports before use instead of failing after the
run, and register the evaluator in hand-rolled loops.python-satmoved into the
examplesextra, which example 04 needs. release.ymllint and test commands had drifted fromci.ymland it lacked the CPU
torch index; both now match. README and the docs corrected where they had drifted.