Conversation
The structured-grid search anchored its every_n_cell stride to the full array shape, so a zone whose active cells never lined up with that fixed grid (or a zone smaller than one stride step) could be silently skipped entirely, leaving par_info empty and crashing on pd.concat([]). Rework the search to anchor per zone instead of the whole array, with a bucketed, aspect-ratio-aware fallback (capped and decimated toward the requested density) for zones the direct stride still misses or covers unevenly. Add a hard MIN_KRIGE_PPOINTS floor (raise) since <3 points can never support kriging, and warnings for fallback engagement and persistent under-delivery relative to what every_n_cell requested. Add autotest/investigate_pp_search_impact.py, a standalone comparison tool used to validate the fix and its warning thresholds against several synthetic and real-model ibound scenarios. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #710 +/- ##
===========================================
+ Coverage 76.33% 76.38% +0.04%
===========================================
Files 38 38
Lines 19872 19907 +35
===========================================
+ Hits 15170 15206 +36
+ Misses 4702 4701 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| @@ -137,17 +148,6 @@ def setup_pilotpoints_grid( | |||
| ycentergrid = np.reshape(ycentergrid, (ycentergrid.shape[0], 1)) | |||
|
|
|||
|
|
|||
There was a problem hiding this comment.
This was to handle xsection models...is that covered in the new functionality?
There was a problem hiding this comment.
Old mate has added some tests for the single row and single col array use-case. They cover of xsection?
jtwhite79
left a comment
There was a problem hiding this comment.
Looks good to me...as long as the support for xsection models is retained...
…ints_grid Addresses PR review comment asking whether develop's "fix for x-section models" special case is still covered after the per-zone bounding-box rewrite; these tests confirm the new thin-zone/midpoint logic subsumes it without further changes to pp_utils.py. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
| ) | ||
|
|
||
|
|
||
| def test_setup_pp_xsection_single_row(tmp_path): |
There was a problem hiding this comment.
@jtwhite79 , added these for xsection usage
|
@briochh does the pp setup now fail if < 3 pps are found in a given zone? I think even 1 pps in a zone is mechanically ok - every cell just gets the single scalar value. Is the reason to force >= 3 conceptual? |
|
Yeah fair call 3 is purely conceptual. Claude is a little officious with these assertions: "minimum for ordinary kriging to be well-posed". |
Summary
pyemu.pp_utils.setup_pilotpoints_grid()'s structured-grid search anchored itsevery_n_cellstride to the full ibound/zone array shape, rather than to each zone individually. As a result, a zone whose active cells didn't happen to line up with that fixed, globally-anchored grid could be silently skipped entirely — even though the zone had plenty of valid (>0) cells. When this happened for every layer/prefix in a call,par_info(built up as a list) stayed empty and:raised an opaque
ValueError: No objects to concatenate, with no indication that the real cause was a zone geometry / stride mismatch.This PR reworks the structured-grid search to anchor per-zone instead of per-array, adds a shape-aware fallback for zones the direct search still misses or covers unevenly, and adds explicit validation (a hard floor + warnings) around how many pilot points a zone actually ends up with, since "how many points is enough to support kriging" was previously never checked at all.
Root cause
pp_utils.py's row/col search loop (for i in range(start_row, ib.shape[0] - start_row//2, every_n_cell): ...) computed its stride start/stop purely from the array's shape, so a zone confined to (or shaped within) a sub-region of that array had no guarantee any candidate location would land on one of its cells.pd.concat([])crash if it happened for every layer/zone.Fix
In
pyemu/utils/pp_utils.py, the structured-grid branch ofsetup_pilotpoints_grid()now:iboundvalue whenuse_ibound_zones=True, or the single "all active cells" group otherwise), the search computes the zone's own bounding box and anchors theevery_n_cellstride to it, rather than to the full array. This is mathematically identical to the old behavior whenever a zone's bbox equals the full array (the common case — verified against all pre-existing tests).<= every_n_cell, that axis just uses its midpoint instead of striding (this also subsumes the old, separate x-section-model special case).len(hits) < max(MIN_KRIGE_PPOINTS, n_requested * PPOINT_UNDER_DELIVERY_FRACTION)— i.e. either below the hard floor, or well below whatevery_n_cellnominally requested for that zone. This catches both "zone entirely missed" and "zone unevenly covered" (e.g. only one part of a multi-segment zone got sampled even though the total hit count looked acceptable).sqrt(target) x sqrt(target)grid, which breaks for elongated/1D-like zones) so a large zone that aliases against the stride doesn't explode into one pilot point per active cell, while a small zone gets properly represented instead of raising unnecessarily.MIN_KRIGE_PPOINTS = 3(the minimum for ordinary kriging to be well-posed) is now a hard floor: if even the fallback can't produce 3 points for a zone/layer,setup_pilotpoints_grid()raises a clear exception naming the zone/layer/active-cell-count, instead of the previous silent under-determined result (or the opaquepd.concat([])crash if this happened everywhere).PyemuWarning) fire when the fallback engages (reporting direct vs. requested vs. final counts) and when a zone still under-delivers relative to what was requested even after using every one of its active cells (i.e. the zone genuinely doesn't have enough active area, and nothing more can be done).Testing
Added to
autotest/utils_tests.py:test_setup_pp_misses_offgrid_active_zone— reproduces the original bug (a zone whose active cells don't align with the global stride) and confirms it's no longer dropped.test_setup_pp_too_few_points_raises— a single-active-cell zone raises a clear exception instead of silently returning 1 point.test_setup_pp_small_zone_uses_fallback_instead_of_raising— a small (3x3) but not-too-small zone uses the fallback instead of raising.test_setup_pp_under_delivery_warns— a sparse zone that clears the floor but is far below the requested density warns rather than silently under-delivering.test_setup_pp_dense_fallback_is_capped— a large zone that fully aliases against the stride gets capped/decimated, not exploded into one point per active cell.test_setup_pp_uneven_coverage_triggers_fallback— a two-segment zone that clears the point-count floor via the direct search, but with all hits clustered in one segment, now gets coverage in both segments.All pre-existing pilot-point tests (
test_setup_pp,test_ppu_geostatsinutils_tests.py;shortname_conversion_testandtest_hyperpars— the one existing test exercisinguse_pp_zones=True— inpst_from_tests.py) continue to pass unchanged, confirming the fix is a no-op for the common case (verified the stride math reduces to exactly the old formula whenever a zone's bbox spans the full array).Also added
autotest/investigate_pp_search_impact.py, a standalone dev/investigation tool (not part of the pytest suite) comparing the pre-fix search against the current one across several synthetic and real-model (Freyberg)iboundscenarios, with a plot overlaying the zone raster and old-vs-new pilot point locations for each. This was used throughout development to validate the fix's impact and tune the warning thresholds, and is left in the repo in case it's useful for future work on this function.Test plan
pytest autotest/utils_tests.py -k "test_setup_pp or test_ppu_geostats or test_setup_pp_misses_offgrid_active_zone or test_setup_pp_too_few_points_raises or test_setup_pp_small_zone_uses_fallback_instead_of_raising or test_setup_pp_under_delivery_warns or test_setup_pp_dense_fallback_is_capped or test_setup_pp_uneven_coverage_triggers_fallback"— 8 passedpytest autotest/pst_from_tests.py -k "shortname_conversion_test or test_hyperpars"— 2 passedpython autotest/investigate_pp_search_impact.py— no regressions against the real Freyberg model or the full-active-grid baseline case