Magnitude pipeline: all-relocated catalog + qc_pass, Route A displacement (Mw), docs - #15
Merged
Merged
Conversation
…k set A single raw_wa_amplitudes.csv is fine for the ~1M associated picks (~200 MB) but becomes multi-GB (and untrackable/unwieldy) if amplitudes are measured for the full ~39.6M ELEP detection set. - route_a_wa_amplitudes.py: add --chunk-rows N to rotate the output into raw_wa_amplitudes_partNNN.csv every N rows. Part index is derived from the ABSOLUTE row index, so --start-index shards/resumes write stable, non-colliding parts. Default 0 = single file (unchanged behavior). Files match reconstruct_split_csvs.py's pattern. - route_a_build_dataset.py: --raw now accepts a glob; each part is SNR/ok-filtered before concatenation, so the full raw set never has to fit in memory at once. - ROUTE_A_RUNBOOK.md: document the chunked full-set workflow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… stack)
The 'internal' env pulls the full default stack (torch, seisbench, pygmt, basemap) +
pnwstore, which is multi-GB and can exhaust disk on the compute host (e.g. psound).
Route A only needs obspy + numpy/pandas/scipy + pnwstore, so add a lean 'amplitude'
environment (no-default-feature) for the Wood-Anderson amplitude rerun:
pixi install --environment amplitude
pixi run -e amplitude python route_a_wa_amplitudes.py ...
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n authoritative copy Dry-run-first cleanup tool for a cramped working dir (e.g. $HOME on psound). Classifies files under --dir as dup-of-ref (byte-identical to a same-named file under --reference, e.g. /wd1/.../data -> safe to delete here), dup-in-dir, junk (*_temp/_test/_old), superseded (ver1/ver2/dated w_amp), or keep. Prints the plan + GB freeable; --apply deletes the dup/junk buckets, writes CLEAN_MANIFEST.csv, and never touches --reference. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both catalogs are local magnitude, so filter ANSS to its ml-typed events (drop Mw/Md) for a fair completeness comparison. Offshore, ANSS has very few ml-typed events (Mendocino 130, Endeavour 4) so the ensemble catalog dominates by 1-3 Mc units there; onshore Puget the two are comparable. (The large offshore events are Mw-typed and thus excluded from an ML-only comparison -- they need the separate Mw analysis.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ction funnel - Methods: new "Magnitude estimation" section documenting the amplitude measurement, the joint amplitude-distance inversion (relative magnitude M_i + station terms C_j, k>=0 anelastic term, n=1), absolute ML calibration, per-event uncertainty (station scatter ~0.3, median ML_unc ~0.2), the ML->Mw conversion (Mw ~ 1.7 + 0.8 ML, RMS ~0.7) with the saturation/low-frequency caveat, the response-removal upgrade (Route A) as the recommended basis, and the decision to release ALL relocated events with quality metrics + a QC flag rather than only the QC subset. - Results: "From picks to catalog" subsection with the full data-selection funnel and criteria (39.6M ELEP picks -> 1.09M associated picks / 116,591 events -> 63,887 relocated (55%) -> 31,020 QC (49%); final QC = 27% of associated). - New prose wrapped to ~72 cols for split-screen reading. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… amp for Mw Data product: release ALL relocated events, not just the QC subset. - phase10: classify the full relocated catalog (63,887) and add a qc_pass flag (= membership in the published QC catalog, 31,020) + carry ML_unc. cascadia_catalog_ classified.csv now has every relocated event with a flag users can filter on. - phase5: the catalog map + regional zooms now join the full relocated origin table (all 63,798 ML'd events) instead of the QC-only subset. fig4 + zooms regenerated. - phase14 completeness now runs over all relocated events (offshore Mc drops further: Blanco 1.7, Endeavour 1.8 vs ANSS-ml ~3.2). - Captions/text (fig4, completeness, supplement) updated to "all relocated + qc_pass". Route A / Mw: route_a_wa_amplitudes.py now measures a second amplitude per pick, disp_amp_um -- broadband displacement (response removed to DISP, low band --disp-lo/ --disp-hi, default 0.05-2 Hz). Wood-Anderson (wa_amp_mm) saturates for large events, so this low-frequency displacement is the amplitude to calibrate toward Mw. RUNBOOK updated. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Copilot stopped reviewing on behalf of
mdenolle due to an error
August 17, 2026 14:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to PR #14. Prepares the amplitude/magnitude rerun and broadens the released catalog.
Data product
qc_passflag (= membership in the published QC catalog) +ML_unc, so users filter to taste.phase10rebuilt over the full relocated catalog; catalog map + regional zooms + completeness now show all events.Route A / Mw
route_a_wa_amplitudes.pynow measures two amplitudes per pick:wa_amp_mm(Wood-Anderson → ML) anddisp_amp_um(broadband displacement, response removed to DISP, low band--disp-lo/--disp-hidefault 0.05–2 Hz → moment-scale amplitude for Mw). WA saturates for large events; the low-frequency displacement is the amplitude to calibrate toward Mw. Output chunking (--chunk-rows) + glob-awarebuild_datasetfor the full ~40M-pick scale.Environment / ops
amplitudepixi env (obspy + pnwstore only) for space-constrained hosts.clean_home_data.py: dry-run-first tool to remove data redundant vs an authoritative copy (e.g. /wd1) — never touches the reference.Docs
🤖 Generated with Claude Code