Recoil hit removal processing for 2e WAB veto

[oscarplewis]

I am updating ldmx-sw, here are the details. (Resolves #2081.)

This is the processor and associated machinery implementing the recoil hit removal strategy for vetoing multi-electron wide-angle bremsstrahlung (WAB) events. This strategy consists of identifying the electron tracks in the recoil tracker, propagating them forward into the ECal, and drawing a radius of containment around the tracks. All hits within these radii are then discarded, and cuts and BDTs are applied to what remains in the ECal. This technique at the moment provides insufficient veto power (using both 68% and 95% RoCs) to warrant using the present form as a finished product, but the code is here provided in the hope that future work will improve upon the results.

Check List

• I successfully compiled ldmx-sw with my developments.
• I read, understood and follow the coding rules.

• I ran my developments and the following shows that they are successful.

Here is a single event example of the technique at work, in a "best case scenario." The straight lines are truth trajectories of the electron and photon in a 1e WAB event, and the dots are reconstructed hits in the ECal.

Here is the performance of the 2e_wab_vs_signal10mev_v15_1RoC_68_recrem.onnx BDT, applied together with the simple trigger, on two 100k 2e samples of WAB+pileup and 10 MeV signal+pileup. Performance is reasonable but non-ideal.

[tvami]

@tomeichlersmith the ONNX is 1MB in total, you think that should go to the dedicated data repo or it's fine here?

[tvami]

/run-validation

[github-actions]

The validation workflow is running here: https://github.com/LDMX-Software/ldmx-sw/actions/runs/27727120728.

[tvami]

@oscarplewis please run the automatic tool like just format-cpp-diff and just lint-python-fix if you havent done so yet (from the CI it kinda seems like you didnt, but maybe that's just on the CI)

[oscarplewis]

Strange, I didn't run the automatic tools but I ran both of those commands before making the PR and I thought it had caught everything. It must not have though. I couldn't get the automatic tools to work, so I just ran the commands for a second time instead and pushed a new commit.

Hope that fixed whatever the problem was.

[github-actions]

Validation Results

All validation samples passed! ✅

Sample	Status
cascade_history	✅ PASS
deep_ecal_gun	✅ PASS
eat_signal	✅ PASS
ecal_pn	✅ PASS
hcal	✅ PASS
inclusive	✅ PASS
it_pileup	✅ PASS
kaon_enhanced	✅ PASS
reduced_ldmx	✅ PASS
signal	✅ PASS
signal_target_al	✅ PASS
target_genie	✅ PASS
target_pn_lyso	✅ PASS
target_ti_en	✅ PASS
wab_lhe	✅ PASS

cascade_history:

• Text Differences Between Logs (2 lines differ)
• Timing for cascade_history: gold=5968s, new=5993s
• Timing within 0% of gold (tolerance 10%)

deep_ecal_gun:

• Text Differences Between Logs (22 lines differ)
• Timing for deep_ecal_gun: gold=4091s, new=4124s
• Timing within 0% of gold (tolerance 10%)

eat_signal:

• Timing for eat_signal: gold=1602s, new=1642s
• Timing within 2% of gold (tolerance 10%)

ecal_pn:

• Text Differences Between Logs (40 lines differ)
• Timing for ecal_pn: gold=6232s, new=6563s
• Timing within 5% of gold (tolerance 10%)

hcal:

• Text Differences Between Logs (2 lines differ)
• Timing for hcal: gold=829s, new=820s
• Timing within 1% of gold (tolerance 10%)

inclusive:

• Text Differences Between Logs (28 lines differ)
• Timing for inclusive: gold=6805s, new=6673s
• Timing within 1% of gold (tolerance 10%)

it_pileup:

• Text Differences Between Logs (44 lines differ)
• Timing for it_pileup: gold=577s, new=594s
• Timing within 2% of gold (tolerance 10%)

kaon_enhanced:

• Text Differences Between Logs (24 lines differ)
• Timing for kaon_enhanced: gold=3122s, new=3120s
• Timing within 0% of gold (tolerance 10%)

reduced_ldmx:

• Text Differences Between Logs (22 lines differ)
• Timing for reduced_ldmx: gold=211s, new=216s
• Timing within 2% of gold (tolerance 10%)

signal:

• Text Differences Between Logs (34 lines differ)
• Timing for signal: gold=1184s, new=1220s
• Timing within 3% of gold (tolerance 10%)

signal_target_al:

• Text Differences Between Logs (36 lines differ)
• Timing for signal_target_al: gold=1047s, new=1043s
• Timing within 0% of gold (tolerance 10%)

target_genie:

• Text Differences Between Logs (4480 lines differ)
• Timing for target_genie: gold=6488s, new=6600s
• Timing within 1% of gold (tolerance 10%)

target_pn_lyso:

• Text Differences Between Logs (44 lines differ)
• Timing for target_pn_lyso: gold=6455s, new=6465s
• Timing within 0% of gold (tolerance 10%)

target_ti_en:

• Text Differences Between Logs (36 lines differ)
• Timing for target_ti_en: gold=3226s, new=3152s
• Timing within 2% of gold (tolerance 10%)

wab_lhe:

• Text Differences Between Logs (26 lines differ)
• Timing for wab_lhe: gold=6770s, new=7299s
• Timing within 7% of gold (tolerance 10%)

[tvami]

Thanks Oscar! I'm approving it pending the decision about where to put the ONNX files

[tomeichlersmith]

My personal opinion is that all non-text files should be in some other data repo, BUT we don't have another good way of distributing them and they are relatively light (they are smaller individually than other text files which were procedurally generated from a CAD model).

$ fd -tf -x stat -c '%s %n' | sort -nr
15208647 ./MagFieldMap/BmapCorrected3D_13k_unfolded_scaled_1.15384615385.dat.tar.gz
5614878 ./SimCore/G4DarkBreM/docs/doxygen-awesome-css/docs/img/fancy_scrollbars_webkit.gif
5614878 ./docs/doxygen.conf/doxygen-awesome-css/docs/img/fancy_scrollbars_webkit.gif
2399274 ./SimCore/G4DarkBreM/docs/G4DarkBreM_HEP_Seminar.pdf
1607080 ./Trigger/Algo_HLS/TS/ip/TSTriggerAlgo.dcp
1208320 ./Detectors/data/archived/ldmx-det-v13.tar.gz
1208320 ./Detectors/data/archived/ldmx-det-v12.tar.gz
898261 ./Detectors/data/ldmx-ti-v15-8gev/magnet.gdml
898261 ./Detectors/data/ldmx-lyso-r4-v15-8gev/magnet.gdml
898261 ./Detectors/data/ldmx-det-v15-8gev/magnet.gdml
898261 ./Detectors/data/ldmx-al-v15-8gev/magnet.gdml
897517 ./Detectors/data/ldmx-vertTS-v14-8gev/magnet.gdml
897517 ./Detectors/data/ldmx-det-v14/magnet.gdml
897517 ./Detectors/data/ldmx-det-v14-8gev-no-cals/magnet.gdml
897517 ./Detectors/data/ldmx-det-v14-8gev/magnet.gdml
830160 ./Hcal/data/visibles.onnx
263525 ./Tools/HLS_Arbitrary_Precision_Types/include/ap_impl/ap_private.h
216518 ./SimCore/G4DarkBreM/docs/doxygen-awesome-css/img/screenshot.png
216518 ./docs/doxygen.conf/doxygen-awesome-css/img/screenshot.png
142587 ./Detectors/data/ldmx-reduced-v3/ecal_support_box.gdml
142522 ./Detectors/data/ldmx-ti-v15-8gev/ecal_support_box.gdml
142522 ./Detectors/data/ldmx-lyso-r4-v15-8gev/ecal_support_box.gdml
142522 ./Detectors/data/ldmx-det-v15-8gev/ecal_support_box.gdml
142522 ./Detectors/data/ldmx-al-v15-8gev/ecal_support_box.gdml
142451 ./Detectors/data/ldmx-vertTS-v14-8gev/ecal_support_box.gdml
142451 ./Detectors/data/ldmx-det-v14/ecal_support_box.gdml
142451 ./Detectors/data/ldmx-det-v14-8gev-no-cals/ecal_support_box.gdml
142451 ./Detectors/data/ldmx-det-v14-8gev/ecal_support_box.gdml
130965 ./Trigger/Algo_HLS/TS/reports/syn/TSTriggerAlgo_csynth.rpt
124238 ./docs/doxygen.conf/doxygen-awesome-css/Doxyfile
117315 ./SimCore/G4DarkBreM/docs/doxygen-awesome-css/Doxyfile
107977 ./Trigger/ruckus/Doxyfile
106498 ./Detectors/data/archived/ldmx-lyso-v12.tar.gz
103213 ./Detectors/data/archived/ldmx-det-v9.tar.gz
102744 ./Detectors/data/archived/ldmx-det-v5.tar.gz
102001 ./Detectors/data/archived/ldmx-det-v3.tar.gz
101167 ./Detectors/data/archived/ldmx-ti-v9.tar.gz
99445 ./docs/doxygen.conf/doxyfile
93871 ./SimCore/G4DarkBreM/docs/doxyfile
85825 ./Tools/HLS_Arbitrary_Precision_Types/include/ap_fixed_base.h
80396 ./DQM/python/dqm.py
...stopped now that we are below 1k