Create preprocessing module

[brady-ryan]

This PR implements a preprocessing module whose usefulness is twofold. Firstly, the ImageCutout class provides a user-friendly interface for making a cutout from the full science mosaic and generating the necessary kwargs_data dictionary. Secondly, this PR eases and accelerates the creation of a PSF from a full science mosaic. It includes:

1. psf.py: the core PSF preprocessing module
2. preprocessing_util.py: utility module which includes some functions that are shared between PSF and the ImageCutout preprocessing class.
3. test_psf.py: an initial suite of test functions. At the moment, the entire code is probably not covered, just the main components while we discuss what changes/modifications might be necessary.
4. test_preprocessing.py: an initial suite of test functions for the preprocessing_util module.
5. files.py: additions for preprocessing I/O
6. Example of PSF.ipynb: example notebook walking through the creation of a PSF with both the PSFr and STARRED methodologies.
7. image_cutout.py: the core Image cutout preprocessing module
8. test_image_cutout: suite of test functions for ImageCutout
9. Example of Image Cutout.ipynb: example walking through using the ImageCutout to create a science image.
10. requirements.txt: includes the required software for PSF preprocessing.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[brady-ryan]

@ajshajib in general this is ready for review. According to the logs, the crash is occurring because astroObjectAnalyser.DataAnalysis cannot be found. This is odd, as the structure clearly exists here and runs fine on my local machine.

[ajshajib]

Thanks a lot, @brady-ryan! Unfortunately, there is no way around getting the tests passed. However, I would also prefer not to depend on/use AstroObjectAnalyzer, the required functionalities can be achieved through astropy, and that saves the user from one more package download. Please see the attached notebook for an example of how I prepare the PSF.

starred_psf_f200lp.ipynb

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (4c2cbf2) to head (06ac562).
⚠️ Report is 26 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main      #188      +/-   ##
===========================================
+ Coverage   99.95%   100.00%   +0.04%     
===========================================
  Files          15        18       +3     
  Lines        2139      2827     +688     
===========================================
+ Hits         2138      2827     +689     
+ Misses          1         0       -1     

Files with missing lines	Coverage Δ
dolphin/preprocessing/image_cutout.py	100.00% <100.00%> (ø)
dolphin/preprocessing/preprocessing_util.py	100.00% <100.00%> (ø)
dolphin/preprocessing/psf.py	100.00% <100.00%> (ø)
dolphin/processor/files.py	100.00% <100.00%> (ø)

... and 4 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[brady-ryan]

@ajshajib, these new updates remove the dependency on AstroObjectAnalyser and instead use Cutout2D() from astropy, as suggested.

@ahuang314, I have added check=True to subprocess.run(), as well as added some functionality to automatically mask out PSF candidate objects which contain NaN values.

Other optimizations include:

1. Changing the PSF mask structure from .reg files to .npy files, which is more consistent with the model masks expected by Dolphin. I also wrote a new helper function in the PSF module where one can make a mask for a specific object and save said mask accordingly.
2. I moved the make_image_catalog and get_background functions to a new module called preprocessing_util.py. Also included in there is the mask making logic for the PSF candidates. I am currently in the process of creating an ImageCutout class to ease the process of making cutouts from the full mosaics, and found it easier to create a new module where commonly used functions could reside (instead of having a version in each respective module, or importing from the other).

If we like this current structure, then I will proceed forward with finishing the test functions!

[ajshajib]

@brady-ryan, just a quick comment (I can give a more thorough look a little later), it looks like there is a dependency on SourceExtractor. I would prefer to avoid that. If I remember correctly, it involved the installation of a Fortran code and its Python wrapper, so it's rather complicated, which can be replaced with Astropy functionalities. For example, look at my notebook, which uses only Python. Is SourceExtractor only needed to compute the background rms, or for something else too? Background rms can also be computed using astropy.

[brady-ryan]

@brady-ryan, just a quick comment (I can give a more thorough look a little later), it looks like there is a dependency on SourceExtractor. I would prefer to avoid that. If I remember correctly, it involved the installation of a Fortran code and its Python wrapper, so it's rather complicated, which can be replaced with Astropy functionalities. For example, look at my notebook, which uses only Python. Is SourceExtractor only needed to compute the background rms, or for something else too? Background rms can also be computed using astropy.

Hi @ajshajib, alongside background estimation, Source Extractor is used to build a catalog of source objects within the full mosaic. After this, a masking criterion is utilized to select candidate PSF stars from the catalog. I see in your notebook that you also used Astropy to find candidate stars, so, if you prefer, I can rewrite get_psf_candidates() around Astropy and see how it compares with Source Extractor. The installation of Source Extractor itself is simple, while the complicated portion lies in setting up the config files. On that note, I agree then it may be useful to only work through Astropy for simplicity purposes.

*Edit - you use photutils for the background and retrieving candidate stars, not Astropy. I suppose that would still be preferable to Source Extractor.

[ajshajib]

Yes, using a fully Pythonic method will be preferable, so astropy/photutils over SourceExtractor is preferred.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

ajshajib commented on 2026-07-17T05:17:41Z
----------------------------------------------------------------

Dolphin

dolphin

D should be in small letter in dolphin. Please fix everywhere else in the notebook.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

ajshajib commented on 2026-07-17T05:17:42Z
----------------------------------------------------------------

Modify the requirement for the full science image to be put in the io_directory, as per my other comment in the code.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

ajshajib commented on 2026-07-17T05:17:42Z
----------------------------------------------------------------

What's the utility of having flux values on top of each panel. If there's none, better to remove it to keep things cleaner. Otherwise, please state the utility for the users.

brady-ryan commented on 2026-07-25T18:15:18Z
----------------------------------------------------------------

This was leftover from when I sorted objects by magnitude with Source Extractor. Indeed, it is no longer necessary here.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

ajshajib commented on 2026-07-17T05:17:43Z
----------------------------------------------------------------

Is it very necessary to provide custom masks made by hand? In my experience, I have never come across a situation where making a custom mask was worth the time, instead I could simply choose some other stars for the PSF. Easy to make masks can be made on the fly through kwargs_mask .

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

ajshajib commented on 2026-07-17T05:17:44Z
----------------------------------------------------------------

their documentation

Make this a linked text pointing to the doc's url.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

ajshajib commented on 2026-07-17T05:17:44Z
----------------------------------------------------------------

STARRED documentation

Please make it linked to the URL.

[brady-ryan]

This was leftover from when I sorted objects by magnitude with Source Extractor. Indeed, it is no longer necessary here.

---

View entire conversation on ReviewNB

[brady-ryan]

Hi @ajshajib, thank you again for the comments and suggestions. Apologies for the delay in addressing them, I was away this week. Anyways, I have modified the workflow such that the user provides the star cutouts and noise map cutouts directly into the PSF fitting functions. Previously, the user would need to save the data first, and it would be automatically reloaded with the fitting functions. As per your suggestion, one no longer needs to save the cutouts. I have also implemented a cleaning function for those who may want to save the cutouts initially, but then remove them before transfer between computers with clean_psf_workspace(). In addition, the masking has been modified such that the user also need not save each individual mask, instead providing a list of arrays into the PSF fitting functions.

The example notebook has been modified accordingly to reflect these changes. I also added a few sentences at the start to discuss the STARRED vs. PSFr results from Millon et al. (2024), recommending the user to utilize the STARRED methodology.

Lastly, there seem to be some new errors in Pre-Commit QA that are not a result of the PSF preprocessing class. Was something in the pre-commit flagging changed?

[brady-ryan]

Hi @ajshajib, I thought it appropriate that as the PSF module nears completion, I completed the preprocessing module with ImageCutout, which provides a user-friendly interface for making a science cutout and building kwargs_data. As such, I have renamed the PR appropriately. I have also accordingly added an example notebook for the ImageCutout module. This module is not nearly as complex as PSF, so I do not anticipate it taking as much of your time for review. I will also work on ensure that the code coverage is 100%. Of course, I look forward to hearing your thoughts. Like before, the failures seem to be attributed to things external to this module.

*Edit - all new code now covered by test functions.