CI: No module named 'pkg_resources'

[victorlin]

This seemed to have turned from a warning into an error between yesterday and today, likely due to upstream dependency changes.

  File "/Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/snakemake/scheduler.py", line 758, in job_selector_ilp
    from stopit import ThreadingTimeout as Timeout, TimeoutException
  File "/Users/runner/miniconda3/envs/test/lib/python3.10/site-packages/stopit/__init__.py", line 10, in <module>
    import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'

Only two jobs are affected: test-dist with Python 3.10 on ubuntu-latest and macos-latest.

[victorlin]

Not sure what's going on. The conda environment looks identical between the warning and error runs linked above (see full diff of logs).

I've triggered a rebuild of conda-base to see if that has any affect.

[victorlin]

Oh, conda-base has nothing to do with this error which comes from the ambient runtime check.

The relevant environment info is from the previous successful setup-integration-tests step. Diffing conda list output from those logs is more revealing:

263c263
< setuptools                   81.0.0           pyh332efcf_0         conda-forge
---
> setuptools                   82.0.0           pyh332efcf_0         conda-forge

[victorlin]

Opened an issue and PR upstream in the stopit package feedstock.

[victorlin]

Actually, Snakemake has already moved on from stopit as of snakemake/snakemake#2938. The error is lingering in this repo because setup-integration-tests installs an old version of Snakemake, 7.32.4.

There is a pin to exclude a specific version 7.30.2, but this shouldn't prevent it from installing the latest versions 9.x.

cli/.github/actions/setup-integration-tests/action.yaml

Line 55 in 4330e94

run: mamba install augur auspice 'snakemake !=7.30.2'

I couldn't reproduce locally with conda or micromamba. I'll try changing from conda-incubator/setup-miniconda to mamba-org/setup-micromamba to see if that helps.

[victorlin]

Here are the missing pieces to the puzzle, explaining why the breaking change in setuptools 82.0.0 only affected Python 3.10:

• Snakemake 7.32.4 was the last version to support Python 3.10 (ref).
• setuptools 80.9.0 was the last version to support Python 3.9 (ref).

I think the easiest thing to do is to add an extra step to downgrade setuptools in the Python 3.10 jobs. The workaround will eventually go away when support for Python 3.10 is dropped.

[victorlin]

Created a PR #508 but looks like things are moving along with the upstream issue conda-forge/stopit-feedstock#4. I'll just wait for those changes to show up in tomorrow's scheduled run.

[victorlin]

Upstream fix made its way into today's scheduled run:

Run mamba install augur auspice 'snakemake !=7.30.2'
…

  Downgrade:
─────────────────────────────────────────────────────────────────────────────────────────

  - setuptools                      82.0.0  pyh332efcf_0         conda-forge     Cached
  + setuptools                      81.0.0  pyh332efcf_0         conda-forge      677kB