Support pandas 2 and 3 - #56
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates Numerai Tools’ dependency and CI matrix to support both pandas 2.x and 3.x while addressing a few pandas-3-related compatibility issues in core scoring/submission code and tests.
Changes:
- Relax runtime pandas constraint to
>=2.2.2,<4.0.0and update dev stubs accordingly. - Expand GitHub Actions CI to test multiple pandas ranges (2.2.x, 2.3.x, 3.x) across supported Python versions (including 3.14 where applicable).
- Adjust scoring/submission logic and tests to avoid pandas 3 behavioral changes (e.g., copy-on-write / indexing semantics).
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
tests/test_submissions.py |
Tweaks remap_ids test setup for pandas compatibility (but currently leaves the test logically incorrect). |
tests/test_scoring.py |
Updates correlation test indexing to be pandas-3-friendly (.iloc). |
pyproject.toml |
Relaxes pandas and pandas-stubs constraints to allow pandas 3.x. |
poetry.lock |
Refreshes lockfile for new dependency ranges / newer resolved versions. |
numerai_tools/submissions.py |
Avoids in-place mutation by using assign when string-casting id columns. |
numerai_tools/scoring.py |
Avoids pandas matmul/indexing pitfalls; reduces copy-on-write issues in neutralize. |
.github/workflows/test-and-deploy.yml |
Adds pandas-version matrix, Python 3.14 coverage, and FutureWarning-as-error test runs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b67d7bc11c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
fcd55ac to
8ad7227
Compare
Summary
0.6.0.>=2.2.2,<4.0.0.poetry installso Python 3.14 jobs do not install the default locked pandas first.Validation
poetry checkpoetry buildbuildsnumerai_tools-0.6.0.tar.gzandnumerai_tools-0.6.0-py3-none-any.whlpoetry run ruff check .poetry run mypy .poetry run pytest -W error::FutureWarningpip checkpip checkpip checkpip checkpip check.Output Equivalence
Compared
origin/masteragainst this PR in isolated Python 3.11.11 worktrees using each side's lockfile. The serialized output diff was empty forcorrelation_contribution,neutralize,_validate_ids,numerai_corr, and the same-nameremap_idstest setup. Both environments usedpandas 2.3.1andnumpy 2.3.2; the PR lock usesscikit-learn 1.9.0whileorigin/masteruses1.7.1.