Skip to content

Support pandas 2 and 3 - #56

Merged
idan192 merged 6 commits into
masterfrom
idan/20260610/pandas-2-and-3-support
Jun 10, 2026
Merged

Support pandas 2 and 3#56
idan192 merged 6 commits into
masterfrom
idan/20260610/pandas-2-and-3-support

Conversation

@idan192

@idan192 idan192 commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Bump package version to 0.6.0.
  • Relax pandas runtime support to >=2.2.2,<4.0.0.
  • Add CI coverage for pandas 2.2.x, 2.3.x, and 3.x across supported Python versions.
  • Add Python 3.14 CI coverage for pandas versions that publish Python 3.14 wheels.
  • Clean up pandas compatibility issues in tests, scoring, and submission validation.
  • Select the matrix pandas/NumPy constraints before poetry install so Python 3.14 jobs do not install the default locked pandas first.

Validation

  • poetry check
  • poetry build builds numerai_tools-0.6.0.tar.gz and numerai_tools-0.6.0-py3-none-any.whl
  • poetry run ruff check .
  • poetry run mypy .
  • poetry run pytest -W error::FutureWarning
  • Python 3.11 + pandas 2.2.3: tests + pip check
  • Python 3.11 + pandas 3.0.3: tests + pip check
  • Python 3.14 + pandas 2.3.3: tests + pip check
  • Python 3.14 + pandas 3.0.3: tests + pip check
  • Isolated workflow-order checks: Python 3.14 + pandas 2.3.3 and Python 3.14 + pandas 3.0.3 both lock before install, install cleanly, pass tests, and pass pip check.

Output Equivalence

Compared origin/master against this PR in isolated Python 3.11.11 worktrees using each side's lockfile. The serialized output diff was empty for correlation_contribution, neutralize, _validate_ids, numerai_corr, and the same-name remap_ids test setup. Both environments used pandas 2.3.1 and numpy 2.3.2; the PR lock uses scikit-learn 1.9.0 while origin/master uses 1.7.1.

@idan192
idan192 requested a review from ndharasz June 10, 2026 19:58
@idan192
idan192 marked this pull request as ready for review June 10, 2026 19:58
@idan192
idan192 requested a review from Copilot June 10, 2026 19:59

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Numerai Tools’ dependency and CI matrix to support both pandas 2.x and 3.x while addressing a few pandas-3-related compatibility issues in core scoring/submission code and tests.

Changes:

  • Relax runtime pandas constraint to >=2.2.2,<4.0.0 and update dev stubs accordingly.
  • Expand GitHub Actions CI to test multiple pandas ranges (2.2.x, 2.3.x, 3.x) across supported Python versions (including 3.14 where applicable).
  • Adjust scoring/submission logic and tests to avoid pandas 3 behavioral changes (e.g., copy-on-write / indexing semantics).

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
tests/test_submissions.py Tweaks remap_ids test setup for pandas compatibility (but currently leaves the test logically incorrect).
tests/test_scoring.py Updates correlation test indexing to be pandas-3-friendly (.iloc).
pyproject.toml Relaxes pandas and pandas-stubs constraints to allow pandas 3.x.
poetry.lock Refreshes lockfile for new dependency ranges / newer resolved versions.
numerai_tools/submissions.py Avoids in-place mutation by using assign when string-casting id columns.
numerai_tools/scoring.py Avoids pandas matmul/indexing pitfalls; reduces copy-on-write issues in neutralize.
.github/workflows/test-and-deploy.yml Adds pandas-version matrix, Python 3.14 coverage, and FutureWarning-as-error test runs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread tests/test_submissions.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b67d7bc11c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/test-and-deploy.yml Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 7 changed files in this pull request and generated no new comments.

@idan192
idan192 force-pushed the idan/20260610/pandas-2-and-3-support branch from fcd55ac to 8ad7227 Compare June 10, 2026 21:37
@idan192
idan192 merged commit 54ef5f1 into master Jun 10, 2026
12 checks passed
@idan192
idan192 deleted the idan/20260610/pandas-2-and-3-support branch June 10, 2026 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants