Skip to content

Added Python 3.14 experimental support, updated SKILLS.md files, better predict.py error handling #63

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
joshterrill merged 2 commits into from
Jul 24, 2026
Merged

Added Python 3.14 experimental support, updated SKILLS.md files, better predict.py error handling #63

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
19d1769
Add support for gitflow (#60)
ethanrucinski Apr 14, 2026
c44391d
fixed skills format, added python 3.14 support to experimental featur…
joshterrill Jul 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions .github/workflows/deploy-images.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,21 @@ name: Deploy All Images
env:
REGISTRY: ghcr.io
IMAGE_NAME: numerai_predict
MODEL_UPLOADS_ECR_ACCOUNT_ID: ${{ github.ref == 'refs/heads/staging' && secrets.MODEL_UPLOADS_STAGING_ACCOUNT_ID || secrets.MODEL_UPLOADS_ACCOUNT_ID }}

on:
workflow_dispatch:
push:
branches:
- master
- staging

jobs:
public-images:
if: github.ref == 'refs/heads/master'
strategy:
matrix:
python-version: ['3{0}10', '3{0}11', '3{0}12', '3{0}13']
python-version: ['3{0}10', '3{0}11', '3{0}12', '3{0}13', '3{0}14']
runs-on: ubuntu-latest-4core
permissions:
packages: write
Expand Down Expand Up @@ -63,9 +66,10 @@ jobs:
labels: ${{ steps.meta.outputs.labels }}

internal-images:
if: github.ref == 'refs/heads/master' || github.ref == 'refs/heads/staging'
strategy:
matrix:
python-version: ['3{0}10', '3{0}11', '3{0}12', '3{0}13']
python-version: ['3{0}10', '3{0}11', '3{0}12', '3{0}13', '3{0}14']
runs-on: ubuntu-latest-4core
permissions:
id-token: write
Expand All @@ -76,7 +80,7 @@ jobs:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ secrets.MODEL_UPLOADS_ACCOUNT_ID }}:role/github-actions
role-to-assume: arn:aws:iam::${{ env.MODEL_UPLOADS_ECR_ACCOUNT_ID }}:role/github-actions
aws-region: us-west-2

- name: Set up QEMU
Expand All @@ -86,4 +90,4 @@ jobs:
uses: docker/setup-buildx-action@v3

- name: Build and Push Docker Image
run: ACCOUNT_ID=${{ secrets.MODEL_UPLOADS_ACCOUNT_ID }} make push_latest_${{ format(matrix.python-version, '_') }}
run: ACCOUNT_ID=${{ env.MODEL_UPLOADS_ECR_ACCOUNT_ID }} make push_latest_${{ format(matrix.python-version, '_') }}
9 changes: 6 additions & 3 deletions .github/workflows/deploy-shell.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,18 @@ name: Deploy Shell Image (3.11 only)
env:
REGISTRY: ghcr.io
IMAGE_NAME: numerai_predict
MODEL_UPLOADS_ECR_ACCOUNT_ID: ${{ github.ref == 'refs/heads/staging' && secrets.MODEL_UPLOADS_STAGING_ACCOUNT_ID || secrets.MODEL_UPLOADS_ACCOUNT_ID }}

on:
workflow_dispatch:
push:
branches:
- master
- staging

jobs:
public-py311-image:
if: github.ref == 'refs/heads/master'
runs-on: ubuntu-latest-4core
permissions:
packages: write
Expand All @@ -34,7 +37,7 @@ jobs:
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ github.repository_owner }}/${{ env.IMAGE_NAME }}_shell

- name: Set up QEMU
uses: docker/setup-qemu-action@v3

Expand Down Expand Up @@ -70,8 +73,8 @@ jobs:
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: arn:aws:iam::${{ secrets.MODEL_UPLOADS_ACCOUNT_ID }}:role/github-actions
role-to-assume: arn:aws:iam::${{ env.MODEL_UPLOADS_ECR_ACCOUNT_ID }}:role/github-actions
aws-region: us-west-2

- name: Build and Push Docker Image
run: ACCOUNT_ID=${{ secrets.MODEL_UPLOADS_ACCOUNT_ID }} make push_latest_shell
run: ACCOUNT_ID=${{ env.MODEL_UPLOADS_ECR_ACCOUNT_ID }} make push_latest_shell
2 changes: 1 addition & 1 deletion .github/workflows/test-all.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
test-docker:
strategy:
matrix:
python-version: ['3_10', '3_11', '3_12', '3_13']
python-version: ['3_10', '3_11', '3_12', '3_13', '3_14']
runs-on: ubuntu-latest-4core
steps:
- uses: actions/checkout@v2
Expand Down
49 changes: 40 additions & 9 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ lint: ## Run linter
ruff check . --fix

.PHONY: build
build: build_3_10 build_3_11 build_3_12 build_3_13 ## Build all Python containers
build: build_3_10 build_3_11 build_3_12 build_3_13 build_3_14 ## Build all Python containers

.PHONY: build_3_10
build_3_10: ## Build Python 3.10 container
Expand All @@ -30,39 +30,54 @@ build_3_12: ## Build Python 3.12 container
build_3_13: ## Build Python 3.13 container
docker build --platform=linux/amd64 --build-arg GIT_REF=${GIT_REF} -t ${NAME}_py_3_13:${GIT_REF} -t ${NAME}_py_3_13:latest -f py3.13/Dockerfile .

.PHONY: build_3_14
build_3_14: ## Build Python 3.14 container
docker build --platform=linux/amd64 --build-arg GIT_REF=${GIT_REF} -t ${NAME}_py_3_14:${GIT_REF} -t ${NAME}_py_3_14:latest -f py3.14/Dockerfile .

.PHONY: build_shell
build_shell: ## Build Python 3.11 container
docker build --platform=linux/amd64 --build-arg GIT_REF=${GIT_REF} -t ${NAME}_shell:${GIT_REF} -t ${NAME}_shell:latest -f shell/Dockerfile .

.PHONY: test
test: test_predict test_3_10 test_3_11 test_3_12 test_3_13 ## Test all container versions
test: test_predict test_3_10 test_3_11 test_3_12 test_3_13 test_3_14 ## Test all container versions

.PHONY: test_predict
test_predict: build_shell ## Test predict script
test_predict: ## Test predict script
@if ! docker image inspect ${NAME}_shell:latest >/dev/null 2>&1; then $(MAKE) build_shell; fi
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_shell:latest python -m unittest tests.test_predict

.PHONY: test_3_10
test_3_10: build_3_10 ## Test Python 3.10 pickle
test_3_10: ## Test Python 3.10 pickle
@if ! docker image inspect ${NAME}_py_3_10:latest >/dev/null 2>&1; then $(MAKE) build_3_10; fi
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_10:latest --model /tests/models/model_3_10_legacy.pkl
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_10:latest --model /tests/models/model_3_10.pkl

.PHONY: test_3_11
test_3_11: build_3_11 ## Test Python 3.11 pickle
test_3_11: ## Test Python 3.11 pickle
@if ! docker image inspect ${NAME}_py_3_11:latest >/dev/null 2>&1; then $(MAKE) build_3_11; fi
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_11:latest --model /tests/models/model_3_11_legacy.pkl
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_11:latest --model /tests/models/model_3_11.pkl

.PHONY: test_3_12
test_3_12: build_3_12 ## Test Python 3.12 pickle
test_3_12: ## Test Python 3.12 pickle
@if ! docker image inspect ${NAME}_py_3_12:latest >/dev/null 2>&1; then $(MAKE) build_3_12; fi
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_12:latest --model /tests/models/model_3_12_legacy.pkl
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_12:latest --model /tests/models/model_3_12.pkl

.PHONY: test_3_13
test_3_13: build_3_13 ## Test Python 3.13 pickle
test_3_13: ## Test Python 3.13 pickle
@if ! docker image inspect ${NAME}_py_3_13:latest >/dev/null 2>&1; then $(MAKE) build_3_13; fi
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_13:latest --model /tests/models/model_3_13_legacy.pkl
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_13:latest --model /tests/models/model_3_13.pkl

.PHONY: test_3_14
test_3_14: ## Test Python 3.14 pickle
@if ! docker image inspect ${NAME}_py_3_14:latest >/dev/null 2>&1; then $(MAKE) build_3_14; fi
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_14:latest --model /tests/models/model_3_14_legacy.pkl
docker run -i --rm -v ./tests/:/tests/ -v /tmp:/tmp ${NAME}_py_3_14:latest --model /tests/models/model_3_14.pkl

.PHONY: push_latest
push_latest: push_latest_3_10 push_latest_3_11 push_latest_3_12 push_latest_3_13 ## Push latest docker containers
push_latest: push_latest_3_10 push_latest_3_11 push_latest_3_12 push_latest_3_13 push_latest_3_14 ## Push latest docker containers

.PHONY: push_latest_3_10
push_latest_3_10: build_3_10 ## Release Python 3.10 container tagged latest
Expand Down Expand Up @@ -96,6 +111,14 @@ push_latest_3_13: build_3_13 ## Release Python 3.13 container tagged latest
docker push ${ECR_REPO}/${NAME}_py_3_13:${GIT_REF}
docker push ${ECR_REPO}/${NAME}_py_3_13:latest

.PHONY: push_latest_3_14
push_latest_3_14: build_3_14 ## Release Python 3.14 container tagged latest
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin ${ECR_REPO}
docker tag ${NAME}_py_3_14:${GIT_REF} ${ECR_REPO}/${NAME}_py_3_14:${GIT_REF}
docker tag ${NAME}_py_3_14:latest ${ECR_REPO}/${NAME}_py_3_14:latest
docker push ${ECR_REPO}/${NAME}_py_3_14:${GIT_REF}
docker push ${ECR_REPO}/${NAME}_py_3_14:latest

.PHONY: push_latest_shell
push_latest_shell: build_shell ## Release Python 3.11 container tagged latest
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin ${ECR_REPO}
Expand All @@ -105,7 +128,7 @@ push_latest_shell: build_shell ## Release Python 3.11 container tagged latest
docker push ${ECR_REPO}/${NAME}_shell:latest

.PHONY: push_stable
push_stable: push_stable_3_10 push_stable_3_11 push_stable_3_12 push_stable_3_13 ## Push all container tagged stable
push_stable: push_stable_3_10 push_stable_3_11 push_stable_3_12 push_stable_3_13 push_stable_3_14 ## Push all container tagged stable

.PHONY: push_stable_3_10
push_stable_3_10: build_3_10 ## Release Python 3.10 container tagged stable
Expand Down Expand Up @@ -138,3 +161,11 @@ push_stable_3_13: build_3_13 ## Release Python 3.13 container tagged stable
docker tag ${NAME}_py_3_13:latest ${ECR_REPO}/${NAME}_py_3_13:stable
docker push ${ECR_REPO}/${NAME}_py_3_13:${GIT_REF}
docker push ${ECR_REPO}/${NAME}_py_3_13:stable

.PHONY: push_stable_3_14
push_stable_3_14: build_3_14 ## Release Python 3.14 container tagged stable
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin ${ECR_REPO}
docker tag ${NAME}_py_3_14:${GIT_REF} ${ECR_REPO}/${NAME}_py_3_14:${GIT_REF}
docker tag ${NAME}_py_3_14:latest ${ECR_REPO}/${NAME}_py_3_14:stable
docker push ${ECR_REPO}/${NAME}_py_3_14:${GIT_REF}
docker push ${ECR_REPO}/${NAME}_py_3_14:stable
Loading
Loading