Pilot pixi for lecture repo environments (lockfiles, faster CI, task runner)

[mmcky]

Proposal: pilot pixi for lecture environment management

Summary

This issue proposes a single-repo pilot of pixi (prefix-dev/pixi) as the environment manager for the lecture repositories. The goal is to make lecture builds fully reproducible via lockfiles — so upstream library releases stop breaking builds ambiently and instead arrive as reviewable, CI-tested pull requests — while also speeding up CI environment setup and giving contributors a one-command build.

This is maintainer/CI tooling only. Nothing changes for readers, who continue to follow the "install Anaconda" instructions in the lectures.

The problem

Our current environment.yml (checked 31 July 2026 on lecture-python.myst and lecture-python-programming.myst) looks like:

name: quantecon
channels:
  - default
dependencies:
  - python=3.13
  - anaconda=2026.07
  - pip
  - pip:
    - jupyter-book>=1.0.4post1,<2.0
    - quantecon-book-theme==0.21.0
    - sphinx-tojupyter==0.6.0
    - sphinxext-rediraffe==0.3.0
    - sphinx-exercise==1.2.1
    # ...

This has two distinct version-drift vectors:

1. The pip layer is only pinned at the top level. Direct dependencies are pinned (quantecon-book-theme==0.21.0, etc.), but everything jupyter-book<2 pulls in transitively — Sphinx, myst-nb, docutils, and friends — re-resolves fresh on every CI run. A new upstream point release can break a build with zero commits in our repositories. This is the recurring "some library version got updated" failure mode.

2. Metapackage bumps are big-bang. anaconda=2026.07 is Anaconda's curated snapshot, so it is stable within a version — but bumping it moves hundreds of packages at once, and lectures break in batches rather than one at a time.

What pixi is

A project-scoped package manager for the conda ecosystem with cargo-like ergonomics:

• A pixi.toml manifest plus a multi-platform pixi.lock that pins the entire transitive closure, with conda and PyPI dependencies solved together.
• A built-in task runner (pixi run <task>).
• A first-class GitHub Action, prefix-dev/setup-pixi, with caching.
• Written in Rust on the rattler library, developed by prefix.dev (the mamba authors). BSD-3-Clause.

Status as of 31 July 2026: latest release v0.75.0 (29 July 2026). The project describes itself as production-ready with a compatibility guarantee on file formats, but it is pre-1.0 with a fast release cadence.

What we would gain

• Deterministic builds. Nothing updates until someone runs pixi update; upgrades arrive as a lockfile diff in a PR that CI tests before merge. Breakage becomes scheduled rather than ambient.
• Granular updates become possible. If we unbundle from the metapackage (see design decision below), pixi update numpy moves one package at a time instead of the whole distribution.
• Faster CI environment setup via setup-pixi with caching, compared to setup-miniconda + pip install.
• One-command onboarding. pixi run build encodes the jb build invocation in-repo, so contributors and CI run literally the same thing.
• Multiple environments per manifest. Directly useful for the Jupyter Book 2 / mystmd evaluation: a jb2 feature can carry mystmd plus nodejs (available on conda-forge) alongside the existing 1.x stack in the same repository, with no separate Node install step.

The key design decision

The anaconda metapackage is deliberate: our build environment mirrors the reader instruction to install Anaconda. Pixi forces us to make this explicit, with two options:

	Option A — keep anaconda=2026.07	Option B — unbundle to explicit deps
Reader-mirror philosophy	Preserved	Build env diverges from reader env
Channel	Requires declaring Anaconda's main channel explicitly (the metapackage is not on conda-forge, and pixi has no implicit defaults)	conda-forge only
Update granularity	Still big-bang for the conda layer; pip/transitive layer becomes locked	Fully granular (pixi update <pkg>)
Env size / solve speed	Unchanged (full distribution)	Smaller and faster
Anaconda channel ToS	Applies (fine for non-profit/edu use)	Sidestepped entirely

The pilot answers which option is viable within the first hour, because Option A's channel configuration is the first thing that will either work or fail.

Risks and caveats

• Pre-1.0 cadence. Pin the pixi version in setup-pixi and update it deliberately.
• Lockfile churn. pixi.lock is large; mark it linguist-generated in .gitattributes so PR diffs collapse by default.
• One more tool for maintainers to install (single binary; curl -fsSL https://pixi.sh/install.sh | sh or Homebrew).
• No reader-facing change — this does not by itself fix anything about the environments readers build at home.

Alternative considered

conda-lock bolts lockfiles onto the existing environment.yml workflow with minimal disruption. It is a reasonable fallback if the pilot finds pixi too invasive, but it lacks the task runner, the joint conda+PyPI solve, and the install speed.

Proposed pilot

On a branch of lecture-python-programming.myst (the smallest series):

pixi init --import environment.yml    # writes pixi.toml, solves pixi.lock
pixi task add build "jb build lectures --path-output ./"   # match the current CI invocation
pixi run build

Then, on the same branch, swap setup-miniconda for a pinned prefix-dev/setup-pixi (with cache: true) in the CI workflow.

Success criteria

• Full lecture build passes from the lockfile on Linux CI and on an Apple Silicon (osx-arm64) machine.
• anaconda=2026.07 resolves (or fails) under pixi — this decides Option A vs. Option B above.
• Wall-clock environment setup time measured against the current conda path.
• A deliberate pixi update on a throwaway branch surfaces upstream breakage as a red PR rather than a red main.
• Contributor ergonomics check: pixi run build works from a fresh clone with no other setup.

Timing

The monorepo consolidation will already touch every series' configuration and CI. Folding an environment-tooling change into that restructure pays the disruption cost once rather than twice — the same sequencing logic as the Jupyter Book 2 evaluation.

References

• pixi repository: https://github.com/prefix-dev/pixi
• pixi documentation: https://pixi.sh
• Lockfile docs: https://pixi.sh/latest/workspace/lockfile/
• GitHub Action: https://github.com/prefix-dev/setup-pixi
• conda-lock (alternative): https://github.com/conda/conda-lock