[Experimental] Use jupyter-book 2.0 for building the lectures

[kp992]

Experimental setup for building the lectures with the QuantEcon MyST stack. This PR is built on top of #345.

Current toolchain

Both workflows now build with the QuantEcon fork of the MyST CLI and the new theme, so the Netlify preview shows the latest content rendered by the latest tooling.

Piece	What the branch uses	Where it is set
MyST CLI	QuantEcon/mystmd main, cloned and built from source with bun; quantecon/VERSION.yml is printed to the log so every build records the qe-vN fork state it ran with (currently v1.10.1 (qe-v8))	.github/workflows/ci.yml, .github/workflows/build-ipynb.yml
Theme	quantecon-theme.mystmd v2.2.0, pinned to the release zip	lectures/myst.yml (site.template)
Build command	myst build --html --execute	.github/workflows/ci.yml
Content	merged up to date with main	—

The theme moves off the archived QuantEcon/quantecon-theme bundle (which the branch was still pulling as a floating main.zip) onto a pinned release of the new theme repo. Bump the vX.Y.Z in the site.template URL to take a newer theme; the fork stays on main deliberately, so the preview tracks the latest tooling.

The build was not actually using the fork

Worth calling out separately, because it had been green and invisible: until this update the HTML preview was built by jupyter book build, which cannot use the fork. jupyter-book 2.x is a thin Python shim — jupyter_book/__main__.py execs a compiled mystmd bundle vendored inside its own wheel (jupyter_book/dist/jupyter-book.cjs) and exposes no hook to point it at another CLI. The globally installed fork was simply ignored, and only the build-ipynb workflow (which calls myst directly) ever exercised it.

The last run before the fix shows it plainly: the jupyter-book HTML build emitted 27 × 'output' unknown export output extension: exports/<page>.ipynb, because the per-page ipynb export is a fork feature (myst-to-ipynb, qe-v1) that upstream does not know about. The same commit built by the fork's myst build --ipynb emitted none and exported 23 clean notebooks. The HTML build now runs myst build --html --execute — the same engine and the same flags, since jupyter-book 2.x is mystmd, just our build of it.

Everything else in this update

• Merged main. Brings in the new Polars lecture, the numba rewrite, the pandas_panel switch to remote CSVs, and the translation-sync workflows. The one conflict was ci.yml, where main and this branch have rewritten the same job for different builders — resolved in favour of this branch's job, keeping main's runner spec and action bumps.
• myst.yml TOC re-synced with _toc.yml. It had drifted: autodiff and the new polars lecture were missing, workspace appeared twice, and one part title differed. The two files now list exactly the same 27 pages.
• project.github corrected to lecture-python-programming — it pointed at lecture-python-programming.myst, which now only survives as a rename redirect. The theme derives the notebook-launch repo from this value plus a .notebooks suffix, so a stale name means broken launch buttons.
• JAX is now installed in the build. Previous preview builds were rendering ModuleNotFoundError cells for jax_intro, autodiff and numpy_vs_numba_vs_jax, because this job installed only myst_requirements.txt while the jupyter-book 1.x job on main installs jax[cuda13] separately. Same install, same runner family.
• CI actions moved to current majors and Node 20 → 24 (the theme's .nvmrc; it runs as a Remix server during myst build --html). This also clears the Node 20 runtime deprecation warnings. The unused configure-pages step is gone — this job deploys to Netlify, not Pages.
• Removed committed execution artifacts (foo.py, newfile.txt, numbers.txt, output.txt, output2.txt, test_table.csv, us_cities.txt). All are written by %%file / %%writefile cells in the lectures themselves, all are matched by .gitignore, and none exist on main.

Still open

• Nothing fails the build on a notebook execution error. Execution errors are reported and the build carries on, which is how the JAX failures above survived green builds. main gained a gate for the jupyter-book 1.x path in CI: gate the first jb build so notebook errors cannot pass green #588; the fork's CLI has myst build --strict for this, but it should be turned on only once the remaining build diagnostics are triaged, or it will fail on unrelated warnings.
• Launch-button URLs are left on the theme defaults (<repo>.notebooks, branch main, notebooks at the repo root), matching the layout of lecture-python-programming.notebooks. Worth clicking through in the preview to confirm the page location resolves as expected.

[github-actions]

🚀 Deployed on https://6a6c3f4f5fa2c9c4f7aebb82--epic-agnesi-957267.netlify.app

[mmcky]

@kp992 thanks for opening this. Excellent work! It is exciting to see how close we are now.

From my review it looks like there a just a few issues and features we need before launch.

Bugs:

1. LHS TOC menu flickers when using the Home page or a top level Part section. It is like it has an auto-stow feature that is triggering when pages are reloaded. It doesn't look great when pages are refreshed.
2. Code block formatting has too much indentation for Solution blocks

Features:

1. I see a bunch of new export notebooks being added to the yaml headers of each lecture. Do we need to do some development work on supporting the download notebook link in the theme?
2. Is index support on the jb2 / mystmd roadmap?

[mmcky]

Thanks @mmcky for pushing this forward and fixing the issues. I saw the generated notebooks and they look great! One issue I found was about the solution rendering. The solution shows up as the normal markdown ```python code block instead of an executable block.

Thanks @kp992 would you mind raising an issue on https://github.com/QuantEcon/mystmd

Thanks @kp992 I think I have identified the root cause of this. QuantEcon/mystmd#5
Working on a fix.

[kp992]

Thank you @mmcky. The issue looks resolved and the generated notebooks are awesome!!

[mmcky]

• check if theme regression is occurring. Clicking on a chapter briefly shows a malformed webpage that disappears to the LEFT. NEEDS REVIEW.

[mmcky]

Refreshed against the current toolchain

Branch is up to date with main and now builds with the QuantEcon fork of the MyST CLI and the new theme repo. The PR description has been rewritten to describe the resulting setup.

Piece	Before	Now
MyST CLI (HTML build)	upstream mystmd, vendored inside the jupyter-book wheel	QuantEcon/mystmd main — v1.10.1 (qe-v8)
Theme	QuantEcon/quantecon-theme floating main.zip (repo archived)	quantecon-theme.mystmd v2.2.0, pinned release zip
Content	2 months behind main	merged, conflict resolved — PR is mergeable again

The HTML build was never using the fork

This is the part worth reading. jupyter book build cannot use our fork, and had been quietly building the preview with upstream mystmd this whole time. jupyter-book 2.x is a thin Python shim: jupyter_book/__main__.py execs a compiled mystmd bundle vendored inside its own wheel (jupyter_book/dist/jupyter-book.cjs) and exposes no hook to point it at a different CLI. The npm install -g of the fork in this workflow was installing a CLI that the build step then ignored. Only build-ipynb.yml, which calls myst directly, was ever exercising the fork.

The evidence is in the last run before the fix — the jupyter-book HTML build emitted 27 × 'output' unknown export output extension: exports/<page>.ipynb, since the per-page ipynb export is a fork feature (myst-to-ipynb, qe-v1) that upstream doesn't know about. The same commit built by the fork's myst build --ipynb emitted none and exported 23 clean notebooks.

The HTML build now runs myst build --html --execute with the fork's CLI, and the pip install jupyter-book step is gone — it was only putting a second, conflicting mystmd on the image. Same engine and same flags either way; jupyter-book 2.x is mystmd, this is just our build of it.

Three other things the review turned up

• The JAX lectures were rendering ModuleNotFoundError cells. This job installed only myst_requirements.txt, while the jupyter-book 1.x job on main installs jax[cuda13] separately, so jax_intro, autodiff and numpy_vs_numba_vs_jax had been executing to import errors through green builds. Same install added here.
• myst.yml's TOC had drifted from _toc.yml — autodiff and the new polars lecture missing, workspace listed twice, one part title stale. The two files now list exactly the same 27 pages.
• project.github pointed at lecture-python-programming.myst, which only survives as a rename redirect. The theme derives the notebook-launch repo from that value plus a .notebooks suffix, so a stale name means broken launch buttons. Corrected.

Also: Node 20 → 24 (the theme's .nvmrc), remaining actions to current majors (clears the Node 20 removal deprecation), the unused configure-pages step dropped since this job deploys to Netlify rather than Pages, and seven committed execution artifacts removed (foo.py, us_cities.txt, test_table.csv and friends — all written by %%file cells in the lectures, all gitignored, none present on main).

Left for a decision

Nothing fails the build on a notebook execution error — that is exactly how the JAX breakage stayed invisible. main gained a gate for the jupyter-book 1.x path in #588; the fork's CLI has myst build --strict for the same job here, but it is worth turning on only after the remaining build diagnostics are triaged, otherwise it will fail on unrelated warnings.