Summary
#864 fixed 31 broken docstring examples that a pytest --doctest-modules sweep turned up — stale pre-NumPy-1.14 array spacing, NumPy 2 scalar reprs, missing imports, and two outright NameErrors. Those examples had drifted precisely because nothing ever ran them. After that PR, 83 of the 85 modules are doctest-clean when each module is run in isolation.
The obvious follow-up is to run doctests in CI so they cannot drift again. Two things stand in the way, and neither has an obviously right answer, so raising it rather than deciding it in #864.
Blocker 1: global print state leaks between modules
Six game_theory docstrings call np.set_printoptions(precision=4) and never restore it:
game_theory/lemke_howson.py (two examples)
game_theory/support_enumeration.py (two examples)
game_theory/vertex_enumeration.py (two examples)
Print options are process-global. Under a package-wide --doctest-modules run, game_theory is collected before markov, optimize, random and util, so every array printed in those later modules comes out with 4 significant digits instead of 8. That produces 9 spurious failures in modules whose examples are individually correct — for instance markov/random.py, random/utilities.py::probvec, markov/ddp.py and optimize/nelder_mead.py all pass on their own and fail in the full run.
This is pre-existing and entirely harmless to the rendered documentation: a reader copying any one of those blocks gets exactly what is shown. It only bites the test harness.
The options are to drop set_printoptions from those six examples and paste full-precision expected output (more faithful, but noisier examples), to add a restoring call after each block (visible clutter in the published docs), or to reset print options between modules from a conftest fixture (invisible to readers, but the fixture has to be discovered and maintained). My weak preference is the conftest route, since it keeps the examples readable, but it is the least obvious to a future contributor.
Blocker 2: two modules cannot pass honestly
| Module |
Why it cannot pass |
util/notebooks.py::fetch_nb_dependencies |
Fetches over the network |
util/timing.py::Timer and ::timeit |
Reports wall-clock durations, which vary per run |
#864 deliberately left both untouched rather than annotating them with # doctest: +SKIP, because doctest directives render visibly in the published HTML and would be a cost paid by every reader to satisfy CI. If we do wire doctests into CI, we need to decide whether that visible cost is acceptable, or whether to exclude these modules at the pytest level instead — for example via --ignore, or collect_ignore in a conftest, which keeps the rendered docs clean.
util/array.py::searchsorted is a third, milder case: it is deliberately deprecated and emits a DeprecationWarning, which a strict warning filter would turn into a failure.
Scope
Once those are settled the change itself is small — a step in ci.yml alongside the existing test run. The value is that docstring examples become executable specifications rather than prose that silently rots, which is what let a call to a nonexistent lorenz() sit in _inequality.py.
Summary
#864 fixed 31 broken docstring examples that a
pytest --doctest-modulessweep turned up — stale pre-NumPy-1.14 array spacing, NumPy 2 scalar reprs, missing imports, and two outrightNameErrors. Those examples had drifted precisely because nothing ever ran them. After that PR, 83 of the 85 modules are doctest-clean when each module is run in isolation.The obvious follow-up is to run doctests in CI so they cannot drift again. Two things stand in the way, and neither has an obviously right answer, so raising it rather than deciding it in #864.
Blocker 1: global print state leaks between modules
Six
game_theorydocstrings callnp.set_printoptions(precision=4)and never restore it:game_theory/lemke_howson.py(two examples)game_theory/support_enumeration.py(two examples)game_theory/vertex_enumeration.py(two examples)Print options are process-global. Under a package-wide
--doctest-modulesrun,game_theoryis collected beforemarkov,optimize,randomandutil, so every array printed in those later modules comes out with 4 significant digits instead of 8. That produces 9 spurious failures in modules whose examples are individually correct — for instancemarkov/random.py,random/utilities.py::probvec,markov/ddp.pyandoptimize/nelder_mead.pyall pass on their own and fail in the full run.This is pre-existing and entirely harmless to the rendered documentation: a reader copying any one of those blocks gets exactly what is shown. It only bites the test harness.
The options are to drop
set_printoptionsfrom those six examples and paste full-precision expected output (more faithful, but noisier examples), to add a restoring call after each block (visible clutter in the published docs), or to reset print options between modules from a conftest fixture (invisible to readers, but the fixture has to be discovered and maintained). My weak preference is the conftest route, since it keeps the examples readable, but it is the least obvious to a future contributor.Blocker 2: two modules cannot pass honestly
util/notebooks.py::fetch_nb_dependenciesutil/timing.py::Timerand::timeit#864 deliberately left both untouched rather than annotating them with
# doctest: +SKIP, because doctest directives render visibly in the published HTML and would be a cost paid by every reader to satisfy CI. If we do wire doctests into CI, we need to decide whether that visible cost is acceptable, or whether to exclude these modules at the pytest level instead — for example via--ignore, orcollect_ignorein a conftest, which keeps the rendered docs clean.util/array.py::searchsortedis a third, milder case: it is deliberately deprecated and emits aDeprecationWarning, which a strict warning filter would turn into a failure.Scope
Once those are settled the change itself is small — a step in
ci.ymlalongside the existing test run. The value is that docstring examples become executable specifications rather than prose that silently rots, which is what let a call to a nonexistentlorenz()sit in_inequality.py.