[Roadmap] Consolidate CTParser — separate parsing from semantics via an IR

[ocots]

Objective

Refactor CTParser following the control-toolbox Handbook philosophy to build a better base for future extensions.

Context

CTParser (v0.8.15) turns the @def DSL into executable model-building code, through two parsing backends living side by side in onepass.jl (~1500 lines): :fun generates function-based CTModels code, :exa generates a full ExaModels transcription (variables on a grid, scheme-dependent dynamics defects, cost quadrature, constraints). The package is a flat module with generated code addressed through mutable prefix Refs. A detailed roadmap has been written, companion to the CTDirect roadmap.

Guiding principle

The parser describes the problem; it does not discretize it. Scheme-dependent code (defects, quadrature, endpoint-control fixes, grids) belongs to CTDirect.

Key points (from the detailed roadmap)

1. ExaModels 0.11 migration (§1) — Compat blocker for the whole ecosystem (CTDirect and CTSolvers also pin ExaModels = "0.9"). Audit all generated ExaModels call sites, bump compat, coordinate the bump across packages.
2. Move the discretization out of the parser (§2, the structural item) — The :exa backend stops generating scheme-dependent transcription code. Two routes: (1) function tracing — CTDirect traces CTModels functions into ExaModels generators, CTParser needs no :exa backend at all; (2) expression IR — CTParser stores a neutral expression representation, CTDirect generates the Exa transcription from it. Joint with CTDirect §5. Absorbs issues [Dev] Add non uniform grid for exa #65 (non-uniform grid), [Dev] Constrain endpoint controls #161 (endpoint-control constraints), [Dev] Exa retrieve multipliers for boundary and path constraints #134 (exa multipliers).
3. "1-D = scalar" simplification (§3) — Emitted closures should work when callers pass scalars for 1-D state/control/variable. Coordinate with CTDirect §7 and CTModels.
4. Remove prefixes (§4, [Dev] Remove prefixes #224) — Replace mutable prefix Refs (PREFIX_FUN, PREFIX_EXA) with direct module-qualified code (CTModels.…, CTBase.…). Removes global mutable state, makes generated code readable and greppable.
5. Parsing robustness (§5) — Bug sweep ([Dev] Code improvement and bug fixes #197: match-pattern typos, as_range issues, __wrap scoping), stricter parsing with ParsingError for silent mis-parses ([Dev] Stricter parsing #193: leftover symbols, [Dev] Improve parsing #139: undefined time arguments), finalize slice notation ([Bug] Comprehensive Slice Notation Tests #260: x[1:3](t), x[1:end](t), x[1:n](t)).
6. Syntax extensions (§6) — Coordinate-derivative aliases (ẋ₁(t) as sugar for ∂(x₁)(t), [Dev] Coordinate derivatives aliases #71), tensors/matrix-valued state and control ([Dev] Adding tensors #181, cross-package design note first), incremental @def! with $-interpolation (Incremental @def! #32).
7. Parsing engine redesign (§7) — Staged compiler-style pipeline over a typed IR: Stage 0 (surface normalization), Stage 1 (grammar → typed IR nodes), Stage 2 (semantic validation: symbol table, constraint classification, dimension resolution, strictness checks), Stage 3 (backend emission). Replaces the 1500-line monolithic onepass.jl. The IR is route 2's neutral representation — engine redesign and Exa repatriation converge on the same object. Submodule split follows the stages (IR, Grammar, Semantics, FunBackend, ExaBackend, InitialGuess, Defaults).
8. Test suite restructuring (§8) — Adopt the Handbook testing template, organize by functionality (grammar, strictness, codegen_fun, codegen_exa, initial_guess, integration), keep :exa tests as golden oracle for CTDirect's repatriation.
9. Documentation (§9) — Vitepress migration + a complete DSL grammar reference page derived from the §7 IR productions, with doctests.

Why

Separating parsing from semantics via an IR will make the DSL more maintainable and extensible:

• New transformations (multi-phase, penalization, symbolic reduction) can operate at the IR level without touching the parser
• Adding a production = adding one file (node struct + parse rule + validate method + emit methods) — plain open multiple dispatch
• Adding a backend = adding emit methods, not duplicating the grammar
• Removing the ExaModels transcription clarifies the responsibility boundary with CTDirect
• Strictness rules become centralized stage-2 checks on data instead of guards scattered through codegen
• Error accumulation: one @def can report all its errors with positions in one shot

References

• CTParser roadmap discussion
• CTDirect roadmap discussion (companion)
• Handbook
• Roadmap comment