|
| 1 | +PEP: 841 |
| 2 | +Title: Adding Frozen Syntax to Optimize Immutable Types |
| 3 | +Author: Donghee Na <donghee.na@python.org>, |
| 4 | + Nikita Sobolev <mail@sobolevn.me> |
| 5 | +Discussions-To: https://discuss.python.org/t/pep-841-adding-frozen-syntax-to-make-immutable-types-optimizable/108219 |
| 6 | +Status: Draft |
| 7 | +Type: Standards Track |
| 8 | +Created: 20-Jul-2026 |
| 9 | +Python-Version: 3.16 |
| 10 | +Post-History: `20-Jul-2026 <https://discuss.python.org/t/pep-841-adding-frozen-syntax-to-make-immutable-types-optimizable/108219>`__ |
| 11 | + |
| 12 | + |
| 13 | +Abstract |
| 14 | +======== |
| 15 | + |
| 16 | +This PEP proposes *frozen display* syntax: ``f{1, 2, 3}`` evaluates to a |
| 17 | +:class:`frozenset`, and ``f{'a': 1}`` evaluates to a ``frozendict``. |
| 18 | +Because immutability is guaranteed by the syntax itself rather than |
| 19 | +inferred from usage, the compiler can treat frozen displays as first-class |
| 20 | +citizens of its optimization pipeline: constant displays are folded into a |
| 21 | +single ``LOAD_CONST`` with an exact result type at compile time and |
| 22 | +cached in ``.pyc`` files. |
| 23 | + |
| 24 | + |
| 25 | +Motivation |
| 26 | +========== |
| 27 | + |
| 28 | +Python has display syntax for its mutable containers but none for its |
| 29 | +immutable ones. Today an immutable set must be written as |
| 30 | +``frozenset({1, 2, 3})`` and an immutable mapping as |
| 31 | +``frozendict({'a': 1})``. Each of these: |
| 32 | + |
| 33 | +* builds a mutable set or dict, then copies it into the immutable type. |
| 34 | +* looks up the name ``frozenset`` or ``frozendict`` at runtime on every |
| 35 | + execution. |
| 36 | +* cannot be easily optimized by the compiler, because either name may be |
| 37 | + rebound and the call may have arbitrary side effects, or |
| 38 | + value can be of unexpected type for the optimization, |
| 39 | + value can be non uniquely referenced. |
| 40 | + |
| 41 | +CPython already hints at the opportunity: the peephole optimizer rewrites |
| 42 | +a constant set display into a frozenset, but only as the right operand |
| 43 | +of ``in``. Assign the same display to a variable and the optimization is |
| 44 | +gone. The root cause is that the compiler can never prove immutability |
| 45 | +of a ``set`` or ``dict`` display, so it must rebuild it on every |
| 46 | +execution. A display whose *semantics* guarantee immutability removes |
| 47 | +that barrier once and for all. |
| 48 | + |
| 49 | +One of the goals of this PEP is to increase the use of immutable containers |
| 50 | +in CPython, preparing for a concurrent future in which free-threading and |
| 51 | +subinterpreters become significantly more common. |
| 52 | +Efficient creation of immutable data structures and convenient syntax would |
| 53 | +encourage the use of ``frozendict`` and ``frozenset``. This would reduce bugs |
| 54 | +caused by accidental mutation of shared mutable containers while also |
| 55 | +improving performance. For example, subinterpreters already share |
| 56 | +``frozenset`` objects, and there are plans to share ``frozendict`` objects as |
| 57 | +well. |
| 58 | + |
| 59 | +Immutable container displays have been requested and discussed by the |
| 60 | +community several times, most recently in the `frozenset and frozendict |
| 61 | +comprehensions |
| 62 | +<https://discuss.python.org/t/frozenset-and-frozendict-comprehensions/101584>`__ |
| 63 | +thread on Discourse. |
| 64 | + |
| 65 | + |
| 66 | +Rationale |
| 67 | +========= |
| 68 | + |
| 69 | +Syntax, not a builtin call |
| 70 | +-------------------------- |
| 71 | + |
| 72 | +Only syntax gives the compiler a semantic guarantee. A call to |
| 73 | +``frozenset(...)`` can be shadowed, but a frozen display cannot. Every |
| 74 | +optimization described below follows from this single property. |
| 75 | + |
| 76 | +Static analysis benefits in the same way. Today, tools |
| 77 | +that don't perform semantic analysis must assume |
| 78 | +that ``frozenset(...)`` refers to the builtin. A frozen display turns |
| 79 | +that assumption into a syntactic guarantee, so analyzers can treat the |
| 80 | +result as immutable with full confidence. This holds even for purely |
| 81 | +syntactic tools that perform no name resolution. |
| 82 | + |
| 83 | +Linters and formatters can automatically rewrite ``frozenset({1, 2, 3})`` |
| 84 | +and ``frozendict({1: 2})`` to be ``f{1, 2, 3}`` and ``f{1: 2}`` |
| 85 | +on newer Python versions. |
| 86 | + |
| 87 | +Why ``f{...}`` |
| 88 | +-------------- |
| 89 | + |
| 90 | +The ``f`` prefix reads as *frozen*, mirroring the familiar f-string |
| 91 | +prefix convention. Sharing the letter with f-strings is not a problem: |
| 92 | +strings are immutable too, so either way an ``f`` prefixed expression |
| 93 | +evaluates to an immutable value. ``f{`` is a syntax error in all |
| 94 | +current Python versions, so the syntax is fully backward compatible. The tokenizer |
| 95 | +emits a single ``FLBRACE`` token for ``f{``, so ``f {1}`` (with a space) |
| 96 | +remains an error and there is no ambiguity with the name ``f`` or with |
| 97 | +f-strings. |
| 98 | + |
| 99 | + |
| 100 | +Specification |
| 101 | +============= |
| 102 | + |
| 103 | +Grammar |
| 104 | +------- |
| 105 | + |
| 106 | +Our goal is to make new syntax and grammar identical |
| 107 | +to existing ``set`` and ``dict`` syntax and grammar rules. |
| 108 | + |
| 109 | +New alternatives are added to ``atom``, mirroring ``set`` and ``dict`` |
| 110 | +displays and comprehensions: |
| 111 | + |
| 112 | +.. code-block:: peg |
| 113 | +
|
| 114 | + fset: FLBRACE star_named_expressions '}' |
| 115 | + fsetcomp: FLBRACE star_named_expression for_if_clauses '}' |
| 116 | + fdict: FLBRACE [double_starred_kvpairs] '}' |
| 117 | + fdictcomp: FLBRACE kvpair for_if_clauses '}' |
| 118 | +
|
| 119 | +* ``f{1, 2, 3}`` is a frozenset display. |
| 120 | +* ``f{'a': 1, 'b': 2}`` is a frozendict display. |
| 121 | +* ``f{}`` is an empty frozendict, mirroring ``{}``. |
| 122 | +* Star unpacking follows the existing displays: ``f{*xs}`` is a |
| 123 | + frozenset display (like ``{*xs}``) and ``f{**d}`` is a frozendict |
| 124 | + display (like ``{**d}``). |
| 125 | +* Comprehensions are supported: ``f{x for x in xs}`` is a frozenset |
| 126 | + comprehension and ``f{k: v for k, v in items}`` is a frozendict |
| 127 | + comprehension. |
| 128 | +* Async comprehensions are also supported |
| 129 | + in async contexts: ``f{x async for arange(5)}`` |
| 130 | + is a frozenset async comprehension |
| 131 | + and ``f{k: v async for k, v in items}`` |
| 132 | + is a frozendict async comprehension. |
| 133 | +* :pep:`798` and unpacking in comprehensions is also supported. |
| 134 | + ``f{*nums for nums in list_of_nums}`` is a frozenset |
| 135 | + compehension with unpacking and ``f{**items for nums in list_of_items}`` |
| 136 | + is a frozendict comprehension with unpacking. |
| 137 | + |
| 138 | +AST |
| 139 | +--- |
| 140 | + |
| 141 | +Four new expression nodes are added: ``FrozenSet(elts)``, |
| 142 | +``FrozenDict(keys, values)``, ``FrozenSetComp(elt, generators)``, and |
| 143 | +``FrozenDictComp(key, value, generators)``, structurally identical to |
| 144 | +their mutable counterparts. Distinct nodes (rather than a flag) let |
| 145 | +every downstream |
| 146 | +consumer (e.g. the symbol table, the AST optimizer, the code generator, |
| 147 | +and third-party tools) dispatch on immutability directly. |
| 148 | + |
| 149 | +Semantics |
| 150 | +--------- |
| 151 | + |
| 152 | +A frozenset display evaluates to exactly what ``frozenset({...})`` |
| 153 | +returns. A frozendict display evaluates to exactly what |
| 154 | +``frozendict({...})`` returns. Both result types are immutable and |
| 155 | +hashable, which is what makes the compile-time treatment below sound. |
| 156 | + |
| 157 | +Bytecode |
| 158 | +-------- |
| 159 | + |
| 160 | +Two new instructions are added: |
| 161 | + |
| 162 | +* ``BUILD_FROZENSET (count)`` works like ``BUILD_SET``, but the freshly |
| 163 | + created, uniquely referenced set is frozen in place with no copy. |
| 164 | +* ``BUILD_FROZENMAP (count)`` works like ``BUILD_MAP``, but creates a |
| 165 | + ``frozendict``. |
| 166 | + |
| 167 | +``count`` is an ordinary oparg with the same format and meaning as in |
| 168 | +the existing ``BUILD_SET`` and ``BUILD_MAP`` instructions. |
| 169 | + |
| 170 | +Displays that use star-unpacking or exceed the stack-use guideline fall |
| 171 | +back to building the mutable container and freezing it in place. The |
| 172 | +result is indistinguishable. Comprehensions take the same path, so |
| 173 | +they need no new opcodes. |
| 174 | + |
| 175 | + |
| 176 | +The optimization pipeline |
| 177 | +========================= |
| 178 | + |
| 179 | +The central claim of this PEP is that frozen displays are not merely |
| 180 | +convenient syntax: they give every stage of the compiler a guarantee it |
| 181 | +can act on. The reference implementation already exercises the full |
| 182 | +pipeline: |
| 183 | + |
| 184 | +1. **AST preprocessing.** ``FrozenSet`` and ``FrozenDict`` participate |
| 185 | + in AST-level constant folding of their elements. |
| 186 | + |
| 187 | +2. **Code generation.** The common case compiles to a single |
| 188 | + ``BUILD_FROZENSET`` / ``BUILD_FROZENDICT`` instruction with no name |
| 189 | + lookup, no temporary copy, and an exact, statically known result type |
| 190 | + (used by the compiler's type inference, e.g. to reject |
| 191 | + ``f{1, 2}[0]`` at compile time). |
| 192 | + |
| 193 | +3. **Control flow graph (CFG) constant folding.** A display whose |
| 194 | + keys and values are all constants is folded into a single |
| 195 | + ``LOAD_CONST``, serialized into the ``.pyc`` by marshal, and shared |
| 196 | + across all executions: zero per-execution construction cost. Unlike |
| 197 | + the existing list/set folds, this is *unconditionally* valid, since |
| 198 | + immutability comes from the language semantics, not from how the |
| 199 | + value is used. A display with a non-constant element, e.g. |
| 200 | + ``f{'key': ['list']}``, is still built at runtime. |
| 201 | + |
| 202 | +4. **Constant deduplication.** Frozen constants participate in |
| 203 | + ``co_consts`` deduplication. For a frozendict the deduplication key |
| 204 | + keeps the insertion order, so a display never loses its iteration |
| 205 | + order to an equal display with a different key order. |
| 206 | + |
| 207 | +.. note:: |
| 208 | + |
| 209 | + This PEP deliberately makes no claims about JIT-level optimization: |
| 210 | + the JIT project is currently on hold following the `Steering |
| 211 | + Council's announcement |
| 212 | + <https://discuss.python.org/t/an-announcement-from-the-steering-council-regarding-the-jit-project/107638>`__. |
| 213 | + |
| 214 | +The pipeline also opens future work that mutable displays can never |
| 215 | +support: sharing folded frozen constants across code objects and |
| 216 | +immortalizing them under free threading. |
| 217 | + |
| 218 | + |
| 219 | +Backwards Compatibility |
| 220 | +======================= |
| 221 | + |
| 222 | +``f{`` is a syntax error today, so no existing code changes meaning. |
| 223 | +The changes visible to tooling are: a new ``FLBRACE`` token, four new |
| 224 | +AST node types, two new opcodes, and a bytecode magic number bump. |
| 225 | + |
| 226 | + |
| 227 | +How to Teach This |
| 228 | +================= |
| 229 | + |
| 230 | +"Prefix a set or dict display with ``f`` to make it frozen". |
| 231 | +Style guidance: prefer ``f{...}`` over |
| 232 | +``frozenset({...})`` for literal values on Python 3.16+. |
| 233 | +Constant frozen displays are free after the first execution. |
| 234 | + |
| 235 | + |
| 236 | +Impact on the Standard Library |
| 237 | +============================== |
| 238 | + |
| 239 | +A quick survey of the standard library (excluding tests) finds about |
| 240 | +105 ``frozenset(...)`` and 65 ``frozendict(...)`` call sites, of which |
| 241 | +about 46 and 22 respectively pass a literal display and could be |
| 242 | +written as ``f{...}``. They spread across widely used modules such as |
| 243 | +``typing``, ``dataclasses``, ``functools``, ``copy``, and |
| 244 | +``traceback``. |
| 245 | + |
| 246 | +These numbers are only an estimate of the potential effect. This PEP |
| 247 | +does not propose a mechanical rewrite of the standard library. |
| 248 | + |
| 249 | + |
| 250 | +Reference Implementation |
| 251 | +======================== |
| 252 | + |
| 253 | +A complete implementation, including the parser, AST, code generator, |
| 254 | +and CFG constant folding, is available in the `fset_fdict branch |
| 255 | +<https://github.com/corona10/cpython/tree/fset_fdict>`__ of the author's |
| 256 | +CPython fork. |
| 257 | + |
| 258 | + |
| 259 | +Rejected Ideas |
| 260 | +============== |
| 261 | + |
| 262 | +Alternative spellings |
| 263 | +--------------------- |
| 264 | + |
| 265 | +Many spellings were considered. ``f`` was chosen simply because it is |
| 266 | +the prefix that best evokes *frozen*: |
| 267 | + |
| 268 | +* Single letter prefixes: ``i{'key': 1}`` (immutable), ``z{'key': 1}``. |
| 269 | + This was rejected because types are called ``frozen``, |
| 270 | + ``f`` as a prefix reads the best. |
| 271 | +* Multi letter prefixes: ``fr{'key': 1}``, ``fz{'key': 1}``, |
| 272 | + ``frz{'key': 1}``. This was rejected because it just adds an extra letter |
| 273 | + to write and read with no extra real value. |
| 274 | +* Symbol prefixes: ``${'key': 1}``, ``+{'key': 1}``. |
| 275 | + This was rejected because most symbols already have a meaning as operators. |
| 276 | + We can't reuse them for this new purpose. |
| 277 | + We also don't want to add new symbols like ``$`` |
| 278 | + to keep them for something else in the future. |
| 279 | +* Bracket variants: ``{{'key': 1}}``, ``|{'key': 1}|``, ``{|'key': 1|}``. |
| 280 | + This was rejected because adding |
| 281 | + a single token ``f{`` is easier than adding two tokens. |
| 282 | + Writing ``f{`` is also easier than writing two different brackets. |
| 283 | + ``{{}}`` is rejected because it is a valid syntax right now. |
| 284 | +* Word prefixes: ``frozen{'key': 1}``, ``fdict{'key': 1}``, |
| 285 | + ``frozendict {'key': 1}``. |
| 286 | + This was rejected because it is rather verbose. |
| 287 | + |
| 288 | +We also rejected using ``F{`` as it is possible with fstrings. |
| 289 | +This was rejected to keep the syntax as minimalistic as possible |
| 290 | +and not to create extra ``F{`` token. |
| 291 | + |
| 292 | +Freezing methods |
| 293 | +---------------- |
| 294 | + |
| 295 | +Methods such as ``{'key': 1}.freeze()`` or |
| 296 | +``{'key': 1}.take_frozendict()`` are not real alternatives: they can be |
| 297 | +added independently of this PEP. |
| 298 | + |
| 299 | +While such methods can be a great feature on its own, |
| 300 | +making this the only way to create immutable containers is not an option: |
| 301 | + |
| 302 | +* Multiline expressions with such methods are really hard to read, |
| 303 | + because you need to read the very last line to know the type of the object. |
| 304 | +* It is quite verbose to write for common cases. |
| 305 | +* It does not provide syntax guarantees for static analysis tools. |
| 306 | +* It may have a different semantics when used |
| 307 | + as ``a = {1: 2}; b(a.take_frozenset())``, |
| 308 | + depending on `how the internals would look like <https://github.com/capi-workgroup/decisions/issues/109>`_, |
| 309 | + it might mean that ``a`` would be cleared. |
| 310 | + |
| 311 | + |
| 312 | +Copyright |
| 313 | +========= |
| 314 | + |
| 315 | +This document is placed in the public domain or under the |
| 316 | +CC0-1.0-Universal license, whichever is more permissive. |
0 commit comments