Skip to content

Commit 9215226

Browse files
corona10sobolevnhugovk
authored
PEP 841: Adding Frozen Syntax to Make Immutable Types Optimizable (#5049)
* pep-841: Adding Frozen Syntax to Make Immutable Types Optimizable * Add CODEOWNERS * Add DPO * Address review Review: corona10@a34737e#r192990084 * Fix CI * Fix CI * Update constant dedup section * Apply suggestions from code review Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> --------- Co-authored-by: sobolevn <mail@sobolevn.me> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
1 parent d42cb1d commit 9215226

2 files changed

Lines changed: 317 additions & 0 deletions

File tree

.github/CODEOWNERS

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -714,6 +714,7 @@ peps/pep-0836.rst @savannahostrowski @Fidget-Spinner @brandtbucher
714714
peps/pep-0837.rst @serhiy-storchaka
715715
peps/pep-0838.rst @AlexWaygood
716716
peps/pep-0840.rst @jeremyhylton @gvanrossum
717+
peps/pep-0841.rst @corona10 @sobolevn
717718
peps/pep-0842.rst @ZeroIntensity
718719
# ...
719720
peps/pep-2026.rst @hugovk

peps/pep-0841.rst

Lines changed: 316 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,316 @@
1+
PEP: 841
2+
Title: Adding Frozen Syntax to Optimize Immutable Types
3+
Author: Donghee Na <donghee.na@python.org>,
4+
Nikita Sobolev <mail@sobolevn.me>
5+
Discussions-To: https://discuss.python.org/t/pep-841-adding-frozen-syntax-to-make-immutable-types-optimizable/108219
6+
Status: Draft
7+
Type: Standards Track
8+
Created: 20-Jul-2026
9+
Python-Version: 3.16
10+
Post-History: `20-Jul-2026 <https://discuss.python.org/t/pep-841-adding-frozen-syntax-to-make-immutable-types-optimizable/108219>`__
11+
12+
13+
Abstract
14+
========
15+
16+
This PEP proposes *frozen display* syntax: ``f{1, 2, 3}`` evaluates to a
17+
:class:`frozenset`, and ``f{'a': 1}`` evaluates to a ``frozendict``.
18+
Because immutability is guaranteed by the syntax itself rather than
19+
inferred from usage, the compiler can treat frozen displays as first-class
20+
citizens of its optimization pipeline: constant displays are folded into a
21+
single ``LOAD_CONST`` with an exact result type at compile time and
22+
cached in ``.pyc`` files.
23+
24+
25+
Motivation
26+
==========
27+
28+
Python has display syntax for its mutable containers but none for its
29+
immutable ones. Today an immutable set must be written as
30+
``frozenset({1, 2, 3})`` and an immutable mapping as
31+
``frozendict({'a': 1})``. Each of these:
32+
33+
* builds a mutable set or dict, then copies it into the immutable type.
34+
* looks up the name ``frozenset`` or ``frozendict`` at runtime on every
35+
execution.
36+
* cannot be easily optimized by the compiler, because either name may be
37+
rebound and the call may have arbitrary side effects, or
38+
value can be of unexpected type for the optimization,
39+
value can be non uniquely referenced.
40+
41+
CPython already hints at the opportunity: the peephole optimizer rewrites
42+
a constant set display into a frozenset, but only as the right operand
43+
of ``in``. Assign the same display to a variable and the optimization is
44+
gone. The root cause is that the compiler can never prove immutability
45+
of a ``set`` or ``dict`` display, so it must rebuild it on every
46+
execution. A display whose *semantics* guarantee immutability removes
47+
that barrier once and for all.
48+
49+
One of the goals of this PEP is to increase the use of immutable containers
50+
in CPython, preparing for a concurrent future in which free-threading and
51+
subinterpreters become significantly more common.
52+
Efficient creation of immutable data structures and convenient syntax would
53+
encourage the use of ``frozendict`` and ``frozenset``. This would reduce bugs
54+
caused by accidental mutation of shared mutable containers while also
55+
improving performance. For example, subinterpreters already share
56+
``frozenset`` objects, and there are plans to share ``frozendict`` objects as
57+
well.
58+
59+
Immutable container displays have been requested and discussed by the
60+
community several times, most recently in the `frozenset and frozendict
61+
comprehensions
62+
<https://discuss.python.org/t/frozenset-and-frozendict-comprehensions/101584>`__
63+
thread on Discourse.
64+
65+
66+
Rationale
67+
=========
68+
69+
Syntax, not a builtin call
70+
--------------------------
71+
72+
Only syntax gives the compiler a semantic guarantee. A call to
73+
``frozenset(...)`` can be shadowed, but a frozen display cannot. Every
74+
optimization described below follows from this single property.
75+
76+
Static analysis benefits in the same way. Today, tools
77+
that don't perform semantic analysis must assume
78+
that ``frozenset(...)`` refers to the builtin. A frozen display turns
79+
that assumption into a syntactic guarantee, so analyzers can treat the
80+
result as immutable with full confidence. This holds even for purely
81+
syntactic tools that perform no name resolution.
82+
83+
Linters and formatters can automatically rewrite ``frozenset({1, 2, 3})``
84+
and ``frozendict({1: 2})`` to be ``f{1, 2, 3}`` and ``f{1: 2}``
85+
on newer Python versions.
86+
87+
Why ``f{...}``
88+
--------------
89+
90+
The ``f`` prefix reads as *frozen*, mirroring the familiar f-string
91+
prefix convention. Sharing the letter with f-strings is not a problem:
92+
strings are immutable too, so either way an ``f`` prefixed expression
93+
evaluates to an immutable value. ``f{`` is a syntax error in all
94+
current Python versions, so the syntax is fully backward compatible. The tokenizer
95+
emits a single ``FLBRACE`` token for ``f{``, so ``f {1}`` (with a space)
96+
remains an error and there is no ambiguity with the name ``f`` or with
97+
f-strings.
98+
99+
100+
Specification
101+
=============
102+
103+
Grammar
104+
-------
105+
106+
Our goal is to make new syntax and grammar identical
107+
to existing ``set`` and ``dict`` syntax and grammar rules.
108+
109+
New alternatives are added to ``atom``, mirroring ``set`` and ``dict``
110+
displays and comprehensions:
111+
112+
.. code-block:: peg
113+
114+
fset: FLBRACE star_named_expressions '}'
115+
fsetcomp: FLBRACE star_named_expression for_if_clauses '}'
116+
fdict: FLBRACE [double_starred_kvpairs] '}'
117+
fdictcomp: FLBRACE kvpair for_if_clauses '}'
118+
119+
* ``f{1, 2, 3}`` is a frozenset display.
120+
* ``f{'a': 1, 'b': 2}`` is a frozendict display.
121+
* ``f{}`` is an empty frozendict, mirroring ``{}``.
122+
* Star unpacking follows the existing displays: ``f{*xs}`` is a
123+
frozenset display (like ``{*xs}``) and ``f{**d}`` is a frozendict
124+
display (like ``{**d}``).
125+
* Comprehensions are supported: ``f{x for x in xs}`` is a frozenset
126+
comprehension and ``f{k: v for k, v in items}`` is a frozendict
127+
comprehension.
128+
* Async comprehensions are also supported
129+
in async contexts: ``f{x async for arange(5)}``
130+
is a frozenset async comprehension
131+
and ``f{k: v async for k, v in items}``
132+
is a frozendict async comprehension.
133+
* :pep:`798` and unpacking in comprehensions is also supported.
134+
``f{*nums for nums in list_of_nums}`` is a frozenset
135+
compehension with unpacking and ``f{**items for nums in list_of_items}``
136+
is a frozendict comprehension with unpacking.
137+
138+
AST
139+
---
140+
141+
Four new expression nodes are added: ``FrozenSet(elts)``,
142+
``FrozenDict(keys, values)``, ``FrozenSetComp(elt, generators)``, and
143+
``FrozenDictComp(key, value, generators)``, structurally identical to
144+
their mutable counterparts. Distinct nodes (rather than a flag) let
145+
every downstream
146+
consumer (e.g. the symbol table, the AST optimizer, the code generator,
147+
and third-party tools) dispatch on immutability directly.
148+
149+
Semantics
150+
---------
151+
152+
A frozenset display evaluates to exactly what ``frozenset({...})``
153+
returns. A frozendict display evaluates to exactly what
154+
``frozendict({...})`` returns. Both result types are immutable and
155+
hashable, which is what makes the compile-time treatment below sound.
156+
157+
Bytecode
158+
--------
159+
160+
Two new instructions are added:
161+
162+
* ``BUILD_FROZENSET (count)`` works like ``BUILD_SET``, but the freshly
163+
created, uniquely referenced set is frozen in place with no copy.
164+
* ``BUILD_FROZENMAP (count)`` works like ``BUILD_MAP``, but creates a
165+
``frozendict``.
166+
167+
``count`` is an ordinary oparg with the same format and meaning as in
168+
the existing ``BUILD_SET`` and ``BUILD_MAP`` instructions.
169+
170+
Displays that use star-unpacking or exceed the stack-use guideline fall
171+
back to building the mutable container and freezing it in place. The
172+
result is indistinguishable. Comprehensions take the same path, so
173+
they need no new opcodes.
174+
175+
176+
The optimization pipeline
177+
=========================
178+
179+
The central claim of this PEP is that frozen displays are not merely
180+
convenient syntax: they give every stage of the compiler a guarantee it
181+
can act on. The reference implementation already exercises the full
182+
pipeline:
183+
184+
1. **AST preprocessing.** ``FrozenSet`` and ``FrozenDict`` participate
185+
in AST-level constant folding of their elements.
186+
187+
2. **Code generation.** The common case compiles to a single
188+
``BUILD_FROZENSET`` / ``BUILD_FROZENDICT`` instruction with no name
189+
lookup, no temporary copy, and an exact, statically known result type
190+
(used by the compiler's type inference, e.g. to reject
191+
``f{1, 2}[0]`` at compile time).
192+
193+
3. **Control flow graph (CFG) constant folding.** A display whose
194+
keys and values are all constants is folded into a single
195+
``LOAD_CONST``, serialized into the ``.pyc`` by marshal, and shared
196+
across all executions: zero per-execution construction cost. Unlike
197+
the existing list/set folds, this is *unconditionally* valid, since
198+
immutability comes from the language semantics, not from how the
199+
value is used. A display with a non-constant element, e.g.
200+
``f{'key': ['list']}``, is still built at runtime.
201+
202+
4. **Constant deduplication.** Frozen constants participate in
203+
``co_consts`` deduplication. For a frozendict the deduplication key
204+
keeps the insertion order, so a display never loses its iteration
205+
order to an equal display with a different key order.
206+
207+
.. note::
208+
209+
This PEP deliberately makes no claims about JIT-level optimization:
210+
the JIT project is currently on hold following the `Steering
211+
Council's announcement
212+
<https://discuss.python.org/t/an-announcement-from-the-steering-council-regarding-the-jit-project/107638>`__.
213+
214+
The pipeline also opens future work that mutable displays can never
215+
support: sharing folded frozen constants across code objects and
216+
immortalizing them under free threading.
217+
218+
219+
Backwards Compatibility
220+
=======================
221+
222+
``f{`` is a syntax error today, so no existing code changes meaning.
223+
The changes visible to tooling are: a new ``FLBRACE`` token, four new
224+
AST node types, two new opcodes, and a bytecode magic number bump.
225+
226+
227+
How to Teach This
228+
=================
229+
230+
"Prefix a set or dict display with ``f`` to make it frozen".
231+
Style guidance: prefer ``f{...}`` over
232+
``frozenset({...})`` for literal values on Python 3.16+.
233+
Constant frozen displays are free after the first execution.
234+
235+
236+
Impact on the Standard Library
237+
==============================
238+
239+
A quick survey of the standard library (excluding tests) finds about
240+
105 ``frozenset(...)`` and 65 ``frozendict(...)`` call sites, of which
241+
about 46 and 22 respectively pass a literal display and could be
242+
written as ``f{...}``. They spread across widely used modules such as
243+
``typing``, ``dataclasses``, ``functools``, ``copy``, and
244+
``traceback``.
245+
246+
These numbers are only an estimate of the potential effect. This PEP
247+
does not propose a mechanical rewrite of the standard library.
248+
249+
250+
Reference Implementation
251+
========================
252+
253+
A complete implementation, including the parser, AST, code generator,
254+
and CFG constant folding, is available in the `fset_fdict branch
255+
<https://github.com/corona10/cpython/tree/fset_fdict>`__ of the author's
256+
CPython fork.
257+
258+
259+
Rejected Ideas
260+
==============
261+
262+
Alternative spellings
263+
---------------------
264+
265+
Many spellings were considered. ``f`` was chosen simply because it is
266+
the prefix that best evokes *frozen*:
267+
268+
* Single letter prefixes: ``i{'key': 1}`` (immutable), ``z{'key': 1}``.
269+
This was rejected because types are called ``frozen``,
270+
``f`` as a prefix reads the best.
271+
* Multi letter prefixes: ``fr{'key': 1}``, ``fz{'key': 1}``,
272+
``frz{'key': 1}``. This was rejected because it just adds an extra letter
273+
to write and read with no extra real value.
274+
* Symbol prefixes: ``${'key': 1}``, ``+{'key': 1}``.
275+
This was rejected because most symbols already have a meaning as operators.
276+
We can't reuse them for this new purpose.
277+
We also don't want to add new symbols like ``$``
278+
to keep them for something else in the future.
279+
* Bracket variants: ``{{'key': 1}}``, ``|{'key': 1}|``, ``{|'key': 1|}``.
280+
This was rejected because adding
281+
a single token ``f{`` is easier than adding two tokens.
282+
Writing ``f{`` is also easier than writing two different brackets.
283+
``{{}}`` is rejected because it is a valid syntax right now.
284+
* Word prefixes: ``frozen{'key': 1}``, ``fdict{'key': 1}``,
285+
``frozendict {'key': 1}``.
286+
This was rejected because it is rather verbose.
287+
288+
We also rejected using ``F{`` as it is possible with fstrings.
289+
This was rejected to keep the syntax as minimalistic as possible
290+
and not to create extra ``F{`` token.
291+
292+
Freezing methods
293+
----------------
294+
295+
Methods such as ``{'key': 1}.freeze()`` or
296+
``{'key': 1}.take_frozendict()`` are not real alternatives: they can be
297+
added independently of this PEP.
298+
299+
While such methods can be a great feature on its own,
300+
making this the only way to create immutable containers is not an option:
301+
302+
* Multiline expressions with such methods are really hard to read,
303+
because you need to read the very last line to know the type of the object.
304+
* It is quite verbose to write for common cases.
305+
* It does not provide syntax guarantees for static analysis tools.
306+
* It may have a different semantics when used
307+
as ``a = {1: 2}; b(a.take_frozenset())``,
308+
depending on `how the internals would look like <https://github.com/capi-workgroup/decisions/issues/109>`_,
309+
it might mean that ``a`` would be cleared.
310+
311+
312+
Copyright
313+
=========
314+
315+
This document is placed in the public domain or under the
316+
CC0-1.0-Universal license, whichever is more permissive.

0 commit comments

Comments
 (0)