Immutability: Fast and Safe sharing of Data across Subinterpreters
- Track:
- Python Core, Internals, Extensions
- Type:
- Talk
- Level:
- intermediate
- Duration:
- 30 minutes
Abstract
Python supports parallelism through multiple sub-interpreters. Parallelism though sub-interpreters avoid the restrictions of a single GIL, and isolating parallel processes from each other is safe from data-races. The downside is that subinterpreter isolation comes with a high cost: sending objects across sub-interpreters typically involves pickling and copying which has negative impact on performance with both CPU and memory overheads. This talk is about PEP795 which adds immutability to Python, which permits sub-interpreters to transfer and share immutable objects directly by reference.
We will present the design and rationale for immutability in Python, the API for creating and managing immutable state, decorators for communicating immutability in code, as well as opting out of immutability where necessary. We will also discuss our prototype implementation on-top of Python 3.15, and demonstrate how immutable sharing across sub-interpreters improves performance, and compare the speed of freezing (the act of making objects immutable) to pickling and unpickling.
Attendees will learn:
- About the semantics of immutability in Python, and what are the challenges of designing immutability to support sharing across sub-interpreters.
- When immutability can improve performance and predictability in concurrent Python.
- How freezing works for built-in types, user‑defined types and functions, and how it interacts with sub-interpreters.
- Performance of pickling vs. immutable sharing, with numbers and failure modes.
- Current limitations of immutability, risks, and what would need to land in CPython for this to be widely usable. Target audience: programmers interested in concurrent programming in Python.
Note: This PEP is not yet accepted; the talk focuses on design, prototype results, and open questions.