What every Python developer should know about the CPython ABI
- Track:
- Python Core, Internals, Extensions
- Type:
- Talk
- Level:
- intermediate
- Duration:
- 30 minutes
Abstract
Most developers interact with the CPython interpreter by writing code in the Python language and executing it with the bytecode interpreter. However, any native extensions and any code that runs alongside or embeds the interpreter needs to communicate with CPython via the CPython C API. This API corresponds to an application binary interface (ABI) that native code written in C, C++, Rust, Zig, or any other low-level programming language needs to use to directly interact with the CPython interpreter runtime. Understanding the ABI allows Python developers to make informed trade-offs when they distribute or depend on packages that contain native extensions.
In this talk, I will explain what the CPython ABI is, how it differs from the CPython C API, and how it determines whether a package from PyPI is installable with a given installation of CPython. I will discuss how CPython's stable ABI and limited C API enable substantially reduced distribution overhead at the cost of some performance overhead. I'll show how maintainers of projects that distribute compiled code in binary wheels can choose to ship wheels using the stable ABI to trade off peak performance with ease of distribution and support for new platforms.
I'll finish by explaining how the ABI is evolving to enable improvements like the free-threaded interpreter. I will share real-world experience of introducing the new free-threaded ABI to the community with the perspective available to a maintainer of open source projects that are core components of the Python ecosystem.
Python 3.15 will enable a new stable ABI that allows distributing one extension per platform for any interpreter newer than Python 3.15. I'll describe work done by myself and others in the community to enable end-to-end testing of the next-generation stable ABI.