Chaos Computer Club - recent events feed (high quality)

Parallel Python at last? Subinterpreters & free-threading in practice (sps24)


Listen Later

Python has never been good at parallel computing. Multi-threading doesn't scale beyond a handful of threads because of the notorious GIL (Global Interpreter Lock). Multi-processing feels like a cumbersome workaround that increases complexity and overhead. And yet, we're firmly in an era of multicore machines, big data, and massive ML models that require all the compute they can get. Python, otherwise the star of data science and ML, doesn't really shine when it comes to parallel workloads. But that is finally changing! There is more focus and progress happening in this area than ever before, and promising leaps forward are on the horizon. Subinterpreters, already merged in 3.12, offer a tentative step towards making the GIL less than "global" … while the free-threading build of Python 3.13 offers a path towards removing the GIL entirely in the future. Let's explore these new developments and look at how they work, what they do and do not solve, and how we can take advantage of them.
about this event: https://c3voc.de
...more
View all episodesView all episodes
Download on the App Store

Chaos Computer Club - recent events feed (high quality)By CCC media team