Free-threaded CPython is ready to experiment with

Original Article ↗ Hacker News Discussion ↗

Free‑threaded CPython, an experimental build of Python 3.13 that can run threads without the Global Interpreter Lock (GIL), is prompting mixed reactions about its benefits and costs. Many see big potential for easier parallelism in areas like data loading, scientific computing, and ML, especially compared to today’s multiprocessing workarounds, but note that the current prototype carries a 30–50% performance hit for single‑threaded code and will take years of optimization and ecosystem porting before it can be a safe default. Others question whether complex multithreading belongs in Python at all, arguing that most heavy computation already lives in C/Fortran extensions or other languages and that removing the GIL risks subtle new bugs for relatively modest real‑world gains.

Release context & tooling

  • Free-threaded / no-GIL builds have been available in Python 3.13 betas for months; this article is tied to a new documentation + tracking site and SciPy 2024.
  • Install instructions exist for various distros and conda/mamba; there’s a tracking page for ecosystem support and porting guides for C extensions.

Performance & benchmarks

  • Multiple comments note substantial single‑threaded slowdowns (often quoted as ~30–50%) in the experimental free‑threaded 3.13 builds.
  • Core contributors (referenced indirectly) say this cost is expected to shrink via optimizations (e.g., deferred refcounting planned for 3.14) and that free‑threaded won’t be default for years.
  • Some argue this regression undercuts the benefit: a 2‑core run could be slower than current single‑threaded Python.
  • Others point out that standard 3.13 with the GIL retains optimizations without the big penalty; the slowdown is specific to the free‑threaded build.
  • Compared with JIT/AOT languages (Java, Go, Rust, JS), commenters expect free‑threaded Python to remain much slower per core.

Use cases: when no‑GIL helps

  • Enthusiasts expect easier parallelism for:
    • ML data loading and preprocessing (today often done with multiprocessing and many bugs/constraints).
    • Mixed Python/extension workloads where pure‑Python glue could finally scale across cores.
    • Background work (I/O + CPU) without separate processes or complex async rewrites.
  • Skeptics argue most heavy numerical/ML work already runs in C/Fortran/CUDA that releases the GIL, so wins will be modest and mostly in glue code or memory/RAM usage.

Threads vs multiprocessing vs async

  • Strong disagreement:
    • Some call multiprocessing “massive overhead and complexity”; others say it scales fine to hundreds of cores and is simpler/safer due to less shared state.
    • Several describe multiprocessing and fork semantics as full of subtle deadlocks and “footguns”; Python 3.14 is expected to change defaults away from raw fork.
  • Async/await is debated:
    • Some dislike “colored functions” and having to rewrite code to be async‑aware; prefer thread pools for simple concurrency.
    • Others say async makes I/O behavior explicit and works well when used with clear boundaries.

Correctness, race conditions & ecosystem impact

  • Many worry about a new class of race conditions, especially for authors who never wrote truly thread‑safe code.
  • Clarifications: the GIL never guaranteed race‑free Python code; it mostly protected interpreter internals, though its removal increases the “attack surface”.
  • The hard work will fall on CPython internals and native extensions; pure‑Python semantics change little, but any naive move from processes to threads risks new bugs.
  • Some fear another ecosystem split (GIL vs no‑GIL builds) and corporate‑driven direction; others see a long, cautious transition as acceptable.