Free-threaded CPython is ready to experiment with

Original Article ↗ Hacker News Discussion ↗

Release context & tooling

  • Free-threaded / no-GIL builds have been available in Python 3.13 betas for months; this article is tied to a new documentation + tracking site and SciPy 2024.
  • Install instructions exist for various distros and conda/mamba; there’s a tracking page for ecosystem support and porting guides for C extensions.

Performance & benchmarks

  • Multiple comments note substantial single‑threaded slowdowns (often quoted as ~30–50%) in the experimental free‑threaded 3.13 builds.
  • Core contributors (referenced indirectly) say this cost is expected to shrink via optimizations (e.g., deferred refcounting planned for 3.14) and that free‑threaded won’t be default for years.
  • Some argue this regression undercuts the benefit: a 2‑core run could be slower than current single‑threaded Python.
  • Others point out that standard 3.13 with the GIL retains optimizations without the big penalty; the slowdown is specific to the free‑threaded build.
  • Compared with JIT/AOT languages (Java, Go, Rust, JS), commenters expect free‑threaded Python to remain much slower per core.

Use cases: when no‑GIL helps

  • Enthusiasts expect easier parallelism for:
    • ML data loading and preprocessing (today often done with multiprocessing and many bugs/constraints).
    • Mixed Python/extension workloads where pure‑Python glue could finally scale across cores.
    • Background work (I/O + CPU) without separate processes or complex async rewrites.
  • Skeptics argue most heavy numerical/ML work already runs in C/Fortran/CUDA that releases the GIL, so wins will be modest and mostly in glue code or memory/RAM usage.

Threads vs multiprocessing vs async

  • Strong disagreement:
    • Some call multiprocessing “massive overhead and complexity”; others say it scales fine to hundreds of cores and is simpler/safer due to less shared state.
    • Several describe multiprocessing and fork semantics as full of subtle deadlocks and “footguns”; Python 3.14 is expected to change defaults away from raw fork.
  • Async/await is debated:
    • Some dislike “colored functions” and having to rewrite code to be async‑aware; prefer thread pools for simple concurrency.
    • Others say async makes I/O behavior explicit and works well when used with clear boundaries.

Correctness, race conditions & ecosystem impact

  • Many worry about a new class of race conditions, especially for authors who never wrote truly thread‑safe code.
  • Clarifications: the GIL never guaranteed race‑free Python code; it mostly protected interpreter internals, though its removal increases the “attack surface”.
  • The hard work will fall on CPython internals and native extensions; pure‑Python semantics change little, but any naive move from processes to threads risks new bugs.
  • Some fear another ecosystem split (GIL vs no‑GIL builds) and corporate‑driven direction; others see a long, cautious transition as acceptable.