FFmpeg lands CLI multi-threading as its "most complex refactoring" in decades

Original Article ↗ Hacker News Discussion ↗

FFmpeg’s command-line interface has just gained end‑to‑end multi‑threading support in what contributors describe as its most complex refactor in decades, enabling different stages of the media pipeline (I/O, filters, encoding) to run in parallel rather than mostly in sequence. Commenters explore where this will and won’t matter in practice, contrasting batch/cloud encoding at scale with one‑off or latency‑sensitive workflows, and noting that many individual codecs were already multi‑threaded. The change also prompts broader debate about how hard concurrency refactors really are, whether tools like LLMs could eventually automate such work, and how FFmpeg compares to alternatives like VapourSynth or custom large‑scale encoding stacks used by Netflix and YouTube.

AI, LLMs, and Code Refactoring

  • Large part of the thread debates whether future LLMs could perform complex refactors like ffmpeg’s new multithreaded CLI overhaul.
  • Proponents say:
    • Current GPT-4 is already useful for smaller refactors (e.g., deduplicating React components, converting serial scripts to parallel forms).
    • The main limitation is context size; with larger windows and automation around tests, LLMs could handle much larger codebases.
    • Refactoring is well-defined (behavior-preserving transformations), making it a good fit for automated tools.
  • Skeptics argue:
    • LLMs lack real “understanding” and struggle with non-trivial, bespoke, or concurrency-heavy code.
    • They often produce plausible but incorrect code, especially out of training distribution.
    • Complex concurrency refactors and performance work require deeper semantic and mathematical reasoning than pattern-based cleanup.
  • Some suggest combining LLMs with existing tools (e.g., semantic patch systems) rather than relying on LLMs alone.

What This Multithreading Change Actually Is

  • Many codecs (H.264, etc.) were already multithreaded; this refactor parallelizes the ffmpeg pipeline itself (reading, decoding, filtering, encoding, writing) so stages can run concurrently.
  • Expected biggest gains: complex filter graphs and multi-output setups, less so simple single-input/single-output transcodes.
  • Some confusion appears in the thread about whether this affects specific codecs (e.g., MP3/LAME); several comments note many older codecs are not well-suited to efficient multithreaded encoding.

Practical Value: Cloud vs Individuals

  • One side: large-scale services (Netflix/YouTube-style) already achieve multicore utilization by running many ffmpeg processes; CLI multithreading adds complexity for little gain at that scale.
  • Other side: for individual users or smaller services running few concurrent jobs, a faster single ffmpeg process is very valuable (e.g., faster one-off encodes, CI pipelines, desktop streaming).

Alternative Pipelines and Chunk-Based Parallelism

  • Some describe using VapourSynth, gstreamer, and tools like av1an to get threaded or per-scene/per-chunk parallelism for years.
  • Discussion on splitting work by keyframe segments vs intra-frame threading:
    • Chunk-based parallelism can improve offline encoding and is used in some pipelines, but adds latency and complexity for live/low-latency streaming.
    • Trade-offs between quality, latency, and CPU utilization are highlighted; no clear consensus on “best” approach.

Project Process and Ecosystem Notes

  • The refactor was large (hundreds of commits) and maintained via mailing-list-style patches rather than modern PR workflows.
  • Some see this as “old school” but accept it as part of long-lived projects like ffmpeg.