Whisper: Nvidia RTX 4090 vs. M1 Pro with MLX
Benchmarks of OpenAI’s Whisper speech-to-text model on Apple’s M‑series chips using the new MLX framework prompted comparisons with an Nvidia RTX 4090, raising questions about how much optimization and software stack choice skew performance results. Commenters note that while a Mac with MLX can approach an unoptimized 4090 run, highly tuned CUDA-based implementations like “insanely-fast-whisper” make the 4090 many times faster in practice. The thread converges on a view that Apple Silicon offers impressive efficiency and good “free” local ML capability for Mac owners, but remains far from Nvidia’s high end for serious or large-scale machine learning workloads.
Benchmark validity and fairness
- Many find the original comparison questionable: it used a relatively unoptimized Whisper implementation on the RTX 4090 versus an MLX‑tuned version on Apple Silicon.
- Commenters who reran the test with an optimized CUDA stack (e.g., insanely‑fast‑whisper) report the 4090 being roughly 6–12× faster than the Mac result, not ~16% faster.
- Several note that different Whisper libraries (OpenAI baseline, whisper.cpp, faster‑whisper, ctranslate2, insanely‑fast‑whisper) vary by large factors, so cross‑platform claims can easily become apples‑to‑oranges.
- Some argue a fair comparison should use the fastest available implementation on each platform; others stress matching algorithmic settings (batching, beam size) to avoid skew.
Whisper implementations and optimization
- Highly optimized Nvidia implementations use batching, custom runtimes, kernel fusion, and features like FlashAttention or BetterTransformer.
- MLX is very new; its Whisper example is considered optimized for Apple Silicon but not “hand‑tuned” at CUDA‑level depth.
- whisper.cpp and CoreML/Metal paths have Apple‑specific optimizations (e.g., encoder on ANE, decoder on GPU/CPU).
- Quality is usually identical when only batching and kernel optimizations change; quantization or search simplifications can hurt accuracy.
Hardware performance and architecture
- Consensus: RTX 4090 (and Nvidia generally) is far ahead in raw ML throughput; M‑series is “impressive for a laptop/desktop SoC” but not truly competitive at the high end.
- Discussion disentangles “cores,” ALUs, shaders, and marketing numbers across Apple and Nvidia architectures.
- Some note different model types stress hardware differently: LLMs are often memory‑bound; diffusion models more compute‑bound; Whisper has its own profile.
Power, cost, and memory
- Apple Silicon often delivers a good fraction of a large GPU’s performance at far lower power, which is valued for laptops and devices like Vision Pro.
- Others counter that when normalized by performance per watt and per dollar, a 4090 in a desktop still looks better, especially if you already own a gaming PC.
- Unified memory on Macs offers large addressable RAM for models that exceed typical consumer VRAM limits, but GPU memory bandwidth still heavily favors high‑end Nvidia cards.
Ecosystem, usability, and broader context
- Nvidia’s biggest advantage is ecosystem maturity: CUDA, containers that “just work,” and many highly optimized libraries. Competing stacks (Apple MLX, AMD ROCm, Intel) are described as earlier in the “just make it work” phase.
- Some users find MLX setup still rough; others report quick success with example repos.
- Several participants frame the real choice pragmatically: use Nvidia for maximum ML performance; use Macs where portability, efficiency, or existing ownership dominates.