M5 Ultra Mac Studio Review

Original Article ↗ Hacker News Discussion ↗

Apple’s new M5 Ultra Mac Studio is being dissected as a high-end machine for running large language models locally, with many weighing its $10k–$15k price against cloud GPU rentals and consumer GPUs like Nvidia’s RTX 5090. Commenters highlight its strengths in huge unified memory and MoE models versus its weaker raw throughput and value compared to dedicated GPUs, especially for dense 27B-class models. The broader debate centers on whether local AI will ever be cost‑effective versus cloud or leased hardware, and how much users should trade speed and model quality for privacy, control, and independence from major AI providers.

Cost and Value Tradeoffs

  • Fully loaded M5 Ultra Studio configurations are estimated around $12k–$15k+, prompting comparisons to:
    • Years of API or “Pro” subscriptions (OpenAI, OpenRouter, etc.).
    • Renting GPUs ($5/hr H200; ~$.20/hr 5090) vs buying hardware.
  • Many argue local AI is not cheaper for most individuals; others note:
    • At high utilization (billions of tokens/month, 24/7 agents), hardware can amortize favorably.
    • Hardware retains resale value and is reusable beyond AI.
  • Some see the M5 Ultra as “hobby hardware” compared to $100k–$1M+ “serious” AI rigs.

Performance vs Nvidia & Other Hardware

  • Benchmarks: M5 Ultra is close to a single RTX 5090 on dense Qwen3.8-27B in tokens/sec, but:
    • Multiple posters claim optimized stacks (vLLM, llama.cpp, NVFP4, MTP, DFlash) push 5090/Pro 6000 to 100–300+ tok/s, far above the review’s numbers.
    • Debate over whether Mac’s lower memory bandwidth vs 5090’s higher bandwidth is the main limiter.
  • Mac’s advantage: huge unified memory (256–512GB) allows running very large MoE/flash models that don’t fit in 32GB VRAM.
  • Counterpoint: once models exceed unified RAM and require SSD streaming, speed collapses (e.g., Kimi K3 estimated at ~0.5 tok/s).
  • Power: 5090 ~575W vs M5 Ultra ~480W; you can underclock GPUs to save power with small throughput loss. Multi-5090 builds are far more power- and infrastructure-intensive.

Dense vs MoE and Architecture

  • Dense 27B benchmarks seen as misleading for Macs; many argue large MoE (Qwen Next, GLM Flash) is where big unified memory shines.
  • Others contend dense models still make sense on unified memory until bandwidth becomes the bottleneck.
  • Discussion of expert caching, speculative decoding, batching, and future software optimizations; impact on real “time to complete task” is emphasized over raw tok/s.

Local vs Cloud: Motivations and Limits

  • Motivations for local:
    • Privacy, avoiding provider data capture and ToS/censorship.
    • Full control (uncensored “cyber-capable” agents, research, hacking).
    • Hobbyist experimentation and “owning the lab.”
  • Acknowledged limitations:
    • Local open-weight models lag frontier cloud models, especially for complex coding and large codebases.
    • For many dev tasks (CRUD apps, assistant work), Sonnet/Astra-level isn’t required; Qwen3.8-27B–class models are often “good enough,” though not for everyone.

macOS, Linux, and Ecosystem

  • Some praise macOS as a polished UNIX-like environment that “just works.”
  • Others see it as limiting for homelab/server use; prefer Linux for containers, cgroups/namespaces, and full-source control.
  • Asahi Linux is noted but constrained by reverse-engineering and funding.

Future Outlook

  • Rumors: M6/M7 generations focusing heavily on AI, with M7 Ultra reportedly targeting up to 1.5TB unified memory and Blackwell-class performance.
  • Debate over long-term balance: local devices (Macs, iPhones) vs hyperscale data centers and custom NPUs/TPUs.