Apple's new SpeechAnalyzer API, benchmarked against Whisper and its predecessor

Original Article ↗ Hacker News Discussion ↗

Apple’s new on‑device SpeechAnalyzer API is being reported as vastly more accurate and faster than Apple’s older SFSpeechRecognizer and even OpenAI’s Whisper Small on standard English benchmarks, while running entirely locally on recent iPhones and Macs. Commenters see this as a major win for privacy‑preserving transcription and live dictation, but note that Apple’s model still lags open alternatives in areas like multilingual support, diarization, and handling of technical jargon, and that it’s locked to Apple platforms with closed weights. Many also question the narrow benchmark choice, asking how SpeechAnalyzer stacks up against newer state‑of‑the‑art models such as Nvidia’s Parakeet, Nemotron, Mistral’s Voxtral, and MOSS‑Transcribe‑Diarize, especially for noisy or non‑English audio.

Benchmark results and scope

  • New Apple SpeechAnalyzer API shows much lower word error rate than the old Apple SFSpeechRecognizer on LibriSpeech test sets, and beats Whisper Small in accuracy and speed in this benchmark.
  • Benchmark is English‑only, read speech (not conversations/meetings), single M2 Pro machine, and only compares to Whisper Tiny/Base/Small, not newer or larger models.
  • Some commenters find the improvement impressive; others dismiss the benchmark as incomplete or misleading because it excludes stronger baselines.

Comparisons to other ASR models

  • Many argue Whisper Small/Base/Tiny are “old” and no longer state of the art; they want comparisons against:
    • Whisper Large V2/V3/Turbo.
    • Nvidia Parakeet (TDT V2/V3, Unified, streaming variants).
    • Nvidia Nemotron, Mistral Voxtral, Cohere Transcribe, OpenAI Whisper V3 Turbo, ElevenLabs Scribe, MOSS‑Transcribe‑Diarize, Qwen ASR, etc.
  • Some report Whisper Large V2/V3 still best on difficult or noisy audio; others claim newer models like Parakeet or MOSS outperform Whisper in their own tests.
  • Several links to ASR leaderboards are shared; some say they favor newer models over Whisper, though anecdotal experiences vary.

On‑device, privacy, and platform issues

  • SpeechAnalyzer runs on‑device and is shared across apps, avoiding bundling large models per app and improving power/thermal behavior on Apple hardware.
  • It is not open‑weight; some see that as fine for privacy (local processing), others distrust Apple or want exportable models for Linux/Home Assistant.
  • There are questions about which iOS/macOS versions and features (keyboard dictation, Siri, Voice Memos) actually use this new engine; several reports say current Siri/dictation still feel poor or slow, even on recent betas.

API behavior and developer notes

  • Important gotcha: ending the audio stream is not enough; finalizeAndFinishThroughEndOfInput() (or equivalent) must be called or streaming results never terminate.
  • Old SFSpeechRecognizer sends audio to Apple servers unless explicitly forced on‑device.
  • SpeechAnalyzer supports streaming partial results, word‑level timestamps (English), and speaker identification, but lacks built‑in diarization and easy multilingual handling; models are downloaded per language.

Real‑world use cases and UX

  • Reported uses include: podcast and lecture transcription, live subtitles for hard‑of‑hearing users, voice memos, dictation, coding workflows, historical archives, and Home Assistant.
  • Many find SpeechAnalyzer fast and accurate in practice, including on mixed‑quality audio; others still prefer their existing tools (e.g., Parakeet‑based or Whisper‑based apps) for certain languages, accents, or domains.
  • Concerns remain about technical jargon, accents (Australian, Texan, British, NYC), language auto‑detection, and Apple’s historical neglect of many languages.