Show HN: Gemini LLM corrects ASR YouTube transcripts

Original Article ↗ Hacker News Discussion ↗

An experimental tool uses Google’s Gemini large language model to clean up YouTube’s often error‑prone automatic captions, sparking debate over how well LLMs can improve speech transcripts without introducing new mistakes. Commenters weigh the trade‑offs between verbatim accuracy and “readable” text, concerns about hallucinations and domain‑specific vocabulary, and the cost of running higher‑quality ASR and LLM pipelines at YouTube scale. The conversation also touches on accessibility and legal pressures around captioning, the relative quality of ASR offerings from major cloud providers, and practical issues like API key safety for third‑party tools.

Use of LLMs to Correct YouTube ASR Transcripts

  • Many see this as a natural, high-value use case: fix “boneheaded” ASR errors and improve readability and domain vocabulary.
  • LLMs can use extra context (video title/description, possibly frames) to pick better words and correct technical terms and names.
  • Some report success using pipelines: ASR (e.g., Whisper/WhisperX) → LLM cleanup → separate LLM for summarization.

Limitations and Risks of LLM Post‑Processing

  • LLMs tend to:
    • Normalize toward “average” language, potentially deleting outliers or unusual but correct phrases (e.g., odd activities, nonsense words that convey tone).
    • Reformat speech into polished “internet text,” reducing fidelity to how people actually talked.
    • Hallucinate, especially over long inputs or when given multiple modalities.
  • People with ASR experience argue generic LLM cleanup often reduces transcript accuracy overall, even if it helps on rare words and readability.
  • Chunking (e.g., ~512 words) is reported to reduce hallucinations versus feeding very long transcripts.

Accessibility, Law, and the Berkeley Lecture Archive

  • Discussion revisits Berkeley removing course videos after an accessibility complaint about missing/poor captions.
  • Some argue modern ASR + LLMs now make captioning cheap enough that such archives could be captioned instead of removed.
  • Others stress the legal issue remains: regulations effectively force “caption well or remove,” leading institutions to pull content when costs or risk are high.
  • Debate over whether such rules protect disabled users or end up harming everyone by reducing available content.

Quality of YouTube/Google ASR vs Alternatives

  • Mixed views on YouTube captions:
    • Some say they’re now “mostly fine” for clear, standard English.
    • Others (especially referencing Deaf/HoH use, accents, domain jargon, and non‑English like Japanese) find them inaccurate, misleading, or useless.
  • One commenter claims Google’s ASR is among the weakest hyperscalers; Azure (via Nuance) is described as significantly better, with several non‑cloud and self‑hosted options (Whisper, Kaldi) mentioned.
  • Re‑transcribing with modern ASR (Whisper variants, commercial APIs) is seen as cheap and often more reliable than “fixing” a bad transcript with an LLM.

Gemini as Product and API

  • Several users report poor experiences with consumer Gemini: refusals, prudish/risk‑averse behavior, weaker quality than GPT‑4o/Claude.
  • Others note Gemini API and multimodal models are strong for long audio/video, but still prone to hallucinations in meeting summaries.
  • Cost: Gemini Flash‑8B is cited as extremely cheap per hour of transcript, making LLM cleanup attractive at scale.

Security and API Keys

  • Users are wary of pasting personal API keys into third‑party tools, even if calls are client‑side.
  • Suggested mitigations: create low‑budget or temporary keys, rotate/delete keys after use, or self‑host the open‑source tool.