Show HN: AI dub tool I made to watch foreign language videos with my 7-year-old

Original Article ↗ Hacker News Discussion ↗

An indie developer has built an AI-powered dubbing service that clones speakers’ voices and synchronizes translated audio and subtitles so videos can be watched in many languages, including with children or non‑English‑speaking relatives. Commenters are impressed by the naturalness of the voices and timing, while probing how the system handles diarization, translation quality, cost, and potential copyright issues when processing YouTube and other content. The thread also explores broader implications for language learning and accessibility, noting both the pedagogical downsides of dubbing for kids and the potential benefits for people who struggle with reading subtitles or hearing the original audio.

Overall reception and use cases

  • Tool widely praised as impressive and “soulful” for many samples; some say cloned voices sound close to real dubbing.
  • Primary use case: letting children or non‑English speakers watch foreign content; others see value for podcasts, YouTube channels, Plex, and accessibility (vision or hearing impairments, elderly viewers).
  • Interest in both “kids” direction (simplifying language) and “parents/grandparents” direction (dubbing/subtitles into their language).

Language support and learning

  • Requests for additional target languages: Ukrainian, Hindi, Portuguese, simplified English (and other graded‑difficulty outputs).
  • Debate on dubbing vs subtitles for language acquisition:
    • Some argue dubbing harms exposure to foreign languages and that kids should watch originals with subtitles.
    • Others counter that reading speed at age ~7 often can’t keep up with subtitles, and that immersion through audio alone can already build strong comprehension.
    • Several anecdotes about children picking up English (or other languages) from TV/YouTube; others stress that real interaction still matters.

Technical approach and challenges

  • Not a wrapper around commercial APIs like ElevenLabs/HeyGen; builder optimized for running on rented GPUs, using open/cheaper components.
  • Pipeline includes: transcription with word/segment timings, translation, voice cloning per speaker, and arranging speech to match timing.
  • Speaker diarization uses embeddings plus heuristics (pauses, sentence boundaries, voice differences), not off‑the‑shelf diarization alone.
  • Timing alignment is hard: length mismatches cause speed‑ups, dropped phrases, or early/late playback; overrun handling and smarter rephrasing are active areas.
  • Idea to self‑evaluate using back‑translation: re‑transcribe the dubbed track and compare to original transcript as a kind of loss function.

UX and product requests

  • Demo deliberately keeps original voice at low volume for context and “AI accountability”; some find it distracting, others think it helps.
  • Users request: toggle to fully mute original voice, better player controls in demo, subtitle display with error reporting, subtitle‑only mode, and browser extensions or Plex integration.
  • Strong demand for a local/offline version; creator prefers centralized service for GPU efficiency and to avoid multi‑platform app support, but is open in principle.

Cost, quality, and legal concerns

  • Cost sensitivity: current cloud pipeline is unoptimized; voice cloning may be the expensive part. Some suggest cheaper tiers with generic voices only.
  • Reported issues: mistranslations (notably Chinese→English), awkward Dutch/Polish output, speed artifacts, and occasional “drunk” or robotic delivery.
  • Plans to monitor output quality automatically, especially for languages the creator doesn’t speak.
  • Copyright concerns around dubbing YouTube and especially movies:
    • Some note this creates derivative works and may trigger rights‑holder actions.
    • Mitigations discussed include focusing on content owners as customers and possibly verifying channel ownership.