Launch HN: Retell AI (YC W24) – Conversational Speech API for Your LLM

Hacker News Discussion ↗

A new YC-backed startup is showcasing a real-time speech API that lets developers plug their own LLMs and audio providers into highly responsive, human‑like phone and voice agents. Commenters are impressed by the low latency, natural speech, and interruption handling, but point out practical gaps such as date/time awareness, occasional conversational loops, and the need for better prompts and memory. Much of the debate centers on economics and ethics: whether per‑minute pricing can compete with human call centers, how to prevent abuse or deception in customer service and crisis‑response scenarios, and what guardrails or disclosures should be built into such systems.

Overall impressions

  • Many commenters found the demo “incredible,” particularly latency, turn-taking, and natural voice quality; several said it felt close to human and past the “uncanny valley.”
  • Others noted the “AI cracks” still show: occasional contradictions, loops, and clearly scripted or inappropriate promises (e.g., claiming to “get back” with info but never doing so).

Capabilities & technical design

  • Key strengths highlighted: low latency (~sub-second), strong interruption handling, and natural prosody.
  • Product is “bring your own LLM” and “bring your own TTS/ASR”: currently integrates with providers like OpenAI TTS, ElevenLabs, Deepgram; plans to train in-house TTS and add voice cloning.
  • Architecture: Retell manages audio in/out, VAD, turn-taking, and calls the customer’s server/LLM; it does not own the reasoning model.
  • Supports function-calling–style hooks so developers can close calls or trigger actions.

Limitations & failure modes

  • Reported issues: incorrect or rigid handling of dates/times, confusion on multi-step constraints, getting stuck in apology loops, failing to actually perform claimed actions, and dropping/“disconnecting” mid-conversation.
  • Some browsers (Firefox, iOS Safari, mic selection in Chrome) caused demo failures.
  • Current demos use GPT‑3.5 and are explicitly described as non-specialized; several commenters felt 3.5 is not strong enough for nuanced phone conversations.

Use cases & product scope

  • Highlighted use cases: appointment booking (dental), logistics, pre-surgery intake, tutoring, therapy/crisis listening, AI assistants/companions, call center agents, Zoom/voicechat participation.
  • Debate over voice vs GUI: some see voice agents as inferior to web forms for straightforward tasks; others value skipping 40–50 minute queues.

Pricing & economics

  • Current per-minute pricing is seen as high relative to bare-bones DIY pipelines or contact-center rates; multiple developers said it’s 5–10× their STT+TTS cost and hard to justify at scale.
  • Team acknowledges this and hints at exploring cheaper tiers.

Ethics, trust, and user experience

  • Strong concern that voice AIs should be required to identify themselves as AI on request or under certain conditions (e.g., confused elderly callers).
  • Mixed feelings about AI “therapist” use cases: some see value as crisis stopgap; others worry about dehumanization and reduced staffing incentives.
  • Several commenters dislike being unknowingly routed to AI when calling a business but would accept it if it solves problems quickly.

Comparisons & alternatives

  • Compared with tools like vocode, Google Dialogflow, KITT/LiveKit, Gridspace, Nero; Retell is viewed as particularly strong on latency and interruption handling.
  • Some would prefer an inexpensive, standalone streaming TTS layer; others request open-source or self-hosted/community options.