Claude 3 beats Google Translate

Original Article ↗ Hacker News Discussion ↗

Overall view of Claude/LLMs vs Google Translate

  • Many commenters say LLMs (Claude 3, GPT‑4, GPT‑3.5, Gemini, etc.) now outperform Google Translate across many language pairs.
  • Translation quality from LLMs is described as more natural, better with metaphor, idiom, style, and long‑range context.
  • Some see this as ironic given transformers originated from translation research at Google.

Comparisons with other machine translation tools

  • DeepL often considered much better than Google Translate for European languages, but several report LLMs surpassing DeepL as well.
  • DeepL is criticized for ignoring or repeating chunks in longer texts and for errors in Japanese.
  • For ex‑USSR languages, Yandex Translate is mentioned as historically better than Google.
  • Gemini is reported by some as stronger than free ChatGPT for translation, but was not used in the paper.

Language- and domain-specific observations

  • Google Translate judged especially weak for Asian languages (Japanese, Chinese, Korean), including basic issues like Japanese particles and misread Kanji.
  • LLMs handle gender consistency and literary/fiction translation notably better, especially for Japanese.
  • Claude is said to dramatically outperform Google Translate on classical Chinese historical texts, avoiding mistranslations and sentence drops.
  • Various anecdotes show Google Translate mishandling slang, numerals, idioms, airports, and polysemy; some users still find it “good enough” for gist and OCR/photo use.
  • LLMs can follow instructions about tone, formality, rhyming, and stylistic tweaks in a way traditional systems cannot.

Hallucinations, reliability, and trust

  • One side claims people prefer Google Translate because it’s more deterministic and less prone to hallucinations.
  • Others counter that Google Translate and DeepL also “hallucinate” or fabricate content, especially on distant language pairs or noisy input, sometimes worse than GPT‑4/Claude.
  • Consensus: for critical content, human translators remain important; both LLMs and classical MT can fail badly.

Cost, speed, censorship, and usage

  • Google Translate is praised for being instant and free at huge scale; LLM inference is seen as more compute‑intensive.
  • Some argue API pricing (e.g., GPT‑3.5 vs Translate) doesn’t map cleanly to energy cost due to subsidies and different business models.
  • Claude and GPT‑4 are criticized for refusing erotic or “unsafe” translations, where Google/DeepL impose fewer content limits.