OpenAI GPT-4 vs. Groq Mistral-8x7B

Original Article ↗ Hacker News Discussion ↗

Benchmarks comparing OpenAI’s GPT‑4 with Groq’s Mixtral‑8x7B highlight a dramatic speed advantage for Groq’s specialized hardware, but prompt debate over whether raw throughput matters when large language models still hallucinate and can’t be trusted for tasks that demand near‑perfect accuracy. Commenters question the wisdom of using LLMs to parse structured HTML—where traditional parsers are faster, cheaper, greener, and deterministic—and instead suggest using models to generate or update parsers, or to drive retrieval‑augmented workflows. The thread also touches on Groq’s hardware economics, the competitive landscape around future models like GPT‑5, and broader concerns about energy use versus real‑world utility compared to alternatives such as crypto.

Meaning of “N/A”

  • Several comments debate whether “N/A” means “not available” or “not applicable.”
  • Some argue “not applicable” is standard (backed by dictionaries), others say they’ve always used it as “not available.”
  • Consensus: in practice it’s overloaded and often just means “no value here,” whichever wording you prefer.

Accuracy vs Speed and Hallucinations

  • Many emphasize that for high‑stakes tasks, correctness matters far more than speed.
  • Examples include a single missing “not” flipping a critical parameter, or repeated confident hallucinations.
  • Multiple responses argue that if an LLM doesn’t know a fact, more sampling won’t fix it; ensembles or retrieval are needed.
  • Others note that models will often change answers when challenged, even if they were correct, so “self‑checking” via a simple prompt isn’t reliable.

Retrieval-Augmented and Hybrid Approaches

  • RAG (LLM + external data source) is highlighted as the standard way to mitigate limited context and factual gaps, but described as brittle.
  • Some argue current LLMs don’t distinguish “facts vs grammar,” which limits how well they can decide when to look things up.

LLMs for HTML Parsing and Scraping

  • Many are skeptical of using LLMs to parse structured formats like HTML, calling it slow, expensive, non‑deterministic, and prone to hallucinations.
  • Suggested alternatives:
    • Use LLMs to generate or update deterministic parsers/selectors.
    • Use LLMs only where HTML is highly variable and human maintenance is costly.
  • Several say the benchmark prompt is underspecified; better results would come from:
    • Extracting one field at a time.
    • Not asking for full JSON in a single step.
    • Investing more in system/“character” setup and workflow over clever wording.

Groq, Hardware, and Performance

  • Groq’s speed is attributed to custom chips and Mixtral’s sparse Mixture‑of‑Experts architecture, with many chips networked together.
  • Commenters question real‑world economics and scalability (memory footprint, multiple models, continuous uptime) versus GPUs.

GPT‑5 and Competition

  • Some speculate OpenAI may delay GPT‑5 due to market dominance and incremental gains; others think competition from Claude and others is eroding that lead.
  • There’s disagreement on how big the next leap will be; improvements are expected but not guaranteed to be dramatic.

Energy Use and Ethics

  • Concerns raised about burning large amounts of compute to do work trivially handled by traditional parsers.
  • Counterpoints: AI has broad practical value whereas proof‑of‑work crypto is seen as wasteful; efficiency is improving, and economics already incentivize lower‑energy methods.