Llama 3.1

Original Article ↗ Hacker News Discussion ↗

Meta’s release of the Llama 3.1 family — including a 405B-parameter model that rivals GPT‑4o on many benchmarks — is seen as a major milestone for openly available AI, even though the flagship model is too large for typical consumer hardware. Commenters focus on practical trade‑offs: smaller 8B and 70B variants can be quantized and run locally, while serious use of the 405B model requires datacenter‑class GPUs or specialized hosts like Groq and cloud providers. There is also active debate over Meta’s “open source” positioning, with many noting that the license and lack of training data make these models better described as “open weights” that still offer substantial value for startups, researchers, and tool builders.

Model capabilities & benchmark results

  • Commenters highlight the 405B model as roughly competitive with GPT‑4o on several public benchmarks (MMLU, coding, math), and near top-tier in some user-run tests (e.g., NYT Connections, coding leaderboards).
  • The 8B and 70B variants show notable gains over Llama 3, especially on MMLU, and are seen as more practical for most users.
  • Some users report that GPT‑4o and Claude 3.5 still feel better in real coding and math tasks despite benchmark parity.
  • Benchmarks are widely treated with caution; LMSys ELO is mentioned as more reflective of “real world” usage, but it has its own limitations.

Hardware requirements & running locally

  • 405B is considered essentially out of reach for typical home hardware, even under 4–8 bit quantization; estimates include ~200 GB+ VRAM and multi‑GPU setups costing around $10k or more.
  • Suggestions include multi‑GPU PC builds, Mac Studio clusters over Thunderbolt with tools like Exo, and CPU-only options that would be extremely slow.
  • 8B and 70B models are commonly run locally via tools like Ollama, llama.cpp, and other frontends on single GPUs or high‑RAM Macs.
  • There is ongoing work and some friction around support for new architectures (e.g., ROPE changes).

Hosting, pricing & ecosystem

  • For serious use of 405B, commenters point to cloud providers (AWS, GCP, Azure), specialized inference platforms (Groq, Hyperbolic, Bedrock), and APIs embedded in products (WhatsApp, Meta AI, Poe, VSCode extensions).
  • Discussion notes that open models don’t automatically mean cheap inference; hosted Llama pricing is often compared to proprietary models.

“Open source” vs “open weights” debate

  • Strong disagreement over calling Llama “open source.”
  • Critics note license restrictions (certain commercial users, military/nuclear use, acceptable‑use clauses) and the absence of training datasets, arguing this breaks with traditional open‑source and open‑science norms.
  • Others argue that releasing weights plus code is still a major positive, and that strict semantic policing may discourage companies from opening anything.
  • Several propose “open weights” or “nearly-open source” as more accurate terms.

Meta’s strategy & competitive landscape

  • Multiple comments frame Meta’s releases as a “scorched earth” play to undercut proprietary labs by collapsing the base‑model moat.
  • There is debate over whether any training “secret sauce” exists or whether compute scale plus open weights will commoditize base models, shifting profit to applications and compute.
  • Some see heavy use of synthetic data for fine‑tuning as a key ingredient and a broader industry trend.

Regulation & regional access

  • Europeans report that Meta’s chat product isn’t available in the EU, likely due to GDPR and upcoming AI/DM rules.
  • Opinions split between viewing this as justified consumer protection vs. evidence that EU regulation slows access to new tech.