Llama-3.3-70B-Instruct
Meta’s release of the Llama‑3.3‑70B‑Instruct model is seen as a major leap for open-weight AI, with many benchmarks suggesting performance close to GPT‑4o at a fraction of the cost and with the ability to run locally. Commenters trade practical notes on quantization, hardware requirements, and tools for running or fine‑tuning the model on consumer GPUs and Macs, highlighting a growing ecosystem around local and customized LLMs. There is also active debate over Meta’s motives, the model’s restrictive license and safety alignment, and whether such powerful “open” models effectively commoditize AI while strengthening Meta’s core advertising and social platforms.
Performance and Benchmarks
- Multiple commenters say Llama‑3.3‑70B‑Instruct performs on par with, or slightly better than, Llama‑3.1‑405B and close to GPT‑4o on several shared benchmarks.
- Some note it does unexpectedly well on independent evals (e.g., Kagi).
- Others give concrete counterexamples (e.g., D&D 5e rules) where it hallucinates; another model (Claude) did better there.
- Overall sentiment: very strong free / “open weight” model, but not universally superior across tasks.
Quantization, Local Use, and Tooling
- Users run 70B locally via Q3–Q5 quantization on consumer GPUs and CPUs, reporting ~2–16 tokens/s depending on hardware (4090, 3090s, M‑series Macs, older GPUs).
- Q4 is widely seen as the practical sweet spot; heavier quantization degrades coherence, prompt adherence, and precision, especially for complex conditionals and chain-of-thought.
- Discussion of model size vs VRAM: rough rule is parameter count ≈ GB at 8‑bit; halve for 4‑bit plus overhead for context and OS.
- Many recommend smaller models (8–20B, ~32B) as “sweet spots” for 12–24GB cards.
- Tools mentioned: Ollama, LM Studio, llama.cpp, Open WebUI, plus Unsloth for more efficient fine‑tuning.
Base Model and “Openness”
- Some are disappointed there’s no separately released base model; others say the base is effectively the earlier 3.1‑70B.
- Strong debate over terminology: several insist this is not “open source” but “open weight” freeware due to license restrictions and lack of full reproducible training pipeline/datasets.
- Others argue practical rebuildability is moot at current training costs, but critics point to the importance of true open source definitions and training‑data legality.
Safety, Censorship, and Uncensoring
- Interest in uncensored/unaligned variants; reports that many “decensored” models feel noticeably dumber or still refuse content.
- One line of work (abliteration) is criticized as often “lobotomizing” models; straightforward fine‑tuning on uncensored data is reported to preserve quality better.
- Some describe manual tricks (editing refusals mid‑response) as local workarounds.
Meta’s Strategy and Ecosystem Impact
- Broad agreement that releasing strong free models pressures proprietary providers (OpenAI/Anthropic), drives prices down, and benefits developers.
- Several frame this as classic “commoditize your complement”: Meta strengthens its social/ads core business while commoditizing general‑purpose AI.
- Views on Meta’s motives and ethics are mixed: some see a “redemption arc” and founder‑driven ambition; others stress past privacy/ads scandals and warn about more powerful ad‑targeting and AI “slop” in feeds.