Llama 3 8B is almost as good as Wizard 2 8x22B
Llama 3 8B is being praised as a major step forward for small, locally runnable language models, approaching early GPT‑4‑like usefulness despite its modest size. Commenters attribute its strength to heavy training (15T tokens), better data curation, and more code in the corpus, while debating trade‑offs between dense vs Mixture‑of‑Experts architectures, training duration, and overfitting. At the same time, they note that the model still struggles with certain reasoning and logic puzzles compared to larger systems, underscoring both the rapid progress and current limits of compact LLMs.
Model architecture & training strategy
- Llama 3 8B is dense (no Mixture-of-Experts); some argue MoE gives theoretical 4× compute savings, others note MoE is harder to fine‑tune and does not reduce GPU memory use.
- Multiple comments frame Llama 3 8B as “small model trained for a very long time”: 15T tokens vs Llama 2’s 2T, going beyond “Chinchilla optimal” for training compute.
- Debate around Chinchilla: some misinterpret it as including inference cost; others correct that it optimizes model size vs training tokens for a fixed training budget, not real‑world deployment.
- Several argue current models are still undertrained (claims of room up to ~500T tokens); others caution about overfitting and diminishing returns for small models.
Data scale, availability & quality
- People question whether enough high‑quality data exists; replies argue there is huge untapped text: non‑English corpora, national libraries, newspapers, and restricted online archives.
- Private data (emails, DMs, chats) is seen as a vast potential corpus but fraught with legal and privacy risks; some speculate about “provably private training” or in‑org/private models.
- There’s interest in better data curation, de‑duplication, and synthetic data generation (e.g., LLM‑generated Q&A, embeddings-based augmentation, or relaxed matching losses).
Capabilities, reasoning & evaluation
- Some users are “blown away” by Llama 3 8B (near early GPT‑4 feel); others find it clearly behind leading frontier models in logic and “world modeling.”
- Classic logic puzzles (e.g., siblings/“how many brothers?”) expose reasoning gaps: 8B often fails, 70B and other proprietary models sometimes succeed but inconsistently across variations.
- Token‑level operation leads to struggles with letter‑based tasks (“countries that start and end with same letter”) unless reframed as code‑generation problems.
- Reports of it being verbose and occasionally imaginative to the point of absurdity (e.g., spit “reaching the clouds,” then self‑correcting).
Quantization, deployment & use cases
- Quality is sensitive to quantization level: advice is to use higher‑quality quantizations (e.g., Q5/Q6) rather than default low‑precision ones to avoid noticeable degradation.
- Some discuss running Llama 3 locally on phones (Termux + llama.cpp, or MLC targeting mobile NPUs), but concrete integration into system‑wide assistants is still unclear.
- For specialized tasks like threat‑intelligence summarization, commenters find both 8B and 70B still behind top proprietary models in consistency and structure.