Llama 3 8B is almost as good as Wizard 2 8x22B

Model architecture & training strategy

• Llama 3 8B is dense (no Mixture-of-Experts); some argue MoE gives theoretical 4× compute savings, others note MoE is harder to fine‑tune and does not reduce GPU memory use.
• Multiple comments frame Llama 3 8B as “small model trained for a very long time”: 15T tokens vs Llama 2’s 2T, going beyond “Chinchilla optimal” for training compute.
• Debate around Chinchilla: some misinterpret it as including inference cost; others correct that it optimizes model size vs training tokens for a fixed training budget, not real‑world deployment.
• Several argue current models are still undertrained (claims of room up to ~500T tokens); others caution about overfitting and diminishing returns for small models.

Data scale, availability & quality

• People question whether enough high‑quality data exists; replies argue there is huge untapped text: non‑English corpora, national libraries, newspapers, and restricted online archives.
• Private data (emails, DMs, chats) is seen as a vast potential corpus but fraught with legal and privacy risks; some speculate about “provably private training” or in‑org/private models.
• There’s interest in better data curation, de‑duplication, and synthetic data generation (e.g., LLM‑generated Q&A, embeddings-based augmentation, or relaxed matching losses).

Capabilities, reasoning & evaluation

• Some users are “blown away” by Llama 3 8B (near early GPT‑4 feel); others find it clearly behind leading frontier models in logic and “world modeling.”
• Classic logic puzzles (e.g., siblings/“how many brothers?”) expose reasoning gaps: 8B often fails, 70B and other proprietary models sometimes succeed but inconsistently across variations.
• Token‑level operation leads to struggles with letter‑based tasks (“countries that start and end with same letter”) unless reframed as code‑generation problems.
• Reports of it being verbose and occasionally imaginative to the point of absurdity (e.g., spit “reaching the clouds,” then self‑correcting).

Quantization, deployment & use cases

• Quality is sensitive to quantization level: advice is to use higher‑quality quantizations (e.g., Q5/Q6) rather than default low‑precision ones to avoid noticeable degradation.
• Some discuss running Llama 3 locally on phones (Termux + llama.cpp, or MLC targeting mobile NPUs), but concrete integration into system‑wide assistants is still unclear.
• For specialized tasks like threat‑intelligence summarization, commenters find both 8B and 70B still behind top proprietary models in consistency and structure.