Extending the context length to 1M tokens
Extending Qwen2.5’s context window to 1M tokens sparks both excitement about handling massive inputs and skepticism about how well models actually retain and reason over such long contexts. Commenters debate whether modern LLMs now surpass most humans in practical intelligence—citing strengths in rapid synthesis, coding help, and everyday creativity—versus their well-known weaknesses in reliable reasoning, recall, and adherence to precise specifications. Others note practical constraints such as degraded performance at large context sizes, unclear open-source availability of the long-context model, and technical quirks when running Qwen locally.
Three-Body Problem demo & long-context claims
- Several commenters note that the page’s Three-Body Problem summaries contain factual errors and hallucinations, undercutting the long-context “read three books” demo.
- Some argue this doesn’t really test 1M-token context, since smaller models already know the books from training data.
- Others highlight that disentangling “using only the provided context” from prior training is ill-defined, since both humans and models inevitably mix prior knowledge with new input.
Do LLMs surpass human intelligence?
- One side claims modern LLMs already “far surpass” average humans on many cognitive tasks, especially fast reading, summarization, retrieval, cross-domain synthesis, and “napkin math.”
- Others strongly disagree: speed and scale are not intelligence; models lack robust reasoning, long-term learning, planning, and grounding in the physical world.
- Comparisons are made to calculators and chainsaws: extremely capable tools, but not “intelligent” in a human sense.
Creativity, novelty, and failure modes
- Supporters cite personal use cases (trip planning, recipes, branding, lyrics, code sketches, theoretical explorations) as evidence of everyday creativity and cross-domain competence beyond most people.
- Skeptics respond that:
- These outputs are recombinations of training data, not evidence of deep understanding.
- Quality often collapses on non-routine tasks (e.g., implementing a real S3 backend, precise drawing constraints, complex reasoning).
- Human and LLM failure modes differ: humans may be slow or unskilled, but models confidently hallucinate and struggle to follow exact specifications.
- There’s recurring debate over whether “hallucination” in LLMs is analogous to human rationalization and memory errors.
Singularity, self-improvement, and sentience
- Some argue we may already have passed a “technological singularity” in practical capability; others insist the true singularity requires autonomous systems that can design and train significantly better successors.
- On sentience, commenters note that:
- We lack clear criteria even for humans.
- A genuinely sentient AI might be dismissed as “just mimicking patterns,” especially given current safety fine-tuning that suppresses such claims.
Qwen 2.5 long-context practicality & availability
- Local users praise Qwen2.5 Coder but struggle with hardware limits and long-context use in GGUF-based setups.
- There’s confusion around enabling 128k+ context via configuration flags and YaRN scaling; some report degradation and hallucinations on very long inputs.
- Needle-in-a-haystack benchmarks showing 100% retrieval at 1M tokens are met with skepticism; individual tests at 32k context already show misses.
- Commenters note that the 1M-token long-context model does not appear to be downloadable; weights are not clearly released.
- One nitpick corrects the blog’s “1M tokens ≈ 1M words,” arguing typical English tokenization yields closer to ~750k words.