Inkling 2 Mini
A new open‑weight language model, Inkling 2 Mini, is drawing attention for matching or rivaling larger models like Qwen 3.8‑27B in coding and reasoning tasks while still being practical to run on consumer hardware, including 32–64 GB Macs and mid‑range GPUs. It combines a looped transformer architecture, mixture‑of‑experts routing, and 4‑bit training to trade more compute per token for lower memory requirements, prompting both excitement about local, fast “good enough” assistants and concern about verbosity, knowledge gaps, and safety/interpretability. Early testers report strong tool use, coding ability, and quantization behavior, but mixed real‑world performance, especially when heavy “reasoning” modes lead to long outputs or runaway loops.
Overall reception & comparisons
- Many commenters are very impressed, calling Inkling 2 Mini “bonkers good” if benchmarks translate to real use.
- Frequently compared to Qwen 3.8 27B / Qwen 3.6 35B and Muse Glimmer; several say it appears competitive or slightly better on coding and some reasoning tasks at lower active parameter counts.
- Some report it feels like a “frontier-lite” model for coding and data work, but not clearly superior for every use case.
Looped transformer architecture
- Strong interest in the looped design: middle layers are reused multiple times per token, effectively increasing depth without more parameters.
- Discussed as related to earlier research (Universal Transformer, recursive models) and DIY “layer repetition” tricks that boosted capability.
- Claimed benefits: more per-token reasoning, potentially less reliance on long chains of text reasoning, better coding performance.
- Concerns: looping increases compute and KV-cache size per token; unclear impact on real-time speeds and interpretability.
Local deployment, quantization & engines
- GGUF quantizations are already available; llama.cpp support landed via recent PRs, though still rough.
- Reported decode speeds vary widely:
- On high-end NVIDIA (5090, 3090) and AMD (7900XTX, 9070XT) GPUs, users see 20–50 tok/s+ after tweaks; some initially see only ~10 tok/s.
- On M1/M2/M4 Macs with 32–64GB RAM, users get ~10–20 tok/s with Q4; considered very usable but below theoretical bandwidth limits.
- Vulkan often outperforms ROCm on AMD in llama.cpp; some recommend specific patches, batch-size tuning, or alternative servers (e.g., radiance).
- FlashAttention and nvfp4 quirks limit some KV quantization choices; support on Metal/MLX is not yet mature.
Practical behavior: coding, tools, prose
- Coding demos impress many; users report concise, good-quality Python and SQL, with strong tool-calling and search when enabled.
- Others see odd behaviors: invented APIs/hooks, stubbornly wrong beliefs even after searching, and occasional failure to actually execute tools it claims it will call.
- Prose quality is generally liked, though some find responses too brief or slightly “off”; for pure writing, some still prefer Gemma 4 or Muse Glimmer.
Token usage, reasoning levels & verbosity
- The model is very “talkative,” especially at “extra high” reasoning settings in some harnesses, burning tens of thousands of tokens on single tasks.
- Some argue this is unrealistic for everyday use and that high reasoning levels trade huge latency and context usage for marginal gains.
- Others note that looping doesn’t increase factual knowledge; it boosts reasoning but still requires tools/search to close knowledge gaps.
Safety, alignment & risk
- Looped models raise concern for some commenters: more “invisible” internal reasoning may reduce monitorability compared to explicit chain-of-thought.
- Others dismiss this as overblown, arguing looping is functionally similar to deeper networks and that all modern LLMs are already opaque.
- There is broader debate about AI-enabled cyber capabilities and bioterror risk, with views ranging from “already here but manageable” to significantly increased “p(doom).”
Role in the ecosystem & business model
- Seen as an important step for serious local models that run on 32–64GB Macs and commodity GPUs, possibly enabling multi-agent setups locally.
- Some frame this as part of a trend toward small, efficient, open weights models (often MoE + looping + low-bit training).
- Business rationale speculated to be a mix of clout, future acquisition prospects, enterprise fine-tuning/services, and a “Red Hat–style” support model.