Edge0-145B: Frontier AI Capabilities on Your Own Computer
A new project called Edge0-145B claims to bring “frontier‑level” AI models to personal machines by modifying and fine‑tuning Alibaba’s Qwen3.8 Flash Next and running it through a fork of the Strata inference engine. Commenters are intrigued by its sparse MoE architecture, aggressive offloading of weights to RAM/SSD, and promising benchmarks, which in theory could let high‑end laptops and desktops run very large models. However, many are skeptical of the marketing, licensing clarity, incomplete macOS support, and benchmark claims, noting that the underlying tech is largely repackaged from existing open‑weight models and tooling.
Provenance and Architecture
- Many note Edge0-145B is derived from Qwen3.8-Flash-Next and runs on a fork of the Strata inference engine (“edge0‑strata”).
- Some see only modest architectural tweaks (e.g., different active parameter count, shared matrices, expert placement heuristics) plus fine‑tuning and quantization.
- Debate on whether this is genuine innovation vs a repackaged fine‑tune with heavy branding.
Licensing and Openness
- Confusion over licensing: some call it “open weights,” others argue the license is too restrictive to qualify.
- Presence of Apache 2.0 in parts of the stack adds to the ambiguity.
Platform, Runtime, and MacOS Confusion
- Blog claims “day‑one support” for macOS, Windows, Linux, and M‑series Macs.
- In practice, released runtime for 120B/145B appears CUDA‑only; the GUI app is Windows‑only.
- Community finds guides for 35B on macOS via other engines, but 145B on Mac is not yet practical; overall messaging is seen as contradictory.
Memory Footprint and Technical Claims
- Headline claim: 145B parameters with ~29GB peak memory, “runs on a 48GB laptop.”
- Commenters dissect this, noting:
- Basic math suggests far more than 29GB for 145B parameters, even at low precision.
- Docs mention additional tens of GB in system RAM plus SSD usage for experts and n‑gram tables.
- Consensus: this is a sparse / offloaded MoE setup where much of the model lives in RAM/SSD; 29GB refers only to GPU or active subset, and marketing is seen as misleading or unclear.
Performance, Benchmarks, and Real‑World Use
- Benchmarks (e.g., TerminalBench) look strong, roughly on par with some well‑regarded local models.
- Several warn about “benchmaxxing”: fine‑tunes that raise benchmark scores without improving general use.
- At least one user test reports the model stalling on a classic coding problem, suggesting immaturity or engine issues.
Use Cases, Tools, and Agents
- Interest in tool use demos, especially Blender automation and coding agents.
- Skepticism about whether generated 3D/PCB artefacts are clean enough to save real time vs manual cleanup.
- Comparisons to other agents/models (Fable, Qwen4, etc.) and suggestions to use text‑based CAD like OpenSCAD for AI‑assisted 3D.
Hardware, Accessibility, and VRAM
- Enthusiasm from users with high‑end GPUs (e.g., 4090/5090, RTX 6000) and large‑RAM Macs; some see this as a reasonable pro investment, others call it “ridiculously expensive.”
- Questions about minimum VRAM (4–8GB vs 16–24GB+) remain unanswered; offloading helps but trade‑offs are unclear.
- Discussion on whether typical “normal” users own hardware capable of running such models; many think the real target is prosumers and professionals.
Model Variants (145B vs 120B vs 35B)
- Confusion about why a 120B model exists when 145B is similar size and both are reported as ~29GB memory in the table.
- Some speculate smaller variants are meant for machines like 48GB MacBooks, but official guidance is missing.
Company Transparency and Origin
- Commenters note very little concrete information about the company on its site (location, team, etc.).
- One comment highlights that this appears to be a PRC startup optimizing a Chinese base model, which some see as relevant context.
Broader Reflections on Local “Frontier” AI
- Debate over marketing language like “frontier AI capabilities on your own computer”:
- Supporters argue it’s impressive progress: large, strong models running on a single high‑end workstation.
- Critics say it’s not truly “frontier,” more like last‑year frontier in a heavily optimized, offloaded format.
- Broader discussion about:
- Trend toward both massive cloud models and smaller/specialized local ones.
- Whether local frontier‑ish models will make it economically viable to give each knowledge worker or hobbyist their own powerful, private AI.