Muse Ember: Our smallest, most efficient model
Meta’s new Muse Ember model is pitched as its smallest, most efficient LLM, competing directly with OpenAI’s Luna/Haiku and DeepSeek’s flash variants on a cost‑per‑task basis, especially for agentic, legal, and finance workloads. Commenters highlight its strong benchmark scores, flexible “reasoning effort” levels, and lower cache costs, but question whether it overfits public evals and underperforms in longer, real‑world interactions. Many also criticize the lack of open weights, Meta’s trust and privacy record, and the accelerating churn of proprietary model updates that are increasingly hard to evaluate meaningfully.
Pelican SVG “Benchmark” and Reasoning Levels
- Multiple SVGs of “pelican on a bike” at different reasoning-effort levels show style changes but no clear quality gain with more tokens.
- Some prefer the “minimal” (10-token) result; others note anatomical issues and layering errors persist even at max effort.
- Discussion suggests models organize SVG elements by concept (bird, bike, background), which hurts correct 3D interleaving (e.g., legs vs frame).
- Pelican images are seen as a personal, consistent benchmark, analogous to tracking one product’s price for inflation.
Model Capabilities, Benchmarks, and Costs
- Muse Ember is framed as a very cost-efficient, small “all-rounder,” often competitive with or better than Luna, Haiku, Grok, and Terra on aggregated benchmarks.
- Detailed Vals and Artificial Analysis numbers show Ember particularly strong and far cheaper for finance and legal tasks, though another model beats it on some coding/terminal benchmarks.
- Some argue benchmarks can be misleading: Muse models may look frontier on paper but feel underwhelming in real-world use (especially compared to certain other products).
Crouton Alpha and Stealth Models
- Strong consensus that OpenRouter’s “Crouton Alpha” was an early Muse Ember, based on identical context window, reasoning options, and other metadata.
- Users report Crouton performed well for short tasks but degraded badly over longer conversations.
Reasoning Effort Levels and UX
- Six reasoning levels are seen by some as an antipattern: too many knobs for users to choose from.
- One suggestion is a “decider model” that picks the effort level, though others doubt such models can reliably predict task complexity.
Open vs Closed Weights and “Smallness”
- Multiple commenters lament that Ember is not open-weight despite being marketed as Meta’s “smallest” model.
- Questions arise: if it only runs in Meta’s datacenter, why does its parameter count or “smallness” matter to users?
- Spark Open is cited as the local alternative, but its open version is reportedly heavily safety-tuned and no longer in the same capability class.
Meta’s Role, Ethics, and Adoption
- Opinions split: some say Meta was irrelevant but is now catching up with Muse/Spark; others see Muse as still behind top competitors or “benchmaxxed.”
- Ethical distrust is strong in parts of the thread; some refuse to send data to Meta’s cloud regardless of quality, preferring local or non-US/Chinese models.
Model-Update Fatigue and Evaluation Practices
- Several participants feel burned out by constant new model releases and subtle price/quality tradeoffs.
- Others argue that if a current model suffices, there’s no need to chase every update—benchmarks and custom eval suites should guide selective adoption.