Amazon saves millions by replacing Sonnet 4.6 with Strands Decider

Amazon has reportedly cut ticket-classification costs by millions of dollars by replacing Anthropic’s Claude Sonnet 4.6 with an in-house “decision model” called Strands Decider, built on small open-weight models like Gemma or Qwen. Commenters see this as a concrete example of a broader shift: LLMs are great for rapid prototyping and data labeling, but often too slow and expensive for high-volume, narrow tasks where classical ML or lightweight specialist models perform as well or better. The thread also highlights tensions between engineering efficiency, organizational inertia, and “AI hype,” with many predicting that specialized decision models and other small models will increasingly displace general-purpose LLMs in production backends.

Cost savings and scale

  • Commenters calculate that at ~$21.7k saved per million tickets, “millions” in savings implies tens of millions of tickets per year; some Amazon workers say 92M+ tickets annually is plausible given tickets are used for almost everything.
  • Several note that for Amazon, a few million is small relative to overall profits but still meaningful at the team/feature level.
  • Some point out that replacing an oversized, outdated, general LLM with a small, task‑specific model and trimming inputs by ~70% obviously cuts costs; the value is in actually doing it.

Why Sonnet and why Strands Decider?

  • Questions arise about why Amazon used Sonnet 4.6 instead of cheaper models (Haiku, Luna, Chinese models, Jev, etc.).
  • Replies: Sonnet was likely the best/only viable model on Bedrock when they started; Haiku was weaker then, Luna/Jev/decision models weren’t yet available.
  • Strands Decider is described as an AWS‑originated decision model (initially Qwen‑based, later Gemma‑based) and part of an AWS agent SDK, so this is seen partly as internal dogfooding and partly as a product case study/advert.

Decision models vs general LLMs

  • Many see ticket routing as a classic classification/decision task where specialized models are a better fit: cheaper, faster, more stable calibration, no need for long text generation.
  • Decision models like Strands/Jev are framed as “system 1” components to sit inside or alongside agents, often providing ~90% of a custom classifier’s power without bespoke training.
  • Some confusion exists about terminology (agent vs single model call; “decision tree” vs “decision model”).

LLMs vs traditional ML and data labeling

  • Large subthread revisits SVMs, Naive Bayes, random forests, BERT, embeddings + small heads, etc.
  • Pro‑LLM points: trivial setup, no labeled data or ML expertise needed, great for rapid prototyping and synthetic labeling; later you can distill into smaller models.
  • Pro‑classical points: LLMs are often 100–1000× more expensive for simple classification, slower, sensitive to prompt and input order, and overused due to hype.
  • Several describe a common pattern: ship an LLM‑based MVP, then refactor to specialized models once costs and scale bite.

Ticketing culture and side discussions

  • Multiple comments confirm that very large organizations generate tickets for nearly every operational, IT, and facilities event, often with auto‑generated tickets and complex routing structures.
  • Some see this as necessary traceability and audit trail; others complain about bureaucracy and noise.

Reliability and quality concerns

  • People flag that JSON key order and minor input changes can shift LLM outputs, underscoring their “text completion” nature and fragility.
  • Some criticize the article for focusing on speed and cost with little discussion of accuracy or error rates, leaving real quality impact unclear.