Qwen 3.8 27B available on Cerebras at 1500 tokens/s

Original Article ↗ Hacker News Discussion ↗

Cerebras has launched ultra‑fast cloud inference for the Qwen 3.8 27B model, delivering around 1,500 tokens per second and impressing many users with its coding capabilities and responsiveness. However, developers highlight significant drawbacks: 128k context instead of the model’s full potential, strict per‑minute token limits, no pricing benefit from prompt caching, and relatively high costs that make sustained or agentic coding workloads impractical. Many conclude the service functions more as a showcase for Cerebras hardware than as a competitive everyday alternative to cheaper GPU‑based or self‑hosted deployments.

Model performance and speed

  • Qwen 3.8 27B on Cerebras is widely praised as very strong for coding and reasoning, with extremely fast output (up to ~1500 tok/s) and low latency.
  • Some users report it “feels” transformative for UX (instant-like answers), especially for search, summarization, and quick code tasks.
  • Others say input processing still dominates time for large prompts, so wall‑clock gains over ~100–200 tok/s models can be modest once tool calls and shell commands are included.

Pricing, caching, and token economics

  • Pricing is around $0.99/M input and $1.49/M output tokens on a pay‑as‑you‑go credit model (no recurring fee, $10 minimum credit).
  • Prompt caching is supported but:
    • Cached input tokens cost the same as fresh tokens.
    • Cached tokens still count toward token‑per‑minute limits.
  • Many see this as “marketing-only” caching that removes the main economic benefit, making agentic workloads significantly more expensive than with competitors that heavily discount cache hits.

Rate limits and suitability for coding/agentic workloads

  • Public endpoint limits: ~150k uncached tokens/min, 450k total/min.
  • At 1500 tok/s, users hit these limits very quickly, especially with large contexts and frequent tool calls.
  • Several report burning through free credits in minutes and hitting limits within 60–90 seconds of sustained agentic coding.
  • Consensus: good for short, bursty tasks; often “unusable” for long‑running coding agents and large projects.

Context window and model configuration

  • Cerebras exposes only 128k context, despite the model supporting more elsewhere.
  • Many consider 128k insufficient for serious coding/agentic setups once you include system prompts, tools, history, and codebase; context gets filled quickly and compaction becomes a choke point.
  • Some note Qwen 3.8’s “extra-high reasoning” default amplifies context needs.

Comparisons: other providers and local hosting

  • Other hosts (e.g., OpenRouter, dedicated GPU providers) offer slower speeds (~80–300 tok/s) but:
    • Better cache discounts.
    • Lower effective cost for long sessions.
  • Multiple users run Qwen 3.8 27B locally (GPUs, ninfer, llama.cpp) at 30–200 tok/s with no rate limits, which many prefer for sustained workloads and sensitive data.
  • For small workloads, Cerebras’ speed premium can be worth the extra cost; for large agentic coding, alternatives are often cheaper overall.

Product strategy, reliability, and support

  • Several commenters see the public API as a demo/marketing channel for selling Cerebras hardware and enterprise deals (e.g., hosting very large models for big customers).
  • Complaints include:
    • Tight quotas, removed/shared‑tier models (e.g., Gemma 4) with little notice.
    • Flaky reliability and confusing errors around billing/access.
    • Heavy reliance on Discord for support, including onboarding issues.
  • Overall sentiment: impressive hardware and raw speed, but public service feels capacity‑constrained, expensive for continuous coding, and not developer‑friendly.