DBRX: A new open LLM

Original Article ↗ Hacker News Discussion ↗

Training data and transparency

  • Several commenters want details on DBRX’s training corpus; blog promises a technical report but doesn’t specify sources.
  • People infer it’s likely similar to other LLMs: large-scale web text, books, cleaned and reweighted.
  • There’s broader concern that training data is now treated as a “moat” and entangled with legal risk, reducing transparency.

Model size, quantization, and hardware

  • Base model card lists ~264GB RAM requirement; many see this as impractical outside server settings.
  • Extensive discussion of 4–6-bit quantization: high-bit (Q5/6) is often near-baseline; Q4 visibly degrades quality, especially for smaller models.
  • Larger models reportedly quantize “better,” but evidence is mixed; some users share anecdotes and a recent paper with nuanced results.
  • Users describe running DBRX/Mixtral-scale MoE models on consumer GPUs or Apple Silicon using quantization and offloading, but often at low tokens/sec. VRAM, not raw FLOPs, is the main bottleneck.

MoE architecture and runtime behavior

  • DBRX is a 132B-parameter MoE with ~36B active parameters per token (16 experts, 4 active), offering 36B-like speed with 132B-like capacity.
  • All 132B parameters must still reside in GPU memory for efficient expert switching.

Licensing and “openness”

  • License forbids using DBRX to train other LLMs and imposes a >700M-MAU carve‑out requiring a separate agreement.
  • Many argue this is “open weights” or “weights-available,” not true open source; comparisons made to LLaMA-style licenses.
  • Some worry custom licenses create legal friction for enterprises; others are fine with large-platform restrictions aimed at hyperscalers.

System prompt and safety behavior

  • The released system prompt enforces neutrality, discourages stereotyping, disallows lyrics/poems/news reproduction, and instructs the model not to discuss its training data.
  • Some see the “not trained on copyrighted X” line as legally/technically dubious but note prompts are about shaping behavior, not truth.

Benchmarks and chart design

  • DBRX modestly outperforms popular open models on MMLU, HumanEval, GSM8K; many note gains are small.
  • Strong criticism of Databricks’ initial bar charts (truncated axes, mis-scaled bars, bar ordering) as misleading “chart crime”; later said to be a mistake and fixed.
  • Some argue benchmarks like MMLU are noisy and contain errors; others say saturated scores make further gains inherently incremental.

LLM landscape, commoditization, and Databricks strategy

  • Several commenters feel transformer LLMs are hitting diminishing returns and converging in capability; models are increasingly interchangeable for many API-style use cases.
  • Consensus that differentiation will shift to fine-tuning, tooling, infra, and business layer rather than raw models.
  • Databricks is seen as using DBRX mainly to:
    • Prove its platform can train state-of-the-art models (Mosaic AI Training, end-to-end stack).
    • Sell compute/model-serving to enterprises preferring in-house or customized models over pure API dependence.
  • Some are bullish on this “commoditize the complement” strategy; others think DBRX lags frontier proprietary models and won’t materially threaten them.

Databricks tooling and notebooks

  • Data engineers/scientists are split: some appreciate Databricks’ managed Spark and infra; others dislike notebook-centric workflows, slow cluster start, and historically weak local-dev/versioning.
  • Databricks advocates respond that local dev, IDE integration, and better packaging/testing support now exist.