Tenstorrent unveils Grayskull, its RISC-V answer to GPUs

Original Article ↗ Hacker News Discussion ↗

Target models and use cases

  • Supported models include BERT, ResNet, Whisper, YOLOv5, and U-Net.
  • Commenters note these were state-of-the-art around 2020–2021 and are still widely used, especially BERT-like models for embeddings and YOLOv5 for edge vision.
  • Many see the boards as a way to learn the programming model, not as turnkey inference appliances.

Dev kit positioning and specs

  • Two dev kits:
    • e75: 75 W, 96 Tensix cores, 96 MB SRAM, 8 GB LPDDR4, ~$599.
    • e150: 200 W, 120 Tensix cores, 120 MB SRAM, 8 GB LPDDR4, ~$799.
  • Multiple low-precision formats supported (FP8/FP4/BFP variants; some 19-bit FP in SFPU).
  • 8 GB VRAM is widely viewed as limiting, especially for LLMs; considered fine for many vision models and quantized experiments.

Architecture & programming model

  • Each Tensix core: tensor math unit (FPU), SIMT engine (SFPU), five tiny RV32I cores, and local memory.
  • Cores are linked by dual torus Network-on-Chip; design emphasizes local memory close to compute.
  • Compared to manycore, transputer-style, and dataflow-like systems; goal is to map ML pipelines across cores and stream data between them.
  • Vector work is not via RISC‑V Vector extension; that’s reserved for future host CPUs (Ascalon).

Memory, bandwidth, and scaling

  • LPDDR4 bandwidth (~100–120 GB/s) is much lower than high-end GPUs; seen as a bottleneck.
  • Large on-chip SRAM is the differentiator; useful for workloads that can reuse data locally.
  • Grayskull is described as inference-only, single-card oriented; next-gen “Wormhole” adds better multi-card scaling with 100 GbE and improved SFPU.

Comparison to GPUs and other accelerators

  • Some question value versus commodity GPUs or APUs with fast DDR5; others note this is a dev kit, not a gaming-competitive product.
  • FLOPs/$ alone deemed less important than memory capacity/bandwidth and software stack.
  • Custom AI accelerators (including this) are seen as more efficient per watt for certain operations but face ecosystem and compiler/toolchain hurdles.

Software, ecosystem, and system requirements

  • Open-ish stack with a high-level (TT-Buda) and low-level (TT-Metalium) API; marketed as having no black-box components.
  • 64 GB host RAM requirement attributed to model compilation needs; PCIe 4.0 likely for bandwidth.
  • Several commenters stress that unseating CUDA’s software moat is harder than beating Nvidia on raw FLOPs or margins.