JEDEC publishes GDDR7 graphics memory standard

Original Article ↗ Hacker News Discussion ↗

JEDEC’s new GDDR7 graphics memory standard prompts debate over how relevant formal standards bodies remain as companies like NVIDIA, Apple and Micron increasingly push proprietary or semi-proprietary DRAM variants. Commenters delve into the technical changes in GDDR7—such as PAM3 signaling, narrower but more numerous channels, on-die ECC and command-bus redesign—and what these mean for latency, bandwidth, power, and controller complexity. The conversation also touches on broader trends in high-speed interconnects (e.g., RF-style modulation, SerDes design), and how memory bandwidth and capacity shape the competitive landscape for GPUs and local AI inference.

JEDEC’s Role and Relevance

  • Some question how relevant JEDEC still is, given proprietary memory variants and non-interchangeable modules.
  • Others argue JEDEC remains critical as a neutral standards body for interoperability, especially for DRAM and HBM, and note many major companies are members.
  • Standardization is seen as enabling economies of scale and giving hardware/firmware engineers a reliable baseline.

Memory Architectures: Bandwidth vs Latency

  • Apple’s SoCs are noted for wide memory interfaces and large unified RAM beneficial for GPUs and LLM inference, but their CPUs don’t fully exploit peak bandwidth.
  • x86 desktops/laptops are criticized for narrow memory buses despite rising core counts.
  • Several commenters stress that for CPUs, latency still matters more than raw bandwidth in typical workloads.
  • Discussion of NUMA effects (e.g., dual 512-bit vs “true” 1024-bit) and optimal core-per-channel ratios for throughput.

Signaling Trends and Interconnects

  • High-speed standards (GDDR, PCIe, USB, Ethernet) are all moving to multi-level signaling (PAM, QAM-like ideas) rather than only higher clocks or more pins.
  • Explanations of why: off-chip links are signal-integrity and power-limited; serial high-order modulation plus SERDES beats very wide parallel buses on boards.
  • Trade-offs highlighted: more complex modulation increases latency, power, and implementation complexity, though it saves pins and board real estate.

GDDR7 Technical Changes

  • Key changes: PAM3 signaling on data pins, on-die ECC, narrower data and command/address buses, added read clock (RCK), more mode registers, and a static data scrambler.
  • Detailed breakdown of GDDR7’s symbol encoding (11b7S, 3b2S, 2b1S) to pack data, CRC, and flags into 176 PAM3 symbols per transaction.
  • Debate on whether latency will increase due to complexity vs potential timing gains; outcome considered unclear.

Industry and AI Implications

  • Some argue Nvidia risks falling behind in consumer AI inference due to limited VRAM capacity, while Apple’s unified memory and AMD’s upcoming designs are improving.
  • Others counter that the real money is in training, where Nvidia’s CUDA ecosystem and software lead dominate, and “consumer inference” may be a minor market.
  • A side thread notes Nvidia’s past use of non-JEDEC GDDR5X/6X and AMD’s earlier consumer HBM cards.

Miscellaneous

  • Curiosity about future interconnects (e.g., optical/QAM, embedded fiber in PCBs).
  • Desire for memory that can dynamically optimize for graphics vs system performance.
  • Brief speculation on whether future HBM generations will reappear in retail GPUs.