Intel Gaudi 3 AI Accelerator

Memory & performance characteristics

• Gaudi 3 offers 128 GB of HBM2e and ~3.7 TB/s memory bandwidth; some note the press release’s initial “terabytes” wording for bandwidth is confusing but later clarified as TB/s.
• Use of HBM2e (older than HBM3) is seen as a pragmatic choice: similar bandwidth to H100 via more stacks, possibly better supply and cost.

Comparison with Nvidia and AMD

• Memory capacity is attractive for large and sparse models (e.g., MoE) and local LLMs.
• AMD’s MI300x has more HBM (192 GB), but several commenters characterize AMD accelerators as held back by ROCm/driver instability; others counter with examples of successful large AMD deployments (Frontier, LUMI, MI250-based training).
• Gaudi 3 is framed as a rare MLPerf-benchmarked non-Nvidia LLM accelerator and a potential competitive pressure on H100/H200 pricing.

Networking architecture

• The 24×200 Gb Ethernet ports surprise many; initially this is imagined as an unmanageable cable explosion.
• Follow-up explains most ports are used as internal all-to-all links between accelerators within and across nodes, surfaced via a smaller number of OSFP ports and standard leaf–spine Ethernet fabrics (often RoCE over twinax or fiber).
• This Ethernet-based approach is contrasted with Nvidia’s InfiniBand ecosystem, which some find expensive and awkward to procure.

Pricing expectations

• HBM cost estimates suggest a theoretical low bill-of-materials, but consensus is that Gaudi 3 will be priced near Nvidia data-center GPUs, not as a workstation part.
• Multiple comments predict >$10k per PCIe card, with references to current A100/H100 street prices.

Software stack and ecosystem

• Gaudi 3 integrates with PyTorch via its own backend (SynapseAI); it is not CUDA-compatible, and CUDA is noted as legally and technically Nvidia-only.
• Some argue porting many CUDA kernels to oneAPI/SYCL-style models is often straightforward, especially for small kernels, but others push back that tooling maturity and debugging still lag CUDA.

Deployment, stability, and ROCm debate

• Several users report serious ROCm instability on AMD cards (resets, kernel faults, version mismatches), especially off the “blessed” kernel/driver combinations.
• An AMD insider-style perspective explains ROCm as a monolithic, release-locked stack heavily tested only on specific OS and hardware combinations; outside those, behavior is effectively “development branch.”
• Others report MI50/MI-series running stably when staying on the officially validated kernel and full ROCm blob.

Open hardware and future reuse

• Gaudi 3 and recent AMD accelerators using the open OAM standard are praised; in contrast, Nvidia’s proprietary SXM pinouts have made large numbers of older P100s practically unusable outside original baseboards.
• Commenters see open interfaces as enabling future “retro supercomputing” and hobbyist reuse instead of e-waste.

Intel strategy and longevity concerns

• The chip is fabbed at TSMC 5 nm, reflecting Gaudi’s origins as an acquisition and Intel’s current process gap.
• Some view using mature memory and external foundry as classic Intel “push old tech hard” strategy; others worry about Intel’s history of killing non-core product lines and question long-term architectural continuity, despite mentions of a planned successor (Falcon Shores).