Intel Gaudi 3 AI Accelerator
Intel’s Gaudi 3 AI accelerator is positioned as a high-memory, high-bandwidth alternative to Nvidia’s H100, pairing 128 GB of HBM2e and 3.7 TB/s bandwidth with 24 integrated 200 Gb Ethernet ports to build large-scale Ethernet-based GPU clusters. Commenters see it as a welcome challenger in a supply-constrained, Nvidia-dominated market but are skeptical about pricing, long-term product commitment, and whether Intel’s software stack (oneAPI/SynapseAI with PyTorch backends) can match CUDA’s maturity. The conversation also contrasts Intel’s approach with AMD’s ROCm ecosystem and highlights the importance of open hardware interfaces like OAM for future reuse and hobbyist access to today’s datacenter hardware.
Memory & performance characteristics
- Gaudi 3 offers 128 GB of HBM2e and ~3.7 TB/s memory bandwidth; some note the press release’s initial “terabytes” wording for bandwidth is confusing but later clarified as TB/s.
- Use of HBM2e (older than HBM3) is seen as a pragmatic choice: similar bandwidth to H100 via more stacks, possibly better supply and cost.
Comparison with Nvidia and AMD
- Memory capacity is attractive for large and sparse models (e.g., MoE) and local LLMs.
- AMD’s MI300x has more HBM (192 GB), but several commenters characterize AMD accelerators as held back by ROCm/driver instability; others counter with examples of successful large AMD deployments (Frontier, LUMI, MI250-based training).
- Gaudi 3 is framed as a rare MLPerf-benchmarked non-Nvidia LLM accelerator and a potential competitive pressure on H100/H200 pricing.
Networking architecture
- The 24×200 Gb Ethernet ports surprise many; initially this is imagined as an unmanageable cable explosion.
- Follow-up explains most ports are used as internal all-to-all links between accelerators within and across nodes, surfaced via a smaller number of OSFP ports and standard leaf–spine Ethernet fabrics (often RoCE over twinax or fiber).
- This Ethernet-based approach is contrasted with Nvidia’s InfiniBand ecosystem, which some find expensive and awkward to procure.
Pricing expectations
- HBM cost estimates suggest a theoretical low bill-of-materials, but consensus is that Gaudi 3 will be priced near Nvidia data-center GPUs, not as a workstation part.
- Multiple comments predict >$10k per PCIe card, with references to current A100/H100 street prices.
Software stack and ecosystem
- Gaudi 3 integrates with PyTorch via its own backend (SynapseAI); it is not CUDA-compatible, and CUDA is noted as legally and technically Nvidia-only.
- Some argue porting many CUDA kernels to oneAPI/SYCL-style models is often straightforward, especially for small kernels, but others push back that tooling maturity and debugging still lag CUDA.
Deployment, stability, and ROCm debate
- Several users report serious ROCm instability on AMD cards (resets, kernel faults, version mismatches), especially off the “blessed” kernel/driver combinations.
- An AMD insider-style perspective explains ROCm as a monolithic, release-locked stack heavily tested only on specific OS and hardware combinations; outside those, behavior is effectively “development branch.”
- Others report MI50/MI-series running stably when staying on the officially validated kernel and full ROCm blob.
Open hardware and future reuse
- Gaudi 3 and recent AMD accelerators using the open OAM standard are praised; in contrast, Nvidia’s proprietary SXM pinouts have made large numbers of older P100s practically unusable outside original baseboards.
- Commenters see open interfaces as enabling future “retro supercomputing” and hobbyist reuse instead of e-waste.
Intel strategy and longevity concerns
- The chip is fabbed at TSMC 5 nm, reflecting Gaudi’s origins as an acquisition and Intel’s current process gap.
- Some view using mature memory and external foundry as classic Intel “push old tech hard” strategy; others worry about Intel’s history of killing non-core product lines and question long-term architectural continuity, despite mentions of a planned successor (Falcon Shores).