GPUs Go Brrr

ThunderKittens, H100, and matrix-centric “AI hardware”

• Many see the post and ThunderKittens as a clear, fun illustration of how to hit H100 tensor cores properly; some find the meme-heavy style off‑putting or confusing.
• The core idea that “AI wants 16×16 tiles” and that tensor cores dominate available FLOPs resonates; several note that not using tensor cores wastes ~94% of H100’s theoretical compute.
• There’s debate over how much further specialized “AI ASICs” can go beyond modern GPUs. Some argue GPUs have already evolved into AI machines; others think trimming graphics baggage and precision could still yield big gains.

GPUs vs NPUs / TPUs / custom accelerators

• Commenters compare NVIDIA tensor cores with Apple NPUs, Google TPUs, Groq, AMD Versal/XDNA, and various NPUs.
• View that NPUs on consumer devices shine for small, low-power tasks but are underpowered for LLMs/large transformers; GPUs still win for big models.
• Some think future will bring simpler, cheaper AI-only chips; others cite NVIDIA research claiming limited benefit from further specialization.

Memory, bandwidth, VRAM, and form factors

• Strong focus on memory bandwidth vs latency; GPUs hide latency but are bottlenecked by bandwidth and VRAM capacity.
• Many lament lack of high‑VRAM consumer cards and absence of expandable VRAM; several argue it’s economically and strategically undesirable for vendors.
• Unified memory (as in Apple silicon or potential advanced packaging) is seen as attractive for large models but not yet standard.

NVIDIA dominance, docs, and competition

• Some accuse NVIDIA of protecting its moat with incomplete or misleading documentation; others see ordinary market segmentation (better docs for “partners”).
• There's a camp calling for NVIDIA to be “broken up”; a larger counter‑camp credits long‑term investment and superior software (CUDA, tooling, PyTorch support) rather than anti‑competitive barriers.
• AMD is viewed as promising but still behind in software and ecosystem, though recent ROCm progress and MI300x interest are noted.

Local vs cloud AI and consumer devices

• Split between those who want strong on‑device models (privacy, offline speech/vision, accessibility) and those who prefer cloud “AI as a service” to save battery/thermals.
• Apple is seen as likely to blend modest on-device AI with iCloud-based services, trading some privacy positioning for better UX.

Beyond current GPUs: neuromorphic and analog ideas

• Some argue real AGI may require insights from neuroscience/psychology; others think current ML is largely decoupled from brain science.
• Analog and neuromorphic approaches are discussed as active but immature: major challenges in precision, noise accumulation, manufacturing, and debugging.