Rust SIMD on the GPU
Rust developers are experimenting with bringing Rust’s portable SIMD abstractions to GPUs, aiming to run existing CPU-oriented libraries efficiently on graphics hardware without rewriting them in CUDA or specialized ML frameworks. Commenters dig into how the proposed trait- and type-driven IR maps to GPU execution models (warps, shuffles, barriers), and what benefits and limitations this has compared to traditional SIMT programming and architecture-specific intrinsics. The thread also touches on broader issues like the maturity of Rust’s SIMD ecosystem, the challenges of “truly portable” SIMD across CPUs and GPUs, and how a startup might open-source its compiler stack while building commercial products on top.
Project goals and design
- Work aims to let existing, unmodified CPU libraries (including
core::simd-using code) run on GPUs. - Uses a strongly-typed IR with traits parameterized by operations (reductions, scans, shuffles, strip mining).
- Execution “shape” is encoded in types; invalid combinations (e.g., warp executor with device barrier) become compile-time errors.
- Design uses type-level constants for shuffle patterns and strip-mining parameters; supposed to catch many GPU misuse patterns at compile time.
- Intended to reuse similar abstractions across CPU and GPU, not to “replace” CPUs.
Compiler, availability, and business model
- Implementation currently relies on a custom compiler fork; not yet public.
- Plan stated: open source compiler and standard library pieces after product launch; business rests on products built on top, not compiler sales.
- Some commenters express frustration that it can’t be tried now or benchmarked.
Use cases vs existing ML stacks
- Skepticism: if you must use an array-programming DSL (scan/gather/etc.), why not just use Torch/TF/JAX/MLIR?
- Response: this adds little for hand-written ML workloads; value is for general CPU libraries transparently gaining GPU acceleration.
GPU vs CPU and SIMD/SIMT discussion
- Several explanations of how GPU “cores” map to SIMD lanes, warps/waves, massive register files, and latency-hiding via many resident threads.
- Contrast: CPUs optimize for low latency and complex cache/speculation; GPUs for throughput and high bandwidth.
- Debate over terminology: GPUs as SIMD vs SIMT; some emphasize they are vector ISAs under a “threads” programming model.
Portable SIMD, nightly Rust, and ecosystem
- Rust’s portable SIMD is nightly-only; some users switched to alternatives (e.g.,
fearless_simd) for stable builds. - Mixed views on long time in nightly: some see it as prudent baking; others frustrated that “basic” utilities and slice/pointer methods stay unstable for years.
- Debate on portable SIMD:
- Critique: most examples pick a fixed vector width, hurting true portability and performance portability.
- Counterpoint: width-agnostic generics are possible; portable SIMD is a useful middle ground when auto-vectorization is insufficient but ISA-specific tuning isn’t needed.
- Acknowledgment that optimal widths differ by architecture and sometimes runtime conditions; selection strategies via
cfg, multi-variants, or JIT are discussed. - Some argue SIMD ISAs differ enough that only a subset of use cases can be truly portable; others note that “trivial” vector/matrix math already covers most practical needs.
Miscellaneous
- Some curiosity about target application domains, especially beyond LLMs; thesis mentioned that most devices ship with underutilized GPUs.
- Request for concrete, competitive benchmarks (e.g., radix sort); none provided in the thread (performance impact remains unclear).
- Minor meta threads on possible bot comments and on the blog’s “pedantic mode” toggle for extra technical detail.