CUDA-oxide: Nvidia's official Rust to CUDA compiler

Original Article ↗ Hacker News Discussion ↗

Role of cuda-oxide vs existing Rust/CUDA tooling

  • cuda-oxide is positioned as a Rust-to-CUDA kernel compiler, not just a host API.
  • It compiles Rust via rustc → MIR → a custom Pliron-based IR → LLVM IR → PTX, which is embedded in the host binary.
  • Existing crates like cudarc focus on host-side CUDA: contexts, memory, launching kernels, calling existing C++/PTX/CUBIN.
  • Several commenters conclude they are complementary: use cuda-oxide to generate PTX, then potentially drive it from host libraries like cudarc.

Build system and performance

  • Many Rust CUDA workflows currently shell out to CMake/nvcc, which slows builds; caching with tools like sccache helps but doesn’t remove nvcc cost.
  • cuda-oxide’s pipeline avoids nvcc for device code, using the Rust/LLVM toolchain instead.
  • One commenter notes they haven’t seen major nvcc overhead in practice when recompiling only on file changes.

Safety model, memory model, and ergonomics

  • There is interest in whether Rust’s memory model and type system can make GPU programming safer than CUDA C++.
  • The docs describe layered safety:
    • “Common case” (one thread per element) is safe by construction.
    • Shared memory, warp intrinsics, etc. are “mostly safe” or require unsafe with contracts.
    • Advanced hardware features remain fully manual/unsafe.
  • Examples of added guardrails vs C++: managed lifetimes instead of manual cudaFree, type-checked kernel arguments, DisjointSlice<T> to prevent multiple threads writing the same element, and restricted memcpy targets.
  • Some worry this is “not Rusty enough” if much of the power surface is still unsafe, questioning the value vs C. Others see it as substantial but incomplete safety.

Host–device data sharing

  • A major selling point is single-source Rust: the same Rust structs can be used on host and device without manual byte-level serialization.
  • cuda-oxide uses rustc’s computed layout so device accesses match host layout, including slices and nested structs.
  • Caveat: host-side heap-owning types (Vec, String, trait objects) still need explicit device-friendly representations.

Open-source, ecosystem, and alternatives

  • Some argue this doesn’t solve objections to closed NVIDIA drivers and toolchain; CUDA remains proprietary overall.
  • Clarification: cuda-oxide does not feed Rust into nvcc, but still requires NVIDIA drivers/toolkit to run PTX.
  • Comparisons are made to Mojo, HIP/ROCm, Slang, and newer IRs (NVIDIA MLIR, Tile IR), with disagreement on how “1:1” or mature these alternatives are relative to CUDA.

AD and scientific computing

  • Some want automatic differentiation (AD) to be first-class in any new GPU language ecosystem.
  • Others note ongoing Rust work on std::autodiff, built on the Enzyme LLVM plugin, and related “Rust for SciComp” goals, but it’s pre-RFC and not yet stable.

AI-written docs and code quality

  • Several comments criticize the cuda-oxide docs’ tone as “LLM-written,” citing common stylistic tics.
  • There’s debate over whether AI authorship is inherently bad:
    • Some see AI-written code/docs as strongly correlated with low quality.
    • Others argue only actual code quality matters, not how it was authored.

General sentiment

  • Many are enthusiastic: easier Rust+CUDA integration, less FFI, less manual serialization, and stronger safety guarantees are seen as big wins.
  • Skeptics worry about remaining unsafety, NVIDIA lock-in, and long-term ecosystem complexity.
  • Some see this as yet another reason they’ll eventually “have to” learn Rust.