The Rust calling convention we deserve

Rust ABI, calling conventions, and performance tradeoffs

  • Many agree C-style ABIs are suboptimal for Rust semantics (Results, enums, large structs, multiple returns).
  • Suggestions include richer register use (e.g., more return registers, flags for Result-like success/error) and different conventions for internal vs escaping functions.
  • Some argue Rust shouldn’t be locked to LLVM’s current limits; others note Rust already inserts checks (e.g., div-by-zero) to avoid LLVM UB.
  • There’s debate over per-function or type-driven calling conventions vs a single fixed ABI; debug support and dynamic linking complicate non‑C‑like approaches.
  • Several participants stress: design must be guided by measurement on real code and hardware, not aesthetics. Surprising wins from “ugly” stack-passing are reported, especially given CPUs tuned on C compiler output.

Inlining, code size, and dynamic dispatch

  • Heavy inlining reduces call overhead, but not all calls can be inlined (e.g., dynamic dispatch such as dyn Trait, cross-crate boundaries).
  • Cheaper calls could allow less aggressive inlining, improving code size and compile times.
  • For small/embedded targets, current Result-heavy patterns are seen as code-size-unfriendly; ideas include returning discriminants in registers while large variants live on the stack.

Struct layout, niches, and Option

  • Rust already reorders fields (unless repr forbids it); there’s an unstable flag to detect layout assumptions.
  • A recurring theme is generalizing niche optimizations to reduce enum and Option size and padding.
  • Example complaint: many Option<T> fields inside a struct waste space because each carries its own discriminant; manual bitpacking with MaybeUninit can be more compact but loses direct &Option<T> borrows.

Interop and managed vs unmanaged memory

  • Rust–Go interop is commonly done via extern "C" and cgo; described as workable but tedious due to manual type marshaling.
  • Strong disagreement over whether mixing managed and unmanaged languages is “usually unwise”: some say it’s broadly problematic, others say it’s routine and efficient in ecosystems designed for it (.NET, Swift/ARC).
  • Techniques like pinning, GC roots, and finalizers are mentioned as necessary but fiddly.

MLIR and related tangents

  • MLIR is described as a framework for intermediate “dialects” and progressive lowering, not directly relevant to Rust calling conventions.
  • It does have a linalg dialect for common linear algebra ops, but that’s treated as orthogonal.

Site and presentation details

  • Several comments note the blog’s skewed headings and minimap; some find it creative, others think it harms readability or accessibility.