Vulkan video extensions for accelerated H.264 and H.265 encode

Original Article ↗ Hacker News Discussion ↗

Khronos’ new Vulkan Video extensions for H.264 and H.265 hardware encode/decode are prompting hopes for a single, cross-vendor API that could simplify video acceleration across GPUs and operating systems, especially on Linux. Commenters weigh Vulkan’s potential against existing OS- and vendor-specific paths like VA-API, Windows Media Foundation, NVDEC/NVENC and others, noting gaps in driver support, codec patent issues, and platform fragmentation (particularly on macOS and Android). There is also interest in upcoming AV1 support and how well modern codecs can be parallelized, both in dedicated hardware blocks and in software decoders like dav1d.

Scope and Goals of Vulkan Video

  • Uses existing fixed-function encode/decode blocks on GPUs/SoCs, similar hardware that VAAPI, NVENC/NVDEC, etc. target.
  • Promoted as a cross-vendor, cross-platform video API, potentially covering Intel, AMD, Nvidia, mobile SoCs, and small devices (e.g. Raspberry Pi).
  • Seen as especially promising for Linux, where there is no unified native media API and driver fragmentation is painful.

Comparison to Existing APIs (VAAPI, DXVA, etc.)

  • Vulkan Video and VAAPI address the same hardware; differences are in API design:
    • Vulkan is more explicit, less codec-specific, and can integrate cleanly with Vulkan rendering (easier zero-copy to textures).
    • VAAPI is widely used on Linux, well supported by Intel/AMD and some third‑party Nvidia shims, but not officially by Nvidia.
  • Some argue VAAPI is no longer “Intel‑only”; others emphasize it and VDPAU/NVDEC/AMF remain vendor-/platform-specific.
  • Skeptics say migrating OBS/Firefox off mature OS-native APIs (DXVA/WMF, VideoToolbox, MediaCodec) adds complexity, especially since macOS and Android lack Vulkan Video and Windows Vulkan is secondary to DirectX.

Firefox and Hardware Decode on Linux

  • Firefox already has hardware decode on Linux via ffmpeg+VAAPI, including on Intel and AMD; Nvidia remains problematic without unofficial shims.
  • Users can verify usage via about:support’s Media section or GPU-specific tools.
  • Mesa disables patent-encumbered codecs by default; many distros keep them off to avoid legal risk, so H.264/H.265 may require extra steps, while AV1 generally works out of the box.

Driver Deployment and Usability Issues

  • On Windows, OEM/Microsoft-provided drivers often ship without a working Vulkan runtime by default; DirectX works but Vulkan may fail until users install an extra runtime, which is acceptable for games but problematic for general apps.

Hardware and Parallelism in Codecs

  • “Software decoding” means CPU-based; “hardware decoding” uses fixed-function units attached to GPUs/iGPUs (e.g., Quick Sync), not general GPU cores.
  • Video decode is partly inherently serial (e.g., entropy decoding, motion prediction chains), but modern codecs (H.265, AV1) provide tiles/slices for parallelism, trading some compression efficiency for multi-threaded decode/encode.