ZFS 2.3 released with ZFS raidz expansion

Original Article ↗ Hacker News Discussion ↗

Release features and general reaction

  • Major additions: online RAIDZ expansion, faster dedup, Direct IO (ARC bypass), optional JSON output, and much longer filename support.
  • Many users see RAIDZ expansion as a long‑awaited, “huge” quality‑of‑life feature, especially for home/hobbyist NAS setups.

RAIDZ expansion: behavior and caveats

  • Expansion adds disks to an existing RAIDZ vdev online while maintaining redundancy and being resumable.
  • Old blocks retain their original data/parity ratio; only new writes use the wider layout. This can leave the pool less space‑efficient than a freshly created wider RAIDZ.
  • Some consider this inefficiency minor and acceptable; others see it as a “huge caveat” for home users who grow arrays over time.
  • Workarounds to “re-layout” data: delete snapshots and rewrite in place, or use zfs send/receive within the pool. Both lose reflink-based deduplication.
  • You still cannot shrink a RAIDZ vdev or remove a RAIDZ top‑level vdev; expansion is one‑way.

Vdev removal and reshaping vs other systems

  • ZFS can remove some top‑level vdevs (typically mirrors) by copying their contents into free space and redirecting reads, but not RAIDZ vdevs yet.
  • Other stacks (Windows Storage Spaces, mdadm+LVM, btrfs, bcachefs) can add/remove devices and reshape arrays more flexibly, sometimes with automatic rebalance.
  • Debate: traditional RAID reshape is easier but can handle corruption poorly; ZFS emphasizes correctness and edge‑case handling over flexibility.

Encryption and Direct IO

  • New features are said to be mostly orthogonal to native encryption; known issues remain around non‑raw send/receive of encrypted datasets.
  • A Linux kernel API change limiting access to SIMD instructions forced ZFS to provide its own equivalents; performance was impacted temporarily but encryption correctness was not.
  • Direct IO is aimed at workloads with their own caching (e.g., databases on fast NVMe). It avoids extra copies through ARC and may significantly help synchronous IO.

Performance, caching, and NVMe

  • ARC vs Linux page cache interaction is discussed; ZFS already flushes the lower‑level buffer cache to avoid double caching.
  • Settings like txg timeout and vdev queue limits can strongly affect small synchronous write amplification and overall throughput.
  • L2ARC now persists across reboots, making SSD caching more useful for desktops (e.g., Steam libraries).

Home/desktop usage and comparisons

  • Many use ZFS at home for: end‑to‑end checksums and scrub, snapshots and easy rollbacks, send/receive backups (including to removable/offsite disks), compression, and encryption.
  • Several emphasize that “enterprise disks”, ECC, huge RAM, or many‑disk arrays are not hard requirements, though ECC is considered beneficial for any filesystem.
  • Compared to btrfs, commenters describe ZFS as more mature and predictable, especially for parity RAID; btrfs is often recommended only for single‑disk or RAID1.
  • bcachefs is noted as promising but still missing key features (full erasure coding, send/receive, complete scrub) and not yet widely trusted for mission‑critical data.

Licensing and platform support

  • CDDL/GPL incompatibility keeps ZFS out of the mainline Linux kernel; ZFS remains a separate module with some friction on fast‑moving distros.
  • Relicensing is described as legally and practically difficult due to many contributors and Oracle’s limited involvement with OpenZFS.
  • Windows and macOS ports of OpenZFS exist; Windows also offers Storage Spaces + ReFS as a partially ZFS‑like alternative, with different trade‑offs.