Why and how we’re migrating many of our servers from Linux to the BSDs

Original Article ↗ Hacker News Discussion ↗

Reaction to the migration story

  • Several commenters appreciate seeing concrete results from Linux→BSD migrations but want more technical depth (e.g., what exactly caused XFS failures, why workloads sped up).
  • Some consider running three different BSDs as “not solving problems” due to operational overhead; others see tailoring OS to task as a positive.

Filesystems: XFS, ext4, ZFS, Btrfs, UFS

  • XFS: Mixed history. Older Linux XFS on cheap hardware was described as fragile under power loss; modern XFS is widely praised as fast and reliable, especially with mdadm. Some want more detail on the author’s specific XFS incident.
  • ext4: Seen as “boring but safe,” with strong recovery tools and design tuned for commodity PCs.
  • ZFS: Many like ZFS (especially on FreeBSD, Solaris, and appliances like TrueNAS); others complain about Linux ZFS kernel-compatibility friction and higher resource use. One notes serious ZFS bugs and a contributor regretting the encryption merge.
  • Btrfs: Deeply polarizing. Some report years of trouble-free use (including large-scale deployments), others repeated catastrophic failures, especially when volumes fill or snapshots hide space usage. RAID5/6 is widely regarded as unsafe.
  • UFS/soft updates: Praised for simplicity, metadata safety, snapshots, and background fsck; seen as slower but robust.
  • General theme: real-world experience varies wildly; hardware quality, RAID level, and usage patterns matter.

BSD vs Linux: strengths and weaknesses

  • BSD positives: cohesive “full OS” design, stable interfaces, simple init (rc.conf), jails, strong docs and manpages, predictable upgrades, and good ZFS integration. Often preferred for NAS, routers, firewalls, and small servers.
  • BSD negatives: weaker desktop, Wi‑Fi, GPU and power‑management support; historically weaker Java support; some rough edges in ZFS booting and /dev semantics; smaller ecosystem and job market.
  • Linux positives: broader hardware and software support, containers/Kubernetes ecosystem, and more people familiar with it.
  • Some recommend FreeBSD (or TrueNAS) specifically for ZFS file servers.

Containers, Kubernetes, and orchestration

  • Lack of a mature Kubernetes equivalent on BSD is a blocker for teams heavily invested in microservices.
  • Opinions on Kubernetes are split:
    • Pro: common platform across enterprises, consistent deployment model, cloud-managed offerings hide much complexity.
    • Skeptical: seen as “ornamental/accidental complexity,” expensive, overkill for non–hyper-growth businesses, and often adopted as cargo cult.
  • Alternatives mentioned: jails plus tools like BastilleBSD, systemd-nspawn, microVMs; some argue traditional VMs plus config management can be simpler.

Operational practices and client trust

  • Major debate over the author migrating client VMs to BSD “without telling them.”
    • One side: the OS is an implementation detail; if SLAs are met and contracts allow infrastructure changes, silent migration is fine.
    • Other side: changing OS carries high risk; clients should be warned of potentially disruptive work even if it ends up improving performance.
  • Some stress the need for maintenance windows and clear communication; others say providers routinely change internals (like cloud vendors) and customers don’t care so long as service quality remains.

Careers, economics, and meaning of IT work

  • A thread explores how IT “optimization” mainly enriches owners while automating away workers’ jobs, feeding a sense of meaninglessness.
  • Mid-career workers describe feeling trapped in “soulless” but well-paid roles, unable to retrain without risking family stability.
  • Others counter that modern living standards and housing expectations complicate claims of generational poverty, but acknowledge decoupling of meaningful work from livelihood.