The drive stats of Backblaze storage pods

Original Article ↗ Hacker News Discussion ↗

Backblaze’s latest report on the performance of its storage pods and hard drives prompts debate over how to best measure drive reliability and why the company is shifting from custom-built “pods” to off‑the‑shelf Dell and Supermicro servers as prices and density improve. Commenters highlight the value of Backblaze’s long-running, model-level drive failure statistics for choosing disks, while also probing how rack placement, vibration, and age affect failure rates. The thread broadens into comparisons of Backblaze B2 with S3, R2, Wasabi, and others on latency, egress pricing, and reliability, along with practical backup tooling choices (e.g., restic vs. alternatives) and encryption trade-offs.

Drive stats, reliability, and hardware behavior

  • Relocating 40 Dell servers to the top of 52U racks coincided with high failure rates; later data shows improvement after changes, suggesting issues like vibration or cooling at the rack top.
  • Commenters note total deployed capacity around 3,000+ PB.
  • Some want failure metrics by deployment age window (e.g., 0–300 days, 301–600 days) rather than lifetime averages, arguing that age-related spikes could be hidden.
  • Shared prior Backblaze curves: SSDs show classic “bathtub” failure rates; HDD AFR reportedly under 2% for the first ~3.5 years due to better factory screening.

From custom storage pods to COTS servers

  • Thread discusses why Backblaze moved from custom pods to Dell/Supermicro: cost, operational simplicity, maturing high-density storage offerings, and better vendor pricing at scale.
  • Early on, few off-the-shelf systems matched their density needs; now many do, reducing the value of custom designs.
  • A Backblaze representative confirms use of Dell PowerEdge R740xd2 in at least one data center and that they use their own drive mix, not expensive OEM-branded disks.
  • Some argue they could still go directly to large OEM/ODM manufacturers instead of Dell; others note that margins on storage servers have fallen due to competition with cloud and DIY.

Drive model data and vendor reputations

  • One commenter complained that the pod article lacked per-model failure data, then realized Backblaze already publishes detailed quarterly drive stats (with model numbers and now datacenter fields).
  • Several people say they choose HDDs based on those stats.
  • Multiple comments express long-running disappointment with Seagate reliability, supported (in their view) by the published data.

B2 vs S3 and other object stores (performance & reliability)

  • One user reports B2 being unreliable at production scale: roughly monthly outages and severe latency spikes (15 ms → ~25 seconds per operation), prompting a return to S3 for primary use.
  • Others find B2 fine for light or personal workloads (e.g., data science, personal backups).
  • Comparisons:
    • Cloudflare R2 praised for free egress but criticized as slow by some; others request more concrete benchmarks.
    • Wasabi and Hetzner Storage Boxes cited as cheaper or faster in some backup scenarios.
  • Several note that S3’s higher cost is tied to very high durability/availability guarantees and rich ecosystem features (events, lifecycle, cross-service free transfer).

Cloudflare CDN / R2 / ToS confusion

  • Detailed debate over whether Cloudflare Workers + CDN can front non-Cloudflare storage (like B2) without violating ToS.
  • Blog posts, ToS language, and employee comments are seen as inconsistent; some users err on the side of caution and follow only the strict written terms.
  • Consensus: R2 behind Cloudflare’s CDN is allowed; Workers used purely as a bandwidth-avoidance proxy are more ambiguous.

B2 pricing structure and egress economics

  • One commenter analyzes B2’s $6/TB-month storage and $10/TB egress with “3× stored” free egress:
    • At 1 TB stored / 1000 TB egress: very expensive.
    • At 334 TB stored / 1000 TB egress: much cheaper, because all egress fits into the “3× storage” free tier.
    • This leads to a theoretical incentive to pad storage with dummy data to reduce total cost.
  • A Backblaze representative contrasts this with hyperscaler egress (~$40–$230/TB list prices), arguing B2’s egress is relatively cheap and explaining they offer a capacity-based B2 Reserve tier (includes egress) starting around 20 TB.

Backup tools, workflows, and security

  • Many use B2 as a backend with tools like restic, rclone, or k8up for servers, NAS, and Kubernetes PVCs; bills can be under $1/month for ~100 GB.
  • Debate: restic vs rustic
    • rustic praised for append-only support and declarative configs but acknowledged as beta and not recommended yet for production.
    • Others note restic has append-only modes (though limited by backend type) and is more mature.
  • Backblaze personal “Computer Backup” vs B2 is clarified:
    • Computer Backup is per-computer, “unlimited” backup with its own client, version history windows, restore via web/mailed drive, and a more complex encryption story.
    • B2 is raw object storage where tools like restic handle encryption end-to-end.
  • Several criticize the Computer Backup encryption design, where the server temporarily accesses decrypted data and/or key material for web-based restore; others emphasize that with B2 plus client-side tools, the provider never sees plaintext.

User experiences with Backblaze products

  • Multiple comments praise B2 as a cheap, effective backup target for personal use (e.g., restic hourly snapshots for ~100 GB costing ~$0.60/month).
  • Enthusiastic support for Backblaze’s openness in publishing detailed drive stats and infrastructure reports; some say they read every release and use the data to guide purchasing.
  • Criticism appears around:
    • B2 latency and reliability for higher-traffic workloads.
    • The usability and robustness of the Computer Backup client, with some preferring legacy solutions like CrashPlan or other backup software.
  • A Backblaze representative explicitly solicits product feedback and feature requests.

Local storage, NAS, and consumer hardware choices

  • For personal media (photos/videos), suggested strategies include:
    • Using B2, Google Photos, iCloud, Dropbox, OneDrive, or a NAS (e.g., Synology) with cloud backup.
    • Combining a NAS with offsite backups (e.g., B2 and/or Glacier) for disaster resilience.
  • For cheap local expansion, external HDDs are preferred over flash drives for reliability; cautions about:
    • Fake-capacity SSDs/flash on marketplaces; rough heuristic price points per TB.
    • Fire/flood risks of only-local disks.
  • Drive purchasing tips: B&H, Microcenter, enterprise wholesalers (for work), and shucking external enclosures from big-box retailers.
  • Small buyers of servers (10–20 units) face high markups and negotiation overhead; suggestions include accepting markup or buying off-lease gear.

Overall sentiment

  • The thread mixes strong appreciation for Backblaze’s transparency and affordable storage with pointed criticism of specific reliability, pricing edge cases, client software design, and some confusing aspects of product behavior.