Big data on the cheapest MacBook

Original Article ↗ Hacker News Discussion ↗

Big data: definitions and moving goalposts

  • Several definitions surface: data too large for a single machine; too large for a maxed-out laptop; too large for a single disk; or simply “doesn’t open in Excel.”
  • Some frame “big data” as workloads that require distributed systems vs “small/medium data” that fit in-memory or on-disk on one box.
  • Others argue most people don’t actually have “big data” (Google-scale) and that the marketing term is overused; some prefer “extreme data.”
  • A meta-point: hardware improved so much (multi‑TB RAM, petabyte disks) that what once demanded “big data” tooling is now “small data.”

MacBook Neo + DuckDB benchmarks

  • Thread sees the DuckDB-on-Neo article partly as a meme and partly a proof you can do nontrivial analytics on very modest hardware.
  • Some note that 100M-row analytical workloads and TPC-DS SF300 are not “big data” by strict definitions, but are meaningful real-world analytics.
  • The author later reports an AWS c6a.4xlarge with 32GB RAM still limited by EBS I/O, only ~2× faster than the Neo on TPC-DS SF300.

8GB RAM, SSD wear, and suitability

  • Strong split: many say 8GB on Apple Silicon is “fine” for dev, office work, and moderate analytics, helped by memory compression and fast NVMe swap.
  • Others insist 8GB is already constraining (Docker, heavy IDEs, LLM tools, many browser tabs) and a poor long-term choice in 2026.
  • Concern about swap-heavy workloads wearing out the soldered SSD resurfaces; countered by claims most 10‑year-old Mac SSDs still function and earlier M1 wear issues were OS-bug related. Overall reliability impact is left unclear.
  • Consensus from the article and many comments: Neo is great as a cheap client and for light/medium work, not for daily heavy data processing.

Cloud vs local compute

  • Benchmarks using EBS-backed AWS instances are criticized as unfair; several suggest instances with local NVMe (c8gd, c8id, i7/i8 families) for a better comparison.
  • One estimate: a top-end c8g.metal-48xl costs enough that ~90 hours of on-demand use equals the purchase price of a Neo.
  • Broader debate: some see cloud compute and bandwidth as massively overpriced and not more reliable than well-managed bare metal; others stress its value for rapid scaling and flexibility, especially for startups.
  • Hybrid patterns (bare metal primary, cloud as DR/burst capacity) are proposed as cost-effective.

Low-end machines and “real work”

  • Many anecdotes of substantial work (startups, BI, iOS apps, PHP/jQuery SaaS, academic work, even some AI orchestration) done on older or very low-end hardware: M1 Airs, old Intel MBPs, cheap Chromebooks, phones with Termux, etc.
  • This is used both to argue that Neo-class machines are sufficient for most dev/analysis and to criticize modern software bloat.
  • Counterpoint: ultra-cheap Windows laptops often ship with slow eMMC, poor build quality, and painful UX; some argue they’re technically capable but unpleasant to use.

DuckDB reception

  • DuckDB is widely praised: easy to embed, great for local analytics, strong performance on columnar workloads, and a “great open source gift.”
  • Examples include replacing hundreds of lines of custom ETL code, large speedups over legacy backends, and outperforming Polars in some large Parquet workflows that caused OOMs.