Warning: $14k BigQuery charge in 2 hours

Original Article ↗ Hacker News Discussion ↗

A researcher querying Google’s BigQuery against the public HTTP Archive dataset was unexpectedly charged $14,000 in two hours, prompting broader scrutiny of how cloud data services expose (or obscure) costs. Commenters explain that the dataset is free but queries are billed per terabyte scanned, and that a poorly optimized, repeated query likely read petabytes of data, yet many argue Google’s UI, lack of hard spending caps, and vague cost indicators make such mistakes too easy. Others counter that BigQuery’s pricing and warnings are documented and that users share responsibility to set quotas and understand billing, using this incident to highlight the wider risk of opaque, usage-based cloud pricing for individuals and small startups.

Incident overview

  • A user ran historical queries against the HTTP Archive public dataset in BigQuery and incurred ~$14k in charges in ~2 hours.
  • The workload appears to have repeatedly full‑scanned large tables (on the order of multiple petabytes scanned total), likely via a loop over many months/sites.
  • Initial support reportedly refused to waive the bill; later in the thread, the user says Google began helping resolve it.

How BigQuery pricing works (per thread)

  • Storage for public datasets is covered by Google; users pay for query processing.
  • On‑demand BigQuery pricing is per TiB of data scanned, with a small free tier.
  • The UI shows estimated bytes processed before running a query, but this is easy to miss and displayed in TB/PB, not dollars.

Responsibility and blame

  • One camp argues this is primarily user error: BigQuery docs and HTTP Archive’s “getting started” guide mention per‑byte billing and free‑tier limits.
  • Another camp argues that while the user made mistakes (e.g., SELECT *, LIKE filters, no limits, looped queries), the system design makes expensive errors too easy and insufficiently warned.

UI/UX and cost transparency

  • Multiple comments criticize BigQuery’s “TBs scanned” abstraction and small, non‑prominent cost indicators as user‑hostile or “dark pattern”–adjacent.
  • Several suggest the UI should explicitly show a dollar estimate and raise a blocking warning for very expensive queries (e.g., “this may cost $14k – confirm?”).

Quotas, caps, and safeguards

  • BigQuery supports custom quotas (e.g., max TB scanned per query/user) and project‑level limits, but they’re nonobvious and not hard spend caps.
  • GCP billing alerts and budget notifications exist but are delayed and cannot guarantee a strict dollar ceiling.
  • Many commenters call for true hard budget caps or prepaid models, especially for individuals, students, and hobbyists.

Advice and alternatives

  • Practical tips: avoid SELECT * on huge columnar tables, use partitions/clusters, pre‑filter into smaller tables, test on samples, and set conservative quotas.
  • Some recommend not using personal cards on major clouds, or using LLCs, budget‑capped providers, or self‑hosted / open‑source alternatives instead.