Calculating the cost of a Google DeepMind paper
Estimating that reproducing a recent Google DeepMind AI paper on rented cloud GPUs could cost around $10 million has raised questions about who can realistically verify cutting‑edge results. Commenters note that Google’s internal costs are far lower due to owned TPUs and spare capacity, but emphasize that opportunity cost, energy use, and researcher salaries still make such work expensive. The thread widens into concerns about reproducibility, how “GPU‑rich” labs can shift expectations for experimental rigor, and parallels to other fields where only well‑funded institutions can afford large‑scale experiments.
Scale of a $10M Experiment
- Many note that $10M is huge at individual or small-company scale but tiny for Alphabet, whose revenue and profit make such losses almost unnoticeable.
- Others argue that the real risk is not a single $10M miss but failing to learn from repeated mistakes, which could signal structural issues.
Internal vs External Compute Costs
- Multiple comments stress the article estimates replication cost at public cloud prices, not Google’s internal cost.
- Google likely used TPUs and internal quota/priority systems, making marginal cost closer to electricity and depreciation than list GPU prices.
- Some argue that if you can buy H100s outright, full ownership can be far cheaper than $3/GPU-hour cloud rates.
Electricity, Infrastructure, and Opportunity Cost
- Disagreement over whether electricity is “negligible”: for hyperscalers it’s small relative to total cost but still in the multi‑million range for heavily utilized clusters.
- Some insist opportunity cost matters: internal cycles displace revenue-generating customer workloads; others counter that utilization is not 100%, so idle capacity makes internal use very cheap.
Best-Effort / Spot Compute and Utilization
- Description of internal “best effort” or low-priority tiers: jobs run on spare capacity and get preempted by higher-priority workloads.
- Others note GPUs/TPUs are harder to preempt efficiently; frequent checkpointing and reloads can make pure “spare cycles” training unrealistic at this scale.
Reproducibility and the “GPU Poor”
- Concern that multi‑million‑dollar experiments are effectively irreproducible for most academics and small labs.
- Some compare this to high-energy physics or space experiments: expensive but still part of science.
- Worry that big labs’ compute-heavy standards crowd out “GPU poor” research and raise reviewer expectations beyond what smaller groups can afford.
Why Big Labs Publish
- Suggested motives: recruiting and retaining researchers, marketing and brand building, impressing investors, and occasionally blocking patents by placing ideas in the public domain.
- Several note that top AI papers function as high-value advertising and career currency, which can distort incentives toward hype and opaque presentation.
Comparisons and Value
- Parallels drawn to other fields: mouse studies and high-throughput drug screens routinely cost hundreds of thousands of dollars or more.
- Some argue similar or greater money is burned across industry on failed projects; research of this scale is not unusual globally.
Critiques of the Cost Estimate
- Technical commenters question assumptions on model-parallelism, hardware choice, utilization (MFU), and pricing, suggesting the blog’s dollar figure could be off by a factor (likely too high).
- Others defend the order of magnitude and emphasize that even with optimization, replication would still be extremely expensive.