PostgreSQL and the OOM killer: Why we use strict memory overcommit

Original Article ↗ Hacker News Discussion ↗

Linux’s memory overcommit and OOM-killer behavior are being re‑examined in light of PostgreSQL’s ability to handle allocation failures cleanly, with some operators favoring strict overcommit to avoid random process deaths on database hosts. Others warn that disabling or tightening overcommit can cause premature allocation failures, crashes in applications that don’t handle ENOMEM, and wasted RAM or disk, especially on desktops and mixed workloads. Comparisons with Windows, macOS, containers, cgroups, and newer kernel features like PSI and MGLRU highlight that there is no universally safe setting, only trade‑offs that depend heavily on workload and tooling.

Linux overcommit modes & PostgreSQL

  • Several commenters stress that strict overcommit (vm.overcommit_memory=2) can break fork() and application restarts, especially if overcommit ratios are already tuned.
  • Recommendation: test strictly in QA/perf, load test, and roll in via deployment scripts before baking it into sysctl.
  • Some prefer heuristic mode (0) with very low overcommit_ratio and heavy use of oom_score_adj (e.g., -1000 for critical services) plus kernel reserve tuning.
  • For PostgreSQL specifically, strict overcommit can be attractive because it handles ENOMEM cleanly and rolls back transactions instead of crashing.

Shared hosts: Go / managed languages vs Postgres

  • Real-world issue: Go backend and Postgres on the same host.
  • Heuristic overcommit caused Postgres allocation failures; strict overcommit made the entire system unstable due to large virtual reservations from Go.
  • Workarounds: empirically tuned overcommit_ratio, using GOMEMLIMIT to cap Go heap, or isolating DB and app on separate nodes or via containers/cgroups.

OOM killer vs allocation failures

  • Strong disagreement over whether disabling overcommit is wise.
  • Some users disable overcommit and prefer short freezes where they manually close apps, rather than having the kernel kill “random” or critical processes.
  • Others argue this just trades OOM kills for crashes on ENOMEM, since most desktop software does not handle allocation failures and may corrupt state.
  • Kernel OOM killer behavior on desktops is widely criticized as slow, unpredictable, and user-hostile.

Windows/macOS and swap / compression

  • Multiple comments contrast Linux with Windows: Windows tracks commit charge, never overcommits, and generally handles low-memory situations more gracefully.
  • Some say Windows effectively has no “OOM killer,” instead rejecting unreasonable allocations; others note it can still kill or prompt to close the offending app.
  • Debate over disabling the Windows page file: some do it to “disable overcommit,” others call this almost always a bad idea and emphasize the page file as safety.
  • On Linux, commenters advocate for swap, zram/zswap, MGLRU, and user-space OOM daemons as saner defaults, especially compared to hyperscaler-style “no swap” setups.

Broader design / language angles

  • Some argue strict overcommit on the desktop is a footgun due to widespread neglect of malloc error handling.
  • Others highlight that robust systems (especially databases) should implement their own memory accounting and OOM strategies.
  • A side thread suggests Rust’s richer types (enums/Result) would have prevented the specific C integer/boolean bug discussed in the article.