Serving 200M requests per day with a CGI-bin

Original Article ↗ Hacker News Discussion ↗

Performance and scalability

  • Many argue there’s no performance reason to use classic CGI today: a single Go or similar server can easily exceed 10k rps without per-request process creation.
  • Others note that for most sites, even a few hundred–thousand rps is enough; 200M/day is feasible with modern hardware even with CGI overhead.
  • Process startup cost is mostly exec, not fork, and on Unix-like systems is now relatively cheap; Windows process creation is reported as dramatically slower.
  • CGI breaks down once you need DB pools, in‑process caches, or batching: connection setup per request is costly, and you lose benefits of warm state.
  • For low-traffic, internal, or batch-like endpoints (seconds per request, or daily/weekly calls), CGI latency isn’t the bottleneck.

Security considerations

  • CGI’s bad reputation is traced mainly to input-validation bugs and misconfigurations (e.g., scripts writable by users, execution from upload dirs), not the protocol itself.
  • Shellshock is cited as a CGI-enabled vulnerability (when bash parsed environment variables), but only for bash-based handlers; not inherent to Python or C CGI.
  • Embedded devices commonly run web servers and CGI as root (BusyBox, routers), which is called out as a continuing anti-pattern.
  • Some stress that process isolation (no cross-request state) is a security advantage compared to long-lived app servers.

Historical context and “serverless is CGI”

  • People recount moving from CGI to mod_php/mod_perl and FastCGI primarily for efficiency and easier deployment while keeping script-like development.
  • PHP’s success is attributed to “drop a file over FTP and it runs fast,” compared to more complex mod_perl setups.
  • Modern FaaS (Lambda, OpenFaaS) is frequently described as CGI-like: per-request sandboxes or processes with a thin HTTP-ish contract.

Python’s cgi removal and language ecosystem debate

  • A long subthread criticizes Python removing the cgi (and related) modules as “boring, unmaintained tech,” seeing it as part of a broader erosion of backward compatibility.
  • Others respond that very few users rely on stdlib CGI, and moving it to external packages (e.g., legacy-cgi) is reasonable to reduce maintenance burden.
  • This branches into comparisons with JS/Node, Lua, Go, PHP, and Perl for prototyping, packaging, async IO, and stability, with no clear consensus.

Why some still like CGI-style setups

  • Advantages cited: extreme simplicity, polyglot support (different languages per path), easy sandboxing via processes, and good fit for “quick and dirty” internal tools.
  • Tools like uwsgi’s CGI plugin and Apache’s .htaccess are used to retain this model, trading some performance for easier deployment and local configurability.