Biggest shell programs

Original Article ↗ Hacker News Discussion ↗

Examples of Large Shell Programs

  • Several large, single-file shell tools are cited: cd ripper abcde (5.5k LOC), ACME client acme.sh (7.1k), firewall tool FireHOL (20k), winetricks (22k), and steamtinkerlaunch (~24–27k).
  • OS-level tools: FreeBSD’s update client (3.6k) and poudriere (3× that) are mentioned as significant sh codebases; Arch’s makepkg is noted but relatively small (<5k).
  • Other sizable systems: backup/restore tool ReaR, rkhunter, a complex containerized pipeline manager (~6k), large test suites (e.g., ~10k lines of bash system tests for mod_pagespeed).

Portability, Environments, and Constraints

  • Shell is valued because it’s nearly always present, including on locked-down or air-gapped systems where installing compilers or new runtimes is hard or forbidden.
  • However, portability across shells and utilities is hard; behavior differences (e.g., DEC/Digital Unix truncating output at $COLUMNS) can dominate script size and complexity.
  • POSIX utilities like bc aren’t guaranteed on Linux, pushing some to rely on AWK or BusyBox as a predictable baseline.

Merits and Drawbacks of Large Shell Scripts

  • Pro: Shell (and Perl) excels at glue code, text processing, and messy OS interactions where Java/Python feel verbose. Some argue loss of Perl expertise has led to more manual drudgery.
  • Pro: Shell scripts are “transparent”: logic and checks are visible rather than hidden behind layers of data structures, making tools like rkhunter effective as human-readable documentation.
  • Con: Beyond a few hundred lines, bash is seen as hard to read, poorly tooled, and fragile (subtle scoping, error handling, quoting, and whitespace issues).
  • Con: Many advocate rewriting large scripts in C, Python, Perl, or other languages, or at least factoring out heavy logic into compiled or higher-level components.

Alternatives and Evolution (Perl, AWK, Oils, PowerShell)

  • Perl is repeatedly praised as the pragmatic migration path from large shell “abominations,” with stories of 50k-line shell systems replaced by ~5k lines of Perl and large performance gains.
  • AWK is used as a more pleasant, portable target language (including for interpreters of DSLs) when shell math and data handling become unwieldy.
  • The Oils project (OSH/YSH) is proposed as an evolutionary path from bash:
    • OSH focuses on high bash-compatibility and better diagnostics.
    • YSH adds strict error checking, real arrays/dicts, modules, and modern features while preserving shell-style pipelines.
    • Emphasis on gradual upgrade of existing bash scripts via shopt options.
  • On Windows, PowerShell is recommended over batch for new work, though legacy batch remains common.

Tooling, Debugging, and Safety

  • ShellCheck is widely recommended to catch common shell “footguns.”
  • Bash tracing (-x) and strict modes are used, but many still feel error handling is primitive.
  • For preventing accidental multi-line paste execution in terminals, bracketed paste mode via inputrc/bind is offered as a mitigation; enforcing rate limits or policies at the server level is seen as contentious because it may annoy users with different workflows.

Anecdotes and Attitudes

  • Numerous war stories describe:
    • Massive cross-Unix installers and CA systems mostly devoted to coping with utility differences.
    • Exotic behaviors on legacy Unixes and odd installers embedding archives/binaries inside shell scripts.
    • Personal “hacker spirit” scripts (VPN endpoint optimizer, weather-based roof sprinkler via X10).
  • Opinions diverge sharply:
    • Some love pushing shell to extremes and find it satisfying and fun.
    • Others see large shell programs as a sign that “life choices” should be reconsidered and favor more conventional programming languages.