The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

Original Article ↗ Hacker News Discussion ↗

Universal binaries and Cosmopolitan / musl ideas

  • Thread centers on whether you can get a “works everywhere” Linux (or multi-OS) executable.
  • Cosmopolitan / APE is cited as going beyond Linux to Mac/Windows/*BSD/BIOS/ARM, but practical issues arise: binfmt_misc setup, Windows Defender, and OpenBSD version quirks. Some see it as technically impressive but not worth the operational friction versus per-OS binaries.
  • musl + static linking is described as very effective for CLI tools and some systems languages (e.g. Nim), but the moment you need graphics, GPU, or complex system libraries the complexity explodes.

Static vs dynamic linking trade-offs

  • Advocates of static linking praise:
    • Portability (“just take the binary and run it”).
    • Dead-code elimination and potentially smaller/faster programs.
    • Avoidance of “DLL hell” and distro fragmentation.
  • Critics point out:
    • Security/update issues: many separate static copies of vulnerable code; vendors slow to rebuild.
    • Practical need to update shared components (OpenSSL, etc.) without rebuilding everything.
    • Historical evidence of dynamic linking enabling long-lived binaries when dependencies are managed carefully.
  • Several argue the real villain is unstable ABIs (especially glibc and many Linux userland libraries), contrasting this with Windows’ relatively stable Win32/WinAPI.

Packaging tools and “one file” solutions

  • Tools mentioned: AppImage (heavily discussed), Flatpak, Snap, nix-bundle, guix pack, Docker-based approaches, Magic Ermine, statifier, Exodus, shappimage, various custom “bundle .so and adjust rpath/LD_LIBRARY_PATH” scripts.
  • Consensus: these tools mostly bundle a mini-root and prefer their own libraries, but:
    • Still depend on libc/kernel compatibility.
    • Often must exclude GPU/graphics libs and other system-specific components.
    • Can be large, occasionally slower to start, and still require careful building on an “old-enough” base system.
  • Some note license and technical issues with bundling proprietary GPU drivers (especially Nvidia), undermining the “truly self-contained” dream.

dlopen, graphics, and libc mixing

  • The original musl+dlopen trick is seen as clever but “asking for trouble”:
    • Mixing a statically linked musl binary with dlopened glibc-based system libraries risks allocator, TLS, and syscall mismatches.
    • Workarounds (e.g. forcing musl’s allocator to avoid brk, careful TLS switching) are possible but fragile and corner-case–sensitive.
    • Graphics stacks (OpenGL, Vulkan, ICD loaders) are highlighted as especially dlopen-heavy and hard to make truly portable.

Who benefits, and how big is the problem?

  • Some claim binary compatibility pain mainly hits proprietary software and commercial games; FOSS inside a distro is mostly fine due to source builds and coherent repos.
  • Others counter with experiences maintaining software across old distros, niche distros (e.g. Nix), and “long-tail” open source projects not packaged by distributions—there binary compatibility and packaging become a major burden.
  • A recurring theme: containers and runtimes (Steam runtimes, Docker images, Wine) are emerging as the most reliable practical answer for compatibility, at the cost of yet another layer of indirection.