From xz to ibus: more questionable tarballs

Original Article ↗ Hacker News Discussion ↗

A mismatch between an ibus release tarball and its corresponding Git tag has revived concerns about the software supply chain, especially in the wake of the xz backdoor that was hidden only in release archives. Commenters debate why Linux distros and language package registries still rely on opaque, sometimes hand-crafted tarballs instead of verifiable builds from immutable source commits, and how features like autogenerated archives, build sandboxes, and trusted publishing can still be subverted. Many argue for reproducible tarballs tied to specific commit hashes, stronger provenance checks in package managers, and tighter, hermetic build processes to restore traceability from source to binary.

Context: ibus tarball vs git tag

  • A release-candidate tarball of ibus contains code not present in the corresponding tagged commit in git.
  • Change seems benign (extra Unicode range), likely from a local working tree being tar’d with uncommitted changes.
  • Core concern: such mismatches weren’t detected before distros imported the checksummed tarball.

Why tarballs differ from VCS trees

  • Traditional reasons:
    • Tarballs include generated files (e.g., configure scripts, codegen outputs, translations) so users don’t need heavy toolchains.
    • Tarballs may exclude dev-only scripts, CI configs, and IDE files.
  • Some projects intentionally do code generation only for releases and not in git.
  • Tools like .gitattributes (export-ignore, export-subst) can shape tarball contents, but are underused and may themselves be an attack surface.

Security implications & reproducibility

  • xz backdoor showed tarballs can hide code not in git, undermining “open source can be reviewed” arguments.
  • Extra files in tarballs enlarge attack surface, even in sandboxed builds.
  • GitHub auto-generated archives have had checksum instability issues; this undermines treating them as stable, signed artifacts.
  • Reproducible builds work has mostly focused on binaries, less on reproducible tarballs from VCS.

Trust: tarballs vs git repositories

  • One side: tarballs signed by maintainers are the authoritative, immutable source; if you don’t trust those, you can’t trust their git either.
  • Other side: public git repos have history, many eyes, and distributed copies — making tampering and quiet rewrites harder to hide.
  • Debate centers on traceability and detectability, not absolute trust: git’s structure and clones raise red flags more easily than opaque tarballs.
  • Counterpoint: the xz backdoor lived in git for a long time; detection came only once activated via tarball.

Ecosystem examples (npm, PyPI, Cargo, etc.)

  • Many package registries allow published artifacts to differ from linked git repos (e.g., npm, Cargo, PyPI wheels).
  • Some registries expose commit hashes or metadata, but do not enforce reproducible builds from those hashes.
  • Examples cited where PyPI wheels contained secrets or extra files not in source dists.
  • Concerns that “trusted publishing” (e.g., PyPI+GitHub Actions) still hinges heavily on CI hosts and user key security.

Package managers and functional builds (Nix/Guix)

  • Nix/Guix improve input pinning and reproducibility but still fetch tarballs or generated archives; builds can still run arbitrary scripts.
  • Nix avoided the xz tarball backdoor by building from GitHub source archives rather than upstream release tarballs, but mass rebuilds are slow.
  • Guix offers tools to compare built binaries against source (guix challenge) and explores deep bootstrappability.

Proposed mitigations and open questions

  • Generate release tarballs deterministically from specific commits and verify them against local git archive.
  • Pin packages to commit hashes and record a verifiable chain from VCS to tarball to binary.
  • Enforce or at least support reproducible tarballs and builds in registries and distros.
  • Use hermetic / pure-functional build systems with strict sandboxes and minimized tooling.
  • Strengthen or replace signature workflows; disagreement over practicality of PGP and current “web of trust.”
  • Some see current tarball-centric workflows as legacy inertia; others argue they remain practical, archivable, and widely understood.