Build a Database in Four Months with Rust and 647 Open-Source Dependencies

Original Article ↗ Hacker News Discussion ↗

A Rust-based observability database built in four months with around 650 open‑source dependencies is prompting questions about modern dependency-heavy software design. Commenters debate whether such deep reliance on third‑party crates improves velocity and code quality or instead creates an unmanageable security and supply‑chain risk, especially in sensitive systems like databases. The thread also touches on alternative models—richer standard libraries, fewer but larger dependencies, or custom implementations—and on the ethics and practicality of building proprietary products atop open‑source ecosystems.

Dependency Count, Security, and Audit Burden

  • Central debate: is 100 direct / 647 total dependencies inherently bad, especially for a database?
  • Critics see this as “points of failure,” increased supply-chain attack surface, and a huge ongoing audit burden.
  • Disagreement on what “auditing” means:
    • Some focus on code review and lines of code.
    • Others focus on vetting and tracking publishers/owners (“trust domains”).
  • Several argue dependency count is a poor metric:
    • Many crates are split for modularity, compilation speed, or derive/proc-macro helpers.
    • Multiple crates often come from a single repo/team.
  • Others still want far fewer dependencies, especially in libraries, and question crates for trivial tasks (e.g., whitespace handling).
  • Databases in particular: some argue “best practice” is near-zero deps; others note compression, crypto, Unicode, HTTP, etc. are reasonable to outsource.
  • The author later publishes Cargo.lock; an audit shows one known advisory in a transitively pulled RSA crate, via an unused MySQL feature.

Rust Ecosystem vs “npm hell”

  • Many compare this to npm/pip “dependency hell,” though others stress Rust’s situation is different:
    • Cargo supports multiple versions of the same crate.
    • Lockfiles plus tools (cargo audit, dependabot, cargo-deny, cargo-supply-chain, etc.) improve visibility and patching.
  • Some blame Rust culture (thin standard library, flat namespace, easy crate publishing) for encouraging many small crates.
  • Others argue the alternative—rewriting everything—invites more bugs, especially in complex domains like crypto and async networking.

Open Source vs Proprietary Product

  • Some readers find it ironic the post praises Rust’s OSS ecosystem yet keeps ScopeDB closed.
  • The author defends this as a standard “open core” / commercial-open-source posture: contribute fixes and shared libraries upstream while keeping the commercial product proprietary.
  • Opinions diverge:
    • Some see this as acceptable and typical of modern stacks (open tooling, proprietary product).
    • Others argue copyleft licenses better protect community value and resent “open source for PR, closed for profit.”

Do Observability Systems Need a Custom Database?

  • Skeptics question why observability data needs a bespoke database rather than an existing DB or observability-focused store.
  • Supporters note:
    • Large vendors often build custom event stores to escape Elasticsearch cost/limitations and data warehouse awkwardness.
    • For a single company, starting with Postgres (or ClickHouse) is suggested, scaling to custom solutions only when needed.

Meta and Expectations

  • Some expected a “build your own DB” tutorial or a deep dive into managing Rust build times with many dependencies.
  • The thread frequently broadens into general debates on software modularity, dependency culture, and modern supply-chain risk.