Chronon, Airbnb's ML feature platform, is now open source

Original Article ↗ Hacker News Discussion ↗

Chronon’s purpose and architecture

  • Chronon is a full rewrite of an earlier feature platform, built to improve scalability of time-travel and reduce serving latency via new algorithms and serialization/fetch strategies.
  • Uses a lambda-style architecture: precomputed intermediate tables for batch/snapshot accuracy plus streaming updates for real-time features.
  • Features are expressed via composable primitives (e.g., GroupBy aggregations, Join enrichments), which can be chained to handle multi-hop feature logic.

Adoption, scale, and operations

  • Offline-only setup for new use cases is described as fast (under a week).
  • Online serving requires more work: teams must validate that their key–value store can handle Chronon’s read/write patterns, typically over a month or more.
  • Platform was motivated by:
    • Inability to back-test real-time features without “log and wait” for months.
    • High cost of building separate batch, streaming, indexing, and serving pipelines for each feature group; Chronon automates this via configuration.

Comparisons to other tools

  • Other feature stores mentioned: Feast, Hopsworks, Feathr, Tecton.
  • Some commenters argue Chronon isn’t a drop-in replacement for Feast; Feast maintainers say it does support some transforms and is still active.
  • Chronon’s authors highlight differences from Tecton (time-travel scalability, composable primitives) and from “pure kappa” systems (need for lambda to fix upstream errors and support GDPR deletions).

Feature store vs OLAP/data warehouse debate

  • One side sees feature stores as largely rebranded mixtures of existing DB technologies (relational, KV, OLAP, etc.).
  • Others argue feature stores add:
    • Point-in-time correct, windowed aggregations for millions of timestamps.
    • Systematic sharing and reuse of partial aggregates.
    • Push-compute-to-write for low-latency serving, rather than expensive read-side OLAP queries.
  • StarRocks, ClickHouse, Rockset, and temporal DBs are cited; some report scalability limits with these for this use case.

Time-travel, mutable data, and caching

  • “Time travel” means reconstructing feature values as of the exact time of past inferences for backtesting.
  • Snapshots and timestamped tables are noted as partial solutions; defenders say they don’t cover the scale and precision Chronon targets.
  • Chronon automatically decides which intermediate results to cache and claims sub-linear scaling of backfills by reusing intermediate state.
  • There is support for mutable OLTP data via an “EntitySource” API, but details for specific patterns (like evolving status fields) are not fully clarified.

LLM and human-in-the-loop use

  • Chronon can provide contextual features to LLM prompts and perform “time-travel evaluations” by replaying historical requests with historical context.
  • A hypothetical refund-bot example sparked concern about delegating refund decisions to LLMs; others emphasize using LLMs mainly for context-gathering and triage, with humans in the loop.

Miscellaneous topics

  • Airflow is the default orchestrator; the project is open to contributions for others (e.g., Cadence).
  • Chronon reportedly powers many ML-guarded or ML-driven UI actions at Airbnb and is also used for some non-ML workloads.
  • Several comments complain about Medium’s UX; alternatives like scribe.rip and the GitHub README are shared.