Every System is a Log: Avoiding coordination in distributed applications

Overall reaction & core idea

• Many commenters like the clear articulation of “everything is a log” and its implications for durable execution and coordination.
• Some see it as a practical extension of ideas from transaction logs, “Turning the Database Inside Out,” and classic distributed systems research.
• Others feel it’s essentially re‑packaging long‑known concepts from accounting and early computing.

“One log” scope and coordination

• There’s confusion around “one log”: some initially read it as literally a single global log.
• Clarification: it means one logical log per entity/handler (e.g., per payment, user, session) that unifies state, communication, and locking; these logical logs are multiplexed over partitioned physical logs.
• Critics argue that coordination needs are ultimately dictated by real‑world causal relationships, so independent logs only work when domains are actually independent.

Scalability, CAP, and consistency

• Multiple comments point out that a single, linearizable log trades availability for consistency (CAP).
• The system is assumed to choose linearizability (like Kafka, Pulsar, Redpanda) and to scale via partitioning/sharding, not a global mutex.
• Some suggest replication or CRDT-like approaches; responses note these only work when temporary inconsistencies are acceptable.

Relation to existing paradigms & systems

• Comparisons are drawn to:
  • Temporal (durable workflows), with this approach seen as broader: also used for RPC and long‑lived state.
  • Actor models/virtual actors (e.g., per-key stateful handlers).
  • CSP-like sequencing via shared logs.
  • ATProto/Bluesky’s architecture, and various log/CRDT projects (gossiplog, OrbitDB, Canvas).
  • Internal “log as a service” systems at large cloud providers.

Log structure, compaction, and ordering

• Discussion on compaction: summarizing log segments, keeping per‑key state in an internal DB, partitioning high‑ vs low‑throughput streams.
• Debate over whether log order must be strict:
  • Some propose peer‑to‑peer, append‑only “ledgers” where order is secondary and corrections are new entries.
  • Others argue that without strong ordering you can’t stream correctly or reason about deletes/negations.

Use cases, edge scenarios, and practicality

• Offline/edge scenarios (e.g., a store losing connectivity) spark discussion of CRDT‑like payments, outboxes, and consensus variants.
• Some see logs as a natural fit for financial systems and reconciliation; analogies to general ledgers and journal entries recur.

Licensing and openness

• Restate uses the Business Source License (not OSI‑open); some are wary of vague restrictions on “Application Platform Service” use and request clearer grants.