RX – a new random-access JSON alternative

Original Article ↗ Hacker News Discussion ↗

A new format called RX aims to store “JSON-shaped” data in a compact, random‑access, mostly ASCII encoding, targeting large read‑only manifests and build artifacts where only small subsets are queried. Commenters contrast it with JSON and binary formats like Protobuf, CBOR, and SQLite-backed stores, weighing RX’s advantages in selective access, reduced heap allocations, and operational copy‑paste friendliness against its poor human readability and limited tooling ecosystem. The emerging consensus is that RX is a niche, no‑SQL‑like embedded store rather than a general replacement for JSON, best suited to performance‑sensitive workloads with large, sparsely accessed datasets.

Purpose and Positioning of RX

  • Framed as a “read-only embedded store for JSON-shaped data” / “no‑SQL sqlite,” not a general JSON replacement.
  • Target use case: large, mostly read-only build artifacts or manifests where clients do many tiny, deep lookups.
  • Designed for random access and lazy reads instead of fully parsing large JSON blobs into memory.

Performance and Random Access

  • Main performance win: selective access. You can jump directly to needed values using built-in indexes and length prefixes.
  • Contrast with JSON: even streaming/SAX parsers must scan most bytes and often allocate full object graphs.
  • Benchmarks cited where a large manifest shrinks from ~92MB JSON to ~5.1MB RX and single-key lookups become tens of thousands of times faster with far fewer heap allocations.
  • Float parsing is acknowledged as expensive in textual formats, but RX’s designer notes many target workloads rarely use many floats.

Text vs Binary and Operational Concerns

  • RX is technically binary-safe but ASCII/UTF‑8 oriented so values can be copy‑pasted through logs, dashboards, terminals, and web UIs.
  • Proponents argue binary formats (CBOR, Protobuf, etc.) complicate production debugging, incident response, and constrained environments.
  • Critics counter that RX’s on-disk text is effectively unreadable by humans, losing a major JSON advantage without full binary efficiency.

Format Characteristics

  • Encoded in reverse (right-to-left) to simplify writing: contents first, length/index metadata later.
  • Uses base64 varints, shared schemas for keys, deduplication, and internal indexes for O(1) array and O(log N) object lookups.
  • Claims large size reductions vs raw JSON (e.g., 18× in some production workloads), but often worse than JSON+zstd; author says compression is a side-effect, not primary goal.

Use Cases and Non-Goals

  • Good fit: large read-only datasets, embedded databases on edge/worker nodes, Lua/LuaJIT environments, server-generated embedded data for web clients.
  • Not recommended: small payloads, float-heavy arrays, scenarios where decompression+JSON parse is affordable, or where existing binary standards already fit well.

Tooling, Documentation, and Adoption Concerns

  • Several commenters find docs and examples unclear, especially the serialization spec and CLI features.
  • Many note that JSON’s dominance comes from ubiquitous tooling; new formats face an adoption hurdle even if technically superior.