Database “sharding” came from Ultima Online? (2009)
Claims that the term “database sharding” originated with the 1997 MMO Ultima Online prompt a broader look at how long data-partitioning techniques have been around and where the word itself came from. Commenters agree that horizontal partitioning long predates games, citing IBM and academic systems like SHARD in the 1980s, but find it plausible that Ultima Online popularized the evocative “shard” terminology in mainstream tech. The thread also highlights how large online games have repeatedly pushed the boundaries of distributed systems and scaling, influencing later web and infrastructure design.
Origin of the term “sharding”
- Consensus that database partitioning long predates Ultima Online; the game is credited mainly for popularizing the name, not the technique.
- Multiple commenters stress the article explicitly says it’s about naming database partitioning “shards” in the game’s lore (a world-crystal split into parallel worlds/servers).
- N-gram and Wikipedia references suggest “database shard/sharding” only appears in print in the 2000s, after Ultima Online (1997), which supports but does not prove the etymology.
- Wikipedia lists two likely sources for the term in databases: Ultima Online and an earlier SHARD system.
Partitioning vs. sharding
- Several comments clarify that sharding is a kind of partitioning, usually:
- Vertical vs. horizontal partitioning as general concepts.
- Sharding ≈ horizontal partitioning, often across multiple instances/machines.
- Others use “shard” and “partition” nearly interchangeably and note even Wikipedia is inconsistent.
- One definition: partitioning can be within a single instance; sharding implies separate instances.
Earlier academic and industry uses
- References to an 1980s “SHARD” system (System for Highly Available Replicated Data) and related papers from 1985–1988.
- Difficulties locating some of these papers; at least one person has a copy but is unsure about copyright.
- Some recall “shards” being used informally with DB2 in the 90s, but this could be influenced by either that system or later gaming usage.
- Several comments emphasize the original question is about word origin in DB context, not whether the technique is new.
MMOs and broader tech impact
- Multiple anecdotes that MMOs drove early large-scale distributed systems, sharding, and clever network tricks.
- Ultima Online cited as pioneering in scaling, social systems, and ethics of online communities.
- Other games and virtual worlds mentioned for similar innovations (e.g., per-user or per-world sharding, BitTorrent-based patching).
Ultima Online culture and legacy
- Many nostalgic stories: unofficial “shards,” player-run worlds, heavy social dynamics, PK vs. “carebear” debates.
- Discussion of ongoing freeshards and modern spiritual successors aiming for single shared universes without sharding.
Debugging and engineering war stories
- Memorable debugging tales: using PC speakers as “caveman debugging,” audio cues tied to code paths, Malloc “Geiger counters,” even AM radios for hardware noise.
- Seen as emblematic of the creativity in early game and systems programming.
Scalability and sharding schemes
- Question raised about indefinitely scalable sharding like S3; no definitive answer, but:
- Examples given of very large sharded deployments (e.g., LiveJournal) using manual user-based shards plus a global lookup.
- Per-customer databases described as a simple but imperfect scaling approach, with “one huge customer” as a typical stress point.
Meta and information-finding
- Frustration expressed that attention-grabbing posts often mix partial truth and speculation; comments are needed to reconstruct nuance.
- Prior HN threads on the same topic are cataloged; a desire is voiced for better automatic surfacing of related discussions.