What If OpenDocument Used SQLite?
XML vs. SQLite and document structure
- Several comments argue an “XML database” would be worse: no widely used, embedded XML engine matches SQLite’s reliability, and XML has no native indexing, forcing full scans or ad‑hoc indexes.
- Others note you can order XML elements to allow binary‑search‑style seeking, but this is fragile due to comments, CDATA, and parsing complexity.
- Breaking a document into many small XML fragments inside SQLite (e.g., per slide, sheet, chapter) is attractive for partial loading, but complicates XSLT, pagination, and cross‑fragment layout changes.
Applying SQLite to ODF, text, and spreadsheets
- Some readers want the thought experiment extended to text and spreadsheet files, not just presentations, and question what benefits beyond versioning and memory use would materialize.
- Ideas floated: linked‑list‑like storage for text, per‑sheet or even cell‑level tables for spreadsheets, and experimentation with different granularity levels.
SQLite as application/file format and on the web
- Multiple examples are cited of tools using SQLite as their project or document format; people like easy introspection and direct SQL access.
- SQLite as a downloadable asset (e.g., from S3) is discussed: clients can fetch full files or query them remotely via HTTP Range requests and a WASM‑based VFS. This works best for modest, mostly public datasets.
Performance, SSD wear, and BLOB storage
- Some doubt the practical importance of avoiding full‑file rewrites: SSD endurance is high, and bulk JSON/XML serialization can be extremely fast and sometimes simpler than a tabular mapping.
- SQLite’s 2 GiB BLOB limit is mentioned as a structural constraint; chunking large binaries across multiple BLOBs is the common workaround, also useful for streaming, compression, and encryption.
Security, reliability, and environments
- Using SQLite as an interchange format raises security issues: need for
secure_delete, hardened settings for untrusted files, and awareness that malicious database files (not just SQL strings) can trigger CVEs. - There’s debate over SQLite’s stance that “few real‑world applications” open untrusted DBs, given this proposed use.
- Networked filesystems can corrupt SQLite if locking is imperfect (e.g., some 9p setups), though others report NFS/CIFS working fine when correctly configured.
Collaboration and editing semantics
- Questions arise about how to reconcile SQL‑level updates with user expectations of “unsaved” changes. Options include long transactions, temp copies with atomic replace, or versioned rows marking the current committed state.
- For collaborative or offline‑first use, suggestions include SQLite’s session extension or CRDT‑based approaches.