Show HN: Comma Separated Values (CSV) to Unicode Separated Values (USV)

Original Article ↗ Hacker News Discussion ↗

Overview of USV Proposal

  • USV (Unicode Separated Values) is a CSV-like text format using Unicode “control picture” symbols as separators:
    • Unit, record, group, and file separators (␟ ␞ ␝ ␜).
    • Escape and other modifiers (␛, ␗, ␖).
  • Intended benefits: fewer escapes than CSV, explicit multi-level hierarchy (unit/record/group/file), and well-defined syntax with an IETF draft and proposed MIME type.

*Comparison with CSV and Existing SV Formats

  • Many see CSV as “good enough,” with huge ecosystem support (databases, Excel, Unix tools, libraries).
  • Complaints about CSV:
    • Inconsistent “standard” and escaping rules.
    • Locale issues (comma vs semicolon separators, decimal commas).
    • Newlines allowed inside fields complicate parsing.
  • Counterpoint: these are mostly tooling/abuse issues; CSV’s human-editability and ubiquity outweigh flaws.
  • Several note ASV (ASCII separators 0x1C–0x1F) already exist and solve many escaping issues but lack editor support.

Unicode Control Pictures vs ASCII Control Characters

  • USV uses the visible glyphs (U+241C–241F), not the actual control codes (U+001C–001F).
  • Proponents: editors and copy/paste work more reliably with visible, letter-width symbols.
  • Critics:
    • These symbols are meant to represent control chars, not be them; this blurs semantics.
    • Multi-byte delimiters require decoding before splitting.
    • Symbols are tiny and hard to visually distinguish; many users would need custom fonts or editor tweaks anyway.
    • Some find the visual equivalence to rendered ASV a “show-stopping” design flaw.

Readability, Editing, and Accessibility

  • Disagreement whether USV is truly human-friendly:
    • Some report mainstream editors (Emacs, etc.) render examples fine; others need to zoom heavily.
    • Optional newlines in the spec initially made long single-line files, breaking line-based tools; the author later updated the converter to insert newlines.
  • Accessibility:
    • Claim that screen readers handle USV well; others test readers that do not announce the symbols.

Escaping, SYN, and Format Complexity

  • There is an escape mechanism; critics argue this undermines “no escaping needed” claims and reproduces CSV-like complexity.
  • Concerns about:
    • Class-dependent escaping rules.
    • Extra modifiers like SYN heartbeat and “ignore next char,” which create multiple byte-level encodings for the same logical document and complicate checksums and parsers.
    • After feedback, the author indicates SYN will be removed and docs updated.

Adoption, Standardization, and Practicality

  • Many doubt USV will gain traction without support from Excel/LibreOffice and major libraries.
  • Some see it as “another standard” per the famous XKCD comic; suggestion is to instead:
    • Use well-specified CSV (RFC 4180), TSV, JSON/JSONLines, or typed binary/tabular formats (Parquet, Avro, SQLite).
  • Supporters argue editors and libraries could evolve, but critics see no compelling upside over existing text or binary formats.

Alternatives and Related Ideas

  • Mentioned alternatives: ASV, RSV (using invalid UTF‑8 bytes as delimiters), SML, JSONLines, Postgres-style COPY TSV, emoji-separated values, and various custom schemes.
  • Several argue that improving support for ASCII separators and better table modes in editors would be a cleaner path than introducing USV.