Show HN: Comma Separated Values (CSV) to Unicode Separated Values (USV)

Original Article ↗ Hacker News Discussion ↗

A new “Unicode Separated Values” (USV) format is proposed as a replacement for CSV, using visible Unicode symbols for traditional ASCII control separators (unit, record, group, file) to reduce escaping and add hierarchy. Commenters find the idea technically interesting but question its practicality: the symbols are hard to read at normal sizes, editors and tools like Excel don’t support it, and using the pictographic versions of control characters is seen as conceptually wrong and needlessly complex. Most conclude that, without strong tooling and adoption, USV is unlikely to displace CSV or simpler alternatives such as TSV, JSON, or ASCII-based separator schemes.

Overview of USV Proposal

  • USV (Unicode Separated Values) is a CSV-like text format using Unicode “control picture” symbols as separators:
    • Unit, record, group, and file separators (␟ ␞ ␝ ␜).
    • Escape and other modifiers (␛, ␗, ␖).
  • Intended benefits: fewer escapes than CSV, explicit multi-level hierarchy (unit/record/group/file), and well-defined syntax with an IETF draft and proposed MIME type.

*Comparison with CSV and Existing SV Formats

  • Many see CSV as “good enough,” with huge ecosystem support (databases, Excel, Unix tools, libraries).
  • Complaints about CSV:
    • Inconsistent “standard” and escaping rules.
    • Locale issues (comma vs semicolon separators, decimal commas).
    • Newlines allowed inside fields complicate parsing.
  • Counterpoint: these are mostly tooling/abuse issues; CSV’s human-editability and ubiquity outweigh flaws.
  • Several note ASV (ASCII separators 0x1C–0x1F) already exist and solve many escaping issues but lack editor support.

Unicode Control Pictures vs ASCII Control Characters

  • USV uses the visible glyphs (U+241C–241F), not the actual control codes (U+001C–001F).
  • Proponents: editors and copy/paste work more reliably with visible, letter-width symbols.
  • Critics:
    • These symbols are meant to represent control chars, not be them; this blurs semantics.
    • Multi-byte delimiters require decoding before splitting.
    • Symbols are tiny and hard to visually distinguish; many users would need custom fonts or editor tweaks anyway.
    • Some find the visual equivalence to rendered ASV a “show-stopping” design flaw.

Readability, Editing, and Accessibility

  • Disagreement whether USV is truly human-friendly:
    • Some report mainstream editors (Emacs, etc.) render examples fine; others need to zoom heavily.
    • Optional newlines in the spec initially made long single-line files, breaking line-based tools; the author later updated the converter to insert newlines.
  • Accessibility:
    • Claim that screen readers handle USV well; others test readers that do not announce the symbols.

Escaping, SYN, and Format Complexity

  • There is an escape mechanism; critics argue this undermines “no escaping needed” claims and reproduces CSV-like complexity.
  • Concerns about:
    • Class-dependent escaping rules.
    • Extra modifiers like SYN heartbeat and “ignore next char,” which create multiple byte-level encodings for the same logical document and complicate checksums and parsers.
    • After feedback, the author indicates SYN will be removed and docs updated.

Adoption, Standardization, and Practicality

  • Many doubt USV will gain traction without support from Excel/LibreOffice and major libraries.
  • Some see it as “another standard” per the famous XKCD comic; suggestion is to instead:
    • Use well-specified CSV (RFC 4180), TSV, JSON/JSONLines, or typed binary/tabular formats (Parquet, Avro, SQLite).
  • Supporters argue editors and libraries could evolve, but critics see no compelling upside over existing text or binary formats.

Alternatives and Related Ideas

  • Mentioned alternatives: ASV, RSV (using invalid UTF‑8 bytes as delimiters), SML, JSONLines, Postgres-style COPY TSV, emoji-separated values, and various custom schemes.
  • Several argue that improving support for ASCII separators and better table modes in editors would be a cleaner path than introducing USV.