Protobuffers Are Wrong (2018)

Article reception and tone

• Many commenters found the technical criticisms interesting but felt the post’s opening (“written by amateurs”) undermined its credibility and came off as an ad hominem.
• Others argued the critique is grounded in type-theoretic concerns and real frustrations, even if the rhetoric is needlessly hostile.
• Several past discussions were referenced; one long, detailed defense of protobuf’s design from one of its original maintainers was repeatedly cited.

Required vs optional fields, defaults, and type-system issues

• A major fault line is protobuf’s treatment of field presence:
  • Frontend/TypeScript users complained that generated types mark almost everything as optional, forcing custom validation and making clients fragile.
  • Critics want “required” fields to express invariants, avoid endless null/empty checks, and make invalid states unrepresentable.
• Defenders say “required” was deliberately removed because it breaks schema evolution in large distributed systems: once something is required and deployed widely, adding/removing it safely is extremely hard.
• Proto3’s “zero == unset” semantics and default values are widely disliked; they can hide bugs where missing data looks valid. Others like defaults because they avoid pervasive presence checks.

Backwards/forwards compatibility and schema evolution

• Supporters emphasize protobuf’s core value: you can add fields and roll out servers/clients in any order, unknown fields are preserved in transit, and huge codebases (search, mail, MapReduce, games, Chrome sync) rely on this.
• Skeptics argue that in practice you still need explicit versioning and migration logic, and many teams re-implement their own back-compat layers on top.
• Long subthreads debate whether version numbers plus explicit upgrade paths are better than “everything optional,” and whether more expressive schema languages (e.g., asymmetric fields in Typical, ASN.1 features) achieve safer evolution.

Protobuf as IDL vs domain model

• Several commenters say protobuf works fine as a wire format/IDL but is a poor core data model; pushing generated types deep into business logic causes pain and extra mapping layers.
• Others explicitly want a language-agnostic IDL as the primary type system to avoid N+1 parallel models.

Tooling, ergonomics, and language experiences

• Complaints:
  • Generated Go types are pointer-heavy, non-copyable, and awkward; some teams generate separate “plain” structs and converters.
  • Older or third‑party TypeScript generators were poor; newer tools (e.g., connect-es) have improved things.
  • Enum keys not allowed in maps, limitations on repeated oneof/maps, and odd composition rules frustrate users, though some of these can be worked around by wrapping in messages.
• Fans argue that despite warts, protoc, linters, and multi-language support remove huge amounts of hand-written serialization code, especially in C/C++ and embedded contexts.

Alternatives and broader trade-offs

• Alternatives mentioned: JSON(+gzip), MessagePack, CBOR(+CDDL), ASN.1, Thrift, Cap’n Proto, FlatBuffers, SBE, Avro, Typical, Arrow, custom TLV.
• No clear “drop‑in better protobuf” emerged:
  • JSON/HTTP is praised for simplicity, debuggability, and good enough performance for many APIs.
  • CBOR and MessagePack get positive mentions, especially where schemas are external or optional.
  • ASN.1 sparks a deep argument: some say it’s powerful and protobuf reinvented a worse wheel; others cite complexity, culture, and tooling gaps.
• Several commenters conclude “everything sucks, protobuf just sucks in a widely supported way,” aligning with a “worse is better” view: it’s imperfect but practical, especially for large, evolving, multi-language systems.