Go Protobuf: The New Opaque API

Original Article ↗ Hacker News Discussion ↗

Reactions to the new Opaque Go Protobuf API

  • Many dislike moving from plain structs to accessor-heavy, “Java-like” APIs; seen as unidiomatic Go, adding boilerplate and mental overhead.
  • Others welcome it as safer and more predictable, especially around optional fields and presence, and report it would have prevented real bugs.
  • Some argue you should convert protobuf messages to domain types at boundaries; others say that creates too much boilerplate and copying.

Generated Code, Ergonomics, and Testing

  • Generated Go protobuf code is widely criticized as bloated, pointer-heavy, and hard to reason about or copy/compare.
  • Several commenters recommend using text-format protos (.textpb) in testdata/ for inputs/outputs, parsed by helpers, instead of gigantic Go literals.
  • Some miss older alternatives (e.g., gogoprotobuf) that generated “pure data” structs and faster code, but were abandoned due to maintenance burden.

Field Presence, Proto2/3, and Editions

  • Proto3’s implicit presence and loss of has_ semantics are widely regarded as a major mistake, later partially and now largely reversed (Edition 2023 uses explicit presence again).
  • The back-and-forth (proto2 → proto3 → editions) is seen as path-dependent and politically driven, with legacy proto2 codebases still hard to migrate.

Performance, Memory Layout, and Lazy Decoding

  • The opaque API is justified as enabling better memory layouts, metadata storage, and potential lazy decoding.
  • Some question whether the complexity and new per-message APIs outweigh the performance gains.
  • There is debate over repeated fields as []*T vs []T; []T would be faster and more compact but conflicts with existing semantics and expectations.

Zero Values, Nullability, and Go’s Design

  • Go’s zero-value philosophy is blamed for many protobuf and API design contortions, especially distinguishing “unset” vs “set to default”.
  • Maps and slices’ nil behavior is called a footgun; defaults and presence semantics keep resurfacing as hard problems.

Protobuf vs Other Formats and Stacks

  • Several report drifting away from gRPC/protobuf toward JSON/REST, JSON-RPC 2.0, MsgPack/CBOR, or GraphQL, citing simpler tooling and ubiquity.
  • Others argue protobuf’s main value is its schema/IDL and evolution rules, not raw speed, and even use it just to define JSON-based APIs.
  • Performance comparisons in the thread suggest protobuf is often 2–10× faster than JSON, but not radically faster than gzipped JSON, and slower than more “proper” binary formats like FlatBuffers/Cap’n Proto in some scenarios.