So you want custom allocator support in your C library

Original Article ↗ Hacker News Discussion ↗

Custom memory allocators for C libraries promise better performance, flexibility, and tooling integration, but their design raises tricky questions about safety and practicality. Commenters debate APIs that require callers to pass object sizes, alignment, and context pointers—arguing this can speed up allocation and aid debugging, while others note that real-world C code often doesn’t reliably track these details and could become more fragile. The thread also contrasts minimalist “single allocator function” designs with clearer but more verbose interfaces, and highlights how choices here affect multithreaded scaling, integration with tools like ASAN/valgrind, and support for patterns like arenas and stack allocators.

Security and correctness of allocator APIs

  • Several commenters argue that requiring size on free/realloc could have prevented many bugs, but others say trusting caller-supplied sizes is dangerous and must still be checked.
  • Some note many allocators already know sizes via metadata or arenas, so extra size arguments often just duplicate work unless they’re treated as hints.
  • There is pushback on the article’s claim that callers “always” have the original size; people give realistic counterexamples where they don’t.

Passing size and alignment information

  • Supporters of passing size to free say it can avoid costly arena/bucket lookups and metadata access.
  • Others point out modern allocators can usually recover size cheaply without walks.
  • Multiple comments argue that extended alignment should be part of the interface; some want an explicit alignment parameter per call, others rely on context or separate aligned-alloc APIs.
  • There’s debate on whether alignment is mandatory (for correct free) or can be “just a hint”.

Single-function vs multi-function allocator interfaces

  • One camp likes a single allocate(ptr, size, ...) style API that encodes malloc/realloc/free by argument combinations, citing simplicity and small footprints (e.g. Lua-like designs).
  • Critics say combining operations complicates reasoning, introduces ambiguous edge cases (alloc(NULL, 0)), and makes review and tooling harder; they prefer distinct alloc/free/realloc.

Caller-managed vs library-managed allocation

  • Some advocate APIs where the caller allocates, possibly via two-pass patterns (size-compute then write), resumable operations, or “pass an allocator callback” as in the article.
  • Others respond this only works for simple cases (e.g. snprintf-like), while complex libraries (XML DOMs, etc.) inevitably allocate internally and benefit from pluggable allocators.

Threading and performance

  • Discussion notes global-state allocators scale poorly; thread-local caches plus mechanisms for cross-thread frees (queues, CAS, per-sizeclass locks) are common mitigations.
  • There is interest in how simply such schemes can be implemented while retaining good multicore scaling.

Real-world C constraints and zero-size allocations

  • Several comments stress that many existing C codebases don’t rigorously track sizes, especially with null-terminated strings and flexible arrays.
  • There is debate over the meaning and usefulness of malloc(0) and over in-band signaling with special numeric values like 0 or -1.

Tooling and integration

  • Some emphasize allocator APIs should integrate with tools like ASan and Valgrind; annotations are considered straightforward but essential.