S3 is showing its age
Amazon S3’s object storage model is praised for its simplicity, durability and long-term stability, but many engineers argue it now lacks capabilities that newer competitors offer, such as append operations, stronger compare‑and‑swap semantics, and first-class multi‑region buckets. Commenters weigh the trade-offs between keeping S3 “boring” and reliable versus layering complexity in clients or DynamoDB, while also criticizing high egress pricing and comparing S3 to alternatives like Azure Blob Storage and Cloudflare R2. Despite feature gaps and costs, many still see S3’s maturity and track record as a key advantage for long-lived data.
“Age” vs. Features and Market Position
- Some argue the title is misleading: S3’s issues are missing features, not age.
- Others tie the slow pace of new features to its maturity and market dominance: as the “top dog,” it doesn’t need to chase niche use cases.
- Several commenters say S3’s long history and stability are its biggest strengths, especially for data that must last a decade or more.
Design Tradeoffs & Missing Capabilities
- S3 is optimized for immutable objects: you can’t modify parts of an object; it’s full replace-or-nothing.
- Lack of append and in-place partial writes is widely lamented; append would simplify many logging and backup workflows.
- Compare-and-swap (CAS) / put-if-absent semantics and If-Match/If-None-Match-style preconditions are viewed as a major missing feature, despite S3’s strong per-object atomicity.
- Some note limited CAS-like behavior on CopyObject, but not full general CAS.
- There’s no true multi-region bucket; users must stitch together replication and multi-region access points.
Workarounds & Higher-Level Patterns
- Common pattern: use DynamoDB as a metadata/coordination layer over S3 for CAS, locking, two-phase commit, “posix-like” filesystems, and big-data table formats.
- This adds complexity and cost, but enables transactional-ish behaviors and safe deduplication and garbage collection.
- Techniques like storing logs as series of immutable objects in a prefix, multipart uploads, and object-lambda processing are cited as partial substitutes for missing append/CAS features.
Cost, Egress, and Alternatives
- Biggest practical complaint is bandwidth/egress pricing, called “highway robbery.”
- Cloudflare R2 (S3-compatible, no egress fees) is frequently mentioned as an attractive alternative, though some distrust the long-term sustainability of “zero egress.”
- Self-hosted S3-compatible systems (MinIO, SeaweedFS) are appreciated for control and avoiding cloud lock-in.
Simplicity vs. Complexity
- Many defend S3’s minimalist feature set: keeping it “boring” and stable is a virtue; new features risk complexity and bugs.
- Others counter that CAS and first-class multi-region support are “boring essentials,” not scope creep.
- API/auth (AWS SigV4, IAM) are seen as the real complexity; the core object model is simple, but the SDKs and permissions are not.