How Pinterest scaled
Pinterest’s early architecture for handling roughly 11 million monthly users prompts debate over whether its large fleet of MySQL, Redis, Memcache and Python/Django servers was necessary or over-engineered. Commenters contrast vertical vs. horizontal scaling costs in cloud vs. bare metal, emphasize operational concerns like fault isolation and blast radius, and note how cloud pricing and free credits push startups toward sharding and microservices. The thread also surfaces broader themes: the tradeoff between developer productivity and runtime efficiency, how tech choices have evolved since 2012, and how organizational incentives and engineer turnover can drive complexity more than pure technical need.
Vertical vs. Horizontal Scaling
- Many comments ask why vertical scaling limits aren’t specified; some argue Silicon Valley is biased toward horizontal scaling, partly due to cloud economics and AWS credits.
- Others note on major clouds, CPU/RAM cost is linear across instance sizes, so vertical scaling loses its usual price advantage vs. bare metal.
- Horizontal scaling is favored for failover, blast-radius reduction, and the ability to adjust capacity to load.
- Several engineers stress operational pain of huge single DB servers: schema changes, backups, restores, and migrations become slow and risky.
Cost, Hardware, and Efficiency
- Disagreement over how much hardware is really needed: some say 11M MAU is modest and could run on far fewer servers with modern hardware and careful optimizations.
- Others counter that personalized, write-heavy workloads and fault tolerance justify larger fleets.
- Debate over cloud vs. dedicated: once load is known, some say dedicated servers are almost always cheaper; clouds win for elasticity, not raw cost.
Language/Framework Choices
- Multiple comments criticize Python/Django as slow and expensive to scale, though good for early productivity.
- Others argue modern languages like Go, C#, Kotlin, Rust can be both fast to develop and performant, making the “productivity vs. speed” trade-off less relevant.
- Some note Pinterest later saved large infra costs by moving parts of the stack away from Python.
Relational vs. NoSQL and Data Modeling
- Database-focused commenters dislike removing joins and complex queries, citing loss of normalization and integrity.
- Others note Pinterest stored key objects as JSON blobs in MySQL mainly for reliability, with performance handled via sharding and caching.
- Some advocate modern NoSQL (e.g., DynamoDB-style single-table design) as best practice for this scale; others say it’s overkill for typical CRUD apps and hard to evolve early in a product’s life.
Product Value and User Perception
- Strong split on Pinterest’s value: some see it as spammy, polluting search results; others describe it as an indispensable visual notebook and recommendation tool.
- Several note Kagi users widely block Pinterest, mainly due to its impact on Google Image Search.
Meta: Article Value and Industry Culture
- Some dismiss the article as rehashed 2012 material; others appreciate having the old talk distilled into a readable summary.
- Broader concerns about over-engineering, resume-driven tech choices, high engineer turnover, and incentive structures that reward complexity.