Are we at peak vector database?

Original Article ↗ Hacker News Discussion ↗

Amid a boom in tools for storing and searching AI-generated embeddings, many engineers question whether specialized “vector databases” are overkill for most real-world use, where data volumes are modest and brute-force search in RAM or PostgreSQL extensions like pgvector work well. Commenters argue that cosine-similarity search is just one feature that traditional databases will likely absorb, and that the harder, more valuable problems lie in model fine‑tuning, hybrid lexical–vector search, and operational concerns like re‑embedding and scaling. There is broad skepticism about the long‑term prospects of numerous vector DB startups, alongside agreement that vector search itself remains important but not yet a fully “solved” technology.

Scope and Need for Vector Databases

  • Many argue most current RAG and “copilot” use cases are small enough that brute-force search over in-memory vectors (NumPy, PyTorch, Pandas, Parquet+FAISS) is sufficient up to ~100k–1M rows, sometimes more.
  • Vector DBs become compelling only at larger scales (millions–billions of vectors), hard latency/QPS requirements, or when data does not fit in RAM.
  • Several note that computing embeddings (especially with LLM-based models) is orders of magnitude more expensive than cosine similarity, so lookup is rarely the primary bottleneck in early stages.
  • Others counter that this underestimates repeated queries, indexing costs, and memory pressure when storing large numbers of vectors.

Postgres, Extensions, and “Feature Not Product” View

  • Strong sentiment that vector search is a “feature” that will be absorbed into mainstream databases (Postgres, MySQL, etc.), as happened with JSON, OLAP, graph, and document stores.
  • pgvector (and similar extensions like Lantern, SQLite VSS) is seen as “good enough” for many production systems up to millions of documents, with recent improvements like multithreaded index builds.
  • Dedicated vector DBs are viewed by some as lacking a durable moat and likely to be displaced as traditional RDBMSes expand vector capabilities and hybrid search.

Algorithmic and Research Limitations

  • ANN / vector indexing is described as still an open research area with unsatisfying tradeoffs; unlike B-trees, current methods are approximate and fragile in high dimensions.
  • Maximum inner product search is highlighted as even harder than standard ANN due to poor exploitable geometry and differing query vs index distributions.
  • Cosine vs Euclidean distance in high dimensions, and the behavior of embeddings as lossy compression / perceptual hashes, are discussed.

Beyond “Cosine Similarity as a Service”

  • Several see “cosine-similarity search as a service” as commoditized or overhyped; the harder and more valuable work is:
    • Fine-tuning embedding models on domain-specific queries.
    • Managing embedding lifecycle: re-embedding when models change, storage, and recomputation.
    • Hybrid retrieval: combining lexical (BM25), metadata filters, and vectors, plus ranking.
    • Preprocessing for RAG: chunking, adding inferred facts or QA pairs, using LLMs to enrich documents before embedding.

Market Hype and Future Directions

  • Opinions differ on whether we are at “peak” vector DB or just in an early gold rush phase.
  • Many expect consolidation: vector capabilities inside LLM platforms and databases, plus a few higher-level “Algolia-for-RAG” services that hide chunking and indexing complexity.
  • On-device / edge vector search (e.g., for private photo search) is noted as an emerging but still underserved area.