Vector Databases: A Technical Primer [pdf]

Original Article ↗ Hacker News Discussion ↗

Vector databases are emerging as core infrastructure for semantic search and retrieval-augmented generation, but engineers are still sorting out when they’re worth adopting over simpler options like brute‑force search in SQLite or Postgres with pgvector. Commenters trade resources and benchmarks, debate cosine versus Euclidean distance and various indexing schemes (HNSW, IVF, ANNOY, PQ), and emphasize that embeddings are produced by external ML models rather than the database itself. A recurring theme is that practical needs—data scale, latency, cost, and hybrid lexical–vector search—should drive the choice between “full” vector databases and lighter-weight extensions or libraries.

Course and Additional Resources

  • Thread centers on a PDF primer derived from a recorded class, with an associated (discounted) Udemy video course.
  • Commenters share many complementary resources: academic surveys on vector DBs, general intros to embeddings and ANN search, and vendor docs.

Cosine Similarity vs Euclidean Distance

  • Extended debate on why cosine similarity is commonly used.
  • Points in favor: magnitude often not semantically meaningful; normalization simplifies to dot product; efficiency benefits.
  • Skepticism: magnitude might matter in some tasks; explanations for cosine often feel hand‑wavy; some argue the real goal is inner product maximization and normalization is mostly convenience.
  • Others bring in intuitions from random hyperplanes and locality-sensitive hashing. Consensus: practice favors cosine, but theory tradeoffs are nuanced.

Brute Force Search vs ANN / Vector Indexes

  • Multiple benchmarks suggest brute force over hundreds of thousands to millions of vectors is surprisingly viable, especially when batched and when LLM generation dominates latency.
  • Discussion on when linear scan “breaks”: depends on vector count, dimensionality, RPS, and RAM.
  • ANN structures (HNSW, IVF, Annoy, LSH, etc.) become important at larger scales or higher QPS, though graph-based methods have memory and build-time costs.

Dedicated Vector Databases vs Traditional DBs

  • Many ask when to move from SQLite/Postgres+pgvector to a specialized vector DB (Pinecone, Qdrant, etc.).
  • For O(100k) vectors and low traffic, simple in‑DB or in‑memory solutions are often “good enough.”
  • Some report scaling custom or lightweight engines to tens of millions of vectors; others highlight cost/complexity of hosted vector DBs and suggest alternatives.
  • There’s demand for clearer guidance on tradeoffs (indexing speed, query latency, cost, operations).

Hybrid Search and RAG

  • Most RAG systems today use vector search; some still rely on keyword/text search.
  • Several emphasize hybrid (vector + lexical) search as increasingly standard; traditional DBs adding vector support and vector DBs adding lexical features.

Embeddings, Feature Selection, and Semantics

  • Important clarification: vector DBs only store and search vectors; embeddings are produced outside by ML models.
  • Substantial discussion on “feature selection” and human judgment:
    • One view: modern deep learning plus attention largely automates feature extraction from raw data for common modalities; no explicit manual feature design needed.
    • Counterview: humans still decide representations (e.g., FFT for audio), objectives, and what “similarity” should mean; missing features can produce systematically wrong “similar” results.
  • Note that embeddings need not be purely semantic; they can encode behavior (e.g., recommender systems) or time/context.

Use Cases and Alternatives

  • Vector DBs mainly used for similarity/semantic search, related-item retrieval, and RAG.
  • For specific visual tasks (e.g., hair/skin color recognition), respondents argue specialized ML models (CNNs, face detectors, multimodal models) outperform naive image similarity search; vector DBs are a component, not a substitute.

Embedded / Lightweight Options

  • Multiple embedded or simple options are mentioned: SQLite with custom functions, DuckDB, Chroma, SQLite extensions, small libraries (Faiss, HNSWlib, usearch), and local/embedded modes of larger systems.

Critiques of the Primer

  • Several point out errors in tables (rows swapped, index types inverted).
  • Objections to describing vector DBs as “clustered by meaning” or “optimized for analytics”:
    • Clustering depends entirely on the embedding and task.
    • Vector DBs are framed as search/retrieval systems, more like search engines than analytical warehouses.
  • Some technical nitpicks: e.g., PQ is characterized more as compression than an indexing strategy; guidance on when to use specific index types is contested.