Pg_vectorize: Vector search and RAG on Postgres

Original Article ↗ Hacker News Discussion ↗

Postgres is increasingly being pushed into the role of a vector database, with projects like pg_vectorize building on pgvector to add high-level APIs for embeddings, vector search, and retrieval-augmented generation (RAG) directly inside the database. Commenters share real-world experiences of pgvector at scale, weigh it against specialized services like Pinecone or Meilisearch, and delve into practical issues such as chunking strategies, recall, performance tuning, and cost control. There is enthusiasm for consolidating search and retrieval in Postgres, but also concern about hiding complex LLM calls in database extensions and skepticism about how reliably RAG improves results compared with simply using larger context windows or fine-tuned models.

Overall reaction to pg_vectorize & Postgres for vectors

  • Many are positive on pushing Postgres as a vector/RAG platform, seeing it as empowering developers vs proprietary “vector DB” products.
  • pg_vectorize is viewed as a convenient high-level wrapper around pgvector, handling embedding generation, index management, and query orchestration.
  • Some prefer using pgvector directly to retain control and avoid hidden complexity.

Costs, hosting, and architecture

  • RAG can be expensive: you pay once to embed the corpus and then per-query for embeddings and LLM tokens.
  • Self-hosted embedding models (and eventually chat models) are highlighted as a way to control cost and keep data private, but add operational complexity.
  • Debate over whether embedding + retrieval should live inside the DB; some like the simpler architecture, others strongly prefer a clean separation and no outbound network calls from Postgres.

pgvector at scale

  • Multiple reports of pgvector working in production, including tens to hundreds of millions of rows.
  • Strengths: open source, transparent indexing, integration with relational data, good observability.
  • Pain points: heavy reliance on vacuum, large/slow HNSW index builds, weak support for filtered ANN, and some API friction (e.g., Python + numpy).

RAG effectiveness and alternatives

  • Mixed experiences: some find naive RAG “bad” or underwhelming; others report excellent results, especially for internal PDFs and large technical corpora.
  • Stronger systems combine RAG with domain structure (e.g., technical support logs, knowledge graphs, boosting by expert signals), achieving much higher recall than “pure vector-only RAG.”
  • Some suggest direct LLM use with large context windows can rival or beat RAG in certain domains, but others argue RAG remains cheaper, faster, and more scalable for large corpora.

Chunking and retrieval strategies

  • Simple fixed-size chunking often underperforms; several comments stress:
    • Hierarchical / semantic chunking (by headings, sections, or context clusters).
    • Using small models to perform semantic chunking and context-aware summaries.
    • “Context cluster” / tree-based approaches and hierarchical summaries (e.g., RAPTOR-like ideas).
    • Careful selection of which chunks to fit within context (knapsack-style optimization).
  • Some projects use sentence-level embeddings followed by clustering and indexing of centroids.

RAG concepts & misconceptions

  • Clarifications that RAG is not just “LLM in front of search,” but LLM generation conditioned on retrieved, vector-ranked context.
  • Several point out that the “R” (retrieval) is a hard problem, and vector similarity is only one piece; hybrid retrieval (keyword, structure, vectors) is often needed.

Comparisons, tools, and alternatives

  • pg_vectorize vs PostgresML: pg_vectorize offloads models to external services and focuses on embeddings/RAG; PostgresML runs models on the DB host and supports broader ML (e.g., supervised training).
  • Alternatives suggested for smaller or different use cases: sqlite with extensions (sqlite-vss), DuckDB, Meilisearch, Elasticsearch/OpenSearch, and dedicated vector DBs.
  • Some argue Postgres is “good enough” and attractive because it’s already in the stack; others prefer specialized search engines for large, complex workloads.