Ingesting PDFs and why Gemini 2.0 changes everything

Original Article ↗ Hacker News Discussion ↗

Perceived strengths of Gemini 2.0 for PDFs

  • Many commenters report Gemini 2.0 Flash / 1.5 Flash as “good enough” or better than legacy OCR for:
    • Financial PDFs (KYC/due diligence, fintech ingestion, SEC filings).
    • Healthcare lab reports.
    • Mixed text/tables/diagrams where schema is defined (JSON output).
  • Ease-of-use, multi‑modal support, huge context windows, and simple prompts (“OCR this PDF into this JSON schema”) are repeatedly cited as major advantages over prior cloud OCR products.
  • Some see it as a breakthrough for RAG ingestion and semantic chunking: model can both extract and suggest meaning‑preserving chunks.

Accuracy, benchmarks, and limitations

  • Reported table benchmark score (~0.84 vs ~0.90 for a specialist model) is debated:
    • Author and others argue many “errors” are superficial structural differences; numerics are “almost never” wrong.
    • Specialist vendors counter that in production, hallucinated rows, checkbox states, and subtle sentence rewrites still occur, and their customers need near‑deterministic behavior.
  • Several practitioners emphasize that traditional OCR is only ~80–85% accurate anyway, but LLM hallucinations are qualitatively worse: they can rewrite or invent entire phrases.
  • For high‑stakes domains (finance, healthcare, legal), multiple commenters say even “very few” numeric errors are unacceptable; they layer multiple models, validation, or human review.

Bounding boxes, layout, and attribution

  • Strong consensus that Gemini currently struggles with precise bounding boxes and spatial reasoning on digital docs, even if text recognition is good.
  • Workarounds:
    • Use classic OCR/layout engines (Textract, Tesseract, Unstructured, Docling, Chunkr, etc.) for boxes + text, then feed segments to an LLM for understanding.
    • Two‑pass LLM approaches: first extract entities, then ask the model to locate them among OCR’d chunks.
    • Some open/commercial systems offer accurate layout segmentation with rich JSON (Docling, Marker, Chunkr, Reducto, others) and then call VLMs only on complex pieces (tables, formulas, charts).

LLMs vs traditional OCR / specialist services

  • Experiences vary:
    • Some replaced well‑known OCR vendors with Gemini, cutting latency from minutes to seconds and cost by an order of magnitude, accepting ~4–10% residual error.
    • Others found Sonnet, GPT‑4o, or Qwen‑VL outperform Gemini on certain PDFs, especially technical papers and long tables.
    • Specialist document‑AI vendors argue that pure‑LLM pipelines are brittle at scale; they combine VLMs with classic CV models, layout detection, heuristics, and human‑in‑the‑loop to meet strict SLAs.
  • Open‑source options (Tesseract + Tika, Docling, Marker+Surya, Qwen2.5‑VL, Chunkr, edgartools, etc.) are widely discussed as cheaper, local, or more controllable, but usually require more engineering.

Cost, scale, context, and determinism

  • Flash models are praised as extremely cheap per page, especially with batch/Vertex pricing, though some commenters recalc lower “pages per dollar” than in the article.
  • Several note that all major LLM APIs are subsidized; long‑term pricing and vendor lock‑in are concerns.
  • Mixed reports on long‑context reliability:
    • Some users successfully work at 100–200K tokens.
    • Others see degradation beyond ~20–40K, with hallucinations when asking multiple questions over large docs.
  • Non‑determinism (even at temperature 0) is flagged as a real issue, especially for pipelines that depend on reproducible outputs.

RAG, semantic chunking, and workflows

  • A recurring pain point: naive fixed‑size chunking of PDFs hurts RAG recall; users are excited about using Gemini to produce semantically coherent chunks directly for indexing.
  • Suggested patterns:
    • Use Gemini for OCR + semantic chunking + schema‑filled JSON.
    • Store both structured data and raw model outputs; sometimes also embed for vector search.
    • Mix lexical search (BM25) with semantic search to reduce “zero‑result” failures.
  • Ideas like multi‑model cross‑checking (two models + a third arbiter), reasoning‑based re‑queries, and explicit citations/bounding boxes are proposed to mitigate hallucinations.

PDFs, standards, and philosophy

  • Many lament PDF as a “dead‑tree emulation” that discards structure; entire industries now exist just to re‑extract machine‑readable data that began life digitally.
  • Some note that PDF does support logical structure and embedded metadata (Tagged PDF, hybrid PDFs, iXBRL, Factur‑X/ZUGFeRD), but these features are underused by real‑world producers.
  • Several argue that, despite hype, Gemini 2.0 doesn’t “change everything”:
    • It meaningfully expands the feasible set of RAG/ingestion tasks and pressures legacy OCR vendors.
    • But fundamental challenges—hallucinations, attribution, high‑stakes accuracy, and messy real‑world layouts—remain unsolved and still demand careful system design.