GPT-4 Turbo with Vision Generally Available

Original Article ↗ Hacker News Discussion ↗

OpenAI’s GPT‑4 Turbo with Vision has moved from beta to general availability, adding JSON mode and function/tool calling so developers can extract structured data directly from images. Commenters compare its capabilities, cost, and reliability to alternatives like Anthropic, Textract, and traditional computer vision tools, noting that GPT‑4 Vision is strong at reading text but still weak at precise layout understanding, counting, and spatial reasoning. The release also raises questions about model naming, rollout timelines (including for future systems like Sora), and how improvements in math and reasoning affect real-world use cases.

Competition and Release Timing

  • Many see GPT-4 Turbo with Vision GA as a response to competitor launches (Anthropic, Google Gemini), though others argue these features were inevitable.
  • Vision was previously available in beta; this is mainly about general availability and feature parity (JSON mode, function/tool calling).

Capabilities: Vision + JSON / Function Calling

  • Vision model now supports JSON mode and function/tool calling.
  • This enables direct structured data extraction from images without multi-step hacks (e.g., vision → text → second LLM pass).
  • Some report strong results when enforcing JSON or schemas (via tools like Pydantic or grammars), but note that semantic accuracy and hallucinations remain unsolved problems.
  • Layout understanding is widely described as poor: tables, spatial relationships, and bounding boxes are often inaccurate.
  • Some prefer specialized OCR (AWS Textract, etc.) plus a text-only LLM for structuring, citing better reliability and lower cost.

Quality: Math, Coding, and Vision Limits

  • OpenAI staff claim significantly better math and reasoning; several users say coding assistance feels improved over previous GPT-4 Turbo.
  • Others question how important “better math” is for everyday chat use, though some note it helps with geometry, algorithms, and technical tasks.
  • Multiple comments describe vision as good at general descriptions and text reading but bad at counting objects and precise detection, with hallucinated objects in complex scenes.
  • For tasks like object counting, traditional CV models (e.g., YOLO) are seen as better than LLM vision.

API vs ChatGPT, Naming, and Availability

  • Thread clarifies: this announcement targets the API; ChatGPT will get the updated model under the GPT-4 label.
  • Model naming/versioning (GPT-4, Turbo, Vision, preview vs dated versions) is seen as confusing.
  • Playground initially showed incorrect context/output token limits but was later fixed.
  • Azure OpenAI support for the new model is uncertain and expected to lag.

Cost, Access, and Tools

  • Some find API experimentation affordable (a few dollars/month); others see GPT-4 as expensive.
  • Suggestions to reduce cost: startup cloud credits, third-party frontends (e.g., alternative chat tools with GPT-4 access), or mixing cheaper/open models.
  • Developers share tools and demos for image-to-JSON pipelines and browser automation using vision + function calling, but fully robust web automation is still described as tricky.