GPT-4 Turbo with Vision Generally Available

Original Article ↗ Hacker News Discussion ↗

Competition and Release Timing

  • Many see GPT-4 Turbo with Vision GA as a response to competitor launches (Anthropic, Google Gemini), though others argue these features were inevitable.
  • Vision was previously available in beta; this is mainly about general availability and feature parity (JSON mode, function/tool calling).

Capabilities: Vision + JSON / Function Calling

  • Vision model now supports JSON mode and function/tool calling.
  • This enables direct structured data extraction from images without multi-step hacks (e.g., vision → text → second LLM pass).
  • Some report strong results when enforcing JSON or schemas (via tools like Pydantic or grammars), but note that semantic accuracy and hallucinations remain unsolved problems.
  • Layout understanding is widely described as poor: tables, spatial relationships, and bounding boxes are often inaccurate.
  • Some prefer specialized OCR (AWS Textract, etc.) plus a text-only LLM for structuring, citing better reliability and lower cost.

Quality: Math, Coding, and Vision Limits

  • OpenAI staff claim significantly better math and reasoning; several users say coding assistance feels improved over previous GPT-4 Turbo.
  • Others question how important “better math” is for everyday chat use, though some note it helps with geometry, algorithms, and technical tasks.
  • Multiple comments describe vision as good at general descriptions and text reading but bad at counting objects and precise detection, with hallucinated objects in complex scenes.
  • For tasks like object counting, traditional CV models (e.g., YOLO) are seen as better than LLM vision.

API vs ChatGPT, Naming, and Availability

  • Thread clarifies: this announcement targets the API; ChatGPT will get the updated model under the GPT-4 label.
  • Model naming/versioning (GPT-4, Turbo, Vision, preview vs dated versions) is seen as confusing.
  • Playground initially showed incorrect context/output token limits but was later fixed.
  • Azure OpenAI support for the new model is uncertain and expected to lag.

Cost, Access, and Tools

  • Some find API experimentation affordable (a few dollars/month); others see GPT-4 as expensive.
  • Suggestions to reduce cost: startup cloud credits, third-party frontends (e.g., alternative chat tools with GPT-4 access), or mixing cheaper/open models.
  • Developers share tools and demos for image-to-JSON pipelines and browser automation using vision + function calling, but fully robust web automation is still described as tricky.