OpenAI o3-pro

Original Article ↗ Hacker News Discussion ↗

Model Proliferation & Naming Confusion

  • Many find the growing set of models (4o, 4.1, 4.5, o3, o3-pro, o4-mini, o4-mini-high, etc.) overwhelming and poorly described, especially in the app.
  • Strong criticism of the naming scheme: “4” vs “3” vs “o3/o4” is seen as actively confusing; some suspect this stems from delayed/failed attempts at a “GPT-5”.
  • Users report OpenAI has publicly acknowledged the naming mess and plans to fix it, but not soon.
  • Suggested alternatives: simple tiered names (e.g., Gen 4 Lite/Pro/Ultra), or even human-style personas with dates, plus long-lived aliases for backward compatibility.
  • Some argue confusing names can obscure value and upsell pricier models.

Access Tiers, Usage Patterns & UX

  • Free users mostly just get 4o and don’t choose; Plus users see too many options; Pro/Teams tiers introduce o3-pro.
  • Several commenters suspect only a small fraction of users ever switch models; power users do switch and have specific workflows (e.g., o4-mini for speed, o3/o3-pro for “gnarly” reasoning, 4.1 for code-interpreter tasks, 4.5 for conversation).
  • Some report flaky UIs (timeouts with o3-pro) and frustrations with other vendors’ frontends and rate limits.

What o3-pro Actually Is

  • Confusion over whether o3-pro is just o3 with maximum “reasoning tokens”.
  • Evidence from docs and staff comments: o3-pro is a distinct product/implementation, not just a parameter change, though marketing copy also emphasizes “more compute to think harder”.
  • o3-pro is much slower and uses a separate Responses API endpoint; o3 already supports high reasoning effort via the regular API.
  • o3-pro is confirmed not to be the same as the earlier o3-preview; some speculation about o3 quantization is pushed back on.

Benchmarks, Quality, and Hallucinations

  • Benchmarks show only modest gains vs o3, prompting debate: incremental “Pro” upgrade vs hitting the top of a sigmoid.
  • Some say benchmarks (MMLU, etc.) badly understate real-world gains; they report qualitatively better code and problem-solving with newer models.
  • Others feel hallucination remains a core unsolved issue and care more about reliability, speed, and domain “taste” than raw benchmark scores.
  • Mixed views on hallucination rates: some claim o3 rarely hallucinates, others strongly disagree and still verify everything.
  • ARC-AGI benchmarks spark long debate: are they good proxies for “intelligence” or overly esoteric puzzles? Humans do well but not perfectly; models still perform poorly on ARC-AGI-2.

Practical Capabilities & Tooling vs Models

  • Several users describe significant real improvements in agentic/vibe coding and complex integration tasks, saying they can now build software they couldn’t before.
  • Counterargument: much of the improvement comes from better tools (Cursor, Claude Code, CLI agents, etc.), not just models; others reply that older models with today’s tools still perform noticeably worse.
  • Desired “killer use case”: robust porting of complex software (e.g., C → Java) or large-scale legacy modernization; current models still struggle on such end-to-end tasks.

Pricing, Value, and Long-Term Outlook

  • Some won’t pay $200/month Pro, using Plus or just API pay-as-you-go instead; others see frontier reasoning as “worth it” for hard problems.
  • One thread worries LLMs may not be the final path to AI and fears another “AI winter” when costs are tallied; others argue that even freezing capabilities at GPT-4-era levels would still be world-changing.
  • Brief concern about AI concentration into a few opaque companies, countered by observations that many labs and open-weight models are advancing quickly.

Miscellaneous Notes

  • The “pelican riding a bicycle” SVG prompt remains a playful de facto visual benchmark; o3-pro’s output is seen as slow and amusing but not obviously superior.
  • Some users want better “utility” image generation (calendars, diagrams) and feel the system should transparently chain reasoning + code/SVG without requiring technical prompts.
  • A few experimenters test models with private, hard algorithmic questions; they avoid sharing details to prevent these from entering training data.