Decisions API is in public beta

Original Article ↗ Hacker News Discussion ↗

OpenAI’s new Decisions API, a fast, low-cost classifier built on the gpt‑6‑luna model, is seen as a direct response to Jev and similar “decision models” that provide calibrated probabilities instead of free-form text. Commenters weigh its higher price and mixed early benchmarks against Jev and open-source alternatives, but note advantages like image input, long context windows, and easier procurement for enterprises already using OpenAI. The broader theme is whether such decision models accelerate AI’s commoditization — pushing providers into price wars and lock‑in strategies — or become a foundational utility for routing, safety checks, and real-time automation in larger AI systems.

Product & API Design

  • New /v1/decisions endpoint wraps the gpt‑6‑luna model to answer structured “questions” about an input, returning probabilities instead of text.
  • Supports predicate (yes/no) and choice questions, parallelization across many questions, 1M-token input, and multimodal (image) input.
  • Output tokens are free; cost is $0.10/M input tokens. No KV caching; likely optimized for lowest latency.

Comparisons to Jev and Other Decision Models

  • Direct comparisons dominate the thread. Jev: ~$0.042/M input tokens, so ~2–3× cheaper than Decisions; both have free output.
  • Some users’ evals: Jev faster and more accurate for their tagging/UI-routing tasks; Decisions slower and more expensive, with occasional failed calls.
  • Others report Decisions performing well on similar prompts, suggesting prompt format and API differences matter.
  • Decisions’ main differentiators vs Jev: image input, long context window, and availability via an existing OpenAI enterprise stack and clouds (e.g., Bedrock/Azure support for OpenAI but not Jev).

Calibration, Probabilities & Technical Behavior

  • Debate over whether classic LLM prompts already suffice; counterargument is that decision models expose calibrated probabilities that can’t be obtained from standard LLM calls.
  • Some experiments show:
    • Predicate-style questions match expected probabilities well.
    • Choice-style questions show bias (overconfident major class, sensitive to option order).
  • Multiple questions are independent; probabilities are not required to sum to 1.
  • Confidence vs per-class probabilities raises questions; one use is to reject low-confidence answers.

Use Cases Discussed

  • Content moderation, support routing, tagging, UI/component selection, PKM helpers.
  • Real-time or near-real-time tasks: video games, CCTV/train platform monitoring, shell command safety, QC for dents/defects, notification triage (“urgent” WhatsApp messages).
  • Some see it primarily as a subcomponent for larger agent systems (“cerebellum” to a larger LLM’s “brain”).

Market, Strategy & Skepticism

  • Many see decision models as evidence that AI infra is becoming a commodity; price wars and fast-following are expected.
  • Arguments for using OpenAI despite price: existing vendor approvals, compliance/ZDR, data-retention contracts, procurement friction, and platform stickiness.
  • Others criticize this release as a rushed response to Jev, underbaked, and unlikely to provide a lasting moat; open-source and local options are highlighted as viable alternatives.