I built non-autoregressive decision models with RL a year ago

Original Article ↗ Hacker News Discussion ↗

An open-source project called Laya claims to replicate the core idea behind Jev, a heavily marketed proprietary “System 1” decision model used for fast, structured classification tasks that don’t need full generative LLMs. Commenters debate how similar Laya really is to Jev, noting major differences in model size, context window, zero‑shot capability, and the need for fine‑tuning, while broadly agreeing that non‑autoregressive, BERT‑style classifiers are not a new research breakthrough. The exchange widens into a critique of AI hype, the importance of branding and distribution over raw research, and a renewed interest in using smaller, specialized models instead of large, expensive LLMs for routine decision tasks.

Relationship between Laya, Jev, and prior work

  • Laya is presented as an open-source, non‑autoregressive “decision model” inspired by the same general idea as Jev: structured, typed outputs and calibrated probabilities for classification/routing.
  • Several commenters note that similar ideas existed before both, e.g., BERT-style classifiers, GLiNER/GLiNER2, tabular/“universal” classifiers, and MMLU-style multiple-choice scoring via logits.
  • Debate over whether Jev “copied” anything: some see likely independent reinvention on top of long-standing prior art; others think Jev should acknowledge more prior work, including open research like Laya’s precursor paper.

Technical characteristics and limitations

  • Laya uses ModernBERT-style backbones (~400M params) with relatively small context windows (512–1024 tokens for checkpoints, though base ModernBERT supports 8k). Jev is advertised as ~32k–64k context.
  • Laya’s paper and HF card emphasize reinforcement learning with proper scoring-rule rewards to produce calibrated probabilities.
  • Laya currently needs fine‑tuning for good performance on the “typed-decisions” benchmark; zero‑shot scores are near random (~0.35) vs ~0.77 after fine‑tuning.
  • Jev and similar models are non‑autoregressive, parallel classifiers, sometimes contrasted with diffusion-style approaches; some open “openjev” implementations emulate the general trick.

Zero-shot vs fine-tuned and quality comparisons

  • Multiple commenters highlight a key distinction: Jev aims to be a strong zero‑shot generalist classifier; Laya is better seen as a fast base to specialize via fine‑tuning.
  • Some users report Jev substantially outperforming LLMs and Laya on real tasks (e.g., inventory categorization, support ticket classification), with large cost and latency gains.
  • At least one person reports Jev ~95% vs Laya ~48% accuracy on a labeled support dataset.
  • Others test Laya on simple probabilistic tasks (coin flips, die rolls) and find its probabilities nonsensical, raising questions about calibration and understanding of numeric reasoning.

Use cases and practical considerations

  • Common target tasks: routing, classification, moderation, sentiment, triaging tickets, inventory tagging, log/error categorization, decision policies in agents.
  • Several people stress that many LLM-based workloads could be cheaper/faster with such classifiers, especially at scale.
  • Small models (~400M) are seen as viable on CPUs and consumer hardware; some are building Jev-compatible servers, vision variants, and suggest spreadsheet add-ins.

Hallucinations, “System 1,” and safety claims

  • Jev markets “can’t hallucinate” and “System 1” framing. Critics argue it still can be wrong; it’s just constrained to a schema and non‑generative.
  • Some see genuine safety benefits from always-structured outputs and schema validation (reduced prompt-injection surface), but note the model can still misclassify.

Marketing, hype, and recognition

  • Strong theme: technology vs branding. Many argue Jev’s success is about clear messaging (“System One model,” simple API, good devex) and VC-powered distribution, not purely novel math.
  • Others see Laya’s author as under-recognized because work was only on arXiv/HF/Reddit, with an academic/healthcare framing rather than productized tooling.
  • There is also skepticism: some find Laya’s original paper low academic quality, point out issues like leakage in old code, and describe the HN post and surrounding discourse as overreaching or “vibe-coded.”
  • Several conclude that prior art is dense, independent rediscovery is common in ML, and that execution, packaging, and ongoing tooling/support matter more than being first.