The gap between open weights LLMs and closed source LLMs
Open‑weight large language models, many from Chinese labs, are rapidly closing the performance gap with proprietary systems from US companies, especially on coding tasks, while remaining far cheaper and locally runnable. Commenters debate whether these gains rely too heavily on distilling closed models, how long open releases will continue amid export controls and political pressure, and whether “open weights” should really be called open source at all. Underneath is a strategic question: if slightly weaker but free or low‑cost models are “good enough” for most uses, they could erode the business models and geopolitical advantages of US AI incumbents.
Terminology: “Open Source” vs “Open Weights”
- Several comments note the article conflates open source with open weights; some see this as an important distinction (reproducible pipeline, licensing, training data), others dismiss it as semantics since users mainly care about modifiable, locally runnable models.
- Some propose neutral terms like “available weights,” but there’s skepticism anyone will adopt them.
Persistence, Control, and Regulation
- Strong emphasis that open-weight models, once released, are hard to “take back,” unlike API models that can be deprecated or accounts shut off.
- Counterpoint: governments can still criminalize usage and deter most people via enforcement and penalties, though others argue such laws are hard to meaningfully enforce (citing piracy, drugs, censorship).
- Concern that propaganda about “harmful” open models could justify bans and OS-level restrictions; others stress civic resistance and open tools as defenses.
Sources of Open Models and Geopolitics
- Many note most strong open-weight models now come from Chinese labs.
- Debate over whether this is philanthropy, marketing, national strategy, or some mix.
- Some argue Chinese models rely heavily on distillation from US frontier models; others point to published research and claim this “copy-only” view underestimates Chinese innovation and hardware capabilities.
- Worry that both US and Chinese governments could eventually restrict open releases once they perceive serious strategic value.
Gap Between Open and Closed Models
- Coding is seen as the area where open weights are closest; some benchmarks show small gaps, and users report open models being “good enough” for many software tasks.
- For domains like legal, biomedical, and high-safety applications, many think frontier closed models still lead.
- Several predict a stabilizing lag: open models trail by the time needed to extract data from frontier models and retrain.
Economics, Sustainability, and Distributed Training
- Training frontier-scale models requires massive capital; concern that open releases rely on a few private actors and could stop.
- Ideas floated: community funding, “fabless” training companies licensing to inference providers, SETI@home-style distributed training, and federated learning.
- Others argue practical constraints (VRAM, bandwidth, reliability) limit large-scale distributed training, though small or specialized models may be feasible.