Chomsky and the two cultures of statistical learning (2011)
Statistical machine learning and rule-based linguistics collide in a long-running debate over how language is best modeled and understood. Commenters contrast probabilistic, data-driven systems such as modern large language models with theories that posit rich innate grammatical structure, arguing over what counts as scientific explanation versus mere engineering. The exchange highlights open questions about how humans actually acquire language, whether current AI offers a meaningful cognitive model, and how far probabilistic methods can go in capturing the “creative” aspects of language use.
Probabilistic models and “simplicity”
- Several commenters challenge the claim that probabilistic models are the “simplest” way to handle uncertainty.
- Objections: they require large computations and vast data; they often work as black boxes that don’t “model” mechanisms, only correlations.
- Others argue “simple” means conceptually natural for inherently stochastic phenomena (e.g., noisy perception), even if computationally heavy.
Other ways to represent uncertainty
- Alternatives mentioned: deterministic models with unknown variables, ternary logic (true/false/unknown), fuzzy logic, backtracking/beam search with heuristics.
- Some see these as either cruder or “probabilities in disguise” (e.g., “unknown” ≈ some interval between 0 and 1).
- In applied domains like weather, ensemble probabilistic forecasting is described as the only practical approach known so far.
Engineering vs science and what models explain
- One camp: successful engineering with probabilistic/statistical methods is evidence that underlying systems are at least partly statistical.
- Opposing view: working technology does not automatically explain natural phenomena (planes vs birds, LLMs vs humans); it may only approximate behavior.
- There is debate over whether current language models are good scientific models of human cognition or just powerful tools.
LLMs, language learning, and innateness
- Some argue LLMs show language can be learned de novo by exposure and that intelligence may be less mysterious than assumed.
- Critics counter: models need many orders of magnitude more text and compute than children; human learners have rich multimodal input and strong innate priors shaped by evolution.
- Ongoing dispute over whether grammar is based on categorical built‑in rules plus some statistics, or is fundamentally statistical with innate priors.
Status of formal linguistics and rule-based theories
- Several comments say classical generative frameworks were scientifically influential but poor for engineering tasks and are now waning in core linguistics.
- Others defend them as having yielded substantial empirical knowledge of syntax, even if their highest‑level theories are no longer widely believed.
- Some criticize dismissals of probabilistic or statistical methods by traditional theorists as dogmatic and obstructive.