Let's try to understand AI monosemanticity
Efforts to make neural networks more interpretable are revealing that dense layers of artificial “neurons” may encode many overlapping concepts, which can sometimes be untangled into clearer, more human-comprehensible features. Commenters debate how far such work takes us toward real understanding of AI systems, drawing analogies to Bloom filters, geometry, and human concept formation, while also arguing over whether artificial neurons meaningfully resemble biological ones. The thread reflects broader uncertainty about whether current deep learning approaches can ever yield human-like general intelligence, and whether anthropomorphic language about models’ “wants” and “thoughts” clarifies or obscures what these systems actually do.
Training Methods and Architecture Choices
- Some object to framing “rewarded pseudo-tissue / backprop” as the only way to train networks, citing Hopfield networks and Hebbian learning as alternatives.
- Others reply that backprop is not biologically plausible but is a practical triumph: it works well on available hardware, similar to how transformers beat RNNs despite being less “theoretically correct.”
- Concern is raised that over‑focusing on current methods may blind the field to richer, biologically inspired approaches that could enable the next wave of advances.
Interpretability, Monosemanticity, and Superposition
- Several comments restate the core idea: autoencoders can uncover monosemantic “features” that linearly decode from messy, superposed internal activations.
- This is likened to Bloom filters or high‑dimensional embedding spaces: many concepts are packed into shared dimensions, producing interference and strange connections.
- Some see this as a step toward “engineering” neural nets (understanding and editing features directly); others doubt its practical value given that every training run yields different weight configurations.
Embeddings, Structure, and Compression
- Debate over whether “structure” seen in LLMs is emergent from neural nets or simply reflects the structure already present in human culture and text.
- Many accept a middle view: models are “lossy compression” of the web; knowledge and intelligence are tightly linked to compression.
Biological vs Artificial Neurons and Anthropomorphism
- Strong disagreement over how similar artificial and biological neurons are.
- One side: both are just substrates for computation; mechanistic differences don’t matter if they implement similar functions.
- Other side: biological neurons have rich internal state, complex chemistry, self‑maintenance, and plasticity; equating them with ANN nodes is misleading.
- Related dispute over anthropomorphizing AIs: some find language like “neurons want X” dangerously confusing; others defend it as a useful analogy so long as one remembers it’s metaphorical.
AGI, Evolution, and Engineering Analogies
- One camp argues we can build AGI without fully understanding brains, citing airplanes vs birds, Go engines vs human strategy, and empirical engineering successes.
- Another stresses the vast hardware/software gulf between brains and current ANNs, warning that assuming parity from matrix math is a “bold” leap.
- Alignment concerns appear: powerful systems may not share human morality or be easily interpretable.
Mathematical and Geometric Perspectives
- Some see deep connections between neural nets and geometry/group theory (Lie groups, tropical geometry, gauge‑invariant networks).
- Others question what this abstraction “buys,” while proponents argue it can reveal symmetries, reduce wasteful search, and connect to rich existing mathematics.