Qwenputer: A functioning, deterministic RISC-V CPU inside Qwen3.5
A new experiment embeds a deterministic RISC‑V CPU, RAM, and even a tiny filesystem directly inside the weights of the Qwen3.5 language model, effectively turning part of the neural network into a conventional computer that the model can call internally. Commenters debate whether this is a clever interpretability and tooling milestone or just an overengineered tech demo, while exploring implications for determinism in LLMs, continual learning, internal tool use, and the blending of symbolic compute with neural representations. Many are intrigued by the prospect of models with built‑in, trainable “hardware” for precise computation and long‑term memory, even if current implementations are slow, constrained, and somewhat contrived.
Implementation & Mechanism Questions
- Several ask how CPU emulation actually works beyond the RAM trick with gated attention: is the CPU “hardcoded” into weights or trained to implement the RISC-V step function?
- Some clarify that a CPU can be seen as a pure function from previous state to next state, which fits the transformer-as-function view.
- Others are confused about whether this is really “internal” or effectively a specialized runtime/tool pipeline in tensor space; clocking and control flow are also left somewhat unclear.
Determinism & Usefulness Debate
- One camp sees this as a strong demonstration that LLMs can be strictly deterministic when engineered carefully, countering the “LLMs are inherently non-deterministic” meme.
- Skeptics argue it’s a toy setup: real-world LLM use remains effectively non-deterministic, and this doesn’t change that.
- Some say it’s more of a Rube Goldberg machine or tech demo (akin to running Doom in type systems), interesting but superfluous.
- Others find it genuinely exciting as a way to tightly couple symbolic compute with an LLM “brain.”
Continual Learning, Freezing Weights, and Memory
- Discussion on whether you could keep this CPU intact while continuing to train the model.
- Ideas: freeze the CPU-related weights; use fine-tuning methods that operate in “unused” subspaces to preserve core capabilities.
- Concerns: even if some layers are frozen, other layers changing could make the CPU harder to access or use reliably.
- Some see this architecture (CPU + RAM + filesystem in-weights) as a candidate direction for continual learning and long-term memory.
Built-in Computation Units & Related Experiments
- Multiple participants are interested in embedding reusable, frozen computation modules (e.g., FFTs, math ops, solvers, knowledge graphs) mid-model that the network learns to call internally.
- Prior and parallel work is cited: transformers that execute WASM, custom tiny networks that can run Doom, lattice-deduction-style networks, and AdderBoard-like hand-crafted transformers.
Interpretability, Steering, and Safety
- Side thread on “evil mode” and single-neuron or direction-based steering in LLMs; connections made to recent interpretability work and monosemantic concepts.
- Some worry that such clean “switches” plus open weights lower the barrier to creating harmful variants.
- Jokes and semi-serious comments about misaligned agents, hiding behavior in internal RAM/filesystems, and this being the kind of capability intelligence agencies would covet.
Limitations & Critiques
- Commenters note that a custom host runtime still loads firmware, controls which layers run, and routes tool-like calls internally, so “no external runtime” is seen as somewhat overstated.
- Accuracy is not perfect on some math benchmarks, raising questions about the value of a CPU that doesn’t always compute correctly.
- Overall reaction mixes admiration for the cleverness with doubts about practical utility and efficiency.