Deep Learning Is Applied Topology

Topology vs. Geometry and Manifolds

• Many argue the post is really about geometry/differential geometry (manifolds with metrics), not topology (which discards distances, angles, and scale).
• Topology is described as what survives after “violent” deformations; deep learning in practice heavily relies on metric structure, loss landscapes, and distances.
• Disagreement over the statement “data lives on a manifold”:
  • Pro: low‑dimensional latent structure seems required for high‑dimensional problems to be learnable.
  • Con: real data can be discrete, noisy, branching, and “thick”; manifolds are at best approximations, not literal.
• Some note alternative meanings of “topology” (e.g., network topology, graphs) and complain the article conflates or overextends the mathematical term.

Theory vs. Empirical “Alchemy”

• One camp: deep learning progress is mostly empirical, advanced by trial‑and‑error, heuristics, and engineering; theory (especially topology) has had little direct impact on widely used methods.
• Counter‑camp: many ideas are adaptations or reinventions of prior theory (linear algebra, optimization, statistics, statistical physics, control, information theory); dismissing theory is short‑sighted technical debt.
• There is no consensus on what a satisfactory “theory of deep learning” should deliver (convergence bounds, architecture guidance, hyperparameters, efficient weight computation, etc.), and some doubt such a theory will be very practical.

Representations, Circuits, and Transformers

• Several comments favor thinking in terms of representation geometry: embeddings, loss landscapes, and trajectories of points under training; visualization via UMAP/t‑SNE; early “violent” manifold reshaping followed by refinement.
• Work on linear representations and “circuits” (features as directions; networks of interacting features) is cited as more productive than purely topological views, including symmetry/equivariance analyses.
• For transformers, attention is framed as a differentiable kernel smoother or distance‑measuring operation over learned embedding manifolds, with feed‑forward layers doing the geometric warping.

AGI, Reasoning, and Manifold Metaphors

• Several object to the article’s casual claim that current methods have reached AGI, seeing it as a credibility hit.
• Debate over whether “reasoning manifolds” or topological views meaningfully capture logical/probabilistic reasoning; some argue all reasoning is fundamentally probabilistic, others maintain discrete logical operations are essential.
• The idea that AGI/ASI are just other points on the same manifold as current next‑token or chain‑of‑thought models is challenged as unjustified.

Usefulness and Limits of the Topology Framing

• Some find the manifold picture a helpful intuition pump for understanding separation surfaces, embeddings, and similarity; others see it as a rebranding of long‑known “manifold learning” with little new.
• Requests for concrete examples where topology (in the strict sense) improved models or understanding mostly go unanswered; topological data analysis is mentioned but seen as niche so far.
• Multiple commenters conclude: deep learning certainly can be described using topology, but calling it “applied topology” adds little unless it yields new, testable design principles or performance gains.