The Google employees who created transformers

Role of R&D, Tax, and Corporate Incentives

• Several comments argue “bloated” R&D is socially valuable; big breakthroughs often come from projects with no immediate revenue.
• Debate over whether R&D is really “tax free”: distinction between normal expense deductions vs R&D tax credits and amortization rules.
• Some see R&D tax structures as subsidies for big tech; others push back on “tax free” framing.

Immigration, Diversity, and Talent Concentration

• Many highlight that the transformer team was overwhelmingly first‑generation immigrants or children of immigrants.
• Some see this as evidence of the US (and California in particular) effectively attracting global talent.
• Others note that US immigration and visa systems are not uniquely welcoming compared to some countries, and that diversity is higher in graduate programs than in the general population.

Google’s Missed Opportunity and Management Critique

• Repeated theme: Google had the people, papers, and infrastructure to be “OpenAI” but failed to aggressively productize transformers.
• Explanations offered: fear of public backlash, impact on search/Assistant, internal politics, AI-safety gatekeeping, and CEO/board conservatism.
• Some see classic “innovator’s dilemma”: protecting an ad business that might be disrupted by chatbots.
• Others argue Google can just acquire winners later, though others say modern VC dynamics make “just buy them” unrealistic.

Impact on Search and Ads

• Some users have already shifted many queries (especially technical) from Google search to LLMs.
• One view: LLMs cannibalize Google’s future ad revenues and threaten the web ecosystem.
• Counterview: LLMs currently absorb unprofitable fact‑finding queries; commercial intent still favors traditional search and ad networks.

What Counts as “Modern AI” and the Role of Transformers

• Disagreement over headlines claiming transformers “invented modern AI.”
• Supporters: transformers are the core architecture behind current LLM hype and enabled today’s systems.
• Skeptics: progress is cumulative—attention, deep nets, big data, GPUs, diffusion models, and earlier work all matter; “modern AI” predates transformers.
• Some emphasize “bitter lesson”–style scaling with simple architectures; others argue future directions may lean more on world models, memory, and more deterministic structures.

Self‑Driving and Transformers

• Some expected self‑driving to define “modern AI” rather than chatbots.
• Others think multimodal transformers will eventually be central to higher‑level driving competence, especially for rare/edge cases, but note current compute and safety limits.

Prior Art and Credit Allocation

• Multiple posts complain about narratives that over‑credit a single company or paper, given extensive prior art in attention and sequence models.
• Comparisons are made to how history sometimes over‑simplifies credit (e.g., in physics or CRISPR).

Industry vs Academia and Compute

• Discussion notes that big industrial labs, not universities, drove much of the last decade’s progress due to compute and salaries.
• Some lament that many foundational contributors (e.g., mathematicians, earlier ML researchers) remain mostly invisible in popular retellings.

Collaboration, Office, and Culture

• Commenters note the transformer authors co‑located in one office; some take this as support for in‑person collaboration with private offices plus deep‑work time.
• Others counter that typical open offices undermine focus, advocating for a balanced or hybrid approach.