AutoDev: Automated AI-driven development by Microsoft
Microsoft’s AutoDev research prototype, which uses AI agents to automate parts of the software development workflow, is prompting debate over how far coding can be offloaded to machines and what work remains for humans. Commenters see clear productivity gains, especially for boilerplate and debugging, but question benchmark relevance, real‑world reliability, liability for AI‑written bugs, and the risk of ossifying current tools and patterns. Many expect engineering roles to shift toward architecture, domain modeling, requirements, and oversight, while warning that broad white‑collar automation could reshape labor markets and social stability.
Scope of AutoDev and Similar Tools
- Seen as an AI “assistant” that automates parts of existing workflows (tests, refactors, simple bugfixes), not full autonomous engineering yet.
- Compared to Devin: one is framed more as a “builder/assistant,” the other as a more autonomous “architect,” though both are early and demo-like.
- Some note similar open-source tools already exist (e.g., IDE-integrated agents, custom DSLs), leading to confusion and calls for clearer naming.
Evolving Role of Software Engineers
- Many predict a shift from “writing code” to:
- System and architecture design
- Requirements gathering and domain modeling
- Integration, verification, and exploratory testing
- Project/program management and stakeholder communication
- Entry- and mid-level “CRUD/plumbing” work is seen as most at risk; niche expertise, legacy systems, and deep debugging may remain safer.
Productivity, Benchmarks, and Limits
- Heavy users report maybe 2–3x personal productivity at best; far from “100x engineers.”
- Benchmarks like HumanEval are criticized as leetcode-style, narrow, and possibly contaminated by training data. Real-world relevance is questioned.
- Some think agents plus LLMs will eventually handle steps 2 and 4–8 of the dev lifecycle (design, coding, testing, troubleshooting), others see major gaps in autonomy and robustness.
Economic and Social Impact
- Strong concern that companies will use AI to reduce headcount rather than hours, shrinking middle-class dev jobs.
- Historical analogies: portrait painters vs photography, textile workers vs mechanization, spreadsheets vs accountants. Some see eventual new roles; others fear a harsher transition without safety nets.
- Debate over whether this is “creative destruction” or a path to mass redundancy, including other white‑collar roles.
Process, Testing, and Requirements
- Many expect AI-driven dev to center on precise specification: tests, property-based specs, or specialized requirement languages, not plain English.
- Some see this as offloading “fun coding” to AI and leaving humans with the hard, ambiguous, and sometimes “shitty” requirements work.
Broader AI/AGI Debate and Skepticism
- Split between those who see LLMs as overhyped “stochastic parrots” and those who view them as early but fast-improving general tools.
- Concerns about ossification: models may reinforce current dominant tools (e.g., pandas, table-based layouts) and slow deeper innovation.
- Liability and alignment questions remain unresolved: who is responsible when AI-generated code fails is seen as legally and practically unclear.