Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out

Original Article ↗ Hacker News Discussion ↗

A startup analyzed about 17,000 runs of coding agents like Claude Code, OpenAI-powered Codex tools, and Cursor to see which developer tools they install and how they choose them, finding that agents often disagree, rarely use web search in some cases, and tend to favor familiar incumbents. Commenters are intrigued by the idea that AI preferences could shape which frameworks, databases, and services become mainstream, but many warn this could recreate the SEO and ad-driven distortions that already plague web search. Several argue for safeguards, transparency, and strong open-source alternatives before “agent optimization” turns into a new, less visible version of growth hacking and paid placement.

Study findings & tool behavior

  • The study ran ~17k coding-agent sessions across varying repos, personas, and org sizes to measure tool choices.
  • Reported patterns: Claude Code rarely uses web search; Codex almost always does; Cursor is in between. Some see the same behavior in practice, others say Claude searches only when explicitly asked.
  • Certain tools (e.g., LangChain, Supabase, Netlify, payment processors) are frequently mentioned but rarely chosen.
  • Repository context and instructions can significantly alter which tool an agent selects.

Influencing agent choices & “SEO for agents”

  • Core business idea: analyze what agents choose, then help vendors nudge agents toward their tools, analogous to SEO/marketing for humans.
  • One commenter reports independently building a similar system and confirms that “selling to agents” feels like traditional marketing.
  • Another notes future models may include affiliate-style revenue for recommendations, though others argue there is immediate opportunity even before that.

Ethics, ads, and trust

  • Strong pushback against “growth hacking” agents, seen as repeating SEO-style degradation of the web and potentially pushing suboptimal tools.
  • Some argue this could counterbalance incumbents baked into model priors and help newer, better tools surface.
  • Many fear “Agent Engine Advertising” and sponsored tool choices, likening it to search ads creep; several assert this is already starting with large labs.

Technical behavior & robustness

  • Agents appear resistant to naive prompt-injection-style attempts to bias them: experiments with a custom search index showed that overly strong or inconsistent bias triggered safety mechanisms and suspicion.
  • Tool-selection differences are attributed to model instructions, system prompts, and cost/efficiency considerations (e.g., preferring shell/Python for multi-file edits to save tokens).

UX, pricing, and ecosystem

  • Several commenters criticize the Armature site for intrusive popups and poor mobile layout; the team responds that fixes are being shipped.
  • Pricing (starting around $5k/month) is seen as high but is framed as a managed “agency” service rather than mere tool access.
  • Broader discussion connects this to concerns about AI centralization, the “golden age” ending, and the importance of open-source/open-weights alternatives.