Anthropic's open-source framework for AI-powered vulnerability discovery

Original Article ↗ Hacker News Discussion ↗

Project status and purpose

  • The repo is explicitly “not maintained” and positioned more as a reference harness than a supported tool.
  • Several commenters see it mainly as marketing for a commercial “Claude Security” / managed scanning offering.
  • Others frame it as a pattern library: useful to study, then recreate or adapt internally rather than adopt directly.

Costs and scalability

  • Token use per agent is high; full-codebase scans, especially with top-tier models (Opus/Mythos), are perceived as expensive.
  • Some argue you’d only run full scans periodically and diffs in CI, but others note many orgs ship every sprint, making recurring cost large.
  • Cost comparisons vary: some say a dedicated security hire might be cheaper; others cite reports claiming AI-based scanning can match many engineers, making it “pennies on the dollar” versus traditional audits.
  • External calculators are referenced showing multi-million–dollar annual token spends for 100+ dev teams if used heavily.

Effectiveness, false positives, and workflow

  • Without a well-designed harness, people report poor results and many false positives (“vibe auditing”).
  • Even with a harness, findings still need expert review and triage, otherwise developers drown in noise, similar to today’s linters and SAST.

Impact on attackers, defenders, and existing tools

  • Some see this as a potential existential threat or eventual feature for traditional SAST vendors.
  • Others note attackers can use the same models, turning vulnerability discovery into a “proof-of-work” arms race, but emphasize the bugs already existed.
  • Concerns include a flood of high-severity reports overwhelming maintainers and bug-bounty programs.

AI-generated code and security lifecycle

  • Many note it now takes far more tokens to secure code than to generate it.
  • There’s skepticism about AI vendors effectively charging first to generate “sloppy” code and then to scan/fix it.
  • Some argue models should be trained to emit secure code, but others reply that serious bugs often span large, dependency-heavy codebases beyond what can be fully reasoned about on each edit.

Business model and ecosystem debates

  • Debate over why vendors sell raw tokens vs vertical SaaS:
    • One side claims if tokens were truly magical, vendors would hoard them and dominate industries directly.
    • Others counter that selling infrastructure (like fabs or tractors) can still be optimal, and building end-user SaaS is a different, distraction-heavy business.
  • Some see security harnesses as part of a broader shift: AI companies turning domain-specific harnesses (design, security, etc.) into packaged products.

Open source harnesses and “shop jig” tools

  • Multiple alternative or similar tools are mentioned; some trip antivirus and are mainly for practitioners comfortable with that.
  • A recurring analogy likens these frameworks to “shop jigs”: custom tooling tuned to a team or individual’s workflow, often better built in-house than used off-the-shelf.
  • Commenters discuss making such harnesses portable across jobs, or shared within organizations to raise the team-wide productivity floor.
  • There’s a broader theme that AI makes bespoke tooling so cheap that generalized libraries/harnesses are increasingly used as inspiration rather than as direct dependencies.

Trust, naming, and practical concerns

  • Confusion around the GitHub account name (“Anthropics” vs “Anthropic”) recurs.
  • Some dismiss the project as “open-source glue to an LLM blob” or criticize that it’s unmaintained and closed to contributions.
  • A few express distrust of sending source code to remote LLMs or being gated by specific browsers/search engines.