Show HN: Skyvern – Browser automation using LLMs and computer vision

Original Article ↗ Hacker News Discussion ↗

Overall Use Cases and Positioning

  • Tool is seen as an LLM + vision-based browser automation / RPA system, not just a testing tool.
  • Highlighted use cases: repetitive form-filling (e.g., government sites, insurance quotes), data entry in enterprise workflows, collecting tax documents, and potentially pen‑testing and virtual assistants.
  • Some see strongest value where a small number of complex workflows are repeated at scale (e.g., accountants handling many clients).

Cost, Performance, and Architecture

  • Current cost estimates: roughly $0.05–$0.20 per page; one user incurred a few dollars on small experiments, finding it “pricey”.
  • Complaints that calling GPT‑4V on every step is expensive and slow; several suggest using LLMs only to discover selectors or generate scripts, then replay cheaply.
  • Project authors mention: max-step limits, per-step cost visibility, future “memorization/caching” of workflows, and an LLM router with support for models like Claude 3 and local LLaMA.
  • Technically, it combines DOM parsing with vision and heuristics rather than pure pixel-based approaches, which are reported as unreliable.

Spam, Abuse, and CAPTCHAs

  • Strong concern that tools like this escalate botting and spam (forms, voting, credit card testing).
  • CAPTCHAs are already often bypassed via cheap human‑in‑the‑loop or AI; this project can solve some CAPTCHAs but that part is not open-sourced and is “gated” by use case.
  • Proposed mitigations discussed: charging money/credit-card verification, manual onboarding, and even eID‑style rate‑limited identity tokens—though some argue identity binding doesn’t solve spam.

Impact on the Web, APIs, and Jobs

  • Some fear a cat‑and‑mouse arms race: more DRM, anti‑bot measures, unscreenshotable UIs, and a web less friendly to humans or accessibility.
  • Others frame it as a “unified API” where agents can use existing UIs when formal APIs are absent, including government sites that don’t want to expose APIs.
  • There’s speculation that such automation will accelerate back-office job automation.

Testing, Scraping, and Code Generation

  • Multiple people want this for frontend/e2e testing: asserting conditions, generating Cypress/Playwright/Selenium code, and producing reusable scrapers.
  • A recurring suggestion: treat the LLM as a “compiler” that outputs automation code or an action plan, only re-running it when UIs change, instead of using LLMs at runtime on every run.

Ethics, Security, and Licensing

  • Serious concern about automating medical billing/EHR workflows due to fraud and loss of human accountability.
  • Interest in using it defensively (e.g., password rotation flows) and in studying prompt‑injection and security models.
  • AGPLv3 licensing is a blocker for some enterprises, despite appreciation for open-sourcing.