Show HN: Let your AI agents paint big arrows, boxes and text on your screen
A small macOS tool that lets AI agents draw oversized arrows, boxes, and text on the screen to tell users where to click is prompting mixed reactions. Supporters see it as a practical aid for onboarding, complex apps like Blender, remote tech support, accessibility, and human-in-the-loop agent workflows, where an AI can guide rather than fully automate actions. Critics argue it exemplifies wasteful “AI slop,” raises concerns about security, manipulation and over‑reliance on cloud models, and reflects how convoluted software UX has become if people now need robots to point at buttons.
Purpose of the Tool
- CLI for macOS that lets AI agents (or humans via scripts) draw big arrows, boxes, and text overlays on the screen.
- Intended mainly for “guiding agent” workflows: AI shows “click here” while the human keeps control, especially for actions agents shouldn’t or can’t perform (passwords, security dialogs, GUI-only tools).
Perceived Use Cases
- Step‑by‑step guidance in complex GUIs (e.g., Blender, media editors, corporate enterprise apps with confusing labels).
- Remote tech support, especially for older or less technical users, as an alternative to repeatedly explaining “the other gear icon” or using remote desktop.
- Better tutorials and documentation: directly highlight on-screen buttons instead of static screenshots and prose.
- Agent debugging and multi‑agent visualization: visual traces of what agents are focusing on.
- Accessibility / training contexts reminiscent of old PC interactive tutorials.
Skepticism and Critique
- Some see it as emblematic of “AI slop”: low‑effort, vibe‑coded code that exists because LLMs make building such tools cheap.
- Others argue it treats humans as “meat bots” doing blind approvals, eroding the step where humans understand consequences.
- Concerns that it’s fixing problems created by poor UX instead of improving the underlying software.
- Several find the arrows visually ugly or “gross,” and dislike the added visual noise, comparing it to annoying popups and Clippy‑style helpers.
Security, Abuse, and Permissions
- Questions about whether overlay power could be abused to hide “decline” buttons or change perceived UI text.
- Counterpoint: if the tool only draws overlays and the agent already has shell access, masking buttons is not the main threat; trust in the software vendor remains the core issue.
- Discussion that current LLM agents often refuse sensitive actions (passwords, security settings) due to system prompts and built‑in norms, not OS‑level permissions alone.
Broader AI & UX Reflections
- Debate over whether this is wasteful overengineering versus a pragmatic fix for decades of bad UX and insufficient in‑app search/tutorials.
- Meta‑discussion about a future where repositories become specs for agents, “prompt = effort,” and AI saturates everyday tooling, for better and worse.