Frontier AI has broken the open CTF format

Impact of LLMs on CTFs

• Many agree frontier LLMs now solve a large fraction of Jeopardy-style CTF challenges quickly, turning open online events into “who has the most/best agents and tokens.”
• This shifts CTFs from human-skill contests to AI-orchestration or spending contests, similar to earlier shifts toward large, tool-heavy “mega-teams.”
• Some see this as strongly confirming the “bitter lesson”: general models beat narrow security tools.

Fairness, Cheating, and Rules

• Historically, heavy tooling and automation were culturally accepted; “attacking the infra” vs “play as intended” already split the community.
• Banning AI is seen as nearly unenforceable in remote CTFs; easy to hide AI usage compared to, say, engine cheating in chess.
• Some events now have dual leaderboards (AI-assisted vs “human”), or explicitly forbid LLMs in onsite finals with modest prizes to reduce incentive to cheat.

Learning, Skill, and Education Parallels

• A core loss identified: the “ladder” for beginners. If top of the board is AI-driven, novices are pushed to outsource instead of struggle and learn.
• Organizers report AI users often can’t explain their solves (“no idea what it did, but here’s the flag”), undermining learning and shared writeups.
• Strong parallels drawn to universities and programming education: students using AI for assignments, then failing later because fundamentals never formed.

Proposed Adaptations

• Move to offline/in‑person CTFs: organizer-provided machines, network isolation, possibly Faraday cages; but this is logistically and financially hard.
• Design AI-hostile challenges: temporal/real‑time, multimodal, game-engine embedded, counterfactual puzzles, physical/lockpicking tasks, or real‑world interaction loops.
• Others warn that “just make it harder” or more obfuscated often degrades educational value into guesswork and further excludes newcomers.

Comparisons and Broader Reflections

• Chess/Go analogies: those games banned engines and built strong anti-cheat; some argue CTFs should similarly have human-only, AI-assisted, and AI-only tracks.
• Others argue CTFs were always artificial training games; if pentesting tasks are automatable by AI, the field itself is changing, not just the competitions.
• Some are optimistic: AI as powerful tutor and productivity tool; others worry about an emerging class of practitioners who can “ship” with AI but can’t understand or debug what they deploy.

Meta: Article, Terminology, and UX

• Multiple complaints about the article/site: hard-to-read styling, lack of defining “CTF,” ambiguous use of “frontier AI,” and a title that confused non‑insiders.
• Several note acronyms and insider language make the discussion opaque to people outside the CTF/security subculture.