Codex Security

Original Article ↗ Hacker News Discussion ↗

OpenAI’s newly open-sourced Codex Security CLI, a tool that scans codebases for vulnerabilities using GPT‑5.6 Sol, is drawing both interest and frustration. Early users report authentication bugs, opaque long-running scans, aggressive safety guardrails that refuse to explain found issues, and unexpectedly high token costs that can exhaust ChatGPT quotas mid-run. The project is praised for its reusable “skills” and CI-focused harness, but many question its practicality for real-world security work without special cyber access or local model support, and raise concerns about sending proprietary code to a hosted AI service.

Launch & Authentication Issues

  • Several users hit authentication failures at launch; a fix was quickly released.
  • Environment variables like OPENAI_API_KEY / CODEX_API_KEY can override ChatGPT/Codex logins, causing confusion.

Guardrails, Refusals & Cyber Access

  • Many reports of the tool running for tens of minutes, consuming significant token quotas, then aborting due to cybersecurity guardrails.
  • Users criticize being charged when the model “finds an issue but refuses to explain,” calling it infuriating and suggesting refunds in such cases.
  • OpenAI staff say the CLI does not bypass model safety; Trusted Access for Cyber (TAC1/Daybreak) can reduce refusals for vetted OSS maintainers and enterprises but is not a blanket bypass.
  • Some see the safety system as making the tool “borderline useless,” especially compared to less-restricted models.

Cost, Rate Limits & UX Problems

  • Multiple anecdotes of scans burning a large fraction of weekly Pro/Plus usage or tens to hundreds of dollars, sometimes failing due to rate limits or repo HEAD changes.
  • --max-cost exists but doesn’t fully prevent failed long runs; partial results are stored but cannot yet be resumed.
  • Users want clearer progress, token usage display, and early warnings about cyber-access requirements.

Scope, Capabilities & Architecture

  • The CLI is described as a wrapper around existing models plus 13 security “skills” (English prompt specs) and a harness: org-wide scans, deduplication, false-positive tracking, budget controls, CI integration.
  • Compared to the Codex plugin, it targets multi-repo, historical, and CI workflows.
  • It is not an offline scanner; code context is sent to OpenAI’s hosted models.

Data Privacy & Alternative Backends

  • Concerns about uploading proprietary code; OpenAI notes business/enterprise data isn’t used for training by default but says organizations forbidding data egress shouldn’t use it yet.
  • Official support for local or OpenAI-compatible endpoints is “in progress”; users already discuss routing through other providers and open-weight models.

Ecosystem, Comparisons & Broader Concerns

  • Comparisons to Snyk, Alibaba’s open-code-review, Strix, Google’s Mantis, and Chinese models (e.g., Kimi) surface; some see Codex Security as mostly a polished harness.
  • Some argue AI tools both introduce and help find vulnerabilities (“fire department run by arsonists”), while others focus on practical value regardless of origin.
  • There is interest but also strong frustration around costs, safety refusals, and the operational polish of a tool from a large, well-funded company.