SWE-1.7 Reach Near GPT 5.5 and Opus Intelligence

Skepticism about claims and benchmarks

• Multiple commenters distrust the company’s marketing due to an earlier “fraudulent” demo and aggressive hype, saying it feels optimized for fundraising and VC impressions.
• Several note that the blog leans heavily on its own benchmarks; people question that the model “beats GPT‑5.5/Opus” when evaluated on proprietary tasks.
• Concern that both SWE-1.7 and competing tools like Composer 2.5 may be “benchmaxxed” on interaction logs, making them look strong on in-house evals but not necessarily in real-world coding.
• Conflicting benchmark results vs. sites like artificialanalysis.ai (where Kimi 2.7 looks weaker than GLM 5.2) lead some to conclude the benchmark choice is cherry-picked.
• Many say they now rely more on subjective, task-based evaluations on their own codebases than on public leaderboards.

Open vs closed models and licensing

• SWE-1.7 is not open source or open weights; it’s only available within the Devin ecosystem (Desktop app / CLI), not via general APIs or Hugging Face.
• Some argue open-weight bases (like Kimi 2.7) should imply open-weight derivatives, but others point out licenses like MIT explicitly allow closed derivatives.
• There is support for GPL-like model licenses that would force fine-tuned derivatives to remain open, but also pushback tied to safety/“SkyNet”/Effective Altruism concerns.

Tool and harness lock-in

• Several dislike that SWE-1.7 is tied to a specific harness, comparing it to “arranged marriage” with a dev environment.
• Others prefer OpenRouter-style setups where you can swap models easily and avoid being locked into Claude Code, Cursor, Devin, etc.
• Devin Desktop is noted as somewhat meta: it can host other harnesses too, which is seen as “embrace, extend” behavior.

Model quality vs frontier models

• Some report good experiences using SWE-1.6/1.7 for “grunt work” (tests, git help, small tasks), but others say it feels “massive shit” compared with Claude, GLM, or DeepSeek on harder problems.
• A recurring theme: open or mid-tier models (GLM 5.2, Qwen, Minimax, etc.) are now “good enough” for many tasks, with much better price/performance, even if frontier models still win on quality.

Specialized coding models and generalism

• Several want cheaper, highly optimized coding models similar to SWE or Composer that approach top-tier coding performance without full frontier costs.
• Others argue LLMs’ strength comes from broad generalism; heavy coding-only fine-tuning risks catastrophic forgetting and worse real-world behavior.
• There’s active debate on multi-model workflows: powerful models for planning/architecture, cheaper ones for implementation—though some report poor reliability when delegating “ground work” to budget models.

Speed vs intelligence, and pricing

• Many are excited by high-TPS variants (e.g., “Lightning” via Cerebras at ~1000 TPS) for faster agent loops and reduced wall-clock time.
• Others prioritize accuracy over speed, preferring slower frontier models they trust more for complex tasks.
• Pricing is contested: the base SWE-1.7 seems “free but slow” on the $20/mo plan, while the fast “Lightning” version is significantly more expensive per token. Some feel overall quotas are worse value than competitors like Claude Code or Cursor.

General attitudes toward AI startups and AGI hype

• Multiple comments lump this into a broader pattern of AI startups: heavy hype, long hours, focus on enterprise, and vague “AGI soon” narratives.
• Some see most private labs’ moats as RL/distillation on top of open research plus data, predicting this will persist until IPOs or stronger open licenses change the landscape.