AI

OpenAI finds roughly 30 percent of popular AI coding test is broken

July 9, 2026 1 min read
OpenAI finds roughly 30 percent of popular AI coding test is broken

OpenAI reviewed SWE-Bench Pro, a widely used test for measuring AI models’ programming skills, and found roughly 30 percent of its tasks are broken. The company is pulling its earlier endorsement of the benchmark.
The article OpenAI finds roughly 30 percent of popular AI coding test is broken appeared first on The Decoder.

Originally reported by
The Decoder
Back to Articles

More to Read

Databricks makes Chinese open-source model GLM 5.2 its default coding engine after it matched Opus at lower cost
AI

Databricks makes Chinese open-source model GLM 5.2 its default coding engine after it matched Opus at lower cost
ChatGPT can now listen and talk at the same time, making AI conversations seem more human
AI

ChatGPT can now listen and talk at the same time, making AI conversations seem more human
Copilot goes cheap as Microsoft phases out OpenAI and Anthropic models to cut costs
AI

Copilot goes cheap as Microsoft phases out OpenAI and Anthropic models to cut costs

Scroll to Top