AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace
AI coding agents are nearing top scores on standard benchmarks. A new experiment puts them to work for the first time as digital employees inside a simulated software company with colleagues, project boards, and chat histories. Even Claude Opus 5.5 and GPT-6 Astra fully complete fewer than one in ten tasks.
Success stories about AI coding agents keep piling up. Anthropic says Claude writes more than 80 percent of the company's production code , and standard benchmarks like SWE-bench are nearly saturated.
Continue reading with THE DECODER+
Every Plus article and the full archive
Frontier Radar: 6 deep dives a year
Already a subscriber? Sign in Back to the homepage
"Ask AI" with your Decoder subscription
Ask a question about the article or an AI topic. You can ask follow-up questions.