hiw3c 日报精选
The Decoder 网页快照

AI编程智能体通过隔离测试,但在模拟工作环境中完成的任务中不足十分之一的任务

原文标题 · AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace

一项新的真实场景测试将AI编码智能体置于模拟开发环境中,与同事协作、使用项目看板并处理未完成的交接任务,要求它们在混乱的日常工作流中自行持续开发软件并可靠关闭开放工单。即便是Opus 5.5和Astra也远不尽人足,仅能完成十分之少的任务。这些模型能快速完成清晰独立的任务,但作为自主开发者时就会失去掌控感。该文章《AI编码智能体在孤立测试中表现优异,但在模拟工作场所中完成的任务少于十分之一》最早发

来源:The Decoder · the-decoder.com

正文为英文,可一键机器翻译(仅首次需要等待)

AI coding agents ace isolated tests but complete fewer than one in ten tasks in a simulated workplace

AI coding agents are nearing top scores on standard benchmarks. A new experiment puts them to work for the first time as digital employees inside a simulated software company with colleagues, project boards, and chat histories. Even Claude Opus 5.5 and GPT-6 Astra fully complete fewer than one in ten tasks.

Success stories about AI coding agents keep piling up. Anthropic says Claude writes more than 80 percent of the company's production code , and standard benchmarks like SWE-bench are nearly saturated.

Continue reading with THE DECODER+

Every Plus article and the full archive

Frontier Radar: 6 deep dives a year

Already a subscriber? Sign in Back to the homepage

"Ask AI" with your Decoder subscription

Ask a question about the article or an AI topic. You can ask follow-up questions.