Go to PegaPoll
PEGAPOLL · NEWS

AI coding agents complete fewer than one in ten tasks in a simulated workplace test

The Decoder (AI) · 2026-11-22
🤖 AI-generated content — The title and summary were produced automatically by artificial intelligence, without human editorial review.

A new workplace-style test treats AI coding agents as digital employees inside a simulated software firm, complete with colleagues, project boards, chat history, and incomplete ticket handoffs. Instead of isolated coding puzzles, the agents must keep developing software on their own and reliably close open tickets in a messy daily workflow. Even the strongest models in the experiment, Claude Opus 5.5 and GPT-6 Astra, fully finish fewer than one in ten tasks.

That result sits uneasily beside the industry’s usual scoreboard. Standard benchmarks such as SWE-bench are nearly saturated, and Anthropic has said Claude already writes more than 80 percent of the company’s production code. Success stories about coding agents have piled up, yet the simulated job shows a sharp gap between those isolated tests and day-to-day autonomy.

The models still close clear, standalone assignments quickly. They lose track, however, when they have to operate as autonomous developers over time: following messy handoffs, staying aligned with colleagues and boards, and seeing tickets through in an incomplete, ongoing workplace rather than a single well-scoped problem.

Continue in the app — vote & join in ➔
Source: The Decoder (AI) · via ahirlevel.hu

Related news