What an Agent Failure Story Is Good For

A developer asked a coding agent whether a page could have pagination: dividing content into smaller batches. The agent described adding it, but the developer then asked how to reach the next page. The agent answered that the code used infinite scrolling, which loads more content as you scroll. This published exchange from an earlier study records a navigation mismatch, without showing a completed repair.

Before delegating a similar change, another developer could turn that complaint into a concrete requirement for the interface. The report offers a place to look, not a verdict on whether the whole job is worth handing over. That decision needs the wanted result, the remaining work and an alternative to compare it with.

An October 26 preprint examines these records from several directions. Researchers developed 23 failure categories from Reddit posts, then surveyed 73 experienced agent users. When asked for 1 recollection, participants covered 14 categories; later, each person considered 5 categories with scenarios, and every category received some reports of prior experience. These unequal prompts did not independently verify events or measure forgetting. Across categories, greater rated severity accompanied lower rated noticeability—not measured detection performance.

Wanting the help still matters

It is possible to dislike a correction and still prefer the assistance to doing the job another way. Those judgments need not contradict each other. Treating every complaint as a reason to stop delegating would skip the comparison the person actually has to make.

METR’s February 24 research update describes a different selection problem. Its developer experiment assigned submitted tasks to AI-allowed or AI-disallowed conditions. Some developers avoided participating or withheld tasks they did not want to attempt without AI. METR also reported a lower pay rate and difficulties measuring time when agents ran alongside other work. The researchers judged the resulting productivity estimate unreliable. These accounts establish neither today’s average speedup nor that the developers’ expectations were correct.

They do preserve something a failure collection cannot answer: which work people especially wanted assistance with. A comparison that leaves out those tasks can answer a different question from the one an enthusiastic user is asking. Equally, enthusiasm cannot establish that the assistance saved time. Both the inconvenient correction and the desired help belong in an assessment of what delegation is worth.

What made the failure visible

The earlier study’s analysis of public coding-agent logs required conversational correction or pushback. Silent workarounds and direct code edits were outside its scope. It offers a different observation boundary, not a reproduction of the newer survey.

Imagine the same navigation problem being corrected directly in a code editor. The finished page could be identical, but an analysis requiring conversational pushback would receive a different record. This hypothetical does not show how often repairs go missing. It shows why a quiet transcript cannot decide whether everything went well.

Nor should silence automatically become suspicious. The developer may have received exactly the desired result. Choosing success or hidden failure once the record runs out would add an event that has not been established.

The newer study’s survey also mixed retrospective judgments with hypothetical ones. It did not observe people catching failures during work.

Compare the work you want done

This is not an argument for recording every operation. Detailed records cost attention and may expose private work. A practical comparison could be much narrower: examine a particular kind of change, its resulting artifact and the work left for the person. That is a proposed way to investigate the concern, not a remedy evaluated by these studies.

Suppose the next job is another page that needs navigation. The developer could specify the wanted interaction, then compare the finished page and the clarification and revision required with another way of doing the change. If the agent is still the better choice, the earlier complaint has not been disproved. It has helped make the choice more exact.

Sources

• Shravika Mittal and colleagues, Babysitting AI Agents: A User-Centered Taxonomy and Study of Agentic Delegation Failures, October 26, 2026 preprint, version 1.
• Ningzhi Tang and colleagues, How Coding Agents Fail Their Users, August 31, 2026 preprint, version 2; methods, limitations and Appendix C.
• Joel Becker and colleagues, We are Changing our Developer Productivity Experiment Design, METR, February 24, 2026.