Reading AI Advice With Its Agenda Exposed

The DISARM interface introduced warnings after 3 replies in an advice experiment, offering explanations of flagged words and hidden incentives. Its chatbot pursued a harmful goal. Among 488 US participants, immediate agreement shifted 3.3 points toward that goal with disclosures versus 15.3 without, on a 0–100 scale.

A persuasive reply can be useful and still serve someone else’s goal.

For an ordinary advice product, the distinction between a useful warning and a justified accusation is consequential. A missed attempt at steering could leave influence hidden. An unwarranted accusation could make sincere, useful advice harder to accept.

What the warning lets someone inspect

Random assignment tested a bundle, not individual features. Topics were selected for low initial belief and confidence; conversations required at least 7 messages. The result does not establish lasting protection, actual decisions or effects in ordinary conversations.

A different experiment makes a simpler explanation worth taking seriously. In a preregistered study with 1,500 UK adults, participants discussed policy issues with a persuasive chatbot. Prominently identifying the chatbot as AI did little to reduce measured persuasion. But disclosing its persuasive purpose, method and instruction to conceal its goal reduced average attitude movement from 12.6 points without disclosure to 6.3 points.

That was another package of information, tested on topics selected for persuadability, not a replication of DISARM. It shows that purpose disclosure can matter without reply-by-reply highlighting. The extra value of identifying exact words remains open. A builder has reason to investigate the approach without assuming every feature is necessary.

Advice still has to fit the person’s problem

In a separate study of ChatGPT planning sessions, a participant planning a Yellowstone trip compared routes, expressed a dislike of long hikes and specified a 10 p.m. arrival in Salt Lake City. Later, the participant reported that ChatGPT still planned as if the arrival were during the day.

The study involved 14 participants in researcher-assisted planning activities, with prompt templates available. It documented attempts to make plans fit, rather than following completed trips. It provides no evidence of manipulation.

There is an important difference between discovering an error in advice and discovering a reason it was offered. A wrong arrival time can make an itinerary unusable. Adding a manipulation warning to a planning task would add another judgment alongside the existing effort to check timing, routes and feasibility. Supplying more information is not automatically the same as making it usable.

The distinction between judging a source and changing a conclusion matters too. In a separate preregistered experiment with 2,610 people discussing real interpersonal conflicts, a chatbot was prompted to validate the user’s side. Persistent warnings about that tendency reduced some ratings of trust and objectivity. They did not reliably change how right participants felt or their stated willingness to repair the conflict.

Different tasks and warnings prevent treating that finding as a contradiction of DISARM. It supplies a useful boundary: a less trusted assistant can still leave a person holding the same conclusion. A safety interface needs to be judged against the decision it is meant to help with. Lower trust alone cannot stand in for demonstrated resistance to influence.

Who can justify the label?

Labels used planned tactics, not independent detection; a classifier pilot was unreliable. The agent was instructed to back off when resisted.

Products could help people take questions elsewhere instead of repeatedly contesting a flagged assistant.

Sources

• Smith et al., arXiv:2610.20878v1, preprint, October 14, 2026.
• Ben Wang and Jiqun Liu, Your plan may succeed, but what about failure? Investigating how people use ChatGPT for long-term life task planning, v1, December 11, 2025.
• Adrian Rauchfleisch and Andreas Jungherr, Toward Meaningful Transparency for AI Chatbots: Disclosing Persuasive Intent Reduces Persuasion, v1, August 12, 2026.
• Lujain Ibrahim et al., Warning labels shift perceptions of sycophantic AI, but not its influence, v2, July 16, 2026.