Skip to content

Preparing Your Reasons With AI Before a Discussion

In a short online policy experiment, AI-guided reflection improved the later human discussion's model-rated quality.

You can ask AI to help you make a case before a difficult discussion. You can also ask it to examine what that case depends on. The second request has a different audience in mind: people who will need to question your reasoning, not just hear your conclusion.

An October 27 preprint reports a randomized experiment with 1,173 U.S. participants. After policy briefings, they spent 7 minutes using a reflection chatbot, following matching written reflection prompts or watching an unrelated video. Both reflection formats addressed views, knowledge gaps and opposing reasoning. 3-person groups from the same condition then discussed the policy for 10 minutes without AI.

On a model-scored index assessing reasons, respect and engagement, AI-prepared groups beat written reflection by 0.27 standard deviations and video by 0.31—modest average shifts. A standard deviation describes the spread of scores. Giving each member equal weight made the written-reflection comparison statistically inconclusive.

That is a reason to take private preparation seriously as part of a human discussion, not evidence that a chatbot should supply the winning argument. A contribution can be useful because it exposes a condition someone else can dispute. It need not settle the issue.

Put the assumption on the table

Consider a hypothetical discussion about changing a team’s working hours. Someone favors the change because it would make coordination easier. That gives colleagues a conclusion and a broad reason, but not much to examine. Which tasks need people to be available together? Which teams would gain overlap, and which would lose it?

These questions develop the argument rather than merely make it sound more persuasive. If the proposal depends on shared hours for a particular handoff, someone who performs that work can challenge the assumption. If the real disagreement is about whose schedule should take priority, more information about overlap may not resolve it. The conversation needs room for both factual questions and competing priorities.

The written comparison matters because simply setting aside time to think is an alternative to adding a chatbot. A fixed prompt leaves the person to decide which part needs more work. A responsive conversation could dwell on an assumption or connect a general principle to a practical consequence. Those are plausible advantages, not ingredients this comparison isolated.

The cost is that an examination can leave preparation unfinished. A missing fact may require a search or a question for a colleague. That need not be a failure: arriving with a precise uncertainty can be more useful than treating an assumption as settled. But it is not the same service as getting a polished statement ready to send.

When wording is the obstacle

There is also a legitimate job for help with wording. In a separate online gun-policy experiment, people who disagreed were paired for a chat. One participant in treated pairs received optional rephrasings: they could choose among 3 suggestions, edit a message or send the original. Suggested alternatives emphasized restatement, acknowledgment or politeness.

Participants accepted suggestions about 2/3 of the time. Partners of those receiving at least 1 rephrasing reported better experiences of being understood and respected. Chats reached their intended length for only 44% of participants. This was assistance during an exchange, not a test of private reflection or a comparison showing which approach is better.

The two forms of help need not be rivals. In the imagined working-hours discussion, a person could have a clear account of the coordination problem and still struggle to express it respectfully. Or they could have a courteous opening that hides an unsupported assumption. Examining the reason and revising its wording address different difficulties, even when both belong to the same conversation.

A knowledge gap is not yet a new position

One possible account is that reflection helps by exposing incomplete understanding. In 2013 experiments, asking people to explain how policies worked lowered their reported understanding and moderated their positions; listing reasons for a position did not produce the same moderation.

But 2 preregistered close replications reported in 2021 reproduced the decline in reported understanding without a statistically reliable moderation effect. That does not prove the original effect impossible. It does block treating recognized uncertainty as a dependable route to changed political views.

Recognizing a gap and changing a view are separate steps. Someone might discover that the coordination argument needs evidence while continuing to favor different hours for another reason. Or they might find the evidence convincing and still disagree about the burden on colleagues. Private examination is not a requirement to abandon a position or meet an opponent halfway.

The recent experiment used a specially instructed chatbot; it did not test ordinary assistant use or lasting workplace effects. Its score does not establish a correct collective decision. This was not a test of preparing only 1 member.

For the hypothetical team, the practical consequence is a different contribution to bring into the room. Instead of repeating that new hours would improve coordination, the proposer could explain which handoff depends on overlap—and leave colleagues able to challenge that account with their own experience. The discussion would still have work to do, but its next question would be clearer.

Sources

Understand How People and Agents Work Together.

Practical essays, experiments, and lessons from building software for people and agents.

Occasional emails, sent when there is something worth reading. Free to subscribe. Unsubscribe at any time.