Skip to content

Deciding What to Tell a Helpful Chatbot

When an AI assistant helps you prepare a difficult conversation, a useful reply does not by itself justify sharing more personal context or keeping the full exchange.

AI companions are chatbots people use for ongoing personal conversation. One companion-AI user had a rule for discussing other people: “Don’t say anything you would be afraid of getting back to them.”

How much of a difficult conversation does an AI assistant need to hear before it can help you reply? The answer depends on the reply you want, not simply on how comfortable the exchange feels. Choosing useful context and choosing what may be kept are separate parts of the task.

An October 28 preprint interviewed 21 U.S. adults selected for perceived chatbot empathy. Almost all reported more sharing in empathic exchanges; most distinguished simulated empathy from genuine caring. Retrospective accounts establish neither causation nor population prevalence.

For everyday AI work, the practical issue is not whether to provide something or nothing. It is what this particular request needs. A draft for a tense meeting, for instance, has a different purpose from a record of everything that made the meeting tense.

Work backward from the reply

Suppose you want wording for asking a client to extend a deadline. The useful ingredients might be the date, why the extra time is needed and how much room there is to negotiate. The client’s name might contribute nothing to that wording. This is a hypothetical example, not a reported use case.

If the question instead concerns how to answer a specific accusation, the sequence of messages may matter. Removing every uncomfortable fact could leave the assistant solving a different problem. That is a cost of withholding, not an argument that the full account belongs in the chat.

The distinction is between information needed for the help you want and information included because you are telling the whole story. It takes work to separate them. There may also be requests whose necessary details you are not willing—or entitled—to give that service.

This is a proposed way to frame a disclosure decision, not a demonstrated privacy remedy. Removing a name does not establish that the remaining account is unidentifiable. Nor does deciding a detail is useful establish permission to share information about somebody else.

The opening interview came from a 15-person companion-AI study. Participants also described withholding names or photos; some valued preserving conversations. These accounts do not establish that their precautions worked.

Withholding and keeping are therefore not opposites. A person might omit names yet want to return to a discussion. Another might want a useful sentence without an ongoing record of the problem. Those possibilities call for different controls, even if both people ask for the same kind of help.

Which part needs to last?

OpenAI’s current Temporary Chat documentation describes a concrete combination: a temporary conversation can use existing personalization without creating memories, and is not used for model improvement while temporary. OpenAI may retain a safety copy for up to 30 days. Saving converts it to a regular chat governed by account settings. Third-party action recipients follow their own policies. This is vendor documentation, not an independent audit.

For the person preparing that deadline request, success might mean leaving with a short message they can send. That does not require wanting the assistant to draw on the underlying dispute later. Conversely, somebody working through an ongoing problem might deliberately prefer continuity. Neither preference is answered by whether the response sounds sympathetic.

A control is useful only in relation to the job you expect it to do. Beginning without personal context, preventing future conversational recall and limiting a recipient’s retention are different aims. Before relying on a setting, the behavior that matters needs to be clear. A pleasant exchange cannot establish it.

The draft asking for more time is an output; the account of the dispute is an input. They may be worth keeping for different reasons. If the useful thing is a paragraph you can send, preserving that paragraph need not mean preserving the entire conversation that produced it. The background deserves its own decision about a lasting record.

Sources

Understand How People and Agents Work Together.

Practical essays, experiments, and lessons from building software for people and agents.

Occasional emails, sent when there is something worth reading. Free to subscribe. Unsubscribe at any time.