Skip to content

What an AI Tutor Asks Next

In one optional module about AI, tutors built on partial answers while fewer students assigned dialogue completed the study than those assigned written responses.

In a published account of a bilingual voice conversation, a learner was asked how a digital service uses data. The answer named a process: “Analyzing the information.”

The AI tutor agreed that analysis mattered. Then it asked what the analysis was for. How did it help a service make decisions or offer advice? Think about YouTube, it suggested. The learner supplied another part of the process: collecting user data.

That still left a connection to explain. Collection supplies something to analyze, but what does the service learn from it? The tutor asked again. When the learner hesitated, it offered a hint that brought the two earlier contributions together. The authors report a later, more specific answer, but the published excerpt stops at the hint.

The exchange comes from an optional module about AI for Kannada-speaking undergraduates in Karnataka, India. It shows a useful thing a tutor can do with an incomplete answer: find the next question inside it. The learner’s contribution gave the tutor something to work with; the tutor’s response gave the learner something further to explain.

The field-study preprint compared written responses with text and voice dialogue. More students assigned written responses completed the study. Among those who finished, dialogue participants contributed more and reported greater interest and confidence in understanding AI. The study does not establish why fewer dialogue participants finished.

For someone studying with an AI tutor, both parts matter. What is the next question helping you understand? And does the activity around those questions get finished? The encouraging exchange cannot answer the second question on its own.

Keeping the question open

The researchers randomly assigned 305 students to write responses, type with a tutor or speak with one. Each format was offered in English or a bilingual Kannada–English version. Students watched the same four short instructional videos and worked on the corresponding exercises, mainly at home on smartphones. Completing the study earned a certificate; participation was optional and did not affect their grades.

Written responses also received AI feedback, on a different schedule. Written participants received comments 12 hours after submitting, while dialogue participants received them immediately after a conversation. Both could retry. The experiment therefore compared combinations of interaction and feedback timing. It did not isolate the effect of having a conversation.

The opening exchange makes the interaction concrete. The tutor did more than mark an answer right or wrong. It acknowledged a relevant contribution, asked for its purpose and, after another partial answer, supplied a clue. The unfinished explanation remained the subject of the conversation.

That invitation can be welcome. A different participant, interviewed after completing the module, credited the tutor’s reassurance with giving them confidence when they did not know an answer. Another question can come with help, rather than simply leaving someone to struggle alone.

But a follow-up can also take the explanation off course. In a separate English voice exchange, a student tried to explain that Google readily answers questions. The tutor mistook this for a request to be given an answer and explained its teaching role. The student tried to clarify. The tutor repeated that it was there to help the student learn.

The learner now had two jobs: explain the digital service and get the tutor to recognize that this was already an explanation. In the data-use exchange, the next question concerned the missing connection. Here, the student first had to repair the tutor’s understanding of what they meant. Neither excerpt establishes its student’s later test score or whether they completed the study.

Who reached the later test

Across the full enrollment, 72 of 105 written-response students completed the study, compared with 56 of 107 in text dialogue and 45 of 93 in voice dialogue. These counts concern finishing the study rather than resolving one conversation.

An institutional interruption mattered. The first deployment’s community computer centers permanently closed after unexpected funding cuts. Yet the later college deployment showed the same direction: all 51 written-response students finished, compared with 44 of 53 text-dialogue students and 27 of 34 voice-dialogue students. The closure cannot explain the entire pattern. The paper does not establish what accounts for the remaining difference.

Most learning and engagement results concern the 173 finishers. Among them, dialogue participants produced more words and had longer logged activity times. The time logs include inactivity. Neither measure directly tells us how much thought occurred, and neither describes the experience of everyone who enrolled.

The knowledge tests found no reliable advantage for one format. That does not establish equal learning, and the later outcomes do not recover what the missing students knew. Random assignment at enrollment cannot make the selected groups of finishers automatically comparable.

The result is a divided view of the module: fewer students assigned dialogue finished, while the dialogue finishers reported a more favorable experience. The data cannot turn the helpful hint into a reason someone stayed, or the misunderstanding into a reason someone left. The work visible inside an exchange and the reasons for completing the study remain separate questions.

Where the exchange should lead

Outside this assigned module, learners choose AI help for different purposes. An older survey at an Israeli university gives a concrete contrast. One student described asking for simple applications of course material to help tackle harder homework. Another described using ChatGPT to finish pass/fail assignments quickly. These were separate self-reported practices, not observed learning gains or explanations of the Karnataka students’ decisions.

For the first student, the reported purpose was to understand a harder problem. The next step could happen away from the chatbot. For the second, finishing the assignment quickly was itself the stated aim. The same label, homework help, covered different destinations. A long exchange is not automatically evidence that either learner is getting closer to theirs.

A practical suggestion follows: when using a tutor, ask yourself which question you want to try answering independently after the exchange. This is a suggested check, not a tested way to improve completion or learning. It gives the conversation a destination against which to judge the next prompt.

For the learner discussing data, the tutor’s last visible move returned to the fragments the learner had already supplied. Collecting information and analyzing it mattered. Now the question was what the service could do with the result.

Sources