Skip to content

Choosing AI's Part in a SQL Repair

When using AI to repair a database query, choosing an explanation, a check of a suspected bug or a replacement query changes what the person needs to verify next.

One participant in a new study of database-query repair came to ChatGPT with a specific suspicion. The query was supposed to find customers whose consumption increased over consecutive months. Was it comparing each month with the starting month, the participant asked, rather than with the previous one?

That question assigns the assistant a different job from “fix this query.” A check of a suspected bug can help someone decide what to change; a replacement supplies a whole candidate solution to evaluate. Choosing between them changes the evidence needed next. For this particular suspicion, a tiny example can distinguish the competing rules more clearly than a reassuring answer.

The November 11 preprint followed 16 people assigned 4 designed debugging tasks using SQL, the language for querying databases. ChatGPT and search were available for 3 assignments, and search alone for 1. Their requests show different ways to divide a repair, not proof that one division works best.

Three months that separate two rules

Consider a hypothetical customer using 10 units in January, 15 in February and 12 in March. Both later amounts exceed January’s 10. Consumption does not increase every month: it falls between February and March.

The study’s growth task required growth across at least 3 consecutive calendar months. Its faulty query kept the initial consumption as the comparison value. On the hypothetical sequence, that distinction matters. A check against the starting value accepts both later months. A check against the previous value rejects the final step.

The query was recursive: it built on its own earlier results. PostgreSQL’s documentation describes how those results feed successive steps. In this calculation, each step needs a value to compare against. Keeping the first amount and carrying forward the latest amount encode different requirements, even if most of the query stays unchanged.

We ran a separate, small SQLite illustration of that comparison. Carrying the initial amount accepted both 10 → 15 → 12 and 10 → 15 → 20. Carrying the latest amount accepted only the steadily rising sequence. This was a check of the two comparison rules, not execution of the study’s exact PostgreSQL query or certification of a complete repair.

A steadily rising example alone would not reveal the difference. Both rules accept 10 → 15 → 20. The falling final month is what gives the first example diagnostic value. It turns an abstract suspicion into a result the query should reject.

That still leaves plenty untested: missing months, duplicate records and a change of calendar year, for example. Showing why the old rule fails is not the same as showing that a replacement meets the full specification. The small example earns a narrower role. It makes one necessary change visible.

What a replacement puts on the table

The study also records someone asking for an explanation without changes, and another participant requesting diagnostic test cases. These are separate requests, not successive steps in one person’s repair. Elsewhere, AI candidates led participants to evaluate a solution before modifying or adopting it.

Each request produces a different starting point for review. An explanation needs to describe the existing query faithfully. A proposed test needs an expected answer and a reason it would expose the suspected error. A replacement needs checking as a candidate implementation of the requirement, including choices the person did not propose.

There are good reasons to ask for that larger contribution. Someone facing an unfamiliar query construction may need a candidate before they can usefully trace the logic. Under a deadline, obtaining a workable repair may matter more than devising it personally. These are practical reasons to consider fuller assistance, not outcomes established by the study.

Nor is a smaller request automatically better. Repeated hints can mean more exchanges, and withholding a requirement can make an answer impossible to judge. An assistant checking the monthly-growth suspicion may need the intended rule, table structure and relevant data even if the person wants to write the eventual fix. Providing context and assigning the repair are separate decisions.

Query checking already had this problem before chat assistants. In 2020 interviews with 20 database users, researchers described breaking queries into components and trying small, targeted inputs. A separate 2021 prototype, Debug-It-Yourself, offered sample data and explanations for inspecting generated queries. These earlier tools and accounts show ways to make behavior inspectable. They do not confirm the effectiveness of a particular chat strategy.

The new study likewise does not establish durable understanding or the best workflow for production systems. Its participants worked on short, constructed tasks. What it offers is a concrete view of people asking AI to do different parts of a repair.

For the monthly-growth query, the useful handoff could include the 3 values and the expected rejection alongside the proposed SQL. Whether the assistant explains, diagnoses or rewrites, the next reviewer would then have something specific to run: a customer who stays above January’s consumption but should not qualify as steadily growing.

Sources

Understand How People and Agents Work Together.

Practical essays, experiments, and lessons from building software for people and agents.

Occasional emails, sent when there is something worth reading. Free to subscribe. Unsubscribe at any time.