Counting the Dollars Behind AI Labels
When AI labels grant descriptions to produce a dollar total, auditing the estimate and settling which purposes count as research are separate jobs.
How much funding supports university research? An AI can read grant descriptions, assign a category and add the amounts. Before relying on its answer, someone needs to check both the money and what the category means. An accurate-looking total does not settle both questions.
A November 9 preprint illustrates the difference. Eight prediction methods assigned $11.2–15.0 billion to research purposes. Correcting their totals against the same person’s labels narrowed the estimates to $11.2–11.6 billion. The target was grants reported as paid in tax years 2020–2024, excluding a separately coded capital category such as research buildings and facilities.
For someone commissioning an AI funding analysis, that convergence is useful. It means quite different machine predictions can lead to similar audited estimates. It does not mean the machines have dependable labels for every grant, or that another person would agree about which descriptions count as research. Auditing the total and settling that boundary are separate jobs.
What one label carries
Consider a hypothetical ledger in which 100 grants share one description. If an analyst labels that phrase once and applies the answer to every linked grant, reviewing the phrase can change all 100 entries. Reading another copy of it does not provide an independent check on the interpretation.
The dollars matter as well as the repetitions. Two wrong labels can count as two errors while moving very different amounts of money. If the deliverable is a funding total, a mistake on one large grant could matter more than many mistakes on small ones. If the deliverable is a searchable list, the individual entries still need to be dependable.
The preprint’s audit used 651 descriptions read independently by 3 people applying the same written classification rules. Its sample grouped texts by machine votes and drew in proportion to their reported dollars, including commitments, with some texts included automatically. The corrections accounted for how each description was sampled before adjusting the predicted totals.
A corrected total can therefore be useful without repairing every entry. In an imagined ledger, two wrong labels might move equal amounts in opposite directions. Their effects would cancel in the sum, but both labels would remain wrong. Agreement on the total is not a substitute for checking the list when the list itself will guide someone’s work.
A different reader, a different total
The analysis was also rerun with its model fitted to other readers’ labels. Readers 2 and 3 produced estimates of $10.12 billion and $14.00 billion, with ranges for sampling uncertainty that did not overlap. The readers’ differences had not been reconciled.
This is a different comparison from correcting several machines against one person. Each range describes uncertainty from sampling around a chosen reading. It does not decide whose interpretation should govern the report. More precise sampling could sharpen either estimate without resolving the disagreement between them.
Nor does disagreement make every reading equally defensible. Someone may have applied the rules incorrectly. The larger estimate is not automatically a better recovery of overlooked research funding. Examining the disputed decisions is necessary to distinguish a mistake in applying the rule from a boundary the rule leaves unsettled.
What the written evidence permits
Grant analysis has to make these choices before choosing an AI. The IRS’s 2024 instructions for private-foundation returns require a purpose more specific than broad labels such as educational or scientific. They also separate grants paid during the year from those approved for future payment.
Even “paid” needs care in a combined database. The report underlying the preprint explains that private-foundation returns report payments while public-charity returns follow the filer’s accounting method, which may recognize a multi-year grant when made. Their combined amounts are not a pure cash-payment series. And a stated research purpose does not show how much was actually spent on research.
Candid’s grants database offers another example of an evidence choice. Its coding procedure combines rules, machine learning and staff indexing. Staff need textual evidence for a code. When a grant description is too vague to code, Candid applies the recipient organization’s codes instead, leaving a category blank when no recipient code exists for it. This is Candid’s stated procedure, not a test of the preprint’s method.
Recipient information can help someone discover grants. But it is not the same evidence as a grant’s stated purpose. Before asking an AI to fill that blank, an analyst must decide whether the intended result permits the substitution. Otherwise, a more complete-looking database may quietly answer a different question.
The preprint does not establish performance on a fresh audit. Its practical value here is to make the review assignment more specific. Before a funding total guides a decision, its recipient needs to know which reported amounts it includes, what evidence qualifies a grant and whose reading defines inclusion. The next reviewer can then examine an ambiguous description with every grant affected by the answer kept in view.
Sources
- George Koomullil, Counting Under Disagreement, November 9, 2026 preprint, version 1.
- George Koomullil, Who Funds University Research?, version 2026.10, October 2026, sections 2.2 and 3.3 and Appendix A. Companion report describing the same database, not independent confirmation.
- Internal Revenue Service, 2024 Instructions for Form 990-PF, Part XIV, page 36.
- Candid, Grants data fact sheet, undated documentation.