{"id":"13a1ad03-5860-4e8e-ad93-862dd040226e","arxiv_id":"2505.08133","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Across 40,514 federal grant notices from 2009 to 2024, only nine include AI-specific review criteria or restrictions, even as agencies increasingly promote AI in grant narratives.","lead":"Federal agencies rarely attach AI-specific rules to their grants, even when funding AI in sensitive areas like policing and health care. This paper maps that gap with a new dataset of more than 40,000 grant notices and argues that grantmaking is an overlooked lever of AI governance.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'only nine' count rests on a 61% full-text corpus and a narrow keyword list; missing NOFOs could contain additional AI-specific conditions.","rationale":"The reader's weakest assumption identifies the same data-coverage issue, and I agree it is the load-bearing point. The finding is an absolute near-zero count, so even a modest number of missed conditions in the missing 39% of notices would change the headline number, even if the qualitative conclusion that conditions are rare survives. That said, the paper's limitations section is candid, the manual coding is documented, and the claim is explicitly about NOFO text rather than all possible post-award conditions. The concern does not require rejection; the appropriate remedy is dataset release plus a coverage-weighted robustness check. I therefore keep the reader's CONDITIONAL verdict rather than moving to ACCEPT or REJECT.","tokens_in":24139,"tokens_out":7783,"duration_ms":79671,"concrete_test":"Retrieve the full text for the roughly 25,876 NOFOs in the filtered 2009-2024 non-defense discretionary universe that lack Grants.gov attachments, using agency websites and archive.org; run the same Table 1 keyword search and manual coding on this complement; then recompute the count of AI-specific review criteria and restrictions. If the recomputed count remains near nine, the conclusion is robust; if it increases substantially, the headline must be re-scoped or the dataset completed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is the near-absence result in Section 4.4: only nine of 407 AI-related NOFOs set AI-specific review criteria or restrictions. For this claim to hold as a statement about federal grantmaking, the 40,514 full-text NOFOs must be representative of all non-defense discretionary notices. That condition is not secure. The funnel in Section 3.1 shows 66,390 notices after exclusions and date filters, but only 40,514 had full-text attachments that could be keyword-searched; roughly 39% of that universe is absent. Section 3.4 concedes that attachment uploading is optional and that Figure 3 shows substantial agency-by-year variation in coverage. If agencies with more mature AI governance, such as NSF, NIH, or DOJ components, are also the ones that host full NOFOs elsewhere, the missing documents could be disproportionately likely to contain AI conditions, pushing the count above nine. A second compounding gap is the keyword list in Table 1, which is taken from OMB memos and omits terms such as 'facial recognition', 'predictive algorithm', 'automated decision-making', and 'computer vision'; conditions written in those terms would not be retrieved. The paper acknowledges both limitations, but the headline is an absolute count, and the direction of the resulting bias is unknown without inspecting the missing records.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper examines how U.S. federal agencies govern grantee use of AI through discretionary grant Notices of Funding Opportunity (NOFOs). The authors assemble 40,514 full-text NOFOs from Grants.gov for 2009–2024, keyword-search them using terms from OMB AI guidance, manually review 407 AI-related notices, and classify them as direct or indirect AI funding and as containing or not containing AI-specific review criteria or restrictions. They find that agencies promote AI in program descriptions and review criteria but that only nine opportunities in the dataset impose AI-specific conditions; this pattern is argued to persist even in rights-impacting contexts such as law enforcement, education, and health care. The paper draws lessons from AI procurement scholarship and discusses grant-specific challenges. The authors acknowledge limitations in NOFO coverage and keyword recall.","tokens_in":24329,"tokens_out":5499,"duration_ms":54613,"significance":"If the empirical findings hold up after robustness checks, the paper makes a valuable contribution by identifying federal discretionary grantmaking as an understudied site of AI governance and by introducing a novel, manually reviewed corpus of 40,514 NOFOs. The comparison with procurement governance is apt, and the paper is honest about the major limitations of data coverage and keyword scope. The claim that agencies rarely impose AI-specific conditions, even while promoting AI in program narratives, is a falsifiable and policy-relevant finding. The paper also provides a useful methodological demonstration that Grants.gov full-text notices capture AI-related grant activity that spending summaries and NOFO metadata miss.","major_comments":[{"comment":"The funnel in Figure 1 and the coverage table in Figure 3 show that only 40,514 of 66,390 eligible notices (61%) had full-text attachments that could be keyword-searched, with substantial agency-by-year variation and the complete absence of NSF and NIH (Footnote 4). Because the paper's headline claim is the absolute count of nine AI-specific conditions, this missing 39% is load-bearing: if agencies with more mature AI-grant governance are also those that host full notices off Grants.gov, the count could be materially higher. I request a sensitivity analysis that searches at least the Grants.gov summary metadata (and, where feasible, agency-hosted full texts) for the missing records, or that restricts every generalization in the abstract and Section 5 to 'notices in our dataset with full-text attachments.'","section":"§3.1, Fig. 1, §3.4, Fig. 3"},{"comment":"The keyword list in Table 1, drawn from OMB memos, omits common AI-governance terms such as 'facial recognition,' 'predictive algorithm,' 'automated decision-making,' and 'computer vision,' and the paper itself notes in Section 3.4 that terminology changes over time. Since the count of nine conditions is produced by keyword screening, the authors should validate recall, for example by running an expanded term list on a random sample of the 40,514 full-text NOFOs and reporting how many additional AI-related opportunities and AI-specific conditions are found; without such a check, the near-absence result could reflect search misses rather than agency practice.","section":"§3.2, Table 1"},{"comment":"The abstract and Section 5 claim that the near-absence of AI-specific conditions 'holds even when agencies fund AI uses in contexts affecting people's rights,' but Section 4.4 includes a 2018 NIJ opportunity on AI tools to combat human trafficking that requires AI prototypes to be delivered for third-party auditing. NIJ appears in Section 4.3 as a law-enforcement-context funder, so either this example is itself an AI-specific condition in a rights-impacting context, or the context coding needs clarification; the claim should be reconciled with the enumerated nine.","section":"§4.4 vs. Abstract"},{"comment":"The manuscript reports that exactly nine opportunities contain AI-specific review criteria or restrictions but provides no table that lists these nine with NOFO identifiers, agencies, years, and coded condition types. Because this count is the central empirical result and Section 4 notes that the boundary between 'conditions' and 'considerations' is contested, a transparent enumeration is necessary for readers to verify the classification and for future work to build on it.","section":"§4.4"}],"minor_comments":[{"comment":"Figure 1 labels the date filter as '2009–2025' while Section 3.1 states 2009–2024; please align these labels.","section":"Fig. 1 and §3.1"},{"comment":"The reference list contains a broken citation 'citejegedeChallengeAcceptedCritique2023' in the sentence about Jegede et al.; please fix the citation.","section":"§5.3"},{"comment":"Table 2 lists 'USDOT United States Department of the Treasury'; if this entry is intended to be the Department of the Treasury, the abbreviation is confusing given that DOT already appears, and it should be clarified or corrected.","section":"Table 2"},{"comment":"Section 3.3 does not report whether manual coding was done by one or multiple coders or how disagreements were resolved; adding this procedural detail would strengthen confidence in the counts.","section":"§3.3"},{"comment":"The dataset is described as available 'upon request from the first author'; for a computational social-science contribution, a public repository with code and coded NOFO identifiers would improve reproducibility.","section":"§3.1"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is a genuinely new empirical result, not a rehash. Bateyko and Levy assembled a 40,514-NOFO corpus from Grants.gov (2009–2024), keyword-searched with terms from OMB memos, hand-coded 407 AI-related notices, and found only nine with AI-specific review criteria or restrictions. That near-absence, alongside the promotion of AI in notice narratives, is a previously overlooked governance lever. The paper also shows that NOFO summaries and USASpending records badly underreport AI activity — only 17% of summaries contained an AI keyword — which is a secondary finding worth citing on its own.\n\nCredit where due: the method is built on earlier single-agency thematic analyses (Lee-Easton et al.; Arkhurst and Green Williams), but scaling to a cross-agency corpus is the contribution. The paper is unusually honest about its own limits — Section 3.4 and Appendix B are transparent about uneven attachment coverage, the narrow keyword list, and the absence of NSF and NIH. That candor makes the work credible.\n\nThe soft spots: the central count rests on a 61% full-text corpus, and the remaining 39% could in principle contain more AI-specific conditions, particularly from agencies like NSF/NIH that host their own clearinghouses. The keyword list also omits phrases like \"facial recognition\" and \"automated decision-making,\" so some conditions could be hiding in plain sight. Both gaps are acknowledged, but the \"only nine\" headline is an absolute count and the direction of bias is unknown. A serious revision should (a) release the dataset, (b) add a robustness check using a broader keyword list, and (c) frame the headline as \"in the notices we could retrieve\" rather than a general claim about federal grantmaking. These are fixable; they don't undermine the paper's core qualitative finding that agencies rarely use conditions to govern AI. The manual coding has no reported inter-rater reliability, which is a standard weakness but minor given the inductive nature of the task.\n\nWho this is for: AI governance scholars, public administration researchers, and anyone working on civil rights oversight of federal funds. It deserves a serious referee. I'd send it out, with the expectation of a conditional accept after the data and robustness issues are addressed.","headline":"A solid, transparent empirical mapping of AI governance in federal grant notices; the 'only nine' count is real for the corpus but cannot carry the full weight of the authors' general claim, so the paper needs a light revision rather than a rewrite.","tokens_in":24909,"tokens_out":2694,"would_cite":true,"duration_ms":25985,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Federal agencies almost never attach AI-specific conditions to grants, a review of 40,514 notices finds, even when the funded AI touches people's rights.","keywords":["AI governance","federal grants","grant policy","Notice of Funding Opportunity","discretionary grants","grant conditions","procurement analogy","algorithmic oversight"],"falsifier":"Inspect the grant announcements hosted on the clearinghouses of the major research funders absent from the dataset, or the other roughly 39% of notices that lack full-text attachments, and count how many impose AI-specific review criteria or conditions; if a substantial number do, the 'only nine' finding and the accompanying claim of general silence would weaken.","tokens_in":23875,"feed_emoji":"🏛️","tokens_out":4931,"duration_ms":49656,"temperature":0.7,"pith_summary":"This paper tries to establish that the U.S. federal government's discretionary grantmaking is an overlooked but powerful lever for governing artificial intelligence, and that agencies currently pull that lever almost nowhere. Analyzing more than 40,000 non-defense grant notices posted between 2009 and 2024, the authors find that agencies often promote AI in their grant narratives, encouraging grantees to use the technology, yet only nine grant programs in the entire dataset impose AI-specific review criteria or conditions. That silence persists even in contexts that affect people's rights, such as law enforcement, education, and healthcare, where a comparable federal procurement regime would trigger extra oversight. The paper matters because if true, it means billions of dollars in federal assistance enable AI uses with little pre-award planning or guardrails, leaving agencies unprepared to prevent or correct misuse after the money is out.","feed_headline":"Only nine of 40,000 grant notices set AI rules","feed_subtitle":"Agencies promote AI in grant announcements but almost never attach conditions, even in rights-impacting fields.","key_machinery":"The central object is the Notice of Funding Opportunity (NOFO), the public document through which agencies announce discretionary grants and set program objectives, judging criteria, restrictions, and eligibility. The paper treats NOFOs as a policy instrument: because agencies can rarely add rules after an award begins, whatever conditions appear in the notice are the main pre-award lever for shaping how grantees use AI. The analysis works by collecting NOFO full texts from the federal grants website, searching them for AI keywords derived from OMB guidance, manually reviewing the 633 matches to discard boilerplate references, and inductively coding the remaining 407 opportunities into categories of direct funding, indirect encouragement, rights-impacting contexts, and explicit conditions.","core_discovery":"The paper's central discovery is that federal agencies rarely use the conditions available to them in discretionary grants to govern grantees' use of AI. Of the 407 grant opportunities that mention AI in a meaningful way, only nine establish AI-specific review criteria or restrictions, and those are mostly broad disclosure requirements, discouragements, or outright bans rather than domain-tuned oversight. The authors find this silence holds even among grants funding AI in contexts that OMB's own guidance flags as high-impact for civil rights and safety, such as recidivism prediction, student monitoring, and HIV-risk identification. They further show that NOFO narratives promote AI in ways that official spending records and summary listings miss: only 17% of Grants.gov summaries for these AI-related notices contain an AI keyword, and about a third of corresponding spending records do not mention AI at all, suggesting the real federal AI footprint is underreported.","pith_inferences":["Beyond the paper: the same near-silence likely extends to state and local grantmaking, and to other financial assistance instruments like prize competitions, where agencies have even fewer pre-award conditions; a state-level search for AI-specific grant conditions could test this.","Beyond the paper: because the dataset underrepresents major research funders that host their own clearinghouses, a fuller count of AI conditions might exceed nine, but the paper's qualitative evidence suggests the increase would be small and unlikely to overturn the claim of general silence.","Beyond the paper: the paper's keyword approach could be adapted as a low-cost screening tool for agencies themselves, letting them audit their own NOFOs for AI mentions and conditions before public release, turning the research method into an internal governance practice.","Beyond the paper: if agencies adopted disclosure-oriented conditions, modeled on the few examples the paper finds, they would create a public record of what AI uses grantees actually deploy, enabling the transparency that current spending records do not provide."],"forward_implications":["If the finding holds, federal agencies are currently funding AI adoption on a large scale without corresponding pre-award planning, meaning the HUD surveillance-camera episode is a systemic risk, not an isolated failure.","The near-absence of AI-specific conditions implies that agencies are failing to exercise the one control they retain after awards are made, since they rarely add rules mid-award; applicants have little notice of what responsible AI use requires.","The underreporting of AI in NOFO summaries and USASpending records suggests that existing measures of federal AI funding, which rely on those records, systematically undercount the true footprint of government-supported AI activity.","The paper's comparison with procurement indicates that if agencies began applying procurement-style AI oversight to grants, they would need new capacity, such as AI-literate review panels and monitoring practices, because grant offices are not currently staffed for that role.","A shift in administrative policy toward AI conditions, such as a future OMB memo covering grants, could quickly change the observed pattern, making the current silence a contingent policy choice rather than a fixed feature of grantmaking."],"supporting_citations":[{"why":"Supplies the overview of grant conditions (administrative versus programmatic) that frames the paper's focus on programmatic conditions.","marker":"[75]"},{"why":"OMB's M-24-10 memo that exempts grants from mandatory AI risk management, establishing the policy gap the paper investigates.","marker":"[92]"},{"why":"Current OMB AI guidance that continues not to apply to grant programs, showing the silence persists across administrations.","marker":"[88]"},{"why":"The procurement-as-policy argument from which the paper draws its main analogy for how grant conditions could govern AI.","marker":"[64]"},{"why":"Empirical finding that agencies fail to inventory their own machine-learning uses, lending support to the paper's argument about underreporting and limited oversight capacity.","marker":"[58]"},{"why":"Shows law enforcement grants funded surveillance equipment without guidance, providing a precedent for the consequences of condition-free grantmaking.","marker":"[46]"},{"why":"The regulation defining the required contents of NOFOs, which grounds the paper's method for extracting program description, review criteria, restrictions, and eligibility.","marker":"[18]"},{"why":"The HUD surveillance-camera episode with which the paper opens, motivating why missing AI conditions matter in practice.","marker":"[61]"}],"fun_headline_variants":["Only 9 of 40,000 grant notices set AI rules","Agencies push AI in grants but set few rules","Grant notices reveal silent AI oversight gap","Federal grants fund AI but skip conditions","AI in grants: 9 rule-setters out of 40,000"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The 40,514 notices with full text attached are representative of all federal discretionary grantmaking, even though attachment coverage is uneven and major research funders that host their own clearinghouses are absent from the dataset.","fun_headline_variants_meta":{"raw":{"variants":["Only 9 of 40,000 grant notices set AI rules","Agencies push AI in grants but set few rules","Grant notices reveal silent AI oversight gap","Federal grants fund AI but skip conditions","AI in grants: 9 rule-setters out of 40,000"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00019,"raw_usage":{"total_tokens":1379,"prompt_tokens":1027,"completion_tokens":352,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":643,"completion_tokens_details":{"reasoning_tokens":273}},"tokens_in":643,"tokens_out":352,"duration_ms":3955,"temperature":1.0,"reasoning_tokens":273,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:01:59.394058+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the grant announcements hosted on the clearinghouses of the major research funders absent from the dataset, or the other roughly 39% of notices that lack full-text attachments, and count how many impose AI-specific review criteria or conditions; if a substantial number do, the 'only nine' finding and the accompanying claim of general silence would weaken.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the overview of grant conditions (administrative versus programmatic) that frames the paper's focus on programmatic conditions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"OMB's M-24-10 memo that exempts grants from mandatory AI risk management, establishing the policy gap the paper investigates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Current OMB AI guidance that continues not to apply to grant programs, showing the silence persists across administrations."},{"cited_title":"Mulligan and Kenneth A","cited_arxiv_id":null,"evidence_quote":"The procurement-as-policy argument from which the paper draws its main analogy for how grant conditions could govern AI."},{"cited_title":"2 CFR § 200.204","cited_arxiv_id":null,"evidence_quote":"The regulation defining the required contents of NOFOs, which grounds the paper's method for extracting program description, review criteria, restrictions, and eligibility."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The HUD surveillance-camera episode with which the paper opens, motivating why missing AI conditions matter in practice."}],"review_version":1}