{"id":"c30c1ed0-1c4c-4ff6-8a09-ca07ca8bb1d5","arxiv_id":"2504.13777","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The paper proposes treating AI hallucinations as a distinct category of misinformation and outlines a research agenda for communication scholars.","lead":"This essay argues that AI hallucinations are a distinct form of misinformation, not just technical errors, because they are generated without human intent. It gives communication researchers a supply-and-demand framework and a macro-meso-micro agenda for studying how these false outputs spread and persuade.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'distinct category' claim rests on a supply-side dichotomy that is not operationalized; the paper's own definition of misinformation includes unintentional errors, so lack of intent cannot carry the weight.","rationale":"The reader correctly identifies the lack-of-intent assumption as load-bearing, but the concern is sharper than 'untested theoretical commitment': the paper's own definition of misinformation includes unintentional human errors, so absence of intent is not even a candidate distinguishing feature. The framework may still be valuable as a programmatic proposal, but its central classification claim needs an operational criterion. The proposed matched-corpus annotation test would directly test whether Table 1's supply-side features can separate the two categories. This does not warrant rejection because the essay is explicitly conceptual and can be revised to state the criterion as a testable hypothesis; it also has independent empirical support from documented hallucination cases and error-rate benchmarks. The conditionality of the reader's verdict remains appropriate, possibly with this demarcation test as a required condition.","tokens_in":15066,"tokens_out":6301,"duration_ms":65876,"concrete_test":"Construct a preregistered, matched corpus: 100 verified LLM hallucinations and 100 verified human-generated misinformation items (e.g., retracted news corrections or fact-check false ratings), matched by topic, length, and plausibility. Have annotators blind to source rate each item on Table 1's supply-side features—'lacks genuine understanding,' 'unaware of knowledge limits,' 'probabilistic generation'—and on intentionality. If coders cannot classify origin above chance from these features, or if human items are rated as equally low in intentionality and awareness, the proposed demarcation does not support a distinct category.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires a workable, operationalizable demarcation between AI hallucinations and human-generated misinformation. The demarcation offered in Table 1 and in the section 'AI Hallucinations are Conceptually Different from Human Misinformation' turns on the absence of human intent and on LLMs' probabilistic generation 'without the understanding or awareness of knowledge limits.' But the paper's own working definition of misinformation explicitly covers 'both unintentional errors and deliberate deceptions' (Scheufele & Krause, 2019), so absence of intent cannot separate hallucinations from ordinary human misinformation. Human misinformation is routinely produced without awareness of knowledge limits and via statistical/heuristic processes (e.g., premature preprint reporting, sensationalized claims, confabulated memories). The paper provides no formal, behavioral, or annotation-based criterion that can place a given false utterance on the AI side of Table 1 rather than the human side. The cited statistical lower bound (Kalai & Vempala, 2024) would help only if it applies to deployed systems; the paper does not state the theorem's conditions (e.g., calibration) or address retrieval-augmented, verified, or abstaining pipelines that may make hallucinations rare in high-stakes settings. Without an operationalized supply-side criterion, 'distinct form of misinformation' is a stipulation rather than a demonstrated category.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript is a conceptual essay arguing that AI hallucinations—false or misleading outputs of large language models—should be treated as a distinct form of misinformation rather than as technical failures or as a mere subset of human misinformation. It grounds this claim in the probabilistic, non-intentional production of LLM outputs, contrasts human and AI misinformation in Table 1, introduces distributed agency as a lens, and outlines a supply-side and demand-side agenda with macro, meso, and micro levels. The essay is written for communication scholars and draws on science-communication theories and recent CS/AI literature.","tokens_in":15294,"tokens_out":5488,"duration_ms":51720,"significance":"If the distinct-category claim is accepted, the paper fills a gap in misinformation theory by giving communication scholars concepts to study non-human generators of falsehoods. Its supply/demand framing and macro-meso-micro agenda generate concrete, falsifiable research questions, and it usefully highlights that hallucinations are socially consequential, not just technical defects. The paper is transparent about its conceptual basis and does not overstate empirical certainty, though it leans on a few empirical anchors that need scrutiny. It also credits existing work and integrates relevant literature from communication, STS, and computer science.","major_comments":[{"comment":"The central claim that hallucinations are a distinct category of misinformation requires an operationalizable demarcation, but the paper does not supply one. Because the working definition of misinformation (Scheufele & Krause, 2019) explicitly includes unintentional errors, the absence of human intent cannot by itself separate AI hallucinations from human-generated misinformation, which also often arises without awareness of knowledge limits and through heuristic or statistical processes (e.g., premature preprints, sensationalized summaries). I do not see a fatal circularity in classifying hallucinations as misinformation under the paper's broad definition; the problem is the stronger claim that they are a distinct form. To make that category distinction load-bearing, the paper should specify observable criteria—annotation rules, production-process indicators, or behavioral/experimental markers—that determine when a false utterance belongs to the AI-hallucination side of Table 1, especially in the hybrid cases acknowledged by Figure 2.","section":"'AI Hallucinations are Conceptually Different from Human Misinformation', Table 1, and Figure 2"},{"comment":"The argument that hallucinations are inherent and statistically inevitable relies on a lower bound (Kalai & Vempala, 2024), but the theorem's conditions are not stated, and the cited rate estimates vary widely (5-29% in Lukens & Ali, 2023; 1.3-4.1% in Vectara, 2024). It is therefore unclear whether the lower bound applies to deployed systems with retrieval augmentation, verification, abstention, and fine-tuning, or only to an idealized next-token prediction setting. The author should specify the theorem's assumptions and explain how it licenses the claim about deployed chatbots and search tools; otherwise the 'inherent limitation' premise is stronger than the cited evidence supports.","section":"Introduction, statistical inevitability argument"},{"comment":"The agenda lists many open questions but few hypotheses or observable implications that would distinguish the AI-hallucination case from human misinformation. For a conceptual framework, this is acceptable, but the claim that hallucinations 'may also trigger these biases' (echo chambers, motivated reasoning) is not enough to establish that the category is distinct. The author should identify at least one empirical prediction that would differ between the two categories—for example, correction effectiveness, sharing patterns, or credibility judgments—so that the proposed framework is testable rather than merely descriptive.","section":"Research agenda, meso-level"}],"minor_comments":[{"comment":"The sentence 'makes them distinct from conventional of misinformation with controllable human intent' is ungrammatical; 'conventional of' should be revised.","section":"Introduction"},{"comment":"The phrase 'the estimated chance of AI hallucination is subject to a statistical lower bound' is vague; it should cite the theorem's formal statement or paraphrase its conditions.","section":"Introduction"},{"comment":"References include Pawitan & Holmes (2025), Xiong et al. (2023), and Druckman & Bolsen (2011) that do not appear to be cited in the text; either cite them or remove them.","section":"References"},{"comment":"The caption ('Hallucination vs. Human-initiated') does not explain the circular layout; please add a legend for the shading and identify the positions of the examples discussed in the text.","section":"Figure 2"},{"comment":"The Vectara (2024) 'Hallucination Leaderboard' is a non-peer-reviewed repository; describe the benchmark or treat the 1.3-4.1% range as an illustrative measurement rather than a general rate.","section":"Introduction, rate estimates"},{"comment":"There are minor typographical issues, such as a stray space in 'customer service , journalism' and 'referstointentionalfalsehoods' in the definition paragraph.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"To the editor: The paper is a timely conceptual contribution appropriate for a communication-oriented venue. The main risk is that the distinct-category claim is currently stipulative; the author should be pushed to supply a demarcation criterion and to engage with the hybrid cases in Figure 2. I would also recommend a careful check of the empirical anchors (Kalai & Vempala's assumptions; the hallucination rate range) and the reference list for uncited entries. The essay's strengths—clear framework, useful research agenda, and honest treatment of limitations—make it worth revising rather than rejecting."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Briefly: this is a well-written conceptual essay that does something useful—it imports supply-and-demand thinking and distributed agency from misinformation research and STS into the AI hallucination space, and it gives communication scholars a concrete macro-meso-micro agenda. I'd send it to a serious referee, but with revision expected.\n\nWhat's new: The paper isn't testing anything; it's a framing piece. It correctly identifies the theoretical gap Schäfer noted and offers a synthesis more specific than prior survey taxonomies: the Swiss cheese model for hallucination causes, the circular spectrum of human agency, and a research agenda with real questions across levels. For a field just starting to study AI-generated falsehoods, that is a legitimate contribution. The author is honest that this is programmatic, not empirical.\n\nWhere it gets soft: the central claim that AI hallucinations are a 'distinct form of misinformation' is not fully earned. The paper's working definition (Scheufele & Krause 2019) already includes unintentional errors, so 'lack of intent' cannot be the separating axis. The stress-test note is right: human misinformation is routinely produced without awareness of knowledge limits—preprint hype, confabulated memories, motivated reasoning. The paper gestures at distributed agency and a continuum, but Table 1 and the prose keep falling back on the intent dichotomy. Without an operationalized criterion (behavioral, annotation-based, or formal), 'distinct' is a stipulation, not a demonstration. That's fixable by softening the claim to 'distinctive in production dynamics and scale' or by offering testable hypotheses.\n\nAlso, several empirical anchors are cited but missing from the reference list: Kalai & Vempala, Lukens & Ali, IPSOS, Gallup, Vectara, and some news examples. That's sloppy for a paper leaning on 'statistical inevitability.' And the Kalai-Vempala lower bound is cited without conditions; it applies to calibrated models, not necessarily deployed ones, and doesn't cover RAG or abstention pipelines.\n\nNone of this is fatal. The framework is coherent on its own terms; the missing references and overstrong dichotomy are fixable. The author shows good command of the communication literature. I'd suggest the editor send it out—it deserves referee time, and reviewers can push on operationalization.\n\nWho it's for: communication and science-communication researchers, especially those starting work on AI misinformation. I wouldn't cite it in my own next paper until the distinct-category claim is sharpened, but I'd keep it in view.","headline":"A useful conceptual synthesis for studying AI hallucinations in communication, but the 'distinct category' claim rests on an intent axis that the paper's own definition undermines.","tokens_in":15801,"tokens_out":2314,"would_cite":false,"duration_ms":22576,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AI hallucinations should be analyzed as a distinct category of misinformation, this essay argues.","keywords":["AI hallucinations","misinformation","science communication","large language models","generative AI","distributed agency","supply and demand","public trust"],"falsifier":"A credible demonstration that an open-domain LLM can be designed or trained—without narrow benchmark constraints—to stop producing false but plausible outputs on unfamiliar facts would falsify the inevitability claim. Short of that, evidence that every observed hallucination traces to a specific human-controllable input, such as a flawed prompt or poisoned retrieval source, would weaken the intent-free and inherent distinction.","tokens_in":14855,"feed_emoji":"🤖","tokens_out":5609,"duration_ms":47714,"temperature":0.7,"pith_summary":"The essay argues that AI hallucinations—false but plausible outputs from generative language models—are not merely technical glitches but a distinct form of misinformation with social consequences. Because these systems generate text by predicting likely tokens rather than by knowing facts, their errors happen without human intent and are statistically inevitable. Communication research, the essay contends, needs its own concepts and agenda for this new source of inaccuracy, built around supply-side causes and demand-side persuasion.","feed_headline":"AI hallucinations deserve their own misinformation category","feed_subtitle":"If false AI outputs are inevitable and intent-free, communicators need new ways to study, verify, and counter them.","key_machinery":"The load-bearing device is a supply-and-demand split inherited from misinformation research, combined with a distributed-agency spectrum. Supply covers how hallucinations originate—training-data flaws, probabilistic token prediction, and imperfect fact-checking—while demand covers why audiences accept and share them, including fluency, confidence, and sycophancy. A circular spectrum of human involvement places hallucinations opposite human-initiated disinformation, clarifying that the falsehood itself forms with minimal direct human control. The essay also uses a layered Swiss cheese model to show how separate vulnerabilities align to let hallucinations through.","core_discovery":"The paper's central claim is that hallucinations produced by LLM-powered generative AI constitute a distinct category of misinformation and should be analyzed as such. The distinction rests on production: hallucinations emerge from probabilistic text generation, training-data limits, and weak downstream gatekeeping, not from a communicator's knowledge, motivation, or oversight. The essay then applies a supply-and-demand view of misinformation, adds distributed agency to locate hallucinations at the low-human-control end of a spectrum of inaccuracy, and proposes a macro–meso–micro research agenda covering institutions, group dissemination, and individual cognition.","pith_inferences":["Editorial inference: if hallucination is statistically inevitable, misinformation prevalence has a nonzero baseline supplied by AI alone; studies of information ecosystems should model this background rate rather than treating falsehood as a human-only variable.","Editorial inference: the lack-of-intent axis suggests treating intent as a continuous variable in misinformation theory, with AI hallucinations as the zero-intent endpoint, which could reorganize typologies currently split into misinformation and disinformation.","Editorial inference: a testable extension would compare matched hallucinations and human falsehoods in experiments to isolate whether fluency or perceived authority explains their persuasiveness; the essay calls for such work but does not run it."],"forward_implications":["Hallucination research becomes a communication problem, not only a machine-learning one: how false outputs are worded, perceived, and spread matters as much as how they are detected.","Existing corrections such as accuracy nudges, inoculation, and fact-checking must be retested against AI-generated content, since there is no human deceiver to blame or trace.","Institutions and platforms need new gatekeeping norms, including disclosure standards and shared definitions of acceptable error, because hallucinations cannot be fully predicted or prevented upstream.","Digital literacy efforts should shift from spotting fake news to coping with fluent, confident AI output, since fluency itself is a persuasive cue.","Media credibility research must grapple with content that has no clear human author, putting source evaluation and trust in a different register."],"supporting_citations":[{"why":"Argues hallucination is an innate limitation of LLMs, grounding the claim that errors are not occasional glitches.","marker":"Xu et al., 2025"},{"why":"Provides a statistical lower bound that makes hallucination inevitable across generative text models.","marker":"Kalai & Vempala, 2024"},{"why":"Defines LLM hallucination and links fluent, authoritative style to perceived credibility.","marker":"Zhang et al., 2023"},{"why":"Supplies the faithfulness/factualness taxonomy used to separate context errors from world-knowledge errors.","marker":"Augenstein et al., 2024"},{"why":"Gives the working definition of misinformation as content that contradicts the best available evidence.","marker":"Scheufele & Krause, 2019"},{"why":"Introduces distributed agency, the lens for locating hallucinations at the low-human-control end of a spectrum.","marker":"Rammert, 2008"},{"why":"Reports current hallucination rates, providing empirical evidence of the phenomenon's real-world frequency.","marker":"Vectara, 2024"}],"fun_headline_variants":["Hallucinations are misinformation—with no human intent","AI hallucinations: a new misinformation species","Study AI hallucinations as communication, not glitch","Why AI hallucinations need their own misinformation lens","Distinct category: AI hallucinations in communication"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework assumes that language models generate text by statistically predicting tokens without understanding or awareness of their knowledge limits, making hallucination an inherent and unavoidable feature rather than a fixable flaw.","fun_headline_variants_meta":{"raw":{"variants":["Hallucinations are misinformation—with no human intent","AI hallucinations: a new misinformation species","Study AI hallucinations as communication, not glitch","Why AI hallucinations need their own misinformation lens","Distinct category: AI hallucinations in communication"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000329,"raw_usage":{"total_tokens":1762,"prompt_tokens":796,"completion_tokens":966,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":412,"completion_tokens_details":{"reasoning_tokens":898}},"tokens_in":412,"tokens_out":966,"duration_ms":7032,"temperature":1.0,"reasoning_tokens":898,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:59:57.733829+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A credible demonstration that an open-domain LLM can be designed or trained—without narrow benchmark constraints—to stop producing false but plausible outputs on unfamiliar facts would falsify the inevitability claim. Short of that, evidence that every observed hallucination traces to a specific human-controllable input, such as a flawed prompt or poisoned retrieval source, would weaken the intent-free and inherent distinction.","supporting_citations":[],"review_version":1}