Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Truth over Tricks: Measuring and Mitigating Shortcut Learning in Misinformation Detection

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Misinformation detectors trained on static benchmarks do not learn authenticity: they learn spurious correlations with surface cues, and LLM rewriting of a text can cut their accuracy by up to 92 percent; retraining on LLM-paraphrased…

desk verdict Large, useful shortcut-injection benchmark, but the headline degradation numbers only appear under deliberately anti-correlated retraining, not on standard-trained detectors. read the letter →

arxiv 2506.02350 v1 pith:U2DYL3EJ submitted 2025-06-03 cs.CL

Herun Wan , Jiaying Wu , Minnan Luo , Zhi Zeng , Zhixiong Su This is my paper
classification cs.CL
keywords shortcutlearningmisinformationdetectionLLMinjectionattackspuriouscorrelationdataaugmentationrobustnessevaluationfakenewsrewriting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Misinformation detectors trained on static benchmarks often do not learn authenticity; they learn spurious associations between the label and surface features such as sentiment, topic, style, and language regularity. This paper sets out to measure that shortcut reliance systematically and to show it is a practical vulnerability, because large language models can rewrite misinformation to change exactly those surface features while preserving the underlying claim. The authors build TruthOverTricks, an evaluation paradigm with two axes—intrinsic shortcuts that already exist in benchmarks and extrinsic shortcuts injected by LLM prompts—and apply it to seven detectors across sixteen datasets. Under explicit LLM injections, trainable detectors lose roughly 80 percent average accuracy, with individual drops reaching 92 percent, while detectors trained on sentiment-injected data measurably acquire sentiment-classification ability. The paper then claims that a data-centric mitigation, SMF, which paraphrases, summarizes, or neutralizes texts before training and inference, consistently improves robustness across the same benchmarks.

What carries the argument

The load-bearing object is a taxonomy: intrinsic shortcut induction, which splits a benchmark so the joint distribution of the label and an indicator (sentiment, style, topic, or perplexity) is reversed between training/validation and test, and extrinsic shortcut injection, which uses Meta-Llama-3-8B-Instruct prompts to rewrite both real and fake texts with explicit attributes (sentiment, tone, word choice) or implicit author attributes (age, gender). The second load-bearing object is SMF, an LLM-augmented data augmentation framework whose three variants—Paraphrase, Summary (keep only fact-relevant content), and Neutral (overwrite a shortcut factor with a neutral alternative)—are applied to text before both training and inference. These objects carry the argument because the extrinsic injection constructs the failure mode, the intrinsic splits demonstrate that the failure exists naturally in existing benchmarks, and SMF supplies the claimed cure.

What would settle it

Train a detector on a standard split, then evaluate it on test texts rewritten by a rule-based, non-LLM paraphrasing engine that changes surface wording while provably introducing no label-correlated sentiment, topic, or style cue; if accuracy stays near the original level, the LLM-rewrite drop is not generic distribution shift, but if it also falls to near chance, the shortcut explanation is not needed to explain the Vanilla results.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that trainable content-based misinformation detectors do not learn to assess whether a claim is true; they learn correlations between the label and shortcut features, so when an LLM rewrites content with a different sentiment, tone, or word choice, accuracy collapses even though the underlying truth label is unchanged. The evidence includes average accuracy drops of about 80 percent, and up to 92 percent, for BERT, DeBERTa, and three debiasing detectors under explicit injection, plus transfer results showing that a detector trained on sentiment-injected data improves on the SST-2 and MTEB sentiment tasks without ever seeing sentiment labels. LLM-based detectors prompted in a zero-shot fashion are less accurate overall but largely immune to the same injections. The paper further claims that rewriting training and test texts with SMF—paraphrase, factual summarization, or sentiment-neutralization—removes enough shortcut signal to substantially recover accuracy, with improvements up to 838.4 percent on the most damaging attacks.

Load-bearing premise

The paper's diagnosis assumes that the large accuracy drop under the Vanilla LLM rewrite (for example, BERT falling from 78.1 to 17.3) is caused by the detector latching onto shortcut cues, but the Vanilla prompt injects no explicit shortcut, so ordinary sensitivity to a changed writing style could in principle produce the same drop without any specific spurious cue being learned.

Editorial extensions

If this is right

  • A detector's high accuracy on a static benchmark should not be read as evidence that it detects misinformation; the same architecture collapses to near chance when the surface form of the text changes.
  • LLM-based prompting detectors, though weaker on standard benchmarks, are the more robust option against rewrite-based attacks, which argues for hybrid designs that combine LLM judgment with trainable components.
  • Data-centric augmentation using SMF is a viable complement to debiasing model architectures, and combining debiasing methods with SMF yields the strongest robustness under explicit injection.
  • Detectors trained under shortcut-inducing conditions acquire transferable ability on unrelated tasks such as sentiment classification, so shortcut reliance can be diagnosed by probing which auxiliary skills a model gained during training.
  • The two new factual misinformation datasets, NQ-Misinfo and Streaming-Misinfo, provide a way to test whether a detector uses factual knowledge, and current detectors perform only slightly above chance on them.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the shortcut diagnosis transfers, the same evaluation design could be applied to other content-classification tasks such as hate speech, spam, and stance detection, where LLM rewriting is equally cheap and would likely reveal similar fragility.
  • The success of the Summary variant suggests that training models on concise, fact-only versions of claims could serve as a general robustness regularizer, independent of the specific attack type used at evaluation time.
  • A testable extension is to apply SMF at inference time only, without retraining, since the framework rewrites text before both training and inference; the paper does not separate these two effects, so it is unknown whether rewriting alone can protect a deployed detector.
  • Because the implicit age and gender injections fail to move detectors, the practical threat model is narrower than 'any LLM rewrite': only attacks that shift an explicit, learnable surface attribute are dangerous, and future adversarial prompts that succeed at implicit author impersonation could close that gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces TruthOverTricks, a two-part evaluation paradigm for shortcut learning in misinformation detection: intrinsic shortcut induction (sentiment, style, topic, perplexity) and extrinsic shortcut injection (Vanilla, Explicit, and Implicit LLM-based rewriting). It evaluates seven detectors across 16 datasets, including two new QA-derived factual misinformation benchmarks (NQ-Misinfo, Streaming-Misinfo), reports that trainable detectors suffer large accuracy drops under the constructed shortcut settings, and proposes SMF, an LLM-based data augmentation framework (Paraphrase, Summary, Neutral variants) that improves accuracy under these settings. The paper also includes human evaluations of injection fidelity and a public release of resources.

Significance. If the claims were accepted as stated, the paper would provide a broad evaluation paradigm and a data-centric mitigation for an important but under-measured failure mode. The manuscript has genuine strengths: a large benchmark coverage (16 datasets, 7 detectors), explicit prompts for six injection strategies, human and automated checks that the injected rewrites preserve authenticity, external sentiment-transfer experiments as evidence of shortcut acquisition, and publicly released resources. However, the central diagnostic claim is currently overstated: the main degradation results are produced by retraining detectors on deliberately anti-correlated splits, not by exposing standard-trained detectors to rewritten inputs, and the paper's own control experiments (Tables 20–21) show only minor performance changes under standard training. Several quantitative claims also rest on very small test sets and best-of-five reporting without variance estimates. The framework and datasets are valuable, but the interpretation needs to be reframed and the evidence needs to be reported with appropriate qualifications.

major comments (4)
  1. [Abstract; §3.2; Appendix J.2; Tables 20–21] The central claim that existing detectors 'suffer severe performance degradation when exposed to' shortcuts is not supported as stated. The large drops in Table 2 are obtained by retraining each trainable detector on training/validation splits where the injected attribute e is perfectly correlated with the label y, then testing on the reversed correlation (Appendix J.2). Appendix F.3, Tables 20–21, shows that when the same detectors are trained on the original data and evaluated on the rewritten test sets, performance changes are minor (e.g., BERT on D01 goes from 74.6 Original to 79.4 Vanilla; DEBERTA on D01 goes from 68.3 to 59.3). The paper should either rephrase the claim as 'detectors suffer severe degradation when retrained under deliberately induced anti-correlated splits' or add standard-trained evaluations to support the broad statement that existing detectors fail on rewritten inputs.
  2. [Table 9; Table 1; Table 14] Several intrinsic shortcut cells have extremely small test sets, making the reported accuracies unstable. For example, Celeberty Style has 2 test instances, COCO Style has 2 test instances, and RumourEval Sentiment has 9 test instances. Appendix H states that experiments were run five times and the best performance is reported, but no standard deviations or error bars are given. This makes it impossible to assess whether the per-dataset drops and occasional improvements in Table 1 and Table 14 are meaningful. The manuscript should report variance across runs or restrict per-cell conclusions to cells with sufficiently large test sets.
  3. [§2.1; Appendix C] The 'intrinsic shortcut induction' protocol constructs training/test splits that are anti-correlated by design, so it measures susceptibility to an induced spurious correlation rather than demonstrating that shortcuts 'naturally emerge within existing benchmarks.' The distribution plots (Figure 5, Table 13) show similar marginal distributions, but the experimental protocol forces the joint distribution to differ between training and test. The manuscript should separate the claim that datasets contain potential shortcut cues (supported by the distribution analyses) from the claim that standard-trained detectors already rely on those cues (which the current protocol does not test).
  4. [§4.2; Table 3] The mitigation results are reported as relative percentages over a near-zero baseline, which is misleading. For example, BERT under Vanilla goes from 17.3 to 38.8 with Paraphrase (123.7% relative), and under Word Choice from 7.0 to 63.4 (800.5% relative), but absolute accuracy remains low and several augmented cells stay near or below chance (e.g., DEBERTA Vanilla with Paraphrase 30.2; BERT Sentiment with Paraphrase 14.7). The claim that SMF 'consistently enhances robustness across 16 benchmarks' should be qualified with per-dataset absolute performance, and statistical significance across the five runs should be reported.
minor comments (6)
  1. [Figure 8; §3.2] The caption says 'before and after training under the Sentiment injection attack,' but the text refers to 'after shortcut-based training' and the figure shows accuracy on SST-2 and MTEB; please clarify that the evaluation is on external sentiment datasets, not on the misinformation test set.
  2. [Appendix B; Table 8] The two new datasets are constructed by converting QA pairs into declarative sentences and using CoEdIT to fix grammar; please report how many instances were discarded due to grammar-correction failure or other filtering, since this could affect the dataset's representativeness.
  3. [Appendix J.2] The notation '∥e∥' for the number of attribute values is nonstandard; consider using '|e|' or explicitly stating 'the number of possible attribute values.'
  4. [Table 7; Appendix H] The hyperparameter table does not include training epochs, early stopping criteria, or validation selection details; adding these would improve reproducibility.
  5. [§2.2; Table 5] The Vanilla strategy is described as 'altering prompt formats,' but the three Vanilla prompts (rewriting, paraphrase, open-ended) change the text substantially, not merely the format; the wording should be adjusted.
  6. [References] The entries for Tianchi, Kaggle1, and Kaggle2 contain only links with no formal citation information; please provide complete citations or dataset documentation references.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claims rest on controlled stress-test experiments with external validation, and the self-citations are contextual rather than load-bearing.

full rationale

The paper's main empirical claims are measurements under an explicitly constructed evaluation protocol, not predictions derived from the protocol's inputs. The intrinsic and extrinsic shortcut settings deliberately create training/test splits in which a chosen attribute e is correlated with the label y in training and anti-correlated in test (Section 2.1 and Appendix C). Downstream accuracy drops under this protocol are a designed stress test for shortcut reliance, not a self-fulfilling derivation: the paper does not claim to predict the drops from the construction; it observes them empirically. The interpretation that detectors rely on shortcuts is further supported by independent evidence such as the transfer of sentiment-injection-trained detectors to external sentiment classification datasets (Figure 8) and the control experiments in Appendix F.3 (Tables 20-21) showing that the same detectors under standard training do not severely degrade on rewritten inputs. The proposed SMF mitigation is evaluated on the same injected shortcut data, but its mechanism (removing the injected attribute) is the intended intervention, and the paper also verifies removal via similarity and distribution analyses (Appendix K.1); this is a designed mitigation experiment rather than a circular validation. The few self-citations (e.g., Refs. [56] and [59], which share authors with this paper) are used for background context about LLM-based attacks and cross-domain generalization struggles, not as the sole justification for the paper's central claims. No load-bearing argument reduces to a self-citation, no fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from the authors' prior work. The paper's limitations—such as the inability to cover all shortcut types and the possibility that vanilla rewriting induces generic distribution shift rather than shortcut learning—are correctness/interpretation concerns, not circularity concerns.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central evaluation depends on arbitrary binarizations and unvalidated proxy classifiers for intrinsic shortcuts, on a sampling cap that changes benchmark composition, and on the assumption that LLM rewriting preserves authenticity. No new physical or theoretical entities are introduced; the new resources are two derived datasets from existing QA benchmarks.

free parameters (5)
  • Perplexity split threshold = Top 50% versus bottom 50%
    The intrinsic perplexity shortcut uses an arbitrary median split computed with GPT-2. Changing this threshold changes the anti-correlated train/test splits and therefore the measured degradation.
  • Topic group binarization = First 4 of 8 topic classes vs last 4
    Topics are manually collapsed into two groups to create the shortcut factor, without evidence that this grouping reflects a natural shortcut.
  • Sentiment binarization = Negative versus positive; neutral excluded
    Neutral examples are dropped from the sentiment shortcut setting, changing dataset composition and making the evaluation not fully representative.
  • Style binarization = Subjective versus neutral
    Style labels come from an external classifier whose accuracy is not reported, and the split into two classes is chosen by hand.
  • Dataset sampling cap = 1,000 instances per dataset
    All 14 existing benchmarks are downsampled to at most 1,000 balanced instances; results may not match full-benchmark behavior.
assumptions (4)
  • domain assumption The external indicator classifiers for sentiment, style, and topic accurately label the shortcut factor e for every instance.
    Intrinsic shortcut splits in Section 2.1 and Appendix D are built from these classifier labels, but no accuracy, agreement, or error analysis for the classifiers is reported.
  • domain assumption LLM rewriting preserves authenticity across all 16 datasets and all injection and mitigation strategies.
    Human evaluation in Appendix F.4 samples 100 positive pairs and reports 91.2% accuracy, but the pipeline applies rewriting to far more texts and to both real and fake content in both train and test.
  • domain assumption A feature e whose marginal distribution is similar across real and fake instances is not causally informative for authenticity, so reliance on it is a shortcut.
    Section 3.1 treats distributional similarity in Figure 5 and Table 13 as evidence of non-causality, which is a modeling assumption about what counts as a shortcut.
  • domain assumption The 14 selected benchmarks plus NQ-Misinfo and Streaming-Misinfo, and the 7 selected detectors, are representative of misinformation detection practice.
    The paper states this coverage in the Limitation section, but representativeness cannot be proven and social-context detectors are excluded.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Truth over Tricks: Measuring and Mitigating Shortcut Learning in Misinformation Detection." pith.science (2026). https://pith.science/paper/U2DYL3EJ

@misc{pith2026250602350,
  author       = {Pith},
  title        = {Pith review of: Truth over Tricks: Measuring and Mitigating Shortcut Learning in Misinformation Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U2DYL3EJ}},
  note         = {Machine review of arXiv:2506.02350}
}
read the original abstract

Misinformation detection models often rely on superficial cues (i.e., \emph{shortcuts}) that correlate with misinformation in training data but fail to generalize to the diverse and evolving nature of real-world misinformation. This issue is exacerbated by large language models (LLMs), which can easily generate convincing misinformation through simple prompts. We introduce TruthOverTricks, a unified evaluation paradigm for measuring shortcut learning in misinformation detection. TruthOverTricks categorizes shortcut behaviors into intrinsic shortcut induction and extrinsic shortcut injection, and evaluates seven representative detectors across 14 popular benchmarks, along with two new factual misinformation datasets, NQ-Misinfo and Streaming-Misinfo. Empirical results reveal that existing detectors suffer severe performance degradation when exposed to both naturally occurring and adversarially crafted shortcuts. To address this, we propose SMF, an LLM-augmented data augmentation framework that mitigates shortcut reliance through paraphrasing, factual summarization, and sentiment normalization. SMF consistently enhances robustness across 16 benchmarks, encouraging models to rely on deeper semantic understanding rather than shortcut cues. To promote the development of misinformation detectors, we have published the resources publicly at https://github.com/whr000001/TruthOverTricks.

Figures

Figures reproduced from arXiv: 2506.02350 by the authors.

Figure 1
Figure 1. Illustration of shortcut learning in misinformation [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. TRUTHOVERTRICKS evaluates misinformation detectors under two types of shortcuts: (1) Intrinsic Shortcut Induction, which captures spurious correlations that naturally emerge within existing benchmarks; and (2) Extrinsic Shortcut Injection, which introduces adversarially crafted variations from LLMs designed to obscure misinformation and challenge detector robustness. • Style. Misinformation is often more subjective … view at source ↗
Figure 3
Figure 3. Token count distributions before and after Word Choice injection. The dis￾tinct distribution patterns and differences in average token counts show that LLMs can ef￾fectively manipulate word choices. 0.2 0.4 0.6 ROUGE-L Paraphrase Open-ended Positive Negative Simple Complex Formal Informal Young Elder Male Female Random .099 0.5 0.6 0.7 BERTScore Random .437 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (9 more)
Figure 5
Figure 5. Figure 5: Distributions of sentiment, style, topic, [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Training loss and accuracy of DE￾BERTA on AntiVax under the shortcut set￾ting, averaged over five runs. Performance degrades over time, indicating that the detec￾tor increasingly overfits to shortcuts rather than learning authenticity. 10 30 50 70 90 Percentage (%) 60 …
Figure 8
Figure 8. Figure 8: Performance of DEBERTA on two sentiment classification datasets before and after training under the Sentiment injection attack. Accuracy improvements indicate that detectors mistakenly learn sentiment as a proxy for authenticity, confirming the presence of shortcut lea…
Figure 9
Figure 9. Figure 9: Calibration of detectors before and af￾ter explicit injection attacks. “Error” indicates the expected calibration error (×100), where lower values denote better calibration. Results show that explicit injection attacks substantially degrade cal￾ibration quality. We als…
Figure 10
Figure 10. Figure 10: The performance of existing detectors on a standard dataset AntiVax and our proposed datasets NQ-Misinfo and Streaming-Misinfo. Detectors fail to achieve acceptable performance on datasets requiring fact-related knowledge and are only slightly better than random guess…
Figure 12
Figure 12. Figure 12: The overviews of standard setting and the shortcut setting. The horizontal axis represents authenticity, while the vertical axis represents other factors (taking positive and negative sentiment as an example). The main difference is that the short￾cut setting induces …
Figure 13
Figure 13. Figure 13: The sentiment distributions in dataset Pheme before (“Original”) and after Sentiment injection attack. It illustrates that LLMs can inject sentiment factors. We further conduct a human evaluation to eval￾uate the injection attacks. We evaluate both ex￾plicit (Tone and…
Figure 14
Figure 14. Figure 14: The perplexity distributions of fake and real instances on the employed 16 datasets. We [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: The sentiment distributions under the Sentiment injection attack and distributions after the three mitigation methods. It illustrates that the mitigation methods except for Paraphrase can remove this shortcut. accuracy should be 80%. For LLM-based detectors, we employ…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation

    cs.SI 2025-10 unverdicted novelty 6.0 of 10

    CrowdNotes+ combines LLM note augmentation and automation with a three-stage evaluation to outperform human contributors on correctness, helpfulness, and evidence utility for health misinformation notes.

Reference graph

Works this paper leans on

71 extracted references · 54 canonical work pages · cited by 1 Pith paper

  1. [1]

    Out of one, many: Using language models to simulate human samples

    Lisa P Argyle, Ethan C Busby, Nancy Fulda, Joshua R Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples. Political Analysis, 31(3):337–351, 2023

    2023

  2. [2]

    Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature

    Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024

    work page 2024

  3. [3]

    Fakingrecipe: Detecting fake news on short video platforms from the perspective of creative process

    Yuyan Bu, Qiang Sheng, Juan Cao, Peng Qi, Danding Wang, and Jintao Li. Fakingrecipe: Detecting fake news on short video platforms from the perspective of creative process. In Proceedings of the 32nd ACM International Conference on Multimedia , pages 1351–1360, 2024

    work page 2024

  4. [4]

    Automatically identifying fake news in popular twitter threads

    Cody Buntain and Jennifer Golbeck. Automatically identifying fake news in popular twitter threads. In 2017 IEEE international conference on smart cloud (smartCloud), pages 208–215. IEEE, 2017

    work page 2017

  5. [5]

    Can llm-generated misinformation be detected? In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11,

    Canyu Chen and Kai Shu. Can llm-generated misinformation be detected? In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11,

    work page 2024

  6. [6]

    Complex claim verification with evidence retrieved in the wild

    Jifan Chen, Grace Kim, Aniruddh Sriram, Greg Durrett, and Eunsol Choi. Complex claim verification with evidence retrieved in the wild. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3569–3587, 2024

    work page 2024

  7. [7]

    Fine-grained analysis of propaganda in news article

    Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov. Fine-grained analysis of propaganda in news article. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 5636–5646, 2019

    work page 2019

  8. [8]

    Semeval-2017 task 8: Rumoureval: Determining rumour veracity and support for rumours

    Leon Derczynski, Kalina Bontcheva, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Arkaitz Zubiaga. Semeval-2017 task 8: Rumoureval: Determining rumour veracity and support for rumours. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 69–76, 2017

    work page 2017

Show all 71 references
  1. [9]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human langu...

    2019

  2. [10]

    Unveiling implicit deceptive patterns in multi-modal fake news via neuro-symbolic reasoning

    Yiqi Dong, Dongxiao He, Xiaobao Wang, Youzhu Jin, Meng Ge, Carl Yang, and Di Jin. Unveiling implicit deceptive patterns in multi-modal fake news via neuro-symbolic reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 8354–8362, 2024

    2024

  3. [11]

    Debiasing graph neural networks via learning disentangled causal substructure

    Shaohua Fan, Xiao Wang, Yanhu Mo, Chuan Shi, and Jian Tang. Debiasing graph neural networks via learning disentangled causal substructure. Advances in Neural Information Processing Systems, 35:24934–24946, 2022

    2022

  4. [12]

    Shortcut learning in deep neural networks

    Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020

    2020

  5. [13]

    Heterogeneous graph reasoning for fact checking over texts and tables

    Haisong Gong, Weizhi Xu, Shu Wu, Qiang Liu, and Liang Wang. Heterogeneous graph reasoning for fact checking over texts and tables. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 100–108, 2024

    2024

  6. [14]

    The llama 3 herd of models

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. 10

    2024 arXiv

  7. [15]

    Public wisdom matters! discourse-aware hyperbolic fourier co-attention for social text classification.Advances in Neural Information Processing Systems, 35:9417–9431, 2022

    Karish Grover, SM Angara, Md Shad Akhtar, and Tanmoy Chakraborty. Public wisdom matters! discourse-aware hyperbolic fourier co-attention for social text classification.Advances in Neural Information Processing Systems, 35:9417–9431, 2022

    2022

  8. [16]

    Language models hallucinate, but may excel at fact verification

    Jian Guan, Jesse Dodge, David Wadden, Minlie Huang, and Hao Peng. Language models hallucinate, but may excel at fact verification. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies ...

    2024

  9. [17]

    On calibration of modern neural networks

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Weinberger. On calibration of modern neural networks. In International conference on machine learning, pages 1321–1330. PMLR, 2017

    2017

  10. [18]

    Applying automatic text summarization for fake news detection

    Philipp Hartl and Udo Kruschwitz. Applying automatic text summarization for fake news detection. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2702–2713, 2022

    2022

  11. [19]

    Anti-vax: a novel twitter dataset for covid-19 vaccine misinformation detection

    Kadhim Hayawi, Sakib Shahriar, Mohamed Adel Serhani, Ikbal Taleb, and Sujith Samuel Mathew. Anti-vax: a novel twitter dataset for covid-19 vaccine misinformation detection. Public health, 203:23–30, 2022

    2022

  12. [20]

    Deberta: decoding-enhanced bert with disentangled attention

    Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: decoding-enhanced bert with disentangled attention. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021

    2021

  13. [21]

    Bad actor, good advisor: Exploring the role of large language models in fake news detection

    Beizhe Hu, Qiang Sheng, Juan Cao, Yuhui Shi, Yang Li, Danding Wang, and Peng Qi. Bad actor, good advisor: Exploring the role of large language models in fake news detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 22105–22113, 2024

    2024

  14. [22]

    A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qiang- long Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. ACM Transactions on Informat...

    2025

  15. [23]

    Fake news classification based on subjective language

    Caio Libanio Melo Jeronimo, Leandro Balby Marinho, Claudio EC Campelo, Adriano Veloso, and Allan Sales da Costa Melo. Fake news classification based on subjective language. In Proceedings of the 21st International Conference on Information Integration and Web-based Application...

    2019

  16. [24]

    Mistral 7b

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023

    arXiv

  17. [25]

    Hale, and Rada Mihalcea

    Ashkan Kazemi, Zehua Li, Verónica Pérez-Rosas, Scott A. Hale, and Rada Mihalcea. Matching tweets with applicable fact-checks across languages. In Amitava Das, Tanmay Chakraborty, Asif Ekbal, and Amit P. Sheth, editors, Proceedings of the Workshop on Multi-Modal Fake News and H...

    2022

  18. [26]

    Natural questions: a benchmark for question answering research

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computa...

    2019

  19. [27]

    Coco: an annotated twitter dataset of covid-19 conspiracy theories

    Johannes Langguth, Daniel Thilo Schroeder, Petra Filkuková, Stefan Brenner, Jesper Phillips, and Konstantin Pogorelov. Coco: an annotated twitter dataset of covid-19 conspiracy theories. Journal of Computational Social Science, 6(2):443–484, 2023

    2023

  20. [28]

    Autobencher: Towards declarative benchmark construction

    Xiang Lisa Li, Farzaan Kaiyom, Evan Zheran Liu, Yifan Mai, Percy Liang, and Tatsunori Hashimoto. Autobencher: Towards declarative benchmark construction. In The Thirteenth International Conference on Learning Representations, 2025. 11

    2025

  21. [29]

    Mcfend: A multi-source benchmark dataset for chinese fake news detection

    Yupeng Li, Haorui He, Jin Bai, and Dacheng Wen. Mcfend: A multi-source benchmark dataset for chinese fake news detection. In Proceedings of the ACM Web Conference 2024 , pages 4018–4027, 2024

    2024

  22. [30]

    Cfever: A chinese fact extraction and verification dataset

    Ying-Jia Lin, Chun-Yi Lin, Chia-Jen Yeh, Yi-Ting Li, Yun-Yu Hu, Chih-Hao Hsu, Mei-Feng Lee, and Hung-Yu Kao. Cfever: A chinese fact extraction and verification dataset. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 18626–18634, 2024

    2024

  23. [31]

    Stream- ingqa: A benchmark for adaptation to new knowledge over time in question answering models

    Adam Liska, Tomas Kocisky, Elena Gribovskaya, Tayfun Terzi, Eren Sezener, Devang Agrawal, Cyprien De Masson D’Autume, Tim Scholtes, Manzil Zaheer, Susannah Young, et al. Stream- ingqa: A benchmark for adaptation to new knowledge over time in question answering models. In Inter...

    2022

  24. [32]

    Infini- gram: Scaling unbounded n-gram language models to a trillion tokens

    Jiacheng Liu, Sewon Min, Luke Zettlemoyer, Yejin Choi, and Hannaneh Hajishirzi. Infini- gram: Scaling unbounded n-gram language models to a trillion tokens. arXiv preprint arXiv:2401.17377, 2024

    2024 arXiv

  25. [33]

    Mmfakebench: A mixed-source multimodal misinformation detection benchmark for lvlms

    Xuannan Liu, Zekun Li, Peipei Li, Shuhan Xia, Xing Cui, Linzhi Huang, Huaibo Huang, Weihong Deng, and Zhaofeng He. Mmfakebench: A mixed-source multimodal misinformation detection benchmark for lvlms. arXiv preprint arXiv:2406.08772, 2024

    2024 arXiv

  26. [34]

    Raemollm: Retrieval augmented llms for cross-domain misinformation detection using in-context learning based on emotional information

    Zhiwei Liu, Kailai Yang, Qianqian Xie, Christine de Kock, Sophia Ananiadou, and Eduard Hovy. Raemollm: Retrieval augmented llms for cross-domain misinformation detection using in-context learning based on emotional information. arXiv preprint arXiv:2406.11093, 2024

    2024

  27. [35]

    Fighting fire with fire: The dual role of llms in crafting and detecting elusive disinformation

    Jason Lucas, Adaku Uchendu, Michiharu Yamashita, Jooyoung Lee, Shaurya Rohatgi, and Dongwon Lee. Fighting fire with fire: The dual role of llms in crafting and detecting elusive disinformation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proc...

    2023

  28. [36]

    Rumor detection on twitter with tree-structured recursive neural networks

    Jing Ma, Wei Gao, and Kam-Fai Wong. Rumor detection on twitter with tree-structured recursive neural networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1980–1989, 2018

    1980

  29. [37]

    A survey on bias and fairness in machine learning

    Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1–35, 2021

    2021

  30. [38]

    Detectgpt: Zero-shot machine-generated text detection using probability curvature

    Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In Interna- tional Conference on Machine Learning, pages 24950–24962. PMLR, 2023

    2023

  31. [39]

    Mteb: Massive text embedding benchmark

    Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. Mteb: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, 2023

    2014

  32. [40]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:2773...

    2022

  33. [41]

    Enhancing society- undermining disinformation detection through fine-grained sentiment analysis pre-finetuning

    Tsung-Hsuan Pan, Chung-Chi Chen, Hen-Hsen Huang, and Hsin-Hsi Chen. Enhancing society- undermining disinformation detection through fine-grained sentiment analysis pre-finetuning. In Findings of the Association for Computational Linguistics: EACL 2024, pages 1371–1377, 2024

    2024

  34. [42]

    On the risk of misinformation pollution with large language models

    Yikang Pan, Liangming Pan, Wenhu Chen, Preslav Nakov, Min-Yen Kan, and William Wang. On the risk of misinformation pollution with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 1389–1403, 2023

    2023

  35. [43]

    Automatic detection of fake news

    Verónica Pérez-Rosas, Bennett Kleinberg, Alexandra Lefevre, and Rada Mihalcea. Automatic detection of fake news. In Proceedings of the 27th International Conference on Computational Linguistics, pages 3391–3401, 2018. 12

    2018

  36. [44]

    Gpt-generated text detection: Benchmark dataset and tensor-based detection method

    Zubair Qazi, William Shiao, and Evangelos E Papalexakis. Gpt-generated text detection: Benchmark dataset and tensor-based detection method. In Companion Proceedings of the ACM on Web Conference 2024, pages 842–846, 2024

    2024

  37. [45]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019

    2019

  38. [46]

    Coedit: Text editing by task- specific instruction tuning

    Vipul Raheja, Dhruv Kumar, Ryan Koo, and Dongyeop Kang. Coedit: Text editing by task- specific instruction tuning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5274–5291, 2023

    2023

  39. [47]

    A domain adaptive graph learning framework to early detection of emergent healthcare misin- formation on social media

    Lanyu Shang, Yang Zhang, Zhenrui Yue, YeonJung Choi, Huimin Zeng, and Dong Wang. A domain adaptive graph learning framework to early detection of emergent healthcare misin- formation on social media. In Proceedings of the International AAAI Conference on Web and Social Media, ...

    2024

  40. [48]

    Zoom out and observe: News environment perception for fake news detection

    Qiang Sheng, Juan Cao, Xueyao Zhang, Rundong Li, Danding Wang, and Yongchun Zhu. Zoom out and observe: News environment perception for fake news detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages...

    2022

  41. [49]

    Causality guided disentanglement for cross-platform hate speech detection

    Paras Sheth, Raha Moraffah, Tharindu S Kumarage, Aman Chadha, and Huan Liu. Causality guided disentanglement for cross-platform hate speech detection. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 626–635, 2024

    2024

  42. [50]

    Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media

    Kai Shu, Deepak Mahudeswaran, Suhang Wang, Dongwon Lee, and Huan Liu. Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data, 8(3):171–188, 2020

    2020

  43. [51]

    Embracing domain differences in fake news: Cross-domain fake news detection using multi-modal data

    Amila Silva, Ling Luo, Shanika Karunasekera, and Christopher Leckie. Embracing domain differences in fake news: Cross-domain fake news detection using multi-modal data. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 557–565, 2021

    2021

  44. [52]

    Recursive deep models for semantic compositionality over a sentiment treebank

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language proce...

    2013

  45. [53]

    Fake news detectors are biased against texts generated by large language models

    Jinyan Su, Terry Yue Zhuo, Jonibek Mansurov, Di Wang, and Preslav Nakov. Fake news detectors are biased against texts generated by large language models. arXiv preprint arXiv:2309.08674, 2023

    2023 arXiv

  46. [54]

    Hg-sl: Jointly learning of global and local user spreading behavior for fake news early detection

    Ling Sun, Yuan Rao, Yuqian Lan, Bingcan Xia, and Yangyang Li. Hg-sl: Jointly learning of global and local user spreading behavior for fake news early detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 5248–5256, 2023

    2023

  47. [55]

    Deepfake video detection via facial action dependencies estimation

    Lingfeng Tan, Yunhong Wang, Junfu Wang, Liang Yang, Xunxun Chen, and Yuanfang Guo. Deepfake video detection via facial action dependencies estimation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 5276–5284, 2023

    2023

  48. [56]

    On the risk of evidence pollution for malicious social text detection in the era of llms

    Herun Wan, Minnan Luo, Zhixiong Su, Guang Dai, and Xiang Zhao. On the risk of evidence pollution for malicious social text detection in the era of llms. arXiv preprint arXiv:2410.12600, 2024

    2024 arXiv

  49. [57]

    Explainable fake news detection with large language model via defense among competing wisdom

    Bo Wang, Jing Ma, Hongzhan Lin, Zhiwei Yang, Ruichao Yang, Yuan Tian, and Yi Chang. Explainable fake news detection with large language model via defense among competing wisdom. In Proceedings of the ACM on Web Conference 2024, pages 2452–2463, 2024

    2024

  50. [58]

    Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks

    Jiaying Wu, Jiafeng Guo, and Bryan Hooi. Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 3367–3378, 2024. 13

    2024

  51. [59]

    Probing spurious correlations in popular event-based rumor detection benchmarks

    Jiaying Wu and Bryan Hooi. Probing spurious correlations in popular event-based rumor detection benchmarks. In Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2022, Grenoble, France, September 19-23, 2022, Proceedings, Part II, pages 274–...

    2022

  52. [60]

    Msynfd: Multi-hop syntax aware fake news detection

    Liang Xiao, Qi Zhang, Chongyang Shi, Shoujin Wang, Usman Naseem, and Liang Hu. Msynfd: Multi-hop syntax aware fake news detection. In Proceedings of the ACM on Web Conference 2024, pages 4128–4137, 2024

    2024

  53. [61]

    Harnessing network effect for fake news mitigation: Selecting debunkers via self-imitation learning

    Xiaofei Xu, Ke Deng, Michael Dann, and Xiuzhen Zhang. Harnessing network effect for fake news mitigation: Selecting debunkers via self-imitation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 22447–22456, 2024

    2024

  54. [62]

    Wsdms: Debunk fake news via weakly supervised detection of misinforming sentences with contextualized social wisdom

    Ruichao Yang, Wei Gao, Jing Ma, Hongzhan Lin, and Zhiwei Yang. Wsdms: Debunk fake news via weakly supervised detection of misinforming sentences with contextualized social wisdom. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages ...

    2023

  55. [63]

    Llm-enhanced multiple instance learning for joint rumor and stance detection with social context information

    Ruichao Yang, Jing Ma, Wei Gao, and Hongzhan Lin. Llm-enhanced multiple instance learning for joint rumor and stance detection with social context information. ACM Transactions on Intelligent Systems and Technology, 2025

    2025

  56. [64]

    Gamc: an unsupervised method for fake news detection using graph autoencoder with masking

    Shu Yin, Peican Zhu, Lianwei Wu, Chao Gao, and Zhen Wang. Gamc: an unsupervised method for fake news detection using graph autoencoder with masking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 347–355, 2024

    2024

  57. [65]

    Contrastive domain adaptation for early misinformation detection: A case study on covid-19

    Zhenrui Yue, Huimin Zeng, Ziyi Kou, Lanyu Shang, and Dong Wang. Contrastive domain adaptation for early misinformation detection: A case study on covid-19. In Proceedings of the 31st ACM international conference on information & knowledge management , pages 2423–2433, 2022

    2022

  58. [66]

    Escnet: Entity-enhanced and stance checking network for multi-modal fact-checking

    Fanrui Zhang, Jiawei Liu, Jingyi Xie, Qiang Zhang, Yongchao Xu, and Zheng-Jun Zha. Escnet: Entity-enhanced and stance checking network for multi-modal fact-checking. In Proceedings of the ACM on Web Conference 2024, pages 2429–2440, 2024

    2024

  59. [67]

    Reinforced adaptive knowledge learning for multimodal fake news detection

    Litian Zhang, Xiaoming Zhang, Ziyi Zhou, Feiran Huang, and Chaozhuo Li. Reinforced adaptive knowledge learning for multimodal fake news detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 16777–16785, 2024

    2024

  60. [68]

    Mining dual emotion for fake news detection

    Xueyao Zhang, Juan Cao, Xirong Li, Qiang Sheng, Lei Zhong, and Kai Shu. Mining dual emotion for fake news detection. In Proceedings of the web conference 2021, pages 3465–3476, 2021

    2021

  61. [69]

    A survey of fake news: Fundamental theories, detection methods, and opportunities

    Xinyi Zhou and Reza Zafarani. A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Comput. Surv., 53(5), September 2020

    2020

  62. [70]

    When”, “Where

    Yongchun Zhu, Qiang Sheng, Juan Cao, Shuokai Li, Danding Wang, and Fuzhen Zhuang. Generalizing to the future: Mitigating entity bias in fake news detection. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, page...

    2022

  63. [2024]

    OpenReview.net, 2024

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.