Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Data

Chen Ding; Drai Paulen-Patterson; Shaina Raza

REVIEW 5 major objections 6 minor 83 references

Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Data

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that, on a GPT-4-annotated and human-reviewed dataset of 10,000 news articles, BERT-like encoder models beat autoregressive LLMs at fake-news classification, while LLMs are more robust to text perturbations.

desk verdict Competent applied comparison undermined by a direct results-section contradiction and unvalidated labels; the central claims as stated cannot be trusted. read the letter →

arxiv 2412.14276 v2 pith:34UOLXJU submitted 2024-12-18 cs.CL cs.AI

Shaina Raza , Drai Paulen-Patterson , Chen Ding This is my paper · ORCID

classification cs.CLcs.AI

keywords fakenewsdetectionBERT-likemodelslargelanguageGPT-4annotationhuman-in-the-looplabelingdistantsupervisiontextperturbationrobustnessbenchmarkdataset

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish two things: that encoder-only BERT-style models are the better classifiers for fake news, and that article-level labels produced by GPT-4 and then checked by human reviewers are a stronger training signal than the source-level distant labels commonly used in this field. On a new 10,000-article political-news dataset, fine-tuned RoBERTa reaches about 89.68% F1, while the best fine-tuned LLM, Mistral-7B, reaches about 80.23% F1. The result matters because it suggests that relatively small, efficient classifiers can still beat much larger generative models on a focused text-classification task, and that the quality of labels, not model size, may be the bottleneck. It also matters for annotation practice: generative AI with human oversight can produce usable training data without full expert annotation.

What carries the argument

The argument is carried by a specific annotation-and-evaluation pipeline. First, GPT-4 Turbo labels 10,000 political news articles as real or fake using a two-shot prompt; eight human reviewers independently re-check every label and resolve disagreements by consensus, yielding roughly 72% inter-rater agreement. Second, the comparison is run with two standardized model recipes: BERT-like models use a softmax classification head over the transformer output, while Llama-2-7B and Mistral-7B are instruction-fine-tuned with QLoRA and then prompted five times per article, with majority voting and an average-confidence threshold of 0.8 deciding the final label. The same two recipes are applied to the NELA-GT-2022 distant-supervision sample, which provides the weak-label control that lets the authors attribute part of the performance gap to label quality rather than only to model family.

What would settle it

Take a random sample of a few hundred articles from this dataset and have independent professional fact-checkers label them, then compare those expert labels to the GPT-plus-human labels; if agreement is near chance, or if models retrained on the expert labels no longer show BERT-like models beating LLMs, the paper's central claim would be overturned. A second check is to run the same two model families on an established fact-checked benchmark such as LIAR and see whether the ordering holds.

Watch

Extended reading notes

Core claim

The central discovery is a performance ordering plus a data-quality finding. Fine-tuned BERT-like encoders, especially RoBERTa, outperform instruction-fine-tuned decoder-only LLMs on fake-news classification when both are trained on the same GPT-4-plus-human-labeled dataset; the paper reports RoBERTa at 89.68% F1 versus 80.23% for the best LLM. The authors also find that both model families score higher on this article-level human-supervised label set than on a matched sample of NELA-GT-2022 articles labeled by source-level distant supervision. In perturbation tests, the LLMs keep more of their accuracy when words are dropped, inserted, negated, or typo-ed, while the BERT-like models misclassify those modified examples more often.

Load-bearing premise

The load-bearing premise is that the GPT-4-generated labels, after consensus review by eight reviewers, are correct enough to serve as ground truth for both training and testing; with roughly 72% inter-rater agreement and no independent gold standard, a systematic labeling error would change the rankings.

Editorial extensions

If this is right

If BERT-like models really are strong classifiers on this data, cost- and compute-constrained fake-news screening systems can rely on smaller encoders and reserve LLMs for harder cases.
If human-supervised GPT labels are superior to source-level distant labels, then article-level annotation with AI-plus-human review is a viable path to better training data than weak supervision.
The robustness gap suggests that a deployment combining a precise BERT-like classifier with an LLM-based check for perturbed or adversarial inputs could cover both accuracy and resilience.
Fine-tuning beats zero-shot and few-shot prompting for both Llama-2 and Mistral on this task, so task-specific adapters matter more than scale when the labels are good.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The BERT-like advantage may partly reflect that GPT-4's labeling style is learnable by encoder models; if the labels encode stylistic tells, the ranking could shrink on independently expert-labeled datasets.
The absence of an independent gold standard means the roughly 72% inter-rater agreement leaves room for systematic label bias; a small expert re-labeling study would be the natural check.
A testable extension is to apply the human-supervised LLM-annotation pipeline to other classification domains, such as hate speech or stance detection, and see whether the same label-quality advantage appears.
The perturbation results suggest a cheap robustness recipe: keep a small encoder for clean inputs and route suspicious or edited items to an LLM, since the two model families fail in complementary ways.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a real empirical study comparing BERT-like encoders and 7B LLMs for fake news detection, with a new 10k-article dataset labeled by GPT-4 plus human review. But the paper contains a direct internal contradiction in the Results section: the Discussion says LLMs outperform BERT-like models for RQ1, while Table 6 and the abstract say the opposite. That is not a minor typo; it is the paper's central finding. A reader cannot tell which result the authors actually stand behind.

What the paper does well: the experimental setup is standard and mostly transparent. The authors fine-tune three encoders and two 7B LLMs with QLoRA, use 5-fold cross-validation, report precision/recall/F1, include a hyperparameter ablation for the BERT models, and compare against NELA-GT with source-level labels. Using GPT-4 with two demonstrations and then having eight human reviewers with consensus resolution is a reasonable annotation approach, and reporting the 72% inter-rater reliability is more honest than many papers would be.

The soft spots are load-bearing. First, the contradiction above. Second, RQ2 is confounded: the comparison between "GPT labels" and "source-level labels" changes the dataset, the articles, the time period, and the label definition at the same time, so any performance difference cannot be attributed to label quality. Third, the robustness claim rests on one article perturbed 15 ways in Table 8; that is illustrative, not evidence for a general conclusion about model families. Fourth, the labels are treated as ground truth without any independent gold standard; 72% raw agreement on a binary decision is moderate, and if GPT-4's labeling has systematic bias, the reported F1 gap may partly reflect which model agrees better with that bias. The paper's own data availability statement says the data are "available upon request" and the code link is a placeholder, so the central artifact cannot be independently checked.

Who is this for? Practitioners choosing between cheap encoders and expensive LLMs for fake news detection, and people working on human-supervised LLM annotation pipelines. The paper could be valuable if revised, because the empirical scaffolding is there.

My recommendation: reject in its current form. The internal contradiction alone is disqualifying, and RQ2/robustness need redesign. But this is not a desk-reject-quality paper; it deserves a serious referee, with the expectation of major revision, not acceptance.

Referee Report

5 major / 6 minor

Summary. The paper introduces a 10,000-article fake-news dataset labeled by GPT-4 Turbo with two-shot prompts and then reviewed by eight human reviewers, with disagreements resolved by team consensus. It fine-tunes three BERT-like models (BERT, DistilBERT, RoBERTa) and two 7B decoder-only LLMs (Llama-2-7B-Chat, Mistral-7B-Instruct) using QLoRA and instruction tuning, and compares them on this dataset and on a NELA-GT-2022 source-labeled subset. The main claims are that BERT-like models outperform LLMs on the GPT-labeled dataset (RoBERTa F1 89.68% vs. Mistral 80.23%), that LLMs are more robust to text perturbations, and that GPT-powered labels with human supervision yield better classifiers than source-level distant labels.

Significance. If the central claims held, the paper would provide a practically useful comparison: encoder-only models as efficient classifiers and decoder-only LLMs as more robust alternatives, together with a scalable human-supervised AI annotation pipeline. The paper has genuine strengths: it compares two model families under controlled fine-tuning, includes zero-/few-shot and instruction-tuned LLM variants, and makes code available. However, the significance is currently contingent on label validity and on the comparability of the two datasets, both of which are load-bearing and insufficiently established. The internal contradiction in the RQ1 answer and the unsupported perturbation claim further weaken the contribution in its present form.

major comments (5)

[§3.1.3, Table 6] The evaluation labels are not validated as ground truth. Section 3.1.3 reports only an inter-rater reliability score of 'approximately 72%' among eight reviewers, with no chance-corrected kappa, no GPT-4-to-human agreement, and no independent fact-checked gold standard. Since the same 10,000 labels are used for both training and test evaluation, any systematic labeling bias (e.g., GPT-4 stylistic or political bias) propagates into every reported metric in Table 6. The F1 gap between RoBERTa and the LLMs, and the superiority claimed for GPT labels over NELA-GT source labels, could reflect which model best fits the annotators' labeling bias rather than which model detects false information. The paper also states that data are 'available upon request' rather than released, so the labels cannot currently be independently checked. I would require an external validation component (e.g., evaluation on a fact-checked benchmark, reporting GPT-4/human agreement and kappa, or releasing the labels and annotation protocol) before the central comparison can be accepted.
[§5.1, Discussion paragraph] The Discussion directly contradicts the paper's central claim. After presenting Table 6, in which RoBERTa (89.68 F1) and Mistral fine-tuned (80.23 F1) are the best models in each family, the text states: 'The results answer that for the task of fake news detection, LLMs outperform BERT-like models (RQ1).' This is the opposite of what Table 6, the abstract, and the rest of the paper report. This is not a minor stylistic slip because RQ1 is the main research question; the manuscript must state the answer consistently and unambiguously.
[§5.1, Table 6, RQ2] RQ2 is not identifiable from the current comparison. The GPT-labeled dataset and the NELA-GT-2022 subset differ in multiple dimensions simultaneously: collection period (May–Oct 2023 vs. calendar year 2022), news sources, minimum article length (150 vs. 50 words), train/validation/test split (80/10/10 vs. 80/20), and likely label distribution. Any performance difference in Table 6 could be due to any of these factors, not specifically to the labeling method. To support the claim that 'GPT-powered labels are more accurate than source-level labels,' the authors would need a matched comparison (e.g., the same articles labeled both ways, or article-level fact-check labels from NELA-GT), or at minimum an ablation that controls for source and time period.
[§5.4, Table 8] The robustness claim is based on a single article with 15 perturbation variants, and the table's own 'Correct per Model' row is inconsistent with the cells. Counting the rows, BERT and RoBERTa are predicted Fake on 6 of 15 perturbations (9 correct), DistilBERT on 5 (10 correct), and Llama-2-7B and Mistral-7B on 0 (15 correct), yet the table reports 10, 10, 10, 14, 14. Even if the counts were corrected, a single example cannot support a general conclusion about 'superior robustness,' and no repeated sampling, aggregate accuracy, variance, or significance testing is provided. The perturbation experiment needs to be run over a larger, representative sample of articles with statistical reporting before the RQ3 answer can be accepted.
[§4.3, Table 3, §3.2.2] The classification threshold is specified inconsistently. Section 4.3 states that 'a threshold of 0.8 is applied to these probabilities' for BERT-like models, while Table 3 lists 'Classification Threshold: 0.5 (default for all),' and §3.2.2 describes a confidence threshold of 0.8 for LLM majority voting. Since the decision threshold directly affects precision, recall, and F1, the paper must state which threshold was actually used for each model family and justify the choice. This is a load-bearing detail for all reported metrics.

minor comments (6)

[Abstract] The abstract says 'we developed an instruction-tuned LLM approach with majority voting during inference for label generation,' but in §3.2.2 majority voting is used for classification during inference, not for label generation. Please rephrase to avoid conflating annotation and classification.
[§5.3] The text says 'The results in Table 4 demonstrate that increasing the number of demonstrations...' but Table 4 reports inference time, throughput, and training time; the relevant results are in Figure 4. Also, the text refers to '10 demonstrations per class,' which is not described in the experimental setup (only 5-shot is mentioned in §3.2.2).
[§3.1.1, Table 2] The numbers are inconsistent: §3.1.1 says about 30,000 articles were curated and 10,000 selected, while Table 2 says 'The curated dataset consists of 40,000 news articles labeled for fake news detection and the labeled dataset is 10,000.' Please reconcile.
[§4.1, §4.2] Training time is contradictory: the text in §4.2 says each LLM was fine-tuned in 'approximately 50 minutes,' but Table 4 reports 3.0 hours for Llama-2-7B and 2.5 hours for Mistral-7B. Please correct one of these.
[Declarations] The ethics statement says 'no human participants were involved in this work,' but §3.1.3 describes eight human reviewers who annotated data. While this may not require REB approval under the stated criteria, the current phrasing is misleading and should be clarified or amended.
[Declarations, Data availability] The contribution states 'We release a dataset,' but the data availability statement says the data 'can be obtained from the first author.' For a dataset contribution to be verifiable, the data and annotation protocol should be released through a persistent repository or DOI.

Circularity Check

1 steps flagged · score 4.0 of 10

One self-referential step: the RQ2 conclusion that GPT labels are 'more accurate' is inferred from classifier F1 measured against those same GPT labels, with no external gold standard; the main BERT-vs-LLM comparison remains independent.

self definitional [Section 5.1, 'Overall Performance' (discussion of Table 6); RQ2 stated in Section 5]
"The use of GPT-powered labeling with human supervision proves to be more effective in detecting fake news than source-level distant labeling. ... On the NELA-GT-2022 dataset with source-level labels, the performance of all models slightly decreases, which indicates that the GPT-powered labels are generally more accurate."

The F1 scores in Table 6 are computed against the test labels of each dataset. For 'Our Dataset,' those test labels are exactly the GPT-4-plus-human labels whose accuracy the paper claims to establish; for NELA-GT, they are the source-level labels being compared. No independent gold standard is introduced (Section 3.1.3 reports only roughly 72% inter-rater agreement and no fact-check validation). Therefore the inference that GPT-powered labels are more accurate reduces by construction to the statement that classifiers trained and evaluated on GPT labels reproduce those same GPT labels better than classifiers reproduce a different distant-labeled corpus. The higher F1 may reflect dataset difficulty, label noise, or annotator bias rather than label accuracy.

full rationale

The paper's central RQ1 comparison—BERT-like encoders versus autoregressive LLMs on a fixed labeled dataset—is a direct empirical benchmark. Given the dataset, the model ranking in Table 6 is self-contained and does not depend on any external derivation chain. The paper does not lean on a load-bearing self-citation chain; its self-citations are background references. The one genuine circular element is the RQ2 claim about label quality. The paper uses its own GPT-4-plus-human labels as the ground truth for training and evaluation, and then treats high classifier F1 on that same labeled test set as evidence that the labels themselves are more accurate than source-level distant labels. Because no independent fact-checked gold standard is used, and because the reported inter-rater reliability is only about 72% without chance-corrected agreement, the 'GPT-powered labels are more accurate' conclusion is a self-referential validation: the labels are both the input and the criterion. This affects a secondary but advertised contribution, not the core model-comparison result, so a moderate score of 4 is appropriate rather than a higher score reserved for cases where the central derivation itself reduces to its inputs.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper's conclusions rest on the quality of its AI-plus-human labels and on several experimental design choices (thresholds, number of votes, applied perturbations). No new theoretical entities are introduced. The lack of an external gold standard and the confounding in the label comparison are the main burdens.

free parameters (3)

confidence threshold = 0.8
Used to accept the majority label for LLMs and to convert BERT probabilities into final classes (Section 4.3). The paper says it was determined through experiments, but Table 3 lists a default of 0.5, creating an inconsistency.
inference repetitions for majority voting = 5
Each article is prompted five times during LLM inference. The authors state the choice balances computational efficiency and prediction stability but do not show a systematic search.
annotation prompt demonstrations = two-shot
GPT-4 Turbo labeling used two in-context examples. This prompt design affects label quality and is a hand-made choice rather than a tested variable.

assumptions (3)

domain assumption GPT-4 with two demonstrations, followed by human consensus review, produces sufficiently accurate ground-truth labels for news veracity.
The entire evaluation uses these labels as ground truth. The paper only reports inter-rater agreement of about 72% and no independent validation.
domain assumption SMOTE can be applied to textual data for transformer fine-tuning to correct class imbalance.
The paper states SMOTE was applied to both BERT and LLM training data but does not describe how (e.g., on embeddings or token representations). Standard SMOTE operates on numeric feature vectors.
domain assumption Text perturbations from TextAttack preserve the original veracity label.
In the robustness test, a single real headline was modified (drop, insertion, negation, etc.), and any misclassification is counted as a robustness failure. The assumption that these perturbations do not change the truth value is not checked.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Data." pith.science (2026). https://pith.science/paper/34UOLXJU

@misc{pith2026241214276,
  author       = {Pith},
  title        = {Pith review of: Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/34UOLXJU}},
  note         = {Machine review of arXiv:2412.14276}
}

read the original abstract

Fake news poses a significant threat to public opinion and social stability in modern society. This study presents a comparative evaluation of BERT-like encoder-only models and autoregressive decoder-only large language models (LLMs) for fake news detection. We introduce a dataset of news articles labeled with GPT-4 assistance (an AI-labeling method) and verified by human experts to ensure reliability. Both BERT-like encoder-only models and LLMs were fine-tuned on this dataset. Additionally, we developed an instruction-tuned LLM approach with majority voting during inference for label generation. Our analysis reveals that BERT-like models generally outperform LLMs in classification tasks, while LLMs demonstrate superior robustness against text perturbations. Compared to weak labels (distant supervision) data, the results show that AI labels with human supervision achieve better classification results. This study highlights the effectiveness of combining AI-based annotation with human oversight and demonstrates the performance of different families of machine learning models for fake news detection

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

83 extracted references · 51 canonical work pages

[1]

International Journal of Data Science and Analytics 13, 335–362 (2022) https://doi.org/10.1007/s41060-021-00302-z

Raza, S., Ding, C.: Fake news detection based on news content and social con- texts: a transformer-based approach. International Journal of Data Science and Analytics 13, 335–362 (2022) https://doi.org/10.1007/s41060-021-00302-z

work page doi:10.1007/s41060-021-00302-z 2022
[2]

Knowledge- Based Systems 294, 111715 (2024)

Truic˘ a, C.-O., Apostol, E.-S., Karras, P.: Danes: Deep neural network ensemble architecture for social and textual context-aware fake news detection. Knowledge- Based Systems 294, 111715 (2024)

2024
[3]

Nature (2024)

Adam, D.: Misinformation might sway elections-but not in the way that you think. Nature (2024)

2024
[4]

ACM Transactions on Intelligent Systems and Technology (TIST) 10(3), 1–42 (2019)

Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., Liu, Y.: Com- bating fake news: A survey on identification and mitigation techniques. ACM Transactions on Intelligent Systems and Technology (TIST) 10(3), 1–42 (2019)

2019
[6]

Multimedia Tools and Appli- cations 80(8), 11765–11788 (2021) https://doi.org/10.1007/s11042-020-10183-2

Kaliyar, R.K., Goswami, A., Narang, P.: FakeBERT: Fake news detection in social media with a BERT-based deep learning approach. Multimedia Tools and Appli- cations 80(8), 11765–11788 (2021) https://doi.org/10.1007/s11042-020-10183-2

doi:10.1007/s11042-020-10183-2 2021
[7]

Big data 8(3), 171–188 (2020)

Shu, K., Mahudeswaran, D., Wang, S., Lee, D., Liu, H.: Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data 8(3), 171–188 (2020)

work page 2020
[8]

In: Rogers, A., Boyd-Graber, J., Okazaki, N

Wang, Z., Hershcovich, D.: On Evaluating Multilingual Compositional Gener- alization with Translated Datasets. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1669–1687. Associa- tion for Computational Linguistics, Toronto, Canada (2023...

work page doi:10.18653/v1/2023.acl-long.93 2023
[9]

arXiv preprint arXiv:2305.13169 (2023)

Longpre, S., Yauney, G., Reif, E., Lee, K., Roberts, A., Zoph, B., Zhou, D., Wei, J., Robinson, K., Mimno, D., et al.: A pretrainer’s guide to training data: Measuring the effects of data age, domain coverage, quality, & toxicity. arXiv preprint arXiv:2305.13169 (2023)

arXiv 2023

Show all 83 references

[10]

arXiv preprint arXiv:1705.00648 (2017)

Wang, W.Y.: ” liar, liar pants on fire”: A new benchmark dataset for fake news detection. arXiv preprint arXiv:1705.00648 (2017)

2017 arXiv
[11]

In: Webber, B., Cohn, T., He, Y., Liu, Y

Vo, N., Lee, K.: Where are the facts? searching for fact-checked information to alleviate the spread of fake news. In: Webber, B., Cohn, T., He, Y., Liu, Y. (eds.) Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7717–7731. As...

2020 doi
[12]

In: Proceedings of the International AAAI Conference on Web and Social Media, vol

Mitra, T., Gilbert, E.: Credbank: A large-scale social media corpus with associated credibility annotations. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 9, pp. 258–267 (2015)

2015
[13]

arXiv preprint arXiv:1911.03854 (2019)

Nakamura, K., Levy, S., Wang, W.Y.: r/fakeddit: A new multimodal benchmark dataset for fine-grained fake news detection. arXiv preprint arXiv:1911.03854 (2019)

2019 arXiv
[14]

arXiv preprint arXiv:2011.04088 (2020)

Li, Y., Jiang, B., Shu, K., Liu, H.: Mm-covid: A multilingual and multi- modal data repository for combating covid-19 disinformation. arXiv preprint arXiv:2011.04088 (2020)

2020 arXiv
[15]

Gruppi, M., Horne, B.D., Adalı, S.: NELA-GT-2019: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles (2020)

2020
[16]

In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pp

Mintz, M., Bills, S., Snow, R., Jurafsky, D.: Distant supervision for relation extraction without labeled data. In: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP,...

2009
[17]

arXiv preprint arXiv:2303.15056 (2023)

Gilardi, F., Alizadeh, M., Kubli, M.: Chatgpt outperforms crowd-workers for text- annotation tasks. arXiv preprint arXiv:2303.15056 (2023)

2023 arXiv
[18]

arXiv preprint arXiv:2402.13446 (2024)

Tan, Z., Beigi, A., Wang, S., Guo, R., Bhattacharjee, A., Jiang, B., Karami, M., Li, J., Cheng, L., Liu, H.: Large language models for data annotation: A survey. arXiv preprint arXiv:2402.13446 (2024)

2024 arXiv
[19]

arXiv preprint arXiv:2303.16854 (2023)

He, X., Lin, Z., Gong, Y., Jin, A., Zhang, H., Lin, C., Jiao, J., Yiu, S.M., Duan, N., Chen, W., et al.: Annollm: Making large language models to be better crowdsourced annotators. arXiv preprint arXiv:2303.16854 (2023)

2023 arXiv
[20]

arXiv preprint arXiv:2310.06825 (2023)

Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, 24 D.d.l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al.: Mistral 7b. arXiv preprint arXiv:2310.06825 (2023)

2023 arXiv
[21]

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi` ere, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G.: LLaMA: Open and Efficient Foundation Language Models (2023)

2023
[22]

https://openai.com/

OpenAI: GPT-3.5. https://openai.com/. Accessed: 2023-12-24 (2023)

2023
[23]

International Journal of Multimedia Information Retrieval 12(2), 28 (2023)

Tufchi, S., Yadav, A., Ahmed, T.: A comprehensive survey of multimodal fake news detection techniques: advances, challenges, and opportunities. International Journal of Multimedia Information Retrieval 12(2), 28 (2023)

2023
[24]

In: Douligeris, C., Karagian- nis, D., Apostolou, D

Liu, C., Wu, X., Yu, M., Li, G., Jiang, J., Huang, W., Lu, X.: A Two-Stage Model Based on BERT for Short Fake News Detection. In: Douligeris, C., Karagian- nis, D., Apostolou, D. (eds.) Knowledge Science, Engineering and Management. Lecture Notes in Computer Science, pp. 172–1...

2019 doi
[25]

In: 2022 Interna- tional Conference on Informatics Electrical and Electronics (ICIEE), pp

Lu, M.F., Renaldy, Ciptadi, V., Nathanael, R., Andaria, K.S., Girsang, A.S.: Fake News Classifier with Deep Learning. In: 2022 Interna- tional Conference on Informatics Electrical and Electronics (ICIEE), pp. 1–4 (2022). https://doi.org/10.1109/ICIEE55596.2022.10010120 . https...

2022
[26]

ACM Computing Surveys 53 (2020) https://doi

Zhou, X., Zafarani, R.: A Survey of Fake News: Fundamental Theories, Detection Methods, and Opportunities. ACM Computing Surveys 53 (2020) https://doi. org/10.1145/3395046

2020 doi
[27]

Journal of economic perspectives 31(2), 211–236 (2017)

Allcott, H., Gentzkow, M.: Social media and fake news in the 2016 election. Journal of economic perspectives 31(2), 211–236 (2017)

2017
[28]

In: Goyal, D., Kumar, A., Piuri, V., Paprzycki, M

Arora, Y., Sikka, S.: Reviewing Fake News Classification Algorithms. In: Goyal, D., Kumar, A., Piuri, V., Paprzycki, M. (eds.) Proceedings of the Third Inter- national Conference on Information Management and Machine Intelligence. Algorithms for Intelligent Systems, pp. 425–42...

2023 doi
[29]

In: 2022 13th International Conference on Computing Communication and Net- working Technologies (ICCCNT), pp

Bonny, A.J., Bhowmik, P., Mahmud, M.S., Sattar, A.: Detecting Fake News in Benchmark English News Dataset Using Machine Learning Classifiers. In: 2022 13th International Conference on Computing Communication and Net- working Technologies (ICCCNT), pp. 1–8 (2022). https://doi.o...

2022
[30]

In: H¨ urriyeto˘ glu, A

Raza, S.: Automatic Fake News Detection in Political Platforms - A Transformer- based Approach. In: H¨ urriyeto˘ glu, A. (ed.) Proceedings of the 4th Work- shop on Challenges and Applications of Automated Extraction of Socio- political Events from Text (CASE 2021), pp. 68–78. ...

2021 doi
[31]

Scientific reports11(1), 23705 (2021)

Szczepa´ nski, M., Pawlicki, M., Kozik, R., Chora´ s, M.: New explainability method for bert-based model in fake news detection. Scientific reports11(1), 23705 (2021)

2021
[32]

In: 2019 IEEE International Conference on Big Data (big Data), pp

Raza, S., Ding, C.: News recommender system considering temporal dynamics and news taxonomy. In: 2019 IEEE International Conference on Big Data (big Data), pp. 920–929 (2019). IEEE

2019
[33]

In: 2019 IEEE International Conference on Data Mining (ICDM), pp

Qi, P., Cao, J., Yang, T., Guo, J., Li, J.: Exploiting multi-domain visual infor- mation for fake news detection. In: 2019 IEEE International Conference on Data Mining (ICDM), pp. 518–527 (2019). IEEE

2019
[34]

Heliyon9(10) (2023) https://doi.org/10.1016/j.heliyon.2023.e20382

Hamed, S.K., Ab Aziz, M.J., Yaakub, M.R.: A review of fake news detection approaches: A critical analysis of relevant studies and highlighting key challenges associated with the dataset, feature representation, and data fusion. Heliyon9(10) (2023) https://doi.org/10.1016/j.hel...

2023 doi
[35]

Expert Systems with Applications 158, 113503 (2020)

Faustini, P.H.A., Covoes, T.F.: Fake news detection in multiple platforms and languages. Expert Systems with Applications 158, 113503 (2020)

2020
[36]

Sitaula, N., Mohan, C.K., Grygiel, J., Zhou, X., Zafarani, R.: Credibility-Based Fake News Detection, pp. 163–182. Springer, ??? (2020). https://doi.org/10.1007/ 978-3-030-42699-6 9

2020
[37]

Electronics 10 (2021) https://doi.org/10.3390/ electronics10111348

Alonso, M.A., Vilares, D., G´ omez-Rodr ´ ıguez, C., Vilares, J.: Sentiment anal- ysis for fake news detection. Electronics 10 (2021) https://doi.org/10.3390/ electronics10111348

2021
[38]

Truica, C.-O., Leordeanu, C.A.: Classification of an imbalanced data set using decision tree algorithms. Univ. Politech. Bucharest Sci. Bull. Ser. C Electr. Eng. Comput. Sci 79, 69–84 (2017)

2017
[39]

Multimedia Tools and Applications 82 (2022) https://doi.org/10.1007/s11042-022-12668-8

Jarrahi, A., Safari, L.: Evaluating the effectiveness of publishers’ features in fake news detection on social media. Multimedia Tools and Applications 82 (2022) https://doi.org/10.1007/s11042-022-12668-8

2022 doi
[40]

Complex & Intelligent Systems 9 (2023) https://doi.org/ 10.1007/s40747-023-01098-0

Essa, E., Omar, K., Alqahtani, A.: Fake news detection based on a hybrid bert and lightgbm models. Complex & Intelligent Systems 9 (2023) https://doi.org/ 10.1007/s40747-023-01098-0

2023 doi
[41]

Information Systems Frontiers26(2), 443–458 (2024)

Olan, F., Jayawickrama, U., Arakpogun, E.O., Suklan, J., Liu, S.: Fake news on 26 social media: the impact on society. Information Systems Frontiers26(2), 443–458 (2024)

2024
[42]

IEEE Access 9, 162122–162146 (2021)

Ilie, V.-I., Truic˘ a, C.-O., Apostol, E.-S., Paschke, A.: Context-aware misin- formation detection: A benchmark of deep learning architectures using word embeddings. IEEE Access 9, 162122–162146 (2021)

2021
[43]

Mathematics 10(4), 569 (2022)

Truic˘ a, C.-O., Apostol, E.-S.: Misrobærta: transformers versus misinformation. Mathematics 10(4), 569 (2022)

2022
[44]

In: CEUR Workshop Proc, vol

Truic˘ a, C.-O., Apostol, E.-S., Paschke, A.: Awakened at checkthat! 2022: Fake news detection using bilstm and sentence transformer. In: CEUR Workshop Proc, vol. 3180, pp. 749–757 (2022)

2022
[45]

Mathematics 11(3), 508 (2023)

Truic˘ a, C.-O., Apostol, E.-S.: It’s all in the embedding! fake news detection using document embeddings. Mathematics 11(3), 508 (2023)

2023
[46]

In: 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp

Jain, S., Sharma, V., Kaushal, R.: Towards automated real-time detection of misinformation on twitter. In: 2016 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 2015–2020 (2016). IEEE

2016
[47]

harmful speech

Petrescu, A., Truic˘ a, C.-O., Apostol, E.-S., Karras, P.: Sparse shield: Social network immunization vs. harmful speech. In: Proceedings of the 30th ACM Inter- national Conference on Information & Knowledge Management, pp. 1426–1436 (2021)

2021
[48]

Information Processing & Management 58(1), 102390 (2021)

Zhao, Y., Da, J., Yan, J.: Detecting health misinformation in online health communities: Incorporating behavioral features into machine learning based approaches. Information Processing & Management 58(1), 102390 (2021)

2021
[49]

Systems 11(9), 458 (2023)

Yuan, L., Jiang, H., Shen, H., Shi, L., Cheng, N.: Sustainable development of information dissemination: A review of current fake news detection research and practice. Systems 11(9), 458 (2023)

2023
[50]

Applied Soft Computing 139, 110235 (2023)

Phan, H.T., Nguyen, N.T., Hwang, D.: Fake news detection: A survey of graph neural network methods. Applied Soft Computing 139, 110235 (2023)

2023
[51]

IEEE Access 11, 125861–125873 (2023)

Truic˘ a, C.-O., Apostol, E.-S., Nicolescu, R.-C., Karras, P.: Mcwdst: a minimum- cost weighted directed spanning tree algorithm for real-time fake news mitigation in social media. IEEE Access 11, 125861–125873 (2023)

2023
[52]

arXiv preprint arXiv:2303.01934 (2023)

Apostol, E.-S., Coban, ¨O., Truic˘ a, C.-O.: Contain: A community-based algorithm for network immunization. arXiv preprint arXiv:2303.01934 (2023)

2023 arXiv
[53]

IEEE Transactions on Knowledge and Data Engineering (2024)

Apostol, E.-S., Truic˘ a, C.-O., Paschke, A.: Contcommrtd: A distributed content- based misinformation-aware community detection system for real-time disaster 27 reporting. IEEE Transactions on Knowledge and Data Engineering (2024)

2024
[54]

International Journal of Data Science and Analytics 13(4), 335–362 (2022)

Raza, S., Ding, C.: Fake news detection based on news content and social con- texts: a transformer-based approach. International Journal of Data Science and Analytics 13(4), 335–362 (2022)

2022
[55]

Social Network Analysis and Mining 13(1), 30 (2023) https://doi.org/10.1007/s13278-023-01028-5

A ¨ ımeur, E., Amri, S., Brassard, G.: Fake news, disinformation and misinformation in social media: a review. Social Network Analysis and Mining 13(1), 30 (2023) https://doi.org/10.1007/s13278-023-01028-5

2023 doi
[56]

arXiv preprint arXiv:2302.04023 (2023)

Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al.: A multitask, multilingual, multimodal eval- uation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023 (2023)

2023 arXiv
[57]

arXiv preprint arXiv:1804.04866 (2018)

Heller, S., Rossetto, L., Schuldt, H.: The ps-battles dataset-an image collection for image manipulation detection. arXiv preprint arXiv:1804.04866 (2018)

2018 arXiv
[58]

In: 2023 International Conference on Data Security and Privacy Protection (DSPP), pp

Yang, H., Zhang, J., Hu, Z., Zhang, L., Cheng, X.: Multimodal relationship-aware attention network for fake news detection. In: 2023 International Conference on Data Security and Privacy Protection (DSPP), pp. 143–149 (2023). IEEE

2023
[59]

IEEE Transactions on Computational Social Systems 8(4), 881–893 (2021)

Verma, P.K., Agrawal, P., Amorim, I., Prodan, R.: Welfake: word embedding over linguistic features for fake news detection. IEEE Transactions on Computational Social Systems 8(4), 881–893 (2021)

2021
[60]

Frontiers in Communication 6, 661801 (2021)

Gaillard, S., Ol´ ah, Z.A., Venmans, S., Burke, M.: Countering the cognitive, lin- guistic, and psychological underpinnings behind susceptibility to fake news: A review of current literature with special focus on the role of age and digital literacy. Frontiers in Communication...

2021
[61]

In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp

Wu, J., Guo, J., Hooi, B.: Fake news in sheep’s clothing: Robust fake news detection against llm-empowered style attacks. In: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3367–3378 (2024)

2024
[62]

arXiv preprint arXiv:2311.05656 (2023)

Chen, C., Shu, K.: Combating misinformation in the age of llms: Opportunities and challenges. arXiv preprint arXiv:2311.05656 (2023)

2023 arXiv
[63]

arXiv preprint arXiv:2409.03067 (2024)

Xu, R., Li, G.: A comparative study of offline models and online llms in fake news detection. arXiv preprint arXiv:2409.03067 (2024)

2024 arXiv
[64]

In: 2023 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp

Wu, G., Wu, W., Liu, X., Xu, K., Wan, T., Wang, W.: Cheap-fake detection with llm using prompt engineering. In: 2023 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 105–109 (2023). IEEE

2023
[65]

arXiv preprint arXiv:2402.11943 (2024)

Xuan, K., Yi, L., Yang, F., Wu, R., Fung, Y.R., Ji, H.: Lemma: Towards 28 lvlm-enhanced multimodal misinformation detection with external knowledge augmentation. arXiv preprint arXiv:2402.11943 (2024)

2024 arXiv
[66]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

Qi, P., Yan, Z., Hsu, W., Lee, M.L.: Sniffer: Multimodal large language model for explainable out-of-context misinformation detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13052– 13062 (2024)

2024
[67]

Scientific Reports 13(1), 8591 (2023)

Raza, S., Schwartz, B.: Constructing a disease database and using natural language processing to capture and standardize free text clinical information. Scientific Reports 13(1), 8591 (2023)

2023
[68]

Science 363(6425), 374–378 (2019)

Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., Lazer, D.: Fake news on twitter during the 2016 us presidential election. Science 363(6425), 374–378 (2019)

2019
[69]

Knowledge-Based Systems 274, 110642 (2023) https: //doi.org/10.1016/j.knosys.2023.110642

Alghamdi, J., Lin, Y., Luo, S.: Towards COVID-19 fake news detection using transformer-based models. Knowledge-Based Systems 274, 110642 (2023) https: //doi.org/10.1016/j.knosys.2023.110642

2023
[70]

arXiv preprint arXiv:2402.18050 (2024)

Kim, H., Mitra, K., Chen, R.L., Rahman, S., Zhang, D.: Meganno+: A human-llm collaborative annotation system. arXiv preprint arXiv:2402.18050 (2024)

2024 arXiv
[71]

arXiv preprint arXiv:2402.11073 (2024)

Ni, J., Shi, M., Stammbach, D., Sachan, M., Ash, E., Leippold, M.: Afacta: Assist- ing the annotation of factual claim detection with reliable llm annotators. arXiv preprint arXiv:2402.11073 (2024)

2024 arXiv
[72]

Addison-Wesley Professional, ??? (2023)

Ozdemir, S.: Quick Start Guide to Large Language Models: Strategies and Best Practices for Using ChatGPT and Other LLMs. Addison-Wesley Professional, ??? (2023)

2023
[73]

https://openai.com/index/gpt-4/

OpenAI: OpenAI GPT-4. https://openai.com/index/gpt-4/. Accessed: 2024-06- 22

2024
[74]

arXiv preprint arXiv:2302.11382 (2023)

White, J., Fu, Q., Hays, S., Sandborn, M., Olea, C., Gilbert, H., Elnashar, A., Spencer-Smith, J., Schmidt, D.C.: A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382 (2023)

2023 arXiv
[75]

Nature Machine Intelligence 5(3), 220–235 (2023)

Ding, N., Qin, Y., Yang, G., Wei, F., Yang, Z., Su, Y., Hu, S., Chen, Y., Chan, C.-M., Chen, W., et al.: Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence 5(3), 220–235 (2023)

2023
[76]

Advances in Neural Information Processing Systems 36 (2024)

Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: Qlora: Efficient fine- tuning of quantized llms. Advances in Neural Information Processing Systems 36 (2024)

2024
[77]

Raschka, S.: Finetuning llms with lora and qlora: Insights from hundreds of 29 experiments (2023)

2023
[78]

Gruppi, M., Horne, B.D., Adalı, S.: NELA-GT-2022: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles (2023)

2023
[79]

Journal of artificial intelligence research 16, 321–357 (2002)

Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. Journal of artificial intelligence research 16, 321–357 (2002)

2002
[80]

arXiv preprint arXiv:2307.03109 (2023)

Chang, Y., Wang, X., Wang, J., Wu, Y., Zhu, K., Chen, H., Yang, L., Yi, X., Wang, C., Wang, Y., et al.: A survey on evaluation of large language models. arXiv preprint arXiv:2307.03109 (2023)

2023 arXiv
[81]

Expert Systems with Applications 237, 121542 (2024)

Raza, S., Garg, M., Reji, D.J., Bashir, S.R., Ding, C.: Nbias: A natural lan- guage processing framework for bias identification in text. Expert Systems with Applications 237, 121542 (2024)

2024
[82]

In: Sil, A., Lin, X.V

Ranasinghe, T., Zampieri, M.: MUDES: Multilingual Detection of Offensive Spans. In: Sil, A., Lin, X.V. (eds.) Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies: Demonstrations, pp. 1...

2021 doi
[83]

Settles, B.: Active learning literature survey (2009)

2009
[84]

In: Proceedings of The 18th Linguistic Annotation Workshop (LA W-XVIII), pp

Rouzegar, H., Makrehchi, M.: Enhancing text classification through llm-driven active learning and human annotation. In: Proceedings of The 18th Linguistic Annotation Workshop (LA W-XVIII), pp. 98–111 (2024) 30

2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

International Journal of Data Science and Analytics 13, 335–362 (2022) https://doi.org/10.1007/s41060-021-00302-z

Raza, S., Ding, C.: Fake news detection based on news content and social con- texts: a transformer-based approach. International Journal of Data Science and Analytics 13, 335–362 (2022) https://doi.org/10.1007/s41060-021-00302-z

work page doi:10.1007/s41060-021-00302-z 2022

[2] [2]

Knowledge- Based Systems 294, 111715 (2024)

Truic˘ a, C.-O., Apostol, E.-S., Karras, P.: Danes: Deep neural network ensemble architecture for social and textual context-aware fake news detection. Knowledge- Based Systems 294, 111715 (2024)

2024

[3] [3]

Nature (2024)

Adam, D.: Misinformation might sway elections-but not in the way that you think. Nature (2024)

2024

[4] [4]

ACM Transactions on Intelligent Systems and Technology (TIST) 10(3), 1–42 (2019)

Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., Liu, Y.: Com- bating fake news: A survey on identification and mitigation techniques. ACM Transactions on Intelligent Systems and Technology (TIST) 10(3), 1–42 (2019)

2019

[5] [6]

Multimedia Tools and Appli- cations 80(8), 11765–11788 (2021) https://doi.org/10.1007/s11042-020-10183-2

Kaliyar, R.K., Goswami, A., Narang, P.: FakeBERT: Fake news detection in social media with a BERT-based deep learning approach. Multimedia Tools and Appli- cations 80(8), 11765–11788 (2021) https://doi.org/10.1007/s11042-020-10183-2

doi:10.1007/s11042-020-10183-2 2021

[6] [7]

Big data 8(3), 171–188 (2020)

Shu, K., Mahudeswaran, D., Wang, S., Lee, D., Liu, H.: Fakenewsnet: A data repository with news content, social context, and spatiotemporal information for studying fake news on social media. Big data 8(3), 171–188 (2020)

work page 2020

[7] [8]

In: Rogers, A., Boyd-Graber, J., Okazaki, N

Wang, Z., Hershcovich, D.: On Evaluating Multilingual Compositional Gener- alization with Translated Datasets. In: Rogers, A., Boyd-Graber, J., Okazaki, N. (eds.) Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1669–1687. Associa- tion for Computational Linguistics, Toronto, Canada (2023...

work page doi:10.18653/v1/2023.acl-long.93 2023

[8] [9]

arXiv preprint arXiv:2305.13169 (2023)

Longpre, S., Yauney, G., Reif, E., Lee, K., Roberts, A., Zoph, B., Zhou, D., Wei, J., Robinson, K., Mimno, D., et al.: A pretrainer’s guide to training data: Measuring the effects of data age, domain coverage, quality, & toxicity. arXiv preprint arXiv:2305.13169 (2023)

arXiv 2023