Pith. sign in

REVIEW 5 cited by

Fake News Detectors are Biased against Texts Generated by Large Language Models

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2309.08674 v1 pith:6PAPLBKW submitted 2023-09-15 cs.CL cs.AI

Jinyan Su , Terry Yue Zhuo , Jonibek Mansurov , Di Wang , Preslav Nakov This is my paper
classification cs.CLcs.AI
keywords newsfakellm-generateddetectorsbiascontentgenuinehuman-written
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The spread of fake news has emerged as a critical challenge, undermining trust and posing threats to society. In the era of Large Language Models (LLMs), the capability to generate believable fake content has intensified these concerns. In this study, we present a novel paradigm to evaluate fake news detectors in scenarios involving both human-written and LLM-generated misinformation. Intriguingly, our findings reveal a significant bias in many existing detectors: they are more prone to flagging LLM-generated content as fake news while often misclassifying human-written fake news as genuine. This unexpected bias appears to arise from distinct linguistic patterns inherent to LLM outputs. To address this, we introduce a mitigation strategy that leverages adversarial training with LLM-paraphrased genuine news. The resulting model yielded marked improvements in detection accuracy for both human and LLM-generated news. To further catalyze research in this domain, we release two comprehensive datasets, \texttt{GossipCop++} and \texttt{PolitiFact++}, thus amalgamating human-validated articles with LLM-generated fake and real news.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Tailored untruths: How personalisation challenges LLM safeguards

    cs.CL 2025-10 conditional novelty 7.0 of 10

    A 1.6-million-text study of eight LLMs in four languages finds that adding demographic personae to disinformation prompts raises jailbreak rates from 78% to 82%.

  2. COMPKE: Complex Question Answering under Knowledge Editing

    cs.CL 2025-06 conditional novelty 6.0 of 10

    COMPKE is a new benchmark with 11,924 complex questions that tests knowledge editing through one-to-many relations and logical operations, where existing editing methods often fail.

  3. Large Language Models in Misinformation Ecosystems: Misuse, Defense, and Vulnerability

    cs.CR 2026-07 conditional novelty 5.0 of 10

    A role-layer survey unifies LLM misuse, LLM-based defense, and LLM-centric verification vulnerabilities across content, social, evidence, and workflow layers, then lists three open challenges.

  4. The Compositional Architecture of Regret in Large Language Models

    cs.CL 2025-06 reject novelty 5.0 of 10

    The paper claims that regret in LLMs is encoded by interacting neuron groups detectable in the final hidden layer, using new S-CDI, RDS, and GIC metrics.

  5. Truth over Tricks: Measuring and Mitigating Shortcut Learning in Misinformation Detection

    cs.CL 2025-06 conditional novelty 5.0 of 10

    Across 16 datasets, trainable misinformation detectors drop sharply under LLM-induced surface rewrites, and LLM-based rewriting helps recover accuracy but with important caveats.

Pith tools