{"paper":{"title":"Are LLMs More Skeptical of Entertainment News?","license":"http://creativecommons.org/licenses/by/4.0/","headline":"Some large language models misclassify legitimate entertainment news as fake at higher rates than hard news.","cross_cats":["cs.CY"],"primary_cat":"cs.AI","authors_text":"Huiqian Lai","submitted_at":"2026-05-03T05:55:00Z","abstract_excerpt":"Large language models (LLMs) are increasingly used for automated news credibility assessment, yet it remains unclear whether they apply even-handed standards across journalistic genres. We examine whether zero-shot LLMs are more likely to misclassify legitimate entertainment news as fake than legitimate hard news, using a within-dataset design on GossipCop from FakeNewsNet. Across four frontier models, we find a clear but model-specific genre asymmetry: DeepSeek-V3.2 and GPT-5.2 show false-positive-rate gaps of 10.1 and 8.8 percentage points, respectively (both $p < .001$), whereas Claude Opus"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"Across four frontier models, we find a clear but model-specific genre asymmetry: DeepSeek-V3.2 and GPT-5.2 show false-positive-rate gaps of 10.1 and 8.8 percentage points, respectively (both p < .001), whereas Claude Opus 4.6 and Gemini 3 Flash show no comparable difference.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That the within-dataset design on GossipCop sufficiently isolates genre effects from confounding differences in topic, source, or unverifiability of private-life claims between entertainment and hard news.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"Certain frontier LLMs exhibit higher false-positive rates on legitimate entertainment news than hard news, with model-specific patterns not explained by style alone.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Some large language models misclassify legitimate entertainment news as fake at higher rates than hard news.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"d190bb12ac227399c2561ea8a717a0fcde89b67c9f70c9a162f7b92cd88cfc60"},"source":{"id":"2605.01727","kind":"arxiv","version":1},"verdict":{"id":"cd3e19a2-ad21-4442-bd5a-43952fb8a946","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-10T16:24:57.696423Z","strongest_claim":"Across four frontier models, we find a clear but model-specific genre asymmetry: DeepSeek-V3.2 and GPT-5.2 show false-positive-rate gaps of 10.1 and 8.8 percentage points, respectively (both p < .001), whereas Claude Opus 4.6 and Gemini 3 Flash show no comparable difference.","one_line_summary":"Certain frontier LLMs exhibit higher false-positive rates on legitimate entertainment news than hard news, with model-specific patterns not explained by style alone.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That the within-dataset design on GossipCop sufficiently isolates genre effects from confounding differences in topic, source, or unverifiability of private-life claims between entertainment and hard news.","pith_extraction_headline":"Some large language models misclassify legitimate entertainment news as fake at higher rates than hard news."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2605.01727/integrity.json","findings":[],"available":true,"detectors_run":[{"name":"ai_meta_artifact","ran_at":"2026-05-20T17:37:31.901475Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"doi_title_agreement","ran_at":"2026-05-20T05:01:23.130086Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"doi_compliance","ran_at":"2026-05-19T17:00:50.742878Z","status":"completed","version":"1.0.0","findings_count":0}],"snapshot_sha256":"c2fa63669aab41d6eb1405d8cf085a4721477d2099b2874aa40e76a38d58131a"},"references":{"count":13,"sample":[{"doi":"","year":2017,"title":"SoK: Machine Learning for Misinformation Detec- tion.arXiv preprint arXiv:2308.12215. Horne, B.; and Adali, S. 2017. This Just In: Fake News Packs A Lot In Title, Uses Simpler, Repetitive Content in T","work_id":"b9dfe36b-8928-403f-94fb-c14ccb029d09","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"A Survey on the Use of Large Language Models (LLMs) in Fake News.Future Internet, 16(8). Pelrine, K.; Mosber, A.; Zheng, J.; Yang, J.-Y .; Peng, A.; Rabbany, R.; and Cheung, J. C. K. 2023. Towards Re-","work_id":"981f6e87-a0bf-410b-8d41-49962f61c18c","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2018,"title":"P´erez-Rosas, V .; Kleinberg, B.; Lefevre, A.; and Mihalcea, R","work_id":"2311371a-907b-4de1-aa69-d3b324fb2fe6","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2017,"title":"In Palmer, M.; Hwa, R.; and Riedel, S., eds.,Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2931–2937","work_id":"ccf60c23-b01a-4517-820e-0aa8ea16f097","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2019,"title":"Hard and Soft News: A Review of Concepts, Oper- ationalizations and Key Findings.Journalism, 13(2): 221– 239. Roberts, S. T. 2019.Behind the screen : content modera- tion in the shadows of social medi","work_id":"7b1e133e-83d7-424b-9f5a-18db4bb4005b","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":13,"snapshot_sha256":"5d9333e09d13e8d2e0b73b8ca026ab4d5144537e2be1f42d59bb0bc76c9458ad","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}