REVIEW 4 major objections 5 minor 40 references
An Emotional Analysis of False Information in Social Media and News Articles
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper claims false information is not emotionally uniform—propaganda, hoax, clickbait, and satire each have a different emotional pattern—and that emotion features added to an LSTM improve false-news detection in both Twitter and news…
desk verdict Solid empirical study of emotion features for fake news detection, but the causal claim in the abstract outruns the source-level labels and the clickbait confound. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the Emotionally-Infused Network (EIN), a two-branch LSTM classifier. One branch reads word embeddings and produces an attention-weighted context vector $c_t=\sum_j \alpha_{tj} h_j$; the other builds a $q$-dimensional emotion vector $v = L_1^{Em}\oplus L_2^{Em}\oplus L_3^{Em}\oplus L_4^{Em}\oplus L_5^{Em}$ by pooling word frequencies over five emotion lexicons (EmoSenticNet, EmoLex, SentiSense, LIWC, Empath) and normalizing by sentence length. The two vectors are concatenated, passed through dense layers, and fed to a softmax over false-information types. The emotion branch is what lets the network exploit the paper's core claim: emotions differ by type, so emotion counts carry classification signal beyond the words themselves.
What would settle it
Take a dataset in which each article or tweet has been individually fact-checked, and run EIN on it. If the accuracy gain from the emotion branch shrinks or disappears once source identity and genre are controlled for, the emotion signal is a signature of the source's style rather than a marker of the content being false.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that false news types are emotionally distinguishable and that emotion features materially improve detection. In the news-articles corpus the top discriminating emotions are joy, anticipation, fear, and disgust; in Twitter they are joy, sadness, fear, and disgust, and the overall ranking shapes are similar across sources. A t-test finds statistically significant differences between false and real news for eight emotions (joy, negative emotion, ambiguous, anticipation, calmness, disgust, trust, surprise) in both datasets. Each type also has its own signature: clickbait leans on surprise and negative emotion, propaganda oscillates between extreme positive and negative emotions with calmness, satire uses disgust for humor, and hoaxes stay emotionally close to real news. The result is operationalized as the Emotionally-Infused Network, which beats content-only baselines and reaches near-96% F1 as a binary clickbait detector.
Load-bearing premise
The labels are trusted at the document level: articles are classified by the reliability of the site that published them and tweets by the reputation of the account that posted them, with no fact-checking of individual items, so the emotional patterns could partly be patterns of source style or topic rather than of falsehood itself.
Editorial extensions
If this is right
- Emotion features are a portable addition to fake-news classifiers: EIN beats the content-only LSTM by about six accuracy points on news articles and 1.5 points on Twitter.
- Because the emotion-importance rankings are similar in both sources, a model trained on news-article emotion patterns may transfer to Twitter, where text is noisier and word-embedding coverage is weaker.
- Clickbait is the most emotionally separable type—surprise and negative emotion dominate—and EIN reaches about 96% F1 on a dedicated clickbait benchmark.
- Propaganda's emotional shifting between extremes (joy and fear, plus calmness or hope) can be read as a manipulation tactic, while satire's disgust and hoaxes' near-neutrality give each type a distinct detection cue.
- Eight emotions show statistically significant differences between false and real news in both datasets, offering explainable features rather than a black-box signal.
Reading between the lines
- The emotion fingerprints may in fact be genre fingerprints: because labels come from unreliable sites and suspicious accounts, the same method applied to individually verified claims might show weaker or different emotion patterns.
- The cross-source similarity in emotion ranking suggests a domain-adaptation experiment the authors did not run: train EIN on news articles and test on Twitter using only emotion features.
- A natural extension is to test whether the emotion signal is causal for engagement—for example, whether clickbait headlines that score higher on surprise actually receive more clicks, linking the classification result to reader psychology.
- The deception-via-emotion account could be probed experimentally by asking readers to rate perceived manipulativeness of items matched on emotion scores; the model predicts that high-surprise and high-fear false items are judged more deceptive.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the emotional content of false information (satire, hoax, propaganda, clickbait) compared with real news, using two datasets: news articles from unreliable versus trusted sites, and tweets from suspicious versus trusted accounts. The authors propose an Emotionally-Infused Network (EIN), an LSTM with an additional branch fed by emotion-lexicon features, and report that EIN improves false-information detection over content-only baselines (news accuracy 80.72 vs. 74.79; Twitter accuracy 64.82 vs. 63.29, Table 4). They also analyze emotion importance via Information Gain, run t-tests for per-emotion differences between false and real news, and list top emotions per false-information type.
Significance. If the results hold, the paper provides evidence that emotion-derived features are a portable and useful signal for automated false-information detection across two very different text sources, and it offers a descriptive emotional profile of specific false-information types. The work is solidly motivated, uses several publicly available resources, compares against multiple baselines, and includes an emotion-only model to isolate the contribution of affect. The main empirical claim, however, rests on dataset labels assigned by source and account reliability rather than per-item verification, and the reported improvements are not accompanied by measures of uncertainty. The analysis is therefore suggestive rather than conclusive, and the abstract's causal phrasing goes beyond what the data can establish.
major comments (4)
- [4.1, Table 1] The dataset labels are assigned by the reliability of the publishing site or Twitter account, not by per-item fact-checking. News articles are labeled false if they come from one of seven unreliable sites, and tweets are labeled from lists of suspicious/trusted accounts taken from prior work. If an unreliable site publishes a true story, or a trusted account posts a neutral statement, the emotion patterns and EIN improvements may reflect source style or topic rather than the truth of individual documents. The clickbait class in the news dataset is also spliced in from a separate headline corpus (Stop Clickbait), which introduces a format confound. This directly affects the abstract's conclusion that 'emotions play a key role in deceiving the reader.' I recommend softening the causal claims and, if possible, testing on a fact-checked subset (e.g., Politifact or Snopes items) to show that the emotional signal survives per-item verification.
- [Table 4, Section 5.2] The central detection result is reported from a single train/test split, with no error bars, confidence intervals, or significance tests. On Twitter, the accuracy gap between EIN and the LSTM baseline is only 64.82 vs. 63.29 (1.53 points); with typical run-to-run variance in LSTM training, this difference may not be stable. Please report results over multiple runs (e.g., 5 or 10 random seeds) with means and standard deviations, or perform paired significance tests between EIN and the best baseline for each dataset. Without this, the claimed improvement is not robustly established.
- [Section 6, RQ3 / Figure 5] The t-tests for statistical significance are performed for 17 emotions without any correction for multiple comparisons. Under the null hypothesis, about one test in 20 will appear significant at the 5% level, so reporting eight or nine 'significant' emotions overstates the evidence. Please apply a multiple-comparison correction (e.g., Benjamini-Hochberg) or report effect sizes and adjusted p-values. This is load-bearing for the claim that false and real news differ systematically across many specific emotions.
- [Section 5.3, Table 5] The clickbait-detector experiment, while a nice extension, is presented as evidence that 'emotions play a key role in deceiving the reader' in clickbait. However, the Stop_Clickbait dataset consists of headlines (clickbait vs. Wikinews headlines), so the near-96% F1 may capture headline-vs-headline format differences rather than emotion-driven deception in full articles. Moreover, the same source is used for the clickbait class in the main news dataset, so the separability visible in Figure 3 may be partly a format artifact. Please discuss this confound explicitly and, if possible, report the contribution of emotional features on a clickbait set that uses full article text.
minor comments (5)
- [Section 3.3, Eq. (3)] The notation in Eq. (3) is inconsistent with Eq. (2): 'LiEM' should be 'LiEm' to match the definition of the emotional vector, and the indexing should be clarified.
- [Section 6, RQ4] The sentence 'we can proof this hypothesis' contains a grammatical error; it should be 'prove this hypothesis.'
- [Figure 3] The t-SNE projections are described in the text but the figure captions do not indicate which class corresponds to which color or symbol; please add a legend.
- [Section 5.2] The description of the validation protocol is a bit confusing: the authors say they 'take another 20% from the training part as a validation set,' which is standard, but the contrast with the 10-fold CV used for the baselines is stated only in a footnote; a sentence in the main text would improve clarity.
- [References] Reference [9] is incomplete in the reference list (the title is cut after 'UPV-INAOE-Autoritas - Check That'); please complete it.
Circularity Check
No significant circularity: the emotional-infusion gains are empirical comparisons against content-only baselines.
full rationale
The paper's central empirical claim—that an emotionally-infused LSTM (EIN) outperforms content-only and feature-based baselines—is not circular: the emotional branch is built from five external lexicons (EmoSenticNet, EmoLex, SentiSense, LIWC, Empath), the content branch is a standard LSTM plus attention, and the comparison is made on a held-out 20% test split with baselines trained under the same protocol (Tables 2 and 4). The emotional-feature gains (e.g., 79.43 vs. 72.26 macro-F1 on news articles, 59.70 vs. 55.41 on Twitter) are therefore an empirical outcome rather than an identity. The interpretive claims that emotions 'play a key role in deceiving the reader' are inferences from classifier performance and from IG/t-test analyses on the same corpora; they raise external-validity questions about source-level labels (Section 4.1) and about the clickbait class having been sampled from the Stop Clickbait dataset, but those are validity concerns, not circular reductions. The paper's few self-citations ([8], [9], [10], [31]) are used for resources or prior motivation and are not load-bearing for the main result. No equation or fitted parameter is shown to be equivalent to the target claim by construction.
Assumptions & free parameters
free parameters (5)
- Article truncation length N =
300 words
- Twitter posts per account M =
1000
- LSTM units =
90 to 180 depending on dataset
- Dense_a and Dense_b units =
Varies per dataset (e.g., 320/60 for news, 100/60 for Twitter)
- Dropout rates (Dropc, Dropd) =
0.1 to 0.5
assumptions (3)
- domain assumption Emotional lexicons provide valid, comparable measures of emotion in short tweets and long news article excerpts.
- domain assumption Source-level labels (unreliable news sites, suspicious Twitter accounts, known clickbait sites) are correct for each individual document.
- ad hoc to paper The first 300 words of a news article contain the emotionally relevant signal.
Cite this review
Pith. "Pith review of An Emotional Analysis of False Information in Social Media and News Articles." pith.science (2026). https://pith.science/paper/GQA33BSI
@misc{pith2026190809951,
author = {Pith},
title = {Pith review of: An Emotional Analysis of False Information in Social Media and News Articles},
year = {2026},
howpublished = {\url{https://pith.science/paper/GQA33BSI}},
note = {Machine review of arXiv:1908.09951}
}
read the original abstract
Fake news is risky since it has been created to manipulate the readers' opinions and beliefs. In this work, we compared the language of false news to the real one of real news from an emotional perspective, considering a set of false information types (propaganda, hoax, clickbait, and satire) from social media and online news articles sources. Our experiments showed that false information has different emotional patterns in each of its types, and emotions play a key role in deceiving the reader. Based on that, we proposed a LSTM neural network model that is emotionally-infused to detect false news.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
Magda B Arnold. 1960. Emotion and Personality. Columbia University Press
work page 1960
-
[2]
Gaurav Bhatt, Aman Sharma, Shivam Sharma, Ankush Nagpal, Balasubramanian Raman, and Ankush Mittal. 2018. Combining Neural, Statistical and External Features for Fake News Stance Identification. In Companion of the The , Vol. 1, No. 1, Article . Publication date: August 2018. 16 Ghanem, B. et al Web Conference 2018 on The Web Conference 2018 . Internationa...
work page 2018
-
[3]
Carlos Castillo, Marcelo Mendoza, and Barbara Poblete. 2011. Information Credibility on Twitter. In Proceedings of the 20th international conference on World wide web . ACM, 675–684
work page 2011
-
[4]
Abhijnan Chakraborty, Bhargavi Paranjape, Sourya Kakarla, and Niloy Ganguly. 2016. Stop Clickbait: Detecting and Preventing Clickbaits in Online News Media. In Proceedings of the 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining . IEEE Press, 9–16
work page 2016
-
[5]
Jorge Carrillo de Albornoz, Laura Plaza, and Pablo Gervás. 2012. SentiSense: An easily scalable concept-based affective lexicon for sentiment analysis.. In LREC. 3562–3567
work page 2012
-
[6]
Paul Ekman. 1992. An Argument for Basic Emotions. Cognition & emotion 6, 3-4 (1992), 169–200
work page 1992
-
[7]
Ethan Fast, Binbin Chen, and Michael S Bernstein. 2016. Empath: Understanding Topic Signals in Large-Scale Text. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems . ACM, 4647–4657
work page 2016
-
[8]
Simona Frenda, Bilal Ghanem, and Manuel Montes-y Gòmez. 2018. Exploration of Misogyny in Spanish and English Tweets. In Third Workshop on Evaluation of Human Language Technologies for Iberian Languages (IberEval 2018) , Vol. 2150. Ceur Workshop Proceedings, 260–267
work page 2018
Show all 40 references
-
[9]
Bilal Ghanem, Manuel Montes-y Gòmez, Francisco Rangel, and Paolo Rosso. 2018. UPV-INAOE-Autoritas - Check That: An Approach based on External Sources to Detect Claims Credibility. In In Working Notes of CLEF 2018 - Conference and Labs of the Evaluation Forum, CLEF ’18
2018
-
[10]
Bilal Ghanem, Paolo Rosso, and Francisco Rangel. 2018. Stance Detection in Fake News A Combined Feature Repre- sentation, In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER). EMNLP 2018, 66–71
2018
-
[11]
Andreas Hanselowski, Avinesh PVS, Benjamin Schiller, Felix Caspelherr, Debanjan Chaudhuri, Christian M Meyer, and Iryna Gurevych. 2018. A Retrospective Analysis of the Fake News Challenge Stance Detection Task. arXiv preprint arXiv:1806.05180 (2018)
2018 arXiv
-
[12]
Delia Irazú Hernańdez Farías, Viviana Patti, and Paolo Rosso. 2016. Irony Detection in Twitter: The Role of Affective Content. ACM Transactions on Internet Technology (TOIT) 16, 3 (2016), 19
2016
-
[13]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory.Neural computation 9, 8 (1997), 1735–1780
1997
-
[14]
Georgi Karadzhov, Preslav Nakov, Lluís Màrquez, Alberto Barrón-Cedeño, and Ivan Koychev. 2017. Fully Automated Fact Checking Using External Sources. In Proceedings of the International Conference Recent Advances in Natural Language Processing, RANLP 2017 . 344–353
2017
-
[15]
Alireza Karduni, Ryan Wesslen, Sashank Santhanam, Isaac Cho, Svitlana Volkova, Dustin Arendt, Samira Shaikh, and Wenwen Dou. 2018. Can You Verifi This? Studying Uncertainty and Decision-Making About Misinformation Using Visual Analytics. (2018)
2018
-
[16]
Elena Kochkina, Maria Liakata, and Arkaitz Zubiaga. 2018. All-in-one: Multi-task Learning for Rumour Verification. arXiv preprint arXiv:1806.03713 (2018)
2018 arXiv
-
[17]
Srijan Kumar, Robert West, and Jure Leskovec. 2016. Disinformation on the Web: Impact, Characteristics, and Detection of Wikipedia Hoaxes. In Proceedings of the 25th international conference on World Wide Web . International World Wide Web Conferences Steering Committee, 591–602
2016
-
[18]
Xian Li, Weiyi Meng, and Clement Yu. 2011. T-verifier: Verifying Truthfulness of Fact Statements. In Data Engineering (ICDE), 2011 IEEE 27th International Conference on . IEEE, 63–74
2011
-
[19]
Mario Livio. 2017. Why?: What Makes Us Curious . Simon and Schuster Publishing
2017
-
[20]
Jing Ma, Wei Gao, Prasenjit Mitra, Sejeong Kwon, Bernard J Jansen, Kam-Fai Wong, and Meeyoung Cha. 2016. Detecting Rumors from Microblogs with Recurrent Neural Networks. In IJCAI. 3818–3824
2016
-
[21]
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing Data Using t-SNE.Journal of machine learning research 9, Nov (2008), 2579–2605
2008
-
[22]
Saif M Mohammad and Peter D Turney. 2010. Emotions Evoked by Common Words and Phrases: Using Mechanical Turk to Create an Emotion Lexicon. In Proceedings of the NAACL HLT 2010 workshop on computational approaches to analysis and generation of emotion in text . Association for ...
2010
-
[23]
Brendan Nyhan and Jason Reifler. 2010. When Corrections Fail: The Persistence of Political Misperceptions. Political Behavior 32, 2 (2010), 303–330
2010
-
[24]
W Gerrod Parrott. 2001. Emotions in Social Psychology: Essential Readings . Psychology Press
2001
-
[25]
Robert Plutchik. 2001. The Nature of Emotions: Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. American scientist 89, 4 (2001), 344–350
2001
-
[26]
Kashyap Popat, Subhabrata Mukherjee, Jannik Strötgen, and Gerhard Weikum. 2016. Credibility Assessment of Textual Claims on the Web. InProceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM, 2173–2178
2016
-
[27]
Kashyap Popat, Subhabrata Mukherjee, Andrew Yates, and Gerhard Weikum. 2018. DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning. arXiv preprint arXiv:1809.06416 (2018). , Vol. 1, No. 1, Article . Publication date: August 2018. An Emotional Analysi...
2018 arXiv
-
[28]
Soujanya Poria, Alexander Gelbukh, Amir Hussain, Newton Howard, Dipankar Das, and Sivaji Bandyopadhyay. 2013. Enhanced SenticNet with Affective Labels for Concept-Based Opinion Mining. IEEE Intelligent Systems 28, 2 (2013), 31–38
2013
-
[29]
Vahed Qazvinian, Emily Rosengren, Dragomir R Radev, and Qiaozhu Mei. 2011. Rumor has it: Identifying Misinforma- tion in Microblogs. In Proceedings of the Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, 1589–1599
2011
-
[30]
Colin Raffel and Daniel PW Ellis. 2015. Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems. arXiv preprint arXiv:1512.08756 (2015)
2015 arXiv
-
[31]
Francisco Rangel and Paolo Rosso. 2016. On the impact of emotions on author profiling. Information processing & management 52, 1 (2016), 73–92
2016
-
[32]
Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2931–2937
2017
-
[33]
Victoria L Rubin, Yimin Chen, and Niall J Conroy. 2015. Deception Detection for News: Three Types of Fakes. In Proceedings of the 78th ASIS&T Annual Meeting: Information Science with Impact: Research in and for the Community . American Society for Information Science, 83
2015
-
[34]
Natali Ruchansky, Sungyong Seo, and Yan Liu. 2017. Csi: A Hybrid Deep Model for Fake News Detection. InProceedings of the 2017 ACM on Conference on Information and Knowledge Management . ACM, 797–806
2017
-
[35]
Yla R Tausczik and James W Pennebaker. 2010. The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods. Journal of language and social psychology 29, 1 (2010), 24–54
2010
-
[36]
Svitlana Volkova, Kyle Shaffer, Jin Yea Jang, and Nathan Hodas. 2017. Separating facts from fiction: Linguistic models to classify suspicious and trusted news posts on twitter. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2...
2017
-
[37]
Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The Spread of True and False News Online. Science 359, 6380 (2018), 1146–1151
2018
-
[38]
Po-Ya Angela Wang. 2013. #Irony or #Sarcasm - A Quantitative and Qualitative Study Based on Twitter. InProceedings of the 27th Pacific Asia Conference on Language, Information, and Computation (PACLIC 27) . 349–356
2013
-
[39]
Savvas Zannettou, Michael Sirivianos, Jeremy Blackburn, and Nicolas Kourtellis. 2018. The Web of False Information: Rumors, Fake News, Hoaxes, Clickbait, and Various Other Shenanigans. arXiv preprint arXiv:1804.03461 (2018)
2018 arXiv
-
[40]
Zhe Zhao, Paul Resnick, and Qiaozhu Mei. 2015. Enquiring Minds: Early Detection of Rumors in Social Media from Enquiry Posts. In Proceedings of the 24th International Conference on World Wide Web . International World Wide Web Conferences Steering Committee, 1395–1405. , Vol. ...
2015
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.