{"id":"52ac6fbe-2b92-459e-8ba7-548d2b279b14","arxiv_id":"1908.09951","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Adding emotion-lexicon features to an LSTM improves false-news detection by about 2 to 7 macro-F1 points and reveals type-specific emotion profiles across Twitter and news articles.","lead":"This paper compares emotional language in four types of false news (propaganda, hoax, clickbait, and satire) with real news in Twitter and online news articles, and builds an emotion-infused LSTM detector. It finds that emotions differ by false-news type and that adding emotion features improves detection over content-only baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The emotional-feature gains may reflect source-level labels rather than per-item falsity, since Section 4.1 labels documents by site/account reliability, and the clickbait class adds a headline-vs-article format confound.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: documents are labeled by source or account reliability, not by item-level truth, and the clickbait class introduces a source-format confound. I agree that this is the most consequential threat to the central claim. The abstract's assertion that emotions play a key role in deceiving the reader requires that the classifier is distinguishing false content from true content; if labels are merely source-level, the learned emotion signals may be distinguishing the stylistic fingerprints of dubious outlets or accounts. The clickbait subset sharpens the problem because its real class is drawn from Wikinews headlines, so the reported clickbait detection accuracy may reflect a trivial format difference. The paper is otherwise a competent empirical study with plausible results, and the EIN gains are reported in both datasets, so an outright rejection would be too harsh. A conditional verdict is appropriate: the authors should release the data and code, add per-item validation or use an existing fact-checked benchmark, and report uncertainty or significance tests. Since the reader already reached CONDITIONAL, my stress-test does not change the verdict.","tokens_in":14044,"tokens_out":3698,"duration_ms":43666,"concrete_test":"Retrain the LSTM and EIN models from Section 5.2 on a dataset with per-claim or per-article fact-check labels, such as FakeNewsNet or LIAR, using the same five emotional lexicons and the same architecture. If the emotion-branch gain over the content-only LSTM largely disappears on per-item verified labels, the source-level labeling in the current corpora is the likely driver of the reported improvement; if the gain persists, the central claim is substantially supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim in Table 4 is that adding an emotional-feature branch to an LSTM improves false-information detection (news: 79.43 vs. 72.26 macro-F1; Twitter: 59.70 vs. 55.41). This claim is load-bearing for the abstract's conclusion that 'emotions play a key role in deceiving the reader.' The weakest point is the dataset labels. In Section 4.1, news articles are labeled as false or true according to the reliability of the publishing site (trusted Gigaword vs. seven unreliable sites), and Twitter posts are labeled from lists of suspicious/trusted accounts taken from prior work [15, 36]. No per-item fact-checking is reported for either corpus. If an unreliable site occasionally posts a true story, or a trusted account posts a neutral statement, the emotion patterns and EIN gains may reflect stylistic or topical properties of particular sources, accounts, or platforms rather than the truth of individual documents. The clickbait class is especially exposed: it is spliced in from the Stop Clickbait dataset [4], where the positive class consists of clickbait headlines and the negative class consists of Wikinews headlines, so the strong clickbait separability in Figure 3 and the 96% F1 in Table 5 may be partly a headline-versus-article format effect. Consequently, the paper's causal wording overstates what the data can show. This is not an internal inconsistency, but it is a serious external-validity gap in the argument that emotions are diagnostic of false information itself.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the emotional content of false information (satire, hoax, propaganda, clickbait) compared with real news, using two datasets: news articles from unreliable versus trusted sites, and tweets from suspicious versus trusted accounts. The authors propose an Emotionally-Infused Network (EIN), an LSTM with an additional branch fed by emotion-lexicon features, and report that EIN improves false-information detection over content-only baselines (news accuracy 80.72 vs. 74.79; Twitter accuracy 64.82 vs. 63.29, Table 4). They also analyze emotion importance via Information Gain, run t-tests for per-emotion differences between false and real news, and list top emotions per false-information type.","tokens_in":14337,"tokens_out":3218,"duration_ms":32492,"significance":"If the results hold, the paper provides evidence that emotion-derived features are a portable and useful signal for automated false-information detection across two very different text sources, and it offers a descriptive emotional profile of specific false-information types. The work is solidly motivated, uses several publicly available resources, compares against multiple baselines, and includes an emotion-only model to isolate the contribution of affect. The main empirical claim, however, rests on dataset labels assigned by source and account reliability rather than per-item verification, and the reported improvements are not accompanied by measures of uncertainty. The analysis is therefore suggestive rather than conclusive, and the abstract's causal phrasing goes beyond what the data can establish.","major_comments":[{"comment":"The dataset labels are assigned by the reliability of the publishing site or Twitter account, not by per-item fact-checking. News articles are labeled false if they come from one of seven unreliable sites, and tweets are labeled from lists of suspicious/trusted accounts taken from prior work. If an unreliable site publishes a true story, or a trusted account posts a neutral statement, the emotion patterns and EIN improvements may reflect source style or topic rather than the truth of individual documents. The clickbait class in the news dataset is also spliced in from a separate headline corpus (Stop Clickbait), which introduces a format confound. This directly affects the abstract's conclusion that 'emotions play a key role in deceiving the reader.' I recommend softening the causal claims and, if possible, testing on a fact-checked subset (e.g., Politifact or Snopes items) to show that the emotional signal survives per-item verification.","section":"4.1, Table 1"},{"comment":"The central detection result is reported from a single train/test split, with no error bars, confidence intervals, or significance tests. On Twitter, the accuracy gap between EIN and the LSTM baseline is only 64.82 vs. 63.29 (1.53 points); with typical run-to-run variance in LSTM training, this difference may not be stable. Please report results over multiple runs (e.g., 5 or 10 random seeds) with means and standard deviations, or perform paired significance tests between EIN and the best baseline for each dataset. Without this, the claimed improvement is not robustly established.","section":"Table 4, Section 5.2"},{"comment":"The t-tests for statistical significance are performed for 17 emotions without any correction for multiple comparisons. Under the null hypothesis, about one test in 20 will appear significant at the 5% level, so reporting eight or nine 'significant' emotions overstates the evidence. Please apply a multiple-comparison correction (e.g., Benjamini-Hochberg) or report effect sizes and adjusted p-values. This is load-bearing for the claim that false and real news differ systematically across many specific emotions.","section":"Section 6, RQ3 / Figure 5"},{"comment":"The clickbait-detector experiment, while a nice extension, is presented as evidence that 'emotions play a key role in deceiving the reader' in clickbait. However, the Stop_Clickbait dataset consists of headlines (clickbait vs. Wikinews headlines), so the near-96% F1 may capture headline-vs-headline format differences rather than emotion-driven deception in full articles. Moreover, the same source is used for the clickbait class in the main news dataset, so the separability visible in Figure 3 may be partly a format artifact. Please discuss this confound explicitly and, if possible, report the contribution of emotional features on a clickbait set that uses full article text.","section":"Section 5.3, Table 5"}],"minor_comments":[{"comment":"The notation in Eq. (3) is inconsistent with Eq. (2): 'LiEM' should be 'LiEm' to match the definition of the emotional vector, and the indexing should be clarified.","section":"Section 3.3, Eq. (3)"},{"comment":"The sentence 'we can proof this hypothesis' contains a grammatical error; it should be 'prove this hypothesis.'","section":"Section 6, RQ4"},{"comment":"The t-SNE projections are described in the text but the figure captions do not indicate which class corresponds to which color or symbol; please add a legend.","section":"Figure 3"},{"comment":"The description of the validation protocol is a bit confusing: the authors say they 'take another 20% from the training part as a validation set,' which is standard, but the contrast with the 10-fold CV used for the baselines is stated only in a footnote; a sentence in the main text would improve clarity.","section":"Section 5.2"},{"comment":"Reference [9] is incomplete in the reference list (the title is cut after 'UPV-INAOE-Autoritas - Check That'); please complete it.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and relevant topic, and the proposed model is reasonable. The main concern is external validity: the dataset construction (source-level labels, clickbait headline confound) makes it hard to attribute the observed emotional signals to the falsity of individual documents. The authors should temper the causal language and add robustness checks. The lack of significance testing for the main result is also concerning, especially given the small margin on Twitter. These are addressable issues, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a decent, workmanlike empirical study of emotion signals in fake news detection. The genuinely new piece is the multi-lexicon emotional analysis across four false-information types (propaganda, hoax, clickbait, satire) and two sources (Twitter and news articles), plus the EIN model that concatenates emotion-lexicon counts with an LSTM. The reported numbers support the operational claim: adding the emotional branch beats the content-only LSTM by about 7 points macro-F1 on news and 4 points on Twitter, and the emotion-only model clearly beats random and majority baselines. The descriptive emotion profiles (clickbait→surprise, satire→disgust, propaganda→joy/fear/calmness) are plausible and line up with definitions.\n\nWhere I'd push back is the gap between the evidence and the conclusion. The abstract says 'emotions play a key role in deceiving the reader.' The data are correlational, and the labels are at the source level, not per-item truth. News articles are labeled by the reliability of the publishing site; tweets are labeled by account lists. If an unreliable site occasionally posts a true story, or a suspicious account posts a neutral tweet, the emotion patterns may reflect source style or topic rather than falsity. The clickbait class is the most exposed: those instances come from a separate headline corpus (Stop Clickbait) with Wikinews headlines as the negative class, so the strong separability and 96% F1 partly reflect a headline-versus-article format difference. That's not an internal inconsistency, but it is a real external-validity problem.\n\nOther soft spots: no error bars or significance tests on the classifier comparisons, no multiple-comparison correction for the 17-emotion t-tests, and no code or data release. Those are fixable in revision and don't undercut the main empirical finding, but they should be addressed before the detection claim is fully trusted.\n\nOverall, this is a credible contribution to a crowded subfield. It deserves a serious referee, and I'd send it out. But the authors should soften the causal language, deal with the clickbait confound explicitly, and release artifacts. It's not a blockbuster, but it's honest, reproducible-in-principle work that moves the needle a little.","headline":"Solid empirical study of emotion features for fake news detection, but the causal claim in the abstract outruns the source-level labels and the clickbait confound.","tokens_in":14898,"tokens_out":2559,"would_cite":false,"duration_ms":24109,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims false information is not emotionally uniform—propaganda, hoax, clickbait, and satire each have a different emotional pattern—and that emotion features added to an LSTM improve false-news detection in both Twitter and news…","keywords":["fake news detection","emotion analysis","false information types","emotionally-infused neural network","LSTM","attention mechanism","Twitter dataset","news articles dataset"],"falsifier":"Take a dataset in which each article or tweet has been individually fact-checked, and run EIN on it. If the accuracy gain from the emotion branch shrinks or disappears once source identity and genre are controlled for, the emotion signal is a signature of the source's style rather than a marker of the content being false.","tokens_in":13789,"feed_emoji":"📰","tokens_out":5733,"duration_ms":54052,"temperature":0.7,"pith_summary":"This paper tries to establish that false information has a distinctive emotional life: propaganda, hoax, clickbait, and satire each carry different emotional patterns, and those patterns can be used to identify false news. The authors show this with an Emotionally-Infused Network (EIN), an LSTM whose word-embedding branch is augmented by a branch of emotion-lexicon vectors. Across news articles the emotion-infused model reaches 80.72% accuracy against 74.79% for the content-only LSTM, and on Twitter 64.82% against 63.29%. The payoff of the claim is practical: emotion is a cheap, portable feature that can supplement content-based fake-news detectors, and the per-type emotional profiles give an explanation of how each deception style works.","feed_headline":"False news types carry distinct emotional fingerprints","feed_subtitle":"Emotion cues lift fake-news detection accuracy by six points on articles and expose each type's deceiving style.","key_machinery":"The central mechanism is the Emotionally-Infused Network (EIN), a two-branch LSTM classifier. One branch reads word embeddings and produces an attention-weighted context vector $c_t=\\sum_j \\alpha_{tj} h_j$; the other builds a $q$-dimensional emotion vector $v = L_1^{Em}\\oplus L_2^{Em}\\oplus L_3^{Em}\\oplus L_4^{Em}\\oplus L_5^{Em}$ by pooling word frequencies over five emotion lexicons (EmoSenticNet, EmoLex, SentiSense, LIWC, Empath) and normalizing by sentence length. The two vectors are concatenated, passed through dense layers, and fed to a softmax over false-information types. The emotion branch is what lets the network exploit the paper's core claim: emotions differ by type, so emotion counts carry classification signal beyond the words themselves.","core_discovery":"On the paper's own terms, the discovery is that false news types are emotionally distinguishable and that emotion features materially improve detection. In the news-articles corpus the top discriminating emotions are joy, anticipation, fear, and disgust; in Twitter they are joy, sadness, fear, and disgust, and the overall ranking shapes are similar across sources. A t-test finds statistically significant differences between false and real news for eight emotions (joy, negative emotion, ambiguous, anticipation, calmness, disgust, trust, surprise) in both datasets. Each type also has its own signature: clickbait leans on surprise and negative emotion, propaganda oscillates between extreme positive and negative emotions with calmness, satire uses disgust for humor, and hoaxes stay emotionally close to real news. The result is operationalized as the Emotionally-Infused Network, which beats content-only baselines and reaches near-96% F1 as a binary clickbait detector.","pith_inferences":["The emotion fingerprints may in fact be genre fingerprints: because labels come from unreliable sites and suspicious accounts, the same method applied to individually verified claims might show weaker or different emotion patterns.","The cross-source similarity in emotion ranking suggests a domain-adaptation experiment the authors did not run: train EIN on news articles and test on Twitter using only emotion features.","A natural extension is to test whether the emotion signal is causal for engagement—for example, whether clickbait headlines that score higher on surprise actually receive more clicks, linking the classification result to reader psychology.","The deception-via-emotion account could be probed experimentally by asking readers to rate perceived manipulativeness of items matched on emotion scores; the model predicts that high-surprise and high-fear false items are judged more deceptive."],"forward_implications":["Emotion features are a portable addition to fake-news classifiers: EIN beats the content-only LSTM by about six accuracy points on news articles and 1.5 points on Twitter.","Because the emotion-importance rankings are similar in both sources, a model trained on news-article emotion patterns may transfer to Twitter, where text is noisier and word-embedding coverage is weaker.","Clickbait is the most emotionally separable type—surprise and negative emotion dominate—and EIN reaches about 96% F1 on a dedicated clickbait benchmark.","Propaganda's emotional shifting between extremes (joy and fear, plus calmness or hope) can be read as a manipulation tactic, while satire's disgust and hoaxes' near-neutrality give each type a distinct detection cue.","Eight emotions show statistically significant differences between false and real news in both datasets, offering explainable features rather than a black-box signal."],"supporting_citations":[{"why":"It supplies the news-articles dataset (real news from Gigaword and false news from unreliable sites) and the hoax, propaganda, and satire categories used in the experiments.","marker":"[32]"},{"why":"It provides the suspicious Twitter account lists and the tweet-crawling design used to build the Twitter dataset.","marker":"[36]"},{"why":"It supplies the clickbait corpus and the Stop_Clickbait system that EIN is compared against in the clickbait-detector experiment.","marker":"[4]"},{"why":"It shows prior emotional differences between true and false rumours, motivating the emotion-based research questions.","marker":"[37]"},{"why":"It is one of the five emotion lexicons whose word-frequency vectors make up the emotional branch of EIN.","marker":"[28]"},{"why":"It is a second emotion lexicon (EmoLex, Plutchik's eight emotions) used in the emotional branch.","marker":"[22]"},{"why":"It is a third emotion lexicon (SentiSense) contributing its emotion categories to the emotional vector.","marker":"[5]"},{"why":"It is the fourth lexicon, LIWC, contributing its emotional categories (sadness, anger, positive emotion, negative emotion) to the emotional branch.","marker":"[35]"},{"why":"It is the fifth lexicon, Empath, providing the primary Parrott emotions used in the emotional vector.","marker":"[7]"},{"why":"It supplies the LSTM architecture that forms the content branch of EIN and the strongest content-only baseline.","marker":"[13]"}],"fun_headline_variants":["Emotion fingerprints unmask fake news types","Fake news styles exposed by emotion patterns","Distinct emotions betray each fake news type","Emotion-aware AI spots fake news patterns"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The labels are trusted at the document level: articles are classified by the reliability of the site that published them and tweets by the reputation of the account that posted them, with no fact-checking of individual items, so the emotional patterns could partly be patterns of source style or topic rather than of falsehood itself.","fun_headline_variants_meta":{"raw":{"variants":["Emotion fingerprints unmask fake news types","Fake news styles exposed by emotion patterns","Distinct emotions betray each fake news type","Emotion-aware AI spots fake news patterns"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000318,"raw_usage":{"total_tokens":1733,"prompt_tokens":818,"completion_tokens":915,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":434,"completion_tokens_details":{"reasoning_tokens":860}},"tokens_in":434,"tokens_out":915,"duration_ms":9972,"temperature":1.0,"reasoning_tokens":860,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T10:57:02.888513+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a dataset in which each article or tweet has been individually fact-checked, and run EIN on it. If the accuracy gain from the emotion branch shrinks or disappears once source identity and genre are controlled for, the emotion signal is a signature of the source's style rather than a marker of the content being false.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the news-articles dataset (real news from Gigaword and false news from unreliable sites) and the hoax, propaganda, and satire categories used in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the suspicious Twitter account lists and the tweet-crawling design used to build the Twitter dataset."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the clickbait corpus and the Stop_Clickbait system that EIN is compared against in the clickbait-detector experiment."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It shows prior emotional differences between true and false rumours, motivating the emotion-based research questions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is one of the five emotion lexicons whose word-frequency vectors make up the emotional branch of EIN."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is a second emotion lexicon (EmoLex, Plutchik's eight emotions) used in the emotional branch."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is a third emotion lexicon (SentiSense) contributing its emotion categories to the emotional vector."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the fourth lexicon, LIWC, contributing its emotional categories (sadness, anger, positive emotion, negative emotion) to the emotional branch."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It is the fifth lexicon, Empath, providing the primary Parrott emotions used in the emotional vector."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It supplies the LSTM architecture that forms the content branch of EIN and the strongest content-only baseline."}],"review_version":1}