{"id":"0ea15559-0c1d-432f-b832-65b9ee9f7a98","arxiv_id":"2501.06274","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"In large-scale debunking discussions on Twitter and Reddit, peripheral users drive toxicity, platforms differ in how toxicity and pessimism relate, and more replies are associated with less toxic received language.","lead":"Analyzing 86 million tweets and 4 million Reddit comments that mention debunking keywords, this study reports that occasional users contribute much of the toxic language in political debunking threads, and that toxic language correlates with lower pessimism. It argues that Twitter and Reddit's different architectures shape these patterns, which could help platform designers and policymakers target moderation efforts.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core claim that these patterns characterize 'debunking' discourse is unsecured: the keyword-only corpus construction does not distinguish debunking from misinformation spreading, and no validation is reported.","rationale":"The paper asks whether debunking discourse is polarized in toxicity and pessimism. For this to mean anything, the units labeled 'debunking' must be acts of debunking. The keyword definition in Methods does not ensure this: 'fake news' and 'misinformation' are used by misinformation spreaders, by journalists, and by ordinary users discussing the topic, while 'debunk.' can match content that is itself conspiratorial or dismissive. The reader's weakest assumption is exactly this, and I agree with it. The uncited 'Table ??' placeholders and the unspecified pessimism extraction rule make a numerical audit impossible, but the corpus-validity check is the first-order test because it determines what phenomenon is actually being studied. No independent support—such as released code, replication data, or formal verification—offsets the gap. The proposed annotation test is feasible with the existing data and would settle the concern. Until it is run, the central claim should not be accepted as referring specifically to debunking discourse, so I would not change the reader's reject verdict.","tokens_in":17708,"tokens_out":5958,"duration_ms":61420,"concrete_test":"Take a stratified random sample of, say, 500 tweets and 500 Reddit comments per topic from the keyword-matched sets and have two independent annotators code each item as (a) active debunking/correcting of a specific false claim, (b) spreading or endorsing misinformation, (c) neutral discussion of misinformation, or (d) other. Report Cohen's kappa and the proportion of category (a). If the proportion of actual debunking acts is not clearly a majority, and kappa is not adequate, then the corpus label 'debunking' should be abandoned or revised, and the central analyses should be rerun on a validated subset.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All three headline findings are attributed to debunking posts, but the Methods section (Data collection) selects posts solely by keyword: 'fact check,' 'fake news,' 'misinformation,' 'debunk.' A post containing 'fake news' may be an act of correction, a dismissal of unfavorable reporting, or itself a misinformation spread. The paper reports no manual or automated validation that these posts actually perform debunking. Thus the 1-degree-user toxicity result, the platform comparison, and the toxicity-pessimism correlation all refer to a corpus whose construct validity is unestablished. This is the load-bearing assumption: if the label is wrong, the abstract's 'debunking tweets/comments' is a misdescription, and all downstream inferences concern an unlabeled mix of content. The causal phrase 'driven by lower community accountability' is also unsupported by the observational design, but it is secondary to corpus validity.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes 86.7 million Twitter posts and 4.7 million Reddit comments retrieved with debunking-related keywords around the 2016 and 2020 U.S. presidential elections and QAnon. Using Perspective API, VADER, a RoBERTa emotion model, and retweet/reply network analyses, it reports three main findings: peripheral 1-degree users contribute disproportionately to toxic discourse; platform architecture shapes polarization, with Twitter amplifying partisan differences and Reddit showing higher overall toxicity; and language toxicity correlates negatively with pessimism, while increased replying is associated with reduced toxicity. The authors interpret these patterns as evidence about the dynamics of debunking discourse on social media.","tokens_in":17876,"tokens_out":5314,"duration_ms":55499,"significance":"If the central claims were supported, the paper would be a valuable large-scale cross-platform investigation of toxicity and sentiment in misinformation-correction contexts, with practical implications for moderation and platform design. The scale of the data, the combination of network and NLP measures, and the manual verification of the political classification are notable strengths. However, the contribution is conditional on resolving the construct-validity and reporting problems described below; in its current form the central findings cannot be evaluated from the manuscript as written.","major_comments":[{"comment":"The corpus is built solely by querying keywords such as \"fact check,\" \"fake news,\" \"misinformation,\" and \"debunk,\" but the paper provides no validation that posts containing these keywords actually perform debunking. A post containing \"fake news\" may be an act of correction, a dismissal of unfavorable reporting, or itself misinformation. Because all three headline findings are attributed to \"debunking tweets/comments,\" this is a load-bearing assumption. Please report a manual or automated validation of a random sample of the retrieved posts, report agreement, and either restrict the corpus to verified debunking content or revise all claims to refer to posts matching debunking-related keywords.","section":"Methods, Data collection"},{"comment":"The paper never operationalizes pessimism. It names the model \"cardiffnlp/twitter-roberta-base-emotion-multilabel-latest\" but does not state which emotion labels are used, how they are combined into a pessimism score, or whether a threshold is applied. Since the negative toxicity-pessimism correlation in the Results is a central finding, the missing mapping makes that analysis unverifiable. Please define the pessimism score explicitly, ideally with an equation or a table of label-to-score weights.","section":"Methods, Sentiment calculation and pessimism detection"},{"comment":"The statistical evidence for nearly every quantitative claim is reported only as \"Table ??\": Pearson correlations, Mann-Whitney U tests, Cliff's delta effect sizes, regression slopes and intercepts, and reply-regression statistics are all referenced to missing tables. For example, the subsection \"Negative relationship between language toxicity and pessimism\" cites \"Table ??\" for all correlation coefficients, and \"Polarization of replying\" references \"Table ??\" for slopes and intercepts. Without the actual tables, the claims cannot be checked. Please supply complete tables with sample sizes, test statistics, and confidence intervals, and correct all cross-references.","section":"Results, throughout"},{"comment":"The assertion that peripheral users shape toxic discourse \"driven by lower community accountability and emotional expression\" is a causal explanation, but the observational design only measures retweet/reply degree, toxicity, and sentiment. No variable for accountability or community investment is measured, and no causal identification is used. Please rephrase this as an observed association and discuss alternative mechanisms, such as selection into peripheral participation or topic-specific activity.","section":"Abstract and Discussion"},{"comment":"The analysis regresses the maximum toxicity and pessimism received by replied-to users on log-transformed reply counts and interprets the negative slopes as evidence that \"sustained interaction\" reduces toxicity. This is an observational correlation and is subject to endogeneity: users who receive few replies may differ systematically from heavily replied-to users in topic, visibility, or prior behavior. Please soften the causal language and consider controlling for user-level confounders or explicitly labeling the finding as correlational.","section":"Results, Polarization of replying"}],"minor_comments":[{"comment":"VADER is misspaced as \"V ADER\" in multiple places; please correct the spacing.","section":"Methods, Sentiment calculation and pessimism detection"},{"comment":"The pseudocode contains the typo \"out put\" for \"output,\" and the step size of 0.1 with strict inequalities should be clarified to avoid floating-point edge cases.","section":"Algorithm 1"},{"comment":"There is a typo \"20120 U.S. presidential election\" in the paragraph discussing the 2020 election; please correct it.","section":"Results, Polarization of replying"},{"comment":"The text reports \"P = 0.0\" in one Mann-Whitney U test; statistical results should be reported as inequalities (for example, P < 0.001) rather than as exactly zero.","section":"Results, Polarization in 1-Degree and 2-Core Users"},{"comment":"Table 2 is labeled as the minimal entropy interval for \"50% of users,\" but the user rates are 50.1% to 53.4%; the threshold and the reason for exceeding 50% should be stated more precisely.","section":"Table 2"},{"comment":"Several reported Pearson correlations are quite small (for example, r = -0.079 and r = -0.126); the text describes them as showing a \"consistent negative relationship,\" but the practical significance and confidence intervals should be discussed once the missing tables are supplied.","section":"Results, Negative relationship between language toxicity and pessimism"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be an unfinished draft, with \"Table ??\" placeholders throughout the Results section and no operational definition of pessimism. The corpus-validity issue raised by the stress-test is real and should be addressed head-on; if the authors cannot validate that the retrieved posts are debunking content, the paper's claims need to be substantially reframed. I would not consider this for acceptance until the missing tables and validation analyses are provided."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things worth knowing about this paper. First, the empirical observations—especially the negative toxicity–pessimism correlation and the disproportionate toxicity from 1-degree users—are novel at this scale and worth taking seriously. Second, the manuscript as submitted cannot be checked: the Results section is littered with 'Table ??' placeholders, and the pessimism variable is never operationalized. You can't verify the central claims from the text.\n\nWhat the paper does well: it assembles a massive corpus (86M tweets, 4.7M Reddit comments) across two platforms and three political topics, applies standard toxicity and sentiment detectors, and reports a manual validation of the political classification with reasonable Cohen's kappa. The cross-platform comparison (Twitter vs Reddit) is a useful design. The negative correlation between toxicity and pessimism, if real, is a genuinely counterintuitive finding that goes against the paper's own prior.\n\nNow the soft spots, in proportion. The most serious is construct validity. The corpus is selected by keywords like 'fake news' and 'misinformation,' which are used by debunkers and by people spreading or dismissing real reporting. The paper reports no validation that these posts actually perform debunking. Without that, all three headline findings describe an unvalidated mixture of content, not 'debunking discourse.' This is the load-bearing issue, and it is not minor.\n\nSecond, the 'pessimism' score is never defined. The methods name a RoBERTa emotion model, but never say how emotion labels become a scalar pessimism score. The main negative correlation is therefore uninterpretable.\n\nThird, the statistical reporting is incomplete to the point of non-verifiability. Many results point to 'Table ??' with no numbers. There are also internal inconsistencies: the QAnon 2-core user count appears as 3,463 in the text and 13,463 in Table 1; the abstract says 86M tweets but the topic subsets sum to about 9M. These may be fixable, but they undermine confidence.\n\nSome minor issues: the causal phrase 'driven by lower community accountability' exceeds the observational design, and the platform comparison is complicated by using retweet networks on Twitter but reply networks on Reddit. The entropy interval algorithm is descriptive but fine.\n\nWho this is for: a reader interested in large-scale descriptive patterns of toxicity and sentiment would find value after revision. The current version should not be cited for its conclusions.\n\nRecommendation: I would not desk-reject this; the questions are worth answering and the scale is a real asset. Send it to peer review with a clear request for major revision: full statistical tables, an operational definition of pessimism, a validation study of the debunking corpus, and removal of causal language. But my own confidence in the findings as stated is low until those pieces appear.","headline":"Potentially interesting large-scale observations, but the manuscript is missing its statistical backbone and conflates keyword mentions with debunking.","tokens_in":18393,"tokens_out":3326,"would_cite":false,"duration_ms":31645,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Analyzing over 86 million Twitter posts and more than 4 million Reddit comments, this paper finds that peripheral users, platform design, and reply frequency shape toxicity and pessimism in debunking debates independently of party…","keywords":["language toxicity","pessimism","debunking","political polarization","social media platforms","Twitter","Reddit","misinformation"],"falsifier":"Take a random sample of the keyword-matched posts, have independent annotators classify each as a genuine debunking attempt (e.g., explicitly labeling a claim false) versus other uses, then recompute the toxicity–pessimism correlations and the 1-degree/2-core effect sizes on the confirmed-debunking subset only. If the negative correlations or peripheral-user effects vanish or flip sign, the keyword operationalization of 'debunking post' is the point of failure.","tokens_in":17515,"feed_emoji":"💬","tokens_out":10770,"duration_ms":94897,"temperature":0.7,"pith_summary":"The paper sets out to show that polarization in online debunking discussions is not only a partisan divide: it also runs along user engagement, platform architecture, and emotional tone. Examining 86.7 million Twitter posts and 4.7 million Reddit comments about the 2016 and 2020 U.S. elections and QAnon, the authors report three regularities: lightly engaged peripheral users carry a disproportionate share of toxic language, Twitter amplifies partisan gaps while Reddit sustains higher overall toxicity, and language toxicity is negatively correlated with pessimism, with more replies associated with less toxicity, especially on Reddit. If these regularities hold, debunking campaigns and moderation tools would need to target peripheral participants and platform-specific interaction structures, not just partisan echo chambers.","feed_headline":"Peripheral users drive toxic debunking debates on Twitter and Reddit","feed_subtitle":"86M tweets and 4M Reddit comments place polarization in user engagement and platform design, not just party.","key_machinery":"The machinery is a set of measurement pairings: a network-degree partition (2-core versus 1-degree users) defines engagement level; automated toxicity scoring and sentiment/emotion classification assign each aggregated user text a toxicity and a pessimism score; and Shannon entropy of user text, summarized by the minimal interval containing roughly half the users, quantifies informational complexity. The 2-core/1-degree partition provides a structural proxy for community accountability, the entropy-minimal-interval procedure makes platform differences in text diversity comparable, and the regression of maximum received toxicity or pessimism on reply count operationalizes whether interaction calms language. These tools are what allow the authors to separate user-engagement polarization from partisan polarization.","core_discovery":"The central claim is that debunking discourse is polarized along three axes in addition to political affiliation: participation depth, platform design, and emotional expression. Users at the network periphery (1-degree users, defined as those absent from the 2-core of the retweet network on Twitter or the reply network on Reddit) are numerically dominant and contribute an outsized share of toxic language. Reddit's threaded, community-structured format is associated with higher overall toxicity and greater informational diversity (wider text-entropy intervals), while Twitter's broadcast format shows more uniform text and sharper partisan separation in the toxicity–entropy relationship. Across both platforms and both party groups, toxicity and pessimism move in opposite directions, and the toxicity of the most extreme replies declines as reply counts increase, with steeper declines on Reddit.","pith_inferences":["Recomputing the analyses on a manually validated subset of genuinely corrective posts (versus keyword-matched posts that quote, mock, or spread misinformation) would show whether the reported patterns are about debunking or about any political talk that mentions 'fake news' and 'misinformation'.","The negative toxicity–pessimism relationship may partly reflect the way the two constructs are measured—one by a reader-perception toxicity model and the other by an emotion classifier—so the 'surprising' sign could be an artifact of label separation; swapping in other measurement tools would test this.","The reply-count regressions are observational: an equally plausible interpretation is that users stop replying to the most toxic threads, which would make the apparent calming effect a selection effect; identifying a change in reply visibility or ranking would separate the two.","If the peripheral-user effect is causal, lightweight interventions targeted at first-time repliers—such as community-rule reminders or delayed publishing—could reduce toxic load more efficiently than post hoc moderation, but the paper stops at correlation."],"forward_implications":["Moderation focused only on highly engaged users or partisan echo chambers will miss the largest numerical source of toxic language, because peripheral 1-degree users dominate the toxic tail.","Platform features that encourage reply-based interaction are a plausible lever for reducing extreme content, since toxicity and pessimism decline as reply counts rise, and the decline is steeper on Reddit.","Toxicity and pessimism should not be treated as a single negative-emotion cluster; their consistent negative correlation means an intervention that lowers toxic speech may not touch pessimism, and vice versa.","Broadcast-style platforms and community-thread platforms polarize through different mechanisms, so platform-specific moderation (accountability prompts for newcomers on Twitter-like feeds, structure-based rules on Reddit-like threads) is implied by the findings.","The three-mechanism account (peripheral users, platform architecture, and emotional expression) is presented as transferable to other controversial topics and newer platforms, though the authors caution that their binary 1-degree/2-core distinction simplifies the spectrum of participation."],"supporting_citations":[{"why":"Supplies the Reddit submissions and comments corpus through the Pushshift dataset.","marker":"[30]"},{"why":"Provides the lexicon-based VADER sentiment scoring used to compute compound sentiment scores.","marker":"[31]"},{"why":"Provides the RoBERTa architecture underlying the fine-tuned emotion classifier used to detect pessimism.","marker":"[32]"},{"why":"Provides the automated toxicity scoring that defines language toxicity in the study.","marker":"[35]"},{"why":"Used to construct the retweet and reply networks from which user degree is computed.","marker":"[39]"},{"why":"Provides the network-based approach to identifying partisan user clusters in QAnon discussions, adapted for Twitter.","marker":"[40]"},{"why":"Defines k-core decomposition, the basis for the 2-core versus 1-degree user partition.","marker":"[41]"},{"why":"Supplies the PoliticalBiasBERT model used to classify Reddit users by party affiliation.","marker":"[42]"},{"why":"Provides the earlier Twitter finding that Democratic users receive more toxic replies, which the paper extends and qualifies on Reddit.","marker":"[46]"},{"why":"Supplies the deindividuation theory used to interpret why peripheral users express more extreme views.","marker":"[47]"}],"fun_headline_variants":["Peripheral users fuel toxic debunking on Twitter and Reddit","Debunking toxicity spikes with peripheral users, platform design","86M tweets, 4M Reddit posts link toxicity to user depth, platform","Toxicity and pessimism diverge in debunking across social media","Platform structure shapes toxic debunking more than party alone"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The operating assumption is that any post containing one of the debunking keywords ('fake news,' 'misinformation,' 'debunk') is actually a debunking post; if many such posts are instead spreading or dismissing misinformation, the whole corpus is mislabeled and the conclusions no longer apply to debunking.","fun_headline_variants_meta":{"raw":{"variants":["Peripheral users fuel toxic debunking on Twitter and Reddit","Debunking toxicity spikes with peripheral users, platform design","86M tweets, 4M Reddit posts link toxicity to user depth, platform","Toxicity and pessimism diverge in debunking across social media","Platform structure shapes toxic debunking more than party alone"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000139,"raw_usage":{"total_tokens":1172,"prompt_tokens":975,"completion_tokens":197,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":104}},"tokens_in":591,"tokens_out":197,"duration_ms":2458,"temperature":1.0,"reasoning_tokens":104,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:06:02.791160+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random sample of the keyword-matched posts, have independent annotators classify each as a genuine debunking attempt (e.g., explicitly labeling a claim false) versus other uses, then recompute the toxicity–pessimism correlations and the 1-degree/2-core effect sizes on the confirmed-debunking subset only. If the negative correlations or peripheral-user effects vanish or flip sign, the keyword operationalization of 'debunking post' is the point of failure.","supporting_citations":[{"cited_title":"& Sasahara, K","cited_arxiv_id":null,"evidence_quote":"Provides the network-based approach to identifying partisan user clusters in QAnon discussions, adapted for Twitter."},{"cited_title":"B., Zhang, J., Wang, C","cited_arxiv_id":null,"evidence_quote":"Supplies the deindividuation theory used to interpret why peripheral users express more extreme views."}],"review_version":1}