Pith. sign in

REVIEW 3 major objections 5 minor 3 references

Incivility in Public Health Policy Discussions Spills Over to Public Engagement with Climate Issues

T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The COVID-19 pandemic measurably spilled hostility into public climate debates on social media along pre-existing populist divides.

desk verdict A solid, well-run observational study of incivility spillover from COVID-19 to climate discourse; the real caveat is the unvalidated toxicity classifier, not the design. read the letter →

arxiv 2502.05255 v3 pith:U7LLPFW6 submitted 2025-02-07 cs.SI cs.CYphysics.soc-ph

Hasti Narimanzadeh , Arash Badie-Modiri , Iuliia Smirnova , Ted Hsuan Yun Chen This is my paper · ORCID
classification cs.SIcs.CYphysics.soc-ph
keywords affectivepolarizationclimatechangeCOVID-19incivilitycontentiousnesssocialmediasciencecommunicationpopulism
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that the hostile, polarized way the public talked about COVID-19 spilled over into how the same publics talked about climate change. Analyzing 134 weeks of English-language posts—38.6 million climate tweets, 2.1 million Reddit comments, and full reply trees of hundreds of thousands of Twitter conversations—it finds that climate posts and replies mentioning COVID-19 were consistently more toxic and more contentious, and that replies to conversations starting with a COVID-19 tweet had roughly 14% higher odds of turning toxic at any given moment. The authors argue this spillover was not random: it activated along pre-pandemic political cleavages, especially anti-internationalist populist beliefs that tied climate-policy opposition to vaccine hesitancy. If true, this means affective polarization does not stay inside one issue but becomes entrenched across science-policy domains, changing how publics engage with climate science.

What carries the argument

The argument is carried by a set of machine-classifier measures and survival/regression models applied to conversation trees. Toxicity, standing in for incivility, is the Perspective API's probability that a post is "rude, disrespectful, or unreasonable," binarized at 0.5; disagreement, standing in for contentiousness, comes from a DistilBERT model fine-tuned on the DEBAGREEMENT dataset of comment-reply pairs; obstructionist climate claims are scored with the Augmented CARDS model. These outcomes are linked to keyword indicators for COVID-19, Fauci, and international organizations through linear probability models with day and account fixed effects, fractional logistic regressions, a Cox proportional hazards model of time-to-toxicity in conversation threads, and BEAST time-series change-point estimation. The Cox model is the piece that gives the headline 14% figure, because it measures whether conversations that begin with COVID-19 content reach a toxic reply sooner.

What would settle it

Take a random sample of climate posts that mention COVID-19 keywords and a matched sample that does not, have human annotators rate incivility blind to the research hypothesis, and check whether the COVID-19 posts are actually ruder; if the human-rated difference is near zero, the central association is an artifact of the toxicity classifier.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that COVID-19 functioned as a cross-domain conduit for affective polarization in public engagement with science. Across Twitter and Reddit, climate-related posts that referenced COVID-19 (and especially Anthony Fauci) had higher toxicity probabilities than otherwise similar posts, and climate conversations related to COVID-19 showed elevated disagreement between message pairs. In general climate conversations, a root tweet mentioning COVID-19 shortened the time until a reply became toxic, with about 14% higher odds of toxicity onset at any given time; in conversations explicitly engaging climate-science publications, COVID-19 content appeared both in the initial tweet and in disagreeing responses, but rarely escalated to personal attacks. The paper further claims that the increase in incivility toward international organizations in climate discussions during the pandemic—measured against a pre-pandemic baseline—shows that the spillover traveled through pre-existing anti-internationalist populist sentiments rather than through COVID-19 content alone.

Load-bearing premise

The load-bearing premise is that the machine-scored measures of toxicity and disagreement genuinely capture rudeness and contentiousness—if the scoring system flags posts merely for mentioning COVID-19 or Fauci, the claimed spillover could be an artifact.

Editorial extensions

If this is right

  • If the spillover is real, crisis-driven antagonism in one science-policy domain should be expected to raise the temperature of debate in other domains that share political cleavage lines.
  • Public-health shocks like COVID-19 can leave a lasting mark on climate discourse: the paper shows toxicity toward international organizations stayed elevated from Q1 2020 until Q3 2021, well after the initial crisis phase.
  • Science communicators and platform moderators should treat incivility signals as cross-domain: a post that looks like ordinary climate skepticism may carry pandemic-era hostility.
  • Because the effect appeared on both Twitter and Reddit and survived account fixed effects, it is not just a selection artifact of which users choose to post about COVID-19; the same individuals became more toxic when discussing it.
  • Conversations that explicitly engage climate-science publications are not immune: they showed more contentious disagreement involving COVID-19, though less personal toxicity, than general climate talk.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the same spillover mechanism should appear in other science-policy clashes—for example, AI governance or gene editing—whenever elite cues align those issues with the same populist cleavages; that is a testable prediction, not a claim established here.
  • Because the data end in August 2021 and rely on keyword matching, the 14% hazard estimate is likely a lower bound for later pandemic waves; one could check with post-2021 data whether the effect persisted, grew, or decayed.
  • A direct extension would repeat the analysis for a later health emergency, such as mpox, using human-annotated incivility labels instead of classifier scores, to see whether cross-domain spillover is a general crisis phenomenon.
  • The anti-internationalist pathway suggests that decoupling vaccine and climate messages from international-elite frames might dampen spillover, but whether such decoupling works is not tested in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. Using approximately 38.6 million climate-related tweets, 311,000 full Twitter conversations, and 2.1 million Reddit comments from February 2019 through August 2021, the paper studies whether the COVID-19 pandemic changed the tone of public engagement with climate science. The authors operationalize incivility with Perspective API toxicity scores (binarized at 0.5), contentiousness with a fine-tuned DistilBERT disagreement classifier, and contrarian climate claims with CARDS. They report that climate posts mentioning COVID-19 or Anthony Fauci are more likely to be toxic across Twitter and Reddit; that COVID-19-related climate conversations show higher disagreement; and that these patterns track pandemic events, with evidence that anti-internationalist populist keywords link climate and vaccine skepticism. The main analyses are supported by account fixed effects, fractional logistic regression, and a Cox proportional hazards model of toxicity onset.

Significance. The paper provides a large-scale, cross-platform descriptive account of a timely question: whether a public health crisis can spill into the tone of climate discussions. The central association—COVID-19 references co-occurring with higher toxicity—is consistent across three social media systems and several model specifications, which is a real empirical strength. If the measurement and sampling-window concerns are addressed, the findings would be a valuable contribution to science communication and affective polarization literatures. The stated commitment to release code and data is also a positive feature. However, the reliance on a single black-box toxicity classifier without any validation on the study corpus is a serious validity threat, and the outcome-driven selection of the analysis window further limits the strength of the causal claims.

major comments (3)
  1. [Sections 4.2.2, 4.3.1, 4.3.2] The validity of the key dependent variable rests entirely on Perspective API toxicity scores binarized at 0.5. The model was trained on Wikipedia talk pages and New York Times comments, not on climate/COVID tweets, and no precision/recall evaluation is reported on a sample from the study corpus. Because the COVID-19 keyword set (Table S3.1) includes terms such as 'mask,' 'vaccin,' and 'Fauci,' which may co-occur with stylistic features (e.g., all-caps, profanity, anti-elite framing) that Perspective flags as toxic regardless of human judgment, the association between COVID-19 content and toxicity could be driven by systematic misclassification. I request a validation sub-study: human annotation of a random sample of posts stratified by platform and COVID-19 keyword presence, reporting agreement statistics, along with a threshold sensitivity analysis (e.g., continuous toxicity score with cutoffs 0.3, 0.5, 0.7) to demonstrate that the main coefficients in Fig. 1.B and Fig. 1.C are not artifacts of the 0.5 cutoff.
  2. [Sections 4.3.1 and S5.2] The main regression and Cox analyses are restricted to the window August 14, 2020–August 26, 2021, which is selected from BEAST change points estimated on the same toxicity time series (Section 4.4). This outcome-driven sample selection can exaggerate the apparent association and yield over-optimistic confidence intervals, because the analysts have essentially chosen the period where the divergence is largest. The authors should report results for alternative analysis windows (e.g., the full pandemic period from March 2020 onward, or a window defined a priori from event dates) and show that the COVID-19 coefficients in Figs. 1.B and 1.C are stable across these choices.
  3. [Section 2.3, Figure 3] The claim that the increase in the toxicity gap for international-organization keywords is 'attributed to the COVID-19 shock' requires the parallel trends assumption and the absence of confounding events beyond the pandemic. No placebo tests are provided (e.g., using a set of unrelated international topics that were not central to COVID-19 politics), and the Q1–Q2 2020 spike could also coincide with the U.S. presidential primary season or other political shocks. Please provide falsification exercises or soften the causal language to 'is associated with the pandemic period.'
minor comments (5)
  1. [Section 4.2.3 and Table S4.1] The text states that the binary classifier returns f1 scores of 0.806 and 0.716 for disagreement and lack of disagreement, respectively, but Table S4.1 reports the three-class disagreement f1 as 0.716. Clarify which value corresponds to which class and reconcile the text with the table.
  2. [Section 2.1] The phrase 'the number of replies until one exceeds 0.5 toxicity probability is significantly lower' is informal; since the outcome is the hazard of toxicity onset, it would be clearer to say 'the hazard of toxicity onset is significantly higher' or 'replies become toxic sooner.'
  3. [Tables 1 and 3] The example tweets are illustrative but the selection criteria are not described. State whether they were hand-picked, randomly sampled, or selected to match specific toxicity ranges, so readers can gauge their representativeness.
  4. [Abstract] The opening sentence 'Affective polarization and political sorting drive public antagonism' states a causal mechanism as fact rather than as a hypothesis the paper tests. Consider rephrasing to avoid overclaiming.
  5. [Figure 1.A] The inset panels for climate science tweets and Reddit comments are very small and difficult to read; consider presenting them as separate panels or with larger font sizes.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the spillover findings rest on external classifiers and independent statistical models, not on inputs that define the outcomes.

full rationale

The paper's central claims—that COVID-19 content is associated with higher toxicity and disagreement in climate discussions—are derived from standard regression and survival models applied to independently measured variables. The predictors are keyword-based topic labels (COVID-19, Fauci, international organizations), while the outcomes are probabilities from external classifiers: Perspective API for toxicity, a DistilBERT model fine-tuned on the DEBAGREEMENT dataset for disagreement, and the Augmented CARDS model for obstructionist content. None of these classifiers was fitted to the paper's target data or to the outcome-predictor relationship under study; they are pre-trained benchmarks with published training sets and, for DistilBERT, reported held-out test performance. There is no equation in the paper that defines a predictor in terms of the outcome or vice versa, and no fitted parameter is renamed as a prediction. The main estimates—LPM coefficients, fractional logistic marginal effects, and Cox hazard ratios—are ordinary empirical associations computed from these independently scored texts. Self-citations such as Chen et al. (2021) and Xia, Chen and Kivelä (2021) are used only to motivate background concepts like partisan sorting or to note prior evidence of antagonistic climate discourse; they are not the load-bearing evidence for the paper's spillover conclusion, which comes from the paper's own regressions and robustness checks. The choice of the August 14, 2020–August 26, 2021 analysis window from BEAST change points estimated on the toxicity outcome is a specification and inference concern, not a circularity: it does not make the COVID-19 coefficients equal to the outcome by construction. Measurement-validity concerns about Perspective's threshold or keyword sets are substantive threats to empirical validity but are not instances of circular derivation. Accordingly, the paper is self-contained against external benchmarks and warrants a score of 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical or theoretical entities are introduced; the paper uses established constructs such as affective polarization and populism, and existing NLP classifiers. The free parameters listed are modeling choices that affect the central estimates. The axioms reflect the measurement and identification assumptions underlying the observational analyses.

free parameters (3)
  • Toxicity cutoff = 0.5
    Used to define 'toxic' replies in the Cox proportional hazards model and in conversation-level analyses. This threshold is chosen by convention rather than data and affects the estimated hazard ratios.
  • BEAST time series constraints = 15 maximum change points; minimum 4 weeks apart
    Hand-specified prior settings for the Bayesian change-point model (Section 4.4). These choices shape the detected 'initial incivility escalation' period that defines the main analysis window.
  • Post-escalation analysis window = 2020-08-14 to 2021-08-26
    The main regression and survival models are restricted to this window, selected from BEAST-detected change points (Section 4.3.1). This data-driven selection can influence effect sizes and is a documented modeling choice.
assumptions (5)
  • domain assumption Perspective API toxicity probability measures incivility as defined by the paper
    Section 4.2.2 operationalizes incivility using the API. If the API does not track the theoretical construct, the main outcome is mis-measured.
  • domain assumption DistilBERT disagreement scores measure contentiousness between conversation participants
    Section 4.2.3; the model is fine-tuned on DEBAGREEMENT with F1 around 0.8 for disagreement, so misclassification is possible and may bias the disagreement analyses.
  • domain assumption Keyword lists for COVID-19, Fauci, and international organizations correctly capture mentions of these topics
    Section 4.2.1 and Table S3.1; false positives and negatives in topic classification can blur the comparison groups and affect the estimated associations.
  • domain assumption English-only filtering yields a representative subset of public engagement
    The paper restricts to English tweets and comments as classified by Twitter and Perspective (Section 4.1); results may not generalize to non-English discussions, and filtering rules could introduce selection.
  • domain assumption Onset of the COVID-19 pandemic is exogenous to climate discussion behavior in the short term
    Section 2.3 uses this assumption to attribute the rise in anti-internationalist toxicity to the pandemic shock. If other concurrent events, such as the U.S. election or BLM protests, drove the change, the causal interpretation is weakened.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Incivility in Public Health Policy Discussions Spills Over to Public Engagement with Climate Issues." pith.science (2026). https://pith.science/paper/U7LLPFW6

@misc{pith2026250205255,
  author       = {Pith},
  title        = {Pith review of: Incivility in Public Health Policy Discussions Spills Over to Public Engagement with Climate Issues},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U7LLPFW6}},
  note         = {Machine review of arXiv:2502.05255}
}
read the original abstract

Affective polarization and political sorting drive public antagonism around climate change and other issues at the science-policy nexus. We study cross-domain spillover of incivility in public engagements with climate change and public health on Twitter and Reddit using the COVID-19 period as a case study. We find strong evidence of the signatures of affective polarization surrounding COVID-19 spilling into the climate change domain. Across different social media systems, COVID-19 content is associated with incivility in climate discussions. These patterns of increased antagonism were responsive to pandemic events that made the link between science and public policy more salient. The observed spillover activated along pre-pandemic political cleavages, specifically anti-internationalist populist beliefs, that linked climate policy opposition to vaccine hesitancy. Our findings show how affective polarization in public engagement with science becomes entrenched across policy domains, which has implications for how the public engages with and communicates about issues such as climate change and public health.

Figures

Figures reproduced from arXiv: 2502.05255 by the authors.

Figure 1
Figure 1. Presence of COVID-19 content and incivility of climate posts. A. [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Presence of COVID-19 content and probability of disagreement between climate [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Quarterly difference in toxicity between tweets containing references to international [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

3 extracted references · 1 canonical work pages

  1. [2]

    a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion

    for Anthony Fauci, we only used his surname, as it is distinct enough to capture relevant references to him and discriminate against false positives; 3) for international organizations, we used a list of keywords that include international entities (i.e., international organizations and public figures). The full keyword lists are in Table S3.1. 4.2.2 Toxi...

    arXiv 2017

  2. [2013]

    Climategate, public opinion, and the loss of trust

    “Climategate, public opinion, and the loss of trust.” American behavioral scientist 57(6):818–837. Leiserowitz, Anthony, Edward Maibach, Seth Rosenthal, John Kotcher, Sanguk Lee, Marija Verner, Matthew Ballew, Jennifer Carman, Teresa Myers, Matthew Goldberg, Nicholas Badullovich and Jennifer Marlon

    work page

  3. [2023]

    Climate Change in the American Mind: Beliefs & Attitudes, Spring 2023

    “Climate Change in the American Mind: Beliefs & Attitudes, Spring 2023.”. URL: https://climatecommunication.yale.edu/publications/climate-change-in-the-american-mind-beliefs- attitudes-spring-2023/ Narimanzadeh & Badie-Modiri et al. 20 Lockwood, Matthew. 2018. “Right-wing populism and the climate change agenda: exploring the linkages.” Environmental polit...

    arXiv 2022

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.