Does Content Moderation Lead Users Away from Fringe Movements? Evidence from a Recovery Community

Giuseppe Russo; Maciej Styczen; Manoel Horta Ribeiro; Robert West

REVIEW 4 major objections 5 minor 12 references

Does Content Moderation Lead Users Away from Fringe Movements? Evidence from a Recovery Community

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Banning fringe subreddits sends users toward a recovery community; quarantines show no such effect.

desk verdict A useful case study with a real but overstated finding: bans, not quarantines, move some Manosphere users into a recovery subreddit, though the causal machinery deserves more scrutiny and the title oversells what was measured. read the letter →

arxiv 2505.03772 v1 pith:CIIAOVYY submitted 2025-04-29 cs.SI

Giuseppe Russo , Maciej Styczen , Manoel Horta Ribeiro , Robert West This is my paper · ORCID

classification cs.SI

keywords contentmoderationderadicalizationrecoverycommunitiesRedditManospherecausalinferenceBayesianstructuraltimeseriesfringe

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This study asks whether platform sanctions against fringe communities can nudge members toward abandoning those movements. It tracks r/exredpill, the largest Reddit recovery community for people leaving Manosphere groups, and measures daily activity, newcomer arrival, and migration from three banned or quarantined subreddits. The paper's central claim is that hard moderation—banning a subreddit—causes large increases in all three outcomes, with activity up 88.4% after the r/Braincels ban and 64.5% after the r/MGTOW ban, while soft moderation in the form of quarantines produces no robust effect. Real-world events tied to the Manosphere also raise recovery participation, but by less than bans do; the post-ban surge appears to be genuine recovery activity rather than brigading or toxic spillover. If the causal estimates hold, banning fringe communities can act as a deradicalization catalyst and platforms could deliberately steer banned users toward support communities.

What carries the argument

The central machinery is Bayesian structural time series (BSTS) with a synthetic control: a state-space model learns the pre-intervention relationship between r/exredpill's outcome series (activity, newcomers, migrants) and 48 control subreddits matched on age, gender, and partisanship, then projects the counterfactual post-intervention trajectory and attributes the gap to the intervention. Complementing it is interrupted time series (ITS) regression, which the paper uses chiefly to visualize level and trend changes around each event. The intervention events themselves are the load-bearing empirical objects: quarantines of r/Braincels and r/TheRedPill on September 27, 2018, and of r/MGTOW on January 31, 2020, and bans of r/Braincels on October 1, 2019, and r/MGTOW on August 3, 2021.

What would settle it

Look for discontinuities at the ban dates in the control subreddits themselves, or apply the same BSTS procedure to a placebo outcome—such as activity in an unrelated recovery subreddit—around October 1, 2019, and August 3, 2021; a spike in the placebo would show that a concurrent shock, not the ban, drove the observed increase.

Watch

Extended reading notes

Core claim

The discovery is a moderation-as-deradicalization result: after Reddit banned r/Braincels and r/MGTOW, participation in r/exredpill rose sharply relative to what a synthetic control built from 48 unaffected subreddits predicts. Bayesian structural time-series estimates put the increases at 88.4% for activity and 174.3% for newcomers after the r/Braincels ban, and 64.5% and 31.7% after the r/MGTOW ban; users migrating from a banned fringe community to r/exredpill increased by 94.6% and 22.8%. The same method finds no significant quarantine effects for the main activity and newcomer outcomes, and the authors read the one positive quarantine result as likely spurious under multiple-comparison correction. Robustness checks—deleted-content rates, toxicity scores, LLM-based moderation judgments, and manual annotation—do not show a toxic influx, and the paper concludes that the measured activity reflects engagement with recovery rather than retaliation. The authors additionally frame the estimate as a lower bound because lurkers, users who leave without joining a recovery space, and users who change usernames are invisible to the measurement.

Load-bearing premise

The causal estimates stand on the assumption that, without the bans, r/exredpill's outcomes would have continued to track the 48 control subreddits exactly as they did before, so that any post-ban deviation is attributable to the ban rather than to some concurrent event affecting recovery communities.

Editorial extensions

If this is right

Banning a radical subreddit increases measured participation in the associated recovery community by tens of percentage points, so hard moderation is not only a suppression tool but also a pathway nudge.
Quarantines do not produce this effect, implying that merely reducing visibility and accessibility of fringe content is not enough to move users toward recovery.
Platform moderation outperforms widely discussed real-world events (Unite the Right, Toronto van attack, Capitol Hill siege) as a driver of recovery-community participation.
The post-ban participation is not accompanied by rises in deleted content, toxicity, or rule-breaking comments, suggesting the new users are engaging with recovery rather than continuing hostility.
Because the measurement misses lurkers, users who leave without joining any recovery space, and users who create new accounts, the true deradicalization effect may be larger than the reported estimates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct platform-level extension follows from the paper's mechanism without being tested: pairing a ban with a visible link to recovery resources should amplify the effect, since the paper shows the causal push operates through disruption rather than visibility reduction.
The same BSTS design could be applied to other movement–recovery pairs (QAnon believers, far-right groups) to test whether the ban-as-turning-point effect generalizes beyond the Manosphere.
The quarantine–ban asymmetry suggests the active ingredient is severing the community, not hiding it; a testable implication is that banning a community while leaving a read-only archive would isolate loss-of-community from loss-of-content.
Re-running the analysis on a different platform or in a later period would check whether the estimated effects are stable or were inflated by Reddit-wide policy changes coinciding with the ban dates.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Read Russo et al. on moderation and r/exredpill. The genuinely new thing is the outcome: instead of measuring hate speech or migration to fringe platforms, they track participation in a recovery subreddit. That is a sensible way to ask whether sanctions nudge people toward disengagement, and as far as I know no one else has done it quantitatively. The comparison across hard and soft moderation and real-world events is also useful, and the results are internally consistent: bans show large BSTS effects across activity, newcomers, and migration; quarantines mostly don't; real-world events are smaller. The robustness checks (deleted content, toxicity, LLM labeling, manual inspection) are real work and push back against the obvious brigading story.

Having said that, the central claim is bigger than the evidence. The abstract calls bans a 'deradicalization catalyst.' What is measured is posting in a recovery forum, not changed beliefs or sustained exit. The paper itself acknowledges this in the limitations, but the title and abstract don't. The absolute migration numbers are also small—46 and 21 users—so the practical significance rests on the relative effects and on the assumption that recovery posting is a meaningful step.

The bigger technical soft spot is the BSTS design. The controls are selected by demographic similarity to the banned subreddits, not by their ability to predict r/exredpill, and no pre-period fit is reported. That is a legitimate gap: if the state-space model cannot track the outcome before the intervention, the counterfactual after it is not worth much. The placebo tests are described too thinly for me to tell whether a -120 day placebo contains the true ban event in its post-period; as written, that column is ambiguous. The ITS results are consistent with BSTS, but ITS shares the same outcome series, so it does not independently validate the counterfactual. I would not call the effect fabrication; the timing is specific to bans and not quarantines, which is what you would expect if something causal were happening. But the causal language should be softened until the controls are validated.

Minor: the multiple-testing correction is applied selectively, and there is still no code or data release, which hurts reproducibility for a paper built on public Reddit data.

Who is this for? Anyone working on content moderation or Manosphere exit. It deserves a serious referee; the design question is fixable, and the framing is fixable. I would send it out, with a request to validate the BSTS controls, clarify the placebos, and align the title with what was actually measured.

Referee Report

4 major / 5 minor

Summary. This paper investigates whether Reddit moderation actions against Manosphere subreddits (r/Braincels, r/MGTOW, r/TheRedPill) increase participation in the recovery subreddit r/exredpill. Using interrupted time series (ITS) regression and Bayesian structural time series (BSTS) modeling, the authors estimate the effects of quarantines and bans on daily activity volume, newcomer counts, and migrating users, and compare these with the effects of three real-world events. They report significant increases after bans (e.g., 88.4% activity, 174.3% newcomers for the r/Braincels ban), no consistent effects for quarantines, and smaller effects for real-world events; robustness checks based on deleted content, toxicity scores, LLM annotations, and manual inspection suggest that the increased activity is not driven by toxic brigading. The paper concludes that content moderation can act as a deradicalization catalyst.

Significance. If the causal estimates are valid, this would be valuable quantitative evidence on how deplatforming affects engagement with recovery communities, a topic previously studied mainly through qualitative methods. The paper's strengths are its use of multiple outcome metrics, the combination of two causal-inference approaches, and the extensive robustness checks including manual annotation. However, the credibility of the central causal claim rests entirely on the BSTS counterfactual, and the current manuscript does not yet demonstrate that the control series can predict the recovery outcomes or that the placebo tests are informative. The result is therefore intriguing but not yet established.

major comments (4)

[§3.2] The BSTS control series are never validated as predictors of the r/exredpill outcomes. The controls are selected by cosine similarity of partisanship, age, and gender to the treated subreddits, not by their ability to forecast the recovery outcomes, and no pre-period fit statistics (e.g., R², RMSE, dynamic regression coefficients, or cross-validation) are reported. Without such validation, the reported effects (activity +88.4% and +64.5%, new users +174.3% and +31.7%, migrating users +94.6% and +22.8% in Table 5) may be artifacts of the local-trend component extrapolating pre-existing growth rather than causal consequences of the bans. Please add explicit pre-period predictive diagnostics for each outcome and each event, and show sensitivity to the construction of the control set.
[§5, Tables 7–8] The placebo tests described in §5 cannot validate the BSTS design. With the stated 120-day observation window, a placebo at −120 days has no pre-intervention period and a placebo at +120 days has no post-intervention period; if the −120 placebo is instead evaluated over the full post-period, that post-period contains the true intervention, so the test cannot separate trend extrapolation from a real effect. Please redesign the placebos using dates that allow a genuine pre-period and a clean post-period (e.g., multiple random dates within an earlier held-out period), and report the distribution of placebo effects against which the observed effects are compared.
[Abstract, §6] The claim that content moderation acts as a deradicalization catalyst is not supported by the measured outcomes, which are activity volume, newcomer counts, and migration into r/exredpill—participation in a support community, not deradicalization or belief change. The manual inspection of 200 comments and the LLM annotations are informative about toxicity and visible alignment with fringe ideologies, but they do not measure ideological change or sustained disengagement from fringe movements. Please restrict the conclusions to increased engagement with a recovery community and explicitly discuss the gap between participation and deradicalization.
[§4.1, §4.2, Table 5] The treatment of multiple comparisons is asymmetric. In §4.1, the quarantine migration effect (p = 0.045) is explicitly discounted with a Bonferroni correction over six tests, but no equivalent correction is applied to the ban results in §4.2 and Table 5. Under the same 0.05/6 threshold, the r/Braincels ban activity effect (p = 0.032) would no longer be significant, while the newcomer effect (p = 0.004) and migration effect (p = 0.001) for r/Braincels and the three r/MGTOW effects (all p ≤ 0.006) would survive. Please apply a consistent multiple-testing policy across all intervention types or justify the selective correction.

minor comments (5)

[Abstract, §4.2] In the abstract, "p = 22.8" for the r/MGTOW migration effect is a typo; the BSTS table reports p = 0.006.
[§4.2] In the Banning and Newcomers paragraph, the coefficient reported as "βBI2 = 0.04; p = 0.04" for the post-ban trend should presumably be βBI3, the post-intervention trend coefficient.
[§5, Manual Inspection] The text states that interannotator agreement is reported with Cohen's κ, but no κ value appears in the manuscript; please include the value and the number of comments annotated by each coder.
[§5, LLM Moderation Analysis] The LLM analysis reports that 97% of comments are guideline-compliant and 99% do not align with fringe ideologies, but the sample size and the exact prompts are only said to be in the appendix; please state how many comments were annotated and provide the complete prompt text in the main text or supplement.
[Table 4] The column header "Braincels+TheRedPill" is not clearly separated from the row labels; consider using a clearer notation such as "Braincels & TheRedPill (combined event)" to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: outcomes are measured independently of treatment and controls are not constructed from the outcome.

full rationale

The causal estimates rest on comparing the observed r/exredpill series (activity volume, newcomers, migrating users) with a BSTS counterfactual built from 48 subreddits selected by demographic similarity to the fringe communities, not by their ability to reproduce the recovery outcome. The outcome series are therefore not an algebraic function of the treatment indicators or of the control series; a ban could in principle have produced no change or a decrease, and the reported increases are data measurements rather than identities. The migrating-users metric is defined by first posting in a fringe subreddit and later posting in r/exredpill, which is a measurement convention applied symmetrically before and after each event; it does not by construction guarantee a post-ban increase. Self-citations in the paper (e.g., Russo, Ribeiro, and West (2024) for assigning users with activity in multiple subreddits to the community with highest activity) are data-processing conventions and are not load-bearing for the central causal claim. Concerns about the placebo tests and the absence of explicit pre-period fit diagnostics for the controls are threats to causal identification and external validity, not circularity. No derivation step in the paper reduces to its own inputs, so the paper is not circular.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on modeling choices and domain assumptions rather than fitted physical parameters. No arbitrary constants are fit to make the result appear; the hand-chosen thresholds and control selection are listed above. The main assumptions are the causal identification continuity and the proxy interpretation of recovery participation.

free parameters (4)

Minimum participation threshold for subreddit membership = 5 comments/posts
Users contributing more than five comments or posts to a subreddit are classified as members; this hand-chosen threshold affects which users count as newcomers or migrants (§3.1).
Observation window around each moderation event = 120 days
Outcomes are measured within a 120-day window before and after each event; sensitivity checks at 60/90/150/180 days confirm consistency (§3.1, §5).
BSTS control group size and composition = 48 subreddits
Control series were selected by cosine similarity on partisanship, age, and gender dimensions; the final set is a modeling choice that determines the synthetic counterfactual (§3.2).
MCMC iterations for BSTS = 1000
CausalImpact was run with 1,000 MCMC iterations, a standard convenience choice (§3.2).

assumptions (4)

domain assumption The pre-intervention relationship between r/exredpill outcomes and the control subreddits would have continued in the absence of the moderation event.
BSTS causal identification rests on this continuity assumption; stated in §3.2: 'As long as the control series received no intervention...'.
domain assumption Participation in r/exredpill is a meaningful proxy for disengagement from Manosphere ideologies.
The paper interprets activity, newcomers, and migration as evidence of deradicalization; the authors acknowledge in §6 that genuine deradicalization is not directly measured.
domain assumption Users who posted at least five times in a fringe subreddit are members of that community.
This threshold defines the population at risk for migration and follows prior work (Kumar et al. 2018; Samory and Mitra 2018), but it is an assumption about membership.
domain assumption The identified sanction dates (quarantines and bans) are accurate.
Dates are drawn from media reports and r/reclassified; any misdating would shift the intervention point in ITS/BSTS (§3.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of Does Content Moderation Lead Users Away from Fringe Movements? Evidence from a Recovery Community." pith.science (2026). https://pith.science/paper/CIIAOVYY

@misc{pith2026250503772,
  author       = {Pith},
  title        = {Pith review of: Does Content Moderation Lead Users Away from Fringe Movements? Evidence from a Recovery Community},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CIIAOVYY}},
  note         = {Machine review of arXiv:2505.03772}
}

read the original abstract

Online platforms have sanctioned individuals and communities associated with fringe movements linked to hate speech, violence, and terrorism, but can these sanctions contribute to the abandonment of these movements? Here, we investigate this question through the lens of exredpill, a recovery community on Reddit meant to help individuals leave movements within the Manosphere, a conglomerate of fringe Web based movements focused on men's issues. We conduct an observational study on the impact of sanctioning some of Reddit's largest Manosphere communities on the activity levels and user influx of exredpill, the largest associated recovery subreddit. We find that banning a related radical community positively affects participation in exredpill in the period following the ban. Yet, quarantining the community, a softer moderation intervention, yields no such effects. We show that the effect induced by banning a radical community is stronger than for some of the widely discussed real-world events related to the Manosphere and that moderation actions against the Manosphere do not cause a spike in toxicity or malicious activity in exredpill. Overall, our findings suggest that content moderation acts as a deradicalization catalyst.

Figures

Figures reproduced from arXiv: 2505.03772 by the authors.

**Figure 2.** Effects of Quarantine on r/exredpill Activity Volume, Newcomers, and Migrants. We show the results obtained after [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Effects of Banning on r/exredpill Activity Volume, Newcomers, and Migrants. We show the results obtained after [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 10 canonical work pages

[1]

For most authors... (a) Would answering this research question advance sci- ence without violating social contracts, such as violat- ing privacy norms, perpetuating unfair profiling, exac- erbating the socio-economic divide, or implying disre- spect to societies or cultures? Yes (b) Do your main claims in the abstract and introduction accurately reflect t...

work page
[2]

Additionally, if your study involves hypotheses testing... (a) Did you clearly state the assumptions underlying all theoretical results? Yes (b) Have you provided justifications for all theoretical results? Yes (c) Did you discuss competing hypotheses or theories that might challenge or complement your theoretical results? NA (d) Have you considered alter...

work page
[3]

(a) Did you state the full set of assumptions of all theo- retical results? NA (b) Did you include complete proofs of all theoretical results? NA

Additionally, if you are including theoretical proofs... (a) Did you state the full set of assumptions of all theo- retical results? NA (b) Did you include complete proofs of all theoretical results? NA

work page
[4]

Additionally, if you ran machine learning experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (ei- ther in the supplemental material or as a URL)? NA (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? NA (c) Did you report error bars (e....

work page
[5]

(a) If your work uses existing assets, did you cite the creators? Yes (b) Did you mention the license of the assets? No, given that the license is ill-defined

Additionally, if you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? Yes (b) Did you mention the license of the assets? No, given that the license is ill-defined. (c) Did you include any new assets in the supplemental material or as a URL? No (d) Did...

work page
[6]

Additionally, if you used crowdsourcing or conducted re- search with human subjects... (a) Did you include the full text of instructions given to participants and screenshots? NA (b) Did you describe any potential participant risks, with mentions of Institutional Review Board (IRB) ap- provals? NA (c) Did you include the estimated hourly wage paid to part...

work page
[11]

Answer: No

"Answer: No" if the comment does not violates the community guidelines ** Community Guidelines of r/exredpill ** : [...] ** Comment posted on r/exredpill **: [...] Does the comment posted on r/exredpill violate the community guidelines? You are tasked with evaluating whether a comment posted by a user in r/exredpill aligns with the ideologies of fringe co...

work page
[12]

Answer: Yes

"Answer: Yes" if the comment posted on r/exredpill violates the community guidelines

work page

Show all 12 references

[13]

Answer: No

"Answer: No" if the comment does not violates the community guidelines ** Description of incels ideology ** : [...] ** Description of MGTOW ideology ** : [...] ** Description of TheRedPill ideology ** : [...] ** Comment posted on r/exredpill **: [...] Is the comment posted on ...
[252]

Samory, M.; and Mitra, T

Accademia University Press. Samory, M.; and Mitra, T. 2018. Conspiracies online: User discussions in a conspiracy community following dramatic events. In Proceedings of the International AAAI Conference on Web and Social Media, volume 12. Schmitz, M.; Muric, G.; and Burghardt,...

2018
[2018]

men going their own way

MEGA X: molecular evolutionary genetics analysis across computing platforms. Molecular biology and evolu- tion, 35(6): 1547. Latona, G. R.; Ribeiro, M. H.; Davidson, T. R.; Veselovsky, V .; and West, R. 2024. The ai review lottery: Widespread ai-assisted peer reviews boost pap...

2024 arXiv
[2019]

I Will Not Drink With You Today

Converging Patterns in Pathways in and out of Violent Extremism. Perspectives on Terrorism, 13(6): 73–89. Binder, M. 2021. Reddit changes its harassment policy and bans major incel community. Brodersen, K. H.; Gallusser, F.; Koehler, J.; Remy, N.; and Scott, S. L. 2015. Inferr...

2021 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

For most authors... (a) Would answering this research question advance sci- ence without violating social contracts, such as violat- ing privacy norms, perpetuating unfair profiling, exac- erbating the socio-economic divide, or implying disre- spect to societies or cultures? Yes (b) Do your main claims in the abstract and introduction accurately reflect t...

work page

[2] [2]

Additionally, if your study involves hypotheses testing... (a) Did you clearly state the assumptions underlying all theoretical results? Yes (b) Have you provided justifications for all theoretical results? Yes (c) Did you discuss competing hypotheses or theories that might challenge or complement your theoretical results? NA (d) Have you considered alter...

work page

[3] [3]

(a) Did you state the full set of assumptions of all theo- retical results? NA (b) Did you include complete proofs of all theoretical results? NA

Additionally, if you are including theoretical proofs... (a) Did you state the full set of assumptions of all theo- retical results? NA (b) Did you include complete proofs of all theoretical results? NA

work page

[4] [4]

Additionally, if you ran machine learning experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (ei- ther in the supplemental material or as a URL)? NA (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? NA (c) Did you report error bars (e....

work page

[5] [5]

(a) If your work uses existing assets, did you cite the creators? Yes (b) Did you mention the license of the assets? No, given that the license is ill-defined

Additionally, if you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators? Yes (b) Did you mention the license of the assets? No, given that the license is ill-defined. (c) Did you include any new assets in the supplemental material or as a URL? No (d) Did...

work page

[6] [6]

Additionally, if you used crowdsourcing or conducted re- search with human subjects... (a) Did you include the full text of instructions given to participants and screenshots? NA (b) Did you describe any potential participant risks, with mentions of Institutional Review Board (IRB) ap- provals? NA (c) Did you include the estimated hourly wage paid to part...

work page

[7] [11]

Answer: No

"Answer: No" if the comment does not violates the community guidelines ** Community Guidelines of r/exredpill ** : [...] ** Comment posted on r/exredpill **: [...] Does the comment posted on r/exredpill violate the community guidelines? You are tasked with evaluating whether a comment posted by a user in r/exredpill aligns with the ideologies of fringe co...

work page

[8] [12]

Answer: Yes

"Answer: Yes" if the comment posted on r/exredpill violates the community guidelines

work page