The Big Ban Theory: A Pre- and Post-Intervention Dataset of Online Content Moderation Actions

Aldo Cerulli; Benedetta Tessa; Lorenzo Cima; Serena Tardelli; Stefano Cresci

REVIEW 2 major objections 3 minor 6 references

The paper introduces The Big Ban Theory (TBBT), a reusable dataset that aligns three months of user activity before and after each of 25 content-moderation interventions across Reddit and Voat.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 10:05 UTC pith:H7X3QQHC

load-bearing objection TBBT is a genuinely useful standardized multi-intervention moderation dataset, but the two largest rows use a month-long date range as t0, which breaks the paper's claim of uniform ±3-month alignment. the 2 major comments →

arxiv 2601.11128 v3 pith:H7X3QQHC submitted 2026-01-16 cs.SI cs.HCcs.IR

The Big Ban Theory: A Pre- and Post-Intervention Dataset of Online Content Moderation Actions

Aldo Cerulli , Lorenzo Cima , Benedetta Tessa , Serena Tardelli , Stefano Cresci This is my paper

classification cs.SI cs.HCcs.IR

keywords content moderationdatasetRedditVoatcommunity bansquarantinepre-post interventionpseudonymization

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces The Big Ban Theory (TBBT), a dataset built around 25 content-moderation interventions—community bans, quarantines, post removals, and ban-triggered migrations—carried out between 2015 and 2023. It claims to cover pseudonymized activity from 339,125 users and 38.7 million messages, with each intervention's data organized into fixed three-month windows before and after the event. The aim is to let researchers study moderation's effects, fairness, and spillovers across many interventions at once, instead of assembling one-off collections for each study. The central design choice is a four-slice model separating time (before/after) and space (inside/outside the moderated community), which also handles cases where the moderated space disappears after a ban.

Core claim

TBBT is a standardized, intervention-centered corpus: for each of 25 moderation events, the paper provides the same kinds of fields (timestamps, text, user hashes, subreddit, scores), collected from three months before and, when possible, three months after the intervention. For community bans, where post-ban activity inside the community is structurally unavailable, the dataset instead tracks the same affected users' activity elsewhere on the platform or on Voat, including matched usernames for migration cases. The paper's claim is that this unified structure turns scattered moderation events into a comparable, reusable resource for descriptive, quasi-experimental, predictive, and governanc

What carries the argument

The load-bearing device is the four-slice data model (IN-BEFORE, IN-AFTER, OUT-BEFORE, OUT-AFTER), built on the idea that a moderation intervention is a discrete event at time t0 with a defined moderated space. Fixed windows t_before = t0 − 3 months and t_after = t0 + 3 months make interventions commensurable; the IN/OUT split separates activity inside the moderated community from activity in other spaces where affected users participate. The model determines which slices exist for each intervention type—post removals and quarantines have IN-AFTER; community bans substitute OUT-BEFORE and OUT-AFTER; migrations pair IN-BEFORE on Reddit with OUT-AFTER on Voat.

Load-bearing premise

The three-month windows are centered on a single, well-defined intervention timestamp t0 for every case; if an intervention unfolded over days or weeks instead of at one moment, the 'before' and 'after' slices are not comparing around the same event.

What would settle it

Look at the timestamps of removed posts in the two post-removal cases spanning '1-30 Jun 22'; if removal timestamps are spread across many days, then the fixed t0 ± 3-month windows are not aligned to a single intervention, which would contradict the paper's claim of uniform and comparable pre/post windows across all 25 interventions.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

Because every intervention is stored in the same four-slice format with the same fields, a single analysis script can be run across all 25 events, making cross-intervention comparisons and robustness checks feasible.
The dataset provides pre-moderation behavioral baselines for hundreds of thousands of users, allowing tests of whether similar communities are treated consistently by moderators and whether outcomes vary by intervention type, community size, or time period.
Pre-intervention activity can serve as input features and post-intervention behavior as ground-truth labels, supporting predictive models of user abandonment, migration, and toxicity change.
The OUT slices for community bans make within-platform and cross-platform spillover measurable, including migration to Voat where usernames were matched across platforms.
The dataset can anchor method comparisons: researchers can benchmark before-after estimators, interrupted time series, and difference-in-differences on the same intervention set, testing sensitivity to window length and aggregation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Our reading: the uniform-window claim should be tested against the two post-removal cases whose reported date is a range ('1-30 Jun 22') rather than a single day; if removals were distributed across the month, the IN-BEFORE and IN-AFTER slices for those cases are not centered on one intervention event, and pooled comparisons should treat them as a separate condition.
Our reading: the rule requiring at least ten pre-intervention messages and the bot filter mean the dataset describes the most active users in a community, not all affected users; any effect estimates generalize to that subpopulation, and replication with relaxed thresholds would show how sensitive conclusions are to this choice.
Our reading: the matched-username migration cases open a route to studying the same individuals' behavior before a ban on one platform and after migration to another; extending this design to other banned communities would require similar cross-platform archives with overlapping identities.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

TBBT is a genuinely useful standardized multi-intervention moderation dataset, but the two largest rows use a month-long date range as t0, which breaks the paper's claim of uniform ±3-month alignment.

read the letter

TBBT is worth a serious look. It assembles 25 moderation interventions across Reddit and Voat, with pseudonymized user activity in four standardized slices (IN/OUT × BEFORE/AFTER), totaling 38.7M comments from 339K users. That is a real resource: prior datasets are mostly single-intervention or general-purpose archives, and this one standardizes the temporal and spatial slicing across nine years. The pipeline is explicit and reproducible, the aggregate counts check out (I summed Table 1 myself), and the limitations section is honest about selection bias, bot filtering, and observational scope. I would use it as a benchmark or comparative testbed.

The soft spot is exactly what the stress-test flagged. The data model assumes each intervention has a discrete timestamp t0, and all windows are t0 ± 3 months. But Table 1 lists the two post-removal interventions (r/AskReddit and r/science) with date '1-30 Jun 22' — a month-long period, not a point. Those two rows cover roughly 278K of the 339K distinct users. If t0 is ambiguous, the IN-BEFORE and IN-AFTER slices cannot both be exactly three months around the same event. The paper doesn't mention this. It's not a fatal flaw: researchers can still use those slices with care, and the rest of the dataset is aligned. But the 'consistent and comparable windows' claim is weakened for the largest part of the dataset, and the paper should either re-define those interventions with a specific t0 or add a caveat.

The other issues are minor and mostly disclosed: migration slices depend on exact username matches, and the ten-post threshold filters out low-activity users. Both are stated in the methodology. The intervention selection leans on prior literature and the authors' own case studies, which limits novelty but is not circular — the dataset is a resource, not a test of a hypothesis.

Overall, this deserves a serious referee. I would accept it conditional on addressing the t0 ambiguity for the two post-removal rows, either by picking a defined date or by releasing separate pre/post windows for those cases. The dataset will be useful to anyone studying moderation effects, and the descriptive analysis is appropriately hedged.

Recommendation: send to peer review with a request for a revision note on the date-range issue. It's a solid dataset paper that needs one clarification.

Referee Report

2 major / 3 minor

Summary. The paper introduces TBBT, a dataset of 25 content-moderation interventions on Reddit and Voat, spanning 2015–2023. For each intervention, the authors provide standardized metadata and pseudonymized user activity in four slices (IN/OUT × BEFORE/AFTER) aligned to fixed three-month windows around an intervention timestamp t0. The dataset reportedly contains 38,700,732 comments from 339,125 distinct users. A six-step collection pipeline is described, and the dataset is released on Zenodo with a DOI. The paper also presents descriptive exploratory analyses and discusses use cases for moderation research.

Significance. If the dataset is as described, it is a valuable multi-intervention resource for studying moderation effects, biases, and spillovers, enabling comparative and reproducible research that existing single-intervention datasets do not support. Strengths include the explicit and mostly transparent pipeline, the public DOI, pseudonymization, FAIR compliance, and the inclusion of cross-platform migration data for selected interventions. The paper is honest about limitations such as platform skew, observational design, and user-selection bias. However, the central value proposition—consistent and comparable pre/post windows across all interventions—is materially compromised for the two largest rows by a mismatch between the data model's discrete-t0 assumption and the month-long date range reported for those interventions.

major comments (2)

[Methodology, Data Model and Data Collection step 1; Table 1 rows 1 and 6] The data model assumes a discrete, time-stamped intervention t0, with pre/post windows defined as t0 ± 3 months (Figure 1; Data Collection step 1). However, Table 1 lists the date for the two post-removal interventions, r/AskReddit and r/science, as '1-30 Jun 22' — a month-long range, not a single timestamp. For these rows, the IN-BEFORE and IN-AFTER slices cannot both be aligned to the same t0: taking t0 as June 1 excludes June from the pre-window; taking it as June 30 includes June in the pre-window and shifts the post-window. Either way, the claim of 'systematically aligned observations from fixed windows' fails for these two interventions, which together cover roughly 278K of the 339K distinct users. The paper does not disclose or discuss this discrepancy in Methodology or Limitations. The authors must either provide exact enforcement timestamps for these post-removal cases or redefi
[Dataset Description, Table 1 and Limitations] The total of 339,125 distinct users is presented as a headline number, but the row-level user counts in Table 1 sum to 352,461 in the IN-BEFORE column alone, indicating that users appear in multiple interventions. The paper does not state whether the 339K figure is de-duplicated across the four slices and across interventions, or how a user active in multiple rows is counted. This is relevant for any cross-intervention analysis and should be clarified, even if the figure is correct.

minor comments (3)

[Table 1] The column headings contain apparent artifacts: '/user-friends' and '/commen◎s' likely should read 'users' and 'comments'. Please correct these rendering issues.
[Dataset Overview and Descriptive Statistics; Figure 5] The descriptive analysis reports that activity decreases 'across nearly all interventions' but provides no statistical tests or uncertainty measures. For a dataset paper this is acceptable, but adding paired tests or effect sizes would strengthen the claim.
[Data Collection and Preparation, step 4] For migration cases, the matching of usernames across Reddit and Voat is asserted but not described in detail. Please specify the matching procedure and any validation performed, since this determines the integrity of the OUT-AFTER slices for those interventions.

Circularity Check

0 steps flagged

No significant circularity: the dataset is assembled from external public archives; no derivation or prediction reduces to its inputs.

full rationale

This is a dataset description paper, not a derivation or prediction paper. The central contribution is an assembled corpus: TBBT is constructed from publicly available archives (an academic torrent of Reddit data and a Zenodo Voat repository), then filtered, standardized, and pseudonymized through a fixed six-step pipeline. There is no fitted parameter that is later called a prediction, no uniqueness theorem invoked to force a choice, and no equation that reduces to itself by construction. The descriptive before/after comparisons in Figure 5 are explicitly descriptive and hedged ('These descriptive patterns are consistent with prior findings'), and the paper repeatedly acknowledges that the dataset is observational and does not support direct causal claims. The selection of interventions draws partly on prior work by the same authors, but the dataset content itself is not derived from those prior papers; it is re-collected from external sources, and the self-citations serve as context for which interventions are well-studied, not as evidence that the dataset's contents are correct. The one notable issue — Table 1 listing '1-30 Jun 22' as the date for the two post-removal interventions, which is inconsistent with the paper's discrete t0 data model — is a data-quality or internal-consistency concern, not a circularity concern. It does not make the dataset's claims equivalent to their inputs. No circular step can be identified under the required standard of quoting a specific reduction, so the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The dataset rests on hand-chosen thresholds, third-party archives, and prior literature rather than fitted parameters. The main unexamined load-bearing premises are archive completeness, the accuracy of prior intervention labels, and the mapping of each intervention to a single timestamp.

free parameters (4)

min_pre_intervention_messages = 10
User-selection threshold in pipeline step 3; excludes low-activity users and shapes every cohort in the dataset.
bot_filter_identical_timestamps = >=2 identical timestamps
Bot filter heuristic in pipeline step 2; may both miss bots and remove fast-posting humans.
observation_window_months = 3
Fixed pre/post window used for all slices; sensitivity of the comparison to this choice is not analyzed.
cross_platform_username_match = exact username string
Migration slices include only users whose Reddit username exactly matches their Voat username, a non-random subset of migrants.

axioms (4)

domain assumption Public archives (Pushshift torrent and Voat Zenodo dump) contain complete and accurate data for the selected communities and windows.
Data collection step 1 relies entirely on these archives; no independent completeness check is reported.
domain assumption Intervention dates, targets, and types derived from prior literature are accurate.
The intervention selection step relies on reported events; errors propagate to all t0 alignments.
domain assumption Same username across Reddit and Voat identifies the same person for migration slices.
Migrations section; no verification beyond username equality is provided.
domain assumption Hashing identifiers in metadata and text preserves data utility while sufficiently mitigating re-identification.
Pseudonymization step and Ethical Considerations; no re-identification risk assessment or utility benchmark is supplied.

pith-pipeline@v1.3.0-alltime-deepseek · 14765 in / 15922 out tokens · 169642 ms · 2026-08-03T10:05:33.598003+00:00 · methodology

0 comments

read the original abstract

Online platforms rely on moderation interventions to curb harmful behavior such as hate speech, toxicity, and the spread of mis- and disinformation. Yet research on the effects and possible biases of such interventions faces multiple limitations. For example, existing works frequently focus on single or a few interventions, due to the absence of comprehensive datasets. As a result, researchers must typically collect the necessary data for each new study, which limits opportunities for systematic comparisons. To overcome these challenges, we introduce The Big Ban Theory (TBBT) -- a large dataset of moderation interventions. TBBT covers 25 interventions of varying type, severity, and scope, comprising in total over 339K users and nearly 39M posted messages on Reddit and Voat. For each intervention, we provide standardized metadata and pseudonymized user activity collected three months before and after its enforcement, enabling consistent and comparable analyses of intervention effects. In addition, we provide a descriptive exploratory analysis of the dataset, along with several use cases of how it can support research on content moderation. With this dataset, we aim to support researchers studying the effects of moderation interventions and to promote more systematic, reproducible, and comparable research.

Figures

Figures reproduced from arXiv: 2601.11128 by Aldo Cerulli, Benedetta Tessa, Lorenzo Cima, Serena Tardelli, Stefano Cresci.

**Figure 1.** Figure 1: For an intervention occurring at time t0, we identify four data slices: IN-BEFORE and IN-AFTER (if available) capture activity in the moderated space before and after the intervention, while OUT-BEFORE and OUT-AFTER capture activity by the same users outside that space. such as interrupted time series analysis or Bayesian structural time series models to estimate both immediate and longer-term interven… view at source ↗

**Figure 2.** Figure 2: Overview of the six steps of the data collection and processing pipeline used to construct the dataset. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Figure 3: Dataset structure, showing the top-level fold [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

**Figure 4.** Figure 4: Frequency of interventions in the TBBT dataset, by type (y axis) and year (x axis) (central heatmap), with marginal distributions (top and right bar charts). unfold through platform migration rather than within the original moderated space and platform (Horta Ribeiro et al. 2021). Finally, TBBT includes some noteworthy cases in which the same community is affected by multiple moderation interventions and … view at source ↗

**Figure 5.** Figure 5: Number of comments (top panels) and active users (bottom panels) before and after each moderation intervention. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

6 extracted references · 2 linked inside Pith

[6]

I Won the Election!

Community notes reduce engagement with and dif- fusion of false information online.PNAS, 122(38). Tessa, B.; Amram, D.; Monreale, A.; and Cresci, S. 2025a. Improving regulatory oversight in online content modera- tion.ECML-PKDD Workshops. Tessa, B.; Cima, L.; Trujillo, A.; Avvenuti, M.; and Cresci, S. 2025b. Beyond trial-and-error: Predicting user abandon...

arXiv 2022
[2019]

Learn the facts about COVID-19

Procedural justice in algorithmic fairness: Leveraging transparency and outcome control for fair algorithmic medi- ation. InACM CSCW. Ling, C.; Gummadi, K. P.; and Zannettou, S. 2023. “Learn the facts about COVID-19”: Analyzing the use of warning labels on TikTok videos. InAAAI ICWSM. Mekacher, A.; and Papasavva, A. 2022. “I Can’t Keep It Up.” A dataset f...

2023
[2020]

Russo, G.; Verginer, L.; Horta Ribeiro, M.; and Casiraghi, G

The implied truth effect: Attaching warnings to a sub- set of fake news headlines increases perceived accuracy of headlines without warnings.Management Science, 66(11). Russo, G.; Verginer, L.; Horta Ribeiro, M.; and Casiraghi, G
[2022]

Video Un- available

Are proactive interventions for Reddit communities feasible? InAAAI ICWSM. Habib, H.; and Nithyanand, R. 2022. Exploring the magni- tude and effects of media influence on Reddit moderation. InAAAI ICWSM. Horta Ribeiro, M.; Cheng, J.; and West, R. 2023. Auto- mated content moderation increases adherence to commu- nity guidelines. InACM Web Conference. Hort...

Pith/arXiv arXiv 2022
[2023]

In AAAI ICWSM

Spillover of antisocial behavior from fringe platforms: The unintended consequences of community banning. In AAAI ICWSM. Seckin, O. C.; Pote, M.; Nwala, A. C.; Yin, L.; Luceri, L.; Flammini, A.; and Menczer, F. 2025. Labeled datasets for research on information operations. InAAAI ICWSM. Shahi, G. K.; Tessa, B.; Trujillo, A.; and Cresci, S. 2025. A year of...

2025
[2025]

Monti, C.; Cinelli, M.; Valensise, C.; Quattrociocchi, W.; and Starnini, M

Post-Post-API Age: Studying digital platforms in scant data access times.arXiv:2505.09877. Monti, C.; Cinelli, M.; Valensise, C.; Quattrociocchi, W.; and Starnini, M. 2023. Online conspiracy communities are more resilient to deplatforming.PNAS Nexus, 2(10). Niverthi, M.; Verma, G.; and Kumar, S. 2022. Characteriz- ing, detecting, and predicting online ban...

Pith/arXiv arXiv 2023

[1] [6]

I Won the Election!

Community notes reduce engagement with and dif- fusion of false information online.PNAS, 122(38). Tessa, B.; Amram, D.; Monreale, A.; and Cresci, S. 2025a. Improving regulatory oversight in online content modera- tion.ECML-PKDD Workshops. Tessa, B.; Cima, L.; Trujillo, A.; Avvenuti, M.; and Cresci, S. 2025b. Beyond trial-and-error: Predicting user abandon...

arXiv 2022

[2] [2019]

Learn the facts about COVID-19

Procedural justice in algorithmic fairness: Leveraging transparency and outcome control for fair algorithmic medi- ation. InACM CSCW. Ling, C.; Gummadi, K. P.; and Zannettou, S. 2023. “Learn the facts about COVID-19”: Analyzing the use of warning labels on TikTok videos. InAAAI ICWSM. Mekacher, A.; and Papasavva, A. 2022. “I Can’t Keep It Up.” A dataset f...

2023

[3] [2020]

Russo, G.; Verginer, L.; Horta Ribeiro, M.; and Casiraghi, G

The implied truth effect: Attaching warnings to a sub- set of fake news headlines increases perceived accuracy of headlines without warnings.Management Science, 66(11). Russo, G.; Verginer, L.; Horta Ribeiro, M.; and Casiraghi, G

[4] [2022]

Video Un- available

Are proactive interventions for Reddit communities feasible? InAAAI ICWSM. Habib, H.; and Nithyanand, R. 2022. Exploring the magni- tude and effects of media influence on Reddit moderation. InAAAI ICWSM. Horta Ribeiro, M.; Cheng, J.; and West, R. 2023. Auto- mated content moderation increases adherence to commu- nity guidelines. InACM Web Conference. Hort...

Pith/arXiv arXiv 2022

[5] [2023]

In AAAI ICWSM

Spillover of antisocial behavior from fringe platforms: The unintended consequences of community banning. In AAAI ICWSM. Seckin, O. C.; Pote, M.; Nwala, A. C.; Yin, L.; Luceri, L.; Flammini, A.; and Menczer, F. 2025. Labeled datasets for research on information operations. InAAAI ICWSM. Shahi, G. K.; Tessa, B.; Trujillo, A.; and Cresci, S. 2025. A year of...

2025

[6] [2025]

Monti, C.; Cinelli, M.; Valensise, C.; Quattrociocchi, W.; and Starnini, M

Post-Post-API Age: Studying digital platforms in scant data access times.arXiv:2505.09877. Monti, C.; Cinelli, M.; Valensise, C.; Quattrociocchi, W.; and Starnini, M. 2023. Online conspiracy communities are more resilient to deplatforming.PNAS Nexus, 2(10). Niverthi, M.; Verma, G.; and Kumar, S. 2022. Characteriz- ing, detecting, and predicting online ban...

Pith/arXiv arXiv 2023