REVIEW 2 major objections 4 minor 45 references
Collaborative Content Moderation in the Fediverse
T0 review · 2 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Mastodon servers moderate better by sharing model weights
desk verdict Real evaluation leak in an otherwise solid system paper: author overlap between train and test inflates the headline numbers, and the proxy-label tasks make the moderation claims shakier than the prose admits. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The core mechanism is parameter exchange among similar instances: instead of sharing posts or labels, servers share the weights of their partially trained classifiers and average those weights together. Peer similarity is computed as the Jaccard similarity of each instance's trending hashtags (hashsim), which the paper shows predicts whether sharing model updates will help; a pre-sampling stage ranks candidate peers by how many of their users the local instance's users follow, reducing the number of similarity computations by 90%. The parameter averaging is what does the work: it pools linguistic patterns and moderates the effect of individual administrators' labeling mistakes.
What would settle it
Re-run the FedMod and local-only comparison on the same 50 instances using ground-truth labels from real administrator moderation actions (or a held-out set of human moderation judgments) instead of the three automated and crowd-sourced proxies; if collaborative training does not beat local-only training on that human-labeled test set, the central claim fails.
Extended reading notes
Core claim
FedMod is a collaborative content-moderation protocol built on federated learning. Each instance first fine-tunes a multilingual transformer classifier on its own small labeled set; it then picks k peers whose weekly trending hashtags are most Jaccard-similar to its own, and the peer group exchanges model parameters and locally averages them with FederatedAveraging to form a shared model. The paper reports that this procedure beats training on local data alone for harmful-content detection, bot-content detection, and content-warning assignment across all 50 instances in the study, reaching average per-instance macro-F1 scores of 0.71, 0.73, and 0.58, respectively. The gains persist when a quarter of the training labels are randomly flipped and when instances use different harmfulness thresholds, and they are largest when instances have very few labeled posts. The optional pre-sampling step cuts peer-search computations by 90% with only a small drop in performance. The paper also shows a limit: if a large fraction of collaborating instances hold completely opposing moderation policies, the shared model degrades below local-only performance.
Load-bearing premise
The load-bearing assumption is that the proxy labels used for training and testing—automated toxicity scores, self-declared bot flags, and author-set content warnings—are good stand-ins for what real instance administrators would actually moderate; if those proxies misrepresent moderation decisions, the measured improvements may not appear in live Fediverse moderation.
Editorial extensions
If this is right
- An instance with roughly 128 labeled posts can reach macro-F1 of 0.71 on harmful content, 0.73 on bots, and 0.58 on content warnings by collaborating with two peers, and more peers help when labels are scarcer.
- Hashtag-based peer selection (hashsim) beats random selection on every task, so topical affinity is a usable signal for choosing moderation partners.
- Because only model parameters change hands, instances gain moderation knowledge without disclosing their labeled posts.
- Label noise is partly absorbed by collaboration: with 25% of harmful-content labels flipped, FedMod still beats local models by 14.54%.
- Collaborating with instances that enforce opposite moderation policies can hurt; FedMod remains ahead of local models when 25% of peers differ, but not when 50% do.
Reading between the lines
- A natural next test is to replace the three proxy label sources (an automated toxicity scorer, self-declared bot flags, and author-set content warnings) with moderation decisions made by real instance administrators; FedMod's advantage should survive that substitution if the mechanism, rather than the label source, is what drives the gains.
- Peer selection could be made from signals of actual moderation-policy alignment, such as shared domain blocks or suspension lists, rather than hashtag similarity; this may mitigate the opposing-policy failure mode the paper identifies.
- The same parameter-exchange recipe could be applied to other label-scarce decentralized tasks such as spam detection, and the paper's pre-sampling logic suggests it should scale to the full Fediverse.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. FedMod proposes a federated-learning-based content moderation system for Mastodon instances. Using a dataset of 219,577 posts from 50 instances, the paper evaluates three tasks: harmful content detection (with Perspective API labels), bot content detection (self-declared bot flags), and content warning assignment (self-tagged sensitive posts). Each instance trains a local mBERT classifier on 128 labeled posts; FedMod lets each instance average parameters with k similar peers selected by hashtag Jaccard similarity. The paper reports that FedMod improves macro-F1 over local-only training for all three tasks (12.69%, 5.79%, and 9.43%) and shows robustness to label perturbation, peer-count variations, pre-sampling, and divergent moderation policies.
Significance. The problem is timely and important: decentralized moderation is a real bottleneck, and a privacy-preserving parameter-exchange scheme is a sensible proposal. The paper's empirical methodology is thorough in several respects: it uses a multi-instance dataset, reports bootstrapped confidence intervals and Wilcoxon test results, and systematically ablates design choices (number of peers, pre-sampling, label noise, and policy disagreement). These strengths make the headline claims credible if the evaluation-validity concerns are resolved. However, the current evaluation does not fully establish that the reported gains correspond to real moderation effectiveness, because of the author-leakage and proxy-label issues detailed below. The work would be a solid contribution to the measurement and systems community after addressing those points.
major comments (2)
- [§4 "Training & Testing Sets" and §3.1.3] The 80:20 split of posts is performed at the post level without grouping by author or bot account. Because the bot detection task contains 42,566 posts from only 3,371 bot accounts and the content-warning task contains 11,074 posts from 3,905 authors (Section 3.1.3), the same account almost certainly appears in both training and test splits of an instance. The fine-tuned mBERT can then memorize account-specific style rather than learn a general moderation rule, which inflates the absolute macro-F1 scores. More importantly for the central claim, the leakage may affect local models and FedMod differently: a local model trained on 128 posts can exploit within-instance account overlap directly, whereas FedMod's parameter averaging with peers may dilute that memorization, making the reported FedMod gains either conservative or spurious depending on the task. I request an author-disjoint (or account-disjoint) split, or a per-account held-out evaluation, for at least the bot and content-warning tasks to confirm that the FedMod-vs-local ranking is not an artifact of this leakage.
- [§3.1.3 and §6.1] The harmful-content task uses Perspective API scores as ground truth for both training and testing. Consequently, the reported macro-F1 measures how well FedMod agrees with the Perspective API, not how well it moderates according to actual instance administrators. The paper acknowledges this proxy issue (e.g., Section 3.1.3 and Appendix A.3), but the abstract and introduction state improvements in 'automated content moderation' without this qualification. The perturbed-label experiment simulates label noise but does not establish that Perspective labels are a valid surrogate for administrator decisions. I recommend reframing the claims as improvements on surrogate labels, or adding a small human-annotated evaluation set (even a few hundred posts) to show the ranking between FedMod and local models transfers to a more direct measure of moderation quality.
minor comments (4)
- [Figure 1 and §3.2] The caption of Figure 1 mentions only the distribution of posts per instance, but the text refers to it as showing distributions of harmful posts, bot posts, and content-warning posts. Please align the caption with the actual panels or the references in the text.
- [§4] Please clarify whether the online learning sampling of 16 posts at each step is with or without replacement from the training set; this affects whether the reported N=128 corresponds to 128 unique posts.
- [Appendix A.5] There is a typo in 'Perspcetive' (should be 'Perspective') in the Appendix A.5 setup description.
- [§6.1] The phrase 'we also re-plot the results of the local content moderation models' is imprecise; these results were not plotted earlier in the same form. Consider saying 'we also plot'.
Circularity Check
No circularity: FedMod's gains are empirical comparisons under shared labels and splits; Perspective proxy and author leakage are validity risks, not derivation-level circularity.
full rationale
The paper contains no derivation chain of the kind that can be circular: FedMod is an application of standard federated averaging (§5, Step 3), and every headline number (0.71/0.73/0.58; 12.69%/5.79%/9.43% improvements) is an empirical macro-F1 measured on held-out post splits, not an analytic consequence of the system's definition. The strongest candidate for 'self-definitional' circularity is the harmful-content task: both training and test labels come from Perspective API (§3.1.3). But the paper explicitly defines the target as Perspective-derived: it 'utilize[s] the Perspective API [1] to label all Mastodon posts, serving as a proxy for the actual labels from instance administrators,' and Appendix A.3 explains why Perspective cannot be used directly. Evaluating a classifier against the same label source used for training is standard supervised learning, not a case of predicting Y from a definition of Y. The FedMod-vs-local comparison is fair in the sense that both systems are trained and tested under identical label definitions, split procedures, and per-instance training budgets (§4, §6.1). The post-level 80/20 split without author grouping is a data-leakage risk that can inflate absolute macro-F1 or distort the relative gain, but it is an experimental-validity concern, not circularity: neither baseline's output is an input to the other by construction. Self-citations [3,7,16,43] appear in motivation and related work only; the federated-learning result does not rest on a self-cited theorem or dataset. No step reduces to its own inputs.
Assumptions & free parameters
free parameters (6)
- Perspective threshold =
0.5 (0.8 in sensitivity check)
- Number of peers k =
2 (varied to 3, 4, 10)
- Per-instance training budget =
8 steps x 16 posts = 128 labeled posts
- Label perturbation rate =
25% label flips per step
- mBERT fine-tuning hyperparameters =
lr 5e-5, batch 16, 2 epochs
- Hashsim top-k and pre-sampling size =
k=2, i=5 or 10
assumptions (5)
- domain assumption Perspective API scores are a valid proxy for instance administrators' harmfulness judgments.
- domain assumption Self-declared bot flags and self-tagged content warnings are sufficiently accurate labels.
- domain assumption Instances with similar trending hashtags have sufficiently similar content and label distributions to benefit from federated averaging.
- domain assumption Federated averaging of fine-tuned transformer parameters is effective in this low-resource, non-IID setting.
- domain assumption The random sample of 50 instances is representative of the Fediverse.
Cite this review
Pith. "Pith review of Collaborative Content Moderation in the Fediverse." pith.science (2026). https://pith.science/paper/GFF33IBA
@misc{pith2026250105871,
author = {Pith},
title = {Pith review of: Collaborative Content Moderation in the Fediverse},
year = {2026},
howpublished = {\url{https://pith.science/paper/GFF33IBA}},
note = {Machine review of arXiv:2501.05871}
}
read the original abstract
The Fediverse, a group of interconnected servers providing a variety of interoperable services (e.g. micro-blogging in Mastodon) has gained rapid popularity. This sudden growth, partly driven by Elon Musk's acquisition of Twitter, has created challenges for administrators though. This paper focuses on one particular challenge: content moderation, e.g. the need to remove spam or hate speech. While centralized platforms like Facebook and Twitter rely on automated tools for moderation, their dependence on massive labeled datasets and specialized infrastructure renders them impractical for decentralized, low-resource settings like the Fediverse. In this work, we design and evaluate FedMod, a collaborative content moderation system based on federated learning. Our system enables servers to exchange parameters of partially trained local content moderation models with similar servers, creating a federated model shared among collaborating servers. FedMod demonstrates robust performance on three different content moderation tasks: harmful content detection, bot content detection, and content warning assignment, achieving average per-server macro-F1 scores of 0.71, 0.73, and 0.58, respectively.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[7]
Haris Bin Zia, Aravindh Raman, Ignacio Castro, Ishaku Hassan Anaobi, Emil- iano De Cristofaro, Nishanth Sastry, and Gareth Tyson. 2022. Toxicity in the decentralized web and the potential for model sharing. Proceedings of the ACM on Measurement and Analysis of Computing Systems 6, 2 (2022), 1–25
work page 2022
- [1]
-
[2]
ActivityPub. 2018. https://www.w3.org/TR/activitypub/
work page 2018
-
[3]
Ishaku Hassan Anaobi, Aravindh Raman, Ignacio Castro, Haris Bin Zia, Damilola Ibosiola, and Gareth Tyson. 2023. Will Admins Cope? Decentralized Moderation in the Fediverse. In Proceedings of the ACM Web Conference 2023 . 3109–3120
work page 2023
-
[4]
Galen Andrew, Om Thakkar, Brendan McMahan, and Swaroop Ramaswamy
-
[5]
Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva Varma. 2017. Deep learning for hate speech detection in tweets. In Proceedings of the 26th international conference on World Wide Web companion . 759–760
work page 2017
-
[6]
Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Javier Fernandez- Marques, Yan Gao, Lorenzo Sani, Kwing Hei Li, Titouan Parcollet, Pedro Porto Buarque de Gusmão, et al . 2020. Flower: A friendly federated learning research framework. arXiv preprint arXiv:2007.14390 (2020)
arXiv 2020
-
[8]
Devin Coldewey. 2019. Racial bias observed in hate speech detection algorithm from Google. https://techcrunch.com/2019/08/14/racial-bias-observed-in-hate- speech-detection-algorithm-from-google/
work page 2019
Show all 45 references
-
[9]
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guil- laume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116 (2019)
2019 arXiv
-
[10]
Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. 2017. Automated hate speech detection and the problem of offensive language. In Proceedings of the international AAAI conference on web and social media , Vol. 11. 512–515
2017
-
[11]
Decentralized social media platform Mastodon deals with an influx of Gab users
-
[12]
Fabio Del Vigna12, Andrea Cimino23, Felice Dell’Orletta, Marinella Petrocchi, and Maurizio Tesconi. 2017. Hate me, hate me not: Hate speech detection on facebook. In Proceedings of the first Italian conference on cybersecurity (ITASEC17) . 86–95
2017
-
[13]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
2018 arXiv
-
[14]
Diaspora. 2010. https://diasporafoundation.org
2010
-
[15]
Tarleton Gillespie. 2020. Content moderation, AI, and the question of scale. Big Data & Society 7, 2 (2020), 2053951720943234
2020
-
[16]
Anaobi Ishaku Hassan, Aravindh Raman, Ignacio Castro, Haris Bin Zia, Emiliano De Cristofaro, Nishanth Sastry, and Gareth Tyson. 2021. Exploring content moderation in the decentralised web: The pleroma case. InProceedings of the 17th International Conference on emerging Network...
2021
-
[17]
Deepak Kumar, Jeff Hancock, Kurt Thomas, and Zakir Durumeric. 2023. Under- standing the behaviors of toxic accounts on reddit. In Proceedings of the ACM Web Conference 2023. 2797–2807
2023
-
[18]
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436–444
2015
-
[19]
Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated learning: Challenges, methods, and future directions. IEEE signal processing magazine 37, 3 (2020), 50–60
2020
-
[20]
Mastodon. 2016. https://joinmastodon.org
2016
-
[21]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep net- works from decentralized data. In Artificial intelligence and statistics . PMLR, 1273–1282
2017
-
[22]
H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. 2017. Learning differentially private recurrent language models. arXiv preprint arXiv:1710.06963 (2017)
2017 arXiv
-
[23]
Endang Wahyu Pamungkas, Valerio Basile, and Viviana Patti. 2020. Misogyny de- tection in twitter: a multilingual and cross-domain study. Information Processing & Management 57, 6 (2020), 102360
2020
-
[24]
Is it a qoincidence?
Antonis Papasavva, Jeremy Blackburn, Gianluca Stringhini, Savvas Zannettou, and Emiliano De Cristofaro. 2021. “Is it a qoincidence?”: An exploratory study of QAnon on Voat. In Proceedings of the Web Conference 2021 . 460–471
2021
-
[25]
Antonis Papasavva, Savvas Zannettou, Emiliano De Cristofaro, Gianluca Stringh- ini, and Jeremy Blackburn. 2020. Raiders of the lost kek: 3.5 years of augmented 4chan posts from the politically incorrect board. InProceedings of the international AAAI conference on web and socia...
2020
-
[26]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing system...
2019
-
[27]
PeerTube. 2018. https://joinpeertube.org
2018
-
[28]
Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. How multilingual is multi- lingual BERT? arXiv preprint arXiv:1906.01502 (2019)
2019 arXiv
-
[29]
Pleroma. 2016. https://pleroma.social
2016
-
[30]
Sara Rosenthal, Pepa Atanasova, Georgi Karadzhov, Marcos Zampieri, and Preslav Nakov. 2020. SOLID: A large-scale semi-supervised dataset for offensive language identification. arXiv preprint arXiv:2004.14454 (2020)
2020 arXiv
-
[31]
Paul Röttger, Debora Nozza, Federico Bianchi, and Dirk Hovy. 2022. Data- Efficient Strategies for Expanding Hate Speech Detection into Under-Resourced Languages. arXiv preprint arXiv:2210.11359 (2022)
2022 arXiv
-
[32]
Paul Röttger, Bertram Vidgen, Dong Nguyen, Zeerak Waseem, Helen Margetts, and Janet B Pierrehumbert. 2020. HateCheck: Functional tests for hate speech detection models. arXiv preprint arXiv:2012.15606 (2020). Zia et al
2020 arXiv
-
[33]
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Dis- tilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108 (2019)
2019 arXiv
-
[34]
Mohit Singhal, Chen Ling, Nihal Kumarswamy, Gianluca Stringhini, and Shirin Nilizadeh. 2022. SoK: content moderation in social media, from guidelines to enforcement, and research to practice. arXiv preprint arXiv:2206.14855 (2022)
2022 arXiv
-
[35]
Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. How to fine-tune bert for text classification?. In Chinese Computational Linguistics: 18th China National Conference, CCL 2019, Kunming, China, October 18–20, 2019, Proceedings
2019
-
[36]
Leanne Townsend and Claire Wallace. 2017. The ethics of using social media data in research: A new framework. In The ethics of online research . Emerald Publishing Limited, 189–207
2017
-
[37]
Bertie Vidgen and Taha Yasseri. 2020. Detecting weak and strong Islamophobic hate speech on social media. Journal of Information Technology & Politics 17, 1 (2020), 66–78
2020
-
[38]
William Warner and Julia Hirschberg. 2012. Detecting hate speech on the world wide web. In Proceedings of the second workshop on language in social media . 19–26
2012
-
[39]
Wikipedia. 2023. https://en.wikipedia.org/wiki/Online_machine_learning
2023
-
[40]
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement De- langue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-of-the-art natural language processing. In Pro- ceedings of the 2020 conference on empirical met...
2020
-
[41]
Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th international conference on world wide web. 1391–1399
2017
-
[42]
Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Sara Rosenthal, Noura Farra, and Ritesh Kumar. 2019. Predicting the type and target of offensive posts in social media. arXiv preprint arXiv:1902.09666 (2019)
2019 arXiv
-
[43]
harm- ful
Haris Bin Zia, Jiahui He, Aravindh Raman, Ignacio Castro, Nishanth Sastry, and Gareth Tyson. 2023. Flocking to mastodon: Tracking the great twitter migration. arXiv preprint arXiv:2302.14294 (2023). A APPENDIX A.1 Ethics We exclusively collect publicly available posts, followi...
2023 arXiv
-
[2019]
https://www.tsf.foundation/blog/decentralized-social-media-platform- mastodon-deals-with-an-influx-of-gab
-
[2021]
Advances in Neural Information Processing Systems 34 (2021), 17455–17466
Differentially private learning with adaptive clipping. Advances in Neural Information Processing Systems 34 (2021), 17455–17466
2021
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.