{"id":"31cc5cef-9bbc-4eb3-9d20-cd666ab23214","arxiv_id":"2507.05455","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A community-annotated toxicity dataset with conversational context shows that models trained on ingroup labels outperform state-of-the-art moderation APIs.","lead":"This paper introduces ModelCitizens, a dataset of about 6,800 social media posts with 40,000 toxicity ratings from both community insiders and outsiders, plus AI-generated conversation context. It also releases two fine-tuned models that beat OpenAI's GPT-o4-mini by 5.5% on this new benchmark.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Statement-level leakage between train and test likely inflates the reported 5.5% and 9% accuracy gains.","rationale":"The reader's weakest assumption (ingroup-as-gold) is a genuine epistemic debate, but it is an explicitly stated design position rather than an internal inconsistency; the paper even includes a limitations section acknowledging subjectivity. The more decisive risk to the central empirical claim is statement-level leakage. The dataset's construction guarantees that most base statements appear in multiple instances, and a stratified instance-level split will scatter variants of the same statement across train and test. This gives the finetuned model an unfair advantage over all zero-shot baselines, directly undermining the quantitative comparison that the abstract and conclusion rest on. I therefore propose a grouped-split rerun as the single check that would settle whether the headline gain is real. If the gain persists after statement-level deduplication, the community-informed training claim is substantially strengthened; if it collapses, the paper's main quantitative contribution is invalid. This is a correctness issue, not a normative disagreement, so it should take priority in the revision.","tokens_in":17437,"tokens_out":5963,"duration_ms":66629,"concrete_test":"Group all 6,822 instances by the original Toxigen statement (or exact statement text) and re-run the 10% stratified split so no base statement appears in both train and test. Retrain LLAMACITIZEN-8B and the best baselines on the grouped train split and report accuracy on the grouped test split and on the context-augmented subset. If LLAMACITIZEN's margin over GPT-o4-mini falls from 5.5%/9% to near zero or becomes non-significant, the central claim is an artifact of leakage. Also report the fraction of test instances whose base statement appears in train under the original split.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"MODELCITIZENS is constructed from 2,520 Toxigen base statements, each appearing once as a standalone post and again in up to two LLM-generated context versions (4,302 context-augmented posts; total 6,822). The train/test split is described only as a 10% stratified sample with 'no overlap' between train and test. Unless deduplication was performed at the base-statement level, the same STATEMENT text appears in both splits under different contexts. For a base statement with three variants, the probability that at least one variant falls in test while another remains in train is about 27%; with two variants, about 19%. A finetuned model can exploit statement-level regularities learned from the sibling variant, while the zero-shot baselines cannot. This would directly inflate the headline 5.5% overall gain and the 9% context-augmented gain, making the comparison to baselines unfair. The paper's contamination check needs to be at the level of the original Toxigen statement, not the packaged instance.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MODELCITIZENS, a toxicity annotation dataset built from 2,520 Toxigen statements, each appearing as a single post and, for a subset, as the core of an LLM-generated Reddit-style conversation, yielding 6,822 instances and about 40K ingroup/outgroup annotations across eight identity groups. The authors report that ingroup and outgroup annotators disagree on 27.5% of posts, that commercial moderation APIs underperform on the dataset (average accuracy 63.6%, dropping to 59.6% on context-augmented posts), and that fine-tuned models LLAMACITIZEN-8B and GEMMACITIZEN-12B beat the best baseline by 5.5% and 9% on the full test set and context-augmented subset, respectively. They also present annotation analyses, a label-choice ablation, a data-scaling curve, and out-of-distribution evaluations on HateModerate, Counter-Context, and Toxigen.","tokens_in":17636,"tokens_out":6512,"duration_ms":70527,"significance":"If the headline results survive a correctly deduplicated split, MODELCITIZENS would be a valuable resource for pluralistic and context-aware toxicity detection: the dataset includes balanced ingroup/outgroup annotations, annotator demographics, LLM-generated conversational contexts, and a clear release plan for data, models, and code. The annotation analyses with Krippendorff's alpha and Wilcoxon tests are appropriate, and the cross-dataset generalization checks on HateModerate and Counter-Context are a useful addition. However, the central quantitative claims rest on a small test set without uncertainty quantification and on a split whose contamination control is described only at the instance level, so the magnitude of the reported gains is not yet established.","major_comments":[{"comment":"The train/test split appears to be performed only at the instance level. Section 5.1 states that the test set is a 10% stratified sample with no overlap between train and test, but Section 3.2 and Appendix D.2 construct up to three instances from each base Toxigen statement (the single post plus harmful- and benign-context versions). Unless the split is deduplicated on the original Toxigen statement ID, a test instance can share its statement text with a training sibling; for a three-variant statement the probability that at least one variant lands in test while another remains in train is roughly 27%, and for two-variant statements roughly 19%. This gives the fine-tuned CITIZEN models an advantage over zero-shot baselines, which cannot see sibling variants in training, and it directly affects the headline 5.5% and 9% gains. The authors should re-run the split with statement-level deduplication, report the overlap rate, and re-report Tables 4 and 6.","section":"§5.1, §D.2"},{"comment":"The headline results are single point estimates on a 669-instance test set with no confidence intervals or significance tests. The 5.5% average gain corresponds to roughly 37 instances, and per-group cells contain only about 70-100 instances (based on Table 3), so small label changes could move group-level rankings. Additionally, the binary toxicity threshold is data-dependent (footnote 5 says it was chosen to maximize inter-annotator agreement on the same data), making the point estimate sensitive to that choice. The authors should report bootstrap confidence intervals or paired significance tests (e.g., McNemar) for the main comparisons, and show how the headline gain varies with the threshold.","section":"§5.1, Table 4"},{"comment":"The label-choice ablation evaluates models trained on ingroup, outgroup, and aggregated labels against the ingroup gold on the same test set. Since the ingroup-trained model's training distribution matches the test gold by construction, this comparison does not establish that ingroup labels are 'more reliable signals' (final paragraph of Section 5.3); it shows only that a model trained on a label source performs best when evaluated against that same source. To support the reliability claim, the authors should also evaluate the three models on outgroup gold and aggregated gold, or use a held-out set of annotations from each source, and report agreement/disagreement statistics.","section":"§5.3, Table 7"},{"comment":"The paper repeatedly describes ingroup annotations as 'gold' and reports baseline performance as 'accuracy', which conflates agreement with one annotation policy with correctness. The 5.5% gain is a gain in alignment with ingroup judgments, not an objective measure of toxicity detection. Because the paper's contribution includes the normative claim that ingroup judgments should be authoritative, the evaluation should state this framing explicitly and, to be informative to practitioners who may use consensus or outgroup labels, also report results under those alternative gold standards for at least the main baselines and LLAMACITIZEN-8B.","section":"§5.1, Abstract"}],"minor_comments":[{"comment":"The main text says labels are created by applying a threshold of 3.5, while the footnote says scores greater than 3 are considered toxic; please clarify the exact threshold and how it was selected.","section":"§5.1, footnote 5"},{"comment":"Section 3.1 reports sampling 2,520 posts, while Table 3 reports 2,502 single statement posts; with 4,302 context-augmented posts the implied total is 6,804, not the stated 6,822. Please reconcile these numbers.","section":"§3.1, Table 3"},{"comment":"Section 3.2 reports that 86% of generated contexts were high-quality, while Appendix D.3 reports 85%; please align these figures.","section":"§3.2, Appendix D.3"},{"comment":"Section 3.3 contains two consecutive paragraphs that both begin 'Toxigen includes 13 identity groups...'; one duplicate should be removed.","section":"§3.3"},{"comment":"The 'ΔBase LLaMA' row applies only to LLAMACITIZEN-8B; please add the corresponding Gemma delta or relabel the row to make its scope clear.","section":"Table 4"},{"comment":"Appendix D.3 contains an incomplete sentence fragment ('85We find that...') before the quotation of the validation prompt; please fix the typo.","section":"Appendix D.3"},{"comment":"The abstract and introduction describe MODELCITIZENS as 'social media posts,' although the post content is drawn from synthetic Toxigen statements and the conversational contexts are LLM-generated; please clarify in the dataset description that the posts are generated rather than collected from a platform.","section":"Abstract, §3"}],"recommendation":"major_revision","confidential_remarks":"The statement-level leakage concern is the most serious issue: if the authors cannot re-run the split with deduplication at the Toxigen statement level, the central comparison to zero-shot baselines is not interpretable. I would ask for that re-analysis, along with confidence intervals, before considering acceptance. The paper otherwise fits the journal's scope as a dataset and benchmark contribution, and the authors' release plans are appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read on arXiv:2507.05455. The dataset is the real contribution: 6.8K posts, 40K annotations from ingroup/outgroup annotators across eight identity groups, plus LLM-generated conversational contexts. That's a genuinely useful resource for anyone working on pluralistic toxicity detection, and it fills a gap – Goyal et al. only had two groups, Lee et al. had no conversational context. The disagreement analysis (missed vs. amplified harm) is measured carefully and the released models are practical.\n\nBut the headline numbers should not be trusted as-is. The 5.5% gain over GPT-o4-mini and 9% on the context-augmented subset are suspect because of a likely statement-level leak. The corpus is built from 2,520 Toxigen base statements, each appearing as a standalone post and again in up to two LLM-generated context versions. The train/test split is described only as 'no overlap' between instances. Unless the split was done at the base-statement level, the same statement text appears in both train and test under different contexts. A fine-tuned model can exploit that; the zero-shot baselines can't. The fix is straightforward: deduplicate by original statement and re-report. The authors need to tell us what they did.\n\nOther soft spots are minor in comparison: no error bars on single-seed runs, a 669-instance test set, and a binary threshold chosen from the data. The label-choice ablation in Table 7 is also structurally tilted: training on ingroup labels and evaluating against ingroup gold tells you ingroup labels are best for predicting ingroup labels. That's less damning than it sounds because the paper's whole premise is that ingroup judgments are the right target, but it doesn't provide independent evidence for that premise.\n\nThat said, the central resource is solid. The out-of-distribution eval on Toxigen, HateModerate, and Counter-Context shows real gains over the base model, so the dataset has value even if the in-distribution numbers shrink. The authors are transparent about limitations and release everything.\n\nMy recommendation: send it to peer review, but with a clear request to fix the split. As it stands, the paper is a good dataset paper with an overstated performance claim. I'd want to see the re-split numbers before accepting the 5.5% as real.","headline":"A genuinely useful community-annotated toxicity dataset, but the 5.5% headline gain is probably inflated by statement-level train/test leaks.","tokens_in":18180,"tokens_out":3229,"would_cite":true,"duration_ms":34790,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Toxicity is community-specific: a dataset built with ingroup labels as gold enables finetuned models to outperform commercial moderation APIs.","keywords":["toxicity detection","community-informed annotation","ingroup and outgroup disagreement","content moderation","hate speech","conversational context","LLM-generated context","dataset"],"falsifier":"Re-annotate the same posts with a fresh, larger panel of ingroup annotators from each of the eight communities; if the new ingroup labels disagree with the original ones on more than a third of posts, the 'ingroup as gold' anchor is not stable enough to support the benchmark. Alternatively, take a sample of MODELCITIZENS posts and have an independent adjudication panel (including community members) resolve each label; if the finetuned model's predictions match the adjudicated labels no better than GPT-o4-mini does, the claimed 5.5% gain would not replicate on a stronger gold standard.","tokens_in":17275,"feed_emoji":"🛡️","tokens_out":5560,"duration_ms":61538,"temperature":0.7,"pith_summary":"Automatic toxicity detection typically trains on labels averaged across annotators, hiding that members of a targeted community often judge a post differently from outsiders. This paper introduces MODELCITIZENS, a dataset of 6,822 social media posts across eight identity groups with 40,000 toxicity annotations split between ingroup and outgroup annotators, plus LLM-generated conversational contexts for about two-thirds of the posts. The paper shows that ingroup and outgroup annotators disagree on 27.5% of posts and that commercial moderation tools, including OpenAI's Moderation API and GPT-o4-mini, average only 63.6% accuracy against ingroup labels. Finetuning LLaMA and Gemma models on this data produces LLAMACITIZEN-8B and GEMMACITIZEN-12B, which beat the best baseline by 5.5 percentage points overall and by 9 points on the context-augmented subset. The central claim is that centering community voices in annotation and training makes toxicity detection both more accurate on community standards and more inclusive.","feed_headline":"Community-voice models beat GPT-o4-mini by 5.5%","feed_subtitle":"Ingroup annotations as gold standard help small finetuned models beat commercial moderation APIs.","key_machinery":"The central object is the MODELCITIZENS dataset with its dual ingroup/outgroup annotation design and the ingroup-as-gold label convention: each post is rated by people who self-identify with the targeted group and by people who do not, and the ingroup scores are treated as the correct answer for training and evaluation. The mechanism is the combination of community-aligned annotation, LLM-generated conversational context around each post, and full finetuning of open-weights language models on those ingroup labels, with disagreements operationalized as missed harm and amplified harm to show where outgroup labels mislead.","core_discovery":"The paper claims that toxicity is not a single objective property of a post but is determined by the target community's own perception, and that incorporating these community (ingroup) judgments as the gold standard during dataset construction and model training yields detectors that better reflect those perspectives. To demonstrate this, it introduces MODELCITIZENS, 6,822 social media posts across eight identity groups with 40K annotations, half from ingroup and half from outgroup annotators, and shows ingroup and outgroup disagree on 27.5% of posts. Commercial moderation tools average 63.6% accuracy against ingroup labels, falling to 59.6% on posts with LLM-generated conversational context; finetuned LLaMA- and Gemma-based models, LLAMACITIZEN-8B and GEMMACITIZEN-12B, outperform the best baseline (GPT-o4-mini) by 5.5 points overall and by 9 points on the context-augmented subset.","pith_inferences":["If ingroup labels are adopted as the evaluation standard, content moderation policies may need to be community-specific rather than one global classifier; a service could route moderation decisions to the relevant community's model, though the paper does not test deployment effects.","The amplified-harm pattern suggests current moderation systems may over-censor benign in-group speech (such as reclaimed language) for some groups, which could be tested by deploying the finetuned model on real moderation queues and measuring user appeals.","Replacing LLM-generated contexts with real human conversations might change the measured gains, so an explicit comparison of human versus synthetic context is a natural next experiment that the paper leaves open.","The monotonic data-scaling curve implies that collecting more community-annotated data, especially for the lowest-accuracy groups (Jewish, Latino), may yield the largest improvements; this extrapolates from the paper's 25/50/100% scaling plot."],"forward_implications":["Finetuned models trained on ingroup labels outperform commercial moderation APIs by 5.5% average accuracy on the in-distribution test set and by 9% on the context-augmented subset.","Ingroup and outgroup annotators disagree on 27.5% of posts, and outgroup annotators more frequently label content as toxic, so a single global toxicity score is not a faithful representation of community perceptions.","Adding conversational context changes toxicity labels for a substantial fraction of posts and degrades all baseline models' accuracy, whereas the finetuned model retains a 9% edge on this harder subset.","Models trained on MODELCITIZENS generalize to out-of-distribution toxicity datasets, including unseen identity groups in Toxigen, as shown by higher F1 scores on HateModerate and Counter-Context.","The label-choice ablation shows that training on ingroup labels outperforms training on outgroup or aggregated labels, suggesting that community-grounded supervision carries signal that other annotation schemes dilute or miss."],"supporting_citations":[{"why":"Supplies the Toxigen source posts used to sample MODELCITIZENS and the out-of-distribution evaluation set on unseen identity groups.","marker":"Hartvigsen et al. (2022)"},{"why":"Provides prior evidence that rater identity changes toxicity judgments, motivating the ingroup/outgroup annotation design.","marker":"Goyal et al. (2022)"},{"why":"Documents racial bias in hate speech detectors, motivating the amplified-harm analysis and community grounding.","marker":"Sap et al. (2019)"},{"why":"Shows conversational context affects toxicity, motivating the context-augmented subset of the dataset.","marker":"Pavlopoulos et al. (2020)"},{"why":"Provides the Counter-Context dataset used for out-of-distribution evaluation of conversational context.","marker":"Yu et al. (2022)"},{"why":"Supplies CREHate, a comparison dataset with community-aligned annotators and cross-cultural findings.","marker":"Lee et al. (2024)"},{"why":"Argues that aggregating labels erases perspectives, the conceptual basis for keeping ingroup labels separate.","marker":"Fleisig et al. (2024)"}],"fun_headline_variants":["Ingroup labels help 8B models beat GPT-o4-mini by 5.5%","Community voices: small finetuned models top commercial toxicity APIs","Toxic detection: community standards beat one-size-fits-all by 5.5%","Context-aware community models outperform GPT-o4-mini by 5.5 points","6.8K posts, 40K labels: community-first toxicity models win"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire benchmark treats the targeted community's own toxicity rating as the correct answer; if a reader does not accept that communities' self-assessments are the gold standard, the reported accuracy gains lose their normative meaning.","fun_headline_variants_meta":{"raw":{"variants":["Ingroup labels help 8B models beat GPT-o4-mini by 5.5%","Community voices: small finetuned models top commercial toxicity APIs","Toxic detection: community standards beat one-size-fits-all by 5.5%","Context-aware community models outperform GPT-o4-mini by 5.5 points","6.8K posts, 40K labels: community-first toxicity models win"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000322,"raw_usage":{"total_tokens":1833,"prompt_tokens":988,"completion_tokens":845,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":604,"completion_tokens_details":{"reasoning_tokens":737}},"tokens_in":604,"tokens_out":845,"duration_ms":9302,"temperature":1.0,"reasoning_tokens":737,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T19:25:20.016207+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate the same posts with a fresh, larger panel of ingroup annotators from each of the eight communities; if the new ingroup labels disagree with the original ones on more than a third of posts, the 'ingroup as gold' anchor is not stable enough to support the benchmark. Alternatively, take a sample of MODELCITIZENS posts and have an independent adjudication panel (including community members) resolve each label; if the finetuned model's predictions match the adjudicated labels no better than GPT-o4-mini does, the claimed 5.5% gain would not replicate on a stronger gold standard.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Toxigen source posts used to sample MODELCITIZENS and the out-of-distribution evaluation set on unseen identity groups."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Argues that aggregating labels erases perspectives, the conceptual basis for keeping ingroup labels separate."}],"review_version":1}