{"id":"e423244f-a920-4f54-ae18-fe4540de109f","arxiv_id":"2412.11745","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A new multilingual radical-content dataset plus an analysis showing that annotation disagreement and socio-demographic factors shift model performance and bias metrics.","lead":"This paper introduces COUNTER, a multilingual dataset of online radical content in English, French, and Arabic, and studies how annotator disagreements and demographic factors affect detection models. It shows that label aggregation choices and synthetic socio-demographic profiles change model performance and fairness measurements.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Synthetic bias analysis confounds demographic attributes with ideology: profile prompts set political view/religion, so performance gaps may reflect ideology difficulty, not model bias.","rationale":"Good-faith reading: the paper's central contribution is a multilingual radical-content dataset with multiple annotations and an analysis of annotation variation. The aggregation experiments (Table 3) convincingly show that the choice of gold labels (contractor vs. MACE vs. majority) changes model performance by up to 15 Macro-F1 points, and that inter-annotator agreement is low (Fleiss' kappa 0.26-0.50 for radicalization level). Those findings are internal to the real annotations and are not threatened by our concern. The dataset itself, pseudonymized and multilingual, is a useful resource despite restricted release and no code.\n\nThe load-bearing weakness is the bias analysis. The synthetic generation pipeline in Section 3.4 uses Wizard-Vicuna-13B-Uncensored with hand-crafted profiles. Critically, the profile variables include political view and religion/culture, which are the very variables that determine the ideology of the radical content. Thus, comparing model performance across political views or religions is comparing performance on far-right vs. far-left vs. jihadist text, not isolating the effect of demographic group membership. The fairness metrics (Section 5.2, Table 4) are then applied to these groups. Demographic parity requires equal selection rates across groups; if the generated data has different base rates of the positive class per group (because the prompt induced different content), even a perfect model would fail demographic parity. Equalized odds conditions on the true label, but the label distribution and text distribution still differ across groups, so the metric reflects distributional differences. The paper's claim that 'sociodemographic variations influence data and model outcomes' is true by construction, but the normative conclusion about model bias is not identified.\n\nThe reader's verdict (CONDITIONAL) is appropriate: the dataset and annotation-variation analysis stand, but the bias analysis needs either a within-ideology stratified reanalysis, a controlled generation design that holds content fixed while varying demographic markers, or a reframing of the findings as exploratory and not evidence of model bias. Our proposed test would settle whether the demographic gaps persist after controlling for ideology.","tokens_in":23389,"tokens_out":6264,"duration_ms":63427,"concrete_test":"Using the released synthetic English/French sets, stratify by the profile's political view and religion/culture categories, then recompute Macro-F1 and the Table 4 fairness metrics for gender, age, and ethnicity within each stratum. If the large cross-group disparities largely disappear within strata, the reported 'sociodemographic biases' are driven by ideology content, not demographic treatment. As a complementary check, compute the per-group base rates of the positive class; if they vary substantially, report the demographic parity of a Bayes-optimal classifier on the same data to show that the metric is not isolating model bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing weakness is in the synthetic bias analysis of Section 5.2. The profiles in Appendix E.1/Table 10 include 'Political view' (e.g., far-right) and 'Religion/Culture' (e.g., Islam (Jihadism)) as generation conditions. These variables determine the ideology and rhetorical content of the generated posts, so the socio-demographic attribute is not varied independently of the text's topic. Consequently, the average Macro-F1 differences across political views and religions reported in Figure 1, and the demographic-parity/equalized-odds differences in Table 4, conflate model bias with per-ideology classification difficulty. Moreover, demographic parity is evaluated on data where the base rate of the positive class (Call for Action=0) is allowed to vary across groups; even a Bayes-optimal model would show nonzero demographic-parity differences under such label shift. The paper's conclusion that 'sociodemographic variations influence data and model outcomes' is thus supported only in the trivial sense that the prompt variables influence generation; the fairness claim is not identified. The authors acknowledge stereotypical generation (Section 3.4) and synthetic-data limitations, but the confounding is an internal-validity issue that remains even if one accepts the synthetic proxy.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces COUNTER, a pseudo-anonymized multilingual dataset (English, French, Arabic) annotated for radicalization level, call-for-action, and named entities, and uses it to study how annotation choices and socio-demographic variation affect radical-content detection. The paper reports (i) an annotation-variation experiment comparing contractor labels, MACE, majority voting, repeated labeling, and an annotator-classifier; (ii) multi-task training results; (iii) a synthetic bias analysis in which Wizard-Vicuna posts generated from persona profiles are used to measure model performance and fairness metrics across socio-demographic groups; and (iv) a comparison of classification and regression for the ordinal call-for-action label. The central conclusion is that the subjectivity of annotations and sociodemographic variations materially influence measured performance and fairness. The annotation-variation and dataset contributions are substantial, but the synthetic bias analysis has internal-validity problems that bear on the main fairness claims.","tokens_in":23616,"tokens_out":4416,"duration_ms":40833,"significance":"The dataset and annotation-variation experiments are valuable: COUNTER is a substantial multilingual resource with multiple annotations, inter-annotator agreement statistics, pseudonymization checks, and a transparent data statement. The label-aggregation comparison in Section 5.1 is carefully executed and gives a useful demonstration that gold-label choices change model evaluation. If the synthetic bias analysis were valid, the fairness findings would be a significant contribution. As written, however, the bias analysis confounds demographic attributes with the ideological content used to generate the test data and relies on a fairness metric that is not identifiable under group-dependent base rates, so its central fairness conclusions are not yet established. The regression-versus-classification analysis is a useful auxiliary result.","major_comments":[{"comment":"The main fairness conclusion is not identified because the synthetic generation protocol varies the tested attribute jointly with the textual topic and ideology. Table 10 lists \"Political view (e.g., Far-Right)\" and \"Religion/Culture\" (e.g., \"Islam (Jihadism)\") as profile variables that are \"crucial for producing radical content,\" so the generated posts for different political-view or religion groups are about different ideologies by construction. The Macro-F1 gaps in Figure 1 and the fairness gaps in Table 4 therefore conflate model bias with per-ideology classification difficulty. This confound is not removed by the acknowledged stereotypical generation limitation in Section 3.4; it is an internal-validity problem. To support the claim that socio-demographic attributes, rather than ideology or topic, drive the gaps, the authors need to control for ideology across groups (e.g., generate the same ideology/topic under different demographic profiles) or re-scope the claims to model sensitivity to generation prompts.","section":"Section 5.2, Appendix E.1 (Table 10)"},{"comment":"The demographic-parity calculation in Table 4 is not a valid bias diagnostic under the experimental setup. The authors treat class 0 as the favorable outcome and compute selection-rate differences across groups. If the base rate P(Y=0 | A=a) differs by group, even a perfectly calibrated Bayes-optimal classifier will exhibit nonzero demographic-parity difference, so the reported values cannot be read as evidence of model bias. The paper should report per-group base rates and either condition on the true label (as equalized odds does) or use a metric that accounts for label shift, and it should state that demographic parity is confounded with label distribution when base rates vary across groups.","section":"Section 5.2 and Appendix D"},{"comment":"The bias analysis rests on an unvalidated synthetic proxy: posts generated by Wizard-Vicuna-13B-Uncensored from hand-crafted persona profiles. The authors acknowledge in Section 3.4 that the generated posts \"sometimes rely heavily on stereotypical keywords\" and in the Limitations that synthetic data \"might not accurately represent the complexity and diversity of radical content in the actual world,\" yet the conclusion in Section 6 that \"sociodemographic variations influence data and model outcomes\" depends on this proxy. There is no external validation that the generated posts have a label distribution, error structure, or demographic signal resembling real online radical content. Without such validation, or a re-scoping of the conclusions to \"performance of models on LLM-generated persona-conditioned content,\" the fairness findings remain a property of the generation procedure as much as of the models.","section":"Section 3.4, Limitations"}],"minor_comments":[{"comment":"\"Pseudononymization\" is misspelled; it should be \"Pseudonymization.\"","section":"Section 3.3"},{"comment":"The labels are extremely small and some are truncated (e.g., \"T aoism\"), making it difficult to assess the claimed differences; a zoomable or table-based version with per-group standard deviations would be clearer.","section":"Figure 1 and Figures 8-9"},{"comment":"The human-label-variation analysis is reported only for French (Table 3), but Section 3.3 states that a large English sample was also double-annotated; the authors should explain why English is excluded from this analysis and whether the same patterns hold.","section":"Section 5.1"},{"comment":"No confidence intervals or significance tests are reported for the fairness differences, and some synthetic categories are very small (e.g., African American at 1% in Table 11), so the reported gaps may be unstable.","section":"Section 5.2, Table 4"},{"comment":"The regression models are compared using rounded predictions for Macro-F1, but the rounding threshold and the treatment of predictions outside the [0,4] range are not specified.","section":"Section 5.3, Table 5"},{"comment":"The full prompt template used for synthetic generation is not included; Section 3.4 refers to it, but the appendix only provides a screenshot and variable descriptions, which is insufficient for reproducibility.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The synthetic bias analysis is the main obstacle to publication as-is. The annotation-variation experiments and the dataset itself are worthwhile contributions, but the fairness conclusions in Section 5.2 are currently confounded by the generation protocol and by the choice of fairness metric. If the authors can control for ideology/topic in the synthetic generation or substantially re-scope their fairness claims, the paper should be publishable. If the confounds cannot be addressed within the current design, the fairness claims should be removed rather than qualified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: the dataset is the contribution; the bias analysis is not. I would send this to reviewers, but with a clear note that the fairness claims need reframing or re-analysis.\n\nWhat is new: COUNTER is a genuinely useful trilingual dataset—English, French, Arabic—with radicalization level, call-for-action, NER, annotator disagreement, guidelines, and pseudonymization. That fills a real gap. The annotation-variation experiments are the strongest part: three labels for French, aggregation via MACE, majority vote, repeated labeling, and an annotator-classifier, with results that honestly show how much the choice of gold standard changes measured performance. The regression-versus-classification comparison is a nice, modest addition; the adjacent-class error pattern is worth having.\n\nThe soft spot is Section 5.2. The synthetic profiles in Table 10 include political view and religion/culture as generation conditions. That means the demographic variable is not varied independently of the ideology and content of the generated post. The Macro-F1 gaps across far-right versus far-left and across religions are as easily read as per-class difficulty as model bias. On top of that, demographic parity is computed on data where the positive-class base rate varies across groups; even a perfectly calibrated model would show parity differences under that label shift. The authors are transparent that the synthetic data may be stereotyped and not real-world, and the Limitations section acknowledges this, but the conclusion still leans on the bias result. I would want the paper to either re-analyze with ideology fixed, or report per-attribute results conditional on political view and religion, and to state plainly that the fairness numbers are not identified.\n\nMinor: no code is shipped and the dataset is on request, which is reasonable for sensitive data but lowers reproducibility below what the phrase \"publicly available\" implies. The annotation section could also say more about the small number of annotators and the narrow demographic range.\n\nOverall, this is an honest paper with a good resource and known limitations acknowledged. The reader's CONDITIONAL verdict is right, and the stress-test note lands once you actually read the appendix. This is not a desk reject; it deserves referee time and likely major revision on the bias section.","headline":"The COUNTER dataset and annotation-aggregation experiments are the real contribution; the synthetic bias analysis is confounded and should be reframed as exploratory.","tokens_in":24102,"tokens_out":1831,"would_cite":true,"duration_ms":19033,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Radical-content detection is shaped by who labels and who tests the model.","keywords":["radical content detection","multilingual dataset","annotation disagreement","label aggregation","synthetic persona data","bias probing","fairness metrics","pseudonymization"],"falsifier":"Collect genuine extremist or radical posts whose authors' demographic attributes are known or reliably inferred, annotate them with the same guidelines, and measure the same Macro-F1 gaps across nationality, ethnicity, religion, and political view. If the real-data gaps do not reproduce the synthetic gaps — for example, if performance is roughly equal across groups once topic is controlled — the claimed sociodemographic influence on model outcomes would be an artifact of the generation process rather than a property of radical-content detection.","tokens_in":23203,"feed_emoji":"⚖️","tokens_out":6945,"duration_ms":63790,"temperature":0.7,"pith_summary":"Detecting radical content online is not a stable labeling task, the paper argues: different human annotators disagree enough that the same model can look noticeably better or worse purely because of how labels were aggregated and which test slices were used. To make that case, the paper presents COUNTER, a pseudo-anonymized dataset of English, French, and Arabic posts from platforms such as Twitter, Telegram, and 4chan, annotated for radicalization level, call for action, and named entities. The authors show that contractor annotations, majority vote, and MACE-derived labels disagree substantially, and that training on one gold standard and testing on another drops French Macro-F1 by up to about 15 points. They then generate synthetic posts from hand-crafted socio-demographic profiles and report systematic performance gaps across nationality, ethnicity, political view, religion, and place of living, with the largest fairness differences for place of living, ethnicity, and nationality. If these findings hold, evaluation and fairness audits of radical-content detectors must be reported as conditional on annotation and demographic choices.","feed_headline":"Who labels and who tests shifts radical-content scores","feed_subtitle":"A multilingual dataset with multiple annotations shows aggregation and demographic profiles change measured model fairness.","key_machinery":"The load-bearing machinery is the COUNTER resource itself plus two controlled perturbations of it: (1) the multiple annotation views — prescriptive contractor labels, descriptive annotator labels, and MACE and majority aggregations — used to show that label construction changes what a model learns and how it scores; and (2) the synthetic persona-prompting setup, in which Wizard-Vicuna-13B-Uncensored generates posts from hand-written profiles varying gender, age, ethnicity, nationality, political view, religion, job, language register, and place of living, used to probe group-level performance with demographic parity and equalized odds.","core_discovery":"The central claim is that the label a radical-content detector learns is co-produced by the annotation protocol and the annotators' socio-demographic positions, and that this propagates into model evaluation. The paper demonstrates this with an English, French, and Arabic corpus whose Call for Action and Radicalization Level labels come from a prescriptive contractor pass plus a descriptive double-annotation pass; inter-annotator agreement ranges from slight (English Call for Action, Fleiss' kappa 0.13) to moderate (French Radicalization Level, 0.50). Models trained and evaluated on different aggregation views of the same posts show large gaps, and in the synthetic persona experiment all tested models show uneven Macro-F1 across demographic attribute values, with XLM-T showing both the best overall performance and the largest disparities. The authors conclude that subjectivity in human annotations and sociodemographic variations measurably influence data and model outcomes, so robust radical-content detection requires reporting and accounting for these choices.","pith_inferences":["My reading: the synthetic-data evidence should be treated as an upper bound on sociodemographic bias, because the generator's own stereotypes may produce the attribute-correlated language; validating against real posts with known author demographics is the decisive test of the fairness claim.","My reading: the kappa asymmetry between English (0.13–0.26) and French (0.43–0.50) may reflect differences in the annotators' cultural proximity to the two corpora; a controlled study varying annotator cultural background would separate task ambiguity from annotator subjectivity.","My reading: for deployment, these results argue for reporting a performance range over aggregation methods and demographic slices, and for training on disagreement-aware objectives rather than a single aggregated gold label.","My reading: the paper's release of multi-view labels invites predicting full label distributions, which would make moderation decisions more transparent about uncertainty than thresholding a single class."],"forward_implications":["Reported performance on radical-content detection is partly an artifact of label aggregation: models trained and tested on the same aggregation look best, and switching the test gold standard can drop French Macro-F1 by roughly 15 points.","Multi-task auxiliary signals such as radicalization level, ideology prediction, and named-entity recognition do not reliably improve Call for Action detection, while separate per-language classifiers consistently beat a shared classifier.","A regression formulation on the ordinal Call for Action scale avoids distant misclassifications but earns lower Macro-F1, making the classification-versus-regression choice a trade-off between discrete accuracy and error severity.","Fairness audits on synthetic demographic profiles reveal large demographic-parity and equalized-odds differences for place of living, ethnicity, and nationality, with English nationality reaching an equalized-odds difference of 0.66 for XLM-T.","Synthetic bias patterns are model-dependent: XLM-T attains higher overall scores but larger demographic disparities than XLM-R and mBERT, suggesting that social-media pretraining amplifies attribute-correlated errors."],"supporting_citations":[{"why":"Supplies the prescriptive versus descriptive annotation paradigm distinction used to structure the two annotation passes.","marker":"Rottger et al., 2022"},{"why":"Provides MACE, the Bayesian annotation aggregation method used as one of the gold-label constructions.","marker":"Hovy et al., 2013"},{"why":"Defines human label variation, the framing used to interpret annotator disagreement in the paper.","marker":"Plank, 2022"},{"why":"Shows that annotator beliefs and identities bias toxic-language labels, supporting the socio-demographic annotation analysis.","marker":"Sap et al., 2022"},{"why":"Demonstrates sociodemographic factors such as age affect subjectivity in sentiment annotation, grounding the socio-demographic hypothesis.","marker":"Diaz et al., 2018"},{"why":"Provides the LLM persona simulation method used to generate synthetic profiles for the bias analysis.","marker":"Aher et al., 2023"},{"why":"Shows large language models can be used to represent subjective socio-demographic opinions, the antecedent for the synthetic profile approach.","marker":"Durmus et al., 2024"},{"why":"Shows annotator identity influences hate-speech annotation, supporting the claim that subjectivity affects radical-content labeling.","marker":"Waseem, 2016"}],"fun_headline_variants":["Annotation choices and annotator bias skew radical-content scores","Who annotates radical posts changes what models learn","Annotator demographics tilt radical-detection fairness","Synthetic personas expose demographic skew in radical-content models","Radical-content labels inherit annotator bias"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The fairness conclusion rests on treating posts that an uncensored language model wrote from hand-crafted demographic profiles as representative of how real online radical content varies across demographic groups.","fun_headline_variants_meta":{"raw":{"variants":["Annotation choices and annotator bias skew radical-content scores","Who annotates radical posts changes what models learn","Annotator demographics tilt radical-detection fairness","Synthetic personas expose demographic skew in radical-content models","Radical-content labels inherit annotator bias"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000574,"raw_usage":{"total_tokens":2677,"prompt_tokens":876,"completion_tokens":1801,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":492,"completion_tokens_details":{"reasoning_tokens":1729}},"tokens_in":492,"tokens_out":1801,"duration_ms":14676,"temperature":1.0,"reasoning_tokens":1729,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:36:38.618992+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Collect genuine extremist or radical posts whose authors' demographic attributes are known or reliably inferred, annotate them with the same guidelines, and measure the same Macro-F1 gaps across nationality, ethnicity, religion, and political view. If the real-data gaps do not reproduce the synthetic gaps — for example, if performance is roughly equal across groups once topic is controlled — the claimed sociodemographic influence on model outcomes would be an artifact of the generation process rather than a property of radical-content detection.","supporting_citations":[],"review_version":1}