{"id":"d24ae855-d6d4-4b14-ab2b-aee2aa0c5665","arxiv_id":"2506.05635","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"LLMs struggle with extremist in-group jargon, but prompting with example posts and domain-adapting encoders substantially improves detection and decoding.","lead":"Extremist groups use coded language that general-purpose LLMs often fail to understand. This paper builds a large dataset from two extremist forums and shows that LLMs can be measurably improved with specialized prompts and domain adaptation, which could inform automated content moderation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The zero-shot tasks rely on single-annotator labels with explicit inclusion bias, and the same annotation instructions are reused as model prompts, so the reported F1 values may not cleanly measure LLM ability on cryptolects.","rationale":"The reader's CONDITIONAL verdict is appropriate, and this stress-test does not move it. The strongest pillar of the paper is Section 7: domain adaptation of xlmt on the IYKYK corpus yields consistent macro-F1 gains over 5 runs with significance tests on the existing Counter benchmark, which does not depend on the authors' own gold labels and is credible. The weakest pillar is the zero-shot evaluation in Section 6. The test sets for Tasks 1-3 are built from LISTN-C candidate lists and annotated by a single expert per platform under instructions that explicitly bias toward positive labels. Those same instructions, including example words and the inclusion policy, are then reused as the model prompt in the 'instructions' framing, creating a circularity that the reader's weakest_assumption did not fully emphasize. A model that simply over-predicts in-group status in line with the instructions can achieve inflated F1, particularly on the Incels test set with 46% positive labels. The paper's own Limitations section acknowledges the single-annotator issue but does not quantify its effect on the reported numbers. A re-annotation study with neutral instructions and multiple experts would directly settle whether the headline zero-shot F1 values are robust. If they are not, the claim that general-purpose LLMs cannot consistently detect or decode extremist language would need to be softened, although the domain-adaptation conclusion would likely survive. The concern is methodological, not a reflection on author intent; the paper is transparent about many of its limitations, and the domain-adaptation experiments are well executed.","tokens_in":16830,"tokens_out":6828,"duration_ms":78481,"concrete_test":"Re-annotate a stratified random sample of 300 candidate words per platform (or the full 3050 if feasible) with two additional expert annotators using a neutral instruction set that removes the 'opt to include' sentence and does not disclose the authors' intended positives. Measure inter-annotator agreement (e.g., Cohen's kappa) and recompute Task 1 F1 for the best model (Llama-3.3 70B) under the definition and instructions framings using majority-vote and full-agreement gold sets. If F1 shifts by more than about 5 points or agreement is below 0.7, the zero-shot claims are not robust to annotation bias.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The claim that general-purpose LLMs cannot consistently detect or decode extremist cryptolects rests primarily on the zero-shot results in Sections 6.1-6.3. The gold labels for these tasks are produced by one expert per platform on candidate words generated by the authors' LISTN-C method, and the annotation instructions (App. B.1) explicitly tell annotators: 'in the case of borderline words, you should opt to include them.' This positive-label bias is especially consequential for Incels, where 1401 of 3050 candidates (46%) are labeled positive. Critically, the 'instructions' prompt framing in Task 1 and Task 3 is constructed from the same human annotation instructions, including the positive/negative example lists and the inclusion policy (App. C.1.2, C.3.2). The model is therefore scored against labels generated under the same decision rule it is explicitly prompted to follow, creating a circularity: a model that over-predicts in-group status in line with the instructions can achieve spuriously high F1. The paper acknowledges the single-annotator limitation but treats the labels as ground truth throughout the evaluation. If the labels are noisy or biased, the zero-shot F1 values are not a valid measure of whether LLMs 'cannot consistently detect or decode' extremist language, and the central claim loses one of its two main pillars. The supervised domain-adaptation results in Section 7 are less affected because they use the existing Counter dataset and show consistent, statistically significant gains, so the overall hypothesis may still hold, but the zero-shot evidence is not secure.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces IYKYK, a new dataset of 19.4 million public posts from the extremist forums Stormfront and Incels, and uses it to test the central hypothesis that general-purpose language technologies require specialised techniques when applied to extremist cryptolects. The authors construct gold-standard lexicons of in-group language using the LISTN-C candidate-generation method followed by expert annotation, and then evaluate seven instruction-tuned LLMs on three zero-shot tasks (classification, retrieval, and decoding of in-group terms) under four prompt framings. They also evaluate domain adaptation of the encoder model xlmt on radical-content prediction tasks from the Counter dataset and on hate-speech detection, reporting significant gains for English radical-content tasks and small but consistent gains for hate-speech tasks. The paper releases datasets and models to verified researchers and includes extensive ethical and limitation discussion.","tokens_in":17065,"tokens_out":5011,"duration_ms":49517,"significance":"If the reported results are valid, this is a substantial contribution to a relatively under-studied area of NLP: it provides a large-scale, long-range dataset from two extremist communities, a transparent lexicon-induction and annotation pipeline, and evidence that domain adaptation on in-domain data can improve radical-content and hate-speech classification. The supervised experiments in Section 7 are well-designed, use a standard toolkit, and include significance tests; the hate-speech transfer results, though small, are a useful sanity check. The paper also ships reproducibility details for hyperparameters and prompts. However, the zero-shot evidence base has several validity concerns that affect the strength of the central claim that LLMs cannot consistently detect or decode extremist language, so the contribution is currently somewhat uneven.","major_comments":[{"comment":"Task 2, labeled 'Retrieval,' is not evaluated as retrieval: scoring only checks whether the single target word is included or excluded in the model's output, ignoring all other tokens the model might retrieve. The paper itself states in §6.2 that 'the same objective is evaluated in both' Task 1 and Task 2, and reports label coincidence of 71% (Incels) and 79% (Stormfront). As a result, the 'six tasks' framing overstates the diversity of the evaluation, and the claim that this task is 'more demanding than Task 1' is not supported by the metric. Please either re-frame Task 2 as a contextualized target-word classification variant or implement a true token-level retrieval metric that accounts for all retrieved and gold tokens.","section":"§4.1, §6.2"},{"comment":"The decoding task (Task 3) is evaluated with only one model, Llama-3.3-70B, while the abstract and introduction describe 'eight models across six tasks.' This means that the conclusion that 'general purpose LLMs cannot consistently detect or decode extremist language' is supported by a single model for the decoding component. Please either expand the decoding evaluation to more models or revise the abstract and summary statements to accurately scope the decoding results as a case study with one model.","section":"§4.1, §6.3, Abstract"},{"comment":"The gold labels for Tasks 1–3 are produced by a single expert per platform on candidate words from the authors' own LISTN-C method, and the annotation instructions include an explicit inclusion bias ('in the case of borderline words, you should opt to include them'). The 'instructions' prompt framing then provides the model with the same positive/negative example lists and inclusion policy that the annotator used. This creates a potential alignment between the annotation rule and the model prompt, so the reported zero-shot F1 values (e.g., 80.0% for Incels) may not be a neutral measure of LLM ability to recognize cryptolects. The Limitations section acknowledges the single-annotator risk but does not discuss the reuse of annotation instructions as prompts. Please add an explicit analysis of this possible circularity, for example by ablating the instruction prompt or by reporting annotator-model agreement on the borderline cases.","section":"§5, App. B, App. C"},{"comment":"The paper claims 'state-of-the-art results on the Call for Action task' without comparing against any published result other than the xlmt baseline from Riabi et al. (2025). If the Counter dataset has been used in other supervised systems, those should be cited and compared; otherwise, the claim should be qualified as 'best under this experimental configuration.'","section":"§7, Table 3"}],"minor_comments":[{"comment":"The phrase 'eight models across six tasks' is misleading because Task 3 uses only one model; please specify the exact number of models per task in the abstract or in the task descriptions.","section":"Abstract and §4.1"},{"comment":"There is a duplicated word in 'expressing expressing a radical perspective'; please remove the duplicate.","section":"§2.2"},{"comment":"The text reads 'usingxmlt'; there is a missing space between 'using' and 'xlmt'.","section":"§4.2"},{"comment":"The sentence 'discussed shown in Table 2' is grammatically awkward; consider rephrasing to 'as discussed and shown in Table 2.'","section":"§6.1"},{"comment":"The table caption reports one-sided t-tests without correction for multiple comparisons; please add a note on whether the significance thresholds remain after any multiple-testing correction, or justify the use of uncorrected tests.","section":"§7, Table 3"},{"comment":"In Section 7 the text cites 'the model of Riabi et al. (2024)' but the Counter dataset reference appears as Riabi et al. (2025) in the bibliography; please check the intended year and citation.","section":"References"},{"comment":"The annotation instructions for Incels and Stormfront are nearly identical except for the platform name and ideology; a note that they were intentionally kept parallel would help readers understand the design choice.","section":"App. B"},{"comment":"The caption says the posts are 'in gray,' but the figure uses colored text for the translations; please ensure the visual distinction is clear in black-and-white printing.","section":"Figure 1"},{"comment":"The statement that researcher risk management plans are in place is reassuring, but a one-sentence description of what those plans include (e.g., counseling, monitoring) would be more informative.","section":"Ethics"},{"comment":"In the list of positive examples, 'linguistic exports' cites 'incel, mogging' as exports from the incel community, while 'linguistic imports' also lists 'mogging' as an import from elsewhere; clarify the intended status of 'mogging.'","section":"App. B.1.1"}],"recommendation":"major_revision","confidential_remarks":"This paper addresses an important and sensitive topic, and the dataset and domain-adaptation results are potentially valuable. The main risk is that the zero-shot evaluation, which underpins the headline claim about LLM limitations, has methodological gaps that a careful reviewer or reader could exploit: the retrieval task is effectively classification, the decoding task uses a single model, and the gold labels are generated with the same instructions that are later given to the models. I believe these issues are fixable within a revision, but they need to be addressed head-on rather than hidden in the limitations section. I would also encourage the authors to make the dataset access process clearer, as the 'upon request' policy may hinder reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on IYKYK. The real contribution is the corpus and the domain-adaptation results. Nineteen million posts from two extremist forums, expert-validated lexicons, and a careful supervised evaluation showing that continued MLM adaptation on this data improves radical content detection on the Counter benchmark, with significance tests over five runs. The cross-ideology transfer finding is genuinely interesting. That part of the paper is in good shape.\n\nThe zero-shot evaluation is where I'd push back. The gold standard for Tasks 1-3 is built by one expert per platform using annotation instructions that explicitly say 'in the case of borderline words, you should opt to include them.' Those same instructions, including that sentence, are then used to build the model prompts. The model is scored against labels produced under the same decision rule it is told to follow. That is a real circularity: both the gold and the model are biased toward inclusion, so the reported F1 values can't cleanly tell us whether LLMs 'cannot consistently detect or decode extremist language.' This doesn't make the zero-shot results worthless, but it does mean the headline numbers are not secure.\n\nA couple of smaller issues, in proportion. Task 2 is called retrieval, but the evaluation only scores whether the target word appears in the model's output, so it is effectively the same classification objective as Task 1 in a different wrapper. The paper halfway admits this, but presenting it as a separate task is generous. Task 3 uses only one model, Llama-3.3-70B, so the 'eight models across six tasks' framing overstates the breadth of the decoding results. The data and models are gated behind a memorandum of understanding, which is defensible for ethical reasons but limits independent verification.\n\nMy overall read: the central hypothesis may well be right, and the supervised evidence supports it. But the zero-shot evidence is weaker than the abstract implies. The paper needs careful revision: re-annotate at least a sample without the inclusion instruction, report sensitivity to that bias, and re-frame Tasks 2 and 3 honestly. This is a paper for anyone working on online extremism, hate speech, or evaluation of LLMs on coded language. It deserves a proper peer review, not a desk reject, but it should go back for major revision. I'd cite it for the corpus and the domain-adaptation results, and I'd bring it to a reading group to discuss the annotation/prompt circularity.","headline":"Worth reading for the 19.4M-post corpus and the solid domain-adaptation results; the zero-shot F1 numbers are provisional because the gold labels and model prompts share the same inclusion-biased instruction.","tokens_in":17714,"tokens_out":3079,"would_cite":true,"duration_ms":28722,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that general-purpose LLMs cannot consistently detect or decode the coded language of extremist forums, and that prompting context and domain adaptation substantially improve both detection and decoding.","keywords":["cryptolects","in-group language","online extremism","large language models","domain adaptation","hate speech detection","prompt engineering","Stormfront"],"falsifier":"Re-annotate the Task 1 and Task 3 test sets with multiple independent experts per platform, then recompute zero-shot F1 and definition-correctness rates against majority-vote labels; if inter-annotator agreement is low or reported scores drop sharply, the paper's central comparison does not survive. A complementary check is to run the same classifiers on a random sample of frequent forum words that LISTN-C did not propose: if many of those are in-group terms, the gold standard misses part of the vocabulary and the reported miss rates understate the problem.","tokens_in":16595,"feed_emoji":"🔍","tokens_out":7799,"duration_ms":70072,"temperature":0.7,"pith_summary":"The paper sets out to show that off-the-shelf language models are not equipped to detect or decode the deliberately coded vocabularies ('cryptolects') of extremist online communities, and that performance on these tasks can be substantially recovered by adding context to prompts and by domain-adapting models on forum data. To test this, it builds a corpus of 19.4 million posts from Stormfront and Incels, creates expert-validated lexicons of in-group terms, and evaluates eight models across six tasks. The headline results are that the best zero-shot model reaches F1 of 64.4% on Stormfront and 80.0% on Incels for word classification, and that providing ten example posts lifts correct definition generation for Incels from 44.7% to 90.3%. Domain adaptation of the multilingual encoder xlmt improves radical content detection, reaching state-of-the-art macro-F1 on the English Call for Action task and improving hate speech detection on the matching ideology. If correct, these findings imply that moderation systems cannot simply point a general LLM at extremist forums; they need platform-aware prompting or specialised adaptation, and the released datasets give the field a place to build those tools.","feed_headline":"Ten example posts lift LLM decoding of incel slang to 90%","feed_subtitle":"Study of 19.4M forum posts shows off-the-shelf models miss coded in-group language; prompting and domain adaptation close much of the gap.","key_machinery":"Three components carry the argument. (1) The IYKYK corpus: 19.4M public posts from Stormfront and Incels, with metadata, used both as adaptation data and as the source of test cases. (2) A lexicon pipeline that combines LISTN-C, a socio-temporal method that jointly embeds words and users and scores words by distance to community clusters, with paid expert annotation to produce validated positive/negative lexicons for each platform. (3) A two-track evaluation design: instruction-tuned LLMs (Gemma, Llama, Mixtral, Qwen) are probed with four prompt framings (definition only, full instructions, one example post, ten example posts); the encoder model xlmt is continued-pretrained on the corpus and then fine-tuned on the Counter radical-content tasks and a hate speech task. The prompt framing and the continued-pretraining step are the levers that, per the paper, close most of the gap.","core_discovery":"The paper's central claim is that generalised language technologies—LLMs trained on broadly filtered internet text—are not directly reliable on extremist cryptolects, and that specialised techniques (prompt context and continued pretraining on platform data) are required to make them useful. On zero-shot classification of in-group words, the best model (Llama-3.3 70B) achieves F1 of 64.4% on Stormfront and 80.0% on Incels; adding the full annotation instructions raises F1 by about 10 percentage points, and including ten example posts raises the share of correct definitions for Incels from 44.7% to 90.3%. Domain-adapting the xlmt encoder on 19.4M forum posts significantly improves macro-F1 on the Counter tasks, reaching state-of-the-art results on English Call for Action prediction and establishing a first baseline for Radicalization Level prediction. The same adaptation improves hate speech detection when the adaptation platform matches the hate speech target (Stormfront for anti-immigrant, Incels for misogynist).","pith_inferences":["The paper's explanation—that toxic-content filtering during pretraining removes exactly the data these tasks need—implies a structural tension between safe generation and monitoring capability; a plausible design consequence is separate 'monitoring' models that keep toxic text for representation learning while general chatbots remain filtered.","The Incels results are inflated by productive morphemes such as '-cel', '-maxx', and '-oid'; a natural stress test is whether adversarial mutations of those suffixes (e.g., novel spellings or unusual compounds) cause the same models to fail, since cryptolects evolve by exactly this kind of recombination.","The pattern that example posts outperform explicit definitions for decoding suggests a general recipe for decoding any community-specific jargon, not just extremist groups: give the model distributional evidence (examples) rather than only a rule, because examples carry usage frequencies and collocational cues that definitions omit.","The combined-forums benefit suggests pooling data across many radical communities could cover rarer terms, but the paper does not test how long adapted models remain useful as these languages evolve; that temporal robustness is an open question."],"forward_implications":["Unadapted general LLMs are not safe arbiters of extremist content: the best zero-shot in-group word classifier still reaches only 64.4% F1 on Stormfront, meaning many coded terms slip through or are mislabelled in moderation pipelines.","Adding a few in-context example posts is an inexpensive remedy: correct decoding for Incels rises from 44.7% to 90.3% with ten examples, so API-based systems can be steered without retraining.","Continued pretraining on extremist forum text transfers beyond the original platforms: it improves radical-content detection even when the adaptation data comes from a different ideology, and it improves hate speech detection when the platform matches the target bias.","The new 19.4M-post corpus and expert-validated lexicons provide a shared testbed for moderation and intervention research, and the first baseline for Radicalization Level prediction gives future work a number to beat."],"supporting_citations":[{"why":"Supplies the LISTN-C method that generates the candidate in-group words for the gold lexicons.","marker":"de Kock, 2024a"},{"why":"Provides the Counter dataset, task definitions, and the xlmt baseline scores that domain adaptation must beat.","marker":"Riabi et al., 2025"},{"why":"Frames the dogwhistle retrieval and definition tasks that Tasks 2-3 extend with confirmed negatives and larger scale.","marker":"Mendelsohn et al., 2023"},{"why":"Provides the hate speech corpora and split used to test cross-domain transfer of adapted models.","marker":"Montariol et al., 2022"},{"why":"Introduces xlmt, the multilingual social-media encoder that the paper domain-adapts.","marker":"Barbieri et al., 2022"},{"why":"Supplies prior lexica whose frequency distributions help construct Task 3 decoding test cases.","marker":"de Kock and Hovy, 2024"}],"fun_headline_variants":["LLMs can't decode extremist slang without domain adaptation","19.4M extremist posts aid LLM understanding of cryptolects","Off-the-shelf LLMs miss extremist code; tuning recovers","Adapting LLMs to 19.4M posts helps decode extremist cryptolects","Ten examples raise LLM definition accuracy from 45% to 90%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything about model performance on in-group words rests on gold labels produced by a single expert annotator per platform, applied to candidate words selected by the authors' LISTN-C method, and the annotation instructions tell annotators to include borderline words, which pushes the positive class larger and can inflate LLM scores.","fun_headline_variants_meta":{"raw":{"variants":["LLMs can't decode extremist slang without domain adaptation","19.4M extremist posts aid LLM understanding of cryptolects","Off-the-shelf LLMs miss extremist code; tuning recovers","Adapting LLMs to 19.4M posts helps decode extremist cryptolects","Ten examples raise LLM definition accuracy from 45% to 90%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000995,"raw_usage":{"total_tokens":4182,"prompt_tokens":877,"completion_tokens":3305,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":3210}},"tokens_in":493,"tokens_out":3305,"duration_ms":25568,"temperature":1.0,"reasoning_tokens":3210,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T10:13:07.402011+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-annotate the Task 1 and Task 3 test sets with multiple independent experts per platform, then recompute zero-shot F1 and definition-correctness rates against majority-vote labels; if inter-annotator agreement is low or reported scores drop sharply, the paper's central comparison does not survive. A complementary check is to run the same classifiers on a random sample of frequent forum words that LISTN-C did not propose: if many of those are in-group terms, the gold standard misses part of the vocabulary and the reported miss rates understate the problem.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Counter dataset, task definitions, and the xlmt baseline scores that domain adaptation must beat."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Frames the dogwhistle retrieval and definition tasks that Tasks 2-3 extend with confirmed negatives and larger scale."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the hate speech corpora and split used to test cross-domain transfer of adapted models."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces xlmt, the multilingual social-media encoder that the paper domain-adapts."}],"review_version":1}