Pith. sign in

REVIEW 4 major objections 5 minor 45 references

Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Toxicity classifiers can recover large chunks of accuracy against adversarial attack by zeroing the outputs of a few attention heads that are distinct from the heads required for clean performance.

desk verdict The suppression claim is an in-sample fit; the demographic head maps are the parts worth keeping. read the letter →

arxiv 2509.12672 v2 pith:JILGHLAV submitted 2025-09-16 cs.CL

Shaz Furniturewala , Arkaitz Zubiaga This is my paper · ORCID
classification cs.CL
keywords toxiccontentmoderationadversarialattacksmechanisticinterpretabilityattentionheadsactivationpatchingBERTRoBERTademographicfairness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fine-tuned BERT and RoBERTa toxicity classifiers can be made substantially more robust to adversarial text perturbations by mechanically identifying and then zeroing out a small set of attention heads. The paper shows that the heads that are vulnerable to attack are not the same heads that are crucial for correct classification on clean input, so suppressing the vulnerable ones improves accuracy on attacked samples by 8 to 9 percentage points while barely hurting clean accuracy. Vulnerability is uneven: on the machine-generated ToxiGen dataset, adversarial attack collapses accuracy from 84.5% to 2.8%, and the best per-demographic-group head suppressions recover accuracy by widely differing amounts (51.35% for women versus 29.83% for muslim). The authors interpret this as evidence that bias is structurally encoded in specific attention heads, and that circuit-level intervention can proactively harden moderation models beyond reactive adversarial training.

What carries the argument

The central mechanism is activation patching by zero-ablation of individual attention heads: each head's output is set to zero, and the change in accuracy or loss is measured on clean and adversarially perturbed inputs. This intervention is used to partition heads into crucial (clean-accuracy important) and vulnerable (attack-loss reducing), with adversarial inputs generated by PGD BERT Attack, a white-box embedding-space attack based on projected gradient descent. The demographic analysis repeats the patching per group to trace which heads carry vulnerability for which minority group.

What would settle it

Use a different adversarial attack (e.g., TextFooler or a genetic attack) to generate a held-out attack set, then apply the heads selected by PGD BERT Attack on the original dataset and measure accuracy gain. If the gains do not transfer, the vulnerable-head selection is attack-specific rather than a general property of the model. Alternatively, replace zero-ablation with mean-activation patching; if a different head set emerges, the intervention is not robustly identifying the same circuit.

Watch

Extended reading notes

Core claim

Using activation patching that zeros each attention head in turn, the authors label heads as crucial if ablation degrades accuracy on clean input and as vulnerable if ablation reduces loss on adversarial input produced by PGD BERT Attack. On the Jigsaw and ToxiGen datasets with fine-tuned BERT and a RoBERTa classifier, they observe that the crucial and vulnerable sets are disjoint. For example, head L0H4 is crucial (zeroing it costs 24% clean accuracy), while heads L0H2 and L1H3 are not needed for clean inputs but suppressing them raises accuracy on adversarial inputs by 8 to 9 percentage points. Head-level ablation also reveals demographic specificity: certain heads are vulnerable only for

Load-bearing premise

The method assumes that zeroing an attention head is a faithful causal probe—that a head whose ablation lowers the adversarial loss is genuinely the mechanism of vulnerability, rather than an arbitrary distribution shift that happens to help on the attacked test set, and that gains are not just in-sample selection on that same set.

Editorial extensions

If this is right

  • Suppressing a single vulnerable head can recover 8 to 9 percentage points of accuracy on attacked inputs with only small loss on clean examples, so circuit-level intervention is a viable, cheap hardening step.
  • Demographic-specific vulnerable heads mean robustness interventions should be group-aware; a single global suppression may not equalize error rates across groups.
  • The disjointness of crucial and vulnerable heads implies that targeted ablation can be applied without destroying the classifier's core toxic/benign decision machinery.
  • Monitoring activations of known vulnerable heads during inference could flag adversarial inputs in real time, as the authors propose as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the zero-ablation gains generalize beyond the PGD BERT Attack distribution used for head selection, head-suppression could serve as a defense-in-depth layer on top of existing filters; if not, the method is attack-specific and needs re-selection per attack family.
  • The per-group variation suggests that measuring aggregate adversarial accuracy can hide large demographic disparities; a natural testable extension is to report worst-group robust accuracy as a standard metric.
  • Because the paper only tests single-head ablations, a direct next step is testing whether jointly suppressing multi-head circuits yields larger or more transferable gains than the best single head.
  • The finding that layer-9 heads are vulnerable only for specific demographics hints at token-level encoding of group identity; probing those heads' attention patterns could reveal what lexical features the attack exploits.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a mechanistic-interpretability pipeline for hardening toxicity classifiers against adversarial attacks. The authors fine-tune BERT on Jigsaw and ToxiGen, use PGD BERT-Attack to generate adversarial examples, zero-ablate individual attention heads to identify 'crucial' heads (clean-input accuracy drop) and 'vulnerable' heads (loss reduction on adversarial input), and then report that suppressing vulnerable heads improves accuracy on adversarial inputs by 8–9% overall and by 29–51% for specific demographic groups. They also report that crucial and vulnerable heads are disjoint and that vulnerability patterns differ across demographic groups. The stated contribution is a proactive, interpretable defense for content moderation.

Significance. If the central claim were supported by out-of-sample evaluation, this would be a worthwhile contribution: it connects mechanistic interpretability to adversarial robustness and fairness in a practical moderation setting, and the demographic disaggregation is a useful descriptive analysis. The paper is honest about several limitations (runtime, multi-head circuits, multilingual scope), and the proposed intervention is concrete and testable. However, the headline improvements are computed on the same examples used to select the heads, so the current evidence is in-sample; the causal-circuit interpretation also lacks controls against random ablation. The contribution is therefore not yet established at the level claimed.

major comments (4)
  1. [Methodology Step 3; Results; Figure 2b] The central claim that suppressing vulnerable heads improves adversarial accuracy is tested on the same adversarial set used to define 'vulnerable.' In Step 3, heads whose ablation reduces loss on the adversarial test set are labeled vulnerable; the Results then report accuracy after suppressing those heads on that same set. This is an in-sample selection procedure: any head that happens to reduce loss on the specific attacked examples is rewarded, and the reported 8–9% improvement is not evidence of generalization. The paper provides no held-out adversarial split, no evaluation on a different attack family, and no comparison against randomly selected heads. This must be fixed before the claim can be assessed.
  2. [Results; Figure 3] The per-group improvements are obtained by selecting the 'best ablated head' for each demographic group on the adversarial test data for that group. Selecting the maximum over heads on the test set makes the reported 29.83–51.35% improvements optimistic order statistics; a null model that picks the best of several random ablations would likely show positive gains. Please report a pre-specified head selection rule, hold out the adversarial examples used for selection, and compare to random ablation or leave-one-out controls.
  3. [Datasets and Metrics; Experimental Setup] The Jigsaw subset is described as '50k samples ... split between 46k toxic and 4k non-toxic,' which is inconsistent with the original Jigsaw distribution and with the reported 92.86% accuracy (a 92% toxic base rate would make trivial accuracy 92%). This is likely a reversed split, but the text must be corrected and the class balance of every evaluation set reported. Similarly, after filtering successful perturbations by cosine similarity >0.95, only 54% (Jigsaw) and 19% (ToxiGen) of successful adversarial samples remain; all reported accuracies are on these filtered subsets, and the class composition of those subsets is not given.
  4. [Results; 'Classifiers have disjoint crucial and vulnerable attention heads'] The disjointness claim is not operationalized. 'Crucial' and 'vulnerable' are not defined by thresholds; 'do not significantly contribute' is not quantified; and no statistical test is provided for the claim that the sets are distinct. More fundamentally, zero-ablation is treated as revealing the causal role of a head, but an alternative explanation is that zeroing a head acts as a generic regularizer that helps the attacked examples. The paper should include controls (e.g., random heads, heads selected on clean input, different perturbation budgets) before interpreting the improved accuracy as evidence for a distinct 'adversarial circuit.'
minor comments (5)
  1. [Results; notation] The notation L0H4, L0H2, L1H3 is introduced without explanation. A brief description of the layer/head indexing and the total number of heads would help the reader.
  2. [Figures 2 and 4] The heatmaps lack colorbars and axis labels. The text refers to 'warmer/cooler' colors, but the scale is not visible, making quantitative interpretation difficult.
  3. [References] The reference list contains a probable duplicate: Wu et al. 2025a and 2025b have identical titles and venues; one appears to be a duplicate of the other.
  4. [Experimental Setup; RoBERTa] The RoBERTa classifier from Logacheva et al. (2022) is used only for an accuracy-degradation measurement; no head-level analysis is reported for RoBERTa. The title/abstract claim about 'toxicity classifiers' should be narrowed to BERT unless RoBERTa results are added.
  5. [General] Minor typographical and capitalization inconsistencies: 'Toxigen' vs 'ToxiGen', 'Bert' vs 'BERT', and accent artifacts in 'Garc´ıa-Carrasco' should be cleaned up.

Circularity Check

1 steps flagged · score 7.0 of 10

Vulnerable heads are selected on the adversarial test set and then evaluated on that same set, making the reported suppression gains in-sample fits rather than out-of-sample predictions.

  1. fitted input called prediction [Methodology Step 3 / Results, Figure 2b and Figure 3]
    "In this adversarial input, heads whose ablation reduces loss are considered vulnerable. [...] Suppressing these heads, as can be seen in Figure 2b, led to an 8–9% increase in accuracy on adversarial inputs, with a comparatively small loss in accuracy on clean examples. [...] From Figure 3 it is evident that zeroing the most adversarially sensitive head per demographic group showed wide variation in accuracy improvement: 51.35% for "women", but only 29.83% for "muslim" on the Toxigen dataset."

    The vulnerable heads are defined by the effect of ablation on the adversarial test set itself: Step 3 labels a head 'vulnerable' if zeroing it reduces loss on that set. The Results then report accuracy gains after suppressing exactly those selected heads on that same adversarial set. Figure 3 is even more direct: it picks the 'most adversarially sensitive head' per demographic group and reports that group's accuracy improvement, so the improvement is the selection statistic. No held-out adversarial split, no alternative attack family, and no random-head baseline is reported. Thus the 8–9% and 29–51% gains are in-sample fitted values, not validated evidence for a generalizable 'adversarial circuit' defense.

full rationale

The paper's central defense claim—that suppressing vulnerable heads improves accuracy on adversarial inputs—is compromised by in-sample selection. The heads are chosen because their ablation reduces loss on the adversarial test set, and the reported accuracy improvement is measured on that same set; the per-demographic improvements are even more explicitly the maximum over ablated heads on the same test data. This is a fitted-input-called-prediction circularity rather than a pure definitional tautology, since loss reduction and accuracy are distinct but highly related metrics. The demographic head maps in Figure 4 are descriptive and not circular, and the critical/vulnerable head distinction is evaluated on separate input types, so the paper is not entirely circular. However, the headline robustness improvement lacks any out-of-sample validation, leaving the causal 'adversarial pathway' interpretation unsupported. No load-bearing self-citation issue was found; the cited prior work is used for methodology, not to force the conclusions.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No new theoretical entities are proposed. The term vulnerable circuits is a descriptive label for heads whose ablation reduces loss, not a new mechanism or object.

free parameters (3)
  • Cosine similarity threshold for adversarial sample filtering = 0.95
    Hand-selected threshold to keep only near-identical paraphrases; no sensitivity analysis reported.
  • Vulnerable head selection criterion = heads whose zero-ablation decreases loss on the adversarial test set
    This criterion is applied to the same adversarial test set later used to measure the accuracy improvement, making the improvement in-sample.
  • PGD attack parameters (perturbation budget, steps) = not reported
    Attack strength is a free choice that controls how many samples are perturbed (53.5% Jigsaw, 98.2% ToxiGen); without epsilon and iterations, results are not reproducible.
assumptions (3)
  • domain assumption Zero-ablation is a faithful causal intervention that isolates a head's role
    Methodology Step 1 sets head outputs to zero and interprets the accuracy drop as the head's contribution; zeroing can push later layers off-distribution.
  • domain assumption The PGD BERT-Attack adversarial examples preserve toxicity and semantics
    The paper filters by cosine similarity >0.95 but provides no human rating or label verification that perturbed inputs remain toxic; if many perturbed inputs are garbled, the 2.8% ToxiGen accuracy reflects corruption, not vulnerability.
  • domain assumption The stated Jigsaw class balance and split are correct
    The paper reports a 50k identity-annotated subset split into 46k toxic and 4k non-toxic, which inverts the known Jigsaw distribution and is inconsistent with the later 160k train / 63k test counts; the accuracy metric is then close to a majority-class baseline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content." pith.science (2026). https://pith.science/paper/JILGHLAV

@misc{pith2026250912672,
  author       = {Pith},
  title        = {Pith review of: Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JILGHLAV}},
  note         = {Machine review of arXiv:2509.12672}
}
read the original abstract

The volume of machine-generated content online has grown dramatically due to the widespread use of Large Language Models (LLMs), leading to new challenges for content moderation systems. Conventional content moderation classifiers, which are usually trained on text produced by humans, suffer from misclassifications due to LLM-generated text deviating from their training data and adversarial attacks that aim to avoid detection. Present-day defence tactics are reactive rather than proactive, since they rely on adversarial training or external detection models to identify attacks. In this work, we aim to identify the vulnerable components of toxicity classifiers that contribute to misclassification, proposing a novel strategy based on mechanistic interpretability techniques. Our study focuses on fine-tuned BERT and RoBERTa classifiers, testing on diverse datasets spanning a variety of minority groups. We use adversarial attacking techniques to identify vulnerable circuits. Finally, we suppress these vulnerable circuits, improving performance against adversarial attacks. We also provide demographic-level insights into these vulnerable circuits, exposing fairness and robustness gaps in model training. We find that models have distinct heads that are either crucial for performance or vulnerable to attack and suppressing the vulnerable heads improves performance on adversarial input. We also find that different heads are responsible for vulnerability across different demographic groups, which can inform more inclusive development of toxicity detection models.

Figures

Figures reproduced from arXiv: 2509.12672 by the authors.

Figure 1
Figure 1. One step of the BERT Attack Pipeline. Binary Cross Entropy Loss. It quantifies the difference between predicted probabilities and actual binary class la￾bels (0 or 1). It is calculated as: Loss = − [y · log(p) + (1 − y) · log(1 − p)] where y is the true label and p is the predicted probability for class 1, i.e. the toxic class. The lower the value, the better the model’s performance. Accuracy. The fraction of labels… view at source ↗
Figure 2
Figure 2. Change in accuracy by attention head on regular and adversarial input of the Jigsaw dataset. Cooler colors indicate a decrease in accuracy and warmer colors indicate an increase. turbations in the embedding space using Projected Gradi￾ent Descent (PGD). It first computes the gradient of the loss with respect to the input embeddings and iteratively applies small changes using another BERT model to maximize the loss, … view at source ↗
Figure 3
Figure 3. Accuracy on adversarial data by demographic group with and without ablation across two datasets. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Change in accuracy by attention head on adversarial inputs, disaggregated by demographic group, for two datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 9 linked inside Pith

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Barman, D.; Guo, Z.; and Conlan, O. 2024. The Dark Side of Language Models: Exploring the Potential of LLMs in Multimedia Disinformation Generation and Dissemination. Machine Learning with Applications, 16: 100545

    2024

  4. [4]

    Bensalem, I.; Rosso, P.; and Zitouni, H. 2024. Toxic language detection: A systematic review of Arabic datasets. Expert Systems, 41(8): e13551

    2024

  5. [5]

    Bereska, L.; and Gavves, E. 2024. Mechanistic interpretability for AI safety--a review. arXiv preprint arXiv:2404.14082

    arXiv 2024

  6. [6]

    Bespalov, D.; Bhabesh, S.; Xiang, Y.; Zhou, L.; and Qi, Y. 2024. Towards Building a Robust Toxicity Predictor. arXiv:2404.08690

    arXiv 2024

  7. [7]

    Brooks, C.; Eggert, S.; and Peskoff, D. 2024. The Rise of AI -Generated Content in W ikipedia. In Lucie-Aim \'e e, L.; Fan, A.; Gwadabe, T.; Johnson, I.; Petroni, F.; and van Strien, D., eds., Proceedings of the First Workshop on Advancing Natural Language Processing for Wikipedia, 67--79. Miami, Florida, USA: Association for Computational Linguistics

    2024

  8. [8]

    Caselli, T.; Basile, V.; Mitrovi \'c , J.; Kartoziya, I.; and Granitzer, M. 2020. I feel offended, don’t be abusive! implicit/explicit messages in offensive and abusive language. In Proceedings of the twelfth language resources and evaluation conference, 6193--6202

    2020

Show all 45 references
  1. [9]

    Chen, Z.; Xu, Z.; et al. 2022. When Should You Trust Toxicity Classifiers? A Framework for Evaluation and Adversarial Robustness. In Proceedings of the 2022 Conference on Fairness, Accountability, and Transparency (FAccT)

    2022

  2. [10]

    cjadams; Borkan, D.; inversion; Sorensen, J.; Dixon, L.; Vasserman, L.; and nithum. 2019. Jigsaw Unintended Bias in Toxicity Classification. https://kaggle.com/competitions/jigsaw-unintended-bias-in-toxicity-classification. Kaggle

    2019

  3. [11]

    Conmy, A.; Elhage, N.; et al. 2023. Towards Automated Circuit Discovery for Mechanistic Interpretability. arXiv preprint arXiv:2304.14997

    2023 arXiv

  4. [12]

    Devlin, J.; Chang, M.-W.; Lee, K.; and Toutanova, K. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805

    2019 arXiv

  5. [13]

    Dinan, E.; Fan, A.; Wu, L.; Weston, J.; and Kiela, D. 2020. Multi-dimensional Gender Bias Classification in Social Media. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 6046--6063

    2020

  6. [14]

    Dixon, L.; Li, J.; Sorensen, J.; Thain, N.; and Vasserman, L. 2018. Measuring and Mitigating Unintended Bias in Text Classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society (AIES), 67--73

    2018

  7. [15]

    Ebrahimi, J.; Rao, A.; Lowd, D.; and Dou, D. 2023. Gradient-based adversarial attacks on text transformers. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  8. [16]

    Fr \"o hling, L.; and Zubiaga, A. 2021. Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover. PeerJ Computer Science, 7: e443

    2021

  9. [17]

    García-Carrasco, J.; Maté, A.; and Trujillo, J. 2024. Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretability. In Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence, IJCAI-2024, 385–393. Internationa...

    2024

  10. [18]

    Hanna, M.; Balesni, A.; et al. 2023. Does GPT-2 Know Your Phone Number? Mechanistic Interpretability Applied to Emergent Representations. In Proceedings of the 2023 NeurIPS Workshop on Mechanistic Interpretability

    2023

  11. [19]

    Hartvigsen, T.; Gabriel, S.; Palangi, H.; Sap, M.; Ray, S.; and Choi, Y. 2022. ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL),...

    2022

  12. [20]

    Jiang, A.; and Zubiaga, A. 2024. Cross-lingual offensive language detection: A systematic review of datasets, transfer approaches and challenges. arXiv preprint arXiv:2401.09244

    2024

  13. [21]

    Khan, A.; Ahmed, A.; Jan, S.; Bilal, M.; and Zuhairi, M. F. 2024. Abusive language detection in urdu text: Leveraging deep learning and attention mechanism. IEEE Access, 12: 37418--37431

    2024

  14. [22]

    Krak, I.; Zalutska, O.; Molchanova, M.; Mazurets, O.; Bahrii, R.; Sobko, O.; and Barmak, O. 2024. Abusive Speech Detection Method for Ukrainian Language Used Recurrent Neural Network. In COLINS (3), 16--28

    2024

  15. [23]

    Lees, A.; Borkan, D.; Kivlichan, I.; Nario, J.; Dixon, L.; Sorensen, J.; and Thain, N. 2022. A new generation of perspective API: Efficient multilingual toxicity classification. arXiv preprint arXiv:2202.11176

    2022 arXiv

  16. [24]

    Li, J.; Ji, S.; Du, T.; Li, B.; and Wang, T. 2019. TextBugger: Generating Adversarial Text Against Real-world Applications. In Proceedings 2019 Network and Distributed System Security Symposium, NDSS 2019. Internet Society

    2019

  17. [25]

    Logacheva, V.; Dementieva, D.; Ustyantsev, S.; Moskovskiy, D.; Dale, D.; Krotova, I.; Semenov, N.; and Panchenko, A. 2022. P ara D etox: Detoxification with Parallel Data. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lon...

    2022

  18. [26]

    D.; and Finn, C

    Mitchell, E.; Lee, Y.; Khazatsky, A.; Manning, C. D.; and Finn, C. 2023. DetectGPT: zero-shot machine-generated text detection using probability curvature. In Proceedings of the 40th International Conference on Machine Learning, 24950--24962

    2023

  19. [27]

    R.; Li, G.; and Crespi, N

    Mnassri, K.; Farahbakhsh, R.; Chalehchaleh, R.; Rajapaksha, P.; Jafari, A. R.; Li, G.; and Crespi, N. 2024. A survey on multi-lingual offensive language detection. PeerJ Computer Science, 10: e1934

    2024

  20. [28]

    Mollas, I.; Chrysopoulou, Z.; Karlos, S.; and Tsoumakas, G. 2022. ETHOS : an online hate speech detection dataset. Complex & Intelligent Systems, 8(6): 4663--4678

    2022

  21. [29]

    Nobata, C.; Tetreault, J.; Thomas, A.; Mehdad, Y.; and Chang, Y. 2016. Abusive language detection in online user content. In Proceedings of the 25th international conference on world wide web, 145--153

    2016

  22. [30]

    Olah, C.; Cammarata, N.; Schubert, L.; Goh, G.; Petrov, M.; and Carter, S. 2020. Zoom In: An Introduction to Circuits. Distill, 5(3)

    2020

  23. [31]

    Olsson, C.; Elhage, N.; Nanda, N.; Joseph, N.; DasSarma, N.; Henighan, T.; Mann, B.; Askell, A.; Bai, Y.; Chen, A.; Conerly, T.; Drain, D.; Ganguli, D.; Hatfield-Dodds, Z.; Hernandez, D.; Johnston, S.; Jones, A.; Kernion, J.; Lovitt, L.; Ndousse, K.; Amodei, D.; Brown, T.; Cla...

    2022

  24. [32]

    Pan, L.; Liu, A.; He, Z.; Gao, Z.; Zhao, X.; Lu, Y.; Zhou, B.; Liu, S.; Hu, X.; Wen, L.; et al. 2024. MarkLLM: An Open-Source Toolkit for LLM Watermarking. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, 61--71

    2024

  25. [33]

    Roth, T.; Gao, Y.; Abuadbba, A.; Nepal, S.; and Liu, W. 2021. Token-Modification Adversarial Attacks for Natural Language Processing: A Survey. ArXiv, abs/2103.00676

    2021 arXiv

  26. [34]

    Sadrizadeh, S.; Dolamic, L.; and Frossard, P. 2023. A Classification-Guided Approach for Adversarial Attacks against Neural Machine Translation. In Conference of the European Chapter of the Association for Computational Linguistics

    2023

  27. [35]

    o ck, F.; and Wagner, C. 2021. “Call me sexist, but

    Samory, M.; Sen, I.; Kohne, J.; Fl \"o ck, F.; and Wagner, C. 2021. “Call me sexist, but...”: Revisiting sexism detection using psychological scales and adversarial samples. In Proceedings of the international AAAI conference on web and social media, volume 15, 573--584

    2021

  28. [36]

    Vasilatos, C.; Alam, M.; Rahwan, T.; Zaki, Y.; and Maniatakos, M. 2023. Howkgpt: Investigating the detection of chatgpt-generated university student homework through context-aware perplexity analysis. arXiv preprint arXiv:2305.18226

    2023 arXiv

  29. [37]

    Waghela, H.; Sen, J.; and Rakshit, S. 2024. Enhancing Adversarial Text Attacks on BERT Models with Projected Gradient Descent. arXiv:2407.21073

    2024 arXiv

  30. [38]

    Wang, K.; Variengien, A.; Conmy, A.; Shlegeris, B.; and Steinhardt, J. 2022 a . Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small. arXiv:2211.00593

    2022 arXiv

  31. [39]

    Wang, K.; Zou, A.; Mu, J.; et al. 2022 b . Interpretability in the Wild: A Circuit for Indirect Object Identification in GPT-2 Small. In Proceedings of the 2022 International Conference on Learning Representations (ICLR)

    2022

  32. [40]

    Warner, M.; Strohmayer, A.; Higgs, M.; and Coventry, L. 2025. A critical reflection on the use of toxicity detection algorithms in proactive content moderation systems. International Journal of Human-Computer Studies, 198: 103468

    2025

  33. [41]

    S.; and Wong, D

    Wu, J.; Yang, S.; Zhan, R.; Yuan, Y.; Chao, L. S.; and Wong, D. F. 2025 a . A Survey on LLM -Generated Text Detection: Necessity, Methods, and Future Directions. Computational Linguistics, 51(1): 275--338

    2025

  34. [42]

    S.; and Wong, D

    Wu, J.; Yang, S.; Zhan, R.; Yuan, Y.; Chao, L. S.; and Wong, D. F. 2025 b . A survey on llm-generated text detection: Necessity, methods, and future directions. Computational Linguistics, 51(1): 275--338

    2025

  35. [43]

    Wulczyn, E.; Thain, N.; and Dixon, L. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web, 1391--1399

    2017

  36. [44]

    Yin, W.; and Zubiaga, A. 2021. Towards generalisable hate speech detection: a review on obstacles and solutions. PeerJ computer science, 7: e598

    2021

  37. [45]

    Yoo, K.; Park, S.; Hwang, S.-w.; and Shin, J.-G. 2022. Detection of adversarial text attacks via neural representation analysis. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, 2315--2328

    2022

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.