RECORD
Cross-Domain Toxic Spans Detection
Computation and Language · Submitted 16 Jun 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Our findings indicate that a simple method using off-the-shelf lexicons performs best in the cross-domain setup.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Given the dynamic nature of toxic language use, automated methods for detecting toxic spans are likely to encounter distributional shift. To explore this phenomenon, we evaluate three approaches for detecting toxic spans under cross-domain conditions: lexicon-based, rationale extraction, and fine-tuned language models. Our findings indicate that a simple method using off-the-shelf lexicons performs best in the cross-domain setup. The cross-domain error analysis suggests that (1) rationale extraction methods are prone to false negatives, while (2) language models, despite performing best for the in-domain case, recall fewer explicitly toxic words than lexicons and are prone to certain types of false positives. Our code is publicly available at: https://github.com/sfschouten/toxic-cross-domain.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Given the dynamic nature of toxic language use, automated methods for detecting toxic spans are likely to encounter distributional shift. | background 0.99 | 0.07 | 0.03 | hedged possibility 1.00 |
| s1 | To explore this phenomenon, we evaluate three approaches for detecting toxic spans under cross-domain conditions: lexicon-based, rationale extraction, and fine-tuned language models. | method 0.70 | 0.83 | 0.03 | asserted 0.90 |
| s2 | Our findings indicate that a simple method using off-the-shelf lexicons performs best in the cross-domain setup. | result 1.00 | 0.96 | 0.86 | asserted 0.73 |
| s3 | The cross-domain error analysis suggests that (1) rationale extraction methods are prone to false negatives, while (2) language models, despite performing best for the in-domain case, recall fewer explicitly toxic words than lexicons and are prone to certain types of false positives. | result 0.73 | 0.93 | 0.22 | hedged possibility 0.71 |
| s4 | Our code is publicly available at: https://github.com/sfschouten/toxic-cross-domain. | contribution 0.90 | 0.41 | 0.01 | asserted 0.88 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Given the dynamic nature of toxic language use, automated methods for detecting toxic spans are likely to encounter distributional shift. To explore this phenomenon, we evaluate three approaches for detecting toxic spans under cross-domain conditions: lexicon-based, rationale extraction, and fine-tuned language models. Our findings indicate that a simple method using off-the-shelf lexicons performs best in the cross-domain setup. The cross-domain error analysis suggests that (1) rationale extraction methods are prone to false negatives, while (2) language models, despite performing best for the in-domain case, recall fewer explicitly toxic words than lexicons and are prone to certain types of false positives. Our code is publicly available at: https://github.com/sfschouten/toxic-cross-domain.
Human review
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.