Pith. sign in

REVIEW 3 major objections 3 minor 44 references

On the Role of Speech Data in Reducing Toxicity Detection Bias

T0 review · 3 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Access to raw speech at inference time reduces false positives on demographic group mentions in toxicity detection.

desk verdict The group annotations are the real contribution; the bias-reduction claim does not hold up as stated because the key comparison is confounded. read the letter →

arxiv 2411.08135 v2 pith:UJVYUNYS submitted 2024-11-12 cs.CL cs.AIcs.LGcs.SDeess.AS

Samuel J. Bell , Mariano Coria Meglioli , Megan Richards , Eduardo Sánchez , Christophe Ropers , Skyler Wang , Adina Williams , Levent Sagun
show 1 more author
Marta R. Costa-jussà
This is my paper · ORCID
classification cs.CLcs.AIcs.LGcs.SDeess.AS
keywords speechtoxicitydetectiongroupmentionbiasfalsepositiverateMUTOXdatasetannotationsmultimodalinferenceambiguoussamplestranscriptionerror
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether speech-based toxicity detectors are less biased against mentions of demographic groups than text-based ones. To answer it, the authors annotate 1,954 English and Spanish clips from MUTOX for toxicity, group mentions, and transcript accuracy, then compare four classifiers: a wordlist, a text neural network, and the MUTOX speech model with and without access to audio at inference. They report that the full speech model produces fewer false positives on group-mention clips than the same model fed only text transcripts, and that the difference concentrates on clips human annotators found ambiguous. They also find that correcting ASR transcripts does little to reduce the bias, pointing to the classifier rather than the transcription pipeline as the main lever. If right, the result argues for keeping raw audio available at test time in speech toxicity systems and focusing fairness effort on classifiers.

What carries the argument

The central object is the paired comparison MUTOX versus MUTOX-ASR: the same classifier, trained jointly on speech and text with SONAR embeddings (a sentence-level multilingual representation space), run either with raw audio plus ASR text (MUTOX) or with only the ASR text's SONAR embedding (MUTOX-ASR). This pairing is meant to isolate inference-time access to speech. The other machinery is a new annotation layer over MUTOX's test set, covering toxicity, group mentions, and corrected transcripts, which turns the dataset into a bias audit instrument, along with the false-positive-rate measurement on group-mention and ambiguous subsets.

What would settle it

Synthesize neutral-prosody renderings of the same transcripts and feed them to MUTOX: if its false-positive rate on group-mention clips rises to the level of MUTOX-ASR, the benefit is prosodic; if it stays low, the inference-time-speech explanation would need revision.

Watch

Extended reading notes

Core claim

The paper's central discovery is that a toxicity classifier trained on speech and text and given both at inference (MUTOX) has a lower false-positive rate on samples mentioning demographic groups than the same model given only text at inference (MUTOX-ASR), while text-only baselines show the opposite pattern. On ambiguous clips, those labeled Cannot say or No consensus, MUTOX and DETOXIFY have zero false positives, whereas MUTOX-ASR's false-positive rate rises when groups are mentioned. The paper interprets this as evidence that speech carries prosodic and contextual cues that let the model avoid treating neutral group mentions as toxicity, and it argues that the effect comes from inference-time access, not from training, because both MUTOX variants were trained identically. It also shows that replacing ASR transcripts with annotator-corrected transcripts barely changes the false-positive rate, which the paper takes to mean transcription is not the driver of group bias.

Load-bearing premise

The main conclusion rests on comparing the speech model with a version of itself that is denied audio at test time; if that denial is not a fair proxy for a genuinely text-based system, the measured bias reduction could be an artefact of the model being trained with speech but tested without it.

Editorial extensions

If this is right

  • Deployed speech toxicity detectors that can listen to raw audio at test time should produce fewer false positives on benign group-mention speech than cascaded ASR-to-text systems, with the largest gains on clips annotators find ambiguous.
  • Models trained jointly on speech and text should not be converted to text-only inference; removing audio at test time appears to make them lean harder on group mentions as toxicity cues.
  • For English and Spanish, better ASR transcription is unlikely to reduce group-mention false positives; modifying the classifier is the more direct lever.
  • The released MUTOX group annotations provide a reusable benchmark for auditing future speech toxicity systems across gender, race and ethnicity, and religion in two languages.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, if prosody is the active signal, resynthesizing the same utterances with flat intonation should make MUTOX's false positives approach MUTOX-ASR's; running that manipulation would test the mechanism directly.
  • Beyond the paper, the released group annotations could double as an ASR fairness audit, checking whether clips mentioning marginalized groups are disproportionately mistranscribed.
  • Beyond the paper, the transcription result likely generalizes only where ASR is strong; for lower-resourced languages, better transcripts may still be the cheaper fix.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper presents a new set of group annotations for the English and Spanish test partitions of the MuTox speech toxicity dataset, and uses them to compare four toxicity classifiers: ETOX (wordlist), Detoxify (text-only neural network), MuTox-ASR (jointly speech-text trained, but with only text at inference), and MuTox (jointly trained, with raw speech and text at inference). The authors report that MuTox, the only model with access to speech at inference, shows reduced false-positive rates on utterances mentioning demographic groups and zero false positives on ambiguous samples, while MuTox-ASR shows elevated false-positive rates on group mentions. They also find that correcting ASR transcripts has little effect on false-positive rates, concluding that improving classifiers rather than transcription pipelines is more helpful for reducing group bias.

Significance. If the central claim holds, the paper makes a useful contribution: it provides the first fairness-audit annotations for a multilingual speech toxicity dataset and offers practical guidance for speech-first toxicity detection. The annotation protocol is rigorous and the public release of the group annotations is a concrete asset for the community. However, the main empirical conclusion rests on a single ablation—MuTox versus MuTox-ASR—that is confounded by a train/test modality mismatch, and the computation of false-positive rates on ambiguous samples is underspecified. These issues must be addressed before the abstract's causal claim about 'access to speech data during inference' can be accepted.

major comments (3)
  1. [§4.1, Table 2, §4.3] This is the central confound and must be fixed or the claims must be reframed to only state that MuTox-ASR, a speech-trained model evaluated without audio, exhibits higher FPR under this modality dropout, which is a different and weaker claim.
  2. [§4.2, §4.3, Fig. 3b/3c] The same issue affects the comparison of group versus no-group ambiguous samples in Fig. 3c.
  3. [§8 and Fig. 4] The 'no significance testing' limitation may be acceptable for a purely exploratory dataset-release paper, but the paper makes strong comparative claims (e.g., 'eliminates false positives', 'reduces bias') that go beyond description and need some measure of uncertainty.
minor comments (3)
  1. [§4.2] This would help readers assess whether the FPR differences are robust or an artifact of a particular operating point.
  2. [§5, Fig. 5] Consider adding a caveat in the abstract or conclusion.
  3. [General] No further action needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims are empirical comparisons, not derivations, and no fitted parameter or self-citation forces the conclusion.

full rationale

This paper reports an empirical evaluation: the authors annotate a subset of the MuTox dataset and compare four classifiers on the resulting labels. There is no derivation chain in which a predicted quantity is defined in terms of a fitted parameter, and no equation in the paper reduces the central claim to its own inputs. The key comparison, MuTox versus MuTox-ASR, is an ablation in which the same joint speech-text model is deprived of speech at inference time; the observed difference is a measured outcome, not a tautology. The paper does not invoke a uniqueness theorem, does not smuggle an ansatz in via citation, and does not rename an existing result. The self-citations to MuTox and SONAR establish the dataset and encoder used in the experiments, but the bias-reduction claim is not justified by those citations alone; it rests on the new annotations and the reported classifier comparisons. The main weaknesses are experimental-design and measurement concerns: MuTox-ASR is not a pure text-only system, the FPR denominator for 'Cannot say' or 'No consensus' samples is not explicitly defined, and the annotations were produced by the authors. These are validity and independence issues, not circularity. The paper also explicitly characterizes the work as exploratory and declines statistical hypothesis testing, further indicating that no confirmatory derivation is being claimed. Therefore no circular step is present and the appropriate score is 0.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The paper's conclusions rest on the accuracy of author-produced toxicity and group labels used as ground truth, on treating MUTOX-ASR as a representative text-based system despite its modality mismatch, and on computing FPR from ambiguous labels without specifying the binarization rule. Per-language thresholds tuned on the MuTox validation set are the main free parameters. No new theoretical entities are introduced.

free parameters (1)
  • per-language decision threshold = unknown (tuned on MuTox validation to match ETOX precision)
    Each neural classifier's continuous score is binarized with a per-language threshold selected on the MuTox validation set to match ETOX precision; this choice directly affects FPR, the paper's main bias metric. Location: Section 4.2.
assumptions (3)
  • domain assumption The three author-annotators' consensus labels for toxicity and group mentions are correct ground truth.
    The entire evaluation rests on these labels; annotators are paper authors, and no inter-annotator agreement metric beyond raw vote counts is reported. Location: Sections 3.2 to 3.4.
  • ad hoc to paper MUTOX-ASR, a jointly trained model with audio dropped at inference, is a valid proxy for a text-based cascaded detector.
    The key comparison that supports the speech-inference claim uses a model trained with both modalities but evaluated without audio, which introduces a train/test modality mismatch absent in genuinely text-only systems. Location: Section 4.1, Table 2.
  • domain assumption FPR can be computed on samples labeled 'Cannot say' or 'No consensus'.
    Figure 3b reports FPR on ambiguous samples, which requires treating these labels as a binary ground truth; the paper does not describe how these labels enter the FPR denominator or whether they are counted as non-toxic. Location: Section 4.3, Figure 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of On the Role of Speech Data in Reducing Toxicity Detection Bias." pith.science (2026). https://pith.science/paper/UJVYUNYS

@misc{pith2026241108135,
  author       = {Pith},
  title        = {Pith review of: On the Role of Speech Data in Reducing Toxicity Detection Bias},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UJVYUNYS}},
  note         = {Machine review of arXiv:2411.08135}
}
read the original abstract

Text toxicity detection systems exhibit significant biases, producing disproportionate rates of false positives on samples mentioning demographic groups. But what about toxicity detection in speech? To investigate the extent to which text-based biases are mitigated by speech-based systems, we produce a set of high-quality group annotations for the multilingual MuTox dataset, and then leverage these annotations to systematically compare speech- and text-based toxicity classifiers. Our findings indicate that access to speech data during inference supports reduced bias against group mentions, particularly for ambiguous and disagreement-inducing samples. Our results also suggest that improving classifiers, rather than transcription pipelines, is more helpful for reducing group bias. We publicly release our annotations and provide recommendations for future toxicity dataset construction.

Figures

Figures reproduced from arXiv: 2411.08135 by the authors.

Figure 1
Figure 1. (a) Number of samples marked as toxic (“Yes”), not toxic (“No”), impossible to decide (“Cannot say”), or [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. (a) F-score, (b) precision, and (c) recall of each classifier, for samples [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. (a) Classifier false positive rate (FPR) for samples [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: False positive rate (FPR) of MUTOX and MUTOX-ASR on samples mentioning specific (a) gender identities, (b) racial or ethnic groups, (c) religious groups. (a) Mutox ASR shows a higher FPR for samples mentioning women than for other samples, whereas MUTOX’s FPR decreases…
Figure 5
Figure 5. Figure 5: (a) F1-score of cascaded ASR-based clas￾sifiers with original ASR transcripts and annotator￾corrected transcripts. (b) FPR on samples mentioning groups. Corrected transcripts only marginally improve model performance but have little to no impact on FPR. 6 Ambiguity in …
Figure 6
Figure 6. Figure 6: Overview of inter-annotator agreement and review for (a) toxicity, (b) group annotation, and (c) transcrip [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Annotation interface. Annotators could respond with free text if no checkbox was suitable. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Transcription correction interface. Annotators were only asked to correct the transcript if they marked it [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 21 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    CJ Adams, Jeffrey Sorensen, Julia Elliott, Lucas Dixon, Mark McDonald, Nithum Thain, and Will Cukierski. 2017. Toxic comment classification challenge. https://kaggle.com/competitions/jigsaw-toxic-comment-classification-challenge

    work page 2017

  4. [4]

    Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. https://proceedings.neurips.cc/paper/2020/hash/92d1e1eb1cd6f9fba3227870bb6d7f07-Abstract.html Wav2vec 2.0: A framework for self-supervised learning of speech representations . In Advances in Neural Information Processing Systems , volume 33, pages 12449--12460. Curran Associates, Inc

    work page 2020

  5. [5]

    Lo \"i c Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, Christopher Klaiber, Pengwei Li, Daniel Licht, Jean Maillard, Alice Rakotoarison, Kaushik Ram Sadagopan, Guillaume Wenzek, Ethan Ye, Bapi Akula, Peng-Jen Chen, Naji El Hachem, Brian Ellis, Gabriel...

    2025

  6. [6]

    Perspectives on Machine Learning from Psychology's Reproducibility Crisis

    Samuel J. Bell and Onno P. Kampman. 2021. https://doi.org/10.48550/arXiv.2104.08878 Perspectives on machine learning from psychology's reproducibility crisis . Preprint, arXiv:2104.08878

    work page Pith review arXiv doi:10.48550/arxiv.2104.08878 2021

  7. [7]

    Tomashenko, and Yannick Est \` e ve

    Marcely Zanon Boito, Laurent Besacier, Natalia A. Tomashenko, and Yannick Est \` e ve. 2022. https://doi.org/10.21437/INTERSPEECH.2022-353 A study of gender impact in self-supervised models for speech-to-text systems . In 23rd Annual Conference of the International Speech Communication Association, Interspeech 2022, pages 1278--1282. ISCA

    work page doi:10.21437/interspeech.2022-353 2022

  8. [8]

    Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. https://doi.org/10.1145/3308560.3317593 Nuanced metrics for measuring unintended bias with real data for text classification . In Companion Proceedings of The 2019 World Wide Web Conference , WWW '19, pages 491--500. Association for Computing Machinery

    work page arXiv 2019

Show all 44 references
  1. [9]

    Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, and Furu Wei. 2022. https://doi.org/10.1109/JSTSP.2022.3188...

    2022

  2. [10]

    Costa-juss \`a , Christine Basta, and Gerard I

    Marta R. Costa-juss \`a , Christine Basta, and Gerard I. G \'a llego. 2022. https://aclanthology.org/2022.lrec-1.230/ Evaluating gender bias in speech translation . In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2141--2147. European Langua...

    2022

  3. [11]

    Costa-juss \`a , Mariano Meglioli, Pierre Andrews, David Dale, Prangthip Hansanti, Elahe Kalbassi, Alexandre Mourachko, Christophe Ropers, and Carleigh Wood

    Marta R. Costa-juss \`a , Mariano Meglioli, Pierre Andrews, David Dale, Prangthip Hansanti, Elahe Kalbassi, Alexandre Mourachko, Christophe Ropers, and Carleigh Wood. 2024. https://aclanthology.org/2024.findings-acl.340 M u T ox: U niversal multilingual audio-based toxicity da...

    2024

  4. [12]

    Costa-juss \`a , Eric Smith, Christophe Ropers, Daniel Licht, Jean Maillard, Javier Ferrando, and Carlos Escolano

    Marta R. Costa-juss \`a , Eric Smith, Christophe Ropers, Daniel Licht, Jean Maillard, Javier Ferrando, and Carlos Escolano. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.642 Toxicity in multilingual machine translation at scale . In Findings of the Association for Comp...

    2023 doi

  5. [13]

    Richard L Daft, Robert H Lengel, and Linda Klebe Trevino. 1987. Message equivocality, media selection, and manager performance: Implications for information systems. MIS Quarterly, pages 355--366

    1987

  6. [14]

    Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. https://doi.org/10.18653/v1/W19-3504 Racial bias in hate speech and abusive language detection datasets . In Proceedings of the Third Workshop on Abusive Language Online , pages 25--35. Association for Computatio...

    2019 doi

  7. [15]

    Thiago Dias Oliva, Dennys Marcelo Antonialli, and Alessandra Gomes. 2021. https://doi.org/10.1007/s12119-020-09790-w Fighting hate speech, silencing drag queens? A rtificial intelligence in content moderation and risks to LGBTQ voices online . Sexuality & Culture, 25(2):700--732

    2021 doi

  8. [16]

    Lucas Dixon, John Li, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2018. https://doi.org/10.1145/3278721.3278729 Measuring and mitigating unintended bias in text classification . In Proceedings of the 2018 AAAI / ACM Conference on AI , Ethics , and Society , AIES '18, p...

    2018

  9. [17]

    Paul-Ambroise Duquenne, Holger Schwenk, and Beno \^i t Sagot. 2023. https://doi.org/10.48550/arXiv.2308.11466 SONAR : S entence-level multimodal and language-agnostic representations . Preprint, arXiv:2308.11466

    2023 doi arXiv

  10. [18]

    Siyuan Feng, Olya Kudina, Bence Mark Halpern, and Odette Scharenborg. 2021. https://doi.org/10.48550/arXiv.2103.15122 Quantifying bias in automatic speech recognition . Preprint, arXiv:2103.15122

    2021 doi arXiv

  11. [19]

    Tanmay Garg, Sarah Masud, Tharun Suresh, and Tanmoy Chakraborty. 2023. https://doi.org/10.1145/3580494 Handling bias in toxic speech detection: A survey . ACM Computing Surveys, 55(13s)

    2023 doi

  12. [20]

    Mahault Garnerin, Solange Rossato, and Laurent Besacier. 2019. https://doi.org/10.1145/3347449.3357480 Gender representation in French broadcast corpora and its impact on ASR performance . In Proceedings of the 1st International Workshop on AI for Smart TV Content Production ,...

    2019

  13. [21]

    Sreyan Ghosh, Samden Lepcha, Sakshi Singh, Rajiv Ratn Shah, and Srinivasan Umesh. 2022. https://doi.org/10.21437/INTERSPEECH.2022-10752 Detoxy: A large-scale multimodal dataset for toxicity classification in spoken utterances . In 23rd Annual Conference of the International Sp...

    2022 doi

  14. [22]

    Kristina Gligoric, Myra Cheng, Lucia Zheng, Esin Durmus, and Dan Jurafsky. 2024. https://doi.org/10.18653/v1/2024.naacl-long.331 NLP systems that can't tell use from mention censor counterspeech, but teaching the distinction helps . In Proceedings of the 2024 Conference of the...

    2024 doi

  15. [23]

    Kivlichan, Rachel Rosen, and Lucy Vasserman

    Nitesh Goyal, Ian D. Kivlichan, Rachel Rosen, and Lucy Vasserman. 2022. https://doi.org/10.1145/3555088 Is your toxicity my toxicity? E xploring the impact of rater identity on toxicity annotation . In Proceedings of the ACM on Human-Computer Interaction, volume 6 (CSCW2). Ass...

    2022 doi

  16. [25]

    Laura Hanu. 2020. Detoxify. https://github.com/unitaryai/detoxify

    2020

  17. [26]

    Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2020. https://proceedings.neurips.cc/paper/2020/hash/1b84c4cee2b8b3d823b30e2d604b1878-Abstract.html The hateful memes challenge: D etecting hate speech in multim...

    2020

  18. [27]

    Robert Kraut, Jolene Galegher, Robert Fish, and Barbara Chalfonte. 1992. Task requirements and media choice in collaborative writing. Human--Computer Interaction, 7(4):375--407

    1992

  19. [28]

    Kumar, Saurav Sahay, Sahisnu Mazumder, Eda Okur, Ramesh Manuvinakurike, Nicole Beckage, Hsuan Su, Hung-yi Lee, and Lama Nachman

    Shachi H. Kumar, Saurav Sahay, Sahisnu Mazumder, Eda Okur, Ramesh Manuvinakurike, Nicole Beckage, Hsuan Su, Hung-yi Lee, and Lama Nachman. 2024. https://doi.org/10.48550/arXiv.2408.03907 Decoding biases: A utomated methods and LLM judges for gender bias detection in language m...

    2024 doi arXiv

  20. [29]

    Yi-Cheng Lin, Tzu-Quan Lin, Chih-Kai Yang, Ke-Han Lu, Wei-Chih Chen, Chun-Yi Kuan, and Hung-Yi Lee. 2024. https://doi.org/10.1109/SLT61566.2024.10832317 Listen and speak fairly: A study on semantic gender bias in speech integrated large language models . In IEEE Spoken Languag...

    2024

  21. [30]

    Joseph Liu, Mahesh Kumar Nandwana, Janne Pylkk \"o nen, Hannes Heikinheimo, and Morgan McGuire. 2024. https://arxiv.org/abs/2406.10325v1 Enhancing multilingual voice toxicity detection with speech-text alignment . In 25th Annual Conference of the International Speech Communica...

    2024 arXiv

  22. [31]

    Liu, and Hung - yi Lee

    Yen Meng, Yi - Hui Chou, Andy T. Liu, and Hung - yi Lee. 2022. https://doi.org/10.1109/ICASSP43922.2022.9747897 Don't speak too fast: The impact of data bias on self-supervised speech models . In IEEE International Conference on Acoustics, Speech and Signal Processing ( ICASSP...

    2022

  23. [32]

    Mahesh Nandwana, Yifan He, Joseph Liu, Xiao Yu , Charles Shang, Eloi Du Bois, Morgan McGuire, and Kiran Bhat. 2024. https://doi.org/10.1109/ICASSP48485.2024.10448289 Voice toxicity detection using multi-task learning . In 2024 IEEE International Conference on Acoustics , Speec...

    2024

  24. [33]

    Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. https://doi.org/10.18653/v1/D18-1302 Reducing gender bias in abusive language detection . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages 2799--2804. Association for Computational...

    2018 doi

  25. [34]

    John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, and Ion Androutsopoulos. 2020. https://doi.org/10.18653/v1/2020.acl-main.396 Toxicity detection: Does context really matter? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistic...

    2020 doi

  26. [35]

    Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, Alexei Baevski, Yossi Adi, Xiaohui Zhang, Wei-Ning Hsu, Alexis Conneau, and Michael Auli. 2024. http://jmlr.org/papers/v25/23-1318.h...

    2024

  27. [36]

    Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine Mcleavey, and Ilya Sutskever. 2023. https://proceedings.mlr.press/v202/radford23a.html Robust speech recognition via large-scale weak supervision . In Proceedings of the 40th International Conference on Machine Lear...

    2023

  28. [37]

    Resende, Luiz F

    Guilherme H. Resende, Luiz F. Nery, Fabrício Benevenuto, Savvas Zannettou, and Flavio Figueiredo. 2024. https://arxiv.org/abs/2401.12720 A comprehensive view of the biases of toxicity and sentiment analysis methods towards utterances with african american english expressions ....

    2024 arXiv

  29. [38]

    Nihar Sahoo, Himanshu Gupta, and Pushpak Bhattacharyya. 2022. https://doi.org/10.18653/v1/2022.conll-1.10 Detecting unintended social bias in toxic language datasets . In Proceedings of the 26th Conference on Computational Natural Language Learning (CoNLL), pages 132--143. Ass...

    2022 doi

  30. [39]

    Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. https://doi.org/10.18653/v1/P19-1163 The risk of racial bias in hate speech detection . In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 1668--1678....

    2019 doi

  31. [40]

    Maarten Sap, Swabha Swayamdipta, Laura Vianna, Xuhui Zhou, Yejin Choi, and Noah A. Smith. 2022. https://doi.org/10.18653/v1/2022.naacl-main.431 Annotators with attitudes: How annotator beliefs and identities bias toxic language detection . In Proceedings of the 2022 Conference...

    2022 doi

  32. [41]

    Khetam Al Sharou and Lucia Specia. 2022. https://aclanthology.org/2022.eamt-1.20 A taxonomy and study of critical errors in machine translation . In Proceedings of the 23rd Annual Conference of the European Association for Machine Translation , pages 171--180. European Associa...

    2022

  33. [42]

    Rachael Tatman. 2017. https://doi.org/10.18653/v1/W17-1606 Gender and dialect bias in Y ou T ube's automatic captions . In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing , pages 53--59. Association for Computational Linguistics

    2017 doi

  34. [43]

    Maxim Tkachenko, Mikhail Malyuk, Andrey Holmanyuk, and Nikolai Liubimov. 2020. https://github.com/HumanSignal/label-studio Label Studio : D ata labeling software . https://github.com/HumanSignal/label-studio

    2020

  35. [44]

    Ahmet \"U st \"u n, Viraat Aryabumi, Zheng Yong, Wei-Yin Ko, Daniel D ' souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, Freddie Vargus, Phil Blunsom, Shayne Longpre, Niklas Muennighoff, Marzieh Fadaee, Julia Kreutzer, and Sara Hooker. 2024. ht...

    2024 doi

  36. [45]

    Alexandros Xenos, John Pavlopoulos, Ion Androutsopoulos, Lucas Dixon, Jeffrey Sorensen, and Leo Laugier. 2021. https://doi.org/10.48550/arXiv.2111.10223 Toxicity detection can be sensitive to the conversational context . Preprint, arXiv:2111.10223

    2021 doi arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.