Social and Information Networks · arXiv:2610.07953 v1 · submitted 6 Oct 2026 · 47 references
Benchmarking System One Models in Online Moderation
The paper says
“Online moderation systems must apply changing platform policies, community rules, and prior decisions while producing decisions that can be audited and routed to human review.”
more of the abstract
We evaluate whether System One Models, which accept natural-language context but return typed choices, probabilities, or scores, can support this setting. Across five moderation benchmarks, Jev is competitive with specialized reference systems, matching or exceeding them in several policy-grounded and harmful-content settings. We then use controlled information conditions to separate written rules, retrieved precedents, and restrictions on the candidate answer space. Jev generally benefits from retrieved precedents, improving exact policy selection and harmful-content discrimination when the answer space is held fixed. Laya is less consistent: retrieval often shifts its positive prediction rate or no-violation rate without improving discrimination. Jev confidence can support selective review in several settings, although it is not consistently calibrated; Laya confidence is less useful for ranking errors. These results show a promising future for SOM-powered content moderation.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 7 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A well-designed policy-selection evaluation with a real but fixable leakage problem in the harmful-content arm; deserves review. This line is wrong
1 major objection · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 47 references say.
The paper says, at Section 3.2 and Section 4.2 (Tables 6 and 7)
The review did not quote this passage.
Pith says: Objection 1 of 1 · major
The harmful-content retrieval contrasts draw precedents leave-one-out from the same cleaned evaluation population: Section 3.2 states this for OLID and HateXplain, and Section 4.2 applies the same protocol to ModerationBench. Under the paper's own framing, a precedent is a previously decided case, so other evaluation-population labels are not available at inference time in any deployed pipeline; the reported Jev gains …
more of objection 1
… (AUROC +.027 on OLID, +.005 on HateXplain, and +.043 and +.038 on ModerationBench) are optimistic estimates with unknown bias, and the abstract's claim that precedents improve harmful-content discrimination when the answer space is held fixed is not yet supported. For OLID and HateXplain the evaluation population appears to be the full cleaned released corpus rather than a held-out split, so both the Direct baselines and the RAD contrasts are transductive. This concern is specific to the harmful-content leg: the policy-selection claim stands, because the PluRule test intersection, HateModerate, and the Appendix B six-community replication all use training-only precedent sources (Section 4.1 states that validation and test labels are never used as precedent sources). The Limitations section does not acknowledge the label-availability assumption. I ask for a held-out replication using official training-split precedents, or, failing that, an explicit reframing of the harmful-content retrieval results as transductive upper bounds with the abstract revised accordingly.
4 minor objections
The paper says, at Throughout
Pith says: minor objection 1 of 4
There are several typographical and spacing errors to correct: 'reseaarch' in Section 2; 'Jevis' and 'Layais' in the Abstract; 'F edericoMz' in the GitHub URL in Section 3; 'byembeddingmessages, usingasembeddingmodel' in Section 3.2; and 'distinguishoperating-point' in Section 3.5.
The paper says, at Tables 6 and 7 captions
Pith says: minor objection 2 of 4
The OLID and HateXplain evaluation populations are the full cleaned released corpora, not held-out splits; the table captions should state this explicitly and note that the RAD precedents are transductive, so the AUROC contrasts are not deployment estimates.
show 2 more
The paper says, at Table 8
Pith says: minor objection 3 of 4
Table 8 mixes policy and harmful-content conditions whose Exact Acc. values exclude deterministic single-option decisions; adding the number of model-inferred decisions behind each row would help readers reconcile these values with the population-level EMA in Tables 2 through 4.
The paper says, at Section 3.2
Pith says: minor objection 4 of 4
The retrieval depth is fixed at k=5 throughout and the paper reports no sensitivity to k; since k is one of only two free parameters in the design, a sentence reporting stability (or pointing to the Limitations future-work item) would strengthen the main contrasts.
What would settle it
Run a controlled replication of the harmful-content Noul Direct-versus-RAD contrast, plus the ModerationBench Choice and Noul contrasts, in which every precedent is sourced from labels that predate the target: OLID and HateXplain official training splits, and for ModerationBench a temporally earlier slice or an external seed set, keeping retrieval k=5, qwen3-embedding:4b, prompts, and threshold identical. Compare Jev AUROC deltas to the reported +.027 (OLID), +.005 (HateXplain), and +.038/+.043 (ModerationBench), with paired bootstrap confidence intervals. If the deltas shrink to or below zero, the general claim that precedents improve harmful-content discrimination is unsupported; if they survive, the leave-one-out protocol is not the driver.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 7 Oct 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 1 major objection, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Read it. The policy-selection core is the paper's real contribution and it holds up. The matched General-all versus NAD-all contrasts on PluRule and HateModerate are clean: same answer space, same interface, bootstrap confidence intervals, Jev-full/compact sensitivity checks, and an appendix replicating the six-community analysis with train-only precedents. The +12.35, +5.20, and +22.20 EMA gains are credible. The decomposition of written rules, retrieved precedents, and answer-space restriction is genuinely useful, and the paper is unusually careful about separating operating point from discrimination, as the Laya analysis shows.
What's new: this is the first broad moderation evaluation of System One Models, and it supplies a controlled framework for separating information conditions. That is enough to make it worth citing.
The soft spot is exactly where the reader put it. OLID, HateXplain, and ModerationBench RAD conditions retrieve precedents leave-one-out from the same cleaned evaluation population. A precedent is supposed to be a previously decided case; at inference time, labels of other evaluation items are not available. The reported Jev AUROC gains (+.027, +.005, and +.038) are therefore optimistic estimates of the retrieval benefit, with unknown bias. This is not acknowledged in the Limitations section, and the abstract's claim about harmful-content discrimination overstates what the design supports. It is fixable: run the harmful-content arms with train-sourced precedents, or at minimum frame them as transductive estimates and provide a separate held-out replication.
Minor points: Jev is proprietary, the repository has no commit hash, and the external references (Detoxify, published PluRule and ModerationBench numbers) are descriptive rather than input-matched. The paper mostly labels them that way, so this is a caveat, not a flaw. The System One Model category is sourced from vendor documentation, which is the normal situation when evaluating a new proprietary model class.
Who gets value: anyone building or benchmarking policy-conditioned moderation, and people designing evaluations that separate rules, examples, and answer space. The paper deserves a serious referee. I would conditionally accept with a required harmful-content replication using train-sourced precedents and a toned-down abstract.
The strongest objection
The central claim as stated in the abstract is twofold: retrieved precedents improve exact policy selection, and they improve harmful-content discrimination when the answer space is fixed. The first part is well supported and should not be tarred by this concern: PluRule test-intersection retrieval uses only training decisions, with Section 4.1 explicitly stating that validation and test labels are never used as precedent sources; HateModerate retrieval uses training examples; and Appendix B repeats the transductive six-community result with train-sourced precedents. The second part is not yet supported because the harmful-content RAD conditions draw precedents leave-one-out from the same cleaned evaluation population: Section 3.2 says so for OLID and HateXplain, and Section 4.2 applies the same protocol on ModerationBench. Under the paper's own framing, a precedent is a previously decided case; labels of other evaluation-population items are not known at inference time in any deployment. This makes the reported Jev AUROC gains (+.027 on OLID, +.005 on HateXplain, and +.038 or +.043 on ModerationBench) optimistic estimates of the retrieval benefit, with unknown bias. The protocol is disclosed as a design choice, but the Limitations section does not acknowledge the label-availability issue, and no held-out precedent-source replication is supplied for the harmful-content tasks. Thus the policy-selection result is robust, while the harmful-content generalization rests on an unchecked assumption.
The reader's weakest assumption identifies exactly the load-bearing soft spot: leave-one-out precedents from the evaluation population in harmful-content experiments. I agree with that concern. The paper's policy-selection claim, which the reader's strongest claim emphasizes, is independently supported: the PluRule test-intersection and HateModerate experiments use training-only precedent sources, and Appendix B supplies a train-sourced held-out replication of the transductive six-community PluRule result, so the +12.35, +5.20, and +22.20 EMA gains are not threatened by the leakage issue. What is threatened is the abstract's broader claim that precedents generally improve harmful-content discrimination, which rests entirely on OLID, HateXplain, and ModerationBench, all of which use evaluation-population labels as precedents. The paper states the protocol but does not treat it as a limitation or provide a held-out version for these datasets, so the concern is unresolved. Because the reader already conditioned the verdict on this issue and the policy-selection core remains credible, I do not recommend moving the verdict; the conditional verdict stands unchanged.
The referee's report
This paper makes a genuinely useful methodological contribution: by varying written rules, retrieved precedents, and the candidate-answer inventory independently, it shows what a controlled information-condition evaluation of moderation systems can measure, and the typed-output interfaces of the SOMs make the required contrasts (General-all versus NAD-all, semantic versus opaque RAD, Direct versus RAD) well defined. The policy-selection core is well supported: matched contrasts with paired bootstrap CIs, Jev-full/Jev-compact serialization sensitivity runs, the semantic-opaque control, a train-sourced PluRule test-intersection protocol, and the held-out train-sourced six-community replication in Appendix B. The ModerationBench caption sensitivity analysis (Appendix E) and the public code repository are additional strengths, as is the paper's care in labeling published reference systems as descriptive rather than input-matched. If the harmful-content leg is repaired or appropriately reframed, the result that typed SOM outputs can be decomposed into policy, precedent, and answer-space effects would be of clear value to the moderation community.
Pith's reading of the paper (a machine's summary, not the authors')
The paper asks whether System One Models—systems that read natural-language context but return typed choices, probabilities, or scores—can support online content moderation. It claims that one such system, Jev, benefits from retrieved similar past decisions: when the full candidate rule set is held fixed, precedents improve exact policy selection by 12.35, 5.20, and 22.20 accuracy points across three policy evaluations, and they improve harmful-content discrimination on the offensiveness and moderation benchmarks studied. A second system, Laya, does not respond the same way, with retrieval more often shifting how often it flags content than improving its ability to rank violations. The paper also argues that Jev's confidence scores can support selective human review, and that separating written rules, retrieved precedents, and the candidate answer space is essential for measuring moderation systems honestly.
What carries the argument. The argument is carried by a set of controlled information conditions that separate three sources of decision context: written rules, retrieved precedents, and the candidate answer space. The conditions are Direct (target only), NAD (written rules, no precedents), RAD (precedents, no written rules), and General (both), with variants that expose all, represented, or retrieved rules as possible answers. The load-bearing comparison is General-all versus NAD-all, which adds retrieved precedents while holding the complete rule inventory fixed; on the harmful-content tasks, the matched comparison is Direct versus semantic RAD. Precedents are retrieved by cosine similarity of message embeddings, with five nearest cases per target, and exact-match accuracy (EMA) requires either the annotated violated rule or a correct 'no rules broken' decision. This machinery lets the paper attribute gains to the precedent signal itself rather than to a restricted answer space.
Core claim. The central claim is that a System One Model can act as a policy-grounded moderation engine when it is given both the written rules and retrieved prior decisions, rather than only one of the two. In the controlled comparison that holds the complete rule inventory fixed, adding five similar precedents raises Jev's exact-match accuracy from 52.7 to 65.1 on the six-community PluRule setting, from 48.4 to 53.6 on the PluRule test intersection, and from 29.4 to 51.6 on HateModerate, with the corresponding violated-rule recall increasing in each case. On harmful-content detection, precedents raise Jev's AUROC on OLID, HateXplain, and ModerationBench, with the largest discrimination gains on the moderation benchmark. Laya behaves differently: retrieval often increases its rate of positive or no-violation predictions without improving ranking, and in some cases degrades AUROC sharply. The paper therefore concludes that precedent-augmented SOM decisions can identify the exact violated rule rather than merely flag content, provided the answer space is held fixed and the model's probabilities and confidence are evaluated separately from its operating point.
The premise it rests on. The main harmful-content gains and the high-support PluRule gain assume that labels from the same evaluation population are available as previously decided cases at inference time, because those precedents are retrieved leave-one-out from that population.
If the paper is right (Pith's inferences, not the authors')
A moderation system can identify which specific rule was broken, not just whether content violates policy, when similar past decisions are supplied alongside the full rule list.
Holding the candidate answer space fixed is essential: restricting allowed rules to those found in retrieved precedents can inflate nominal accuracy by pushing decisions toward 'no rules broken'.
Retrieval-based harmful-content moderation can improve discrimination as measured by AUROC, but accuracy and F1 can still decline if the operating point moves, so the two must be reported together.
Confidence thresholds of at least 0.8 can route a substantial fraction of decisions to automation with higher accuracy in several settings, though calibration is not reliable enough to treat confidence as a single guarantee.
The precedent benefit is model-specific: one evaluated system improves consistently, while the other mainly changes its flagging rate, so retrieval should be tuned per model rather than assumed to help universally.
If the leave-one-out precedent design were replaced with a genuinely external precedent store, the harmful-content AUROC gains might shrink; the exact magnitude is testable and is not settled by the paper's design.
The semantic-versus-opaque policy-label comparison hints that the human-readable names of rules carry decision-relevant information, so rewording a policy could shift moderation outcomes without any retraining.
A deployed system would need a governance loop that feeds confirmed human-review outcomes back into the precedent store; otherwise retrieval would propagate past biases and errors forward.
The single-choice formulation cannot represent content that violates multiple policies at once, so an immediate extension is a multi-label version of the same information-condition framework.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Online moderation systems must apply changing platform policies, community rules, and prior decisions while producing decisions that can be audited and routed to human review. We evaluate whether System One Models, which accept natural-language context but return typed choices, probabilities, or scores, can support this setting. Across five moderation benchmarks, Jev is competitive with specialized reference systems, matching or exceeding them in several policy-grounded and harmful-content settings. We then use controlled information conditions to separate written rules, retrieved precedents, and restrictions on the candidate answer space. Jev generally benefits from retrieved precedents, improving exact policy selection and harmful-content discrimination when the answer space is held fixed. Laya is less consistent: retrieval often shifts its positive prediction rate or no-violation rate without improving discrimination. Jev confidence can support selective review in several settings, although it is not consistently calibrated; Laya confidence is less useful for ranking errors. These results show a promising future for SOM-powered content moderation.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-07
The authors' main claim, as the classifier picked it out 0.92
“These results show a promising future for SOM-powered content moderation.”
How the main claim is stated: asserted 0.65
What the abstract itself states, as probabilities: claims first 0.03 · states limitation 0.88 · human participants 0.11 · controlled comparison 0.76 · randomized assignment 0.03 · states quantitative result 0.11 · compares with prior results 0.92 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Online moderation systems must apply changing platform policies, community rules, and prior decisions while producing decisions that can be audited and routed to human review. | background 1.00 | 0.05 | 0.03 | asserted 1.00 |
| We evaluate whether System One Models, which accept natural-language context but return typed choices, probabilities, or scores, can support this setting. | contribution 0.92 | 0.60 | 0.03 | hedged possibility 0.45 |
| Across five moderation benchmarks, Jev is competitive with specialized reference systems, matching or exceeding them in several policy-grounded and harmful-content settings. | result 1.00 | 0.93 | 0.49 | asserted 1.00 |
| We then use controlled information conditions to separate written rules, retrieved precedents, and restrictions on the candidate answer space. | method 0.95 | 0.94 | 0.02 | asserted 0.87 |
| Jev generally benefits from retrieved precedents, improving exact policy selection and harmful-content discrimination when the answer space is held fixed. | result 1.00 | 0.94 | 0.11 | asserted 0.98 |
| Laya is less consistent: retrieval often shifts its positive prediction rate or no-violation rate without improving discrimination. | result 1.00 | 0.94 | 0.04 | asserted 0.99 |
| Jev confidence can support selective review in several settings, although it is not consistently calibrated; Laya confidence is less useful for ranking errors. | result 1.00 | 0.94 | 0.08 | asserted 0.98 |
| These results show a promising future for SOM-powered content moderation. | interpretation 0.96 | 0.48 | 0.81 | asserted 0.65 |
What kind of paper the classifier reports this is: empirical study 1.00 · evidence from observation or experiment 0.93
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-07: 1 request, 6,116 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
References (47)
- [1] D. Almeida. Introducing system one models and jev, Sept. 2026. TypeSafe AI on Pith
- [2] A. Arango, J. Pérez, and B. Poblete. Hate speech detection is not as easy as you may think: A closer look at model validation. InProceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 45–54, 2019 on Pith
- [3] K. D. Ashley.Modeling Legal Argument: Reasoning with Cases and Hypotheticals. MIT Press, 1990. 27 on Pith
- [4] E. Chandrasekharan, C. Gandhi, M. W. Mustelier, and E. Gilbert. Crossmod: A cross- community learning-based system to assist reddit moderators.Proceedings of the ACM on Human-Computer Interaction, 3(CSCW):1–30, 2019 on Pith
- [5] C. Chen, W. Qu, S. Su, Y. Feng, and T. Li. A comprehensive review of llm-based con- tent moderation: Advancements, challenges, and future directions.Knowledge-Based Systems, 330:114689, 2025 on Pith
- [6] T. Davidson, D. Warmsley, M. Macy, and I. Weber. Automated hate speech detection and the problem of offensive language. InProceedings of the International AAAI Conference on Web and Social Media, volume 11, pages 512–515, 2017 on Pith
- [7] L. Dixon, J. Li, J. Sorensen, N. Thain, and L. Vasserman. Measuring and mitigating unintended bias in text classification. InProceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pages 67–73, 2018 on Pith
- [8] E. Douek. Content moderation as systems thinking.Harvard Law Review, 136(2):526–607, 2022 on Pith
- [9] C. Fiesler, J. A. Jiang, J. McCann, K. Frye, and J. R. Brubaker. Reddit rules! characterizing an ecosystem of governance. InProceedings of the International AAAI Conference on Web and Social Media, volume 12, 2018 on Pith
- [10] P. Fortuna and S. Nunes. A survey on automatic detection of hate speech in text.ACM Computing Surveys, 51(4):1–30, 2018 on Pith
- [11] T. Gillespie.Custodians of the Internet: Platforms, Content Moderation, and the Hidden Decisions That Shape Social Media. Yale University Press, 2018 on Pith
- [12] T. Gillespie. Content moderation, ai, and the question of scale.Big Data & Society, 7(2):2053951720943234, 2020 on Pith
- [13] V. U. Gongane, M. V. Munot, and A. D. Anuse. Detection and moderation of detrimental con- tent on social media platforms: Current status and future directions.Social Network Analysis and Mining, 12(1):129, 2022 on Pith
- [14] R. Gorwa, R. Binns, and C. Katzenbach. Algorithmic content moderation: Technical and political challenges in the automation of platform governance.Big data & society, 7(1):2053951719897945, 2020 on Pith
- [15] L. Hanu and Unitary team. Detoxify. Github. https://github.com/unitaryai/detoxify, 2020 on Pith
- [16] H. Inan, K. Upasani, J. Chi, R. Rungta, K. Iyer, Y. Mao, E. Tontchev, Q. Hu, B. Fuller, D. Testuggine, and M. Khabsa. Llama guard: Llm-based input-output safeguard for human-ai conversations.arXiv preprint arXiv:2312.06674, 2023 on PithPith's reviewarXiv
- [17] Z. Kachwala, B. T. Truong, R. Muralidharan, H. Kwak, J. An, and F. Menczer. Plurule: A benchmark for moderating pluralistic communities on social media. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 344... on Pith
- [18] D. Kahneman.Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011. 28 on Pith
- [19] K. S. Kolla, H. Salunkhe, and E. Chandrasekharan. Llm-mod: Can large language models assist content moderation? InExtended Abstracts of the 2024 CHI Conference on Human Factors in Computing Systems, pages 1–7, 2024 on Pith
- [20] J. L. Kolodner.Case-Based Reasoning. Morgan Kaufmann, 1993 on Pith
show 27 more
- [21] D. Kumar, Y. AbuHashem, and Z. Durumeric. Watch your language: Investigating content moderation with large language models. InProceedings of the International AAAI Conference on Web and Social Media, volume 18, pages 865–878, 2024 on Pith
- [22] P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems, 33:9459–9474, 2020 on Pith
- [23] J. Liu, D. Shen, Y. Zhang, W. B. Dolan, L. Carin, and W. Chen. What makes good in- context examples for gpt-3? InProceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd workshop on knowledge extraction and integration for deep learning architectures, pages 100–114, 2022 on Pith
- [24] Y. Lu, M. Bartolo, A. Moore, S. Riedel, and P. Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page... on Pith
- [25] A. Majumdar, S. Paul, P. Singh, I. Abdelaziz, S. Jarollahi, S. Lee, K. P. Gummadi, I. Weber, and A. Dash. Can foundation models moderate online content? evaluating instruction-vs. example-driven policy operationalization.arXiv preprint arXiv:2609.10410, 2026 on PithPith's reviewarXiv
- [26] T. Markov, C. Zhang, S. Agarwal, F. EloundouNekoul, T.Lee, S. Adler, A. Jiang, and L. Weng. A holistic approach to undesired content detection in the real world. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 15009–15018, 2023 on Pith
- [27] B. Mathew, P. Saha, S. M. Yimam, C. Biemann, P. Goyal, and A. Mukherjee. Hatexplain: A benchmarkdatasetforexplainablehatespeechdetection. InProceedings of the AAAI conference on artificial intelligence, volume 35, pages 14867–14875, 2021 on Pith
- [28] S. Min, X. Lyu, A. Holtzman, M. Artetxe, M. Lewis, H. Hajishirzi, and L. Zettlemoyer. Re- thinking the role of demonstrations: What makes in-context learning work? InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048–11064, 2022 on Pith
- [29] N. Mukkunnoth. Laya: Multilingual System One Decision Engine, 2026. ConvAI Innovations on Pith
- [30] S. S. Mullick, M. Bhambhani, S. Sinha, A. Mathur, S. Gupta, and J. Shah. Content moderation for evolving policies using binary question answering. InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track), pages 561–573, 2023 on Pith
- [31] G. Nogara, F. Pierri, S. Cresci, L. Luceri, P. Törnberg, and S. Giordano. Toxic bias: Perspective api misreads german as more toxic. InProceedings of the international AAAI conference on web and social media, volume 19, pages 1346–1357, 2025. 29 on Pith
- [32] K. Palla, J. L. R. García, C. Hauff, F. Fabbri, A. Damianou, H. Lindström, D. Taber, and M. Lalmas. Policy-as-prompt: Rethinking content moderation in the age of large language models. InProceedings of the 2025 ACM Conference on Fairness, Accountability, and Trans- parency, pa... on Pith
- [33] J. Pan, T. Gao, H. Chen, and D. Chen. What in-context learning “learns” in-context: Disen- tangling task recognition and task learning. InFindings of the Association for Computational Linguistics: ACL 2023, pages 8298–8319, 2023 on Pith
- [34] F. Poletto, V. Basile, M. Sanguinetti, C. Bosco, and V. Patti. Resources and benchmark corpora for hate speech detection: A systematic review.Language Resources and Evaluation, 55:477–523, 2021 on Pith
- [35] S. T. Roberts.Behind the Screen: Content Moderation in the Shadows of Social Media. Yale University Press, 2019 on Pith
- [36] P. Röttger, B. Vidgen, D. Nguyen, Z. Waseem, H. Margetts, and J. Pierrehumbert. Hatecheck: Functional tests for hate speech detection models. InProceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference o... on Pith
- [37] O. Rubin, J. Herzig, and J. Berant. Learning to retrieve prompts for in-context learning. InProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2655–2671, 2022 on Pith
- [38] M. Sap, D. Card, S. Gabriel, Y. Choi, and N. A. Smith. The risk of racial bias in hate speech detection. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1668–1678, 2019 on Pith
- [39] B. Schaffner, A. N. Bhagoji, S. Cheng, J. Mei, J. L. Shen, G. Wang, M. Chetty, N. Feamster, G. Lakier, and C. Tan. “community guidelines make this the best party on the internet”: An in-depth study of online platforms’ content moderation policies. InProceedings of the 2024 CHI... on Pith
- [40] F. Schauer. Precedent.Stanford Law Review, 39(3):571–605, 1987 on Pith
- [41] A. Schmidt and M. Wiegand. A survey on hate speech detection using natural language processing. InProceedings of the Fifth International Workshop on Natural Language Processing for Social Media, pages 1–10, 2017 on Pith
- [42] B. Vidgen and L. Derczynski. Directions in abusive language training data, a systematic review: Garbage in, garbage out.PLOS ONE, 15(12):e0243300, 2020 on Pith
- [43] J. Wei, J. Wei, Y. Tay, D. Tran, A. Webson, Y. Lu, X. Chen, H. Liu, D. Huang, D. Zhou, and T. Ma. Larger language models do in-context learning differently.arXiv preprint arXiv:2303.03846, 2023 on PithPith's reviewarXiv
- [44] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, and R. Kumar. Semeval-2019 task 6: Identifying and categorizing offensive language in social media (offenseval). InProceedings of the 13th International Workshop on Semantic Evaluation, pages 75–86, 2019. 30 on Pith
- [45] Y. Zhang, M. Li, D. Long, X. Zhang, H. Lin, B. Yang, P. Xie, A. Yang, D. Liu, J. Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176, 2025 on PithPith's reviewarXiv
- [46] Z. Zhao, E. Wallace, S. Feng, D. Klein, and S. Singh. Calibrate before use: Improving few- shot performance of language models. InProceedings of the 38th International Conference on Machine Learning, pages 12697–12706. PMLR, 2021 on Pith
- [47] J. Zheng, X. Liu, M. Haque, X. Qian, G. Yang, and W. Yang. Hatemoderate: Testing hate speech detectors against content moderation policies. InFindings of the association for com- putational linguistics: NAACL 2024, pages 2691–2710, 2024. 31 Table 19: Published ModerationBench ... on Pith
Cited by 2 Pith papers
Reviewed papers on Pith that cite this one. The full citation record
- Cross-Primitive Consistency in Jev cs.CV · Oct 2026
- VisualJevBench: A Benchmark and Black-Box Attacks for Visual Typed Decision Models cs.CR · Nov 2026
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment