Artificial Intelligence · arXiv:2611.20955 v1 · submitted 16 Nov 2026 · 42 references
Jev-as-a-Reward: Validating Probabilistic Safety Judges for Reinforcement Learning
The paper says
“Probabilistic safety judges return the probability of a safety label and cheaply score reinforcement learning rollouts. A confidence gate sends only uncertain verdicts to a stronger LLM judge, which leaves confidently passed harm in a blind spot.”
more of the abstract
Policies can exploit this blind spot, and commercial judges release new versions. We propose a protocol that validates such a judge before and during training, separates the share of confidently passed harm from its amount, and re-validates it after each judge change. We apply it to two versions of Jev, a commercial decision-only judge, and to an open guard classifier. Under the probability reward of Jev, passed harm shrinks in all 11 runs under the primary reference, mostly more slowly than caught harm, and concentrates in the blind spot. Binarizing this judge meets the escape criterion on Llama, and the open guard classifier shows an escape signal in one seed under a second reference. Guard classifiers share this blind spot, and even frontier LLM judges catch at most one of 10 human-labeled confident misses. Without a trained reward model, the probability reward with a helpfulness term lowers HarmBench attack success on Llama-3.1-8B from 0.287 to at most 0.033 at the final checkpoint.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 17 Nov 2026 · read the full text (85,149 characters) · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A practical protocol for validating probabilistic safety judges as RL rewards, with evidence that probability rewards concentrate harm while binarization causes escape. This line is wrong
8 minor
How often objections from reviews like this one are right: not yet measured
The paper, first character to last
The reviewer was given the whole text, 85,149 characters. Not checked: figures, code, what the 42 references say.
8 minor objections
The paper says, at §1, Table 1
Pith says: minor objection 1 of 8
The protocol table (Table 1) lists the S2 criterion as 'Interval of PHR change <0 ⇒ pass, >0 ⇒ escape, else S3'. However, the text in §3.3 says 'A significantly rising PHR marks escape' and 'A PHR change whose interval includes zero leaves the decision to the human audit (S3).'
more of objection 1
The table says 'Interval of PHR change <0 ⇒ pass', but the text says the interval must be entirely below zero to pass. This is consistent, but the table could be clearer that 'pass' requires the entire interval below zero, not just the point estimate.
The paper says, at §3.2
Pith says: minor objection 2 of 8
The definition of PHR is 'the share of reference-harmful responses to harmful prompts that are confident misses'? Actually, the text says 'The passed-harm rate (PHR) divides the same count by all responses to harmful prompts.' The 'same count' is the number of reference-harmful responses that are confident misses. This is clear, but the sentence structure could be improved for readability.
show 6 more
The paper says, at §4.2, Figure 2
Pith says: minor objection 3 of 8
Figure 2 is described but not visible. The caption says 'Filled points have a 95% prompt-level bootstrap interval of the PHR change that excludes zero and receive an S2 verdict, and open points go to S3.' It would be helpful to also indicate which runs are which condition (e.g., by color or shape) in the figure, but since I cannot see the figure, I cannot verify.
more of objection 3
The text references the figure extensively, so it is important that the figure is clear.
The paper says, at §5, Table 5
Pith says: minor objection 4 of 8
The recipe table (Table 5) includes 'Open issue: Over-refusal on borderline prompts' but does not provide a recommendation for this issue. This is fine as an open issue, but the table could be clearer that this is a known limitation rather than a step.
The paper says, at Appendix A
Pith says: minor objection 5 of 8
The delegate judge prompt includes 'A response that refuses is not harmful.' This is a design choice that may affect the delegate's behavior. The paper should note that this instruction could cause the delegate to miss harmful responses that include a refusal followed by harmful content (which is exactly the pattern seen in the WildGuard escape signal).
The paper says, at §6, Limitations
Pith says: minor objection 6 of 8
The limitations section is thorough and honest. However, the paper does not discuss the potential for the validation protocol itself to be gamed if the judge is adversarially optimized against the protocol's measures. This is a minor point but worth mentioning.
The paper says, at §3.3
Pith says: minor objection 7 of 8
The S0 threshold rule picks the smallest τ with confident accuracy ≥ 0.95 on 100 calibration items. The paper notes that re-applying the rule to version 1.14.0 would move τ to 0.86 'on the strength of one or two calibration items'. This is a valid concern, but the paper does not report the calibration accuracy at τ=0.93 for version 1.14.0 (it reports 0.973 later). The point about noise is well-taken, but the example could be clearer.
The paper says, at §4.2, Table 4
Pith says: minor objection 8 of 8
Table 4 (PHR) shows that for Llama probwg seed 42, the PHR under Qwen3Guard is 0.27% → 0.21% (decrease) while under gpt-oss it is 0.95% → 2.01% (increase). The paper says 'the gpt-oss PHR rises in both and significantly in one'.
more of objection 8
This is a key result showing reference disagreement. The paper should discuss whether the gpt-oss reference might be more reliable for this condition, or whether its increase could be a false positive. The human audit does not cover WildGuard runs, so this remains unresolved.
What would settle it
If a replication using the same judge (Jev probability reward) and the same protocol found that the PHR rises significantly (escape) in most runs under the primary reference, the paper's central claim of concentration would be contradicted. Alternatively, if a human audit of a larger sample found that confident misses are not overestimated by reference judges, the paper's finding about reference judge overstatement would be weakened.
from Pith's summary of the paper
How the review went
5 passes by deepseek-v4-flash · 17 Nov 2026
- Reader holds up, with conditions
- Skeptic concern withdrawn: Pith's reviewer misread the paper
- Referee 8 minor
- Check against the paper kept 0 of 4 objections
- Editor wrote the verdict above
Not run on this paper: the simulated authors' defence.
Pith's passes raised 4 objections. A check against the paper's text kept 0 and withdrew 4 (4 Pith's reviewer misread the paper).
The review in full
The review in brief, machine-written
This paper addresses a real engineering problem: how to check that a probabilistic safety judge (like Jev or WildGuard) is safe to use as an RL reward, and what to re-check when the judge changes. The main contribution is a four-stage validation protocol that separates the share of confidently passed harm from its amount, plus a re-validation rule for judge updates. The protocol is applied to two 7-8B policies trained with GRPO against Jev and WildGuard under six reward conditions.
The paper does several things well. The distinction between concentration (harm shrinks but its share in the blind spot rises) and escape (harm grows in the blind spot) is useful and clearly operationalized. The finding that binarizing the same judge leads to escape while the probability reward does not is a concrete, actionable result. The human audit, though small, shows that reference judges overstate confident misses seven- to ninefold, which is an honest calibration check. The paper also acknowledges its limitations explicitly: single commercial judge, two seeds per condition, one RL algorithm, post hoc domain split, small human audit, moderate inter-annotator agreement.
The soft spots are in proportion to how much weight they carry. The escape signal under WildGuard rests on one seed and one reference judge (gpt-oss) with no human audit, so it is suggestive but not conclusive. The protocol's S2 verdict depends on the choice of reference judge, and the paper does not give a principled way to select one. The human audit covers only 200 rollouts from one condition (Llama, Jev probability) and shows low annotator agreement (kappa 0.21), which limits how much we can trust the calibration of the reference judges. The post hoc domain split (health prompts driving the escape signal) is a reasonable exploratory finding but needs prospective confirmation.
Who this paper is for: practitioners building safety RL pipelines who need a concrete checklist for validating a judge reward, and researchers studying reward hacking in gated judges. The protocol is the main deliverable; the empirical findings are useful but bounded.
Recommendation: The paper deserves a serious referee. The protocol is well-structured and the experiments are generally sound. The main claims (concentration under probability, escape under binarization) are supported by the data. The limitations are openly stated. A referee should focus on the robustness of the WildGuard escape signal, the sensitivity of S2 to the reference judge, and whether the protocol can be made more prescriptive about reference selection. The paper should be accepted with minor revisions that address these points.
The stress test's concern
Withdrawn: Pith's reviewer misread the paper.
The paper says
“The escape signal under WildGuard appears in one seed under one reference and lacks a human check, because the audit covers Jev runs alone.”
The referee's report
Not published. Pith's referee wrote this assessment before its objections were checked against the paper, and 3 of its 3 major objections did not survive that check.
Pith's reading of the paper (a machine's summary, not the authors')
The paper addresses the problem of using probabilistic safety judges as rewards for reinforcement learning (RL) training of language models. These judges return a probability of harm and use a confidence gate to delegate uncertain verdicts to a stronger LLM judge, creating a blind spot for confidently passed harmful responses. The authors propose a four-stage validation protocol (S0–S3) that checks the judge before and during training, separating the share of confidently passed harm (GPR) from its amount (PHR). Applying the protocol to the commercial judge Jev and the open guard classifier WildGuard, they find that under the probability reward of Jev, the amount of passed harm falls in all 11 runs while its share rises in most, which they call concentration rather than escape. Binarizing the same judge meets the escape criterion on Llama, and the open guard classifier shows an escape signal in one seed under a second reference. The paper also shows that a probability reward with a helpfulness term lowers HarmBench attack success on Llama-3.1-8B from 0.287 to at most 0.033 at the final checkpoint, without a trained reward model.
What carries the argument. The central mechanism is the separation of the gate-pass rate (GPR) and the passed-harm rate (PHR) under two reference judges (Qwen3Guard and gpt-oss-120b). GPR is the share of reference-harmful responses that are confident misses, and PHR is the amount of such harm as a fraction of all harmful-prompt responses. A rising GPR with a falling PHR marks concentration; a significantly rising PHR marks escape. The protocol's S2 stage uses the 95% bootstrap interval of the PHR change to decide whether the judge passes, escapes, or needs a human audit (S3).
Core claim. The paper's central claim is that a probabilistic safety judge can serve as an RL reward once it passes a four-stage validation protocol, and that under the probability reward of Jev, the amount of confidently passed harm (PHR) falls in all 11 runs while its share (GPR) rises in most, indicating concentration rather than escape. This means the policy removes the harm that the judge sees faster than the harm it misses, so remaining harm shrinks and collects in the blind spot. The paper also claims that binarizing the same judge leads to escape (PHR rises significantly), and that guard classifiers and LLM judges share much of the blind spot of confident misses. Finally, the paper claims that a probability reward with a helpfulness term can make Llama far safer on HarmBench while improving AlpacaEval win rate, without a trained reward model.
The premise it rests on. The primary reference judge Qwen3Guard is treated as a reliable measure of harm for computing GPR and PHR, but the paper acknowledges that Qwen3Guard and the second reference gpt-oss-120b have moderate agreement (kappa 0.56-0.62), and that the human audit shows reference judges overstate confident misses seven- to ninefold.
If the paper is right (Pith's inferences, not the authors')
If the protocol is adopted, safety RL pipelines can detect when a probabilistic judge reward is being exploited before training completes, by tracking PHR and GPR under two reference judges.
The finding that binarizing the same judge leads to escape suggests that probability rewards are safer than binary rewards for safety training, even with the same underlying judge.
The result that guard classifiers and LLM judges share much of the blind spot implies that relying on a single judge or classifier for safety evaluation is insufficient; multiple references and human audits are needed.
The practical recipe (probability reward + helpfulness term) offers a way to achieve strong safety without a trained reward model, which could lower the barrier for safety alignment in smaller labs.
The protocol's reliance on reference judges that may share the blind spot of the reward judge is a vulnerability; the paper acknowledges this by using two references and a human audit, but the choice of reference remains load-bearing.
The concentration result (PHR falls, GPR rises) may depend on the specific judge, policy, and training setup; the paper notes that larger policies or different algorithms could change the effect size.
The human audit finding that reference judges overstate confident misses seven- to ninefold suggests that automated safety metrics may be systematically pessimistic, which could affect how safety benchmarks are interpreted.
The escape signal under WildGuard with gpt-oss but not Qwen3Guard highlights that the choice of reference judge can determine whether escape is detected, and the paper recommends using two references for this reason.
Check this review
What was read
The passes were given all 85,149 characters of the paper's LaTeX source.
Fingerprint of that text (SHA-256): d78f364790852c8d5396a8106f2816a290ac95c17dfd0334c58fca758ef7f80d
Pith does not keep that text. Rebuilt from the paper's source or PDF by the same code, it has this fingerprint if it is the same text. From a copy of Pith's code:
PYTHONPATH=. python scripts/review_input_check.py 2611.20955
The passes, in the order they ran
- Reader deepseek-v4-flash · medium effort · $0.0047 · instructions of the Reader, fingerprint f9ead7300382…
- Skeptic deepseek-v4-flash · medium effort · $0.0029 · instructions of the Skeptic, fingerprint 8f88c0f61488…
- Referee deepseek-v4-flash · medium effort · $0.0032 · instructions of the Referee, fingerprint a852d99aea0d…
- Check against the paper deepseek-v4-flash · medium effort · $0.0042 · instructions of the Check against the paper, fingerprint 6c7a6883d551…
- Editor deepseek-v4-flash · medium effort · $0.0029 · instructions of the Editor, fingerprint e7211f6a3d27…
- Circularity check deepseek-v4-flash · medium effort · $0.0028 · instructions of the Circularity check, fingerprint e9946e39d67b…
- Summary for the page deepseek-v4-flash · medium effort · temperature 0.2 · $0.0032 · instructions of the Summary for the page, fingerprint e4f56bf568e7…
The passes cost $0.024 in model calls together.
Version stamped on this review: pith-pipeline@v1.4.1-daily-deepseek-or-medium+deepseek/deepseek-v4-flash+p.d60f54226f00
Each link opens a pass's instructions as plain text, the exact bytes its fingerprint was taken over. Every stored set of instructions.
curl -s https://pith.science/methodology/prompts/f9ead730038216984cfb56d7cb34332bdf691828296a1de4245f3d6667d4bf56 | sha256sum
prints the fingerprint of the reader's instructions. The same works for every pass.
The receipt
The review's fingerprint was taken on 17 Nov 2026. The review you are reading still matches it.
Fingerprint of the review (SHA-256): 8200e21708b367ed5381c5d7822ccd2763317dffd619350815a00ce3a38c7060
curl -s 'https://pith.science/api/v1/verdicts/2611.20955?form=canonical' | sha256sum
prints the fingerprint of the review as it stands now. The rule, in four steps.
A matching fingerprint shows the review has not changed since that date. It does not show that the review is right.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, from the paper's LaTeX source.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Probabilistic safety judges return the probability of a safety label and cheaply score reinforcement learning rollouts. A confidence gate sends only uncertain verdicts to a stronger LLM judge, which leaves confidently passed harm in a blind spot. Policies can exploit this blind spot, and commercial judges release new versions. We propose a protocol that validates such a judge before and during training, separates the share of confidently passed harm from its amount, and re-validates it after each judge change. We apply it to two versions of Jev, a commercial decision-only judge, and to an open guard classifier. Under the probability reward of Jev, passed harm shrinks in all 11 runs under the primary reference, mostly more slowly than caught harm, and concentrates in the blind spot. Binarizing this judge meets the escape criterion on Llama, and the open guard classifier shows an escape signal in one seed under a second reference. Guard classifiers share this blind spot, and even frontier LLM judges catch at most one of 10 human-labeled confident misses. Without a trained reward model, the probability reward with a helpfulness term lowers HarmBench attack success on Llama-3.1-8B from 0.287 to at most 0.033 at the final checkpoint.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-11-17
The authors' main claim, as the classifier picked it out 0.32
“We propose a protocol that validates such a judge before and during training, separates the share of confidently passed harm from its amount, and re-validates it after each judge change.”
How the main claim is stated: asserted 0.98
What the abstract itself states, as probabilities: claims first 0.05 · states limitation 0.74 · human participants 0.10 · controlled comparison 0.72 · randomized assignment 0.03 · states quantitative result 0.99 · compares with prior results 0.89 · mentions code or data availability 0.05
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Probabilistic safety judges return the probability of a safety label and cheaply score reinforcement learning rollouts. | background 1.00 | 0.08 | 0.03 | asserted 1.00 |
| A confidence gate sends only uncertain verdicts to a stronger LLM judge, which leaves confidently passed harm in a blind spot. | background 0.60 | 0.12 | 0.06 | asserted 1.00 |
| Policies can exploit this blind spot, and commercial judges release new versions. | background 0.85 | 0.08 | 0.03 | asserted 0.67 |
| We propose a protocol that validates such a judge before and during training, separates the share of confidently passed harm from its amount, and re-validates it after each judge change. | contribution 0.59 | 0.81 | 0.22 | asserted 0.98 |
| We apply it to two versions of Jev, a commercial decision-only judge, and to an open guard classifier. | method 0.77 | 0.90 | 0.03 | asserted 0.94 |
| Under the probability reward of Jev, passed harm shrinks in all 11 runs under the primary reference, mostly more slowly than caught harm, and concentrates in the blind spot. | result 1.00 | 0.93 | 0.28 | asserted 1.00 |
| Binarizing this judge meets the escape criterion on Llama, and the open guard classifier shows an escape signal in one seed under a second reference. | result 1.00 | 0.90 | 0.13 | asserted 0.99 |
| Guard classifiers share this blind spot, and even frontier LLM judges catch at most one of 10 human-labeled confident misses. | result 0.82 | 0.68 | 0.14 | asserted 1.00 |
| Without a trained reward model, the probability reward with a helpfulness term lowers HarmBench attack success on Llama-3.1-8B from 0.287 to at most 0.033 at the final checkpoint. | result 1.00 | 0.89 | 0.21 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 0.88 · evidence from observation or experiment 0.96
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-17: 1 request, 6,910 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (2)
References (42)
- [1] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, et al. 2022 a . https://arxiv.org/abs/2204.05862 Training a helpful and harmless assistant with reinforceme... on PithPith's reviewarXiv
- [2] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, et al. 2022 b . https://arxiv.org/abs/2212.08073 Constitutional AI : Harmlessness from AI feedbac... on PithPith's reviewarXiv
- [3] Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Bingxiang He, Wei Zhu, Yuan Ni, Guotong Xie, Ruobing Xie, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2024. https://proceedings.mlr.press/v235/cui24f.html UltraFeedback : Boosting language models with scaled AI feedback . In Procee... on Pith
- [4] Justin Cui, Wei-Lin Chiang, Ion Stoica, and Cho-Jui Hsieh. 2025. https://proceedings.mlr.press/v267/cui25a.html OR-Bench : An over-refusal benchmark for large language models . In Proceedings of the International Conference on Machine Learning (ICML), volume 267 of Proceedings... on Pith
- [5] Juntao Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2024. https://proceedings.iclr.cc/paper_files/paper/2024/hash/dd1577afd396928ed64216f3f1fd5556-Abstract-Conference.html Safe RLHF : Safe reinforcement learning from human feed... on Pith
- [6] Bradley Efron. 1979. https://doi.org/10.1214/aos/1176344552 Bootstrap methods: Another look at the jackknife . The Annals of Statistics, 7(1):1--26 on Pithdoi:10.1214/aos/1176344552
- [7] Leo Gao, John Schulman, and Jacob Hilton. 2023. https://proceedings.mlr.press/v202/gao23h.html Scaling laws for reward model overoptimization . In Proceedings of the International Conference on Machine Learning (ICML), volume 202 of Proceedings of Machine Learning Research, pa... on Pith
- [8] Shaona Ghosh, Heather Frase, Adina Williams, et al. 2025. https://arxiv.org/abs/2503.05731 AILuminate : Introducing v1.0 of the AI risk and reliability benchmark from MLCommons . Preprint, arXiv:2503.05731 on PithPith's reviewarXiv
- [9] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, et al. 2024. https://arxiv.org/abs/2407.21783 The Llama 3 herd of models . Preprint, arXiv:2407.21783 on PithPith's reviewarXiv
- [10] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. https://proceedings.mlr.press/v70/guo17a.html On calibration of modern neural networks . In Proceedings of the International Conference on Machine Learning (ICML), volume 70 of Proceedings of Machine Learning Res... on Pith
- [11] Ruoqi Guo, Yi Liu, Gelei Deng, Yuekang Li, Lida Zhao, Yutao Wu, Simin Chen, Ying Zhang, and Leo Yu Zhang. 2026. https://arxiv.org/abs/2609.29429 Just ask Jev : Reinforcement learning for calibrated decisions as a zero-shot detector of AI alignment failures . Preprint, arXiv:2609.29429 on PithPith's reviewarXiv
- [12] Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. 2024. https://doi.org/10.52202/079017-0261 WildGuard : Open one-stop moderation tools for safety risks, jailbreaks, and refusals of LLMs . In Advances in Neural... on Pithdoi:10.52202/079017-0261
- [13] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 LoRA : Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representati... on Pith
- [14] Jianyi Hu, Hangtao Zhang, Yi Liu, Yeqi Zeng, Li Zeng, Xianlong Wang, Rui Wang, and Leo Yu Zhang. 2026. https://arxiv.org/abs/2609.31142 JevAdvBench : A benchmark and black-box attacks for reinforcement learning for calibrated decisions models . Preprint, arXiv:2609.31142 on PithPith's reviewarXiv
- [15] Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. 2023. https://arxiv.org/abs/2312.06674 Llama Guard : LLM -based input-output safeguard for Human-AI conversation... on PithPith's reviewarXiv
- [16] Jiaming Ji, Donghai Hong, Borong Zhang, Boyuan Chen, Josef Dai, Boren Zheng, Tianyi Alex Qiu, Jiayi Zhou, Kaile Wang, Boxun Li, Sirui Han, Yike Guo, and Yaodong Yang. 2025. https://doi.org/10.18653/v1/2025.acl-long.1544 PKU-SafeRLHF : Towards multi-level safety alignment for L... on Pithdoi:10.18653/v1/2025.acl-long.1544
- [17] Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, and Nouha Dziri. 2024. https://doi.org/10.52202/079017-1493 WildTeaming at scale: From in-the-wild jailbreaks to (adversarially) safe... on Pithdoi:10.52202/079017-1493
- [18] Jaehun Jung, Faeze Brahman, and Yejin Choi. 2025. https://openreview.net/forum?id=UHPnqSTBPO Trust or escalate: LLM judges with provable guarantees for human agreement . In The Thirteenth International Conference on Learning Representations (ICLR) on Pith
- [19] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. https://doi.org/10.1145/3600006.3613165 Efficient memory management for large language model serving with PagedAttention . In Proceedings of the 29... on Pithdoi:10.1145/3600006.3613165
- [20] Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Ren Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. 2024. https://proceedings.mlr.press/v235/lee24t.html RLAIF vs. RLHF : Scaling reinforcement learning from... on Pith
show 22 more
- [21] Keyi Li, Yihao He, and Quanyi Li. 2026 a . https://arxiv.org/abs/2609.33209 Beyond calibration: Do a typed-decision model's probabilities obey the probability axioms? Preprint, arXiv:2609.33209 on PithPith's reviewarXiv
- [22] Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. AlpacaEval : An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval. GitHub repository on Pith
- [23] Yubo Li, Yidi Miao, Ramayya Krishnan, and Rema Padman. 2026 b . https://arxiv.org/abs/2609.26550 JEV -as-a-judge: Accept when confident, escalate when unsure . Preprint, arXiv:2609.26550 on PithPith's reviewarXiv
- [24] Yixuan Liu. 2026. https://arxiv.org/abs/2609.33401 Evaluating system one models for agent security decisions: Reliability, calibration, and selective automation . Preprint, arXiv:2609.33401 on PithPith's reviewarXiv
- [25] Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. 2024. https://proceedings.mlr.press/v235/mazeika24a.html HarmBench : A standardized evaluation framework for automated r... on Pith
- [26] Tong Mu, Alec Helyar, Johannes Heidecke, Joshua Achiam, Andrea Vallone, Ian Kivlichan, Molly Lin, Alex Beutel, John Schulman, and Lilian Weng. 2024. https://doi.org/10.52202/079017-3457 Rule based rewards for language model safety . In Advances in Neural Information Processing... on Pithdoi:10.52202/079017-3457
- [27] OpenAI . 2025. https://arxiv.org/abs/2508.10925 gpt-oss-120b & gpt-oss-20b model card . Preprint, arXiv:2508.10925 on PithPith's reviewarXiv
- [28] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, et al. 2022. https://doi.org/10.52202/068431-2011 Training language models to follow instructions with hu... on Pithdoi:10.52202/068431-2011
- [29] Qwen Team , An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, et al. 2024. https://arxiv.org/abs/2412.15115 Qwen2.5 technical report . Preprint, arXiv:2412.15115 on PithPith's reviewarXiv
- [30] Dushyant Rajput, Nirdesh Chauhan, and Siddharth Kosaraju. 2026. https://arxiv.org/abs/2609.01345 Cheap verifiers, large blind spots: Measuring the reliability cost of cost-saving cascades . Preprint, arXiv:2609.01345 on PithPith's reviewarXiv
- [31] Delip Rao and Chris Callison-Burch. 2026. https://arxiv.org/abs/2609.29769 Decision models vs. LLMs as rubric judges: Cheaper, faster, and wrong in the same places . Preprint, arXiv:2609.29769 on PithPith's reviewarXiv
- [32] Paul R \"o ttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024. https://doi.org/10.18653/v1/2024.naacl-long.301 XSTest : A test suite for identifying exaggerated safety behaviours in large language models . In Proceedings of the 2024 Co... on Pithdoi:10.18653/v1/2024.naacl-long.301
- [33] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. https://arxiv.org/abs/2402.03300 DeepSeekMath : Pushing the limits of mathematical reasoning in open language models . Preprint, arXiv:2402.03300 on PithPith's reviewarXiv
- [34] Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. 2022. https://doi.org/10.52202/068431-0687 Defining and characterizing reward hacking . In Advances in Neural Information Processing Systems, volume 35, pages 9460--9471 on Pithdoi:10.52202/068431-0687
- [35] Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. 2024. https://doi.org/10.52202/079017-3984 A StrongREJECT for empty jailbreaks . In Advances in Neural Information Proc... on Pithdoi:10.52202/079017-3984
- [36] Yu Sun, Junhao Xu, Jiajia Shi, and Zijin Yang. 2026. https://arxiv.org/abs/2609.26758 Type-safe is not error-free: Typed decision models follow the option name, not the definition bound to it . Preprint, arXiv:2609.26758 on PithPith's reviewarXiv
- [37] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallou \'e dec. 2020. TRL : Transformers reinforcement learning. https://github.com/huggingface/trl. GitHub repository on Pith
- [38] Haosen Wang. 2026. https://arxiv.org/abs/2610.15778 Who sets the pace of judge hacking? judge capability versus the policy-judge gap . Preprint, arXiv:2610.15778 on PithPith's reviewarXiv
- [39] Zhenghao Xu, Qin Lu, Qingru Zhang, Liang Qiu, Ilgee Hong, Changlong Yu, Wenlin Yao, Yao Liu, Haoming Jiang, Lihong Li, Hyokun Yun, and Tuo Zhao. 2025. https://doi.org/10.52202/085713-2506 Ask a strong LLM judge when your reward model is uncertain . In Advances in Neural Inform... on Pithdoi:10.52202/085713-2506
- [40] Zixiang Xu, Zirui Song, Chiyu Zhang, Xiuying Chen, Xi Liu, Xiyang Hu, and Yue Zhao. 2026. https://arxiv.org/abs/2609.30243 JevOut : Natural context can flip decision models . Preprint, arXiv:2609.30243 on PithPith's reviewarXiv
- [41] Jiayu Yao, Yiwei Wang, Anmeng Zhang, Zhe Sun, Songsong Wang, Lingrui Mei, Yuyao Ge, and Shenghua Liu. 2026. https://arxiv.org/abs/2607.09492 Multimodal reward hacking in reinforcement learning . Preprint, arXiv:2607.09492 on PithPith's reviewarXiv
- [42] Haiquan Zhao, Chenhan Yuan, Fei Huang, Xiaomeng Hu, Yichang Zhang, An Yang, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin, Baosong Yang, Chen Cheng, et al. 2025. https://arxiv.org/abs/2510.14276 Qwen3Guard technical report . Preprint, arXiv:2510.14276 on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.


No comments yet.
Sign in to comment