Pith. sign in

REVIEW 3 major objections 4 minor 26 references

The paper claims that a black-box LLM's safety–helpfulness trade-off can be resolved at inference time by solving a small linear program over candidate answers, with no retraining and no access to model internals.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 10:28 UTC pith:CFYXZRRT

load-bearing objection A promising black-box safety reranker with strong SafetyBench results, but the paper's central safety guarantee is false: the optimization actually solved is a soft-penalty reranker, and the key proposition has a direct counterexample. the 3 major comments →

arxiv 2510.09330 v3 pith:CFYXZRRT submitted 2025-10-10 cs.LG

Safety Game: Inference-Time Alignment of Black-Box LLMs via Constrained Optimization

Tuan Nguyen , Long Tran-Thanh This is my paper
classification cs.LG
keywords inference-time alignmentblack-box LLMsafety alignmenthelpfulnesszero-sum gamelinear programmingconstrained optimizationmultiple-choice QA
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to prove that safety alignment of a proprietary, black-box LLM is feasible at inference time, without retraining or any internal access. It models the safety-helpfulness dilemma as a two-player zero-sum game: the model picks a distribution over candidate answers, and the minimax equilibrium of that game is computed by a small linear program. The LP maximizes expected helpfulness subject to a per-prompt risk cap relative to a safe fallback, so the worst case is always the safe no-information answer. On three standard multiple-choice safety benchmarks the method outperforms existing reranking baselines in most comparisons, with the largest gains on the biggest benchmark. If the claim is right, any third party with API access could enforce a safety cap on a closed model by wrapping it in a solver.

Core claim

The paper's central claim is that the safety–helpfulness dilemma can be cast as a two-player zero-sum game whose minimax equilibrium, computed by a linear program at inference time, gives a per-prompt safety guarantee: the chosen mixed strategy over candidate answers is never expected-riskier than always returning a safe fallback, but it can still deliver useful answers whenever a candidate stays within the risk budget. Helpfulness and risk scores are obtained by asking the frozen black-box model itself binary YES/NO questions—'Is this answer helpful?' and 'Is this answer risky?'—and converting the log-likelihoods into normalized scores. The LP then maximizes expected helpfulness lift subjec

What carries the argument

The load-bearing object is a small constrained optimization: maximize Σ π_i M_i subject to Σ π_i Δ_i ≤ T, where M_i is a candidate's helpfulness lift over a safe fallback answer, Δ_i is its extra risk, and T is a per-prompt risk cap. The scores come from binary probes on the same black-box model, normalized log-likelihoods of YES versus NO. The paper reformulates this as a minimax problem with a bounded multiplier, then uses a sigmoid penalty to avoid cap-hugging and boundary sensitivity. The safe fallback r_s anchors the guarantee: because it has zero helpfulness and zero extra risk by construction, any feasible mixture is never worse than emitting the fallback.

Load-bearing premise

The whole safety cap depends on the model's own YES/NO answers to 'Is this answer risky?' being a trustworthy measure of real harm; if those answers are miscalibrated or blind to certain harms, the LP's cap constrains only a fiction.

What would settle it

Find one prompt where the probe says a known-dangerous answer is safe (for example, by adversarial phrasing or using a model whose risk probe cannot recognize the harm), and show that Safety Game's selected answer is judged harmful by an independent human or external audit; that single counterexample refutes the per-prompt safety guarantee.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Any party with API access can enforce a per-prompt safety cap on a closed model by wrapping it with an LP solver—no retraining, no weights, no architecture knowledge.
  • The selected response is guaranteed, in expectation, to be no riskier than always giving the safe fallback, even under the worst-case reading of the user's intent.
  • New safety rules or stricter policies can be imposed by changing the cap T or the probe wording, which is far cheaper than fine-tuning.
  • The same machinery applies to any finite candidate pool, including sampled drafts, retrieved passages, or tool-call options in deployed systems.
  • On the largest benchmark, the method beats all six reranking baselines on four of five models, with large gains that survive a standard paired-significance test.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: Because the scores are the model's own self-assessments, the method's safety guarantee is really a cap on self-reported risk; plugging an external calibrated safety classifier into the same LP would extend the guarantee to independently measured harm.
  • Editorial inference: The finite-candidate setup is less restrictive than it looks—any best-of-N or retrieval-augmented pipeline already produces a finite pool—so the main obstacle to open-ended use is generating diverse candidates that include a safe, useful option.
  • Editorial inference: The cap T and the dual bound β could be set by a deployer or regulator as policy knobs, meaning 'how safe' becomes a configurable per-prompt parameter rather than a fixed training objective.
  • Editorial inference: The guarantee is about the expected risk of the mixture, not the risk of a single sampled answer; deployments that require a hard per-response guarantee would need to pair the LP with a deterministic final fallback decision.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes 'Safety Game', an inference-time, black-box alignment method for LLMs in multiple-choice QA. For each prompt, candidate answers are scored by two binary probes (helpfulness and risk) applied to the same frozen LLM; the method then solves a linear program that maximizes expected helpfulness subject to a per-prompt cap on expected extra risk, with a safe fallback answer as the zero-risk baseline. The LP is reformulated first with a bounded dual multiplier and then with a sigmoid penalty, and this is interpreted as the value of a two-player zero-sum game. Experiments on HHH, TruthfulQA, and SafetyBench compare the method with Consensus-Game-style rerankers and report that it outperforms baselines in 11 of 15 test cases, with particularly large gains on SafetyBench. The central claim is that the method enforces a hard per-prompt safety guarantee while remaining competitive on helpfulness.

Significance. If the central claim were correct, the paper would make a useful contribution: a training-free, API-only method for enforcing a per-prompt safety cap would be practically relevant for third-party deployers. The paper deserves credit for a transparent LP-based selection procedure, a clearly specified safe-fallback construction, and careful statistical reporting on SafetyBench. However, the load-bearing theoretical guarantee is not established. Proposition 3.1 is false as stated, the bounded-multiplier and sigmoid objectives are not equivalent to the hard-cap LP, and the experiments use the soft sigmoid penalty. The empirical results may still be interesting as a soft-penalty reranker, but they do not support the claimed safety guarantee. Since the advertised contribution is precisely that guarantee, the manuscript in its current form cannot be accepted.

major comments (3)
  1. [Section 3.3, Proposition 3.1 / Appendix B.1] Proposition 3.1 is false. Take two candidates: the fallback (M=0, Δ=0) and a candidate with M_j=100, Δ_j=1, with T=0 and β=1. The optimum of (3.2) is the pure candidate, whose objective is min_{0≤μ≤1}[100 - μ(1)] = 99, giving R(π)=1 > T, not R(π)=T. The proof in Appendix B.1 assumes that mass can be moved from the fallback to candidate j, but the optimal π need not put mass on the fallback. This directly invalidates the statement that optimizing (3.2) enforces the cap in (3.1).
  2. [Sections 3.3 and 3.4] Problem (3.2) is a Lagrangian relaxation with the dual multiplier bounded by β, not an equivalent reformulation of (3.1) unless β is at least the optimal dual multiplier. For any finite β, a candidate with M_j > β Δ_j can be chosen even when it violates the cap by a large amount. The sigmoid objective (3.3) is explicitly soft, and the paper itself concedes that it 'does not enforce a hard cutoff' and may allow a small increase in expected risk above T. Since the main experiments (Section 4.3) use the sigmoid with β=10, the reported results do not demonstrate a hard safety cap. The statement in Section 3.4 that optimizing (3.4) 'yields the same solution as the hard-cap program (3.1)' is therefore unsupported.
  3. [Section 3.1] The scores h_i and s_i are obtained by asking the same black-box model to evaluate its own candidate answers with binary YES/NO probes. The risk cap T therefore constrains only the model's self-reported risk score, not any independently validated notion of actual harm. If the model's self-assessments are miscalibrated or it cannot recognize harmful content in its own outputs, the 'safety guarantee' does not constrain real-world risk. No external safety classifier or calibration analysis is provided, so this premise is load-bearing for the safety claims and is not supported.
minor comments (4)
  1. [Algorithm 1] Step 4 appears to contain a typo: the safety score uses pyes_h in the denominator, so it should presumably be pyes_s + pno_s. Also, the surrounding text calls this 'log-odds' but the formula in Section 3.1 is a normalized log probability; please align the terminology.
  2. [Section 3.4] The paragraph says the outcome of the chance event is 'known to the model', yet the model is simultaneously required to commit to a single π shared across both branches. If the mode is known, a rational player would condition π on the mode. Clarify whether this is purely an algebraic reinterpretation or a genuine game-theoretic model; as written, the strategic interpretation is unclear.
  3. [Appendix C, Tables 8 and 9] Table 9 is said to report HHH (N≈221), but the main text states HHH has 200 items. The discrepancy should be reconciled. Also, several HHH and TruthfulQA comparisons are statistically tied, which is acknowledged in the text; consider making that more prominent in the abstract-level claims.
  4. [General] The paper claims '11 of 15 test cases' as top performance, but in several rows of Tables 1 and 2 the differences are within noise. Please state the number of statistically significant wins separately from raw best-of-table counts.

Circularity Check

0 steps flagged

No significant circularity: the game framing is an explicit rescaling of the LP, and the benchmark results are externally anchored.

full rationale

The derivation chain is: binary probes define h_i and s_i, margins M_i and Δ_i are formed relative to a safe fallback, the constrained LP (3.1) maximizes helpfulness subject to a risk cap, (3.2) introduces a bounded multiplier, (3.3) replaces the linear penalty with a sigmoid, and (3.4) rescales the objective into a two-branch form. Section 3.4 then interprets (3.4) as a two-player zero-sum game, explicitly stating that the branch weights 'exactly match the coefficients in (3.4)' and that solving (3.4) 'yields the same solution as the hard-cap program (3.1).' This is a deliberate algebraic equivalence, not a hidden reduction of a prediction to a fitted input: the game is constructed from the LP, so its equilibrium is the LP solution by design. No test labels are used to fit the method's parameters, and the empirical comparisons on HHH, TruthfulQA, and SafetyBench are assessed against external ground truth, so the headline accuracy results are not circular. The safety guarantee is weakened by the paper's own admission that the sigmoid 'does not enforce a hard cutoff' and by the self-reported nature of the probe scores; these are correctness and validity concerns, not instances of a prediction being equivalent to its inputs by construction. Proposition 3.1 also appears to contain a flawed proof, but again this is a mathematical-error issue rather than circularity. There are no load-bearing self-citations and no fitted parameters renamed as predictions. Overall, the paper is self-contained against external benchmarks and does not exhibit significant circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical or formal entities; the S1/S2 modes, the chance split, and the multiplier λ are interpretive constructs used to recast the LP as a game. The main external inputs the reader must accept are the probe-score reliability, the hand-set hyperparameters (T, β, κ), and the soft-cap approximation.

free parameters (4)
  • Risk cap T = T=1.0 (TruthfulQA); tuned per task on a dev slice
    Controls the safety-helpfulness tradeoff; the paper's 'per-prompt safety cap' depends on this hand-set value, and the guarantee is only relative to it.
  • Dual upper bound β = β=10
    Scales the safety penalty and the probability of the safety-enforcement branch; chosen by hand without sensitivity-based justification beyond the ablation.
  • Sigmoid steepness κ (reported as α=30) = α=30
    Sets the softness/sharpness of the sigmoid penalty near the cap; directly changes whether the hard cap is approximated.
  • Candidate sampling hyperparameters = temperature=0.7, top-p=0.9, repetition penalty=1.1, k=10 (TruthfulQA)
    Determines which candidate answers exist; the method cannot select a balanced response if the generator never produces one.
axioms (5)
  • domain assumption Normalized YES/NO log-likelihoods from the black-box probes measure true helpfulness and safety risk.
    Section 3.1 defines h_i and s_i from probe log-likelihoods; all later claims about safety caps depend on these scores being meaningful.
  • domain assumption The candidate set R contains a safe fallback r_s with zero helpfulness and zero risk relative to itself, and contains a balanced answer.
    The method's feasibility and its 'never worse than fallback' argument rely on r_s having M_s=Δ_s=0 and on the candidate pool containing a good safe-and-helpful option.
  • standard math LP duality and the bounded-multiplier reformulation (3.2) are equivalent to the hard-cap LP (3.1).
    Invoked in Section 3.3, but this equivalence is not generally true for bounded β; the proof is absent and Proposition 3.1 is false.
  • ad hoc to paper The chance split (1/(β+1), β/(β+1)) represents the user's benign/adversarial intent distribution.
    Introduced in Section 3.4 to reinterpret the LP as a two-branch game; the probabilities are chosen to match the objective, not measured or independently motivated.
  • ad hoc to paper The sigmoid penalty with steepness κ approximates a hard risk cap closely enough to claim a safety guarantee.
    Used in Eq. (3.3) and in the main experiments; the paper admits it tolerates slight violations, so the hard-cap guarantee is not actually enforced.

pith-pipeline@v1.3.0-alltime-deepseek · 18182 in / 14568 out tokens · 130156 ms · 2026-08-04T10:28:27.088026+00:00 · methodology

0 comments
read the original abstract

Ensuring that large language models (LLMs) comply with safety requirements is a central challenge in AI deployment. Existing alignment approaches primarily operate during training, such as through fine-tuning or reinforcement learning from human feedback, but these methods are costly and inflexible, requiring retraining whenever new requirements arise. Recent efforts toward inference-time alignment mitigate some of these limitations but still assume access to model internals, which is impractical, and not suitable for third party stakeholders who do not have access to the models. In this work, we propose a model-independent, black-box framework for safety alignment that does not require retraining or access to the underlying LLM architecture. As a proof of concept, we address the problem of trading off between generating safe but uninformative answers versus helpful yet potentially risky ones. We formulate this dilemma as a two-player zero-sum game whose minimax equilibrium captures the optimal balance between safety and helpfulness. LLM agents operationalize this framework by leveraging a linear programming solver at inference time to compute equilibrium strategies. Our results demonstrate the feasibility of black-box safety alignment, offering a scalable and accessible pathway for stakeholders, including smaller organizations and entities in resource-constrained settings, to enforce safety across rapidly evolving LLM ecosystems.

Figures

Figures reproduced from arXiv: 2510.09330 by Long Tran-Thanh, Tuan Nguyen.

Figure 1
Figure 1. Figure 1: Reward distributions on HHH. SG (Sigmoid) concentrates near the HHH reference mean (dashed line), exhibit a positive skew, and substantially suppress the negative left tail compared to baselines. 4.4 Main Results HHH: SG matches or exceeds the best baseline on 3/5 models ( [PITH_FULL_IMAGE:figures/full_fig_p011_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 16 linked inside Pith

  1. [1]

    • Retrieval-augmented generation.Systems select among retrieved passages, answer drafts, or template completions derived from a fixed retrieval pool

    p-value Llama-2-13B D 23 38 0.0722 ER-D 35 42 0.4944 ER-G 35 43 0.4282 G 42 33 0.3557 MI 42 34 0.4222 SC 41 42 1.0000 Llama-2-7B D 27 31 0.6940 ER-D 35 29 0.5323 ER-G 35 29 0.5323 G 47 35 0.2242 MI 47 35 0.2242 SC 43 27 0.0722 Llama-3.1-8B D 51 35 0.1052 ER-D 50 33 0.0784 ER-G 50 33 0.0784 G 40 33 0.4828 MI 39 33 0.5560 SC 49 33 0.0970 Llama-3.2-1B D 37 5...

    1980

  2. [4]

    13 Zina Chkirbene, Ridha Hamila, Ala Gouissem, and Unal Devrim

    MA VIS: Multi- Objective Alignment via Value-Guided Inference-Time Search.arXiv preprint arXiv:2508.13415 (2025). 13 Zina Chkirbene, Ridha Hamila, Ala Gouissem, and Unal Devrim

    arXiv 2025

  3. [5]

    Nicolai Dorka

    Deep reinforcement learning from human preferences.Advances in neural information processing systems30 (2017). Nicolai Dorka

    2017

  4. [8]

    https: //openreview.net/forum?id=JV84NVo1em

    Safe and Robust Subgame Exploitation in Imperfect-Information Games. https: //openreview.net/forum?id=JV84NVo1em. Introduces the notion of Adaptation Safety in opponent exploitation. Ian Gemp, Roma Patel, Yoram Bachrach, Marc Lanctot, Vibhavari Dasagi, Luke Marris, Georgios Piliouras, Siqi Liu, and Karl Tuyls. 2024a. Steering Language Models with Game-The...

    Pith/arXiv arXiv 2024

  5. [10]

    InProceedings of the 60th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers)

    TruthfulQA: Measuring How Models Mimic Human Falsehoods. InProceedings of the 60th Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 3214–3252. doi:10.18653/v1/2022.acl-long.229 14 Sidharth Mudgal, Jong Lee, Harish Ganapathy, YaGuang Li, Tao Wang, Yanping Huang, Zhifeng Ch...

    doi:10.18653/v1/2022.acl-long.229 2022

  6. [12]

    InProceedings of the 41st International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol

    Nash Learning from Human Feedback. InProceedings of the 41st International Conference on Machine Learning (ICML) (Proceedings of Machine Learning Research, Vol. 235). PMLR, 36743–36768. arXiv:2312.00886 [cs.LG] https://proceedings. mlr.press/v235/munos24a.html Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher ...

    Pith/arXiv arXiv

  7. [13]

    Alexander Tobias Neumann, Yue Yin, Sulayman Sowe, Stefan Decker, and Matthias Jarke

    Webgpt: Browser-assisted question-answering with human feedback.arXiv preprint arXiv:2112.09332(2021). Alexander Tobias Neumann, Yue Yin, Sulayman Sowe, Stefan Decker, and Matthias Jarke

    Pith/arXiv arXiv 2021

  8. [14]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al

    An llm-driven chatbot in higher education for databases and information systems.IEEE Transactions on Education(2024). Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al

    2024

  9. [15]

    Saurabh Pahune, Zahid Akhtar, Venkatesh Mandapati, and Kamran Siddique

    Training language models to follow instructions with human feedback.Advances in neural information processing systems35 (2022), 27730–27744. Saurabh Pahune, Zahid Akhtar, Venkatesh Mandapati, and Kamran Siddique

    2022

  10. [16]

    The Importance of AI Data Governance in Large Language Models.Big Data and Cognitive Computing9, 6 (2025),

    2025

  11. [18]

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D

    Navigating the safety landscape: Measuring risks in finetuning large language models.Advances in Neural Information Processing Systems37 (2024), 95692–95715. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn

    2024

  12. [19]

    arXiv preprint arXiv:2305.18290(2023).https://arxiv.org/abs/2305.18290 Aarohi Srivastava, Abhinav Rastogi, et al

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model. arXiv preprint arXiv:2305.18290(2023).https://arxiv.org/abs/2305.18290 Aarohi Srivastava, Abhinav Rastogi, et al

    Pith/arXiv arXiv 2023

  13. [20]

    Cited as a general reference for HHH-style evaluation context

    Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models (BIG-bench).arXiv preprint arXiv:2206.04615 (2022). Cited as a general reference for HHH-style evaluation context. Pengyu Wang, Dong Zhang, Linyang Li, Chenkun Tan, Xinghao Wang, Ke Ren, Botian Jiang, and Xipeng Qiu

    Pith/arXiv arXiv 2022

  14. [21]

    InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP)

    InferAligner: Inference-Time Alignment for Harmlessness through Cross-Model Guidance. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP). arXiv:2401.11206 [cs.CL] https://aclanthology.org/ 2024.emnlp-main.585/ Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V . Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, ...

    Pith/arXiv arXiv 2024

  15. [22]

    InInternational Conference on Learning Representations (ICLR)

    Self-Consistency Improves Chain-of-Thought Reason- ing in Language Models. InInternational Conference on Learning Representations (ICLR). arXiv:2203.11171 [cs.CL]https://openreview.net/forum?id=1PL1NIMMrw Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al

    Pith/arXiv arXiv

  16. [23]

    15 Yuheng Zhang, Dian Yu, Baolin Peng, Linfeng Song, Ye Tian, Mingyue Huo, Nan Jiang, Haitao Mi, and Dong Yu

    Ethical and social risks of harm from language models.arXiv preprint arXiv:2112.04359(2021). 15 Yuheng Zhang, Dian Yu, Baolin Peng, Linfeng Song, Ye Tian, Mingyue Huo, Nan Jiang, Haitao Mi, and Dong Yu

    Pith/arXiv arXiv 2021

  17. [24]

    https://arxiv.org/abs/ 2407.00617ICLR 2025 (OpenReview) version available

    Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning.arXiv preprint arXiv:2407.00617(2024). https://arxiv.org/abs/ 2407.00617ICLR 2025 (OpenReview) version available. Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang

    Pith/arXiv arXiv 2024

  18. [25]

    I cannot provide advice on this topic

    SafetyBench: Evaluating the Safety of Large Language Models.arXiv preprint arXiv:2309.07045(2023).https://arxiv.org/abs/2309.07045 16 A Prompts and Decoding This appendix specifies the exact prompt strings and decoding constraints used in our experiments, organized by dataset. We separate (i)GENERATORprompts used to elicit candidates or select MCQ options...

    Pith/arXiv arXiv 2023

  19. [2002]

    InProceedings of the 40th Annual Meeting of the Association for Computational Linguistics

    BLEU: a Method for Automatic Evaluation of Machine Translation. InProceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 311–318. doi:10.3115/1073083.1073135 Sheng Y Peng, Pin-Yu Chen, Matthew Hull, and Duen H Chau

    arXiv

  20. [2017]

    InAdvances in Neural Information Processing Systems (NeurIPS)

    Safe and Nested Subgame Solving for Imperfect-Information Games. InAdvances in Neural Information Processing Systems (NeurIPS). arXiv:1705.02955 [cs.GT] https://papers.nips.cc/paper/2017/hash/ 7fe1f8abaad094e0b5cb1b01d712f708-Abstract.html Jeremy Carleton, Debajoy Mukherjee, Srinivas Shakkottai, and Dileep Kalathil

    Pith/arXiv arXiv 2017

  21. [2018]

    AI Safety via Debate.arXiv preprint arXiv:1805.00899(2018).https://arxiv.org/abs/1805.00899 Athul Paul Jacob, Yikang Shen, Gabriele Farina, and Jacob Andreas. 2024a. The Consensus Game: Language Model Generation via Equilibrium Search. InInternational Conference on Learning Representations (ICLR). arXiv:2310.09139 [cs.CL] https://openreview.net/forum?id= ...

    Pith/arXiv arXiv 2018

  22. [2021]

    A General Language Assistant as a Laboratory for Alignment.arXiv preprint arXiv:2112.00861(2021). https: //arxiv.org/abs/2112.00861 Suhana Bedi, Yutong Liu, Lucy Orr-Ewing, Dev Dash, Sanmi Koyejo, Alison Callahan, Jason A Fries, Michael Wornow, Akshay Swaminathan, Lisa Soleymani Lehmann, et al

    Pith/arXiv arXiv 2021

  23. [2022]

    Zifan Ge, Yile Wu, Bowen Zhang, Xiaotie Chen, Georgios Piliouras, and Panayotis Mertikopou- los

    Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned.arXiv preprint arXiv:2209.07858 (2022). Zifan Ge, Yile Wu, Bowen Zhang, Xiaotie Chen, Georgios Piliouras, and Panayotis Mertikopou- los

    Pith/arXiv arXiv 2022

  24. [2023]

    Controlled decoding from language models.arXiv preprint arXiv:2310.17022(2023). Rémi Munos, Michal Valko, Daniele Calandriello, Mohammad Gheshlaghi Azar, Mark Rowland, Zhaohan Daniel Guo, Yunhao Tang, Matthieu Geist, Thomas Mesnard, Côme Fiegel, Andrea Michi, Marco Selvi, Sertan Girgin, Nikola Momchev, Olivier Bachem, Daniel J. Mankowitz, Doina Precup, an...

    Pith/arXiv arXiv 2023

  25. [2024]

    Quantile Regression for Distributional Reward Models in RLHF.arXiv preprint arXiv:2409.10164(2024).https://arxiv.org/abs/2409.10164 Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al

    Pith/arXiv arXiv 2024

  26. [2025]

    Noam Brown and Tuomas Sandholm

    Testing and evaluation of health care applications of large language models: a systematic review.Jama(2025). Noam Brown and Tuomas Sandholm

    2025