Avoiding Obfuscation with Prover-Estimator Debate

Geoffrey Irving; Georgios Piliouras; Jiawei Li; Jonah Brown-Cohen; Lijie Chen; Zhiyang Xun

REVIEW 3 major objections 3 minor 2 cited by

Avoiding Obfuscation with Prover-Estimator Debate

T0 review · 3 major / 3 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Prover-estimator debate lets an honest debater win against obfuscation while keeping recursive efficiency.

desk verdict Novel prover-estimator protocol with a genuinely useful stability framing, but the main soundness proof has a load-bearing convexity error: Theorem 6.1 is unproven as written. read the letter →

arxiv 2506.13609 v2 pith:MVA64KQQ submitted 2025-06-16 cs.AI cs.CCcs.DS

Jonah Brown-Cohen , Geoffrey Irving , Georgios Piliouras , Lijie Chen , Jiawei Li , Zhiyang Xun This is my paper · ORCID

classification cs.AIcs.CCcs.DS

keywords AIdebatescalableoversightobfuscatedargumentsproblemprover-estimatoroutcomeindistinguishabilityargumentstabilityrecursivedecompositionStackelbergequilibrium

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Recursive debate promises to let humans supervise AI on problems too hard to check directly by having two AIs recursively decompose a hard question into simpler ones. But a dishonest debater can exploit this by decomposing an easy question into subquestions so hard that the honest opponent cannot find the flaw, which is the obfuscated arguments problem. This paper proposes changing who chooses where the debate goes: the estimator assigns probabilities to all subclaims, and the prover must pick a specific probability and claim it is wrong in a specific direction. The paper proves that under a stability condition on the argument, an honest prover can guarantee a winning reward against any estimator, while a dishonest prover's expected reward can be capped by an estimator that uses only slightly more computation. If the proof is right, it gives a recursive debate protocol that avoids obfuscation without sacrificing efficiency.

What carries the argument

The mechanism that carries the argument is an asymmetric division of labour plus a probability-penalty structure. In each round the estimator B assigns conditional probabilities to the prover's branching subclaims, and the prover A then chooses one subclaim and asserts that B's probability there is over- or underestimated. A receives reward proportional to the size of B's error, but only if A points the accusation in the correct direction, and the reward rate grows geometrically toward a final round judged directly by the human oracle. Three ingredients make the bounds work: outcome indistinguishability, produced by an online gradient descent construction so that B's probabilities cannot be distinguished from the truth by any small circuit that queries A; (ε,ρ)-stability, which says perturbing subclaim probabilities by less than ε changes the argument's outcome by less than ρε and stops B from hiding a big error behind many small ones; and majority voting over independent evidence, which converts s-supported languages into stable decompositions by error-correcting each oracle query.

What would settle it

A concrete refutation would be to instantiate the protocol on a recursive decomposition that is deliberately unstable, say a single subclaim whose probability crossing a threshold flips the verdict, with an honest prover and an estimator whose probabilities are wrong by just under ε in every branch. If the honest prover's expected reward falls short of (1−ε)/n, the completeness direction is false. For the soundness direction, one would need a prover strategy A and input distribution such that no estimator circuit of size O($q^{2}$ 2^q n $d^{2}$ / $η^{2}$) can hold A's expected payoff below P[A(x)=L(x)]/n + η.

Watch

Extended reading notes

Core claim

The central claim is that prover-estimator debate resolves the obfuscated arguments problem for languages in the class of problems for which both yes and no answers have proofs checkable in polynomial time with access to a human-judgement oracle. The protocol works by having the estimator B output conditional probabilities for the prover's subclaims and having the prover A choose a subclaim where B is alleged to be wrong in a specific direction, with rewards scaled by a factor of 1/r per round and ending in direct human judgement. Theorem 6.1 states two bounds with the same 1/n scale: if the recursive decomposition is A-provable and (ε,1/2)-stable, an honest A that always answers correctly earns expected reward at least (1−ε)/n against any B; and for every A there is a B computable by a size O($q^{2}$ 2^q n $d^{2}$ / $η^{2}$) circuit querying A such that A's expected reward is at most P[A(x)=L(x)]/n + η. The matching 1/n scale means truthfulness beats dishonesty by a gap that grows polynomially with n, so the protocol supplies a usable training signal rather than requiring exponentially many debates to catch one flaw. The paper further shows that any language whose human-judgement queries have enough independent confirming evidence can be transformed into one with a stable decomposition, so the stability condition is plausibly satisfiable in practice.

Load-bearing premise

The load-bearing premise is (ε,ρ)-stability—that nudging each subclaim probability by less than ε can shift an argument's verdict by at most ρε—because without it an estimator can bury a large error in many small probability changes, and the protocol never verifies on its own the independent evidence needed to guarantee stability.

Editorial extensions

If this is right

With an A-provable, (ε,1/2)-stable decomposition, an honest prover earns expected reward at least (1−ε)/n no matter how powerful the estimator is.
Against any dishonest prover, an estimator circuit of size O(q^2 2^q n d^2 / η^2) that queries the prover caps the prover's expected reward at P[A(x)=L(x)]/n + η.
Training to approximate Stackelberg equilibria in this game makes the prover answer correctly with probability greater than 1−4ε, so debate success can be read off from the prover's payoff.
Because the separation between honest and dishonest reward is Ω(1/n), catching a flaw requires only O(n) debates in expectation instead of navigating the full q^d = n^c proof tree.
Stability is needed only for completeness; the soundness bound holds even on unstable, obfuscated decompositions, so the safety-relevant guarantee does not depend on stability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The protocol's dependence on probability estimates suggests a direct empirical implementation: use an LLM's token probabilities as B's estimates and have A accuse specific estimates of being too high or too low; the paper gives qualitative recommendations but no experiments, so this would be a natural testbed.
The soundness construction is built from indistinguishability of probability vectors rather than from solving subproblems, so the same technique could be applied to other oversight settings, such as reward modelling or amplification, wherever one model evaluates another's claims.
The exponential factor 2^q in the estimator's circuit size indicates the protocol is practical only for small branching factors; an open and testable question is whether the same guarantees can be obtained with q growing with n.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The protocol idea is the real contribution: giving the estimator the job of assigning probabilities, and the prover the choice of where to recurse, is a clever way to attack the obfuscated arguments problem. The stability condition is also a useful way to say when recursive debate can work, and the paper is refreshingly honest about the fact that s-supportedness cannot be verified by the protocol itself.

Unfortunately, the main soundness proof does not hold. Lemma A.3 claims to construct an indistinguishable h by online gradient descent, but the loss is not convex in the embedded vector. The test function f(x,z,h(x)) depends on the current h, so Psi(F) is a function of h, and the mapping h -> Psi(F) is nonlinear. Their line (3) is not the absolute value of an affine function; the concrete example in the stress-test note is correct: with X={x0}, Z={0,1}, g(x0)=delta_1, and f=(z-p)^2, the loss becomes |2p^2-3p+1|, which is concave on [1/2,1]. So the online gradient descent regret bound does not apply, and Lemma A.4, which uses Lemma A.3 to bound every round's reward and the initial reward, cannot be trusted. Since the dishonest-prover bound in Theorem 6.1 is the protocol's main new guarantee, that claim is unproven as written.

Completeness (Lemma A.5) looks less obviously broken: the stability invocation appears workable once you grant the uniform pointwise bound. But it does not rescue the paper, because soundness is the part that distinguishes this protocol from naive recursive debate. The stability/s-supportedness assumption is strong and admittedly unverifiable through oracle calls; that is a real limitation, but secondary to the proof gap.

Who gets value from this? Researchers in scalable oversight theory. The protocol design and the stability definition are worth building on even if the theorem falls. The paper deserves a serious referee—the novelty is high and the question is important—but the referee should be prepared to catch the non-convexity issue. As submitted, I would not accept; with a corrected proof or a weakened soundness claim, it could become a solid contribution.

Recommendation: send it to peer review, but expect major revision.

Referee Report

3 major / 3 minor

Summary. The paper introduces prover-estimator debate, a recursive debate protocol in which the prover (A) decomposes a claim into q subclaims and the estimator (B) outputs probabilities for those subclaims, after which A selects a subclaim for recursion. The main theorem (Theorem 6.1) claims two guarantees: completeness, stating that under A-provability and (ε,1/2)-stability an honest prover can guarantee expected verifier reward at least (1−ε)/n against any estimator, and soundness, stating that for every prover A there is an estimator B of circuit size O(q^2 2^q n d^2/η^2) with oracle access to A such that A's expected reward is at most P[A(x)=L(x)]/n + η. The soundness proof in Appendix A constructs B's probability assignments via online gradient descent so that they are outcome-indistinguishable from the truth for any test function available to A. Section 8 provides a majority-voting construction of stable decompositions under an s-supportedness condition. The paper is candid that s-supportedness cannot be verified through oracle calls and that stability is necessary only for the completeness side.

Significance. If the results were established, this would be a meaningful theoretical contribution to the scalable-oversight literature: it proposes a recursive debate protocol with a polynomial reward gap in n, potentially mitigating the obfuscated-arguments problem while retaining the efficiency of recursive debate. The introduction of the (ε,ρ)-stability condition is a useful conceptual contribution, and the paper is commendably explicit about its assumptions and limitations. However, the soundness proof is currently invalid at a central point: the convexity claim in the key indistinguishability lemma is false as written, and the soundness half of the main theorem is therefore not proven. The result is promising but the manuscript is not yet in a publishable state.

major comments (3)

[Appendix A, Lemma A.3, Eq. (3)] The loss function l_t(h)=|<Ψg,ΨF>−<h,ΨF>| is claimed to be the absolute value of an affine function in h and hence convex. This is false because F(x,z)=f(x,z,h(x)) depends on h, so ΨF is not a fixed vector independent of the optimization variable. For example, take X={x0}, Z={0,1}, μ(x0)=1, g(x0)=δ_1, and f(x,z,h)=(z−h(x))^2; writing p=h(x0)(1), the loss becomes |(1−p)^2−p(1−p)|=|2p^2−3p+1|, which is concave on [1/2,1]. Thus the regret bound of Theorem A.2 does not apply, and the termination and circuit-size conclusions of Lemma A.3 do not follow. A possible repair is to fix the test function at the current iterate, defining F_t(x,z)=f(x,z,h_t(x)) before writing the loss; if that was the intent, the proof must be rewritten to define F_t explicitly and to recompute the subgradient accordingly.
[Appendix A, Lemma A.4] Because Lemma A.3 is the only mechanism used to construct B and to bound both the initialization reward (Eq. 5) and each per-round reward (Eq. 11), the invalidity in Lemma A.3 propagates directly into the soundness half of Theorem 6.1. Without a valid indistinguishability construction, the claimed bound E_{x∼μ}[V_{A,B}(x)] ≤ P[A(x)=L(x)]/n + η is unsupported. The soundness guarantee, which is the paper's main new contribution over prior doubly-efficient debate, therefore needs a corrected proof.
[Section 8, Definition 8.1 and Theorem 8.3] The completeness case of Theorem 6.1, and consequently Theorem 6.2, depend on the existence of an (ε,1/2)-stable recursive decomposition. The only construction of such a decomposition given in the paper, Theorem 8.3, requires s-supportedness, and the paper itself states that s-supportedness cannot be verified via oracle calls and must be established by outside knowledge. This is an acknowledged limitation rather than an internal inconsistency, but it means the protocol's completeness guarantee is conditional on external information that the debate protocol never checks. The main theorems should state this condition prominently rather than leaving it only to the discussion in Section 8.

minor comments (3)

[Lemma A.3, circuit-size accounting] The proof of Lemma A.3 treats an evaluation of f∈F as O(1) when counting the circuit size of h, but f itself is a circuit of size O(nq) making q queries to A. With |Z|=2^q, the bound stated in Lemma A.4, O(q^2 2^q n/δ^2), does not follow directly from Lemma A.3; the paper should clarify the circuit model and the cost of implementing F-evaluations, or adjust the bound.
[Proof of Lemma A.5] The phrase 'last round l' is ambiguous because rounds are indexed in decreasing order from d to 0. The subsequent inequalities appear to use l as the final (smallest-index) round in which the probability error exceeds ε, but the surrounding text should be rephrased to avoid off-by-one confusion.
[Definition 8.1] The independence in s-supportedness is a strong modeling assumption that could be violated by correlated human errors. The discussion in Section 9 acknowledges this informally; a formal remark stating that the stability construction inherits this assumption and does not check it would improve the paper's precision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the prover-estimator debate results are proved from declared stability and provability assumptions, not from their conclusions, and self-citations are background rather than load-bearing.

full rationale

The paper's derivation chain is self-contained. Theorem 6.1 is proved via Lemma A.4 (soundness) and Lemma A.5 (completeness), which in turn rely on Lemma A.3 (online gradient descent yielding outcome indistinguishability) and on the explicitly stated assumptions of A-provability and (ε,ρ)-stability. These assumptions are not defined in terms of the theorem's conclusion: A-provability posits a strategy A that answers all decomposition queries correctly, and stability is a Lipschitz-type condition on the recursive checker MD with respect to subclaim probabilities; stability is used only in the completeness half. The route to stability in Theorem 8.3 assumes s-supportedness, which the paper itself flags as unverifiable by the protocol ('We caution that s-supportedness cannot be verified via calls to the oracles: additional causal structure or other knowledge about L must be brought to bear, and this is a limitation of our paper'). That is a strong declared hypothesis and an honest limitation, not a circular reduction. The soundness construction's online gradient descent loss uses F(x,z)=f(x,z,h(x)) so the loss depends on the current h; whether that loss is convex is a mathematical correctness concern about Lemma A.3, not a circularity, because the constructed B is not an input being relabeled as a prediction and no fitted parameter is renamed as a theorem output. Citations to Brown-Cohen et al. 2024 and to Irving/Pfau are background or open-problem pointers, and no uniqueness or stability theorem is imported from prior work by the same authors as a forced choice. Accordingly, no step reduces by construction to its own inputs, and the paper merits a non-circularity score of 0.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

No data-fitting constants appear. The central guarantees rest on the protocol parameters ε, ρ, η chosen by the designer (not fitted to data), and on three conceptual assumptions: the oracle model of human judgement, A-provability of the honest prover, and (ε,ρ)-stability of the decomposition. Stability is the paper's own condition; its concrete realization in Theorem 8.3 requires s-supportedness, whose independence condition is explicitly admitted to be unverifiable via the protocol. No new physical or mathematical entities are postulated.

free parameters (4)

epsilon (ε) = not fitted; chosen by designer
Controls the stability requirement, the reward rate r=1/(4ε(1-ρ)), and the final accuracy bound 1-4ε in Theorem 6.2. Guesses on its value are not validated by data.
rho (ρ) = set to 1/2 in main results
Stability Lipschitz constant; the completeness proof requires balancing error growth via ρ and the reward rate.
eta (η) = not fitted; approximation parameter
Bounds the additive error in the soundness guarantee; larger η allows smaller estimator circuits.
gamma (γ) = not fitted; failure probability
Probability that A-provability or stability fails; completeness holds with probability 1-2γ.

assumptions (6)

standard math Computational complexity framework (oracle Turing machines, NTIME^O, Γ^O) and standard results such as online gradient descent and Chernoff bounds.
Used throughout as background; no proof is given for these foundational tools.
domain assumption The human judge is modeled as an oracle O that answers individual queries correctly.
This is the standard scalable oversight formalization from Irving et al. 2018; the protocol's guarantees are relative to this oracle model.
domain assumption For every L in Γ^O(T), a recursive decomposition M_D exists (Definition 5.1), constructed by splitting the verifier transcript.
The authors note every such language has a natural depth log T, width 2 decomposition.
domain assumption There exists an honest prover A that is A-provable: it correctly answers all queries in the decomposition with probability 1-γ (Definition 5.2).
Needed for completeness; the protocol guarantees nothing about honesty if no debater can answer subqueries.
ad hoc to paper The decomposition M_D is (ε,ρ)-stable with probability 1-γ (Definition 5.3).
A condition introduced by this paper, required for the completeness case; the paper calls stability the primary limitation.
ad hoc to paper For Theorem 8.3, L is s-supported: each oracle query has s independent supporting pieces of evidence, independent under a distribution P over oracles.
Introduced to construct stable decompositions via majority voting; as the paper states, s-supportedness cannot be verified via oracle calls and must be established by outside knowledge.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Avoiding Obfuscation with Prover-Estimator Debate." pith.science (2026). https://pith.science/paper/MVA64KQQ

@misc{pith2026250613609,
  author       = {Pith},
  title        = {Pith review of: Avoiding Obfuscation with Prover-Estimator Debate},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MVA64KQQ}},
  note         = {Machine review of arXiv:2506.13609}
}

read the original abstract

Training powerful AI systems to exhibit desired behaviors hinges on the ability to provide accurate human supervision on increasingly complex tasks. A promising approach to this problem is to amplify human judgement by leveraging the power of two competing AIs in a debate about the correct solution to a given problem. Prior theoretical work has provided a complexity-theoretic formalization of AI debate, and posed the problem of designing protocols for AI debate that guarantee the correctness of human judgements for as complex a class of problems as possible. Recursive debates, in which debaters decompose a complex problem into simpler subproblems, hold promise for growing the class of problems that can be accurately judged in a debate. However, existing protocols for recursive debate run into the obfuscated arguments problem: a dishonest debater can use a computationally efficient strategy that forces an honest opponent to solve a computationally intractable problem to win. We mitigate this problem with a new recursive debate protocol that, under certain stability assumptions, ensures that an honest debater can win with a strategy requiring computational efficiency comparable to their opponent.

Figures

Figures reproduced from arXiv: 2506.13609 by the authors.

**Figure 2.** The prover-estimator debate protocol. subproblems, and debater B gives probabilities for the answers to each subproblem. Debater A then selects a subproblem for recursion where B’s probability is allegedly incorrectly estimated. Rewards are assigned in each round to A for correctly predicting inconsistencies between B’s probabilities in the current round versus the previous round, with the available reward growing b… view at source ↗

**Figure 3.** While prover-estimator debate mitigates the obfuscated arguments problem that occurs in [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

How to Avoid Debate: Scalable AI Safety via Doubly-Efficient Interactive Proofs
cs.AI 2026-07 accept novelty 7.5 of 10

Doubly-efficient single-prover interactive proofs and arguments exist for robust oracle circuits and for low-degree oracles, enabling relativizing verification without debate.
How to Verify Consistency of Probabilistic Claims
cs.CC 2026-08 reject novelty 7.0 of 10

Places approximate probabilistic consistency of explicit claims in NP with short certificates, and gives an interactive PCP for circuit-specified predictors, but the main protocol's soundness proof has an unproven step.

Reference graph

Works this paper leans on

30 extracted references · 20 canonical work pages · cited by 2 Pith papers

[1]

Models that prove their own correctness

Noga Amit, Shafi Goldwasser, Orr Paradise, and Guy Rothblum. Models that prove their own correctness. arXiv preprint arXiv:2405.15722, 2024

arXiv 2024
[2]

Learning to give checkable answers with prover-verifier games

Cem Anil, Guodong Zhang, Yuhuai Wu, and Roger Grosse. Learning to give checkable answers with prover-verifier games. arXiv preprint arXiv:2108.12099, 2021

arXiv 2021
[3]

Debate update: Obfuscated arguments problem, 2020

Beth Barnes. Debate update: Obfuscated arguments problem, 2020. URL https://www.alignmentforum.org/posts/PJLABqQ962hZEqhdB/debate-update-obfuscated-arguments-problem

work page 2020
[4]

Superintelligent agents pose catastrophic risks: Can scientist AI offer a safer path? arXiv preprint arXiv:2502.15657, 2025

Yoshua Bengio, Michael Cohen, Damiano Fornasiere, Joumana Ghosn, Pietro Greiner, Matt MacDermott, S \"o ren Mindermann, Adam Oberman, Jesse Richardson, Oliver Richardson, et al. Superintelligent agents pose catastrophic risks: Can scientist AI offer a safer path? arXiv preprint arXiv:2502.15657, 2025

arXiv 2025
[5]

Scalable AI safety via doubly-efficient debate

Jonah Brown-Cohen, Geoffrey Irving, and Georgios Piliouras. Scalable AI safety via doubly-efficient debate. In Forty-first International Conference on Machine Learning, 2024

work page 2024
[6]

An alignment safety case sketch based on debate

Marie Davidsen Buhl, Jacob Pfau, Benjamin Hilton, and Geoffrey Irving. An alignment safety case sketch based on debate. arXiv preprint arXiv:2505.03989, 2025

work page Pith review arXiv 2025
[7]

Refereed delegation of computation

Ran Canetti, Ben Riva, and Guy N Rothblum. Refereed delegation of computation. Information and Computation, 226: 0 16--36, 2013

work page 2013
[8]

Chandra, Dexter C

Ashok K. Chandra, Dexter C. Kozen, and Larry J. Stockmeyer. Alternation. J. ACM, 28 0 (1): 0 114–133, January 1981. ISSN 0004-5411. doi:10.1145/322234.322243. URL https://doi.org/10.1145/322234.322243

arXiv 1981

Show all 30 references

[9]

Playing large games with oracles and AI debate

Xinyi Chen, Angelica Chen, Dean Foster, and Elad Hazan. Playing large games with oracles and AI debate. arXiv preprint arXiv:2312.04792, 2023

2023 arXiv
[10]

Supervising strong learners by amplifying weak experts

Paul Christiano, Buck Shlegeris, and Dario Amodei. Supervising strong learners by amplifying weak experts. arXiv preprint arXiv:1810.08575, 2018

2018 arXiv
[11]

Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra

John C. Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra. Efficient projections onto the l1-ball for learning in high dimensions. In International Conference on Machine Learning, 2008. URL https://api.semanticscholar.org/CorpusID:1226433

2008
[12]

Outcome indistinguishability

Cynthia Dwork, Michael P Kim, Omer Reingold, Guy N Rothblum, and Gal Yona. Outcome indistinguishability. In Proceedings of the 53rd Annual ACM SIGACT Symposium on Theory of Computing, pages 1095--1108, 2021

2021
[13]

Kim, Omer Reingold, Guy N

Cynthia Dwork, Michael P. Kim, Omer Reingold, Guy N. Rothblum, and G. Yona. Beyond Bernoulli : Generating random outcomes that cannot be distinguished from nature. In International Conference on Algorithmic Learning Theory, 2022. URL https://api.semanticscholar.org/CorpusID:247682041

2022
[14]

Making games short

Uriel Feige and Joe Kilian. Making games short. In Proceedings of the twenty-ninth annual ACM symposium on Theory of computing, pages 506--516, 1997

1997
[15]

Neural interactive proofs

Lewis Hammond and Sam Adam-Day. Neural interactive proofs. In The Thirteenth International Conference on Learning Representations, 2025

2025
[16]

Introduction to online convex optimization

Elad Hazan et al. Introduction to online convex optimization. Foundations and Trends in Optimization, 2 0 (3-4): 0 157--325, 2016

2016
[17]

Dodging systematic human errors in scalable oversight, 2025

Geoffrey Irving. Dodging systematic human errors in scalable oversight, 2025. URL https://www.alignmentforum.org/posts/EgRJtwQurNzz8CEfJ/dodging-systematic-human-errors-in-scalable-oversight

2025
[18]

AI safety via debate

Geoffrey Irving, Paul Christiano, and Dario Amodei. AI safety via debate. arXiv preprint arXiv:1805.00899, 2018

2018 arXiv
[19]

On scalable oversight with weak LLMs judging strong LLMs

Zachary Kenton, Noah Yamamoto Siegel, Janos Kramar, Jonah Brown-Cohen, Samuel Albanie, Jannis Bulian, Rishabh Agarwal, David Lindner, Yunhao Tang, Noah Goodman, et al. On scalable oversight with weak LLMs judging strong LLMs . In Forty-first International Conference on Machine...

2024
[20]

Debating with more persuasive LLMs leads to more truthful answers

Akbir Khan, John Hughes, Dan Valentine, Laura Ruis, Kshitij Sachan, Ansh Radhakrishnan, Edward Grefenstette, Samuel R Bowman, Tim Rockt \"a schel, and Ethan Perez. Debating with more persuasive LLMs leads to more truthful answers. In Forty-first International Conference on Mac...

2024
[21]

Prover-verifier games improve legibility of LLM outputs

Jan Hendrik Kirchner, Yining Chen, Harri Edwards, Jan Leike, Nat McAleese, and Yuri Burda. Prover-verifier games improve legibility of LLM outputs. arXiv preprint arXiv:2407.13692, 2024

2024 arXiv
[22]

Scalable agent alignment via reward modeling: a research direction

Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. Scalable agent alignment via reward modeling: a research direction. arXiv preprint arXiv:1811.07871, 2018

2018 arXiv
[23]

Debate helps supervise unreliable experts

Julian Michael, Salsabila Mahdi, David Rein, Jackson Petty, Julien Dirani, Vishakh Padmakumar, and Samuel R Bowman. Debate helps supervise unreliable experts. arXiv preprint arXiv:2311.08702, 2023

2023 arXiv
[24]

Riemann's hypothesis and tests for primality

Gary L Miller. Riemann's hypothesis and tests for primality. In Proceedings of the seventh annual ACM symposium on Theory of computing, pages 234--239, 1975

1975
[25]

Unexploitable search: blocking malicious use of free parameters, 2025

Jacob Pfau and Geoffrey Irving. Unexploitable search: blocking malicious use of free parameters, 2025. URL https://www.alignmentforum.org/posts/CuneN5HmLnztsLRzD/unexploitable-search-blocking-malicious-use-of-free-1

2025
[26]

Probabilistic algorithm for testing primality

Michael O Rabin. Probabilistic algorithm for testing primality. Journal of number theory, 12 0 (1): 0 128--138, 1980

1980
[27]

A method for obtaining digital signatures and public-key cryptosystems

Ronald L Rivest, Adi Shamir, and Leonard Adleman. A method for obtaining digital signatures and public-key cryptosystems. Communications of the ACM, 21 0 (2): 0 120--126, 1978

1978
[28]

Probabilistic logics and the synthesis of reliable organisms from unreliable components

John Von Neumann. Probabilistic logics and the synthesis of reliable organisms from unreliable components. Automata studies, 34 0 (34): 0 43--98, 1956

1956
[29]

Merlin-Arthur classifiers: Formal interpretability with interactive black boxes

Stephan W \"a ldchen, Kartikey Sharma, Max Zimmer, and Sebastian Pokutta. Merlin-Arthur classifiers: Formal interpretability with interactive black boxes. arXiv preprint arXiv:2206.00759, 2022

2022 arXiv
[30]

Online convex programming and generalized infinitesimal gradient ascent

Martin Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Machine Learning, pages 928--936, 2003

2003

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Models that prove their own correctness

Noga Amit, Shafi Goldwasser, Orr Paradise, and Guy Rothblum. Models that prove their own correctness. arXiv preprint arXiv:2405.15722, 2024

arXiv 2024

[2] [2]

Learning to give checkable answers with prover-verifier games

Cem Anil, Guodong Zhang, Yuhuai Wu, and Roger Grosse. Learning to give checkable answers with prover-verifier games. arXiv preprint arXiv:2108.12099, 2021

arXiv 2021

[3] [3]

Debate update: Obfuscated arguments problem, 2020

Beth Barnes. Debate update: Obfuscated arguments problem, 2020. URL https://www.alignmentforum.org/posts/PJLABqQ962hZEqhdB/debate-update-obfuscated-arguments-problem

work page 2020

[4] [4]

Superintelligent agents pose catastrophic risks: Can scientist AI offer a safer path? arXiv preprint arXiv:2502.15657, 2025

Yoshua Bengio, Michael Cohen, Damiano Fornasiere, Joumana Ghosn, Pietro Greiner, Matt MacDermott, S \"o ren Mindermann, Adam Oberman, Jesse Richardson, Oliver Richardson, et al. Superintelligent agents pose catastrophic risks: Can scientist AI offer a safer path? arXiv preprint arXiv:2502.15657, 2025

arXiv 2025

[5] [5]

Scalable AI safety via doubly-efficient debate

Jonah Brown-Cohen, Geoffrey Irving, and Georgios Piliouras. Scalable AI safety via doubly-efficient debate. In Forty-first International Conference on Machine Learning, 2024

work page 2024

[6] [6]

An alignment safety case sketch based on debate

Marie Davidsen Buhl, Jacob Pfau, Benjamin Hilton, and Geoffrey Irving. An alignment safety case sketch based on debate. arXiv preprint arXiv:2505.03989, 2025

work page Pith review arXiv 2025

[7] [7]

Refereed delegation of computation

Ran Canetti, Ben Riva, and Guy N Rothblum. Refereed delegation of computation. Information and Computation, 226: 0 16--36, 2013

work page 2013

[8] [8]

Chandra, Dexter C

Ashok K. Chandra, Dexter C. Kozen, and Larry J. Stockmeyer. Alternation. J. ACM, 28 0 (1): 0 114–133, January 1981. ISSN 0004-5411. doi:10.1145/322234.322243. URL https://doi.org/10.1145/322234.322243

arXiv 1981