{"id":"cfc38c69-bed7-4f0c-ba80-a03a6831c2f8","arxiv_id":"2508.07699","paper_version":1,"verdict":"REJECT","confidence":"LOW","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"RTCFR, a reward-transformation form of CFR for perturbed games, is claimed to converge to an epsilon-EFPE with last-iterate dynamics and adaptive perturbations via the new ISNE metric.","lead":"The paper proposes a new algorithm, RTCFR, for computing approximate Extensive-Form Perfect Equilibria in two-player zero-sum games, with last-iterate convergence and an adaptive perturbation scheme. The significance is a potentially faster, more accurate way to compute a game-theoretic refinement that models player mistakes, useful for game-solving AI.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma 16's proof contains an invalid algebra step: printed inequality (35) forces x_ref = x*_n, so the contraction central to Theorem 11 is unproven.","rationale":"The reader's weakest-assumption analysis correctly identifies the fatal flaw in Lemma 16's proof. I independently checked the algebra: the printed inequality (35) does not follow from (34) and, if taken as written, implies x*_n = x_ref, contradicting the lemma's premise. This invalidates the proof of the contraction that drives the convergence of the saddle-point sequence and Theorem 11. The paper's additional caveat that only best-iterate convergence is achieved for finite T further weakens the 'last-iterate' claim, but the Lemma 16 issue is the most load-bearing because it breaks the theoretical foundation before that caveat even applies. The experimental results may be valid, and the algorithm could be salvageable with a corrected proof, but as printed the central theoretical guarantee is unsupported. Thus the reader's REJECT verdict is appropriate, albeit with low confidence because a corrected proof of Lemma 16 may exist.","tokens_in":11985,"tokens_out":5923,"duration_ms":56923,"concrete_test":"Independently re-derive the implications of Equation (34). The correct valid inequality is ⟨x*_n − x_ref, x* − x*_n⟩ ≥ 0; verify whether the claimed contraction (32) and the quantitative bound (33) can be derived from this corrected inequality alone. Additionally, numerically test Lemma 16 in a small EFG (e.g., Kuhn Poker) by computing exact saddle points x*_n of the RT-BSPP for several reference strategies x_ref and the exact NE x* of the perturbed game; check whether inequalities (32) and (33) hold with a game-dependent constant C. If either the derivation fails without extra assumptions or (33) is violated numerically, the central convergence theorem is not established.","verdict_should_be":"REJECT","load_bearing_attack":"The proof of Lemma 16, which provides the contraction that underpins Theorem 11's last-iterate convergence, contains a demonstrably invalid step. Starting from (34), the paper claims it 'simplifies to' ⟨x*_n − x_ref, x* − x*_n⟩ ≥ ⟨x*_n − x_ref, x* − x_ref⟩  (35). But (34) rearranges to ⟨x* − x*_n, x*_n − x_ref⟩ ≥ 0, i.e., ⟨x*_n − x_ref, x* − x*_n⟩ ≥ 0. The printed (35) is strictly stronger and, since x* − x_ref = (x*_n − x_ref) + (x* − x*_n), implies ||x*_n − x_ref||^2 ≤ 0, contradicting the assumption x*_n ≠ x_ref. Consequently, the subsequent derivation of (32)–(33) is incoherent: the 'adding' step from the printed (35) yields 0 ≥ ||x_ref − x*_n||^2, not the stated distance contraction. Even if Lemma 16's statement is true, the provided proof does not establish it, so the convergence of the saddle-point sequence {x*_n} to x*—and hence the asymptotic last-iterate convergence of RTCFR to the ϵ-EFPE—is unsupported. This is load-bearing because Theorem 11 and the adaptive perturbation guarantees in Section 5 both rely on this contraction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies computation of approximate extensive-form perfect equilibria (EFPE) in two-player zero-sum extensive-form games. It introduces RTCFR, a reward-transformation variant of CFR that operates on epsilon-perturbed sequence-form strategy spaces, and claims asymptotic last-iterate convergence to an epsilon-EFPE (Theorem 11), supported by a contraction lemma (Lemma 16). It also defines an information-set regret metric (ISNE) and an adaptive schedule that reduces the perturbation when the metric falls below a threshold. Experiments on Kuhn Poker, Leduc Poker, Goofspiel, and Liar's Dice compare RTCFR+ against CFR+, EGT, and RegOMWU variants.","tokens_in":12355,"tokens_out":9092,"duration_ms":103719,"significance":"The problem is well motivated: computing EFPE efficiently with last-iterate convergence and adaptive perturbations would be a genuine advance over average-iterate CFR-based methods. The paper proposes a concrete algorithm with available code and benchmark comparisons. However, the theoretical core is not currently established: the proof of the key contraction lemma contains a demonstrable algebraic error, and the paper's own caveat about finite T reducing the guarantee to best-iterate convergence contradicts the last-iterate claim. The experimental results are suggestive but cannot compensate for the missing theoretical support.","major_comments":[{"comment":"The claimed simplification from Eq. (34) to Eq. (35) is algebraically incorrect. From (34) the correct inequality is ⟨x*_n − x_ref, x* − x*_n⟩ ≥ 0. The printed (35), however, is equivalent to ⟨x*_n − x_ref, x* − x*_n⟩ ≥ ⟨x*_n − x_ref, x* − x_ref⟩, which reduces to −‖x*_n − x_ref‖² ≥ 0 and hence contradicts the assumption x*_n ≠ x_ref. The subsequent 'adding' step and the derivation of (32)–(33) are therefore not justified as written. Since Theorem 11 and the adaptive perturbation guarantees in Section 5 rely on this contraction, the central convergence proof is unsupported.","section":"Appendix B, Lemma 16, Eq. (35)"},{"comment":"The paper states that a strict theoretical guarantee requires T → ∞ in each RT-BSPP and that with finite T only best-iterate convergence (Theorem 10) is achieved, yet it concludes 'thus fulfilling Theorem 11.' This is a non sequitur: Theorem 10 guarantees the existence of some t ≤ T with small error, not that the last iterate x_T^{n,T} is close. Algorithm 1 sets the reference strategy to the last inner iterate, so the asymptotic last-iterate claim does not follow from the stated theorems. The authors need either a genuine last-iterate guarantee for the inner loop or a revised claim (e.g., best-iterate convergence of the output sequence).","section":"Section 4, Theorem 11 and preceding paragraph"},{"comment":"The adaptive schedule changes the perturbed game Γ_ϵ whenever rmax falls below δ. The paper claims that this preserves convergence to an EFPE as ϵ → 0, but Theorems 9–11 concern a fixed perturbed game. No theorem or continuity argument tracks the convergence of the RTCFR iterates as the target saddle point changes with ϵ. This is a load-bearing gap because the adaptive perturbation is a central contribution; the statement 'ensuring asymptotic convergence to the EFPE as ϵ → 0' is not proven by the supplied results.","section":"Section 5.2, Algorithm 1"}],"minor_comments":[{"comment":"Several equations and symbols are corrupted or missing in the provided manuscript (e.g., the RT-BSPP definitions and the statement of Lemma 14), making verification difficult. The authors should ensure the final PDF renders all notation correctly.","section":"Throughout"},{"comment":"The information set regret notation is inconsistent: q'_i(I) is used both as a regret vector and later as a scalar maximum, and the relationship to counterfactual regret via Bayes' rule is stated without derivation.","section":"Section 5.2, Definition 12"},{"comment":"For Liar's Dice, the paper sets T = 1, which nullifies the RT term and makes RTCFR equivalent to CFR+. Thus those experiments do not test the proposed reward-transformation mechanism, weakening the empirical claim of broad superiority.","section":"Section 6, Liar's Dice experiments"}],"recommendation":"major_revision","confidential_remarks":"The algebra error in Lemma 16 is easy to verify and should be decisive for the authors: if a corrected proof cannot be supplied, the theoretical claim should be downgraded to best-iterate convergence or removed. The paper's central contribution is potentially useful, but the current proof and the best-iterate/last-iterate gap are substantial enough that the manuscript is not suitable for acceptance in its present form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a real attempt at an important problem, and the algorithm and experiments are worth a look, but the central theorem is not supported by the submitted text. The claimed last-iterate convergence to ε-EFPE rests on Lemma 16, and Lemma 16's proof is demonstrably wrong as written. Starting from (34), they drop a nonnegative exploitability term and get (35): ⟨x*_n − x_ref, x* − x*_n⟩ ≥ ⟨x*_n − x_ref, x* − x_ref⟩. Since x* − x_ref = (x*_n − x_ref) + (x* − x*_n), this rearranges to ||x*_n − x_ref||² ≤ 0. So the printed (35) can only hold when the reference equals the saddle point; the later “adding” step then derives the contraction from a false premise. That's load-bearing: Theorem 11 and the adaptive perturbation claims in Section 5 depend on it. If the lemma has a correct proof, it's not in this version.\n\nWhat's actually new and good: Meng et al. proposed RTCFR+ for perturbed games without convergence theory; this paper supplies a convergence theorem (modulo the broken lemma) and an adaptive perturbation heuristic, ISNE, which is a reasonable novelty. ISNE as an information-set-regret threshold is a sensible way to decide when to shrink ε, and the experiments cover standard benchmarks with code released. The paper also honestly says the finite-T guarantee is best-iterate (Theorem 10) and that T must go to infinity per block for the strict last-iterate statement; that honesty is good, but it means the practical claim is weaker than the title suggests.\n\nSoft spots beyond the lemma: the adaptive method has no formal convergence proof of its own—it relies on the unproven Theorem 11. The experiments tune many hyperparameters per game, and in Liar's Dice they set T=1, which makes RTCFR identical to plain CFR+; those runs don't test RTCFR. The comparison to Bernasconi et al. could also be fairer, because their RegOMWU(adp) is modified substantially from the original and the tuning is not symmetric. Citation practice is otherwise honest: Meng et al. and Bernasconi et al. are credited properly.\n\nBottom line: this deserves a serious referee because the problem is central and the proof may be repairable, but in the current form I wouldn't trust the main theorem. I'd send it back for major revision with a request to fix Lemma 16 and re-check the adaptive guarantee.","headline":"RTCFR is a plausible algorithm with a nice adaptive idea, but the central last-iterate theorem isn't proven as printed: Lemma 16's key algebra step is invalid.","tokens_in":12832,"tokens_out":4012,"would_cite":false,"duration_ms":46955,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["91A05","91A18","91A26"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper's central claim is that RTCFR, a reward-transformed counterfactual regret minimizer, attains last-iterate convergence to an approximate extensive-form perfect equilibrium in two-player zero-sum games, and that an adaptive informat","keywords":["last-iterate convergence","counterfactual regret minimization","extensive-form perfect equilibrium","reward transformation","adaptive perturbation","information-set regret","two-player zero-sum games","trembling-hand perfection"],"falsifier":"Re-derive inequality (35) in Lemma 16 from equation (34); if the step implies $\\|\\pi^* - \\pi^{ref}\\|^2 \\le 0$ for distinct strategies, the contraction argument collapses. Separately, on a small game such as Kuhn Poker, solve each RT-BSPP to high precision and plot $\\|\\pi^* - \\pi_{*n}^*\\|$ across $n$; if the distances do not decrease monotonically, the claimed last-iterate convergence mechanism is falsified.","tokens_in":11873,"feed_emoji":"🎲","tokens_out":14392,"duration_ms":133688,"temperature":0.7,"pith_summary":"The paper tries to establish a last-iterate guarantee for a refinement of Nash equilibrium in two-player zero-sum extensive-form games. It combines reward transformation—a regularization trick that nudges a player's plan toward a reference plan during play—with counterfactual regret minimization on a deliberately perturbed game. The claimed result is that the final strategy produced by the algorithm, RTCFR, converges asymptotically to an $\\epsilon$-extensive-form perfect equilibrium, and that a new information-set regret metric can shrink the perturbation adaptively so that equilibrium accuracy improves as training proceeds. If this holds, equilibrium solvers no longer need to keep and average a long history of strategies, and off-equilibrium robustness is obtained without the fixed-perturbation trade-off between accuracy and convergence speed.","feed_headline":"Last-iterate regret method reaches approximate perfect equilibrium","feed_subtitle":"An adaptive perturbation schedule lets the final strategy converge to a refined equilibrium in zero-sum games.","key_machinery":"RT-BSPP: the $n$-th reward-transformed bilinear saddle-point problem, in which the payoff in the perturbed game is augmented by $\\mu(\\pi_i - \\pi_i^{ref})$ for a reference strategy; its saddle point becomes the next reference. Affine basis matrices $B(I)$ map behavioral strategies between the original and the $\\epsilon$-perturbed space, enforcing the minimum action probability. The ISNE metric, defined as the largest information-set regret (counterfactual regret renormalized to unit opponent reach probability), serves as the adaptive stopping threshold that decides when to decay $\\epsilon$ by a factor $\\gamma$.","core_discovery":"On its own terms, the paper's central claim is Theorem 11: in an $\\epsilon$-perturbed extensive-form game, the last strategy produced by RTCFR converges asymptotically to an $\\epsilon$-EFPE as both the number of reward-transformed bilinear saddle-point subproblems (RT-BSPPs) and the number of regret-matching iterations inside each subproblem go to infinity. The argument runs through a sequence of saddle points $\\pi_{*n}^*$: each RT-BSPP is the perturbed game with reward $\\tilde{u}_i = u_i + \\mu(\\pi_i - \\pi_i^{ref})$, so its saddle point is pulled toward the Nash equilibrium of the perturbed game, and the updated reference is the previous saddle point. The paper also introduces the Informatio","pith_inferences":["A repaired proof of the contraction lemma would likely extend the same template to quasi-perfect equilibria or to multi-player settings, since the reward transformation is applied locally at information sets rather than to global strategies.","The ISNE metric could serve as a practical stopping rule in deep reinforcement learning for imperfect-information games, where global exploitability is expensive but per-information-set regrets are directly available.","A testable corollary of the paper's mechanism: in games with a unique strict Nash equilibrium, the reward-transformation term becomes irrelevant (the paper itself notes this for Liar's Dice), so a purely adaptive version of plain CFR+ should match RTCFR's performance; the adaptive schedule's value should show up mainly in games with multiple equilibria or deep low-reach subgames."],"forward_implications":["If Theorem 11 holds, average-strategy computation can be discarded for perturbed-game solving: the very last strategy already converges, removing the memory and bias costs of averaging.","The adaptive ISNE schedule dissolves the fixed-perturbation dilemma: large early $\\epsilon$ gives fast refinement, while geometric decay preserves NE approximation accuracy later.","As $\\epsilon \\to 0$, the same algorithm recovers Nash-equilibrium computation, and the paper's experiments report practical exploitability decay near $O(1/T)$ although the proven bound is only $O(1/\\sqrt{T})$.","The ISNE metric provides a local certificate: it detects poorly converged branches with low reach probability, something global exploitability alone cannot pinpoint.","Because RTCFR needs one tree traversal per iteration (unlike EGT's three and CFR+'s two in the paper's accounting), the method scales to comparatively large instances such as Liar's Dice with six dice."],"supporting_citations":[{"why":"Supplies the behavioral-constraint affine maps used to define strategy spaces of perturbed games, the foundation RTCFR builds on.","marker":"[9]"},{"why":"Introduces Reward Transformation Regret Matching+ (RTRM+) and RTCFR+ from which RTCFR extends; provides the reward-transformation regret-matching mechanism.","marker":"[22]"},{"why":"Defines the extensive-form perfect equilibrium and the notion of trembles that the paper targets.","marker":"[31]"},{"why":"Motivates reward transformation as a way to escape cyclic behavior in iterative game solving and reach an equilibrium.","marker":"[26]"},{"why":"A fixed-perturbation smoothing method for approximate EFPE whose accuracy-versus-rate trade-off motivates the adaptive schedule.","marker":"[16]"},{"why":"A recent optimistic mirror-descent method for EFPE with last-iterate convergence, used as a baseline and a reference point for adaptive perturbation.","marker":"[4]"}],"fun_headline_variants":["Last-iterate regret algorithm approximates perfect equilibrium","Adaptive perturbation schedule yields last-iterate EFPE","RTCFR achieves last-iterate convergence to EFPE","Adaptive regret minimization for approximate perfect equilibrium","Perturbed games solved adaptively: last-iterate perfect equilibrium"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that each reward-transformed saddle point is strictly closer to the perturbed game's Nash equilibrium than the reference strategy was; the paper's printed proof of that contraction (Lemma 16) contains an algebraic step that, taken literally, would force a squared distance to be non-positive, so the guarantee is not yet established.","fun_headline_variants_meta":{"raw":{"variants":["Last-iterate regret algorithm approximates perfect equilibrium","Adaptive perturbation schedule yields last-iterate EFPE","RTCFR achieves last-iterate convergence to EFPE","Adaptive regret minimization for approximate perfect equilibrium","Perturbed games solved adaptively: last-iterate perfect equilibrium"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000179,"raw_usage":{"total_tokens":1145,"prompt_tokens":761,"completion_tokens":384,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":307}},"tokens_in":505,"tokens_out":384,"duration_ms":4654,"temperature":1.0,"reasoning_tokens":307,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T21:55:03.761907+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-derive inequality (35) in Lemma 16 from equation (34); if the step implies $\\|\\pi^* - \\pi^{ref}\\|^2 \\le 0$ for distinct strategies, the contraction argument collapses. Separately, on a small game such as Kuhn Poker, solve each RT-BSPP to high precision and plot $\\|\\pi^* - \\pi_{*n}^*\\|$ across $n$; if the distances do not decrease monotonically, the claimed last-iterate convergence mechanism is falsified.","supporting_citations":[{"cited_title":"Farina, C","cited_arxiv_id":null,"evidence_quote":"Supplies the behavioral-constraint affine maps used to define strategy spaces of perturbed games, the foundation RTCFR builds on."},{"cited_title":"Efficient Last-iterate Convergence Algorithms in Solving Games","cited_arxiv_id":"2308.11256","evidence_quote":"Introduces Reward Transformation Regret Matching+ (RTRM+) and RTCFR+ from which RTCFR extends; provides the reward-transformation regret-matching mechanism."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the extensive-form perfect equilibrium and the notion of trembles that the paper targets."},{"cited_title":"Perolat, R","cited_arxiv_id":null,"evidence_quote":"Motivates reward transformation as a way to escape cyclic behavior in iterative game solving and reach an equilibrium."},{"cited_title":"Kroer, G","cited_arxiv_id":null,"evidence_quote":"A fixed-perturbation smoothing method for approximate EFPE whose accuracy-versus-rate trade-off motivates the adaptive schedule."},{"cited_title":"Bernasconi, A","cited_arxiv_id":null,"evidence_quote":"A recent optimistic mirror-descent method for EFPE with last-iterate convergence, used as a baseline and a reference point for adaptive perturbation."}],"review_version":1}