Pith. sign in

REVIEW 2 major objections 2 minor 15 references

GARIP: A Running-Average Moving Reference for Last-Iterate Self-Play in Two-Player Zero-Sum Games

T0 review · 2 major / 2 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read GARIP anchors self-play to a running average to minimize peak lag and restore last-iterate convergence in zero-sum games.

desk verdict GARIP gives a clean lag-minimization theorem and local convergence at fixed β via running-average anchoring, but the scaling step in the proof needs verification and global convergence stays conjectural. read the letter →

arxiv 2606.22688 v1 pith:7FTE5IJ6 submitted 2026-06-21 cs.MA cs.GTcs.LG

Can Savc{i} This is my paper · ORCID
classification cs.MAcs.GTcs.LG
keywords self-playlast-iterateconvergencezero-sumgamesmovingreferencerunningaverageanchorstrengthmatrixboard
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Self-play with plain gradient ascent produces orbiting last iterates around equilibrium in two-player zero-sum games. Anchoring updates to a moving reference can pull the orbit inward, but the choice of reference determines how much lag is introduced before the pull takes effect. GARIP uses the running average of past policies as that reference. The paper proves that, among all causal convex averages with the same mean lag, the running average is the unique choice that minimizes the maximum lag experienced by any iterate, while a periodic snapshot reference experiences twice that maximum lag. This lag reduction supplies the contraction needed for a local convergence proof at fixed anchor strength and explains why GARIP matches the empirical robustness of snapshot-based methods while requiring less careful hyperparameter tuning.

What carries the argument

The collapse mechanism, which tracks the peak lag of the reference and converts that lag into a contraction factor of (1-β) on the base map's rotation.

What would settle it

Run naive self-play and GARIP on the same 2-by-2 matrix game with known cycling behavior and measure whether the distance of the last iterate to equilibrium drops below a fixed threshold within a fixed number of steps only under the running-average anchor.

Watch

Extended reading notes

Core claim

The central discovery is a one-line comparison of lag profiles: collapse of the last-iterate orbit is governed by the peak lag of the reference; for any fixed mean lag the flat profile of the running average sets peak equal to mean, whereas the sawtooth profile of a snapshot sets peak equal to twice the mean. This identity directly yields local last-iterate convergence because the anchor scales the rotation angle of the base map by exactly (1-β), moving the fixed point from the unit circle into the interior of the unit disk.

Load-bearing premise

The anchor multiplies the base map's rotation angle by exactly the factor (1-β).

Editorial extensions

If this is right

  • Local last-iterate convergence holds at any constant anchor strength β in (0,1) once the rotation scaling crosses the stability boundary.
  • Global convergence is expected for sufficiently small β; large β produces a consensus failure in which both players lock onto the same suboptimal policy.
  • On matrix games, the Coin Game, Connect Four and Othello, GARIP reaches the same peak performance as snapshot references but collapses reliably under conventional hyperparameter choices where snapshots require explicit shortening of the snapshot interval.
  • An anticipatory reference with negative weights further reduces stale-side lag, but only improves outcomes in games where naive self-play already exhibits sustained cycling.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same lag-minimization argument could be tested in non-zero-sum or many-player settings where cycling is observed but no equilibrium concept is assumed.
  • Replacing the running average with other low-pass filters whose lag profile is flatter than a snapshot might yield further gains without changing the mean lag.
  • The local proof suggests that any base dynamics whose linearization has eigenvalues on the unit circle could be stabilized by a sufficiently strong running-average anchor, offering a general template beyond gradient ascent.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper introduces GARIP, which anchors self-play updates in two-player zero-sum games to the running average of past policies rather than a fixed reference or periodic snapshot. Its central claims are (i) a one-line peak-lag theorem establishing that, among causal convex averages with fixed mean lag, the running average uniquely minimizes peak lag (flat profile, peak = mean) while a snapshot yields peak = 2× mean, (ii) a local last-iterate convergence proof at constant anchor strength β obtained by showing that the anchor multiplies the rotational eigenvalues of the base map by exactly (1-β), crossing the stability boundary from recurrent to contractive behavior, and (iii) empirical robustness on matrix games, the Coin Game, Connect Four and Othello that matches R-NaD peak performance while being less sensitive to hyperparameter choice.

Significance. If the scaling property and lag theorem hold, the work supplies a simple, theoretically grounded default for last-iterate convergence that avoids the need to tune snapshot periods K. The mechanism cleanly separates the effect of reference shape from mean lag and supplies a falsifiable prediction (collapse rate depends on peak rather than mean lag). Experiments are reported as fully reproducible JAX code; the local-convergence result at constant β is a concrete theoretical advance over purely empirical moving-reference methods.

major comments (2)
  1. [local convergence argument / peak-lag theorem] Local convergence argument: the manuscript states that the anchored map multiplies the base map's rotational eigenvalues by precisely (1-β). The linearization must be exhibited (Jacobian of the joint update including the running-average reference) to confirm that no payoff-matrix cross terms or higher-order contributions from the average alter the multiplicative factor; without this explicit calculation the claimed boundary crossing from recurrent to contractive behavior remains unverified and is load-bearing for both the constant-β convergence claim and the large-β consensus-failure characterization.
  2. [experiments] Experimental comparison: the claim that GARIP and R-NaD are statistically indistinguishable over the full grid but that a matched-mean-lag GARIP collapses in 0/40 seeds versus 10/40 for R-NaD at conventional parameterizations requires the exact grid ranges, the definition of 'collapse,' and the statistical test to be stated in a single table or section so that the robustness advantage can be assessed.
minor comments (2)
  1. [peak-lag theorem] The abstract and text should clarify whether the peak-lag theorem assumes a fixed mean lag or derives it; the one-line statement is elegant but its precise hypotheses (convexity, causality, stationarity of the mean) need to be listed.
  2. [method] Notation for the running-average reference (e.g., how the anchor is updated each step) should be introduced once with an equation rather than described only in prose.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the careful review and constructive comments. We address each major point below and will revise the manuscript accordingly to improve clarity and verifiability.

read point-by-point responses

  1. Referee: [local convergence argument / peak-lag theorem] Local convergence argument: the manuscript states that the anchored map multiplies the base map's rotational eigenvalues by precisely (1-β). The linearization must be exhibited (Jacobian of the joint update including the running-average reference) to confirm that no payoff-matrix cross terms or higher-order contributions from the average alter the multiplicative factor; without this explicit calculation the claimed boundary crossing from recurrent to contractive behavior remains unverified and is load-bearing for both the constant-β convergence claim and the large-β consensus-failure characterization.

    Authors: We agree that the explicit Jacobian is required to substantiate the eigenvalue scaling. In the revised manuscript we will derive and display the full linearization of the joint map (policy gradient step together with the running-average update), confirming that the anchor multiplies the rotational eigenvalues of the base map by exactly (1-β) with no additional cross terms arising from the payoff matrix or the averaging operation at linear order. This will directly verify the claimed transition from recurrent to contractive behavior. revision: yes

  2. Referee: [experiments] Experimental comparison: the claim that GARIP and R-NaD are statistically indistinguishable over the full grid but that a matched-mean-lag GARIP collapses in 0/40 seeds versus 10/40 for R-NaD at conventional parameterizations requires the exact grid ranges, the definition of 'collapse,' and the statistical test to be stated in a single table or section so that the robustness advantage can be assessed.

    Authors: We will add a dedicated table (or consolidated subsection) that reports the precise hyperparameter grid ranges, the operational definition of collapse (last-iterate distance exceeding a fixed threshold for a minimum number of iterations), and the statistical procedure used to compare collapse rates. This will allow readers to evaluate the robustness claims directly from the reported data. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; central claims rest on independent one-line theorem and explicit scaling argument

full rationale

The peak-lag theorem is stated as a one-line result comparing running-average (peak = mean) vs. snapshot (peak = 2× mean) profiles for fixed mean lag; it does not reduce to any fitted parameter or prior result by construction. Local convergence is derived by showing the anchor multiplies base-map rotational eigenvalues by exactly (1-β) to cross the stability boundary; this scaling step is invoked directly rather than obtained from a fit or self-citation chain. No self-citations appear in the provided text that carry the load of either the theorem or the convergence claim. The derivation therefore remains self-contained against the stated external benchmark of linear stability analysis.

Assumptions & free parameters 1 free parameters · 2 assumptions · 0 invented entities

The central claims rest on the existence of a base map whose rotation is scaled linearly by (1-β), the convexity and causality of the reference averages, and the definition of collapse as tracking peak lag; no new entities are postulated.

free parameters (1)
  • anchor strength β
    Constant anchor strength used for local convergence; its value affects the scaling factor (1-β) and the large-β failure mode.
assumptions (2)
  • domain assumption The base update map without anchor has rotational dynamics that become contractive when scaled by (1-β).
    Invoked in the local convergence proof and stability-boundary crossing argument.
  • domain assumption References are causal convex averages with fixed mean lag.
    Required for the one-line theorem comparing running average to snapshot.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GARIP: A Running-Average Moving Reference for Last-Iterate Self-Play in Two-Player Zero-Sum Games." pith.science (2026). https://pith.science/paper/7FTE5IJ6

@misc{pith2026260622688,
  author       = {Pith},
  title        = {Pith review of: GARIP: A Running-Average Moving Reference for Last-Iterate Self-Play in Two-Player Zero-Sum Games},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7FTE5IJ6}},
  note         = {Machine review of arXiv:2606.22688}
}
abstract

Self-play with naive gradient ascent cycles in two-player zero-sum games: the last iterate orbits the equilibrium. Modern methods restore last-iterate convergence by regularizing toward a reference policy -- MMD a fixed one (reaching only the regularized equilibrium), R-NaD a periodic snapshot (the engine of DeepNash). We study GARIP, which anchors to the running average, and isolate what the choice of reference controls. Our central result is a mechanism: collapse tracks the peak lag of the reference, and among causal convex averages of a fixed mean lag the running average (flat profile, peak $=$ mean) uniquely minimizes that peak, while a snapshot's sawtooth has peak $= 2\times$ mean (a one-line theorem). Two consequences follow. Convergence: we prove local last-iterate convergence at constant anchor strength -- the anchor scales the base map's rotation by $1-\beta$, crossing the stability boundary and turning a recurrent base into a contraction (global convergence is conjectured at small $\beta$; we characterize a large-$\beta$ consensus failure). Robustness: GARIP matches R-NaD's peak performance -- on matrix games, the Coin Game, and the board games Connect Four/Othello, both moving references are far more robust than fixed-magnet and magnet-free baselines -- but is the better hyperparameter default; we report it both ways: over the full grid collapse rates are statistically indistinguishable, yet at conventional parameterizations a matched-mean-lag setting collapses in 0/40 vs 10/40 seeds (a snapshot matches it only by knowing to shorten $K$). The boundaries: an anticipatory (negative-weight) reference does better still on the stale side, and the advantage appears only where naive self-play cycles (five deep self-play loops). All experiments are pure JAX and reproducible.

Figures

Figures reproduced from arXiv: 2606.22688 by the authors.

Figure 1
Figure 1. Collapse rate vs. effective reference lag (deep [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Coin Game proxy exploitability vs. training [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Best-response win-rate across four board [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Deep-RL hyperparameter sensitivity (10 seeds; GARIP λ×ρ, R-NaD λ×K). Both methods have a stale-collapse region (red): R-NaD at large K, GARIP at small ρ (the two leftmost columns). The point is lo￾cation, not absence — GARIP’s red region sits at aver￾aging rates ≤ 0.00…
Figure 5
Figure 5. Figure 5: Last-iterate exploitability vs. iteration on four matrix games (10 seeds, shaded IQR). GARIP (red) [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Row-player strategy trajectory on the RPS [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Leduc hold’em: exact exploitability of the [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Matrix-game hyperparameter sensitivity. GARIP (η, β) has a wide low-exploitability basin; R￾NaD (α, K) degrades sharply outside a narrow band [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Leduc collapse heatmaps (exact exploitability, [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 1 canonical work pages

  1. [1]

    Perolat et al

    J. Perolat et al. Mastering the game of Stratego with model-free multiagent reinforcement learning.Science, 378(6623), 2022. (R-NaD / DeepNash.)

    2022

  2. [2]

    Sokota et al

    S. Sokota et al. A unified approach to reinforcement learning, quantal response equilibria, and two-player zero-sum games.ICLR, 2023. (Magnetic Mirror De- scent.)

    2023

  3. [3]

    Deep Reinforcement Learning from Self-Play in Imperfect-Information Games

    J. Heinrich and D. Silver. Deep reinforcement learn- ing from self-play in imperfect-information games. arXiv:1603.01121, 2016. (NFSP.)

    work page Pith review arXiv 2016

  4. [4]

    Brown, A

    N. Brown, A. Lerer, S. Gross, T. Sandholm. Deep coun- terfactual regret minimization.ICML, 2019

    2019

  5. [5]

    Unpairedimage-to- image translation using cycle-consistent adversarial net- works.ICCV, 2017

    J.-Y.Zhu, T.Park, P.Isola, A.Efros. Unpairedimage-to- image translation using cycle-consistent adversarial net- works.ICCV, 2017. (CycleGAN.)

    2017

  6. [6]

    G. M. Korpelevich. The extragradient method for find- ing saddle points and other problems.Ekonomika i Matematicheskie Metody, 12, 1976

    1976

  7. [7]

    L. D. Popov. A modification of the Arrow–Hurwicz method for search of saddle points.Mathematical Notes, 28(5), 1980. (Optimistic/past-gradient update.)

    1980

  8. [8]

    Rakhlin and K

    A. Rakhlin and K. Sridharan. Online learning with predictable sequences.COLT, 2013. (Optimism / pre- dictable sequences.)

    2013

Show all 15 references
  1. [9]

    Daskalakis, A

    C. Daskalakis, A. Ilyas, V. Syrgkanis, H. Zeng. Training GANs with optimism.ICLR, 2018. (OGDA, last-iterate in bilinear games.)

    2018

  2. [10]

    Wei, C.-W

    C.-Y. Wei, C.-W. Lee, M. Zhang, H. Luo. Linear last- iterateconvergencein constrainedsaddle-point optimiza- tion.ICLR, 2021. (OMWU linear last-iterate.)

    2021

  3. [11]

    Mertikopoulos, B

    P. Mertikopoulos, B. Lecouat, H. Zenati, C.-S. Foo, V. Chandrasekhar, G. Piliouras. Optimistic mirror de- scent in saddle-point problems: going the extra (gradi- ent) mile.ICLR, 2019

    2019

  4. [12]

    Golowich, S

    N. Golowich, S. Pattathil, C. Daskalakis, A. Ozdaglar. Last iterate is slower than averaged iterate in smooth convex-concave saddle point problems.COLT, 2020. (Last-iterate vs. averaged.)

    2020

  5. [13]

    B. Halpern. Fixed points of nonexpanding maps.Bull. Amer. Math. Soc., 73, 1967. (Halpern anchoring itera- tion.)

    1967

  6. [14]

    Yoon and E

    T. Yoon and E. K. Ryu. Accelerated algorithms for smooth convex-concave minimax problems withO(1/k2) rate on squared gradient norm.ICML, 2021. (Halpern- anchored acceleration for minimax.)

    2021

  7. [15]

    no collapse region

    Y. Cai, A. Oikonomou, W. Zheng. Finite-time last- iterate convergence for learning in multi-player games. NeurIPS, 2022. 9 A Proofs Proof of Prop. 1 (peak-lag minimality).ℓmax = maxt ℓt ≥T −1P t ℓt = ¯ℓ, with equality iff allℓ t coincide. The EMA’s weightswt,s =ρ(1−ρ) t−s give...

    2022

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.