REVIEW 2 major objections 3 minor 50 references
The α=0 speed-up in Newman's ranking iterations is real only with asynchronous updates; synchronous α=0 can diverge.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 05:25 UTC pith:UL2OQQTQ
load-bearing objection First real local-convergence theory for Newman's α-scheme, mostly good — but the central async-monotonicity theorem has a symmetrization error that must be fixed. the 2 major comments →
Convergence analysis of a family of Zermelo-type iterations for the Bradley--Terry model
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
For Newman's α-family of fixed-point iterations for the Bradley–Terry maximum likelihood estimator, the paper proves that the local convergence factor under asynchronous (sequential, Gauss–Seidel-type) updates is always strictly below 1 for every α≥0 on any strongly connected comparison outcome. Under the population Bradley–Terry model on a consistently ordered bipartite comparison graph, this asynchronous factor is monotonically increasing in α, so α=0 is the optimal parameter; at the same time the synchronous update at α=0 has convergence factor exactly 1 and can diverge. Therefore the practical superiority of Newman's α=0 algorithm is attributable to the combination of the parameter with
What carries the argument
The analysis reduces local convergence factors to spectral radii of Jacobian matrices at the MLE. For asynchronous updates the Jacobian of the composed mapping is a Gauss–Seidel-type splitting J_Aα = (I−J_Fα,L)^{-1}(J_Fα,D+J_Fα,U); under a consistently ordered bipartite graph its eigenvalues are squares of the roots of the quadratic eigenvalue problem det(μ²(C̄*+α R̄*) − μ W̄* − α R̄*)=0, whose non-unit roots φ_i(α) are monotonic in α. That monotonicity, together with the Möbius-transform containment argument for general graphs, yields the optimality of α=0 and the always-convergent guarantee.
Load-bearing premise
The finite-sample optimality of α=0 rests on the observed comparison data being close to Bradley–Terry-generated outcomes, with either many comparisons per pair or a dense comparison graph; when the data are strongly non-BT, as in the cyclic-outcome example, the population approximation can fail badly.
What would settle it
Run Newman's α=0 scheme synchronously on a bipartite comparison graph generated from a Bradley–Terry model and observe whether it diverges (ρ_sync(0)=1 predicted); or compute the asynchronous local convergence factor for a consistently ordered bipartite graph and check whether it is strictly larger at α=1 than at α=0, which would contradict Theorem 4.3's monotonicity. A more direct check is to compute the eigenvalues of the Jacobian at the MLE for a non-BT cyclic outcome and confirm that ρ_sync(0)>1.
If this is right
- Synchronous updates with α<1 can fail to converge even on Bradley–Terry data: on bipartite graphs ρ_sync(0)=1, and on non-BT cyclic outcomes ρ_sync(0) can exceed 1, so α=0 offers no acceleration without asynchronous updating.
- Asynchronous updates are always locally convergent for any α≥0 on any strongly connected comparison graph, with local factor ρ_async(α)<1.
- Under the population Bradley–Terry model on a consistently ordered bipartite graph, ρ_async(α) increases monotonically in α, so α=0 is provably the fastest parameter choice; for small α, ρ_async(α)<ρ_sync(α).
- Population convergence factors accurately approximate observed finite-sample factors as the number of comparisons per pair grows (fixed n), and numerical experiments confirm the approximation on real-world datasets; the rigorous large-n bound for |ρ_async−ρ̄_async| is explicitly left open.
- On real-world datasets, the convergence factor of the fitted BT model tracks the observed factor, and asynchronous updates offer the largest gain at α=0.
Where Pith is reading between the lines
- The monotonicity result is proved only for consistently ordered bipartite graphs, but the eigenvalue-containment result for general graphs suggests α=0 is likely optimal or near-optimal in many practical non-bipartite settings as well; a natural extension is to prove monotonicity under weaker structural assumptions.
- A practically testable consequence: whenever synchronous α=0 iteration diverges on a given dataset, switching to asynchronous updates should restore local convergence; the theory predicts the convergence factor will be near the population value when the data are near-Bradley–Terry.
- The left-open large-n gap for |ρ_async−ρ̄_async| could likely be closed by bounding eigenvalue condition numbers of the asynchronous Jacobian; the paper observes empirically that these condition numbers stay constant as n grows, which would make the perturbation bound scale correctly.
- The cyclic-outcome counterexample implies that the population approximation is invalid under strong model misspecification; a practical diagnostic for when the theory applies is whether the fitted BT model's expected outcome matrix is close to the observed one.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the local convergence of Newman's α-scheme, a family of fixed-point iterations for the Bradley–Terry (BT) maximum likelihood estimator, under both synchronous and asynchronous updates. For synchronous updates, the authors derive a spectral expression for the local convergence factor and prove, under the population BT model, that it is quasi-convex in α and can exceed 1 for α<1. For asynchronous updates, they prove local convergence for all α and establish, for population BT models on consistently ordered bipartite comparison graphs, that the convergence factor is monotonically increasing in α, thereby identifying α=0 as optimal. They also provide fixed-n and large-n approximation theorems relating observed convergence factors to their population counterparts, supported by numerical experiments on synthetic and real-world data. The paper explicitly acknowledges that the large-n factor approximation for asynchronous updates is not fully rigorous.
Significance. If the main theorems hold, the paper gives a principled explanation of the empirically observed acceleration of Newman's α=0 algorithm: the acceleration is due less to the parameter choice alone than to the combination of α=0 with asynchronous updating. The strengths of the paper are its closed-form spectral characterizations, the derivation of the convergence-factor dependence on α from a quadratic eigenvalue problem, and the extensive numerical validation, including real datasets and reproducible code. The paper is also commendably explicit about its limitations, in particular the open gap between the Jacobian approximation in Theorem 5.7 and a rigorous bound on the asynchronous convergence-factor difference. The principal concern is a false symmetry claim in Lemma 4.3(b), which is load-bearing for the asynchronous monotonicity theorem.
major comments (2)
- [§4.2, Lemma 4.3(b), Eq. (4.18)] The proof of Lemma 4.3(b) claims that diag(π*)W* is symmetric 'by the definition of W(π*) in (2.12)'. This is not correct. With [W*]_ij = m_ij π*_i/(π*_i+π*_j)^2, the (i,j) entry of diag(π*)W* is m_ij (π*_i)^2/(π*_i+π*_j)^2, while the (j,i) entry is m_ij (π*_j)^2/(π*_i+π*_j)^2; equality holds only if π*_i=π*_j. Hence the linearized pencil (A_α, B_α) in (4.18) is not symmetric as written, so the asserted reality and monotonicity of the QEP eigenvalues—and therefore Theorem 4.3—are not supported by the current proof. The fix appears local: replacing fW* := diag(π*)W* with fW* := W* diag(π*) makes A_α symmetric and preserves the QEP up to a nonsingular right factor, and the derivative formula for φ'_i(α) is unchanged because eC* and eR* are diagonal. Please correct this step and verify the stated bounds.
- [§5.2, Theorem 5.7] The paper explicitly leaves open whether Theorem 5.7 yields a rigorous bound on |ρ_async(α) − ρbar_async(α)| in the large-n regime. This gap is load-bearing for the practical-relevance claim attached to the asynchronous optimality result: Theorem 4.3 is a population-model statement, and the numerical success of approximating ρ by ρbar is not guaranteed for non-BT outcomes, as the cyclic-outcome example in §6.1 demonstrates (ρ_sync(0)≈1.5 while ρbar_sync(0)≤1). I would ask the authors to qualify the abstract and introduction accordingly, or to add a concrete sufficient condition under which the convergence-factor approximation holds in the large-n asynchronous setting.
minor comments (3)
- [Notation, §2.2] The symbol W is used for the set of observed outcomes, for the matrix (w_ij), and for the population/expected outcome; the overloading makes some formulas hard to follow. A consistent notation such as W, W_obs, and W̄ would improve readability.
- [Figure 4] The caption 'Backward Cyclic' appears to be a leftover label; the text refers to a cyclic comparison graph. Please align the caption with the text.
- [§5.2 last paragraph] The heuristic explanation that eigenvalue condition numbers 'remain asymptotically constant' is not proven. Since the paper already flags this as beyond scope, it would be clearer to state explicitly that this is an empirical observation rather than a conjecture backed by evidence.
Circularity Check
No circular reduction; central theorems are self-contained, and the self-citations are to independent published results.
full rationale
The paper's central claims (Theorems 3.2 and 4.3) are derived directly from the Jacobian of Newman's alpha-scheme and the population BT model; alpha is a free algorithm parameter, not a fitted constant, and no equation is defined in terms of the convergence factor it is said to predict. The only substantive self-citation is Theorem 5.4, deferred to Han, Xu and Chen [23], a published JASA result with stated assumptions that do not include the target result; under the reviewing rules this is independent support and does not raise the circularity score. The real-data 'predicted' convergence factors in Section 6.2 are computed from a BT model fitted by MLE on the same data, an in-sample model check that the paper itself qualifies by noting 'This proximity is partly explained by the fact that the expected outcome matrix W remains close to the original W'; it is not a load-bearing derivation. The paper also explicitly flags the large-n gap for |rho_async - rho_bar_async| as open in Section 5.2, which is a stated limitation rather than a circular step. Separately, the skeptic's concern about Lemma 4.3(b) ('The diagonal scaling ensures A_alpha is symmetric, recalling that diag(pi*)W* is symmetric by the definition of W(pi*) in (2.12)') is a proof-correctness issue: the claimed symmetry is false as written, so Theorem 4.3's proof has a gap. However, this is not a circular reduction — the monotonicity conclusion is not assumed in the definition of its inputs.
Axiom & Free-Parameter Ledger
axioms (5)
- domain assumption Pairwise comparison outcomes follow the Bradley–Terry model: w_ij ~ Bin(m_ij, π*_i/(π*_i+π*_j)), with comparison counts m_ij fixed and independent of strength.
- domain assumption The comparison graph G([n], W) or G([n], M) is strongly connected.
- domain assumption Assumption 1: the comparison graph is bipartite and vertices are ordered consistently with the bipartition.
- domain assumption Large-n regime: comparison graph is generated by a two-community stochastic block model with min{p,q} n / (log n)^3 → ∞ and dynamic range κ_n bounded by a constant.
- standard math Standard matrix-analysis results: Perron–Frobenius theorem, analytic eigenvalue perturbation theory, Weyl and Ostrowski–Elsner perturbation bounds, matrix Bernstein inequality, properties of Möbius transforms.
read the original abstract
Zermelo's algorithm is a classical method for computing the maximum likelihood estimator in the Bradley--Terry (BT) model, but its convergence can be slow in practice. To accelerate computation, Newman introduced a family of Zermelo-type fixed-point iterations parameterized by $\alpha$, with Zermelo's algorithm recovered at $\alpha=1$. Empirical evidence suggests that the choice $\alpha=0$ often converges substantially faster, making it a promising alternative, yet the mechanism underlying this acceleration remains elusive. This paper provides theoretical insight into this phenomenon through a systematic local convergence analysis. We derive closed-form expressions for local convergence factors under synchronous and asynchronous updates and analyze their dependence on $\alpha$ via spectral analysis of the associated Jacobian matrices. For synchronous updates, we show that the algorithm may fail to converge when $\alpha<1$, and its local convergence factor is quasi-convex in $\alpha$ under the population BT model. In contrast, asynchronous updates are always locally convergent, and their local convergence factor is provably monotonically increasing in $\alpha$ under the population BT model of consistently ordered bipartite comparison graphs, establishing the optimality of $\alpha=0$ in this setting. We further establish asymptotic approximation results for the population convergence factors under the BT model, justifying their practical relevance. Numerical experiments on synthetic and real-world datasets confirm the theory. Our analysis complements existing convergence results and shows that the acceleration of $\alpha=0$ arises not only from the parameter choice but, more importantly, from the use of asynchronous updates.
Figures
Reference graph
Works this paper leans on
-
[1]
Community detection and stochastic block models: recent developments
E. Abbe. “Community detection and stochastic block models: recent developments”.J. Mach. Learn. Res.18.177 (2018), pp. 1–86
2018
-
[2]
Accelerated spectral ranking
A. Agarwal, P. Patil, and S. Agarwal. “Accelerated spectral ranking”. In:ICML. PMLR. 2018, pp. 70– 79
2018
-
[3]
A dynamic paired comparisons model: Who is the greatest tennis player?
R. D. Baker and I. G. McHale. “A dynamic paired comparisons model: Who is the greatest tennis player?”Eur. J. Oper. Res.236.2 (2014), pp. 677–684
2014
-
[4]
Berman and R
A. Berman and R. J. Plemmons.Nonnegative matrices in the mathematical sciences. SIAM, 1994
1994
-
[5]
Billingsley.Convergence of probability measures
P. Billingsley.Convergence of probability measures. John Wiley & Sons, 2013
2013
-
[6]
An application of incomplete pairwise comparison matrices for ranking top tennis players
S. Boz´ oki, L. Csat´ o, and J. Temesi. “An application of incomplete pairwise comparison matrices for ranking top tennis players”.Eur. J. Oper. Res.248.1 (2016), pp. 211–218
2016
-
[7]
Rank analysis of incomplete block designs: I. the method of paired comparisons
R. A. Bradley and M. E. Terry. “Rank analysis of incomplete block designs: I. the method of paired comparisons”.Biometrika39.3/4 (1952), pp. 324–345
1952
-
[8]
Optimal full ranking from pairwise comparisons
P. Chen, C. Gao, and A. Y. Zhang. “Optimal full ranking from pairwise comparisons”.Ann. Statist. 50.3 (2022), pp. 1775–1805
2022
-
[9]
Partial recovery for top-k ranking: optimality of MLE and suboptimality of the spectral method
P. Chen, C. Gao, and A. Y. Zhang. “Partial recovery for top-k ranking: optimality of MLE and suboptimality of the spectral method”.Ann. Statist.50.3 (2022), pp. 1618–1652
2022
-
[10]
Item response theory—A statistical framework for educational and psychological measurement
Y. Chen, X. Li, J. Liu, and Z. Ying. “Item response theory—A statistical framework for educational and psychological measurement”.Statist. Sci.40.2 (2025), pp. 167–194
2025
-
[11]
Spectral method and regularized MLE are both optimal for top-K ranking
Y. Chen, J. Fan, C. Ma, and K. Wang. “Spectral method and regularized MLE are both optimal for top-K ranking”.Ann. Statist.47.4 (2019), p. 2204
2019
-
[12]
Deep reinforcement learning from human preferences
P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei. “Deep reinforcement learning from human preferences”. In:NeurIPS. 2017, pp. 4299–4307
2017
-
[13]
Statistical ranking with dynamic covariates
P. Dong, R. Han, B. Jiang, and Y. Xu. “Statistical ranking with dynamic covariates”.J. R. Stat. Soc. Ser. B Stat. Methodol.88.1 (2026), pp. 221–238
2026
-
[14]
A Note on the Rank Analysis of Incomplete Block Designs – Applications beyond the Scope of Existing Tables
O. Dykstra. “A Note on the Rank Analysis of Incomplete Block Designs – Applications beyond the Scope of Existing Tables”.Biometrics12.3 (1956), p. 301
1956
-
[15]
Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models
L. Fahrmeir and H. Kaufmann. “Consistency and asymptotic normality of the maximum likelihood estimator in generalized linear models”.Ann. Statist.13.1 (1985), pp. 342–368
1985
-
[16]
Recent advances in the Bradley–Terry Model: theory, algorithms, and applications
S. Fang, R. Han, Y. Luo, and Y. Xu. “Recent advances in the Bradley–Terry Model: theory, algorithms, and applications”.arXiv preprint arXiv:2601.14727(2026)
arXiv 2026
-
[17]
Addressing the assessment challenge with an online system that tutors as it assesses
M. Feng, N. Heffernan, and K. Koedinger. “Addressing the assessment challenge with an online system that tutors as it assesses”.User Model. User-Adapt. Interact.19.3 (2009), pp. 243–266
2009
-
[18]
Solution of a ranking problem from binary comparisons
L. R. Ford Jr. “Solution of a ranking problem from binary comparisons”.Amer. Math. Monthly64.8, part II (1957), pp. 28–33
1957
-
[19]
Uncertainty quantification in the Bradley–Terry–Luce model
C. Gao, Y. Shen, and A. Y. Zhang. “Uncertainty quantification in the Bradley–Terry–Luce model”. Inf. Inference12.2 (2023), pp. 1073–1140
2023
-
[20]
A survey of statistical network models
A. Goldenberg, A. X. Zheng, S. E. Fienberg, and E. M. Airoldi. “A survey of statistical network models”.Found. Trends Mach. Learn.2.2 (2010), pp. 129–233
2010
-
[21]
The many routes to the ubiquitous Bradley-Terry model
I. Hamilton, N. Tawn, and D. Firth. “The many routes to the ubiquitous Bradley-Terry model”.arXiv preprint arXiv:2312.13619(2023)
Pith/arXiv arXiv 2023
-
[22]
A unified analysis of likelihood-based estimators in the Plackett–Luce model
R. Han and Y. Xu. “A unified analysis of likelihood-based estimators in the Plackett–Luce model”. Ann. Statist.53.5 (2025), pp. 2077–2102
2025
-
[23]
A general pairwise comparison model for extremely sparse networks
R. Han, Y. Xu, and K. Chen. “A general pairwise comparison model for extremely sparse networks”. J. Amer. Statist. Assoc.118.544 (2023), pp. 2422–2432. 44
2023
-
[24]
Asymptotic theory of sparse Bradley–Terry model
R. Han, R. Ye, C. Tan, and K. Chen. “Asymptotic theory of sparse Bradley–Terry model”.Ann. Appl. Probab.30.5 (2020), pp. 2491–2515
2020
-
[25]
Detecting a Definite Hermitian Pair and a Hyperbolic or Elliptic Quadratic Eigenvalue Problem, and Associated Nearness Problems
N. J. Higham, F. Tisseur, and P. M. Van Dooren. “Detecting a Definite Hermitian Pair and a Hyperbolic or Elliptic Quadratic Eigenvalue Problem, and Associated Nearness Problems”.Linear Algebra Appl. 351–352 (2002), pp. 455–474
2002
-
[26]
Stochastic blockmodels: First steps
P. W. Holland, K. B. Laskey, and S. Leinhardt. “Stochastic blockmodels: First steps”.Soc. Networks 5.2 (1983), pp. 109–137
1983
-
[27]
MM algorithms for generalized Bradley-Terry models
D. R. Hunter. “MM algorithms for generalized Bradley-Terry models”.Ann. Statist.32.1 (2004), pp. 384–406
2004
-
[28]
Kato.Perturbation theory for linear operators
T. Kato.Perturbation theory for linear operators. Vol. 132. Springer Science & Business Media, 2013
2013
-
[29]
Testing the power of arguments in referendums: A Bradley–Terry approach
P. J. Loewen, D. Rubenson, and A. Spirling. “Testing the power of arguments in referendums: A Bradley–Terry approach”.Elect. Stud.31.1 (2012), pp. 212–221
2012
-
[30]
Conditional logit analysis of qualitative choice behavior
D McFadden. “Conditional logit analysis of qualitative choice behavior”.Frontiers in Economics (1973), pp. 105–142
1973
-
[31]
Rank centrality: Ranking from pairwise comparisons
S. Negahban, S. Oh, and D. Shah. “Rank centrality: Ranking from pairwise comparisons”.Oper. Res. 65.1 (2017), pp. 266–287
2017
-
[32]
Efficient computation of rankings from pairwise comparisons
M. Newman. “Efficient computation of rankings from pairwise comparisons”.J. Mach. Learn. Res. 24.238 (2023), pp. 1–25
2023
-
[33]
J. M. Ortega and W. C. Rheinboldt.Iterative solution of nonlinear equations in several variables. SIAM, 2000
2000
-
[34]
On sinkhorn’s algorithm and choice modeling
Z. Qu, A. Galichon, W. Gao, and J. Ugander. “On sinkhorn’s algorithm and choice modeling”.Oper. Res.(2025)
2025
-
[35]
Direct preference optimization: Your language model is secretly a reward model
R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn. “Direct preference optimization: Your language model is secretly a reward model”. In:NeurIPS. Vol. 36. 2023, pp. 53728– 53741
2023
-
[36]
Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests
G. Rasch. “Studies in mathematical psychology: I. Probabilistic models for some intelligence and attainment tests.” (1960)
1960
-
[37]
Rellich.Perturbation theory of eigenvalue problems
F. Rellich.Perturbation theory of eigenvalue problems. CRC Press, 1969
1969
-
[38]
Saad.Iterative methods for sparse linear systems
Y. Saad.Iterative methods for sparse linear systems. SIAM, 2003
2003
-
[39]
G. W. Stewart and J.-G. Sun.Matrix Perturbation Theory. Boston, MA: Academic Press, 1990
1990
-
[40]
Rethinking Reward Modeling in Preference-based Large Language Model Alignment
H. Sun, Y. Shen, and J.-F. Ton. “Rethinking Reward Modeling in Preference-based Large Language Model Alignment”. In:ICLR. 2025
2025
-
[41]
The method of paired comparisons for social values
L. L. Thurstone. “The method of paired comparisons for social values.”J. Abnorm. Psychol.21.4 (1927), p. 384
1927
-
[42]
The Quadratic Eigenvalue Problem
F. Tisseur and K. Meerbergen. “The Quadratic Eigenvalue Problem”.SIAM Rev.43.2 (2001), pp. 235– 286
2001
-
[43]
User-friendly tail bounds for sums of random matrices
J. A. Tropp. “User-friendly tail bounds for sums of random matrices”.Found. Comput. Math.12.4 (2012), pp. 389–434
2012
-
[44]
Statistical Modelling of Citation Exchange Between Statistics Journals
C. Varin, M. Cattelan, and D. Firth. “Statistical Modelling of Citation Exchange Between Statistics Journals”.J. R. Stat. Soc. Ser. A179.1 (Nov. 2015), pp. 1–63.issn: 0964-1998
2015
-
[45]
Comparing dominance hierarchy methods using a data-splitting approach with real-world data
C. Vilette, T. Bonnell, P. Henzi, and L. Barrett. “Comparing dominance hierarchy methods using a data-splitting approach with real-world data”.Behav. Ecol.31.6 (2020), pp. 1379–1390
2020
-
[46]
Accelerated mm algorithms for inference of ranking scores from comparison data
M. Vojnovi´ c, S.-Y. Yun, and K. Zhou. “Accelerated mm algorithms for inference of ranking scores from comparison data”.Oper. Res.71.4 (2023), pp. 1318–1342
2023
-
[47]
Quantifying hierarchy and dynamics in US faculty hiring and retention
K. H. Wapman, S. Zhang, A. Clauset, and D. B. Larremore. “Quantifying hierarchy and dynamics in US faculty hiring and retention”.Nature610.7930 (2022), pp. 120–127. 45
2022
-
[48]
Efficient inference of rankings from multibody comparisons
J. Yeung, D. Kaiser, and F. Radicchi. “Efficient inference of rankings from multibody comparisons”. Phys. Rev. E112.1 (2025), p. 014305
2025
-
[49]
D. M. Young.Iterative solution of large linear systems. Elsevier, 2014
2014
-
[50]
Die berechnung der turnier-ergebnisse als ein maximumproblem der wahrscheinlichkeit- srechnung
E. Zermelo. “Die berechnung der turnier-ergebnisse als ein maximumproblem der wahrscheinlichkeit- srechnung”.Math. Z.29.1 (1929), pp. 436–460. 46
1929
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.