REVIEW 4 major objections 6 minor 1 cited by
Elo Ratings in the Presence of Intransitivity
T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read This paper proves that when a game's win probabilities are intransitive, the final Elo ratings depend on which players are scheduled to play whom, not just on their skill.
desk verdict The fixed-point theorem is real and clean, but the paper oversells it as a statement about observed Elo ratings when the stochastic convergence bridge is not proved. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the stability equation $\operatorname{div}(Q \odot P) = \operatorname{div}(Q \odot \sigma(\operatorname{grad}(r)))$, which states that at a final Elo score the conditionally expected one-step change in ratings is zero, where $Q$ is the selection matrix, $P$ the win-probability matrix, $\sigma$ the logistic function, and $\operatorname{grad}$ and $\operatorname{div}$ the combinatorial gradient and divergence from Hodge theory. Existence of a solution follows from applying Brouwer's fixed-point theorem to the expected-update map $f(r) = r + \eta \operatorname{div}(Q \odot (P - \sigma(\operatorname{grad}(r))))$ on a large enough hypersphere. Uniqueness is proved by showing the map $g(r) = \operatorname{div}(Q \odot \sigma(\operatorname{grad}(r)))$ is injective when $Q$'s graph is strongly connected, using the orthogonality of the kernel of $\operatorname{div}$ to the image of $\operatorname{grad}$. The schedule-dependence result uses spanning trees: on a tree, ratings are forced by summing $\sigma^{-1}(P)$ along the unique paths, and path-dependence of these sums is exactly intransitivity.
What would settle it
Run a long simulation of an intransitive game with a fixed connected selection matrix Q, record the time-averaged Elo ratings, and compare them to the unique solution of the stability equation; if the averages do not track the fixed point and instead cycle or converge elsewhere, the paper's notion of long-term behaviour is not the behaviour the system exhibits. A second check: compute the final ratings for two different spanning-tree selection matrices and verify empirically that the long-run averages land at the two different fixed points, as Theorem 4.1 predicts.
Extended reading notes
Core claim
The central claim is Theorem 4.1: if the advantage matrix $M = \sigma^{-1}(P)$ is not a strongly transitive additive comparison matrix, meaning the win probabilities cannot be written as $\sigma(r_i - r_j)$ for any skill vector $r$, then there exist two selection matrices $Q_1$ and $Q_2$ whose associated final Elo scores are different. The proof constructs two spanning interaction trees that contain different paths between the same pair of players; because the sums of $\sigma^{-1}(P)$ along the paths differ, the ratings forced by the stability equation differ. Sections 5 and 6 show the complementary result: for any fixed selection matrix whose graph has one strongly connected component, the stability equation has a unique solution, found by Brouwer's fixed-point theorem and an injectivity argument. Thus an intransitive game has a whole set of possible final ratings, called the Elotope, and Elo selects one point of it depending on the fixture.
Load-bearing premise
The paper defines the final Elo score as a fixed point of the expected one-step update and assumes this deterministic object describes where the stochastic Elo Markov chain really goes in the long run, even though it does not prove the chain has a limiting distribution or that it converges to this fixed point.
Editorial extensions
If this is right
- In any game whose win probabilities are intransitive, Elo ratings are schedule-dependent: the same set of players can receive different final scores under different match fixtures.
- Despite that, Elo still has a unique final rating for each fixed, connected schedule, so the system does not lose its fixed-point property when transitivity fails.
- The set of all possible final ratings over all schedules, the Elotope, grows with the cyclic component of the advantage matrix, so the spread of possible ratings is a measure of how much intransitivity distorts the ratings.
- The schedule-dependence explains why adaptive matchmaking, where opponents are chosen from current ratings, can prevent convergence: changing $Q$ moves the target fixed point while the process is chasing it.
- The proposed intransitivity measure $I(A)$ stays near or below 1 for predominantly transitive games and rises above 1 when the cyclic component dominates, matching what one would expect on Rock-Paper-Scissors examples.
Reading between the lines
- The schedule-dependence mechanism is not specific to Elo: any pairwise scalar-rating update whose fixed point is a weighted sum of edge contributions will inherit the dependence on the edge weights of the schedule.
- A more direct index of rating instability than $I(A)$ would be the diameter of the Elotope, the maximum difference between final scores achievable by varying $Q$; the paper describes the Elotope but does not formalize this diameter.
- Because the paper proves only that the mean-field fixed point exists and is unique, not that the stochastic Elo chain converges to it, the practical relevance depends on whether time averages of real Elo updates track the fixed point; this could be tested by simulation across different learning rates $\eta$ and selection matrices.
- The intransitivity measure requires full pairwise win probabilities, which are rarely available in real sports data; a useful extension would be to estimate $I(A)$ from the sparse match data that Elo itself uses, avoiding the circularity the paper notes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the Elo rating system when pairwise win probabilities are not consistent with a transitive, additive skill model. It defines the "final Elo score" as a fixed point of the conditional expected one-step update (the stability equation), proves that this fixed point depends on the schedule of matches when the advantage matrix is not a strongly transitive additive comparison matrix (Theorem 4.1), claims existence and uniqueness of the fixed point for connected selection matrices (Theorems 5.1 and 6.1), and introduces an intransitivity measure I(A) validated on simulated Rock-Paper-Scissors data. The spanning-tree construction behind Theorem 4.1 is elegant, and the proposed measure is clearly described and behaves sensibly in the simulations. The main gaps are the missing connection between the mean-field fixed point and the actual stochastic Elo chain, and several incomplete proofs in the existence and uniqueness arguments.
Significance. If the results were fully established, the paper would make an important point for practitioners: scalar Elo ratings in intransitive games are not determined by the players' abilities alone, and the match schedule can change the final ratings. The uniqueness claim would also provide a useful stability property. The paper correctly identifies the gap between the fixed point of the expected update and the long-run behavior of the stochastic chain, but it does not bridge that gap. The intransitivity measure and its simulations are a useful contribution, though they validate only the metric, not the convergence claims. The use of combinatorial Hodge theory is appropriate, and the authors are honest about several limitations, but the advertised practical conclusion currently exceeds what the proofs establish.
major comments (4)
- [Section 3.2.1 and Theorem 4.1] The central practical claim that "estimated ratings are dependent on who plays who" is not established for the stochastic Elo process. The paper defines the final Elo score as a fixed point of the conditional expected one-step update and explicitly states that no proof of stationarity, recurrence, or convergence to this fixed point is given. With a constant gain eta, the Markov chain cannot converge to a point; at best one would need a stationary-distribution concentration result as eta -> 0, and no such result or bound on the spread is provided. Theorem 4.1 therefore describes the deterministic mean-field equation, not necessarily "the ratings users observe." The simulations in Section 7 validate the intransitivity measure, not this bridge. The authors should either prove a concentration statement, state clearly that all results concern the mean-field fixed point, or both.
- [Section 5.3, Lemma 5.4 and Step 1.3] The existence proof via Brouwer's fixed-point theorem is incomplete. Lemma 5.4 is justified by "can be verified visually" for m = 3, with no rigorous argument for general m. Step 1.3 reasons about the limit as the radius of K approaches infinity and then asserts that the inward components "will stay bounded from below" without proof; the passage from asymptotic behavior to a fixed finite radius is not justified. Section 5.4 then extends the radius by sqrt(2 eta) and asserts that the inward-direction property persists on the inner shell, which does not follow from the boundary argument alone. As written, Theorem 5.1's existence claim lacks a complete proof.
- [Section 6, Theorem 6.1] The uniqueness proof is not valid as written. The argument that tangent vectors to sigma(im(grad)) and vectors in ker(div) lie in different orthants is unsupported: Euclidean orthogonality of the two subspaces does not imply sign-opposition of the entries of their vectors. The stated sufficient condition for global injectivity of a projection is only a local condition; without a properness or covering argument it does not imply global injectivity. The "if and only if" direction for disconnected Q is asserted rather than proved. The theorem may be true and can be proved by a monotonicity argument: for connected Q, <g(r)-g(s), r-s> > 0 for r != s because (x-y)(sigma(x)-sigma(y)) > 0 for the logistic function. I recommend replacing the current proof with such an argument.
- [Abstract and Introduction] The claim that the fixed point "represents the maximum likelihood estimator of the Elo ratings, given the observed data" is not proven anywhere in the manuscript. The stability equation is an expected score equation, and connecting it to the MLE for realized data requires a likelihood calculation or an explicit stochastic-approximation theorem. As written, this advertised contribution is unsupported and should be either proved or removed.
minor comments (6)
- [Title] The title contains a typo: "Presense" should be "Presence."
- [Section 3.2, Eq. (4)] With div(A) defined as the row-mean in Section 2, Eq. (3) equals m * eta * div(Q circ (P - sigma(grad(r)))), not eta * div(...). The factor m cancels in the stability equation, but the displayed equality is false as written.
- [Section 5.1] The sum index in "the subspace sum_{i=0}^m r_i = 0" should be i=1, not i=0.
- [Section 6] The text uses "simply connected interaction network" to mean a connected graph and refers to "strongly connected component" for an undirected graph; the standard term is "connected component."
- [Sections 2 and 7] The symbol A denotes both the cyclic component in the Hodge decomposition and the advantage matrix in the intransitivity measure (Eq. (8)); this overloaded notation is confusing and should be changed.
- [Section 7.1.2, Figure 9] The "vertical asymptote at t = 1" is a limiting statement because logit(0) and logit(1) are infinite; the text should describe it as a limit of the ground-truth calculation rather than a property that the finite simulations display.
Circularity Check
No significant circularity: the central theorems are proven from explicit stability-equation assumptions, with no fitted input renamed as prediction.
full rationale
The paper's derivation chain is self-contained for what it proves. Theorem 4.1 takes the paper's own definition of final Elo score as a fixed point of the expected one-step update (Eq. 5) and shows by an explicit spanning-tree construction that a non-STACM advantage matrix yields two selection matrices with different fixed-point ratings; this is a mathematical property of Eq. 5, not an input to Eq. 5. Theorems 5.1 and 6.1 prove existence and uniqueness from Brouwer's fixed-point theorem and an injectivity argument, with no fitted constants. Section 7's intransitivity measure, Eq. (8), is a definition rather than a fitted prediction, and the simulations compare it to ground-truth values as a sanity check; the paper explicitly avoids estimating the advantage matrix through Elo because this would create a circular chain of reasoning (Section 7). The only self-citation, [19], supplies optional Elotope geometry and is not load-bearing. The gap the paper itself acknowledges—that it proves properties of the fixed point of the expected update, not stationarity or convergence of the stochastic Elo chain (Section 3.2.1)—is a scope limitation for the practical interpretation of the theorems, not circular reasoning.
Assumptions & free parameters
free parameters (1)
- Additive constant 1 in intransitivity measure I(A) =
1
assumptions (6)
- domain assumption The Elo model: P_ij = sigma(r_i - r_j) for a logistic sigmoid sigma, and ratings update by transferring a fraction of the difference between observed and expected outcome.
- ad hoc to paper Long-term behavior of the Elo Markov chain is characterized by the fixed point of the conditional expected one-step update (the stability equation).
- domain assumption Selection matrix Q is constant and independent of game outcomes.
- standard math Hodge decomposition of skew-symmetric matrices into gradient and cyclic components (Jiang et al. 2011).
- standard math Brouwer's fixed-point theorem.
- ad hoc to paper Global injectivity of the projection of sigma(grad(r)) onto the rating subspace follows from the tangent space avoiding the kernel of div.
invented entities (2)
-
Elotope
-
Intransitivity measure I(A)
Cite this review
Pith. "Pith review of Elo Ratings in the Presence of Intransitivity." pith.science (2026). https://pith.science/paper/Q2MZ2LST
@misc{pith2026241214427,
author = {Pith},
title = {Pith review of: Elo Ratings in the Presence of Intransitivity},
year = {2026},
howpublished = {\url{https://pith.science/paper/Q2MZ2LST}},
note = {Machine review of arXiv:2412.14427}
}
read the original abstract
This paper studies how the Elo rating system behaves when the underlying modelling assumptions are not met.
Figures
Forward citations
Cited by 1 Pith paper
-
Capturing Intransitive Dominance in Tennis Forecasting: A Graph Neural Network Approach
A MagNet GNN on temporal directed head-to-head graphs matches Elo-level tennis forecast accuracy and earns small but statistically significant betting profits on high-intransitivity matchups, though the headline combi...
Reference graph
Works this paper leans on
-
[1]
Aldous, D. (2017). Elo Ratings and the Sports Model: A Neglected Topic in Applied Probability?Statistical Science 32 616 – 629. https://doi. org/10.1214/17-STS628
-
[2]
Balduzzi, D. , Tuyls, K. , Perolat, J. and Graepel, T. (2018). Re-evaluating evaluation. In Advances in Neural Information Processing Systems (S. Bengio , H. W allach, H. Larochelle , K. Grauman , N. Cesa-Bianchi and R. Garnett, eds.) 31. Curran Associates, Inc
work page 2018
- [3]
-
[4]
Bertrand, Q., Czarnecki, W. M. and Gidel, G. (2022). On the Lim- itations of Elo: Real-World Games, are Transitive, not Additive.CoRR abs/2206.12301. https://doi.org/10.48550/arXiv.2206.12301
-
[5]
Bolsinova, M., Gergely, B. and Brinkhuis, M. (2024). Keeping Elo alive: Evaluating and improving measurement properties of learning sys- tems based on Eloratings.preprint. Elo Ratings in the Presense of Intransitivity 27
work page 2024
-
[6]
Brooks, S. (2011). Handbook of Markov chain Monte Carlo, 1st ed. ed. Chapman & Hall CRC handbooks of modern statistical methods. Taylor & Francis, Boca Raton, Fla
work page 2011
-
[7]
chess. com (2017). How Many Chess Players Are There In The World? Accessed on 2023-04-24 14:57:39 +0930
work page 2017
- [8]
Show all 42 references
-
[9]
Czarnecki, W. M. , Gidel, G. , Tracey, B. , Tuyls, K. , Omid- shafiei, S., Balduzzi, D. and Jaderberg, M. (2020).RealWorldGames Look like Spinning Tops. InProceedings of the 34th International Confer- ence on Neural Information Processing Systems. NIPS’20. Curran Asso- ciates ...
2020
-
[10]
, Lambrecht, M
Duersch, P. , Lambrecht, M. and Oechssler, J. (2020). Measuring skill and chance in games. European Economic Review 127. https:// doi.org/10.1016/j.euroecorev.2020
2020 doi
-
[11]
Dumoulin, V., Johnson, D. D. , Castro, P. S. , Larochelle, H. and Dauphin, Y. (2024). A density estimation perspective on learning from pairwise human preferences
2024
-
[12]
, TORREGROSSA, M
Düring, B. , TORREGROSSA, M. and WOLFRAM, M.-T. (2018). On a kinetic Elo rating model for players with dynamical strength.arXiv preprint arXiv:1806.06648
2018 arXiv
-
[13]
, Torregrossa, M
Düring, B. , Torregrossa, M. and Wolfram, M.-T. (2019). Boltz- mann and Fokker–Planck equations modelling the Elo rating system with learning effects.Journal of Nonlinear Science29 1095–1128
2019
-
[14]
and Liu, P
Ebtekar, A. and Liu, P. (2021). Elo-MMR: A Rating System for Mas- sive Multiplayer Competitions. InProceedings of the Web Conference 2021. WWW ’21 1772–1784. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3442381.3450091
2021
-
[15]
Elo, A. E. (2008). The Rating of Chess Players, Past and Present. Ishi Press International
2008
-
[16]
Firth, D. (2005). Bradley-Terry Models in R.Journal of Statistical Soft- ware 12 1–12. https://doi.org/10.18637/jss.v012.i01
2005 doi
-
[17]
Glickman, M. (2022). Glicko Ratings:. Accessed on 2023-06-27 15:57:39 +0930
2022
-
[18]
and Rodríguez, J
Gómez-Abejón, G. and Rodríguez, J. T. (2024). Stochastic Extensions of the Elo Rating System. Applied Sciences 14. https://doi.org/10. 3390/app14178023
2024
-
[19]
Hamilton, A. (2023). A Multi-Dimensional Extension of the Elo Rating System. PhD Thesis, University of Adelaide
2023
-
[20]
Harris, S. N. and Tauritz, D. R. (2021). Competitive Coevolution for Defense and Security: Elo-Based Similar-Strength Opponent Sampling. In Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO ’21 1898–1906. Association for Computing Machin- ery,...
2021
-
[21]
, Chang, C.-J
Hua, H.-F. , Chang, C.-J. , Lin, T.-C. and Weng, R. C.-H. (2024). RatingplayersbyLaplace’sapproximationanddynamicmodeling. Interna- tional Journal of Forecasting40 1152-1165. https://doi.org/10.1016/ 28 A. Hamilton et al. j.ijforecast.2023.10.004
2024
-
[22]
Ingram, M. (2021). How to extend Elo: a Bayesian perspective.Journal of Quantitative Analysis in Sports 17 203–219. https://doi.org/doi: 10.1515/jqas-2020-0066
2021 doi
-
[23]
(2011).Statisticalrankingand combinatorial Hodge theory.Mathematical Programming127 203–244
Jiang, X., Lim, L.-H., Yao, Y.and Ye, Y. (2011).Statisticalrankingand combinatorial Hodge theory.Mathematical Programming127 203–244
2011
-
[24]
W., Mikhailov, L., Redlarski, G., Soltys, M., Szy- bowski, J., Tamazian, G., W ajch, E.and Yuen, K
Koczkodaj, W. W., Mikhailov, L., Redlarski, G., Soltys, M., Szy- bowski, J., Tamazian, G., W ajch, E.and Yuen, K. K. F. (2016). Im- portant Facts and Observations about Pairwise Comparisons (the special issue edition).Fundamenta Informaticae144 291–307
2016
-
[25]
Kovalchik, S. (2020). Extension of the Elo rating system to margin of victory. International Journal of Forecasting36 1329-1341. https://doi. org/10.1016/j.ijforecast.2020.01.006
2020 doi
-
[26]
Langville, A. N. and Meyer, C. (2012). Who’s #1? The Science of Rating and Ranking. Princeton University Press, Princeton
2012
-
[27]
Lyons, K. (2014). What are the World Football Elo Ratings? Accessed on 2023-05-1 14:57:39 +0930
2014
-
[28]
Milnor, J. (1965). Topology from the Differentiable Viewpoint. University Press of Virginia
1965
-
[29]
and Probst, C
Pieters, W., van der Ven, S. and Probst, C. (2012). A move in the security measurement stalemate: Elo-style ratings to quantify vulnerability. In Proceedings of the 2012 New Security Paradigms Workshop. NSPW ’12 1-14. ACM
2012
-
[30]
Sackmann, J. (2019). An Introduction to Tennis Elo. Accessed on 2023- 05-1 14:57:39 +0930
2019
-
[31]
Sizemore, R. K. (2013). Hodgerank: Applying combinatorial Hodge the- ory to sports ranking
2013
-
[32]
(2022).G-Elo:generalizationoftheEloalgorithmbymod- eling the discretized margin of victory.Journal of Quantitative Analysis in Sports 18 1–14
Szczecinski, L. (2022).G-Elo:generalizationoftheEloalgorithmbymod- eling the discretized margin of victory.Journal of Quantitative Analysis in Sports 18 1–14. https://doi.org/doi:10.1515/jqas-2020-0115
2022 doi
-
[33]
and Djebbi, A
Szczecinski, L. and Djebbi, A. (2019). Understanding and Pushing the Limits of the Elo Rating Algorithm.arXiv preprint arXiv:1910.06081
2019 arXiv
-
[34]
and Djebbi, A
Szczecinski, L. and Djebbi, A. (2020). Understanding draws in Elo rating algorithm. Journal of Quantitative Analysis in Sports16 211–220. https://doi.org/doi:10.1515/jqas-2019-0102
2020 doi
-
[35]
and Tihon, R
Szczecinski, L. and Tihon, R. (2023). Simplified Kalman filter for on- line rating: one-fits-all approach.Journal of Quantitative Analysis in Sports 19 295–315
2023
-
[36]
Tenkanen, S. (2019). Rating National Hockey League teams: the predic- tive power of Elo rating models in ice hockey, Bachelor’s thesis, Aalto Uni- versity, Espoo Finland Available athttps://aaltodoc.aalto.fi/items/ d163b9aa-bcdc-4d64-90d0-6efee35a3cf5
2019
-
[37]
Tran, N. M. (2013). Pairwise ranking: choice of method can produce arbitrarily different rank order. Linear Algebra and its Applications438 1012–1024
2013
-
[38]
Tsang, C. S. C. , Ngan, H. Y. T. and Pang, G. K. H. (2016). Fabric Elo Ratings in the Presense of Intransitivity 29 inspection based on the Elo rating method.Pattern Recognition51378-394. https://doi.org/10.1016/j.patcog.2015.09.022
2016 doi
-
[39]
and Gerrard, H
V aughan-Williams, L., Liu, C., Dixon, L. and Gerrard, H. (2021). How well do Elo-based ratings predict professional tennis matches?Journal of Quantitative Analysis in Sports17 91–105
2021
-
[40]
and Yao, Y
Xu, Q., Huang, Q., Jiang, T., Yan, B., Lin, W. and Yao, Y. (2012). HodgeRank on random graphs for subjective video quality assessment. IEEE Transactions on Multimedia14 844–857
2012
-
[41]
and Chen, X
Yan, X., Du, Y., Ru, B., W ang, J., Zhang, H. and Chen, X. (2022). Learning to Identify Top Elo Ratings: A Dueling Bandits Approach In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI6375–6383. AAAI Press
2022
-
[42]
Yudelson, M. (2019). Elo, I Love You Won’t You Tell Me Your K. In Transforming Learning with Meaningful Technologies(M. Scheffel , J. Broisin , V. Pammer-Schindler, A. Ioannou and J. Schneider , eds.) 213–223. Springer International Publishing, Cham
2019
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.