{"id":"ae9129b3-bc08-4881-aa40-48396f37941d","arxiv_id":"2412.14427","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Elo ratings in intransitive games are unique for a given schedule but shift with the schedule, so they cannot represent a single transitive skill scale.","lead":"This paper proves that when a game's true win probabilities are intransitive, the final Elo ratings depend on the schedule of who plays whom, not just on skill. It also shows a unique rating fixed point exists for any connected schedule and proposes a statistic to measure how much intransitivity a game has.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4.1 is sound for the fixed-point equation, but the paper never proves that the stochastic Elo chain actually converges to this fixed point, so the claimed schedule-dependence of real Elo ratings is not established.","rationale":"The paper's core mathematical contribution, Theorem 4.1, is a theorem about the stability equation (5), and its spanning-tree proof is correct: for a tree, the leaf equations force r_i - r_j = σ^{-1}(P_ij) along every edge, so path-sum ratings satisfy the fixed-point equation, and non-STACM M yields two trees with different induced rating vectors. The existence and uniqueness theorems are plausible and likely true (the uniqueness claim can be proven by strict convexity of the weighted log-likelihood on the sum-zero subspace), though the written proofs have gaps—the Brouwer argument is informal and the uniqueness proof gives only local injectivity. These gaps are fillable and would not by themselves overturn the theorem. The more serious gap is interpretive: the paper defines 'final Elo score' as a fixed point of the conditional expected update and asserts, without proof, that this captures the long-term behavior of the actual Elo Markov chain. Since the practical claim in the title and abstract is about Elo ratings as used in real systems, this assumption is load-bearing. The paper explicitly states that it cannot show stationarity, and it provides no stochastic-approximation argument (e.g., Kushner-Clark or Borkar) to connect the constant-step-size chain to the ODE. The empirical section validates an intransitivity metric, not the convergence of Elo scores. Thus the central claim, while mathematically correct as a statement about Eq. (5), is not fully established as a statement about the ratings the Elo algorithm produces. This matches the reader's weakest-assumption analysis, so I agree with the conditional verdict: the paper should be accepted only if the fixed-point-to-chain bridge is either proven, or clearly positioned as a mean-field approximation with supporting simulations.","tokens_in":19709,"tokens_out":16994,"duration_ms":132692,"concrete_test":"Simulate the Elo Markov chain for a three-player intransitive game (e.g., a cyclic advantage matrix) with a small learning rate η, under at least two different connected selection matrices Q1 and Q2, including two distinct spanning trees and a full-support matrix. For each Q, run many independent long trajectories, discard burn-in, and compute the time-averaged rating vector. Compare these averages to the unique solutions of the stability equation (5) for the same Q, and test whether the Q1 and Q2 averages differ in the direction predicted by Theorem 4.1. Repeat for η ∈ {0.1, 0.05, 0.01} and for increasing trajectory lengths to check whether the time averages concentrate on the fixed points as η → 0.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theorem, Theorem 4.1, is internally sound: for a spanning-tree selection matrix Q, the stability equation reduces to the path-sum ratings assigned along the tree, and a non-STACM advantage matrix yields two trees with different r_i - r_j for some pair (i,j). However, the paper's headline conclusion—that Elo ratings depend on who plays whom—requires that the fixed point of the expected one-step drift is what the stochastic Elo process actually settles near. Section 3.2.1 explicitly disclaims any proof of stationarity, recurrence, or convergence to the fixed point. The deterministic map f(r) = r + η div(Q⊙(P−σ(grad(r)))) has the fixed point as an equilibrium of its mean-field ODE, but the actual chain uses a constant step size η and cannot converge to a point; at best, its stationary distribution (if it exists) would be concentrated near the fixed point only as η→0. The paper neither proves existence of such a stationary distribution nor bounds the spread around the fixed point. Without this bridge, Theorem 4.1 describes the mean-field equation, not 'the ratings users observe.' The simulations in Section 7 validate the intransitivity metric, not the convergence of Elo scores to the fixed point, so there is no empirical support for the bridge either. If the stochastic process has a different long-run behavior—for example, a limit cycle or a stationary distribution with substantial width that varies with Q in the opposite direction—the paper's main practical claim could fail even though Eq. (5) is solved correctly. This is the load-bearing gap: the fixed point may be well-defined and unique, but its relevance to actual Elo ratings is assumed, not demonstrated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the Elo rating system when pairwise win probabilities are not consistent with a transitive, additive skill model. It defines the \"final Elo score\" as a fixed point of the conditional expected one-step update (the stability equation), proves that this fixed point depends on the schedule of matches when the advantage matrix is not a strongly transitive additive comparison matrix (Theorem 4.1), claims existence and uniqueness of the fixed point for connected selection matrices (Theorems 5.1 and 6.1), and introduces an intransitivity measure I(A) validated on simulated Rock-Paper-Scissors data. The spanning-tree construction behind Theorem 4.1 is elegant, and the proposed measure is clearly described and behaves sensibly in the simulations. The main gaps are the missing connection between the mean-field fixed point and the actual stochastic Elo chain, and several incomplete proofs in the existence and uniqueness arguments.","tokens_in":20072,"tokens_out":12627,"duration_ms":118736,"significance":"If the results were fully established, the paper would make an important point for practitioners: scalar Elo ratings in intransitive games are not determined by the players' abilities alone, and the match schedule can change the final ratings. The uniqueness claim would also provide a useful stability property. The paper correctly identifies the gap between the fixed point of the expected update and the long-run behavior of the stochastic chain, but it does not bridge that gap. The intransitivity measure and its simulations are a useful contribution, though they validate only the metric, not the convergence claims. The use of combinatorial Hodge theory is appropriate, and the authors are honest about several limitations, but the advertised practical conclusion currently exceeds what the proofs establish.","major_comments":[{"comment":"The central practical claim that \"estimated ratings are dependent on who plays who\" is not established for the stochastic Elo process. The paper defines the final Elo score as a fixed point of the conditional expected one-step update and explicitly states that no proof of stationarity, recurrence, or convergence to this fixed point is given. With a constant gain eta, the Markov chain cannot converge to a point; at best one would need a stationary-distribution concentration result as eta -> 0, and no such result or bound on the spread is provided. Theorem 4.1 therefore describes the deterministic mean-field equation, not necessarily \"the ratings users observe.\" The simulations in Section 7 validate the intransitivity measure, not this bridge. The authors should either prove a concentration statement, state clearly that all results concern the mean-field fixed point, or both.","section":"Section 3.2.1 and Theorem 4.1"},{"comment":"The existence proof via Brouwer's fixed-point theorem is incomplete. Lemma 5.4 is justified by \"can be verified visually\" for m = 3, with no rigorous argument for general m. Step 1.3 reasons about the limit as the radius of K approaches infinity and then asserts that the inward components \"will stay bounded from below\" without proof; the passage from asymptotic behavior to a fixed finite radius is not justified. Section 5.4 then extends the radius by sqrt(2 eta) and asserts that the inward-direction property persists on the inner shell, which does not follow from the boundary argument alone. As written, Theorem 5.1's existence claim lacks a complete proof.","section":"Section 5.3, Lemma 5.4 and Step 1.3"},{"comment":"The uniqueness proof is not valid as written. The argument that tangent vectors to sigma(im(grad)) and vectors in ker(div) lie in different orthants is unsupported: Euclidean orthogonality of the two subspaces does not imply sign-opposition of the entries of their vectors. The stated sufficient condition for global injectivity of a projection is only a local condition; without a properness or covering argument it does not imply global injectivity. The \"if and only if\" direction for disconnected Q is asserted rather than proved. The theorem may be true and can be proved by a monotonicity argument: for connected Q, <g(r)-g(s), r-s> > 0 for r != s because (x-y)(sigma(x)-sigma(y)) > 0 for the logistic function. I recommend replacing the current proof with such an argument.","section":"Section 6, Theorem 6.1"},{"comment":"The claim that the fixed point \"represents the maximum likelihood estimator of the Elo ratings, given the observed data\" is not proven anywhere in the manuscript. The stability equation is an expected score equation, and connecting it to the MLE for realized data requires a likelihood calculation or an explicit stochastic-approximation theorem. As written, this advertised contribution is unsupported and should be either proved or removed.","section":"Abstract and Introduction"}],"minor_comments":[{"comment":"The title contains a typo: \"Presense\" should be \"Presence.\"","section":"Title"},{"comment":"With div(A) defined as the row-mean in Section 2, Eq. (3) equals m * eta * div(Q circ (P - sigma(grad(r)))), not eta * div(...). The factor m cancels in the stability equation, but the displayed equality is false as written.","section":"Section 3.2, Eq. (4)"},{"comment":"The sum index in \"the subspace sum_{i=0}^m r_i = 0\" should be i=1, not i=0.","section":"Section 5.1"},{"comment":"The text uses \"simply connected interaction network\" to mean a connected graph and refers to \"strongly connected component\" for an undirected graph; the standard term is \"connected component.\"","section":"Section 6"},{"comment":"The symbol A denotes both the cyclic component in the Hodge decomposition and the advantage matrix in the intransitivity measure (Eq. (8)); this overloaded notation is confusing and should be changed.","section":"Sections 2 and 7"},{"comment":"The \"vertical asymptote at t = 1\" is a limiting statement because logit(0) and logit(1) are infinite; the text should describe it as a limit of the ground-truth calculation rather than a property that the finite simulations display.","section":"Section 7.1.2, Figure 9"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid core idea and the spanning-tree proof of Theorem 4.1 is elegant. The main obstacles are the missing stochastic-process bridge and the incomplete existence and uniqueness proofs. The uniqueness proof is likely repairable via a monotonicity argument, and the authors may be able to reframe the paper as a mean-field analysis if a full stochastic concentration result is out of reach. I would encourage the editor to invite a revision rather than reject."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. The paper proves a real theorem: for the fixed-point equation defining Elo's expected one-step update, the solution depends on the schedule matrix Q whenever the advantage matrix is not a strongly transitive additive comparison matrix. The spanning-tree argument is clean and genuinely new. Second, the paper does not prove that the stochastic Elo chain actually converges to that fixed point. The authors admit as much in Section 3.2.1, but the abstract and introduction claim more than the theorems support. So the schedule-dependence result is solid for the mean-field equation, not for the ratings users observe.\n\nWhat is good: Theorem 4.1 is a genuine advance over the informal claims in Balduzzi et al. The existence and uniqueness theorems for connected Q are plausible and likely correct, though the proofs are not fully rigorous. The intransitivity measure is a reasonable definition, and the simulated validation is a useful sanity check even without code or error bars.\n\nSoft spots: Lemma 5.4 is justified by \"can be verified visually\" for m=3 and then asserted to generalize by symmetry; that is a real gap. The uniqueness proof in Section 6 leans on an orthant argument that is not convincing as written; the result is probably true by monotonicity, but the written proof needs work. The bigger issue is the stochastic bridge: the fixed point of the expected update is not shown to be the limit of the Elo chain. The paper is honest about this in the technical sections, but the framing overreaches. If the chain has a stationary distribution that is not concentrated around the fixed point, or if the step size matters, the schedule-dependence could look different. The simulations only validate the intransitivity measure, not the convergence claim.\n\nWho this is for: people working on rating systems, Elo variants, and learning from pairwise comparisons. They will get a useful formal result and a clear statement of a gap that matters. The paper deserves a serious referee; with the proofs tightened and the claims scaled back to the fixed-point equation, it would be publishable.\n\nRecommendation: engage with it. Send it to review and ask the authors to either prove a concentration result for the stochastic chain or rewrite the framing to match the theorems.","headline":"The fixed-point theorem is real and clean, but the paper oversells it as a statement about observed Elo ratings when the stochastic convergence bridge is not proved.","tokens_in":20609,"tokens_out":3619,"would_cite":true,"duration_ms":34033,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60G99","00X00","91A99"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proves that when a game's win probabilities are intransitive, the final Elo ratings depend on which players are scheduled to play whom, not just on their skill.","keywords":["Elo rating system","Intransitivity","Selection matrix","Stability equation","Hodge decomposition","Brouwer fixed point","Bradley-Terry model","Rock-Paper-Scissors"],"falsifier":"Run a long simulation of an intransitive game with a fixed connected selection matrix Q, record the time-averaged Elo ratings, and compare them to the unique solution of the stability equation; if the averages do not track the fixed point and instead cycle or converge elsewhere, the paper's notion of long-term behaviour is not the behaviour the system exhibits. A second check: compute the final ratings for two different spanning-tree selection matrices and verify empirically that the long-run averages land at the two different fixed points, as Theorem 4.1 predicts.","tokens_in":19478,"feed_emoji":"♟️","tokens_out":7825,"duration_ms":61545,"temperature":0.7,"pith_summary":"The paper asks what happens to the Elo rating system when the game being rated is intransitive, meaning that player A may usually beat B, B usually beats C, yet C usually beats A. It proves that in that case the final Elo ratings are not a property of the players alone: they depend on the selection matrix, the probabilities with which each pair of players is scheduled to play. At the same time, it shows that for any fixed schedule with a connected comparison graph there is exactly one rating vector satisfying the stability equation, so Elo still has a well-defined fixed point. The paper also defines a scalar measure of intransitivity based on the Hodge decomposition of the win-probability matrix, and validates it on simulated Rock-Paper-Scissors data.","feed_headline":"When wins are intransitive, Elo ratings depend on who plays whom","feed_subtitle":"The same players get different final scores under different fixtures, and a new metric quantifies the effect.","key_machinery":"The load-bearing object is the stability equation $\\operatorname{div}(Q \\odot P) = \\operatorname{div}(Q \\odot \\sigma(\\operatorname{grad}(r)))$, which states that at a final Elo score the conditionally expected one-step change in ratings is zero, where $Q$ is the selection matrix, $P$ the win-probability matrix, $\\sigma$ the logistic function, and $\\operatorname{grad}$ and $\\operatorname{div}$ the combinatorial gradient and divergence from Hodge theory. Existence of a solution follows from applying Brouwer's fixed-point theorem to the expected-update map $f(r) = r + \\eta \\operatorname{div}(Q \\odot (P - \\sigma(\\operatorname{grad}(r))))$ on a large enough hypersphere. Uniqueness is proved by showing the map $g(r) = \\operatorname{div}(Q \\odot \\sigma(\\operatorname{grad}(r)))$ is injective when $Q$'s graph is strongly connected, using the orthogonality of the kernel of $\\operatorname{div}$ to the image of $\\operatorname{grad}$. The schedule-dependence result uses spanning trees: on a tree, ratings are forced by summing $\\sigma^{-1}(P)$ along the unique paths, and path-dependence of these sums is exactly intransitivity.","core_discovery":"The central claim is Theorem 4.1: if the advantage matrix $M = \\sigma^{-1}(P)$ is not a strongly transitive additive comparison matrix, meaning the win probabilities cannot be written as $\\sigma(r_i - r_j)$ for any skill vector $r$, then there exist two selection matrices $Q_1$ and $Q_2$ whose associated final Elo scores are different. The proof constructs two spanning interaction trees that contain different paths between the same pair of players; because the sums of $\\sigma^{-1}(P)$ along the paths differ, the ratings forced by the stability equation differ. Sections 5 and 6 show the complementary result: for any fixed selection matrix whose graph has one strongly connected component, the stability equation has a unique solution, found by Brouwer's fixed-point theorem and an injectivity argument. Thus an intransitive game has a whole set of possible final ratings, called the Elotope, and Elo selects one point of it depending on the fixture.","pith_inferences":["The schedule-dependence mechanism is not specific to Elo: any pairwise scalar-rating update whose fixed point is a weighted sum of edge contributions will inherit the dependence on the edge weights of the schedule.","A more direct index of rating instability than $I(A)$ would be the diameter of the Elotope, the maximum difference between final scores achievable by varying $Q$; the paper describes the Elotope but does not formalize this diameter.","Because the paper proves only that the mean-field fixed point exists and is unique, not that the stochastic Elo chain converges to it, the practical relevance depends on whether time averages of real Elo updates track the fixed point; this could be tested by simulation across different learning rates $\\eta$ and selection matrices.","The intransitivity measure requires full pairwise win probabilities, which are rarely available in real sports data; a useful extension would be to estimate $I(A)$ from the sparse match data that Elo itself uses, avoiding the circularity the paper notes."],"forward_implications":["In any game whose win probabilities are intransitive, Elo ratings are schedule-dependent: the same set of players can receive different final scores under different match fixtures.","Despite that, Elo still has a unique final rating for each fixed, connected schedule, so the system does not lose its fixed-point property when transitivity fails.","The set of all possible final ratings over all schedules, the Elotope, grows with the cyclic component of the advantage matrix, so the spread of possible ratings is a measure of how much intransitivity distorts the ratings.","The schedule-dependence explains why adaptive matchmaking, where opponents are chosen from current ratings, can prevent convergence: changing $Q$ moves the target fixed point while the process is chasing it.","The proposed intransitivity measure $I(A)$ stays near or below 1 for predominantly transitive games and rises above 1 when the cyclic component dominates, matching what one would expect on Rock-Paper-Scissors examples."],"supporting_citations":[{"why":"Supplies the definition of final Elo score that the paper adopts, and the prior unproved claim that Elo fails under intransitivity, which this work proves and refines.","marker":"[2]"},{"why":"Provides the combinatorial Hodge theory, including the orthogonal decomposition into gradient and cyclic components and the kernel of the divergence, used in the uniqueness proof.","marker":"[23]"},{"why":"Establishes the path-independence property of strongly transitive additive comparison matrices that underlies the spanning-tree construction in Theorem 4.1.","marker":"[40]"},{"why":"Derives the expected one-step update formula for Elo ratings that is the basis of the stability equation.","marker":"[1]"},{"why":"Reports non-convergence of Elo ratings under adaptive opponent selection, which the paper explains as a consequence of final scores depending on Q.","marker":"[5]"},{"why":"Shows that different rating methods can produce arbitrarily different rankings when intransitivity is present, motivating the need to study Elo's own behaviour.","marker":"[37]"},{"why":"Supplies the statement of Brouwer's fixed-point theorem used to prove existence of a solution to the stability equation.","marker":"[28]"}],"fun_headline_variants":["Intransitive games make Elo ratings fixture-dependent","When wins are circular, Elo rankings aren't unique","Elo scores shift with match order in intransitive games","Intransitivity breaks Elo: ratings depend on the schedule"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper defines the final Elo score as a fixed point of the expected one-step update and assumes this deterministic object describes where the stochastic Elo Markov chain really goes in the long run, even though it does not prove the chain has a limiting distribution or that it converges to this fixed point.","fun_headline_variants_meta":{"raw":{"variants":["Intransitive games make Elo ratings fixture-dependent","When wins are circular, Elo rankings aren't unique","Elo scores shift with match order in intransitive games","Intransitivity breaks Elo: ratings depend on the schedule"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000237,"raw_usage":{"total_tokens":1402,"prompt_tokens":732,"completion_tokens":670,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":348,"completion_tokens_details":{"reasoning_tokens":603}},"tokens_in":348,"tokens_out":670,"duration_ms":5960,"temperature":1.0,"reasoning_tokens":603,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T12:15:32.266712+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a long simulation of an intransitive game with a fixed connected selection matrix Q, record the time-averaged Elo ratings, and compare them to the unique solution of the stability equation; if the averages do not track the fixed point and instead cycle or converge elsewhere, the paper's notion of long-term behaviour is not the behaviour the system exhibits. A second check: compute the final ratings for two different spanning-tree selection matrices and verify empirically that the long-run averages land at the two different fixed points, as Theorem 4.1 predicts.","supporting_citations":[{"cited_title":", Tuyls, K","cited_arxiv_id":null,"evidence_quote":"Supplies the definition of final Elo score that the paper adopts, and the prior unproved claim that Elo fails under intransitivity, which this work proves and refines."},{"cited_title":"(2011).Statisticalrankingand combinatorial Hodge theory.Mathematical Programming127 203–244","cited_arxiv_id":null,"evidence_quote":"Provides the combinatorial Hodge theory, including the orthogonal decomposition into gradient and cyclic components and the kernel of the divergence, used in the uniqueness proof."},{"cited_title":"and Yao, Y","cited_arxiv_id":null,"evidence_quote":"Establishes the path-independence property of strongly transitive additive comparison matrices that underlies the spanning-tree construction in Theorem 4.1."},{"cited_title":"and Brinkhuis, M","cited_arxiv_id":null,"evidence_quote":"Reports non-convergence of Elo ratings under adaptive opponent selection, which the paper explains as a consequence of final scores depending on Q."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that different rating methods can produce arbitrarily different rankings when intransitivity is present, motivating the need to study Elo's own behaviour."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the statement of Brouwer's fixed-point theorem used to prove existence of a solution to the stability equation."}],"review_version":1}