{"id":"83a21162-ac49-4cfa-9baf-b096a6e9e8ca","arxiv_id":"2603.05789","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"In a multi-agent resource race, independent Q-learning agents alternate less often than random policies, while conventional fairness/efficiency metrics hide the deficit.","lead":"This paper introduces alternation metrics that measure turn-taking in repeated multi-agent games and reports that simple Q-learning agents coordinate worse than random action selection even when standard fairness scores look high. The metrics are aimed at diagnosing temporal fairness failures that conventional payoff-based metrics miss.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Q-learning 'worse than random' claim rests on an ambiguous evaluation protocol: Table 4 may average training-phase behavior rather than post-convergence learned policies, with no multi-seed variance.","rationale":"The reader's weakest assumption identifies the evaluation protocol and missing seeds; this is indeed the most load-bearing weakness. I agree with the need for a stated post-convergence protocol and multi-seed variance. However, I would phrase the direction more carefully: Figure 5 shows CALT decreasing as epsilon decays, so if Table 4 averages the full training history, early exploration would inflate Q-learning ALT values and make the 'worse than random' gap conservative rather than an artifact. That does not remove the protocol mismatch — the reported numbers still are not measurements of learned policies and no confidence interval is given — but it means the concern is about what was measured, not that early exploration explains the deficit. The abstract vs Table 4 numerical inconsistencies reinforce the need for a precise protocol. Since the likely direction of the bias favors the authors' conclusion, the appropriate disposition is not rejection but conditional acceptance pending a clean post-training evaluation with seeds. This matches the reader's CONDITIONAL verdict, so no adjustment is needed.","tokens_in":18159,"tokens_out":10463,"duration_ms":104422,"concrete_test":"Recompute Table 4 using the final learned policies only: freeze each agent's Q-table at the end of scheduled training (epsilon set to the stated minimum or 0), roll out 10,000 fresh episodes per run for every configuration (Type-A/B x ILF/IQF x n in {2,3,5,8,10}), and repeat over at least 10 independent seeds. Compute Relative Change and Coordination Score against the same random baselines and report per-seed distributions/confidence intervals. If all seed-level gaps remain negative and intervals exclude zero, the central claim is supported; if any include zero or reverse, it must be weakened. Simultaneously disclose whether original Table 4 values are full-training or post-convergence; if full-training, the rerun replaces them as the primary evidence.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Section 5.5.2) is that 'Q-learning agents have learned policies that are actively worse than random action selection.' The evidence in Table 4 does not establish this. Section 5.2 defines a training schedule with epsilon-greedy decay and episode counts per configuration, but never states whether the Q-learning ALT values in Table 4/5 are computed over the full training history, over a post-training evaluation, or over a final-policy roll-out. Figure 5 shows CALT decreasing during training as epsilon decays; hence the metric is phase-dependent. If Table 4 uses full-history averages, early near-random exploration inflates CALT, so the comparison is not 'learned policies vs random' but 'learning trajectory average vs random.' If instead it uses final policies, the protocol is not documented and the random baseline (10,000 fresh uniform-random episodes) is not matched in episode count or stochastic structure. The paper also reports no multiple independent seeds: the only spread cited (Fig. 1) is across the four Type-A/B x ILF/IQF modes, not stochastic replicates, so the 'systematically below random' gap has no confidence interval. The claim may be true — Figure 5 suggests post-convergence values would be even lower — but the reported numbers do not demonstrate it. Abstract/full-text percentage inconsistencies (e.g., 'up to 92% on EALT' vs Table 4's -75.9%) compound the ambiguity about which protocol generated the headline figures.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper formalizes a multi-agent variant of the Battle of the Exes as an episodic Markov game and proposes Perfect Alternation (PA) as a reference regime, together with six ALT metrics (CALT, EALT, AALT, FALT, qFALT, qEALT) intended to measure temporal coordination. It also proposes an AltRatio regression mapping to PA-equivalent coordination, and random-policy baselines as null processes. In Q-learning experiments with n ∈ {2,3,5,8,10} and four state/reward configurations, the paper reports that Q-learning policies fall below random baselines on all primary ALT metrics while traditional fairness/efficiency metrics remain high, concluding that conventional metrics mask temporal coordination failure. The central empirical claim is that 'Q-learning agents have learned policies that are actively worse than random action selection for achieving turn-taking coordination' (Section 5.5.2).","tokens_in":18596,"tokens_out":4862,"duration_ms":46307,"significance":"If the central empirical claim were cleanly established, the paper would make a useful contribution: it provides a concrete diagnostic family for temporal coordination, an explicit null baseline for turn-taking evaluation, and a striking demonstration that outcome-based fairness metrics can be high while temporal alternation is below chance. The availability of code and archived data is a strength, as is the attempt to ground PA in the temporal fair division literature. However, the current manuscript does not yet support the headline quantitative claims because the evaluation protocol for the Q-learning ALT values is not specified and no seed-level variance is reported.","major_comments":[{"comment":"The manuscript never states whether the Q-learning ALT values in Table 4 are computed over the full training history, over a post-convergence evaluation, or over final-policy rollouts. Figure 5 shows CALT decreasing during training as epsilon decays, so the metric is phase-dependent. If Table 4 uses full-history averages, the comparison is not 'learned policies vs random' but 'learning-trajectory average vs random', while the random baseline uses 10,000 fresh uniform-random episodes. This is load-bearing for the Section 5.5.2 claim. The authors must specify the protocol and report post-convergence evaluation of final greedy policies over a matched number of fresh episodes.","section":"§5.2, Table 4, Figure 5"},{"comment":"No multiple independent seeds are reported. The only spread shown in Figure 1 is across the four Type-A/B × ILF/IQF configurations, not across stochastic replicates. Consequently, there is no confidence interval or standard error for any Q-learning ALT value, and the statement that Q-learning is 'systematically' below random is unsupported statistically. Multiple seeds (at least 10, with typical dispersion reported) and matched episode counts are needed for the central empirical claim.","section":"§5.5.2, Figure 1"},{"comment":"The reported deficit magnitudes are inconsistent. The abstract states 'up to 92% on EALT', but Table 4 gives the worst EALT Relative Change as -75.9%; the full-text abstract says 'up to 81% below random' and Section 7.1 says '-81% (qEALT)', a secondary metric not in Table 4. The reader cannot tell which numbers are the headline result or which evaluation protocol generated them. These inconsistencies must be reconciled, with all headline numbers traceable to a named table entry.","section":"Abstract vs Table 4"},{"comment":"The claim that Perfect Alternation is the unique allocation achieving exact Temporal Envy-Freeness 'at every n-episode cycle boundary' is asserted via Theorem 17 of Choi and Li (2026), but the mapping from their scheduling model to MBoE is not proved, and uniqueness is not established. Since this is presented as independent theoretical grounding for PA, the authors should either provide a self-contained proof of the mapping and uniqueness or explicitly soften the claim to a correspondence that holds under stated assumptions.","section":"§4.1, Contribution 2"},{"comment":"The AltRatio/PA-equivalent mapping is central to the abstract claim that 10-agent coordination is 'roughly one-fifth of the population', but the main text only says regression equations are in Supplementary Section S1. Table 5 states 'ALT Ratio = sqrt(CALT − 1.879e−10)' with no derivation or explanation of the constant. The PA-equivalent percentages in Section 5.5.3 depend on this unexplained mapping. The regression construction should be summarized in the main text, or the PA-equivalent claims should be clearly labeled as dependent on a supplementary calibration.","section":"§5.3, Table 5"}],"minor_comments":[{"comment":"The CALT definition uses β_j^{qFALT} before qFALT is defined. Reorder the metric presentations or add a forward reference so Eq. (9) is self-contained.","section":"§4.3, Eq. (9)"},{"comment":"The caption says 'Error bars represent standard deviation across Type-A/Type-B and ILF/IQF configurations.' This is a spread across configurations, not a statistical uncertainty estimate. Clarify this in the caption to avoid misleading readers into inferring seed-level variance.","section":"Figure 1 caption"},{"comment":"The claim 'Relative Change reaches as low as -81% (qEALT)' and the single positive '+5.4% for 10-agent qEALT' are not visible in any main-text table; they are deferred to supplementary material. Since these are used as summary statistics, include the secondary-metric table or at least the relevant rows in the main text.","section":"§7.1"},{"comment":"The episode-count schedule is described as heuristic, and the paper states that 'pilot runs with alternative scalings produced qualitatively similar trends' without showing these runs. A brief description of the pilot sensitivity analysis would strengthen confidence that the reported patterns are not an artifact of the chosen schedule.","section":"§5.2, Eq. (15)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is heavily self-referential and extends the authors' prior ALT framework; the novelty relative to those papers should be stated more crisply. The central empirical claim is potentially interesting but needs a clean post-convergence, multi-seed evaluation before it can be accepted. I do not see a fatal flaw in the metric definitions themselves; the issue is evidentiary support for the headline 'worse than random' statement, plus internal numeric inconsistencies."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the takeaway: the paper's central claim — independent Q-learners in the MBoE game do worse than random on turn-taking metrics even when traditional fairness looks high — is plausible and worth testing, but as written it is not demonstrated. The main reason is that the evaluation protocol is under-specified. I cannot tell from the text or tables whether the Q-learning ALT values come from full training histories or post-convergence rollouts. Figure 5 shows CALT decreasing as epsilon decays, so this matters. If the numbers average over training, then \"worse than random\" is about learning trajectories, not learned policies.\n\nWhat is genuinely new: the MBoE Markov-game formalization, the PA reference regime, the six ALT metrics, and the systematic random-baseline comparison. Prior BoE work has mostly stayed two-agent and rarely reports chance baselines. The reported pattern — high reward fairness alongside below-random alternation — is a real observation if it survives scrutiny. The authors also ship code and data, which is more than many papers in the area.\n\nNow the soft spots, in rough order of importance. First, the abstract and Table 4 disagree on magnitudes: the abstract says 'up to 92% on EALT' but Table 4 shows -75.9% as the worst EALT relative change. That suggests the headline numbers came from a different pipeline than the main tables. Second, no multiple seeds. The only spread cited is across the four configuration modes, not stochastic replicates, so the 'systematically worse than random' gap has no confidence interval. Third, the PA-equivalent values come from a regression in Supplementary S1 that is not described in the main text; a headline number should not depend on an invisible fit. Fourth, the 'PA is uniquely TEF' claim is imported from Choi & Li (2026) and asserted rather than proven for MBoE; smaller issue, since the metrics don't stand or fall on it.\n\nThese are fixable. The core idea is good, the code exists, and the direction is probably right. What is missing is a clear evaluation protocol, seed variance, and consistent numbers between abstract and tables.\n\nWho should read it: people working on MARL evaluation, temporal fair division, or coordination metrics. It deserves a serious referee — an editor should send it out — but the referee should ask for the protocol to be pinned down and the numbers regenerated with seeds and post-convergence evaluation.","headline":"Plausible and worth testing, but the current write-up doesn't pin down what the reported 'worse than random' numbers actually measure.","tokens_in":19049,"tokens_out":3486,"would_cite":false,"duration_ms":32516,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In a repeated multi-agent coordination game, Q-learning policies that look fair by conventional metrics actually alternate access to a shared reward worse than random action selection, by 34–74% on the paper's primary alternation metric.","keywords":["temporal fairness","turn-taking","alternation metrics","multi-agent reinforcement learning","repeated games","Battle of the Exes","coordination","Q-learning"],"falsifier":"Compute the ALT metrics only on the final block of episodes after epsilon has reached its minimum, and average over many independent seeds; if post-convergence Q-learning then meets or exceeds the random baselines on CALT, the paper's central claim that learned policies are actively worse than random is falsified.","tokens_in":18054,"feed_emoji":"🤖","tokens_out":3995,"duration_ms":37669,"temperature":0.7,"pith_summary":"Repeated interactions among self-interested agents can produce high aggregate payoffs while the agents are failing to actually take turns. This paper argues that standard fairness and efficiency metrics are temporally blind: they can score coordinated rotation, random access, and monopolization almost identically. To expose the difference, the authors define Perfect Alternation (PA) as a reference turn-taking regime and introduce six Alternation (ALT) metrics, then compare simple Q-learning agents against analytically derived random-policy baselines in a multi-agent Battle of the Exes game. The central finding is that the learned policies score worse than random on every primary ALT metric, even as reward fairness often exceeds 0.9, so conventional metrics can conceal severe coordination failure as agent count grows. The paper thereby makes a measurement point: temporal coordination must be evaluated relative to an explicit chance-level null process.","feed_headline":"Q-learning turn-taking falls 34–74% below random","feed_subtitle":"Standard fairness scores stay high while coordination worsens as agent count grows.","key_machinery":"The load-bearing object is Perfect Alternation (PA): the n-periodic round-robin sequence in which each of n agents wins exactly once per block of n episodes. In the MBoE setting this is the unique allocation achieving exact Temporal Envy-Freeness at every cycle boundary. Against this reference, six ALT metrics (CALT, EALT, AALT, and secondary FALT, qFALT, qEALT) score overlapping batches of n consecutive episodes, penalizing ties and rewarding exclusive wins and winner diversity. A regression-based AltRatio framework converts raw ALT scores into a PA-equivalent count of perfectly alternating agents, and uniformly random action policies are run as explicit null processes so that reported coor","core_discovery":"On the paper's own terms, the discovery is that in the Multi-Agent Battle of the Exes (MBoE)—a Markov game where n agents compete for exclusive access to a single high-reward state—independent tabular Q-learning agents systematically under-coordinate relative to chance. Across 20 configurations of state representation, reward scheme, and agent count, Q-learning achieved reward fairness values between 0.49 and 0.993 and efficiency up to 0.677, yet all primary Alternation metrics (CALT, EALT, AALT, FALT) were below random-policy baselines, with CALT 34–74% lower and PA-equivalent coordination falling to roughly one-fifth of the population at n=10. The paper interprets this as a measurement fai","pith_inferences":["Editorial extension: the same temporal blindness likely affects other aggregate metrics in multi-agent systems beyond this game—any setting where fairness of cumulative shares is reported without checking the sequence of allocations.","A testable extension: adding an alternation-based reward bonus or using ALT scores as an early-stopping signal would likely push independent learners above the random baseline, which would confirm that the deficit is a credit-assignment failure rather than an intrinsic impossibility.","The result suggests that comparing learned policies to an analytic random null is a cheap and general diagnostic that could be standardized across coordination benchmarks, not just turn-taking games."],"forward_implications":["If the claim holds, any evaluation of emergent coordination that reports only efficiency, reward fairness, or turn-taking fairness is incomplete and can certify failure as success.","Reported coordination results in repeated games should include a random-policy null baseline; without one, high traditional metrics are uninterpretable.","Temporal fairness measures such as the ALT family should accompany outcome-based fairness in multi-agent reinforcement learning evaluation, especially as agent counts grow.","The sharp decline from 56.8% to roughly 22% PA-equivalent coordination between 2 and 8–10 agents suggests that coordination difficulty in this class of game scales non-linearly with population size.","Independent tabular Q-learning, in this setting, does not spontaneously converge to turn-taking behavior, meaning decentralized learners need additional structure (memory, signaling, or centralized training) to achieve temporal fairness."],"fun_headline_variants":["Fairness metrics blind to turn-taking collapse","Q-learning turn-taking worse than random by 34-74%","Turn-taking deficit: Q-learning under random despite fair payoffs","Coordination gap: fairness scores stay high, alternation drops","Temporal fairness unmasks multi-agent coordination failure"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The central comparison assumes the Q-learning runs are evaluated fairly and represent converged learners; because ALT scores are computed over full training histories, early-exploration and epsilon-decay artifacts may contribute to the below-random result.","fun_headline_variants_meta":{"raw":{"variants":["Fairness metrics blind to turn-taking collapse","Q-learning turn-taking worse than random by 34-74%","Turn-taking deficit: Q-learning under random despite fair payoffs","Coordination gap: fairness scores stay high, alternation drops","Temporal fairness unmasks multi-agent coordination failure"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000537,"raw_usage":{"total_tokens":2475,"prompt_tokens":861,"completion_tokens":1614,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":605,"completion_tokens_details":{"reasoning_tokens":1533}},"tokens_in":605,"tokens_out":1614,"duration_ms":15327,"temperature":1.0,"reasoning_tokens":1533,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T18:42:08.348597+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the ALT metrics only on the final block of episodes after epsilon has reached its minimum, and average over many independent seeds; if post-convergence Q-learning then meets or exceeds the random baselines on CALT, the paper's central claim that learned policies are actively worse than random is falsified.","supporting_citations":[],"review_version":1}