{"id":"42867eb4-8b01-4bd0-90c6-34cea1326929","arxiv_id":"2505.13834","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Hierarchical MARL with fictitious self-play trains quadruped soccer teams in simulation and transfers them zero-shot to real robots, enabling onboard, decentralized 1v1 and 2v1 soccer with emergent passing and role allocation.","lead":"This paper builds a two-layer learning system that lets teams of quadruped robots play soccer autonomously, with low-level skills such as walking, dribbling, and kicking under a learned team strategy. It deploys the system on real Unitree Go1 robots that sense with onboard LiDAR, in 1v1 and 2v1 games with no external motion capture.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Real-world soccer claim rests on an unquantified zero-shot perception gap: policies trained on clean simulator state are deployed with noisy LiDAR/FAST-LIO observations, with no noise model, latency model, or quantitative real-world trials.","rationale":"The reader's weakest assumption already identifies the sim-to-real perception gap; my read agrees. I have sharpened it to a specific missing channel: the high-level policy is trained on clean state (Table 4) and no perceptual noise or latency is modeled, while the real stack relies on FAST-LIO and reflective-point thresholding (Sec. G). The qualitative real-world section cannot distinguish a robust transfer from a selected demonstration. The paper's own Sec. 6 occlusion caveat supports the concern. This is an evidential weakness, not an internal inconsistency. If the proposed perturbation replay preserves high win rates, the central claim would be substantially strengthened; if not, the real-world demonstration would need to be re-scoped. The reader's CONDITIONAL verdict already captures the need for quantitative real-world evidence, so I do not adjust the verdict.","tokens_in":12561,"tokens_out":7299,"duration_ms":75449,"concrete_test":"Calibrate the observation-error distribution from the real system: run one 2v1 session while logging the policy's LiDAR/FAST-LIO inputs and an independent ground-truth pose/ball trajectory (manual annotation or a temporary reference system used only for evaluation). Then replay the trained attacker high-level policy in IsaacGym against the Appendix E ball-chasing defender with observations corrupted by that calibrated noise plus a 0.2 s latency, under the in-domain protocol. If the 89.2% in-domain win rate drops below roughly 50%, the zero-shot real-world claim is not supported; if the win rate remains above about 80%, the concern is largely resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 4.3 claims that the policy can be zero-shot transferred to physical robots, and the abstract/first-contribution claim is a fully decentralized real-world multi-quadruped soccer system. The load-bearing condition is that the learned policies are robust to the observations the real stack actually produces. In Sec. 3.3 and Table 4, the high-level policy is trained on ground-truth relative positions of ball, teammates, opponents, and goals; in Sec. G the deployed system obtains these from FAST-LIO localization and thresholded LiDAR reflective-ball detection. The training pipeline (Sec. 3.4, Appendix D) contains no sensor-noise model, detection latency, or occlusion process, and the real-world evidence in Sec. 4.3 is qualitative (selected episodes, value-map stills, video), with no goal counts, possession times, pass-completion rates, or comparison to a scripted baseline. Appendix E reports generalization only in simulation with clean observations. Sec. 6 explicitly concedes that the LiDAR ball detection may suffer from occlusions and proposes simulating occlusion as future work, confirming that the deployed perception channel was not modeled during training. If real perception errors or latency lie outside the policy's implicit robustness envelope, the zero-shot transfer claim and the central real-world demonstration fail.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a hierarchical MARL approach to quadruped soccer. Low-level skills (Walk, Dribble, Kick) are trained separately in IsaacGym, and a shared high-level GRU policy trained with MAPPO and fictitious self-play selects a skill type and an eight-direction command at 5 Hz using only egocentric relative positions. Experiments cover 1v1, 2v1, and 2v2 simulation matches, ablations against a flat end-to-end policy and against reduced skill sets, out-of-domain initialization tests, and qualitative real-world deployments on Unitree Go1 robots using onboard LiDAR, FAST-LIO localization, and broadcast positions, including robot-robot and robot-human games. The central claim is that this is the first fully decentralized multi-quadruped soccer system in the real world and that the policies transfer zero-shot from simulation.","tokens_in":12725,"tokens_out":5166,"duration_ms":48601,"significance":"If the real-world claims were backed by quantitative trials, this would be a notable advance for sim-to-real multi-agent legged robotics: the hierarchical decomposition is sensible, the FSP curriculum is a principled way to avoid opponent overfitting, the ablations support the contribution of skill diversity, and the out-of-domain initialization numbers in Appendix E (89.2%, 81.9%, 68.9%) are genuine holdout evaluations. The value-map analysis is a useful interpretability device. The main gap is that the paper's headline contribution is the real-world system, and that part of the evidence is currently anecdotal and leaves the perception sim-to-real gap unquantified.","major_comments":[{"comment":"The real-world evidence consists of selected episodes and value-map stills (Fig. 7) plus representative 1v1 cases in Appendix B, but no aggregate numbers are reported: there is no count of trials, goal totals, possession time, pass-completion rate, or comparison against a scripted or teleoperated baseline. Because the paper's headline contribution is a real-world multi-robot soccer system, this evidence base is anecdotal relative to the claim. Please add quantitative field-trial statistics, or, if such statistics are not available, explicitly label the real-world results as demonstrations and soften the 'first fully decentralized system' claim accordingly.","section":"Section 4.3 and Appendix G"},{"comment":"The zero-shot sim-to-real transfer claim rests on an unmodeled perception channel. The high-level policy is trained on ground-truth relative positions of the ball, teammates, opponents, and goals, and the low-level skills receive exact ball state; deployment obtains these quantities from FAST-LIO localization and thresholded LiDAR reflective-ball detection, with no sensor-noise model, detection-latency model, or occlusion process in training. Section 6 explicitly concedes that LiDAR-based ball detection may suffer from occlusions and proposes simulating occlusion as future work. Please quantify real-world perception quality (detection rate, localization error, latency) and its effect on policy performance, add corresponding robustness training and evaluation, or restrict the zero-shot transfer claim to the conditions under which the perception is accurate.","section":"Section 3.3, Table 4, Appendix G, Section 6"},{"comment":"The out-of-domain tests vary only initialization positions in simulation with clean observations; they do not evaluate generalization over perception noise, physical parameter variations, or opponent distributions beyond the FSP population. The reported degradation from 89.2% in-domain to 68.9% when attackers start out-of-domain is useful, but it is not evidence about the real-world deployment gap identified above. Please either add domain-randomization tests over sensor and dynamics parameters or state explicitly that the out-of-domain robustness is limited to the simulated observation model.","section":"Appendix E"}],"minor_comments":[{"comment":"The action-space table is incomplete in the displayed text: the 'Direction Options' column for Walk, Dribble, and Kick is blank, while the Stop row reads '– None'. Please fill in the eight direction options and the per-skill command magnitudes, and repair the table formatting.","section":"Table 1"},{"comment":"Fig. 8 is captioned as showing four low-level motor skills and includes Receive, but the final skill library in Sec. 3.2 contains only Walk, Dribble, and Kick. Please clarify that Receive was ablated during training and is not part of the deployed policy.","section":"Appendix A"},{"comment":"The text says 'no explicit reward is given for coordinated behaviors,' but Table 5 includes 'base2ball' (reward for approaching the ball when the agent is the closest teammate) and 'opponent near ball' penalties, which are coordination-relevant shaping terms. Please qualify the statement to distinguish the absence of a direct team-reward term from the presence of teammate-relative shaping rewards.","section":"Section 3.3 and Table 5"},{"comment":"The caption says '20 trails' and should say '20 trials.' In addition, the End2End comparison reports win rates against a static opponent without stating the number of seeds or the variance across seeds; please add this information.","section":"Figure 4 caption and Section 4.1"},{"comment":"The robot-human collaboration experiment is under-specified: state whether the human teammate is treated as a policy-controlled teammate whose position is broadcast to the robot, how the human's actions are synchronized with the robot's decision loop, and whether the robot policy was ever trained with a human-like teammate. This matters because the claim of robot-human collaboration depends on these protocol details.","section":"Section 4.3, Fig. 7(g)"},{"comment":"The phrase 'for the first time' should be scoped precisely. The abstract and introduction should state whether the claimed novelty is the first fully autonomous, learning-based, decentralized multi-quadruped soccer system, and should explicitly compare with prior real-world systems such as Labiosa et al. [9] to make the novelty claim verifiable.","section":"Introduction, Contribution (3)"}],"recommendation":"major_revision","confidential_remarks":"Confidentially, the simulation-side method and ablations are solid enough to be worth publishing if the real-world claims are either quantified or reframed as demonstrations. The main risk is that the abstract and contribution list overstate the evidence in Sec. 4.3 and Appendix G. I do not see a fundamental flaw in the learning framework itself."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the headline real-world claim is real but not yet proven at the strength they state. What is proven is that a hierarchical MARL pipeline with learned skills plus FSP produces strong, generalizing soccer behavior in simulation, and that selected episodes of that behavior transfer to three Go1 robots with onboard LiDAR. Those are different claims, and the paper blurs them.\n\nWhat is new is the integration. Prior real legged soccer was 1v1 or rule-based at the high level; this is the first fully learning-based, decentralized multi-agent legged soccer with no external mocap, as far as I know. The hierarchical decomposition into reusable skills plus a GRU high-level policy trained by MAPPO/FSP is sensible. The ablations are real evidence: 98.3% vs 37.5% win rate against a static opponent, faster convergence with all three skills, and OOD win rates that stay respectable when positions are randomized. The 2v2 simulator extension helps. They also state limitations honestly, including occlusion and scaling to larger teams.\n\nSoft spots, in proportion. The real-world section is qualitative. No aggregate goals, possession times, pass-completion rates, or comparison to a scripted baseline appear in the paper. The stress-test note is fair: the high-level policy is trained on ground-truth relative positions (Table 4), deployed with FAST-LIO localization and thresholded LiDAR ball detection, and Section 6 concedes that occlusion is not modeled. Appendix D includes no sensor-noise or latency model. So \"zero-shot transfer\" should be read as \"we tried it and it worked in the shown clips,\" not as a measured robustness result. Also, the fully learned claim is slightly overstated: Appendix D.2 describes rule-based transitions that map Dribble and Kick to walking or stopping under certain conditions. That is a hand-coded wrapper around the learned policy. Not fatal, but it qualifies the claim. And there is no code or release of configs, so the simulation numbers are not independently reproducible.\n\nThe citation pattern is fine; they engage with the actual predecessors. The reader's conditional verdict is about right, though I would weight the simulation side a bit higher and the real-world evidence a bit lower than the numeric scores suggest.\n\nThis paper deserves a serious referee. It is useful for anyone doing legged robot MARL or sim-to-real multi-agent systems. With quantitative real-world evaluation and released code or detailed configs, I would support acceptance. Recommend conditional acceptance with those changes.","headline":"A credible simulation-side systems result with a genuine first demonstration of decentralized multi-robot legged soccer, but the real-world evidence is anecdotal and the zero-shot transfer claim is stronger than the data.","tokens_in":13392,"tokens_out":2394,"would_cite":true,"duration_ms":24227,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Quadruped teams play real soccer without a central planner","keywords":["quadruped robots","robot soccer","multi-agent reinforcement learning","fictitious self-play","hierarchical reinforcement learning","sim-to-real transfer","decentralized control","LiDAR perception"],"falsifier":"Run a controlled 2v1 match where a second defender stands between the attacker and the ball so that the ball leaves the attacker's LiDAR field of view; if the attacker does not consistently recover or re-acquire the ball, the onboard-perception premise behind the real-world claim fails.","tokens_in":12269,"feed_emoji":"⚽","tokens_out":8271,"duration_ms":76233,"temperature":0.7,"pith_summary":"This paper tries to establish that fully autonomous, cooperative and competitive soccer is possible for teams of quadruped robots operating in the real world with no external infrastructure. The proposed recipe is a hierarchical multi-agent reinforcement learning framework: separately trained low-level skills for walking, dribbling, and kicking are composed by a high-level strategy policy trained with Multi-Agent Proximal Policy Optimization under Fictitious Self-Play. A sympathetic reader would care because the authors show the resulting policies transfer zero-shot from simulation to physical robots for multi-agent games, producing coordinated passing, interception, counterattacks, and even robot-human play using only onboard LiDAR, onboard computation, and broadcast positions. The paper positions this as evidence that long-horizon team coordination in adversarial, dynamic settings can emerge purely through learning rather than handcrafted rules.","feed_headline":"Quadruped teams play real soccer without a central planner","feed_subtitle":"Simulation-trained policies transfer zero-shot to Go1 robots in 1v1, 2v1, and robot-human matches.","key_machinery":"The load-bearing object is the two-level policy stack. At the bottom, a reusable skill library (Walk, Dribble, Kick, and Stop) maps velocity commands to joint targets at 50 Hz under fixed PD gains, with each skill trained separately by reinforcement learning. At the top, a shared recurrent policy observes only egocentric relative positions (ball, teammates, opponents, goals) and outputs a skill type plus one of eight discretized direction commands at 5 Hz; this policy is trained with multi-agent proximal policy optimization under Fictitious Self-Play, which freezes one side and trains the other against a growing population of past opponent snapshots. Cooperation is not rewarded directly; it emerges from the shared scoring objective and the diversity of opponents in the population. The real-world transfer is carried by onboard perception: a LiDAR detects the high-reflectivity ball, LiDAR-inertial odometry provides localization against a pre-mapped environment, and a wireless broadcast shares poses, so no external infrastructure is needed.","core_discovery":"The central claim is that a fully decentralized multi-quadruped soccer system can be trained entirely in simulation and then run in the real world on quadruped robots without motion capture, centralized planning, or external tracking. The framework stacks a library of low-level motor skills (walk, dribble, kick) under a recurrent high-level policy that selects a skill and a command direction every 0.2 seconds; the high-level policy is trained with MAPPO under Fictitious Self-Play, so attackers and defenders co-evolve against populations of past opponents. In simulation the full hierarchical method beats a flat end-to-end baseline (98.3% versus 37.5% win rate against a static opponent) and converges faster than variants missing the dribble or kick skills. On physical robots the same policies produce coordinated passing, interception, counterattacks, and one robot-human scoring play, with each robot perceiving via laser-based ball detection and LiDAR-inertial localization and sharing only minimal position messages.","pith_inferences":["One extension the paper points to but does not test is training with simulated ball occlusion, which could address the LiDAR-occlusion bottleneck the authors list as the main obstacle to larger teams.","The critic value maps could double as an interpretability tool for debugging physical multi-robot coordination, a use the paper only hints at.","If the same hierarchy were applied to other adversarial ball games, only the low-level skills and the high-level observation encoding would need to change; this is a conjecture, not a result reported in the paper."],"forward_implications":["The reusable skill library means a new team configuration, such as 2v2 or 3v3, would require retraining only the high-level policy, not the low-level motor skills.","Because execution is decentralized and relies only on onboard sensing, the system can play in outdoor spaces where motion-capture systems and centralized planners are impractical.","The same architecture supports robot-human teams: a human teammate is treated as another agent in the broadcast observation, and the learned attacker still coordinates passes and scores.","FSP's population-based opponent pool prevents strategy collapse, so the final policy exhibits multi-modal behaviors rather than converging to a single local optimum.","The paper's own limitation section indicates that scaling to more robots is gated by LiDAR occlusion and sample complexity, not by the hierarchical framework itself."],"supporting_citations":[{"why":"Introduces Fictitious Self-Play, the training regime that co-evolves attackers and defenders against a population of past opponent policies.","marker":"[14]"},{"why":"Supplies the multi-agent PPO algorithm used to train the high-level strategy policy with shared team parameters.","marker":"[15]"},{"why":"Provides the GPU-accelerated simulation where all low-level and high-level policies are trained.","marker":"[29]"},{"why":"Prior real-world state-based 1v1 bipedal soccer system that the paper contrasts with its decentralized multi-robot deployment.","marker":"[10]"},{"why":"Prior vision-based egocentric soccer policy whose field-of-view and motion-blur limitations motivate the paper's LiDAR-based perception.","marker":"[11]"},{"why":"A decentralized MARL team-play baseline in simulation that this work moves to real robots.","marker":"[13]"},{"why":"Prior hierarchical 5v5 sim-to-real soccer system with handcrafted high-level strategy, the approach this work replaces with learned strategies.","marker":"[9]"},{"why":"Provides the LiDAR-inertial odometry used for each robot's decentralized localization without external infrastructure.","marker":"[35]"}],"fun_headline_variants":["Quadruped robots play soccer with no central planner","Robot dogs learn soccer teamwork in sim, play for real","Decentralized quadruped soccer: sim-to-real teamwork","Quadruped teams pass and score without a central brain","Sim-trained robot dogs play soccer with human teammates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simulation-trained policies behave on the physical robots just as they do in the simulator, which in turn requires the onboard laser-based ball detection and self-localization to be accurate enough during real play; the paper reports real demonstrations but no quantitative measurement of this accuracy.","fun_headline_variants_meta":{"raw":{"variants":["Quadruped robots play soccer with no central planner","Robot dogs learn soccer teamwork in sim, play for real","Decentralized quadruped soccer: sim-to-real teamwork","Quadruped teams pass and score without a central brain","Sim-trained robot dogs play soccer with human teammates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000945,"raw_usage":{"total_tokens":4039,"prompt_tokens":954,"completion_tokens":3085,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":3007}},"tokens_in":570,"tokens_out":3085,"duration_ms":21159,"temperature":1.0,"reasoning_tokens":3007,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:09:25.706219+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a controlled 2v1 match where a second defender stands between the attacker and the ball so that the ball leaves the attacker's LiDAR field of view; if the attacker does not consistently recover or re-acquire the ball, the onboard-perception premise behind the real-world claim fails.","supporting_citations":[{"cited_title":"Heinrich, M","cited_arxiv_id":null,"evidence_quote":"Introduces Fictitious Self-Play, the training regime that co-evolves attackers and defenders against a population of past opponent policies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the multi-agent PPO algorithm used to train the high-level strategy policy with shared team parameters."},{"cited_title":"MARLadona -- Towards Cooperative Team Play Using Multi-Agent Reinforcement Learning","cited_arxiv_id":"2409.20326","evidence_quote":"A decentralized MARL team-play baseline in simulation that this work moves to real robots."},{"cited_title":"Reinforcement Learning Within the Classical Robotics Stack: A Case Study in Robot Soccer","cited_arxiv_id":"2412.09417","evidence_quote":"Prior hierarchical 5v5 sim-to-real soccer system with handcrafted high-level strategy, the approach this work replaces with learned strategies."},{"cited_title":"Xu and F","cited_arxiv_id":null,"evidence_quote":"Provides the LiDAR-inertial odometry used for each robot's decentralized localization without external infrastructure."}],"review_version":1}