{"id":"f323570b-704e-4e4d-912f-303846d9a607","arxiv_id":"2505.12811","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A UCB meta-controller that selects per-episode observation ranges improves MARL performance in several benchmark settings, but the paper's claim of consistent improvement is contradicted by its own results.","lead":"This paper wraps MARL training in a sliding-window UCB meta-controller that picks a different sight range for each episode, then tests it on LBF, RWARE, and SMAC with QMIX, MAPPO, and other algorithms. The authors report faster learning and higher final scores in many settings, though their own tables show some settings where DSR performs worse than a fixed sight range.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"UCB meta-controller rewards are confounded with the shared policy's learning, so the claimed 'optimal sight range' discovery lacks support without a random-selection ablation.","rationale":"The reader's weakest assumption identifies exactly the same load-bearing issue: the UCB meta-controller in Eq. (3) treats recent episode returns as independent, non-stationary signals of sight-range quality, but those returns are generated by a policy that is itself learning and shared across all ranges via one replay buffer. This confounding is not modeled or tested. I agree this is the most critical concern because the paper's headline contribution includes 'automatically discovers the optimal sight range' — if the UCB scores are biased by policy familiarity rather than intrinsic range quality, that claim fails, and the method reduces to an unspecified curriculum schedule. The paper does provide a fixed-scheduling baseline (Figure 6) that DSR outperforms, but that does not isolate the UCB mechanism; a random-selection baseline is the decisive ablation. I also note a secondary issue: Table 1 shows DSR underperforms the baseline in RWARE small-2ag-5s and small-2ag-3s, so the abstract's word 'consistently' is not supported. However, that overclaim is more about wording and can be fixed by reporting significance and effect sizes, while the confounding concern affects the core mechanism. Therefore the reader's CONDITIONAL verdict remains appropriate; no change is needed, provided the authors add the random-selection ablation, discuss the confounding, and narrow their claims.","tokens_in":44582,"tokens_out":6557,"duration_ms":67466,"concrete_test":"Run DSR with a uniform-random meta-controller over the same D, same budget, same seeds, and compare final test returns and selected ranges. Additionally, after training, freeze the policy and evaluate it under each fixed sight range to see which range actually maximizes performance; if the random scheduler matches UCB, or the UCB-chosen range does not equal the best fixed range, the optimal-range claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The meta-controller (Eq. 3) treats each sight range di as a bandit arm whose recent episode returns are noisy samples of a fixed quality. But all arms are evaluated under a single policy trained on a shared replay buffer that mixes data from all ranges. Thus the return for di at episode e is a function of the policy's current parameters, which have been shaped by previous selections. Early dominance of one range biases the policy toward it, inflating its future returns and creating a self-reinforcing loop. SW-UCB's confidence intervals then reflect familiarity, not intrinsic quality. The paper never models this feedback or checks whether the selected range maximizes final performance under a fixed policy. Hence the central claim 'automatically discovers the optimal sight range' is unsupported. An ablation replacing UCB with random selection over the same D is needed; if random matches DSR, the improvement stems from training on a mixture of sight ranges, not from UCB's selection.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Dynamic Sight Range Selection (DSR), a meta-controller that uses a sliding-window UCB algorithm to select, at the beginning of each episode, one sight range from a hand-chosen set D. The selected range is applied to all agents' observations, while the underlying MARL algorithm (QMIX, IQL, VDN, IPPO, MAPPO, IM-Qatten) trains as usual on the resulting episodes. The authors evaluate DSR on LBF, RWARE, SMAC, and SMAC-DT, reporting that DSR outperforms fixed-range baselines, accelerates training, and 'automatically discovers the optimal sight range.' The paper includes an extensive appendix with fixed-range comparisons, hyperparameter sensitivity for c and w, and a comparison against the communication-based method CAMA.","tokens_in":44767,"tokens_out":3184,"duration_ms":33338,"significance":"If the claims hold, DSR is an attractive drop-in wrapper: it removes the need to manually select an observation range, requires no global information or communication, and could guide sensor design in real-world applications. The experimental breadth is a genuine strength: three environments, five MARL algorithms, many maps, fixed-range baselines for every setting, five seeds for the main QMIX experiments, and an integration with the CAMA codebase. However, the central claims of 'consistently improves' and 'discovers the optimal sight range' need additional support: several Table 1 rows contradict the consistency claim, no significance testing is reported, and the UCB meta-controller's reward signal is confounded with the shared policy's ongoing learning. The paper's practical value is real but currently overstated.","major_comments":[{"comment":"The abstract and §4.2 claim that DSR 'consistently improves performance across three common MARL environments,' but Table 1 contains multiple rows where the DSR mean is below the baseline mean: 10x10-4p-2f-coop-6s (0.957 vs 0.972), 10x10-4p-2f-6s (0.987 vs 0.998), small-2ag-5s (0.050 vs 0.074), small-2ag-3s (0.036 vs 0.182), 3s_vs_5z-9s (0.676 vs 0.716), 3s5z-15s (0.770 vs 0.808), and 3s5z-21s (0.736 vs 0.784). Additionally, the standard deviations are large relative to the differences (e.g., tiny-2ag-5s: 4.762 ± 4.702 vs 1.486 ± 1.361), and no significance tests or confidence intervals are provided. To support the consistency claim, the authors should either weaken the wording to 'often improves' or provide paired significance tests (e.g., bootstrap or Wilcoxon across the five seeds) and report how many of the 22 settings show a statistically significant improvement.","section":"Section 4.2, Table 1"},{"comment":"The meta-controller's UCB scores are computed from episode returns that are generated by a single policy shared across all sight ranges and trained on one replay buffer containing data from all ranges. Consequently, the return r_j(d_i) in Eq. (3) is not a sample of a stationary arm quality; it is a function of the current policy, which has itself been shaped by previous sight-range selections and by data collected under other ranges. The paper therefore does not establish that DSR 'automatically discovers the optimal sight range,' because 'optimal' is defined operationally as the argmax of recent episode returns within a hand-chosen set D, with no evaluation of each candidate range under a fixed, converged policy. This confounding is a load-bearing issue: the observed benefits could arise from training on a mixture of sight ranges rather than from UCB's selection mechanism. I recommend adding an ablation that replaces the UCB meta-controller with random selection over the same set D, and an additional evaluation where the final policy is tested under each fixed sight range to check whether the range selected by DSR is actually the best at convergence.","section":"Section 3.2, Eq. (3) and Algorithm 1"},{"comment":"The SMAC experiments use a non-standard state construction: the global state is formed by concatenating all agents' observations, rather than using the environment-provided global state. This choice likely accentuates the sight range dilemma and may make the SMAC results not directly comparable to standard SMAC benchmarks. The appendix reports 'w/ Given State' comparisons (Figures 10 and 26–31), but the main text and abstract do not qualify the SMAC claim accordingly. The authors should clearly state in §4.1 and the abstract that the SMAC results use this observation-concatenation variant, or present the standard-state results as the primary SMAC evidence.","section":"Section 4.1 and Appendix B.1"}],"minor_comments":[{"comment":"Line 8 contains the condition 'if d_{e-t} ≠ d*_e'; the index 'e-t' appears to be a typo (likely 'e-1' or a comparison of the previous selection). The intended semantics for updating N_e(d_i,w) are unclear, especially in relation to the definition in Eq. (3). Please clarify.","section":"Algorithm 1, line 8"},{"comment":"The row for '10m_vs_11m' lists '3 Stalkers + 5 Zealots' for both sides, which appears to be a copy-paste error; 10m_vs_11m in SMAC consists of Marines. Please correct the table.","section":"Table 2"},{"comment":"The SMAC-DT comparison with CAMA reports no standard deviations, seeds, or number of runs for Figures 9, 36, and 37. Given that the main experiments use five seeds, the dynamic team composition results should report the same level of uncertainty to support the claim that DSR 'outperforms' CAMA.","section":"Section 4.6 and Appendix B.2"},{"comment":"The training acceleration claim is supported only by visual inspection of two LBF curves. Please provide a quantitative summary, such as the number of steps to reach a given return threshold or the area under the training curve, for all settings where final performance is comparable.","section":"Section 4.2, Figure 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the empirical base is broader than typical, but the central 'consistently improves' and 'discovers optimal' claims need substantive revision. The random-selection ablation is essential: without it, the reader cannot distinguish the contribution of UCB from the simple effect of training on multiple sight ranges. If the authors add that ablation, significance tests, and qualify the SMAC state variant, the paper could become acceptable. I would not reject at this stage because the method is simple, practical, and the fixed-range comparisons are valuable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful engineering wrapper, not a new principle. DSR wraps any MARL algorithm with a sliding-window UCB that picks the agent sight range at the start of each episode. That is a sensible, practical idea, and the experimental appendix is unusually thorough: three environments, five algorithms, many maps, five seeds, plus a comparison with a fixed curriculum and with CAMA. Credit where due: the fixed-scheduling comparison (Figure 6) is the right control to isolate the benefit of adaptive selection, and the paper honestly reports the selected ranges over training.\n\nThe soft spots are in the claims, not the experiments. The abstract says DSR 'consistently improves performance,' but Table 1 has several rows where DSR is worse: LBF 10x10-4p-2f-coop-6s (0.957 vs 0.972), 10x10-4p-2f-6s (0.987 vs 0.998), RWARE small-2ag both settings, and SMAC 3s5z-15s and 3s5z-21s. The wins are often large, but the losses are real. There are no significance tests or effect sizes, and the standard deviations are big, so 'consistently' is not supported.\n\nThe deeper issue is the 'automatically discovers the optimal sight range' framing. The optimal range is by construction the argmax of recent episode returns within a hand-chosen set D. Moreover, the UCB reward is collected under a single shared policy that is learning from a replay buffer mixing all ranges, so the return for each range is confounded with the policy's history. Early dominance of one range can create a self-reinforcing loop. The paper never checks whether the selected range is optimal under a fixed policy, and it does not run a random-selection ablation over the same D. That ablation matters: if random selection matches DSR, the gain comes from training on a mixture, not from UCB. The fixed-scheduling comparison partly addresses this, since a non-adaptive schedule does worse, but it is not the same as random.\n\nWho this is for: practitioners tuning sensor ranges in cooperative MARL; the method is drop-in and cheap. It deserves a serious referee. My advice: send it to review, but push the authors to narrow the 'consistent' claim, report significance or effect sizes, and add a random-selection ablation.","headline":"A practical drop-in sight-range selector with broad experiments, but the 'consistent improvement' and 'optimal range' claims overreach given the confounded UCB rewards and several counter rows in Table 1.","tokens_in":45262,"tokens_out":1984,"would_cite":true,"duration_ms":20049,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A sliding-window UCB bandit that picks each agent's sight range episode by episode can replace manual range selection in cooperative MARL, lifting performance in LBF, RWARE, and SMAC.","keywords":["multi-agent reinforcement learning","sight range dilemma","upper confidence bound","sliding-window UCB","cooperative MARL","partial observability","observation range selection"],"falsifier":"Take an environment where exhaustive fixed-range training has already identified the best sight range, run DSR with matched total compute, and compare the range DSR converges to against that best range; if the converged range is not the empirically best one, the UCB is selecting the easiest range during policy learning rather than the best range. A sharper version freezes a fully trained policy and runs the meta-controller without further policy updates: DSR's design gives no signal in that regime, which would show that it tracks learning progress under each range, not the range's intrinsic value.","tokens_in":44402,"feed_emoji":"👁️","tokens_out":10939,"duration_ms":106543,"temperature":0.7,"pith_summary":"Cooperative multi-agent reinforcement learning faces a trade-off the paper calls the sight range dilemma: an observation window too small hides the partners and objects an agent needs to coordinate with, while a window too large floods the learner with redundant information that slows or degrades training. This paper tries to show that the dilemma can be resolved without global information or communication, by adding a lightweight meta-controller that picks the sight range at the start of every episode. The meta-controller is a sliding-window upper-confidence-bound bandit (SW-UCB) whose arms are candidate ranges and whose reward is the episode return. The paper reports that this wrapper, called DSR, improves final test scores or win rates across Level-Based Foraging, Multi-Robot Warehouse, and StarCraft Multi-Agent Challenge, works unchanged with QMIX, MAPPO, IQL, VDN, and IPPO, and accelerates early training by starting with narrow views and widening them. If true, DSR turns sight-range selection from a hand-tuned hyperparameter into an automatic by-product of training.","feed_headline":"Dynamic sight-range selection beats fixed ranges in MARL","feed_subtitle":"A bandit meta-controller picks each agent's observation range during training, lifting scores in LBF, RWARE, and SMAC.","key_machinery":"The load-bearing object is the sliding-window UCB meta-controller of equation (3): each candidate sight range $d_i$ is an arm, the reward is the episode return collected while that range was active, and the controller scores arms by the windowed mean return plus an exploration bonus $c\\sqrt{\\log\\min(e,w)/N_e(d_i,w)}$, with window size $w=5000$ episodes and exploration constant $c=2$. It sits on top of a modified observation function $Z(s,n_i,d)$ that masks the observation to the selected range, cropping grid views in LBF and RWARE and limiting the visibility radius in SMAC, with out-of-range entries set to a default or zero. The meta-controller operates at episode granularity while the underlying MARL algorithm trains on a single replay buffer fed by all ranges, and the paper argues this lets the controller steadily concentrate episodes on whatever range yields the best recent returns.","core_discovery":"The central discovery claimed is that the sight range dilemma can be handled by selection alone: keep the MARL algorithm untouched, modify only the observation function $Z(s,n_i,d)$ to crop each agent's local view to the chosen range $d$, and let a meta-controller decide $d$ at the start of each episode from the recent episode returns of each candidate range. The meta-controller uses the sliding-window UCB rule (equation 3), with a window of 5000 episodes and an exploration constant $c=2$, treating each range as a bandit arm. Across their experiments the authors report that DSR matches or beats the fixed-range baseline in nearly every setting, with the largest improvements exactly where large fixed ranges are known to hurt: for example, LBF 10x10-4p-4f-coop-10s improves from 0.338 to 0.798 and SMAC MMM2-21s from 0.190 to 0.714. They further report that the selected range typically starts small and grows during training, and that a hand-designed fixed expansion schedule underperforms DSR, which they read as evidence that dynamic selection both accelerates learning and reveals how much information the task actually needs.","pith_inferences":["My read is that a large share of the reported gain may come from a curriculum effect rather than from discovering the truly optimal range: small ranges pay off early in training, so the UCB naturally lingers there before widening, and a well-timed manual schedule might capture part of the same benefit.","A testable extension the paper invites but does not pursue: feed the selected range $d$ to the policy as an explicit conditioning input, so the network can specialize per range instead of inferring the range from observation statistics; this would also make the UCB's reward signal cleaner.","The authors explicitly leave per-agent, heterogeneous ranges and continuous range spaces to future work, so the current claim is bounded to one shared, discrete range per environment; whether the selection signal survives finer granularity is open."],"forward_implications":["Practitioners no longer need to sweep sight ranges by hand: DSR wraps an existing MARL algorithm and converges to a range on its own, so the observation width becomes an output of training rather than an input.","Training is accelerated because the controller tends to start agents on small, simple observations and widen them as the policy matures; the reported curves rise faster than fixed-range baselines while reaching equal or better final scores.","The converged range is an interpretable design signal: it states how much of the environment the agents actually rely on, which the authors propose as guidance for sensor design in applications such as autonomous driving.","Because DSR relies only on per-agent observation cropping, it applies where no global state or communication channel exists, which is the regime the authors argue prior communication-based solutions cannot serve."],"supporting_citations":[{"why":"Supplies the sliding-window UCB rule that the meta-controller applies to choose among sight ranges.","marker":"[8]"},{"why":"Introduces the sight range dilemma and provides the communication-based baseline and SMAC-DT tasks that DSR is compared against.","marker":"[24]"},{"why":"Provides prior evidence that redundant global information can hurt coordination, motivating the dilemma DSR targets.","marker":"[9]"},{"why":"QMIX, the primary MARL algorithm DSR is wrapped around in the main experiments.","marker":"[20]"},{"why":"MAPPO, the actor-critic algorithm used to show DSR transfers across algorithm families.","marker":"[36]"},{"why":"SMAC, one of the three benchmark environments and the base of the dynamic-team-composition comparison.","marker":"[21]"},{"why":"Supplies the benchmark training pipeline and the LBF and RWARE environment implementations used in the experiments.","marker":"[18]"}],"fun_headline_variants":["Adaptive sight-range bandit lifts MARL on three benchmarks","UCB-chosen observation ranges speed MARL and boost scores","Pick each agent's sight range on the fly for better MARL","Dynamic sight range selection outlearns fixed schedules in MARL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the recent episode returns collected under each sight range honestly measure that range's quality, even though those returns come from a single policy that is still learning and is shared across all ranges through one replay buffer.","fun_headline_variants_meta":{"raw":{"variants":["Adaptive sight-range bandit lifts MARL on three benchmarks","UCB-chosen observation ranges speed MARL and boost scores","Pick each agent's sight range on the fly for better MARL","Dynamic sight range selection outlearns fixed schedules in MARL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000201,"raw_usage":{"total_tokens":1410,"prompt_tokens":1008,"completion_tokens":402,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":330}},"tokens_in":624,"tokens_out":402,"duration_ms":5082,"temperature":1.0,"reasoning_tokens":330,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:25:40.088973+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an environment where exhaustive fixed-range training has already identified the best sight range, run DSR with matched total compute, and compare the range DSR converges to against that best range; if the converged range is not the empirically best one, the UCB is selecting the easiest range during policy learning rather than the best range. A sharper version freezes a fully trained policy and runs the meta-controller without further policy updates: DSR's design gives no signal in that regime, which would show that it tracks learning progress under each range, not the range's intrinsic value.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the sight range dilemma and provides the communication-based baseline and SMAC-DT tasks that DSR is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides prior evidence that redundant global information can hurt coordination, motivating the dilemma DSR targets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"QMIX, the primary MARL algorithm DSR is wrapped around in the main experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MAPPO, the actor-critic algorithm used to show DSR transfers across algorithm families."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SMAC, one of the three benchmark environments and the base of the dynamic-team-composition comparison."}],"review_version":1}