{"id":"73a3c62f-133e-484c-83cb-5c286b03a0dd","arxiv_id":"2507.01378","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RALLY couples a two-stage LLM consensus module with a QMIX-style role-assignment network and reports higher reward and better generalization than three baselines in drone-swarm coverage simulations.","lead":"This paper presents RALLY, a control system that pairs a large language model's common-sense reasoning with reinforcement learning to coordinate drone swarms as they cover targets, dodge obstacles, and flee an enemy. A smart generalist would read it to see how LLM reasoning is being fused with multi-agent learning for robot swarms, and where that fusion still lacks rigorous evidence.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline advantage may reflect privileged game information in the prompt: Fig. 4 gives the LLM the enemy's exact pursuit strategy and scoring rules that CIHRL must discover from rewards, so the central claim of algorithmic superiority is not yet established.","rationale":"I read RALLY as a serious engineering integration: the two-stage LLM consensus module, the RMIX role-value mixing network, and the offline distillation to a local model are coherent, and the SITL demonstration is useful qualitative evidence. The central claim, however, is an empirical superiority claim, and the empirical comparison has a control problem that the paper does not address. The strongest_claim requires that the margin over CIHRL, CoNavGPT, and DITTO reflect the algorithmic design. The prompts shown in Fig. 4 hand the LLM the adversary's exact strategy, the scoring rule, and the target coordinates; Section IV.A confirms the adversary is trained to chase the nearest cluster of at least three agents, so this is not a generic prior but the exact ground truth. Baselines receive only raw observations and rewards. The generalization sections strengthen the concern because RALLY's prompt encodes the maximum permitted formation size while CIHRL must generalize from learned behavior. The paper reports no ablation that strips this information. For these reasons, the main load-bearing concern is information asymmetry, not the circularity of Theorem 1, which is real but secondary. A prompt-content ablation is cheap and decisive; until it is run, the conditional verdict remains appropriate. I agree with the reader's weakest_assumption and keep the verdict unchanged.","tokens_in":20390,"tokens_out":2973,"duration_ms":36022,"concrete_test":"Sanitize the RALLY prompts and rerun the Fig. 6 comparison: delete from X_task and M_CoT all game-rule content, namely the enemy pursuit strategy, the scoring threshold, the candidate-point coordinates, and the role-specific decision rules, leaving only the raw observations o_i^t and neighbor-communication data that CIHRL receives through Eq. (1) and Eq. (7). If RALLY's mean reward and convergence gap versus CIHRL collapse or materially shrink, the central claim should be reframed as prompt-information gain rather than algorithmic superiority; if the gap persists, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing requirement for the central claim is that RALLY's advantage over CIHRL, CoNavGPT, and DITTO in Fig. 6 comes from its architecture, not from information that only RALLY receives. The published prompts in Fig. 4 and Fig. 18 violate this: X_task states \"The strategy of a known enemy node is to directly pursue the nearest cluster of 3 or more nodes\", the scoring condition \"more than 3 agents covering one of scoring points\", the candidate-point coordinates, and the role decision rules in M_CoT. Section IV.A separately says the adversary is trained with PPO to chase the nearest cluster of at least three agents, so the prompt is encoding the exact ground-truth policy that the baselines must infer from rewards. CIHRL observes only o_i^t per Eq. (1) and receives scalar rewards; it never sees the enemy's strategy or scoring rule. Even the LLM baselines are not given this task-specific X_task in the reported comparison, so the 30-episode comparison in Fig. 6 is effectively between an informed system and uninformed systems. The generalization result in Figs. 9 and 10 makes the concern sharper because RALLY's prompt encodes the maximum permitted formation size, while CIHRL must generalize from learned behavior. The paper reports no ablation that removes this content. This is not an accusation; it is a missing control. The circularity of Theorem 1, via Assumption 2 and Eq. (13), is real but secondary to this empirical confound.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RALLY, a framework for the DS-CEFC UAV swarm task that combines a two-stage LLM-based semantic consensus mechanism (LLM_init followed by LLM_cons) with a role-value mixing network (RMIX) for dynamic role assignment and credit assignment. The authors claim that RALLY outperforms the MARL baseline CIHRL, the LLM planner CoNavGPT, and the role-play baseline DITTO in task coverage, convergence speed, and generalization, based on experiments in the Multi-Agent Particle Environment (MPE) and a Gazebo-ROS-PX4 software-in-the-loop (SITL) platform. They also present a capacity-migration method that fine-tunes a smaller Qwen model from GPT-4o-generated data, with runtime and memory measurements on an RTX 4090.","tokens_in":20601,"tokens_out":5312,"duration_ms":58757,"significance":"The problem is timely and the integration of LLM semantic reasoning with MARL value decomposition for role selection is a plausible and potentially useful direction. The paper is commendably transparent about its prompts (Figs. 4 and 18) and provides a real SITL deployment with a fine-tuned 1.5B model, which is a strength. If the privileged-information confound I describe below is resolved and the statistical evidence is added, the approach would be a meaningful advance over both pure MARL and pure LLM planners for the DS-CEFC setting. As it stands, however, the empirical advantage and the theoretical superiority claim are not yet established.","major_comments":[{"comment":"The headline comparison in Fig. 6 is confounded by privileged information. The X_task prompt in Fig. 4 tells the LLM that 'the strategy of a known enemy node is to directly pursue the nearest cluster of 3 or more nodes', states the scoring rule ('Scoring happens only when more than 3 agents covering one of scoring points'), and lists the eight candidate target coordinates, while Section IV.A separately states that the adversary is trained with PPO to chase the nearest cluster of at least three agents. CIHRL, positioned as the SOTA MARL baseline, receives only the local observation o_i^t of Eq. (1) and scalar rewards, so it must infer exactly the regularities that RALLY is handed in text. The Fig. 6 result, and the generalization results in Figs. 9 and 10, therefore do not establish that RALLY's architecture is superior; they may only establish that RALLY's prompt contains task information the baselines lack. Please add a control that removes the enemy-strategy and scoring details from X_task, or grant the baselines the same information, and report the comparison.","section":"§IV.B.1, Fig. 4, Eq. (1)"},{"comment":"Theorem 1 is circular as stated. Assumption 2 asserts that 'extra contextual reasoning ... leads to higher-quality Q-values', which is precisely the conclusion of the theorem, and Eq. (13) claims Qi(o_i^t, k_i^t, g_i^t) >= Qi(o_i^t, k_i^t, g'_i^t) 'by construction of LLM_cons' rather than by any property proved from the Dec-POMDP structure. The proof then applies Assumption 1 to a strict improvement that has been assumed, not derived. Please either prove a nontrivial sufficient condition under which the refinement step improves expected return, or explicitly downgrade the result to a conditional proposition whose premise is validated empirically, for example by the missing one-stage ablation.","section":"§III.A, Assumption 2, Eq. (13), Theorem 1"},{"comment":"The empirical claims are not supported with statistical evidence. Fig. 6 is described as 'average rewards over 30 test episodes', but no error bars, number of independent seeds, standard deviations, or significance tests are reported anywhere, and the text's assertion that RALLY has the 'narrowest variance distribution' is not backed by displayed variance. Figs. 9, 10, and 12 make generalization and role-number claims from what appear to be single curves or box plots without confidence information. Please report multiple seeds with confidence intervals and appropriate statistical tests for all headline comparisons.","section":"§IV.B, Figs. 6, 9, 10, 12"},{"comment":"The two-stage consensus component LLM_cons is never empirically ablated. The only support for the second stage is Theorem 1, which is circular as noted above, and the experiments compare the full RALLY against external baselines, not against RALLY with the consensus refinement removed. Please add a one-stage versus two-stage ablation to the main comparison; without it, the contribution of the consensus stage to the Fig. 6 result is untested.","section":"§III.A and §IV.B"}],"minor_comments":[{"comment":"The expression for Qtot has an unmatched parenthesis, with 'at' appearing before the closing parenthesis; please fix the typo.","section":"Eq. (10)"},{"comment":"The prompt says 'clusters of three or more nodes' but the scoring rule says 'more than 3 agents covering one of scoring points', while the CoT prompt says 'needs cluster with other two teammates'; please harmonize these thresholds.","section":"Fig. 4"},{"comment":"The reproduced figures lack axis labels and legends; please ensure all figures are self-contained and readable.","section":"Figs. 7 and 8"},{"comment":"The abstract says 'offline training of role selection strategies', while Section III.B describes an offline pre-collection phase followed by online RMIX training; please align the terminology.","section":"Abstract and Section III.B"},{"comment":"The term 'yoked' in the title is never defined in the paper; please add a sentence explaining the intended meaning.","section":"Title and Section I"}],"recommendation":"major_revision","confidential_remarks":"The privileged-information issue is the main risk to the central claim; if the authors can show the advantage survives prompt-information controls, I would be willing to revisit. The paper is within the journal's scope, but the theorem should be reframed regardless, and the statistical reporting needs to meet the standard for the empirical claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: RALLY is a sensible engineering integration of an LLM two-stage consensus module with a QMIX-style role-value mixing network for the DS-CEFC pursuit-evasion task. The distillation results from GPT-4o to a 1.5B model with measured memory footprints are practically useful. But the headline empirical claim is not yet established: the LLM is given privileged task information in the prompt that the MARL baselines must learn from rewards, and the superiority theorem is circular.\n\nWhat is actually new is the specific combination: LLMinit for local intent, LLMcons for neighborhood consensus, RMIX for dynamic role assignment, and the LoRA fine-tuning path to Qwen2.5-1.5B. Generalization experiments to 9–11 agents and different grid layouts address a real pain point. Publishing the prompts in the appendix is good practice.\n\nThe soft spots are serious but fixable. The stress test is right: Fig. 4 states the enemy's actual policy ('directly pursue the nearest cluster of 3 or more nodes') and the scoring rules, while CIHRL only sees local observations and scalar rewards. That makes the comparison one between an informed system and uninformed systems. The generalization result is even more affected because the prompt encodes the maximum formation size. There is no ablation removing that content, so the margin in Fig. 6 could be largely prompt leakage. This is a missing control, not an accusation, but for a paper whose central claim is empirical superiority it is load-bearing.\n\nThe theory is weaker. Theorem 1 assumes exactly what it tries to prove: Assumption 2 states extra contextual reasoning 'leads to higher-quality Q-values', and Eq. (13) asserts the improvement 'by construction'. The monotonicity from QMIX only says larger individual Q's do not hurt the total; it says nothing about whether LLMcons actually produces larger Q's. The theorem should be reframed as a conditional statement or dropped.\n\nMinor issues: no error bars, seeds, or statistical tests; SITL validation is qualitative; no code or data released.\n\nFor a reader working on LLM-MARL integration, this is a useful template and a cautionary example in evaluation. It deserves a serious referee, with expectations of major revision: ablate prompt content, report seeds and variance, and release code. I would not cite it as evidence for a working advantage until those are addressed.","headline":"A useful LLM-MARL integration with a prompt-leakage confound and a circular theorem; engineering is real, but the empirical claim needs a missing control.","tokens_in":21358,"tokens_out":4117,"would_cite":false,"duration_ms":42562,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a two-stage LLM consensus module paired with a role-value mixing network gives UAV swarms higher task coverage, faster convergence, and better generalization than MARL-only, LLM-only, or fixed-role baselines on the…","keywords":["UAV swarm control","large language model","multi-agent reinforcement learning","role heterogeneity","agentic AI","consensus inference","credit assignment","DS-CEFC"],"falsifier":"Run the DS-CEFC benchmark with RALLY's prompts stripped of strategy leakage: remove the sentence describing the enemy's 'pursue the nearest cluster of 3 or more nodes' behavior and drop the explicit scoring and role-choice rules, leaving only the raw positions, velocities, and targets of Eq. (1). If the Fig. 6 mean-reward gap between RALLY and CIHRL collapses to noise while CoNavGPT and DITTO also shift, the central claim of algorithmic superiority is refuted.","tokens_in":20015,"feed_emoji":"🚁","tokens_out":8288,"duration_ms":75262,"temperature":0.7,"pith_summary":"This paper claims that a UAV swarm coordinates better when a large language model and a multi-agent reinforcement learning network share the job: the LLM translates local sensor readings into natural-language target intentions and consensus proposals, while a learned role-assignment network decides in real time which agents act as Commander, Coordinator, or Executor. The proposed system, RALLY, is tested on a pursuit-evasion coverage task in which agents must split into clusters of three or more to score on target regions while dodging a faster chasing enemy. The authors report that RALLY reaches higher mean reward, converges faster, and generalizes to swarm sizes (9-11 agents) and target layouts its training never saw, beating a MARL baseline (CIHRL), a pure-LLM planner (CoNavGPT), and a role-play baseline (DITTO). If the claim holds, it points to a practical division of labor between semantic priors and online learning for robotic swarms.","feed_headline":"Two-stage LLM consensus beats MARL and LLM baselines in UAV swarms","feed_subtitle":"A role-value mixing network adapts swarm roles online and keeps scoring as formations grow to 11 agents.","key_machinery":"Three components carry the argument. First, LLMHC, the two-stage LLM consensus module: LLMinit maps local observation $o_i^t$ to an initial intention $g'_i$, and LLMcons, given neighbors' intents and roles, outputs the consensus goal $g_i^t$; role definitions and threat logic are injected through structured prompts with chain-of-thought guidance. Second, RMIX (Role-value Mixing Network): a monotonic mixing network in the style of QMIX that combines individual role-values $Q_i(o_i^t, k_i^t)$ into $Q_{tot}$ with non-negative hypernetwork weights, guaranteeing $\\partial Q_{tot}/\\partial Q_i \\ge 0$ and making role selection $\\arg\\max_k Q_i$ trainable by TD updates over a buffer seeded with GPT-4o role assignments. Third, capacity migration: LoRA fine-tuning of a small Qwen2.5 model (1.5B-7B) on roughly 8,231 filtered GPT-4o samples shrinks memory to under 5GB so consensus inference can run distributively, while a contingency rule handles illegal LLM outputs: a Commander keeps its intent, a Coordinator defers to a valid Commander, and an Executor follows any available superior.","core_discovery":"RALLY's central claim is that two-stage LLM consensus inference yoked to a role-value mixing network outperforms both learning-only and LLM-only approaches on the DS-CEFC task. Each agent first asks the LLM, prompted with its local observation and task rules, for an initial target intention; after exchanging intentions and roles with neighbors, a second LLM pass refines them into a consensus goal. The RMIX network, a QMIX-style monotonic mixer with non-negative weights, aggregates per-agent role-values $Q_i(o_i^t, k_i^t)$ into a global $Q_{tot}$, so role assignments are chosen by $\\arg\\max_k Q_i$ and trained semi-offline on a replay buffer seeded by GPT-4o role suggestions. The paper formalizes the two-stage design as Theorem 1: under monotonic value factorization and the assumption that contextual reasoning improves Q-values, the two-stage policy's expected return strictly exceeds the one-stage policy's whenever the refinement helps at least one reachable agent. Empirically (Figs. 6, 9, 10), RALLY achieves higher mean reward with smaller variance than CIHRL, CoNavGPT, and DITTO, keeps scoring as swarm size grows to 9-11, and holds its performance across $3\\times 3$, $2\\times 4$, and $4\\times 2$ target grids.","pith_inferences":["The headline comparison may overstate the algorithmic gain: RALLY's task prompt discloses the enemy's exact pursuit strategy and the scoring rules, while CIHRL must infer them from raw observations and rewards, so a prompt-restricted ablation could shrink the reported margin substantially.","A concrete test: give the MARL baseline the same privileged knowledge as an auxiliary observation or shaped reward; if RALLY's advantage persists, it is the semantic-consensus mechanism, and if not, it is prompt-side information rather than the architecture.","The RMIX monotonicity guarantee could carry over to other LLM+MARL role hierarchies, not just DS-CEFC: as long as a role improvement never decreases joint value, injecting LLM priors into other hierarchical task decompositions should inherit the same convergence argument.","Because the LLM stage is memoryless, the paper's own stated remedies-test-time training and diversified reasoning paths to escape CoT local optima-are the natural next experiments, and the appendix's prompt-sensitivity examples suggest performance depends heavily on prompt wording."],"forward_implications":["Generalization to unseen swarm sizes: because the maximum permitted formation size is encoded in the prompt, RALLY forms split sub-clusters as the swarm grows, avoiding the 'habitual grouping' collapse the authors document for CIHRL at 9-11 agents.","Practical deployment: the fine-tuned 1.5B model runs in 2.9 GB memory with about 14.5 s average inference on an RTX 4090, and the SITL experiment shows the full Gazebo-ROS-PX4 loop executing distributed consensus with dynamic role transitions mid-episode.","Sample-efficient role learning: seeding the RMIX replay buffer with LLM-generated role assignments shrinks the cold-start problem in a $3^8$ joint role space, letting a small MLP learn role selection without large online exploration budgets.","Three roles is the design sweet spot: a role-count ablation (Fig. 12) finds that one role collapses performance, two roles raise variance, and a fourth 'Decoy' role lowers mean reward, so the three-role hierarchy is the claimed optimum.","Interpretability as a byproduct: the consensus output is a natural-language recommendation with role alignment, so a human can audit why a swarm split or regrouped at each decision step."],"supporting_citations":[{"why":"Defines the DS-CEFC task, the reward components, and the mid/low-layer policies, and serves as the primary MARL baseline (CIHRL) that RALLY must beat.","marker":"[2]"},{"why":"Supplies the monotonic value factorization (Assumption 1) that the RMIX mixing network adapts for role-value aggregation.","marker":"[21]"},{"why":"Generates the offline role assignments and reference samples that seed the RMIX replay buffer and the fine-tuning dataset.","marker":"[58]"},{"why":"Co-NavGPT is the pure-LLM global-planner baseline used for comparison.","marker":"[30]"},{"why":"DITTO is the role-play LLM baseline that demonstrates fixed-role heterogeneity and is compared on role-based collaboration.","marker":"[51]"},{"why":"Provides the PPO policy used for the adversarial enemy that chases clusters, defining the evasion pressure of the task.","marker":"[52]"},{"why":"The Multi-Agent Particle Environment where the main reward, convergence, and generalization experiments are run.","marker":"[49]"},{"why":"Supplies the Gazebo-PX4 Software-In-The-Loop platform used to validate RALLY under realistic quadrotor dynamics.","marker":"[50]"},{"why":"LoRA is the parameter-efficient fine-tuning method behind the capacity-migration step to a small local LLM.","marker":"[62]"},{"why":"Chain-of-thought prompting is the technique embedded in the task prompts to improve LLM reasoning in the consensus stages.","marker":"[55]"}],"fun_headline_variants":["Two-stage LLM consensus and role mixing beat baselines in UAV swarms","Role-adaptive LLM yokes UAV swarm navigation to beat baselines","LLM-driven role switches scale UAV swarm coverage to 11 agents","RALLY uses adaptive roles and LLM consensus to outperform MARL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes RALLY's margin reflects its algorithm and not the privileged information baked into its prompts: the task prompt states the enemy's exact strategy and the scoring rules, while MARL baselines must discover those regularities from raw observations and rewards; if prompts were limited to baseline-equivalent information, the reported advantage could shrink or vanish.","fun_headline_variants_meta":{"raw":{"variants":["Two-stage LLM consensus and role mixing beat baselines in UAV swarms","Role-adaptive LLM yokes UAV swarm navigation to beat baselines","LLM-driven role switches scale UAV swarm coverage to 11 agents","RALLY uses adaptive roles and LLM consensus to outperform MARL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000509,"raw_usage":{"total_tokens":2571,"prompt_tokens":1128,"completion_tokens":1443,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":744,"completion_tokens_details":{"reasoning_tokens":1365}},"tokens_in":744,"tokens_out":1443,"duration_ms":112038,"temperature":1.0,"reasoning_tokens":1365,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:55:58.295872+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the DS-CEFC benchmark with RALLY's prompts stripped of strategy leakage: remove the sentence describing the enemy's 'pursue the nearest cluster of 3 or more nodes' behavior and drop the explicit scoring and role-choice rules, leaving only the raw positions, velocities, and targets of Eq. (1). If the Fig. 6 mean-reward gap between RALLY and CIHRL collapses to noise while CoNavGPT and DITTO also shift, the central claim of algorithmic superiority is refuted.","supporting_citations":[{"cited_title":"Decentralized consensus inference- based hierarchical reinforcement learning for multi-constrained UA V pursuit-evasion game,","cited_arxiv_id":null,"evidence_quote":"Defines the DS-CEFC task, the reward components, and the mid/low-layer policies, and serves as the primary MARL baseline (CIHRL) that RALLY must beat."},{"cited_title":"QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learn- ing,","cited_arxiv_id":null,"evidence_quote":"Supplies the monotonic value factorization (Assumption 1) that the RMIX mixing network adapts for role-value aggregation."},{"cited_title":"Large language models are superposi- tions of all characters: Attaining arbitrary role-play via self-alignment,","cited_arxiv_id":null,"evidence_quote":"DITTO is the role-play LLM baseline that demonstrates fixed-role heterogeneity and is compared on role-based collaboration."},{"cited_title":"Multi-agent actor-critic for mixed cooperative-competitive environments,","cited_arxiv_id":null,"evidence_quote":"The Multi-Agent Particle Environment where the main reward, convergence, and generalization experiments are run."},{"cited_title":"Vision-Based software- in-the-loop-simulation for unmanned aerial vehicles using gazebo and PX4 open source,","cited_arxiv_id":null,"evidence_quote":"Supplies the Gazebo-PX4 Software-In-The-Loop platform used to validate RALLY under realistic quadrotor dynamics."},{"cited_title":"LoRA: Low-rank adaptation of large language models,","cited_arxiv_id":null,"evidence_quote":"LoRA is the parameter-efficient fine-tuning method behind the capacity-migration step to a small local LLM."},{"cited_title":"Chain-of-thought prompting elicits reasoning in large language models,","cited_arxiv_id":null,"evidence_quote":"Chain-of-thought prompting is the technique embedded in the task prompts to improve LLM reasoning in the consensus stages."}],"review_version":1}