{"id":"8cc8ee53-3b40-4b92-a8ca-cdb6e0e27d6e","arxiv_id":"2505.06513","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"LLM-Flock combines per-robot LLM planning with an influence-based plan-copying rule to stabilize decentralized multi-robot formations.","lead":"Each robot asks a language model for a formation plan, then copies the plan of the robot with the most neighbors until all agree. The authors show this reduces the disorder seen when robots plan independently, in simulation and with small Crazyflie drones.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Consensus selects plans by neighbor count, not geometric validity; Appendix D.1 shows a faulty high-influence plan propagating team-wide, so stable plan agreement is not yet stable correct formation.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: influence measures communication centrality, not geometric validity, and Appendix D.1 shows a faulty plan propagating team-wide. I agree with that assessment. The paper is honest in documenting failure cases and limitations, but the abstract and Section 4.2 overstate what the consensus protocol can guarantee. The central claim requires an unproven assumption that the most-connected robot's plan is sufficiently good; the protocol itself only enforces plan agreement. My concrete test would determine whether this assumption is actually needed by injecting a known bad plan at a high-influence node and checking whether the system fails. The reader's verdict of CONDITIONAL is appropriate: the method is plausible and clearly described, but the scope of the claim should be narrowed, and a plan-quality check or a comparison to previous LLM-based methods is needed before acceptance. My read does not move the verdict.","tokens_in":16822,"tokens_out":3471,"duration_ms":41596,"concrete_test":"Re-run the 10-robot circle experiment from Appendix D.1 with the faulty Claude 3.5 plan assigned to the robot with the highest initial neighbor count, and valid o3-mini plans assigned to all other robots. Measure final Procrustes error over 10 trials. If the team adopts the faulty plan and final error remains high, the consensus protocol cannot rescue bad plans. Then compare against two ablations on the same setup: (a) random leader selection among neighbors instead of highest influence, and (b) a consensus rule that rejects any proposed plan violating pairwise-distance or shape constraints. If (b) restores a low-error circle while (a) does not, the influence score is not the operative correctness mechanism, and the central claim would need to be restated as synchronization without geometric validation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The core novelty is the influence-based consensus protocol, but its adoption rule (Algorithm 1, Eq. 1) selects a plan purely by degree centrality: I_i(t) = |N_i(t)|/N, and a robot adopts the neighbor's plan whenever that neighbor has a higher influence score and a different plan. There is no geometric validity check, no plan-quality term, and no mechanism to reject a plan that violates the requested shape or distance constraints. Thus the protocol guarantees agreement on the plan of the most-connected robot, not agreement on a correct formation. This assumption is load-bearing because the abstract and Section 4.2 claim 'coherent and stable flocking formation' and 'stable convergence across all tested language models'. The paper's own Appendix D.1 is a direct counterexample: Claude 3.5 generates a distorted 10-robot circle plan, and 'this faulty plan ends up propagating to the entire team, and the final formation does not represent an ideal uniformly spaced circular shape.' Consensus here amplifies a single bad plan instead of correcting it. The influence score is orthogonal to plan correctness, and no experiment in the paper measures whether the highest-influence plan is, on average, better than the median or best plan available locally. The failure in D.1 is not an edge case irrelevant to the central claim; it shows that the claimed stable convergence is really convergence in plan agreement, with final accuracy dependent on the most-connected robot's LLM output. A secondary execution-side assumption, documented in D.2, is that LLMs will move toward assigned goals without overshooting; that also fails with Qwen-Max, but the plan-selection issue is more fundamental because consensus can propagate errors team-wide. This does not invalidate the method as a heuristic, but it narrows the central claim: LLM-Flock synchronizes plans, it does not validate them.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LLM-Flock, a decentralized multi-robot formation control framework in which each robot uses a large language model to generate a candidate global formation plan and then iteratively negotiates plan adoption through an influence-based consensus protocol. The influence score of a robot is its degree centrality (Eq. 1), and a robot adopts a neighbor's plan when that neighbor has a higher influence score and a different plan (Algorithm 1). Motion execution is also LLM-driven, with each robot querying its model for the next waypoint toward its assigned goal. The framework is evaluated in simulation across five LLM backends and several formation geometries, with Procrustes shape error as the quantitative metric, and in physical experiments with Crazyflie drones. The central claim is that the influence-based consensus protocol prevents the collapse-to-centroid and divergence observed in unstructured LLM-only planning, yielding stable convergence to the desired formations.","tokens_in":17079,"tokens_out":5614,"duration_ms":56775,"significance":"If the central claim is established, the paper makes a useful empirical contribution: it shows that a simple, communication-based plan-selection rule can reduce the failure modes of independent LLM planners in multi-robot formation tasks. The strengths of the paper are its reproducible setting (code and video are provided), the breadth of LLM backends considered, the inclusion of real-robot Crazyflie experiments, and the honest discussion of failure cases in Appendix D. However, the significance is conditional because the consensus protocol selects plans by communication centrality rather than by geometric validity, so the protocol guarantees plan agreement but not formation correctness. In addition, the quantitative evidence shows that weaker models still produce high residual errors with wide confidence intervals, and the abstract's claim of improvement over 'previous LLM-based methods' is not tested against any prior method. The work is a reasonable step, but the claims need to be reframed or the protocol needs to be strengthened.","major_comments":[{"comment":"The adoption rule selects the plan of the neighbor with the largest degree centrality and contains no geometric validity check or plan-quality term. Consequently, the protocol guarantees convergence to a common plan, not to a correct formation. This distinction is not merely theoretical: Appendix D.1 shows a faulty Claude 3.5 plan propagating to the entire team (Fig. 13(b)), and the failure is attributed to the influence score being orthogonal to plan correctness. The abstract's claim that consensus 'drives the system toward a coherent and stable flocking formation' and Section 4.2's claim of 'stable convergence across all tested language models' therefore overstate what the protocol establishes. The claims should be reformulated as convergence in plan agreement, or the protocol should incorporate a mechanism that rejects plans violating the formation constraints.","section":"Section 3.2, Eq. (1), Algorithm 1, Appendix D.1"},{"comment":"The goal-assignment procedure is described as a decentralized greedy rule, but as written each robot constructs its own available set from its local copy of the adopted plan and removes only its own choice. Nothing in Algorithm 1 synchronizes the 'available' sets or prevents two robots from selecting the same waypoint. The text states that 'once a goal position is assigned to a robot, it becomes unavailable for others,' but no communication or tie-breaking rule in the protocol implements this. The manuscript should specify the claim/coordination mechanism for goal assignment, or prove that the nearest-available rule is conflict-free under the assumed communication model; otherwise the reported formations may not be reproducible from the protocol alone.","section":"Section 3.2, Eq. (2), Algorithm 1, lines 8-10"},{"comment":"The baseline comparison confounds the influence-based consensus protocol with the goal-assignment rule. In the baseline, each robot follows its initially assigned LLM goal, while in LLM-Flock robots reassign goals by nearest-available-goal selection after adopting a plan (Eq. 2). The improved performance reported in Fig. 2 and Fig. 11 could therefore be partly due to the goal-reassignment mechanism rather than to the consensus protocol itself. To support the attribution, the baseline should use the same nearest-available-goal assignment on the initial plan, or the experiments should include an ablation that changes only the adoption rule while holding the goal-assignment mechanism fixed.","section":"Section 4.1, Eq. (2)"},{"comment":"The abstract claims 'notable improvements ... over previous LLM-based methods,' but no comparison to prior LLM-based multi-robot coordination methods (e.g., RoCo, SMART-LLM, ZeroCAP) is reported anywhere in the paper. The only comparative experiment is against the no-consensus baseline using the same LLM backend. Either the authors should add direct comparisons to at least one prior LLM-based formation-control method, or the abstract and conclusion should be revised to limit the claim to improvement over the unstructured-LLM baseline.","section":"Abstract, Section 4"},{"comment":"The sentence 'LLM-Flock enables stable convergence across all tested language models and team sizes' is not supported by the quantitative data. In Fig. 11(e), after 19 timesteps, Claude 3.5 and Qwen-Max show mean Procrustes errors well above 50 with wide 95% confidence intervals for the 10-robot circle, and Appendix D.2 documents a Qwen-Max execution failure in a triangle task where the robot oscillates without stabilizing despite full plan consensus. The claim should be qualified to the strongest reasoning models, or the quantitative threshold for 'stable convergence' should be defined and shown to be met by all models.","section":"Section 4.2, Fig. 11, Appendix D.2"}],"minor_comments":[{"comment":"The notation in Eq. (3) is confusing: it refers to 'xi(t) and yi denote the aligned positions of robot i and actual target formation, respectively,' while Appendix C defines the Procrustes error through R*(xi - xbar) - (yi - ybar). Please make the notation consistent and define the correspondence between robot indices and target points.","section":"Section 4.2, Eq. (3), Appendix C"},{"comment":"The text says 'More results are included in Appendix C,' but the extended quantitative results appear in Appendix B.2, while Appendix C defines the Procrustes metric. Please correct the cross-reference.","section":"Section 4.2 and Appendix B"},{"comment":"The goal-assignment step appears only inside the branch where a robot adopts a new plan. Please specify whether a robot that retains its current plan also recomputes its goal, and how the 'available' set is initialized in that case.","section":"Algorithm 1"},{"comment":"The extracted plan in the text lists '[47.11, 56]' as Vertex 1, but the LLM output above it says '[47.11, 55]'. Please correct the transcription.","section":"Appendix A.2"},{"comment":"The paper uses 'flocking' to describe the task of converging to a static geometric formation. Classical flocking typically involves ongoing motion with separation, alignment, and cohesion terms. Please clarify that the task is static formation acquisition rather than dynamic flocking, or justify the terminology.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's framing around 'previous LLM-based methods' is not supported by the experimental design, which only compares against a no-consensus baseline. I would ask the editor to require either a comparative experiment against at least one prior method or a revision of the abstract and title claims. The real-robot experiments are qualitative only (snapshots), so the 'practical viability' statement should be tempered accordingly. The code- and video-release policy is a positive signal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nWhat you should know: this is a reasonably solid systems paper, not a breakthrough. The new bit is influence-based consensus—each robot adopts the plan of the neighbor with the most visible links—applied to LLM-generated formation plans. It works: in simulation, the rule beats the no-consensus baseline, and the authors show real Crazyflie drones forming circles and crosses. The code and video are up, and the failure appendix is refreshingly honest—they show a distorted Claude 3.5 plan propagating to the whole team and a Qwen-Max motion overshoot.\n\nThe main soft spot is the gap between the claim and the mechanism. The influence score is pure degree centrality; it never looks at geometric validity. So the protocol guarantees agreement on the most-connected robot's plan, not on a correct formation. Appendix D.1 is the smoking gun: a bad plan propagates team-wide. That doesn't kill the method as a heuristic, but it narrows the central claim to 'stable plan agreement.' The abstract also says 'notable improvements over previous LLM-based methods,' yet the only comparison is against their own no-consensus baseline—no RoCo, SMART-LLM, or ZeroCAP runs. That overclaim is easy to fix in revision.\n\nOther soft spots are minor: 10 trials per configuration with wide CIs for weaker models, and the motion-execution failure in D.2 shows the LLM step controller can overshoot. Both are documented in the paper, which is to their credit.\n\nIf you're working on LLM-based multi-robot coordination, this is worth reading. It's a clean demonstration that a lightweight coordination layer can fix a lot of LLM inconsistency. It deserves a serious referee—the experiments are reproducible and the method is simple enough to check. I'd accept it for review but push for a softened abstract and an explicit statement that consensus selects influence, not quality.\n\nBest","headline":"A simple degree-based plan consensus rule does stabilize LLM-generated formations, but the paper overclaims: it synchronizes plans, not correctness, and never compares to prior LLM methods.","tokens_in":17674,"tokens_out":2590,"would_cite":true,"duration_ms":25054,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"LLM-Flock claims that coupling independent LLM-generated plans with an influence-based consensus protocol prevents centroid collapse and divergence, producing stable multi-robot formations.","keywords":["multi-robot systems","flocking","formation control","large language models","decentralized consensus","influence-based planning","plan adoption","Crazyflie drones"],"falsifier":"In a three-robot triangle run with a strong LLM, deliberately inject the distorted circle plan shown in Appendix D.1 as the plan of the most-connected robot; if the team adopts that plan wholesale and the final Procrustes error remains above the convergence threshold used in Section 4.2, then influence-based consensus alone does not guarantee stable convergence.","tokens_in":16591,"feed_emoji":"🤖","tokens_out":7982,"duration_ms":69295,"temperature":0.7,"pith_summary":"The paper claims that the instability of LLM-only multi-robot flocking can be fixed without central control by adding a lightweight consensus layer. Each robot asks its own LLM for a full formation plan, then repeatedly replaces that plan with the plan of whichever neighbor can talk to the largest fraction of the team; the team thereby converges on one plan and moves to the corresponding goal positions. The authors argue that this influence-based plan consensus prevents the centroid collapse and chaotic divergence that unstructured LLM planners exhibit, and they support it with simulations across five language models and formations of three to ten robots, plus physical Crazyflie drone tests. If right, it means language-model reasoning and classical decentralized coordination are complementary rather than competing: the LLM proposes, the communication graph disposes.","feed_headline":"Copying the most-connected robot's plan stabilizes LLM robot flocks","feed_subtitle":"Adopting the most-connected neighbor's plan lets drone teams form shapes without a central controller.","key_machinery":"The load-bearing mechanism is influence-based plan consensus. Influence is defined as the fraction of the team within communication range, $I_i(t)=|\\mathcal{N}_i(t)|/N$, so a robot with more neighbors is deemed more influential; each robot adopts the plan of the neighbor with strictly higher influence, then assigns itself the nearest available goal in the adopted plan. This makes communication topology, not geometric correctness, the arbiter of which plan wins, and it is what turns inconsistent LLM outputs into a single shared formation plan.","core_discovery":"The paper's central discovery is that independent LLM-generated formation plans, which by themselves collapse to the team centroid or scatter chaotically, can be reconciled into a single coherent plan by a simple decentralized adoption rule: at each negotiation round a robot replaces its plan with the plan of whichever neighbor has the largest fraction of the team within communication range, provided that fraction exceeds its own. Because plans propagate outward from the most-connected robots, the team converges to a shared formation and then moves to the corresponding goal positions, with each robot choosing the nearest unassigned waypoint. The authors report stable convergence across five different language models and team sizes of three to ten robots, in simulation and on physical Crazyflie drones.","pith_inferences":["Because influence is pure communication degree, the protocol is effectively a max-degree leader-election rule on the communication graph; an immediate testable consequence is that convergence time should scale with graph diameter and that a poorly connected robot holding the best plan can never win.","The Appendix D.1 failure with Claude 3.5 suggests a cheap fix the authors do not test: validating candidate plans against the desired inter-robot distance before adoption would filter faulty high-influence plans without changing the decentralized structure.","The consensus protocol is agnostic to what generates the plans, so replacing the LLM with any stochastic or rule-based planner should reproduce the same convergence dynamics; if true, the paper's contribution is the negotiation protocol rather than the language model."],"forward_implications":["Teams of LLM-driven robots can reach a shared formation plan using only local communication, with no central controller and no retraining of the language models.","Adding the influence-based adoption rule turns failure modes documented for unstructured LLM planners, such as centroid collapse and divergence, into convergent formations across triangle, square, circle, and cross shapes.","Convergence speed and residual error depend on the reasoning strength of the underlying LLM, with stronger models converging faster and to lower Procrustes error, while general-purpose models still improve under consensus.","The same protocol transfers from simulation to physical hardware, as Crazyflie drones form the target shapes in a motion-capture arena, showing the framework is not simulation-only."],"supporting_citations":[{"why":"Documents the centroid-collapse and divergence failures of LLM-only multi-agent flocking that LLM-Flock targets.","marker":"[24]"},{"why":"Establishes that unstructured LLM outputs are unreliable for multi-agent coordination, motivating the need for plan consensus.","marker":"[25]"},{"why":"Provides a prior LLM-based pattern-formation method that relies on centralized global knowledge, the approach this work makes decentralized.","marker":"[36]"},{"why":"Shows LLMs can serve as multi-robot coordinators through structured communication, supporting the feasibility of LLM planners.","marker":"[14]"},{"why":"Names the Crazyflie platform used in the physical validation experiments.","marker":"[37]"},{"why":"o3-mini is the LLM backend used in the qualitative, quantitative, and real-robot demonstrations.","marker":"[40]"}],"fun_headline_variants":["LLM robot flocks go stable by copying the most-connected neighbor","Stable drone swarms: LLMs plus influence-based consensus","Copy the most-connected plan to keep LLM robot flocks stable","LLM-driven flocking: adopt the hub's plan, avoid the chaos"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach assumes that the plan offered by the most-connected neighbor is a good enough formation plan and that each robot's LLM will actually move it toward its assigned goal, since influence scores measure communication centrality rather than geometric validity.","fun_headline_variants_meta":{"raw":{"variants":["LLM robot flocks go stable by copying the most-connected neighbor","Stable drone swarms: LLMs plus influence-based consensus","Copy the most-connected plan to keep LLM robot flocks stable","LLM-driven flocking: adopt the hub's plan, avoid the chaos"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000448,"raw_usage":{"total_tokens":2262,"prompt_tokens":949,"completion_tokens":1313,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":1239}},"tokens_in":565,"tokens_out":1313,"duration_ms":12551,"temperature":1.0,"reasoning_tokens":1239,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:40:01.111801+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In a three-robot triangle run with a strong LLM, deliberately inject the distorted circle plan shown in Appendix D.1 as the plan of the most-connected robot; if the team adopts that plan wholesale and the final Procrustes error remains above the convergence threshold used in Section 4.2, then influence-based consensus alone does not guarantee stable convergence.","supporting_citations":[{"cited_title":"https://www.bitcraze.io/","cited_arxiv_id":null,"evidence_quote":"Names the Crazyflie platform used in the physical validation experiments."},{"cited_title":"Openai o3-mini, January 2025","cited_arxiv_id":null,"evidence_quote":"o3-mini is the LLM backend used in the qualitative, quantitative, and real-robot demonstrations."}],"review_version":1}