{"id":"00b01f5e-9bed-4af1-be6c-9d38dc454c7e","arxiv_id":"2508.14387","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DEXTER-LLM couples multi-stage LLM-based subtask generation with branch-and-bound search and mixed-integer programming to achieve dynamic, human-verified multi-robot task planning in unknown environments.","lead":"A new framework, DEXTER-LLM, combines large language model reasoning with mathematical optimization to coordinate teams of robots in unknown, changing environments. The system automatically decomposes missions into subtasks, assigns them to robots, and replans when new objects or tasks are discovered.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"100% success is a human-in-the-loop result: without verification SR drops to 65–71%, so the headline understates the human's load-bearing role.","rationale":"The strongest claim is empirical: 100% success, roughly 3x more tasks completed, and superior plan quality. The paper's own ablation in §IV-D-4 directly measures the contribution of human verification to that success. Removing verification drops SR to 65% and 71% in the two scenarios. That means the 'exceptional performance' is not produced by the LLM-plus-optimization pipeline alone; it requires a human to catch and correct errors after each module. Since the abstract and conclusion do not qualify the 100% success claim with this dependency, the central claim, as stated, is not established for the automated system. This is the most load-bearing concern because it targets the headline success metric directly. I considered whether the unproven 'optimality' claim (Section III-D: branch-and-bound plus integer optimization) is more central; it is also a weakness—no formal proof or optimality gap is reported—but the headline numbers are about success/completed tasks, and the optimality claim is secondary and applies to assignment given LLM-generated strategies. The human-dependency concern is backed by the paper's own data, so it is the strongest. I do not see an internal inconsistency: the authors explicitly design human verification into the framework and argue it requires only 2–3 interventions. Therefore a conditional verdict is appropriate; the paper should be accepted only if the claims are reframed as human-in-the-loop and additional no-verification/error-injection data are reported. The reader's conditional verdict already captures this, so no change is needed.","tokens_in":12566,"tokens_out":3813,"duration_ms":44483,"concrete_test":"Rerun the Scenario-I and II evaluations under the exact reporting protocol but with the human-in-the-loop verification module disabled; report SR, number of invalid plans, and the types of errors caught. If SR remains 100% or within statistical noise, the concern is resolved. If SR drops to the reported 65–71%, then the headline metric must be reported both with and without verification, and the human dependency must be stated explicitly in the abstract and contributions. Additionally, to test human reliability, inject scripted operator errors in a fraction (e.g., 10%) of verification calls to measure how sensitive the system's success rate is to imperfect feedback.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim of 100% success across all scenarios is only achieved with human-in-the-loop verification after every module. Section IV-D-4 reports SR 100% with verification vs 65% (Scenario-I) and 71% (Scenario-II) without it, with 2–3 interventions per trial. This is the paper's own evidence that the automated pipeline alone is correct in only about two-thirds of trials. Therefore the central claim 'robust, generalizable, verifiable coordination' is not supported as a property of the automated framework; it is a property of a human-assisted system. If the human operator is unavailable, delayed, or incorrect, the 100% success rate collapses to the ablation rates. The abstract and contributions do not qualify the headline by this dependency; they present the framework as if the success is intrinsic. This is not an internal inconsistency (the design explicitly includes human verification), but it is the load-bearing condition for the empirical claims.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DEXTER-LLM, a centralized framework for online multi-robot task coordination in unknown environments. It combines four modules: (i) mission comprehension that converts natural-language or LTL missions into task DAGs; (ii) a multi-stage LLM-based subtask generator that produces layered DAGs with precedence/exclusion constraints; (iii) a branch-and-bound plus mixed-integer-programming module for subtask assignment and scheduling; and (iv) an event-triggered online adaptation module with human-in-the-loop verification. The authors report experiments in two simulated scenarios (industrial emergency and arctic protection) against five baselines, claiming 100% success rates, roughly three times more completed tasks, lower LLM query counts, and higher SPL for compound tasks. Ablations assess different LLMs, removal of prompting stages, module-triggering rates, and the effect of human verification.","tokens_in":12813,"tokens_out":4279,"duration_ms":52350,"significance":"If taken at face value, the framework is a plausible and potentially impactful architecture: it separates LLM-based open-world reasoning from a model-based optimizer and routes online events to the correct module, which addresses a real gap in LLM-based multi-robot planning. The work has clear strengths: the multi-stage prompting design is explicit and principled; the comparison covers five relevant baselines; and the module-triggering analysis in Table III directly addresses the cost of LLM calls. However, the central quantitative claims are not yet established. The 'optimal' assignment guarantee is conditional on exhaustive branch-and-bound search, yet the algorithm also stops on a time cutoff; the headline 100% success rate is achieved only with human verification, and the same table shows 65% and 71% success without it; and the empirical comparisons rest on three trials without error bars. The contribution is worth publishing after the claims are scoped and the evidence is strengthened.","major_comments":[{"comment":"The paper repeatedly claims 'optimality' and 'performance guarantees' (abstract, Section I-B, Section V), but the branch-and-bound procedure terminates 'until the planning time elapsed or all combinations of possible strategies ... are exhausted.' Unless the experiments always exhaust the search tree, or the lower bound can be shown to prune all suboptimal nodes, the result is a best-found-within-budget heuristic, not an optimum. Please provide a pseudocode block with the lower/upper bound definitions, prove the admissibility of the lower bound, and report how many times the time cutoff was actually hit. If the cutoff is commonly active, revise all 'optimal' wording accordingly.","section":"Section III-D-2"},{"comment":"The human-in-the-loop verification is load-bearing. The ablation shows that success rate drops from 100% to 65% (Scenario-I) and from 100% to 71% (Scenario-II) when human verification is removed, with 2–3 human interventions per trial. Thus the automated framework alone succeeds in only about two-thirds of trials on these scenarios, and the headline '100% success' is a property of the human-assisted system, not of the automated pipeline. The abstract and contributions should state this dependence explicitly, report the verification-free success rates as the automatic success rate, and specify how human feedback is solicited, validated, and what happens if the human is unavailable or errs.","section":"Table III and Section IV-D-4"},{"comment":"Each scenario comprises only three trials, yet Table I reports single-point means with no error bars or confidence intervals. Comparisons such as SR 1.00 versus 0.67 are based on n=3 and are not statistically meaningful. Please report per-trial results, full distributions, or significantly more repetitions, and state explicitly which performance differences are robust across trials. This is essential because the paper's central empirical claim is superiority over baselines in task completion and SPL.","section":"Table I and Section IV-A"},{"comment":"The abstract states '160 tasks and 480 subtasks completed on average (3 times the baselines),' but Table I reports 'Tasks Completed' values such as 160.14 (Scenario-I, compound) and 101.07 (Scenario-II, compound), and no subtask-completion metric is tabulated. The '480 subtasks' figure does not appear in the experimental sections. Please add the subtask-completion metric to the tables or rephrase the abstract. Similarly, '2 times higher plan quality for compound tasks' should be tied to a specific baseline and scenario in the results.","section":"Abstract and Table I"}],"minor_comments":[{"comment":"The formal problem statement does not define the optimization objective precisely: 'maximize task completion' is used, but the later scheduler minimizes makespan. Please define the overall objective and how task-completion quantity trades off with makespan.","section":"Section II and III-B"},{"comment":"The sentence 'Algorithmic details are omitted here due to limited space' is problematic because the optimality claim depends on those details. A compact pseudocode listing and explicit definitions of the lower/upper bounds are needed.","section":"Section III-D-2"},{"comment":"The phrase 'only 3 interventions are required during 11 tests' is ambiguous: does this mean 3 interventions across all 11 trials, or 3 per trial? Clarify, and also describe what kind of feedback the human provides (e.g., correcting the DAG, rejecting a strategy, adjusting a schedule).","section":"Section IV-D-4"},{"comment":"There is an inconsistency: the 'DeepSeek-V3' row for Scenario-II reports Tasks = 101.18, while the 'Ours (full)' row for Scenario-II reports Tasks = 106.29, although these should be the same configuration. Please explain or correct.","section":"Table II"},{"comment":"Several baselines are 'modified' or 'enhanced' with components of DEXTER-LLM (LiP-LLM, COHERENT, SMART-LLM). Please specify exactly which modules were replaced or added, so that the comparison is reproducible and fair.","section":"Section IV-C"},{"comment":"Minor typos and typesetting issues: 'Secnario-I' in Section IV-B, malformed mathematical symbols in Section III-B (e.g., the LTL formula), and inconsistent use of 'Scenario-I' vs 'Scenario I'. Please proofread the manuscript.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The ablation study is candid and the authors do not hide the human-dependence of their headline result, which is commendable. The main concern for the editor is over-claiming: 'optimality' and '100% success' are stated too strongly relative to the evidence. I would encourage the editor to request the code/data or a detailed reproducibility appendix, since the numerical results rest on only three trials and several baseline modifications are described only briefly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe real story of arXiv:2508.14387 is an architecture, not a breakthrough result. The architecture is a genuine integration: NL2TL mission comprehension, multi-stage LLM subtask generation, branch-and-bound/MIP assignment, and a seven-event adaptation taxonomy that decides which module re-triggers on each event type. That taxonomy is clean and practical, and the multi-rate triggering does cut LLM calls substantially. The ablation of the prompting stages is useful and shows the meta-policy stage matters. I do not know another system with this exact stack, so it is a plausible and coherent engineering contribution.\n\nThe headline, though, belongs to the human-assisted system, not the framework. The paper's own Table III shows success falling from 100% to 65–71% when human verification is removed, with only two to three interventions needed. The paper does state this in Section IV-D-4, so it is not hidden, but the abstract and contributions present 100% success as an intrinsic property. That framing overstates what the automated pipeline alone delivers. The empirical basis is also thin: three trials per scenario, no error bars, no code or data release, and an SPL metric that compares against the authors' own expert-defined ground-truth strategies. That makes the SPL advantage less convincing than it looks.\n\nThe optimality claim is not proven either. The branch-and-bound stops on a time cutoff, and the MIP details are omitted, so what remains is an anytime heuristic with strong engineering rather than a guarantee. The mathematical core is sketched, not derived, but the components are standard and the citation pattern is appropriate—it engages the relevant hybrid LLM+optimization baselines and prior work.\n\nSoft spots are in proportion: the human-verification dependency is load-bearing and should be faced directly; the variance and artifact issues are fixable with more trials and open code; the optimality language needs qualification. This paper is worth reading for anyone working on LLM-based multi-robot task planning and human-in-the-loop verification. It deserves referee time rather than a desk reject, but a serious referee should push for more honest claims, variance reporting, and ideally hardware experiments. I would not cite it in my own work yet.","headline":"A genuinely integrated LLM+optimization framework for multi-robot coordination, but the 100% success rate is a human-in-the-loop result, and the evidence is thinner than the claims.","tokens_in":13262,"tokens_out":2547,"would_cite":false,"duration_ms":31156,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"DEXTER-LLM combines LLM reasoning with search-based scheduling to coordinate robot fleets in unknown environments.","keywords":["multi-robot task planning","large language models","human-in-the-loop verification","online adaptation","temporal logic","task allocation and scheduling","unknown environments","explainable planning"],"falsifier":"Run the same two scenarios with the human-in-the-loop verification replaced by an automated formal check, such as LTL model checking of the layered DAG and the final schedule, and compare success rates. If automated verification cannot reproduce the human-level success rate, the framework's reliability depends on the operator; if it can, the human bottleneck is removable and the framework is fully automatic.","tokens_in":12499,"feed_emoji":"🤖","tokens_out":4776,"duration_ms":47064,"temperature":0.7,"pith_summary":"The paper proposes DEXTER-LLM, a four-module pipeline that lets a heterogeneous robot fleet plan and replan missions in environments whose features and tasks are not known in advance. It claims that by splitting work between LLMs (which reason about what subtasks make sense) and model-based search (which assigns and schedules those subtasks optimally), the system can keep a 100% success rate across simulated emergency and conservation scenarios while completing about three times more tasks than the baselines. The authors also claim the architecture is explainable: each module produces a checkable intermediate representation, and a human operator verifies the outputs. The central bet is that LLM open-world reasoning and formal scheduling are complementary rather than competing.","feed_headline":"Language model plus optimizer runs 160 robot tasks at 100%","feed_subtitle":"Splitting subtask reasoning from scheduling keeps multi-robot plans explainable and online-adaptable.","key_machinery":"The layered directed acyclic graph (DAG) is the backbone of the framework. Task-level nodes come from an LTL mission formula via an R-poset abstraction; each task node expands into a strategy-specific DAG generated by the LLM; the scheduler operates on the layered DAG. The key design is that LLM output is never used as the final plan—it is converted into structured constraints that a branch-and-bound search with mixed-integer programming optimizes, and every intermediate representation is human-verifiable.","core_discovery":"The paper claims that dynamic multi-robot coordination in unknown environments can be made reliable and explainable by enforcing a strict division of labor. LLMs generate candidate task-decomposition strategies in four prompted stages, and those strategies are encoded as layered directed acyclic graphs. A branch-and-bound search combined with mixed-integer programming then assigns the subtasks to robots so that temporal constraints hold and makespan is minimized. New events—new task instances, new priorities, new feature types, delays, robot failures—trigger only the affected module rather than a full replan, cutting LLM calls by 62–81%. The system is reported to achieve 100% success in all","pith_inferences":["The event-driven module-triggering discipline could be generalized: instead of asking an LLM to replan from scratch, let it propose deltas and let a solver verify them, which may cut costs in other LLM-in-the-loop systems.","Replacing the human verification step with a formal model checker on the layered DAGs would test whether the framework's reliability can become independent of operator availability; the ablation suggests this is the natural next experiment.","The scenario-dependence of LLM choice (GPT-4o best in one scenario, Qwen-2.5-max in another) hints that an ensemble or model selector could raise completed-task counts without changing the scheduler.","The 62–81% reduction in LLM calls is a quantitative argument for hybrid architectures in robotics, but it likely depends on the rate and type of online events, so stress tests with frequent novel feature types would reveal its limits."],"forward_implications":["Natural-language missions can be turned into verifiable LTL constraints and then into a task DAG, so plan correctness is checkable rather than probabilistic.","Online events can be handled by re-triggering only the affected module, lowering LLM query counts by 62–81% and keeping scheduling adaptation latency in the seconds range.","The framework retains 100% success when the underlying LLM is swapped (DeepSeek-V3, GPT-4o, Qwen-2.5-max, Grok-3), suggesting the reasoning stage is replaceable.","Compound tasks requiring three-robot coordination show the largest gains over baselines, indicating the method handles collaborative subtask dependencies.","The human-in-the-loop verification stage is essential: removing it drops success from 100% to 65–71%, so 'verifiable' in this paper means human-verified, not automatically checked."],"supporting_citations":[{"why":"Supplies the NL-to-LTL translator used by the mission comprehension module to turn natural-language missions into verifiable formulas.","marker":"[26]"},{"why":"Provides the R-poset abstraction used to convert LTL formulas into task DAGs with precedence and exclusion relations.","marker":"[3]"},{"why":"Off-the-shelf solver used to solve the mixed-integer program that assigns and schedules subtasks.","marker":"[28]"},{"why":"Baseline method for comparison and source of the SPL metric used to measure plan quality.","marker":"[24]"},{"why":"Baseline with closed-loop recovery, compared against in the compound-task experiments.","marker":"[6]"},{"why":"Baseline multi-robot LLM collaboration approach compared in the same scenarios.","marker":"[7]"},{"why":"Hybrid LLM-plus-optimization baseline included in the comparisons.","marker":"[5]"}],"fun_headline_variants":["LLM + search hybrid: 160 tasks, 100% success","Split LLM reasoning from scheduling for 100% robot task success","Divide LLM planning from optimizer assignment: 160 tasks, all success","Dynamic LLM coordination: 160 tasks online, zero failures"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that a qualified human operator is always available to verify every module's output; the paper's ablation shows success falls from 100% to 65–71% when that verification is removed, so the headline reliability claim rests on continuous human attention.","fun_headline_variants_meta":{"raw":{"variants":["LLM + search hybrid: 160 tasks, 100% success","Split LLM reasoning from scheduling for 100% robot task success","Divide LLM planning from optimizer assignment: 160 tasks, all success","Dynamic LLM coordination: 160 tasks online, zero failures"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000754,"raw_usage":{"total_tokens":3231,"prompt_tokens":827,"completion_tokens":2404,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":2328}},"tokens_in":571,"tokens_out":2404,"duration_ms":19324,"temperature":1.0,"reasoning_tokens":2328,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T18:34:03.716855+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same two scenarios with the human-in-the-loop verification replaced by an automated formal check, such as LTL model checking of the layered DAG and the final schedule, and compare success rates. If automated verification cannot reproduce the human-level success rate, the framework's reliability depends on the operator; if it can, the human bottleneck is removable and the framework is fully automatic.","supporting_citations":[{"cited_title":"Nl2tl: Transforming natural languages to temporal logics using large language models,","cited_arxiv_id":null,"evidence_quote":"Supplies the NL-to-LTL translator used by the mission comprehension module to turn natural-language missions into verifiable formulas."},{"cited_title":"Time minimization and online synchronization for multi- agent systems under collaborative temporal logic tasks,","cited_arxiv_id":null,"evidence_quote":"Provides the R-poset abstraction used to convert LTL formulas into task DAGs with precedence and exclusion relations."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Off-the-shelf solver used to solve the mixed-integer program that assigns and schedules subtasks."},{"cited_title":"Lip-llm: Integrating linear programming and de- pendency graph with large language models for multi-robot task planning,","cited_arxiv_id":null,"evidence_quote":"Baseline method for comparison and source of the SPL metric used to measure plan quality."},{"cited_title":"Roco: Dialectic multi-robot collaboration with large language models,","cited_arxiv_id":null,"evidence_quote":"Baseline multi-robot LLM collaboration approach compared in the same scenarios."},{"cited_title":"Smart-llm: Smart multi-agent robot task planning using large language models,","cited_arxiv_id":null,"evidence_quote":"Hybrid LLM-plus-optimization baseline included in the comparisons."}],"review_version":1}