{"id":"8f2f0de9-25c8-4a83-b7a3-2f50cf45560c","arxiv_id":"2511.21886","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A learned world model predicts per-agent execution times from Action Dependency Graphs, and steering CBS/MAPF-LNS with those predictions cuts simulated deadline penalties by up to 20%.","lead":"Warehouse robots run on plans that assume perfect, synchronized movement, but real robots slow down, turn, and wait. This paper builds a learned world model that predicts how long each robot action will actually take, then feeds those predictions into standard multi-robot planners to meet real deadlines — cutting deadline penalties by up to 20% in simulated warehouses.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline 20% gain is demonstrated only on the training-map pool; planner-level OOD generalization is never tested.","rationale":"I focused on map-level generalization because the paper's scientific claim — that a learned execution-time model can replace constant-speed estimates as a search signal — depends on transfer to environments not seen during training. The body supports transfer only at the prediction level (Table I), not at the planner level: WH-L and boston appear in the ExecTimeNet evaluation but not in the REMAP planner experiments. Since training graphs come from the same six maps used in the planner evaluation, the 20% improvement could be in-distribution interpolation. This is a sharper, more testable version of the reader's closed-loop concern: even within SMART, the evaluation should hold out maps. The abstract/body mismatch is real but concerns submission integrity more than the scientific argument itself; the OOD planner test directly probes the load-bearing claim. Verdict remains CONDITIONAL because the concern is addressable and does not by itself show the in-distribution result is wrong.","tokens_in":15504,"tokens_out":5274,"duration_ms":52168,"concrete_test":"Rerun the REMAP-vs-baseline protocol from Section IV-B on WH-L and boston (the two OOD maps in Table I): 25 instances per map, same K_D selection rule from Appendix F, same K_u in {0.1,0.05,0.03}, same 60s budget, and report penalty-gap curves for MAPF-LNS and CBS (equivalent of Figs. 3-5). If REMAP no longer beats ConstExec by a similar margin on these unseen maps, the 20% claim is in-distribution only and the generalization part of the abstract is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The body's central claim is that REMAP's learned execution-time model yields up to 20% better solution quality than constant-speed baselines. The weakest support for that claim is map-level generalization. ExecTimeNet's OOD generalization is tested only as prediction MAPE: Table I reports WH-L and boston (marked *) with MAPE 6-8% vs 4-6% in-distribution. But the REMAP planner evaluation in Section IV-B is described as 'six MovingAI Benchmark maps' and never includes WH-L or boston; Figs. 3-5 and the fixed-iteration Fig. 4 all use maps from the same six-map pool (empty, maze, random, room, den312d, WH-S) from which Section IV-A1 collected the 9,433 training graphs. Thus the headline 20% may reflect interpolation over ADG structures the model has already seen, not transfer to new topologies — the property the abstract promises and the premise for a 'world model' of execution. If REMAP's advantage is an in-distribution artifact, the strongest claim fails. The K_D grid search in Appendix F tunes the deadline regime on each map/agent count but does not introduce unseen environments. The body never reports REMAP's planner-level performance on the two OOD maps, so the generalization claim in the abstract has no planner-level evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript (full text) proposes REMAP, a planning framework for MAPF with Real-world Deadlines (MAPF-RD). REMAP wraps existing search-based MAPF planners (MAPF-LNS and CBS) with a learned execution-time predictor, ExecTimeNet, which takes an Action Dependency Graph (ADG) encoding of a candidate plan and predicts per-agent wall-clock completion times (or distributions thereof). These predictions are used to compute deadline penalties and guide the search. ExecTimeNet is trained on SMART simulator traces generated from six MovingAI maps, and REMAP is evaluated by re-simulating its output plans in SMART. The paper reports up to a 20% improvement in solution quality (penalty gap relative to a Virtual Best Solver) over constant-speed baselines on up to 300 agents. The supplied arXiv abstract additionally claims an ESADG post-processing method, up to 40% improvement, and 15.3% total execution-time reduction on physical hardware, but none of these appear in the full text, which is a REMAP-only simulation study.","tokens_in":15627,"tokens_out":3447,"duration_ms":32603,"significance":"If the central claim holds, the paper's contribution is useful: it shows that a learned surrogate of MAPF execution, taking an ADG structure as input, can act as a search signal and improve deadline-related objectives over simple constant-speed estimates. The paper includes several strengths: a fixed-iteration control in Fig. 4, which mitigates the concern that the performance gap is only an artifact of differing iteration counts; a feature ablation in Table II; a distributional variant with a proper likelihood loss; and a careful discussion of CBS non-optimality in Appendix E. However, the significance is currently limited by the absence of any planner-level out-of-distribution evaluation, by the closed SMART-only loop, and by the fact that the abstract promises hardware/ESADG results that are not present in the body.","major_comments":[{"comment":"The supplied abstract claims 'ESADG ... up to 40% normalized improvement' and 'on physical hardware ... reduces total execution time by up to 15.3%'. The full text contains no ESADG, no hardware experiment, and no such results. The body's conclusion (Section V) claims only 'up to 20%' improvement in 'realistic simulations'. This is a load-bearing mismatch: the advertised contribution is absent from the manuscript. The authors must either include the ESADG and hardware experiments, or revise the abstract to match the REMAP-only simulation content.","section":"Abstract vs. Sections I–V"},{"comment":"The planner-level evaluation of REMAP is performed only on the six-map pool (empty, maze, random, room, den312d, WH-S) that was used to train ExecTimeNet in Section IV-A1. The two out-of-distribution maps, WH-L and boston, appear only in Table I for prediction MAPE, not in the REMAP planner comparisons. The claim in Section V that the model 'generalizes across ... unseen maps' is therefore supported only at the prediction level, not at the task level. The headline 20% improvement could be an in-distribution artifact. This is fixable: report REMAP penalty-gap results on WH-L and boston (or explicitly scope the planner claim to in-distribution maps).","section":"Section IV-B vs. Section V"},{"comment":"The deadline regime is selected post hoc: Appendx F describes a grid search over K_D (8 to 16, step 2) and chooses the value that gives roughly 50% of agents meeting their deadlines on each map/agent-count. This makes the comparison regime-dependent, and it is unclear how sensitive the reported gaps are to this choice. The authors should justify that 50% pass rate is a principled operating point a priori and, at minimum, report results for neighboring K_D values. Without this, the headline improvement may be concentrated in a deliberately selected easy/hard regime.","section":"Section IV-B1 and Appendix F"},{"comment":"The empirical pipeline is closed entirely inside the SMART simulator: ExecTimeNet is trained on SMART labels, the planner optimizes against ExecTimeNet's surrogate of SMART, and the final evaluation re-simulates in SMART. This is not circular by equation, but it leaves the central premise—that optimizing against a learned model of SMART transfers to physical robots—unsupported. The abstract promises 'effective transfer from simulation to real-world deployment' with a 15.3% hardware figure, but no real-robot experiment appears. Either add hardware validation or explicitly restrict the paper's claims to the SMART simulator and remove the hardware/transfer language.","section":"Section IV-A1 and Section IV-B"}],"minor_comments":[{"comment":"Definition 2 contains a typo: 'communication delays).' has an unbalanced parenthesis. Also, the end of the definition should be punctuated consistently.","section":"Section II-A"},{"comment":"The normalization with VBS is described as 'results are normalized using a Virtual Best Solver ... divided by the number of agents', but the exact formula for the 'penalty gap' is not stated. Please write the normalization explicitly to allow reproduction.","section":"Section IV-B1"},{"comment":"No error bars or confidence intervals are shown for the penalty-gap curves. Given only 25 instances per map and intrinsic SMART variability, reporting standard errors or confidence bands, at least for the headline comparisons, would strengthen the presentation.","section":"Figures 3–5"},{"comment":"The document's internal title is 'Bridging Planning and Execution: Multi-Agent Path Finding Under Real-World Deadlines', which differs from the arXiv title 'From Discrete Plans to Real-World Execution: A World-Model-Driven Framework for Execution-Aware Multi-Agent Path Finding'. The authors should align these and ensure the abstract matches the body.","section":"Title and front matter"},{"comment":"The distributional head is described as modeling a Gaussian in log-space, but the main text (Section III-B2) calls it 'log-normal'. Please reconcile the terminology—this is the same model, but the presentation is confusing.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"As submitted, the artifact appears to be two different papers: the arXiv abstract promises ESADG and a physical-robot evaluation (15.3%), while the full text is a REMAP-only simulation study. If the hardware/ESADG results exist and are omitted, they must be integrated; if not, the abstract must be corrected. The planner-level out-of-distribution gap is the most substantive technical issue and is the main reason I cannot recommend acceptance at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the arXiv abstract promises ESADG, hardware experiments, 40% improvement, and 15.3% physical-robot gains; none of that appears in the full text, which is a different paper: REMAP/ExecTimeNet only, up to 20%, simulation only. That is a serious mismatch and the deciding issue for me. Second, the body's 20% claim is real but narrow: REMAP beats three constant-speed baselines in SMART on the same six maps used to train ExecTimeNet. The stress-test note is right: planner-level out-of-distribution transfer is never tested. Table I shows the predictor transfers to WH-L and Boston, but the planner experiments never include those maps, so we don't know if the search-guidance advantage survives new topologies.\n\nWhat's new: encoding MAPF plans as ADGs with action, structural, and edge features, then using a Transformer/GATv2 execution-time predictor as a search objective inside CBS and MAPF-LNS. I don't know of prior work doing that. The predictor itself is well-designed and the ablation in Table II is informative. The paper is also honest where it counts: Appendix E concedes CBS is non-optimal for MAPF-RD with a concrete example, Fig. 4 controls for iteration count, and Section IV-B2 openly blames inference overhead for degradation at high agent counts.\n\nSoft spots beyond the abstract mismatch. The evaluation loop is entirely internal to SMART — training labels, planner objective, and evaluation all come from the same simulator. That is not circular by equation, but it means the sim-to-real claim is unsubstantiated. There are no error bars on the headline penalty-gap curves, and the deadline scaling factor K_D is grid-searched to produce the 'about 50% pass' regime, which is post hoc tuning of the evaluation itself. The fixed-iteration control mitigates the runtime concern but does not touch generalization.\n\nWho this is for: researchers working on execution-aware MAPF, deadline-aware planning, or learned surrogate models for multi-agent systems. The core idea is worth engaging with. I would send it to peer review, but I would require the author to fix the abstract/body mismatch, release the SMART data and code, add a planner-level OOD experiment, and ideally run at least one independent execution benchmark. The body is a solid simulation study; the submission as a whole overclaims.","headline":"The abstract and full text describe different papers, and the body's headline 20% gain is only tested on training maps; the core idea is still worth referee time.","tokens_in":16323,"tokens_out":1778,"would_cite":false,"duration_ms":16297,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that using a learned model of real-world execution time to guide multi-agent path planning cuts deadline violations by up to 20% compared to constant-speed baselines.","keywords":["multi-agent path finding","execution time prediction","action dependency graph","deadline-aware planning","graph neural networks","world model","warehouse robotics","MAPF-RD"],"falsifier":"Run the same REMAP evaluation on physical robots with the same maps, agent counts, and deadlines, and compare realized penalties against the constant-speed baselines; if the improvement over baselines disappears or reverses, the claim that learned execution-time prediction transfers to reality is falsified.","tokens_in":15197,"feed_emoji":"⏱️","tokens_out":5780,"duration_ms":54122,"temperature":0.7,"pith_summary":"The paper tries to close the gap between discrete multi-agent path finding (MAPF) and physical execution. It introduces a neural network, ExecTimeNet, that takes the dependency graph of a candidate plan and predicts each agent's wall-clock completion time. These predictions are fed back into search-based planners so they can optimize for real-world deadlines rather than abstract path length. In the paper's simulations with up to 300 agents, this execution-aware search improves solution quality by up to 20% over baselines that assume a constant execution speed. The significance is that a single learned model, trained once, generalizes to unseen maps and agent counts, making deadline-aware warehouse-scale planning feasible.","feed_headline":"Learned execution times cut robot deadline misses by up to 20%","feed_subtitle":"A graph-based world model predicts when each robot actually arrives, guiding planners toward plans that meet wall-clock deadlines.","key_machinery":"The Action Dependency Graph (ADG) encoding, augmented with node and edge features, is the central object. It turns a discrete MAPF plan into a graph whose nodes are actions and whose edges are intra-agent sequencing and inter-agent passing constraints. ExecTimeNet—a Transformer plus graph-attention model—processes this graph to predict per-agent wall-clock execution times, capturing how delays propagate across agents. The ADG-based representation is map-agnostic, which is what allows the model to transfer to unseen maps and agent counts.","core_discovery":"The central claim is that execution time in MAPF is predictable from the structure of the plan itself. Given a plan, the Action Dependency Graph encodes both the order of each agent's actions and the passing order of agents at shared locations. ExecTimeNet reads this graph and outputs either a point estimate or a distribution of each agent's completion time. REMAP uses these estimates to compute deadline penalties and guides the search of two leading search-based planners, producing plans that, when re-simulated, incur up to 20% lower penalty than plans chosen by constant-speed execution estimates.","pith_inferences":["The paper's evaluation uses the same simulator for both training and testing; a fair transfer test would require a different simulator or physical robots to confirm the 20% improvement persists.","The paper's abstract describes a post-planning procedure and physical-hardware results that are not present in the full text; the body's evidence is simulation-only.","The deadline protocol is tuned so that roughly half the agents pass; a natural extension is to evaluate under tight-deadline regimes where the planning-execution gap matters most.","Because the ADG encoding is map-agnostic but trained on plans from one planner family, one could test whether the model retains accuracy when plans come from a different planner or with different robot dynamics."],"forward_implications":["Deadline-aware planning can be added to existing search-based MAPF solvers with modest changes, without redesigning the planner.","A single trained model can serve fleets of varying sizes and unseen warehouse layouts, removing the need to re-calibrate per map.","Distributional predictions allow planners to hedge against execution uncertainty, not just optimize the expected time.","The runtime cost of ADG construction and inference is small enough to support many search refinement iterations within a minute budget.","The approach extends beyond deadline satisfaction to any time-sensitive multi-agent scheduling problem where execution times are hard to model analytically."],"fun_headline_variants":["World model predicts robot timing, slashing delays by 20%","Execution-aware planning cuts robot delays by up to 20%","Neural world model forecasts robot arrival times in MAPF","Predicting real robot execution times improves plans by 20%","From plans to reality: AI predicts robot completion times"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The results depend on the simulator used for both training and evaluation faithfully reproducing the physical execution of robots; if that simulator's speed, acceleration, and controller models diverge from real hardware, the predicted improvements may not transfer to deployed fleets.","fun_headline_variants_meta":{"raw":{"variants":["World model predicts robot timing, slashing delays by 20%","Execution-aware planning cuts robot delays by up to 20%","Neural world model forecasts robot arrival times in MAPF","Predicting real robot execution times improves plans by 20%","From plans to reality: AI predicts robot completion times"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000448,"raw_usage":{"total_tokens":2138,"prompt_tokens":826,"completion_tokens":1312,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":1228}},"tokens_in":570,"tokens_out":1312,"duration_ms":10548,"temperature":1.0,"reasoning_tokens":1228,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T19:55:59.352959+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same REMAP evaluation on physical robots with the same maps, agent counts, and deadlines, and compare realized penalties against the constant-speed baselines; if the improvement over baselines disappears or reverses, the claim that learned execution-time prediction transfers to reality is falsified.","supporting_citations":[],"review_version":1}