{"id":"d2a46f0d-7f6f-47a5-969d-c568b9d13d9e","arxiv_id":"2607.05683","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Average-reward independent PPO for multi-block AMR fleets jointly learns charging station choice, charge duration, and depot returns, improving order completion by up to ~6 pp over the best baselines in simulation.","lead":"A multi-agent PPO system learns when warehouse robots should charge, which station to use, and how long to stay, under random order arrivals. In simulated multi-block warehouses it raises order completion by up to about six percentage points over strong fixed-threshold and prior DRL baselines while cutting idle charging and queue time.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.5","headline":"Headline 6% gain rests on single-run tables without multi-seed variance or significance tests, so the superiority claim is not yet statistically secured.","rationale":"The reader’s weakest-assumption note (collision-free, constant η, b_min reachability) is a genuine sim-to-real limitation and correctly keeps the verdict CONDITIONAL. However, that limitation is common to all methods evaluated inside the paper’s own MDP (Eqs. 1–3, §3.3); it does not differentially threaten the relative 6-point claim. The more immediate load-bearing gap for the strongest claim is the absence of multi-seed statistics around Tables 8–9. Because the paper already ships code and a fully specified simulator, the missing check is cheap and decisive. I therefore leave the verdict at CONDITIONAL (same as the reader) but re-center the primary condition on statistical reliability of the reported gains rather than solely on the shared modeling assumptions. Agreement is partial: both identify real caveats that prevent unconditional acceptance, yet the single most load-bearing concern for the numerical headline is different.","tokens_in":23048,"tokens_out":617,"duration_ms":6284,"concrete_test":"Retrain IPPO and the two strongest baselines (Fixed 100_15 / Fixed 100_20 and the adapted Bischoff PPO) for ≥5 independent random seeds on E-I µ=0.6 and E-II µ=0.9; report mean ± std of overall completion rate and a paired t-test (or bootstrap CI) on the IPPO–baseline difference. If the mean gap falls below ~3 points or the 95% CI includes zero, the “up to 6%” headline weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (Abstract; Tables 8–9; Fig. 7) is that average-reward IPPO raises order-completion rates by up to 6 percentage points versus the strongest baseline (FixedThreshold / Bischoff) while cutting charge/wait time. All reported numbers are point estimates from one training trajectory and one evaluation horizon per (environment, µ) cell; no multi-seed means, standard deviations, or hypothesis tests appear. In multi-agent PPO the policy is known to be sensitive to random seeds, entropy schedules, and early exploration. Without that variance, it is impossible to know whether the 6-point gap (e.g., 76 vs 70 in E-I µ=0.6, or 60 vs 55 in E-II µ=0.9) is reliable or an artifact of a favorable seed. The reader correctly flags the collision-free / constant-η assumptions as a transfer risk, but those assumptions are shared by every baseline inside the same simulator; they therefore do not explain the relative ranking. The ranking itself is the load-bearing empirical claim, and it is currently unsecured by statistical evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.5","summary":"The paper formulates dynamic battery management for multi-block warehouse AMRs under Poisson order arrivals as a multi-agent MDP and solves it with average-reward Independent PPO (shared actor, per-agent critics, differential GAE, action masks). Agents jointly learn when to charge, which fixed station to use, when to stop charging, and when to return to the depot. Experiments on 4- and 6-block layouts compare IPPO to an adapted Bischoff masked-PPO baseline, DQN, CTDE PPO, FixedThreshold, and HighLow heuristics. The central empirical claim is up to ~6 percentage-point higher order-completion rates versus the strongest baseline, with lower charge and queue time; SHAP and mean-state analyses are used to interpret the learned policy, and sensitivity checks cover train/test horizons and time-varying arrivals.","tokens_in":23316,"tokens_out":1114,"duration_ms":22363,"significance":"Battery coordination under shared fixed chargers is a practically important and still under-modeled piece of warehouse AMR control. Strengths include a carefully specified MDP with feasibility masks (Section 4), a well-motivated average-reward objective with differential GAE (Section 5.1–5.3; Figure 2), a transparent IPPO training loop (Algorithm 1), public code, multi-baseline comparisons including a literature DRL adaptation, and useful SHAP/operational-rule interpretation (Section 6.3). If the reported gains hold under proper statistical validation and the stated modeling assumptions, the work is a solid applied contribution to DRL for warehouse logistics and offers transferable decision heuristics for practitioners.","major_comments":[{"comment":"Abstract and Tables 8–9 / Figure 7 report order-completion gains of up to ~6 points (e.g., 76 vs 70 in E-I µ=0.6; 60 vs 55 in E-II µ=0.9) as single point estimates from one training trajectory and one evaluation horizon per cell. No multi-seed means, standard deviations, or significance tests are provided. Multi-agent PPO is known to be seed- and entropy-schedule sensitive; without that evidence the headline superiority claim is not statistically secured. Please re-run evaluation (and ideally training) over multiple seeds and report mean±std (or confidence intervals) for completion rate, charge time, and wait time.","section":null},{"comment":"Section 3.3 and Eqs. (1)–(3) assume collision-free motion and payload-independent constant depletion η, with b_min always sufficient to reach a station. These assumptions are shared by all in-simulator baselines, so they do not invalidate relative rankings inside the study, but they are load-bearing for any claim of operational transfer. The paper should either (i) stress-test relative ranking under mild collision costs or load-dependent drain, or (ii) clearly bound the claim to the idealized simulator and strengthen the limitations/future-work discussion so the Abstract does not over-generalize.","section":null},{"comment":"Hyperparameters (Table 7b) are tuned via Optuna on E-I only (Figure 3) and then reused for E-II and dynamic-arrival tests. Given that the strongest baseline sometimes changes with layout and µ, please either retune (or at least ablate) key PPO/GAE settings on E-II, or show that the ranking is stable under a small hyperparameter perturbation so the cross-environment robustness claim is not confounded by E-I-specific tuning.","section":null}],"minor_comments":[{"comment":"Table 7(a) lists “γ” with values 0.50/0.60 and 0.75/0.90; the text and figures use these as order-arrival rates µ, and the method is average-reward (no discount). Fix the symbol so γ is not confused with the discarded discount factor.","section":null},{"comment":"Abstract says the model learns “two key decisions” (station and duration), while the contributions and action space also include when to charge and when to go to the depot. Align the Abstract wording with Section 1 contributions.","section":null},{"comment":"Figures 5 and 7–9 would benefit from error bars (once multi-seed runs exist) and consistent axis labels; several panels show blank “=” where µ should appear (extraction/typesetting issue).","section":null},{"comment":"Clarify in Section 6.2 exactly which Bischoff reward/interrupt/A_full settings were selected as “best” and whether any re-tuning of that baseline’s thresholds was allowed, so the comparison is fully reproducible.","section":null},{"comment":"Minor prose: “deport” appears once for “depot” in Section 6.3; check consistency of CS vs charging station notation.","section":null}],"recommendation":"major_revision","confidential_remarks":"Methodologically careful applied MARL paper with code and interpretability; main barrier to acceptance is the single-seed reporting of the quantitative headline claim, which is fixable. Scope fits OR/warehouse logistics and applied RL venues. No integrity concerns from the manuscript alone."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is a carefully specified average-reward IPPO controller that jointly learns station choice, when to stop charging, and depot return under shared queues and Poisson arrivals in multi-block layouts. That joint action space plus the explicit queue modeling is the real increment over Mu et al., Bischoff et al., and the usual fixed-threshold heuristics.\n\nWhat they do well: the MDP (action masks, differential GAE, shared actor + independent critics) is written cleanly in Sections 4–5; they adapt Bischoff’s masked PPO fairly rather than straw-manning it; they add dynamic-arrival and train/test-horizon checks; and the SHAP / mean-feature policy analysis actually produces usable operational rules (charge near 25%, stop early when neighbors are low, depot only when capacity is exhausted). Code is linked. Relative ranking versus FixedThreshold, HighLow, DQN, and CTDE-PPO is consistent across E-I/E-II and two µ values, and charge/wait time drops with the completion gains.\n\nSoft spots, in proportion: the headline “up to 6%” (Tables 8–9, Fig. 7) is single-trajectory point estimates. No multi-seed means, SDs, or tests. In multi-agent PPO that is a real gap; the 76-vs-70 or 60-vs-55 gaps could shrink under seed variation. The collision-free / constant-η / b_min-always-reaches assumptions are load-bearing for transfer, but they are shared by every baseline inside the same simulator, so they do not explain the ranking—only the absolute numbers. Novelty is solid application, not a foundational RL advance. Reward shaping is simple and works; free parameters are the usual PPO set plus the entropy decay horizon.\n\nThis is for people who run warehouse AMR fleets or who build multi-agent RL for logistics. A serious editor should send it to referees; the formulation and empirical design are already referee-ready. I would cite the joint formulation and the policy interpretation if I were writing on AMR energy management. Tighten the statistics and the sim-to-real caveats and it is a clean contribution.","headline":"Solid applied multi-agent RL for joint charging/depot decisions in multi-block warehouses; the ~6% completion lift is directionally supported but rests on single-run point estimates without seed variance.","tokens_in":24001,"tokens_out":545,"would_cite":true,"duration_ms":6473,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A multi-agent PPO policy that jointly decides when, where, and how long warehouse robots charge raises order-completion rates by up to 6% over the best fixed-rule and DRL baselines while cutting recharging time.","keywords":["warehouse logistics","deep reinforcement learning","battery charging","autonomous mobile robots","proximal policy optimization","multi-agent systems","average-reward RL","order picking"],"falsifier":"Re-run the identical multi-block layouts with payload-dependent drain and a simple collision or path-conflict model; if the 6% completion gap over Fixed-100-15 / Fixed-100-20 disappears or reverses, the central claim fails under realistic kinematics.","tokens_in":23861,"feed_emoji":"🔋","tokens_out":982,"duration_ms":10073,"temperature":0.7,"pith_summary":"Warehouse robots keep stopping work to recharge, and simple battery thresholds plus nearest-station rules leave stations congested and orders unfinished when arrivals are random. This paper shows that a shared-actor, independent-critic PPO controller can learn, from local observations of battery, queues, neighbors, and distances, four linked decisions at once: whether to charge now, which fixed station to join, when to stop charging short of full, and when to return to the depot. The controller is trained with average reward rather than discounted return so every future order counts equally. In four- and six-block simulated warehouses the learned policy finishes up to 6% more orders than the strongest fixed-threshold, high-low, DQN, CTDE-PPO, and prior masked-PPO baselines, while spending less total time charging and waiting. The same policy remains superior under piecewise-constant real arrival patterns and under longer test horizons, and SHAP plus mean-state analysis recover transparent operating rules that managers can adopt even without running the network.","feed_headline":"Robot chargers learn to wait less, finish 6% more orders","feed_subtitle":"Average-reward multi-agent PPO beats fixed battery rules and prior DRL on stochastic warehouse workloads","key_machinery":"Average-reward Independent PPO (shared actor + per-agent local critics) with differential GAE, action masks, and a sparse reward of +20 per pick start minus 1 per time-step; the agent observes its own battery/capacity/distances plus neighbors’ batteries and station queues so the policy can coordinate without a joint critic.","core_discovery":"When multi-block AMRs with shared fixed chargers must serve stochastic Poisson orders, an average-reward Independent PPO policy that jointly chooses station, charging duration (including early stop), and depot return raises order-completion rates by up to 6% over the strongest fixed-threshold and DRL baselines while simultaneously lowering charging and queue time per completed order.","pith_inferences":["The same observation-and-mask design should transfer to other shared-resource fleets (AGVs, drones, last-mile robots) whose downtime is dominated by queuing rather than pure travel.","Because the policy already conditions on neighbor battery, adding a light communication channel for planned charge times could further cut residual queue waste without enlarging the action space.","If load-dependent drain is introduced later, the critic’s value estimates will shift mainly through the battery feature; retraining only the later layers may be sufficient.","Positioning chargers nearer the farthest blocks is a low-cost layout change that the SHAP analysis already flags as high-leverage."],"forward_implications":["Fixed lower/upper battery thresholds and nearest-station rules are systematically suboptimal once queues and stochastic arrivals matter.","Allowing early termination of charging when a neighbor’s battery is low measurably reduces station congestion and raises fleet throughput.","Depot trips should be deferred until capacity is nearly exhausted; the learned policy recovers this batching rule automatically.","Average-reward training (no discount factor) is necessary for long-horizon order-completion objectives; discounted PPO plateaus lower.","The extracted decision rules (charge near 25%, prefer nearest free station, stop early for low-battery neighbors) can be hard-coded as improved heuristics even without online DRL."],"fun_headline_variants":["PPO robots cut charge waits, clear 6% more warehouse orders","Average-reward DRL lifts AMR completions 6% vs fixed rules","Multi-block AMRs learn station and duration for higher throughput","Stochastic order pickers slash queue time with Independent PPO","DRL battery policy finishes more orders while charging less"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"Robots never collide, battery drains only while moving and at a constant rate independent of load, and the minimum battery threshold is always enough to reach the nearest charger from anywhere in the block.","fun_headline_variants_meta":{"raw":{"variants":["PPO robots cut charge waits, clear 6% more warehouse orders","Average-reward DRL lifts AMR completions 6% vs fixed rules","Multi-block AMRs learn station and duration for higher throughput","Stochastic order pickers slash queue time with Independent PPO","DRL battery policy finishes more orders while charging less"]},"model":"grok-4.5","effort":"low","cost_usd":0.00398,"raw_usage":{"total_tokens":1200,"prompt_tokens":757,"num_sources_used":0,"completion_tokens":89,"cost_in_usd_ticks":39800000,"prompt_tokens_details":{"text_tokens":757,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":354,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":757,"tokens_out":89,"duration_ms":3371,"temperature":1.0,"reasoning_tokens":354,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T03:49:39.363925+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Re-run the identical multi-block layouts with payload-dependent drain and a simple collision or path-conflict model; if the 6% completion gap over Fixed-100-15 / Fixed-100-20 disappears or reverses, the central claim fails under realistic kinematics.","supporting_citations":[],"review_version":1}