{"id":"209afa85-a1ef-4b90-a66a-0adaf305a15e","arxiv_id":"2608.00337","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A lookahead scheduler that simulates each robot's action-queue state before choosing batches improves throughput in heterogeneous multi-robot policy serving by up to 18% in real-world tests.","lead":"Robot foundation models are too heavy to run on every robot, so this paper serves many robots from one shared GPU and decides which robots get the next batch of actions. It introduces a 'lookahead' scheduler that plans batches using a model of each robot's remaining action buffer, and reports up to 18% higher task throughput on a real 10-robot fleet when robots have different responsiveness needs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 18% real-world gain may stem from Lookahead's chunk start-index bias interacting with naive async execution, a confound the paper identifies in Appendix A.8 but does not control for.","rationale":"The paper's own Appendix A.8 identifies the chunk start-index bias as a likely confound, and Figure 10 shows a clear difference between Lookahead and the baselines. This is the most load-bearing threat because it directly challenges the causal interpretation of the headline throughput improvement: if the gain comes from reducing jerky motions via index-0 starts, then the 'closed-loop scheduling' mechanism is not what drives the result, and the contribution's novelty (MDP-based scheduling) is not supported. The reader's weakest assumption matches this exactly, and I agree with the CONDITIONAL verdict. I considered whether the lack of statistical rigor (3 seeds, 1-minute rollouts, no error bars) is more load-bearing; but that concern affects confidence in the existence of the effect, whereas the confound affects attribution of the effect to the claimed mechanism. The confound is more specific and the paper itself raises it, so it is the appropriate focus. The proposed test—switching to RTC or VLASH—directly isolates the confound and is feasible given the existing testbed. If the gain persists under RTC/VLASH, the central claim survives; if it disappears, the claim needs substantial qualification. Thus the verdict remains CONDITIONAL rather than ACCEPT, and no change to the reader's verdict is needed.","tokens_in":16721,"tokens_out":3286,"duration_ms":31768,"concrete_test":"Re-run the one-fast and half-fast real-world experiments (10 PiPER arms, same tasks, same schedulers, same batch-size caps) with a real-time chunking strategy such as RTC or VLASH instead of naive async execution, while recording first-executed-index distributions. If Lookahead's total-throughput advantage over EDF/RR falls below the reported 18% (ideally to within noise), the confound is the cause and the central claim must be restricted to naive-async deployments.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central real-world claim (up to 18% throughput gain in heterogeneous settings) is asserted to arise from a closed-loop MDP scheduling mechanism that accounts for robot heterogeneity. However, Appendix A.8 and Figure 10 document a confound: Lookahead dispatches chunks that start execution at index 0 far more often than RR or EDF, which often start mid-chunk. With naive async execution, mid-chunk starts produce jerky/discontinuous motions that reduce task success. Thus Lookahead's measured throughput advantage could be due to this execution-pattern side effect rather than to smarter batch selection based on starvation prediction. The paper explicitly says 'Predicting chunks with more sophisticated asynchronous strategies such as RTC [8] or VLASH [9] may lead to different results' (Appendix A.8), admitting the effect may not transfer. This concern is amplified by simulation results (Appendix A.7): in simulation, Lookahead does not improve total system throughput, only reallocates it across tiers. The real-world 18% gain appears only in the one-fast configuration (98.67 vs 83.33 successes/min), based on 3 one-minute rollouts with no error bars. Without controlling for the chunk-start-index confound, the real-world numbers cannot be attributed to the MDP scheduling mechanism, and the generality of the claim is unestablished.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces Armory, a serving system for batched inference of robot policies (e.g., VLA models) on a single remote GPU shared by multiple robots. The authors formulate action-chunk scheduling as an MDP over server-side robot state (action indices, queues, latency estimates) and propose Lookahead, a one-step receding-horizon scheduler that scores candidate batches by predicted total executed robot time normalized by inference latency. They evaluate Lookahead against Round Robin and EDF in simulation (LIBERO) and on ten real AgileX PiPER arms with a π0.5 policy. The main empirical claims are that simple schedulers suffice for homogeneous fleets, while Lookahead with tuned per-robot weights improves fast-robot service and, in the one-fast real-world configuration, increases system throughput by up to 18% over baselines.","tokens_in":17035,"tokens_out":4776,"duration_ms":47783,"significance":"If the empirical claims hold, the paper opens a useful new problem area: multi-robot closed-loop policy serving, where inference latency changes robot behavior rather than being a pure SLO. The MDP formulation and the software-mirror abstraction are coherent, and the paper provides detailed pseudocode, extensive ablations (batch-size cap, lookahead depth, network delay/jitter), and a real-robot testbed with two task types. The claimed 18% throughput gain in heterogeneous settings is the paper's headline contribution and would be of interest to the robotics-systems community. However, the real-world evidence currently does not establish the causal mechanism behind that gain, and the simulation results do not reproduce it, so the significance is contingent on additional experiments.","major_comments":[{"comment":"The headline 18% throughput gain is not attributable to the MDP/lookahead mechanism because the paper itself identifies an uncontrolled confound: Lookahead dispatches a large fraction of chunks starting at index 0, while RR and EDF frequently start mid-chunk; with naive async execution, mid-chunk starts cause jerky/discontinuous motions that reduce task success. Figure 10 shows a strong first-executed-index bias, and the text states that using RTC or VLASH 'may lead to different results.' This means the measured gain could be an execution-pattern side effect rather than the result of smarter starvation-aware batch selection. The authors should either control the first-executed-index distribution across schedulers, repeat with RTC/VLASH, or otherwise provide evidence that the scheduling decision drives the throughput difference.","section":"§4.2 / Appendix A.8, Table 5, Figure 10"},{"comment":"The real-world throughput result rests on three one-minute rollouts per condition, with no error bars, confidence intervals, or significance tests. The appendix explicitly notes that 'metrics may be noisy due to small sample size' and that real-world evaluations are noisy. For the key one-fast comparison (LA@5 total 98.67 vs RR 83.33), the reader cannot tell whether the 18% difference is signal or seed variation. The paper should report per-seed results, variance, and at least a simple statistical comparison, or temper the throughput claim accordingly.","section":"§4.1 / Table 5 / A.8"},{"comment":"The simulation results contradict the claimed real-world transfer. Table 4 and Figure 9 show that Lookahead does not improve aggregate system throughput in simulation; Appendix A.7 explicitly states that 'the core contribution of Lookahead is not in raising aggregate throughput, but in reallocating it across tiers.' Yet Section 4.2 says 'Scheduling gains transfer from simulation to real hardware' and the abstract claims a throughput improvement. This inconsistency undercuts the claim that the lookahead mechanism itself improves total throughput. The authors should either present simulation evidence of aggregate gains or revise the transfer claim to reflect the real evidence.","section":"§4.2 vs Appendix A.7, Table 4"},{"comment":"The choice of batch-size cap may favor Lookahead independently of its scheduling logic. In the heterogeneous real-world experiments b=5, while Appendix A.4 shows that Lookahead chooses smaller batches on average (mean 3.3) whereas EDF and RR gravitate toward the cap. Since the paper notes that over-batching causes starvation, the advantage could come from Lookahead's dynamic batch-size selection rather than from its forward simulation of execution state. The claim that b=5 'favors RR and EDF' is not backed by the reported real-world sweep. Please show the batch-size tuning for the real-world baselines or run an ablation that restricts Lookahead to the same batch-size choices as the baselines.","section":"§4.1 / Appendix A.4"}],"minor_comments":[{"comment":"The term 'naive async' is used in the appendix and in the confound discussion but is not formally defined in the main text. Please define it early (e.g., in Section 2 or 3) so the reader understands the execution model that the confounding discussion relies on.","section":"§2 / §A.8"},{"comment":"The score in Eq. (13) normalizes by d_infer only, not by the full epoch time including communication delays. This is reasonable for a heuristic, but the choice should be justified, especially since Section 3.1's transition includes d_obs and d_action.","section":"§3.2, Eq. (13)"},{"comment":"The caption says 'legos per minute' while the text uses 'successes per minute' and 'Lego pieces per minute.' Use consistent units and clarify what 'success' means for the real-world tasks.","section":"Figure 4 caption"},{"comment":"Reference [12] is a personal blog post. For a systems claim about execution index continuity, a peer-reviewed or archival source would be more appropriate if one exists.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is clearly written and the systems work appears solid, but the central empirical claim is currently undercut by the authors' own appendix: the admitted chunk-start-index confound and the small sample size make the 18% number insufficiently supported. The simulation results further weaken the throughput claim. I believe the issues are addressable with additional experiments or a substantially softened claim, hence major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Start with the punchline: this paper gives you a genuinely new way to think about batched robot policy serving—an MDP over action queues, starvation, and task-dependent horizons—and it builds a real system around that model. But the headline claim, up to 18% real-world throughput gain, is not cleanly attributable to the scheduling policy. The authors themselves show in Appendix A.8 that Lookahead biases chunks to start at index 0, which interacts with naive asynchronous execution to reduce jerky motions, and they admit that with RTC or VLASH the results may differ. That's a self-identified confound, and it's the right one.\n\nWhat's actually new: the MDP formulation is a real step beyond Kairos and ROSA, which treat latency as a target and don't model the closed-loop effect of a late chunk. The reward weights w_j give operators a controllable tradeoff between fast and slow robots. The software mirror, with push-based communication and latency profiling, is sensible engineering, and the simulation ablations (batch-size cap, network jitter, lookahead depth) are thorough. No circularity: the scheduler and reward model are proposed mechanisms, not fitted to the measured throughput.\n\nSoft spots, in order. First, the real-world evidence for total throughput gains is thin: three 1-minute rollouts per scheduler, no error bars, and the gain appears only in the one-fast configuration. Second, the Appendix A.8 confound is acknowledged but not controlled; without an alternative async strategy or a direct measure of motion smoothness, you cannot attribute the 18% to the scheduling logic. The authors deserve credit for flagging it, but the abstract's claim is stronger than the evidence. Third, in simulation Lookahead does not increase total throughput; it reallocates it across tiers. So the contribution is better framed as a control surface for trading service between robot classes, not as a universal throughput win. The Limitations section is honest about the single-GPU batching assumption—scope, not a flaw.\n\nWho this is for: systems people working on multi-robot or cloud VLA serving, and robot learning folks who care about deployment. The MDP formulation and the scheduler code (assuming release) are worth building on.\n\nWould I send this to peer review? Yes. The core formulation is solid and the authors are transparent about the main caveat. A revision that either controls for the async confound or downscopes the abstract's claim, plus basic statistics on the real-world runs, would make it a strong paper. That's a referee conversation, not a desk reject.","headline":"Solid new formulation, honest system work, but the headline real-world gain is likely a naive-async artifact the authors themselves flag.","tokens_in":17580,"tokens_out":3305,"would_cite":true,"duration_ms":28944,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Batched robot policy serving should be a closed-loop scheduling problem, and a lookahead scheduler recovers up to 18% throughput on heterogeneous fleets.","keywords":["robot policy serving","action chunking","cloud inference","batching","scheduling","closed-loop control","heterogeneous fleets","starvation"],"falsifier":"Take the heterogeneous real-world setting and force all schedulers to execute only from chunk start—or use a chunking scheme without the index-0 bias—then compare throughput; if the Lookahead advantage disappears or shrinks to noise, the scheduling policy was not the driver.","tokens_in":16554,"feed_emoji":"🤖","tokens_out":4953,"duration_ms":43778,"temperature":0.7,"pith_summary":"The paper's claim is that when one GPU serves many robots, batching decisions should be made by simulating how each batch changes each robot's future action buffer, not by generic heuristics. The authors formulate batched policy serving as a Markov Decision Process whose state mirrors each robot's action queue, executed index, and network delay, and whose reward is total robot time executed. They propose Lookahead, which rolls candidate batches forward and picks the one with the best reward per unit inference time. In simulation and on ten real robots, simple round-robin and deadline scheduling match Lookahead when fleets are homogeneous, but under heterogeneous reactivity demands Lookahead raises real-world system throughput by up to 18% and exposes a tunable weight for trading off fast and slow robots. The paper matters because it reframes multi-robot policy deployment as a control-aware systems problem rather than a pure throughput problem.","feed_headline":"Lookahead scheduling recovers 18% throughput for batched robot fleets","feed_subtitle":"When robots drain action chunks at different rates, planning batches against their buffers beats round-robin and deadline heuristics.","key_machinery":"The central object is the server-side execution mirror: for each robot, the action index of the latest executed step, the latest step received, and the queue of generated chunks with arrival times—where an action chunk is a sequence of future actions produced by one policy query. Scheduling is an MDP over these states; a batch transition advances robots' indices only when a covering chunk has arrived, and the per-epoch reward is total executed robot time, optionally weighted per task. Lookahead is a one-step receding-horizon algorithm (the paper finds deeper lookahead does not improve results) that scores candidate batches by predicted reward divided by inference time and dispatches the best","core_discovery":"The paper's central discovery is that batching decisions in a shared GPU policy server alter robot behavior in closed loop: a late chunk starves a robot, and the harm depends on the robot's task. The authors build Armory, which maintains a server-side mirror of each robot's action queue, latest executed index, in-flight chunks, and communication delay, and cast scheduling as an MDP whose reward is total executed robot time. Their Lookahead scheduler simulates candidate batches forward and dispatches the first batch of the highest-scoring schedule, normalizing reward by inference time. Evaluations on a simulation benchmark and a ten-robot real fleet show that when all robots consume chunks at","pith_inferences":["The paper's own appendix flags that Lookahead tends to start chunks at executed index 0, avoiding the jerky mid-chunk starts that naive asynchronous execution produces; if that execution-pattern effect rather than batch selection drives the measured gains, the advantage may vanish under real-time chunking strategies that eliminate the bias.","A natural next step is closed-loop weight adaptation: use task-phase or progress signals to raise a robot's weight during dynamic segments and lower it when quasi-static, which the MDP formulation already supports.","The same mirror-based formulation could extend to multi-GPU or disaggregated serving by treating prefill/decode or model shards as scheduling resources, though the paper only assumes a single GPU.","Because starvation cost is task-dependent, scheduler weights could be learned from policy-success statistics instead of being set by hand, making the tradeoff data-driven."],"forward_implications":["Operators can choose a weight per robot class and move along a throughput/starvation tradeoff instead of accepting whatever a fixed heuristic delivers.","In homogeneous fleets the paper's experiments indicate round-robin or earliest-deadline scheduling is sufficient, so the extra machinery is only needed when reactivity demands differ.","Choosing small, efficient batches matters more than filling the batch: Lookahead keeps throughput flat as the batch-size cap grows while fixed-batch heuristics starve robots.","Because Armory is model-agnostic, its scheduling gains compose with any per-model inference speedup.","Network jitter and median delays up to about 50 ms degrade all schedulers similarly; scheduling policy cannot compensate for raw transport delay, but the system remains practical."],"fun_headline_variants":["Lookahead scheduling recovers 18% throughput for robot fleets","When robot action rates differ, lookahead scheduling wins","Batched robot policies: Heterogeneity-aware lookahead gains 18%","Armory's lookahead scheduler beats round-robin for mixed robots","Action chunk serving: Lookahead boosts robot fleet throughput 18%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The real-world 18% throughput gain is assumed to come from Lookahead's better batch selection, but the paper's own appendix shows Lookahead also avoids mid-chunk starts; if that execution smoothness is the true cause, the gain does not validate the scheduling mechanism.","fun_headline_variants_meta":{"raw":{"variants":["Lookahead scheduling recovers 18% throughput for robot fleets","When robot action rates differ, lookahead scheduling wins","Batched robot policies: Heterogeneity-aware lookahead gains 18%","Armory's lookahead scheduler beats round-robin for mixed robots","Action chunk serving: Lookahead boosts robot fleet throughput 18%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000246,"raw_usage":{"total_tokens":1350,"prompt_tokens":695,"completion_tokens":655,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":439,"completion_tokens_details":{"reasoning_tokens":571}},"tokens_in":439,"tokens_out":655,"duration_ms":6495,"temperature":1.0,"reasoning_tokens":571,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T00:40:43.172319+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the heterogeneous real-world setting and force all schedulers to execute only from chunk start—or use a chunking scheme without the index-0 bias—then compare throughput; if the Lookahead advantage disappears or shrinks to noise, the scheduling policy was not the driver.","supporting_citations":[],"review_version":1}