{"id":"7975b8b8-655c-47a5-b1f1-58e34efc98ff","arxiv_id":"2607.20293","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A training-free evolutionary search over global cache schedules cuts diffusion-policy inference FLOPs by up to 8x with near-full rollout success.","lead":"EVO uses evolutionary search to decide which transformer blocks and denoising steps a diffusion policy should recompute, reusing cached activations elsewhere. On manipulation benchmarks it preserves most task success while cutting FLOPs by up to 8x, though measured wall-clock speedup is closer to 3.4x.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Near-full performance may be a selection artifact: the schedule is chosen by a single-seed evolutionary search that stops at the first candidate passing a noisy 50-episode acceptance test, so reported success may not replicate under a different search seed.","rationale":"The reader identified the single-seed search and the 20-episode quick-evaluation acceptance rule as the weakest assumption. My reading of the full text confirms this is the load-bearing issue: the evolutionary search is explicitly goal-directed, terminating at the first schedule that passes a noisy formal-evaluation threshold (Sec. 3.3, Eqs. 8–9; Appendix A.6). Because only one search seed is used per task (Appendix A.1), the reported success is a single draw from a selection process, not a demonstration that the method reliably finds near-full-performance schedules. The speedup-metric mismatch (8.05× FLOPs ratio vs. ~3.4× wall-clock from Appendix A.7) is a real reporting concern but does not threaten the existence of a schedule; the search-seed/selection concern directly threatens the replicability of the main performance claim. A multi-seed search experiment with reported acceptance thresholds would settle whether the concern lands. The reader's CONDITIONAL verdict is appropriate; I would not change it, so I mark UNCHANGED.","tokens_in":20997,"tokens_out":6273,"duration_ms":63659,"concrete_test":"Run the released EVO search from scratch 10 times on one representative task (e.g., Square-MH with M=8), each with a different search seed but the same reported configuration and target; evaluate every accepted schedule on the standard 3×50 test protocol. Report the min/mean/max final success across the 10 search seeds. If the worst accepted schedule falls below q_base − δ_acc, or if the spread exceeds about 0.05 success rate, the near-full performance claim is not robust to search seed. Also release δ_acc, ε, and γ so the acceptance threshold can be audited.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that EVO finds a training-free schedule that preserves near-full policy performance at large computation reduction. The evidence for this rests on a search procedure that stops as soon as a candidate satisfies Eq. (9): q_formal(S) ≥ q_base − δ_acc, after a 20-episode quick screening (Eq. 8). With 50 formal episodes, the standard error of q_formal is roughly sqrt(p(1−p)/50) ≈ 0.07 for p ≈ 0.5, and many candidates are screened over generations. Selecting the first schedule that clears a noisy threshold can therefore yield a schedule whose true success is below the target, especially since δ_acc is not reported. Appendix A.1 states that each task is searched with one search seed, so the paper provides no estimate of search-to-search variability. The final 3×50 evaluation is independent of the search seed, but it evaluates only the one accepted schedule; it does not tell us whether a different search run would also find an acceptable schedule. If the reported near-full performance is partly a consequence of selecting a lucky schedule, the claim that EVO reliably preserves performance is not established. The open-source code is a positive, but the single-seed search and unreleased acceptance hyperparameters (δ_acc, ε, γ) leave this as the main load-bearing uncertainty.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"EVO formulates cache-based acceleration of transformer-based diffusion policies as a fixed-budget subset-selection problem over the block–timestep lattice, and solves it with an evolutionary algorithm that uses closed-loop rollout success as fitness. To reduce search cost, EVO seeds part of the population using activation-dissimilarity priors and stops early once a candidate passes a quick screen and then a formal acceptance test. The paper reports that on RoboMimic, Push-T, Block Push, and Kitchen benchmarks, EVO keeps success rates close to the full policy while reducing FLOPs from 15.77G to as low as 1.96G, corresponding to a nominal 8.05x FLOPs reduction, and it compares favorably to EfficientVLA and BAC. The authors also provide wall-clock measurements in the appendix showing a mean inference-time reduction from 448.91 ms to 132.17 ms.","tokens_in":21297,"tokens_out":4486,"duration_ms":42264,"significance":"The core idea—using black-box evolutionary search to allocate a global cache-refresh budget across blocks and denoising steps—is a reasonable and potentially useful extension of per-block caching methods such as BAC. The paper ships source code, and the ablation in Table 4 gives some evidence that global budget allocation and redundancy-aware initialization each contribute to performance and reduce search cost. The motivation analysis in Fig. 1/Fig. 3 is also a useful empirical observation. If the search reliably produces schedules that preserve near-full policy success at one-eighth of the FLOPs, this would be a practical training-free acceleration result. However, the current evidence is materially weakened by (i) a stopping rule that accepts the first schedule clearing a noisy, target-conditioned threshold with an unreported tolerance, (ii) only one search seed per task, and (iii) a headline 'speedup' that conflates FLOPs reduction with wall-clock time. These issues are fixable but are load-bearing for the central claim.","major_comments":[{"comment":"The abstract and main text describe 'up to 8.05x action-generation speedup', but the measured wall-clock speedup in Table 6 of Appendix A.7 is 448.91/132.17 ≈ 3.40x for EVO (M=8). The value 8.05 is the FLOPs ratio 15.77/1.96, not an action-generation speedup. The Speed× columns in Tables 1–3 appear to be FLOPs ratios, not wall-clock measurements. Please separate FLOPs reduction from wall-clock speedup and adjust the abstract accordingly; the current wording is misleading.","section":"Abstract and Sec. 4.2 / Appendix A.7"},{"comment":"The acceptance rule accepts only schedules with q_formal(S) ≥ q_base − δ_acc, so 'near-full performance' is partly enforced by the selection criterion rather than independently demonstrated. δ_acc is never reported. In addition, Appendix A.1 states that each task is searched with one search seed. With 50-episode formal evaluations (standard error roughly 0.07 for p≈0.5) and first-pass acceptance, the selected schedule may be a lucky draw whose true success is below the target. Please report δ_acc per task, run the search with multiple seeds (e.g., 5–10) per task, and report the distribution of final success rates over search seeds, including q_base and the accepted q_formal value. Without this, the reliability claim is not established.","section":"Sec. 3.3, Eq. (9); Sec. 4.1 and Appendix A.1"},{"comment":"The text states that 'the selected schedule is robust to different initializations', but no experiment varies the search initialization or search seed; every task is searched once. The 'Search Steps to Target' metric is introduced as an average without specifying what it is averaged over. Please either supply multi-seed search results supporting robustness or soften this claim, and clarify the averaging for Search Steps.","section":"Sec. 4.3, Table 4"},{"comment":"The hyperparameters ε and γ in the initialization sampling distribution and δ_acc in the acceptance rule are never specified. These directly affect the search behavior and the acceptance threshold, so the experiments are not reproducible from the text alone. Please provide concrete values (or a sensitivity study if they are not critical).","section":"Sec. 3.3, Eq. (7) and Sec. 4.1"}],"minor_comments":[{"comment":"Typo: 'Specificly' should be 'Specifically'.","section":"Sec. 4.3"},{"comment":"The column 'AVG' is inconsistently formatted (sometimes with a space) and the caption says 'speedups are measured with full DP-T inference', which is ambiguous given the FLOPs-vs-wall-clock issue above.","section":"Tables 1–3"},{"comment":"The metric 'GPU-days' is attributed to reference [2], which is a differentiable architecture search paper; please cite or define the GPU-days convention directly.","section":"Sec. 4.3 / Table 4"},{"comment":"The full schedule tables (Tables 3–5) are very dense. Consider reporting aggregate statistics (e.g., per-block refresh counts, step histograms) in the main paper and moving the complete schedules to a repository, while keeping one representative schedule as an example.","section":"Appendix A.5"}],"recommendation":"major_revision","confidential_remarks":"The paper has a useful idea and the code release is a plus, but the current presentation overstates the result. The '8.05x speedup' is a FLOPs ratio, not the measured wall-clock speedup of about 3.4x, and this will draw immediate criticism. More substantively, the acceptance rule in Eq. (9) makes the near-full performance claim partly a selection artifact, and the single-seed search provides no evidence of run-to-run robustness. These are fixable with additional experiments and reporting, so I recommend major revision rather than rejection. I would also ask the authors to state δ_acc plainly; its absence is the single most important missing number in the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: EVO is a plausible training-free acceleration method with one genuinely new idea — globally allocating a fixed cache-refresh budget over the block-timestep lattice and searching it with an evolutionary algorithm using rollout success as fitness. The ablation study supports the design choices, the code is public, and the main tables show EVO beating both fixed-interval (EfficientVLA) and per-block (BAC) caching on average success at comparable FLOPs. That part is solid.\n\nNow the soft spots, in proportion. First, the abstract's \"8.05x action-generation speedup\" is a FLOPs ratio (15.77G to 1.96G), not what you'd see on a clock. Their own wall-clock measurements in Sec. A.7 give about 3.4x (448.91 ms to 132.17 ms). That's a real conflation and should be corrected before publication.\n\nSecond, the near-full performance claim is partly a selection artifact, though less damning than the stress-test note suggests. The search accepts the first schedule whose 50-episode formal success meets q_base − δ_acc, with δ_acc unreported. Selection across generations on a noisy 20-episode screen can inflate the chosen schedule's apparent quality, and the final 3×50 evaluation is on the same acceptance criterion's distribution. The paper would be far more convincing with multiple search seeds and a reported δ_acc. Appendix A.1 says each task is searched with one seed, so we have no estimate of search-to-search variability. That's a genuine reproducibility gap.\n\nThird, key hyperparameters (epsilon, gamma in Eq. 7, delta_acc in Eq. 9) are not released. Given the code is public, this is fixable, but as submitted the reader can't rerun the search.\n\nOne more thing: the BAC baseline shares a co-author (Kangye Ji) with this paper, and that isn't disclosed. It doesn't invalidate the comparison, but reviewers should know.\n\nWho should read this: anyone working on cache-based acceleration for diffusion policies. It's a reasonable extension of known components, and the global-formulation idea is worth taking seriously as a baseline. It deserves a real referee, and with the speedup claim corrected, the search variability reported, and the missing hyperparameters released, it could be a useful contribution. My recommendation: send to peer review, but expect the authors to tighten the claims before acceptance.","headline":"EVO has a genuinely new global cache-scheduling formulation and the ablations support it, but the 8x speedup is a FLOPs ratio (wall-clock is ~3.4x) and the near-full performance claim rests on a single-seed search with an unreported acceptance threshold.","tokens_in":21804,"tokens_out":3348,"would_cite":true,"duration_ms":31026,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EVO claims that a globally searched cache schedule over the block-timestep lattice can cut diffusion-policy action-generation FLOPs from 15.77G to as low as 1.96G—up to 8.05x speedup—while keeping task success near the uncached policy.","keywords":["diffusion policy","feature caching","evolutionary search","cache schedule","inference acceleration","robot manipulation","denoising","block-timestep lattice"],"falsifier":"Take one task from the paper's set and run the EVO search from multiple independent search seeds (e.g., 10) with the same budget and final evaluation protocol; if the mean final success rate across those seeds falls more than the allowed δ_acc below the full model, the claim that EVO reliably finds near-full-performance schedules is refuted. A complementary check: compare the best found schedule against a randomly sampled schedule of the same budget under identical 50-episode evaluation; if random schedules match its success, rollout-driven search is not the source of the result.","tokens_in":20856,"feed_emoji":"🤖","tokens_out":5746,"duration_ms":49392,"temperature":0.7,"pith_summary":"EVO claims that the best way to accelerate a pretrained transformer-based diffusion policy is not a fixed or per-block refresh rule but a global schedule that decides, for each denoising step and each transformer module, whether to recompute or reuse a cached residual. It searches this discrete schedule space with an evolutionary algorithm that scores candidates by real closed-loop rollout success, using feature dissimilarity only to bias the starting population and early stopping to cap search cost. If the paper is right, a single offline search per task yields a schedule that cuts action-generation FLOPs from 15.77G to as low as 1.96G—an up to 8.05x speedup—while keeping task success within a couple of points of the uncached policy. A careful reader should care because it offers a training-free, plug-in acceleration path for diffusion policies that does not touch policy weights or the sampler.","feed_headline":"Evolutionary search slashes diffusion-policy FLOPs 8x","feed_subtitle":"EVO finds which denoising computations to skip, keeping task success near full-model levels without retraining.","key_machinery":"The load-bearing object is the cache schedule: a set S of K positions in the B×T lattice formed by the B cacheable residual computations (self-attention, cross-attention, FFN per layer) and T denoising steps. At inference, positions in S run the original residual branch and refresh a per-module cache; positions outside S reuse the most recent cached residual. EVO's evolutionary search—tournament selection, set-level crossover, mutation, repair, and elitism—treats each schedule as a fixed-size individual and scores it by empirical rollout success, so the budget can move freely from redundant to sensitive modules. Redundancy-aware initialization samples part of the initial population from a di","core_discovery":"The paper's central claim is that heterogeneous redundancy across a diffusion policy's transformer blocks makes uniform or per-block cache budgets suboptimal, and that a globally budgeted cache schedule—a subset of the block-timestep lattice of fixed size K—can be found by evolutionary search using rollout success rate as the fitness. The discovered schedule, evaluated in the paper on multiple manipulation benchmarks, preserves near-full success (average 0.78–0.99 across datasets vs 0.79–0.99 for the full model) while reducing FLOPs from 15.77G to about 1.96–2.34G and raising policy inference frequency from 2.23 Hz to 7.57 Hz, without retraining or modifying the diffusion sampler.","pith_inferences":["Because the search is seeded from activations of the uncached policy on the target task's demonstrations, the found schedules are likely task- and distribution-specific; a natural extension is to test whether a schedule found on one task transfers to a different task or to a real robot without re-searching.","The weak similarity-sensitivity correlation reported in the paper suggests a cheaper alternative—random schedule search with the same rollout budget—might produce comparable schedules; comparing EVO against such a baseline would clarify how much of the gain comes from the evolutionary search itself versus from global allocation.","The method's fitness is simulator rollouts; before real-world deployment, robustness to sim-to-real gaps would need checking, since the frozen schedule is optimized under simulation dynamics and a single search seed.","Since the schedule space grows as binomial(BT, K), the evolutionary approach could also be applied to other discrete inference-optimization problems—e.g., where to skip layers or tokens in large multimodal models—where the bottleneck is the same: a non-differentiable objective defined by downstream performance."],"forward_implications":["If EVO's central claim holds, training-free cache acceleration of diffusion policies can be treated as a global combinatorial search problem rather than a hand-designed refresh rule, and the same search machinery should apply to any transformer-based diffusion policy without retraining.","A frozen per-task schedule found offline can be deployed directly in the inference loop, so the one-time search cost is amortized over every action query; the paper reports the schedule is reused across episodes from the same environment distribution without online adaptation.","Allowing the refresh budget to move across blocks improves both performance and search efficiency: the paper's ablation shows global block-timestep allocation raises average success from 0.87 to 0.91 while cutting search cost from 1.48 to 1.00 GPU-days.","Feature similarity, while a useful starting prior, is too weakly correlated with task sensitivity to serve as the scheduling objective; EVO's results imply rollout-based fitness is needed to find schedules that actually preserve closed-loop behaviour.","The up-to-8.05x speedup and 1.96G FLOPs at M=8 are measured with 100 denoising steps on the evaluated benchmarks; the same budget settings (M=8, M=10) correspond to fixed total refresh positions K=192 and K=240."],"fun_headline_variants":["Evolutionary search trims diffusion-policy FLOPs 8.05x","EVO: Evolutionary cache scheduling cuts diffusion-policy compute 8x","8x faster diffusion policy inference via learned cache schedules","Skip redundant denoising: evolutionary cache schedules cut FLOPs 8x","EVO: Which denoising steps to skip? Search speeds policies 8x"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The single search seed per task, combined with 20-episode quick rollouts as the screening signal, is the load-bearing premise: if that one search trajectory overfits to the quick-evaluation seed or to the simulation distribution, the reported near-full performance on held-out episodes may not replicate.","fun_headline_variants_meta":{"raw":{"variants":["Evolutionary search trims diffusion-policy FLOPs 8.05x","EVO: Evolutionary cache scheduling cuts diffusion-policy compute 8x","8x faster diffusion policy inference via learned cache schedules","Skip redundant denoising: evolutionary cache schedules cut FLOPs 8x","EVO: Which denoising steps to skip? Search speeds policies 8x"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00131,"raw_usage":{"total_tokens":5194,"prompt_tokens":781,"completion_tokens":4413,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":4314}},"tokens_in":525,"tokens_out":4413,"duration_ms":27073,"temperature":1.0,"reasoning_tokens":4314,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T10:13:37.259899+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one task from the paper's set and run the EVO search from multiple independent search seeds (e.g., 10) with the same budget and final evaluation protocol; if the mean final success rate across those seeds falls more than the allowed δ_acc below the full model, the claim that EVO reliably finds near-full-performance schedules is refuted. A complementary check: compare the best found schedule against a randomly sampled schedule of the same budget under identical 50-episode evaluation; if random schedules match its success, rollout-driven search is not the source of the result.","supporting_citations":[],"review_version":1}