REVIEW 3 major objections 5 minor 37 references
Robot policies that shorten action chunks when the observation is most sensitive succeed more often without extra computation.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-11 14:17 UTC pith:O4WBJ64V
load-bearing objection Clean optimization-derived sensitivity metric for adaptive chunk horizons in diffusion policies, with matched-compute gains on sim/real; optimality claim rests on an unchecked power-law disturbance model that experiments never validate. the 3 major comments →
Spatial Attention: Adapting Execution Horizons for Diffusion Policies via Observation Sensitivity
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Under a fixed budget of chunk samplings, the execution horizon that minimizes the cumulative likelihood drop caused by observation disturbances decreases as Spatial Attention increases. Forecasting Spatial Attention along each sampled chunk and terminating when the integrated rate reaches a threshold therefore improves success rates while preserving the average execution horizon.
What carries the argument
Spatial Attention—the expected squared norm of the gradient of the action log-likelihood with respect to the observation—together with the optimal resampling rate ρ(t) ∝ Att(o(t))^{1/(2γ+1)} that solves the constrained minimization of cumulative likelihood drop.
Load-bearing premise
The paper models the observation disturbance that accumulates during open-loop execution as zero-mean isotropic Gaussian noise whose variance grows exactly as a power of elapsed time.
What would settle it
On a high-precision task such as Tool Hang, replace the power-law disturbance model with a non-Gaussian or non-power-law process; if the Spatial-Attention adaptive horizon then fails to raise success rate above the matched fixed-horizon baseline, the central claim is refuted.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses the fixed-execution-horizon trade-off in generative (diffusion/consistency) policies that sample action chunks. It defines Spatial Attention as Att(o) = E_a[||∇_o log π(a|o)||²], the expected squared sensitivity of the action log-likelihood to the observation. Under a fixed sampling budget N, it formulates minimization of cumulative expected log-likelihood drop caused by open-loop disturbances, approximates the drop via second-order Taylor expansion plus the Fisher-information identity, and obtains the optimal resampling rate ρ(t) ∝ Att(o(t))^{1/(2γ+1)}. At inference a sequence-to-sequence forecaster predicts future Att values from the current observation and sampled chunk; the execution horizon T_a is the first index at which the integrated rate exceeds a threshold C_att. Score networks (via Bayes decomposition) and a VAE latent for vision make Att computable without an explicit policy density. Experiments on Robomimic (standard and object-perturbation variants) and a real Franka task with a moving cube show higher success rates than fixed-horizon DDPM/DDIM/CP baselines while matching average T_a.
Significance. If the result holds, the work supplies a modular, architecture-agnostic criterion for adaptive chunk execution that is derived from an explicit optimization problem rather than a pure heuristic. The experimental protocol is careful: C_att is chosen so that mean T_a lies in [T_avg−0.5, T_avg+0.5], results are averaged over 20 checkpoints (4 seeds × 5 best), and both state and vision settings plus real-robot disturbance are reported. The real-robot lift of success rate from 0.42 to 0.92 at essentially identical average horizon is a concrete practical contribution. The framework applies to any chunk-generating policy and is therefore of interest beyond the diffusion models used for evaluation. The main theoretical claim is conditional on a modeling assumption (power-law isotropic Gaussian disturbance growth); when that assumption is accepted or the method is viewed as a principled heuristic, the paper advances the responsiveness–compute trade-off for generative robot policies.
major comments (3)
- [§III-A, Eq. (4); Abstract; Eqs. 8, 14] The optimality claim (Abstract; §III-B–C; Eqs. 8, 14) is derived under the modeling assumption that accumulated observation disturbance after open-loop time τ is zero-mean isotropic Gaussian with variance exactly ε²τ^{2γ} (Eq. 4). The subsequent Taylor + Fisher reduction yields D(o,τ)≈(1/2)Att(o)ε²τ^{2γ} (Eqs. 9–11), which is the sole justification for the continuous objective and the power-law rate. The experimental disturbances (orthogonal object translation once the gripper is near; constant-velocity camera-slider motion) are deterministic, structured, and non-Gaussian. The manuscript never checks whether residual observation error under these regimes grows as a power of open-loop time, nor whether the second-order approximation remains accurate. Consequently the claimed optimality of the Spatial-Attention schedule does not automatically transfer to the reported experiments. Either va
- [Eqs. (14), (20); §V experimental protocol] The free parameter γ enters the optimal rate exponent 1/(2γ+1) (Eq. 14) and the discrete accumulation rule (Eq. 20). The manuscript does not report the numerical value of γ used in any experiment, nor any sensitivity of success rate or realized T_a distribution to γ. Without this information the mapping from forecasted Att to T_a is not fully reproducible, and it is unclear how much of the reported gains depend on a particular choice of the disturbance-growth exponent versus the qualitative idea of shortening horizons in high-Att phases. Report γ (and the resulting effective exponent) for all tables, and preferably include a brief sensitivity sweep or an argument that results are stable over a plausible range of γ.
- [§II-B; Tables I–III] Related work (§II-B) criticizes Mixture of Horizons (MoH) as heuristic and notes that it was not evaluated under external disturbances, yet the experimental section contains no comparison to MoH or to any other adaptive-horizon baseline. Because the central empirical claim is that Spatial Attention improves success rates at matched average horizon, a direct head-to-head (or at least a discussion of why MoH is inapplicable to the disturbance settings) is needed to substantiate superiority over the closest published alternative. If computational or architectural reasons preclude a full re-implementation, state them and provide a controlled ablation that isolates the contribution of the Att-based schedule versus simpler adaptive rules (e.g., fixed short horizon near contact).
minor comments (5)
- [Fig. 2] Figure 2 caption and surrounding text usefully illustrate high-Att phases, but the quantitative scale of Att and the precise C_att threshold used for that rollout are not stated; adding them would make the figure self-contained.
- [§IV-B, Eq. (18)] The VAE latent proxy for vision (Eq. 18) is presented as practical; a short remark on whether Att computed in latent space correlates with Att computed (where feasible) in a lower-dimensional state space would strengthen the proxy claim.
- [Throughout; §IV-C heading] Notation: T_H, T_a, T_avg, T_max appear with slight typographic inconsistency (subscripts vs. plain); unify. Also “F orecasted” and similar spacing artifacts appear in headings.
- [§IV-C] The forecasting model Att_φ is a transformer trained offline; no quantitative forecast error (e.g., MAE of predicted vs. oracle Att along held-out trajectories) is reported. Even a brief number would help readers judge how much of the gain comes from accurate foresight versus the Att signal itself.
- [Table II] Table II omits the T_avg / steps columns present in Table I; for consistency and readability they should be restored or explicitly cross-referenced.
Circularity Check
No significant circularity; the optimality relation and adaptive-horizon rule are derived from an explicit approximate objective under a stated disturbance model, and C_att is used only to equalize average compute for fair comparison.
full rationale
The derivation chain is self-contained. Spatial Attention is defined directly as Att(o) ≜ E_a[‖∇_o log π(a|o)‖²] (Eq. 15). Under the modeling assumption δo ∼ N(0, ε² τ^{2γ} I) (Eq. 4), a second-order Taylor expansion plus the Fisher-information identity produces the closed-form drop D(o,τ) ≈ (1/2) Att(o) ε² τ^{2γ} (Eqs. 9–11). Substituting into the cumulative-drop objective and taking the continuum limit yields the tractable problem (Eq. 8). A standard Lagrangian then gives the pointwise stationarity condition ρ(t) ∝ Att(o(t))^{1/(2γ+1)} (Eq. 14). At inference the same quantity is forecasted by a sequence model trained on offline labels and the execution horizon is cut when the discrete integral of Att^{1/(2γ+1)} reaches a threshold C_att (Eq. 20). C_att is chosen solely so that the realized mean horizon lies inside [T_avg−0.5, T_avg+0.5], equalizing the sampling budget with the fixed-horizon baselines; it is not fitted to success rates. The success-rate gains reported in Tables I–III are therefore empirical outcomes, not tautological consequences of the fit. The sole self-citation ([14], coauthored by Park) appears only as motivational background for the phrase “minimum-attention control” and is not used to import uniqueness, an ansatz, or any load-bearing lemma. No equation reduces a claimed prediction to its own inputs by construction.
Axiom & Free-Parameter Ledger
free parameters (3)
- γ (disturbance growth exponent)
- C_att (attention accumulation threshold)
- ε (noise scale)
axioms (4)
- ad hoc to paper Accumulated observation disturbance after open-loop interval τ is zero-mean isotropic Gaussian with variance ε²τ^{2γ}.
- standard math Second-order Taylor expansion of log π(a|o+δo) plus Fisher-information identity yields D(o,τ)≈(1/2)Att(o)ε²τ^{2γ}.
- domain assumption Att(o(·)) and ρ(·) are continuous, allowing the discrete bound (Eq. 12) to pass to the continuous objective (Eq. 8).
- domain assumption Latent-space gradient norm of a VAE encoder is a faithful proxy for observation-space sensitivity of the policy.
invented entities (2)
-
Spatial Attention Att(o)=E_a[‖∇_o log π(a|o)‖²]
no independent evidence
-
Sequence-to-sequence Spatial Attention forecaster Att_ϕ
no independent evidence
read the original abstract
Sampling action chunks via generative models has become a widely adopted methodology for robotic learning from demonstration. However, existing methods often struggle to balance responsiveness and computational cost because they execute each action chunk for a fixed execution horizon. In this paper, we adaptively adjust the execution horizon of sampled action chunks, balancing responsiveness and computational efficiency. We introduce Spatial Attention -- defined as the expected squared norm of the gradient of the action log-likelihood with respect to the observation -- which indicates the sensitivity of the policy's action distribution to variations in the observation. We show that, under a fixed budget of chunk samplings, the execution horizon that minimizes the cumulative likelihood drop induced by disturbances decreases as Spatial Attention increases. By forecasting future Spatial Attention values alongside the action chunk, our framework dynamically assigns shorter execution horizons to phases with high Spatial Attention, and longer horizons to phases with low Spatial Attention. Experiments on standard and perturbed tasks, in both simulation and on a real robot, show that our method significantly improves success rates over fixed-horizon baselines while maintaining the average execution horizon.
Figures
Reference graph
Works this paper leans on
-
[1]
Diffusion policy: Visuomotor policy learning via ac- tion diffusion,
C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via ac- tion diffusion,”The International Journal of Robotics Research, p. 02783649241273668, 2023
2023
-
[2]
Consistency policy: Accelerated visuomotor policies via consistency distillation,
A. Prasad, K. Lin, J. Wu, L. Zhou, and J. Bohg, “Consistency policy: Accelerated visuomotor policies via consistency distillation,”arXiv preprint arXiv:2405.07503, 2024
Pith/arXiv arXiv 2024
-
[3]
One-step diffusion policy: Fast visuomotor policies via diffusion distillation,
Z. Wang, Z. Li, A. Mandlekar, Z. Xu, J. Fan, Y . Narang, L. Fan, Y . Zhu, Y . Balaji, M. Zhouet al., “One-step diffusion policy: Fast visuomotor policies via diffusion distillation,”arXiv preprint arXiv:2410.21257, 2024
Pith/arXiv arXiv 2024
-
[4]
Learning fine- grained bimanual manipulation with low-cost hardware,
T. Z. Zhao, V . Kumar, S. Levine, and C. Finn, “Learning fine- grained bimanual manipulation with low-cost hardware,”arXiv preprint arXiv:2304.13705, 2023
Pith/arXiv arXiv 2023
-
[5]
Responsive noise-relaying diffusion policy: Responsive and efficient visuomotor control,
Z. Chen, X. Yuan, T. Mu, and H. Su, “Responsive noise-relaying diffusion policy: Responsive and efficient visuomotor control,”arXiv preprint arXiv:2502.12724, 2025
Pith/arXiv arXiv 2025
-
[6]
Reactive diffusion policy: Slow-fast visual-tactile policy learning for contact-rich manipulation,
H. Xue, J. Ren, W. Chen, G. Zhang, Y . Fang, G. Gu, H. Xu, and C. Lu, “Reactive diffusion policy: Slow-fast visual-tactile policy learning for contact-rich manipulation,” inProceedings of Robotics: Science and Systems (RSS), 2025
2025
-
[7]
Subconscious robotic imitation learning,
J. Xie, Z. Wang, J. Tan, H. Lin, and X. Ma, “Subconscious robotic imitation learning,”arXiv preprint arXiv:2412.20368, 2024
Pith/arXiv arXiv 2024
-
[8]
Openvla: An open- source vision-language-action model,
M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketiet al., “Openvla: An open- source vision-language-action model,”arXiv preprint arXiv:2406.09246, 2024
Pith/arXiv arXiv 2024
-
[9]
Real-time execution of action chunking flow policies,
K. Black, M. Y . Galliker, and S. Levine, “Real-time execution of action chunking flow policies,”arXiv preprint arXiv:2506.07339, 2025
Pith/arXiv arXiv 2025
-
[10]
Adaptive online replanning with diffusion models,
S. Zhou, Y . Du, S. Zhang, M. Xu, Y . Shen, W. Xiao, D.-Y . Yeung, and C. Gan, “Adaptive online replanning with diffusion models,”Advances in Neural Information Processing Systems, vol. 36, pp. 44 000–44 016, 2023
2023
-
[11]
Adaptive planning with generative models under uncertainty,
P. Jutras-Dub ´e, R. Zhang, and A. Bera, “Adaptive planning with generative models under uncertainty,” in2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2024, pp. 7664–7670
2024
-
[12]
Dynamic obstacle avoidance through uncertainty-based adaptive planning with diffusion,
V . Punyamoorty, P. Jutras-Dub ´e, R. Zhang, V . Aggarwal, D. Conover, and A. Bera, “Dynamic obstacle avoidance through uncertainty-based adaptive planning with diffusion,”arXiv preprint arXiv:2409.16950, 2024
Pith/arXiv arXiv 2024
-
[13]
Minimum attention control,
W. Brockett, “Minimum attention control,” inProceedings of the 36th IEEE Conference on Decision and Control, vol. 3. IEEE, 1997, pp. 2628–2632
1997
-
[14]
A minimum attention control law for ball catching,
C. Jang, J.-e. Lee, S. Lee, and F. C. Park, “A minimum attention control law for ball catching,”Bioinspiration & Biomimetics, vol. 10, no. 5, p. 055008, 2015
2015
-
[15]
What matters in learning from offline human demonstrations for robot manipulation,
A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y . Zhu, and R. Mart ´ın-Mart´ın, “What matters in learning from offline human demonstrations for robot manipulation,” inConference on Robot Learning (CoRL), 2021
2021
-
[16]
T. T. Zhang, D. Pfrommer, C. Pan, N. Matni, and M. Simchowitz, “Action chunking and exploratory data collection yield exponential im- provements in behavior cloning for continuous control,”arXiv preprint arXiv:2507.09061, 2025
arXiv 2025
-
[17]
π 0: A vision-language-action flow model for general robot control,
K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichteret al., “π 0: A vision-language-action flow model for general robot control,”arXiv preprint arXiv:2410.24164, 2024
Pith/arXiv arXiv 2024
-
[18]
GR00T N1: An open foundation model for generalist humanoid robots,
NVIDIA, J. Bjorck, N. C. Fernando Casta ˜neda, X. Da, R. Ding, L. J. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, J. Jang, Z. Jiang, J. Kautz, K. Kundalia, L. Lao, Z. Li, Z. Lin, K. Lin, G. Liu, E. Llontop, L. Magne, A. Mandlekar, A. Narayan, S. Nasiriany, S. Reed, Y . L. Tan, G. Wang, Z. Wang, J. Wang, Q. Wang, J. Xiang, Y . Xie, Y . Xu, Z. Xu, S. Ye, Z. Yu, ...
2025
-
[19]
Planning with diffu- sion for flexible behavior synthesis,
M. Janner, Y . Du, J. B. Tenenbaum, and S. Levine, “Planning with diffu- sion for flexible behavior synthesis,”arXiv preprint arXiv:2205.09991, 2022
Pith/arXiv arXiv 2022
-
[20]
Is conditional generative modeling all you need for decision-making?
A. Ajay, Y . Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal, “Is conditional generative modeling all you need for decision-making?” arXiv preprint arXiv:2211.15657, 2022
Pith/arXiv arXiv 2022
-
[21]
Mixture of horizons in action chunking,
D. Jing, G. Wang, J. Liu, W. Tang, Z. Sun, Y . Yao, Z. Wei, Y . Liu, Z. Lu, and M. Ding, “Mixture of horizons in action chunking,”arXiv preprint arXiv:2511.19433, 2025
Pith/arXiv arXiv 2025
-
[22]
Generative modeling by estimating gradients of the data distribution,
Y . Song and S. Ermon, “Generative modeling by estimating gradients of the data distribution,”Advances in neural information processing systems, vol. 32, 2019
2019
-
[23]
Improved techniques for training score-based generative models,
——, “Improved techniques for training score-based generative models,” Advances in neural information processing systems, vol. 33, pp. 12 438– 12 448, 2020
2020
-
[24]
Score-based generative modeling through stochastic differ- ential equations,
Y . Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole, “Score-based generative modeling through stochastic differ- ential equations,”arXiv preprint arXiv:2011.13456, 2020
Pith/arXiv arXiv 2011
-
[25]
A connection between score matching and denoising au- toencoders,
P. Vincent, “A connection between score matching and denoising au- toencoders,”Neural computation, vol. 23, no. 7, pp. 1661–1674, 2011
2011
-
[26]
Denoising diffusion probabilistic models,
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840– 6851, 2020
2020
-
[27]
Deep unsupervised learning using nonequilibrium thermodynamics,
J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli, “Deep unsupervised learning using nonequilibrium thermodynamics,” in International conference on machine learning. pmlr, 2015, pp. 2256– 2265
2015
-
[28]
Diffusion models beat gans on image synthesis,
P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,”Advances in neural information processing systems, vol. 34, pp. 8780–8794, 2021
2021
-
[29]
High- resolution image synthesis with latent diffusion models,
R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10 684–10 695
2022
-
[30]
Auto-encoding variational bayes,
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,”arXiv preprint arXiv:1312.6114, 2013
Pith/arXiv arXiv 2013
-
[31]
Attention is all you need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017
2017
-
[32]
Denoising diffusion implicit models,
J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” arXiv preprint arXiv:2010.02502, 2020
Pith/arXiv arXiv 2010
-
[33]
Consistency trajectory models: Learning probability flow ode trajectory of diffusion,
D. Kim, C.-H. Lai, W.-H. Liao, N. Murata, Y . Takida, T. Uesaka, Y . He, Y . Mitsufuji, and S. Ermon, “Consistency trajectory models: Learning probability flow ode trajectory of diffusion,”arXiv preprint arXiv:2310.02279, 2023
Pith/arXiv arXiv 2023
-
[34]
Parallel sam- pling of diffusion models,
A. Shih, S. Belkhale, S. Ermon, D. Sadigh, and N. Anari, “Parallel sam- pling of diffusion models,”Advances in Neural Information Processing Systems, vol. 36, pp. 4263–4276, 2023
2023
-
[35]
Foundationpose: Unified 6d pose estimation and tracking of novel objects,
B. Wen, W. Yang, J. Kautz, and S. Birchfield, “Foundationpose: Unified 6d pose estimation and tracking of novel objects,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, pp. 17 868–17 879
2024
-
[36]
Fast policy synthesis with variable noise diffusion models,
S. H. Høeg, Y . Du, and O. Egeland, “Fast policy synthesis with variable noise diffusion models,” in2025 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2025, pp. 4821–4828
2025
-
[37]
Streaming flow policy: Simplifying diffusion/flow-matching policies by treating action trajectories as flow trajectories,
S. Jiang, X. Fang, N. Roy, T. Lozano-P ´erez, L. P. Kaelbling, and S. Ancha, “Streaming flow policy: Simplifying diffusion/flow-matching policies by treating action trajectories as flow trajectories,” inConference on Robot Learning. PMLR, 2025, pp. 238–257
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.