REVIEW 5 major objections 8 minor 32 references
A PAC-Bayes regularizer on diffusion-policy weights improves robot manipulation, especially when data is scarce.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-31 18:52 UTC pith:WPVBUYTS
load-bearing objection Useful KL-regularized Bayesian DP with real low-data gains; the PAC-Bayes framing is mostly heuristic because the bound does not actually apply to the unbounded loss or the mean predictor they deploy. the 5 major comments →
PAC-DP: PAC-Bayesian Diffusion Policy Learning
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Modeling a diffusion policy’s noise predictor as a Bayesian network and minimizing a PAC-Bayes surrogate—empirical denoising risk plus a weighted KL divergence between posterior and prior weight distributions—yields better validation denoising error, lower variational negative log-likelihood, and higher robotic success rates than standard diffusion policies and unregularized Bayesian diffusion policies, especially in low-data regimes and on complex tasks.
What carries the argument
The PAC-DP training objective (Eq. 10): the expected denoising loss under a factorized-Gaussian posterior Q, plus the square-root and linear PAC-Bayes complexity terms built from λ D_KL(Q∥P) and a confidence term. Weights are sampled via the reparameterization trick; inference can use the posterior mean.
Load-bearing premise
The method treats a PAC-Bayes bound written for bounded losses as a valid training guide for the unbounded squared denoising loss, on the grounds that the loss quickly falls below one so clipping would barely change it.
What would settle it
On the same low-data PushT and ToolHang setups, train PAC-DP versus standard DP and unregularized Bayesian DP and check whether PAC-DP still shows lower held-out noise-prediction error, a substantially tighter variational NLL bound, and higher success rate when the loss is actually clipped to satisfy the bound, or when λ is set so the KL term is truly active.
If this is right
- Robot imitation with few demonstrations can gain from adding a closed-form KL regularizer on a Gaussian posterior over denoiser weights rather than only from more data or architecture changes.
- Bayesian parameterization alone is not enough; the PAC-Bayes KL term is what drives the reported gains over plain Bayesian diffusion policies.
- Keeping conditioning layers deterministic while making only the denoising U-Net Bayesian preserves a stable observation signal and avoids the collapse seen when the full network is Bayesian.
- The same objective is a candidate regularizer for longer-horizon and harder multi-stage manipulation where memorization of scarce demos is most damaging.
Where Pith is reading between the lines
- If the bound-as-surrogate story holds, similar KL-from-PAC-Bayes terms could be tried on other generative imitation losses (score matching, flow matching) without redesigning the policy network.
- Data-dependent or hierarchical priors, mentioned only as future work, are a natural next lever if the fixed factorized Gaussian prior is the main source of bound looseness.
- The gap between better denoising metrics and sometimes flat or lower success on Square/Transport suggests pairing this regularizer with a task-aware or closed-loop objective when control performance, not likelihood, is the true target.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes PAC-DP, a training objective for diffusion policies in which the denoising network is treated as a Bayesian neural network with a factorized-Gaussian posterior Q(w), and the standard denoising loss is augmented with a KL(Q‖P) regularizer derived from the PAC-Bayes-kl bound via a refined-Pinsker relaxation (Eqs. 5, 9, 10). Training uses the reparameterization trick with softplus-parameterized posterior scales (Eqs. 7–8); inference uses the posterior mean. Experiments on PushT and five Robomimic tasks compare PAC-DP against standard DP and an unregularized Bayesian ablation (BDP), reporting held-out noise-prediction error, a variational NLL bound, and success rates over 1,000 rollouts per seed (3 seeds), plus a low-data sweep (20–200 demos), hyperparameter sensitivity and architecture ablations, and a 200-trial real-robot deployment on a Franka Panda. PAC-DP improves NPE and NLL on every task and achieves the clearest SR gains on PushT, ToolHang, and the low-data regime, while slightly underperforming DP on Square and Transport at 200 demos.
Significance. If the results hold, this is a useful contribution to diffusion-based imitation learning: a training objective with a learning-theoretic motivation that demonstrably improves held-out denoising error and variational NLL on every task tested, with the largest task-level gains exactly where the field needs them (low-data regimes, hard long-horizon tasks such as ToolHang, +19.4 SR points over DP at 200 demos). The work would be, to my knowledge, the first to bring PAC-Bayes regularization into diffusion policy training, and the empirical protocol is strong by the standards of the venue. The theoretical grounding, however, currently rests on an unproven boundedness assumption, and the claimed "guarantees" are never instantiated numerically; until that is repaired, the contribution is best described as a well-validated principled regularizer rather than a guaranteed one.
major comments (5)
- [§III-C, Eqs. (5) and (10)] The bounded-loss gap is load-bearing for the paper's theoretical claim and the given justification does not close it. The PAC-Bayes-kl inequality underlying Eq. (5) requires ℓ(w,z) ∈ [0,1] for all z, but the defense offered — 'the denoising loss rapidly decreases below one during the first few training iterations, and remains in this regime' — is a statement about the empirical average training objective, not about per-sample or population losses. Two concrete problems: (i) the per-sample loss ‖ε − ε_w(x_t,t,o)‖²₂ at high noise levels t is on the order of E‖ε‖², which for the summed squared norm in Eq. (2) scales with the action-sequence dimension (T_a × action dim), so individual draws plausibly exceed 1 throughout training even when the running average is 0.02–0.19 (Table I); (ii) the population risk R(Q) in Eq. (4) integrates the unbounded squared error over unseen data, so near-equal
- [§I, §III-B, §V — certificate never computed] The paper repeatedly invokes generalization guarantees (abstract: 'theoretically grounded framework'; §I: 'establishes theoretical guarantees on expected generalization performance') but never evaluates the certificate in Eq. (5) for the learned (Q,P) on any task. Given a U-Net-scale KL term, the certificate may be vacuous even under a clipped loss — which is acceptable in the PAC-Bayes literature, but only if reported and discussed. If the certificate is vacuous, the contribution is a principled *motivation* for a KL regularizer, not a guarantee, and the abstract/§I/§V wording should be revised accordingly. Reporting at least one evaluated bound (e.g., on PushT, with stated δ and N) would substantially strengthen the letter either way.
- [§III-C, Eq. (10) — the λ weighting] The weighted objective in Eq. (10) inserts λ only in front of D_KL(Q‖P) inside the Pinsker-relaxed bound, and λ is then tuned per task (λ=10⁻⁵ for Lift/Can/ToolHang, 10⁻⁶ for Square/Transport; §IV-B2). With λ ≠ 1, Eq. (10) is no longer a generalization bound in any stated sense, and 'following common practice in PAC-Bayesian optimization' is doing real work here — the analogous λ-weighted objectives in the literature are known to break the bound's interpretation and in some settings its empirical behavior. Combined with per-task λ selection, the 'derived directly from the PAC-Bayes generalization bound' framing in §III-C and §V should be moderated, or the λ-weighted objective should be justified formally (e.g., via a bound that tolerates the rescaling).
- [§IV-B2, Table I — success-rate results are mixed] The central empirical claim ('higher success rates across multiple robotic manipulation benchmarks') is only partially supported by the headline 200-demonstration table. PAC-DP loses clearly on Transport (0.701 vs 0.761 for DP, and also below BDP at 0.768) and on Square (0.900 vs 0.920), while Lift and Can are saturated ties. The genuine SR wins are PushT (Fig. 3) and ToolHang, plus the low-data sweep in Table II. The text in §IV-B2 is commendably honest about Square/Transport, but the abstract and conclusion should be calibrated to 'improved denoising/NLL across all tasks, with SR gains concentrated in low-data regimes and the hardest tasks' rather than the current blanket phrasing. Relatedly, Table I reports λ differing across tasks, which readers should weigh when interpreting 'consistent' improvements.
- [§IV metrics and §IV-B1, Fig. 3 — NPE/NLL evaluation protocol] The NPE and NLL evaluation protocol for the Bayesian models is underspecified, and the magnitude of the reported gap makes this consequential. It is not stated whether held-out NPE for PAC-DP/BDP is computed with mean weights, a single posterior sample, or a posterior average; nor how the variational NLL bound of [32] is evaluated for a stochastic denoiser. The PushT result — DP/BDP saturating at −3.24 (the random-denoiser floor) while PAC-DP reaches −8.275 ± 0.142 — is an enormous qualitative difference, and before accepting it as evidence of 'more informative denoisers' the reader needs assurance the comparison is like-for-like. Please specify the exact estimator and number of weight samples used for both metrics.
minor comments (8)
- [Algorithm 2, line 8] The noise predictor is defined as ε_w(x_t, t, o) in Eq. (2) and §III-A, but Algorithm 2 line 8 calls ε_w(x_t, c, t) with an undefined 'c' and swapped argument order. Please make notation consistent.
- [§III-B, Eq. (5)] The bound in Eq. (5) depends on N and δ, but neither is instantiated: is N the number of demonstrations (140), demonstration fragments, or per-timestep denoising samples? δ is never given a value anywhere in the paper. Even if the ln(2√N/δ)/N term is negligible in practice, this should be stated explicitly with the chosen values.
- [§III-B vs §IV-A — status of ρ] §III-B (Eq. 8) presents ρ as a learnable posterior parameter updated by SGD, while §IV-A says 'the scale parameter ρ is chosen as a constant hyperparameter.' I believe the intended meaning is that ρ is learned but its *initialization* is a fixed hyperparameter (ρ₀ ∈ {−3.5,…,−5.0}), with softplus keeping σ near that scale early in training. Please clarify, since Table III's sensitivity analysis otherwise reads as sweeping a fixed σ.
- [§IV-C — real-world experiments] The real-robot evaluation reports 100% success over 200 trials for PAC-DP but includes no DP or BDP baseline on hardware, so it demonstrates deployability rather than an improvement over the baseline. The text mostly frames it correctly ('not intended as a comprehensive real-world benchmark'), but §V's 'real-world experiments confirmed successful deployment' should keep the same care. Also state that the five initial configurations per task are shared or comparable across policies if a baseline is added.
- [§IV-B4] Typo: 'Robomimic benchnark'. Also in the same paragraph, 'λ > 10⁻⁵ consistently degraded performance' — please quantify briefly (even one number), since this bounds the usable regularization range and is practically important for reproduction.
- [References] Ref. [27] (Seldin) is cited for the refined Pinsker relaxation as an arXiv preprint of lecture notes; please cite the primary source of the inequality if one exists. Ref. [4] is listed as NeurIPS vol. 38, 2026 — please verify the venue/year metadata before publication.
- [§IV (metrics), Algorithms 1–2] Inference uses the posterior mean throughout (§IV), but Algorithm 2 offers a sampled-weights mode. A one-line note on whether sampled inference was tried (even negatively) would preempt an obvious reader question, since sampled inference is the mode the PAC-Bayes analysis actually covers.
- [Fig. 3 and Fig. 4] Fig. 3's shaded std-over-seeds regions are nearly invisible at the printed scale; consider reporting the std numerically in the caption (already done for the final values) or widening the bands. Fig. 4 would benefit from timestamps and a brief statement of task success criteria for the hardware trials.
Circularity Check
No significant circularity: PAC-DP objective is a standard PAC-Bayes surrogate applied to DP, evaluated on independent held-out metrics.
full rationale
The derivation chain is externally grounded and non-circular. The paper imports the PAC-Bayes-kl inequality (Eq. 5) from the literature (Seeger; Refined Pinsker), interprets the standard denoising loss as empirical risk of a stochastic predictor (Eq. 9), and forms the weighted surrogate LPAC-DP (Eq. 10). That surrogate is then minimized and judged on held-out noise-prediction error, variational NLL bounds, and task success rates—quantities that are not algebraically identical to the KL(Q∥P) term or to the training objective by construction. The BDP ablation (λ=0) further separates Bayesian parameterization from the regularizer. Hyperparameter choices (λ, ρ) are ordinary tuning, not fitted inputs renamed as predictions of the main claim. No load-bearing self-citation, uniqueness import, or definitional loop appears. The bounded-loss gap (ℓ theoretically unbounded vs. PAC-Bayes-kl assumptions) is a validity/correctness concern about whether Eq. 10 is a true certificate surrogate, not a circularity in the derivation chain.
Axiom & Free-Parameter Ledger
free parameters (4)
- λ (KL weight in L_PAC-DP) =
typically 1e-4 to 1e-6
- ρ (posterior softplus scale init) =
often -4.5
- δ (PAC-Bayes failure probability) =
not numerically specified in experiments
- Prior/posterior Gaussian factorization and init =
as in [19]; Q=P at start
axioms (5)
- standard math PAC-Bayes-kl inequality plus Refined Pinsker relaxation yields the population-risk bound (Eq. 5) for stochastic predictors under i.i.d. data.
- domain assumption Demonstrations are i.i.d. samples from an unknown distribution D.
- ad hoc to paper After early training, unbounded L_DP may be optimized in place of a clipped loss min(ℓ,1) without breaking the bound’s applicability.
- domain assumption Mean-field factorized Gaussian posterior/prior suffice to express useful policy uncertainty and closed-form KL.
- domain assumption Improving a bound on denoising risk is a justified proxy for better closed-loop task success.
invented entities (1)
-
PAC-DP training objective (L_PAC-DP)
no independent evidence
read the original abstract
Diffusion Policies (DPs) are able to perform complex manipulation tasks. However, DPs are typically trained by minimizing a denoising objective, which provides limited control over generalization in the finite-data regimes common in robotics. In this letter, we propose PAC-DP, an approach that increases the performance of DPs in robotic manipulation tasks. By modeling the DP as a Bayesian neural network, and defining a PAC-Bayes generalization bound, we derive a novel training objective that augments the standard denoising loss with a Kullback-Leibler divergence regularizer between the posterior and prior parameter distributions. From the theoretical perspective, our approach provides a principled approach to regularize the training of DPs without significantly increasing the training time. From the practical point of view, experimental results demonstrate improved denoising performance, lower variational negative log-likelihood, and higher success rates across multiple robotic manipulation benchmarks. Crucially, the largest improvements are observed in low-data training regimes and complex tasks, establishing PAC-DP as a theoretically grounded framework for robot policy learning.
Figures
Reference graph
Works this paper leans on
-
[1]
Diffusion models for robotic manipulation: A survey,
R. Wolf, Y . Shi, S. Liu, and R. Rayyes, “Diffusion models for robotic manipulation: A survey,”Frontiers in Robotics and AI, vol. 12, p. 1606247, 2025
2025
-
[2]
Diffusion policy: Visuomotor policy learning via action diffusion,
C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,”The International Journal of Robotics Research, vol. 44, no. 10-11, pp. 1684–1704, 2025
2025
-
[3]
A good score does not lead to a good generative model,
S. Li, S. Chen, and Q. Li, “A good score does not lead to a good generative model,”arXiv preprint arXiv:2401.04856, 2024
Pith/arXiv arXiv 2024
-
[4]
Why diffusion models don’t memorize: The role of implicit dynamical regularization in training,
T. Bonnaire, R. Urfin, G. Biroli, and M. M ´ezard, “Why diffusion models don’t memorize: The role of implicit dynamical regularization in training,”Advances in Neural Information Processing Systems, vol. 38, pp. 141 266–141 286, 2026
2026
-
[5]
On memorization in diffusion models,
X. Gu, C. Du, T. Pang, C. Li, M. Lin, and Y . Wang, “On memorization in diffusion models,”Transactions on Machine Learning Research, 2025
2025
-
[6]
Memorization and regularization in generative diffusion models,
R. Baptista, A. Dasgupta, N. B. Kovachki, A. Oberai, and A. M. Stuart, “Memorization and regularization in generative diffusion models,”arXiv preprint arXiv:2501.15785, 2025
Pith/arXiv arXiv 2025
-
[7]
3D diffusion policy: Generalizable visuomotor policy learning via simple 3D representations,
Y . Ze, G. Zhang, K. Zhang, C. Hu, M. Wang, and H. Xu, “3D diffusion policy: Generalizable visuomotor policy learning via simple 3D representations,” inRobotics: Science and Systems, 2024
2024
-
[8]
3D diffuser actor: Policy diffusion with 3D scene representations,
T.-W. Ke, N. Gkanatsios, and K. Fragkiadaki, “3D diffuser actor: Policy diffusion with 3D scene representations,” inConference on Robot Learning, 2025, pp. 1949–1974
2025
-
[9]
Chaineddiffuser: Unifying trajectory diffusion and keypose prediction for robotic manipulation,
Z. Xian, N. Gkanatsios, T. Gervet, T.-W. Ke, and K. Fragkiadaki, “Chaineddiffuser: Unifying trajectory diffusion and keypose prediction for robotic manipulation,” inConference on Robot Learning, 2023
2023
-
[10]
Goal-conditioned imitation learning using score-based diffusion policies,
M. Reuss, M. Li, X. Jia, and R. Lioutikov, “Goal-conditioned imitation learning using score-based diffusion policies,” inRobotics: Science and Systems, 2023
2023
-
[11]
Octo: An open-source generalist robot policy,
O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xuet al., “Octo: An open-source generalist robot policy,” inRobotics: Science and Systems, 2024
2024
-
[12]
User-friendly introduction to pac-bayes bounds,
P. Alquier, “User-friendly introduction to pac-bayes bounds,”Founda- tions and Trends in Machine Learning, vol. 17, no. 2, pp. 174–303, 2024
2024
-
[13]
Denoising diffusion probabilistic models,
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems, vol. 33, pp. 6840– 6851, 2020
2020
-
[14]
Tighter risk certificates for neural networks,
M. P ´erez-Ortiz, O. Rivasplata, J. Shawe-Taylor, and C. Szepesv ´ari, “Tighter risk certificates for neural networks,”Journal of Machine Learning Research, vol. 22, no. 227, pp. 1–40, 2021
2021
-
[15]
The franka emika robot: a standard platform in robotics research,
S. Haddadin, “The franka emika robot: a standard platform in robotics research,”IEEE Robotics & Automation Magazine, vol. 31, no. 4, pp. 136–148, 2024
2024
-
[16]
Openvla: An open- source vision-language-action model,
M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketiet al., “Openvla: An open- source vision-language-action model,” inConference on Robot Learning, 2025, pp. 2679–2713
2025
-
[17]
General- ization of diffusion models arises with a balanced representation space,
Z. Zhang, X. Li, X. Li, L. Shi, M. Wu, M. Tao, and Q. Qu, “General- ization of diffusion models arises with a balanced representation space,” inInternational Conference on Learning Representations, 2026
2026
-
[18]
Detecting, explaining, and mit- igating memorization in diffusion models,
Y . Wen, Y . Liu, C. Chen, and L. Lyu, “Detecting, explaining, and mit- igating memorization in diffusion models,” inInternational Conference on Learning Representations, 2024
2024
-
[19]
Tighter pac-bayes generalisation bounds by leveraging example difficulty,
F. Biggs and B. Guedj, “Tighter pac-bayes generalisation bounds by leveraging example difficulty,” inInternational conference on artificial intelligence and statistics, 2023, pp. 8165–8182
2023
-
[20]
Pac-bayes control: Synthesizing con- trollers that provably generalize to novel environments,
A. Majumdar and M. Goldstein, “Pac-bayes control: Synthesizing con- trollers that provably generalize to novel environments,” inConference on robot learning, 2018, pp. 293–305
2018
-
[21]
Pac-bayes control: learning poli- cies that provably generalize to novel environments,
A. Majumdar, A. Farid, and A. Sonar, “Pac-bayes control: learning poli- cies that provably generalize to novel environments,”The International Journal of Robotics Research, vol. 40, no. 2-3, pp. 574–593, 2021
2021
-
[22]
Probably approximately correct vision-based planning using motion primitives,
S. Veer and A. Majumdar, “Probably approximately correct vision-based planning using motion primitives,” inConference on Robot Learning, 2021, pp. 1001–1014
2021
-
[23]
Generalization guarantees for imitation learning,
A. Ren, S. Veer, and A. Majumdar, “Generalization guarantees for imitation learning,” inConference on Robot Learning, 2021, pp. 1426– 1442
2021
-
[24]
A pac-bayesian framework for optimal control with stability guarantees,
M. G. Boroujeni, C. L. Galimberti, A. Krause, and G. Ferrari-Trecate, “A pac-bayesian framework for optimal control with stability guarantees,” inIEEE Conference on Decision and Control, 2024, pp. 8237–8244
2024
-
[25]
Denoising diffusion implicit models,
J. Song, C. Meng, and S. Ermon, “Denoising diffusion implicit models,” inInternational Conference on Learning Representations, 2021
2021
-
[26]
Pac-bayesian generalisation error bounds for gaussian process classification,
M. Seeger, “Pac-bayesian generalisation error bounds for gaussian process classification,”Journal of machine learning research, vol. 3, no. 10, pp. 233–269, 2002
2002
-
[27]
Machine learning. the science of selection under uncertainty,
Y . Seldin, “Machine learning. the science of selection under uncertainty,” arXiv preprint arXiv:2509.21547, 2025
arXiv 2025
-
[28]
Weight uncertainty in neural network,
C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra, “Weight uncertainty in neural network,” inInternational conference on machine learning. PMLR, 2015, pp. 1613–1622
2015
-
[29]
Auto-encoding variational bayes,
D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in International Conference on Learning Representations (ICLR), 2014
2014
-
[30]
Implicit behavioral cloning,
P. Florence, C. Lynch, A. Zeng, O. A. Ramirez, A. Wahid, L. Downs, A. Wong, J. Lee, I. Mordatch, and J. Tompson, “Implicit behavioral cloning,” inConference on robot learning, 2022, pp. 158–168
2022
-
[31]
What matters in learning from offline human demonstrations for robot manipulation,
A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y . Zhu, and R. Mart ´ın-Mart´ın, “What matters in learning from offline human demonstrations for robot manipulation,” inConference on Robot Learning, 2022, pp. 1678–1690
2022
-
[32]
Information-theoretic diffusion,
X. Kong, R. Brekelmans, and G. V . Steeg, “Information-theoretic diffusion,” inInternational Conference on Learning Representations, 2023
2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.