pith. sign in

arxiv: 2605.21489 · v2 · pith:THBHO7MOnew · submitted 2026-05-20 · 💻 cs.LG · cs.AI· cs.CV· stat.CO· stat.ML

Variance Reduction for Expectations with Diffusion Teachers

Jesse Bettencourt , Xindi Wu , Matan Atzmon , James Lucas , Jonathan Lorraine This is my paper

Pith reviewed 2026-05-25 05:39 UTC · model grok-4.3

classification 💻 cs.LG cs.AIcs.CVstat.COstat.ML
keywords variance reductionmonte carlo estimationdiffusion modelsimportance samplingstratified samplingdistillationtext-to-3D
0
0 comments X

The pith

CARV amortizes expensive diffusion teacher computations over multiple noise samples to cut Monte Carlo variance.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Pretrained diffusion models act as frozen teachers whose gradients appear as Monte Carlo expectations over noise levels and samples. Each sample triggers costly upstream work such as rendering or encoding, so estimator variance drives most of the compute budget. The paper introduces CARV, a compute-aware variance-accounting framework whose hierarchical estimator reuses the expensive upstream result across many cheap noise resamples while adding timestep importance sampling and a stratified inverse-CDF construction. Experiments on text-to-3D distillation and attribution show 2-3x effective compute multipliers, most from amortization and roughly one-quarter from the sampling refinements, all without changing the target objective. In single-step distillation the same changes reduce gradient variance by an order of magnitude yet leave downstream FID unchanged, indicating the point at which Monte Carlo variance stops being the limiting factor.

Core claim

CARV supplies a hierarchical Monte Carlo estimator that amortizes the costly upstream computation (rendering, simulation, encoding) across multiple cheap diffusion-noise resamples, sharpened by timestep importance sampling and stratified-inverse-CDF sampling. The construction preserves the exact expectation required by the downstream pipeline while lowering estimator variance.

What carries the argument

CARV, the compute-aware variance-accounting framework that motivates the hierarchical MC estimator with amortization over noise resamples plus importance sampling and stratification.

If this is right

  • Text-to-3D distillation and attribution pipelines obtain 2-3x effective compute multipliers.
  • Single-step distillation sees gradient variance reduced by roughly an order of magnitude.
  • The target objective remains exactly the same; only the estimator changes.
  • In some regimes Monte Carlo variance ceases to be the dominant bottleneck once the proposed reductions are applied.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same amortization pattern could be applied to any pipeline that repeatedly evaluates an expensive forward map before adding cheap stochastic perturbations.
  • When variance reduction no longer improves final metrics, attention should shift to other sources of error such as optimization dynamics or model capacity.
  • The importance-sampling and stratification components can be tuned independently of the amortization layer, allowing incremental adoption.

Load-bearing premise

The expensive upstream computation can be amortized over multiple cheap diffusion-noise resamples while preserving the exact Monte Carlo expectation that the downstream pipeline requires.

What would settle it

A side-by-side comparison in which the hierarchical estimator and the standard Monte Carlo estimator produce statistically different mean values on the same downstream objective would show that the amortization step alters the expectation.

Figures

Figures reproduced from arXiv: 2605.21489 by James Lucas, Jesse Bettencourt, Jonathan Lorraine, Matan Atzmon, Xindi Wu.

Figure 1
Figure 1. Figure 1: Importance sampling for timestep allocation: [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Stratified Sampling Visualization: We show 3 realizations/batches of 8 timestep samples for both IID and stratified sampling. Notably, the stratified method creates bins for each sample and requires each batch to contain one sample from each bin, often result￾ing in lower-variance estimators. 2.3 Diffusion Model Applications 2.3.1 Diffusion Priors for Optimization Score Distillation Sampling (SDS) uses a f… view at source ↗
Figure 3
Figure 3. Figure 3: Compute Re-use Visualization: Compu￾tational graph comparing baseline (left, K = 1) and our re-noising (right, K > 1). Both take θ (e.g., NeRF weights or generator), render, encode, noise, de￾noise, combine into a residual, and backpropagate. Re￾noising helps when (a) (t, ϵ) drives variance and (b) de￾noising is cheaper than rendering. From [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Combining Stratified Sampling with Im￾portance Weighting: We illustrate how to use inverse￾transform sampling to map a stratified sample uni￾formly in [0, 1] (see [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Quantifying variance reduction from IW and stratification (SDS). Top: Variance (tr(Cov(∇θ)) late in training) vs. compute. Colors: uniform baseline and IW+Strat. Points annotated by (R, K). Bottom: Effective compute multiplier vs. uniform baseline. Lines trace (R = 1, K), peaking at (1, 8): ∼2.6× (uniform), ∼3.3× (IW+Strat). Ab￾lations in App [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Performance Gains from Vari￾ance Reduction: CLIP score versus op￾timization iteration, averaged across 30 prompts, 3 seeds, and multiple views (± std. dev.). Equal per-iteration cost (∼ 300 − 400ms/iter, App. Sec. D.1.1), so the iteration axis is wall-clock up to a known constant: baseline vs. ours (stratified+IS+re-noising). Higher CLIP at fixed iteration count from lower per-iteration variance ( [PITH_F… view at source ↗
Figure 9
Figure 9. Figure 9: Geometric intuition for efficiency metrics: [PITH_FULL_IMAGE:figures/full_fig_p024_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Variance reduction with Monte-Carlo seed error bars (single SDS prompt). [PITH_FULL_IMAGE:figures/full_fig_p029_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Quantifying variance reduction from hierarchical cost awareness with importance weighting [PITH_FULL_IMAGE:figures/full_fig_p029_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative Results from Variance Reduction: [PITH_FULL_IMAGE:figures/full_fig_p030_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Variance reduction across training, low classifier-free guidance ( [PITH_FULL_IMAGE:figures/full_fig_p031_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Variance reduction measured via latent-space residual norm. [PITH_FULL_IMAGE:figures/full_fig_p032_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Cosine similarity to ground-truth gradient versus compute budget. [PITH_FULL_IMAGE:figures/full_fig_p032_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Variance reduction in the low guidance regime ( [PITH_FULL_IMAGE:figures/full_fig_p033_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Performance gains from variance reduction at low guidance ( [PITH_FULL_IMAGE:figures/full_fig_p033_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Qualitative SDS trajectories at low classifier-free guidance ( [PITH_FULL_IMAGE:figures/full_fig_p034_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Qualitative SDS trajectories at low classifier-free guidance ( [PITH_FULL_IMAGE:figures/full_fig_p034_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Pair probability matrices Q˜(i, j) for N = 2 sampling strategies, computed on gradient data from a single SDS prompt at the end of training. Each panel shows the probability of selecting pair (i, j) on a log scale (brighter = higher probability, gray = zero). (a) IID places equal mass on all pairs (1.00×, baseline). (b) Index-based stratification concentrates mass in off-diagonal blocks. (c) Importance we… view at source ↗
Figure 21
Figure 21. Figure 21: Sensitivity of variance reduction to render-vs-denoise cost ratio. [PITH_FULL_IMAGE:figures/full_fig_p036_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Weight function closely tracks gradient magnitude across timesteps. [PITH_FULL_IMAGE:figures/full_fig_p037_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Importance Sampling Strategy Comparison: Weight-Based Heuristic versus Oracle. [PITH_FULL_IMAGE:figures/full_fig_p037_23.png] view at source ↗
Figure 24
Figure 24. Figure 24: Comparing Per-Render and Global Stratification Strategies. [PITH_FULL_IMAGE:figures/full_fig_p038_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Quantifying variance reduction against compute cost for one-step distillation. [PITH_FULL_IMAGE:figures/full_fig_p039_25.png] view at source ↗
Figure 26
Figure 26. Figure 26: FID convergence during DMD training for student-step resampling. [PITH_FULL_IMAGE:figures/full_fig_p040_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Best FID achieved during training for fake-score-step resampling strategies. [PITH_FULL_IMAGE:figures/full_fig_p040_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: (Extended) Quantifying Changes in Data Attribution: [PITH_FULL_IMAGE:figures/full_fig_p042_28.png] view at source ↗
Figure 29
Figure 29. Figure 29: Is there an improvement from importance sampling for data attribution? [PITH_FULL_IMAGE:figures/full_fig_p042_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: Example Videos for Attribution: We show assorted clips from VIDGEN-1M [78] used for our video data attribution experiments, where the influence is being calculated for Wan2.1-T2V-1.3B [81] Sora [5], CogVideoX [92], and Wan [81]. Diffusion transformers (DiT) [57] and related architec￾tures scaled these models with transformer backbones. We treat pretrained teachers as given and target gradient-estimator va… view at source ↗
read the original abstract

Pretrained diffusion models serve as frozen teachers feeding downstream pipelines such as text-to-3D, single-step distillation, and data attribution. The teacher gradients these pipelines consume are Monte Carlo (MC) expectations over noise levels and Gaussian noise samples; their estimator variance dominates compute cost because each draw requires expensive upstream work (rendering, simulation, encoding). We introduce CARV, a compute-aware variance-accounting framework that motivates a hierarchical MC estimator: amortize the expensive upstream computation over cheap diffusion-noise resamples, sharpened by timestep importance sampling and a stratified-inverse-CDF construction. In our text-to-3D distillation and attribution experiments, CARV delivers 2-3x effective compute multipliers (most from amortized reuse; ~25% additional from IS+stratification) without changing the objective; in single-step distillation, the same techniques cut gradient variance by an order of magnitude but do not improve downstream FID, marking the regime where MC variance is no longer the bottleneck.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes CARV, a compute-aware variance-accounting framework for reducing variance in Monte Carlo expectations over diffusion noise levels and samples when pretrained diffusion models serve as frozen teachers for downstream pipelines (text-to-3D distillation, single-step distillation, data attribution). It introduces a hierarchical MC estimator that amortizes expensive upstream computations (rendering, simulation, encoding) over multiple cheap noise resamples, sharpened by timestep importance sampling and a stratified-inverse-CDF construction. Experiments report 2-3x effective compute multipliers (mostly from amortization, ~25% from IS+stratification) without changing the objective, plus an order-of-magnitude gradient variance reduction in single-step distillation (without FID gains).

Significance. If the estimator remains exactly unbiased, the amortization approach could yield substantial practical efficiency gains in diffusion-teacher pipelines by reusing upstream work across noise samples. The reported empirical multipliers and variance reductions provide concrete, task-specific evidence of utility in text-to-3D and attribution settings, and the observation that variance reduction does not always translate to better FID usefully delineates when MC variance ceases to be the bottleneck.

major comments (2)
  1. [Abstract] Abstract: the central claim that the hierarchical MC estimator 'delivers 2-3x effective compute multipliers ... without changing the objective' requires that amortizing upstream computation over noise resamples preserves the exact original expectation. No derivation is supplied showing that the upstream function factors out of the integral over noise in a measure-preserving way (or that upstream and noise are unentangled), which is load-bearing for the unbiasedness assertion.
  2. [Experiments] Experiments (text-to-3D and attribution results): the reported 2-3x multipliers and ~25% additional gain from IS+stratification are presented without error bars, explicit baseline definitions, or details on how effective compute is measured, undermining assessment of whether the gains are statistically reliable or reproducible.
minor comments (2)
  1. The single-step distillation experiment notes an order-of-magnitude variance cut but no FID improvement; the manuscript would benefit from a brief discussion of why downstream performance is unaffected (e.g., other bottlenecks).
  2. Notation for the stratified-inverse-CDF construction and the precise form of the hierarchical estimator could be introduced with explicit equations early in the methods section for clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive report. We address each major comment below and will incorporate clarifications and additions in the revised manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the central claim that the hierarchical MC estimator 'delivers 2-3x effective compute multipliers ... without changing the objective' requires that amortizing upstream computation over noise resamples preserves the exact original expectation. No derivation is supplied showing that the upstream function factors out of the integral over noise in a measure-preserving way (or that upstream and noise are unentangled), which is load-bearing for the unbiasedness assertion.

    Authors: We agree an explicit derivation strengthens the presentation. The upstream computation (rendering, simulation, or encoding) operates on clean inputs or model parameters and is independent of the diffusion timestep and noise sample; it therefore factors out of the outer expectation over the noise measure. In the revision we will add a short formal derivation in Section 3 establishing that the hierarchical estimator remains exactly unbiased under this independence. revision: yes

  2. Referee: [Experiments] Experiments (text-to-3D and attribution results): the reported 2-3x multipliers and ~25% additional gain from IS+stratification are presented without error bars, explicit baseline definitions, or details on how effective compute is measured, undermining assessment of whether the gains are statistically reliable or reproducible.

    Authors: We will strengthen the experimental reporting. The revision will include error bars from at least five independent runs, explicitly define the baseline as the standard single-sample Monte Carlo estimator without amortization or importance sampling, and specify that effective compute is measured as the ratio of wall-clock time (or equivalent FLOPs) needed to reach a target variance level. revision: yes

Circularity Check

0 steps flagged

No circularity; results are empirical measurements of variance reduction

full rationale

The paper presents CARV as a hierarchical MC estimator whose gains are measured directly in text-to-3D, attribution, and distillation experiments. The abstract states that the techniques deliver 2-3x multipliers 'without changing the objective' and that variance is cut 'by an order of magnitude' in one regime, but these are reported as observed outcomes rather than quantities derived from fitted parameters or self-referential definitions. No equations, uniqueness theorems, or ansatzes are shown that reduce the claimed multipliers to inputs by construction. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The framework rests on the standard assumption that Monte Carlo expectations over diffusion noise can be rewritten as hierarchical estimators without bias; no free parameters or invented entities are described in the abstract.

axioms (1)
  • domain assumption Monte Carlo expectations over noise levels and Gaussian samples can be estimated with variance reduction while preserving the original objective.
    Invoked to justify the hierarchical estimator and amortization without changing the target expectation.

pith-pipeline@v0.9.0 · 5711 in / 1226 out tokens · 18947 ms · 2026-05-25T05:39:10.942365+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

111 extracted references · 111 canonical work pages · 10 internal anchors

  1. [1]

    Training data attribution via approximate unrolled differentiation

    Juhan Bae, Wu Lin, Jonathan Lorraine, and Roger Grosse. Training data attribution via approximate unrolled differentiation. InAdvances in Neural Information Processing Systems, volume 37, 2024. URLhttps://arxiv.org/abs/2405.12186. 44

    work page arXiv 2024

  2. [2]

    Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, and David B. Lindell. 4D-fy: Text-to-4d generation using hybrid score distillation sampling. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7996–8006,

    work page

  3. [3]

    Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting.ACM Transactions on Graphics (Proc

    Benedikt Bitterli, Chris Wyman, Matt Pharr, Peter Shirley, Aaron Lefohn, and Wojciech Jarosz. Spatiotemporal reservoir resampling for real-time ray tracing with dynamic direct lighting.ACM Transactions on Graphics (Proc. SIGGRAPH), 39(4):148:1–148:16, 2020. doi: 10.1145/3386569.3392481. 46

    work page doi:10.1145/3386569.3392481 2020

  4. [4]

    Montrage: Monitoring training for attribution of generative diffusion models

    Jonathan Brokman, Omer Hofman, Roman Vainshtein, Amit Giloni, Toshiya Shimizu, Inder- jeet Singh, Oren Rachmil, Alon Zolfi, Asaf Shabtai, Yuki Unno, et al. Montrage: Monitoring training for attribution of generative diffusion models. InEuropean Conference on Computer Vision, pages 1–17. Springer, 2024. 44

    work page 2024

  5. [5]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. OpenAI technical report.https://openai. com/research/video-generation-models-as-world-simulators, 2024. 43

    work page 2024

  6. [6]

    Fantasia3D: Disentangling geometry and appearance for high-quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fantasia3D: Disentangling geometry and appearance for high-quality text-to-3d content creation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 22246–22256, 2023. 43

    work page 2023

  7. [7]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 44(10-11):1684–1704, 2025. 44

    work page 2025

  8. [8]

    Perception prioritized training of diffusion models

    Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. URLhttps://arxiv.org/ abs/2204.00227. 43

    work page arXiv 2022

  9. [9]

    Sinkhorn distances: Lightspeed computation of optimal transport

    Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. InAdvances in Neural Information Processing Systems, volume 26, pages 2292–2300, 2013. 33

    work page 2013

  10. [10]

    FlashTex: Fast relightable mesh texturing with LightCon- trolNet

    Kangle Deng, Timothy Omernick, Alexander Weiss, Deva Ramanan, Jun-Yan Zhu, Tinghui Zhou, and Maneesh Agrawala. FlashTex: Fast relightable mesh texturing with LightCon- trolNet. InEuropean Conference on Computer Vision, pages 90–107. Springer, 2024. 3, 44

    work page 2024

  11. [11]

    Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons

    Zach Evans, Julian D. Parker, CJ Carr, Zack Zukowski, Josiah Taylor, and Jordi Pons. Stable audio open. InICASSP 2025 – 2025 IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1–5. IEEE, 2025. 42

    work page 2025

  12. [12]

    The journey, not the destination: How data guides diffusion models.arXiv preprint arXiv:2312.06205, 2023

    Kristian Georgiev, Joshua Vendrow, Hadi Salman, Sung Min Park, and Aleksander Madry. The journey, not the destination: How data guides diffusion models.arXiv preprint arXiv:2312.06205, 2023. 44

    work page arXiv 2023

  13. [13]

    threestudio: A unified framework for 3D content generation.https://github.com/ threestudio-project/threestudio, 2023

    Yuan-Chen Guo, Ying-Tian Liu, Ruizhi Shao, Christian Laforte, Vikram V oleti, Guan Luo, Chia-Hao Chen, Zi-Xin Zou, Chen Wang, Yan-Pei Cao, and Song-Hai Zhang. threestudio: A unified framework for 3D content generation.https://github.com/ threestudio-project/threestudio, 2023. 7, 17, 26, 47 10

    work page 2023

  14. [14]

    Efficient diffusion training via Min-SNR weighting strategy

    Tiankai Hang, Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, and Baining Guo. Efficient diffusion training via Min-SNR weighting strategy. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 7441–7451, 2023. 43, 46

    work page 2023

  15. [15]

    CLIPScore: A reference-free evaluation metric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: A reference-free evaluation metric for image captioning. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514–7528, 2021. 7

    work page 2021

  16. [16]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pages 6840–6851, 2020. 2, 42, 43

    work page 2020

  17. [17]

    simple diffusion: End-to-end diffu- sion for high resolution images

    Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: End-to-end diffu- sion for high resolution images. InInternational Conference on Machine Learning, volume 202, pages 13213–13232. PMLR, 2023. 43

    work page 2023

  18. [18]

    JacNet: Learning functions with structured jacobian

    Safwan Hossain and Jonathan Lorraine. JacNet: Learning functions with structured jacobian. InFirst Workshop on Invertible Neural Nets and Normalizing Flows (INNF), ICML, 2019. 47

    work page 2019

  19. [19]

    Planning with Diffusion for Flexible Behavior Synthesis

    Michael Janner, Yilun Du, Joshua B. Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. InInternational Conference on Machine Learning, volume 162, pages 9902–9915. PMLR, 2022. URLhttps://arxiv.org/abs/2205.09991. 44

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  20. [20]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. InAdvances in Neural Information Processing Systems, volume 35, pages 26565–26577, 2022. 43

    work page 2022

  21. [21]

    Analyzing and improving the training dynamics of diffusion models

    Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. URLhttps: //arxiv.org/abs/2312.02696. 43

    work page arXiv 2024

  22. [22]

    Denoising diffusion restora- tion models

    Bahjat Kawar, Michael Elad, Stefano Ermon, and Jiaming Song. Denoising diffusion restora- tion models. InAdvances in Neural Information Processing Systems, volume 35, pages 23593–23606, 2022. 44

    work page 2022

  23. [23]

    Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation

    Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, and Il-Chul Moon. Soft truncation: A universal training technique of score-based diffusion model for high precision score estimation. InInternational Conference on Machine Learning, volume 162, pages 11201–11228. PMLR, 2022. URLhttps://arxiv.org/abs/2106.05527. 43

    work page arXiv 2022

  24. [24]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. InInter- national Conference on Learning Representations, 2015. 26

    work page 2015

  25. [25]

    Kingma, Tim Salimans, Ben Poole, and Jonathan Ho

    Diederik P. Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. InAdvances in Neural Information Processing Systems, volume 34, 2021. URL https://arxiv.org/abs/2107.00630. 2, 3, 19, 43, 46

    work page arXiv 2021

  26. [26]

    Understanding black-box predictions via influence functions

    Pang Wei Koh and Percy Liang. Understanding black-box predictions via influence functions. InInternational conference on machine learning, pages 1885–1894. PMLR, 2017. 4, 22, 44

    work page 2017

  27. [27]

    Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. In F. Pereira, C.J. Burges, L. Bottou, and K.Q. Weinberger, editors,Advances in Neural Information Processing Systems, volume 25, pages 1097–1105. Curran Associates, Inc., 2012. URLhttps://proceedings.neurips.cc/paper_files/ paper/2012...

    work page 2012

  28. [28]

    DataInf: Efficiently estimating data in- fluence in LoRA-tuned LLMs and diffusion models

    Yongchan Kwon, Eric Wu, Kevin Wu, and James Zou. DataInf: Efficiently estimating data in- fluence in LoRA-tuned LLMs and diffusion models. InInternational Conference on Learning Representations, 2024. URLhttps://arxiv.org/abs/2310.00902. 44 11

    work page arXiv 2024

  29. [29]

    Magic3D: High-resolution text-to-3d content creation

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3D: High-resolution text-to-3d content creation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 300–309, 2023. 7, 43

    work page 2023

  30. [30]

    Generalized resampled importance sampling: Foundations of ReSTIR.ACM Trans- actions on Graphics (Proc

    Daqi Lin, Markus Kettunen, Benedikt Bitterli, Jacopo Pantaleoni, Cem Yuksel, and Chris Wyman. Generalized resampled importance sampling: Foundations of ReSTIR.ACM Trans- actions on Graphics (Proc. SIGGRAPH), 41(4):75:1–75:23, 2022. doi: 10.1145/3528223. 3530158. 46

    work page doi:10.1145/3528223 2022

  31. [31]

    Diffusion attribution score: Evalu- ating training data influence in diffusion models

    Jinxu Lin, Linwei Tao, Minjing Dong, and Chang Xu. Diffusion attribution score: Evalu- ating training data influence in diffusion models. InInternational Conference on Learning Representations, 2025. URLhttps://arxiv.org/abs/2410.18639. 44

    work page arXiv 2025

  32. [32]

    Align your Gaussians: Text-to-4d with dynamic 3D Gaussians and composed diffusion models

    Huan Ling, Seung Wook Kim, Antonio Torralba, Sanja Fidler, and Karsten Kreis. Align your Gaussians: Text-to-4d with dynamic 3D Gaussians and composed diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8576–8588, 2024. 44

    work page 2024

  33. [33]

    Physics3D: Learning physical properties of 3D gaussians via video diffusion.arXiv preprint arXiv:2406.04338, 2024

    Fangfu Liu, Hanyang Wang, Shunyu Yao, Shengjun Zhang, Jie Zhou, and Yueqi Duan. Physics3D: Learning physical properties of 3D gaussians via video diffusion.arXiv preprint arXiv:2406.04338, 2024. 3, 44

    work page arXiv 2024

  34. [34]

    DARTS: Differentiable architecture search

    Hanxiao Liu, Karen Simonyan, and Yiming Yang. DARTS: Differentiable architecture search. InInternational Conference on Learning Representations, 2019. URLhttps: //openreview.net/forum?id=S1eYHoC5FX. 47

    work page 2019

  35. [35]

    PhD thesis, University of Toronto, 2024

    Jonathan Lorraine.Scalable Nested Optimization for Deep Learning. PhD thesis, University of Toronto, 2024. 44, 47

    work page 2024

  36. [36]

    Task selection for automl system evaluation.arXiv preprint arXiv:2208.12754,

    Jonathan Lorraine, Nihesh Anderson, Chansoo Lee, Quentin De Laroussilhe, and Mehadi Hassen. Task selection for automl system evaluation.arXiv preprint arXiv:2208.12754,

    work page arXiv

  37. [37]

    Lyapunov exponents for diversity in differentiable games

    Jonathan Lorraine, Paul Vicol, Jack Parker-Holder, Tal Kachman, Luke Metz, and Jakob Fo- erster. Lyapunov exponents for diversity in differentiable games. InInternational Conference on Autonomous Agents and Multiagent Systems, pages 842–852, 2022. 47

    work page 2022

  38. [38]

    ATT3D: Amortized text-to-3d object synthesis

    Jonathan Lorraine, Kevin Xie, Xiaohui Zeng, Chen-Hsuan Lin, Towaki Takikawa, Nicholas Sharp, Tsung-Yi Lin, Ming-Yu Liu, Sanja Fidler, and James Lucas. ATT3D: Amortized text-to-3d object synthesis. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 17946–17956, 2023. 44, 45, 46, 47

    work page 2023

  39. [39]

    Lorraine, David Acuna, Paul Vicol, and David Duvenaud

    Jonathan P. Lorraine, David Acuna, Paul Vicol, and David Duvenaud. Complex momentum for optimization in games. InInternational Conference on Artificial Intelligence and Statis- tics, volume 151, pages 7742–7765. PMLR, 2022. 47

    work page 2022

  40. [40]

    RePaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. RePaint: Inpainting using denoising diffusion probabilistic models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11461– 11471, 2022. 44

    work page 2022

  41. [41]

    Artem Lukoianov, Haitz S ´aez de Oc´ariz Borde, Kristjan Greenewald, Vitor Guizilini, Timur Bagautdinov, Vincent Sitzmann, and Justin M. Solomon. Score distillation via reparametrized DDIM. InAdvances in Neural Information Processing Systems, volume 37, pages 26011– 26044, 2024. 43, 45, 46

    work page 2024

  42. [42]

    Diff-Instruct: A universal approach for transferring knowledge from pre-trained diffusion models

    Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-Instruct: A universal approach for transferring knowledge from pre-trained diffusion models. InAdvances in Neural Information Processing Systems, volume 36, 2023. 44

    work page 2023

  43. [43]

    Scale- Dreamer: Scalable text-to-3d synthesis with asynchronous score distillation

    Zhiyuan Ma, Yuxiang Wei, Yabin Zhang, Xiangyu Zhu, Zhen Lei, and Lei Zhang. Scale- Dreamer: Scalable text-to-3d synthesis with asynchronous score distillation. InEuropean Conference on Computer Vision, pages 1–19. Springer, 2024. 43 12

    work page 2024

  44. [44]

    Gradient-based hyperparameter op- timization through reversible learning

    Dougal Maclaurin, David Duvenaud, and Ryan Adams. Gradient-based hyperparameter op- timization through reversible learning. InInternational Conference on Machine Learning, volume 37, pages 2113–2122. PMLR, 2015. 44, 47

    work page 2015

  45. [45]

    Jacobs, Alexei A

    David McAllister, Songwei Ge, Jia-Bin Huang, David W. Jacobs, Alexei A. Efros, Aleksander Holynski, and Angjoo Kanazawa. Rethinking score distillation as a bridge between image distributions. InAdvances in Neural Information Processing Systems, volume 37, 2024. URL https://arxiv.org/abs/2406.09417. 43

    work page arXiv 2024

  46. [46]

    Improving hyperparameter optimization with checkpointed model weights

    Nikhil Mehta, Jonathan Lorraine, Steve Masson, Ramanathan Arunachalam, Zaid Pervaiz Bhat, James Lucas, and Arun George Zachariah. Improving hyperparameter optimization with checkpointed model weights. InEuropean Conference on Computer Vision Workshop on Efficient Deep Learning for Foundation Models (EFM), pages 75–96, 2024. doi: 10.1007/ 978-3-031-91979-4 8. 47

    work page 2024

  47. [47]

    SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2022. URLhttps://arxiv.org/ abs/2108.01073. 44

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  48. [48]

    Influence functions for scalable data attribution in diffu- sion models

    Bruno Kacper Mlodozeniec, Runa Eschenhagen, Juhan Bae, Alexander Immer, David Krueger, and Richard E Turner. Influence functions for scalable data attribution in diffu- sion models. InThe Thirteenth International Conference on Learning Representations, 2025. 44

    work page 2025

  49. [49]

    T2I-Adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2I-Adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. InProceedings of the AAAI Conference on Artificial Intelligence, pages 4296–4304, 2024. 44

    work page 2024

  50. [50]

    Instant neural graph- ics primitives with a multiresolution hash encoding.ACM Transactions on Graphics (Proc

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graph- ics primitives with a multiresolution hash encoding.ACM Transactions on Graphics (Proc. SIGGRAPH), 41(4):102:1–102:15, 2022. doi: 10.1145/3528223.3530127. 26, 45, 47

    work page doi:10.1145/3528223.3530127 2022

  51. [51]

    SwiftBrush: One-step text-to-image diffusion model with variational score distillation

    Thuan Hoang Nguyen and Anh Tran. SwiftBrush: One-step text-to-image diffusion model with variational score distillation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7807–7816, 2024. 44

    work page 2024

  52. [52]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. InInternational Conference on Machine Learning, volume 139, pages 8162–8171. PMLR, 2021. 2, 18, 43

    work page 2021

  53. [53]

    NVIDIA FastGen: Fast generation from diffusion models.https://github.com/NVlabs/FastGen, 2026

    Weili Nie, Julius Berner, Chao Liu, and Arash Vahdat. NVIDIA FastGen: Fast generation from diffusion models.https://github.com/NVlabs/FastGen, 2026. 8, 17

    work page 2026

  54. [54]

    Art B. Owen. Monte carlo theory, methods and examples.https://artowen.su.domains/ mc/, 2013. 31, 43

    work page 2013

  55. [55]

    Trak: Attributing model behavior at scale

    Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry. Trak: Attributing model behavior at scale. InInternational Conference on Machine Learning, pages 27074–27113. PMLR, 2023. 4, 23, 41, 44

    work page 2023

  56. [56]

    Hyperparameter optimization with approximate gradient

    Fabian Pedregosa. Hyperparameter optimization with approximate gradient. InInternational Conference on Machine Learning, volume 48, pages 737–746. PMLR, 2016. 44, 47

    work page 2016

  57. [57]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceed- ings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023. 8, 37, 43, 50

    work page 2023

  58. [58]

    DreamFusion: Text-to-3D using 2D Diffusion

    Ben Poole, Ajay Jain, Jonathan T. Barron, and Ben Mildenhall. DreamFusion: Text-to-3d using 2D diffusion. InInternational Conference on Learning Representations, 2023. URL https://arxiv.org/abs/2209.14988. 3, 7, 43 13

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  59. [59]

    Estimating training data influence by tracing gradient descent

    Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. Estimating training data influence by tracing gradient descent. InAdvances in Neural Information Processing Systems, volume 33, pages 19920–19930, 2020. 4, 23, 44

    work page 2020

  60. [60]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InInter- national Conference on Machine Learning, pages 8748–8763. PMLR, 2021. 27

    work page 2021

  61. [61]

    DreamGaussian4D: Generative 4D gaussian splatting.arXiv preprint arXiv:2312.17142,

    Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, and Ziwei Liu. DreamGaussian4D: Generative 4D gaussian splatting.arXiv preprint arXiv:2312.17142,

    work page arXiv

  62. [62]

    Input convex gradient net- works

    Jack Richter-Powell, Jonathan Lorraine, and Brandon Amos. Input convex gradient net- works. InAdvances in Neural Information Processing Systems Optimal Transport and Ma- chine Learning Workshop, 2021. 47

    work page 2021

  63. [63]

    Score distillation sampling for audio: Source separation, synthesis, and beyond

    Jessie Richter-Powell, Antonio Torralba, and Jonathan Lorraine. Score distillation sampling for audio: Source separation, synthesis, and beyond. arXiv preprint arXiv:2505.04621, 2025. Presented at the ICML 2025 AI Heard That! Workshop on Machine Learning for Audio. 3, 44, 47

    work page arXiv 2025

  64. [64]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Om- mer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695,

    work page

  65. [65]

    Rubinstein and Dirk P

    Reuven Y . Rubinstein and Dirk P. Kroese.Simulation and the Monte Carlo Method. John Wiley & Sons, 3 edition, 2016. 2, 18, 43

    work page 2016

  66. [66]

    Align your flow: Scaling continuous- time flow map distillation

    Amirmojtaba Sabour, Sanja Fidler, and Karsten Kreis. Align your flow: Scaling continuous- time flow map distillation. InAdvances in Neural Information Processing Systems, 2025. URLhttps://arxiv.org/abs/2506.14603. 44

    work page arXiv 2025

  67. [67]

    Progressive Distillation for Fast Sampling of Diffusion Models

    Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. InInternational Conference on Learning Representations, 2022. URLhttps://arxiv. org/abs/2202.00512. 43

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  68. [68]

    Multistep distilla- tion of diffusion models via moment matching

    Tim Salimans, Thomas Mensink, Jonathan Heek, and Emiel Hoogeboom. Multistep distilla- tion of diffusion models via moment matching. InAdvances in Neural Information Processing Systems, volume 37, 2024. URLhttps://arxiv.org/abs/2406.04103. 43, 44

    work page arXiv 2024

  69. [69]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. InEuropean Conference on Computer Vision. Springer, 2024. URLhttps: //arxiv.org/abs/2311.17042. 44

    work page arXiv 2024

  70. [70]

    MVDream: Multi-view Diffusion for 3D Generation

    Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. MVDream: Multi-view diffusion for 3D generation. InInternational Conference on Learning Represen- tations, 2024. URLhttps://arxiv.org/abs/2308.16512. 43, 47

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  71. [71]

    Deep un- supervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep un- supervised learning using nonequilibrium thermodynamics. InInternational Conference on Machine Learning, volume 37, pages 2256–2265. PMLR, 2015. 42

    work page 2015

  72. [72]

    Improved Techniques for Training Consistency Models

    Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In International Conference on Learning Representations, 2024. URLhttps://arxiv.org/ abs/2310.14189. 44

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  73. [73]

    Generative modeling by estimating gradients of the data dis- tribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data dis- tribution. InAdvances in Neural Information Processing Systems, volume 32, pages 11918– 11930, 2019. 42 14

    work page 2019

  74. [74]

    Score-Based Generative Modeling through Stochastic Differential Equations

    Yang Song, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. InInternational Conference on Learning Representations, 2021. URLhttps://arxiv. org/abs/2011.13456. 42

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  75. [75]

    Solving inverse problems in med- ical imaging with score-based generative models

    Yang Song, Liyue Shen, Lei Xing, and Stefano Ermon. Solving inverse problems in med- ical imaging with score-based generative models. InInternational Conference on Learning Representations, 2022. URLhttps://arxiv.org/abs/2111.08005. 44

    work page arXiv 2022

  76. [76]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning, volume 202, pages 32211–32252. PMLR,

    work page

  77. [77]

    URLhttps://arxiv.org/abs/2303.01469. 44

    work page internal anchor Pith review Pith/arXiv arXiv

  78. [78]

    Multi-student diffusion distillation for better one-step generators.arXiv preprint arXiv:2410.23274, 2024

    Yanke Song, Jonathan Lorraine, Weili Nie, Karsten Kreis, and James Lucas. Multi-student diffusion distillation for better one-step generators.arXiv preprint arXiv:2410.23274, 2024. 44, 46, 47

    work page arXiv 2024

  79. [79]

    VidGen-1M: A large-scale dataset for text-to-video generation

    Zhiyu Tan, Xiaomeng Yang, Luozheng Qin, and Hao Li. VidGen-1M: A large-scale dataset for text-to-video generation. arXiv preprint arXiv:2408.02629, 2024. 8, 41, 43

    work page arXiv 2024

  80. [80]

    Mean-shift distillation for diffusion mode seeking

    Vikas Thamizharasan, Nikitas Chatzis, Iliyan Georgiev, Matthew Fisher, Difan Liu, Nanxuan Zhao, Evangelos Kalogerakis, and Michal Luk ´aˇc. Mean-shift distillation for diffusion mode seeking. InIEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2026. URLhttps://arxiv.org/abs/2502.15989. 46

    work page arXiv 2026

Showing first 80 references.