Pith. sign in

REVIEW 3 major objections 4 minor 54 references

Any unbiased Monte Carlo estimator with finite variance is a diffusion process: its SDE matches the reverse SDE of a variance-exploding diffusion, so rendered noise can be injected into a diffusion model at a matched timestep.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 21:15 UTC pith:XSHSERUG

load-bearing objection The paper's central claim that Monte Carlo estimators are exactly a diffusion SDE doesn't hold up, but the specular/diffuse variance analysis and the empirical recipe are worth a second look. the 3 major comments →

arxiv 2602.20725 v2 pith:XSHSERUG submitted 2026-02-24 cs.CV

Bridging Rendering and Generative Modeling with Monte Carlo Transport Scheduling

Junwei Shu , Wenjie Liu , Hantang Liu , Changbo Wang , Yang Li This is my paper
classification cs.CV
keywords Monte Carlo path tracingdiffusion modelsstochastic differential equationsvariance timenoise scheduledenoisingmaterial editingspecular dominance
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This work seeks to show that physically based rendering and diffusion-based generation are not just superficially similar but mathematically identical in their noisy-to-clean evolution. The authors derive a stochastic differential equation for Monte Carlo estimation under the Central Limit Theorem, choose a sample-count schedule that makes the noise vanish, and prove the resulting MC-SDE coincides with the reverse SDE of a clean-start variance-exploding forward process. That identification supplies a one-line mapping from render sample count to diffusion timestep, and a variance-ordering argument that specular content stabilizes later than diffuse content during denoising. If correct, it lets pretrained diffusion models denoise and edit low-sample path-traced images without specialized training, and gives a principled handle on material appearance.

Core claim

The central discovery is that a Monte Carlo estimator built from i.i.d. samples with mean µ and variance σ², viewed as a process indexed by the variance time τ (with τ→0 meaning infinitely many samples), converges under the CLT to the SDE dY(τ) = (µ−Y(τ)) N′(τ)/N(τ) dτ + σ√(|N′(τ)|)/N(τ) dW. With the schedule N(τ)=τ^{-2}, this becomes dY(τ)=2(Y(τ)−µ)/τ dτ + σ√(2τ) dW. The paper shows this is exactly the reverse-time SDE of the drift-free VE forward process dX=σ√(2τ)dW, X(0)=µ, whose marginal variance is σ²τ², matching the CLT variance. Hence the renderer's variance time and a diffusion model's timestep can be aligned by log-SNR matching, and the ordering of per-component variances (specular

What carries the argument

The load-bearing object is the MC-SDE (Eq. 24): dY(τ)=2(Y(τ)−µ)/τ dτ + σ√(2τ) dW, obtained by choosing N(τ)=τ^{-2} in the general CLT limit. Its equivalence to the reverse SDE of the clean-start VE forward process dX=σ√(2τ)dW, X(0)=µ, is the bridge: it makes the renderer's sample-count axis isomorphic to the diffusion model's noise axis, enabling the τ↔t mapping and the specular-later ordering t_spec ≥ t_diff.

Load-bearing premise

The paper assumes that the step-by-step change in the Monte Carlo estimate behaves like a Gaussian diffusion with independent increments, so the whole discrete averaging process can be replaced by a well-behaved SDE—a convergence the paper asserts rather than proves.

What would settle it

Render a simple scene with known mean and variance, record the estimator's increments over small Δτ under N(τ)=τ^{-2}, and test whether their standardized distribution is approximately Gaussian with covariance 2τΣ and whether the marginal variance tracks σ²τ²; a systematic deviation would falsify the MC-SDE approximation. Also, checking whether the drift integral over [0,τ₀] diverges as τ₀→0 would test the well-posedness near the clean limit.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Low-sample path-traced images can be mapped to a diffusion model's noise level and then denoised by the pretrained model, producing clean renders with better structure than feeding the raw noisy image.
  • The mapping from sample count to timestep is explicit: τ=N^{-1/2} paired with log-SNR alignment gives a one-line conversion usable with any VE/VP diffusion schedule.
  • Because specular variance dominates diffuse variance, specular content stabilizes later in the diffusion trajectory; a simple attention modulation based on this ordering yields fine-grained, reversible material editing.
  • The method works without retraining the diffusion model: a training-free τMapper already improves results, and a lightweight time-conditioned adapter further improves distribution match and quantitative quality.
  • The variance-based analysis extends the physics of path tracing to the geometry of diffusion sampling, suggesting that renderer-side scene statistics can predict where in the denoising trajectory a given visual feature will appear.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the SDE equivalence holds, a natural next step is an instance-specific mapper t(N, x_N) that adapts the alignment to each render's measured noise statistics, as the authors note when discussing the fixed constant A.
  • The variance-ordering argument likely generalizes beyond diffuse/specular to any transport component with high- versus low-variance path contributions (e.g., caustics vs. indirect soft light), offering new handles for inverse rendering and relighting.
  • The derivation assumes i.i.d. sampling without importance sampling; extending the MC-SDE to MIS and guided sampling would test whether the clean-start VE form survives real production path tracers.
  • A continuous-time training objective that learns from real render endpoints rather than synthetic interpolants could change how diffusion models are fine-tuned for image formation, a direction the paper proposes but does not explore.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a unified stochastic formulation connecting Monte Carlo path tracing and diffusion-based generative models. The authors derive a 'Monte Carlo SDE' (MC-SDE) by taking a continuous-time limit of the sample-mean estimator under the CLT (Sec. 3.1), identify it with the reverse SDE of a clean-start VE forward process (Sec. 3.3), and align the two processes by matching log-SNR, yielding a mapping from sample count to diffusion timestep. This mapping is used to inject low-SPP path-traced images into pretrained diffusion models, and the variance ordering is used to justify material-editing via attention modulation. Experiments include render refinement (Table 1) and material tuning (Figs. 4-6), with supplementary derivations and implementation details.

Significance. If the central SDE derivation were valid, the paper would offer a principled continuous-time bridge between Monte Carlo rendering and diffusion models, with practical implications for low-spp denoising and physically grounded material editing. The empirical framework is well motivated, and the reported quantitative gains with the adapter (PSNR 20.72 vs. baseline 11.24 in Table 1) suggest a practically useful approach even independent of the theoretical framing. The supplementary provides detailed implementation and additional experiments. However, the load-bearing theoretical claim—that the discrete MC estimator converges to the MC-SDE (13)—is not established, and the experiments are performed precisely in the low-sample-count regime where the approximation is least justified.

major comments (3)
  1. [Sec. 3.1, Eqs. (5)-(8)] The derivation of Eq. (8) is not a valid weak-convergence argument. With the chosen schedule N(τ)=τ^{-2}, the increment |ΔN|≈2τ^{-3}Δτ tends to 0 as Δτ→0, so the local CLT in Eq. (6) is applied to a vanishing, often fractional, number of new samples. No regime exists where both Δτ is small and |ΔN| is large. Additionally, for Δτ>0, ΔN is negative, so the summation limits in Eq. (5) are inconsistent; the direction of time (increasing vs. decreasing τ) is never defined. The supplementary's check that the drift is O_p(1) near τ=0 (Sec. 6.2) addresses only the apparent singularity, not the Gaussian-increment assumption. Consequently, Eq. (13) is not established as a continuous-time limit of the discrete MC estimator.
  2. [Sec. 4.1, Table 1] All experiments use sample counts N=1,2,3,5,7, where the estimator X_N is a sum of one to seven samples. For N=1, X_1=F_1 has the distribution of a single radiance sample, which is arbitrary and not Gaussian; the jump size is O(1), not O(√Δτ). Thus the diffusion approximation (13) is qualitatively wrong in the tested regime. This invalidates the use of the variance-time mapping (27) as a theoretically grounded initialization for the τ-mapper. The empirical improvements may stem from the adapter and the diffusion model's generative prior, but they do not validate the MC-SDE bridge.
  3. [Sec. 3.3, Eqs. (21)-(24)] The identification of the MC-SDE with the reverse VE SDE is algebraically correct as a statement about two SDEs, but it inherits the invalid derivation of the MC-SDE. Moreover, the marginal variance Var=σ²τ² in Eq. (22) is a direct consequence of the chosen normalization N(τ)=τ^{-2} and the definition of τ; it is not an independent physical confirmation. The log-SNR alignment (27) contains a free constant A=log κ−log σ², tuned from anchors as described in Sec. 4.1 and the Limitations, so the later ordering t_spec†≥t_diff† (Eq. (32)) is not a parameter-free prediction but a consequence of the fitted alignment.
minor comments (4)
  1. [Sec. 3.2, Eq. (17)] The notation in Eq. (17) appears swapped: it defines μ_d=E[F^s_k] and μ_s=E[F^d_k], but the subscripts d and s should match the superscripts. Please clarify.
  2. [Sec. 6.1, Eq. (34)] There are typographical errors in the integrals: 'P(ω ⟩)' should be 'P(ω_i)' or similar. Also, the notation for the separated incoming light L_c is introduced without a clear definition.
  3. [General] The paper frequently refers to 'Eq. (53)' before it is defined (e.g., Sec. 3.1 and Sec. 6.3); the numbering is confusing and should be reordered. The phrase 'This makes sense with common observation' is informal and should be rewritten.
  4. [Sec. 7.1, Eq. (64)] The definition of x_t^DM uses z_ref, but the target distribution for the adapter is the noisy latent at timestep t. It should be stated explicitly that z_ref is a VAE-encoded reference image and that the adapter learns to match the distribution of x_t^DM. The dependence on z_ref is clear, but the notation could be improved.

Circularity Check

0 steps flagged

No significant circularity: the MC-SDE is a self-contained CLT-based construction, the VE reverse-SDE match is an explicit algebraic identity, and the empirical claims are externally benchmarked.

full rationale

The paper's core derivation starts from the discrete Monte Carlo estimator (Eq. 1), applies CLT and the explicit schedule N(τ)=τ^{-2} (Eqs. 2-11), and obtains the MC-SDE (Eq. 13). This is a self-contained derivation from stated assumptions. The later claim that the MC-SDE is the reverse SDE of the clean-start VE process (Eqs. 21-24) is a direct application of the standard reverse-time SDE formula (Eq. 23), where the forward process is explicitly introduced as 'fictitious' and its noise g(τ)=σ√(2τ) is chosen so that the marginal variance is σ²τ². This is a construction/representation, not a hidden reuse of the target claim as an input. The τ→t mapper involves an empirically anchored constant A; the paper itself flags in Sec. 8 that N_min=0.001 and N_max=5000 are empirically derived and that an instance-specific mapper would be preferable. However, that is a calibration choice, not a fitted parameter renamed as a prediction. Crucially, the specular-ordering prediction (Eq. 32) follows from σ_spec ≫ σ_diff plus monotone log-SNR alignment, and the constant A cancels, so it does not reduce to the fitted value. No load-bearing self-citations appear; the reverse-SDE formalism cites standard external work (Song et al. 2021). The reviewer concern that the local CLT is applied to small ΔN blocks at low SPP is a mathematical-validity issue, not a circularity: it does not show that any claim was assumed as an input. The method is also tested against external ground-truth renders and baselines (Tables 1-2), so the empirical results are not forced by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 1 invented entities

The central derivation relies on a heuristic CLT-to-SDE conversion, an arbitrary variance-time parameterization, an empirically anchored log-SNR alignment, and a trained adapter. Most of the mathematical content is a reparameterization of the CLT, with the constants that make the predictions concrete fitted to the chosen range.

free parameters (5)
  • Exponent p in N(tau)=tau^{-p} = p=2
    Chosen from the polynomial family p>1 because it makes the diffusion coefficient vanish as tau approaches 0; the main MC-SDE and all downstream tau-to-t mapping depend on this choice (Sec. 3.1, Eqs. 11-13).
  • Alignment constant A = log kappa - log sigma^2 = set by N_min=0.001, N_max=5000 average
    Fixes the tau-to-t mapping in Eq. (27); the paper admits N_min and N_max are empirically chosen and A varies per image (Sec. 8 Limitations).
  • Reference signal power kappa = unspecified
    Used in the SNR_MC definition Eq. (26); arbitrary scale folded into A.
  • Material-editing scaler M_at and thresholds Con_m, Con_r = hand-designed thresholds
    The attention modulation rule in Sec. 4.2 sets parameter tokens by thresholds; no ablation or principled derivation is provided.
  • Adapter network G_theta = trained on 30 scenes
    Learned distribution-mapping network used in the main experiment; its output depends on training data and loss weights, and it is a fitted component rather than a derived one.
axioms (6)
  • domain assumption Monte Carlo samples are i.i.d. with finite variance; CLT applies to the estimator and to local increments.
    Invoked from Eq. (2) onward; the paper's own Limitations section says importance sampling, MIS, and path correlations distort the effective variance, so this does not hold for practical path tracers.
  • ad hoc to paper The process X_{N(tau)} can be approximated by a Markov diffusion SDE with drift and diffusion given by Eq. (8).
    The discrete estimator sequence is not Markov nor Gaussian-increment except asymptotically; the paper does not establish weak convergence to the stated SDE (Sec. 3.1).
  • ad hoc to paper Choosing N(tau)=tau^{-p}, p>1, is a valid representation of Monte Carlo variance time.
    Any monotone N would do; p=2 is selected for convenience and is not derived from rendering (Eqs. 54-61).
  • domain assumption Specular and diffuse components can be separated and share a common noise source via uniform hemisphere sampling.
    Sec. 3.2; production renderers use importance sampling and correlated paths, so sigma_s >> sigma_d bounds are for a simplified estimator.
  • domain assumption The spectral power-law model S_x0(f) ~= |f|^{-p} and the frequency-stabilization argument in Eqs. (28)-(32).
    Assumed without empirical validation; used to conclude that specular content stabilizes later along the denoising trajectory.
  • standard math The diffusion reverse-SDE formula (Anderson, Eq. 23) is valid.
    Standard result in stochastic calculus; not a point of contention.
invented entities (1)
  • Variance time tau no independent evidence
    purpose: Continuous coordinate that maps Monte Carlo sample count N to a diffusion timestep via tau = N^{-1/2}; used throughout the MC-SDE and tau-to-t alignment.
    A reparameterization, not an independently evidenced physical quantity; its value is fixed by the ad hoc N(tau)=tau^{-2} and the fitted constant A.

pith-pipeline@v1.3.0-alltime-deepseek · 18021 in / 21926 out tokens · 202057 ms · 2026-08-02T21:15:48.249173+00:00 · methodology

0 comments
read the original abstract

Monte Carlo rendering and modern generative models both transform uncertain states into structured images, yet they are usually studied as separate processes. We introduce Monte Carlo Transport Scheduling, a framework that treats progressive path tracing as a continuous sampling-driven transport process. Our key observation is that the renderer already produces physically valid states along this process: nested Monte Carlo estimates trace a refinement trajectory whose natural time coordinate follows from sampling variance. This view leads to a continuous training framework that learns from real render endpoints rather than synthetic interpolants, preserving the statistical structure of Monte Carlo estimation while enabling arbitrary-step neural refinement. We evaluate the framework on a controlled rendering benchmark designed to separate transport difficulty from scene context, and show that it yields stable render refinement, supports continuous stopping between rendering states, and transfers as a physical prior for frozen generative samplers. These results suggest a common continuous-time substrate for rendering and generation, where Monte Carlo sampling provides both the physical states and the supervision for learning image transport.

Figures

Figures reproduced from arXiv: 2602.20725 by Changbo Wang, Hantang Liu, Junwei Shu, Wenjie Liu, Yang Li.

Figure 1
Figure 1. Figure 1: Monte Carlo Sampling Based Rendering (first row) vs. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: MC–SDE ↔ Diffusion-SDE: variance-time alignment and unified noise. Top (Path Tracing). Monte Carlo estimation is cast as a continuous process Y (τ ) indexed by the variance time τ (large τ : few samples, high variance; τ ↓ 0: noise-free), contracting toward the mean radiance µ. Middle (Reverse SDE). The evolution matches the reverse dynamics of a clean-start VE process with noise scale g(τ ); this provides… view at source ↗
Figure 3
Figure 3. Figure 3: Effect of stage order on appearance and color statis￾tics. Top-left inset: reference render IS. We edit the refer￾ence image into to different materials, a specular one xs (left) and a diffuse one xes (right). The bottom row plots the normal￾ized color histogram fCH(·) (RGB aggregated). Colors: blue = fCH(IS), orange = fCH(xs), green = fCH(xes). Allocating a larger denoising range to the high-variance spec… view at source ↗
Figure 4
Figure 4. Figure 4: Diffusion Model takes over noisy path traced images. Top left is the path traced high spp images, as the ground truth. N is the sample count of the path traced image in its column. The first row indicates the input into the diffusion model. For the second row, we initialize the baseline latent noise with the corresponding sample count image. Regardless of the specific sample counts per image, we then subje… view at source ↗
Figure 5
Figure 5. Figure 5: a simple framework for fine-grained material tuning. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Result of reversed ConMat. m = 0.5, r = 0.5. Left: original image. Middle: our normal method result. Right: Re￾versed result. It could be found that the reversed result has lower respond and intention to be a metallic material specular part), even with the same parameter. during the early stages of the denoising process, according to Eq. (32). So we design this process reversed, which strengthen the diffus… view at source ↗
Figure 7
Figure 7. Figure 7: No prompt (left, cfg 1.0) vs prompt (right, cfg 7.5) results. The prompt is [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Quality comparison in different base models of our [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: This figure depicts more complex scenarios, including sophisticated illumination environments and rendering results from [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 18 linked inside Pith

  1. [1]

    Blended diffusion for text-driven editing of natural images

    Omri Avrahami, Dani Lischinski, and Ohad Fried. Blended diffusion for text-driven editing of natural images. InCVPR, pages 18208–18218, 2022. 2

    2022

  2. [2]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. InCVPR, pages 18392–18402, 2023. 2

    2023

  3. [3]

    Physically-based shading at disney

    Brent Burley and Walt Disney Animation Studios. Physically-based shading at disney. InAcm Siggraph, pages 1–7. vol. 2012, 2012. 4

    2012

  4. [4]

    Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing.arXiv preprint arXiv:2304.08465, 2023

    Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xi- aohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing.arXiv preprint arXiv:2304.08465, 2023. 2

    Pith/arXiv arXiv 2023

  5. [5]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation. InICCV, 2023. 2

    2023

  6. [6]

    Attribute-centric compositional text-to-image generation.arXiv preprint arXiv:2301.01413, 2023

    Yuren Cong, Martin Renqiang Min, Li Erran Li, Bodo Rosenhahn, and Michael Ying Yang. Attribute-centric compositional text-to-image generation.arXiv preprint arXiv:2301.01413, 2023. 2

    Pith/arXiv arXiv 2023

  7. [7]

    R. L. Cook and K. E. Torrance. A reflectance model for computer graphics.ACM Trans. Graph., 1(1):7–24, 1982. 1, 4, 5

    1982

  8. [8]

    Expressive text-to-image generation with rich text

    Songwei Ge, Taesung Park, Jun-Yan Zhu, and Jia-Bin Huang. Expressive text-to-image generation with rich text. InCVPR, pages 7545–7556, 2023. 2

    2023

  9. [9]

    Prompt-to-prompt im- age editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022. 2

    Pith/arXiv arXiv 2022

  10. [10]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. InNeurIPS, 2021. 2

    2021

  11. [11]

    Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. 2

    2020

  12. [12]

    Neural gaffer: Relighting any object via diffusion

    Haian Jin, Yuan Li, Fujun Luan, Yuanbo Xiangli, Sai Bi, Kai Zhang, Zexiang Xu, Jin Sun, and Noah Snavely. Neural gaffer: Relighting any object via diffusion. InAdvances in Neural Information Processing Systems, 2024. 2

    2024

  13. [13]

    James T. Kajiya. The rendering equation. InProceedings of the 13th Annual Conference on Computer Graphics and Interactive Techniques, page 143–150, New York, NY , USA,

  14. [14]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. InNeurIPS, 2022. 2

    2022

  15. [15]

    Imagic: Text-based real image editing with diffusion models

    Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani. Imagic: Text-based real image editing with diffusion models. In CVPR, pages 6007–6017, 2023. 2

    2023

  16. [16]

    Dif- fusionclip: Text-guided diffusion models for robust image manipulation

    Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Dif- fusionclip: Text-guided diffusion models for robust image manipulation. InCVPR, pages 2426–2435, 2022. 2

    2022

  17. [17]

    In- trinsic image diffusion for single-view material estimation

    Peter Kocsis, Vincent Sitzmann, and Matthias Nießner. In- trinsic image diffusion for single-view material estimation. arXiv preprint arXiv:2312.12274, 2023. 2

    Pith/arXiv arXiv 2023

  18. [18]

    Diffusion- renderer: Neural inverse and forward rendering with video diffusion models.arXiv preprint arXiv: 2501.18590, 2025

    Ruofan Liang, Zan Gojcic, Huan Ling, Jacob Munkberg, Jon Hasselgren, Zhi-Hao Lin, Jun Gao, Alexander Keller, Nan- dita Vijaykumar, Sanja Fidler, and Zian Wang. Diffusion- renderer: Neural inverse and forward rendering with video diffusion models.arXiv preprint arXiv: 2501.18590, 2025. 2

    Pith/arXiv arXiv 2025

  19. [19]

    Zero-1-to-3: Zero-shot one image to 3d object

    Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tok- makov, Sergey Zakharov, and Carl V ondrick. Zero-1-to-3: Zero-shot one image to 3d object. InICCV, pages 9298– 9309, 2023. 2

    2023

  20. [20]

    Open-edit: Open- domain image manipulation with open-vocabulary instruc- tions

    Xihui Liu, Zhe Lin, Jianming Zhang, Handong Zhao, Quan Tran, Xiaogang Wang, and Hongsheng Li. Open-edit: Open- domain image manipulation with open-vocabulary instruc- tions. InECCV, pages 89–106. Springer, 2020. 2

    2020

  21. [21]

    Subject-diffusion:open domain personalized text-to-image generation without test-time fine-tuning.arXiv preprint arXiv:2307.11410, 2023

    Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Subject-diffusion:open domain personalized text-to-image generation without test-time fine-tuning.arXiv preprint arXiv:2307.11410, 2023. 2

    Pith/arXiv arXiv 2023

  22. [22]

    SDEdit: Guided image synthesis and editing with stochastic differential equa- tions

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia- jun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equa- tions. InICLR, 2022. 2

    2022

  23. [23]

    Null-text inversion for editing real images using guided diffusion models

    Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. InCVPR, pages 6038–6047,

  24. [24]

    T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.arXiv preprint arXiv:2302.08453, 2023

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2i- adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.arXiv preprint arXiv:2302.08453, 2023. 2

    Pith/arXiv arXiv 2023

  25. [25]

    Glide: Towards photorealis- tic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealis- tic image generation and editing with text-guided diffusion models. InICML, pages 16784–16804, 2022. 1, 2

    2022

  26. [26]

    Hierarchical text-conditional image gen- eration with clip latents.arXiv preprint arXiv:2204.06125, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gen- eration with clip latents.arXiv preprint arXiv:2204.06125, 2022

    Pith/arXiv arXiv 2022

  27. [27]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. InCVPR, pages 10684– 10695, 2022. 1, 2

    2022

  28. [28]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. InCVPR, pages 22500–22510, 2023. 2

    2023

  29. [29]

    Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models.arXiv preprint arXiv:2307.06949, 2023

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. Hyperdreambooth: Hypernetworks for fast personalization of text-to-image models.arXiv preprint arXiv:2307.06949, 2023. 2

    Pith/arXiv arXiv 2023

  30. [30]

    Photorealistic text-to-image diffusion models with deep language understanding.NeurIPS, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding.NeurIPS, 35:36479–36494, 2022. 1, 2

    2022

  31. [31]

    Matfusion: A generative dif- fusion model for svbrdf capture

    Sam Sartor and Pieter Peers. Matfusion: A generative dif- fusion model for svbrdf capture. InSIGGRAPH Asia 2023 Conference Papers, pages 1–10, 2023. 2

    2023

  32. [32]

    Freeman, and Mark Matthews

    Prafull Sharma, Varun Jampani, Yuanzhen Li, Xuhui Jia, Dmitry Lagun, Fredo Durand, William T. Freeman, and Mark Matthews. Alchemist: Parametric control of ma- terial properties with diffusion models.arXiv preprint arXiv:2312.02970, 2023. 2

    Pith/arXiv arXiv 2023

  33. [33]

    Large-scale text-to-image model with inpainting is a zero-shot subject-driven image generator

    Chaehun Shin, Jooyoung Choi, Heeseung Kim, and Sungroh Yoon. Large-scale text-to-image model with inpainting is a zero-shot subject-driven image generator. 2024. 8

    2024

  34. [34]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. InICLR, 2021. 2, 4, 5

    2021

  35. [35]

    Stable diffusion v2.1, 2022.https : / / huggingface

    Stability AI. Stable diffusion v2.1, 2022.https : / / huggingface . co / stabilityai / stable - diffusion-2-1. 1

    2022

  36. [36]

    Plug-and-play diffusion features for text-driven image-to-image translation

    Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image-to-image translation. InCVPR, pages 1921–1930,

    1921

  37. [37]

    Controlmat: A controlled generative approach to material capture.arXiv preprint arXiv:2309.01700, 2023

    Giuseppe Vecchio, Rosalie Martin, Arthur Roullier, Adrien Kaiser, Romain Rouffet, Valentin Deschaintre, and Tamy Boubekeur. Controlmat: A controlled generative approach to material capture.arXiv preprint arXiv:2309.01700, 2023. 2

    Pith/arXiv arXiv 2023

  38. [38]

    Sketch-guided text-to-image diffusion models

    Andrey V oynov, Kfir Aberman, and Daniel Cohen-Or. Sketch-guided text-to-image diffusion models. InACM SIG- GRAPH 2023 Conference Proceedings, 2023. 2

    2023

  39. [39]

    P+: Extended textual conditioning in text-to- image generation.arXiv preprint arXiv:2303.09522, 2023

    Andrey V oynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. P+: Extended textual conditioning in text-to- image generation.arXiv preprint arXiv:2303.09522, 2023. 2

    Pith/arXiv arXiv 2023

  40. [40]

    Novel view synthesis with diffusion models.arXiv preprint arXiv:2210.04628, 2022

    Daniel Watson, William Chan, Ricardo Martin-Brualla, Jonathan Ho, Andrea Tagliasacchi, and Mohammad Norouzi. Novel view synthesis with diffusion models.arXiv preprint arXiv:2210.04628, 2022. 2

    Pith/arXiv arXiv 2022

  41. [41]

    3d-aware image generation using 2d diffusion mod- els.arXiv preprint arXiv:2303.17905, 2023

    Jianfeng Xiang, Jiaolong Yang, Binbin Huang, and Xin Tong. 3d-aware image generation using 2d diffusion mod- els.arXiv preprint arXiv:2303.17905, 2023. 2

    Pith/arXiv arXiv 2023

  42. [42]

    Freeman, Fr ´edo Durand, and Song Han

    Guangxuan Xiao, Tianwei Yin, William T. Freeman, Fr ´edo Durand, and Song Han. Fastcomposer: Tuning-free multi- subject image generation with localized attention.arXiv preprint arXiv:2305.10431, 2023. 2

    Pith/arXiv arXiv 2023

  43. [43]

    Mat- laber: Material-aware text-to-3d via latent brdf auto-encoder

    Xudong Xu, Zhaoyang Lyu, Xingang Pan, and Bo Dai. Mat- laber: Material-aware text-to-3d via latent brdf auto-encoder. arXiv preprint arXiv:2308.09278, 2023. 2

    Pith/arXiv arXiv 2023

  44. [44]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models.arXiv preprint arXiv:2308.06721,

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models.arXiv preprint arXiv:2308.06721,

    Pith/arXiv arXiv

  45. [45]

    Dilightnet: Fine-grained light- ing control for diffusion-based image generation

    Chong Zeng, Yue Dong, Pieter Peers, Youkang Kong, Hongzhi Wu, and Xin Tong. Dilightnet: Fine-grained light- ing control for diffusion-based image generation. InACM SIGGRAPH 2024 Conference Papers, 2024. 2

    2024

  46. [46]

    Paint3d: Paint anything 3d with lighting-less texture diffusion models

    Xianfang Zeng, Xin Chen, Zhongqi Qi, Wen Liu, Zibo Zhao, Zhibin Wang, Bin Fu, Yong Liu, and Gang Yu. Paint3d: Paint anything 3d with lighting-less texture diffusion models. arXiv preprint arXiv:2312.13913, 2023. 2

    Pith/arXiv arXiv 2023

  47. [47]

    Rgb↔x: Image decomposition and synthe- sis using material- and lighting-aware diffusion models

    Zheng Zeng, Valentin Deschaintre, Iliyan Georgiev, Yannick Hold-Geoffroy, Yiwei Hu, Fujun Luan, Ling-Qi Yan, and Miloˇs Haˇsan. Rgb↔x: Image decomposition and synthe- sis using material- and lighting-aware diffusion models. In ACM SIGGRAPH 2024 Conference Papers, New York, NY , USA, 2024. Association for Computing Machinery. 2

    2024

  48. [48]

    Dreamface: Progressive generation of animat- able 3d faces under text guidance.ACM Trans

    Longwen Zhang, Qiwei Qiu, Hongyang Lin, Qixuan Zhang, Cheng Shi, Wei Yang, Ye Shi, Sibei Yang, Lan Xu, and Jingyi Yu. Dreamface: Progressive generation of animat- able 3d faces under text guidance.ACM Trans. Graph., 42 (4), 2023. 2

    2023

  49. [49]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In CVPR, pages 3836–3847, 2023. 2

    2023

  50. [50]

    Adding conditional control to text-to-image diffusion models, 2023

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models, 2023. 1 Bridging Physically Based Rendering and Diffusion Models with Stochastic Differential Equation Supplementary Material

    2023

  51. [52]

    variance time

    Mathematical Explanation 6.1. Analysis ofσ s andσ d This section we will discuss whyσ s is often greater than σd. First, we separate the last ray into camerax [0] k out as fol- lows: xk =x [0] k ˆLi,k = (kd c π + DF G 4(ωo·n)(ωi,k·n) )n·ω i,k P(ω i,k) ˆLi,k = kd c π n·ω i,k P(ω i,k) ˆLi,k + DF G 4(ωo ·n)(ω i,k ·n)P(ω i,k) n·ω i,k ˆLi,k =x k,d +x k,s, wher...

  52. [53]

    variance time

    Experiments Details This section we will introduce our extra experiments and more details. The first subsection presents the implementa- tion details of our adapter training. Then more experiments are conducted and analyzed. 7.1. Adapter Training Details We train a small distribution adapterG θ that maps Monte Carlo (MC) low-SPP latents to diffusion-model...

  53. [54]

    variance time

    Limitations and Future Work Selection ofτmapper.In the main paper, we use t(N) = arg min t λdif f−(A−2log τ(N))|, and we use twoN min, Nmax to calculate the averageA. In practical, we chooseN min = 0.001, Nmax = 5000. These two val- ues are empirically derived. While they demonstrate broad applicability across most common scenarios, they are not theoretic...

  54. [1986]

    Association for Computing Machinery. 1