Pith. sign in

REVIEW 3 major objections 6 minor 71 references

Advancing Text-to-3D Generation with Linearized Lookahead Variational Score Distillation

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes L2-VSD, a linearized lookahead correction to variational score distillation, and claims it reliably improves text-to-3D generation quality.

desk verdict A credible linearization fix for VSD's stale LoRA update—worth a serious referee despite thin quantitative support. read the letter →

arxiv 2507.09748 v1 pith:TMQ6DN4M submitted 2025-07-13 cs.CV

Yu Lei , Bingde Liu , Qingsong Xie , Haonan Lu , Zhijie Deng This is my paper
classification cs.CV
keywords text-to-3DgenerationscoredistillationvariationalLoRAlookaheadcorrectionlinearizationdiffusionmodelsNeRF
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Text-to-3D generation by score distillation uses a pretrained 2D diffusion model to guide a 3D scene, and variational score distillation (VSD) adds a trainable LoRA score model, a low-rank adapter of the pretrained model, to correct that guidance. This paper argues that practical VSD suffers from a timing mismatch: the LoRA model is estimated on the previous 3D state rather than the current one, so its corrections are stale. The proposed fix, L2-VSD, updates the LoRA model one step ahead and then keeps only the first-order Taylor term of that update in the distillation gradient, discarding a higher-order term that the authors find is mostly noise. The paper claims this reliably improves geometry and texture, works even at low rendering resolutions, and adds only one forward pass per iteration.

What carries the argument

The load-bearing object is the linearized lookahead LoRA score: after one SGD step on the LoRA denoising loss, the updated score $\epsilon_{\phi_{i+1}}$ is Taylor-expanded around $\phi_i$, and only the first-order term $\Delta\epsilon_{\mathrm{first}} = -2\eta\,\Delta\phi_i J_{\phi_i}^T(x_t,t,c,y)$ is kept. This term can be computed with one additional forward pass using forward-mode automatic differentiation, since $\Delta\phi_i$ is already available from the LoRA backward pass. The result is a corrected distillation gradient with two score contrasts: the original VSD term plus a lookahead correction whose Jacobian product acts as a preconditioning matrix.

What would settle it

A controlled experiment across diverse prompts, LoRA ranks, and training stages that decodes the first-order and higher-order residual maps, as in Fig. 5, would settle the claim: if the higher-order term ever displays recognizable object structure or if dropping it fails to improve over vanilla VSD in a new setting, the decomposition L2-VSD relies on would be undermined.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that the difference between ordinary VSD and a lookahead variant can be Taylor-expanded into a first-order term that carries coherent object semantics and a higher-order term that acts as unstructured noise, and that using only the first-order term yields stable, higher-quality generation. Concretely, the VSD update gradient in Eq. (5) is replaced by Eq. (11), where the LoRA score is replaced by its linearized lookahead version, computed as the LoRA prediction plus a vector-Jacobian product. The paper supports this with pivot experiments showing that simply improving LoRA convergence does not help, that a naive lookahead scheme suffers from over-saturation and overfitting, and that the linearized version avoids collapse while retaining the lookahead benefit.

Load-bearing premise

The improvement from L2-VSD rests on the claim that the first-order Taylor term of the LoRA update carries the semantic signal while the higher-order term is mostly noise, a split the authors support by visual inspection and norm plots rather than by a formal argument.

Editorial extensions

If this is right

  • If L2-VSD is right, first-stage NeRF generation in VSD-style pipelines becomes sharper and more complete, even at 64 by 64 rendering resolution, so later refinement stages inherit better geometry and texture.
  • The correction term is cheap: one extra forward pass per iteration, and using only last-layer LoRA Jacobians trades a little quality for a measurable speedup.
  • Because ESD and HiFA build on VSD, the linearized lookahead correction can be inserted into them to improve their outputs without changing their own objectives.
  • Total wall-clock time need not grow: each iteration costs about 0.3 seconds more than VSD, but the method converges in roughly 10k steps instead of 15k, yielding a slightly shorter total training time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is whether the semantic-versus-noise split survives across LoRA ranks, base diffusion backbones, and prompt difficulty; if the higher-order term carries useful signal in some regimes, a hybrid weighting of the two terms could outperform dropping it entirely.
  • The vector-Jacobian product in Eq. (12) effectively reweights the VSD gradient by a local curvature-like map of the LoRA model, suggesting possible connections to natural-gradient or preconditioned score distillation that the paper only hints at.
  • Because the correction is computed on the same noisy rendered image, it may reduce the number of views needed per step; the paper's multi-sample Gaussian example hints that more rendered views mitigate the mismatch, so L2-VSD could pair well with multi-view diffusion priors to stabilize low-sample regimes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper analyzes a practical mismatch in Variational Score Distillation (VSD) for text-to-3D generation: the LoRA score model used in the gradient update characterizes the distribution of the previous 3D state, not the current one. The authors first propose Lookahead-VSD (L-VSD), which updates the LoRA model before the 3D model, and show that this improves structure acquisition but leads to overfitting and over-saturation. They then derive a Taylor expansion of the LoRA prediction after its update, identifying a first-order term and a higher-order remainder. Based on norm diagnostics and decoded residual visualizations, they argue the first-order term carries semantic content while the higher-order term is noise, and propose L2-VSD, which keeps only the linearized lookahead correction. The gradient is expressed as the original VSD term plus an additional correction involving a vector-Jacobian product, computable with forward-mode autodiff. Experiments on the threestudio framework compare L2-VSD with SDS, VSD, ESD, L-VSD, and HiFA using qualitative renderings, CLIP similarity, FID, ablations on the LoRA learning rate and last-layer approximation, and integration with ESD and HiFA.

Significance. If the central claim holds, L2-VSD is an attractive plug-in improvement for VSD-based text-to-3D pipelines: it is conceptually simple, requires only one additional forward-mode pass per iteration, and the ablations (learning-rate robustness, last-layer approximation, and the high-order-term collapse in Fig. 20) provide initial evidence that the design choices matter. The paper is also honest in Section 6 about not having a distribution-based objective for the linearized correction. However, the evidence for 'clear superiority' is currently thin: the quantitative evaluation uses 20 prompts with no error bars or significance tests, and the semantic-vs-noise distinction is supported mainly by visual inspection of a few residual maps. These weaknesses leave the core claim plausible but not yet established at the level implied by the paper's conclusions.

major comments (3)
  1. [Sec. 5.3, Table 1] The quantitative evidence for 'clear superiority' rests on a single run over 20 prompts, with no error bars, no per-prompt breakdown, and no significance test. The reported differences, e.g., CLIP angle 0.324 for VSD vs 0.285 for L2-VSD and FID 301.54 vs 284.06, are within ranges that could easily arise from run-to-run variation in score-distillation methods. I request repeated seeds with variance reporting, or at least paired per-prompt statistics and a significance test, before the superiority claim can be accepted.
  2. [Sec. 4.1, Figs. 4c and 5] The decision to discard the higher-order term Δϵ_high is load-bearing, but it is supported only by norm curves and decoded residual images for a few prompts and training stages, plus a two-prompt qualitative ablation in Fig. 20. Section 6 explicitly admits that no distribution-based objective is known for why the first-order term should dominate. This means Eq. (11) is a heuristic modification rather than a derived approximation to a known objective. Please provide quantitative evidence across timesteps, prompts, and training stages that the higher-order term is consistently harmful, or a formal argument for the semantic/noise separation; otherwise the method may not transfer to other LoRA architectures or prompt regimes.
  3. [Sec. 4.2, Eq. (12), Appendix B.2, Table 2] The efficiency claim needs a more precise cost accounting. The vector-Jacobian product in Eq. (12) is not obtained from the ordinary forward pass at xt; it requires a forward-mode differentiation pass with a tangent vector, which in most frameworks roughly doubles the cost of that forward computation relative to a plain evaluation. The text should state explicitly how many LoRA forward and backward passes VSD and L2-VSD perform per iteration, and clarify that the three-forward-pass procedure in Appendix B.2 is used only for the diagnostic computation of Δϵ_high during L-VSD, not for L2-VSD training. This will also address the apparent inconsistency between the 'one additional forward pass' claim and the measured 0.3 s/iteration overhead.
minor comments (6)
  1. [Eq. (7) and Eq. (9)] The Jacobian notation is ambiguous: J_phi is used without specifying whether it is d×p or p×d, and the row/column orientation of Δϕ_i is not defined. Please make the conventions explicit so that the dimensions in Eqs. (9) and (12) are unambiguous.
  2. [Appendix B.2] There are typos in the appendix headings and text, such as 'comppute' and '∆ϵhighomputation'. Please proofread the appendix titles.
  3. [Fig. 4b] The caption says the LoRA prediction norm is shown for 'various γ in L-VSD', but the legend also includes 'VSD'. Clarify which curves correspond to which method and whether the VSD curve is shown for reference.
  4. [Table 1] The notation 'CLIP sim (↓)' is confusing because CLIP similarity is ordinarily higher-is-better. The text explains that the angle is used, but the table should state this directly (e.g., 'CLIP angle (↓)').
  5. [Sec. 5.5] The integration experiments with ESD and HiFA use only two prompts per method and are evaluated qualitatively. Please state that these results are illustrative rather than conclusive, or provide additional quantitative comparison.
  6. [Sec. 5.2] The phrase 'some representative results without bias' is not verifiable. Since the prompts for the main qualitative figures are not explicitly listed as a predefined random set, please state how the shown examples were selected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: L2-VSD is derived from a Taylor expansion of the lookahead update, not from a fitted value or self-citation.

full rationale

The central derivation (Sec. 4.1-4.2, Eqs. 7-12) starts from the L-VSD update, Taylor-expands the LoRA score, and keeps only the first-order term. Eq. (12) follows algebraically from this expansion; no fitted parameter is renamed as a prediction, and the only hand-set parameter is the LoRA learning rate eta. All supporting citations, including VSD [52], HiFA [66], ESD [51], and SiD [65], are external works and are not self-citations. The empirical claim that the first-order term carries semantics while the high-order term is noise is an explicit hypothesis supported by norm plots, decoded residual visualizations, and ablations (Figs. 4c, 5, 20); the paper itself acknowledges in Sec. 6 that no distribution-based objective has been formulated, which is an honest limitation rather than a circular argument. The efficiency statement that L2-VSD requires 'only one additional forward pass' is consistent with VSD already computing epsilon_phi_i(xt) and epsilon_phi_i(xt'); Appendix B.2's three-forward-pass accounting applies to the diagnostic high-order computation, not to the L2-VSD update itself. No circular step was found.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the VSD framework, the smoothness of the LoRA model, and the denoising objective. No new physical or algorithmic entities are postulated; the only user-set parameter is the LoRA learning rate eta.

free parameters (1)
  • LoRA learning rate eta = 1e-3 to 1 in ablations; default 1e-2 in analysis
    Scales the linearized lookahead correction in Eq. (12). Ablated in Sec. 5.4.1; the paper claims robustness over this range, so it is a user-chosen hyperparameter rather than a fitted constant.
assumptions (4)
  • domain assumption VSD theory: the LoRA model should estimate the score of the distribution of images rendered from the current 3D model (Theorem 2 of ProlificDreamer [52]).
    The entire mismatch diagnosis in Sec. 3.2 is framed against this theorem; the paper assumes it is valid.
  • domain assumption Smoothness of the LoRA model output with respect to its parameters, so the Taylor expansion in Eq. (9) converges and the first-order truncation is meaningful.
    The method drops the O(delta^2) term based on an assumed smoothness; no bound or regularity condition is given.
  • standard math The denoising score-matching objective in Eq. (2) adequately trains the LoRA model.
    Used for the LoRA update in Eq. (6) and for the linearized model in Eq. (10).
  • domain assumption The rendered image distribution is supported where the pretrained 2D diffusion prior is reliable.
    The distillation gradient assumes the diffusion model score is meaningful on rendered images; this is standard for score distillation but unverified in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Advancing Text-to-3D Generation with Linearized Lookahead Variational Score Distillation." pith.science (2026). https://pith.science/paper/TMQ6DN4M

@misc{pith2026250709748,
  author       = {Pith},
  title        = {Pith review of: Advancing Text-to-3D Generation with Linearized Lookahead Variational Score Distillation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TMQ6DN4M}},
  note         = {Machine review of arXiv:2507.09748}
}
abstract

Text-to-3D generation based on score distillation of pre-trained 2D diffusion models has gained increasing interest, with variational score distillation (VSD) as a remarkable example. VSD proves that vanilla score distillation can be improved by introducing an extra score-based model, which characterizes the distribution of images rendered from 3D models, to correct the distillation gradient. Despite the theoretical foundations, VSD, in practice, is likely to suffer from slow and sometimes ill-posed convergence. In this paper, we perform an in-depth investigation of the interplay between the introduced score model and the 3D model, and find that there exists a mismatching problem between LoRA and 3D distributions in practical implementation. We can simply adjust their optimization order to improve the generation quality. By doing so, the score model looks ahead to the current 3D state and hence yields more reasonable corrections. Nevertheless, naive lookahead VSD may suffer from unstable training in practice due to the potential over-fitting. To address this, we propose to use a linearized variant of the model for score distillation, giving rise to the Linearized Lookahead Variational Score Distillation ($L^2$-VSD). $L^2$-VSD can be realized efficiently with forward-mode autodiff functionalities of existing deep learning libraries. Extensive experiments validate the efficacy of $L^2$-VSD, revealing its clear superiority over prior score distillation-based methods. We also show that our method can be seamlessly incorporated into any other VSD-based text-to-3D framework.

Figures

Figures reproduced from arXiv: 2507.09748 by the authors.

Figure 1
Figure 1. Qualitative comparison with high resolution of 256. L 2 -VSD can generate meticulously detailed and realistic 3D assets from easy to very complex prompts and scenes. L 2 -VSD tends to yield more complete structures than HiFA. head Variational Score Distillation (L 2 -VSD), to reliably and consistently boost the generation quality of VSD. L 2 -VSD is both easy to implement and computationally efficient—the added line… view at source ↗
Figure 2
Figure 2. Comparison of convergence between VSD and L￾VSD with an illustrative 2D Gaussian example. In this toy ex￾ample we consider x ∈ R 2 from a single Gaussian distribution. We optimize q(x) towards ground truth distribution p(x) and use r(x) to function as LoRA which is used to estimate q(x). This example validates the existence of mismatching issue in VSD and we leave the details in Sec. 3.2. 2 [PITH_FULL_IMAGE:figures… view at source ↗
Figure 3
Figure 3. Qualitative Examples of training LoRA model for multiple steps per optimization iteration for VSD and L-VSD. are probably the root of the unstable performance and some￾times corrupted outcomes of VSD, and perform an in-depth investigation regarding them below. 3. Diagnose the Issues of VSD To assess whether the identified gaps contribute to the issues of VSD, we conduct two sets of pivot experiments in this section.… view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Curve Plots of Pivot Experimentsin Sec. 3. problem of VSD which hinders the distribution matching process. Correcting the optimization order can lead to better results. But, if we overfit the r(x) on the samples xsample, the results cannot converge normally towards the…
Figure 5
Figure 5. Figure 5: Visualization of ∆ϵf irst and ∆ϵhigh. The left column in each group represents the decoded first-order term while the right column represents the decoded high-order term. Prompt for each group: (left) ”a delicious hamburger”; (mid) ”an astronaut riding a horse”;(right)…
Figure 6
Figure 6. Figure 6: More Qualitative comparison with high resolution. generation quality by introducing novel techniques. We compare with them in order to position our performance more rigorously. We can find that our method can gener￾ate high-quality 3D assets with more realistic appeara…
Figure 7
Figure 7. Figure 7: Qualitative results of ablation studies. ESD 𝐿2-ESD ESD 𝐿2-ESD [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 10
Figure 10. Figure 10: Overview of VSD, L-VSD and L 2 -VSD training. We present an illustrative overview about the updating pipeline of VSD, L-VSD and L 2 -VSD respectively. As stated in Sec 2.2, we use θi and ϕi to represent the 3D and the LoRA models at ith iteration respectively. We use …
Figure 11
Figure 11. Figure 11: Comparison of VSD and L-VSD with more render samples. In this example, we sample 4 points in each iteration. , GT Dist. p(x) Init Dist. qinit(x) LoRA Init Dist. rinit(x) Fitted Dist. q(x) LoRA Dist. r(x) Optimized Sample Trajetory of Optimization GT Dist. p(x) Init Di…
Figure 12
Figure 12. Figure 12: Exploring the impact of r(x) overfitting on rendered samples. In this example, r(x) is delta distribution as we overfit it on x in each iteration. vector and text embedding vector through the image and text encoder of a CLIP model. We use the CLIP ViT-B/16 model [30].…
Figure 13
Figure 13. Figure 13: Qualitative comparison with low resolution of 64. L 2 -VSD can generate highly detailed 3D assets even with low resolution, while the other baselines (except for HiFA), suffering from geometry-texture co-training, tend to be blurry and have floaters. C. More Experimen…
Figure 14
Figure 14. Figure 14: Visualization of Failure Process. The upper row result is generated with original learning rate while the lower one is generated with scaling the learning rate by 0.1. Each row corresponds to a continue optimization process. Our prompt is ”an astronaut riding a horse”…
Figure 15
Figure 15. Figure 15: Results of L-VSD. These results are generated with the same prompts in [PITH_FULL_IMAGE:figures/full_fig_p016_15.png]
Figure 16
Figure 16. Figure 16: Comparison at second and third stages. We initial the objects with first-stage’s results and compare the geometry and texture refinement. As shown in the figure, the geometry generated by our method is more complete and texture generated by our method is much more rea…
Figure 17
Figure 17. Figure 17: Comparison on texture representation. We use VSD and our method to generate texture conditioned on the same geometry initialization. Prompts: (Upper)”a completely destroyed car” ;(Bottom)”a zoomed out DSLR photo of a pair of floating chopsticks picking up noodles out …
Figure 18
Figure 18. Figure 18: More Loss Curve. , into clear shape yet. So the predicted noises are all likely to be gaussian. Also, to verify the generality of this phenomenon, we test on multiple samples and measure the average LoRA loss to provide more convincing results, which is shown in Fig. …
Figure 19
Figure 19. Figure 19: VSD LoRA Comparison [PITH_FULL_IMAGE:figures/full_fig_p018_19.png]
Figure 20
Figure 20. Figure 20: Results comparison with using high-order term. Prompts: (upper)”A rotary telephone carved out of wood” ;(Bottom)”a DSLR photo of an exercise bike in a well lit room” D. Other Related works D.1. Text-to-Image Diffusion Models Text-to-image diffusion models [31, 32] are…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 57 canonical work pages

  1. [1]

    Genie: Gen- erative interactive environments, 2024

    Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker- Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, Lucy Gonzalez, Simon Osindero, Sherjil Ozair, Scott Reed, Jingwei Zhang, Konrad Zolna, Jeff Clune, Nando de Freitas, Satinder S...

    work page 2024

  2. [2]

    Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation, 2023

    Rui Chen, Yongwei Chen, Ningxin Jiao, and Kui Jia. Fan- tasia3d: Disentangling geometry and appearance for high- quality text-to-3d content creation, 2023. 1, 19

    work page 2023

  3. [3]

    Cinematic techniques in narrative visualization,

    Matthew Conlen, Jeffrey Heer, Hillary Mushkin, and Scott Davidoff. Cinematic techniques in narrative visualization,

    work page

  4. [4]

    Inclusive ar/vr: accessibility barriers for immersive technologies

    Chris Creed, Maadh Al-Kalbani, Arthur Theil, Sayan Sarcar, and Ian Williams. Inclusive ar/vr: accessibility barriers for immersive technologies. Universal Access in the Information Society, 23(1):59–73, 2023. 1

    work page 2023

  5. [5]

    threestudio: A unified framework for 3d content generation

    Yuan-Chen Guo, Ying-Tian Liu, Ruizhi Shao, Christian Laforte, Vikram V oleti, Guan Luo, Chia-Hao Chen, Zi- Xin Zou, Chen Wang, Yan-Pei Cao, and Song-Hai Zhang. threestudio: A unified framework for 3d content generation. https://github.com/threestudio- project/ threestudio, 2023. 4, 6

    work page 2023

  6. [6]

    CLIPScore: a reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. CLIPScore: a reference-free evaluation met- ric for image captioning. In EMNLP, 2021. 7

    work page 2021

  7. [7]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium. Advances in neural information processing systems , 30, 2017. 7, 13

    work page 2017

  8. [8]

    Classifier-free diffusion guidance, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance, 2022. 3, 8

    work page 2022

Show all 71 references
  1. [9]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. In Advances in Neural Infor- mation Processing Systems, pages 6840–6851. Curran Asso- ciates, Inc., 2020. 1, 3

    2020

  2. [10]

    Debi- asing scores and prompts of 2d diffusion for robust text-to-3d generation

    Susung Hong, Donghoon Ahn, and Seungryong Kim. Debi- asing scores and prompts of 2d diffusion for robust text-to-3d generation. arXiv preprint arXiv:2303.15413, 2023. 19

    2023 arXiv

  3. [11]

    Lora: Low- rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low- rank adaptation of large language models. In International Conference on Learning Representations, 2021. 3

    2021

  4. [12]

    Dreamtime: An improved optimiza- tion strategy for text-to-3d content creation

    Yukun Huang, Jianan Wang, Yukai Shi, Xianbiao Qi, Zheng- Jun Zha, and Lei Zhang. Dreamtime: An improved optimiza- tion strategy for text-to-3d content creation. arXiv preprint arXiv:2306.12422, 2023. 5, 19

    2023 arXiv

  5. [13]

    Vectorfusion: Text-to-svg by abstracting pixel-based diffusion models

    Ajay Jain, Amber Xie, and Pieter Abbeel. Vectorfusion: Text-to-svg by abstracting pixel-based diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR) , pages 1911–1920,

    1911

  6. [14]

    A survey on text-to-3d contents generation in the wild, 2024

    Chenhan Jiang. A survey on text-to-3d contents generation in the wild, 2024. 1

    2024

  7. [15]

    Noise-free score distillation, 2023

    Oren Katzir, Or Patashnik, Daniel Cohen-Or, and Dani Lischinski. Noise-free score distillation, 2023. 1, 3

    2023

  8. [16]

    Automatic differentiation in deep learning

    Nikhil Ketkar, Jojo Moolayil, Nikhil Ketkar, and Jojo Moolayil. Automatic differentiation in deep learning. Deep Learning with Python: Learn Best Practices of Deep Learn- ing Models with PyTorch, pages 133–145, 2021. 2

    2021

  9. [17]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Comput...

    2023

  10. [18]

    Kingma, Tim Salimans, Ben Poole, and Jonathan Ho

    Diederik P. Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models, 2023. 3

    2023

  11. [19]

    Instant-3d: Instant neural radiance field training towards on- device ar/vr 3d reconstruction, 2024

    Sixu Li, Chaojian Li, Wenbo Zhu, Boyang, Yu, Yang, Zhao, Cheng Wan, Haoran You, Huihong Shi, Yingyan, and Lin. Instant-3d: Instant neural radiance field training towards on- device ar/vr 3d reconstruction, 2024. 1

    2024

  12. [20]

    Magic3d: High-resolution text-to-3d content creation, 2023

    Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Liu, and Tsung-Yi Lin. Magic3d: High-resolution text-to-3d content creation, 2023. 1, 19

    2023

  13. [21]

    Point- voxel cnn for efficient 3d deep learning

    Zhijian Liu, Haotian Tang, Yujun Lin, and Song Han. Point- voxel cnn for efficient 3d deep learning. Advances in neural information processing systems, 32, 2019. 18

    2019

  14. [22]

    Scaledreamer: Scalable text-to- 3d synthesis with asynchronous score distillation

    Zhiyuan Ma, Yuxiang Wei, Yabin Zhang, Xiangyu Zhu, Zhen Lei, and Lei Zhang. Scaledreamer: Scalable text-to- 3d synthesis with asynchronous score distillation. In Euro- pean Conference on Computer Vision, pages 1–19. Springer,

  15. [23]

    A variational perspective on solving inverse problems with diffusion models

    Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vah- dat. A variational perspective on solving inverse problems with diffusion models. arXiv preprint arXiv:2305.04391 ,

    arXiv

  16. [24]

    Occupancy networks: Learning 3d reconstruction in function space

    Lars Mescheder, Michael Oechsle, Michael Niemeyer, Se- bastian Nowozin, and Andreas Geiger. Occupancy networks: Learning 3d reconstruction in function space. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4460–4470, 2019. 18

    2019

  17. [25]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis, 2020. 3

    2020

  18. [26]

    Instant neural graphics primitives with a mul- tiresolution hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexan- der Keller. Instant neural graphics primitives with a mul- tiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1–15, 2022. 13

    2022

  19. [27]

    Robust deep reinforce- ment learning through adversarial loss

    Tuomas Oikarinen, Wang Zhang, Alexandre Megretski, Luca Daniel, and Tsui-Wei Weng. Robust deep reinforce- ment learning through adversarial loss. Advances in Neural Information Processing Systems, 34:26156–26167, 2021. 19

    2021

  20. [28]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Al- ban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. 2017. 2, 6

    2017

  21. [29]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. In The 9 Eleventh International Conference on Learning Representa- tions, 2022. 1, 3, 7

    2022

  22. [30]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  23. [31]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In Proceedings of the 38th International Conference on Machine Learning , pages 8821–8831. PMLR, 2021. 18

    2021

  24. [32]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 18

    2022 arXiv

  25. [33]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1, 6, 18

    2022

  26. [34]

    Let 2d diffusion model know 3d- consistency for robust text-to-3d generation

    Junyoung Seo, Wooseok Jang, Min-Seop Kwak, Hyeonsu Kim, Jaehoon Ko, Junho Kim, Jin-Hwa Kim, Jiyoung Lee, and Seungryong Kim. Let 2d diffusion model know 3d- consistency for robust text-to-3d generation. arXiv preprint arXiv:2303.07937, 2023. 19

    2023 arXiv

  27. [35]

    Deep marching tetrahedra: a hybrid repre- sentation for high-resolution 3d shape synthesis

    Tianchang Shen, Jun Gao, Kangxue Yin, Ming-Yu Liu, and Sanja Fidler. Deep marching tetrahedra: a hybrid repre- sentation for high-resolution 3d shape synthesis. Advances in Neural Information Processing Systems , 34:6087–6101,

  28. [36]

    Mvdream: Multi-view diffusion for 3d gen- eration

    Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d gen- eration. arXiv:2308.16512, 2023. 13

    2023 arXiv

  29. [37]

    Graphvae: Towards generation of small graphs using variational au- toencoders

    Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational au- toencoders. In Artificial Neural Networks and Machine Learning–ICANN 2018: 27th International Conference on Artificial Neural Networks, Rhodes, Greece, October 4-7, 2018, ...

    2018

  30. [38]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  31. [39]

    Text-to-4d dy- namic scene generation

    Uriel Singer, Shelly Sheynin, Adam Polyak, Oron Ashual, Iurii Makarov, Filippos Kokkinos, Naman Goyal, Andrea Vedaldi, Devi Parikh, Justin Johnson, et al. Text-to-4d dy- namic scene generation. arXiv preprint arXiv:2301.11280 ,

    arXiv

  32. [40]

    Deep- voxels: Learning persistent 3d feature embeddings

    Vincent Sitzmann, Justus Thies, Felix Heide, Matthias Nießner, Gordon Wetzstein, and Michael Zollhofer. Deep- voxels: Learning persistent 3d feature embeddings. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2437–2446, 2019. 18

    2019

  33. [41]

    Implicit neural representa- tions with periodic activation functions

    Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. Implicit neural representa- tions with periodic activation functions. Advances in neural information processing systems, 33:7462–7473, 2020. 18

    2020

  34. [42]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International confer- ence on machine learning, pages 2256–2265. PMLR, 2015. 3

    2015

  35. [43]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In International Conference on Learning Represen- tations, 2020. 3

    2020

  36. [44]

    Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction

    Cheng Sun, Min Sun, and Hwann-Tzong Chen. Direct voxel grid optimization: Super-fast convergence for radiance fields reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5459– 5469, 2022. 18

    2022

  37. [45]

    Hierarchical amortized training for memory-efficient high resolution 3d gan

    Li Sun, Junxiang Chen, Yanwu Xu, Mingming Gong, Ke Yu, and Kayhan Batmanghelich. Hierarchical amortized training for memory-efficient high resolution 3d gan. arXiv preprint arXiv:2008.01910, 2020. 18

    2008 arXiv

  38. [46]

    Sta- ble score distillation for high-quality 3d generation, 2024

    Boshi Tang, Jianan Wang, Zhiyong Wu, and Lei Zhang. Sta- ble score distillation for high-quality 3d generation, 2024. 3, 5

    2024

  39. [47]

    SIMA Team, Maria Abi Raad, Arun Ahuja, Catarina Bar- ros, Frederic Besse, Andrew Bolt, Adrian Bolton, Bethanie Brownfield, Gavin Buttimore, Max Cant, Sarah Chakera, Stephanie C. Y . Chan, Jeff Clune, Adrian Collister, Vikki Copeman, Alex Cullum, Ishita Dasgupta, Dario de Ce- s...

    2024

  40. [48]

    Textmesh: Gen- 10 eration of realistic 3d meshes from text prompts

    Christina Tsalicoglou, Fabian Manhardt, Alessio Tonioni, Michael Niemeyer, and Federico Tombari. Textmesh: Gen- 10 eration of realistic 3d meshes from text prompts. arXiv preprint arXiv:2304.12439, 2023. 19

    2023 arXiv

  41. [49]

    Atlasnet: Multi-atlas non-linear deep networks for medical image segmentation

    Maria Vakalopoulou, Guillaume Chassagnon, Norbert Bus, Rafael Marini, Evangelia I Zacharaki, M-P Revel, and Nikos Paragios. Atlasnet: Multi-atlas non-linear deep networks for medical image segmentation. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018:...

    2018

  42. [50]

    A connection between score matching and denoising autoencoders

    Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661– 1674, 2011. 3

    2011

  43. [51]

    Taming mode col- lapse in score distillation for text-to-3d generation, 2024

    Peihao Wang, Dejia Xu, Zhiwen Fan, Dilin Wang, Sreyas Mohan, Forrest Iandola, Rakesh Ranjan, Yilei Li, Qiang Liu, Zhangyang Wang, and Vikas Chandra. Taming mode col- lapse in score distillation for text-to-3d generation, 2024. 1, 2, 3, 6, 8

    2024

  44. [52]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion, 2023

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion, 2023. 1, 3, 4

    2023

  45. [53]

    Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion

    Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distilla- tion. Advances in Neural Information Processing Systems , 36, 2024. 13

    2024

  46. [54]

    Adversarial score distillation: When score distillation meets gan, 2023

    Min Wei, Jingkai Zhou, Junyao Sun, and Xuesong Zhang. Adversarial score distillation: When score distillation meets gan, 2023. 1

    2023

  47. [55]

    Adversarial score distillation: When score distillation meets gan

    Min Wei, Jingkai Zhou, Junyao Sun, and Xuesong Zhang. Adversarial score distillation: When score distillation meets gan. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 8131–8141,

  48. [56]

    Video2game: Real-time, interactive, realistic and browser-compatible environment from a single video, 2024

    Hongchi Xia, Zhi-Hao Lin, Wei-Chiu Ma, and Shenlong Wang. Video2game: Real-time, interactive, realistic and browser-compatible environment from a single video, 2024. 1

    2024

  49. [57]

    Neurallift-360: Lifting an in-the-wild 2d photo to a 3d object with 360deg views

    Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Yi Wang, and Zhangyang Wang. Neurallift-360: Lifting an in-the-wild 2d photo to a 3d object with 360deg views. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4479–4489, 2023. 18

    2023

  50. [58]

    Neurallift-360: Lifting an in-the-wild 2d photo to a 3d object with 360deg views

    Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Yi Wang, and Zhangyang Wang. Neurallift-360: Lifting an in-the-wild 2d photo to a 3d object with 360deg views. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4479–4489, 2023. 19

    2023

  51. [59]

    Pointflow: 3d point cloud generation with continuous normalizing flows

    Guandao Yang, Xun Huang, Zekun Hao, Ming-Yu Liu, Serge Belongie, and Bharath Hariharan. Pointflow: 3d point cloud generation with continuous normalizing flows. In Proceed- ings of the IEEE/CVF international conference on computer vision, pages 4541–4550, 2019. 18

    2019

  52. [60]

    Learn to optimize denoising scores for 3d generation: A unified and improved diffusion prior on nerf and 3d gaussian splatting,

    Xiaofeng Yang, Yiwen Chen, Cheng Chen, Chi Zhang, Yi Xu, Xulei Yang, Fayao Liu, and Guosheng Lin. Learn to optimize denoising scores for 3d generation: A unified and improved diffusion prior on nerf and 3d gaussian splatting,

  53. [61]

    Graphrnn: Generating realistic graphs with deep auto-regressive models

    Jiaxuan You, Rex Ying, Xiang Ren, William Hamilton, and Jure Leskovec. Graphrnn: Generating realistic graphs with deep auto-regressive models. In International conference on machine learning, pages 5708–5717. PMLR, 2018. 18

    2018

  54. [62]

    Text-to-3d with classifier score distillation, 2023

    Xin Yu, Yuan-Chen Guo, Yangguang Li, Ding Liang, Song- Hai Zhang, and Xiaojuan Qi. Text-to-3d with classifier score distillation, 2023. 1, 3, 7

    2023

  55. [63]

    Efficientdreamer: High-fidelity and robust 3d cre- ation via orthogonal-view diffusion prior

    Minda Zhao, Chaoyi Zhao, Xinyue Liang, Lincheng Li, Zeng Zhao, Zhipeng Hu, Changjie Fan, and Xin Yu. Efficientdreamer: High-fidelity and robust 3d cre- ation via orthogonal-view diffusion prior. arXiv preprint arXiv:2308.13223, 2023. 19

    2023 arXiv

  56. [64]

    Dreampropeller: Supercharge text-to-3d generation with parallel sampling

    Linqi Zhou, Andy Shih, Chenlin Meng, and Stefano Er- mon. Dreampropeller: Supercharge text-to-3d generation with parallel sampling. arXiv preprint arXiv:2311.17082 ,

    arXiv

  57. [65]

    Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation

    Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. arXiv preprint arXiv:2404.04057 ,

    arXiv

  58. [66]

    Hifa: High- fidelity text-to-3d generation with advanced diffusion guid- ance

    Junzhe Zhu, Peiye Zhuang, and Sanmi Koyejo. Hifa: High- fidelity text-to-3d generation with advanced diffusion guid- ance. arXiv preprint arXiv:2305.18766, 2023. 6, 8 11 Contents

    2023 arXiv

  59. [67]

    Diffusion Models

    Background 3 2.1. Diffusion Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2.2. Text-to-3D Generation with Score Distillation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

  60. [68]

    Better Convergence? Maybe No

    Diagnose the Issues of VSD 4 3.1. Better Convergence? Maybe No. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.2. Lookahead? Maybe Yes! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 3.3. L...

  61. [69]

    Compare VSD with L-VSD

    Methodology 5 4.1. Compare VSD with L-VSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 4.2. Linearized Lookahead VSD . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

  62. [70]

    Settings

    Experiments 6 5.1. Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.2. Qualitative Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 5.3. Quantitati...

  63. [71]

    sum") + F.mse_loss(q_sigmao, r_sigma, reduction=

    Discussion, Conclusion and Limitation 8 A . Illustrative Comparison of Optimizations 13 B . Experiment Implementation 13 B.1. Main Experiments Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 B.2. High Order ∆ϵhighomputation ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.