Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

What Makes for Text to 360-degree Panorama Generation with Stable Diffusion?

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-tuning Stable Diffusion for 360-degree panoramas succeeds because the value and output projections of cross-attention carry the spherical structure, while query and key projections do not; freezing query/key and strengthening output…

desk verdict A solid empirical study with a clean method whose headline mechanistic claim ('Wq/Wk learn shared knowledge') is not actually supported by its own experiments. read the letter →

arxiv 2505.22129 v1 pith:22VMUH32 submitted 2025-05-28 cs.CV

Jinhong Ni , Chang-Bin Zhang , Qiang Zhang , Jing Zhang This is my paper
classification cs.CV
keywords text-to-panoramagenerationStableDiffusionLoRAfine-tuningcross-attentionanalysisequirectangularprojectionmixtureofexpertsparameter-efficientmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper aims to explain why a few low-rank updates to a pre-trained perspective diffusion model such as Stable Diffusion are enough to generate 360-degree equirectangular panoramas from text. By fine-tuning each of the four cross-attention projection matrices in isolation and by deactivating subsets after joint fine-tuning, the authors conclude that the query and key matrices carry information shared with ordinary perspective images and do not capture spherical distortion, whereas the value and output matrices are the ones that adapt the model to panoramic geometry. If correct, this division of labor implies a simple recipe: leave query and key weights untouched and give the output projection more capacity with a mixture-of-experts layer. The paper verifies the recipe in the UniPano framework, which reports better FAED and horizontal-view FID than the prior dual-branch approach while using roughly half the memory and training time, and scales end-to-end to 1024×2048 panoramas when built on Stable Diffusion 3.

What carries the argument

The argument is carried by the four LoRA weight matrices of the cross-attention block, $W_q$, $W_k$, $W_v$, $W_o$, with $Q = z_t W_q$, $K = y W_k$, $V = y W_v$, and the output projection $W_o$. Two probing procedures do the analytical work: training each matrix in isolation to see whether it can learn the spherical structure, and deactivating subsets of LoRAs after joint fine-tuning to see which are load-bearing for panorama versus perspective generation. UniPano turns the findings into a design: it freezes $W_q$ and $W_k$, keeps $W_v$ as a LoRA, and replaces the $W_o$ LoRA with a mixture of experts (four LoRA experts, top-2 routing, load-balancing auxiliary loss), which the ablations show is the best of the compared capacity-boosting choices.

What would settle it

Measure the relative size and effective rank of each LoRA update after joint fine-tuning: if $\|\Delta W_q\|/\|W_q\|$ and $\|\Delta W_k\|/\|W_k\|$ are negligible compared with the value/output updates, the 'shared knowledge' reading is falsified, because the only observed effect of removing query/key LoRAs would be explained by their smallness. Alternatively, apply the trained query/key LoRAs to ordinary perspective-image prompts and compare outputs against the base model with random updates of the same scale: no detectable difference would show that these LoRAs carry no usable shared knowledge.

Watch

Extended reading notes

Core claim

The central discovery is a functional division of labor inside the cross-attention modules during LoRA fine-tuning for panorama generation. The paper shows that training only $W_q$ or $W_k$ on panoramic data fails to produce spherical/equirectangular structure, while training only $W_v$ or $W_o$ succeeds. After joint fine-tuning, removing the $W_v$ and $W_o$ LoRAs restores the model's ability to generate perspective images, and removing $W_q$ and $W_k$ does not change panoramic outputs. From this the authors conclude that $W_q$ and $W_k$ learn domain-shared semantic information that is irrelevant to the panorama, while $W_v$ and $W_o$ specialize in the equirectangular structure; UniPano then freezes $W_q$/$W_k$, strengthens $W_o$ with a mixture of LoRA experts, and shows this beats or matches the dual-branch baseline at lower cost.

Load-bearing premise

The load-bearing premise is that the deactivation experiment reveals what the query/key low-rank updates learned, but the same visual outcomes would occur if those updates are simply small enough to be harmless; UniPano itself needs only the weaker claim that they are unnecessary.

Editorial extensions

If this is right

  • Freezing $W_q$ and $W_k$ during panoramic fine-tuning does not hurt quality: in the ablations it improves FID while keeping FAED comparable to full LoRA, so roughly half the trainable attention weights can be dropped.
  • Extra capacity for panorama generation should be spent on $W_o$, not on query/key or on extra branches; replacing the $W_o$ LoRA with a mixture of experts yields the largest gains in the paper's ablations.
  • The dual-branch design of the prior strongest system is not necessary to reach its quality: UniPano reports better FAED and horizontal-view FID than PanFusion while using roughly half the peak GPU memory and about half the training time.
  • Cheaper training makes end-to-end higher-resolution panoramas practical: the same recipe builds on Stable Diffusion 3 to generate 1024×2048 images directly, without a separate super-resolution stage.
  • A single jointly fine-tuned checkpoint can serve both domains: dropping only the $W_v$/$W_o$ LoRAs returns the model to perspective generation, while keeping them produces panoramas.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The deactivation result also follows if the query/key LoRA updates are merely small; a direct comparison of the norms or effective ranks of the four updates, or a transfer test that applies the q/k LoRAs to perspective prompts, would separate 'shared knowledge' from 'effectively unchanged'.
  • If the pattern is general, other geometric or domain shifts handled by diffusion fine-tuning — depth estimation, outpainting, stylization — might also be carried mainly by value/output projections, so the freeze-query/key recipe could be tested there.
  • The saturation of the mixture of experts at only four experts suggests the fine-tuning task needs modest extra capacity, so a single higher-rank LoRA on $W_o$ might capture much of the benefit without routing overhead; the paper does not test this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper studies LoRA fine-tuning of Stable Diffusion for text-to-360-degree panorama generation. It isolates the four cross-attention weight matrices Wq, Wk, Wv, Wo and reports two findings: (1) training Wv or Wo in isolation captures equirectangular structure while Wq or Wk does not; (2) after joint fine-tuning, deactivating Wv/Wo restores perspective generation, while deactivating Wq/Wk leaves panorama generation unchanged. From this the authors conclude that Wq/Wk encode shared perspective-panorama knowledge and Wv/Wo specialize in panoramic structure. Based on this analysis, they propose UniPano, which freezes Wq/Wk and replaces the Wo LoRA with a mixture of LoRA experts. UniPano is reported to outperform prior methods on Matterport3D at 512x1024 and to scale to 1024x2048 with Stable Diffusion 3, while using less memory and training time than PanFusion.

Significance. If the central mechanistic claim is supported, the paper would make a useful contribution to understanding parameter-efficient fine-tuning for domain transfer, and the UniPano recipe itself is a simple, memory-efficient baseline that the community could adopt. The strengths of the paper are its clean ablation structure, the direct comparison of alternative capacity-boosting designs for Wo, the explicit memory/time reporting, and the promise of code release. However, the paper's main interpretative conclusion is currently underdetermined by the presented evidence, and the quantitative comparison lacks variance estimates; these issues need to be resolved before the significance can be fully assessed.

major comments (3)
  1. [Sec. 3.2 (Conclusion 3.5, Fig. 4)] The deactivation experiment does not support the strong mechanistic claim that Wq/Wk 'learn shared knowledge' and 'preserve/enhance' the perspective prior. The observation that removing Wv/Wo restores perspective generation (Fig. 4c) is exactly what would be expected if the Wq/Wk LoRAs were near-identity or functionally dead, because the base perspective model would then dominate regardless of those LoRAs; similarly, Fig. 4d shows only that Wq/Wk are unnecessary for panorama generation. No norm, scale, or functional-difference measurement of the learned LoRA deltas is reported, so the evidence cannot distinguish 'shared knowledge' from 'no contribution.' I request a control in which random or zero-initialized Wq/Wk LoRAs are used instead, plus quantitative measures (e.g., delta norms, projection alignment, or head-level ablation) that positively identify what Wq/Wk encode. The UniPano recipe depends only on the weaker claim that Wq/Wk can be frozen, which the ablations support, but the paper's abstract and conclusion rest on the stronger interpretation.
  2. [Table 3 and Appendix B] The state-of-the-art comparison is not fully supported as reported. The caption states that results are reproduced with FP32 precision, but Appendix B states that SD+LoRA, Pano Only, and PanFusion metrics are directly taken from [56]; no own runs, seeds, or confidence intervals are provided. The reported FAED gap between UniPano (5.90) and PanFusion (6.04) is small and may be within run-to-run noise, especially given the 20-view sampling randomness. Please reproduce all baselines under identical conditions with at least three seeds, report mean and standard deviation, and reconcile the footnote with the appendix.
  3. [Sec. 3.3, Tables 2, 4, 5] The design choice of MoE over the other Wo-capacity strategies is made on the same Matterport3D evaluation set used for the final comparison, and the MoE hyperparameters (n=4, k=2) are selected from Table 5 on that same set; this creates a selection-circularity risk. In addition, Table 4 shows that freezing Wq/Wk yields mixed results relative to the LoRA baseline (FAED 7.99 vs. 7.90, FID 48.62 vs. 50.40), so the claim that freezing improves performance is not uniformly supported. Please validate the key choices on a held-out prompt set or with multiple seeds, or temper the claim accordingly.
minor comments (5)
  1. [Sec. 4.3] The phrase 'non-parametric-specific information' should probably read 'non-panorama-specific information' or 'panorama-agnostic information'; as written it is confusing.
  2. [Fig. 9 caption] The caption lists '1024 x 2024', while the text and Fig. 1 state 1024 x 2048; please correct the typo.
  3. [Appendix B] For MVDiffusion, the statement that reported results are 'directly taken from [56]' conflicts with the description of a resolution change; please specify exactly which numbers were recomputed and which were copied.
  4. [Sec. 4.4] The high-resolution experiments are qualitative only and are described as illustrations; the abstract should not imply a fully evaluated scalability result without a clarifying caveat.
  5. [General] Please include a reproducibility statement with exact LoRA ranks, learning rates, MoE routing loss coefficient, and the number of training epochs for each of Wq, Wk, Wv, and Wo in the main text or supplementary material.

Circularity Check

0 steps flagged · score 1.0 of 10

No circularity found: the empirical analysis and UniPano design are based on direct experiments and external baselines; the main weakness is an underdetermined mechanistic interpretation, not a circular step.

full rationale

The paper's central claims are derived from isolation experiments (training Wq, Wk, Wv, Wo separately, Table 1 and Fig. 3) and deactivation experiments after joint fine-tuning (Fig. 4). These are direct empirical observations, not definitions or fitted-input predictions. The conclusion that Wq/Wk learn 'shared knowledge across panoramic and perspective domains' is an interpretation of Observation 3.4, which is also consistent with the null hypothesis that those LoRAs are near-identity or inactive. However, this underdetermination is a scientific-evidence limitation, not circularity: the claimed conclusion does not reduce by construction to the experimental setup, and the paper does not define Wq/Wk in terms of the outcome. The UniPano recipe (freezing Wq/Wk, increasing Wo capacity via MoE) is subsequently validated on the same Matterport3D benchmark, but this is standard empirical model selection rather than a fitted parameter being relabeled as a prediction. There are no load-bearing self-citations; references such as PanFusion [56] are prior external work, and no uniqueness theorem from the authors is invoked. The quantitative results are compared against external baselines, and the design ablation (Table 4) explicitly tests the proposed strategy versus the full-LoRA baseline. Thus the derivation chain is self-contained, and any weakness concerns interpretative rigor, not circular reasoning.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on standard PEFT practice, a single dataset, chosen metrics, and the interpretability of the cross-attention decomposition. No new physical or theoretical entities are introduced.

free parameters (3)
  • LoRA rank for Wv (and Wq/Wk when active) = r=4
    Default rank from prior work [15,56]; not derived from theory.
  • Number of MoE experts for Wo = n=4
    Chosen by ablation (Table 5); n=4 best on FID/FAED.
  • Top-k selected experts in MoE = k=2
    Chosen by ablation (Table 5); k=2 outperforms k=4.
assumptions (4)
  • domain assumption LoRA on cross-attention weights W{q,k,v,o} is sufficient to adapt Stable Diffusion for panorama generation.
    Borrowed from [56] and standard PEFT practice (Sec 3.1); not proven in this paper.
  • domain assumption Matterport3D with BLIP-2 captions and the split of [44,56] is an adequate evaluation corpus.
    Used in Sec 4.1; no independent justification of dataset representativeness.
  • domain assumption FID/FAED/IS/CS faithfully measure panorama quality.
    Sec 4.1 acknowledges limitations of IS and 20-view FID, yet uses them for ranking.
  • domain assumption The base Stable Diffusion model has a perspective prior that can be cleanly separated from panoramic structure.
    Central to the analysis in Sec 3.2; not directly measured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What Makes for Text to 360-degree Panorama Generation with Stable Diffusion?." pith.science (2026). https://pith.science/paper/22VMUH32

@misc{pith2026250522129,
  author       = {Pith},
  title        = {Pith review of: What Makes for Text to 360-degree Panorama Generation with Stable Diffusion?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/22VMUH32}},
  note         = {Machine review of arXiv:2505.22129}
}
read the original abstract

Recent prosperity of text-to-image diffusion models, e.g. Stable Diffusion, has stimulated research to adapt them to 360-degree panorama generation. Prior work has demonstrated the feasibility of using conventional low-rank adaptation techniques on pre-trained diffusion models to generate panoramic images. However, the substantial domain gap between perspective and panoramic images raises questions about the underlying mechanisms enabling this empirical success. We hypothesize and examine that the trainable counterparts exhibit distinct behaviors when fine-tuned on panoramic data, and such an adaptation conceals some intrinsic mechanism to leverage the prior knowledge within the pre-trained diffusion models. Our analysis reveals the following: 1) the query and key matrices in the attention modules are responsible for common information that can be shared between the panoramic and perspective domains, thus are less relevant to panorama generation; and 2) the value and output weight matrices specialize in adapting pre-trained knowledge to the panoramic domain, playing a more critical role during fine-tuning for panorama generation. We empirically verify these insights by introducing a simple framework called UniPano, with the objective of establishing an elegant baseline for future research. UniPano not only outperforms existing methods but also significantly reduces memory usage and training time compared to prior dual-branch approaches, making it scalable for end-to-end panorama generation with higher resolution. The code will be released.

Figures

Figures reproduced from arXiv: 2505.22129 by the authors.

Figure 1
Figure 1. Our UniPano can synthesize realistic 360-degree panoramic images by fine-tuning Stable Diffusion. (Top) [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Illustration of fine-tuning cross-attention blocks within diffusion models with low-rank adaptation (LoRA). MCA, MatMul, h denotes the multi-head cross attention, matrix multiplication, and the number of attention heads respectively. 3. What Makes for Panorama Generation? 3.1. Preliminary Diffusion models [14, 40, 41] generate images by iteratively transforming the noise sampled from the prior distribution into the … view at source ↗
Figure 3
Figure 3. Qualitative comparison for training W{q,k,v,o} in isolation separately. Training Wq or Wk in isolation fails to capture the spherical structure, as in (a) and (b); whereas training Wv or Wo in isolation successfully captures the spherical distortion of the panoramic images, as in (c) and (d). All visualizations are generated with the text prompt “a kitchen with stainless steel appliances”. 3.2. Motivation and Insigh… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Illustration of roles of W{q,k,v,o} when jointly fine-tuned. We first fine-tune W{q,k,v,o} jointly with LoRA on panoramic data, then optionally deactivate some LoRAs for inference with different purposes. (a) the panoramic image generated by fine-tuning W{q,k,v,o} join…
Figure 5
Figure 5. Figure 5: Selected qualitative comparisons between UniPano (Ours) and PanFusion. We show the generated panoramic image for each text prompt and 4 randomly sampled horizontal perspective views below. We highlight notable artifacts such as non-perspective lines with red boxes. Mor…
Figure 6
Figure 6. Figure 6 [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Additional qualitative comparisons. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Additional qualitative comparisons. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Additional high-resolution (1024 × 2024) results. Note that all results are generated using UniPano based on Stable Diffusion 3. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Additional high-resolution results for out-of-distribution prompts. [PITH_FULL_IMAGE:figures/full_fig_p015_10.png]
Figure 11
Figure 11. Figure 11: Additional high-resolution results for out-of-distribution prompts. [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]
Figure 12
Figure 12. Figure 12: Additional high-resolution results for out-of-distribution prompts. [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Additional high-resolution results with complex prompts. [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PanoLora: Bridging Perspective and Panoramic Video Generation with LoRA Adaptation

    cs.CV 2025-09 reject novelty 5.0 of 10

    Fine-tuning a pretrained video diffusion model with LoRA rank 16 on about 1,000 synthetic videos produces panoramic video with good seam closure, but the claim that rank must exceed 8 degrees of freedom is not proven.

Reference graph

Works this paper leans on

59 extracted references · 48 canonical work pages · cited by 1 Pith paper

  1. [56]

    Taming stable diffusion for text to 360° panorama image generation

    Cheng Zhang, Qianyi Wu, Camilo Cruz Gambardella, Xi- aoshui Huang, Dinh Phung, Wanli Ouyang, and Jianfei Cai. Taming stable diffusion for text to 360° panorama image generation. In CVPR, 2024. 1, 2, 3, 4, 6, 7, 8, 9, 11, 12, 13

    work page 2024

  2. [1]

    https : / / www

    Chief architect 360° panorama renderings. https : / / www . chiefarchitect . com / products / 360 - panorama-viewer. Accessed: 2025-03-03. 11

    work page 2025

  3. [2]

    https : / / renderstuff

    Online 360° panorama viewer vr. https : / / renderstuff . com / tools / 360 - panorama - web-viewer. Accessed: 2025-03-03. 11

    work page 2025

  4. [3]

    Di- verse plausible 360-degree image outpainting for efficient 3dcg background creation

    Naofumi Akimoto, Yuhi Matsuo, and Yoshimitsu Aoki. Di- verse plausible 360-degree image outpainting for efficient 3dcg background creation. In CVPR, 2022. 3

    work page 2022

  5. [4]

    Multidiffusion: Fusing diffusion paths for controlled image generation

    Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. Multidiffusion: Fusing diffusion paths for controlled image generation. In ICML, 2023. 3

    work page 2023

  6. [5]

    Matterport3d: Learning from rgb-d data in indoor environments

    Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Hal- ber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. In 3DV, 2017. 1, 6, 11

    work page 2017

  7. [6]

    Dif- fusiondet: Diffusion model for object detection

    Shoufa Chen, Peize Sun, Yibing Song, and Ping Luo. Dif- fusiondet: Diffusion model for object detection. In ICCV,

    work page

  8. [7]

    Text2light: Zero-shot text-driven hdr panorama generation

    Zhaoxi Chen, Guangcong Wang, and Ziwei Liu. Text2light: Zero-shot text-driven hdr panorama generation. TOG, 2022. 6, 8

    work page 2022

Show all 59 references
  1. [8]

    Text-to-3d using gaussian splatting

    Zilong Chen, Feng Wang, Yikai Wang, and Huaping Liu. Text-to-3d using gaussian splatting. In CVPR, 2024. 3

    2024

  2. [9]

    Scaling recti- fied flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M ¨uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling recti- fied flow transformers for high-resolution image synthesis. In ICML, 2024. 3, 8

    2024

  3. [10]

    Scaling diffusion transformers to 16 bil- lion parameters

    Zhengcong Fei, Mingyuan Fan, Changqian Yu, Debang Li, and Junshi Huang. Scaling diffusion transformers to 16 bil- lion parameters. arXiv preprint arXiv:2407.11633, 2024. 6, 8

    2024 arXiv

  4. [11]

    An image is worth one word: Personalizing text-to-image gen- eration using textual inversion

    Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image gen- eration using textual inversion. In ICLR, 2023. 3

    2023

  5. [12]

    Neighborhood attention transformer

    Ali Hassani, Steven Walton, Jiachen Li, Shen Li, and Humphrey Shi. Neighborhood attention transformer. In CVPR, 2023. 6

    2023

  6. [13]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 11

    2022 arXiv

  7. [14]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. In NeurIPS, 2020. 3

    2020

  8. [15]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In ICLR, 2022. 1, 3, 6, 7, 8, 11

    2022

  9. [16]

    Squeeze-and-excitation net- works

    Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation net- works. In CVPR, 2018. 6

    2018

  10. [17]

    Cubediff: Repurposing diffusion-based image models for panorama generation

    Nikolai Kalischek, Michael Oechsle, Fabian Manhardt, Philipp Henzler, Konrad Schindler, and Federico Tombari. Cubediff: Repurposing diffusion-based image models for panorama generation. In ICLR, 2025. 1, 3

    2025

  11. [18]

    Repurpos- ing diffusion-based image generators for monocular depth estimation

    Bingxin Ke, Anton Obukhov, Shengyu Huang, Nando Met- zger, Rodrigo Caye Daudt, and Konrad Schindler. Repurpos- ing diffusion-based image generators for monocular depth estimation. In CVPR, 2024. 1, 3

    2024

  12. [19]

    Panogen: Text-conditioned panoramic environment generation for vision-and-language navigation

    Jialu Li and Mohit Bansal. Panogen: Text-conditioned panoramic environment generation for vision-and-language navigation. In NeurIPS, 2023. 1, 3

    2023

  13. [20]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML,

  14. [21]

    4k4dgen: Panoramic 4d generation at 4k resolution

    Renjie Li, Panwang Pan, Bangbang Yang, Dejia Xu, Shi- jie Zhou, Xuanyang Zhang, Zeming Li, Achuta Kadambi, Zhangyang Wang, Zhengzhong Tu, et al. 4k4dgen: Panoramic 4d generation at 4k resolution. arXiv preprint arXiv:2406.13527, 2024. 1

    2024 arXiv

  15. [22]

    Panofree: Tuning-free holistic multi- view image generation with cross-view self-guidance

    Aoming Liu, Zhong Li, Zhang Chen, Nannan Li, Yi Xu, and Bryan A Plummer. Panofree: Tuning-free holistic multi- view image generation with cross-view self-guidance. In ECCV, 2024. 3

    2024

  16. [23]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In ICLR, 2023. 3

    2023

  17. [24]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019. 6

    2019

  18. [25]

    Autoregressive omni-aware outpainting for open- vocabulary 360-degree image generation

    Zhuqiang Lu, Kun Hu, Chaoyue Wang, Lei Bai, and Zhiy- ong Wang. Autoregressive omni-aware outpainting for open- vocabulary 360-degree image generation. In AAAI, 2024. 3

    2024

  19. [26]

    Repaint: Inpainting using denoising diffusion probabilistic models

    Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting using denoising diffusion probabilistic models. In CVPR,

  20. [27]

    On distillation of guided diffusion models

    Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. In CVPR, 2023. 3

    2023

  21. [28]

    Bips: Bi-modal in- door panorama synthesis via residual depth-aided adversarial learning

    Changgyoon Oh, Wonjune Cho, Yujeong Chae, Daehee Park, Lin Wang, and Kuk-Jin Yoon. Bips: Bi-modal in- door panorama synthesis via residual depth-aided adversarial learning. In ECCV, 2022. 3, 6

    2022

  22. [29]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023. 11

    2023

  23. [30]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. In ICLR, 2024. 3 9

    2024

  24. [31]

    Dreamfusion: Text-to-3d using 2d diffusion

    Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Milden- hall. Dreamfusion: Text-to-3d using 2d diffusion. In ICLR,

  25. [32]

    Do transformers need deep long- range memory? In ACL, 2020

    Jack Rae and Ali Razavi. Do transformers need deep long- range memory? In ACL, 2020. 6

    2020

  26. [33]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, 2021. 1, 3

    2021

  27. [34]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022

    2022 arXiv

  28. [35]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2022. 1, 3, 7, 11

    2022

  29. [36]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In MICCAI, 2015. 11

    2015

  30. [37]

    Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, 2023. 1, 3

    2023

  31. [38]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. In NeurIPS, 2022. 1

    2022

  32. [39]

    Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer. In ICLR, 2017. 8

    2017

  33. [40]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In ICLR, 2020. 3, 11

    2020

  34. [41]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In ICLR, 2021. 3

    2021

  35. [42]

    Consistency models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In ICML, 2023. 3

    2023

  36. [43]

    Rethinking the inception ar- chitecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception ar- chitecture for computer vision. In CVPR, 2016. 6

    2016

  37. [44]

    Mvdiffusion: Enabling holistic multi- view image generation with correspondence-aware diffusion

    Shitao Tang, Fuyang Zhang, Jiacheng Chen, Peng Wang, and Yasutaka Furukawa. Mvdiffusion: Enabling holistic multi- view image generation with correspondence-aware diffusion. In NeurIPS, 2023. 1, 3, 6, 7, 11

    2023

  38. [45]

    Diffusers: State-of-the-art diffu- sion models

    Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, Dhruv Nair, Sayak Paul, William Berman, Yiyi Xu, Steven Liu, and Thomas Wolf. Diffusers: State-of-the-art diffu- sion models. https://github.com/huggingface/ diffusers...

    2022

  39. [46]

    Stylelight: Hdr panorama generation for lighting estimation and editing

    Guangcong Wang, Yinuo Yang, Chen Change Loy, and Zi- wei Liu. Stylelight: Hdr panorama generation for lighting estimation and editing. In ECCV, 2022. 3

    2022

  40. [47]

    Customizing 360-degree panoramas through text-to-image diffusion models

    Hai Wang, Xiaoyu Xiang, Yuchen Fan, and Jing-Hao Xue. Customizing 360-degree panoramas through text-to-image diffusion models. In WACV, 2024. 3

    2024

  41. [48]

    Panodif- fusion: 360-degree panorama outpainting via diffusion

    Tianhao Wu, Chuanxia Zheng, and Tat-Jen Cham. Panodif- fusion: 360-degree panorama outpainting via diffusion. In ICLR, 2024. 3

    2024

  42. [49]

    Mixture of lora experts

    Xun Wu, Shaohan Huang, and Furu Wei. Mixture of lora experts. In ICLR, 2024. 6, 8

    2024

  43. [50]

    Vision transformer with deformable attention

    Zhuofan Xia, Xuran Pan, Shiji Song, Li Erran Li, and Gao Huang. Vision transformer with deformable attention. In CVPR, 2022. 6

    2022

  44. [51]

    Smartbrush: Text and shape guided object inpainting with diffusion model

    Shaoan Xie, Zhifei Zhang, Zhe Lin, Tobias Hinz, and Kun Zhang. Smartbrush: Text and shape guided object inpainting with diffusion model. In CVPR, 2023. 3

    2023

  45. [52]

    Open-vocabulary panop- tic segmentation with text-to-image diffusion models

    Jiarui Xu, Sifei Liu, Arash Vahdat, Wonmin Byeon, Xiao- long Wang, and Shalini De Mello. Open-vocabulary panop- tic segmentation with text-to-image diffusion models. In CVPR, 2023. 3

    2023

  46. [53]

    Dreamspace: Dreaming your room space with text-driven panoramic texture propa- gation

    Bangbang Yang, Wenqi Dong, Lin Ma, Wenbo Hu, Xiao Liu, Zhaopeng Cui, and Yuewen Ma. Dreamspace: Dreaming your room space with text-driven panoramic texture propa- gation. In IEEE VR, 2024. 1

    2024

  47. [54]

    Diffpano: Scalable and con- sistent text to panorama generation with spherical epipolar- aware diffusion

    Weicai Ye, Chenhao Ji, Zheng Chen, Junyao Gao, Xiaoshui Huang, Song-Hai Zhang, Wanli Ouyang, Tong He, Cairong Zhao, and Guofeng Zhang. Diffpano: Scalable and con- sistent text to panorama generation with spherical epipolar- aware diffusion. In NeurIPS, 2024. 3

    2024

  48. [55]

    Long-term photometric consistent novel view synthesis with diffusion models

    Jason J Yu, Fereshteh Forghani, Konstantinos G Derpanis, and Marcus A Brubaker. Long-term photometric consistent novel view synthesis with diffusion models. In ICCV, 2023. 3

    2023

  49. [57]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, 2023. 1

    2023

  50. [58]

    Unipc: A unified predictor-corrector framework for fast sampling of diffusion models

    Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen Lu. Unipc: A unified predictor-corrector framework for fast sampling of diffusion models. In NeurIPS, 2023. 3

    2023

  51. [59]

    a bedroom with a ceiling fan

    Kaiwen Zheng, Cheng Lu, Jianfei Chen, and Jun Zhu. Dpm- solver-v3: Improved diffusion ode solver with empirical model statistics. In NeurIPS, 2023. 3 10 A. Preliminary on Diffusion Models For completeness sake, we provide preliminary on diffusion models, particularly latent di...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.