REVIEW 3 major objections 5 minor 2 cited by
OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read OmniDrag claims the first motion-control system for omnidirectional image-to-video generation, letting users drag points to steer both camera and objects.
desk verdict Useful and plausible extension of drag control to 360 video, but the accuracy claim rests on an unvalidated tracker used for both training and evaluation, so the quantitative evidence needs stronger grounding. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing components are the spherical motion estimator (SME) and the Omni Controller. SME uses a HEALPix equal-area iso-latitude grid to place tracking points uniformly on the sphere instead of on the distorted equirectangular image, tracks those points through the video with a point tracker, keeps trajectories whose spherical (great-circle) displacement exceeds a threshold, and at inference time reconstructs full trajectories from user-supplied handle and target points via spherical interpolation. The Omni Controller is a lightweight two-ResBlock convolutional module that encodes the trajectory condition and injects it into the first UNet block of Stable Video Diffusion using cross-normalization, while the temporal attention layers of the denoiser are jointly fine-tuned so the model learns spherical motion patterns rather than only 2D translations. The two together are what the paper credits for accurate, distortion-free control of both camera and object motion.
What would settle it
Render synthetic 360° videos with a known camera rotation or object translation, run the spherical motion estimator on them, and compare estimated trajectories with ground truth; if endpoint errors grow substantially for points near the poles or for large spherical rotations, the learned condition maps are built on wrong motion signals and control quality would be expected to degrade correspondingly.
Extended reading notes
Core claim
On its own terms, OmniDrag establishes that omnidirectional motion control is achievable by making every stage of the control pipeline spherical: uniform point initialization on the sphere, spherical-distance-based selection of dominant motions, spherical interpolation from user clicks, and joint fine-tuning of temporal attention layers with a lightweight condition-injection module on a diffusion image-to-video model. The method is claimed to be the first to support both scene-level and object-level drag control for omnidirectional video, and experiments report lower FVD on equirectangular frames, lower FID and FVD on viewports, and better human ratings than DragNUWA, MotionCtrl, and DragAnything.
Load-bearing premise
The whole training signal rests on the point tracker's output on equirectangular frames being a faithful measurement of true spherical motion, including in the heavily distorted polar regions; the paper does not provide a validation of tracking accuracy on omnidirectional content.
Editorial extensions
If this is right
- A user can generate 360° videos with specified camera or object motion by drawing just two points, making motion control accessible for VR content creation.
- Spherical-aware handling—uniform initialization, great-circle distances, and spherical interpolation—becomes the standard ingredient for ODV motion control, since the paper shows 2D-based alternatives distort.
- Jointly fine-tuning temporal attention with the controller is necessary for learning spherical motion, so future ODV control models should not freeze the base UNet.
- Training on a dataset with large motions, like Move360, is what enables scene-level control; low-motion datasets limit controllability.
Reading between the lines
- A straightforward next test is to measure how SME behaves on synthetic spherical motion (e.g., pure pan, pure tilt, or rotation around the optical axis); the paper's construction assumes great-circle paths, so non-geodesic motions like spirals or abrupt jerks may be out of reach for the current inference interpolation.
- Because the inference stage interpolates between one handle and one target, fine-grained multi-point control—for instance, dragging along a curved path or shifting camera and object simultaneously—may require extending the estimator to several control points or user-drawn splines.
- Since Move360 is captured by a car with four degrees of freedom, it is likely biased toward forward/backward and lateral translations; a dataset with more varied camera motions (tilts, rolls, orbits) would test whether OmniDrag generalizes beyond the capture setup.
- If the CoTracker-based training signal turns out to be inaccurate in polar regions, an alternative estimator using spherical-aware tracking or projection to tangent planes could be plugged into the same Omni Controller architecture without changing the rest of the pipeline.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces OmniDrag, a method for drag-based motion control in omnidirectional image-to-video (ODV) generation. Building on Stable Video Diffusion, it adds a lightweight Omni Controller jointly fine-tuned with temporal attention layers, a spherical motion estimator (SME) for extracting training trajectories and estimating user trajectories via spherical interpolation, and the new Move360 dataset with large scene and object motions. Experiments compare against DragNUWA, MotionCtrl, and DragAnything on a 12-pair test set, reporting FID, FVD, an object-motion-control metric (ObjMC), human evaluation, and ablations of the proposed components.
Significance. If the central claims hold, OmniDrag would be a notable step toward practical controllable ODV generation, offering a drag-style interface that works on equirectangular images and supporting both scene- and object-level control. The SME design is well motivated and the component ablations in Tables 2 and Figures 6-7 directly support the importance of HEALPix initialization, spherical distance filtering, spherical interpolation, joint fine-tuning, and the Move360 dataset. The paper also provides a new dataset that addresses a real scarcity of high-motion ODV data. However, the strength of the evidence depends critically on whether CoTracker's tracks on ERP frames can be treated as ground-truth spherical motion, and on whether a 12-sample test set with no error bars can support the quantitative superiority claims.
major comments (3)
- [Sec. 3.4, Eq. (6); Sec. 4.1; Tables 1-2] The tracking function Ft in Eq. (6) is CoTracker applied directly to equirectangular (ERP) frames, and the ObjMC metric used in Tables 1 and 2 is computed with the same tracking methodology. The HEALPix initialization and spherical-distance filter only change which 2D tracks are kept; they do not correct tracking errors in the heavily distorted polar regions or across the 0/2π seam. The paper provides no validation that CoTracker yields accurate spherical trajectories on omnidirectional content (e.g., synthetic ERP videos with known ground truth, per-latitude error analysis, or a comparison against a spherical-aware tracker). Without such validation, the agreement between the training condition maps and the ObjMC evaluation could stem from a shared tracker bias rather than true spherical motion accuracy, leaving the central claim of 'accurate' control unsupported.
- [Sec. 4.2, test set description] The quantitative comparison is based on only twelve pairs of input as the test set, with no error bars, confidence intervals, or significance tests. For such a small sample, the reported differences in FID, FVD, ObjMC, and the human-preference percentages may not be statistically reliable. This weakens the claim of 'significant superiority' over DragNUWA, MotionCtrl, and DragAnything. The authors should report per-sample results or bootstrap confidence intervals, and also clarify the number of human evaluation samples and the evaluation protocol.
- [Sec. 4.3, Table 2, 'w/o Move360 Dataset' row] The ablation without Move360 achieves a better FID (167.56) than the full OmniDrag (171.41) and a comparable FVD (941.58 vs 933.73), while having much worse ObjMC (0.327 vs 0.044). The paper acknowledges this but does not explain the mechanism; the abstract and conclusion claim 'high-quality' generation. If FID is better without Move360, the quality claim depends entirely on the motion-control metric, which is built on the same tracking stack as the training signal. The authors should discuss whether the FID difference is within noise, whether ObjMC alone should be the primary criterion for the paper's quality claim, and how the trade-off is resolved in Fig. 6.
minor comments (5)
- [Table 2] The header contains two typos: 'w/o Shperical Filter' and 'w/o Shperical Interp.' should be 'Spherical'.
- [Sec. 3.4, Eq. (7)] The notation for spherical coordinates is inconsistent: Eq. (7) uses 'ϕ' while the surrounding text uses 'φ'. Please unify.
- [Sec. 3.4, text after Ninit] The formula 'Ninit = 12 × Nside 2' is missing the superscript; it should read 'Nside^2'.
- [Sec. 3.4, Eq. (9)] The spherical interpolation formula is undefined for the degenerate cases ω = 0 and ω = π; please state how these are handled.
- [Sec. 4.2, human evaluation] The human evaluation description is underspecified: there is no information on how many videos each volunteer rated, the rating scale, or inter-rater agreement. This affects the interpretability of the 75.7% and 76.1% preference scores.
Circularity Check
No significant circularity: the spherical motion estimator is an external-tracking-plus-geometric-filtering pipeline, and the central claim is not equivalent to its own inputs.
full rationale
Walking the derivation chain, the spherical motion estimator (Sec. 3.4) uses CoTracker as the external tracking function Ft in Eq. (6), with HEALPix initialization and spherical-distance filtering (Eqs. (7)-(8)) as geometric preprocessing, and spherical interpolation (Eq. (9)) as the inference-time trajectory model. None of these quantities is defined in terms of the evaluation metrics or fitted to them. The training condition maps are extracted from real ODV training videos, not from the test set or from the reported ObjMC values. The central claim of accurate motion control is supported by FVD/FID, viewport metrics, and an independent human evaluation (76.1% motion matching), so it does not rest on a self-consistent training/evaluation loop. The overlapping-author citations (ReVideo [42], 360DVD [61]) supply implementation details—trajectory-count sampling and latent rotation—and are not load-bearing for the central claim. The closest candidate for circularity is that ObjMC shares CoTracker with the training-condition extraction; however, CoTracker is a frozen, externally pretrained model and the diffusion loss is not optimized against ObjMC, so a shared tracker is a metric-validity concern, especially given unvalidated equirectangular behavior, rather than an equation-level circular reduction. The Limitations section's statement that decoupling camera- and object-level motion remains open is an acknowledged limitation, not a circular step. Therefore the paper's derivation is self-contained against external benchmarks and no significant circularity is found.
Assumptions & free parameters
free parameters (4)
- dth =
not reported
- Nside =
not reported
- Nsamp =
random between 1 and 10
- gamma =
not reported
assumptions (5)
- domain assumption Equirectangular projection with the spherical coordinate conversion in Eq. (7) correctly represents distances and motions on the viewing sphere.
- domain assumption CoTracker point tracks computed on ERP frames correspond to genuine physical point motions in the 360 degree scene.
- domain assumption Stable Video Diffusion's pretrained video prior is a suitable base for ODV generation after fine-tuning a controller and temporal attention layers.
- ad hoc to paper Spherical interpolation in Eq. (9) matches user intentions and aligns inference-time control signals with training-time SME trajectories.
- standard math HEALPix provides a uniform initialization for point tracking on the sphere.
Cite this review
Pith. "Pith review of OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation." pith.science (2026). https://pith.science/paper/ZWRH4YBM
@misc{pith2026241209623,
author = {Pith},
title = {Pith review of: OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZWRH4YBM}},
note = {Machine review of arXiv:2412.09623}
}
read the original abstract
As virtual reality gains popularity, the demand for controllable creation of immersive and dynamic omnidirectional videos (ODVs) is increasing. While previous text-to-ODV generation methods achieve impressive results, they struggle with content inaccuracies and inconsistencies due to reliance solely on textual inputs. Although recent motion control techniques provide fine-grained control for video generation, directly applying these methods to ODVs often results in spatial distortion and unsatisfactory performance, especially with complex spherical motions. To tackle these challenges, we propose OmniDrag, the first approach enabling both scene- and object-level motion control for accurate, high-quality omnidirectional image-to-video generation. Building on pretrained video diffusion models, we introduce an omnidirectional control module, which is jointly fine-tuned with temporal attention layers to effectively handle complex spherical motion. In addition, we develop a novel spherical motion estimator that accurately extracts motion-control signals and allows users to perform drag-style ODV generation by simply drawing handle and target points. We also present a new dataset, named Move360, addressing the scarcity of ODV data with large scene and object motions. Experiments demonstrate the significant superiority of OmniDrag in achieving holistic scene-level and fine-grained object-level control for ODV generation. The project page is available at https://lwq20020127.github.io/OmniDrag.
Figures
Figures from the paper (8 more)
Forward citations
Cited by 2 Pith papers
-
VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning
VQ-Insight uses progressive reinforcement learning with temporal shuffle and task rewards to teach a vision-language model to score and compare AI-generated videos, with gains on multiple video quality benchmarks.
-
MIND-Edit: MLLM Insight-Driven Editing via Language-Vision Projection
MIND-Edit combines instruction rewriting with MLLM-derived visual embeddings to guide diffusion-based image editing, but the reported numbers only partly support the claim of state-of-the-art performance.
Reference graph
Works this paper leans on
-
[1]
Deep learning for omnidirectional vision: A survey and new perspectives
Hao Ai, Zidong Cao, Jinjing Zhu, Haotian Bai, Yucheng Chen, and Lin Wang. Deep learning for omnidirectional vision: A survey and new perspectives. arXiv preprint arXiv:2205.10468, 2022. 2
arXiv 2022
-
[2]
Dream360: Diverse and immersive outdoor virtual scene creation via transformer-based 360° image outpainting
Hao Ai, Zidong Cao, Haonan Lu, Chen Chen, Jian Ma, Pengyuan Zhou, Tae-Kyun Kim, Pan Hui, and Lin Wang. Dream360: Diverse and immersive outdoor virtual scene creation via transformer-based 360° image outpainting. IEEE Transactions on Visualization and Computer Graphics (TVCG), 2024. 3
2024
-
[3]
Di- verse plausible 360-degree image outpainting for efficient 3dcg background creation
Naofumi Akimoto, Yuhi Matsuo, and Yoshimitsu Aoki. Di- verse plausible 360-degree image outpainting for efficient 3dcg background creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11441–11450, 2022. 3
work page 2022
-
[4]
Uniedit: A unified tuning- free framework for video motion and appearance editing
Jianhong Bai, Tianyu He, Yuchi Wang, Junliang Guo, Haoji Hu, Zuozhu Liu, and Jiang Bian. Uniedit: A unified tuning- free framework for video motion and appearance editing. arXiv preprint arXiv:2402.13185, 2024. 4
arXiv 2024
-
[5]
Stable video diffusion: Scaling latent video diffusion models to large datasets
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 3, 7
arXiv 2023
-
[6]
Align your latents: High-resolution video synthesis with la- tent diffusion models
Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22563–22575, 2023. 2
2023
-
[7]
Video generation models as world simu- lators
Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simu- lators. https://openai.com/research/video- generation - models - as - world - simulators,
-
[8]
Mingdeng Cao, Chong Mou, Fanghua Yu, Xintao Wang, Yinqiang Zheng, Jian Zhang, Chao Dong, Gen Li, Ying Shan, Radu Timofte, et al. Ntire 2023 challenge on 360deg omnidirectional image and video super-resolution: Datasets, methods and results. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition Work- shop (CVPRW), pages 173...
work page 2023
Show all 81 references
-
[9]
Invertible diffusion models for compressed sensing
Bin Chen, Zhenyu Zhang, Weiqi Li, Chen Zhao, Jiwen Yu, Shijie Zhao, Jie Chen, and Jian Zhang. Invertible diffusion models for compressed sensing. arXiv preprint arXiv:2403.17006, 2024. 3
2024 arXiv
-
[10]
360+x: A panoptic multi- modal scene understanding dataset
Hao Chen, Yuqi Hou, Chenyuan Qu, Irene Testini, Xiao- han Hong, and Jianbo Jiao. 360+x: A panoptic multi- modal scene understanding dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19373–19382, 2024. 6
2024
-
[11]
Control-a-video: Controllable text-to-video generation with diffusion models
Weifeng Chen, Yatai Ji, Jie Wu, Hefeng Wu, Pan Xie, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video generation with diffusion models. arXiv preprint arXiv:2305.13840, 2023. 2
2023 arXiv
-
[12]
Text2light: Zero-shot text-driven hdr panorama generation
Zhaoxi Chen, Guangcong Wang, and Ziwei Liu. Text2light: Zero-shot text-driven hdr panorama generation. ACM Trans- actions on Graphics (TOG), 41(6):1–16, 2022. 3
2022
-
[13]
Hybrid transformer and cnn attention net- work for stereo image super-resolution
Ming Cheng, Haoyu Ma, Qiufang Ma, Xiaopeng Sun, Weiqi Li, Zhenyu Zhang, Xuhan Sheng, Shijie Zhao, Junlin Li, and Li Zhang. Hybrid transformer and cnn attention net- work for stereo image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...
2023
-
[14]
Inout: Diverse image outpainting via gan inversion
Yen-Chi Cheng, Chieh Hubert Lin, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, and Ming-Hsuan Yang. Inout: Diverse image outpainting via gan inversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11431–11440, 2022. 3
2022
-
[15]
Guided co-modulated gan for 360° field of view extrapolation
Mohammad Reza Karimi Dastjerdi, Yannick Hold-Geoffroy, Jonathan Eisenmann, Siavash Khodadadeh, and Jean- Franc ¸ois Lalonde. Guided co-modulated gan for 360° field of view extrapolation. In 2022 International Conference on 3D Vision (3DV), pages 475–485. IEEE, 2022. 3
2022
-
[16]
Lau-net: Latitude adaptive upscaling network for omnidirectional image super-resolution
Xin Deng, Hao Wang, Mai Xu, Yichen Guo, Yuhang Song, and Li Yang. Lau-net: Latitude adaptive upscaling network for omnidirectional image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 9189–9198, 2021. 7
2021
-
[17]
Diffusion mod- els beat gans on image synthesis
Prafulla Dhariwal and Alexander Nichol. Diffusion mod- els beat gans on image synthesis. Proceedings of the Ad- vances in Neural Information Processing Systems (NeurIPS), 34:8780–8794, 2021. 2
2021
-
[18]
Structure and content-guided video synthesis with diffusion models
Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7346–7356, 2023. 2
2023
-
[19]
Healpix: A framework for high- resolution discretization and fast analysis of data distributed on the sphere
Krzysztof M Gorski, Eric Hivon, Anthony J Banday, Ben- jamin D Wandelt, Frode K Hansen, Mstvos Reinecke, and Matthia Bartelmann. Healpix: A framework for high- resolution discretization and fast analysis of data distributed on the sphere. The Astrophysical Journal, 622(2):759,...
2005
-
[20]
Animatediff: Animate your personalized text-to- image diffusion models without specific tuning.Proceedings of the International Conference on Learning Representations (ICLR), 2024
Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning.Proceedings of the International Conference on Learning Representati...
2024
-
[21]
Controllable video generation with sparse trajectories
Zekun Hao, Xun Huang, and Serge Belongie. Controllable video generation with sparse trajectories. In Proceedings of 9 the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7854–7863, 2018. 2
2018
-
[22]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016. 4, 13
2016
-
[23]
Denoising dif- fusion probabilistic models
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) , 33: 6840–6851, 2020. 2
2020
-
[24]
Imagen video: High definition video generation with diffusion mod- els
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els. arXiv preprint arXiv:2210.02303, 2022. 2
-
[25]
Mo- tionmaster: Training-free camera motion transfer for video generation
Teng Hu, Jiangning Zhang, Ran Yi, Yating Wang, Hongrui Huang, Jieyu Weng, Yabiao Wang, and Lizhuang Ma. Mo- tionmaster: Training-free camera motion transfer for video generation. arXiv preprint arXiv:2404.15789, 2024. 4
2024 arXiv
-
[26]
Peekaboo: Interactive video generation via masked- diffusion
Yash Jain, Anshul Nasery, Vibhav Vineet, and Harkirat Behl. Peekaboo: Interactive video generation via masked- diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8079–8088, 2024. 3
2024
-
[27]
Taming encoder for zero fine-tuning image customization with text-to-image diffusion models
Xuhui Jia, Yang Zhao, Kelvin CK Chan, Yandong Li, Han Zhang, Boqing Gong, Tingbo Hou, Huisheng Wang, and Yu-Chuan Su. Taming encoder for zero fine-tuning image customization with text-to-image diffusion models. arXiv preprint arXiv:2304.02642, 2023. 3, 4
2023 arXiv
-
[28]
Co- tracker: It is better to track together
Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- tracker: It is better to track together. Proceedings of the Eu- ropean Conference on Computer Vision (ECCV), 2024. 5, 7, 13
2024
-
[29]
Elucidating the design space of diffusion-based generative models
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 35:26565–26577, 2022. 4
2022
-
[30]
Anyv2v: A tuning-free framework for any video-to- video editing tasks
Max Ku, Cong Wei, Weiming Ren, Harry Yang, and Wenhu Chen. Anyv2v: A tuning-free framework for any video-to- video editing tasks. arXiv preprint arXiv:2403.14468, 2024. 4
2024 arXiv
-
[31]
Panogen: Text-conditioned panoramic environment generation for vision-and-language navigation
Jialu Li and Mohit Bansal. Panogen: Text-conditioned panoramic environment generation for vision-and-language navigation. In Proceedings of the Advances in Neural Infor- mation Processing Systems (NeurIPS), 2023. 3
2023
-
[32]
4k4dgen: Panoramic 4d generation at 4k resolution
Renjie Li, Panwang Pan, Bangbang Yang, Dejia Xu, Shijie Zhou, Xuanyang Zhang, Zeming Li, Achuta Kadambi, Zhangyang Wang, and Zhiwen Fan. 4k4dgen: Panoramic 4d generation at 4k resolution. arXiv preprint arXiv:2406.13527, 2024. 3
2024 arXiv
-
[33]
Omnissr: Zero-shot omnidirectional image super-resolution using sta- ble diffusion model
Runyi Li, Xuhan Sheng, Weiqi Li, and Jian Zhang. Omnissr: Zero-shot omnidirectional image super-resolution using sta- ble diffusion model. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pages 198–216. Springer,
-
[34]
D3c2-net: Dual-domain deep convolutional coding network for compressive sensing
Weiqi Li, Bin Chen, Shuai Liu, Shijie Zhao, Bowen Du, Yongbing Zhang, and Jian Zhang. D3c2-net: Dual-domain deep convolutional coding network for compressive sensing. IEEE Transactions on Circuits and Systems for Video Tech- nology, 2024. 3
2024
-
[35]
Scenedreamer360: Text-driven 3d-consistent scene generation with panoramic gaussian splatting
Wenrui Li, Yapeng Mi, Fucheng Cai, Zhe Yang, Wangmeng Zuo, Xingtao Wang, and Xiaopeng Fan. Scenedreamer360: Text-driven 3d-consistent scene generation with panoramic gaussian splatting. arXiv preprint arXiv:2408.13711, 2024. 3
2024 arXiv
-
[36]
Resvr: Joint rescaling and view- port rendering of omnidirectional images
Weiqi Li, Shijie Zhao, Bin Chen, Xinhua Cheng, Junlin Li, Li Zhang, and Jian Zhang. Resvr: Joint rescaling and view- port rendering of omnidirectional images. In Proceedings of the 32nd ACM International Conference on Multimedia (ACM MM), pages 78–87, 2024. 5
2024
-
[37]
Coco- gan: Generation by parts via conditional coordinating
Chieh Hubert Lin, Chia-Che Chang, Yu-Sheng Chen, Da- Cheng Juan, Wei Wei, and Hwann-Tzong Chen. Coco- gan: Generation by parts via conditional coordinating. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4512–4521, 2019. 3
2019
-
[38]
Infinitygan: Towards infinite-pixel image synthesis
Chieh Hubert Lin, Hsin-Ying Lee, Yen-Chi Cheng, Sergey Tulyakov, and Ming-Hsuan Yang. Infinitygan: Towards infinite-pixel image synthesis. Proceedings of the Inter- national Conference on Learning Representations (ICLR) ,
-
[39]
Hybrid attention based residual network for pansharpening
Qin Liu, Letong Han, Rui Tan, Hongfei Fan, Weiqi Li, Hong- ming Zhu, Bowen Du, and Sicong Liu. Hybrid attention based residual network for pansharpening. Remote Sensing, 13(10):1962, 2021. 3
1962
-
[40]
Decoupled weight decay regularization
I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 7
2017 arXiv
-
[41]
Trailblazer: Trajectory control for diffusion-based video generation
Wan-Duo Kurt Ma, John P Lewis, and W Bastiaan Kleijn. Trailblazer: Trajectory control for diffusion-based video generation. arXiv preprint arXiv:2401.00896, 2023. 3
2023 arXiv
-
[42]
Revideo: Remake a video with motion and content control
Chong Mou, Mingdeng Cao, Xintao Wang, Zhaoyang Zhang, Ying Shan, and Jian Zhang. Revideo: Remake a video with motion and content control. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2024. 4, 5, 7
2024
-
[43]
T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models
Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 4296–4...
2024
-
[44]
Bips: Bi-modal in- door panorama synthesis via residual depth-aided adversar- ial learning
Changgyoon Oh, Wonjune Cho, Yujeong Chae, Daehee Park, Lin Wang, and Kuk-Jin Yoon. Bips: Bi-modal in- door panorama synthesis via residual depth-aided adversar- ial learning. In Proceedings of the European Conference on Computer Vision (ECCV), pages 352–371. Springer, 2022. 3
2022
-
[45]
Controlnext: Powerful and effi- cient control for image and video generation
Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation. arXiv preprint arXiv:2408.06070, 2024. 2, 4, 13
2024 arXiv
-
[46]
Freetraj: Tuning-free tra- jectory control in video diffusion models
Haonan Qiu, Zhaoxi Chen, Zhouxia Wang, Yingqing He, Menghan Xia, and Ziwei Liu. Freetraj: Tuning-free tra- jectory control in video diffusion models. arXiv preprint arXiv:2406.16863, 2024. 3
2024 arXiv
-
[47]
Learn- ing transferable visual models from natural language super- vision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, 10 Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In Proceedings of the International Conferen...
-
[48]
Hierarchical text-conditional image gener- ation with clip latents
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2
2022 arXiv
-
[49]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022. 2, 3, 4
2022
-
[50]
Photorealistic text-to-image diffusion models with deep language understanding
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Proceedings of the Advances in...
2022
-
[51]
pytorch-fid: Fid score for pytorch, 2020
Maximilian Seitzer. pytorch-fid: Fid score for pytorch, 2020. 7
2020
-
[52]
Panoflow: Learning 360° optical flow for surrounding tem- poral understanding
Hao Shi, Yifan Zhou, Kailun Yang, Xiaoting Yin, Ze Wang, Yaozu Ye, Zhe Yin, Shi Meng, Peng Li, and Kaiwei Wang. Panoflow: Learning 360° optical flow for surrounding tem- poral understanding. IEEE Transactions on Intelligent Transportation Systems (TITS), 24(5):5570–5585, 2023. 2
2023
-
[53]
Opdn: Omnidirectional position- aware deformable network for omnidirectional image super- resolution
Xiaopeng Sun, Weiqi Li, Zhenyu Zhang, Qiufang Ma, Xuhan Sheng, Ming Cheng, Haoyu Ma, Shijie Zhao, Jian Zhang, Junlin Li, et al. Opdn: Omnidirectional position- aware deformable network for omnidirectional image super- resolution. In Proceedings of the IEEE/CVF Conference on Co...
2023
-
[54]
Boundless: Generative adversarial networks for image ex- tension
Piotr Teterwak, Aaron Sarna, Dilip Krishnan, Aaron Maschinot, David Belanger, Ce Liu, and William T Freeman. Boundless: Generative adversarial networks for image ex- tension. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV) , pages 10521–10530,
-
[55]
To- wards accurate generative models of video: A new metric & challenges
Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018. 7
2018 arXiv
-
[56]
Neural discrete representation learning
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Proceedings of the Advances in Neu- ral Information Processing Systems (NeurIPS), 30, 2017. 3, 4
2017
-
[57]
Stylelight: Hdr panorama generation for light- ing estimation and editing
Guangcong Wang, Yinuo Yang, Chen Change Loy, and Zi- wei Liu. Stylelight: Hdr panorama generation for light- ing estimation and editing. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 477–492. Springer, 2022. 3
2022
-
[58]
Customizing 360-degree panoramas through text-to-image diffusion models
Hai Wang, Xiaoyu Xiang, Yuchen Fan, and Jing-Hao Xue. Customizing 360-degree panoramas through text-to-image diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision (WACV), pages 4933–4943, 2024. 3
2024
-
[59]
360-degree panorama generation from few unregis- tered nfov images
Jionghao Wang, Ziyu Chen, Jun Ling, Rong Xie, and Li Song. 360-degree panorama generation from few unregis- tered nfov images. In Proceedings of the 31th ACM Interna- tional Conference on Multimedia (ACM MM), 2023. 3
2023
-
[60]
Boximator: Generat- ing rich and controllable motions for video synthesis
Jiawei Wang, Yuchen Zhang, Jiaxin Zou, Yan Zeng, Guo- qiang Wei, Liping Yuan, and Hang Li. Boximator: Generat- ing rich and controllable motions for video synthesis. arXiv preprint arXiv:2402.01566, 2024. 3
2024 arXiv
-
[61]
360dvd: Controllable panorama video generation with 360-degree video diffusion model
Qian Wang, Weiqi Li, Chong Mou, Xinhua Cheng, and Jian Zhang. 360dvd: Controllable panorama video generation with 360-degree video diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 3, 6, 7, 8
2024
-
[62]
Motionctrl: A unified and flexible motion controller for video generation
Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Pa- pers, pages 1–11, 2024. 2, 3, 4, 6, 7, 13
2024
-
[63]
Cross-view panorama image synthesis
Songsong Wu, Hao Tang, Xiao-Yuan Jing, Haifeng Zhao, Jianjun Qian, Nicu Sebe, and Yan Yan. Cross-view panorama image synthesis. IEEE Transactions on Multimedia (TMM),
-
[64]
Panodif- fusion: 360-degree panorama outpainting via diffusion
Tianhao Wu, Chuanxia Zheng, and Tat-Jen Cham. Panodif- fusion: 360-degree panorama outpainting via diffusion. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. 3, 7
2024
-
[65]
Draganything: Motion control for any- thing using entity representation
Weijia Wu, Zhuang Li, Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. Draganything: Motion control for any- thing using entity representation. In Proceedings of the Eu- ropean Conference on Computer Vision (ECCV), pages 331–
-
[66]
Recognizing scene viewpoint using panoramic place representation
Jianxiong Xiao, Krista A Ehinger, Aude Oliva, and Anto- nio Torralba. Recognizing scene viewpoint using panoramic place representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2695–2702. IEEE, 2012. 1, 7
2012
-
[67]
Dynamicrafter: Animating open-domain images with video diffusion priors
Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Xin- tao Wang, Tien-Tsin Wong, and Ying Shan. Dynamicrafter: Animating open-domain images with video diffusion priors. Proceedings of the European Conference on Computer Vi- sion (ECCV), 2023. 2
2023
-
[68]
Layer- pano3d: Layered 3d panorama for hyper-immersive scene generation
Shuai Yang, Jing Tan, Mengchen Zhang, Tong Wu, Yixuan Li, Gordon Wetzstein, Ziwei Liu, and Dahua Lin. Layer- pano3d: Layered 3d panorama for hyper-immersive scene generation. arXiv preprint arXiv:2408.13252, 2024. 3
2024 arXiv
-
[69]
Fourier123: One image to high- quality 3d object generation with hybrid fourier score dis- tillation
Shuzhou Yang, Yu Wang, Haijie Li, Jiarui Meng, Xiandong Meng, and Jian Zhang. Fourier123: One image to high- quality 3d object generation with hybrid fourier score dis- tillation. arXiv preprint arXiv:2405.20669, 2024. 3
2024 arXiv
-
[70]
Diffpano: Scalable and con- sistent text to panorama generation with spherical epipolar- aware diffusion
Weicai Ye, Chenhao Ji, Zheng Chen, Junyao Gao, Xiaoshui Huang, Song-Hai Zhang, Wanli Ouyang, Tong He, Cairong Zhao, and Guofeng Zhang. Diffpano: Scalable and con- sistent text to panorama generation with spherical epipolar- aware diffusion. In Proceedings of the Advances in Ne...
2024
-
[71]
Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory
Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089, 2023. 2, 3, 4, 6, 7, 14, 15
2023 arXiv
-
[72]
Taming stable diffusion for text to 360 panorama image gen- eration
Cheng Zhang, Qianyi Wu, Camilo Cruz Gambardella, Xi- aoshui Huang, Dinh Phung, Wanli Ouyang, and Jianfei Cai. Taming stable diffusion for text to 360 panorama image gen- eration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , page...
2024
-
[73]
Adding conditional control to text-to-image diffusion models
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3836–3847, 2023. 2
2023
-
[74]
Diffcollage: Parallel generation of large content with diffusion models
Qinsheng Zhang, Jiaming Song, Xun Huang, Yongxin Chen, and Ming-Yu Liu. Diffcollage: Parallel generation of large content with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10188–10198. IEEE, 2023. 3
2023
-
[75]
I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models
Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023. 2
2023 arXiv
-
[76]
Editguard: Versatile image watermarking for tamper localization and copyright protection
Xuanyu Zhang, Runyi Li, Jiwen Yu, Youmin Xu, Weiqi Li, and Jian Zhang. Editguard: Versatile image watermarking for tamper localization and copyright protection. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11964–11974, 2024. 3
2024
-
[77]
V2a-mark: Versatile deep visual- audio watermarking for manipulation localization and copy- right protection
Xuanyu Zhang, Youmin Xu, Runyi Li, Jiwen Yu, Weiqi Li, Zhipei Xu, and Jian Zhang. V2a-mark: Versatile deep visual- audio watermarking for manipulation localization and copy- right protection. In Proceedings of the 32nd ACM Interna- tional Conference on Multimedia (ACM MM) , pa...
2024
-
[78]
Controlvideo: Training-free controllable text-to-video generation.Proceed- ings of the International Conference on Learning Represen- tations (ICLR), 2023
Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation.Proceed- ings of the International Conference on Learning Represen- tations (ICLR), 2023. 2
2023
-
[79]
Tora: Trajectory-oriented diffu- sion transformer for video generation
Zhenghao Zhang, Junchao Liao, Menghao Li, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffu- sion transformer for video generation. arXiv preprint arXiv:2407.21705, 2024. 3
2024 arXiv
-
[80]
Scalable 360 video stream delivery: Challenges, solutions, and opportunities
Michael Zink, Ramesh Sitaraman, and Klara Nahrstedt. Scalable 360 video stream delivery: Challenges, solutions, and opportunities. Proceedings of the IEEE , 107(4):639– 650, 2019. 1 12 OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation Supplementar...
2019
-
[348]
2, 3, 4, 6, 7, 14, 15
Springer, 2025. 2, 3, 4, 6, 7, 14, 15
2025
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.