Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read OmniDrag claims the first motion-control system for omnidirectional image-to-video generation, letting users drag points to steer both camera and objects.

desk verdict Useful and plausible extension of drag control to 360 video, but the accuracy claim rests on an unvalidated tracker used for both training and evaluation, so the quantitative evidence needs stronger grounding. read the letter →

arxiv 2412.09623 v1 pith:ZWRH4YBM submitted 2024-12-12 cs.CV

classification cs.CV
keywords omnidirectionalvideogenerationimage-to-videomotioncontroldrag-styleinteractionsphericalestimationequirectangularprojectiondiffusionmodelstrajectory
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a drag-style interaction—clicking a handle point and a target point on a 360° reference image—can control both scene-level and object-level motion in generated omnidirectional video, something prior trajectory-based methods could not do without spatial distortion. The reason prior methods fail, it argues, is that they treat the equirectangular frame as a flat image and ignore spherical geometry, so motion signals are oversampled near the poles and paths are wrong. OmniDrag answers with a spherical motion estimator that initializes tracking points uniformly on the sphere, selects trajectories by great-circle distance, and interpolates user trajectories along spherical paths, plus a lightweight controller jointly fine-tuned with temporal attention to learn spherical motion. If true, this gives VR and 360° content creators a simple, visual way to specify camera moves and object motion instead of writing text prompts or drawing long paths.

What carries the argument

The load-bearing components are the spherical motion estimator (SME) and the Omni Controller. SME uses a HEALPix equal-area iso-latitude grid to place tracking points uniformly on the sphere instead of on the distorted equirectangular image, tracks those points through the video with a point tracker, keeps trajectories whose spherical (great-circle) displacement exceeds a threshold, and at inference time reconstructs full trajectories from user-supplied handle and target points via spherical interpolation. The Omni Controller is a lightweight two-ResBlock convolutional module that encodes the trajectory condition and injects it into the first UNet block of Stable Video Diffusion using cross-normalization, while the temporal attention layers of the denoiser are jointly fine-tuned so the model learns spherical motion patterns rather than only 2D translations. The two together are what the paper credits for accurate, distortion-free control of both camera and object motion.

What would settle it

Render synthetic 360° videos with a known camera rotation or object translation, run the spherical motion estimator on them, and compare estimated trajectories with ground truth; if endpoint errors grow substantially for points near the poles or for large spherical rotations, the learned condition maps are built on wrong motion signals and control quality would be expected to degrade correspondingly.

Watch

Extended reading notes

Core claim

On its own terms, OmniDrag establishes that omnidirectional motion control is achievable by making every stage of the control pipeline spherical: uniform point initialization on the sphere, spherical-distance-based selection of dominant motions, spherical interpolation from user clicks, and joint fine-tuning of temporal attention layers with a lightweight condition-injection module on a diffusion image-to-video model. The method is claimed to be the first to support both scene-level and object-level drag control for omnidirectional video, and experiments report lower FVD on equirectangular frames, lower FID and FVD on viewports, and better human ratings than DragNUWA, MotionCtrl, and DragAnything.

Load-bearing premise

The whole training signal rests on the point tracker's output on equirectangular frames being a faithful measurement of true spherical motion, including in the heavily distorted polar regions; the paper does not provide a validation of tracking accuracy on omnidirectional content.

Editorial extensions

If this is right

  • A user can generate 360° videos with specified camera or object motion by drawing just two points, making motion control accessible for VR content creation.
  • Spherical-aware handling—uniform initialization, great-circle distances, and spherical interpolation—becomes the standard ingredient for ODV motion control, since the paper shows 2D-based alternatives distort.
  • Jointly fine-tuning temporal attention with the controller is necessary for learning spherical motion, so future ODV control models should not freeze the base UNet.
  • Training on a dataset with large motions, like Move360, is what enables scene-level control; low-motion datasets limit controllability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A straightforward next test is to measure how SME behaves on synthetic spherical motion (e.g., pure pan, pure tilt, or rotation around the optical axis); the paper's construction assumes great-circle paths, so non-geodesic motions like spirals or abrupt jerks may be out of reach for the current inference interpolation.
  • Because the inference stage interpolates between one handle and one target, fine-grained multi-point control—for instance, dragging along a curved path or shifting camera and object simultaneously—may require extending the estimator to several control points or user-drawn splines.
  • Since Move360 is captured by a car with four degrees of freedom, it is likely biased toward forward/backward and lateral translations; a dataset with more varied camera motions (tilts, rolls, orbits) would test whether OmniDrag generalizes beyond the capture setup.
  • If the CoTracker-based training signal turns out to be inaccurate in polar regions, an alternative estimator using spherical-aware tracking or projection to tangent planes could be plugged into the same Omni Controller architecture without changing the rest of the pipeline.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces OmniDrag, a method for drag-based motion control in omnidirectional image-to-video (ODV) generation. Building on Stable Video Diffusion, it adds a lightweight Omni Controller jointly fine-tuned with temporal attention layers, a spherical motion estimator (SME) for extracting training trajectories and estimating user trajectories via spherical interpolation, and the new Move360 dataset with large scene and object motions. Experiments compare against DragNUWA, MotionCtrl, and DragAnything on a 12-pair test set, reporting FID, FVD, an object-motion-control metric (ObjMC), human evaluation, and ablations of the proposed components.

Significance. If the central claims hold, OmniDrag would be a notable step toward practical controllable ODV generation, offering a drag-style interface that works on equirectangular images and supporting both scene- and object-level control. The SME design is well motivated and the component ablations in Tables 2 and Figures 6-7 directly support the importance of HEALPix initialization, spherical distance filtering, spherical interpolation, joint fine-tuning, and the Move360 dataset. The paper also provides a new dataset that addresses a real scarcity of high-motion ODV data. However, the strength of the evidence depends critically on whether CoTracker's tracks on ERP frames can be treated as ground-truth spherical motion, and on whether a 12-sample test set with no error bars can support the quantitative superiority claims.

major comments (3)
  1. [Sec. 3.4, Eq. (6); Sec. 4.1; Tables 1-2] The tracking function Ft in Eq. (6) is CoTracker applied directly to equirectangular (ERP) frames, and the ObjMC metric used in Tables 1 and 2 is computed with the same tracking methodology. The HEALPix initialization and spherical-distance filter only change which 2D tracks are kept; they do not correct tracking errors in the heavily distorted polar regions or across the 0/2π seam. The paper provides no validation that CoTracker yields accurate spherical trajectories on omnidirectional content (e.g., synthetic ERP videos with known ground truth, per-latitude error analysis, or a comparison against a spherical-aware tracker). Without such validation, the agreement between the training condition maps and the ObjMC evaluation could stem from a shared tracker bias rather than true spherical motion accuracy, leaving the central claim of 'accurate' control unsupported.
  2. [Sec. 4.2, test set description] The quantitative comparison is based on only twelve pairs of input as the test set, with no error bars, confidence intervals, or significance tests. For such a small sample, the reported differences in FID, FVD, ObjMC, and the human-preference percentages may not be statistically reliable. This weakens the claim of 'significant superiority' over DragNUWA, MotionCtrl, and DragAnything. The authors should report per-sample results or bootstrap confidence intervals, and also clarify the number of human evaluation samples and the evaluation protocol.
  3. [Sec. 4.3, Table 2, 'w/o Move360 Dataset' row] The ablation without Move360 achieves a better FID (167.56) than the full OmniDrag (171.41) and a comparable FVD (941.58 vs 933.73), while having much worse ObjMC (0.327 vs 0.044). The paper acknowledges this but does not explain the mechanism; the abstract and conclusion claim 'high-quality' generation. If FID is better without Move360, the quality claim depends entirely on the motion-control metric, which is built on the same tracking stack as the training signal. The authors should discuss whether the FID difference is within noise, whether ObjMC alone should be the primary criterion for the paper's quality claim, and how the trade-off is resolved in Fig. 6.
minor comments (5)
  1. [Table 2] The header contains two typos: 'w/o Shperical Filter' and 'w/o Shperical Interp.' should be 'Spherical'.
  2. [Sec. 3.4, Eq. (7)] The notation for spherical coordinates is inconsistent: Eq. (7) uses 'ϕ' while the surrounding text uses 'φ'. Please unify.
  3. [Sec. 3.4, text after Ninit] The formula 'Ninit = 12 × Nside 2' is missing the superscript; it should read 'Nside^2'.
  4. [Sec. 3.4, Eq. (9)] The spherical interpolation formula is undefined for the degenerate cases ω = 0 and ω = π; please state how these are handled.
  5. [Sec. 4.2, human evaluation] The human evaluation description is underspecified: there is no information on how many videos each volunteer rated, the rating scale, or inter-rater agreement. This affects the interpretability of the 75.7% and 76.1% preference scores.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the spherical motion estimator is an external-tracking-plus-geometric-filtering pipeline, and the central claim is not equivalent to its own inputs.

full rationale

Walking the derivation chain, the spherical motion estimator (Sec. 3.4) uses CoTracker as the external tracking function Ft in Eq. (6), with HEALPix initialization and spherical-distance filtering (Eqs. (7)-(8)) as geometric preprocessing, and spherical interpolation (Eq. (9)) as the inference-time trajectory model. None of these quantities is defined in terms of the evaluation metrics or fitted to them. The training condition maps are extracted from real ODV training videos, not from the test set or from the reported ObjMC values. The central claim of accurate motion control is supported by FVD/FID, viewport metrics, and an independent human evaluation (76.1% motion matching), so it does not rest on a self-consistent training/evaluation loop. The overlapping-author citations (ReVideo [42], 360DVD [61]) supply implementation details—trajectory-count sampling and latent rotation—and are not load-bearing for the central claim. The closest candidate for circularity is that ObjMC shares CoTracker with the training-condition extraction; however, CoTracker is a frozen, externally pretrained model and the diffusion loss is not optimized against ObjMC, so a shared tracker is a metric-validity concern, especially given unvalidated equirectangular behavior, rather than an equation-level circular reduction. The Limitations section's statement that decoupling camera- and object-level motion remains open is an acknowledged limitation, not a circular step. Therefore the paper's derivation is self-contained against external benchmarks and no significant circularity is found.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several hand-chosen hyperparameters (dth, Nside, Nsamp, gamma) and on domain assumptions about equirectangular tracking and spherical interpolation. No invented physical entities are introduced; Move360 is a dataset, not a postulated entity.

free parameters (4)
  • dth = not reported
    Spherical-distance threshold in Eq. (8) that filters training trajectories; chosen by hand and not reported, it directly affects which motions the model learns to control.
  • Nside = not reported
    HEALPix resolution parameter controlling the number of initialized tracking points Ninit = 12 * Nside^2; affects the density and coverage of training trajectories.
  • Nsamp = random between 1 and 10
    Number of sampled trajectories per training video, following ReVideo; controls the sparsity of the trajectory condition map.
  • gamma = not reported
    Scaling hyper-parameter in the cross-normalization equation, Eq. (5); controls the magnitude of injected control features.
assumptions (5)
  • domain assumption Equirectangular projection with the spherical coordinate conversion in Eq. (7) correctly represents distances and motions on the viewing sphere.
    SME's spherical distance filter and inference interpolation both rely on this mapping; inaccuracies would bias trajectory selection. Invoked in Sec. 3.4, Eq. (7).
  • domain assumption CoTracker point tracks computed on ERP frames correspond to genuine physical point motions in the 360 degree scene.
    Training trajectories T = Ft(P0, V) in Eq. (6) are taken as ground truth motion; no validation of tracking accuracy on omnidirectional ERP content is provided. Invoked in Sec. 3.4, Eq. (6).
  • domain assumption Stable Video Diffusion's pretrained video prior is a suitable base for ODV generation after fine-tuning a controller and temporal attention layers.
    The method inherits SVD's biases and resolution limits, as stated in Sec. 3.1 and acknowledged in the Limitations section.
  • ad hoc to paper Spherical interpolation in Eq. (9) matches user intentions and aligns inference-time control signals with training-time SME trajectories.
    This is a new modeling choice introduced by the paper; if the interpolated path deviates from the user's intended spherical path, control quality degrades. Invoked in Sec. 3.4, Eq. (9).
  • standard math HEALPix provides a uniform initialization for point tracking on the sphere.
    Borrowed from astronomy; the equal-area property is well established. Invoked in Sec. 3.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation." pith.science (2026). https://pith.science/paper/ZWRH4YBM

@misc{pith2026241209623,
  author       = {Pith},
  title        = {Pith review of: OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZWRH4YBM}},
  note         = {Machine review of arXiv:2412.09623}
}
read the original abstract

As virtual reality gains popularity, the demand for controllable creation of immersive and dynamic omnidirectional videos (ODVs) is increasing. While previous text-to-ODV generation methods achieve impressive results, they struggle with content inaccuracies and inconsistencies due to reliance solely on textual inputs. Although recent motion control techniques provide fine-grained control for video generation, directly applying these methods to ODVs often results in spatial distortion and unsatisfactory performance, especially with complex spherical motions. To tackle these challenges, we propose OmniDrag, the first approach enabling both scene- and object-level motion control for accurate, high-quality omnidirectional image-to-video generation. Building on pretrained video diffusion models, we introduce an omnidirectional control module, which is jointly fine-tuned with temporal attention layers to effectively handle complex spherical motion. In addition, we develop a novel spherical motion estimator that accurately extracts motion-control signals and allows users to perform drag-style ODV generation by simply drawing handle and target points. We also present a new dataset, named Move360, addressing the scarcity of ODV data with large scene and object motions. Experiments demonstrate the significant superiority of OmniDrag in achieving holistic scene-level and fine-grained object-level control for ODV generation. The project page is available at https://lwq20020127.github.io/OmniDrag.

Figures

Figures reproduced from arXiv: 2412.09623 by the authors.

Figure 1
Figure 1. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overall pipeline of proposed OmniDrag. (a) During training, spherical motion is extracted by the proposed spherical motion estimator. The Omni Controller and temporal attention layers in the UNet denoiser are jointly fine-tuned. (b) During inference, OmniDrag allows users to simply select handle and target points on the reference image and generates ODVs with the corresponding motion. proves ControlNet for lightweig… view at source ↗
Figure 3
Figure 3. Illustration of our spherical motion estimator (SME). In the training stage, given the input video V, P 0 is firstly initial￾ized through equal area iso-latitude pixelation. Then trajectories T are tracked, and finally filtered as T ′ according to spherical dis￾tance via Eqs. (6-8). During inference, given point pairs by users, the trajectories are estimated through spherical interpolation. where ym represents the u… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Our Move360 dataset. (a) We mount Insta360 Titan on a filming car, enabling its movement along four degrees of freedom. (b) Sample frames from the Move360 dataset showcasing a wide range of scenes, including indoor spaces, green landscapes, urban environments, and nigh…
Figure 5
Figure 5. Figure 5: Visual comparisons between DragNUWA [71], MotionCtrl [62], DragAnything [65], and our OmniDrag. Our SME estimates reasonable trajectories on the sphere, and OmniDrag achieves precise and stable control under both scene-level (the top case: go forward on the road) and o…
Figure 6
Figure 6. Figure 6: Ablation study on jointly fine-tuning temporal attention layers, and training with proposed Move360 dataset. For each ERP image, we show a corresponding viewport at specific perspective. dataset named Move360. Specifically, we mount an In￾sta360 Titan camera on a filmi…
Figure 7
Figure 7. Figure 7: Ablation study on proposed spherical motion estimator (SME). The “w/o HEALPix init.” variant fails to control the car, the “w/o spherical dist. filter” variant generates unstable result, and the “w/o spherical interp.” variant leads to unintended path. In contrast, our…
Figure 8
Figure 8. Figure 8: Illustration of our Omni Controller. Our Omni Con [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Some sample videos in our Move360 dataset. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Visual comparisons of between DragNUWA [71], DragAnything [65], and our OmniDrag. For each ERP image, we show two corresponding viewports at specific perspectives. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Visual comparisons of between DragNUWA [71], DragAnything [65], and our OmniDrag on the same reference image under different drag controls. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning

    cs.CV 2025-06 conditional novelty 6.0 of 10

    VQ-Insight uses progressive reinforcement learning with temporal shuffle and task rewards to teach a vision-language model to score and compare AI-generated videos, with gains on multiple video quality benchmarks.

  2. MIND-Edit: MLLM Insight-Driven Editing via Language-Vision Projection

    cs.CV 2025-05 reject novelty 4.0 of 10

    MIND-Edit combines instruction rewriting with MLLM-derived visual embeddings to guide diffusion-based image editing, but the reported numbers only partly support the claim of state-of-the-art performance.

Reference graph

Works this paper leans on

81 extracted references · 56 canonical work pages · cited by 2 Pith papers

  1. [1]

    Deep learning for omnidirectional vision: A survey and new perspectives

    Hao Ai, Zidong Cao, Jinjing Zhu, Haotian Bai, Yucheng Chen, and Lin Wang. Deep learning for omnidirectional vision: A survey and new perspectives. arXiv preprint arXiv:2205.10468, 2022. 2

  2. [2]

    Dream360: Diverse and immersive outdoor virtual scene creation via transformer-based 360° image outpainting

    Hao Ai, Zidong Cao, Haonan Lu, Chen Chen, Jian Ma, Pengyuan Zhou, Tae-Kyun Kim, Pan Hui, and Lin Wang. Dream360: Diverse and immersive outdoor virtual scene creation via transformer-based 360° image outpainting. IEEE Transactions on Visualization and Computer Graphics (TVCG), 2024. 3

  3. [3]

    Di- verse plausible 360-degree image outpainting for efficient 3dcg background creation

    Naofumi Akimoto, Yuhi Matsuo, and Yoshimitsu Aoki. Di- verse plausible 360-degree image outpainting for efficient 3dcg background creation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11441–11450, 2022. 3

  4. [4]

    Uniedit: A unified tuning- free framework for video motion and appearance editing

    Jianhong Bai, Tianyu He, Yuchi Wang, Junliang Guo, Haoji Hu, Zuozhu Liu, and Jiang Bian. Uniedit: A unified tuning- free framework for video motion and appearance editing. arXiv preprint arXiv:2402.13185, 2024. 4

  5. [5]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. 2, 3, 7

  6. [6]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22563–22575, 2023. 2

  7. [7]

    Video generation models as world simu- lators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simu- lators. https://openai.com/research/video- generation - models - as - world - simulators,

  8. [8]

    Ntire 2023 challenge on 360deg omnidirectional image and video super-resolution: Datasets, methods and results

    Mingdeng Cao, Chong Mou, Fanghua Yu, Xintao Wang, Yinqiang Zheng, Jian Zhang, Chao Dong, Gen Li, Ying Shan, Radu Timofte, et al. Ntire 2023 challenge on 360deg omnidirectional image and video super-resolution: Datasets, methods and results. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition Work- shop (CVPRW), pages 173...

Show all 81 references
  1. [9]

    Invertible diffusion models for compressed sensing

    Bin Chen, Zhenyu Zhang, Weiqi Li, Chen Zhao, Jiwen Yu, Shijie Zhao, Jie Chen, and Jian Zhang. Invertible diffusion models for compressed sensing. arXiv preprint arXiv:2403.17006, 2024. 3

  2. [10]

    360+x: A panoptic multi- modal scene understanding dataset

    Hao Chen, Yuqi Hou, Chenyuan Qu, Irene Testini, Xiao- han Hong, and Jianbo Jiao. 360+x: A panoptic multi- modal scene understanding dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19373–19382, 2024. 6

  3. [11]

    Control-a-video: Controllable text-to-video generation with diffusion models

    Weifeng Chen, Yatai Ji, Jie Wu, Hefeng Wu, Pan Xie, Jiashi Li, Xin Xia, Xuefeng Xiao, and Liang Lin. Control-a-video: Controllable text-to-video generation with diffusion models. arXiv preprint arXiv:2305.13840, 2023. 2

  4. [12]

    Text2light: Zero-shot text-driven hdr panorama generation

    Zhaoxi Chen, Guangcong Wang, and Ziwei Liu. Text2light: Zero-shot text-driven hdr panorama generation. ACM Trans- actions on Graphics (TOG), 41(6):1–16, 2022. 3

  5. [13]

    Hybrid transformer and cnn attention net- work for stereo image super-resolution

    Ming Cheng, Haoyu Ma, Qiufang Ma, Xiaopeng Sun, Weiqi Li, Zhenyu Zhang, Xuhan Sheng, Shijie Zhao, Junlin Li, and Li Zhang. Hybrid transformer and cnn attention net- work for stereo image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern...

  6. [14]

    Inout: Diverse image outpainting via gan inversion

    Yen-Chi Cheng, Chieh Hubert Lin, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, and Ming-Hsuan Yang. Inout: Diverse image outpainting via gan inversion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11431–11440, 2022. 3

  7. [15]

    Guided co-modulated gan for 360° field of view extrapolation

    Mohammad Reza Karimi Dastjerdi, Yannick Hold-Geoffroy, Jonathan Eisenmann, Siavash Khodadadeh, and Jean- Franc ¸ois Lalonde. Guided co-modulated gan for 360° field of view extrapolation. In 2022 International Conference on 3D Vision (3DV), pages 475–485. IEEE, 2022. 3

  8. [16]

    Lau-net: Latitude adaptive upscaling network for omnidirectional image super-resolution

    Xin Deng, Hao Wang, Mai Xu, Yichen Guo, Yuhang Song, and Li Yang. Lau-net: Latitude adaptive upscaling network for omnidirectional image super-resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR), pages 9189–9198, 2021. 7

  9. [17]

    Diffusion mod- els beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion mod- els beat gans on image synthesis. Proceedings of the Ad- vances in Neural Information Processing Systems (NeurIPS), 34:8780–8794, 2021. 2

  10. [18]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 7346–7356, 2023. 2

  11. [19]

    Healpix: A framework for high- resolution discretization and fast analysis of data distributed on the sphere

    Krzysztof M Gorski, Eric Hivon, Anthony J Banday, Ben- jamin D Wandelt, Frode K Hansen, Mstvos Reinecke, and Matthia Bartelmann. Healpix: A framework for high- resolution discretization and fast analysis of data distributed on the sphere. The Astrophysical Journal, 622(2):759,...

  12. [20]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning.Proceedings of the International Conference on Learning Representations (ICLR), 2024

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning.Proceedings of the International Conference on Learning Representati...

  13. [21]

    Controllable video generation with sparse trajectories

    Zekun Hao, Xun Huang, and Serge Belongie. Controllable video generation with sparse trajectories. In Proceedings of 9 the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7854–7863, 2018. 2

  14. [22]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016. 4, 13

  15. [23]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS) , 33: 6840–6851, 2020. 2

  16. [24]

    Imagen video: High definition video generation with diffusion mod- els

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els. arXiv preprint arXiv:2210.02303, 2022. 2

  17. [25]

    Mo- tionmaster: Training-free camera motion transfer for video generation

    Teng Hu, Jiangning Zhang, Ran Yi, Yating Wang, Hongrui Huang, Jieyu Weng, Yabiao Wang, and Lizhuang Ma. Mo- tionmaster: Training-free camera motion transfer for video generation. arXiv preprint arXiv:2404.15789, 2024. 4

  18. [26]

    Peekaboo: Interactive video generation via masked- diffusion

    Yash Jain, Anshul Nasery, Vibhav Vineet, and Harkirat Behl. Peekaboo: Interactive video generation via masked- diffusion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8079–8088, 2024. 3

  19. [27]

    Taming encoder for zero fine-tuning image customization with text-to-image diffusion models

    Xuhui Jia, Yang Zhao, Kelvin CK Chan, Yandong Li, Han Zhang, Boqing Gong, Tingbo Hou, Huisheng Wang, and Yu-Chuan Su. Taming encoder for zero fine-tuning image customization with text-to-image diffusion models. arXiv preprint arXiv:2304.02642, 2023. 3, 4

  20. [28]

    Co- tracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- tracker: It is better to track together. Proceedings of the Eu- ropean Conference on Computer Vision (ECCV), 2024. 5, 7, 13

  21. [29]

    Elucidating the design space of diffusion-based generative models

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 35:26565–26577, 2022. 4

  22. [30]

    Anyv2v: A tuning-free framework for any video-to- video editing tasks

    Max Ku, Cong Wei, Weiming Ren, Harry Yang, and Wenhu Chen. Anyv2v: A tuning-free framework for any video-to- video editing tasks. arXiv preprint arXiv:2403.14468, 2024. 4

  23. [31]

    Panogen: Text-conditioned panoramic environment generation for vision-and-language navigation

    Jialu Li and Mohit Bansal. Panogen: Text-conditioned panoramic environment generation for vision-and-language navigation. In Proceedings of the Advances in Neural Infor- mation Processing Systems (NeurIPS), 2023. 3

  24. [32]

    4k4dgen: Panoramic 4d generation at 4k resolution

    Renjie Li, Panwang Pan, Bangbang Yang, Dejia Xu, Shijie Zhou, Xuanyang Zhang, Zeming Li, Achuta Kadambi, Zhangyang Wang, and Zhiwen Fan. 4k4dgen: Panoramic 4d generation at 4k resolution. arXiv preprint arXiv:2406.13527, 2024. 3

  25. [33]

    Omnissr: Zero-shot omnidirectional image super-resolution using sta- ble diffusion model

    Runyi Li, Xuhan Sheng, Weiqi Li, and Jian Zhang. Omnissr: Zero-shot omnidirectional image super-resolution using sta- ble diffusion model. In Proceedings of the European Confer- ence on Computer Vision (ECCV), pages 198–216. Springer,

  26. [34]

    D3c2-net: Dual-domain deep convolutional coding network for compressive sensing

    Weiqi Li, Bin Chen, Shuai Liu, Shijie Zhao, Bowen Du, Yongbing Zhang, and Jian Zhang. D3c2-net: Dual-domain deep convolutional coding network for compressive sensing. IEEE Transactions on Circuits and Systems for Video Tech- nology, 2024. 3

  27. [35]

    Scenedreamer360: Text-driven 3d-consistent scene generation with panoramic gaussian splatting

    Wenrui Li, Yapeng Mi, Fucheng Cai, Zhe Yang, Wangmeng Zuo, Xingtao Wang, and Xiaopeng Fan. Scenedreamer360: Text-driven 3d-consistent scene generation with panoramic gaussian splatting. arXiv preprint arXiv:2408.13711, 2024. 3

  28. [36]

    Resvr: Joint rescaling and view- port rendering of omnidirectional images

    Weiqi Li, Shijie Zhao, Bin Chen, Xinhua Cheng, Junlin Li, Li Zhang, and Jian Zhang. Resvr: Joint rescaling and view- port rendering of omnidirectional images. In Proceedings of the 32nd ACM International Conference on Multimedia (ACM MM), pages 78–87, 2024. 5

  29. [37]

    Coco- gan: Generation by parts via conditional coordinating

    Chieh Hubert Lin, Chia-Che Chang, Yu-Sheng Chen, Da- Cheng Juan, Wei Wei, and Hwann-Tzong Chen. Coco- gan: Generation by parts via conditional coordinating. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4512–4521, 2019. 3

  30. [38]

    Infinitygan: Towards infinite-pixel image synthesis

    Chieh Hubert Lin, Hsin-Ying Lee, Yen-Chi Cheng, Sergey Tulyakov, and Ming-Hsuan Yang. Infinitygan: Towards infinite-pixel image synthesis. Proceedings of the Inter- national Conference on Learning Representations (ICLR) ,

  31. [39]

    Hybrid attention based residual network for pansharpening

    Qin Liu, Letong Han, Rui Tan, Hongfei Fan, Weiqi Li, Hong- ming Zhu, Bowen Du, and Sicong Liu. Hybrid attention based residual network for pansharpening. Remote Sensing, 13(10):1962, 2021. 3

  32. [40]

    Decoupled weight decay regularization

    I Loshchilov. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 7

  33. [41]

    Trailblazer: Trajectory control for diffusion-based video generation

    Wan-Duo Kurt Ma, John P Lewis, and W Bastiaan Kleijn. Trailblazer: Trajectory control for diffusion-based video generation. arXiv preprint arXiv:2401.00896, 2023. 3

  34. [42]

    Revideo: Remake a video with motion and content control

    Chong Mou, Mingdeng Cao, Xintao Wang, Zhaoyang Zhang, Ying Shan, and Jian Zhang. Revideo: Remake a video with motion and content control. In Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), 2024. 4, 5, 7

  35. [43]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 4296–4...

  36. [44]

    Bips: Bi-modal in- door panorama synthesis via residual depth-aided adversar- ial learning

    Changgyoon Oh, Wonjune Cho, Yujeong Chae, Daehee Park, Lin Wang, and Kuk-Jin Yoon. Bips: Bi-modal in- door panorama synthesis via residual depth-aided adversar- ial learning. In Proceedings of the European Conference on Computer Vision (ECCV), pages 352–371. Springer, 2022. 3

  37. [45]

    Controlnext: Powerful and effi- cient control for image and video generation

    Bohao Peng, Jian Wang, Yuechen Zhang, Wenbo Li, Ming- Chang Yang, and Jiaya Jia. Controlnext: Powerful and effi- cient control for image and video generation. arXiv preprint arXiv:2408.06070, 2024. 2, 4, 13

  38. [46]

    Freetraj: Tuning-free tra- jectory control in video diffusion models

    Haonan Qiu, Zhaoxi Chen, Zhouxia Wang, Yingqing He, Menghan Xia, and Ziwei Liu. Freetraj: Tuning-free tra- jectory control in video diffusion models. arXiv preprint arXiv:2406.16863, 2024. 3

  39. [47]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, 10 Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In Proceedings of the International Conferen...

  40. [48]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

  41. [49]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022. 2, 3, 4

  42. [50]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Proceedings of the Advances in...

  43. [51]

    pytorch-fid: Fid score for pytorch, 2020

    Maximilian Seitzer. pytorch-fid: Fid score for pytorch, 2020. 7

  44. [52]

    Panoflow: Learning 360° optical flow for surrounding tem- poral understanding

    Hao Shi, Yifan Zhou, Kailun Yang, Xiaoting Yin, Ze Wang, Yaozu Ye, Zhe Yin, Shi Meng, Peng Li, and Kaiwei Wang. Panoflow: Learning 360° optical flow for surrounding tem- poral understanding. IEEE Transactions on Intelligent Transportation Systems (TITS), 24(5):5570–5585, 2023. 2

  45. [53]

    Opdn: Omnidirectional position- aware deformable network for omnidirectional image super- resolution

    Xiaopeng Sun, Weiqi Li, Zhenyu Zhang, Qiufang Ma, Xuhan Sheng, Ming Cheng, Haoyu Ma, Shijie Zhao, Jian Zhang, Junlin Li, et al. Opdn: Omnidirectional position- aware deformable network for omnidirectional image super- resolution. In Proceedings of the IEEE/CVF Conference on Co...

  46. [54]

    Boundless: Generative adversarial networks for image ex- tension

    Piotr Teterwak, Aaron Sarna, Dilip Krishnan, Aaron Maschinot, David Belanger, Ce Liu, and William T Freeman. Boundless: Generative adversarial networks for image ex- tension. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision (ICCV) , pages 10521–10530,

  47. [55]

    To- wards accurate generative models of video: A new metric & challenges

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717, 2018. 7

  48. [56]

    Neural discrete representation learning

    Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Proceedings of the Advances in Neu- ral Information Processing Systems (NeurIPS), 30, 2017. 3, 4

  49. [57]

    Stylelight: Hdr panorama generation for light- ing estimation and editing

    Guangcong Wang, Yinuo Yang, Chen Change Loy, and Zi- wei Liu. Stylelight: Hdr panorama generation for light- ing estimation and editing. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 477–492. Springer, 2022. 3

  50. [58]

    Customizing 360-degree panoramas through text-to-image diffusion models

    Hai Wang, Xiaoyu Xiang, Yuchen Fan, and Jing-Hao Xue. Customizing 360-degree panoramas through text-to-image diffusion models. In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision (WACV), pages 4933–4943, 2024. 3

  51. [59]

    360-degree panorama generation from few unregis- tered nfov images

    Jionghao Wang, Ziyu Chen, Jun Ling, Rong Xie, and Li Song. 360-degree panorama generation from few unregis- tered nfov images. In Proceedings of the 31th ACM Interna- tional Conference on Multimedia (ACM MM), 2023. 3

  52. [60]

    Boximator: Generat- ing rich and controllable motions for video synthesis

    Jiawei Wang, Yuchen Zhang, Jiaxin Zou, Yan Zeng, Guo- qiang Wei, Liping Yuan, and Hang Li. Boximator: Generat- ing rich and controllable motions for video synthesis. arXiv preprint arXiv:2402.01566, 2024. 3

  53. [61]

    360dvd: Controllable panorama video generation with 360-degree video diffusion model

    Qian Wang, Weiqi Li, Chong Mou, Xinhua Cheng, and Jian Zhang. 360dvd: Controllable panorama video generation with 360-degree video diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 2, 3, 6, 7, 8

  54. [62]

    Motionctrl: A unified and flexible motion controller for video generation

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In ACM SIGGRAPH 2024 Conference Pa- pers, pages 1–11, 2024. 2, 3, 4, 6, 7, 13

  55. [63]

    Cross-view panorama image synthesis

    Songsong Wu, Hao Tang, Xiao-Yuan Jing, Haifeng Zhao, Jianjun Qian, Nicu Sebe, and Yan Yan. Cross-view panorama image synthesis. IEEE Transactions on Multimedia (TMM),

  56. [64]

    Panodif- fusion: 360-degree panorama outpainting via diffusion

    Tianhao Wu, Chuanxia Zheng, and Tat-Jen Cham. Panodif- fusion: 360-degree panorama outpainting via diffusion. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. 3, 7

  57. [65]

    Draganything: Motion control for any- thing using entity representation

    Weijia Wu, Zhuang Li, Yuchao Gu, Rui Zhao, Yefei He, David Junhao Zhang, Mike Zheng Shou, Yan Li, Tingting Gao, and Di Zhang. Draganything: Motion control for any- thing using entity representation. In Proceedings of the Eu- ropean Conference on Computer Vision (ECCV), pages 331–

  58. [66]

    Recognizing scene viewpoint using panoramic place representation

    Jianxiong Xiao, Krista A Ehinger, Aude Oliva, and Anto- nio Torralba. Recognizing scene viewpoint using panoramic place representation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2695–2702. IEEE, 2012. 1, 7

  59. [67]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Xin- tao Wang, Tien-Tsin Wong, and Ying Shan. Dynamicrafter: Animating open-domain images with video diffusion priors. Proceedings of the European Conference on Computer Vi- sion (ECCV), 2023. 2

  60. [68]

    Layer- pano3d: Layered 3d panorama for hyper-immersive scene generation

    Shuai Yang, Jing Tan, Mengchen Zhang, Tong Wu, Yixuan Li, Gordon Wetzstein, Ziwei Liu, and Dahua Lin. Layer- pano3d: Layered 3d panorama for hyper-immersive scene generation. arXiv preprint arXiv:2408.13252, 2024. 3

  61. [69]

    Fourier123: One image to high- quality 3d object generation with hybrid fourier score dis- tillation

    Shuzhou Yang, Yu Wang, Haijie Li, Jiarui Meng, Xiandong Meng, and Jian Zhang. Fourier123: One image to high- quality 3d object generation with hybrid fourier score dis- tillation. arXiv preprint arXiv:2405.20669, 2024. 3

  62. [70]

    Diffpano: Scalable and con- sistent text to panorama generation with spherical epipolar- aware diffusion

    Weicai Ye, Chenhao Ji, Zheng Chen, Junyao Gao, Xiaoshui Huang, Song-Hai Zhang, Wanli Ouyang, Tong He, Cairong Zhao, and Guofeng Zhang. Diffpano: Scalable and con- sistent text to panorama generation with spherical epipolar- aware diffusion. In Proceedings of the Advances in Ne...

  63. [71]

    Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089, 2023. 2, 3, 4, 6, 7, 14, 15

  64. [72]

    Taming stable diffusion for text to 360 panorama image gen- eration

    Cheng Zhang, Qianyi Wu, Camilo Cruz Gambardella, Xi- aoshui Huang, Dinh Phung, Wanli Ouyang, and Jianfei Cai. Taming stable diffusion for text to 360 panorama image gen- eration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , page...

  65. [73]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 3836–3847, 2023. 2

  66. [74]

    Diffcollage: Parallel generation of large content with diffusion models

    Qinsheng Zhang, Jiaming Song, Xun Huang, Yongxin Chen, and Ming-Yu Liu. Diffcollage: Parallel generation of large content with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10188–10198. IEEE, 2023. 3

  67. [75]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023. 2

  68. [76]

    Editguard: Versatile image watermarking for tamper localization and copyright protection

    Xuanyu Zhang, Runyi Li, Jiwen Yu, Youmin Xu, Weiqi Li, and Jian Zhang. Editguard: Versatile image watermarking for tamper localization and copyright protection. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11964–11974, 2024. 3

  69. [77]

    V2a-mark: Versatile deep visual- audio watermarking for manipulation localization and copy- right protection

    Xuanyu Zhang, Youmin Xu, Runyi Li, Jiwen Yu, Weiqi Li, Zhipei Xu, and Jian Zhang. V2a-mark: Versatile deep visual- audio watermarking for manipulation localization and copy- right protection. In Proceedings of the 32nd ACM Interna- tional Conference on Multimedia (ACM MM) , pa...

  70. [78]

    Controlvideo: Training-free controllable text-to-video generation.Proceed- ings of the International Conference on Learning Represen- tations (ICLR), 2023

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation.Proceed- ings of the International Conference on Learning Represen- tations (ICLR), 2023. 2

  71. [79]

    Tora: Trajectory-oriented diffu- sion transformer for video generation

    Zhenghao Zhang, Junchao Liao, Menghao Li, Long Qin, and Weizhi Wang. Tora: Trajectory-oriented diffu- sion transformer for video generation. arXiv preprint arXiv:2407.21705, 2024. 3

  72. [80]

    Scalable 360 video stream delivery: Challenges, solutions, and opportunities

    Michael Zink, Ramesh Sitaraman, and Klara Nahrstedt. Scalable 360 video stream delivery: Challenges, solutions, and opportunities. Proceedings of the IEEE , 107(4):639– 650, 2019. 1 12 OmniDrag: Enabling Motion Control for Omnidirectional Image-to-Video Generation Supplementar...

  73. [348]

    2, 3, 4, 6, 7, 14, 15

    Springer, 2025. 2, 3, 4, 6, 7, 14, 15

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.