{"id":"d000ea91-5034-4445-a28b-58be08a7afed","arxiv_id":"2412.09623","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A drag-style motion control method for 360 degree image-to-video generation, built on spherical trajectory estimation and joint fine-tuning of a pretrained video diffusion model.","lead":"OmniDrag lets a user click a starting point and a target point on a 360 degree photo, then generates a video in which the scene or an object moves along that path. It adapts drag-style motion control to omnidirectional video, which is harder than ordinary video because the spherical view is stored in a distorted flat format.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"CoTracker tracks on ERP frames are treated as ground-truth spherical motion in Eq. (6), and the same tracker is used for the ObjMC metric, so a systematic tracking bias would be invisible in the quantitative evidence supporting the central control-quality claim.","rationale":"The reader's weakest assumption correctly identifies the unvalidated CoTracker-on-ERP training signal as the most load-bearing point. I agree with the conditional verdict: the paper's method is plausible, the ablations are consistent with the design choices, and the qualitative results support the approach, but the absence of any validation that CoTracker tracks correspond to true spherical motion leaves the accuracy claim unverified. My concern strengthens the reader's point by noting that the same tracking paradigm is used in the ObjMC evaluation, making the quantitative evidence unable to reveal such a bias. The paper has real independent value in its proposed spherical-aware trajectory initialization and filtering, the Move360 dataset, and the user-friendly spherical interpolation, but none of these contributions address the underlying validity of the tracking function. A synthetic ground-truth experiment would settle the concern directly. Since the reader already marked the paper conditional, no verdict adjustment is needed; the concern is a reason to enforce the conditional, not to reject.","tokens_in":16474,"tokens_out":4820,"duration_ms":58932,"concrete_test":"Render synthetic omnidirectional video clips with known camera and object motion, using an engine such as Blender, with trajectories that deliberately include high-latitude and seam-crossing regions. Run the SME training pipeline (HEALPix initialization, CoTracker tracking, spherical-distance filtering) on the rendered ERP frames and compare extracted trajectories to ground-truth spherical trajectories, reporting per-latitude endpoint error and seam-crossing error. If error is small and uniform across latitude, the training signal is validated; if error is concentrated near the poles or the seam, the learned controller is trained on incorrect conditions and the ObjMC metric cannot certify control accuracy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that OmniDrag provides accurate spherical motion control rests on the assumption that CoTracker, applied directly to equirectangular frames, yields ground-truth spherical trajectories. Eq. (6) uses CoTracker as the tracking function Ft with no adaptation for spherical geometry; the HEALPix initialization in Sec. 3.4 only changes where points start, and the spherical-distance filter in Eqs. (7)-(8) only selects which tracks to keep. It does not correct tracking errors in the heavily distorted polar regions or across the 0/2π seam of the ERP format. CoTracker is a 2D correspondence model trained on perspective video, and its behavior on ERP input is unvalidated here. This matters doubly because ObjMC, the quantitative motion-control metric in Tables 1 and 2, is also based on the same CoTracker-style trajectory comparison. If CoTracker is biased on omnidirectional content, the training condition maps and the evaluation metric can agree with each other while both disagree with true spherical motion, making the quantitative superiority evidence circular. The paper contains no sanity check: no synthetic ODV with known ground truth, no comparison against a spherical-aware tracker or perspective re-projection, and no per-latitude accuracy analysis. The paper itself lists decoupling camera- and object-level motion as an open problem, which compounds the risk because Move360 is captured from a moving car, so CoTracker trajectories conflate ego-motion and object motion. Without independent validation of the tracking signal, the accuracy component of the central claim is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OmniDrag, a method for drag-based motion control in omnidirectional image-to-video (ODV) generation. Building on Stable Video Diffusion, it adds a lightweight Omni Controller jointly fine-tuned with temporal attention layers, a spherical motion estimator (SME) for extracting training trajectories and estimating user trajectories via spherical interpolation, and the new Move360 dataset with large scene and object motions. Experiments compare against DragNUWA, MotionCtrl, and DragAnything on a 12-pair test set, reporting FID, FVD, an object-motion-control metric (ObjMC), human evaluation, and ablations of the proposed components.","tokens_in":16786,"tokens_out":4845,"duration_ms":49078,"significance":"If the central claims hold, OmniDrag would be a notable step toward practical controllable ODV generation, offering a drag-style interface that works on equirectangular images and supporting both scene- and object-level control. The SME design is well motivated and the component ablations in Tables 2 and Figures 6-7 directly support the importance of HEALPix initialization, spherical distance filtering, spherical interpolation, joint fine-tuning, and the Move360 dataset. The paper also provides a new dataset that addresses a real scarcity of high-motion ODV data. However, the strength of the evidence depends critically on whether CoTracker's tracks on ERP frames can be treated as ground-truth spherical motion, and on whether a 12-sample test set with no error bars can support the quantitative superiority claims.","major_comments":[{"comment":"The tracking function Ft in Eq. (6) is CoTracker applied directly to equirectangular (ERP) frames, and the ObjMC metric used in Tables 1 and 2 is computed with the same tracking methodology. The HEALPix initialization and spherical-distance filter only change which 2D tracks are kept; they do not correct tracking errors in the heavily distorted polar regions or across the 0/2π seam. The paper provides no validation that CoTracker yields accurate spherical trajectories on omnidirectional content (e.g., synthetic ERP videos with known ground truth, per-latitude error analysis, or a comparison against a spherical-aware tracker). Without such validation, the agreement between the training condition maps and the ObjMC evaluation could stem from a shared tracker bias rather than true spherical motion accuracy, leaving the central claim of 'accurate' control unsupported.","section":"Sec. 3.4, Eq. (6); Sec. 4.1; Tables 1-2"},{"comment":"The quantitative comparison is based on only twelve pairs of input as the test set, with no error bars, confidence intervals, or significance tests. For such a small sample, the reported differences in FID, FVD, ObjMC, and the human-preference percentages may not be statistically reliable. This weakens the claim of 'significant superiority' over DragNUWA, MotionCtrl, and DragAnything. The authors should report per-sample results or bootstrap confidence intervals, and also clarify the number of human evaluation samples and the evaluation protocol.","section":"Sec. 4.2, test set description"},{"comment":"The ablation without Move360 achieves a better FID (167.56) than the full OmniDrag (171.41) and a comparable FVD (941.58 vs 933.73), while having much worse ObjMC (0.327 vs 0.044). The paper acknowledges this but does not explain the mechanism; the abstract and conclusion claim 'high-quality' generation. If FID is better without Move360, the quality claim depends entirely on the motion-control metric, which is built on the same tracking stack as the training signal. The authors should discuss whether the FID difference is within noise, whether ObjMC alone should be the primary criterion for the paper's quality claim, and how the trade-off is resolved in Fig. 6.","section":"Sec. 4.3, Table 2, 'w/o Move360 Dataset' row"}],"minor_comments":[{"comment":"The header contains two typos: 'w/o Shperical Filter' and 'w/o Shperical Interp.' should be 'Spherical'.","section":"Table 2"},{"comment":"The notation for spherical coordinates is inconsistent: Eq. (7) uses 'ϕ' while the surrounding text uses 'φ'. Please unify.","section":"Sec. 3.4, Eq. (7)"},{"comment":"The formula 'Ninit = 12 × Nside 2' is missing the superscript; it should read 'Nside^2'.","section":"Sec. 3.4, text after Ninit"},{"comment":"The spherical interpolation formula is undefined for the degenerate cases ω = 0 and ω = π; please state how these are handled.","section":"Sec. 3.4, Eq. (9)"},{"comment":"The human evaluation description is underspecified: there is no information on how many videos each volunteer rated, the rating scale, or inter-rater agreement. This affects the interpretability of the 75.7% and 76.1% preference scores.","section":"Sec. 4.2, human evaluation"}],"recommendation":"major_revision","confidential_remarks":"The paper relies heavily on CoTracker both for creating training conditions and for the ObjMC evaluation, making the evaluation potentially circular if CoTracker is biased on ERP content. I am not asking for an entirely new evaluation, but a targeted sanity check (e.g., synthetic ERP videos with known ground-truth spherical motion, or a per-latitude accuracy plot) would substantially increase confidence. The small test set and the better FID of the 'w/o Move360' ablation also need to be addressed. The dataset and code are not explicitly released in the paper, which limits reproducibility; please consider adding a clear availability statement. The high citation count of the authors' own prior work (e.g., ReVideo, 360DVD) is not problematic on its own, but the manuscript should make clear which components are reused verbatim versus newly proposed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something new and sensible: it adapts trajectory-based motion control to omnidirectional video. The HEALPix initialization, spherical-distance filtering, and spherical interpolation are real adaptations to the ERP domain, not just 2D methods applied blindly. Jointly fine-tuning temporal attention with the controller is also a reasonable choice, and the ablations support that decision. The Move360 dataset is a contribution in itself, even if it is not yet released.\n\nWhere the paper is soft is the evidence for the word “accurate” in the central claim. The training signal for the controller is CoTracker tracks on equirectangular frames, treated as ground-truth spherical motion in Eq. (6). The same kind of tracked trajectory is then used to compute ObjMC, the metric that shows the method’s superiority. If CoTracker is biased in the polar regions or across the seam, the training maps and the evaluation metric can agree with each other while both disagree with real spherical motion. The paper offers no sanity check: no synthetic ODV with known ground truth, no per-latitude accuracy analysis, no comparison against a spherical-aware tracker or perspective re-projection. That is a genuine gap, and it is addressable rather than fatal, but the authors should be pushed on it.\n\nThe quantitative evaluation is also thin: twelve test pairs, no error bars, no significance tests. The ablation table even shows the w/o Move360 variant achieving better FID while having much worse ObjMC, which tells you the metric is capturing motion matching but not overall quality. The paper’s own limitation about decoupling camera-level and object-level motion is honest, but it compounds the risk because Move360 was captured from a moving car, so CoTracker trajectories mix ego-motion and object motion.\n\nNone of this sinks the paper. The method is plausible, the qualitative results look consistent with the claims, and the design choices are well motivated. The citation pattern is fine; the self-citations to 360DVD and ReVideo are legitimate, and the core idea does not reduce to those works. The main weaknesses are evaluation discipline and the lack of validation for the tracking signal.\n\nFor readers in controllable video generation or VR content creation, this paper is worth a look. It deserves a serious referee, with the clear expectation that the tracker validation and evaluation robustness be addressed before publication. I would not desk-reject it, but I would not accept it as is.","headline":"Useful and plausible extension of drag control to 360 video, but the accuracy claim rests on an unvalidated tracker used for both training and evaluation, so the quantitative evidence needs stronger grounding.","tokens_in":675,"tokens_out":1110,"would_cite":false,"duration_ms":29143,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OmniDrag claims the first motion-control system for omnidirectional image-to-video generation, letting users drag points to steer both camera and objects.","keywords":["omnidirectional video generation","image-to-video generation","motion control","drag-style interaction","spherical motion estimation","equirectangular projection","diffusion models","trajectory control"],"falsifier":"Render synthetic 360° videos with a known camera rotation or object translation, run the spherical motion estimator on them, and compare estimated trajectories with ground truth; if endpoint errors grow substantially for points near the poles or for large spherical rotations, the learned condition maps are built on wrong motion signals and control quality would be expected to degrade correspondingly.","tokens_in":16288,"feed_emoji":"🌐","tokens_out":6643,"duration_ms":57446,"temperature":0.7,"pith_summary":"The paper claims that a drag-style interaction—clicking a handle point and a target point on a 360° reference image—can control both scene-level and object-level motion in generated omnidirectional video, something prior trajectory-based methods could not do without spatial distortion. The reason prior methods fail, it argues, is that they treat the equirectangular frame as a flat image and ignore spherical geometry, so motion signals are oversampled near the poles and paths are wrong. OmniDrag answers with a spherical motion estimator that initializes tracking points uniformly on the sphere, selects trajectories by great-circle distance, and interpolates user trajectories along spherical paths, plus a lightweight controller jointly fine-tuned with temporal attention to learn spherical motion. If true, this gives VR and 360° content creators a simple, visual way to specify camera moves and object motion instead of writing text prompts or drawing long paths.","feed_headline":"Drag control comes to 360° video generation","feed_subtitle":"OmniDrag maps user clicks to spherical trajectories for scene- and object-level control.","key_machinery":"The load-bearing components are the spherical motion estimator (SME) and the Omni Controller. SME uses a HEALPix equal-area iso-latitude grid to place tracking points uniformly on the sphere instead of on the distorted equirectangular image, tracks those points through the video with a point tracker, keeps trajectories whose spherical (great-circle) displacement exceeds a threshold, and at inference time reconstructs full trajectories from user-supplied handle and target points via spherical interpolation. The Omni Controller is a lightweight two-ResBlock convolutional module that encodes the trajectory condition and injects it into the first UNet block of Stable Video Diffusion using cross-normalization, while the temporal attention layers of the denoiser are jointly fine-tuned so the model learns spherical motion patterns rather than only 2D translations. The two together are what the paper credits for accurate, distortion-free control of both camera and object motion.","core_discovery":"On its own terms, OmniDrag establishes that omnidirectional motion control is achievable by making every stage of the control pipeline spherical: uniform point initialization on the sphere, spherical-distance-based selection of dominant motions, spherical interpolation from user clicks, and joint fine-tuning of temporal attention layers with a lightweight condition-injection module on a diffusion image-to-video model. The method is claimed to be the first to support both scene-level and object-level drag control for omnidirectional video, and experiments report lower FVD on equirectangular frames, lower FID and FVD on viewports, and better human ratings than DragNUWA, MotionCtrl, and DragAnything.","pith_inferences":["A straightforward next test is to measure how SME behaves on synthetic spherical motion (e.g., pure pan, pure tilt, or rotation around the optical axis); the paper's construction assumes great-circle paths, so non-geodesic motions like spirals or abrupt jerks may be out of reach for the current inference interpolation.","Because the inference stage interpolates between one handle and one target, fine-grained multi-point control—for instance, dragging along a curved path or shifting camera and object simultaneously—may require extending the estimator to several control points or user-drawn splines.","Since Move360 is captured by a car with four degrees of freedom, it is likely biased toward forward/backward and lateral translations; a dataset with more varied camera motions (tilts, rolls, orbits) would test whether OmniDrag generalizes beyond the capture setup.","If the CoTracker-based training signal turns out to be inaccurate in polar regions, an alternative estimator using spherical-aware tracking or projection to tangent planes could be plugged into the same Omni Controller architecture without changing the rest of the pipeline."],"forward_implications":["A user can generate 360° videos with specified camera or object motion by drawing just two points, making motion control accessible for VR content creation.","Spherical-aware handling—uniform initialization, great-circle distances, and spherical interpolation—becomes the standard ingredient for ODV motion control, since the paper shows 2D-based alternatives distort.","Jointly fine-tuning temporal attention with the controller is necessary for learning spherical motion, so future ODV control models should not freeze the base UNet.","Training on a dataset with large motions, like Move360, is what enables scene-level control; low-motion datasets limit controllability."],"supporting_citations":[{"why":"supplies the pretrained Stable Video Diffusion model that OmniDrag extends and partially fine-tunes.","marker":"[5]"},{"why":"provides the equal-area iso-latitude (HEALPix) sphere grid used to initialize tracking points.","marker":"[19]"},{"why":"is the point tracker used inside the spherical motion estimator to extract trajectories.","marker":"[28]"},{"why":"supplies the trajectory sampling strategy and trajectory-count schedule used during training.","marker":"[42]"},{"why":"provides the lightweight controller design and cross-normalization used for control-signal injection.","marker":"[45]"},{"why":"provides the prior ODV generation method and the WEB360 dataset used for comparison and training.","marker":"[61]"},{"why":"serves as the scene-level motion control baseline compared in experiments.","marker":"[62]"},{"why":"serves as a drag-based trajectory control baseline compared for object and scene control.","marker":"[65]"},{"why":"serves as a drag-based trajectory control baseline compared for object and scene control.","marker":"[71]"}],"fun_headline_variants":["Drag control for 360° video generation","OmniDrag enables scene- and object-level motion control","Spherical motion control for omnidirectional video","Turn clicks into motion for 360° video generation","First drag-based motion control for 360° videos"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole training signal rests on the point tracker's output on equirectangular frames being a faithful measurement of true spherical motion, including in the heavily distorted polar regions; the paper does not provide a validation of tracking accuracy on omnidirectional content.","fun_headline_variants_meta":{"raw":{"variants":["Drag control for 360° video generation","OmniDrag enables scene- and object-level motion control","Spherical motion control for omnidirectional video","Turn clicks into motion for 360° video generation","First drag-based motion control for 360° videos"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000197,"raw_usage":{"total_tokens":1341,"prompt_tokens":902,"completion_tokens":439,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":366}},"tokens_in":518,"tokens_out":439,"duration_ms":4474,"temperature":1.0,"reasoning_tokens":366,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:50:58.101357+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render synthetic 360° videos with a known camera rotation or object translation, run the spherical motion estimator on them, and compare estimated trajectories with ground truth; if endpoint errors grow substantially for points near the poles or for large spherical rotations, the learned condition maps are built on wrong motion signals and control quality would be expected to degrade correspondingly.","supporting_citations":[{"cited_title":"Healpix: A framework for high- resolution discretization and fast analysis of data distributed on the sphere","cited_arxiv_id":null,"evidence_quote":"provides the equal-area iso-latitude (HEALPix) sphere grid used to initialize tracking points."},{"cited_title":"Co- tracker: It is better to track together","cited_arxiv_id":null,"evidence_quote":"is the point tracker used inside the spherical motion estimator to extract trajectories."},{"cited_title":"Revideo: Remake a video with motion and content control","cited_arxiv_id":null,"evidence_quote":"supplies the trajectory sampling strategy and trajectory-count schedule used during training."},{"cited_title":"360dvd: Controllable panorama video generation with 360-degree video diffusion model","cited_arxiv_id":null,"evidence_quote":"provides the prior ODV generation method and the WEB360 dataset used for comparison and training."},{"cited_title":"Motionctrl: A unified and flexible motion controller for video generation","cited_arxiv_id":null,"evidence_quote":"serves as the scene-level motion control baseline compared in experiments."},{"cited_title":"Draganything: Motion control for any- thing using entity representation","cited_arxiv_id":null,"evidence_quote":"serves as a drag-based trajectory control baseline compared for object and scene control."}],"review_version":1}