{"id":"f3b88258-ad54-4a16-bf83-e050c578ad91","arxiv_id":"2412.09620","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A real-world dataset of 99k drone camera trajectories and an autoregressive transformer, DVGFormer, learn to generate smooth, collision-avoiding camera movements without hand-crafted filming heuristics.","lead":"The authors built a system that learns how to move a drone camera by watching 99,000 real drone videos from YouTube, then tested it in 3D simulations. It matters because it offers a cheaper, more scalable way to train automated cameramen than recording expert pilots or building virtual worlds by hand.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Scale ambiguity is never resolved end-to-end: dataset normalization assumes uniform drone speed (Sec. 3.2), and the paper never states how normalized action outputs are mapped to metric simulation units at inference, leaving collision-rate and low-altitude claims undefined.","rationale":"The reader identified the scale-normalization assumption in Sec. 3.2 as the weakest assumption, and I agree that this is the root of the most load-bearing concern. My stress-test sharpens it: even if the uniform-speed assumption were approximately true, the paper still omits the inference-time mapping from normalized action predictions to the metric units of the Blender evaluation scenes. This omission is more specific than the dataset-level assumption alone, and it directly affects the headline quantitative claims in Table 2 and the qualitative claims about navigating obstacles, maintaining low altitude, and orbiting buildings. The concern is not that the method is internally inconsistent, but that the reported evaluation is under-specified at a critical point: without knowing the scale factor, one cannot tell whether the collision rate reflects learned 3D awareness or merely an advantageous choice of units. The reader's CONDITIONAL verdict already captures the need for additional validation, so my recommendation does not change the verdict. I would condition acceptance on the authors providing the exact inference-time scaling procedure and demonstrating that the reported metrics are stable across a reasonable range of scale calibrations. This is a concrete, addressable issue rather than a fundamental flaw, and the paper's main contribution of a large real-world trajectory dataset remains valuable regardless of the outcome of this check.","tokens_in":15624,"tokens_out":3257,"duration_ms":37746,"concrete_test":"Ask authors to specify and release the exact de-normalization or scale factor applied to DVGFormer's predicted normalized actions before execution in Blender (Sec. 4.1, 5.2). Then rerun the Table 2 evaluation on the same 184 videos with that global scale multiplied by 0.5 and 2.0. If collision rate, user preference, or smoothness metrics change by more than a few points, the reported results depend on an arbitrary unit, so the claim of effective obstacle navigation and low-altitude control is not established. As a complementary check, run the dataset pipeline on synthetic videos with known metric trajectories to measure the spread of recovered per-clip scales; if the spread is large, the stable-speed assumption in Sec. 3.2 is violated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 3.2 normalizes each Colmap reconstruction by the average inter-frame camera distance, explicitly assuming stable drone speed across videos. This makes per-clip average speed unitless and discards clips with max speed more than 3 times the average (Appendix A). The model then predicts continuous 6-DoF motions that are normalized according to dataset statistics (Sec. 4.1, 4.3). The paper never specifies the inverse operation used when executing these normalized predictions in the Blender evaluation platform, where scenes (InfiniGen, Google Earth meshes) have real metric extent. If predictions are executed directly in normalized units, the absolute scale of every generated trajectory is arbitrary; collision rate, low-altitude flyover, and obstacle-navigation behavior all depend on that scale. If predictions are de-normalized with some fixed scalar, that scalar must be reported and justified, and it inherits the uniform-speed assumption, which is likely violated across FPV and cruising footage. Without a defined scale mapping, Table 2 metrics are not reproducible and the central claim about physically meaningful camera movement is under-specified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DroneMotion-99k, a dataset of 99,003 camera trajectories recovered from YouTube drone videos via Colmap reconstruction and Kalman-filter-based quality filtering, and DVGFormer, an autoregressive transformer that predicts 6-DoF camera motion from past frames, camera poses, and depth estimates. The system is evaluated in Blender on 38 synthetic natural scenes and 7 Google Earth city scans against an RT-1-inspired baseline, reporting higher user preference, lower collision rate, and smoother motion. The core idea is scalable and avoids hand-crafted cinematography heuristics, but the evaluation is thin and the scale-normalization step is load-bearing for the main claims.","tokens_in":15862,"tokens_out":4801,"duration_ms":51428,"significance":"If the scale ambiguity is resolved, this is a valuable contribution to AI videography: it provides a large real-world trajectory dataset, an automatic annotation pipeline, and a model that exploits long-horizon context. The authors state that data and code will be released, which would aid reproducibility. However, the current evidence for the central claim rests on a single baseline, a simulation-only evaluation without error bars or significance tests, and an unverified assumption about uniform drone speed during dataset normalization. The authors' own limitation section acknowledges the scale issue and the undesirable crash rate, which is honest but also indicates that the main claims are not yet fully supported.","major_comments":[{"comment":"The scale-normalization assumption is load-bearing and is never resolved end-to-end. Section 3.2 normalizes each 3D reconstruction by the average inter-frame camera distance, explicitly assuming stable drone speed across videos, and Appendix A further discards clips whose maximum speed exceeds 3 times the average. Section 4.1 says predicted motions are normalized according to dataset statistics, but the paper never states how these normalized outputs are mapped to metric units when executed in the Blender evaluation platform, where city scans have real metric extent (e.g., roughly 1 km regions). Without this mapping, the collision rate in Table 2 and the qualitative claims about low-altitude flight and obstacle navigation are not well-defined: direct execution in normalized units makes absolute trajectory scale arbitrary, while any fixed de-normalization constant inherits the unverified uniform-speed assumption. Please report the exact inference-time scale mapping and justify it, or evaluate how collision rate and user preference vary under a plausible range of scale factors.","section":"Sec. 3.2, Sec. 4.1, Sec. 5.2"},{"comment":"The quantitative evidence for the central claim is not statistically grounded. Table 2 reports user preference, collision rate, and smoothness on 184 videos, but no confidence intervals, number of human raters, or significance tests are provided, so the word 'significantly' in the Abstract and Section 5.4 is not supported. Moreover, the only comparison is an RT-1-inspired baseline; existing cinematography methods are set aside in Section 5.3 because they require human actors, but a learning-based method such as Jiang et al. [38] or a simple heuristic controller could be adapted to the same 3D scenes. Please add variance estimates, a paired significance test for preference and collision rate, and at least one additional baseline, or soften the corresponding claims.","section":"Sec. 5.4, Table 2"},{"comment":"The Abstract claims that the system 'effectively learns' to navigate through obstacles, maintain low altitude, and orbit towers and buildings. These behaviors appear only as qualitative examples in Fig. 1 and Fig. 5 and are not quantified. Define and report metrics such as distance to the nearest scene mesh, altitude above ground, and orbit coverage for both DVGFormer and the baseline, and compare against a random-trajectory control. Without such metrics, the distinguishing behaviors claimed in the Abstract are not testable and the central qualitative claim is under-supported.","section":"Abstract, Sec. 5.5"},{"comment":"The data-filtering pipeline relies on two thresholds whose sensitivity is not analyzed: the Kalman-filter distance threshold of 0.2 selected on roughly 1,000 labeled reconstructions, and the rule discarding clips whose maximum speed exceeds 3 times the average speed. The paper does not report whether the labeled reconstructions were split to avoid overfitting the threshold, nor how downstream model performance changes when these thresholds are varied. Please report this sensitivity or justify the choices with cross-validation, since these thresholds determine the final dataset composition.","section":"Sec. 3.3, Appendix A"}],"minor_comments":[{"comment":"The section heading contains a typo: 'Limitatinons' should be 'Limitations'.","section":"Sec. 6"},{"comment":"In the generalization study description, 'we the under water setting in InfiniGen' appears to be missing a verb, likely 'we use the under water setting in InfiniGen'.","section":"Appendix C"},{"comment":"The phrase 'natural and civic scenes' should probably be 'natural and city scenes' for clarity.","section":"Sec. 5.2"},{"comment":"The caption should state how the user preference percentage is computed (e.g., pairwise votes, fraction of videos preferred) and report the number of human raters.","section":"Table 2"},{"comment":"Minor grammatical issue: 'the models predicts' should be 'the model predicts'; also, the sentence 'we break the action at into N steps' is missing the subscript t in the running text, which can confuse readers.","section":"Sec. 4.1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is in scope for a computer vision journal and the dataset/model release is a plus. The main risk is the scale-mapping gap between normalized training trajectories and metric simulation scenes; this is central rather than peripheral because it affects every training trajectory and every evaluation metric such as collision rate. The authors' own limitation section acknowledges the issue, but it needs to be resolved or explicitly analyzed, not just listed. The evaluation should also be strengthened with error bars and at least one additional baseline; otherwise the paper may be better positioned as a dataset-and-method report with more modest claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a serious systems paper. DroneMotion-99k is a genuinely new resource—99k real camera trajectories extracted from online drone videos via Colmap and Kalman filtering, with a labeled 1k reconstruction set used to set the filter threshold. The DVGFormer architecture, an autoregressive transformer that consumes long image, depth, pose, and motion tokens, is a coherent and well-motivated design. The paper is honest about its limitations, and the user-preference result (70.5% vs. 29.5%) is a solid if preliminary signal.\n\nThe soft spots are real but mostly repairable. Evaluation is against a single baseline with no error bars or significance tests; the user study is a forced choice on 184 videos and deserves a sentence on protocol. Everything is simulation-only, which the Discussion acknowledges. The deeper issue is the scale mapping. The dataset is normalized by average inter-frame camera distance under the 'drone speed is stable' assumption (Sec. 3.2), and the model outputs normalized motion (Sec. 4.1). But the paper never states how those normalized predictions are converted into metric units for the Blender evaluation platform. Without that inverse operation, collision rates and low-altitude claims are undefined. The stress test is right on this. This doesn't kill the relative comparison if both models go through the same pipeline, but it blocks reproducibility and weakens the physical interpretation of Table 2. The authors should specify the mapping or, failing that, treat the numbers as dimensionless.\n\nCitation practice is fine, and there is no sign of circularity. The paper does not oversell itself; the Discussion even flags the remaining crash rate.\n\nWho should read it: anyone working on autonomous cinematography, drone control, or camera trajectory learning. It deserves a serious referee. Send it out, but with the expectation that the authors will clarify the scale mapping and add at least one more baseline or some error bars.","headline":"A useful dataset and a clean architecture, but the scale-normalization-to-execution gap leaves the headline numbers under-specified.","tokens_in":16346,"tokens_out":3145,"would_cite":true,"duration_ms":31259,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An AI cameraman can learn skilled drone camera movement from 99,003 trajectories automatically extracted from real YouTube drone videos, and beats a robotics baseline on user preference, collision rate, and smoothness in simulation.","keywords":["drone videography","camera movement control","autoregressive transformer","structure-from-motion","Kalman filter","behavior cloning","real-world dataset","3D camera trajectory"],"falsifier":"Record drone flights whose true three-dimensional trajectories are known from onboard GPS/IMU telemetry, deliberately covering a wide range of flight speeds, run the paper's Colmap-and-normalization pipeline on that footage, and compare each normalized camera path to the true flight speed; if the normalized trajectories do not track the true speeds, the speed-stability assumption fails and the learned camera motions would be miscalibrated.","tokens_in":15464,"feed_emoji":"🚁","tokens_out":10140,"duration_ms":85625,"temperature":0.7,"pith_summary":"The paper aims to show that camera movement control for drone videography can be learned at scale from real-world online videos, without hand-coded cinematography rules or expensive expert teleoperation. To do this, the authors automatically extract 99,003 three-dimensional camera trajectories from YouTube drone footage using structure-from-motion and a Kalman filter, then train DVGFormer, an autoregressive transformer that predicts the next camera motion from all past images, poses, and motions. In simulation over 38 synthetic natural scenes and 7 real city 3D scans, the model learns challenging moves such as navigating through obstacles, flying low to increase perceived speed, and orbiting towers and buildings, and it records higher user preference, a lower collision rate, and smoother motion than an RT-1-inspired baseline. The significance is that this points to a scalable path to AI videography that does not require pixel generation or heuristic objective design.","feed_headline":"99,000 YouTube clips teach an AI cameraman how to move","feed_subtitle":"Trained on real footage, it learns obstacle dodging and orbiting without hand-coded rules.","key_machinery":"The load-bearing machinery is the automatic trajectory-extraction pipeline that builds DroneMotion-99k together with the DVGFormer architecture that consumes it. The pipeline splits scraped YouTube videos into shots with shot-change detection, recovers per-frame camera poses with Colmap structure-from-motion, connects consecutive poses into three-dimensional paths, and uses an Unscented Kalman Filter plus a labeled threshold to discard reconstructions whose camera locations jump between neighboring frames; a scale normalization divides each scene by the average inter-frame camera distance, assuming drone speed is stable across videos. DVGFormer is a GPT-2-style autoregressive transformer of about 40 million parameters that tokenizes camera pose and motion with MLPs, image patches with DINOv2 features, and monocular depth from Depth-Anything, breaks each predicted action into five sub-steps, and predicts the next camera motion from up to 10 seconds of past frames using bi-level positional embeddings.","core_discovery":"On the paper's own terms, the central discovery is that a long-horizon behavior-cloning model can acquire sophisticated drone cinematography skills purely from automatically recovered real-world camera paths. The authors train DVGFormer on the DroneMotion-99k dataset, which contains 99,003 trajectories totaling more than 180 hours of footage, and report that compared with an RT-1-inspired baseline it achieves significantly better user preference (70.5% versus 29.5%), a lower collision rate (15.2% versus 33.7%), and smoother motion, measured as smaller maximum relative changes in velocity and angular velocity, across 184 simulated videos covering 38 natural scenes and 7 real city scans. Qualitatively, they observe behaviors such as obstacle navigation, low-altitude flyovers, building orbits, and even transfer to underwater scenes, all without hand-written cinematography heuristics.","pith_inferences":["A natural test the paper leaves implicit is to validate the extraction pipeline on footage with known UAV telemetry; if flight speeds vary widely in practice, the normalization would distort recovered trajectory magnitudes and speeds.","The same automatic-trajectory recipe could transfer to other camera platforms such as handheld gimbals, dashcams, or underwater ROVs, potentially yielding a general camera-movement prior for videography.","The reported crash rate suggests a learned movement prior alone is not yet safe for real-world flight; adding an explicit collision-avoidance term or sequence-level 3D perception at inference time is a concrete next step the paper itself flags.","The 99k real trajectories could serve as a camera-motion prior for generative video models that condition on camera paths, narrowing the search space without pixel-space supervision."],"forward_implications":["Videography control for drones can be built from already-existing online video at low annotation cost, moving the data bottleneck from expert teleoperation to automatic reconstruction and filtering.","Because the model outputs executable three-dimensional trajectories rather than pixels, the same trained system can be deployed in any renderable 3D scene, including ones unlike its training footage.","The action-chunking design and long context window let the model continue trajectories beyond its training length, producing smooth 20-second videos without post-processing.","The <Cond> token provides a stochastic control handle: the same starting frame can yield multiple distinct but feasible camera paths.","The comparison with the RT-1-inspired baseline indicates that adding camera pose and motion tokens plus a longer temporal receptive field is what separates smooth videography from a generalist behavior-cloning model."],"supporting_citations":[{"why":"Colmap structure-from-motion is the method that recovers per-frame 3D camera poses from video frames; the entire dataset of trajectories is built on these reconstructions.","marker":"[55]"},{"why":"The Unscented Kalman Filter smooths camera paths and identifies low-quality reconstructions whose camera locations jump between neighboring frames, forming the data-filtering step.","marker":"[63]"},{"why":"RT-1 inspires the baseline model that DVGFormer is compared against for user preference, collision rate, and smoothness.","marker":"[12]"},{"why":"DINOv2 provides the image patch features that DVGFormer tokenizes to perceive scene appearance.","marker":"[50]"},{"why":"Depth-Anything supplies monocular depth estimates that inject per-frame 3D information into the model.","marker":"[65]"},{"why":"The GPT-2 transformer architecture is the backbone of the auto-regressive sequence model in DVGFormer.","marker":"[53]"},{"why":"PySceneDetect performs the shot-change detection that splits raw videos into clips before reconstruction.","marker":"[18]"},{"why":"Example-driven cinematography learned from 3D animations represents the costly, limited-appearance alternative that the paper's real-world-data approach is positioned against.","marker":"[38]"},{"why":"A prior online-video drone dataset using SLAM without camera intrinsics illustrates the reconstruction-quality problem the paper's SfM pipeline addresses.","marker":"[6]"},{"why":"InfiniGen generates the 38 synthetic natural scenes used for evaluation alongside 7 real city 3D scans.","marker":"[54]"}],"fun_headline_variants":["AI learns drone cinematography from 99k real trajectories","No hand-coded rules: AI masters drone moves from real video","99k drone videos teach AI to orbit and dodge obstacles","DVGFormer: AI cameraman trained on 180 hours of real drone footage"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole dataset is normalized by the average inter-frame camera distance under the assumption that drone flight speed is roughly stable across different videos; if real speeds vary substantially across the scraped footage, the recovered trajectories carry inconsistent scale and the model would learn distorted motion magnitudes and speeds.","fun_headline_variants_meta":{"raw":{"variants":["AI learns drone cinematography from 99k real trajectories","No hand-coded rules: AI masters drone moves from real video","99k drone videos teach AI to orbit and dodge obstacles","DVGFormer: AI cameraman trained on 180 hours of real drone footage"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000426,"raw_usage":{"total_tokens":2196,"prompt_tokens":973,"completion_tokens":1223,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":1147}},"tokens_in":589,"tokens_out":1223,"duration_ms":8887,"temperature":1.0,"reasoning_tokens":1147,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T16:50:46.152656+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Record drone flights whose true three-dimensional trajectories are known from onboard GPS/IMU telemetry, deliberately covering a wide range of flight speeds, run the paper's Colmap-and-normalization pipeline on that footage, and compare each normalized camera path to the true flight speed; if the normalized trajectories do not track the true speeds, the speed-stability assumption fails and the learned camera motions would be miscalibrated.","supporting_citations":[{"cited_title":"Structure-from-motion revisited","cited_arxiv_id":null,"evidence_quote":"Colmap structure-from-motion is the method that recovers per-frame 3D camera poses from video frames; the entire dataset of trajectories is built on these reconstructions."},{"cited_title":"The unscented kalman filter for nonlinear estimation","cited_arxiv_id":null,"evidence_quote":"The Unscented Kalman Filter smooths camera paths and identifies low-quality reconstructions whose camera locations jump between neighboring frames, forming the data-filtering step."},{"cited_title":"Language models are unsu- pervised multitask learners","cited_arxiv_id":null,"evidence_quote":"The GPT-2 transformer architecture is the backbone of the auto-regressive sequence model in DVGFormer."},{"cited_title":"Home - PySceneDetect — scenede- tect.com","cited_arxiv_id":null,"evidence_quote":"PySceneDetect performs the shot-change detection that splits raw videos into clips before reconstruction."},{"cited_title":"Example-driven virtual cinematography by learning camera behaviors","cited_arxiv_id":null,"evidence_quote":"Example-driven cinematography learned from 3D animations represents the costly, limited-appearance alternative that the paper's real-world-data approach is positioned against."},{"cited_title":"A drone video clip dataset and its applications in automated cinematography","cited_arxiv_id":null,"evidence_quote":"A prior online-video drone dataset using SLAM without camera intrinsics illustrates the reconstruction-quality problem the paper's SfM pipeline addresses."},{"cited_title":"Infinite photore- alistic worlds using procedural generation","cited_arxiv_id":null,"evidence_quote":"InfiniGen generates the 38 synthetic natural scenes used for evaluation alongside 7 real city 3D scans."}],"review_version":1}