{"id":"d1b8e4ea-b7b0-4dea-9335-bb8577db40a8","arxiv_id":"2411.17765","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"I2VControl unifies camera, drag, and brush controls into a single point-trajectory-based adapter for image-to-video diffusion models, enabling conflict-free combined motion control.","lead":"This paper presents a video-generation controller that combines camera movement, object dragging, and motion-brush effects in one pipeline. It may matter because it lets users apply several motion controls at once without the conflicts that arise when single-purpose tools are stacked.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The unified trajectory input is not actually dense: Eq. 10 feeds only the rigid term R^(p), while the residual G^(p) is compressed to a scalar speed m^(p) (Eqs. 7/9), so brush control is underdetermined and the claimed conflict-free motion brush rests on an unverified information bottleneck.","rationale":"The reader identified the factorization and upstream tracking/depth accuracy as the weakest assumption. My concern is related but distinct: even assuming the factorization in Eq. 5 is exact and the data pipeline recovers E_λ, R^(p)_λ, and m^(p)_λ perfectly, the control signal fed to the adapter in Eq. 10 omits G^(p) entirely and summarizes it by a scalar. This creates an information-theoretic underdetermination for brush-units (and for the residual part of drag-units). The paper's stated motivation is to unify all controls as dense point trajectories; that unification is incomplete exactly where motion brush is concerned, because the brush trajectory is static after camera projection and the non-rigid component is represented only by speed. This does not make the method internally inconsistent or obviously wrong: a scalar 'motion strength' brush may be a deliberate, coarse user control. But the abstract and Sec. 1 claim a unified trajectory representation and a conflict-free combination of all three controls, and the empirical section provides no quantitative test of combined control or of whether the scalar bottleneck limits brush fidelity. The reader's CONDITIONAL verdict already demands stronger benchmarking and artifact release; this concern adds a specific correctness risk that should be tested before accepting the 'first framework' claim at face value. An ablation with the full dense trajectory would settle whether the omitted G is load-bearing. Since the existing reader verdict already captures the need for additional evidence, I do not propose changing the verdict.","tokens_in":13353,"tokens_out":9236,"duration_ms":90489,"concrete_test":"Train a controlled ablation on the same data and base model: (A) the submitted input (T_λ,M_λ,P_λ,C_λ), and (B) a variant where the trajectory input for every unit is the full dense camera-space trajectory, T_full(λ,x)=Π(E_λ^{-1}∘D(λ,x)), with m still provided as an auxiliary signal. Evaluate both on Object/Fluid Brush (MSC, IoU, FID) and on combined camera+drag+brush examples with manual annotation. If variant B is significantly better on brush metrics and combined-control consistency, the scalar compression of G is the active limitation; if B is not better, then the scalar bottleneck is not load-bearing and the current claim can stand as a restricted brush definition.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that all three controls are reformulated as dense point trajectories and can be combined without conflict. In Eq. 5 the world-space motion is D(λ,x)=R^(p)_λ ∘ x + G^(p)(λ,x). But the trajectory control signal T_λ in Eq. 10 is computed from the rigid term only: T_λ = Σ_p Π(E_λ^{-1} ∘ R^(p)_λ ∘ (χ^(p) ⊙ Ω)), with no G^(p) term. For brush-units, R^(p)≡I, so T_λ is just the static, camera-parallax-projected point set; the only brush information is the scalar m^(p)_λ = M(Ω^(p),G^(p),λ) in Eq. 7. Thus the 'dense trajectory' is not dense for brush control, and the residual non-rigid motion is reduced to a single average-speed number. This is an identification limit, not just a tracking-error issue: infinitely many ground-truth videos share the same image, mask, camera, rigid drag, and scalar strength m yet have very different G^(p). The model cannot learn a deterministic mapping from the control signal to the specific residual; it must either regress to the conditional mean or hallucinate. The paper's own brush comparison against Motion-I2V is therefore against a method that feeds explicit optical flow, while I2VControl feeds only a scalar for brush units. The 'without conflicts' claim is only validated qualitatively on a restricted, scalar-defined brush, and no quantitative benchmark isolates whether combined camera+drag+brush preserves user intent for the residual component. The paper should either include the residual in the trajectory signal, or explicitly reframe 'motion brush' as a coarse speed-control that does not specify the spatial pattern of non-rigid motion.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes I2VControl, an image-to-video motion control framework that unifies camera extrinsics control, object dragging, and motion brush into a single dense point-trajectory representation. It introduces a motion-unit spatial partitioning scheme in which the image is divided into brush-units, drag-units, and a borderland, each governed by a dedicated formulation, and it presents a plug-in adapter for pre-trained video diffusion models along with a data pipeline that extracts control signals from RGB videos. The method is trained on 970K clips and evaluated on six datasets against DragAnything, MOFA-Video, MotionCtrl, CameraCtrl, and Motion-I2V, reporting strong quantitative results on single-control benchmarks. The central claim is that the three control types can be combined without logical conflicts in a single inference pipeline.","tokens_in":13687,"tokens_out":5149,"duration_ms":48887,"significance":"If the central claim holds, I2VControl would be a practically significant contribution: it is the first single pipeline supporting camera, drag, and brush control together, and it introduces a unified representation that makes combined control feasible. The paper contains substantial experimental work, including six datasets, three newly curated test sets, user studies, and adapter training on multiple base models (Seedance and Wanx 2.1), with a code release mentioned for Wan 2.1. The adapter design is architecture-agnostic and the data pipeline is clearly described. However, the flagship 'without conflicts' claim currently rests on qualitative examples, and the brush-control signal is information-limited by construction, so the contribution is not yet established at the level claimed in the abstract.","major_comments":[{"comment":"The 'dense trajectory' control signal is not dense for brush units. In Eq. (5), D decomposes into R^(p) and G^(p), but Eq. (10) feeds only R^(p) into T_λ. For brush units Eq. (7) sets R^(p) ≡ I, so T_λ is the camera-parallax projection of a static point set; the only brush information is the scalar m^(p)_λ computed by Eq. (7). Infinitely many ground-truth residual motions G^(p) are consistent with the same control signal. The model therefore cannot learn a deterministic mapping from the control signal to the requested residual motion; it must regress to a conditional mean or hallucinate. Please either include the residual in the trajectory signal (e.g., sampled trajectories from G^(p)) or provide an explicit analysis or bound showing that the scalar bottleneck preserves user intent for the brush task.","section":"Sec. 4.2 (Eqs. 5, 7, 10)"},{"comment":"The paper's first contribution is conflict-free combined control, but the only evidence for the combined setting is qualitative (Fig. 7, columns 7–8). There is no quantitative metric for multi-control consistency, such as camera RotErr/TransErr measured under simultaneous drag and brush, ObjMC under camera motion, brush mask IoU/MSC under drag or camera, or an ablation that removes one control and measures the effect on the others. Without such a benchmark, the central 'without conflicts' claim is not empirically supported.","section":"Sec. 5.3.2 and Fig. 7"},{"comment":"The brush metric is circular with respect to the training objective. The motion strength M in Eq. (3) is taken from the authors' own I2VControl-Camera [8], used as supervision for brush and drag units in Eqs. (7) and (9), and then reused as the MSC evaluation metric in Table 4. High MSC therefore partly measures adherence to the training definition rather than agreement with an external notion of requested motion. Please supplement MSC with an external metric, such as tracked optical-flow consistency with the user-supplied motion strength or a forced-choice user study comparing different residual motions at a fixed scalar strength.","section":"Sec. 4.2 (Eq. 3) and Table 4"},{"comment":"The data pipeline assumes that SpatialTracker plus the I2VControl-Camera decomposition recovers E_λ and D(λ, x) accurately, and that Unidepth provides a reliable initial point set Ω during inference. No error analysis or validation of these upstream estimates is reported. Since T_λ and M_λ are computed from these quantities, systematic tracking or depth errors would corrupt the training signal and the projected trajectories at inference. Please report tracking and depth accuracy on a validation subset, or provide an ablation that perturbs E_λ and Ω, to bound the sensitivity of the control signal to these assumptions.","section":"Sec. 4.3"}],"minor_comments":[{"comment":"Please report the number of raters and videos in the user study, and provide confidence intervals or significance tests; the reported score differences are large, but the scale and variance are not stated.","section":"Sec. 5.3.2, Tables 2–4"},{"comment":"The notation (B, T, 5, H, W) for the control signal is inconsistent with the text description of a (T, 5, H, W)-shaped tensor and with the channel breakdown (T_λ, M_λ, P_λ, C_λ); please clarify how the five channels are arranged.","section":"Fig. 6"},{"comment":"The masks χ^(p) are not explicitly required to be disjoint; please state the partition constraint (e.g., Σ_p χ^(p) = 1) to make the spatial partitioning well-defined.","section":"Eq. (4)"},{"comment":"The user-facing motion-strength values (e.g., 300, 1000, 5000) are not normalized or calibrated; please specify the scale, a default value, or the mapping from user input to m^(p)_λ.","section":"Sec. 4.2 and Figs. 1, 7"},{"comment":"References [2] and [3] are duplicated (both cite the same Stable Video Diffusion preprint); please merge or correct the citation list.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper leans heavily on the authors' own I2VControl-Camera work for both the training signal and the evaluation metric, which creates a self-confirmation risk; I would ask the editor to ensure the revision addresses this with an external evaluation. The 'first framework' claim is also strong and should be carefully scoped in revision. The manuscript fits the journal's scope, and the core idea is worth pursuing, but the current evidence does not yet support the conflict-free combined-control claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you work on controllable video generation. The paper's real contribution is integration: camera extrinsics, object drag, and motion brush in a single adapter, with a clean spatial partitioning scheme. As far as I can tell from Table 1, that combination is new, and the adapter design is plausibly architecture-agnostic. The formulations in Secs. 4.1–4.2 are clear, and the authors re-trained MotionCtrl and CameraCtrl on their own base for fairer camera comparisons. They also say they released code for the Wan 2.1 variant, which is more than many arXiv papers do.\n\nThe soft spots are material. First, the stress-test note is right: Eq. 10 builds the trajectory signal from the rigid term only. Brush units have R=I, so the 'dense point trajectory' for a brushed region is just a static point cloud under camera motion; all residual motion information enters only as the scalar speed m. So the brush control is not actually a dense trajectory—it is a region-level average speed. That is arguably consistent with the motion-brush task definition (users only specify strength), but it undercuts the paper's unified-dense-trajectory narrative and makes the comparison against Motion-I2V, which feeds explicit optical flow, less than apples-to-apples.\n\nSecond, the 'without conflicts' claim is the centerpiece, yet it is only demonstrated qualitatively (Fig. 7). There is no metric that isolates whether combined camera+drag+brush preserves all three intents simultaneously. Third, the brush metric MSC is computed from the same motion-strength definition used to create training labels, so part of the reported brush performance measures adherence to the training definition rather than an external ground truth. The mask IoU is independent, which helps, but the story is weaker than the numbers suggest. Fourth, no ablations or error bars; the six datasets are a plus, but comparisons across different base models are hard to interpret.\n\nNone of these are fatal. The central idea—trajectory-based spatial partitioning plus a plug-in adapter—is sound enough to build on, and the paper is honest about the difficulty of matching baselines. The authors should either fold the residual into the trajectory signal for brush units or explicitly reframe the motion brush as coarse speed control, and they should add a quantitative combined-control benchmark with per-control metrics.\n\nMy bottom line: send it to review, but expect major revision. A serious referee will want the multi-control benchmark and a clearer statement of what the brush signal does and does not specify. I'd cite it for the unified-control capability; I'd bring it to a reading group if video generation is on the agenda.","headline":"A genuinely new three-in-one control adapter whose 'dense trajectory' framing overstates what brush units actually receive, and whose conflict-free claim needs a quantitative benchmark.","tokens_in":14308,"tokens_out":3125,"would_cite":true,"duration_ms":28657,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"I2VControl claims a single pipeline can orchestrate camera motion, object dragging, and motion brush without conflicts.","keywords":["image-to-video synthesis","motion controllability","camera control","object dragging","motion brush","dense point trajectories","spatial partitioning","diffusion adapter"],"falsifier":"On held-out videos of a moving camera with a strongly non-rigid object (cloth, liquid, or hair), run the paper's pipeline to extract $E_\\lambda$ and per-unit $R_\\lambda^{(p)}$ and measure the residual $G^{(p)}(\\lambda,x)$. If that residual grows with deformation speed and cannot be summarized by the scalar $m_\\lambda^{(p)}$, the factorization fails; equivalently, generated brush-control videos on such scenes should visibly fail to reproduce the non-rigid motion, which a reader can check directly.","tokens_in":13085,"feed_emoji":"🎬","tokens_out":7649,"duration_ms":71107,"temperature":0.7,"pith_summary":"This paper argues that the three standard families of video-motion control—camera extrinsics, object dragging, and region-based motion brush—are not separate problems but one problem: specifying dense point trajectories. I2VControl tries to show that by rewriting every control as a trajectory signal, partitioning the image into motion units (drag, brush, borderland), and injecting per-unit controls through a small adapter into a frozen image-to-video diffusion model, all three controls can be exercised in a single inference pass without logical conflicts. If this holds, a video editor would no longer choose between moving the camera and moving an object: both can be specified together, and the model resolves which pixels obey which command. The paper reports single-task results that match or beat dedicated baselines on camera, dragging, and brush benchmarks, and demonstrates combined controls that purely single-task methods cannot express.","feed_headline":"One pipeline now drives camera, drag, and brush control together","feed_subtitle":"I2VControl rewrites all three controls as point trajectories with per-region masks, so editors can combine them in one pass.","key_machinery":"The load-bearing device is the unit-wise trajectory decomposition with the control tensor. The image domain is sliced into motion units $\\Omega^{(p)}$ (brush-units, drag-units, and the borderland) with masks $\\chi^{(p)}$, and on each unit the world-frame trajectory is decomposed as $D(\\lambda,x)=R_\\lambda^{(p)}\\circ x+G^{(p)}(\\lambda,x)$, where $R_\\lambda^{(p)}\\in SE(3)$ is the best-fit rigid transform recovered from tracked points (the identity for brush-units and borderland) and $G^{(p)}$ carries residual motion summarized by the scalar strength $m_\\lambda^{(p)}$. A global camera transform $E_\\lambda$ is applied afterward, giving the final camera-frame trajectory $T_\\lambda=\\sum_p \\Pi(E_\\lambda^{-1}\\circ R_\\lambda^{(p)}\\circ(\\chi^{(p)}\\odot\\Omega))$. The control signal handed to the network is the concatenation of trajectory, motion-strength, partition-index, and category maps $(T_\\lambda,M_\\lambda,P_\\lambda,C_\\lambda)$, encoded by convolutional layers and injected through a few trainable attention layers (the adapter) before the frozen diffusion model's self-attention, then removed after attention.","core_discovery":"The paper tries to establish that camera control, object dragging, and motion brush are not fundamentally separate mechanisms but three instances of one mechanism: prescribing dense point trajectories. Camera motion is a global 6-DOF transform $E_\\lambda$ acting on all points; a dragged region is a per-unit rigid transform $R_\\lambda^{(p)}$ fit to the tracked motion of that mask; a brushed region is governed only by a scalar motion strength $m_\\lambda^{(p)}$, with an identity rigid part; the unselected borderland is left to natural motion. With the image partitioned into such units and each pixel carrying its trajectory, the generated video motion is specified in a single coordinate frame with explicit per-unit instructions, so the previous conflicts between camera motion and object or background motion are removed at the representation level. The paper substantiates this by reporting camera rotation and translation errors, drag accuracy (ObjMC), and brush motion-strength metrics on six datasets, along with qualitative combined-control sequences that single-task baselines cannot produce.","pith_inferences":["If the trajectory-factorization view generalizes, other motion controls—box trajectories, text-prompt motion, audio-driven motion—could be encoded as additional per-unit signal channels inside the same adapter rather than as separate frameworks; the paper only builds the three categories it evaluates.","The scalar motion-strength per unit is intentionally coarse, so for strongly non-rigid deformations such as cloth, liquid, or hair a single residual scalar may under-specify motion; a natural extension would carry per-unit residual flow fields alongside the rigid transform, and this is testable with the paper's own brush datasets.","The paper's data pipeline can be read as a video-motion-analysis tool: the same decomposition into global camera transform, per-unit rigid motion, and residual strength could be used to retarget or edit the motion of existing videos, an application the paper does not demonstrate.","The claimed conflict-free combination depends on the per-unit masks covering the relevant motion; a testable extension is whether partially overlapping or erroneously segmented masks degrade the camera-drag-brush combination more than single-control settings."],"forward_implications":["A user can set a camera path, drag masks with 6-DOF motion, and brush masks with motion-strength scalars in one inference call, and the generated frames follow all these signals simultaneously.","Because the adapter plugs into a frozen base model and the control representation is expressed as pixel-wise maps, the same adapter design transfers to other image-to-video diffusion backbones, as demonstrated in the paper on Seedance and Wan 2.1.","The automatic data pipeline turns ordinary RGB video clips into training supervision through tracking and motion-unit decomposition, so unified control training does not require per-task 3D or optical-flow datasets.","Single-task performance is not sacrificed for unification: the paper's reported camera errors, drag ObjMC, and brush motion-strength metrics improve on or match the dedicated methods it compares against.","Spatial controls can be composed dynamically per frame from masks, so switching from dragging one object to brushing a fluid region is only a change of category map and strength value, not a retrained model."],"supporting_citations":[{"why":"Supplies the camera-extrinsic and motion-strength formulation, and the world-to-camera trajectory decomposition, that this work extends to multiple control types.","marker":"[8]"},{"why":"The frozen image-to-video diffusion base model into which the adapter is plugged for the main experiments.","marker":"[37]"},{"why":"Provides the per-pixel 2D trajectories in 3D space used by the training data pipeline to recover motion.","marker":"[43]"},{"why":"Segments the first frame into candidate motion units for the spatial partitioning step.","marker":"[21]"},{"why":"Estimates metric depth at inference time to lift user-selected masks to the initial 3D point set.","marker":"[27]"},{"why":"Serves as the dragging-control baseline whose dataset and ObjMC metric are used for comparison.","marker":"[40]"},{"why":"Serves as the unified motion-control and camera-control baseline retrained on the same base model for camera comparison.","marker":"[38]"},{"why":"Serves as the extrinsics-based camera-control baseline compared on RealEstate10K and MO.","marker":"[15]"},{"why":"Defines the motion-brush task with mask and scalar strength, and provides the comparison for brush controls.","marker":"[30]"}],"fun_headline_variants":["One trajectory language kills camera-drag-brush conflicts","Camera, drag, brush unified as point trajectories","Three controls, zero conflicts via point trajectories","I2VControl: all motion controls as one trajectory"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole scheme assumes that real video motion can be decomposed into a global camera transform, per-region rigid motion, and a single scalar residual per region, and that the tracker and depth estimator used to build training labels recover that decomposition accurately.","fun_headline_variants_meta":{"raw":{"variants":["One trajectory language kills camera-drag-brush conflicts","Camera, drag, brush unified as point trajectories","Three controls, zero conflicts via point trajectories","I2VControl: all motion controls as one trajectory"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000572,"raw_usage":{"total_tokens":2690,"prompt_tokens":919,"completion_tokens":1771,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":1710}},"tokens_in":535,"tokens_out":1771,"duration_ms":11383,"temperature":1.0,"reasoning_tokens":1710,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:31:17.308935+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On held-out videos of a moving camera with a strongly non-rigid object (cloth, liquid, or hair), run the paper's pipeline to extract $E_\\lambda$ and per-unit $R_\\lambda^{(p)}$ and measure the residual $G^{(p)}(\\lambda,x)$. If that residual grows with deformation speed and cannot be summarized by the scalar $m_\\lambda^{(p)}$, the factorization fails; equivalently, generated brush-control videos on such scenes should visibly fail to reproduce the non-rigid motion, which a reader can check directly.","supporting_citations":[{"cited_title":"Spatialtracker: Tracking any 2d pixels in 3d space","cited_arxiv_id":null,"evidence_quote":"Provides the per-pixel 2D trajectories in 3D space used by the training data pipeline to recover motion."},{"cited_title":"UniDepth: Universal monocular metric depth estimation","cited_arxiv_id":null,"evidence_quote":"Estimates metric depth at inference time to lift user-selected masks to the initial 3D point set."},{"cited_title":"Draganything: Motion control for anything using entity representation, 2024","cited_arxiv_id":null,"evidence_quote":"Serves as the dragging-control baseline whose dataset and ObjMC metric are used for comparison."},{"cited_title":"Motion-i2v: Consistent and controllable image-to-video generation with explicit motion modeling","cited_arxiv_id":null,"evidence_quote":"Defines the motion-brush task with mask and scalar strength, and provides the comparison for brush controls."}],"review_version":1}