{"id":"a466265f-976c-4dd1-a740-5339307a6caf","arxiv_id":"2502.10059","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Metric-scale depth alignment plus scene-constrained noise shaping improves camera controllability and video quality for image-to-video generation on RealEstate10K.","lead":"RealCam-I2V makes camera-controlled video generation work with real-world images by first reconstructing a metric-scale 3D scene from a single image, then letting users draw camera paths in that scene. It reports better camera-control accuracy and video quality than prior trajectory-based models on RealEstate10K.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method never specifies how camera intrinsics K are obtained for arbitrary user-supplied images at inference; without K, the claimed metric 3D scene from Section 3.3 is not well defined, so the central scale-consistency argument has a missing input.","rationale":"Good-faith reading: the paper's contribution is to replace relative-scale trajectories with a metric-scale 3D scene derived from monocular depth. For that to work at inference on arbitrary images, the 3D reconstruction must be geometrically consistent with the scale used in training. The paper carefully aligns COLMAP trajectories to Depth Anything V2 depth for RealEstate10K, but that alignment is performed on data with known intrinsics. At inference, the interactive interface in Section 3.3 projects depth to 3D using K without explaining where K comes from. This is not a minor implementation detail: the same metric depth expressed through a different focal length produces a different 3D scene aspect ratio and different camera translations, so the learned Plücker-conditioning would receive inputs from a distribution it was not trained on. The reader's weakest assumption concerned the correctness of the depth predictor's absolute scale; the intrinsics issue is a more fundamental and more easily fixed gap, because it remains even if the depth predictor is perfect. A focused experiment varying K on calibrated images would determine whether the concern actually lands. If it does, the paper should be accepted only after specifying and validating K handling for arbitrary inputs; since the reader already set CONDITIONAL, my recommended verdict is unchanged.","tokens_in":19388,"tokens_out":10308,"duration_ms":106057,"concrete_test":"Run the inference pipeline on images with known ground-truth intrinsics (e.g., ScanNet++ or ARKit/Record3D frames) using three K values: the ground-truth K, K with focal length halved, and K with focal length doubled, keeping the same metric depth map and the same user trajectory. Measure RotErr, TransErr, and CamMC as in Table 1, plus visual fidelity of the generated video. If camera-control metrics degrade by more than about 10% under wrong K, the missing intrinsics specification is load-bearing because the method cannot be applied to arbitrary images without calibration; if the results are insensitive to K, a default intrinsics assumption would suffice and the concern is minor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on reconstructing a metric-scale 3D scene from a single arbitrary image (Section 3.3) and on keeping that scale consistent with the metric-aligned training data (Section 3.2). The paper's projection equation pc = D(u,v)K^{-1}[u v 1] requires the camera intrinsics matrix K. On RealEstate10K, K is known; but for the real-world images that motivate the paper, and for the out-of-domain demos in Figure 9, the paper never states whether K is user-provided, estimated, or set to a default value. A metric depth map fixes depth along the optical axis, but the lateral 3D coordinates and the resulting camera translation scale depend on the assumed focal length and principal point. If a default or wrongly estimated K is used, the interactive 3D scene is not metric, and a trajectory drawn in it is not in the same scale as the training trajectories from RealEstate10K. This breaks the claimed metric scene-scale guarantee independently of whether Depth Anything V2's depth output is accurate. Appendix E discusses robustness to depth-predictor error via consistent error propagation, but that argument assumes the same K mapping; it does not cover unknown intrinsics. Without a stated K handling procedure, the core one-round inference scenario is underspecified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RealCam-I2V, an image-to-video generation framework that adds interactive, precise camera control by integrating monocular metric depth as a preprocessing step. During training, camera poses from RealEstate10K are rescaled from relative to metric units by aligning COLMAP point clouds with Depth Anything V2 metric depth. During inference, the metric depth is used to build an interactive 3D point-cloud scene where users draw camera trajectories and receive preview feedback, and a scene-constrained noise-shaping mechanism modifies early denoising steps using rendered preview frames. Experiments on RealEstate10K and out-of-domain images, with ablations of the two main components, report improved camera-control metrics and FVD relative to reproduced MotionCtrl, CameraCtrl, and CamI2V baselines, and the paper additionally demonstrates applications such as looping, interpolation, and transitions.","tokens_in":19592,"tokens_out":8867,"duration_ms":83943,"significance":"If the results hold, the paper addresses a genuine usability and consistency problem in camera-controlled video generation: replacing relative-scale trajectory conditioning with a single, depth-derived metric scale and giving users an interactive 3D preview is a practical and plausible design. The paper is also commendable for including a component-wise ablation of metric-scale alignment and noise shaping, sensitivity analysis for the noise-shaping threshold (Table 4), generalization tests on an additional dynamic dataset (RealCam-Vid), and applications beyond basic generation. The core conceptual contribution—unifying the scale of training data and user input through one metric-depth model—is clearly stated and internally consistent. However, the verification of the central claim is weakened by (i) an underspecified inference procedure for camera intrinsics, (ii) omission of the closest concurrent metric-depth-based methods from the comparison, (iii) an evaluation protocol that filters difficult test clips and lacks error bars, and (iv) a depth backbone whose stated indoor range conflicts with the claimed out-of-domain generality.","major_comments":[{"comment":"The inference pipeline is underspecified for arbitrary user-supplied images because the camera intrinsics matrix K is never defined for such inputs. On RealEstate10K, K is known from the dataset; for the out-of-domain images in Figure 9 and the claimed real-world use case, the paper does not state whether K is user-provided, estimated, or set to a heuristic default. Since the lateral 3D coordinates and the resulting camera translation scale depend on K, a wrong or arbitrary K breaks the claimed metric-scale consistency between the interactive scene and the metric-aligned training data. The consistency argument in Appendix E covers depth-predictor error but assumes the same K mapping. Please specify how K is obtained for arbitrary images, or explicitly state the assumption and its consequences for the metric-scale guarantee.","section":"Section 3.3, Eq. (pc = D(u,v)K^{-1}[u v 1])"},{"comment":"The quantitative comparison omits 4DiM [70] and AC3D [1], both of which are named in Section 2 as methods that also use monocular metric depth to address scale inconsistency. Because these are the closest prior approaches to RealCam-I2V, the paper's claim of establishing 'a new sota' is not supported without comparing against them. Please add these baselines under the same base model and evaluation protocol, or justify their exclusion.","section":"Section 4.3, Table 1"},{"comment":"The evaluation protocol filters the dataset by removing the top and bottom 2% of maximum and minimum frame-level scaling factors before splitting into training and test sets. This removes clips with unstable scale alignment and likely inflates the measured performance, especially for metrics that depend on scale. Additionally, although five SfM trials are run per clip, the paper does not report standard deviations or confidence intervals for RotErr, TransErr, CamMC, or FVD. The headline 'over 30% improvement' in Table 1 is therefore not accompanied by evidence of statistical robustness. Please report error bars for the main results and, if possible, evaluate on the unfiltered test set.","section":"Sections 4.1 and 4.2"},{"comment":"The paper uses Depth Anything V2 Large Indoor as the metric depth predictor, which is stated to estimate metric depth only up to 20 m and is fine-tuned on indoor data. The method is nevertheless presented as a general real-world solution and demonstrated on outdoor and diverse-domain images (e.g., landscape, food, pets in Figure 9). The consistency argument in Appendix E assumes that the depth model's errors are propagated consistently between training and inference, but for outdoor images beyond the 20 m range, truncation or domain-dependent scale errors can break that consistency. The paper should validate the metric depth against ground-truth metric scale for the out-of-domain images, or restrict the claims to scenes within the depth model's reliable range.","section":"Section 4.1, Appendix C, Appendix E, Figure 9"}],"minor_comments":[{"comment":"\"We further enables applications\" should read \"We further enable applications.\"","section":"Abstract"},{"comment":"\"an metric-scale camera-to-world transformation\" should be \"a metric-scale camera-to-world transformation.\"","section":"Section 3.2"},{"comment":"The figure contains a garbled fragment \"RelativScalefailson\" that appears to be a placeholder; please clean up the figure text.","section":"Figure 1"},{"comment":"The asterisk notation is confusing: the caption says \"* denotes our reproduced results on DynamiCrafter\" but the table lists DynamiCrafter itself as the first row; clarify which rows are reproduced and which are original.","section":"Table 2"},{"comment":"The noise-shaping threshold tNS is described as a value in the range [0,1000], but the noise level t in the main text is normalized to [0,1] (Section 3.1). Please make the scale consistent or explicitly explain the conversion.","section":"Table 4 / Appendix D"},{"comment":"Metric3D v2 [28] and MonST3R [90] are cited only in the appendix but are relevant to the main text's discussion of depth predictors; consider citing them in Section 2 or Section 3.2.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper would be strengthened by a clear statement of how camera intrinsics are handled at inference and by comparisons with 4DiM and AC3D. The evaluation also needs error bars and perhaps a result on the unfiltered test set. The authors should be asked to clarify whether the 'metric scale' evaluation metrics in Eq. (4)-(5) truly measure absolute scale or a GT-normalized relative error. The paper relies on the authors' own prior work (CamI2V, RealCam-Vid); the novelty relative to these works should be made explicit."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a genuine engineering contribution: aligning COLMAP camera trajectories to monocular metric depth during training, then reusing the same metric 3D scene at inference, closes the relative-scale gap that makes camera-conditioned video models awkward to use. The interactive 3D UI and scene-constrained noise shaping are new, sensible components, and the ablation cleanly shows that both earn their keep. Second, don't take the real-world numbers at face value until the intrinsics question is answered: the paper never says where K comes from for arbitrary user images, and the metric guarantee in Section 3.3 depends on it.\n\nWhat the paper does well: the ablation study separates metric-scale alignment from noise shaping, and both components justify themselves. Appendix E is unusually thorough — a second depth predictor (Metric3D v2), different SfM tools (GLOMAP, MonST3R), a DiT backbone (CogVideoX), and noise-shaping threshold sensitivity. The RealCam-Vid reproduction on genuinely dynamic scenes is real evidence, and the paper is honest that RealEstate10K is mostly static.\n\nSoft spots, in order of softness. The intrinsics gap is real. The projection p_c = D(u,v)K^{-1}[u v 1] needs K; RealEstate10K has fixed intrinsics, but the out-of-domain demos never state whether K is user-provided, estimated, or a default. A wrong focal length distorts the lateral scene, so a trajectory drawn in the interactive 3D scene is not in the same scale as the training data — which quietly breaks the one-round metric-scale story for the exact use case the paper targets. Easy to fix, but it needs stating. Second, 4DiM and AC3D already use monocular metric depth for scale consistency; they're cited but never compared, which thins the empirical case and makes the novelty hard to judge. Third, the test set drops the worst scale-alignment clips, and the camera metrics carry no error bars despite trial averaging. Fourth, the \"over 30% improvement\" claim is really about CamMC; metric-scale TransErr improves only ~4%, so the prose overstates. Minor: noise shaping trades dynamics for control, visible in the threshold-sensitivity table; the paper discloses this, so I read it as a design trade-off, not a flaw. The reader's \"dynamics drop\" point is slightly loose — Dynamic Degree holds against the base model; the drop is only relative to the MSA-only ablation.\n\nWho benefits: anyone working on camera-conditioned I2V, especially on scale-consistent conditioning and usable trajectory interfaces. It deserves a serious referee. My recommendation: send it to review, with requests for the K-handling procedure, direct comparisons against 4DiM and AC3D, and error bars.","headline":"RealCam-I2V is a credible, well-ablated extension that unifies camera scale via monocular metric depth, but the unspecified intrinsics for arbitrary images and missing 4DiM/AC3D baselines keep the real-world claim from being fully proven.","tokens_in":20238,"tokens_out":7358,"would_cite":true,"duration_ms":65037,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RealCam-I2V claims that anchoring camera trajectories to metric-scale depth makes complex camera control precise and practical for real-world images, reporting over 30% gains on camera-control metrics and over 10% gains on FVD compared…","keywords":["image-to-video generation","camera control","metric depth estimation","diffusion model","3D scene interaction","noise shaping","camera trajectory","RealEstate10K"],"falsifier":"Take a real indoor scene with ground-truth depth from a depth sensor, run the full RealCam-I2V pipeline using the sensor depth instead of the monocular prediction, and compare the camera-control metrics; if metric-scale alignment only helps when the depth comes from the learned predictor, or if the sensor-depth version does not match the monocular one, then the claimed benefit is not the metric scale itself.","tokens_in":19091,"feed_emoji":"🎬","tokens_out":10080,"duration_ms":86338,"temperature":0.7,"pith_summary":"The paper argues that the main obstacle to practically useful camera-controlled image-to-video generation is scene scale: existing models are trained on relative-scale camera trajectories, so the camera motion they learn is ambiguous for real images of unknown size. RealCam-I2V inserts a monocular metric-depth step that reconstructs the input image as a metric-scale 3D point cloud, and uses that reconstruction in two places. During training, it rescales each video clip's structure-from-motion camera poses to metric units by aligning them with the depth-based point cloud; during inference, the same metric scene lets a user draw a camera trajectory directly in 3D and preview it as a static rendering in under a second. A scene-constrained noise-shaping step then pastes visible pixels of that preview into the diffusion model's high-noise stages, pinning the early layout and camera motion while leaving later stages free for dynamics. On RealEstate10K the method reports camera-control improvements over 30% on metric-scale measurements and FVD improvements over 10% relative to the prior state of the art, as well as generalization to out-of-domain images.","feed_headline":"Metric depth lets users draw the camera path a video must follow","feed_subtitle":"One image becomes a metric 3D scene; drawing a trajectory in it lifts camera-control accuracy by over 30%.","key_machinery":"The load-bearing object is metric scene-scale alignment: camera translations are rescaled by a per-scene factor obtained by aligning the monocular metric-depth point cloud with the structure-from-motion point cloud, which makes camera-conditioned diffusion training operate on physically meaningful units. The interactive 3D scene reprojects the depth map into a point cloud that the user can drag a trajectory through, and a rendered static preview gives immediate feedback. Scene-constrained noise shaping masks that preview and pastes visible, non-edge pixels into the predicted clean latent at noise levels $t>0.9$, constraining early layout while leaving later denoising steps free to synthesize dynamics.","core_discovery":"The paper's central claim is that camera-trajectory-guided image-to-video generation becomes precise and practical once the trajectory is expressed in metric scale, because the model no longer has to guess the scene's true size. It establishes this by using a monocular metric-depth network to reconstruct the reference image as a metric-scale 3D point cloud, rescaling every training video's structure-from-motion camera poses to match that metric scale, and letting users draw trajectories directly in the reconstructed scene while feeding a static preview into the diffusion model's high-noise stages via scene-constrained noise shaping. RealCam-I2V reports that this combination lowers camera-control errors by over 30% on metric-scale measurements and improves FVD by over 10% relative to the strongest prior method, while preserving the base model's ability to generate dynamic content and generalizing to out-of-domain images.","pith_inferences":["If monocular depth is trustworthy, the core idea turns camera control into a 3D drag-and-preview interaction, which could lower the entry barrier for camera-controlled generation enough for consumer editing tools.","The alignment recipe is not tied to a particular base model: because it only rescales the conditioning signal, it could be applied to other diffusion backbones (the paper reports one transformer-based variant) with the same expected benefit.","The noise-shaping step is a general recipe: any cheap geometric proxy of the desired video, such as a depth map, point-cloud render, or coarse layout, could be injected at high noise levels to steer early structure, suggesting a testable extension to object motion or lighting control.","A direct test of the paper's load-bearing premise would be to swap the monocular depth for sensor depth on a few real indoor scenes; if the gains persist with imperfect monocular depth but not with perfect depth, the claimed benefit is not actually caused by metric scale."],"forward_implications":["With metric-scale conditioning, camera-control error measures such as RotErr, TransErr, and CamMC drop sharply, and training the same base model on aligned trajectories improves even existing camera-conditioned baselines.","Users can produce a precise trajectory in one round: draw in the 3D scene, see a static preview immediately, then generate; the slow error-then-adjust loop of text or 2D input disappears.","Scene-constrained noise shaping at high noise levels keeps camera adherence strong without freezing the scene; lowering the shaping threshold to 600 or 800 trades dynamics for stricter layout matching.","The same model supports camera-controlled loops, frame interpolation, and continuation through simple changes to the condition-frame concatenation, enabling smooth transitions and long videos.","The method generalizes outside its training domain to pets, landscapes, anime, and food, while preserving the base model's motion dynamics."],"supporting_citations":[{"why":"Supplies the monocular metric-depth predictions used to reconstruct absolute-scale 3D scenes for both training alignment and inference interaction.","marker":"[81]"},{"why":"Provides the structure-from-motion point clouds whose relative scale is aligned to the metric depth during training.","marker":"[54]"},{"why":"Is the image-to-video diffusion base model that RealCam-I2V extends with metric-scale camera conditioning and noise shaping.","marker":"[76]"},{"why":"Is the strongest camera-controlled baseline whose relative-scale training is compared and whose reproduced results are improved in metric scale.","marker":"[97]"},{"why":"Is the training and evaluation dataset containing relative-scale camera annotations used for alignment.","marker":"[101]"},{"why":"Is a camera-conditioning baseline that uses Plücker embeddings, providing the comparison for metric-scale conditioning benefits.","marker":"[22]"},{"why":"Is a trajectory-conditioned baseline whose relative-scale results improve when retrained with metric-scale alignment.","marker":"[69]"},{"why":"Inspires the noise-shaping idea by showing that clean reference content can be overlaid on noised latents to guide synthesis.","marker":"[44]"},{"why":"Inspires the noise-shaping mechanism through inversion-style reuse of clean features during early denoising steps.","marker":"[56]"}],"fun_headline_variants":["Draw a 3D camera path in metric space to guide video gen","Metric depth makes camera control as easy as drawing a line","One image becomes a metric 3D scene for precise video camera control","Turn any photo into a 3D set you can drag the camera through","RealCam-I2V: draw trajectories in 3D to steer video generation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything depends on the monocular depth network's prediction being the true real-world scale of the scene; if that scale is wrong for an image, the training alignment, the 3D drawing interface, and the noise-shaping preview all inherit the same error.","fun_headline_variants_meta":{"raw":{"variants":["Draw a 3D camera path in metric space to guide video gen","Metric depth makes camera control as easy as drawing a line","One image becomes a metric 3D scene for precise video camera control","Turn any photo into a 3D set you can drag the camera through","RealCam-I2V: draw trajectories in 3D to steer video generation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000207,"raw_usage":{"total_tokens":1404,"prompt_tokens":956,"completion_tokens":448,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":572,"completion_tokens_details":{"reasoning_tokens":352}},"tokens_in":572,"tokens_out":448,"duration_ms":4283,"temperature":1.0,"reasoning_tokens":352,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T19:33:52.410096+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a real indoor scene with ground-truth depth from a depth sensor, run the full RealCam-I2V pipeline using the sensor depth instead of the monocular prediction, and compare the camera-control metrics; if metric-scale alignment only helps when the depth comes from the learned predictor, or if the sensor-depth version does not match the monocular one, then the claimed benefit is not the metric scale itself.","supporting_citations":[{"cited_title":"Structure-from-motion revisited","cited_arxiv_id":null,"evidence_quote":"Provides the structure-from-motion point clouds whose relative scale is aligned to the metric depth during training."},{"cited_title":"Stereo magnification: Learning view synthesis using multiplane images","cited_arxiv_id":null,"evidence_quote":"Is the training and evaluation dataset containing relative-scale camera annotations used for alignment."},{"cited_title":"Motionctrl: A unified and flexible motion controller for video generation","cited_arxiv_id":null,"evidence_quote":"Is a trajectory-conditioned baseline whose relative-scale results improve when retrained with metric-scale alignment."}],"review_version":1}