{"id":"321c8100-3925-451e-996a-7883e0fd5f30","arxiv_id":"2412.00851","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A pose-free two-image pipeline decomposes a dynamic scene into rigid objects and fits per-Gaussian SE(3) motions to synthesize novel views of moving scenes.","lead":"This paper presents a method that builds a moving 3D Gaussian scene model from just two photos taken at different times, without knowing where the camera was. It could make realistic new views of dynamic scenes possible from minimal snapshots, which matters for robotics, autonomous driving, and augmented reality.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The evaluation target is used during inference: Sec. 4.3 optimizes test camera pose and per-object SE(3) ratios against Itest (Eq. 6), so Table 1 measures target fitting, not held-out two-view novel view synthesis.","rationale":"The paper's central claim, as stated in the abstract and in the reader's strongest_claim, is that two unposed images suffice to fit dynamic Gaussians and render a temporally intermediate novel view. That claim requires the intermediate view to be produced without using that view's pixels during inference. Section 4.3 violates this requirement: Eq. (6) feeds robj into the renderer and the optimization objective minimizes photometric error against Itest, jointly refining test camera pose and per-object SE(3) ratios. The target is thus used to set the temporal positions of dynamic objects. Because the metrics in Table 1 are computed on that same Itest, they report how well the method can adapt the model to the target, not how well it predicts an unseen view from two images. Table 2 provides a direct estimate of the effect: dropping only the ratio alignment (while still optimizing camera pose on the target) reduces Kubric PSNR from 33.86 to 32.14. This is a large fraction of the reported advantage over baselines, and the baselines are not allowed an analogous per-object temporal fitting knob. I therefore do not regard the current experiments as establishing the two-view claim. The static-background assumption identified by the reader as the weakest assumption is real, since T^(0) is the only anchor for separating camera motion from object motion, but it limits applicability rather than invalidating the measured numbers. The target-leakage issue invalidates, or at least heavily qualifies, all of them. The remedy is a held-out evaluation with fixed ratios and a BA-only pose, plus a target-adapted baseline for fairness. If that re-evaluation preserves the margins, the central claim would be supported. Until then, conditional acceptance with this specific re-evaluation requirement is the appropriate verdict.","tokens_in":12440,"tokens_out":9162,"duration_ms":86484,"concrete_test":"Re-run the Kubric and KITTI evaluations in a strictly held-out mode: train on I0 and I2 only, fix robj=0.5 for all objects, and use the BA-only camera pose T^(0) (no optimization against I1); render the intermediate frame and recompute PSNR/SSIM/LPIPS. Compare with Table 1. If Kubric PSNR drops by roughly 1.7 dB or more, the reported margin over baselines is substantially target fitting rather than two-view generalization. Additionally, run at least one baseline with the same target-based pose/ratio adaptation to verify fairness of the comparison.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing concern is the evaluation protocol in Sec. 4.3, not the static-background assumption. To render the intermediate frame, the method fixes the Gaussians trained on the two unposed frames and then minimizes photometric loss against the actual test image Itest, optimizing the test camera pose Tcam and, in particular, each object's SE(3) interpolation ratio robj (Eq. 6). The target image is therefore used at inference to place objects at their target positions and to align the camera: the rendered frame is a fit to Itest, not a prediction from the two input images. This affects every number in Table 1. The ablation in Table 2 quantifies the leak: fixing robj=0.5 lowers Kubric PSNR from 33.86 to 32.14, so about 1.7 dB of the headline result comes directly from target-based ratio optimization. Even if optimizing an unknown test camera pose against its own image is a common concession in pose-free NVS, the per-object SE(3) ratio is not a camera pose; it is a temporal motion parameter adjusted with the answer image in hand. Baselines are not given an equivalent target-adaptive per-object temporal parameter, so the comparison is not apples-to-apples. The central claim 'use only two images' is thus not what is currently demonstrated: the intermediate view is rendered with the aid of the third image. This is a fixable evaluation flaw rather than a fatal flaw, but it must be fixed before the paper can support its headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DynSUP, a two-stage method for fitting dynamic 3D Gaussian Splatting (3DGS) from two unposed images with known intrinsics. In the first stage, an object-level dense bundle adjustment decomposes the scene into a static background and piecewise rigid dynamic objects, jointly estimating camera pose and per-object SE(3) motions using optical flow and monocular depth. In the second stage, each Gaussian is initialized with its object's SE(3) motion and a dense per-Gaussian SE(3) field is optimized jointly with Gaussian parameters and camera pose. At test time, the method additionally optimizes the camera pose and per-object SE(3) interpolation ratios against the actual test image (Sec. 4.3, Eq. 6). Experiments are reported on KITTI and Kubric, with comparisons against 4DGS, SC-GS, and InstantSplat, plus ablations of the SE(3) initialization and the test-time ratio alignment.","tokens_in":12741,"tokens_out":6466,"duration_ms":55868,"significance":"If the evaluation protocol is corrected, this would be a meaningful step: it is the first 3DGS formulation that explicitly combines object-level two-view bundle adjustment with a per-Gaussian SE(3) motion field for dynamic scenes from sparse, pose-free input. The ablation study does provide evidence that the SE(3) initialization matters (Kubric PSNR 26.00 without initialization vs 33.86 with it), and the piecewise-rigid decomposition is a sensible way to constrain an under-determined problem. The paper also honestly lists non-rigid motion and segmentation quality as limitations. However, the current test-time procedure uses the target image to optimize both camera pose and per-object temporal interpolation ratios, so the reported numbers partly measure fitting to the target rather than prediction from the two training images. The central claim of 'use only two images' is therefore not yet supported as stated.","major_comments":[{"comment":"The evaluation protocol leaks the target image into inference. The text states that at test time the method fixes the trained Gaussians and minimizes the photometric loss against the actual test image Itest, jointly optimizing the camera pose Tcam and, in particular, each object's SE(3) interpolation ratio robj. This means the intermediate frame is rendered after aligning the camera and placing each object at its target temporal position using the answer image. The per-object ratio is not a camera pose, so this is not an ordinary pose-free NVS concession. The ablation in Table 2 quantifies the effect: fixing robj=0.5 reduces Kubric PSNR from 33.86 to 32.14, so about 1.7 dB of the headline result comes directly from target-based ratio optimization. Baselines are not given an equivalent target-adaptive per-object temporal parameter, so the comparison is not apples-to-apples. Please re-evaluate with robj fixed at 0.5 (or otherwise determined from the two training frames alone), and report both settings so the reader can see the predictive performance of the two-view model without the test-time alignment.","section":"Sec. 4.3, Eq. (6); Table 1; Table 2"},{"comment":"The formula T_obj^(i) = Tcam (T^(i))^{-1} appears inconsistent with the stated definitions. Since T^(i) maps a 3D point from the I0 camera coordinate system to the I1 camera coordinate system, and T^(0)=Tcam for the static region, for a dynamic region the combined transformation should satisfy T^(i) = Tcam * T_obj^(i) (assuming the world frame coincides with the I0 camera frame). Then T_obj^(i) = Tcam^{-1} T^(i), not Tcam (T^(i))^{-1}. As printed, the initialization of the SE(3) field would be the inverse/conjugation of the intended object motion. Given that the ablation shows SE(3) initialization is crucial for the method's performance, please clarify the transformation convention or correct the formula, and verify that the implemented initialization matches the corrected expression.","section":"Sec. 4.1, object transformation formula"},{"comment":"The method relies on a detectable static background region P^(0) whose apparent motion between the two frames is purely camera motion, because the camera pose Tcam is taken directly from T^(0). If a scene has no reliable static region, or the static region is too small, camera motion cannot be separated from object motion, and the SE(3) initialization and the subsequent rendering pipeline lose their anchor. Both datasets used in the paper (KITTI driving scenes, Kubric with static floor and walls) contain prominent static backgrounds, so this assumption is never stress-tested. The conclusion lists non-rigid motion and segmentation quality as limitations but does not mention the static-background requirement. Please add a discussion of this requirement and, if possible, an experiment or qualitative example on a sequence without a dominant static background.","section":"Sec. 4.1, static background assumption"}],"minor_comments":[{"comment":"The column header 'LPIPS ↑' should be 'LPIPS ↓', since lower LPIPS is better.","section":"Table 2"},{"comment":"The related-work text cites 'MASt3R [25]' and 'Splatt3R [13]', but reference [25] is the Splatt3R paper and reference [13] is Pf3plat; the labels and the reference list need to be corrected to match the intended works.","section":"References"},{"comment":"The notation in Eq. (3) is confusing: Lreproj and Ld already include sums over pixels, so writing Lba = sum_{p in P(i)} (lambda1 Lreproj + lambda2 Ld) reads as a sum of sums. Please rewrite using per-region losses, e.g., Lba = sum_i (lambda1 Lreproj^(i) + lambda2 Ld^(i)).","section":"Eq. (3)"},{"comment":"The statements 'use only two images' and 'two images captured at distinct moments with unknown poses' should also mention that known camera intrinsics are required, since K appears in all projection equations.","section":"Abstract and Fig. 1 caption"},{"comment":"The SE(3) regularization Lreg is applied only to dynamic regions (i>0). It would be helpful to state explicitly whether background Gaussians are allowed to drift during optimization and, if so, how background rigidity is enforced.","section":"Sec. 4.2, regularization"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising, and the two-stage pipeline is coherent. The main reason for major revision is the evaluation protocol: the test-time optimization of per-object SE(3) ratios against the target image, combined with the lack of an equivalent parameter for baselines, makes Table 1 difficult to interpret. I would also encourage the authors to double-check the object transformation formula in Sec. 4.1 and to provide code or a detailed reproducibility statement, since no code is currently linked and several reference labels are mismatched."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is the first method I've seen that fits dynamic 3D Gaussians from two unposed images, and the pipeline is coherent. But the headline numbers in Table 1 are partly a fit to the test image: at inference you optimize the test camera pose and, more importantly, each object's SE(3) interpolation ratio by minimizing photometric loss against Itest itself (Sec. 4.3, Eq. 6). So the intermediate view is not a pure prediction from the two input frames.\n\nWhat's genuinely new: the combination of object-level two-view bundle adjustment with a per-Gaussian SE(3) field. The decomposition into piecewise rigid components is a reasonable way to get initialization for the Gaussians, and the ablation shows SE(3) init matters a lot (26.00 vs 33.86 PSNR on Kubric). The paper is clearly written and the limitations are acknowledged (non-rigid motion, segmentation dependence).\n\nThe soft spot is the evaluation. The stress-test note is right: optimizing robj against the test image leaks the target into the result. The ablation quantifies this: fixing robj=0.5 drops PSNR from 33.86 to 32.14, so about 1.7 dB of the headline number comes directly from target-based ratio optimization. Baselines don't get an equivalent target-adaptive per-object temporal parameter, so the comparison isn't apples-to-apples. This is fixable: either freeze robj and camera pose, or give baselines the same test-time optimization, or evaluate on a held-out fourth frame. The static-background assumption is a real limitation but the paper flags segmentation quality; it just doesn't stress-test scenes without a dominant static region.\n\nMinor: no error bars, no code release, and Monst3R is discussed but not compared quantitatively despite being the closest prior work on dynamic geometry from sparse views.\n\nBottom line: the method is plausible and the setting is new, so it deserves a serious referee. I'd want the re-evaluation before believing the absolute numbers, but the core idea stands. Send to peer review with a request for an honest protocol.","headline":"Novel and coherent pipeline for dynamic GS from two unposed images, but the test image is used at inference to tune pose and per-object SE(3) ratios, inflating Table 1.","tokens_in":13359,"tokens_out":2184,"would_cite":false,"duration_ms":18374,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A method that fits dynamic 3D scenes as Gaussians from just two unposed images, jointly recovering camera pose and object motion.","keywords":["3D Gaussian Splatting","dynamic scene reconstruction","pose-free novel view synthesis","bundle adjustment","SE(3) motion field","two-view reconstruction","temporal consistency","motion segmentation"],"falsifier":"Take two frames of a dynamic scene that contains no static background, run DynSUP, and check whether camera pose and object motions are still recovered and whether novel-view rendering remains sharp; if the pipeline fails or the camera pose is grossly wrong, the static-background anchor is required.","tokens_in":12197,"feed_emoji":"🎥","tokens_out":8884,"duration_ms":63877,"temperature":0.7,"pith_summary":"This paper claims that two images taken at different moments from unknown camera poses are sufficient to fit a dynamic scene as 3D Gaussians and render a new view at an intermediate time. The method splits the scene into piecewise-rigid components, solves the camera pose and every object's motion in a single bundle adjustment, then lets each Gaussian refine its own SE(3) transformation. If correct, it is the first demonstration of dynamic Gaussian fitting from two unposed views, removing the usual need for dense views, known poses, and a static scene. Experiments on KITTI driving scenes and the synthetic Kubric dataset report consistent gains over methods that assume static scenes, multiple images, or known poses, and ablations show the object-level initialization is what makes the ambiguous two-view problem converge.","feed_headline":"Rebuild a dynamic 3D scene from two images with unknown poses","feed_subtitle":"Camera pose and per-object motion are solved jointly, then each Gaussian refines its own motion to render new views.","key_machinery":"The load-bearing mechanism is a two-stage pipeline: first, an object-level two-view bundle adjustment that uses optical flow, monocular depth, and motion segmentation masks to divide the scene into static background and rigid objects, then jointly optimizes camera pose, per-object SE(3) transformations, and depths; second, an SE(3) field-driven Gaussian splatting stage in which each 3D Gaussian carries its own learnable SE(3) transformation, initialized from the object-level motion, with a Huber-based regularization that keeps motions smooth within each object and a 6D rotation representation for stable optimization. This bridges geometry (reprojection, depth consistency) and photometry (rendering loss), and the same pipeline with per-object interpolation ratios is reused at test time to align the temporal position of objects.","core_discovery":"The paper's central claim is that a dynamic scene can be fitted with 3D Gaussian Splatting from a single, unposed pair of images. An object-level dense bundle adjustment decomposes the scene into static background and piecewise-rigid objects, jointly estimating the camera motion and each object's SE(3) motion while regularizing depth. These motions seed an SE(3) field in which every Gaussian has its own learnable rotation and translation, refined jointly with the Gaussian parameters and camera pose under photometric loss. The resulting representation supports rendering a novel view at an intermediate timestamp, including a test-time alignment that optimizes per-object SE(3) interpolation ratios. The paper argues this is the first demonstration of dynamic Gaussian fitting from two unposed views, with experiments showing consistent improvement over methods that assume static scenes, multiple images, or known poses.","pith_inferences":["The method's dependence on a detectable static background suggests a natural stress test: scenes where all visible content moves, such as a train interior or a flock of birds filling the frame, should break the camera-object decomposition unless the ambiguity is resolved by other cues.","A natural extension is to chain the two-view bundle adjustment across additional frames, which would give richer observations for the SE(3) field and might relax the static-background requirement.","The per-object interpolation-ratio alignment at test time is effectively a temporal pose estimate, so the same mechanism could estimate the time offset between two unposed frames, turning the method into a synchronization tool for uncalibrated cameras.","Because the pipeline leans on learned depth and flow priors, its reliability on out-of-distribution motion is bounded by those priors; a controlled test with fast motion and heavy occlusion would clarify how much geometric constraint the two-view objective alone provides."],"forward_implications":["Dynamic novel-view synthesis becomes possible from a two-frame capture with no pose metadata, which matters for casual photography and autonomous driving logs where pose estimation is unreliable.","The explicit SE(3) field gives an interpretable per-object motion model that can be interpolated in time, enabling temporal alignment of test frames rather than just spatial rendering.","The object-level bundle adjustment provides the initialization that makes the otherwise ambiguous two-view dynamic problem converge; the ablation study shows that removing it degrades rendering sharply in large-motion regions.","Because the representation is explicit 3D Gaussians with rigid motions, it can be rendered in real time with existing rasterizers, unlike deformation-field methods that need dense temporal observations."],"supporting_citations":[{"why":"This paper defines the 3D Gaussian splatting representation and differentiable rasterizer that the whole pipeline builds on.","marker":"[17]"},{"why":"This paper provides the forward and backward optical flow used to establish correspondences and confidence weights in the bundle adjustment.","marker":"[35]"},{"why":"This paper supplies monocular depth maps that initialize and regularize the optimized depths in the object-level bundle adjustment.","marker":"[39]"},{"why":"This paper detects rigid motion regions from two frames, producing the initial dynamic-object boxes.","marker":"[38]"},{"why":"This paper refines the motion boxes into precise pixel masks that partition the scene into static background and dynamic objects.","marker":"[23]"},{"why":"This paper provides the dense point map and camera poses used to initialize the comparison baselines, since COLMAP fails on two-view dynamic input.","marker":"[30]"},{"why":"This paper represents a dense-video dynamic Gaussian baseline whose deformation field needs many observations and known poses, used as a comparison.","marker":"[33]"},{"why":"This paper is a sparse-control-point dynamic Gaussian baseline requiring dense videos with poses, used as a comparison.","marker":"[16]"},{"why":"This paper is a pose-free static-scene Gaussian splatting baseline showing the gap that dynamics and unknown poses introduce.","marker":"[6]"}],"fun_headline_variants":["Two unposed photos rebuild dynamic 3D scenes","Dynamic Gaussian splatting from a single unposed image pair","Joint camera and object motion from just two views","Per-Gaussian SE(3) motion from an unposed pair","Dynamic 3D Gaussian fitting without any poses"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes a detectable static background region whose apparent motion between the two frames is purely camera motion, because the camera pose is taken directly from the static region's motion. If no static region can be found, camera and object motion cannot be separated.","fun_headline_variants_meta":{"raw":{"variants":["Two unposed photos rebuild dynamic 3D scenes","Dynamic Gaussian splatting from a single unposed image pair","Joint camera and object motion from just two views","Per-Gaussian SE(3) motion from an unposed pair","Dynamic 3D Gaussian fitting without any poses"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000226,"raw_usage":{"total_tokens":1453,"prompt_tokens":916,"completion_tokens":537,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":532,"completion_tokens_details":{"reasoning_tokens":458}},"tokens_in":532,"tokens_out":537,"duration_ms":5672,"temperature":1.0,"reasoning_tokens":458,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T04:56:30.458544+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two frames of a dynamic scene that contains no static background, run DynSUP, and check whether camera pose and object motions are still recovered and whether novel-view rendering remains sharp; if the pipeline fails or the camera pose is grossly wrong, the static-background anchor is required.","supporting_citations":[{"cited_title":"Unifying flow, stereo and depth estimation","cited_arxiv_id":null,"evidence_quote":"This paper provides the forward and backward optical flow used to establish correspondences and confidence weights in the bundle adjustment."},{"cited_title":"Learning to seg- ment rigid motions from two frames","cited_arxiv_id":null,"evidence_quote":"This paper detects rigid motion regions from two frames, producing the initial dynamic-object boxes."},{"cited_title":"Dust3r: Geometric 3d vi- sion made easy","cited_arxiv_id":null,"evidence_quote":"This paper provides the dense point map and camera poses used to initialize the comparison baselines, since COLMAP fails on two-view dynamic input."},{"cited_title":"4d gaussian splatting for real-time dynamic scene render- ing","cited_arxiv_id":null,"evidence_quote":"This paper represents a dense-video dynamic Gaussian baseline whose deformation field needs many observations and known poses, used as a comparison."}],"review_version":1}