{"id":"656e4019-6e8b-449e-8392-fcf25db60f11","arxiv_id":"2412.16155","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Generating intermediate frames with a video model and feeding them to DUSt3R improves pairwise pose estimation for low-overlap images, when a medoid-based self-consistency score selects the best generated video.","lead":"This paper tests whether videos generated between two photos can help a pose estimation model recover the relative camera positions. The method, InterPose, feeds generated intermediate frames to a standard pose estimator and selects the most self-consistent video, giving small but consistent accuracy gains on low-overlap image pairs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed world-prior effect is not isolated: no control with non-generative added frames, and the D_bias-only ablation suggests gains may not stem from the video model's content.","rationale":"The reader's weakest assumption (self-consistency score) is valid, but the Oracle gap does not invalidate the central claim; it only shows the selection is suboptimal. A more fundamental threat is that the experiments do not isolate the cause of the reported improvement. DUSt3R is a multi-view method; adding any frames changes its optimization. The D_bias-only ablation (Table 4, Cambridge, Dream Machine: 11.25 vs 13.28) shows that simply selecting the medoid pose closest to the baseline yields much of the gain, consistent with a variance-reduction or regularization effect rather than the video model encoding a geometric prior. A control using non-generative frames under the same protocol is necessary to attribute the improvement to the generative model's world prior. The Oracle results do show that generated videos contain useful geometric information, so the idea is promising, but the paper's headline claim overreaches the evidence. I therefore recommend conditional acceptance with the added requirement of a non-generative-frame control, or a rewording of the claim. This is a partial agreement with the reader: we both flag the selection procedure as a concern, but the missing control is the load-bearing issue.","tokens_in":22160,"tokens_out":9580,"duration_ms":88885,"concrete_test":"Run the identical InterPose pipeline on Cambridge Landmarks and ScanNet, but replace the generated frames with three controls per pair: (i) copies of IA and IB, (ii) linear alpha blends between IA and IB, and (iii) Gaussian-noise images. Apply the same 4-video creation strategy (using different orderings or noise seeds to create candidate videos) and the same Dmed+Dbias scoring with m=11 subsets. If the control's MRE/MTE on Cambridge and ScanNet are within the margin of the Dream Machine rows in Table 1, the improvement cannot be attributed to the generative video model's world prior, and the central claim must be weakened.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim (Section 1) is that a generative video model improves pose estimation 'by acting as a world prior.' The experiments compare DUSt3R on the pair alone against DUSt3R on the pair plus generated frames with medoid selection, but never against DUSt3R on the pair plus non-generative additional frames. DUSt3R is a multi-view estimator; supplying it with any extra images changes its optimization. Critically, Table 4 (ablation on distance metrics) shows that for Dream Machine on Cambridge, selection using D_bias alone (closeness to the pair-only estimate) yields MRE 11.25, already below the 13.28 pair-only baseline. This raises the possibility that the reported gains come from taking a medoid over multiple DUSt3R runs on subsets and anchoring near the baseline, rather than from geometric information in the generated frames. Without a control that replaces generated frames with e.g. duplicated or blended input images under the same 11-subset medoid protocol, the central claim that the generative model's learned prior is the cause is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes InterPose, a method that uses off-the-shelf generative video models to synthesize interpolated frames between two input images with low or no overlap, feeds the original pair plus generated frames into a multi-view pose estimator (DUSt3R, and MASt3R in the supplement), and selects among multiple generated videos using a medoid-based self-consistency score (Eqs. 8-9). The experiments cover four datasets (Cambridge Landmarks, ScanNet, DL3DV-10K, NA VI), three video generators (DynamiCrafter, Runway, Dream Machine), and report mean rotation/translation errors as well as accuracy/AUC metrics. The central claim is that generated video frames act as a world prior and consistently improve DUSt3R on low-overlap pairs; the paper also reports an Oracle upper bound and ablations over distance metrics and number of frames.","tokens_in":22368,"tokens_out":5858,"duration_ms":51579,"significance":"If the central claim held, this would be a valuable demonstration that large generative video models trained on web-scale data can provide useful geometric priors for 3D vision. The paper has genuine strengths: it evaluates three different video models on four datasets, includes an oracle analysis that quantifies headroom, tests a second pose estimator (MASt3R), and ablates the number of frames and the yaw range. However, the claim as stated is stronger than the evidence: several gains are small, no significance tests are reported, and the selection metric is partly anchored to the baseline. The manuscript's own supplementary results (Table 7) show a case with slightly worse rotation error on non-overlapping ScanNet pairs, and the conclusion concedes that all generated videos can be low quality. These caveats make the contribution interesting but in need of revision.","major_comments":[{"comment":"The central claim that a generative video model improves pose estimation 'by acting as a world prior' is not isolated by any control condition in which non-generative additional frames are supplied to DUSt3R under the same protocol. Since DUSt3R is a multi-view estimator, feeding it extra images changes its optimization even when the images carry no new geometric information; Table 4 (D_bias-only row for Dream Machine on Cambridge, MRE 11.25 vs. 13.28) shows that selecting predictions close to the pair-only estimate already yields a large share of the reported gain. A control with duplicated, blended, or randomly warped input images under the same 4-video, 11-subset medoid protocol is needed to attribute the improvement to the learned content of the generated frames.","section":"Sec. 4.2, Tables 1-2"},{"comment":"The selection score D_total includes dist(T_med, fpose({IA,IB})), which anchors the chosen video's pose toward the baseline DUSt3R estimate. This makes the method partially a 'stay near the baseline' heuristic rather than a pure measure of the generated video's consistency. The Oracle gap in Tables 1-2 (e.g., Cambridge MRE 3.65 vs. 10.78 for the reported Medoid result) further shows that consistency across subsets is a weak proxy for correctness, so the method's practical gains are contingent on this proxy holding. The paper should quantify how often the selected video differs from the baseline-selected video and report results with the bias term removed or reweighted on a validation split.","section":"Sec. 3.2, Eq. (9), Table 4"},{"comment":"No error bars, confidence intervals, or significance tests are reported for any of the main comparisons. Several improvements are small (e.g., DynamiCrafter MRE on Cambridge: 12.70 vs. 13.28; several accuracy differences under 2 points), and with 290-300 pairs per dataset, paired bootstrap or per-pair significance testing would be needed to support the repeated claim of 'consistent improvements.' Without these, the universal claim is not established even though the larger reductions, such as ScanNet MTE from 24.72 to 15.88, are themselves notable.","section":"Sec. 4.5, Tables 1-2"},{"comment":"The metric D_total appears to have been designed after observing that D_med alone fails on Cambridge with Dream Machine (MRE 19.37 vs. 11.96 for D_total), and the paper states 'we found it helpful to bias the metric.' Because the same four test sets were used to motivate the metric, a validation-based protocol or an independent test set is needed to rule out overfitting to the benchmark. In addition, the supplementary results for non-overlapping ScanNet pairs (Table 7: MRE 83.94 for Ours vs. 83.48 for DUSt3R) and the main paper's admission that all generated videos can be low quality show that the 'consistent improvements' claim does not extend to the full yaw range; the abstract and contributions should be qualified accordingly.","section":"Sec. 3.2 and Supp. Sec. D.1, Table 4; Supp. Table 7"}],"minor_comments":[{"comment":"The sentence 'For each pair, we the settings described in the main paper' is incomplete; it should read 'we use the settings described in the main paper.'","section":"Supp. Sec. B"},{"comment":"Table 4's caption contains the typo 'Abltion' instead of 'Ablation', and Table 3 uses 'Navi' inconsistently with 'NA VI' in the main text.","section":"Supp. Tables 3-4"},{"comment":"Clarify that the medoid is the sample minimizing the average distance to the other samples and that D_med is the value of that minimized average; the current notation could be misread as a separate quantity from T_med.","section":"Sec. 3.2, Eq. (8)"},{"comment":"The project page link appears without the URL scheme ('Inter-Pose.github.'); ensure a complete, working hyperlink is provided.","section":"Abstract and Supp. Sec. A"},{"comment":"The three video models produce very different frame counts (16 for DynamiCrafter, 112 for Runway, 114 for Dream Machine); a note on how these counts interact with the k=5 frame-subset sampling would improve the comparability of the three generators.","section":"Sec. 4.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript's conclusions rest on a comparison that lacks a non-generative control and on a selection metric with a built-in baseline bias. These are fixable with additional experiments and a more qualified claim. The oracle analysis is a useful strength, but the gap between Oracle and Medoid selection should be discussed as a limitation of the proposed selection heuristic rather than a minor detail. The paper is within scope for a computer vision venue, though the 'world prior' phrasing overstates what is currently demonstrated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely new application and the experiments are mostly solid. The authors show that feeding DUSt3R generated intermediate frames from DynamiCrafter, Runway, or Dream Machine, plus a medoid-based selection over sampled frame subsets, reduces relative pose error on low-overlap pairs across four datasets. That holds for all three video models on the DUSt3R backbone, including the harder outward-facing Cambridge and ScanNet splits. The oracle experiments are honestly reported and the gap between oracle and medoid selection frames the remaining challenge well.\n\nThe paper does good things: three video models, four datasets, an ablation of the distance metric, a frame-count ablation, MASt3R results, yaw sweeps in the supplement, and a clear statement of limitations and cost. That is a solid preliminary study, not a toy.\n\nThe soft spots are real but not disqualifying. First, there is no control where non-generative extra frames are added to DUSt3R under the same eleven-subset medoid protocol. DUSt3R is a multi-view estimator, so part of the gain could come simply from optimizing over more frames and from the Dbias term that anchors selection near the pair-only estimate. The Table 4 ablation shows Dbias alone on Cambridge with Dream Machine gives 11.25, below the 13.28 baseline, so the attribution of the gain to video-model content is not isolated. The same table shows Dtotal is better on ScanNet and the other datasets, so the video content is probably contributing, but I want the control before using the phrase 'world prior' without qualification.\n\nSecond, there are no error bars or significance tests. Many of the wins are small (a degree or two on DL3DV/NAVI), and the no-overlap ScanNet split in the supplement shows rotation error actually slightly worse (83.94 vs 83.48) with Dream Machine. The MASt3R results are also mixed on the center-facing datasets, which the text underplays. Third, the self-consistency score is a weak proxy for correctness—the oracle gap makes that clear—and the metric was adjusted after seeing behavior; the paper says this transparently, but it means the headline effect should be read as preliminary.\n\nWho this is for: people working on pose estimation, wide-baseline matching, or using generative models as scene priors. It deserves a serious referee; I would ask for the control, error bars, and a more careful discussion of the mixed cases, then accept with revisions.","headline":"A worthwhile first demonstration that video-model interpolation can feed pose estimators, but the causal claim that the world prior is doing the work needs a control and error bars.","tokens_in":22911,"tokens_out":2417,"would_cite":true,"duration_ms":21205,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Video hallucinations cut pose error on low-overlap image pairs.","keywords":["generative video models","pose estimation","video interpolation","DUSt3R","self-consistency score","low-overlap image pairs","world prior","camera pose"],"falsifier":"Run InterPose on a held-out set of low-overlap pairs and record both $D_{\\mathrm{total}}$ and actual pose error for each generated video; a positive correlation between score and error would confirm the mechanism, while any substantial fraction of low-score, high-error videos (or cases where the lowest-score video is wrong while a higher-score one is right) would falsify the claim that self-consistency drives the improvement.","tokens_in":21955,"feed_emoji":"🎥","tokens_out":6448,"duration_ms":52443,"temperature":0.7,"pith_summary":"Can a generative video model help estimate camera pose? This paper claims that yes, it can, when the two input images barely overlap. The idea is to let a video model hallucinate a visual bridge between the images, then feed those hallucinated frames to an off-the-shelf pose estimator such as DUSt3R, which gains enough context to recover the relative pose more accurately. Because generated videos are sometimes geometrically inconsistent, the paper selects among multiple samples using a self-consistency score based on pose estimates from random subsets of frames. Across four datasets spanning indoor, outdoor, and object-centric scenes, the selected predictions consistently beat DUSt3R run on the original image pair alone, for all three video models tested. The broader claim is that web-scale video data contains spatial priors that 3D training data does not, and those priors can be tapped without changing the pose model.","feed_headline":"Video hallucinations cut pose error on low-overlap image pairs","feed_subtitle":"Feeding interpolated frames to DUSt3R beats using the original pair alone on all four test datasets.","key_machinery":"The load-bearing mechanism is the medoid-based self-consistency score. From each generated interpolation, the paper samples $m$ subsets of $k=5$ frames (always containing both input images), runs DUSt3R on each subset to get relative pose estimates $\\hat T^{(i)}$, and computes the medoid distance $D_{\\mathrm{med}} = \\min_i \\frac{1}{m-1}\\sum_{j\\ne i} \\operatorname{dist}(\\hat T^{(i)}, \\hat T^{(j)})$ in pose space. To avoid degenerate videos that are consistently wrong, the selection score adds a bias term anchoring the medoid to the pose estimated from the original pair alone: $D_{\\mathrm{total}} = D_{\\mathrm{med}} + \\operatorname{dist}(\\hat T_{\\mathrm{med}}, f_{\\mathrm{pose}}(\\{I_A,I_B\\}))$. The video with the smallest $D_{\\mathrm{total}}$ is chosen and its medoid pose is returned; this score is what turns unreliable video hallucinations into a usable prior.","core_discovery":"InterPose claims the first demonstration that a generative video model can improve relative camera pose estimation by acting as a world prior: given two images with small or no overlap, it samples several videos that interpolate between them, feeds frames from each video together with the original pair into DUSt3R, and selects the most self-consistent result. On the paper's benchmark of challenging pairs from Cambridge Landmarks, ScanNet, DL3DV-10K, and NAVI, the selected prediction consistently reduces rotation and translation error relative to DUSt3R on the original pair alone, for all three video models tested. The strongest numbers come on outward-facing indoor and outdoor scenes: Cambridge mean rotation error drops from 13.28° to 10.78°, and ScanNet mean rotation and translation errors drop from (21.31°, 24.72°) to (17.65°, 15.88°). An oracle that picks the best pose among all generated predictions is far better, which the paper takes as evidence that the selection heuristic, not the video prior, is the current bottleneck.","pith_inferences":["The paper leaves implicit that the large gap between medoid selection and the oracle suggests a learned or geometry-aware verifier for generated videos could roughly halve pose errors on these benchmarks.","The same mechanism could transfer to other sparse-view 3D tasks such as depth estimation or novel view synthesis, since the paper only tests pose but the core idea is generic context bridging.","A testable extension would be to use the generated interpolations as training-data augmentation for pose estimators, recovering pseudo-labels through the same pipeline and adding them to image-pair training sets.","Because the paper notes video models are biased toward certain camera motions, explicitly debiasing generation direction could remove part of the remaining error without changing the pose estimator."],"forward_implications":["Pose estimators that fail on low-overlap pairs can be improved without retraining by feeding them frames hallucinated by a video model.","The improvement persists across three different video models, so the effect is not tied to one generator's artifacts.","The medoid self-consistency score prevents the generated frames from degrading results, since simply averaging all predictions is worse than ignoring the video model.","On center-facing datasets where DUSt3R already performs well, the generated frames still give small gains and do not hurt on the main benchmark.","The large oracle improvements bound the headroom: better selection of generated videos would yield substantially larger pose accuracy gains."],"supporting_citations":[{"why":"DUSt3R is the state-of-the-art pose estimator whose predictions the paper feeds generated frames into and compares against.","marker":"[59]"},{"why":"DynamiCrafter is one of the three video interpolation models used to hallucinate intermediate frames.","marker":"[61]"},{"why":"Runway Gen-3 Alpha Turbo is the commercial video model whose interpolations are tested.","marker":"[40]"},{"why":"Luma Dream Machine is the third video model tested and the one used in the ablations.","marker":"[34]"},{"why":"Cambridge Landmarks supplies the outdoor image pairs with 50-65 degree yaw changes used to evaluate rotation accuracy.","marker":"[25]"},{"why":"ScanNet supplies the indoor image pairs used for the main low-overlap evaluation and the yaw-range ablations.","marker":"[11]"},{"why":"DL3DV-10K supplies the outdoor center-facing image pairs used to test performance when DUSt3R is already strong.","marker":"[31]"},{"why":"NAVI supplies the object-centric image pairs used to test the method on category-agnostic objects.","marker":"[21]"}],"fun_headline_variants":["Video-generated intermediate frames cut pose error on low-overlap pairs","Hallucinating frames between views improves pose estimation","Video models as world priors: pose from disjoint images","InterPose: video hallucination beats DUSt3R on non-overlap","Generative video models fill image gaps to sharpen relative pose"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's gains depend on the medoid self-consistency score reliably flagging videos that yield correct poses; the paper's own oracle numbers show that a perfect selector would outperform the medoid selector by a wide margin, so if consistency across sampled subsets stops correlating with correctness, generated frames can mislead DUSt3R and degrade pose estimates.","fun_headline_variants_meta":{"raw":{"variants":["Video-generated intermediate frames cut pose error on low-overlap pairs","Hallucinating frames between views improves pose estimation","Video models as world priors: pose from disjoint images","InterPose: video hallucination beats DUSt3R on non-overlap","Generative video models fill image gaps to sharpen relative pose"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001156,"raw_usage":{"total_tokens":4802,"prompt_tokens":969,"completion_tokens":3833,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":3748}},"tokens_in":585,"tokens_out":3833,"duration_ms":24763,"temperature":1.0,"reasoning_tokens":3748,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:44:14.487602+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run InterPose on a held-out set of low-overlap pairs and record both $D_{\\mathrm{total}}$ and actual pose error for each generated video; a positive correlation between score and error would confirm the mechanism, while any substantial fraction of low-score, high-error videos (or cases where the lowest-score video is wrong while a higher-score one is right) would falsify the claim that self-consistency drives the improvement.","supporting_citations":[{"cited_title":"Dust3r: Geometric 3d vi- sion made easy","cited_arxiv_id":null,"evidence_quote":"DUSt3R is the state-of-the-art pose estimator whose predictions the paper feeds generated frames into and compares against."},{"cited_title":"Tools for human imagination, 2024","cited_arxiv_id":null,"evidence_quote":"Runway Gen-3 Alpha Turbo is the commercial video model whose interpolations are tested."},{"cited_title":"Luma dream machine, 2024","cited_arxiv_id":null,"evidence_quote":"Luma Dream Machine is the third video model tested and the one used in the ablations."},{"cited_title":"Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner","cited_arxiv_id":null,"evidence_quote":"ScanNet supplies the indoor image pairs used for the main low-overlap evaluation and the yaw-range ablations."},{"cited_title":"Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision","cited_arxiv_id":null,"evidence_quote":"DL3DV-10K supplies the outdoor center-facing image pairs used to test performance when DUSt3R is already strong."},{"cited_title":"Navi: Category-agnostic image collections with high-quality 3d shape and pose annotations","cited_arxiv_id":null,"evidence_quote":"NAVI supplies the object-centric image pairs used to test the method on category-agnostic objects."}],"review_version":1}