Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Can Generative Video Models Help Pose Estimation?

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Video hallucinations cut pose error on low-overlap image pairs.

desk verdict A worthwhile first demonstration that video-model interpolation can feed pose estimators, but the causal claim that the world prior is doing the work needs a control and error bars. read the letter →

arxiv 2412.16155 v1 pith:IZ54LRCL submitted 2024-12-20 cs.CV

Ruojin Cai , Jason Y. Zhang , Philipp Henzler , Zhengqi Li , Noah Snavely , Ricardo Martin-Brualla This is my paper · ORCID
classification cs.CV
keywords generativevideomodelsposeestimationinterpolationDUSt3Rself-consistencyscorelow-overlapimagepairsworldpriorcamera
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Can a generative video model help estimate camera pose? This paper claims that yes, it can, when the two input images barely overlap. The idea is to let a video model hallucinate a visual bridge between the images, then feed those hallucinated frames to an off-the-shelf pose estimator such as DUSt3R, which gains enough context to recover the relative pose more accurately. Because generated videos are sometimes geometrically inconsistent, the paper selects among multiple samples using a self-consistency score based on pose estimates from random subsets of frames. Across four datasets spanning indoor, outdoor, and object-centric scenes, the selected predictions consistently beat DUSt3R run on the original image pair alone, for all three video models tested. The broader claim is that web-scale video data contains spatial priors that 3D training data does not, and those priors can be tapped without changing the pose model.

What carries the argument

The load-bearing mechanism is the medoid-based self-consistency score. From each generated interpolation, the paper samples $m$ subsets of $k=5$ frames (always containing both input images), runs DUSt3R on each subset to get relative pose estimates $\hat T^{(i)}$, and computes the medoid distance $D_{\mathrm{med}} = \min_i \frac{1}{m-1}\sum_{j\ne i} \operatorname{dist}(\hat T^{(i)}, \hat T^{(j)})$ in pose space. To avoid degenerate videos that are consistently wrong, the selection score adds a bias term anchoring the medoid to the pose estimated from the original pair alone: $D_{\mathrm{total}} = D_{\mathrm{med}} + \operatorname{dist}(\hat T_{\mathrm{med}}, f_{\mathrm{pose}}(\{I_A,I_B\}))$. The video with the smallest $D_{\mathrm{total}}$ is chosen and its medoid pose is returned; this score is what turns unreliable video hallucinations into a usable prior.

What would settle it

Run InterPose on a held-out set of low-overlap pairs and record both $D_{\mathrm{total}}$ and actual pose error for each generated video; a positive correlation between score and error would confirm the mechanism, while any substantial fraction of low-score, high-error videos (or cases where the lowest-score video is wrong while a higher-score one is right) would falsify the claim that self-consistency drives the improvement.

Watch

Extended reading notes

Core claim

InterPose claims the first demonstration that a generative video model can improve relative camera pose estimation by acting as a world prior: given two images with small or no overlap, it samples several videos that interpolate between them, feeds frames from each video together with the original pair into DUSt3R, and selects the most self-consistent result. On the paper's benchmark of challenging pairs from Cambridge Landmarks, ScanNet, DL3DV-10K, and NAVI, the selected prediction consistently reduces rotation and translation error relative to DUSt3R on the original pair alone, for all three video models tested. The strongest numbers come on outward-facing indoor and outdoor scenes: Cambridge mean rotation error drops from 13.28° to 10.78°, and ScanNet mean rotation and translation errors drop from (21.31°, 24.72°) to (17.65°, 15.88°). An oracle that picks the best pose among all generated predictions is far better, which the paper takes as evidence that the selection heuristic, not the video prior, is the current bottleneck.

Load-bearing premise

The method's gains depend on the medoid self-consistency score reliably flagging videos that yield correct poses; the paper's own oracle numbers show that a perfect selector would outperform the medoid selector by a wide margin, so if consistency across sampled subsets stops correlating with correctness, generated frames can mislead DUSt3R and degrade pose estimates.

Editorial extensions

If this is right

  • Pose estimators that fail on low-overlap pairs can be improved without retraining by feeding them frames hallucinated by a video model.
  • The improvement persists across three different video models, so the effect is not tied to one generator's artifacts.
  • The medoid self-consistency score prevents the generated frames from degrading results, since simply averaging all predictions is worse than ignoring the video model.
  • On center-facing datasets where DUSt3R already performs well, the generated frames still give small gains and do not hurt on the main benchmark.
  • The large oracle improvements bound the headroom: better selection of generated videos would yield substantially larger pose accuracy gains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the large gap between medoid selection and the oracle suggests a learned or geometry-aware verifier for generated videos could roughly halve pose errors on these benchmarks.
  • The same mechanism could transfer to other sparse-view 3D tasks such as depth estimation or novel view synthesis, since the paper only tests pose but the core idea is generic context bridging.
  • A testable extension would be to use the generated interpolations as training-data augmentation for pose estimators, recovering pseudo-labels through the same pipeline and adding them to image-pair training sets.
  • Because the paper notes video models are biased toward certain camera motions, explicitly debiasing generation direction could remove part of the remaining error without changing the pose estimator.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes InterPose, a method that uses off-the-shelf generative video models to synthesize interpolated frames between two input images with low or no overlap, feeds the original pair plus generated frames into a multi-view pose estimator (DUSt3R, and MASt3R in the supplement), and selects among multiple generated videos using a medoid-based self-consistency score (Eqs. 8-9). The experiments cover four datasets (Cambridge Landmarks, ScanNet, DL3DV-10K, NA VI), three video generators (DynamiCrafter, Runway, Dream Machine), and report mean rotation/translation errors as well as accuracy/AUC metrics. The central claim is that generated video frames act as a world prior and consistently improve DUSt3R on low-overlap pairs; the paper also reports an Oracle upper bound and ablations over distance metrics and number of frames.

Significance. If the central claim held, this would be a valuable demonstration that large generative video models trained on web-scale data can provide useful geometric priors for 3D vision. The paper has genuine strengths: it evaluates three different video models on four datasets, includes an oracle analysis that quantifies headroom, tests a second pose estimator (MASt3R), and ablates the number of frames and the yaw range. However, the claim as stated is stronger than the evidence: several gains are small, no significance tests are reported, and the selection metric is partly anchored to the baseline. The manuscript's own supplementary results (Table 7) show a case with slightly worse rotation error on non-overlapping ScanNet pairs, and the conclusion concedes that all generated videos can be low quality. These caveats make the contribution interesting but in need of revision.

major comments (4)
  1. [Sec. 4.2, Tables 1-2] The central claim that a generative video model improves pose estimation 'by acting as a world prior' is not isolated by any control condition in which non-generative additional frames are supplied to DUSt3R under the same protocol. Since DUSt3R is a multi-view estimator, feeding it extra images changes its optimization even when the images carry no new geometric information; Table 4 (D_bias-only row for Dream Machine on Cambridge, MRE 11.25 vs. 13.28) shows that selecting predictions close to the pair-only estimate already yields a large share of the reported gain. A control with duplicated, blended, or randomly warped input images under the same 4-video, 11-subset medoid protocol is needed to attribute the improvement to the learned content of the generated frames.
  2. [Sec. 3.2, Eq. (9), Table 4] The selection score D_total includes dist(T_med, fpose({IA,IB})), which anchors the chosen video's pose toward the baseline DUSt3R estimate. This makes the method partially a 'stay near the baseline' heuristic rather than a pure measure of the generated video's consistency. The Oracle gap in Tables 1-2 (e.g., Cambridge MRE 3.65 vs. 10.78 for the reported Medoid result) further shows that consistency across subsets is a weak proxy for correctness, so the method's practical gains are contingent on this proxy holding. The paper should quantify how often the selected video differs from the baseline-selected video and report results with the bias term removed or reweighted on a validation split.
  3. [Sec. 4.5, Tables 1-2] No error bars, confidence intervals, or significance tests are reported for any of the main comparisons. Several improvements are small (e.g., DynamiCrafter MRE on Cambridge: 12.70 vs. 13.28; several accuracy differences under 2 points), and with 290-300 pairs per dataset, paired bootstrap or per-pair significance testing would be needed to support the repeated claim of 'consistent improvements.' Without these, the universal claim is not established even though the larger reductions, such as ScanNet MTE from 24.72 to 15.88, are themselves notable.
  4. [Sec. 3.2 and Supp. Sec. D.1, Table 4; Supp. Table 7] The metric D_total appears to have been designed after observing that D_med alone fails on Cambridge with Dream Machine (MRE 19.37 vs. 11.96 for D_total), and the paper states 'we found it helpful to bias the metric.' Because the same four test sets were used to motivate the metric, a validation-based protocol or an independent test set is needed to rule out overfitting to the benchmark. In addition, the supplementary results for non-overlapping ScanNet pairs (Table 7: MRE 83.94 for Ours vs. 83.48 for DUSt3R) and the main paper's admission that all generated videos can be low quality show that the 'consistent improvements' claim does not extend to the full yaw range; the abstract and contributions should be qualified accordingly.
minor comments (5)
  1. [Supp. Sec. B] The sentence 'For each pair, we the settings described in the main paper' is incomplete; it should read 'we use the settings described in the main paper.'
  2. [Supp. Tables 3-4] Table 4's caption contains the typo 'Abltion' instead of 'Ablation', and Table 3 uses 'Navi' inconsistently with 'NA VI' in the main text.
  3. [Sec. 3.2, Eq. (8)] Clarify that the medoid is the sample minimizing the average distance to the other samples and that D_med is the value of that minimized average; the current notation could be misread as a separate quantity from T_med.
  4. [Abstract and Supp. Sec. A] The project page link appears without the URL scheme ('Inter-Pose.github.'); ensure a complete, working hyperlink is provided.
  5. [Sec. 4.3] The three video models produce very different frame counts (16 for DynamiCrafter, 112 for Runway, 114 for Dream Machine); a note on how these counts interact with the k=5 frame-subset sampling would improve the comparability of the three generators.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an empirical, externally benchmarked comparison with a disclosed and non-equivalent selection heuristic.

full rationale

The central claim — that frames hallucinated by pretrained video models improve DUSt3R pose estimates on low-overlap pairs — is tested against held-out ground-truth poses (Cambridge, ScanNet, DL3DV-10K, NAVI) and against a genuine black-box baseline (DUSt3R on the input pair alone). Generated frames are additional inputs to DUSt3R; they are not used to define the error metric. The selection score Dtotal = Dmed + dist(Tmed, fpose({IA,IB})) (Eqs. 8-9) does bias selection toward the pair-only DUSt3R estimate, but this is a disclosed regularizer, not a reduction of the output to the baseline: the output is the medoid pose from subsets that include generated frames, and the paper's own ablation (Table 4) isolates Dmed, Dbias, and Dtotal, showing that Dmed-only still improves most settings and that the bias term mainly protects against confidently wrong videos. The Oracle baseline is an upper bound, not a claimed prediction. Self-citations (e.g., [5] for the Cambridge pair subset) are data-source or related-work citations, not load-bearing theoretical premises. The absence of a non-generative extra-frames control is a limitation in attributing the gain specifically to the video model's learned prior, but that is a validity concern, not a circular derivation.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method introduces no new entities, but the pipeline depends on several hand-picked parameters (n, k, m, prompt templates, additive bias weight) and on assumptions about video model plausibility and DUSt3R behavior. The self-consistency proxy is the most fragile assumption, as the oracle gap demonstrates.

free parameters (5)
  • number of generated videos n = 4
    Chosen for cost/benefit; not justified in the paper.
  • frames per subset k = 5
    Ablation shows 5 works best on ScanNet; chosen after evaluation.
  • frame subsets m = 11
    10 random + 1 uniform; arbitrary, no sensitivity analysis.
  • bias term weighting in Dtotal = 1.0 (additive)
    Dtotal = Dmed + dist(Tmed, fpose(pair)); weight fixed at 1 without ablation.
  • prompt templates = two GPT-4o prompts
    Specific prompts affect output; only templates given, exact GPT-4o outputs not released.
assumptions (3)
  • domain assumption Video models produce geometrically plausible interpolations between the two input images
    Section 3.2 and Fig. 2 acknowledge this is often false; the method relies on filtering, but the assumption that at least one of n=4 videos is plausible underlies the approach.
  • domain assumption DUSt3R can extract accurate relative pose from a set of 5 images including generated frames
    Section 3.1 assumes a black-box pose estimator that works on N images; the paper does not analyze how DUSt3R's optimization behaves with synthetic frames.
  • ad hoc to paper Medoid distance is a valid proxy for pose correctness
    Section 3.2, Eq. 8. The Oracle results show this proxy is far from optimal, making it an unvalidated assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can Generative Video Models Help Pose Estimation?." pith.science (2026). https://pith.science/paper/IZ54LRCL

@misc{pith2026241216155,
  author       = {Pith},
  title        = {Pith review of: Can Generative Video Models Help Pose Estimation?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IZ54LRCL}},
  note         = {Machine review of arXiv:2412.16155}
}
read the original abstract

Pairwise pose estimation from images with little or no overlap is an open challenge in computer vision. Existing methods, even those trained on large-scale datasets, struggle in these scenarios due to the lack of identifiable correspondences or visual overlap. Inspired by the human ability to infer spatial relationships from diverse scenes, we propose a novel approach, InterPose, that leverages the rich priors encoded within pre-trained generative video models. We propose to use a video model to hallucinate intermediate frames between two input images, effectively creating a dense, visual transition, which significantly simplifies the problem of pose estimation. Since current video models can still produce implausible motion or inconsistent geometry, we introduce a self-consistency score that evaluates the consistency of pose predictions from sampled videos. We demonstrate that our approach generalizes among three state-of-the-art video models and show consistent improvements over the state-of-the-art DUSt3R on four diverse datasets encompassing indoor, outdoor, and object-centric scenes. Our findings suggest a promising avenue for improving pose estimation models by leveraging large generative models trained on vast amounts of video data, which is more readily available than 3D data. See our project page for results: https://inter-pose.github.io/.

Figures

Figures reproduced from arXiv: 2412.16155 by the authors.

Figure 1
Figure 1. Improving pose estimation by interpolating frames using a video model. Given two images of a scene with almost no overlap, we aim to recover their relative camera pose. Without being able to rely on visual correspondences, existing methods struggle in this setting (left). We propose to use an off-the-shelf video generation model to interpolate a video connecting the two images. Augmented with the frames generated by… view at source ↗
Figure 2
Figure 2. Common failure modes of video models. We show some failure modes of interpolating between two images. In the first row, a microwave suddenly appears over the sink. In the second and third row, the video model morphs and blends images without consistent changes to the under￾lying scene geometry. In the fourth row, the object’s appearance changes in an unrealistic way. tributions [10], energy-based models [29, 62], or… view at source ↗
Figure 3
Figure 3. Qualitative comparison of the three video models: DynamiCrafter (DC), Runway (RW), and Dream Machine (DM), using the same text prompt [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Self-consistency scores for poses derived from generated videos. (a) From a pair of input frames A and B, we generate several candidate videos from a given video interpolation method. For each video, we sample subsets of frames and compute a relative pose from A to B f…
Figure 5
Figure 5. Figure 5: Qualitative results of pose estimation from DUSt3R taking only image pair as input and taking additional video frames. We show the input image pair in the first two columns, and the DUSt3R prediction using the image pair alone in the third column. The 3D reconstruction…
Figure 6
Figure 6. Figure 6: Left-to-right bias. We observed that video models exhibit a tendency to generate similar camera motions (e.g., both left-to-right pans) regardless of the intended direction of interpolation (i.e., transitioning from image A to image B or from image B to image A). This …
Figure 7
Figure 7. Figure 7: Camera Pose Estimation Performance vs. Yaw Angle Change on the ScanNet and DL3DV-10K Datasets. Comparison of Mean Rotation Error (MRE), Mean Translation Error (MTE), and Area Under Curve at 30° (AUC30◦ ) across different yaw angle change intervals (0°, 20°, 40°, 60°, e…
Figure 8
Figure 8. Figure 8: Failure examples of MASt3R. We show instances where MASt3R fails to accurately predict poses on non-overlapping pairs from the Cambridge Landmarks (top row) and ScanNet (bottom row) datasets. MASt3R relies on feature matching for pose refinement, which is insufficient …

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Emergent Temporal Correspondences from Video Diffusion Transformers

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Video diffusion transformers encode temporal correspondences primarily in query-key similarities of a few specific attention layers, which can be extracted for zero-shot point tracking and used for training-free motio...

Reference graph

Works this paper leans on

63 extracted references · 31 canonical work pages · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Frozen in time: A joint video and image encoder for end-to-end retrieval

    Max Bain, Arsha Nagrani, G ¨ul Varol, and Andrew Zisser- man. Frozen in time: A joint video and image encoder for end-to-end retrieval. In ICCV, 2021. 7

    work page 2021

  3. [3]

    Lumiere: A space- time diffusion model for video generation

    Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Her- rmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, et al. Lumiere: A space- time diffusion model for video generation. arXiv preprint arXiv:2401.12945, 2024. 2

    arXiv 2024

  4. [4]

    Surf: Speeded up robust features

    Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. Surf: Speeded up robust features. In ECCV, 2006. 2

    2006

  5. [5]

    Extreme rotation estimation in the wild

    Hana Bezalel, Dotan Ankri, Ruojin Cai, and Hadar Averbuch-Elor. Extreme rotation estimation in the wild. arXiv preprint arXiv:2411.07096, 2024. 2, 5

    arXiv 2024

  6. [6]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In CVPR, 2023. 2

    2023

  7. [7]

    G. Bradski. The OpenCV Library. Dr. Dobb’s Journal of Software Tools, 2000. 5

    2000

  8. [8]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators

Show all 63 references
  1. [9]

    Extreme rotation estimation using dense cor- relation volumes

    Ruojin Cai, Bharath Hariharan, Noah Snavely, and Hadar Averbuch-Elor. Extreme rotation estimation using dense cor- relation volumes. In CVPR, 2021. 2

    2021

  2. [10]

    Wide- baseline relative camera pose estimation with directional learning

    Kefan Chen, Noah Snavely, and Ameesh Makadia. Wide- baseline relative camera pose estimation with directional learning. In CVPR, 2021. 3

    2021

  3. [11]

    Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner

    Angela Dai, Angel X. Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 5, 11

    2017

  4. [12]

    Stochastic video generation with a learned prior

    Emily Denton and Rob Fergus. Stochastic video generation with a learned prior. In ICML, 2018. 2

    2018

  5. [13]

    Superpoint: Self-supervised interest point detection and description

    Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabi- novich. Superpoint: Self-supervised interest point detection and description. In CVPRW, 2018. 2

    2018

  6. [14]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981

    Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981. 2, 5

    1981

  7. [15]

    Photorealistic video generation with diffusion models

    Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Fei-Fei Li, Irfan Essa, Lu Jiang, and Jos ´e Lezama. Photorealistic video generation with diffusion models. In ECCV, 2024. 2

    2024

  8. [16]

    In defense of the eight-point algorithm

    Richard I Hartley. In defense of the eight-point algorithm. IEEE TPAMI, 19(6):580–593, 1997. 1, 2

    1997

  9. [17]

    Denoising diffu- sion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models. NeurIPS, 2020. 2

    2020

  10. [18]

    Imagen video: High definition video generation with diffusion mod- els

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els. arXiv preprint arXiv:2210.02303, 2022. 2

    arXiv

  11. [19]

    Video dif- fusion models

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models. NeurIPS, 2022. 2

    2022

  12. [20]

    Learning to decompose and disen- tangle representations for video prediction

    Jun-Ting Hsieh, Bingbin Liu, De-An Huang, Li F Fei-Fei, and Juan Carlos Niebles. Learning to decompose and disen- tangle representations for video prediction. NeurIPS, 2018. 2

    2018

  13. [21]

    Navi: Category-agnostic image collections with high-quality 3d shape and pose annotations

    Varun Jampani, Kevis-Kokitsi Maninis, Andreas Engelhardt, Arjun Karpur, Karen Truong, Kyle Sargent, Stefan Popov, Andr´e Araujo, Ricardo Martin Brualla, Kaushal Patel, et al. Navi: Category-agnostic image collections with high-quality 3d shape and pose annotations. NeurIPS, 2023. 5

    2023

  14. [22]

    Omniglue: Generalizable feature match- ing with foundation model guidance

    Hanwen Jiang, Arjun Karpur, Bingyi Cao, Qixing Huang, and Andre Araujo. Omniglue: Generalizable feature match- ing with foundation model guidance. In CVPR, 2024. 2

    2024

  15. [23]

    Image matching across wide baselines: From paper to practice

    Yuhe Jin, Dmytro Mishkin, Anastasiia Mishchuk, Jiri Matas, Pascal Fua, Kwang Moo Yi, and Eduard Trulls. Image matching across wide baselines: From paper to practice. IJCV, 2021. 7

    2021

  16. [24]

    Lfm-3d: Learnable feature matching across wide baselines using 3d signals

    Arjun Karpur, Guilherme Perrotta, Ricardo Martin-Brualla, Howard Zhou, and Andr´e Araujo. Lfm-3d: Learnable feature matching across wide baselines using 3d signals. In 3DV, pages 11–20. IEEE, 2024. 2

    2024

  17. [25]

    Posenet: A convolutional network for real-time 6-dof camera relocalization

    Alex Kendall, Matthew Grimes, and Roberto Cipolla. Posenet: A convolutional network for real-time 6-dof camera relocalization. In ICCV, 2015. 5

    2015

  18. [26]

    Kling ai, 2024

    Kuaishou. Kling ai, 2024. https://klingai.com/ [Accessed: (September 2024)]. 2

    2024

  19. [27]

    Stochastic adversarial video prediction

    Alex X Lee, Richard Zhang, Frederik Ebert, Pieter Abbeel, Chelsea Finn, and Sergey Levine. Stochastic adversarial video prediction. arXiv preprint arXiv:1804.01523 , 2018. 2

    2018 arXiv

  20. [28]

    Ground- ing image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and J´erˆome Revaud. Ground- ing image matching in 3d with mast3r. In ECCV, pages 71–

  21. [29]

    Relpose++: Recovering 6d poses from sparse-view observations

    Amy Lin, Jason Y Zhang, Deva Ramanan, and Shubham Tul- siani. Relpose++: Recovering 6d poses from sparse-view observations. arXiv preprint arXiv:2305.04926, 2023. 3

    2023 arXiv

  22. [30]

    Lightglue: Local feature matching at light speed

    Philipp Lindenberger, Paul-Edouard Sarlin, and Marc Polle- feys. Lightglue: Local feature matching at light speed. In ICCV, 2023. 2

    2023

  23. [31]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. In CVPR, 2024. 5, 11

    2024

  24. [32]

    A computer algorithm for reconstructing a scene from two projections

    H Christopher Longuet-Higgins. A computer algorithm for reconstructing a scene from two projections. Nature, 293 (5828):133–135, 1981. 2 9

    1981

  25. [33]

    Distinctive image features from scale- invariant keypoints

    David G Lowe. Distinctive image features from scale- invariant keypoints. IJCV, 2004. 1, 2, 5

    2004

  26. [34]

    Luma dream machine, 2024

    LumaAI. Luma dream machine, 2024. https : / / lumalabs.ai/dream-machine [Accessed: (Septem- ber 2024)]. 2, 3, 7

    2024

  27. [35]

    Fast approximate nearest neighbors with automatic algorithm configuration

    Marius Muja and David G Lowe. Fast approximate nearest neighbors with automatic algorithm configuration. VISAPP (1), 2(331-340):2, 2009. 2

    2009

  28. [36]

    An efficient solution to the five-point relative pose problem

    David Nist ´er. An efficient solution to the five-point relative pose problem. IEEE TPAMI, 26(6):756–770, 2004. 2

    2004

  29. [37]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 2

    2022 arXiv

  30. [38]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, 2021. 2

    2021

  31. [39]

    Orb: An efficient alternative to sift or surf

    Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. Orb: An efficient alternative to sift or surf. InICCV,

  32. [40]

    Tools for human imagination, 2024

    RunwayML. Tools for human imagination, 2024. https: / / runwayml . com / product[Accessed: (November 2024)]. 2, 3, 7

    2024

  33. [41]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 2022. 2

    2022

  34. [42]

    Tempo- ral generative adversarial nets with singular value clipping

    Masaki Saito, Eiichi Matsumoto, and Shunta Saito. Tempo- ral generative adversarial nets with singular value clipping. In ICCV, 2017. 2

    2017

  35. [43]

    Superglue: Learning feature matching with graph neural networks

    Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. Superglue: Learning feature matching with graph neural networks. In CVPR, 2020. 2

    2020

  36. [44]

    Structure-from-motion revisited

    Johannes Lutz Sch ¨onberger and Jan-Michael Frahm. Structure-from-motion revisited. In CVPR, 2016. 3

    2016

  37. [45]

    Pixelwise view selection for un- structured multi-view stereo

    Johannes Lutz Sch ¨onberger, Enliang Zheng, Marc Pollefeys, and Jan-Michael Frahm. Pixelwise view selection for un- structured multi-view stereo. In ECCV, 2016. 3

    2016

  38. [46]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  39. [47]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In ICML, 2015. 2

    2015

  40. [48]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. 2

    2010 arXiv

  41. [49]

    LoFTR: Detector-free local feature matching with transformers

    Jiaming Sun, Zehong Shen, Yuang Wang, Hujun Bao, and Xiaowei Zhou. LoFTR: Detector-free local feature matching with transformers. CVPR, 2021. 2, 5

    2021

  42. [50]

    Quadtree attention for vision transformers

    Shitao Tang, Jiahui Zhang, Siyu Zhu, and Ping Tan. Quadtree attention for vision transformers. ICLR, 2022

    2022

  43. [51]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In ECCV, 2020. 2

    2020

  44. [52]

    Mocogan: Decomposing motion and content for video generation

    Sergey Tulyakov, Ming-Yu Liu, Xiaodong Yang, and Jan Kautz. Mocogan: Decomposing motion and content for video generation. arxiv. arXiv preprint arXiv:1707.04993 ,

    arXiv

  45. [53]

    Disk: Learning local features with policy gradient

    Michał Tyszkiewicz, Pascal Fua, and Eduard Trulls. Disk: Learning local features with policy gradient. NeurIPS, 2020. 2

    2020

  46. [54]

    Hier- archical long-term video prediction without supervision

    Ruben Villegas, Dumitru Erhan, Honglak Lee, et al. Hier- archical long-term video prediction without supervision. In ICML, 2018. 2

    2018

  47. [55]

    Phenaki: Variable length video generation from open domain textual descriptions

    Ruben Villegas, Mohammad Babaeizadeh, Pieter-Jan Kin- dermans, Hernan Moraldo, Han Zhang, Mohammad Taghi Saffar, Santiago Castro, Julius Kunze, and Dumitru Erhan. Phenaki: Variable length video generation from open domain textual descriptions. In ICLR, 2022. 2

    2022

  48. [56]

    Generating videos with scene dynamics

    Carl V ondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics. NeurIPS, 2016. 2

    2016

  49. [57]

    Posediffusion: Solving pose estimation via diffusion-aided bundle adjustment

    Jianyuan Wang, Christian Rupprecht, and David Novotny. Posediffusion: Solving pose estimation via diffusion-aided bundle adjustment. In ICCV, 2023. 3, 7

    2023

  50. [58]

    Pf-lrm: Pose-free large reconstruction model for joint pose and shape prediction

    Peng Wang, Hao Tan, Sai Bi, Yinghao Xu, Fujun Luan, Kalyan Sunkavalli, Wenping Wang, Zexiang Xu, and Kai Zhang. Pf-lrm: Pose-free large reconstruction model for joint pose and shape prediction. arXiv preprint arXiv:2311.12024, 2023. 3

    2023 arXiv

  51. [59]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. In CVPR, 2024. 1, 2, 3, 5

    2024

  52. [60]

    Croco v2: Improved cross-view completion pre- training for stereo matching and optical flow

    Philippe Weinzaepfel, Thomas Lucas, Vincent Leroy, Yohann Cabon, Vaibhav Arora, Romain Br ´egier, Gabriela Csurka, Leonid Antsfeld, Boris Chidlovskii, and J ´erˆome Revaud. Croco v2: Improved cross-view completion pre- training for stereo matching and optical flow. InICCV, 2023. 3

    2023

  53. [61]

    Dynamicrafter: Animating open-domain images with video diffusion priors

    Jinbo Xing, Menghan Xia, Yong Zhang, Haoxin Chen, Wangbo Yu, Hanyuan Liu, Gongye Liu, Xintao Wang, Ying Shan, and Tien-Tsin Wong. Dynamicrafter: Animating open-domain images with video diffusion priors. In ECCV,

  54. [62]

    Rel- pose: Predicting probabilistic relative rotation for single ob- jects in the wild

    Jason Y Zhang, Deva Ramanan, and Shubham Tulsiani. Rel- pose: Predicting probabilistic relative rotation for single ob- jects in the wild. In ECCV, 2022. 3

    2022

  55. [63]

    Cameras as rays: Pose estimation via ray diffusion

    Jason Y Zhang, Amy Lin, Moneish Kumar, Tzu-Hsuan Yang, Deva Ramanan, and Shubham Tulsiani. Cameras as rays: Pose estimation via ray diffusion. arXiv preprint arXiv:2402.14817, 2024. 3 10 Can Generative Video Models Help Pose Estimation? Supplementary Material A. Qualitative Re...

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.