Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation

T0 review · 3 major / 7 minor · reviewed 2026-07-08 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Video diffusion model tracks 6-DoF motion without 3D models or depth

desk verdict Genuinely novel reframing of 6-DoF tracking as video-to-video translation; SOTA on benchmarks but high variance and short evaluation windows need scrutiny read the letter →

arxiv 2607.06555 v1 pith:ICLJZT72 submitted 2026-07-07 cs.CV

Ruihang Zhang , Felix Taubner , Pooja Ravi , Kiriakos N. Kutulakos , David B. Lindell This is my paper · ORCID
classification cs.CV
keywords 6-DoFposetrackingvideodiffusionmodelvideo-to-videotranslationmonocularRGBestimationproxyPnPsolversyntheticfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ProxyPose recasts six-degree-of-freedom pose tracking from monocular video as a video-to-video translation problem. Given a video and a single marked pixel, a fine-tuned video diffusion model generates a synthetic proxy video showing a colored cube undergoing the same local rigid-body motion as the surface at that pixel. Because the cube's geometry is fully known, recovering the full 6-DoF trajectory reduces to classical pose estimation with standard solvers. The approach requires no 3D models, depth maps, object masks, or task-specific feature extractors—only RGB video and a focal length estimate. The key claim is that large video generation models implicitly encode sufficient 3D motion understanding to translate real-world motion—including non-rigid, transparent, reflective, or occluded surfaces—into equivalent rigid-body proxy motion, and that this translation can be learned from purely synthetic training data while generalizing to real scenes.

What carries the argument

Proxy video generation via fine-tuned video diffusion model (Wan-14B with LoRA adapters), Perspective-n-Point (PnP) solver, multi-query bundle adjustment for rigid surfaces

What would settle it

Apply ProxyPose to scenes with motion patterns absent from the synthetic training distribution (e.g., extreme non-rigid deformation, fluid surfaces, or unusual lighting) and compare recovered poses against ground truth. Systematic divergence between proxy motion and true motion would indicate the diffusion model's motion priors do not generalize as needed.

Watch

Extended reading notes

Core claim

The central mechanism is the proxy video: a synthetic rendering of a known polyhedron whose motion mirrors the local rigid-body motion at a queried surface point. This converts the hardest part of pose tracking—handling challenging materials, occlusions, and deformations—into a video translation step handled by the diffusion model's learned motion priors, while leaving the geometric reasoning to off-the-shelf PnP solvers operating on the proxy's known geometry. The paper shows this pipeline achieves state-of-the-art accuracy on HO3D and YCBInEOAT benchmarks while requiring strictly fewer inputs than all competing methods, and extends qualitatively to face tracking, camera pose estimation, in

Load-bearing premise

The method assumes the video diffusion model, fine-tuned on synthetic rigid-body motion, faithfully translates real-world complex motion into equivalent rigid-body proxy motion without systematic drift or hallucination. If the model generates proxy motion that does not reflect the true 3D kinematics of the queried surface, the downstream PnP solver will recover incorrect poses with no signal that an error occurred.

Editorial extensions

If this is right

  • 6-DoF tracking becomes accessible without CAD models, depth sensors, or segmentation pipelines, lowering the barrier for robotics, AR, and scientific video analysis.
  • The video-to-video-translation-to-classical-solver pattern could extend to other perception tasks such as articulated body tracking, non-rigid surface reconstruction, or dense scene flow estimation.
  • Video diffusion models may serve as general-purpose backbones for 3D motion understanding, complementing or replacing task-specific foundation models.
  • Tracking can operate at the pixel level without assumptions about object identity or boundaries, enabling tracking of arbitrary surface regions including non-rigid or fragmenting surfaces.
  • The finding that fine-tuning on only 300 synthetic sequences already yields compelling performance suggests the approach transfers efficiently from synthetic to real data.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. ProxyPose introduces a novel approach to 6-DoF pose tracking from monocular RGB video by recasting it as a video-to-video translation problem. A fine-tuned video diffusion model (Wan-14B with LoRA adapters) translates an input video and a single marked pixel into a 'proxy video' depicting a colored cube undergoing the same local rigid-body motion as the surface at the marked pixel. Because the proxy geometry is known, 6-DoF pose recovery reduces to classical PnP-based tracking. The method is trained solely on 35,000 synthetic sequences and evaluated on HO3D and YCBInEOAT benchmarks, where it achieves competitive or superior accuracy compared to baselines that require 3D models, depth, or object masks. The paper also demonstrates qualitative results on challenging in-the-wild scenes including transparent, reflective, and non-rigid surfaces, as well as applications to face tracking and camera pose estimation.

Significance. The core idea of using a video diffusion model as a general-purpose motion prior for 6-DoF tracking, without task-specific 3D models or depth, is genuinely novel and well-motivated. The method ships several commendable components: a parameter-light fine-tuning scheme (LoRA rank 64, ~2.2% of backbone parameters), a well-designed synthetic training pipeline, a principled noise schedule offset for proxy stability, and a multi-query bundle adjustment formulation with depth scalars. The ablation study (Table 2) justifies key design choices, and the focal-length sensitivity analysis (Table S6) provides useful robustness information. The qualitative in-the-wild results on transparent, reflective, and non-rigid surfaces are compelling and go beyond what existing model-based methods can handle. The approach opens a promising research direction.

major comments (3)
  1. §4, Table S5 and Table S3: The SOTA claim on YCBInEOAT is undermined by high-variance, bimodal performance. Table S5 reports ProxyPose (one query) ATE = 30.1 ± 29.9 mm and ARE = 15.1 ± 24.0° on YCBInEOAT — standard deviations approximately equal to or exceeding the means. Table S3 confirms this is driven by a catastrophic outlier: sequence 00006_obj3 yields ATE = 104.4 mm and ARE = 79.94°, roughly 3–5× the mean. This bimodal pattern (accurate tracking vs. catastrophic failure) is not analyzed in the paper. No failure rate, median, or trimmed-mean metrics are reported. The SOTA claim rests on the failure fraction being low enough that outliers don't dominate the mean, but this is not demonstrated. The authors should report median metrics and/or failure rates, and discuss what causes these failures (e.g., specific object geometries, motion patterns, or proxy generation artifacts). Without此
  2. §C.1 (Supp.): The evaluation protocol selects 49-frame windows that 'maximize the rotation delta between the first and last frames,' discarding windows with <90% object visibility. This selection criterion may systematically favor regimes where the method performs well, since high-rotation segments are precisely where the proxy cube's motion is most observable. Segments where the diffusion model struggles (e.g., slow motion, heavy occlusion, or motion blur) may be underrepresented. The paper should justify why this selection criterion does not bias the evaluation, or report results on randomly selected windows or across all valid windows for transparency.
  3. Table S4: On the held-out synthetic dataset, ProxyPose (one query) reports ATE = 480.3 mm while achieving ARE = 19.79° and RPE-r = 1.920°. This 480 mm translation error is very large and suggests the translation component of the recovered poses is substantially off, even though rotation is competitive. The paper does not discuss this discrepancy. Since the synthetic dataset is the one domain where the training and test distributions match, this large ATE raises questions about the method's translation accuracy in general. The authors should explain the source of this error (e.g., depth/scale ambiguity, PnP failure modes) and discuss whether it indicates a systematic limitation of the recovered poses.
minor comments (7)
  1. §3.1, Eq. (3): The noise schedule offset is described as a fixed parameter (Δoffset = 500 steps), but the ablation in Table 2 only tests Δoffset ∈ {0, 500}. It would help to clarify whether intermediate values were tried and whether the choice is sensitive to the specific backbone or schedule.
  2. §3.4: The text prompt used for conditioning is quite detailed (Supp. §A.1). Was any ablation performed on prompt content? It would be useful to know whether the prompt details (e.g., color descriptions, motion descriptions) matter or whether a minimal prompt suffices.
  3. Table 1: The 'Obj. Mask' column for ProxyPose (two/three queries) is marked with a checkmark, but the text states the mask is only used 'in the first frame to place all queries on the same object.' This should be clarified in the table caption to avoid the impression that per-frame masks are required.
  4. §4.1: The ablation (Table 2) is conducted only on the synthetic dataset. It would strengthen the paper to confirm that the design choices (LoRA rank 64, 35k samples, noise offset) also hold on real benchmarks, or to note this limitation.
  5. Figure 3: The visualization showing tracked points, orientation axes, and trajectories is helpful, but the coordinate axes are small. Consider enlarging or adding zoomed insets for clarity.
  6. §5: The limitations discussion mentions VAE blur from fast motion and drift for textureless/reflective objects, but does not mention the high-variance failure mode visible in the YCBInEOAT results. This should be acknowledged in the limitations section.
  7. Table S6: The focal-length sensitivity results show that halving f causes ATE to jump from 15.79 to 175.2 mm while ARE only increases from 5.1° to 15.2°. This extreme translation sensitivity should be discussed more explicitly, as it suggests the method's translation estimates are fragile to focal-length misestimation.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive report. The referee raises three major points: (1) the bimodal performance on YCBInEOAT and the need for median/failure-rate metrics and failure analysis, (2) potential evaluation bias from the window selection criterion, and (3) the large ATE on the synthetic dataset. We agree that all three points warrant additional discussion and reporting in the revision. Below we address each in turn.

read point-by-point responses

  1. Referee: §4, Table S5 and Table S3: The SOTA claim on YCBInEOAT is undermined by high-variance, bimodal performance. Table S5 reports ProxyPose (one query) ATE = 30.1 ± 29.9 mm and ARE = 15.1 ± 24.0° on YCBInEOAT — standard deviations approximately equal to or exceeding the means. Table S3 confirms this is driven by a catastrophic outlier: sequence 00006_obj3 yields ATE = 104.4 mm and ARE = 79.94°, roughly 3–5× the mean. This bimodal pattern (accurate tracking vs. catastrophic failure) is not analyzed in the paper. No failure rate, median, or trimmed-mean metrics are reported. The SOTA claim rests on the failure fraction being low enough that outliers don't dominate the mean, but this is not demonstrated. The authors should report median metrics and/or failure rates, and discuss what causes these failures (e.g., specific object geometries, motion patterns, or proxy generation artifacts).

    Authors: The referee is correct that the YCBInEOAT results exhibit bimodal behavior driven by the 00006_obj3 outlier, and we agree that the current presentation is insufficient. We will add median metrics and per-sequence failure analysis to the revision. To preview the numbers: the median ATE across the 9 YCBInEOAT sequences is 22.0 mm (vs. mean 30.1 mm) and the median ARE is 6.83° (vs. mean 15.1°), confirming that the mean is inflated by the single outlier. Excluding 00006_obj3, the mean ATE drops to 20.8 mm and mean ARE to 7.10°. We will report these in a revised Table S5. Regarding the failure cause: sequence 00006_obj3 involves the large mustard bottle undergoing rapid in-hand rotation, and inspection of the generated proxy video reveals that the cube identity is lost mid-sequence due to motion blur in the VAE encoding during fast rotation—after which the PnP tracker locks onto an incorrect face orientation. This is consistent with the limitation already noted in Section 5 (fast motion exceeding VAE encoding capabilities). We will add this analysis explicitly, including a failure case figure showing the degraded proxy frames. We also note that even with the outlier included, ProxyPose (one query) achieves the best mean ATE, ARE, RPE-t, and RPE-r among all methods in Table 1, and the next-best method (BundleSDF, which uses depth and a 3D model) has ATE = 42.1 ± 59.3 mm with its own large standard deviation—so the bimodal issue is not unique to our method. That said, we will temper the SOTA claim language to acknowledge the high variance and small sample size (9 sequences). revision: yes

  2. Referee: §C.1 (Supp.): The evaluation protocol selects 49-frame windows that 'maximize the rotation delta between the first and last frames,' discarding windows with <90% object visibility. This selection criterion may systematically favor regimes where the method performs well, since high-rotation segments are precisely where the proxy cube's motion is most observable. Segments where the diffusion model struggles (e.g., slow motion, heavy occlusion, or motion blur) may be underrepresented. The paper should justify why this selection criterion does not bias the evaluation, or report results on randomly selected windows or across all valid windows for transparency.

    Authors: This is a fair concern. The selection criterion was chosen to ensure that the evaluation windows contain sufficient motion to make the 6-DoF tracking task meaningful—windows with near-zero rotation are uninformative for comparing methods since all approaches trivially succeed. However, the referee is right that this could bias results if the method performs systematically worse on low-motion or high-occlusion segments. To address this, we will run evaluation on all valid 49-frame windows (those with ≥90% object visibility in the first frame) across both HO3D and YCBInEOAT and report the results in the supplement. We expect this to include windows with slower motion and partial occlusion. We note that the 90% visibility threshold is applied only to the first frame, not across all frames, so windows with subsequent occlusion are retained. We will also report the number of windows evaluated under each protocol. If the all-windows evaluation reveals a significant performance drop, we will report both and discuss the discrepancy; if results are consistent, this will strengthen the original evaluation. revision: yes

  3. Referee: Table S4: On the held-out synthetic dataset, ProxyPose (one query) reports ATE = 480.3 mm while achieving ARE = 19.79° and RPE-r = 1.920°. This 480 mm translation error is very large and suggests the translation component of the recovered poses is substantially off, even though rotation is competitive. The paper does not discuss this discrepancy. Since the synthetic dataset is the one domain where the training and test distributions match, this large ATE raises questions about the method's translation accuracy in general. The authors should explain the source of this error (e.g., depth/scale ambiguity, PnP failure modes) and discuss whether it indicates a systematic limitation of the recovered poses.

    Authors: The referee correctly identifies a discrepancy that we should have discussed. The large ATE on the synthetic dataset is primarily caused by the depth/scale ambiguity inherent to monocular pose estimation. Although we align scale at the first frame using ground-truth depth, the recovered per-frame translations accumulate depth drift over the sequence because the proxy cube's apparent size in the image does not uniquely constrain its depth—this is the standard perspective scale ambiguity. The synthetic dataset is particularly affected because the rendered scenes include objects at large depths (up to several meters from the camera) with significant depth variation across the 64-frame sequences, amplifying the effect of per-frame depth errors. On HO3D and YCBInEOAT, where objects are closer to the camera and depth variation is smaller, ATE is much lower (15.8 mm and 30.1 mm respectively). We also note that the relative pose errors (RPE-t = 29.9 mm, RPE-r = 1.92°) and 2D reprojection distance (15.2 px) are more moderate, indicating that the local frame-to-frame motion is captured reasonably well even when absolute translation drifts. The rotation metrics (ARE = 19.79°) are competitive with or better than all baselines. We will add this discussion to the supplement, explicitly noting that absolute translation accuracy is a systematic limitation of the monocular formulation, that the multi-query bundle adjustment with depth scalars (Section 3.3) partially mitigates this (ATE drops to 435.1 mm with two queries), and that incorporating additional constraints (e.g., known object size or multi-view input) would be needed to fully resolve the scale ambiguity. revision: yes

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the derivation chain is self-contained with only minor non-load-bearing self-citations for rendering protocols.

full rationale

ProxyPose's central claim—that a video diffusion model fine-tuned on synthetic data can translate real video into proxy videos for 6-DoF pose tracking—is evaluated against external benchmarks (HO3D, YCBInEOAT) using standard PnP solvers. The fine-tuning uses synthetic data with known ground-truth poses, and the evaluation data (real-world benchmarks) is disjoint from the training data. No prediction is fitted to test data and then re-presented as a result. The self-citations present (Taubner et al. 2025a,b; Liang et al. 2025b; Zhang et al. 2026) are for rendering protocols and related architectures, none of which are load-bearing for the core contribution. The method's pipeline (Eq. 1: G_theta generates proxy video, T extracts pose) is a genuine two-stage architecture, not a definitional identity. The PnP and bundle adjustment stages (Eqs. 5, S1) are standard geometric solvers applied to the proxy video output. No step reduces to its inputs by construction.

Assumptions & free parameters 4 free parameters · 2 assumptions · 1 invented entities

The method introduces a few hyperparameters for the proxy geometry and training schedule, which are standard for diffusion model fine-tuning. The core axioms are reasonable domain assumptions.

free parameters (4)
  • scube = 0.15
    Sets the projected area of the proxy cube relative to the image height. Chosen to balance tracking visibility against available image area.
  • Δoffset = 500
    Noise schedule offset for the first proxy frame. Chosen empirically to stabilize video generation.
  • wt, wr = 200, 40
    Weights for the temporal smoothness penalty in bundle adjustment. Chosen to enforce temporal consistency.
  • LoRA rank = 64
    Rank of the low-rank adapters. Selected via ablation (Table 2).
assumptions (2)
  • domain assumption The camera's focal length is known or can be coarsely approximated (e.g., 45-degree FOV).
    Stated in Section 3. The PnP solver requires intrinsics. Sensitivity analysis in Table S6 shows graceful degradation but assumes a known aspect ratio and square pixels.
  • ad hoc to paper Large video diffusion models implicitly encode rich information about 3D motion from 2D appearance changes.
    Stated in Section 1. This is the core premise enabling the approach. While empirically supported by results, it is an assumption about the model's internal representations.
invented entities (1)
  • Proxy video independent evidence
    purpose: A synthetic video depicting a colored polyhedron undergoing the same local rigid-body motion as the surface region at the marked pixel.
    The proxy video is a generated artifact, not a physical entity. Its utility is validated by the downstream pose recovery accuracy on real benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation." pith.science (2026). https://pith.science/paper/ICLJZT72

@misc{pith2026260706555,
  author       = {Pith},
  title        = {Pith review of: ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ICLJZT72}},
  note         = {Machine review of arXiv:2607.06555}
}
read the original abstract

Tracking the six-degree-of-freedom (6-DoF) pose of objects and surfaces from monocular video is a long-standing problem in computer vision. To tackle this problem, existing methods require inputs beyond the video itself-such as 3D models, depth maps, object masks, or task-specific learned features-and they struggle with textureless, transparent, reflective, or deformable surfaces. Here, we introduce ProxyPose, which recasts 6-DoF pose tracking as video-to-video translation. Given only a video and a single marked pixel in the first frame, a fine-tuned video diffusion model translates the input into a proxy video-a synthetic video depicting a colored polyhedron undergoing the same local rigid-body motion as the surface region at the marked pixel. Because the proxy's geometry and appearance are known by construction, recovering its full 6-DoF trajectory reduces to classical pose estimation with off-the-shelf solvers. This formulation leverages large-scale video pre-training to absorb the hardest aspects of pose tracking-handling challenging materials, occlusions, and deformations-into the translation step, while operating at the pixel level with no assumptions about object identity, boundaries, or global rigidity. ProxyPose achieves state-of-the-art 6-DoF pose tracking accuracy without the additional inputs required by competing methods and after fine-tuning the video model only on synthetic data. We further demonstrate that ProxyPose extends to face tracking, camera pose estimation, and challenging in-the-wild scenes that are beyond the reach of existing approaches. Project page: https://ruihangzhang97.github.io/proxypose/.

Figures

Figures reproduced from arXiv: 2607.06555 by the authors.

Figure 1
Figure 1. Our approach enables tracking relative 6-DoF pose in diverse, highly dynamic scenes [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. ProxyPose pipeline overview. Given a source video and a single marked query pixel on a [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Results on HO3D and YCBInEOAT. For each method, we show the tracked point (white [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Challenging in-the-wild videos. ProxyPose successfully tracks surface regions on a specular [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Additional applications. Face tracking compared to FlowFace (Taubner et al., 2024): [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Limitations. Fast motion (mar￾bles) can result in blur from the VAE, degrading pose recovery. Tracking can also drift for fluid surfaces where locally rigid motion is ill-defined (waves) or for textureless/reflective objects (balloons). Despite its generality, ProxyPos…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. C2Dex: Contact-Consistent Reconstruction and Retargeting for Dexterous Manipulation from Monocular Video

    cs.RO 2026-08 conditional novelty 7.0 of 10

    C2Dex converts monocular human videos into executable dexterous robot manipulation trajectories by using stable object-side contacts as a shared representation for reconstruction and retargeting, achieving 57.78% and ...

Reference graph

Works this paper leans on

157 extracted references · 157 canonical work pages · cited by 1 Pith paper

  1. [1]

    Speeded-up robust features (SURF) , author=. Proc. ECCV , year=

    work page

  2. [2]

    A General and Simple Method for Camera Pose and Focal Length Determination , year=

    Zheng, Yinqiang and Sugimoto, Shigeki and Sato, Imari and Okutomi, Masatoshi , booktitle=. A General and Simple Method for Camera Pose and Focal Length Determination , year=

    work page

  3. [3]

    EPnP: An accurate O(n) solution to the PnP problem , author=. Int. J. Comput. Vis. , year=

    work page

  4. [4]

    A Robust O(n) Solution to the Perspective-n-Point Problem , year=

    Li, Shiqi and Xu, Chi and Xie, Ming , journal=. A Robust O(n) Solution to the Perspective-n-Point Problem , year=

    work page

  5. [5]

    IEEE Trans

    A method for registration of 3-D shapes , author=. IEEE Trans. Pattern Anal. Mach. Intell. , volume=

    work page

  6. [6]

    Model globally, match locally: Efficient and robust 3D object recognition , author=. Proc. CVPR , year=

    work page

  7. [7]

    Tracking and modeling non-rigid objects with rank constraints , author=. Proc. CVPR , year=

    work page

  8. [8]

    BB8: A scalable, accurate, robust to partial occlusion method for predicting the 3D poses of challenging objects without using depth , author=. Proc. ICCV , year=

    work page

Show all 157 references
  1. [9]

    CVPR , year=

    PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation , author=. CVPR , year=

  2. [10]

    CosyPose: Consistent multi-view multi-object 6D pose estimation , author=. Proc. ECCV , year=

  3. [11]

    2020 , eprint=

    EfficientPose: An efficient, accurate and scalable end-to-end 6D multi object pose estimation approach , author=. 2020 , eprint=

    2020

  4. [12]

    FFB6D: A full flow bidirectional fusion network for 6D pose estimation , author=. Proc. CVPR , year=

  5. [13]

    Normalized object coordinate space for category-level 6D object pose and size estimation , author=. Proc. CVPR , year=

  6. [14]

    Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimation , author=. Proc. CVPR , year=

  7. [15]

    Proceedings of the European Conference on Computer Vision (ECCV) , month =

    Tian, Meng and Ang Jr, Marcelo H and Lee, Gim Hee , title =. Proceedings of the European Conference on Computer Vision (ECCV) , month =

  8. [16]

    MegaPose: 6D pose estimation of novel objects via render and compare , author=. Proc. CoRL , year=

  9. [17]

    One2Any: One-Reference 6D Pose Estimation for Any Object , author=. Proc. CVPR , year=

  10. [18]

    2025 , eprint=

    One2Any: One-Reference 6D Pose Estimation for Any Object , author=. 2025 , eprint=

    2025

  11. [19]

    LoFTR: Detector-free local feature matching with transformers , author=. Proc. CVPR , year=

  12. [20]

    2024 , eprint=

    NOPE: Novel Object Pose Estimation from a Single Image , author=. 2024 , eprint=

    2024

  13. [21]

    Open-vocabulary object 6D pose estimation , author=. Proc. CVPR , year=

  14. [22]

    GigaPose: Fast and robust novel object pose estimation via one correspondence , author=. Proc. CVPR , year=

  15. [23]

    OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints , author=. Proc. CVPR , year=

  16. [24]

    Diffusion Renderer: Neural Inverse and Forward Rendering with Video Diffusion Models , author=. Proc. CVPR , year=

  17. [25]

    He, Kai and Liang, Ruofan and Munkberg, Jacob and Hasselgren, Jon and Vijaykumar, Nandita and Keller, Alexander and Fidler, Sanja and Gilitschenski, Igor and Gojcic, Zan and Wang, Zian , booktitle=

  18. [26]

    Objaverse: A universe of annotated

    Deitke, Matt and Schwenk, Dustin and Salvador, Jordi and Weihs, Luca and Michel, Oscar and VanderBilt, Eli and Schmidt, Ludwig and Ehsani, Kiana and Kembhavi, Aniruddha and Farhadi, Ali , booktitle=. Objaverse: A universe of annotated

  19. [27]

    2025 , eprint=

    Wan: Open and Advanced Large-Scale Video Generative Models , author=. 2025 , eprint=

    2025

  20. [28]

    Attention is All you Need , url =

    Vaswani, Ashish and Shazeer, Noam and Parmar, Niki and Uszkoreit, Jakob and Jones, Llion and Gomez, Aidan N and Kaiser, ukasz and Polosukhin, Illia , booktitle =. Attention is All you Need , url =

  21. [29]

    arXiv preprint arXiv:2207.12598 , year=

    Classifier-Free Diffusion Guidance , author=. arXiv preprint arXiv:2207.12598 , year=

    arXiv

  22. [30]

    2022 , journal=

    Scalable Diffusion Models with Transformers , author=. 2022 , journal=

    2022

  23. [31]

    Edward J Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen , booktitle=. Lo. 2022 , url=

    2022

  24. [32]

    Bundle Adjustment---A Modern Synthesis , author=. Proc. Vision Algorithms Workshop , year=

  25. [33]

    Andrew W Fitzgibbon and Andrew Zisserman , title=. Proc. ECCV , year=

  26. [34]

    PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization , author=. Proc. ICCV , year=

  27. [35]

    2017 , eprint=

    PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , author=. 2017 , eprint=

    2017

  28. [36]

    DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion , author=. Proc. CVPR , year=. 1901.04780 , archivePrefix=

    1901 arXiv

  29. [37]

    Lee, Taeyeop and Lee, Byeong-Uk and Shin, Inkyu and Choe, Jaesung and Shin, Ukcheol and Kweon, In So and Yoon, Kuk-Jin , booktitle=

  30. [38]

    A New Approach to Linear Filtering and Prediction Problems , author=. J. Basic Eng. , volume=

  31. [39]

    Self-Supervised Learning of a Facial Attribute Embedding from Video , author=. Proc. BMVC , year=

  32. [40]

    PoseRBPF: A Rao-Blackwellized Particle Filter for 6D Object Pose Tracking , author=. Proc. RSS , year=

  33. [41]

    DeepIM: Deep Iterative Matching for 6D Pose Estimation , volume=

    Li, Yi and Wang, Gu and Ji, Xiangyang and Xiang, Yu and Fox, Dieter , year=. DeepIM: Deep Iterative Matching for 6D Pose Estimation , volume=. International Journal of Computer Vision , publisher=. doi:10.1007/s11263-019-01250-9 , number=

    doi

  34. [42]

    Deep Learning-Based Object Pose Estimation: A Comprehensive Survey , volume=

    Liu, Jian and Sun, Wei and Yang, Hui and Zeng, Zhiwen and Liu, Chongpei and Zheng, Jin and Liu, Xingyu and Rahmani, Hossein and Sebe, Nicu and Mian, Ajmal , year=. Deep Learning-Based Object Pose Estimation: A Comprehensive Survey , volume=. International Journal of Computer V...

    doi

  35. [43]

    CVPR , year =

    HOnnotate: A method for 3D Annotation of Hand and Object Poses , author=. CVPR , year =

  36. [44]

    CoRR , volume =

    Shreyas Hampali and Sayan Deb Sarkar and Vincent Lepetit , title =. CoRR , volume =. 2021 , url =. 2107.00887 , timestamp =

    2021 arXiv

  37. [45]

    Sensors (Basel, Switzerland) , year=

    A Survey of 6DoF Object Pose Estimation Methods for Different Application Scenarios , author=. Sensors (Basel, Switzerland) , year=

  38. [46]

    arXiv preprint arXiv:2502.14061 , year=

    EfficientPose 6D: Scalable and Efficient 6D Object Pose Estimation , author=. arXiv preprint arXiv:2502.14061 , year=

    arXiv

  39. [47]

    arXiv preprint arXiv:2502.10606 , year=

    HIPPo: Harnessing Image-to-3D Priors for Model-free Zero-shot 6D Pose Estimation , author=. arXiv preprint arXiv:2502.10606 , year=

    arXiv

  40. [48]

    Lee, Taeyeop and Wen, Bowen and Kang, Minjun and Kang, Gyuree and Kweon, In So and Yoon, Kuk-Jin , booktitle=

  41. [49]

    ConceptPose: Training-Free Zero-Shot Object Pose Estimation using Concept Vectors , author=. Proc. CVPR , year=

  42. [50]

    2026 , eprint=

    ConceptPose: Training-Free Zero-Shot Object Pose Estimation using Concept Vectors , author=. 2026 , eprint=

    2026

  43. [51]

    UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimation , author=. Proc. CoRL , year=

  44. [52]

    IEEE Robot

    Liu, Yibo and Jiang, Zhaodong and Xu, Binbin and Wu, Guile and Ren, Yuan and Cao, Tongtong and Liu, Bingbing and Yang, Rui Heng and Rasouli, Amir and Shan, Jinjun , title=. IEEE Robot. Autom. Lett. , year=

  45. [53]

    ORB: An efficient alternative to SIFT or SURF , author=. Proc. ICCV , year=

  46. [54]

    Geometric loss functions for camera pose regression with deep learning , author=. Proc. CVPR , year=

  47. [55]

    Image-based localization using LSTMs for structured feature correlation , author=. Proc. ICCV , year=

  48. [56]

    DPOD: 6D pose object detector and refiner , author=. Proc. ICCV , year=

  49. [57]

    HybridPose: 6D object pose estimation under hybrid representations , author=. Proc. CVPR , year=

  50. [58]

    GDR-Net: Geometry-guided direct regression network for monocular 6D object pose estimation , author=. Proc. CVPR , year=

  51. [59]

    Uni6D: A Unified CNN Framework without Projection Breakdown for 6D Pose Estimation , author=. Proc. CVPR , year=

  52. [60]

    PointFusion: Deep sensor fusion for 3D bounding box estimation , author=. Proc. CVPR , year=

  53. [61]

    PVN3D: A deep point-wise 3D keypoints voting network for 6DoF pose estimation , author=. Proc. CVPR , year=

  54. [62]

    End-to-end learnable geometric vision by backpropagating PnP optimization , author=. Proc. CVPR , year=

  55. [63]

    PointNetLK: Robust & efficient point cloud registration using PointNet , author=. Proc. CVPR , year=

  56. [64]

    Deep closest point: Learning representations for point cloud registration , author=. Proc. ICCV , year=

  57. [65]

    KinectFusion: Real-time dense surface mapping and tracking , author=. Proc. ISMAR , year=

  58. [66]

    Real-time human pose recognition in parts from single depth images , year=

    Shotton, Jamie and Fitzgibbon, Andrew and Cook, Mat and Sharp, Toby and Finocchio, Mark and Moore, Richard and Kipman, Alex and Blake, Andrew , booktitle=. Real-time human pose recognition in parts from single depth images , year=

  59. [67]

    The International Journal of Robotics Research , volume=

    ElasticFusion: Real-time dense SLAM and light source estimation , author=. The International Journal of Robotics Research , volume=. 2016 , publisher=

    2016

  60. [68]

    IEEE transactions on pattern analysis and machine intelligence , volume=

    Gradient response maps for real-time detection of textureless objects , author=. IEEE transactions on pattern analysis and machine intelligence , volume=. 2011 , publisher=

    2011

  61. [69]

    and Fitzgibbon, Andrew W

    Cashman, Thomas J. and Fitzgibbon, Andrew W. , title=. IEEE Trans. Pattern Anal. Mach. Intell. , year=

  62. [70]

    2021 , doi =

    Wen, Bowen and Bekris, Kostas , title =. 2021 , doi =

    2021

  63. [71]

    BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects , author=. Proc. CVPR , year=

  64. [72]

    Sun, Jiaming and Wang, Zihao and Zhang, Siyu and He, Xingyi and Zhao, Hongcheng and Zhang, Guofeng and Zhou, Xiaowei , booktitle=

  65. [73]

    Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images , author=. Proc. ECCV , year=

  66. [74]

    IEEE Trans

    Fitting parameterized three-dimensional models to images , author=. IEEE Trans. Pattern Anal. Mach. Intell. , volume=

  67. [75]

    IEEE Trans

    Efficient region tracking with parametric models of geometry and illumination , author=. IEEE Trans. Pattern Anal. Mach. Intell. , volume=

  68. [76]

    IEEE Trans

    Real-time visual tracking of complex structures , author=. IEEE Trans. Pattern Anal. Mach. Intell. , volume=

  69. [77]

    Active Vision , pages=

    Tracking with rigid models , author=. Active Vision , pages=. 1993 , publisher=

    1993

  70. [78]

    IEEE Trans

    Real-time markerless tracking for augmented reality: The virtual visual servoing framework , author=. IEEE Trans. Vis. Comput. Graph. , volume=

  71. [79]

    Active appearance models , author=. Proc. ECCV , year=

  72. [80]

    A morphable model for the synthesis of

    Blanz, Volker and Vetter, Thomas , booktitle=. A morphable model for the synthesis of

  73. [81]

    Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion , author=. Proc. ICCV , year=

  74. [82]

    Tracking people with twists and exponential maps , author=. Proc. CVPR , year=

  75. [83]

    Optical flow constraints on deformable models with applications to face tracking , author=. Int. J. Comput. Vis. , volume=

  76. [84]

    IEEE Trans

    Closed-form solutions for physically based shape modeling and recognition , author=. IEEE Trans. Pattern Anal. Mach. Intell. , volume=

  77. [85]

    An iterative image registration technique with an application to stereo vision , author=. Proc. IJCAI , year=

  78. [86]

    Good features to track , author=. Proc. CVPR , year=

  79. [87]

    Distinctive image features from scale-invariant keypoints , author=. Int. J. Comput. Vis. , volume=

  80. [88]

    Davison, Andrew J and Reid, Ian D and Molton, Nicholas D and Stasse, Olivier , journal=

  81. [89]

    Parallel tracking and mapping for small

    Klein, Georg and Murray, David , booktitle=. Parallel tracking and mapping for small

  82. [90]

    IEEE Trans

    Mur-Artal, Raul and Tard. IEEE Trans. Robot. , volume=

  83. [91]

    Engel, Jakob and Sch. Proc. ECCV , year=

  84. [92]

    2004 , publisher=

    Multiple View Geometry in Computer Vision , author=. 2004 , publisher=

    2004

  85. [93]

    Structure-from-motion revisited , author=. Proc. CVPR , year=

  86. [94]

    Model based training, detection and pose estimation of texture-less

    Hinterstoisser, Stefan and Lepetit, Vincent and Ilic, Slobodan and Holzer, Stefan and Bradski, Gary and Konolige, Kurt and Navab, Nassir , booktitle=. Model based training, detection and pose estimation of texture-less

  87. [95]

    Kehl, Wadim and Manhardt, Fabian and Tombari, Federico and Ilic, Slobodan and Navab, Nassir , booktitle=

  88. [96]

    Xiang, Yu and Schmidt, Tanner and Narayanan, Venkatraman and Fox, Dieter , booktitle=

  89. [97]

    CoRR , volume =

    Yu Xiang and Tanner Schmidt and Venkatraman Narayanan and Dieter Fox , title =. CoRR , volume =. 2017 , url =. 1711.00199 , timestamp =

    2017 arXiv

  90. [98]

    Segment anything , author=. Proc. ICCV , year=

  91. [99]

    arXiv preprint arXiv:2408.00714 , year=

    Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R. arXiv preprint arXiv:2408.00714 , year=

    arXiv

  92. [100]

    Depth anything: Unleashing the power of large-scale unlabeled data , author=. Proc. CVPR , year=

  93. [101]

    Depth Anything

    Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang , booktitle=. Depth Anything

  94. [102]

    Repurposing diffusion-based image generators for monocular depth estimation , author=. Proc. CVPR , year=

  95. [103]

    Karaev, Nikita and Rocco, Ignacio and Graham, Benjamin and Neverova, Natalia and Vedaldi, Andrea and Rupprecht, Christian , booktitle=

  96. [104]

    Doersch, Carl and Gupta, Ankush and Markeeva, Larisa and Recasens, Adria and Smaira, Lucas and Aytar, Yusuf and Carreira, Jo. Proc. NeurIPS , year=

  97. [105]

    Wen, Bowen and Mitash, Chaitanya and Ren, Baozhang and Bekris, Kostas E , booktitle=

  98. [106]

    Doersch, Carl and Yang, Yi and Vecerik, Mel and Gokay, Dilara and Gupta, Ankush and Aytar, Yusuf and Carreira, Jo. Proc. ICCV , year=

  99. [107]

    Particle video revisited: Tracking through occlusions using point trajectories , author=. Proc. ECCV , year=

  100. [108]

    Wang, Jianyuan and Chen, Minghao and Karaev, Nikita and Vedaldi, Andrea and Rupprecht, Christian and Novotny, David , booktitle=

  101. [109]

    Grounding image matching in

    Leroy, Vincent and Cabon, Yohann and Revaud, J. Grounding image matching in. Proc. ECCV , year=

  102. [110]

    Li, Zhengqi and Tucker, Richard and Cole, Forrester and Wang, Qianqian and Jin, Linyi and Ye, Vickie and Kanazawa, Angjoo and Holynski, Aleksander and Snavely, Noah , booktitle=

  103. [111]

    Seitz and Richard Szeliski , booktitle=

    Sameer Agarwal and Noah Snavely and Ian Simon and Steven M. Seitz and Richard Szeliski , booktitle=. Building

  104. [112]

    Wen, Bowen and Yang, Wei and Kautz, Jan and Birchfield, Stan , booktitle=

  105. [113]

    He, Xingyi and Sun, Jiaming and Wang, Yuang and Huang, Di and Bao, Hujun and Zhou, Xiaowei , booktitle=

  106. [114]

    Video diffusion models , author=. Proc. NeurIPS , year=

  107. [115]

    arXiv preprint arXiv:2311.15127 , year=

    Stable video diffusion: Scaling latent video diffusion models to large datasets , author=. arXiv preprint arXiv:2311.15127 , year=

    arXiv

  108. [116]

    2024 , url=

    Video generation models as world simulators , author=. 2024 , url=

    2024

  109. [117]

    Polyak, Adam and Zohar, Amit and Brown, Andrew and Tjandra, Andros and Sber, Anurag and Jain, Apoorv and others , journal=. Movie

  110. [118]

    Xiao Fu and Wei Yin and Mu Hu and Kaixuan Wang and Yuexin Ma and Ping Tan and Shaojie Shen and Dahua Lin and Xiaoxiao Long , booktitle=

  111. [119]

    ACM Trans

    Stablenormal: Reducing diffusion variance for stable and sharp normal , author=. ACM Trans. Graph. , volume=. 2024 , publisher=

    2024

  112. [120]

    The surprising effectiveness of diffusion models for optical flow and monocular depth estimation , author=. Proc. NeurIPS , year=

  113. [121]

    Zhang, Bowei and Ke, Lei and Harley, Adam W and Fragkiadaki, Katerina , booktitle=

  114. [122]

    Doersch, Carl and Gupta, Ankush and Markeeva, Larisa and Recasens, Adri. Proc. NeurIPS , year=

  115. [123]

    Xiao, Yuxi and Wang, Qianqian and Zhang, Shangzhan and Xue, Nan and Peng, Sida and Shen, Yujun and Zhou, Xiaowei , booktitle=

  116. [124]

    High-Resolution Image Synthesis with Latent Diffusion Models , author=. Proc. CVPR , year=

  117. [125]

    Adding Conditional Control to Text-to-Image Diffusion Models , author=. Proc. ICCV , year=

  118. [126]

    Taubner, Felix and Zhang, Ruihang and Tuli, Mathieu and Lindell, David B , booktitle=

  119. [127]

    Taubner, Felix and Zhang, Ruihang and Tuli, Mathieu and Bahmani, Sherwin and Lindell, David B , journal=

  120. [128]

    ACM Trans

    Generating the Past, Present and Future from a Motion-Blurred Image , author=. ACM Trans. Graph. , volume=

  121. [129]

    Point Prompting: Counterfactual Tracking with Video Diffusion Models , author=. Proc. ICLR , year=

  122. [130]

    Zhang, Jiyao and Huang, Weiyao and Peng, Bo and Wu, Mingdong and Hu, Fei and Chen, Zijian and Zhao, Bo and Dong, Hao , booktitle=

  123. [131]

    Determining optical flow , author=. Artif. Intell. , volume=

  124. [132]

    arXiv preprint arXiv:2503.20314 , year =

    Wan: Open and Advanced Large-Scale Video Generative Models , author =. arXiv preprint arXiv:2503.20314 , year =

    arXiv

  125. [133]

    Flow Matching for Generative Modeling , author =. Proc. ICLR , year =

  126. [134]

    Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow , author =. Proc. ICLR , year =

  127. [135]

    , author=

    The opencv library. , author=. Dr. Dobb's Journal: Software Tools for the Professional Programmer , volume=

  128. [136]

    Structured

    Xiang, Jianfeng and Lv, Zelong and Xu, Sicheng and Deng, Yu and Wang, Ruicheng and Zhang, Bowen and Chen, Dong and Tong, Xin and Yang, Jiaolong , booktitle=. Structured

  129. [137]

    Zhang, Zitian and Georgiev, Iliyan and Fischer, Michael and Hold-Geoffroy, Yannick and Lalonde, Jean-Fran. Proc. CVPR , year =

  130. [138]

    Liang, Ruofan and He, Kai and Gojcic, Zan and Gilitschenski, Igor and Fidler, Sanja and Vijaykumar, Nandita and Wang, Zian , booktitle=

  131. [139]

    Accelerating

    Ravi, Nikhila and Reizenstein, Jeremy and Novotny, David and Gordon, Taylor and Lo, Wan-Yen and Johnson, Justin and Gkioxari, Georgia , journal =. Accelerating

  132. [140]

    2024 , howpublished =

    2024

  133. [141]

    arXiv preprint arXiv:2511.10647 , year=

    Depth anything 3: Recovering the visual space from any views , author=. arXiv preprint arXiv:2511.10647 , year=

    arXiv

  134. [142]

    A benchmark for the evaluation of

    Sturm, J. A benchmark for the evaluation of. Proc. IROS , year=

  135. [143]

    Taubner, Felix and Raina, Prashant and Tuli, Mathieu and Teh, Eu Wern and Lee, Chul and Huang, Jinmiao , booktitle=

  136. [144]

    Inter-Photon-Limited Videography , author=. Proc. CVPR , year=

  137. [145]

    2024 , howpublished=

    Metavision Sample Recordings , author=. 2024 , howpublished=

    2024

  138. [146]

    Mai, Jinjie and Wang, Chaoyang and Qian, Guocheng Gordon and Menapace, Willi and Tulyakov, Sergey and Ghanem, Bernard and Wonka, Peter and Mirzaei, Ashkan , booktitle=

  139. [147]

    Jeong, Hyeonho and Lee, Suhyeon and Ye, Jong Chul , booktitle=

  140. [148]

    Looking Backward: Streaming Video-to-Video Translation with Feature Banks , author=. Proc. ICLR , year=

  141. [149]

    arXiv preprint arXiv:2503.10634 , year=

    V ^2 Edit: Versatile Video Diffusion Editor for Videos and 3D Scenes , author=. arXiv preprint arXiv:2503.10634 , year=

    arXiv

  142. [150]

    StyleMaster: Stylize Your Video with Artistic Generation and Translation , author=. Proc. CVPR , year=

  143. [151]

    VACE: All-in-One Video Creation and Editing , author=. Proc. ICCV , year=

  144. [152]

    Motion Prompting: Controlling Video Generation with Motion Trajectories , author=. Proc. CVPR , year=

  145. [153]

    From slow bidirectional to fast autoregressive video diffusion models , author=. Proc. CVPR , year=

  146. [154]

    and Gojcic, Zan and Fidler, Sanja and Ling, Huan and Gao, Jun and Ren, Xuanchi , booktitle=

    Bahmani, Sherwin and Shen, Tianchang and Ren, Jiawei and Huang, Jiahui and Jiang, Yifeng and Turki, Haithem and Tagliasacchi, Andrea and Lindell, David B. and Gojcic, Zan and Fidler, Sanja and Ling, Huan and Gao, Jun and Ren, Xuanchi , booktitle=. Lyra: Generative

  147. [155]

    Motionctrl: A unified and flexible motion controller for video generation , author=. Proc. ACM SIGGRAPH , year=

  148. [156]

    Bahmani, Sherwin and Skorokhodov, Ivan and Qian, Guocheng and Siarohin, Aliaksandr and Menapace, Willi and Tagliasacchi, Andrea and Lindell, David B and Tulyakov, Sergey , booktitle=

  149. [157]

    Cotracker3: Simpler and better point tracking by pseudo-labelling real videos , author=. Proc. CVPR , year=

Pith tools

Reviewed July 8, 2026 · model on record in the stance chip above.