Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Seurat: From Moving Points to Depth

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Trajectory patterns alone can reveal depth changes over time.

desk verdict Genuinely fresh idea—depth ratios from 2D trajectories—with strong TAPVid-3D results, but the unstated constant-focal-length assumption and the synthetic-to-real tracker gap need explicit handling before the generalization claim is safe. read the letter →

arxiv 2504.14687 v1 pith:VGGVM66W submitted 2025-04-20 cs.CV

Seokju Cho , Jiahui Huang , Seungryong Kim , Joon-Young Lee This is my paper · ORCID
classification cs.CV
keywords monocularvideodepthpointtrackingratiotemporalcoherencezero-shottransfertransformerslidingwindowTAPVid-3D
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that depth changes over time in a monocular video can be read off the 2D trajectories of tracked points: as a surface recedes or approaches the camera, its projected points spread out or bunch together, and that density change encodes the relative depth change. The authors build a transformer-based model that takes a set of 2D trajectories and their occlusion flags, predicts a depth ratio for each point relative to a reference frame, and then fuses these ratios with an off-the-shelf monocular depth estimator to recover metric depth. Trained only on synthetic videos, the model transfers zero-shot to real egocentric, driving, and studio videos, and on TAPVid-3D it reports smoother and more accurate depth changes than per-frame or video-depth baselines. The contribution matters because it points to a depth signal that uses no stereo, no multi-view geometry, no extra sensors, and no pretrained image backbone.

What carries the argument

The load-bearing object is the depth ratio along a trajectory, $r_{i,t}=d_{i,t}/d_{i,t_0}$, which the model predicts as a log ratio relative to the first frame of a sliding window. The architecture is a two-branch transformer: a supporting branch encodes a uniform grid of 24 by 24 trajectories with alternating temporal and spatial attention, capturing global scene motion, and a query branch processes the query trajectories with cross-attention into the supporting branch, so that biased query-point distributions do not contaminate the scene-motion estimate. The supervised target is an L1 loss on log depth ratios, with the theoretical density-depth identity $\rho^{\mathrm{image}}_{t_0}/\rho^{\mathrm{image}}_t = (d_{t_0}/d_t)^2(\cos\theta_t/\cos\theta_{t_0})^2$ as the motivating signal. At inference, ratios from overlapping windows of length 8 are accumulated by exponentiation and stitching, and per-visible-subsequence median scaling against a monocular depth estimator converts them to metric depths; iterative refinement and an auxiliary occlusion-position head stabilize the predictions.

What would settle it

Feed the trained model TAPVid-3D trajectories that have been perturbed with increasing levels of Gaussian noise or with realistic tracker failure modes, and plot 3D-AJ and APD against noise amplitude: if even small perturbations that are well within typical tracker error cause large depth-ratio errors, the synthetic-to-real transfer is riding on trajectory quality rather than on learned geometry.

Watch

Extended reading notes

Core claim

The central claim is that relative depth over time is recoverable from motion alone. Under a pinhole projection, the density of projected points from a small locally rigid patch scales inversely with the square of the depth, so the ratio of densities at two times determines the depth ratio up to an unknown surface-orientation factor; the paper argues that a transformer can learn this mapping implicitly without estimating surface normals explicitly. Concretely, the model predicts the log depth ratio $\ell^{w}_{i,t} = \log(d^{w}_{i,t}/d^{w}_{i,0})$ within short sliding windows, accumulates the ratios across windows, and rescales them to metric depth by matching medians with a monocular depth estimator per visible subsequence. Evaluated on TAPVid-3D with CoTracker and LocoTrack trajectories, the paper reports substantially better position accuracy and temporal coherence than unprojection baselines and video-depth estimators.

Load-bearing premise

The load-bearing premise is that the depth-ratio mapping learned from clean, ground-truth synthetic trajectories still holds when the input comes from real, noisy off-the-shelf trackers; the paper does not measure how quickly predictions degrade as tracker errors grow.

Editorial extensions

If this is right

  • Monocular video depth becomes obtainable without stereo rigs, multi-view setups, inertial sensors, or a pretrained image backbone.
  • Long and dynamic sequences, where per-frame depth estimators flicker, get temporally smooth depth because changes are predicted inside short windows and stitched across the whole video.
  • The depth-ratio output is metric-free, so it can be combined with any monocular metric depth estimator through per-subsequence median scaling.
  • Because the model is trained on synthetic data alone, the same recipe could be applied to new motion domains without collecting real depth labels.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If trajectory geometry alone carries depth-change information, a promising next step is to train the same kind of model without depth supervision, for instance by enforcing consistency with rigid-flow or epipolar constraints between predicted depths.
  • The texture-patch ablation result suggests that appearance can hurt transfer; a purely geometric, appearance-free input may generalize better across rendering styles, so swapping RGB patches out entirely is likely more robust than adding them.
  • The sliding-window accumulation in Eq. (9) concatenates log-ratio errors, so very long videos may drift; adding overlap consistency or a global temporal smoothing term would be a natural robustness extension.
  • Within a frame, the model has limited spatial relative-depth ability by design, so it functions as a temporal coherence module rather than a full depth estimator; end-to-end training with a monocular branch could remove the need for the median-scaling fusion step.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Seurat, a method that predicts per-point depth changes over time from 2D point trajectories and visibility masks. The input trajectories come from off-the-shelf trackers such as CoTracker or LocoTrack. A two-branch transformer architecture processes a dense supporting grid of trajectories and the query trajectories separately, with cross-attention from the supporting branch into the query branch, and predicts log depth ratios within sliding windows. The window-wise predictions are accumulated and then rescaled with a monocular depth estimator such as ZoeDepth or DepthPro to produce metric depth estimates. The method is trained on synthetic Kubric MOVi-F data with ground-truth trajectories and is evaluated zero-shot on the TAPVid-3D benchmark across Aria, DriveTrack, and PStudio, reporting 3D-AJ, APD, and temporal coherence. The paper reports consistent gains over baselines that combine point trackers with per-frame or video depth estimators, plus ablations supporting the two-branch design, sliding windows, and window-wise loss.

Significance. If the claims hold, the paper makes a useful contribution: it shows that temporal depth changes can be extracted from 2D trajectory patterns alone, without stereo, multi-view, or strong depth backbones at the ratio-prediction stage, and it demonstrates zero-shot transfer from a single synthetic dataset to real-world benchmarks. The evaluation is not circular: the ratio model is trained on synthetic ground-truth depth and tested on external real-world data, and the method works with two different trackers, which mitigates the concern that results depend on a particular tracker from the same group. The ablations are informative, including the handcrafted density baseline, the texture-input ablation, and the Gaussian-smoothing control. The main weaknesses are an unstated and untested constant-focal-length assumption, a train/inference mismatch in trajectory quality that is not analyzed, and a missing key hyperparameter for the sliding window.

major comments (3)
  1. [Sec. 3.2, Eqs. (1)-(4)] The derivation cancels the focal length f between times t and t0, which is valid only if the focal length is constant over time. Under optical zoom or digital crop, a static scene produces projected 2D trajectories that are formally indistinguishable from trajectories caused by depth motion, so the mapping from 2D trajectories to depth ratios is non-identifiable without an additional assumption or an input that encodes the changing intrinsics. The paper never states the constant-focal-length assumption, and no experiment with time-varying intrinsics is reported; both training (Sec. 4.2, Kubric with fixed camera intrinsics) and evaluation (TAPVid-3D, fixed-camera videos) avoid exercising this confound. I ask the authors to state the assumption explicitly and to either add an experiment with changing focal length (for example, synthetic sequences with smoothly varying f, or digitally cropped videos) or restrict the abstract and introduction claims to fixed-intrinsics videos.
  2. [Sec. 4.2 and supplementary Sec. 8] The model is trained on ground-truth trajectory positions and ground-truth occlusion masks, as stated in Sec. 8, while at inference it receives trajectories from CoTracker or LocoTrack, which contain tracking noise, drift, and failures. The reported benchmark results show that the pipeline works in practice with both trackers, but the sensitivity of the learned depth-ratio mapping to tracker error is never quantified. I request an analysis that corrupts ground-truth trajectories with Gaussian noise, drift, and incorrect occlusion labels during evaluation, reporting how APD, 3D-AJ, and TC degrade as a function of perturbation level. This would directly test the most fragile premise of the method.
  3. [Sec. 4.2 and Eq. (9)] The sliding-window stride S is a free parameter used in training, in the accumulation formula Eq. (9), and in inference, but the implementation details specify only the window size W = 8 and never state the value of S. Without S, the exact experimental setup and the ratio-accumulation procedure cannot be reproduced. Report the stride (and any overlap schedule) used in all experiments.
minor comments (4)
  1. [Sec. 3.2, Eq. (3)] The cosine factor in Eq. (3) appears to be squared by mistake: from Eq. (2), the density ratio is (d_t0/d_t)^2 * (cosθ_t/cosθ_t0), not (d_t0/d_t)^2 * (cosθ_t/cosθ_t0)^2. The subsequent Eq. (4) is consistent with the corrected form, so this seems to be a typographical error rather than a substantive one.
  2. [Table 7 caption and Table 9 header] Table 7's caption contains the typo 'meidan scaling' (should be 'median scaling'), and Table 9's header cites 'DepthPro [3]' where the reference should be [4].
  3. [Sec. 4.4, Table 3 (left)] The description of ablation (IV), 'ratio depth with respect to the query point,' is unclear: since query points are trajectories, please specify the anchor time and how this loss differs from the proposed window-wise log-ratio loss.
  4. [Table 6] Several numeric entries in Table 6 have irregular spacing (for example, '4.8 8 .6' and '7.7 14 .1'), which makes the table hard to read; reformat the entries.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Seurat's depth-ratio predictions are trained on synthetic ground truth and evaluated on an external benchmark; self-citations are inputs, not load-bearing reductions.

full rationale

The central claim is that temporal depth ratios can be predicted from 2D point trajectories by a transformer trained on synthetic Kubric data with ground-truth depth. The evaluation is on TAPVid-3D, an external real-world benchmark, and no parameter is fitted to the test set; the depth-ratio targets come from synthetic ground truth (Sec. 4.2, Eqs. 7-8), not from the benchmark. The theoretical derivation (Eqs. 1-4) is explicitly qualified as requiring unknown rotation information, and the paper does not use it as a closed-form predictor; Table 4 shows a handcrafted implementation of Eq. 4 underperforms, so the learned model is not equivalent to that formula by construction. The authors' own LocoTrack [9] is used as an off-the-shelf tracker, but the same architecture is also evaluated with CoTracker [24], and the tracker is an input to the depth-ratio model, not a component whose output is renamed as prediction. The focal-length/zoom ambiguity raised by the skeptic is a real generalization risk (the derivation cancels f only if f is constant), but it is an unstated assumption and a correctness concern, not a circular reduction of the prediction to its input. There is no imported uniqueness theorem, no ansatz smuggled in by citation, and no renaming of a known result. The derivation chain is therefore self-contained with respect to circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method relies on a small number of hand-chosen hyperparameters and on domain assumptions about camera model, rigidity, and track quality. No new physical entities are introduced.

free parameters (5)
  • window_size_W = 8
    Temporal window length chosen by hand; affects the resolution of depth ratio prediction and the amount of temporal context. No sensitivity analysis is reported.
  • window_stride_S = not specified
    The overlap between sliding windows is a hyperparameter in Eq. (9) but its value is never given in the paper, making exact inference unreproducible.
  • number_of_transformer_layers_L = 2
    Chosen by hand; the ablation shows performance is similar for L=2,3,4, so the choice is not critical.
  • supporting_grid_size = 24x24
    Sampling density of the supporting grid; affects the richness of global motion context.
  • iterative_refinement_iterations = 4
    Number of refinement passes used in training and inference, following common practice in tracking.
assumptions (4)
  • domain assumption Pinhole camera model with time-invariant intrinsics
    Used in the theoretical derivation (Eq. 1-4) and implicitly in the density-depth relationship. Real cameras may deviate, though experiments on three diverse datasets suggest the assumption holds approximately.
  • domain assumption Local rigidity of surface patches over the temporal window
    Assumed in Sec. 3.2 to relate density change to depth ratio. The paper acknowledges this and relies on the transformer to handle non-rigid motion implicitly.
  • domain assumption Synthetic Kubric data is representative of real-world dynamics for zero-shot transfer
    The model is trained only on synthetic MOVi-F and tested on real Aria, DriveTrack, and PStudio. The success of transfer depends on this domain similarity.
  • domain assumption Off-the-shelf point trackers provide trajectories accurate enough for depth inference
    At inference, both CoTracker and LocoTrack are used. The paper does not analyze failure cases caused by tracking errors, so this is an implicit premise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Seurat: From Moving Points to Depth." pith.science (2026). https://pith.science/paper/VGGVM66W

@misc{pith2026250414687,
  author       = {Pith},
  title        = {Pith review of: Seurat: From Moving Points to Depth},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VGGVM66W}},
  note         = {Machine review of arXiv:2504.14687}
}
read the original abstract

Accurate depth estimation from monocular videos remains challenging due to ambiguities inherent in single-view geometry, as crucial depth cues like stereopsis are absent. However, humans often perceive relative depth intuitively by observing variations in the size and spacing of objects as they move. Inspired by this, we propose a novel method that infers relative depth by examining the spatial relationships and temporal evolution of a set of tracked 2D trajectories. Specifically, we use off-the-shelf point tracking models to capture 2D trajectories. Then, our approach employs spatial and temporal transformers to process these trajectories and directly infer depth changes over time. Evaluated on the TAPVid-3D benchmark, our method demonstrates robust zero-shot performance, generalizing effectively from synthetic to real-world datasets. Results indicate that our approach achieves temporally smooth, high-accuracy depth predictions across diverse domains.

Figures

Figures reproduced from arXiv: 2504.14687 by the authors.

Figure 1
Figure 1. Seurat predicts precise and smooth depth changes for dynamic objects over time by only looking at the 2D point trajectories, which encode depth cues in their motion patterns. The figure illustrates 2D point tracks lifted into 3D space with our depth predictions on videos from the DAVIS dataset [41]. Abstract Accurate depth estimation from monocular videos re￾mains challenging due to ambiguities inherent in single￾vi… view at source ↗
Figure 2
Figure 2. Motivation of our work. (a) By only looking at the tracked points, we can easily perceive that the object (here, a car) is moving away. (b) As a 3D object (here, a sphere) moves away from the camera, the pattern of its projected 2D points on the image plane changes, providing depth cues. In the initial frame (left), points are spaced farther apart on the image plane. As the object recedes (right), these 2D points co… view at source ↗
Figure 3
Figure 3. Overall architecture. We first use off-the-shelf point tracker [9, 24] to extract 2D trajectories of query points and a dense supporting grid, then, these trajectories are processed with a tem￾poral and a spatial transformer in two separate branches. Motion information encoded by the supporting branch is injected into the query branch via cross-attention. Finally, two regression heads output ratio depths of both sup… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative comparisons to baselines. We visualize 3D trajectories using the TAPVid-3D [27] benchmark. Compared to baselines that use combinations such as CoTracker with ZoeDepth, our model achieves superior depth smoothness and accuracy. tire video sequence. Our metho…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Emergent Temporal Correspondences from Video Diffusion Transformers

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Video diffusion transformers encode temporal correspondences primarily in query-key similarities of a few specific attention layers, which can be extracted for zero-shot point tracking and used for training-free motio...

Reference graph

Works this paper leans on

57 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Vivit: A video vision transformer

    Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lu ˇci´c, and Cordelia Schmid. Vivit: A video vision transformer. In Proceedings of the IEEE/CVF inter- national conference on computer vision , pages 6836–6846,

  2. [2]

    Drivetrack: A benchmark for long-range point tracking in real-world videos, 2023

    Arjun Balasingam, Joseph Chandler, Chenning Li, Zhoutong Zhang, and Hari Balakrishnan. Drivetrack: A benchmark for long-range point tracking in real-world videos, 2023. 4, 6, 7, 9, 10

    work page 2023

  3. [3]

    Zoedepth: Zero-shot trans- fer by combining relative and metric depth

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot trans- fer by combining relative and metric depth. arXiv preprint arXiv:2302.12288, 2023. 3, 5, 6, 7, 9, 10

    arXiv 2023

  4. [4]

    Depth pro: Sharp monocular metric depth in less than a second

    Aleksei Bochkovskii, Ama ¨el Delaunoy, Hugo Germain, Marcel Santos, Yichao Zhou, Stephan R Richter, and Vladlen Koltun. Depth pro: Sharp monocular metric depth in less than a second. arXiv preprint arXiv:2410.02073, 2024. 2, 3, 5, 6, 7, 8, 9, 10

    arXiv 2024

  5. [5]

    Recovering non-rigid 3d shape from image streams

    Christoph Bregler, Aaron Hertzmann, and Henning Bier- mann. Recovering non-rigid 3d shape from image streams. In Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No. PR00662), pages 690–696. IEEE, 2000. 3

    work page 2000

  6. [6]

    Efficient geometry-aware 3d generative adversarial networks

    Eric R Chan, Connor Z Lin, Matthew A Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J Guibas, Jonathan Tremblay, Sameh Khamis, et al. Efficient geometry-aware 3d generative adversarial networks. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16123–16133, 2022. 3

    2022

  7. [7]

    Pyramid stereo matching network

    Jia-Ren Chang and Yong-Sheng Chen. Pyramid stereo matching network. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 5410–5418,

  8. [8]

    Flowtrack: Revisiting optical flow for long- range dense tracking

    Seokju Cho, Jiahui Huang, Seungryong Kim, and Joon- Young Lee. Flowtrack: Revisiting optical flow for long- range dense tracking. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19268–19277, 2024. 4, 5, 6

    work page 2024

Show all 57 references
  1. [9]

    Local all-pair correspon- dence for point tracking

    Seokju Cho, Jiahui Huang, Jisu Nam, Honggyu An, Seun- gryong Kim, and Joon-Young Lee. Local all-pair correspon- dence for point tracking. arXiv preprint arXiv:2407.15420,

    arXiv

  2. [10]

    Tap-vid: A benchmark for track- ing any point in a video

    Carl Doersch, Ankush Gupta, Larisa Markeeva, Adria Re- casens, Lucas Smaira, Yusuf Aytar, Joao Carreira, Andrew Zisserman, and Yi Yang. Tap-vid: A benchmark for track- ing any point in a video. Advances in Neural Information Processing Systems, 35:13610–13626, 2022. 2, 4

    2022

  3. [11]

    Tapir: Tracking any point with per-frame initialization and temporal refinement

    Carl Doersch, Yi Yang, Mel Vecerik, Dilara Gokay, Ankush Gupta, Yusuf Aytar, Joao Carreira, and Andrew Zisserman. Tapir: Tracking any point with per-frame initialization and temporal refinement. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pag...

    2023

  4. [12]

    Towards real-time monocular depth estimation for robotics: A survey

    Xingshuai Dong, Matthew A Garratt, Sreenatha G Ana- vatti, and Hussein A Abbass. Towards real-time monocular depth estimation for robotics: A survey. IEEE Transactions on Intelligent Transportation Systems, 23(10):16940–16961,

  5. [13]

    Adaptive colour classifi- cation for structured light systems

    Philipp Fechteler and Peter Eisert. Adaptive colour classifi- cation for structured light systems. IET Computer Vision, 3 (2):49–59, 2009. 1

    2009

  6. [14]

    Fast and high resolution 3d face scanning

    Philipp Fechteler, Peter Eisert, and Jurgen Rurainsky. Fast and high resolution 3d face scanning. In 2007 IEEE Interna- tional Conference on Image Processing, pages III–81. IEEE,

    2007

  7. [15]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In Conference on Computer Vision and Pattern Recog- nition (CVPR), 2012. 2

    2012

  8. [16]

    Vision meets robotics: The kitti dataset

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. Interna- tional Journal of Robotics Research (IJRR), 2013. 2

    2013

  9. [17]

    Unsupervised monocular depth estimation with left- right consistency

    Cl ´ement Godard, Oisin Mac Aodha, and Gabriel J Bros- tow. Unsupervised monocular depth estimation with left- right consistency. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 270–279,

  10. [18]

    Kubric: A scalable dataset generator

    Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapra- gasam, Florian Golemo, Charles Herrmann, et al. Kubric: A scalable dataset generator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2022

  11. [19]

    Particle video revisited: Tracking through occlusions using point trajectories

    Adam W Harley, Zhaoyuan Fang, and Katerina Fragkiadaki. Particle video revisited: Tracking through occlusions using point trajectories. In European Conference on Computer Vi- sion, pages 59–75. Springer, 2022. 2, 5

    2022

  12. [20]

    Multiple view geometry in computer vision

    Richard Hartley. Multiple view geometry in computer vision. Cambridge university press, 2003. 3

    2003

  13. [21]

    Depthcrafter: Generating consistent long depth sequences for open-world videos

    Wenbo Hu, Xiangjun Gao, Xiaoyu Li, Sijie Zhao, Xiaodong Cun, Yong Zhang, Long Quan, and Ying Shan. Depthcrafter: Generating consistent long depth sequences for open-world videos. arXiv preprint arXiv:2409.02095, 2024. 1, 6, 7, 9, 10

    2024 arXiv

  14. [22]

    Panoptic studio: A massively multiview system for social motion capture

    Hanbyul Joo, Hao Liu, Lei Tan, Lin Gui, Bart Nabbe, Iain Matthews, Takeo Kanade, Shohei Nobuhara, and Yaser Sheikh. Panoptic studio: A massively multiview system for social motion capture. In Proceedings of the IEEE inter- national conference on computer vision , pages 3334–3342,

  15. [24]

    Co- tracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- tracker: It is better to track together. arXiv preprint arXiv:2307.07635v2, 2023. 2, 3, 4, 5, 6, 7, 9, 10

    2023 arXiv

  16. [25]

    Fast encoder- based 3d from casual videos via point track processing.arXiv preprint arXiv:2404.07097, 2024

    Yoni Kasten, Wuyue Lu, and Haggai Maron. Fast encoder- based 3d from casual videos via point track processing.arXiv preprint arXiv:2404.07097, 2024. 3

    2024 arXiv

  17. [26]

    Exploring 11 temporally-aware features for point tracking

    In `es Hyeonsu Kim, Seokju Cho, Jiahui Huang, Jung Yi, Joon-Young Lee, and Seungryong Kim. Exploring 11 temporally-aware features for point tracking. arXiv preprint arXiv:2501.12218, 2025. 2

    2025 arXiv

  18. [27]

    Tapvid-3d: A benchmark for tracking any point in 3d

    Skanda Koppula, Ignacio Rocco, Yi Yang, Joe Heyward, Jo˜ao Carreira, Andrew Zisserman, Gabriel Brostow, and Carl Doersch. Tapvid-3d: A benchmark for tracking any point in 3d. arXiv preprint arXiv:2407.05921, 2024. 2, 6, 7, 8, 9, 10

    2024 arXiv

  19. [28]

    Ground- ing image matching in 3d with mast3r

    Vincent Leroy, Yohann Cabon, and J´erˆome Revaud. Ground- ing image matching in 3d with mast3r. arXiv preprint arXiv:2406.09756, 2024. 2, 3

    2024 arXiv

  20. [29]

    Taptrv2: Attention-based position update improves tracking any point

    Hongyang Li, Hao Zhang, Shilong Liu, Zhaoyang Zeng, Feng Li, Tianhe Ren, Bohan Li, and Lei Zhang. Taptrv2: Attention-based position update improves tracking any point. arXiv preprint arXiv:2407.16291, 2024. 2

    2024 arXiv

  21. [30]

    Taptr: Tracking any point with transformers as detection

    Hongyang Li, Hao Zhang, Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, and Lei Zhang. Taptr: Tracking any point with transformers as detection. In European Confer- ence on Computer Vision, pages 57–75. Springer, 2025. 2

    2025

  22. [31]

    Megadepth: Learning single- view depth prediction from internet photos

    Zhengqi Li and Noah Snavely. Megadepth: Learning single- view depth prediction from internet photos. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 2041–2050, 2018. 3

    2018

  23. [32]

    Dynibar: Neural dynamic image-based rendering

    Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. Dynibar: Neural dynamic image-based rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4273– 4284, 2023. 1

    2023

  24. [33]

    Dual-frequency pattern scheme for high-speed 3-d shape measurement

    Kai Liu, Yongchang Wang, Daniel L Lau, Qi Hao, and Lau- rence G Hassebrook. Dual-frequency pattern scheme for high-speed 3-d shape measurement. Optics express, 18(5): 5229–5244, 2010. 1

    2010

  25. [34]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 7

    2017 arXiv

  26. [35]

    Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis, 2023

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis, 2023. 2

    2023

  27. [36]

    Consistent video depth estimation.ACM Transactions on Graphics (ToG), 39(4):71–1, 2020

    Xuan Luo, Jia-Bin Huang, Richard Szeliski, Kevin Matzen, and Johannes Kopf. Consistent video depth estimation.ACM Transactions on Graphics (ToG), 39(4):71–1, 2020. 1

    2020

  28. [37]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 2, 3

    2023 arXiv

  29. [38]

    Aria digital twin: A new benchmark dataset for egocentric 3d machine perception

    Xiaqing Pan, Nicholas Charron, Yongqian Yang, Scott Pe- ters, Thomas Whelan, Chen Kong, Omkar Parkhi, Richard Newcombe, and Yuheng Carl Ren. Aria digital twin: A new benchmark dataset for egocentric 3d machine perception. In Proceedings of the IEEE/CVF International Conference...

    2023

  30. [39]

    Unidepth: Universal monocular metric depth estimation, 2024

    Luigi Piccinelli, Yung-Hsu Yang, Christos Sakaridis, Mattia Segu, Siyuan Li, Luc Van Gool, and Fisher Yu. Unidepth: Universal monocular metric depth estimation, 2024. 6

    2024

  31. [40]

    Detailed real- time urban 3d reconstruction from video.International Jour- nal of Computer Vision, 78:143–167, 2008

    Marc Pollefeys, David Nist ´er, J-M Frahm, Amir Ak- barzadeh, Philippos Mordohai, Brian Clipp, Chris Engels, David Gallup, S-J Kim, Paul Merrell, et al. Detailed real- time urban 3d reconstruction from video.International Jour- nal of Computer Vision, 78:143–167, 2008. 1

    2008

  32. [41]

    The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017. 1

    2017 arXiv

  33. [42]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE transactions on pattern analysis and machine intelligence, 44(3):1623–1637, 2020. 1, 2, 3, 9

    2020

  34. [43]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  35. [44]

    Structure- from-motion revisited

    Johannes L Schonberger and Jan-Michael Frahm. Structure- from-motion revisited. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 4104–4113, 2016. 3

    2016

  36. [45]

    Learning tem- porally consistent video depth from video diffusion priors

    Jiahao Shao, Yuanbo Yang, Hongyu Zhou, Youmin Zhang, Yujun Shen, Matteo Poggi, and Yiyi Liao. Learning tem- porally consistent video depth from video diffusion priors. arXiv preprint arXiv:2406.01493, 2024. 6, 7, 9, 10

    2024 arXiv

  37. [46]

    Track everything everywhere fast and ro- bustly

    Yunzhou Song, Jiahui Lei, Ziyun Wang, Lingjie Liu, and Kostas Daniilidis. Track everything everywhere fast and ro- bustly. In European Conference on Computer Vision, pages 343–359. Springer, 2025. 2

    2025

  38. [47]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aurelien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Et- tinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zhan...

    2020

  39. [48]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part II 16, pages 402–419. Springer,

    2020

  40. [49]

    Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras

    Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neu- ral information processing systems, 34:16558–16569, 2021. 2, 3

    2021

  41. [50]

    Mlp-mixer: An all-mlp architecture for vision

    Ilya O Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lu- cas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Andreas Steiner, Daniel Keysers, Jakob Uszkoreit, et al. Mlp-mixer: An all-mlp architecture for vision. Advances in neural information processing systems, 34:2...

  42. [51]

    Dino-tracker: Taming dino for self-supervised point track- ing in a single video

    Narek Tumanyan, Assaf Singer, Shai Bagon, and Tali Dekel. Dino-tracker: Taming dino for self-supervised point track- ing in a single video. In European Conference on Computer Vision, pages 367–385. Springer, 2025. 2

    2025

  43. [52]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017. 2, 4, 7 12

    2017

  44. [53]

    Tracking everything everywhere all at once

    Qianqian Wang, Yen-Yu Chang, Ruojin Cai, Zhengqi Li, Bharath Hariharan, Aleksander Holynski, and Noah Snavely. Tracking everything everywhere all at once. arXiv preprint arXiv:2306.05422, 2023. 2, 6

    2023 arXiv

  45. [54]

    Spatialtracker: Tracking any 2d pixels in 3d space

    Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng, Yujun Shen, and Xiaowei Zhou. Spatialtracker: Tracking any 2d pixels in 3d space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20406–20417, 2024. 3, 9

    2024

  46. [55]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10371–10381, 2024. 1, 3, 9

    2024

  47. [56]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3836–3847, 2023. 1

    2023

  48. [57]

    Pointodyssey: A large-scale synthetic dataset for long-term point tracking

    Yang Zheng, Adam W Harley, Bokui Shen, Gordon Wet- zstein, and Leonidas J Guibas. Pointodyssey: A large-scale synthetic dataset for long-term point tracking. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 19855–19865, 2023. 4 13

    2023

  49. [2024]

    1, 2, 3, 4, 5, 6, 7, 9, 10

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.