Pith. sign in

REVIEW 3 major objections 7 minor 61 references

AllTracker: Efficient Dense Point Tracking at High Resolution

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AllTracker claims that long-range point tracking is best solved as dense multi-frame optical flow, delivering state-of-the-art accuracy at 768×1024 while running at optical-flow speed with only 16 million parameters.

desk verdict Solid dense point tracker with real practical gains, but the reported SOTA margin rests on an untested evaluation shortcut that could flip the ranking. read the letter →

arxiv 2506.07310 v2 pith:IWSMCL4B submitted 2025-06-08 cs.CV

Adam W. Harley , Yang You , Xinglong Sun , Yang Zheng , Nikhil Raghuraman , Yunqi Gu , Sheldon Liang , Wen-Hsuan Chu
show 6 more authors
Achal Dave Pavel Tokmakov Suya You Rares Ambrus Katerina Fragkiadaki Leonidas J. Guibas
This is my paper
classification cs.CV
keywords densepointtrackinglong-rangeopticalflowmulti-framecorrespondencetemporalattentionslidingwindowinferencejointandtraininghigh-resolutionvideotrajectories
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AllTracker claims that long-range point tracking is best solved as multi-frame optical flow: estimating the correspondence field from one query frame to every other frame, at full pixel density. This reframing lets the model bring spatial awareness from optical flow methods and temporal awareness from point trackers into one 16 million parameter network, and the result is state-of-the-art tracking accuracy at 768×1024, the highest resolution tested, while running about as fast as an optical flow model. A direct consequence, if the paper is right, is that sparse trackers that require pre-selected query points are no longer needed for accurate dense tracking, and all-pixel trajectories become practical for long videos. The paper also finds that jointly training on optical flow datasets and point tracking datasets is crucial for the top results, an ingredient earlier point trackers did not use.

What carries the argument

The load-bearing component is a recurrent refinement module that operates on low-resolution grids (spatial stride 8) and iterates four times with shared weights. It builds multi-scale 4D correlation volumes between a copy of the query frame's feature map and each timestep's feature pyramid, then, for every pixel and timestep, extracts local correlation patches centered at the current position estimate and interleaves 2D ConvNeXt spatial blocks with pixel-aligned temporal attention (attention along the temporal axis for corresponding pixels only). The module decodes explicit revisions to motion, visibility, and confidence, and learns the weights of a pixel-shuffle upsampling layer that restores full resolution. A sliding window of length 16 with stride 8 carries estimates forward between windows, which is what allows inference on long videos. This machinery borrows the low-resolution iterative refinement and upsampling of optical flow models and the per-pixel temporal attention of point trackers, and the ablation study shows that both the transformer temporal module and the query-relative motion representation contribute to the accuracy.

What would settle it

Run the nine-benchmark evaluation under the full protocol — tracking points from every available query frame and keeping full video lengths — and recompute average δavg; if AllTracker's lead over CoTracker3 disappears or reverses, the state-of-the-art claim is falsified. A cheaper check: the paper reports a 3.5-point gap between realtime streaming and offline inference at 512×512, so measuring that gap on the same benchmarks gives a direct bound on the protocol's influence.

Watch

Extended reading notes

Core claim

The paper's central claim is that AllTracker is a state-of-the-art point tracker at high resolution and full density. On nine point-tracking benchmarks it reports an average δavg of 66.1 at 384×512 and 69.5 at 768×1024, versus 65.0 and 66.9 for CoTracker3, the previous state of the art, and it produces 786,432 tracks at once on a 40G GPU. The method's distinctive move is to represent point tracking as long-range flow: the model outputs, for every pixel of the query frame, a flow map to each of hundreds of later frames, plus visibility and confidence maps. This makes the task a direct generalization of optical flow rather than a separate sparse-tracking problem, and it lets the same weights be trained on both flow and tracking supervision.

Load-bearing premise

The paper's leading claim rests on the assumption that evaluating large benchmarks with only the first available query frame and videos trimmed to 600 frames matches the full protocol (all queries, full length); this is asserted without a demonstrated comparison, and since the reported margin over CoTracker3 is only about 1 point at 384×512, a stricter protocol could shift the ranking.

Editorial extensions

If this is right

  • Sparse point trackers of similar speed and accuracy are made redundant: any subset of pixels can be tracked by sampling from dense output, with no per-query cost.
  • Dense all-pixel tracks at 768×1024 become practical on one 40G GPU, a resolution where the strongest sparse baseline required a 96G GPU.
  • Joint training on optical flow and point tracking data gives accuracy comparable to CoTracker3's 15,000-video bootstrapping, without pseudo-labels.
  • Streaming inference enables realtime tracking at about 58 FPS at 512×512, with a modest accuracy penalty (62.6 vs. 66.1 δavg).
  • The model underfits short-range optical flow (SINTEL EPE 1.673 clean vs. SEA-RAFT 1.309), so the same design with more compute should improve further.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: since temporal attention is pixel-aligned, widening the window S (possible with larger GPUs) should directly extend the occlusion horizon without changing the architecture.
  • Beyond the paper: the dense output could serve as a cheap pseudo-label source for training other video models, complementing the bootstrapping approach the paper compares against.
  • Beyond the paper: the negative result on sinusoidal positional embeddings and the preference for query-relative motion suggest that absolute timeline information is unhelpful for windows of this length, which may simplify future designs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper introduces AllTracker, a dense point tracker that reformulates long-range point tracking as multi-frame optical flow from a query frame to every other frame. The architecture combines a ConvNeXt encoder, multi-scale correlation volumes, and a recurrent refinement module that interleaves 2D convolutions with pixel-aligned temporal attention, operating on low-resolution grids and upsampling to full resolution. The model is trained jointly on optical flow and point tracking datasets, and the paper reports state-of-the-art δavg results on nine point-tracking benchmarks at 384×512 as well as at higher resolutions (448×768 and 768×1024), with 16.48M parameters and optical-flow-scale throughput. The paper also presents ablations of temporal modules, motion representations, backbones, and hyperparameters, and releases code and model weights.

Significance. If the reported results hold, AllTracker is a valuable practical contribution: it provides dense all-pixel long-range tracking at high resolution with a memory footprint and speed that sparse point trackers do not offer, and it demonstrates that joint training on optical flow and point tracking data is beneficial. The paper is an empirical systems paper with no circularity in its evaluation design: the model is trained on synthetic data and evaluated on held-out real benchmarks, and the ablations are genuine architecture comparisons. Strengths include the breadth of the evaluation (nine benchmarks, including high-resolution comparisons), the extensive ablation study, the clear disclosure of evaluation-protocol choices, and the promise of released code and weights. However, the central state-of-the-art claim currently rests on an unvalidated evaluation simplification and on small margins over the main baseline, so the significance is conditional on resolving those concerns.

major comments (3)
  1. [Section 4, Evaluation protocol; Table 1; Table 8] The paper states that on larger datasets it tracks only points from the first available query frame and trims videos to 600 frames because this 'gives similar results to using all possible queries and full video lengths,' but no comparison supporting this assertion is provided. This is load-bearing for the central claim: the average margin over CoTracker3 in Table 1 is only 1.1 δavg (66.1 vs. 65.0), so a differential effect of the protocol choice between AllTracker and CoTracker3 larger than 1.1 would flip the ranking. Table 8 only compares query-grouping protocols for CoTracker3 on four datasets and does not isolate the first-query/600-frame-truncation simplification, nor does it test AllTracker under the full protocol. The authors should provide a per-dataset comparison on a subset of benchmarks (or on one or two full benchmarks) between the simplified and full protocols for both AllTracker and CoTracker3, and report the resulting differences.
  2. [Appendix E, Validation dataset; Section 4.2; Tables 5-7] The ablation validation set is constructed from six of the nine main evaluation benchmarks (BADJA, CroHD, TAPVid-DAVIS, DriveTrack, Horse10, RoboTAP), and the design choices validated on it (temporal module, motion representation, backbone, refinement blocks, correlation radius/scales, loss type, frame ordering) are used in the final model whose numbers are reported on the same benchmarks in Table 1. This overlap means the main benchmark results are not an independent test of the architecture choices: the model's hyperparameters have effectively been selected using labels from the evaluation datasets. Please either report main results on benchmarks that were not used in any form for model selection, or provide evidence that the selected choices are not tuned to these particular test labels, for example by showing that the relative ranking of the ablated variants is stable across a disjoint validation set.
  3. [Tables 1, 2, and 3; Section 4.1] All reported results are single evaluation runs without error bars, confidence intervals, or any measure of run-to-run or protocol-induced variability. This is particularly important because the state-of-the-art margin in Table 1 is small (1.1 δavg over CoTracker3) and because the CoTracker3 baseline is evaluated under a protocol that the authors themselves state over-estimates its accuracy. The authors should report variance across at least a few runs of the main comparisons, or otherwise quantify the sensitivity of the rankings to the evaluation protocol, so that the reader can assess whether the claimed advantage is robust.
minor comments (7)
  1. [Section 3.1] The notation 't ∈ T' in the problem formulation is imprecise; it should be something like 't ∈ {0, ..., T−1}'.
  2. [Section 4.2, 'Temporal module'] The phrase 'learning tempral priors' contains a typo; it should read 'temporal priors'.
  3. [Section 2, Related Work] The sentence 'The introduction of the concept of optic flow can attributed to Gibson' is missing a 'be'; it should read 'can be attributed to Gibson'.
  4. [Table 9 caption] The caption contains a typo: 'offical' should be 'official'.
  5. [Section 1, Introduction] The phrase 'we solve awindow of flow problems simultaneously' is missing a space and possibly an article; it should be 'a window of flow problems'.
  6. [Figure 3] The throughput measurements in Figure 3 would benefit from specifying the hardware, batch size, and input resolution used for the throughput axis, since these details are not given in the main text.
  7. [Reference [18]] The reference 'TAG: Tracking at any granularity' appears without a venue or year; if it is a preprint, this should be stated.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AllTracker's claims are empirical, evaluated on held-out benchmarks; the protocol simplification is an unverified assumption, not a circular reduction.

full rationale

This paper is an empirical systems paper. The central claim—state-of-the-art dense point tracking at high resolution—is established by training on synthetic datasets (Kubric, FlyingThings++, PointOdyssey, optical flow mixes) and evaluating on held-out real benchmarks (BADJA, CroHD, TAPVid, DriveTrack, EgoPoints, Horse10, Kinetics, RGB-Stacking, RoboTAP). No metric, loss, or architecture component is defined in terms of the evaluation benchmarks, and no parameter is fitted to the test set. The architecture ablations are genuine controlled experiments, and the SOTA comparison against CoTracker3 is made transparent: the paper discloses that its cheaper CoTracker3 protocol (all queries at once) over-estimates CoTracker3 accuracy, which is a conservative choice. The one load-bearing assumption is the Section 4 statement that on large datasets tracking only points from the first available query frame and trimming to 600 frames 'gives similar results to using all possible queries and full video lengths.' This is an unverified empirical assertion that could affect the ranking if it interacts differently across methods, but it is not circular: it does not define AllTracker's output in terms of a fitted parameter, nor does it reduce the result to its own inputs by construction. The paper's self-citations (e.g., PIPs, PointOdyssey, CoTracker) are used as prior techniques and data sources, not as loaded uniqueness theorems that force the conclusion. Accordingly, the circularity score is 0; the protocol-equivalence concern belongs to correctness risk, not circularity.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central empirical claim rests on hand-chosen hyperparameters and architecture choices, all reported in the paper, plus domain assumptions about benchmark annotations, evaluation protocols, synthetic-to-real transfer, and upsampling precision. No new physical entities are introduced.

free parameters (7)
  • alpha (trajectory loss scale) = 0.05
    Hand-chosen scaling of the L1 trajectory loss (Eq. 1); affects accuracy and training stability.
  • gamma (refinement step weighting) = 0.8
    Hand-chosen exponent weighting later refinement steps in the loss (Eq. 1).
  • window size S = 16
    Subsequence length for temporal attention and sliding window; central to the architecture (Sec 3.1).
  • correlation radius R = 4
    Patch radius for indexing the correlation volume (Sec 3.4); ablation shows it is optimal among tested values.
  • correlation pyramid scales = 5
    Number of pyramid levels in the multi-scale correlation volume (Sec 3.2); ablation shows best, 6 scales causes runtime error.
  • refinement iterations at inference = 4
    Number of recurrent refinement steps; accuracy peaks at 5 and 4 chosen for efficiency (Supplementary Fig. 7).
  • number of space-time blocks = 3
    Number of interleaved spatial and temporal blocks in the recurrent module; ablation shows 3 is best among tested values.
assumptions (4)
  • domain assumption Ground-truth point annotations in the nine benchmarks are accurate enough that a difference of about 1 point in delta-avg reflects a real performance difference.
    The state-of-the-art claim rests on small margins (e.g., 66.1 vs 65.0). If annotation noise is comparable to the margin, rankings could be unreliable. See Tables 1 and 2.
  • domain assumption Tracking only the first available query frame and trimming video lengths yields similar results to the full evaluation protocol.
    Stated without supporting comparison in Section 4. If false, benchmark rankings across methods could change.
  • domain assumption Training on synthetic datasets transfers to real-world videos.
    All training data is synthetic (Kubric, FlyingThings, etc.); evaluation is on real videos. The paper relies on this transfer but does not include real-video training or pseudo-labelling.
  • domain assumption Pixel-shuffle upsampling from 1/8-resolution estimates preserves the spatial precision needed for full-resolution dense tracking.
    The architecture only produces full-resolution output through the upsampling layer (Sec 3.4). If this loses subpixel accuracy, the high-resolution claim weakens.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AllTracker: Efficient Dense Point Tracking at High Resolution." pith.science (2026). https://pith.science/paper/IWSMCL4B

@misc{pith2026250607310,
  author       = {Pith},
  title        = {Pith review of: AllTracker: Efficient Dense Point Tracking at High Resolution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IWSMCL4B}},
  note         = {Machine review of arXiv:2506.07310}
}
read the original abstract

We introduce AllTracker: a model that estimates long-range point tracks by way of estimating the flow field between a query frame and every other frame of a video. Unlike existing point tracking methods, our approach delivers high-resolution and dense (all-pixel) correspondence fields, which can be visualized as flow maps. Unlike existing optical flow methods, our approach corresponds one frame to hundreds of subsequent frames, rather than just the next frame. We develop a new architecture for this task, blending techniques from existing work in optical flow and point tracking: the model performs iterative inference on low-resolution grids of correspondence estimates, propagating information spatially via 2D convolution layers, and propagating information temporally via pixel-aligned attention layers. The model is fast and parameter-efficient (16 million parameters), and delivers state-of-the-art point tracking accuracy at high resolution (i.e., tracking 768x1024 pixels, on a 40G GPU). A benefit of our design is that we can train jointly on optical flow datasets and point tracking datasets, and we find that doing so is crucial for top performance. We provide an extensive ablation study on our architecture details and training recipe, making it clear which details matter most. Our code and model weights are available at https://alltracker.github.io

Figures

Figures reproduced from arXiv: 2506.07310 by the authors.

Figure 1
Figure 1. AllTracker estimates high-resolution optical flow be￾tween a “query frame” and every other frame of a video, using a sliding-window strategy. Point samples from these outputs can be interpreted as long-term point trajectories. occlusions [40]. An attractive shortcut here is to directly compute the flow between a reference frame and each other frame, and thus track drift-free and across occlusions, but estimating “lo… view at source ↗
Figure 2
Figure 2. AllTracker architecture. First, we compute feature maps for all frames, and copy the zeroth (query) feature map to every timestep, and compute multi-scale cost volumes. Then, we iterate a recurrent module, which references the query feature map and cost volume pyramid at each timestep, and estimates a low￾resolution correspondence field, using interleaved 2D convolutions and pixel-aligned temporal attentions. The ou… view at source ↗
Figure 3
Figure 3. AllTracker (top right corner) delivers accurate multi [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: AllTracker produces accurate displacement fields across dozens of frames. Prior optical flow methods struggle to make correspondences across wide time gaps, while our model uses temporal priors to resolve the ambiguity; prior point trackers take multiple minutes to pro…
Figure 5
Figure 5. Figure 5: Detailed view of iterative refinement block. We con￾solidate data from visibility, confidence, correlation, motion, and appearance features into a single feature map, then interleave con￾volutional spatial blocks and pixel-aligned temporal blocks, and output revisions …
Figure 6
Figure 6. Figure 6: Visualization of dense correspondence maps produced by all models. On the far left column we show the ground truth trajectories overlaid on the first frame of the input video, with blue-to-green colormap. (Note that a ground truth flow map does not exist in this data.)…
Figure 7
Figure 7. Figure 7: Accuracy over inference steps. Accuracy rises quickly [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 56 canonical work pages

  1. [1]

    Drivetrack: A benchmark for long-range point tracking in real-world videos

    Arjun Balasingam, Joseph Chandler, Chenning Li, Zhoutong Zhang, and Hari Balakrishnan. Drivetrack: A benchmark for long-range point tracking in real-world videos. In CVPR,

  2. [2]

    Context-pips: Persistent inde- pendent particles demands context features

    Weikang Bian, Zhaoyang Huang, Xiaoyu Shi, Yitong Dong, Yijin Li, and Hongsheng Li. Context-pips: Persistent inde- pendent particles demands context features. NeurIPS, 2024. 3

    work page 2024

  3. [3]

    Creatures great and SMAL: Recovering the shape and motion of animals from video

    Benjamin Biggs, Thomas Roddick, Andrew Fitzgibbon, and Roberto Cipolla. Creatures great and SMAL: Recovering the shape and motion of animals from video. In ACCV, 2018. 5, 6, 8, 12, 13

    work page 2018

  4. [4]

    Large displacement optical flow: descriptor matching in variational motion estimation

    Thomas Brox and Jitendra Malik. Large displacement optical flow: descriptor matching in variational motion estimation. TPAMI, 33(3):500–513, 2010. 2

    work page 2010

  5. [5]

    Flowtrack: Revisiting optical flow for long-range dense tracking

    Seokju Cho, Jiahui Huang, Seungryong Kim, and Joon-Young Lee. Flowtrack: Revisiting optical flow for long-range dense tracking. In CVPR, 2024. 2

    work page 2024

  6. [6]

    Local all-pair correspondence for point tracking

    Seokju Cho, Jiahui Huang, Jisu Nam, Honggyu An, Seungry- ong Kim, and Joon-Young Lee. Local all-pair correspondence for point tracking. In ECCV, 2025. 3, 5, 6

    work page 2025

  7. [7]

    Dense long-term motion estimation via statistical multi-step flow

    Pierre-Henri Conze, Philippe Robert, Tomas Crivelli, and Luce Morin. Dense long-term motion estimation via statistical multi-step flow. In VISAPP, 2014. 2

    work page 2014

  8. [8]

    Multi-reference combinatorial strategy towards longer long-term dense motion estimation

    Pierre-Henri Conze, Philippe Robert, Tomas Crivelli, and Luce Morin. Multi-reference combinatorial strategy towards longer long-term dense motion estimation. Computer Vision and Image Understanding, 150:66–80, 2016

    work page 2016

Show all 61 references
  1. [9]

    From optical flow to dense long term correspon- dences

    Tomas Crivelli, Pierre-Henri Conze, Philippe Robert, and Patrick Pérez. From optical flow to dense long term correspon- dences. In International Conference on Image Processing,

  2. [10]

    Egopoints: Advancing point tracking for ego- centric videos

    Ahmad Darkhalil, Rhodri Guerrier, Adam W Harley, and Dima Damen. Egopoints: Advancing point tracking for ego- centric videos. arXiv:2412.04592, 2024. 5, 6, 12

    2024 arXiv

  3. [11]

    TAP-Vid: A benchmark for tracking any point in a video

    Carl Doersch, Ankush Gupta, Larisa Markeeva, Adrià Re- casens, Lucas Smaira, Yusuf Aytar, João Carreira, Andrew Zisserman, and Yi Yang. TAP-Vid: A benchmark for tracking any point in a video. In NeurIPS Datasets and Benchmarks,

  4. [12]

    TAPIR: Tracking any point with per-frame initialization and temporal refinement

    Carl Doersch, Yi Yang, Mel Vecerik, Dilara Gokay, Ankush Gupta, Yusuf Aytar, Joao Carreira, and Andrew Zisserman. TAPIR: Tracking any point with per-frame initialization and temporal refinement. In ICCV, 2023. 1, 3, 4, 8

    2023

  5. [13]

    Bootstap: Bootstrapped training for tracking-any-point

    Carl Doersch, Pauline Luc, Yi Yang, Dilara Gokay, Skanda Koppula, Ankush Gupta, Joseph Heyward, Ignacio Rocco, Ross Goroshin, João Carreira, et al. Bootstap: Bootstrapped training for tracking-any-point. In ACCV, 2024. 3, 5, 6, 7

    2024

  6. [14]

    Flownet: Learning optical flow with convolutional networks

    Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning optical flow with convolutional networks. In ICCV, 2015. 2, 3, 5

    2015

  7. [15]

    Vision meets robotics: The kitti dataset

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The kitti dataset. Interna- tional Journal of Robotics Research, 2013. 3, 5

    2013

  8. [16]

    The perception of the visual world

    James J Gibson. The perception of the visual world. 1950. 1, 2

    1950

  9. [17]

    Kubric: A scalable dataset generator

    Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapra- gasam, Florian Golemo, Charles Herrmann, et al. Kubric: A scalable dataset generator. In CVPR, 2022. 3, 5

    2022

  10. [18]

    TAG: Tracking at any granularity

    Adam Harley, Yang You, Yang Zheng, Xinglong Sun, Nikhil Raghuraman, Sheldon Liang, Wen-Hsuan Chu, Suya You, Achal Dave, Pavel Tokmakov, et al. TAG: Tracking at any granularity. 2024. 8

    2024

  11. [19]

    Particle video revisited: Tracking through occlusions using point trajectories

    Adam W Harley, Zhaoyuan Fang, and Katerina Fragkiadaki. Particle video revisited: Tracking through occlusions using point trajectories. In ECCV, 2022. 1, 2, 3, 5, 8

    2022

  12. [20]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016. 2

    2016

  13. [21]

    Determining optical flow

    Berthold KP Horn and Brian G Schunck. Determining optical flow. Artificial intelligence, 17(1-3):185–203, 1981. 2

    1981

  14. [22]

    Flownet 2.0: Evolu- tion of optical flow estimation with deep networks

    Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. Flownet 2.0: Evolu- tion of optical flow estimation with deep networks. In CVPR,

  15. [23]

    Dy- namicstereo: Consistent dynamic depth from stereo videos

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Dy- namicstereo: Consistent dynamic depth from stereo videos. CVPR, 2023. 5

    2023

  16. [24]

    Co- tracker3: Simpler and better point tracking by pseudo- labelling real videos

    Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- tracker3: Simpler and better point tracking by pseudo- labelling real videos. arXiv:2410.11831, 2024. 1, 3, 4, 5, 6, 8, 12

    2024 arXiv

  17. [25]

    Co- Tracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- Tracker: It is better to track together. In ECCV, 2024. 1, 2, 3, 5, 6, 8

    2024

  18. [26]

    The HCI benchmark suite: Stereo and flow ground truth with uncertainties for urban autonomous driving

    Daniel Kondermann, Rahul Nair, Katrin Honauer, Karsten Krispin, Jonas Andrulis, Alexander Brock, Burkhard Gusse- feld, Mohsen Rahimimoghaddam, Sabine Hofmann, Claus Brenner, et al. The HCI benchmark suite: Stereo and flow ground truth with uncertainties for urban autonomous dr...

    2016

  19. [27]

    Dense optical tracking: connecting the dots

    Guillaume Le Moing, Jean Ponce, and Cordelia Schmid. Dense optical tracking: connecting the dots. In CVPR, 2024. 1, 3, 13

    2024

  20. [28]

    Beyond pick-and-place: Tackling robotic stacking of diverse shapes

    Alex X Lee, Coline Manon Devin, Yuxiang Zhou, Thomas Lampe, Konstantinos Bousmalis, Jost Tobias Springenberg, Arunkumar Byravan, Abbas Abdolmaleki, Nimrod Gileadi, David Khosid, et al. Beyond pick-and-place: Tackling robotic stacking of diverse shapes. In CoRL, 2021. 5, 6, 12

    2021

  21. [29]

    TAPTRv2: Attention-based position update improves tracking any point

    Hongyang Li, Hao Zhang, Shilong Liu, Zhaoyang Zeng, Feng Li, Tianhe Ren, Bohan Li, and Lei Zhang. TAPTRv2: Attention-based position update improves tracking any point. In NeurIPS, 2024. 3

    2024

  22. [30]

    TAPTR: Tracking any point with transformers as detection

    Hongyang Li, Hao Zhang, Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, and Lei Zhang. TAPTR: Tracking any point with transformers as detection. In ECCV, 2024. 3 9

    2024

  23. [31]

    A convnet for the 2020s

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feicht- enhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In CVPR, 2022. 4, 5, 7, 8

    2022

  24. [32]

    An iterative image registration technique with an application to stereo vision

    Bruce D Lucas, Takeo Kanade, et al. An iterative image registration technique with an application to stereo vision. In IJCAI, 1981. 2

    1981

  25. [33]

    Pretraining boosts out-of-domain robustness for pose estimation

    Alexander Mathis, Thomas Biasi, Steffen Schneider, Mert Yuksekgonul, Byron Rogers, Matthias Bethge, and Macken- zie W Mathis. Pretraining boosts out-of-domain robustness for pose estimation. In WACV, 2021. 5, 6, 8, 12, 13

    2021

  26. [34]

    A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation

    Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In CVPR, 2016. 3, 5

    2016

  27. [35]

    Spring: A high-resolution high- detail dataset and benchmark for scene flow, optical flow and stereo

    Lukas Mehl, Jenny Schmalfuss, Azin Jahedi, Yaroslava Nali- vayko, and Andrés Bruhn. Spring: A high-resolution high- detail dataset and benchmark for scene flow, optical flow and stereo. In CVPR, 2023. 3, 5

    2023

  28. [36]

    Mft: Long-term tracking of every pixel

    Michal Neoral, Jonáš Šer`ych, and Jiˇrí Matas. Mft: Long-term tracking of every pixel. In WACV, 2024. 2

    2024

  29. [37]

    DELTA: Dense efficient long-range 3d tracking for any video

    Tuan Duc Ngo, Peiye Zhuang, Chuang Gan, Evangelos Kalogerakis, Sergey Tulyakov, Hsin-Ying Lee, and Chaoyang Wang. DELTA: Dense efficient long-range 3d tracking for any video. In ICLR, 2025. 1, 2, 5, 6, 8

    2025

  30. [38]

    Optic flow: a history

    Diederick C Niehorster. Optic flow: a history. i-Perception, 12(6):20416695211055766, 2021. 2

    2021

  31. [39]

    Playing for benchmarks

    Stephan R Richter, Zeeshan Hayder, and Vladlen Koltun. Playing for benchmarks. In ICCV, 2017. 3, 5

    2017

  32. [40]

    Sand and S

    P. Sand and S. Teller. Particle video: Long-range motion estimation using point trajectories. In CVPR, 2006. 1, 2

    2006

  33. [41]

    Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

    Wenzhe Shi, Jose Caballero, Ferenc Huszár, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In CVPR, 2016. 2, 3, 4

    2016

  34. [42]

    PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume

    Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In CVPR, 2018. 2, 13

    2018

  35. [43]

    Autoflow: Learning a better training set for optical flow

    Deqing Sun, Daniel Vlasic, Charles Herrmann, Varun Jampani, Michael Krainin, Huiwen Chang, Ramin Zabih, William T Freeman, and Ce Liu. Autoflow: Learning a better training set for optical flow. In CVPR, 2021. 3, 5

    2021

  36. [44]

    Re- fining pre-trained motion models

    Xinglong Sun, Adam W Harley, and Leonidas J Guibas. Re- fining pre-trained motion models. In ICRA, 2024. 3

    2024

  37. [45]

    Dense point trajectories by GPU-accelerated large displace- ment optical flow

    Narayanan Sundaram, Thomas Brox, and Kurt Keutzer. Dense point trajectories by GPU-accelerated large displace- ment optical flow. In ECCV, 2010. 2

    2010

  38. [46]

    Tracking pedestrian heads in dense crowd

    Ramana Sundararaman, Cedric De Almeida Braga, Eric Marc- hand, and Julien Pettre. Tracking pedestrian heads in dense crowd. In CVPR, 2021. 5, 6, 8, 12, 13

    2021

  39. [47]

    RAFT: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. RAFT: Recurrent all-pairs field transforms for optical flow. In ECCV, 2020. 2, 4, 5, 6, 8, 13

    2020

  40. [48]

    Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lu- cas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy

    Ilya O. Tolstikhin, Neil Houlsby, Alexander Kolesnikov, Lu- cas Beyer, Xiaohua Zhai, Thomas Unterthiner, Jessica Yung, Daniel Keysers, Jakob Uszkoreit, Mario Lucic, and Alexey Dosovitskiy. MLP-mixer: An all-mlp architecture for vision. arXiv:2105.01601, 2021. 7, 8

    2021 arXiv

  41. [49]

    Detection and tracking of point

    Carlo Tomasi and Takeo Kanade. Detection and tracking of point. IJCV, 9:137–154, 1991. 2

    1991

  42. [50]

    Dense trajectory fields: Consistent and efficient spatio-temporal pixel tracking

    Marc Tournadre, Catherine Soladié, Nicolas Stoiber, and Pierre-Yves Richard. Dense trajectory fields: Consistent and efficient spatio-temporal pixel tracking. In ACCV, 2024. 1, 2

    2024

  43. [51]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NeurIPS, 2017. 7, 8

    2017

  44. [52]

    RoboTAP: Tracking arbitrary points for few-shot visual imitation

    Mel Vecerik, Carl Doersch, Yi Yang, Todor Davchev, Yusuf Aytar, Guangyao Zhou, Raia Hadsell, Lourdes Agapito, and Jon Scholz. RoboTAP: Tracking arbitrary points for few-shot visual imitation. In ICRA, 2024. 5, 6, 8, 12, 13

    2024

  45. [53]

    TartanAir: A dataset to push the limits of visual slam

    Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. TartanAir: A dataset to push the limits of visual slam. In IROS, 2020. 3, 5

    2020

  46. [54]

    Sea-raft: Simple, efficient, accurate raft for optical flow

    Yihan Wang, Lahav Lipson, and Jia Deng. Sea-raft: Simple, efficient, accurate raft for optical flow. In ECCV, 2024. 2, 3, 5, 6, 7, 11, 13

    2024

  47. [55]

    Accflow: Backward accumulation for long-range optical flow

    Guangyang Wu, Xiaohong Liu, Kunming Luo, Xi Liu, Qingqing Zheng, Shuaicheng Liu, Xinyang Jiang, Guangtao Zhai, and Wenyi Wang. Accflow: Backward accumulation for long-range optical flow. In ICCV, 2023. 5, 6, 13

    2023

  48. [56]

    Spatialtracker: Tracking any 2d pixels in 3d space

    Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng, Yujun Shen, and Xiaowei Zhou. Spatialtracker: Tracking any 2d pixels in 3d space. In CVPR, 2024. 8

    2024

  49. [57]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In CVPR, 2022. 7, 13

    2022

  50. [58]

    Accurate optical flow via direct cost volume processing

    Jia Xu, René Ranftl, and Vladlen Koltun. Accurate optical flow via direct cost volume processing. In CVPR, 2017. 2

    2017

  51. [59]

    Harley, Bokui Shen, Gordon Wet- zstein, and Leonidas J

    Yang Zheng, Adam W. Harley, Bokui Shen, Gordon Wet- zstein, and Leonidas J. Guibas. Pointodyssey: A large-scale synthetic dataset for long-term point tracking. In ICCV, 2023. 1, 3, 5, 6, 8

    2023

  52. [60]

    TAPNext: Tracking any point (TAP) as next token prediction

    Artem Zholus, Carl Doersch, Yi Yang, Skanda Koppula, Vior- ica Patraucean, Xu Owen He, Ignacio Rocco, Mehdi SM Saj- jadi, Sarath Chandar, and Ross Goroshin. TAPNext: Tracking any point (TAP) as next token prediction. arXiv:2504.05579,

    arXiv

  53. [2025]

    space- time

    8 10 Supplementary Material A. Additional model details Recurrent module In the recurrent module, we compress and contextualize the input data in stages [54], following the design ideas of SEA-RAFT [54], as illustrated in Figure 5. We use parallel 2-layer CNNs (with 3 × 3 kern...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.