Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Track-On: Transformer-based Online Point Tracking with Memory

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A causal, frame-by-frame tracker matches offline point-tracking models that see the entire video, and beats them on TAP-Vid DAVIS.

desk verdict The online-SOTA claim is solid; the offline-competitive claim is real but overstated, resting on test-set-tuned memory sizes and no error bars. read the letter →

arxiv 2501.18487 v1 pith:EMDJGDGQ submitted 2025-01-30 cs.CV

Görkay Aydemir , Xiongyi Cai , Weidi Xie , Fatma Güney This is my paper · ORCID
classification cs.CV
keywords onlinepointtrackinglong-termtransformermemorymoduletemporalcorrespondenceTAP-Vidbenchmarkcausalvideoprocessingpatchclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Track-On is a point-tracking model that processes video one frame at a time, with no access to future frames, and still reaches the accuracy of offline models that consume the whole video or a large window. The paper's central claim is that long-term tracking does not need bidirectional temporal modeling: two memory modules, one storing local patches around recent predictions and one storing the track's history of query embeddings, carry the temporal information. On the TAP-Vid benchmark, the model reports AJ 65.0 and $\delta_{\mathrm{avg}}$ 78.0 on DAVIS, ahead of the best offline model CoTracker3 (64.5 and 76.7) trained on comparable data, and it sets a new state of the art among online trackers. The significance, if true, is that streaming applications such as robotics and augmented reality can get offline-level tracking without waiting for future frames.

What carries the argument

The two memory modules are the load-bearing mechanism. Spatial memory $M_s$ holds up to $K$ feature vectors extracted from small neighborhoods around the model's predictions in past frames; before the query decoder runs, the initial query is updated by attending to this memory, which the paper shows increases similarity to the true correspondence by 24% on DAVIS, directly countering feature drift. Context memory $M_c$ stores the decoded query embeddings from previous frames and is attended by the query decoder, providing the track's history including occlusion status. Both are FIFO queues of size $K=12$ at training, and at inference the temporal positional embeddings are linearly interpolated to extend the memory to $K_i$ up to 96, which lets a model trained on 24-frame clips track videos of hundreds of frames. The correspondence pipeline of multi-scale patch classification, top-$k$ re-ranking, and offset refinement carries the localization.

What would settle it

Run the TAP-Vid DAVIS evaluation with one fixed memory size and visibility threshold for all datasets (for example, $K_i=48$ and threshold 0.8) and no per-dataset selection; if the reported AJ 65.0 margin over CoTracker3 shrinks or reverses, the state-of-the-art claim depends on test-set hyperparameter selection. Separately, evaluate on videos longer than 96 frames at $K_i=96$: if accuracy collapses beyond the trained memory length, the untrained interpolation only covers the tested range.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that point tracking can be formulated as a causal matching problem solved by a transformer: points are treated as queries that attend to the current frame, the correspondence is found by classifying which patch contains the point using cosine similarity at multiple scales, a re-ranking step re-scores the top-$k$ candidate patches, and an offset head refines the chosen patch center to sub-patch precision. Temporal continuity comes from two FIFO memories — spatial memory, which stores the features around the model's recent predictions and updates the initial query to counteract feature drift, and context memory, which stores decoded query embeddings from past frames so the decoder can see the track's history, including occlusions. The paper reports that this design sets a new state of the art among online trackers and delivers superior or competitive results against offline approaches on seven datasets including TAP-Vid, with the DAVIS numbers exceeding the offline CoTracker3.

Load-bearing premise

The model is trained with a memory of 12 frames and then evaluated with memories of up to 96 frames created by linearly interpolating the temporal positional embeddings — a length extrapolation the model never trained on — and the per-dataset memory sizes and visibility thresholds are chosen without a described validation split.

Editorial extensions

If this is right

  • Streaming deployment: robotics, augmented reality, and video editing can use a tracker that emits predictions frame by frame with bounded memory and no future frames.
  • Long-video scalability: GPU memory stays in the 0.61–1.03 GB range as the inference memory grows from 12 to 96, so very long videos can be tracked with roughly constant per-frame cost.
  • A new paradigm: patch classification plus offset refinement is a viable alternative to the iterative regression used by prior point trackers.
  • Training and inference length are decoupled: a model trained on 24-frame clips tracks 250-frame videos because the learned memory positional embeddings extrapolate by interpolation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The per-dataset memory sizes (48 for DAVIS, 80 for RGB-Stacking, 96 for Kinetics) suggest the authors treat $K_i$ as an inference hyperparameter matched to video length and motion speed; a testable extension would be a rule for choosing $K_i$ from clip statistics rather than per-dataset tuning.
  • The paper's own failure analysis singles out thin surfaces and uniform regions, both symptoms of stride-4 feature resolution; combining the memory design with learnable upsampling is the natural follow-up the authors flag as future work.
  • Because the largest gains over offline models appear on textureless robotics scenes (RGB-Stacking, RoboTAP), the approach looks particularly suited to robot perception; whether the same memory settings transfer to egocentric internet video is an open question.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. Track-On proposes a causal, frame-by-frame transformer for long-term point tracking. Queries are initialized from DINOv2/ViT-Adapter features, refined by a query decoder, and matched to patches via multi-scale cosine similarity with top-k re-ranking; a separate offset head refines the patch center, and visibility/uncertainty heads produce per-point estimates. Temporal information is carried by two FIFO memory modules: spatial memory stores features around recent predictions, and context memory stores decoded queries from the track history. The model is trained on 24-frame TAP-Vid Kubric clips with memory size K=12; at inference, the temporal positional embeddings are linearly interpolated to extend memory up to Ki=96. Experiments report state-of-the-art results among online trackers on TAP-Vid DAVIS/RGB-Stacking/Kinetics and additional comparisons on RoboTAP, Dynamic Replica, BADJA, and PointOdyssey, with the headline claim of matching or surpassing offline models on several benchmarks.

Significance. The paper's strongest and best-supported result is the online-vs-online comparison: on DAVIS, Track-On improves AJ by 8.3 points over Online TAPIR (64.9/65.0 vs 56.7), a margin far larger than plausible run-to-run noise. The design is clean, the ablations isolate the contributions of re-ranking, offset prediction, the visibility head, and the two memory modules, and the memory-efficiency figures (0.73 GB, 16.8 FPS at Ki=48 on an A100) are a useful practical contribution. If the offline-competitive claim were fully supported, the paper would establish an important fact: causal streaming tracking can match the accuracy of bidirectional window/video-level models on standard benchmarks. The current evidence for that claim is weakened by per-dataset test-set selection of memory size and visibility threshold, by reliance on untrained long-context extrapolation, and by the absence of uncertainty estimates on small datasets. These issues are addressable and do not undermine the online SOTA contribution, but they need to be resolved before the broader claim can be accepted.

major comments (4)
  1. [Section 3.1, Appendix D, Fig. 12] The inference-time memory size Ki is selected per dataset on the evaluation sets. Section 3.1 sets Ki=48 for DAVIS, RoboTAP, and Dynamic Replica; Ki=80 for RGB-Stacking; Ki=96 for Kinetics and PointOdyssey; and Ki=12 for BADJA. Appendix D and Fig. 12 then sweep Ki directly on the TAP-Vid test sets and report that RGB-Stacking gains +2.9 AJ up to Ki=80 and DAVIS peaks at Ki=48. No validation split is described, so the long-video numbers in Tables 1, 4, and 5 are produced with hyperparameters selected on the evaluation data. This matters because the margins supporting the offline-competitive claim are small: 0.5 AJ over CoTracker3 on DAVIS and 1.2 AJ over LocoTrack on RoboTAP. Please select Ki on a validation split or by a principled length-based rule, report the resulting test numbers, and show how the conclusions change when Ki is fixed to 12 or chosen by a data-independent rule.
  2. [Section 2.3.3, Eqs. (8)-(9)] The model is trained with memory size K=12, and at inference the temporal positional embeddings gamma_s and gamma_c are linearly interpolated to sizes up to Ki=96. This is an untrained extrapolation, yet it is active in all long-video results that support the offline-competitive claim. The paper does not justify that linear interpolation preserves the memory read/write behavior learned at K=12; for example, the learned relative-position semantics of the embeddings may not transfer to unseen lengths. Please provide evidence that the interpolation is safe, ideally by training with K=16/32/48 or with variable-length memory on Kubric and comparing against the interpolated model on a held-out split, or by analyzing the stability of memory attention under interpolation.
  3. [Appendix B.1] The visibility threshold delta_v is set to 0.8 for all datasets except RGB-Stacking, where it is 0.5, justified only as 'due to its domain-specific characteristics.' No validation split is described for this choice. Since AJ and OA depend on the predicted visibility threshold, this is a hyperparameter that can change the reported numbers. Please report AJ/OA as a function of delta_v on a validation split, at least for DAVIS and RGB-Stacking, and state the chosen value before test evaluation.
  4. [Tables 1, 4, 5] All reported results are single-run point estimates without error bars, confidence intervals, or multiple seeds. The datasets are small (30 DAVIS videos, 50 RGB-Stacking videos, 20 Dynamic Replica sequences, 7 BADJA sequences, 12 PointOdyssey videos), and the headline offline comparisons are close: 65.0 vs 64.5 AJ on DAVIS, a 0.2 AJ gap on Kinetics, and a 1.2 AJ gap on RoboTAP. Without per-sequence variance, bootstrap intervals, or multiple seeds/checkpoints, it is not possible to assess whether these differences are meaningful. Please add uncertainty estimates for the main comparisons or state the variance across sequences.
minor comments (6)
  1. [Section 3.1] The sentence 'For DA VIS evaluation, we set the memory size Ki to 48, 80, and 96 for DA VIS, RGB-Stacking, and Kinetics' is grammatically unclear; it should say that for evaluation on DAVIS, RGB-Stacking, and Kinetics, Ki is set to 48, 80, and 96, respectively.
  2. [Table 1 caption] The phrase 'the former setting grants access to any frame regardless of video length' is confusing because the preceding text does not clearly establish two numbered settings; use 'offline models with video-level access' instead.
  3. [Figure 12] The y-axis is labeled 'AJ' but the text describes 'the change in AJ compared to the default training memory size of 12'; the axis should be labeled 'ΔAJ' or the caption should explicitly state the plotted quantity.
  4. [Appendix B.1] The description of 'random key masking with a 0.1 ratio during attention calculations for memory read operations' is not defined in the method section; please specify which attention computations are masked.
  5. [Appendix C, Dynamic Replica] The paragraph says 'we evaluate models using delta_vis, consistent with the TAP-Vid benchmark,' but delta_vis is not introduced in the main text and is not a TAP-Vid metric; please define it and provide a reference.
  6. [Reproducibility] No code or trained weights are released. Given that several headline margins are around 0.5 AJ, a reproducibility statement with code/weights would materially help independent verification of the reported numbers.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the model is trained on external ground truth and evaluated on held-out benchmarks; the one self-citation is a non-load-bearing related-work aside, though per-dataset test-set hyperparameter selection is a robustness caveat.

full rationale

Track-On's derivation is self-contained with respect to its performance claims. The architecture is defined by a visual encoder, query decoder, patch-classification/refinement heads, and two FIFO memories, trained with the composite loss in Eq. (6) against ground-truth trajectories and visibility from TAP-Vid Kubric; no equation feeds the reported benchmark scores back into the model definition. The headline results are evaluations on external benchmarks (TAP-Vid DAVIS/RGB-Stacking/Kinetics, RoboTAP, Dynamic Replica, BADJA, PointOdyssey), and the central online-SOTA claim is robust even without inference-time memory extension (Model-D, 64.9 AJ on DAVIS vs 56.7 for Online TAPIR). The only self-citation is the aside in Appendix A that DINO-Tracker's capability 'has been shown (Aydemir et al., 2024)', which is not load-bearing for any architectural choice or benchmark conclusion. The main caveats are evaluation robustness rather than circularity: Appendix D sweeps memory size Ki on the test sets and reports best values (e.g., RGB-Stacking gains +2.9 AJ up to K=80), and Appendix B.1 sets the visibility threshold to 0.5 specifically for RGB-Stacking, with no described validation split; these could shrink the offline-competitive margins but do not make the predictions equivalent to the chosen hyperparameters by construction. Score 2 reflects only the minor non-load-bearing self-citation; no central claim reduces to its inputs.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The central claim is an empirical benchmark claim, so the 'axioms' are domain assumptions about data and architecture rather than mathematical postulates. The main free parameters are inference-time memory sizes and the visibility threshold, which are tuned per evaluation dataset without a described validation protocol; this is the largest source of circularity burden. The memory modules are invented architectural components with no independent falsifiable handle outside the paper's own experiments.

free parameters (6)
  • Softmax temperature τ = 0.05
    Hand-set temperature in patch classification similarity; affects the sharpness of the spatial softmax and therefore patch selection.
  • Visibility threshold δv = 0.8 (0.5 for RGB-Stacking)
    Inference threshold for visible/occluded classification; chosen per dataset without reported validation protocol, and directly determines the OA metric.
  • Uncertainty threshold δu = 8 pixels
    Training-time threshold for labeling predictions as uncertain; influences uncertainty head supervision but not inference.
  • Loss weight λ = 3
    Weight for patch classification loss terms in the total loss (Eq. 6).
  • Inference-time memory size Ki = 48 (DAVIS), 80 (RGB-Stacking), 96 (Kinetics), 48 (RoboTAP), 12 (BADJA)
    Trained with K=12 and extended to per-dataset values; these choices directly affect long-video performance and are not tuned on a described validation split.
  • Top-k candidate patches = 16 (trained), varied at inference up to 64
    Number of patches considered in the re-ranking module; a hand-chosen architecture parameter.
assumptions (5)
  • domain assumption DINOv2 features with ViT-Adapter provide suitable representations for point correspondence at stride 4
    Invoked in Sec 2.2.1; the entire matching pipeline relies on these features, but no independent evidence is given that this backbone is optimal or sufficient for thin structures.
  • domain assumption Training on synthetic TAP-Vid Kubric (24-frame clips) generalizes to real long videos
    Sec 3.1 trains only on Kubric; evaluation is on DAVIS, Kinetics, RoboTAP, etc. The paper does not fine-tune on real data, yet claims competitive offline results.
  • ad hoc to paper Inference-time memory extension by linear interpolation of temporal positional embeddings preserves learned memory behavior for untrained lengths
    Sec 2.3.3 extends K=12 to Ki=96; no theoretical or empirical analysis of why linear interpolation is valid, only benchmark results.
  • domain assumption Ground-truth trajectories in Kubric provide correct patch labels for classification training
    Sec 2.2.3 uses patch labels from ground-truth trajectories; errors in these labels would propagate.
  • domain assumption Benchmark protocol (queried first, 256x256 downsampling) is appropriate
    Sec 3.1; standard protocol but affects comparability.
invented entities (2)
  • Spatial memory module (Ms)
    purpose: Stores local feature content around previous predictions to update query features, mitigating feature drift.
    Introduced in Sec 2.3.1; effectiveness only shown via in-paper ablations, no external falsifiable prediction.
  • Context memory module (Mc)
    purpose: Stores decoded query embeddings from past frames to provide appearance and occlusion history.
    Introduced in Sec 2.3.2; effectiveness only via in-paper ablations.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Track-On: Transformer-based Online Point Tracking with Memory." pith.science (2026). https://pith.science/paper/EMDJGDGQ

@misc{pith2026250118487,
  author       = {Pith},
  title        = {Pith review of: Track-On: Transformer-based Online Point Tracking with Memory},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EMDJGDGQ}},
  note         = {Machine review of arXiv:2501.18487}
}
read the original abstract

In this paper, we consider the problem of long-term point tracking, which requires consistent identification of points across multiple frames in a video, despite changes in appearance, lighting, perspective, and occlusions. We target online tracking on a frame-by-frame basis, making it suitable for real-world, streaming scenarios. Specifically, we introduce Track-On, a simple transformer-based model designed for online long-term point tracking. Unlike prior methods that depend on full temporal modeling, our model processes video frames causally without access to future frames, leveraging two memory modules -- spatial memory and context memory -- to capture temporal information and maintain reliable point tracking over long time horizons. At inference time, it employs patch classification and refinement to identify correspondences and track points with high accuracy. Through extensive experiments, we demonstrate that Track-On sets a new state-of-the-art for online models and delivers superior or competitive results compared to offline approaches on seven datasets, including the TAP-Vid benchmark. Our method offers a robust and scalable solution for real-time tracking in diverse applications. Project page: https://kuis-ai.github.io/track_on

Figures

Figures reproduced from arXiv: 2501.18487 by the authors.

Figure 1
Figure 1. Offline vs. Online Point Tracking. We propose an online model, tracking points frame-by￾frame (right), unlike the dominant offline paradigm where models require access to all frames within a sliding window or the entire video (left). In contrast, our approach allows for frame-by-frame tracking in videos of any length. To capture temporal information, we introduce two memory modules: spatial memory, which tracks chan… view at source ↗
Figure 2
Figure 2. Overview. We introduce Track-On, a simple transformer-based method for online, frame￾by-frame point tracking. The process involves three steps: (i) Visual Encoder, which extracts features from the given frame; (ii) Query Decoder, which decodes interest point queries using the frame’s features; (iii) Point Prediction (highlighted in light blue), where correspondences are estimated in a coarse-to-fine manner, first th… view at source ↗
Figure 3
Figure 3. Top-k Points. In certain cases, a patch with high similarity, though not the most similar, is closer to the ground-truth patch. The top-3 patch centers, ranked by similarity, are marked with dots, while the ground￾truth is represented by a diamond [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Offset Head. Starting with a rough estimation from patch classification (left), where lighter colors indicate higher correlation, we refine the prediction using the offset head (right). The selected patch center and the final prediction are marked by a blue dot and a r…
Figure 7
Figure 7. Figure 7: Memory Modules. Spatial memory Ms t−1 (left) is used to update the initial query q init from the first frame to q init t on the current frame. The goal is to resolve feature drift by storing the content around the model’s predictions in previous frames. Context memory …
Figure 8
Figure 8. Figure 8: Efficiency. Inference speed (frames per second, FPS) vs. maximum GPU memory usage (in GB) where color represents the performance in AJ for different memory sizes (indicated near the nodes), while tracking approximately 400 points on the DAVIS dataset. 4 RELATED WORK Tr…
Figure 9
Figure 9. Figure 9: Details of Different Modules. This figure describes the details for modules in our model: Query Decoder (Φq-dec), Offset Head (Φoff), Query Update Module (Φq-up), Spatial Memory Write Module (Φq-wr), and Visibility Head (Φvis). B.2 IMPLEMENTATION DETAILS All of our mod…
Figure 11
Figure 11. Figure 11: Multiscale Similarity Calculation. This figure illustrates the detailed process of computing multiscale simi￾larity between a given query qt and a feature map ht. The different levels of the feature map (h l t ) are generated by ap￾plying bilinear downsampling at vari…
Figure 12
Figure 12. Figure 12: Memory Size. The effect of varying extended memory sizes during inference, on TAP￾Vid DAVIS and TAP-Vid RGB-Stacking. 12 4 8 16 32 64 k 56 58 60 62 64 AJ [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 14
Figure 14. Figure 14: Similarity Ratio Score. The similarity ratio score ssr > 1 over frames for different tracks, demonstrates increased similarity with ground-truth location on the target frame when utilizing spatial memory. F FAILURE ANALYSIS We identify three common failure cases: (i) …
Figure 15
Figure 15. Figure 15: Common Failure Cases. We identify three common failure cases: tracking points on thin surfaces (a), fast motion or scene cuts (b), and localization on uniform areas (c). We visualize predictions with average error higher than 8 pixels, where predictions are shown as d…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BleedOrigin: Dynamic Bleeding Source Localization in Endoscopic Submucosal Dissection via Dual-Stage Detection and Tracking

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A new ESD bleeding-source dataset and a dual-stage detection-tracking framework report 96.85% onset, 70.24% source, and 96.11% tracking accuracy within defined tolerances.

  2. You Are Your Best Teacher: Semi-Supervised Surgical Point Tracking with Cycle-Consistent Self-Distillation

    cs.CV 2025-05 conditional novelty 5.0 of 10

    SurgTracker adapts synthetic-trained point trackers to surgical video with self-distillation and cycle-consistent pseudo-label filtering, improving STIR accuracy over CoTracker3 without any manual annotations.

Reference graph

Works this paper leans on

62 extracted references · 54 canonical work pages · cited by 2 Pith papers

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    o rkay Aydemir, Weidi Xie, and Fatma G \

    G \"o rkay Aydemir, Weidi Xie, and Fatma G \"u ney. Can visual foundation models achieve long-term point tracking? In Proc. of the European Conf. on Computer Vision (ECCV) Workshops, 2024

    work page 2024

  3. [3]

    Memory consolidation enables long-context video understanding

    Ivana Balazevic, Yuge Shi, Pinelopi Papalampidi, Rahma Chaabouni, Skanda Koppula, and Olivier J Henaff. Memory consolidation enables long-context video understanding. In Proc. of the International Conf. on Machine learning (ICML), 2024

    work page 2024

  4. [4]

    SIFT features tracking for video stabilization

    Sebastiano Battiato, Giovanni Gallo, Giovanni Puglisi, and Salvatore Scellato. SIFT features tracking for video stabilization. In Proc. of the International Conference on Image Analysis and Processing (ICIAP), 2007

    work page 2007

  5. [5]

    Creatures great and SMAL : Recovering the shape and motion of animals from video

    Benjamin Biggs, Thomas Roddick, Andrew Fitzgibbon, and Roberto Cipolla. Creatures great and SMAL : Recovering the shape and motion of animals from video. In Proc. of the Asian Conf. on Computer Vision (ACCV), 2019

    work page 2019

  6. [6]

    SST : Single-stream temporal action proposals

    Shyamal Buch, Victor Escorcia, Chuanqi Shen, Bernard Ghanem, and Juan Carlos Niebles. SST : Single-stream temporal action proposals. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2017

    work page 2017

  7. [7]

    A naturalistic open source movie for optical flow evaluation

    Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. In Proc. of the European Conf. on Computer Vision (ECCV), 2012

    work page 2012

  8. [8]

    End-to-end object detection with transformers

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers. In Proc. of the European Conf. on Computer Vision (ECCV), 2020

    work page 2020

Show all 62 references
  1. [9]

    GateHUB : Gated history unit with background suppression for online action detection

    Junwen Chen, Gaurav Mittal, Ye Yu, Yu Kong, and Mei Chen. GateHUB : Gated history unit with background suppression for online action detection. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2022 a

    2022

  2. [10]

    Vision transformer adapter for dense predictions

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. In Proc. of the International Conf. on Learning Representations (ICLR), 2022 b

    2022

  3. [11]

    XMem : Long-term video object segmentation with an atkinson-shiffrin memory model

    Ho Kei Cheng and Alexander G Schwing. XMem : Long-term video object segmentation with an atkinson-shiffrin memory model. In Proc. of the European Conf. on Computer Vision (ECCV), 2022

    2022

  4. [12]

    Local all-pair correspondence for point tracking

    Seokju Cho, Jiahui Huang, Jisu Nam, Honggyu An, Seungryong Kim, and Joon-Young Lee. Local all-pair correspondence for point tracking. In Proc. of the European Conf. on Computer Vision (ECCV), 2024

    2024

  5. [13]

    Online action detection

    Roeland De Geest, Efstratios Gavves, Amir Ghodrati, Zhenyang Li, Cees Snoek, and Tinne Tuytelaars. Online action detection. In Proc. of the European Conf. on Computer Vision (ECCV), 2016

    2016

  6. [14]

    TAP-Vid : A benchmark for tracking any point in a video

    Carl Doersch, Ankush Gupta, Larisa Markeeva, Adria Recasens, Lucas Smaira, Yusuf Aytar, Joao Carreira, Andrew Zisserman, and Yi Yang. TAP-Vid : A benchmark for tracking any point in a video. In Advances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  7. [15]

    TAPIR : Tracking any point with per-frame initialization and temporal refinement

    Carl Doersch, Yi Yang, Mel Vecerik, Dilara Gokay, Ankush Gupta, Yusuf Aytar, Joao Carreira, and Andrew Zisserman. TAPIR : Tracking any point with per-frame initialization and temporal refinement. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2023

    2023

  8. [16]

    BootsTAP : Bootstrapped training for tracking-any-point

    Carl Doersch, Pauline Luc, Yi Yang, Dilara Gokay, Skanda Koppula, Ankush Gupta, Joseph Heyward, Ignacio Rocco, Ross Goroshin, Jo \ a o Carreira, and Andrew Zisserman. BootsTAP : Bootstrapped training for tracking-any-point. Proc. of the Asian Conf. on Computer Vision (ACCV), 2024

    2024

  9. [17]

    Learning to discriminate information for online action detection

    Hyunjun Eun, Jinyoung Moon, Jongyoul Park, Chanho Jung, and Changick Kim. Learning to discriminate information for online action detection. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020

    2020

  10. [18]

    Motion adaptive pose estimation from compressed videos

    Zhipeng Fan, Jun Liu, and Yao Wang. Motion adaptive pose estimation from compressed videos. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2021

    2021

  11. [19]

    Particle video revisited: Tracking through occlusions using point trajectories

    Adam W Harley, Zhaoyuan Fang, and Katerina Fragkiadaki. Particle video revisited: Tracking through occlusions using point trajectories. In Proc. of the European Conf. on Computer Vision (ECCV), 2022

    2022

  12. [20]

    A twofold siamese network for real-time object tracking

    Anfeng He, Chong Luo, Xinmei Tian, and Wenjun Zeng. A twofold siamese network for real-time object tracking. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018

    2018

  13. [21]

    Motion estimation methods for video compression—a review

    Radu S Jasinschi, T Na Veen, et al. Motion estimation methods for video compression—a review. Journal of the Franklin Institute, 1998

    1998

  14. [22]

    DynamicStereo : Consistent dynamic depth from stereo videos

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. DynamicStereo : Consistent dynamic depth from stereo videos. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2023

    2023

  15. [23]

    CoTracker3 : Simpler and better point tracking by pseudo-labelling real videos

    Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. CoTracker3 : Simpler and better point tracking by pseudo-labelling real videos. arXiv preprint arXiv:2410.11831, 2024 a

    2024 arXiv

  16. [24]

    CoTracker : It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. CoTracker : It is better to track together. In Proc. of the European Conf. on Computer Vision (ECCV), 2024 b

    2024

  17. [25]

    MoViNets : Mobile video networks for efficient video recognition

    Dan Kondratyuk, Liangzhe Yuan, Yandong Li, Li Zhang, Mingxing Tan, Matthew Brown, and Boqing Gong. MoViNets : Mobile video networks for efficient video recognition. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021

    2021

  18. [26]

    Few-shot panoptic segmentation with foundation models

    Markus Käppeler, Kürsat Petek, Niclas Vödisch, Wolfram Burgard, and Abhinav Valada. Few-shot panoptic segmentation with foundation models. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2024

    2024

  19. [27]

    Video stabilization using robust feature trajectories

    Ken-Yi Lee, Yung-Yu Chuang, Bing-Yu Chen, and Ming Ouhyoung. Video stabilization using robust feature trajectories. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2009

    2009

  20. [28]

    TAPTRv2 : Attention-based position update improves tracking any point

    Hongyang Li, Hao Zhang, Shilong Liu, Zhaoyang Zeng, Feng Li, Tianhe Ren, Bohan Li, and Lei Zhang. TAPTRv2 : Attention-based position update improves tracking any point. In Advances in Neural Information Processing Systems (NeurIPS), 2024 a

    2024

  21. [29]

    TAPTR : Tracking any point with transformers as detection

    Hongyang Li, Hao Zhang, Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, and Lei Zhang. TAPTR : Tracking any point with transformers as detection. In Proc. of the European Conf. on Computer Vision (ECCV), 2024 b

    2024

  22. [30]

    Video object segmentation with adaptive feature bank and uncertain-region refinement

    Yongqing Liang, Xin Li, Navid Jafari, and Jim Chen. Video object segmentation with adaptive feature bank and uncertain-region refinement. In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  23. [31]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In Proc. of the International Conf. on Learning Representations (ICLR), 2019

    2019

  24. [32]

    Pose estimation for augmented reality: a hands-on survey

    Eric Marchand, Hideaki Uchiyama, and Fabien Spindler. Pose estimation for augmented reality: a hands-on survey. In IEEE Trans. on Visualization and Computer Graphics (VCG), 2015

    2015

  25. [33]

    A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation

    Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In Proc. IEEE Conf. on Computer Vision and Pattern Recognitio...

    2016

  26. [34]

    MFT : Long-term tracking of every pixel

    Michal Neoral, Jon \'a s S er \`y ch, and Ji r \' Matas. MFT : Long-term tracking of every pixel. In Proc. of the IEEE Winter Conference on Applications of Computer Vision (WACV), 2024

    2024

  27. [35]

    Dynamic kernel distillation for efficient pose estimation in videos

    Xuecheng Nie, Yuncheng Li, Linjie Luo, Ning Zhang, and Jiashi Feng. Dynamic kernel distillation for efficient pose estimation in videos. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2019

    2019

  28. [36]

    Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang-Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicolas B...

    2024

  29. [37]

    Particle video: Long-range motion estimation using point trajectories

    Peter Sand and Seth Teller. Particle video: Long-range motion estimation using point trajectories. In International Journal of Computer Vision (IJCV), 2008

    2008

  30. [38]

    DynOMo : Online point tracking by dynamic online monocular gaussian reconstruction

    Jenny Seidenschwarz, Qunjie Zhou, Bardienus Duisterhof, Deva Ramanan, and Laura Leal-Taix \'e . DynOMo : Online point tracking by dynamic online monocular gaussian reconstruction. In Proc. of the International Conf. on 3D Vision (3DV), 2025

    2025

  31. [39]

    Online real-time multiple spatiotemporal action localisation and prediction

    Gurkirt Singh, Suman Saha, Michael Sapienza, Philip HS Torr, and Fabio Cuzzolin. Online real-time multiple spatiotemporal action localisation and prediction. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2017

    2017

  32. [40]

    PWC-Net : Cnns for optical flow using pyramid, warping, and cost volume

    Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz. PWC-Net : Cnns for optical flow using pyramid, warping, and cost volume. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2018

    2018

  33. [41]

    RAFT : Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. RAFT : Recurrent all-pairs field transforms for optical flow. In Proc. of the European Conf. on Computer Vision (ECCV), 2020

    2020

  34. [42]

    DINO-Tracker : Taming DINO for self-supervised point tracking in a single video

    Narek Tumanyan, Assaf Singer, Shai Bagon, and Tali Dekel. DINO-Tracker : Taming DINO for self-supervised point tracking in a single video. In Proc. of the European Conf. on Computer Vision (ECCV), 2024

    2024

  35. [43]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), 2017

    2017

  36. [44]

    RoboTAP : Tracking arbitrary points for few-shot visual imitation

    Mel Vecerik, Carl Doersch, Yi Yang, Todor Davchev, Yusuf Aytar, Guangyao Zhou, Raia Hadsell, Lourdes Agapito, and Jon Scholz. RoboTAP : Tracking arbitrary points for few-shot visual imitation. In Proc. IEEE International Conf. on Robotics and Automation (ICRA), 2023

    2023

  37. [45]

    Tracking everything everywhere all at once

    Qianqian Wang, Yen-Yu Chang, Ruojin Cai, Zhengqi Li, Bharath Hariharan, Aleksander Holynski, and Noah Snavely. Tracking everything everywhere all at once. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2023

    2023

  38. [46]

    OadTR : Online action detection with transformers

    Xiang Wang, Shiwei Zhang, Zhiwu Qing, Yuanjie Shao, Zhengrong Zuo, Changxin Gao, and Nong Sang. OadTR : Online action detection with transformers. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2021

    2021

  39. [47]

    Towards real-time multi-object tracking

    Zhongdao Wang, Liang Zheng, Yixuan Liu, Yali Li, and Shengjin Wang. Towards real-time multi-object tracking. In Proc. of the European Conf. on Computer Vision (ECCV), 2020

    2020

  40. [48]

    SpatialTracker : Tracking any 2D pixels in 3D space

    Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng, Yujun Shen, and Xiaowei Zhou. SpatialTracker : Tracking any 2D pixels in 3D space. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  41. [49]

    Temporal recurrent networks for online action detection

    Mingze Xu, Mingfei Gao, Yi-Ting Chen, Larry S Davis, and David J Crandall. Temporal recurrent networks for online action detection. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2019

    2019

  42. [50]

    Long short-term transformer for online action detection

    Mingze Xu, Yuanjun Xiong, Hao Chen, Xinyu Li, Wei Xia, Zhuowen Tu, and Stefano Soatto. Long short-term transformer for online action detection. In Advances in Neural Information Processing Systems (NeurIPS), 2021

    2021

  43. [51]

    Real-time object detection for streaming perception

    Jinrong Yang, Songtao Liu, Zeming Li, Xiaoping Li, and Jian Sun. Real-time object detection for streaming perception. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2022 a

    2022

  44. [52]

    Colar: Effective and efficient online action detection by consulting exemplars

    Le Yang, Junwei Han, and Dingwen Zhang. Colar: Effective and efficient online action detection by consulting exemplars. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2022 b

    2022

  45. [53]

    Unsupervised traffic accident detection in first-person videos

    Yu Yao, Mingze Xu, Yuchen Wang, David J Crandall, and Ella M Atkins. Unsupervised traffic accident detection in first-person videos. In Proc. IEEE International Conf. on Intelligent Robots and Systems (IROS), 2019

    2019

  46. [54]

    Flash-VStream: memory-based real-time understanding for long video streams

    Haoji Zhang, Yiqin Wang, Yansong Tang, Yong Liu, Jiashi Feng, Jifeng Dai, and Xiaojie Jin. Flash-VStream: memory-based real-time understanding for long video streams. arXiv preprint arXiv:2406.08085, 2024

    2024 arXiv

  47. [55]

    Learning to rehearse in long sequence memorization

    Zhu Zhang, Chang Zhou, Jianxin Ma, Zhijie Lin, Jingren Zhou, Hongxia Yang, and Zhou Zhao. Learning to rehearse in long sequence memorization. In Proc. of the International Conf. on Machine learning (ICML), 2021

    2021

  48. [56]

    a henb \

    Yue Zhao and Philipp Kr \"a henb \"u hl. Real-time online video detection with temporal smoothing transformers. In Proc. of the European Conf. on Computer Vision (ECCV), 2022

    2022

  49. [57]

    PointOdyssey : A large-scale synthetic dataset for long-term point tracking

    Yang Zheng, Adam W Harley, Bokui Shen, Gordon Wetzstein, and Leonidas J Guibas. PointOdyssey : A large-scale synthetic dataset for long-term point tracking. In Proc. of the IEEE International Conf. on Computer Vision (ICCV), 2023

    2023

  50. [58]

    Streaming dense video captioning

    Xingyi Zhou, Anurag Arnab, Shyamal Buch, Shen Yan, Austin Myers, Xuehan Xiong, Arsha Nagrani, and Cordelia Schmid. Streaming dense video captioning. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2024

    2024

  51. [59]

    Deformable DETR : Deformable transformers for end-to-end object detection

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable DETR : Deformable transformers for end-to-end object detection. In Proc. of the International Conf. on Learning Representations (ICLR), 2021

    2021

  52. [60]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  53. [61]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  54. [62]

    Specifically, we train our model on TAP-Vid Kubric, a synthetic dataset of 11k video sequences, each with a fixed length of 24 frames

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.