Pith. sign in

REVIEW 3 major objections 5 minor 63 references

MFTIQ: Multi-Flow Tracker with Independent Matching Quality Estimation

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MFTIQ separates flow quality from flow computation, so one tracker can wrap any off-the-shelf optical flow method and, with RoMa, match state-of-the-art point trackers while tracking every pixel.

desk verdict Solid plug-and-play dense tracker; the IQ module is a real step beyond MFT, but the universality claim and two reporting gaps need referee attention. read the letter →

arxiv 2411.09551 v1 pith:734KNUL5 submitted 2024-11-14 cs.CV

Jonas Serych , Michal Neoral , Jiri Matas This is my paper · ORCID
classification cs.CV
keywords denselong-termtrackingopticalflowchainingindependentqualityestimationocclusionpointTAP-Vidplug-and-playtrackermulti-flow
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes MFTIQ, a dense long-term point tracker built on multi-flow chaining. Its central claim is that correspondence quality and occlusion can be estimated by a separate 'Independent Quality' network that looks at an already-computed flow chain and the two frames it connects, rather than by modifying the optical flow network itself. Because of this separation, the tracker claims to work with any off-the-shelf optical flow method without retraining or architecture changes, and the paper shows that with the RoMa dense matcher it matches state-of-the-art trackers on TAP-Vid Davis while being substantially faster for dense tracking. If true, dense long-term tracking becomes an interchangeable layer whose accuracy improves automatically as optical flow methods improve.

What carries the argument

The Independent Quality (IQ) module is a small convolutional network that takes a candidate chained flow field together with the template and current frames and outputs an occlusion mask and a cost map. It replaces MFT's accumulation of per-flow uncertainties with a direct quality estimate of the whole chain. The local correlation cost volume, with a maximum displacement of three pixels at quarter resolution, is the only signal that compares warped current-frame features to template features; the cost map is a weighted average of five sigmoid classifiers trained on 1-to-5-pixel end-point-error thresholds. This design is what makes the tracker plug-and-play, because quality estimation no longer lives inside the optical flow network and can therefore be applied to any flow method that outputs dense displacement fields.

What would settle it

Take a real video with known ground-truth point trajectories and run MFTIQ with a deliberately degraded optical flow method whose errors are spatially coherent and larger than three pixels, so they fall outside the IQ cost-volume window. If the IQ module assigns low cost to such flows, the per-pixel chain selection in Eq. (3) will keep choosing them and the tracker will drift; observing that drift would falsify the claim that quality can be read from the local cost volume alone.

Watch

Extended reading notes

Core claim

MFTIQ's discovery is that the hard part of long-term flow chaining, deciding which chain of optical flows is trustworthy at each pixel, can be learned once and then applied to arbitrary flow methods. The Independent Quality module takes the chained flow field, the template frame, and the current frame; warps current-frame features by the flow; computes a local correlation cost volume with displacement up to three pixels at quarter resolution; and predicts both an occlusion map and a cost map. The cost map is trained as five binary classifiers against end-point-error thresholds of 1, 2, 3, 4, and 5 pixels, so it directly scores how far the chained flow is from ground truth. Per pixel, MFTIQ selects the chain with the lowest cost, with a large penalty for predicted occlusion. The paper shows that this selection avoids the drift of MFT's uncertainty chaining and that a single model trained on synthetic Kubric data transfers to RAFT, FlowFormer++, and unseen flows such as RoMa.

Load-bearing premise

The IQ network is trained only on synthetic Kubric image pairs, using flows from ground truth, RAFT, and a ground-truth-initialized FlowFormer++, and the whole argument assumes that the quality signal it learns, read from a small window of pixel similarities around each point, transfers to real videos and to unseen flow methods such as RoMa.

Editorial extensions

If this is right

  • Dense long-term tracking becomes an interchangeable layer: any current or future optical flow method can be plugged in, so users choose the speed/accuracy trade-off by choosing the flow estimator.
  • The tracker's accuracy should rise automatically as optical flow methods improve, because the quality module does not need to be retrained when the flow method changes.
  • A model trained on synthetic data with RAFT and FlowFormer++ flows transfers to unseen flows such as RoMa, which is direct evidence for the plug-and-play claim.
  • Because MFTIQ is causal and dense, it can run online and produce a trajectory for every pixel of the first frame, something that sparse trackers looking into the future cannot do.
  • Selecting chains by direct quality estimation reduces drift and false re-detections compared with MFT's accumulated uncertainty, as shown on TAP-Vid Davis and on the POT-210 rotation and scale subsets.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the per-pixel cost map could serve as a confidence signal for downstream fusion, such as merging tracker output with other correspondence sources, since it already scores how far each flow chain is from the 1-to-5-pixel ground-truth error range.
  • A testable extension would be to fine-tune the IQ module on real video or on flow-error distributions wider than Kubric's, to see whether cross-domain transfer improves even further.
  • Editorial observation: the 1-to-5-pixel thresholds are defined at 256-by-256 evaluation resolution; a version that regresses continuous per-pixel end-point error would test whether the coarse threshold classes are the right training target.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper presents MFTIQ, a dense long-term point tracker built on the MFT flow-chaining framework. MFTIQ computes candidate long-term flow fields by chaining optical flows over logarithmically spaced temporal deltas and, for each pixel, selects the best candidate using an Independent Quality (IQ) module. The IQ module is a neural network trained once on synthetic Kubric data with ground-truth, RAFT, and ground-truth-initialized FlowFormer++ flows; it takes a chained flow and the two images as input and predicts a per-pixel EPE-based cost map and occlusion map. The authors evaluate MFTIQ with 12 off-the-shelf optical flow methods on TAP-Vid DAVIS and report results on TAP-Vid Kinetics (partial), RoboTAP, and POT-210, including a homography-tracking variant. The main claims are that the IQ module decouples quality estimation from flow computation, enabling plug-and-play use with any optical flow method, and that MFTIQ matches state-of-the-art trackers while being faster for dense tracking.

Significance. The central idea is clean and the experiments largely support it for the tested family of supervised learning-based flow estimators: a single IQ network, trained only on synthetic data, improves MFT across several flow methods and transfers to RoMa, which was not seen during training. The ablations of delta sets (Table 7) and feature extractors (Table 6) are informative, and the public code and models are a plus. MFTIQ is also causal, unlike several sparse point trackers. If the plug-and-play property holds beyond the tested distribution, the contribution is practically valuable because users can swap optical flow backbones to trade speed for accuracy without retraining. The main limitations are the scope of the 'any off-the-shelf' claim and the fairness of the speed and Kinetics comparisons, which are addressable in revision.

major comments (3)
  1. [Abstract; Section 4.1; Section 3.2] The claim that MFTIQ 'can be employed with any off-the-shelf optical flow method' (Abstract) is stronger than the evidence in Table 1. All twelve flow estimators evaluated are supervised learning-based networks or dense matchers; no unsupervised or classical flow method is tested. The IQ module's only matching signal is a local correlation cost volume with a maximum displacement of ±3 px at quarter resolution (Section 3.2), so a flow method whose errors are coherent and locally consistent (e.g., smooth drift in textureless regions or repeated-texture hallucination) would not be captured by the learned EPE/occlusion classifiers, and the per-pixel selection in Eq. (3) could choose a poor delta. Please qualify the claim to supervised learning-based flow estimators with error statistics comparable to the training distribution, or add an out-of-family flow method to demonstrate the boundary.
  2. [Table 2 (footnote)] The Kinetics-first evaluation is only on the first 465 of 1189 sequences, while all competitor numbers in the same columns are for the full benchmark. A 465-sequence subset is not guaranteed to be representative, so the reported rank ('second-best' position accuracy) is not directly comparable. Please evaluate on the full Kinetics set or clearly label the result as partial, remove the rank claim, and state how the subset was selected.
  3. [Table 2; Section 4.1] The speed comparison underlying the claim that MFTIQ is 'significantly faster' in dense tracking is not well controlled. The PPS values in Table 2 are measured on different GPUs (RTX A5000, Tesla V100, A100, RTX 3090) as acknowledged in the footnote, and no definition of PPS is given for dense vs. sparse trackers. In the reported table, DOT has PPS = 2473 with slightly higher AJ than MFTIQ's 709, and MFT reports 10671 PPS, so the blanket speed claim is not supported by the table as presented. Please report end-to-end timings on a single GPU with a common protocol, define PPS precisely, and restrict the speed claim to the trackers for which the comparison is valid.
minor comments (5)
  1. [Section 3.2, Eq. (5)] The text calls E a 'weighted average' but Eq. (5) is a weighted sum with weights 1, 2, 4, 8, 16; since only relative ranking matters this does not affect the method, but the wording should be corrected.
  2. [Section 3.1 and Section 3.3] The delta set D = {1, 2, 4, 8, 16, 32, t-1} is defined twice; please consolidate the definitions to avoid confusion.
  3. [Figure 2] The notation in the diagram, such as 'F2 1→7←Q(F2 1→7,I1,I7)', is hard to parse; please make the subscript/superscript notation clearer in the caption.
  4. [Table 3 and Table 4] Please state in the main text which POT-210 ground-truth version is used for each table; the Table 4 caption says 'improved ground-truth from [46]', but Table 3 does not.
  5. [References] References [29] and [30] are the same paper (Le Moing et al., 'Dense optical tracking: connecting the dots') and should be merged.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the IQ module is trained on synthetic ground-truth EPE labels and evaluated with unseen optical flow methods; self-citations are non-load-bearing baselines.

full rationale

The paper's derivation chain is self-contained. The Independent Quality (IQ) module is trained on synthetic Kubric data with dense ground-truth optical flow, occlusion, and validity masks, using EPE thresholds as supervision (Eqs. 5-6). The cost map E is a learned predictor of EPE classes from the chained flow and the two images, not a quantity fitted to the TAP-Vid benchmark. At inference, E and O are used in Eq. (3) to select among candidate flow chains, and the resulting tracks are evaluated on TAP-Vid, RoboTAP, and POT-210. The RoMa optical flow method was not used in training the IQ network, and Table 1 includes multiple unseen flow methods, providing an external generalization check rather than a circular reduction. The authors' self-citations to MFT [39] and MFT-RoMa [25] are used as baselines and as prior chaining formulation; they are not invoked as unverified premises that force the paper's conclusions. The delta set and the constant M are hyperparameters chosen by ablation, not parameters fitted to the evaluation targets in a way that makes the reported predictions equivalent to their inputs. Overall, no step reduces by construction to a fitted quantity or to a self-citation chain.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

All free parameters are design choices or hyperparameters selected by the authors; none are fitted to the target benchmark metrics. The assumptions concern domain transfer from synthetic training data and the sufficiency of the local cost volume for quality prediction. No new physical entities or external objects are introduced.

free parameters (4)
  • EPE thresholds theta (1..5) = 1,2,3,4,5 px
    Used in Eq. (5) to construct the cost map E from five binary classifiers; chosen by the authors as design hyperparameters, not tuned on target benchmarks.
  • Cost map weights 2^(theta-1) = 1,2,4,8,16
    Weighting of the five EPE-threshold classifiers in Eq. (5); hand-chosen exponential weights.
  • Chain selection constant M = not specified (large)
    Large constant in Eq. (3) to prefer unoccluded chains over cost ordering; exact value not reported in the paper.
  • Delta set D = {1,2,4,8,16,32,t-1}
    Set of flow chain lengths; inherited from MFT, ablated in Table 7 showing sensitivity (e.g., base-4 set gives AJ 65.50 vs 65.67 for the default set).
assumptions (4)
  • domain assumption Synthetic Kubric data with replayed motions and random camera shake is representative of real video for learning flow quality
    Section 3.3: the IQ network is trained exclusively on Kubric; real-world generalization is assumed and only indirectly validated on TAP-Vid, Robotap, and POT-210.
  • domain assumption A local cost volume (displacement +-3 px at quarter resolution) between warped current features and template features carries enough information to predict EPE-threshold cost and occlusion
    Section 3.2: the correlation cost volume is the main matching signal feeding the cost and occlusion heads; no theoretical guarantee that it suffices for arbitrary flow error patterns.
  • domain assumption Error patterns of unseen optical flow methods are similar enough to RAFT/FlowFormer++/GT for the trained IQ network to generalize
    Training flows are GT, RAFT, and GT-initialized FlowFormer++; performance with RoMa and other methods is empirical evidence for this assumption, but the plug-and-play property is not guaranteed for any future flow method.
  • domain assumption Per-pixel independent chain selection is optimal or near-optimal for trajectory accuracy
    Eq. (3) selects the best delta per pixel with no spatial context or temporal smoothing; the design follows MFT and is not proven optimal.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MFTIQ: Multi-Flow Tracker with Independent Matching Quality Estimation." pith.science (2026). https://pith.science/paper/734KNUL5

@misc{pith2026241109551,
  author       = {Pith},
  title        = {Pith review of: MFTIQ: Multi-Flow Tracker with Independent Matching Quality Estimation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/734KNUL5}},
  note         = {Machine review of arXiv:2411.09551}
}
read the original abstract

In this work, we present MFTIQ, a novel dense long-term tracking model that advances the Multi-Flow Tracker (MFT) framework to address challenges in point-level visual tracking in video sequences. MFTIQ builds upon the flow-chaining concepts of MFT, integrating an Independent Quality (IQ) module that separates correspondence quality estimation from optical flow computations. This decoupling significantly enhances the accuracy and flexibility of the tracking process, allowing MFTIQ to maintain reliable trajectory predictions even in scenarios of prolonged occlusions and complex dynamics. Designed to be "plug-and-play", MFTIQ can be employed with any off-the-shelf optical flow method without the need for fine-tuning or architectural modifications. Experimental validations on the TAP-Vid Davis dataset show that MFTIQ with RoMa optical flow not only surpasses MFT but also performs comparably to state-of-the-art trackers while having substantially faster processing speed. Code and models available at https://github.com/serycjon/MFTIQ .

Figures

Figures reproduced from arXiv: 2411.09551 by the authors.

Figure 1
Figure 1. Dense long-term tracking – MFT and MFTIQ com￾parison. Visualisation of query positions (red) tracked from frame 1 to frame 80. MFTIQ generates a lower number of false re￾detections than MFT, especially on the grass in bottom-left, which was out-of-view on frame 1. Best viewed zoomed-in and in color. Multi-Flow Tracker (MFT) [39] revisited flow chaining [6, 7] for not only consecutive frames, but also for temporally … view at source ↗
Figure 2
Figure 2. Example of MFTIQ optical flow chaining strategy for estimating the flow between I1 and the current frame It=7. MFTIQ constructs a flow chain F ∆ 1→7 by going through an in￾termediate frame I7−∆. This is done for multiple values of ∆, here shown for ∆ = 2 and ∆ = 4. The most reliable flow chain F ∆⋆ 1→7 is selected independently in each pixel based on flow qual￾ity Q [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Overview of the Independent Quality (IQ) estima￾tion network. First, image features are extracted from the tem￾plate frame I1 and the current frame It. Then, the current frame features are warped using the positions given by the chained flow F ∆ 1→t . The now-aligned feature maps are compared with a lo￾cal (displacement up to ±3) correlation cost-volume. Finally a concatenation of the features extracted from both im… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Comparison of flow candidate selection in MFT (left) and MFTIQ (right). MFT often selects (bottom left) the direct optical flow, i.e. the flow chain with ∆ = t − 1 with probability increasing during the video. The probability of the selected direct flow to be accurate …
Figure 5
Figure 5. Figure 5: Probability of selecting OF with a given [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 50 canonical work pages

  1. [1]

    MAGSAC++, a fast, reliable and accurate robust estimator

    Daniel Barath, Jana Noskova, Maksym Ivashechkin, and Jiri Matas. MAGSAC++, a fast, reliable and accurate robust estimator. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1304–1312,

    work page

  2. [2]

    Visual object tracking using adaptive corre- lation filters

    David S Bolme, J Ross Beveridge, Bruce A Draper, and Yui Man Lui. Visual object tracking using adaptive corre- lation filters. In 2010 IEEE computer society conference on computer vision and pattern recognition, pages 2544–2550. IEEE, 2010. 1

    2010

  3. [3]

    A short note on the kinetics-700 human action dataset

    Joao Carreira, Eric Noland, Chloe Hillier, and Andrew Zis- serman. A short note on the kinetics-700 human action dataset. arXiv preprint arXiv:1907.06987, 2019. 6

    arXiv 1907

  4. [4]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308, 2017. 6

    work page 2017

  5. [5]

    FlowTrack: Revisiting optical flow for long- range dense tracking

    Seokju Cho, Jiahui Huang, Seungryong Kim, and Joon- Young Lee. FlowTrack: Revisiting optical flow for long- range dense tracking. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 19268–19277, 2024. 2, 3, 7

    work page 2024

  6. [6]

    Multi-reference combinatorial strategy towards longer long-term dense motion estimation

    Pierre-Henri Conze, Philippe Robert, Tomas Crivelli, and Luce Morin. Multi-reference combinatorial strategy towards longer long-term dense motion estimation. Computer Vision and Image Understanding, 150:66–80, 2016. 1, 3

    work page 2016

  7. [7]

    From optical flow to dense long term cor- respondences

    Tomas Crivelli, Pierre-Henri Conze, Philippe Robert, and Patrick Pérez. From optical flow to dense long term cor- respondences. In 2012 19th IEEE International Conference on Image Processing, pages 61–64. IEEE, 2012. 1, 3

    work page 2012

  8. [8]

    Atom: Accurate tracking by overlap max- imization

    Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. Atom: Accurate tracking by overlap max- imization. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition , pages 4660–4669,

    work page

Show all 63 references
  1. [9]

    J. Deng, W. Dong, R. Socher, L. J. Li, Kai Li, and Li Fei- Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, June 2009. 12

    2009

  2. [10]

    TAP-Vid: A bench- mark for tracking any point in a video

    Carl Doersch, Ankush Gupta, Larisa Markeeva, Adria Re- casens Continente, Lucas Smaira, Yusuf Aytar, Joao Car- reira, Andrew Zisserman, and Yi Yang. TAP-Vid: A bench- mark for tracking any point in a video. Advances in Neural Information Processing Systems, 2022. 1, 2, 5, 6, 7...

    2022

  3. [11]

    BootsTAP: Boot- strapped training for tracking-any-point, 2024

    Carl Doersch, Yi Yang, Dilara Gokay, Pauline Luc, Skanda Koppula, Ankush Gupta, Joseph Heyward, Ross Goroshin, João Carreira, and Andrew Zisserman. BootsTAP: Boot- strapped training for tracking-any-point, 2024. 1, 2, 7

    2024

  4. [12]

    TAPIR: Tracking any point with per-frame initialization and temporal refinement

    Carl Doersch, Yi Yang, Mel Vecerik, Dilara Gokay, Ankush Gupta, Yusuf Aytar, Joao Carreira, and Andrew Zisserman. TAPIR: Tracking any point with per-frame initialization and temporal refinement. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision (ICCV...

    2023

  5. [13]

    MemFlow: Optical flow es- timation and prediction with memory

    Qiaole Dong and Yanwei Fu. MemFlow: Optical flow es- timation and prediction with memory. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19068–19078, 2024. 2, 6, 12

    2024

  6. [14]

    Flownet: Learn- ing optical flow with convolutional networks

    Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Häusser, Caner Hazırba¸ s, Vladimir Golkov, Patrick van der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learn- ing optical flow with convolutional networks. InICCV, pages 2758–2766, Dec. 2015. 2, 4

    2015

  7. [15]

    DKM: Dense kernelized feature matching for geometry estimation

    Johan Edstedt, Ioannis Athanasiadis, Mårten Wadenbäck, and Michael Felsberg. DKM: Dense kernelized feature matching for geometry estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17765–17775, 2023. 3

    2023

  8. [16]

    RoMa: Revisiting robust losses for dense feature matching

    Johan Edstedt, Qiyu Sun, Georg Bökman, Mårten Waden- bäck, and Michael Felsberg. RoMa: Revisiting robust losses for dense feature matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19790–19800, 2024. 1, 2, 3, 6, 12

    2024

  9. [17]

    Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981

    Martin A Fischler and Robert C Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981. 8

    1981

  10. [18]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In Computer Vision and Pat- tern Recognition (CVPR), 2012 IEEE Conference on , pages 3354–3361, June 2012. 1

    2012

  11. [19]

    Kubric: A scalable dataset generator

    Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J Fleet, Dan Gnanapra- gasam, Florian Golemo, Charles Herrmann, et al. Kubric: A scalable dataset generator. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Re...

    2022

  12. [20]

    Particle video revisited: Tracking through occlusions using point trajectories

    Adam W Harley, Zhaoyuan Fang, and Katerina Fragkiadaki. Particle video revisited: Tracking through occlusions using point trajectories. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXII, pages 59–75. Springer, 2022. 2

    2022

  13. [21]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 4, 12

    2016

  14. [22]

    Determining opti- cal flow

    Berthold KP Horn and Brian G Schunck. Determining opti- cal flow. Artificial intelligence, 17(1-3):185–203, 1981. 2

    1981

  15. [23]

    Flowformer: A transformer architecture for optical flow

    Zhaoyang Huang, Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer: A transformer architecture for optical flow. Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part...

    2022

  16. [24]

    Flownet 2.0: Evolution of optical flow estimation with deep networks

    Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keu- per, Alexey Dosovitskiy, and Thomas Brox. Flownet 2.0: Evolution of optical flow estimation with deep networks. IEEE Conference on Computer Vision and Pattern Recog- nition (CVPR), 2017. 2

    2017

  17. [25]

    Dense matchers for dense tracking

    Tomáš Jelínek, Jonáš Šerých, and Jiˇrí Matas. Dense matchers for dense tracking. In Proceedings of the 27th Computer Vision Winter Workshop (CVWW 2024), 2024. 1, 2, 3, 6, 7

    2024

  18. [26]

    Tracking-learning-detection

    Zdenek Kalal, Krystian Mikolajczyk, and Jiri Matas. Tracking-learning-detection. IEEE transactions on pattern analysis and machine intelligence , 34(7):1409–1422, 2011. 1 9

    2011

  19. [27]

    Co- Tracker: It is better to track together

    Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Co- Tracker: It is better to track together. arXiv preprint arXiv:2307.07635, 2023. 1, 2, 3, 7

    2023 arXiv

  20. [28]

    The eighth visual object tracking vot2020 chal- lenge results

    Matej Kristan, Aleš Leonardis, Ji ˇrí Matas, Michael Fels- berg, Roman Pflugfelder, Joni-Kristian Kämäräinen, Martin Danelljan, Luka ˇCehovin Zajc, Alan Lukeži ˇc, Ondrej Dr- bohlav, et al. The eighth visual object tracking vot2020 chal- lenge results. In European Conference o...

    2020

  21. [29]

    Dense optical tracking: connecting the dots

    Guillaume Le Moing, Jean Ponce, and Cordelia Schmid. Dense optical tracking: connecting the dots. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19187–19197, 2024. 2

    2024

  22. [30]

    Dense optical tracking: Connecting the dots

    Guillaume Le Moing, Jean Ponce, and Cordelia Schmid. Dense optical tracking: Connecting the dots. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19187–19197, 2024. 3, 7

    2024

  23. [31]

    TAPTR: Tracking any point with transformers as detection

    Hongyang Li, Hao Zhang, Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, and Lei Zhang. TAPTR: Tracking any point with transformers as detection. arXiv preprint arXiv:2403.13042, 2024. 1, 7

    2024 arXiv

  24. [32]

    Centroid-based graph matching networks for planar object tracking

    Kunpeng Li, He Liu, and Tao Wang. Centroid-based graph matching networks for planar object tracking. Machine Vi- sion and Applications, 34(2):31, 2023. 8

    2023

  25. [33]

    Planar object tracking benchmark in the wild

    Pengpeng Liang, Haoxuanye Ji, Yifan Wu, Yumei Chai, Liming Wang, Chunyuan Liao, and Haibin Ling. Planar object tracking benchmark in the wild. Neurocomputing, 454:254–267, 2021. 8

    2021

  26. [34]

    Planar object tracking in the wild: A benchmark

    Pengpeng Liang, Yifan Wu, Hu Lu, Liming Wang, Chun- yuan Liao, and Haibin Ling. Planar object tracking in the wild: A benchmark. In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages 651–658. IEEE,

    2018

  27. [35]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. Communications of the ACM , 65(1):99–106, 2021. 3

    2021

  28. [36]

    RAPIDFlow: Recurrent Adaptable Pyramids with Iterative Decoding for Efficient Optical Flow Estimation

    Henrique Morimitsu, Xiaobin Zhu, Roberto M Cesar, Xi- angyang Ji, and Xu-Cheng Yin. RAPIDFlow: Recurrent Adaptable Pyramids with Iterative Decoding for Efficient Optical Flow Estimation. In 2024 IEEE International Con- ference on Robotics and Automation (ICRA) , pages 2946–

    2024

  29. [37]

    Recurrent partial kernel network for efficient optical flow estimation

    Henrique Morimitsu, Xiaobin Zhu, Xiangyang Ji, and Xu- Cheng Yin. Recurrent partial kernel network for efficient optical flow estimation. In Proceedings of the AAAI Confer- ence on Artificial Intelligence, volume 38, pages 4278–4286,

  30. [38]

    Multi-object monocular slam for dynamic environments

    Gokul B Nair, Swapnil Daga, Rahul Sajnani, Anirudha Ramesh, Junaid Ahmed Ansari, and K Madhava Krishna. Multi-object monocular slam for dynamic environments. arXiv preprint arXiv:2002.03528, 2020. 1

    2002 arXiv

  31. [39]

    MFT: Long-term tracking of every pixel

    Michal Neoral, Jonáš Šerých, and Ji ˇrí Matas. MFT: Long-term tracking of every pixel. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 6837–6847, 2024. 1, 2, 3, 4, 6, 7, 13

    2024

  32. [40]

    Continual oc- clusion and optical flow estimation

    Michal Neoral, Jan Šochman, and Ji ˇrí Matas. Continual oc- clusion and optical flow estimation. In Asian Conference on Computer Vision, pages 159–174. Springer, 2018. 2

    2018

  33. [41]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 4

    2023 arXiv

  34. [42]

    Online invariance selection for local feature de- scriptors

    Rémi Pautrat, Viktor Larsson, Martin R Oswald, and Marc Pollefeys. Online invariance selection for local feature de- scriptors. In European Conference on Computer Vision , pages 707–724. Springer, 2020. 8

    2020

  35. [43]

    Perazzi, J

    F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Computer Vision and Pattern Recognition, 2016. 1

    2016

  36. [44]

    The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675v2, 2017

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- beláez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675v2, 2017. 6

    2017 arXiv

  37. [45]

    Particle video: Long-range mo- tion estimation using point trajectories

    Peter Sand and Seth Teller. Particle video: Long-range mo- tion estimation using point trajectories. International Jour- nal of Computer Vision, 80:72–91, 2008. 2

    2008

  38. [46]

    Planar object tracking via weighted optical flow

    Jonáš Šerých and Ji ˇrí Matas. Planar object tracking via weighted optical flow. InProceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 1593–1602, 2023. 8

    2023

  39. [47]

    Videoflow: Exploiting temporal cues for multi-frame optical flow estimation

    Xiaoyu Shi, Zhaoyang Huang, Weikang Bian, Dasong Li, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Videoflow: Exploiting temporal cues for multi-frame optical flow estimation. arXiv preprint arXiv:2303.08340, 2023. 2

    2023 arXiv

  40. [48]

    FlowFormer++: Masked cost volume autoen- coding for pretraining optical flow estimation

    Xiaoyu Shi, Zhaoyang Huang, Dasong Li, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, Jifeng Dai, and Hongsheng Li. FlowFormer++: Masked cost volume autoen- coding for pretraining optical flow estimation. In Proceed- ings of the IEEE/CVF Conference on Computer Vision a...

    2023

  41. [49]

    Super-convergence: Very fast training of neural networks using large learning rates

    Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domain operations applications, volume 11006, pages 369–386. SPIE, 2019. 5

    2019

  42. [50]

    PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume

    Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Janf Kautz. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. CVPR, 2018. 2

    2018

  43. [51]

    RAFT: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. RAFT: Recurrent all-pairs field transforms for optical flow. In European Conference on Computer Vision, pages 402–419. Springer, 2020. 2, 3, 4, 5, 6, 12

    2020

  44. [52]

    RoboTAP: Tracking arbitrary points for few-shot visual imitation

    Mel Vecerik, Carl Doersch, Yi Yang, Todor Davchev, Yusuf Aytar, Guangyao Zhou, Raia Hadsell, Lourdes Agapito, and Jon Scholz. RoboTAP: Tracking arbitrary points for few-shot visual imitation. arXiv preprint arXiv:2308.15975, 2023. 5, 6, 7

    2023 arXiv

  45. [53]

    Tracking everything everywhere all at once

    Qianqian Wang, Yen-Yu Chang, Ruojin Cai, Zhengqi Li, Bharath Hariharan, Aleksander Holynski, and Noah 10 Snavely. Tracking everything everywhere all at once. arXiv:2306.05422, 2023. 2, 3

    2023 arXiv

  46. [55]

    Sea-raft: Sim- ple, efficient, accurate raft for optical flow

    Yihan Wang, Lahav Lipson, and Jia Deng. Sea-raft: Sim- ple, efficient, accurate raft for optical flow. arXiv preprint arXiv:2405.14793, 2024. 6, 12

    2024 arXiv

  47. [56]

    Fu- ture video synthesis with object motion prediction

    Yue Wu, Rongrong Gao, Jaesik Park, and Qifeng Chen. Fu- ture video synthesis with object motion prediction. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5539–5548, 2020. 1

    2020

  48. [57]

    Spatial- tracker: Tracking any 2d pixels in 3d space

    Yuxi Xiao, Qianqian Wang, Shangzhan Zhang, Nan Xue, Sida Peng, Yujun Shen, and Xiaowei Zhou. Spatial- tracker: Tracking any 2d pixels in 3d space. arXiv preprint arXiv:2404.04319, 2024. 1, 2

    2024 arXiv

  49. [58]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8121–8130,

  50. [59]

    Lla-flow: A lightweight local aggregation on cost volume for optical flow estimation

    Jiawei Xu, Zongqing Lu, and Qingmin Liao. Lla-flow: A lightweight local aggregation on cost volume for optical flow estimation. In 2023 IEEE International Conference on Image Processing (ICIP), pages 3220–3224. IEEE, 2023. 6

    2023

  51. [60]

    Ho- mography decomposition networks for planar object track- ing

    Xinrui Zhan, Yueran Liu, Jianke Zhu, and Yang Li. Ho- mography decomposition networks for planar object track- ing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3234–3242, 2022. 8

    2022

  52. [61]

    HVC-Net: Unifying ho- mography, visibility, and confidence learning for planar ob- ject tracking

    Haoxian Zhang and Yonggen Ling. HVC-Net: Unifying ho- mography, visibility, and confidence learning for planar ob- ject tracking. In Computer Vision and Pattern Regognition (CVPR), 2022. 8

    2022

  53. [62]

    Neuflow v2: High-efficiency optical flow estimation on edge devices

    Zhiyong Zhang, Aniket Gupta, Huaizu Jiang, and Hanumant Singh. Neuflow v2: High-efficiency optical flow estimation on edge devices. arXiv preprint arXiv:2408.10161, 2024. 6, 12

    2024 arXiv

  54. [63]

    NeuFlow: Real-time, high-accuracy optical flow esti- mation on robots using edge devices

    Zhiyong Zhang, Huaizu Jiang, and Hanumant Singh. NeuFlow: Real-time, high-accuracy optical flow esti- mation on robots using edge devices. arXiv preprint arXiv:2403.10425v1, 2024. 2, 6, 12 11 Supplementary Materials A. Image Feature Extraction For the D INOV2 features we use t...

    2024 arXiv

  55. [2020]

    8 2https://github.com/serycjon/MFTIQ 8

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.