Pith. sign in

REVIEW 3 major objections 5 minor 86 references

FlowIt turns optical flow into a global matching problem solved by optimal transport, then refines with the confidence and occlusion maps that transport produces, setting new Sintel and zero-shot records.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 16:07 UTC pith:F5DVVW7Q

load-bearing objection Solid systems paper: hierarchical transformers + OT init + conf/occ-guided refinement deliver real Sintel SOTA and strong zero-shot numbers; ablations leave the OT piece partly entangled with capacity and pretraining. the 3 major comments →

arxiv 2603.28759 v2 pith:F5DVVW7Q submitted 2026-03-30 cs.CV

FlowIt: Global Matching via Hierarchical Transformers and Optimal Transport for Optical Flow

Sadra Safadoust , Fabio Tosi , Matteo Poggi , Fatma G\"uney This is my paper
classification cs.CV
keywords optical flowglobal matchingoptimal transporthierarchical transformerocclusion estimationconfidence-guided refinementzero-shot generalization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Optical flow is the dense 2-D motion field between consecutive video frames. Most recent networks still search only locally, so they struggle with large displacements, occlusions and textureless regions. FlowIt instead builds a hierarchical transformer that keeps global context at quarter resolution, casts the initial correspondence problem as entropy-regularized optimal transport (with dustbins for unmatched pixels), and obtains from that transport plan not only a flow field but also explicit occlusion and confidence maps. A subsequent refinement stage is conditioned on those maps so that reliable motion is deliberately propagated into ambiguous areas. The paper shows that this combination yields state-of-the-art numbers on the Sintel benchmark and the strongest published zero-shot cross-dataset results on Sintel, Spring and LayeredFlow, while remaining competitive on KITTI. Multiple model sizes let the same idea trade accuracy for compute.

Core claim

A hierarchical transformer that produces multi-scale features, followed by Sinkhorn optimal transport over the all-pairs correlation volume at 1/4 resolution, yields a robust initial flow together with usable occlusion and confidence maps; conditioning a short RAFT-style residual refinement on those maps, and supervising them jointly, produces more accurate and more generalizable optical flow than local-search or pure-transformer baselines.

What carries the argument

Optimal-transport flow initialization: the 4-D correlation volume is turned into a soft mutual-consistency transport plan via entropy-regularized Sinkhorn (with dustbins). Marginal sums of that plan become the occlusion map; windowed probability mass around the peak becomes the confidence map; both maps then guide residual updates.

Load-bearing premise

That the occlusion and confidence maps extracted from Sinkhorn transport are accurate enough, and informative enough when fed into the residual updates, to be the main reason for the reported gains rather than model size or pre-training alone.

What would settle it

Re-train the identical architecture but replace the Sinkhorn step with ordinary softmax matching (keeping every other weight, loss term and pre-training schedule fixed) and check whether the Sintel Clean / Final and zero-shot Spring / LayeredFlow numbers collapse back to the softmax baseline levels reported in the ablation table.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Global matching need not be abandoned for memory reasons; a hierarchical transformer plus optimal transport at 1/4 resolution is already competitive at full Sintel resolution.
  • Explicit occlusion and confidence maps derived from transport can be used as free auxiliary signals to steer iterative refinement without extra network heads.
  • The same pipeline produces four scalable variants (S–XL) that trade parameters for accuracy, giving practitioners a concrete accuracy–compute curve.
  • Zero-shot gains on Spring and LayeredFlow (transparent / reflective surfaces) indicate that mutual-consistency transport helps more than local search when photometric assumptions break.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same transport-plus-guided-refinement pattern should transfer almost unchanged to stereo or multi-view matching, where mutual consistency and occlusion reasoning are equally central.
  • Because the confidence map is already produced at every iteration, it could be exposed at test time for uncertainty-aware downstream modules (e.g., robust odometry or selective key-frame selection) without any extra training.
  • The residual gap on KITTI relative to multi-frame methods suggests that adding a lightweight temporal memory on top of the same transport initialization could close the remaining real-world gap.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. FlowIt is a two-frame optical flow architecture that extracts multi-scale CNN+FPN features, processes them with Multi-Resolution Transformer (MRT) blocks for hierarchical global context, builds a 1/4-resolution all-pairs correlation volume, and obtains an initial flow via entropy-regularized optimal transport (Sinkhorn with dustbins). The OT plan also yields initial occlusion and confidence maps that condition a RAFT-style iterative residual refinement (axis-wise u/v heads, logit-space accumulation for O and Γ, convex upsampling). Supervision combines Smooth-L1/L1 flow losses with L1 losses on forward–backward occlusion pseudo-labels and EPE-threshold confidence pseudo-labels. Four capacity variants (S/M/L/XL) are trained with a TartanAir o C o T o C+T+S+K+H schedule (plus optional KITTI fine-tune) and evaluated on Sintel/KITTI benchmarks and zero-shot Sintel-train, KITTI-train, Spring, and LayeredFlow. The XL model reports Sintel test Clean 0.85 / Final 1.84 and strong zero-shot numbers; ablations on FlyingChairs isolate MRT, OT vs softmax, conf/occ guidance, refinement steps, axis-wise heads, and the joint loss.

Significance. If the gains are real, the paper supplies a practical and competitive global-matching alternative to the dominant local iterative family (RAFT/SEA-RAFT/WAFT). The explicit OT-derived occlusion and confidence maps that both initialize and guide residual updates are a clean architectural idea, and the multi-variant design plus public-benchmark and zero-shot tables (including the difficult LayeredFlow non-Lambertian setting) make the empirical contribution usable by the community. The work is incremental rather than paradigm-shifting, but the combination of hierarchical transformers, OT initialization, and guided multi-output refinement is well-motivated and the reported Sintel and cross-dataset numbers would be of clear interest if the causal role of the OT+guidance components is better isolated.

major comments (3)
  1. The central causal claim—that Sinkhorn OT initialization plus conf/occ-guided refinement is the main driver of Sintel SOTA and zero-shot gains—is only weakly isolated. Table 4 measures OT vs softmax (0.617 vs 0.631 EPE) and guidance vs none (0.617 vs 0.708) solely on FlyingChairs val with the (S) model; the deltas are small and never re-run after TartanAir pretraining or S2M2 weight transfer (§4.2). Table S1 shows that removing TartanAir alone moves XL Clean from 0.85 to 0.90 and Final from 1.84 to 2.12, while capacity scaling (S o XL) produces larger EPE drops than OT or guidance. A load-bearing ablation on at least one target regime (Sintel train zero-shot or Spring) that holds capacity and pretraining fixed is required to support the attribution in the abstract and §1/§5.
  2. SOTA language is uneven across tables. On KITTI test (Table 1) XL is best only on Non-Occ and trails WAFT-DAv2-a2 and multi-frame methods on Fl-all; on KITTI train zero-shot (Table 2) Fl-all is worse than several WAFT and FlowSeek variants. The abstract and conclusion should qualify “competitive on KITTI” more carefully and avoid implying uniform leadership. Likewise, multi-frame methods in block (A) of Table 1 are not a fair peer group for the two-frame claim; the ranking narrative should keep them clearly separated.
  3. §3.2–3.4 and Eqs. (4)–(5), (9)–(11): the quality of the OT-derived O0/Γ0 maps is never validated against the pseudo-labels used for supervision (forward–backward consistency and EPE < 4 px). Without even a simple correlation or IoU check, it remains possible that the refinement network largely ignores these maps and that the reported gains come from capacity and pretraining. A short quantitative check (or an ablation that freezes/randomizes the maps at inference) would make the guided-propagation story falsifiable.
minor comments (5)
  1. Figure 3 caption and surrounding text claim the correlation volume is “notably smoother” than SEA-RAFT; a quantitative smoothness or peakiness statistic would make the visual comparison more convincing.
  2. Hyper-parameters (r=2, ε=10−6, λ F=1, λ Γ=λ O=0.1, three refinement steps, EPE threshold 4 px, FB threshold 2 px) are stated but never ablated beyond the coarse Table 4; a short sensitivity note would help reproducibility.
  3. Notation: ΓΓΓ and O are used both for maps and for their residual updates; a clearer distinction (e.g., Γ t vs ΔΓ t) would reduce ambiguity in Eqs. (6)–(8).
  4. Supplementary Table S1 is valuable; the main text should explicitly point readers to the “without TartanAir” rows when discussing pretraining impact.
  5. Latency and GMAC numbers in Table 4 are measured on L40S for the (S) model only; reporting the same for XL on a common GPU would better support the efficiency claims made for the smaller variants.

Circularity Check

0 steps flagged

Empirical architecture paper with external-benchmark evaluation; no derivation reduces to its inputs by construction.

full rationale

FlowIt is a supervised optical-flow architecture paper. Its load-bearing claims are empirical performance numbers (Sintel Clean/Final, KITTI Fl-all, Spring/LayeredFlow zero-shot EPE) measured on held-out external benchmarks after standard multi-stage training (TartanAir o Chairs o Things o C+T+S+K+H). The OT initialization (Sinkhorn on the 1/4-resolution correlation volume with dustbins) produces F0, O0, Γ0 by definition of the transport plan (Eqs. 2–5); these maps are then supervised by ordinary pseudo-labels (forward–backward consistency for occlusion, EPE-threshold indicator for confidence) and used as conditioning for residual updates. None of the reported benchmark metrics is forced by those definitions or by the loss weights. Pretraining/weight transfer from S2M2 [48] is ordinary transfer learning from a non-overlapping author set and a different task; it does not constitute a self-citation uniqueness theorem or an ansatz smuggled as a first-principles result. Ablations (Table 4) isolate components on FlyingChairs validation only and do not claim that the SOTA numbers are mathematically entailed by the OT maps. Consequently there is no self-definitional loop, no fitted-input-called-prediction, and no load-bearing self-citation chain. Score 0 is the correct outcome.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 2 invented entities

Load-bearing content is mostly engineering design plus standard CV assumptions, not new physics. Free parameters are training/loss/architecture knobs chosen by the authors. Axioms are domain conventions (cost volumes, Sinkhorn OT, RAFT-style updates, FB occlusion labels). Invented entities are architectural modules (MRT usage for flow, OT-derived conf/occ guidance stack), not new physical objects; independent evidence is only the paper’s own ablations and benchmarks.

free parameters (7)
  • loss weights λ_F, λ_Γ, λ_O = 1 / 0.1 / 0.1
    Set by hand to 1, 0.1, 0.1 for all experiments; relative emphasis of flow vs confidence/occlusion supervision is not derived.
  • local soft-expectation window radius r = 2
    Controls how the initial flow is read out from the OT plan; fixed to 2 without theoretical necessity.
  • number of refinement iterations T = 3
    Chosen as 3 after ablation; accuracy–cost trade-off is empirical.
  • confidence EPE threshold for Γ_gt = 4 px full-res / 1 px at 1/4
    Pixels with endpoint error <4 px (full res) labeled confident; ad hoc pseudo-label threshold.
  • occlusion FB consistency threshold = 2 px
    Non-occluded if forward–backward warping error <2 px; standard but hand-chosen label rule.
  • model capacity knobs (channels, MRT blocks) = S:128/1; M:192/2; L:256/3; XL:384/3
    S/M/L/XL variants (128–384 channels, 1–3 blocks) selected to trade parameters vs accuracy.
  • peak learning rate and training iteration budget = 1e-5; 300k/100k/120k/300k/10k stages
    One-cycle AdamW peak 1e-5 and multi-stage iteration counts follow SEA-RAFT-style schedule; fitted to training practice.
axioms (5)
  • domain assumption Dense optical flow can be recovered from multi-scale CNN+transformer features via a 4D correlation volume at 1/4 resolution.
    Standard deep optical-flow modeling assumption used throughout §3.1–3.2.
  • domain assumption Entropy-regularized optimal transport (Sinkhorn) with dustbins yields a valid soft mutual-consistent matching plan and occlusion mass.
    Imported from stereo/matching literature [8,47,56]; invoked in §3.2 without new proof.
  • domain assumption Forward–backward consistency and EPE thresholds are adequate ground-truth proxies for occlusion and confidence supervision.
    Used to define O_gt and Γ_gt in §3.4; common practice but not ground-truth physics.
  • ad hoc to paper Residual updates accumulated in logit space keep confidence/occlusion in [0,1] and improve stability.
    Design choice in Eqs. (7)–(8); justified by practice, not derived.
  • ad hoc to paper Axis-wise (u/v) separate refinement heads are preferable to coupled two-channel updates.
    Empirically supported in Table 4 ablation; treated as architectural axiom of the final model.
invented entities (2)
  • FlowIt architecture (MRT global matching + OT init + conf/occ-guided multi-output refinement) no independent evidence
    purpose: End-to-end optical flow estimator claiming improved long-range matching and generalization.
    The paper’s main proposed system; evidence is internal benchmarks/ablations, not independent external theory.
  • OT-derived initial confidence map Γ0 and occlusion map O0 used as refinement conditioning no independent evidence
    purpose: Steer residual updates so high-certainty motion propagates into ambiguous regions.
    Defined from the transport plan in Eqs. (4)–(5) and fed into refinement; only validated inside this paper’s experiments.

pith-pipeline@v1.1.0-grok45 · 24690 in / 3822 out tokens · 34708 ms · 2026-07-13T16:07:23.226003+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of FlowIt: Global Matching via Hierarchical Transformers and Optimal Transport for Optical Flow." pith.science (2026). https://pith.science/paper/F5DVVW7Q

@misc{pith2026260328759,
  author       = {Pith},
  title        = {Pith review of: FlowIt: Global Matching via Hierarchical Transformers and Optimal Transport for Optical Flow},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F5DVVW7Q}},
  note         = {Machine review of arXiv:2603.28759}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We present FlowIt, a novel architecture for optical flow estimation that combines global matching with confidence and occlusion-guided refinement. At its core, FlowIt leverages a hierarchical transformer architecture that captures extensive global context, enabling the model to effectively model long-range correspondences. To overcome the limitations of localized matching, we formulate the flow initialization as an optimal transport problem. This formulation yields a highly robust initial flow field, alongside explicitly derived occlusion and confidence maps. These cues are then seamlessly integrated into a guided refinement stage, where the network actively propagates reliable motion estimates from high-confidence regions into ambiguous, low-confidence areas. Extensive experiments across the Sintel, KITTI, Spring, and LayeredFlow datasets validate the effectiveness of our approach. FlowIt achieves state-of-the-art results on the competitive Sintel benchmark and establishes new state-of-the-art cross-dataset zero-shot generalization performance on Sintel, Spring, and LayeredFlow, while also delivering competitive performance on both the KITTI benchmark and KITTI zero-shot generalization settings.

Figures

Figures reproduced from arXiv: 2603.28759 by Fabio Tosi, Fatma G\"uney, Matteo Poggi, Sadra Safadoust.

Figure 1
Figure 1. Figure 1: SHOW3D is the rst dataset of in-the-wild handobject interactions with accurate 3D annotations as well as text descriptions. The dataset was captured with our novel mobile multi-camera rig in diverse indoor and outdoor scenes, and annotated with 3D shapes and poses with our multi-view pipeline. Overlays show 3D annotations projected to egocentric images (hands in red and blue, object in green). Abstract Acc… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

86 extracted references · 4 linked inside Pith

  1. [1]

    Learning optical flow from still images

    Filippo Aleotti, Matteo Poggi, and Stefano Mattoccia. Learning optical flow from still images. InCVPR, 2021

    2021

  2. [2]

    A computational framework and an algorithm for the measurement of visual motion.IJCV, 2:283–310, 1989

    P Anandan. A computational framework and an algorithm for the measurement of visual motion.IJCV, 2:283–310, 1989

    1989

  3. [3]

    MEM- FOF: High-resolution training for memory-efficient multi-frame optical flow estima- tion

    Vladislav Bargatin, Egor Chistov, Alexander Yakovenko, and Dmitriy Vatolin. MEM- FOF: High-resolution training for memory-efficient multi-frame optical flow estima- tion. InICCV, 2025

    2025

  4. [4]

    PMBP: Patch- match belief propagation for correspondence problems

    Frederic Besse, Carsten Rother, Andrew Fitzgibbon, and Jan Kautz. PMBP: Patch- match belief propagation for correspondence problems. InCVPR, 2014

    2014

  5. [5]

    A framework for the robust estimation of optical flow

    Michael J Black and P Anandan. A framework for the robust estimation of optical flow. InICCV, 1993

    1993

  6. [6]

    High accuracy optical flow estimation based on a theory for warping

    Thomas Brox, Andrés Bruhn, Nils Papenberg, and Joachim Weickert. High accuracy optical flow estimation based on a theory for warping. InECCV, 2004

    2004

  7. [7]

    A naturalistic open source movie for optical flow evaluation

    Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for optical flow evaluation. InECCV. Springer, 2012

    2012

  8. [8]

    Sinkhorn distances: Lightspeed computation of optimal transport

    Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In NeurIPS, 2013

    2013

  9. [9]

    Explicit motion disentangling for efficient optical flow estimation

    Changxing Deng, Ao Luo, Haibin Huang, Shaodan Ma, Jiangyu Liu, and Shuaicheng Liu. Explicit motion disentangling for efficient optical flow estimation. InICCV, 2023

    2023

  10. [10]

    Memflow: Optical flow estimation and prediction with memory

    Qiaole Dong and Yanwei Fu. Memflow: Optical flow estimation and prediction with memory. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19068–19078, 2024

    2024

  11. [11]

    Rethinking optical flow from geometric matching consistent perspective

    Qiaole Dong, Chenjie Cao, and Yanwei Fu. Rethinking optical flow from geometric matching consistent perspective. InCVPR, 2023

    2023

  12. [12]

    FlowNet: Learning optical flow with convolutional networks

    Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. FlowNet: Learning optical flow with convolutional networks. InICCV, 2015

    2015

  13. [13]

    Flow-edge guided video completion

    Chen Gao, Ayush Saraf, Jia-Bin Huang, and Johannes Kopf. Flow-edge guided video completion. InECCV. Springer, 2020

    2020

  14. [14]

    Vision meets robotics: The KITTI dataset.IJCV, 32(11):1231–1237, 2013

    Andreas Geiger, Philip Lenz, Christoph Stiller, and Raquel Urtasun. Vision meets robotics: The KITTI dataset.IJCV, 32(11):1231–1237, 2013

    2013

  15. [15]

    RealFlow: EM-based realistic optical flow dataset generation from videos

    Yunhui Han, Kunming Luo, Ao Luo, Jiangyu Liu, Haoqiang Fan, Guiming Luo, and Shuaicheng Liu. RealFlow: EM-based realistic optical flow dataset generation from videos. InECCV, 2022

    2022

  16. [16]

    Determining optical flow.Artificial intelli- gence, 17(1-3):185–203, 1981

    Berthold KP Horn and Brian G Schunck. Determining optical flow.Artificial intelli- gence, 17(1-3):185–203, 1981. S. SAFADOUST ET AL.: FLOWIT21

    1981

  17. [17]

    FlowFormer: A transformer architecture for optical flow

    Zhaoyang Huang, Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. FlowFormer: A transformer architecture for optical flow. InECCV, 2022

    2022

  18. [18]

    Real-time intermediate flow estimation for video frame interpolation

    Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. Real-time intermediate flow estimation for video frame interpolation. InECCV, pages 624–642. Springer, 2022

    2022

  19. [19]

    LiteFlowNet3: Resolving Correspondence Ambi- guity for More Accurate Optical Flow Estimation

    Tak-Wai Hui and Chen Change Loy. LiteFlowNet3: Resolving Correspondence Ambi- guity for More Accurate Optical Flow Estimation. InECCV, 2020

    2020

  20. [20]

    LiteFlowNet: A Lightweight Con- volutional Neural Network for Optical Flow Estimation

    Tak-Wai Hui, Xiaoou Tang, and Chen Change Loy. LiteFlowNet: A Lightweight Con- volutional Neural Network for Optical Flow Estimation. InCVPR, 2018

    2018

  21. [21]

    A Lightweight Optical Flow CNN —Revisiting Data Fidelity and Regularization .IEEE TPAMI, 43(08):2555–2569,

    Tak-Wai Hui, Xiaoou Tang, and Chen Change Loy. A Lightweight Optical Flow CNN —Revisiting Data Fidelity and Regularization .IEEE TPAMI, 43(08):2555–2569,

  22. [22]

    FlowNet 2.0: Evolution of optical flow estimation with deep networks

    Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. FlowNet 2.0: Evolution of optical flow estimation with deep networks. InCVPR, 2017

    2017

  23. [23]

    CCMR: High reso- lution optical flow estimation via coarse-to-fine context-guided motion reasoning

    Azin Jahedi, Maximilian Luz, Marc Rivinius, and Andrés Bruhn. CCMR: High reso- lution optical flow estimation via coarse-to-fine context-guided motion reasoning. In WACV, 2024

    2024

  24. [24]

    MS- RAFT+: high resolution multi-scale raft.IJCV, 132(5):1835–1856, 2024

    Azin Jahedi, Maximilian Luz, Marc Rivinius, Lukas Mehl, and Andrés Bruhn. MS- RAFT+: high resolution multi-scale raft.IJCV, 132(5):1835–1856, 2024

    2024

  25. [25]

    Computer vision for autonomous vehicles: Problems, datasets and state of the art.Foundations and trends® in computer graphics and vision, 12(1–3):1–308, 2020

    Joel Janai, Fatma Güney, Aseem Behl, Andreas Geiger, et al. Computer vision for autonomous vehicles: Problems, datasets and state of the art.Foundations and trends® in computer graphics and vision, 12(1–3):1–308, 2020

    2020

  26. [26]

    DistractFlow: Improving optical flow estimation via realistic distractions and pseudo-labeling

    Jisoo Jeong, Hong Cai, Risheek Garrepalli, and Fatih Porikli. DistractFlow: Improving optical flow estimation via realistic distractions and pseudo-labeling. InCVPR, 2023

    2023

  27. [27]

    Learn- ing to estimate hidden motions with global motion aggregation.arXiv preprint arXiv:2104.02409, 2021

    Shihao Jiang, Dylan Campbell, Yao Lu, Hongdong Li, and Richard Hartley. Learn- ing to estimate hidden motions with global motion aggregation.arXiv preprint arXiv:2104.02409, 2021

    Pith/arXiv arXiv 2021

  28. [28]

    Learning optical flow from a few matches

    Shihao Jiang, Yao Lu, Hongdong Li, and Richard Hartley. Learning optical flow from a few matches. InCVPR, 2021

    2021

  29. [29]

    EffiScene: Efficient per-pixel rigidity inference for unsupervised joint learning of optical flow, depth, camera pose and motion segmentation

    Yang Jiao, Trac D Tran, and Guangming Shi. EffiScene: Efficient per-pixel rigidity inference for unsupervised joint learning of optical flow, depth, camera pose and motion segmentation. InCVPR, 2021

    2021

  30. [30]

    What matters in unsupervised optical flow

    Rico Jonschkowski, Austin Stone, Jonathan T Barron, Ariel Gordon, Kurt Konolige, and Anelia Angelova. What matters in unsupervised optical flow. InECCV, 2020

    2020

  31. [31]

    AnyFlow: Arbitrary scale optical flow with implicit neural representation

    Hyunyoung Jung, Zhuo Hui, Lei Luo, Haitao Yang, Feng Liu, Sungjoo Yoo, Rakesh Ranjan, and Denis Demandolx. AnyFlow: Arbitrary scale optical flow with implicit neural representation. InCVPR, 2023. 22S. SAFADOUST ET AL.: FLOWIT

    2023

  32. [32]

    Deep video inpaint- ing

    Dahun Kim, Sanghyun Woo, Joon-Young Lee, and In So Kweon. Deep video inpaint- ing. InCVPR, 2019

    2019

  33. [33]

    The HCI benchmark suite: Stereo and flow ground truth with uncertainties for urban autonomous driving

    Daniel Kondermann, Rahul Nair, Katrin Honauer, Karsten Krispin, Jonas Andrulis, Alexander Brock, Burkhard Gussefeld, Mohsen Rahimimoghaddam, Sabine Hofmann, Claus Brenner, et al. The HCI benchmark suite: Stereo and flow ground truth with uncertainties for urban autonomous driving. InCVPRW, 2016

    2016

  34. [34]

    Win- Win: Training high-resolution vision transformers from two windows.arXiv preprint arXiv:2310.00632, 2023

    Vincent Leroy, Jerome Revaud, Thomas Lucas, and Philippe Weinzaepfel. Win- Win: Training high-resolution vision transformers from two windows.arXiv preprint arXiv:2310.00632, 2023

    Pith/arXiv arXiv 2023

  35. [35]

    MegaDepth: Learning single-view depth prediction from internet photos

    Zhengqi Li and Noah Snavely. MegaDepth: Learning single-view depth prediction from internet photos. InCVPR, 2018

    2018

  36. [36]

    SIFT flow: Dense correspondence across scenes and its applications.IEEE TPAMI, 33(5):978–994, 2010

    Ce Liu, Jenny Yuen, and Antonio Torralba. SIFT flow: Dense correspondence across scenes and its applications.IEEE TPAMI, 33(5):978–994, 2010

    2010

  37. [37]

    ARFlow: Auto-regressive op- tical flow estimation for arbitrary-length videos via progressive next-frame forecasting

    Jiuming Liu, Mengmeng Liu, Siting Zhu, Yunpeng Zhang, Jiangtao Li, Michael Ying Yang, Francesco Nex, Hao Cheng, and Hesheng Wang. ARFlow: Auto-regressive op- tical flow estimation for arbitrary-length videos via progressive next-frame forecasting. InThe Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum...

    2026

  38. [38]

    Learning by analogy: Reliable su- pervision from transformations for unsupervised optical flow estimation

    Liang Liu, Jiangning Zhang, Ruifei He, Yong Liu, Yabiao Wang, Ying Tai, Donghao Luo, Chengjie Wang, Jilin Li, and Feiyue Huang. Learning by analogy: Reliable su- pervision from transformations for unsupervised optical flow estimation. InCVPR, 2020

    2020

  39. [39]

    Flow2Stereo: Effective self- supervised learning of optical flow and stereo matching

    Pengpeng Liu, Irwin King, Michael R Lyu, and Jia Xu. Flow2Stereo: Effective self- supervised learning of optical flow and stereo matching. InCVPR, 2020

    2020

  40. [40]

    Video frame interpolation via optical flow estimation with image inpainting.IJCV, 35(12):2087–2102, 2020

    Xiaozhang Liu, Hui Liu, and Yuxiu Lin. Video frame interpolation via optical flow estimation with image inpainting.IJCV, 35(12):2087–2102, 2020

    2087

  41. [41]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Pith/arXiv arXiv 2017

  42. [42]

    Learn- ing optical flow with adaptive graph reasoning

    Ao Luo, Fan Yang, Kunming Luo, Xin Li, Haoqiang Fan, and Shuaicheng Liu. Learn- ing optical flow with adaptive graph reasoning. InAAAI, 2022

    2022

  43. [43]

    FlowDif- fuser: Advancing optical flow estimation with diffusion models

    Ao Luo, Xin Li, Fan Yang, Jiangyu Liu, Haoqiang Fan, and Shuaicheng Liu. FlowDif- fuser: Advancing optical flow estimation with diffusion models. InCVPR, 2024

    2024

  44. [44]

    A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation

    Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. InCVPR, 2016

    2016

  45. [45]

    Spring: A high-resolution high-detail dataset and benchmark for scene flow, optical flow and stereo

    Lukas Mehl, Jenny Schmalfuss, Azin Jahedi, Yaroslava Nalivayko, and Andrés Bruhn. Spring: A high-resolution high-detail dataset and benchmark for scene flow, optical flow and stereo. InCVPR, 2023. S. SAFADOUST ET AL.: FLOWIT23

    2023

  46. [46]

    Object scene flow for autonomous vehicles

    Moritz Menze and Andreas Geiger. Object scene flow for autonomous vehicles. In CVPR, 2015

    2015

  47. [47]

    Confidence aware stereo matching for realistic clut- tered scenario

    Junhong Min and Youngpil Jeon. Confidence aware stereo matching for realistic clut- tered scenario. InICIP. IEEE, 2024

    2024

  48. [48]

    S2M2: Scalable stereo matching model for reliable depth estimation

    Junhong Min, Youngpil Jeon, Jimin Kim, and Minyong Choi. S2M2: Scalable stereo matching model for reliable depth estimation. InICCV, 2025

    2025

  49. [49]

    Recurrent partial kernel network for efficient optical flow estimation

    Henrique Morimitsu, Xiaobin Zhu, Xiangyang Ji, and Xu-Cheng Yin. Recurrent partial kernel network for efficient optical flow estimation. InAAAI, 2024

    2024

  50. [50]

    DPFlow: Adaptive optical flow estimation with a dual-pyramid framework.arXiv preprint arXiv:2503.14880, 2025

    Henrique Morimitsu, Xiaobin Zhu, Roberto M Cesar Jr, Xiangyang Ji, and Xu-Cheng Yin. DPFlow: Adaptive optical flow estimation with a dual-pyramid framework.arXiv preprint arXiv:2503.14880, 2025

    arXiv 2025

  51. [51]

    Automatic differentiation in PyTorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in PyTorch. InNeurIPS, 2017

    2017

  52. [52]

    Representation flow for action recognition

    AJ Piergiovanni and Michael S Ryoo. Representation flow for action recognition. In CVPR, 2019

    2019

  53. [53]

    FlowSeek: Optical flow made easier with depth founda- tion models and motion bases

    Matteo Poggi and Fabio Tosi. FlowSeek: Optical flow made easier with depth founda- tion models and motion bases. InICCV, 2025

    2025

  54. [54]

    Optical flow estimation using a spatial pyramid network

    Anurag Ranjan and Michael J Black. Optical flow estimation using a spatial pyramid network. InCVPR, 2017

    2017

  55. [55]

    Playing for benchmarks

    Stephan R Richter, Zeeshan Hayder, and Vladlen Koltun. Playing for benchmarks. In ICCV, 2017

    2017

  56. [56]

    SuperGlue: Learning feature matching with graph neural networks

    Paul-Edouard Sarlin, Daniel DeTone, Tomasz Malisiewicz, and Andrew Rabinovich. SuperGlue: Learning feature matching with graph neural networks. InCVPR, 2020

    2020

  57. [57]

    The surprising effectiveness of diffusion models for optical flow and monocular depth estimation

    Saurabh Saxena, Charles Herrmann, Junhwa Hur, Abhishek Kar, Mohammad Norouzi, Deqing Sun, and David J Fleet. The surprising effectiveness of diffusion models for optical flow and monocular depth estimation. InNeurIPS, 2023

    2023

  58. [58]

    VideoFlow: Ex- ploiting temporal cues for multi-frame optical flow estimation

    Xiaoyu Shi, Zhaoyang Huang, Weikang Bian, Dasong Li, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, Jifeng Dai, and Hongsheng Li. VideoFlow: Ex- ploiting temporal cues for multi-frame optical flow estimation. InICCV, 2023

    2023

  59. [59]

    Flowformer++: Masked cost volume autoencoding for pretraining optical flow estimation

    Xiaoyu Shi, Zhaoyang Huang, Dasong Li, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer++: Masked cost volume autoencoding for pretraining optical flow estimation. InCVPR, 2023

    2023

  60. [60]

    Super-convergence: Very fast training of neural networks using large learning rates

    Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. InArtificial intelligence and machine learning for multi-domain operations applications, volume 11006, pages 369–386. SPIE, 2019

    2019

  61. [61]

    CRAFT: Cross-attentional flow transformer for robust optical flow

    Xiuchao Sui, Shaohua Li, Xue Geng, Yan Wu, Xinxing Xu, Yong Liu, Rick Goh, and Hongyuan Zhu. CRAFT: Cross-attentional flow transformer for robust optical flow. In CVPR, 2022. 24S. SAFADOUST ET AL.: FLOWIT

    2022

  62. [62]

    PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume

    Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. InCVPR, 2018

    2018

  63. [63]

    Models matter, so does training: An empirical study of CNNs for optical flow estimation.IEEE TPAMI, 42(6): 1408–1423, 2019

    Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz. Models matter, so does training: An empirical study of CNNs for optical flow estimation.IEEE TPAMI, 42(6): 1408–1423, 2019

    2019

  64. [64]

    AutoFlow: Learning a better training set for optical flow

    Deqing Sun, Daniel Vlasic, Charles Herrmann, Varun Jampani, Michael Krainin, Hui- wen Chang, Ramin Zabih, William T Freeman, and Ce Liu. AutoFlow: Learning a better training set for optical flow. InCVPR, 2021

    2021

  65. [65]

    Disentangling architecture and training for optical flow

    Deqing Sun, Charles Herrmann, Fitsum Reda, Michael Rubinstein, David J Fleet, and William T Freeman. Disentangling architecture and training for optical flow. InECCV, 2022

    2022

  66. [66]

    SKFlow: Learning optical flow with super kernels

    Shangkun Sun, Yuanqi Chen, Yu Zhu, Guodong Guo, and Ge Li. SKFlow: Learning optical flow with super kernels. InNeurIPS, 2022

    2022

  67. [67]

    Streamflow: streamlined multi-frame optical flow estimation for video sequences.Ad- vances in neural information processing systems, 37:9205–9228, 2024

    Shangkun Sun, Jiaming Liu, Huaxia Li, Guoqing Liu, Thomas H Li, and Wei Gao. Streamflow: streamlined multi-frame optical flow estimation for video sequences.Ad- vances in neural information processing systems, 37:9205–9228, 2024

    2024

  68. [68]

    Optical flow guided feature: A fast and robust motion representation for video action recogni- tion

    Shuyang Sun, Zhanghui Kuang, Lu Sheng, Wanli Ouyang, and Wei Zhang. Optical flow guided feature: A fast and robust motion representation for video action recogni- tion. InCVPR, 2018

    2018

  69. [69]

    RAFT: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. RAFT: Recurrent all-pairs field transforms for optical flow. InECCV. Springer, 2020

    2020

  70. [70]

    Splatflow: Learning multi-frame optical flow via splatting.International Journal of Computer Vision, 132(8):3023–3045, 2024

    Bo Wang, Yifan Zhang, Jian Li, Yang Yu, Zhenping Sun, Li Liu, and Dewen Hu. Splatflow: Learning multi-frame optical flow via splatting.International Journal of Computer Vision, 132(8):3023–3045, 2024

    2024

  71. [71]

    TartanAir: A dataset to push the limits of visual SLAM

    Wenshan Wang, Delong Zhu, Xiangwei Wang, Yaoyu Hu, Yuheng Qiu, Chen Wang, Yafei Hu, Ashish Kapoor, and Sebastian Scherer. TartanAir: A dataset to push the limits of visual SLAM. InIROS. IEEE, 2020

    2020

  72. [72]

    W AFT: Warping-alone field transforms for optical flow

    Yihan Wang and Jia Deng. W AFT: Warping-alone field transforms for optical flow. arXiv preprint arXiv:2506.21526, 2025

    arXiv 2025

  73. [73]

    SEA-RAFT: Simple, efficient, accurate RAFT for optical flow

    Yihan Wang, Lahav Lipson, and Jia Deng. SEA-RAFT: Simple, efficient, accurate RAFT for optical flow. InECCV. Springer, 2024

    2024

  74. [74]

    CroCo v2: Improved cross-view completion pre-training for stereo matching and optical flow

    Philippe Weinzaepfel, Thomas Lucas, Vincent Leroy, Yohann Cabon, Vaibhav Arora, Romain Brégier, Gabriela Csurka, Leonid Antsfeld, Boris Chidlovskii, and Jérôme Revaud. CroCo v2: Improved cross-view completion pre-training for stereo matching and optical flow. InICCV, 2023

    2023

  75. [75]

    Layeredflow: A real-world benchmark for non-lambertian multi-layer optical flow

    Hongyu Wen, Erich Liang, and Jia Deng. Layeredflow: A real-world benchmark for non-lambertian multi-layer optical flow. InECCV, pages 477–495. Springer, 2024

    2024

  76. [76]

    GMFlow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. GMFlow: Learning optical flow via global matching. InCVPR, 2022. S. SAFADOUST ET AL.: FLOWIT25

    2022

  77. [77]

    Unifying flow, stereo and depth estimation.IEEE TPAMI, 2023

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, Fisher Yu, Dacheng Tao, and Andreas Geiger. Unifying flow, stereo and depth estimation.IEEE TPAMI, 2023

    2023

  78. [78]

    Youtube-VOS: A large-scale video object segmentation benchmark

    Ning Xu, Linjie Yang, Yuchen Fan, Dingcheng Yue, Yuchen Liang, Jianchao Yang, and Thomas Huang. Youtube-VOS: A large-scale video object segmentation benchmark. arXiv preprint arXiv:1809.03327, 2018

    Pith/arXiv arXiv 2018

  79. [79]

    Deep flow-guided video inpainting

    Rui Xu, Xiaoxiao Li, Bolei Zhou, and Chen Change Loy. Deep flow-guided video inpainting. InCVPR, 2019

    2019

  80. [80]

    Quadratic video interpolation

    Xiangyu Xu, Li Siyao, Wenxiu Sun, Qian Yin, and Ming-Hsuan Yang. Quadratic video interpolation. InNeurIPS, 2019

    2019

Showing first 80 references.