Pith. sign in

REVIEW 3 major objections 5 minor 38 references

Local Attention Transformers for High-Detail Optical Flow Upsampling

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing the standard convex upsampler with a local-attention transformer improves optical flow accuracy across RAFT, GMA, and FlowFormer, with only the final upsampling step changed.

desk verdict The convex-upsampling-as-attention idea is clean and the gains are likely real, but the abstract's 'solely adapting the convex upsampler' overstates the evidence because the GMA number includes a training-scheme change with no control. read the letter →

arxiv 2412.06439 v1 pith:3ZY7BUIG submitted 2024-12-09 cs.CV

Alexander Gielisse , Nergis Tömen , Jan van Gemert This is my paper · ORCID
classification cs.CV
keywords opticalflowconvexupsamplingneighborhoodattentionlocaltransformersend-pointerrorhierarchicaldataaugmentation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets a small but widely shared component of modern optical flow networks: the convex upsampler that turns a low-resolution flow estimate into a full-resolution one. It argues that the standard 3x3 convex mask is unnecessarily restrictive, because a correct full-resolution pixel value exists only if the low-resolution neighborhood can be combined to produce it. The central move is to reformulate convex upsampling as local neighborhood attention, which decouples mask size from the number of parameters and allows hierarchical upsampling in three 2x steps. With only the upsampler replaced, plus a final training stage that disables interpolation-based augmentation, the authors report Sintel Clean training end-point-error dropping from 1.42 to 1.26 for RAFT, 1.31 to 1.18 for GMA, and 0.94 to 0.90 for FlowFormer. A careful reader would care because the change is presented as a drop-in replacement applicable to nearly every current state-of-the-art flow architecture.

What carries the argument

The central object is the Transformers for Convex Upsampling (TCU) module, a drop-in replacement for the convex upsampler. It is built on the equivalence between convex upsampling and Neighborhood Attention: the softmax-weighted dot product with low-resolution neighbors is identical in both, so local attention maps can serve as convex masks while their size is decoupled from the number of parameters. The module performs three 2x upsampling steps with mask sizes 9, 7, and 5, concatenates context and image features at each scale, and is applied only to the final refinement iteration's flow, giving that last step its own weights. This machinery carries the argument because each proposed change targets a specific failure mode: larger masks make a correct convex combination more likely to exist, hierarchical upsampling reduces the difficulty of aligning sub-pixels, extra features align flow with edges, and decoupling avoids polluting the final upsampler with noisy intermediate flow estimates.

What would settle it

Run a controlled comparison where the only variable is the upsampler: start all models from identical pre-trained weights, apply the exact same fine-tuning protocol with the same number of iterations and learning rates, and vary only the final upsampling module; if the end-point-error gap disappears, the reported gains come from the training setup rather than from attention-based upsampling.

Watch

Extended reading notes

Core claim

The paper's central claim is that convex upsampling, widely used to raise optical flow from one-eighth to full resolution, is best understood and improved as local attention. The authors observe that the softmax-weighted dot product over a sliding window of low-resolution flow, which defines a convex combination, is the same operation as neighborhood attention: both take a dot product between a normalized weight map and local values. They introduce Transformers for Convex Upsampling (TCU), which uses local attention maps as convex masks, allowing larger mask sizes, three hierarchical 2x upsampling steps instead of one 8x step, and the injection of image features at intermediate resolutions. They also decouple the upsampler used for the final refinement iteration from the shared upsampler used for intermediate iterations, and propose a final training stage with interpolation-based augmentations disabled to remove bilinear interpolation artifacts. On the FlyingChairs plus FlyingThings3D training setting, they report reducing the Sintel Clean training end-point-error of RAFT from 1.42 to 1.26, GMA from 1.31 to 1.18, and FlowFormer from 0.94 to 0.90, solely by adapting the upsampler.

Load-bearing premise

The reported improvements assume the comparison models were retrained under exactly the same fine-tuning protocol as the models using the new upsampler, so that the upsampler is the only meaningful difference.

Editorial extensions

If this is right

  • Any RAFT-style optical flow model that currently uses the original 8x convex upsampler can adopt TCU in its final refinement step without retraining the rest of the network, so the reported gains are portable.
  • Larger convex masks are not merely a parameter-count issue: they increase the chance that a correct high-resolution flow value is representable as a convex combination of low-resolution neighbors, so further gains from even larger masks should appear where memory permits.
  • Hierarchical 2x upsampling with multi-scale image features may become the default design for flow upsampling, replacing the single 8x step.
  • The -AUG training scheme shows that bilinear interpolation artifacts in augmented training data are baked into model outputs; disabling interpolation-based augmentation at the end sharpens edges and improves clean-dataset accuracy, though it can hurt generalization to blurry targets such as Sintel Final.
  • Because the upsampler is a small, largely independent part of the network, its improvements compose with other advances in flow prediction, which explains the consistent gains across RAFT, GMA, and FlowFormer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same convex-upsampling-as-attention argument applies to other dense prediction tasks that upsample low-resolution outputs, such as depth estimation, semantic segmentation, or scene flow, where a TCU-style module could improve edge fidelity.
  • The paper's 'existence of a convex combination' framing suggests a quantitative diagnostic: one could measure per patch whether the ground-truth high-resolution flow lies in the convex hull of the low-resolution neighborhood, and use that to decide where larger masks help most.
  • The authors' caution about cross-dataset generalization implies a testable prediction: on blurry datasets like Sintel Final, sharp edge alignment from larger masks will hurt unless the training data also contains blur, so the optimal mask size may be dataset-dependent.
  • The reported gains assume the baselines were fine-tuned under the same protocol; a direct re-run with identical pre-training, learning rate, and iteration counts would settle whether the upsampler alone, rather than training details, is responsible for the improvement.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Transformers for Convex Upsampling (TCU), a drop-in replacement for the convex upsampler used in RAFT-style optical flow networks. The method reformulates convex upsampling as local (neighborhood) attention, which decouples mask size from parameter count and enables hierarchical 2x upsampling steps with multi-scale image features. The authors also propose decoupling the final upsampler's weights and an additional training phase (-AUG) that disables interpolation-based augmentations. On the C+T training setting, they report reduced Sintel Clean training-split EPE for RAFT (1.42 to 1.26), GMA (1.31 to 1.18), and FlowFormer (0.94 to 0.90), attributing the gains to adapting the convex upsampler.

Significance. The core reformulation of convex upsampling as neighborhood attention is elegant and technically plausible; it cleanly decouples mask size from parameter count and makes hierarchical upsampling natural. The paper addresses an under-explored component of optical flow pipelines and provides experiments across three architectures, with a useful honesty about the risk that the changes can hurt cross-dataset generalization (Section 6). However, the central attribution claim is weakened by confounds: the best GMA number includes a training-scheme change, the FlowFormer comparison uses a reduced configuration, and the baseline protocol is not specified. With additional control experiments or a more cautious claim, the contribution would be valuable.

major comments (3)
  1. [Abstract; Section 4; Table 3] The abstract's headline for GMA (1.31 to 1.18) is the GMA+ALL-aug row of Table 3, but that row includes the -AUG training scheme defined in Section 4 (an additional 40K fine-tuning phase with interpolation-based augmentations disabled), not just a change of the upsampler. No comparison is provided for GMA with the original convex upsampler under the -AUG protocol, and Table 2 shows -AUG alone changes Sintel Clean EPE by 0.03 for the TCU model, so the 0.13 improvement cannot be attributed solely to the upsampler. Moreover, the three headline numbers come from different configurations: RAFT+ALL, GMA+ALL-aug, and FlowFormer+ALL, so the claim 'by solely adapting the convex upsampler' is not supported by the reported experiments.
  2. [Section 5; Table 3] It is unclear whether the 'recomputed' baselines in Table 3 (RAFT, GMA, FlowFormer) were re-trained under the same fine-tuning protocol described in Section 5 (100K iterations, learning rates 1e-4/2e-4, batch size 3) or are simply released checkpoints re-evaluated. If the latter, the improvement of the proposed models over these baselines could be partly an effect of additional fine-tuning. The paper should state explicitly how each baseline row was produced; if they are re-trained, give the exact protocol, and if they are not, a control fine-tuning of the original upsampler under the same protocol is needed.
  3. [Table 3 (FlowFormer rows and note)] The FlowFormer+ALL result is reported with the caveat that the transformer models are not fine-tuned and only scales 1/4 and 1/8 are used because scale 1/2 cannot be obtained. This means the comparison is not a clean replacement of the upsampler while keeping everything else fixed. Since the abstract cites the FlowFormer improvement as evidence for the upsampler's effect, the experiment should either be run with the same feature scales and fine-tuning protocol as the baseline, or the claim should be restricted to the configurations that are actually matched.
minor comments (5)
  1. [Section 3.1, Eqs. (12)-(13)] The notation for mask dimensions is inconsistent: Eq. (12) states masks are in R^{f^2 m^2}, while Eq. (13) writes R^{f^2*m*m} and then equates it to R^{f^2 m^2}; please use a single consistent representation throughout.
  2. [Section 5.2, paragraph 2] The statement that 'all our proposed changes result in improvements' is contradicted by Table 2, where +DC+FT (1.24 Sintel Clean, 9.53 FlyingThings3D) is worse than +DC alone (1.23 and 9.38); please qualify this claim.
  3. [Section 4] The claim that 'all public top submissions to the Sintel leaderboard show bilinear interpolation artifacts' is made without a citation or a supporting figure; please add evidence or soften the claim.
  4. [Table 3, FlowFormer note] The note about not fine-tuning the transformer models is important enough to be discussed in the main text, not only as a table footnote, because it directly affects the interpretation of the FlowFormer comparison.
  5. [References] Reference [7] for Neighborhood Attention is missing the publication venue; please provide the full citation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are externally benchmarked and the upsampler/attention equivalence is explicit, not a hidden input-output identity.

full rationale

The paper's central claim is empirical: replacing the last-iteration convex upsampler with a local-attention-based upsampler reduces end-point-error on standard optical-flow benchmarks. The method states in Equations 15 and 26 that the new aggregation is algebraically equivalent to convex upsampling, so the attention reformulation is a transparent design motivation rather than a recycled result. The comparisons are made against public baselines (RAFT, GMA, FlowFormer) on FlyingThings3D, Sintel, and KITTI, using published training setups, and the proposed module is trained end-to-end on those benchmarks. There are no load-bearing self-citations: the cited Neighborhood Attention Transformer [7] and MS-RAFT(+) [12,13] are independent prior works by other groups, and the paper invokes no same-author uniqueness theorem to force its choice. The empirical caveats present in the paper's own discussion, such as the -AUG training scheme being an additional 40K fine-tuning phase rather than a pure upsampler change, and the recomputation protocol for baselines not being fully specified, are experimental attribution and reproducibility concerns, not circularity: no fitted parameter is renamed as a prediction, and no definitional equivalence between input and output is used to produce the claimed result. The derivation chain is therefore self-contained with respect to circularity.

Assumptions & free parameters 10 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on hand-selected hyperparameters (mask sizes, dimensions, training schedules) and on standard pretrained checkpoints and benchmark protocols. No new physical entities are introduced; the TCU is a neural network module, not an invented entity in the ledger sense.

free parameters (10)
  • mask sizes for hierarchical upsampling = (9, 7, 5)
    Chosen by hand for the three factor-2 upsampling steps; larger masks are the paper's central hypothesis, but the exact sizes are tuned, not derived.
  • number of hierarchical upsampling steps = 3
    Chosen to replace one 8x upsampling with three 2x steps; follows MS-RAFT [12,13].
  • number of Neighborhood Attention Transformer blocks = 2
    Specified in Section 3.2; hand-selected.
  • feature dimension D per scale = 128, 64, 32
    D=128 at 1/8 scale, D=64 at 1/4, D=32 at 1/2; hand-selected.
  • head dimensionality = 32
    Hand-selected; attention heads use half the embedding dimension for upsampling.
  • fine-tuning learning rates = 1e-4 pretrained, 2e-4 new upsampler
    Section 5; chosen for fine-tuning procedure.
  • fine-tuning iterations = 100,000
    Section 5; chosen for all experiments.
  • additional -AUG iterations = 40,000
    Section 4; chosen for the artifact-removal training scheme.
  • batch size = 3
    Section 5; chosen for fine-tuning.
  • edge-detection threshold and bin width for detail analysis = 8, 0.02
    Section 5.1; chosen for the high-detail EPE analysis; arbitrary choices that shape Figure 4.
assumptions (4)
  • domain assumption Pre-trained RAFT, GMA, and FlowFormer checkpoints are available and used as initialization for all fine-tuning runs.
    Section 5 states training starts with pre-trained weights; the paper does not release or identify the exact checkpoints.
  • domain assumption The standard FlyingChairs + FlyingThings3D (C+T) training protocol, including the standard augmentation pipeline, is used and is sufficient to produce the reported baseline numbers.
    Section 5 says 'the only difference from the original training setting is the convex upsampler'; the original setting is taken as given.
  • standard math Softmax-normalized local attention weights provide a valid convex combination, making attention maps a drop-in replacement for convex masks.
    Equations 15 and 26 are claimed equivalent; this relies on the standard softmax property that weights are positive and sum to one.
  • domain assumption End-point error (EPE) on Sintel Clean train and FlyingThings3D test is an acceptable proxy for optical flow quality, and single-run results are treated as meaningful comparisons.
    All conclusions are drawn from EPE on these benchmarks without repeated seeds or statistical tests.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Local Attention Transformers for High-Detail Optical Flow Upsampling." pith.science (2026). https://pith.science/paper/3ZY7BUIG

@misc{pith2026241206439,
  author       = {Pith},
  title        = {Pith review of: Local Attention Transformers for High-Detail Optical Flow Upsampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3ZY7BUIG}},
  note         = {Machine review of arXiv:2412.06439}
}
read the original abstract

Most recent works on optical flow use convex upsampling as the last step to obtain high-resolution flow. In this work, we show and discuss several issues and limitations of this currently widely adopted convex upsampling approach. We propose a series of changes, in an attempt to resolve current issues. First, we propose to decouple the weights for the final convex upsampler, making it easier to find the correct convex combination. For the same reason, we also provide extra contextual features to the convex upsampler. Then, we increase the convex mask size by using an attention-based alternative convex upsampler; Transformers for Convex Upsampling. This upsampler is based on the observation that convex upsampling can be reformulated as attention, and we propose to use local attention masks as a drop-in replacement for convex masks to increase the mask size. We provide empirical evidence that a larger mask size increases the likelihood of the existence of the convex combination. Lastly, we propose an alternative training scheme to remove bilinear interpolation artifacts from the model output. Our proposed ideas could theoretically be applied to almost every current state-of-the-art optical flow architecture. On the FlyingChairs + FlyingThings3D training setting we reduce the Sintel Clean training end-point-error of RAFT from 1.42 to 1.26, GMA from 1.31 to 1.18, and that of FlowFormer from 0.94 to 0.90, by solely adapting the convex upsampler.

Figures

Figures reproduced from arXiv: 2412.06439 by the authors.

Figure 1
Figure 1. Current optical flow methods ignore fine details. We [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Original convex optical flow upsampling as proposed [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Left: the original convex upscaling method as proposed by RAFT [30]. Right: our proposed multi-step Transformer convex upsampling network. The feature extractor is adopted from RAFT [30] but we extract the features at 3 all intermediate scales. The Neighborhood Attention Transformer blocks [7] perform local neighborhood attention to enhance features. Attention is used to both upsample the low-resolution flow as well… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The average end-point-error for increasing amount of [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Improvement in end-point-error for increasing amount of detail on FlyingThings3D (test) [19], Sintel Clean (train) and Sintel [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 29 canonical work pages

  1. [1]

    Training stochastic model recognition algo- rithms as networks can lead to maximum mutual information estimation of parameters

    John Bridle. Training stochastic model recognition algo- rithms as networks can lead to maximum mutual information estimation of parameters. Advances in neural information processing systems, 2, 1989

    work page 1989

  2. [2]

    A naturalistic open source movie for opti- cal flow evaluation

    Daniel J Butler, Jonas Wulff, Garrett B Stanley, and Michael J Black. A naturalistic open source movie for opti- cal flow evaluation. In Computer Vision–ECCV 2012: 12th European Conference on Computer Vision, Florence, Italy, October 7-13, 2012, Proceedings, Part VI 12 , pages 611–

    2012

  3. [3]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  4. [4]

    Flownet: Learning optical flow with convolutional networks

    Alexey Dosovitskiy, Philipp Fischer, Eddy Ilg, Philip Hausser, Caner Hazirbas, Vladimir Golkov, Patrick Van Der Smagt, Daniel Cremers, and Thomas Brox. Flownet: Learning optical flow with convolutional networks. In Pro- ceedings of the IEEE international conference on computer vision, pages 2758–2766, 2015

    work page 2015

  5. [5]

    Cognitron: A self-organizing mul- tilayered neural network

    Kunihiko Fukushima. Cognitron: A self-organizing mul- tilayered neural network. Biological cybernetics , 20(3- 4):121–136, 1975

    work page 1975

  6. [6]

    Digital image processing

    Rafael C Gonzalez. Digital image processing. Pearson edu- cation india, 2009

    work page 2009

  7. [7]

    Neighborhood attention transformer

    Ali Hassani, Steven Walton, Jiachen Li, Shen Li, and Humphrey Shi. Neighborhood attention transformer. 2022

    work page 2022

  8. [8]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

Show all 38 references
  1. [9]

    Flowformer: A transformer architecture for optical flow

    Zhaoyang Huang, Xiaoyu Shi, Chao Zhang, Qiang Wang, Ka Chun Cheung, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer: A transformer architecture for optical flow. arXiv preprint arXiv:2203.16194, 2022

    2022 arXiv

  2. [10]

    A lightweight optical flow cnn—revisiting data fidelity and reg- ularization

    Tak-Wai Hui, Xiaoou Tang, and Chen Change Loy. A lightweight optical flow cnn—revisiting data fidelity and reg- ularization. IEEE transactions on pattern analysis and ma- chine intelligence, 43(8):2555–2569, 2020

    2020

  3. [11]

    Flownet 2.0: Evolu- tion of optical flow estimation with deep networks

    Eddy Ilg, Nikolaus Mayer, Tonmoy Saikia, Margret Keuper, Alexey Dosovitskiy, and Thomas Brox. Flownet 2.0: Evolu- tion of optical flow estimation with deep networks. In Pro- ceedings of the IEEE conference on computer vision and pat- tern recognition, pages 2462–2470, 2017

    2017

  4. [12]

    High resolution multi-scale raft (robust vision challenge 2022)

    Azin Jahedi, Maximilian Luz, Lukas Mehl, Marc Rivinius, and Andr´es Bruhn. High resolution multi-scale raft (robust vision challenge 2022). arXiv preprint arXiv:2210.16900 , 2022

    2022 arXiv

  5. [13]

    Multi-scale raft: Combining hierarchical concepts for learning-based optical flow estimation

    Azin Jahedi, Lukas Mehl, Marc Rivinius, and Andr ´es Bruhn. Multi-scale raft: Combining hierarchical concepts for learning-based optical flow estimation. In 2022 IEEE In- ternational Conference on Image Processing (ICIP) , pages 1236–1240. IEEE, 2022

    2022

  6. [14]

    Learning to estimate hidden motions with global motion aggregation

    Shihao Jiang, Dylan Campbell, Yao Lu, Hongdong Li, and Richard Hartley. Learning to estimate hidden motions with global motion aggregation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9772– 9781, 2021

    2021

  7. [15]

    Progressive growing of gans for improved quality, stability, and variation

    Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017

    2017 arXiv

  8. [16]

    The hci benchmark suite: Stereo and flow ground truth with uncertainties for urban autonomous driv- ing

    Daniel Kondermann, Rahul Nair, Katrin Honauer, Karsten Krispin, Jonas Andrulis, Alexander Brock, Burkhard Gusse- feld, Mohsen Rahimimoghaddam, Sabine Hofmann, Claus Brenner, et al. The hci benchmark suite: Stereo and flow ground truth with uncertainties for urban autonomous dr...

    2016

  9. [17]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021

    2021

  10. [18]

    Fully convolutional networks for semantic segmentation

    Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Pro- ceedings of the IEEE conference on computer vision and pat- tern recognition, pages 3431–3440, 2015

    2015

  11. [19]

    A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation

    Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In Proceedings of the IEEE conference on computer vision and ...

    2016

  12. [20]

    Spring: A high-resolution high- detail dataset and benchmark for scene flow, optical flow and stereo, 2023

    Lukas Mehl, Jenny Schmalfuss, Azin Jahedi, Yaroslava Nali- vayko, and Andr ´es Bruhn. Spring: A high-resolution high- detail dataset and benchmark for scene flow, optical flow and stereo, 2023

    2023

  13. [21]

    Object scene flow for au- tonomous vehicles

    Moritz Menze and Andreas Geiger. Object scene flow for au- tonomous vehicles. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3061– 3070, 2015

    2015

  14. [22]

    Optical flow estima- tion using a spatial pyramid network

    Anurag Ranjan and Michael J Black. Optical flow estima- tion using a spatial pyramid network. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 4161–4170, 2017

    2017

  15. [23]

    Kornia: an open source differentiable computer vision library for pytorch

    Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee, and Gary Bradski. Kornia: an open source differentiable computer vision library for pytorch. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3674–3683, 2020

    2020

  16. [24]

    Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network

    Wenzhe Shi, Jose Caballero, Ferenc Husz ´ar, Johannes Totz, Andrew P Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In Proceedings of the IEEE conference on compu...

    2016

  17. [25]

    Flowformer++: Masked cost volume autoen- coding for pretraining optical flow estimation.arXiv preprint arXiv:2303.01237, 2023

    Xiaoyu Shi, Zhaoyang Huang, Dasong Li, Manyuan Zhang, Ka Chun Cheung, Simon See, Hongwei Qin, Jifeng Dai, and Hongsheng Li. Flowformer++: Masked cost volume autoen- coding for pretraining optical flow estimation.arXiv preprint arXiv:2303.01237, 2023

    2023 arXiv

  18. [26]

    Craft: Cross- attentional flow transformer for robust optical flow

    Xiuchao Sui, Shaohua Li, Xue Geng, Yan Wu, Xinxing Xu, Yong Liu, Rick Goh, and Hongyuan Zhu. Craft: Cross- attentional flow transformer for robust optical flow. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17602–17611, 2022

    2022

  19. [27]

    Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume

    Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz. Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8934–8943, 2018

    2018

  20. [28]

    Models matter, so does training: An empirical study of cnns for optical flow estimation

    Deqing Sun, Xiaodong Yang, Ming-Yu Liu, and Jan Kautz. Models matter, so does training: An empirical study of cnns for optical flow estimation. IEEE transactions on pattern analysis and machine intelligence, 42(6):1408–1423, 2019

    2019

  21. [29]

    Computer vision: algorithms and applica- tions

    Richard Szeliski. Computer vision: algorithms and applica- tions. Springer Nature, 2022

    2022

  22. [30]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In European conference on com- puter vision, pages 402–419. Springer, 2020

    2020

  23. [31]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  24. [32]

    Displacement-invariant matching cost learning for accurate optical flow estimation

    Jianyuan Wang, Yiran Zhong, Yuchao Dai, Kaihao Zhang, Pan Ji, and Hongdong Li. Displacement-invariant matching cost learning for accurate optical flow estimation. Advances in Neural Information Processing Systems, 33, 2020

    2020

  25. [33]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8121– 8130, 2022

    2022

  26. [34]

    V olumetric correspon- dence networks for optical flow

    Gengshan Yang and Deva Ramanan. V olumetric correspon- dence networks for optical flow. Advances in neural infor- mation processing systems, 32, 2019

    2019

  27. [35]

    Hierarchical discrete distribution decomposition for match density esti- mation

    Zhichao Yin, Trevor Darrell, and Fisher Yu. Hierarchical discrete distribution decomposition for match density esti- mation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6044–6053, 2019

    2019

  28. [36]

    Separable flow: Learning motion cost volumes for optical flow estimation

    Feihu Zhang, Oliver J Woodford, Victor Adrian Prisacariu, and Philip HS Torr. Separable flow: Learning motion cost volumes for optical flow estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10807–10817, 2021

    2021

  29. [37]

    Maskflownet: Asymmetric feature matching with learnable occlusion mask

    Shengyu Zhao, Yilun Sheng, Yue Dong, Eric I Chang, Yan Xu, et al. Maskflownet: Asymmetric feature matching with learnable occlusion mask. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6278–6287, 2020

    2020

  30. [38]

    Global matching with overlapping at- tention for optical flow estimation

    Shiyu Zhao, Long Zhao, Zhixing Zhang, Enyu Zhou, and Dimitris Metaxas. Global matching with overlapping at- tention for optical flow estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17592–17601, 2022

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.