Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

CrossVideoMAE: Self-Supervised Image-Video Representation Learning with Masked Autoencoders

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CrossVideoMAE claims that pairing a video masked autoencoder with single sampled frames during self-supervised pretraining yields state-of-the-art action recognition on four benchmarks, reaching 97.6% on UCF101 and 83.2% on K400.

desk verdict SOTA claim rests on a test-time adaptation step that baselines don't get, and the paper never reports numbers without it, so the main comparison is confounded. read the letter →

arxiv 2502.07811 v1 pith:TUX6QHUU submitted 2025-02-08 cs.CV

Shihab Aaqil Ahamed , Malitha Gunawardhana , Liel David , Michael Sidorov , Daniel Harari , Muhammad Haris Khan This is my paper · ORCID
classification cs.CV
keywords self-supervisedvideorepresentationlearningmaskedautoencoderscross-modalcontrastiveactionrecognitiontest-timeadaptationKinetics-400Something-Somethingv2spatiotemporal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CrossVideoMAE is a self-supervised video pretraining method that tries to show that masked autoencoders for video can be improved by also aligning the video with still frames sampled from the same clip. The paper argues that video-only MAEs learn generic spatiotemporal patterns but miss action-specific semantics, and that a pre-trained image MAE can supply those semantics through cross-modal contrastive objectives at both video and frame levels. Across UCF101, HMDB51, Kinetics-400, and Something-Something v2, the method reports higher full-fine-tuning accuracy than previous self-supervised methods, reaching 97.6%, 78.4%, 83.2%, and 73.7%, respectively, on a ViT-B/16 backbone. The reported protocol includes 20 gradient updates on evaluation data as a test-time adaptation step, so the comparisons are conditional on that adaptation.

What carries the argument

The load-bearing mechanism is a two-branch, three-tower pretraining setup: two shared-weight SpatioTemporalMAE video encoders process the original and an augmented video, while a pre-trained MAE image encoder processes a single randomly sampled frame from the same clip. Visible tokens from each branch are projected into a shared invariant space, and NT-Xent contrastive losses are applied at two scales—video level and frame level (frame-level vectors are sampled from the video-level embeddings)—both within the video modality and across the video and image modalities. The masking ratios are deliberately asymmetric (90–95% on video, 75–90% on frames) so the two branches see complementary information; the overall loss is a weighted sum of intra-modal contrastive, cross-modal contrastive, reconstruction, and MSE terms. This design is what lets a static image encoder inject semantic attributes into a video encoder without any labels.

What would settle it

Run CrossVideoMAE and a strong baseline such as SpatioTemporalMAE or VideoMAE with the same fine-tuning protocol but with test-time adaptation disabled (or, symmetrically, give the baselines the same 20-step TTA) on SSv2 and K400; if the reported accuracy differences shrink to noise or reverse, the claim that cross-modal pretraining is the source of the gains is falsified.

Watch

Extended reading notes

Core claim

The central claim is that joining a video masked autoencoder with an image masked autoencoder during self-supervised pretraining produces video representations that transfer better than either modality alone. CrossVideoMAE takes a SpatioTemporalMAE video encoder and a pre-trained MAE image encoder, randomly masks each branch at different ratios (90–95% for video, 75–90% for frames), and optimizes NT-Xent contrastive losses that align the two views of the same clip at both video level and frame level, plus reconstruction and MSE losses. On its own terms, the paper states that this joint intra-modal and cross-modal objective distills semantic attributes from sampled frames into the video encoder while preserving invariance to spatiotemporal augmentations. The paper reports state-of-the-art Acc@1 on four benchmarks and ablation studies showing that each objective contributes, with the largest gains on SSv2 (73.7%). The image branch is discarded after pretraining, so downstream inference uses only the video encoder.

Load-bearing premise

The reported comparison assumes that the 20 gradient updates of test-time adaptation performed on the evaluation set (Section 4.4) are not responsible for the margin over baselines that are evaluated without the same adaptation.

Editorial extensions

If this is right

  • If the central claim holds, self-supervised video pretraining can be improved with no extra labels by using a single sampled frame per clip as a second modality.
  • The method's downstream encoder is the same ViT-B/16 as existing video MAEs, so better accuracy on these benchmarks comes without an increase in inference parameters.
  • High masking ratios (95% on video, 90% on frames) are shown to be beneficial when combined with contrastive alignment, pointing to masking-ratio asymmetry as a reusable design choice.
  • Because the image branch is discarded after pretraining, the approach could be applied to any video MAE that has access to unlabeled video and the corresponding frames.
  • Reported gains on SSv2 are larger than on K400, suggesting the cross-modal alignment is most helpful for motion-centric datasets where static scene semantics matter.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper does not pursue is to apply the same test-time adaptation protocol to the leading baselines; if TTA is what drives the gap, the pretraining contribution would need to be evaluated without adaptation.
  • If the cross-modal alignment generalizes, the same sampled-frame distillation should transfer to other spatiotemporal tasks such as video retrieval, and the paper's retrieval results (85.5% R@1 on UCF101, 49.7% on HMDB51) are consistent with that direction.
  • The paper's ablation in the supplement (Table 13) suggests the gain comes from using corresponding video-frame pairs rather than simply adding ImageNet data; a natural next test is varying the number and choice of sampled frames at larger scale.
  • The asymmetry in optimal masking ratios hints that the two branches should be forced to see complementary information; one could test this by tuning the ratio gap on other backbone scales.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CrossVideoMAE, a self-supervised video representation learning method that couples a video MAE branch with an image MAE branch on sampled frames using intra-modal and cross-modal contrastive losses at both video and frame levels, together with reconstruction and MSE losses (Eqs. 1-11). The method is evaluated by full fine-tuning on UCF101, HMDB51, K400, and SSv2, with state-of-the-art accuracy reported in Table 1 (e.g., 97.6% on UCF101, 78.4% on HMDB51, 73.7% on SSv2, 83.2% on K400). The empirical protocol includes a test-time adaptation step (Section 4.4) in which the model performs 20 gradient updates on evaluation-set data before final inference. Ablation studies on SSv2 are used to justify design choices such as masking ratios, number of sampled frames, and loss components.

Significance. If the reported results held under a standard inductive evaluation, the paper would make a useful contribution: the idea of distilling semantic information from randomly sampled frames into a video MAE via contrastive objectives is well motivated, the loss definitions are explicit and reproducible from the text, and supplementary Table 13 provides a sensible control (corresponding sampled frames vs IN-1K) suggesting that data pairing matters. However, the empirical claims as stated are not yet supported because of the transductive TTA protocol; the significance depends on whether the authors can demonstrate the gains without TTA or with TTA applied equally to baselines.

major comments (4)
  1. [Section 4.4, Table 1] The main comparison is confounded by test-time adaptation on the evaluation set. All CrossVideoMAE numbers in Table 1 are produced after 20 gradient updates on evaluation-set data, whereas the cited baselines are evaluated with fixed weights. The paper provides no non-TTA results, so the reported improvements (e.g., 73.7 vs 72.5 on SSv2, 97.6 vs 97.0 on UCF101) cannot be attributed to the proposed pre-training. Please report the same results without TTA, and either drop TTA from the method or apply the same TTA procedure to all baselines.
  2. [Section 4.4] It is ambiguous what weights are being adapted. The text says 'initial inference is conducted using pre-trained weights available from open-source repositories (MAE for the image branch and SpatioTemporalMAE for the video branch)' and later describes TTA as 'dynamically fine-tuning pre-trained weights (from MAE and SpatioTemporalMAE)'. If TTA operates on the open-source checkpoints directly rather than on the CrossVideoMAE-pretrained model, Table 1 would not reflect the proposed pre-training at all. Please specify whether TTA starts from the CrossVideoMAE-pretrained fine-tuned encoder, and describe the exact losses and data used during the 20 updates.
  3. [Tables 3-6 and supplementary Tables 9-14] The ablations also include TTA, so they do not isolate the effect of the proposed components. For example, Table 6's claim that removing cross-modal contrastive training costs 0.7 points is only valid under the TTA protocol; without a non-TTA ablation, the marginal contributions of the contrastive losses, masking ratios, and augmentations are not measured. A non-TTA version of at least Tables 3 and 6 is needed to support the attribution.
  4. [Section 4.4] The description of TTA data is incomplete. The paper says 'due to GPU constraints, we created smaller subsets of K400 and SSv2 from the subset randomly sampled frames to create corresponding pre-sampled frame datasets-our test-time data'. It is unclear how these subsets relate to the test split used for accuracy reporting, whether the same videos appear in both the TTA data and the evaluation set, and how many videos are used. This compounds the transductive-evaluation concern.
minor comments (5)
  1. [Equations (1)-(2)] The denominators in Eq. (1) and Eq. (2) are rendered unreadably with 'NP' and an undefined summation; please typeset the NT-Xent denominator properly as sums over k=1..N with k != i.
  2. [Table 2] The heading '1N-1K' should be 'IN-1K'.
  3. [Sections 4.3 and 4.4] The claim that TTA 'saves GPU memory and reduces pre-training time' is confusing, since TTA is applied during inference; please clarify how it affects pre-training compute.
  4. [Abstract] The abstract contains grammatical errors ('CrossVideoMAE an end-to-end' should be 'CrossVideoMAE is an end-to-end'; 'Our approach aim' should be 'Our approach aims').
  5. [Table 2] Reporting IN-1K accuracy for a video pre-training method is not explained; please clarify how the IN-1K fine-tuning is performed (e.g., treating images as single-frame videos).

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation; reported SOTA comparison is confounded by test-time adaptation but not circular.

full rationale

We walked the claimed derivation chain: CrossVideoMAE combines intra- and cross-modal NT-Xent losses (Eqs. 1-7) with reconstruction (Eq. 10) and MSE (Eqs. 8-9) losses; the overall objective (Eq. 11) is optimized during pre-training on unlabeled video/frame pairs. The downstream 'prediction' is an action-class posterior after supervised fine-tuning, not a re-derivation of the pre-training loss. No parameter is fitted to a subset and then used to predict that same subset; masking ratios, sampled-frame count, decoder depth, and augmentation choices are selected by SSv2 ablations and applied to other datasets, which is standard hyperparameter selection, not circular. The two self-citations ([17] survey, [33] contact-identification study) are contextual and do not carry the load of the SOTA claim. The only serious concern is the test-time adaptation in Sec. 4.4: 20 gradient updates on the evaluation set before final inference are not given to the baselines, so the Table 1 comparison may overstate the benefit of the proposed pre-training. That is a validity/fairness issue, not an equivalence-by-construction; it does not make the derivation circular. Hence no circular step is identified, and the paper is assigned a low score reflecting the minor self-citations and the protocol caveat.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The method depends on standard contrastive and reconstruction losses, plus a few hyperparameters chosen via SSv2 ablations. The most consequential assumption is that test-time adaptation on unlabeled evaluation data does not invalidate the comparison with baselines that lack such adaptation, a premise the paper does not test.

free parameters (6)
  • Video branch masking ratio = 95% (default)
    Tuned on SSv2 ablation (Table 5) and then used for all experiments.
  • Image branch masking ratio = 90% (default)
    Tuned on SSv2 ablation (Table 5).
  • Number of sampled frames n = 1
    Ablation Table 3 shows n=1 matches or beats larger n; used in all experiments.
  • TTA gradient update steps = 20
    Set in Section 4.4 without ablation.
  • Loss weight lambda_c = not specified
    Overall objective Eq. 11; the value is not reported, so results cannot be exactly reproduced.
  • Temperature tau = not specified
    Used in contrastive losses Eq. 1-7; value not reported.
assumptions (4)
  • standard math NT-Xent contrastive loss with cosine similarity and temperature tau provides a valid learning signal.
    Used in Eq. 1-7, following SimCLR.
  • domain assumption Random masking with very high ratios (90-95%) is beneficial.
    Motivated by prior MAE works; the paper's ablation (Table 5) shows 95%/90% best.
  • domain assumption A single randomly sampled frame per video is sufficient to distill semantic knowledge to the video encoder.
    Ablation Table 3 shows n=1 best or equal; used in all experiments.
  • ad hoc to paper Test-time adaptation on unlabeled evaluation-set data is a valid inference-time refinement and does not break comparability with baselines that do not use it.
    The paper applies 20 gradient updates on test-time data (Section 4.4) but baselines do not; this assumption is required for the SOTA claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CrossVideoMAE: Self-Supervised Image-Video Representation Learning with Masked Autoencoders." pith.science (2026). https://pith.science/paper/TUX6QHUU

@misc{pith2026250207811,
  author       = {Pith},
  title        = {Pith review of: CrossVideoMAE: Self-Supervised Image-Video Representation Learning with Masked Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TUX6QHUU}},
  note         = {Machine review of arXiv:2502.07811}
}
read the original abstract

Current video-based Masked Autoencoders (MAEs) primarily focus on learning effective spatiotemporal representations from a visual perspective, which may lead the model to prioritize general spatial-temporal patterns but often overlook nuanced semantic attributes like specific interactions or sequences that define actions - such as action-specific features that align more closely with human cognition for space-time correspondence. This can limit the model's ability to capture the essence of certain actions that are contextually rich and continuous. Humans are capable of mapping visual concepts, object view invariance, and semantic attributes available in static instances to comprehend natural dynamic scenes or videos. Existing MAEs for videos and static images rely on separate datasets for videos and images, which may lack the rich semantic attributes necessary for fully understanding the learned concepts, especially when compared to using video and corresponding sampled frame images together. To this end, we propose CrossVideoMAE an end-to-end self-supervised cross-modal contrastive learning MAE that effectively learns both video-level and frame-level rich spatiotemporal representations and semantic attributes. Our method integrates mutual spatiotemporal information from videos with spatial information from sampled frames within a feature-invariant space, while encouraging invariance to augmentations within the video domain. This objective is achieved through jointly embedding features of visible tokens and combining feature correspondence within and across modalities, which is critical for acquiring rich, label-free guiding signals from both video and frame image modalities in a self-supervised manner. Extensive experiments demonstrate that our approach surpasses previous state-of-the-art methods and ablation studies validate the effectiveness of our approach.

Figures

Figures reproduced from arXiv: 2502.07811 by the authors.

Figure 1
Figure 1. Self-attention maps visualization of the proposed approach.. This demonstrates the efficacy of our method in learning spatiotemporal and semantic representations. The rows depict: original video frames from an action video sequence (first row), masked frames with random mask￾ing applied (second row), reconstructed frames (third row), self-attention heatmaps highlighting spatiotemporal representations (fourth row), o… view at source ↗
Figure 2
Figure 2. A). The proposed CrossVideoMAE framework comprises two branches: the video branch and the image branch. The video branch employs intra￾modal pre-training to ensure that the encoder develops invariance to augmentations within the video domain. The image branch leverages are cross-modal pre-training to distill semantic knowledge from pre-trained MAE [38], transferring insights from sampled frames to corresponding vide… view at source ↗
Figure 1
Figure 1. An example self-attention maps visualization of our [PITH_FULL_IMAGE:figures/full_fig_p020_1.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: An example self-attention maps visualization of our [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]
Figure 6
Figure 6. Figure 6: An example self-attention maps visualization of our [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]
Figure 4
Figure 4. Figure 4: An example self-attention maps visualization of our [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]
Figure 8
Figure 8. Figure 8: An example self-attention maps visualization of our [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: An example self-attention maps visualization of our [PITH_FULL_IMAGE:figures/full_fig_p022_9.png]
Figure 10
Figure 10. Figure 10: An example self-attention maps visualization of our [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]
Figure 14
Figure 14. Figure 14: An example self-attention maps visualization of our [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]
Figure 15
Figure 15. Figure 15: An example self-attention maps visualization of our [PITH_FULL_IMAGE:figures/full_fig_p023_15.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Infinite Video Understanding

    cs.CV 2025-07 conditional novelty 3.0 of 10

    The paper argues that video understanding research should aim at processing streams of arbitrary, unbounded duration and outlines the challenges, directions, and metrics needed.

Reference graph

Works this paper leans on

111 extracted references · 68 canonical work pages · cited by 1 Pith paper

  1. [1]

    Self-supervised multimodal versatile networks

    Jean-Baptiste Alayrac, Adria Recasens, Rosalia Schneider, Relja Arandjelovi ´c, Jason Ramapuram, Jeffrey De Fauw, Lucas Smaira, Sander Dieleman, and Andrew Zisserman. Self-supervised multimodal versatile networks. Advances in neural information processing systems, 33:25–37, 2020. 7

    2020

  2. [2]

    Self-supervised learning by cross-modal audio-video clustering

    Humam Alwassel, Dhruv Mahajan, Bruno Korbar, Lorenzo Torresani, Bernard Ghanem, and Du Tran. Self-supervised learning by cross-modal audio-video clustering. Advances in Neural Information Processing Systems, 33:9758–9770,

  3. [3]

    Look, listen and learn

    Relja Arandjelovic and Andrew Zisserman. Look, listen and learn. In Proceedings of the IEEE international confer- ence on computer vision, pages 609–617, 2017. 3

    2017

  4. [4]

    Objects that sound

    Relja Arandjelovic and Andrew Zisserman. Objects that sound. In Proceedings of the European Conference on Computer Vision, pages 435–451, 2018. 3

    2018

  5. [5]

    Vivit: A video vision transformer

    Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lu ˇci´c, and Cordelia Schmid. Vivit: A video vision transformer. In Proceedings of the IEEE/CVF inter- national conference on computer vision, pages 6836–6846,

  6. [6]

    Adamae: Adaptive masking for efficient spatiotempo- ral learning with masked autoencoders

    Wele Gedara Chaminda Bandara, Naman Patel, Ali Gho- lami, Mehdi Nikkhah, Motilal Agrawal, and Vishal M Pa- tel. Adamae: Adaptive masking for efficient spatiotempo- ral learning with masked autoencoders. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14507–14517, 2023. 3, 7

    2023

  7. [7]

    BEit: BERT pre-training of image transformers

    Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. BEit: BERT pre-training of image transformers. In International Conference on Learning Representations, 2022. 2, 3, 7

    2022

  8. [8]

    Speednet: Learning the speediness in videos

    Sagie Benaim, Ariel Ephrat, Oran Lang, Inbar Mosseri, William T Freeman, Michael Rubinstein, Michal Irani, and Tali Dekel. Speednet: Learning the speediness in videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9922–9931, 2020. 3, 7

    2020

Show all 111 references
  1. [9]

    Is space-time attention all you need for video understanding? In Proceedings of the 38th International Conference on Ma- chine Learning (ICML), pages 813–824

    Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? In Proceedings of the 38th International Conference on Ma- chine Learning (ICML), pages 813–824. PMLR, 2021. 4, 5

    2021

  2. [10]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021. 1

    2021

  3. [11]

    Learning aligned cross-modal representations from weakly aligned data

    Lluis Castrejon, Yusuf Aytar, Carl V ondrick, Hamed Pirsi- avash, and Antonio Torralba. Learning aligned cross-modal representations from weakly aligned data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 2940–2949, 2016. 3

    2016

  4. [12]

    Generative pre- training from pixels

    Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Hee- woo Jun, David Luan, and Ilya Sutskever. Generative pre- training from pixels. In International conference on ma- chine learning, pages 1691–1703. PMLR, 2020. 2

    2020

  5. [13]

    Rspnet: Relative speed perception for unsupervised video representation learning

    Peihao Chen, Deng Huang, Dongliang He, Xiang Long, Runhao Zeng, Shilei Wen, Mingkui Tan, and Chuang Gan. Rspnet: Relative speed perception for unsupervised video representation learning. In Proceedings of the AAAI Con- ference on Artificial Intelligence , pages 1045–1053, 2021. 7

    2021

  6. [14]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PMLR, 2020. 3, 4

    2020

  7. [15]

    Electra: Pre-training text encoders as dis- criminators rather than generators

    Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christo- pher D Manning. Electra: Pre-training text encoders as dis- criminators rather than generators. In International Confer- ence on Learning Representations, 2020. 3

    2020

  8. [16]

    Randaugment: Practical automated data augmen- tation with a reduced search space

    Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmen- tation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702–703, 2020. 8, 3

    2020

  9. [17]

    Unifying video self-supervised learning across families of tasks: A survey

    Ishan Dave, Malitha Gunawardhana, Limalka Sadith, Honglu Zhou, Liel David, Daniel Harari, Mubarak Shah, and Muhammad Khan. Unifying video self-supervised learning across families of tasks: A survey. 2024. 2

    2024

  10. [18]

    Imagenet: A large-scale hierarchical im- age database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical im- age database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 8

    2009

  11. [19]

    Virtex: Learning visual representations from textual annotations

    Karan Desai and Justin Johnson. Virtex: Learning visual representations from textual annotations. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11162–11173, 2021. 2, 3

    2021

  12. [20]

    9 Vi2clr: Video and image for visual contrastive learning of representation

    Ali Diba, Vivek Sharma, Reza Safdari, Dariush Lotfi, Saquib Sarfraz, Rainer Stiefelhagen, and Luc Van Gool. 9 Vi2clr: Video and image for visual contrastive learning of representation. In Proceedings of the IEEE/CVF interna- tional conference on computer vision , pages 1502–1512,

  13. [21]

    Taming transformers for high-resolution image synthesis

    Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. 2

    2021

  14. [22]

    Multiscale vision transformers

    Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, and Christoph Feichten- hofer. Multiscale vision transformers. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 6824–6835, 2021. 4, 5

    2021

  15. [23]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202–6211, 2019. 4, 5

    2019

  16. [24]

    A large-scale study on unsuper- vised spatiotemporal representation learning

    Christoph Feichtenhofer, Haoqi Fan, Bo Xiong, Ross Gir- shick, and Kaiming He. A large-scale study on unsuper- vised spatiotemporal representation learning. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3299–3309, 2021. 3, 4, 5

    2021

  17. [25]

    Masked autoencoders as spatiotemporal learners

    Christoph Feichtenhofer, Yanghao Li, Kaiming He, et al. Masked autoencoders as spatiotemporal learners. Ad- vances in neural information processing systems , 35: 35946–35958, 2022. 3, 6, 7, 8, 1, 2

    2022

  18. [26]

    Mcmae: Masked convolution meets masked autoencoders

    Peng Gao, Teli Ma, Hongsheng Li, Ziyi Lin, Jifeng Dai, and Yu Qiao. Mcmae: Masked convolution meets masked autoencoders. Advances in Neural Information Processing Systems, 35:35632–35644, 2022. 7

    2022

  19. [27]

    Revitalizing cnn attention via transformers in self-supervised visual representation learn- ing

    Chongjian Ge, Youwei Liang, Yibing Song, Jianbo Jiao, Jue Wang, and Ping Luo. Revitalizing cnn attention via transformers in self-supervised visual representation learn- ing. Advances in Neural Information Processing Systems , 34:4193–4206, 2021. 3

    2021

  20. [28]

    Omnimae: Single model masked pretraining on images and videos

    Rohit Girdhar, Alaaeldin El-Nouby, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Omnimae: Single model masked pretraining on images and videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10406– 10417, 2023...

    2023

  21. [29]

    Improving image-sentence embeddings using large weakly annotated photo collec- tions

    Yunchao Gong, Liwei Wang, Micah Hodosh, Julia Hocken- maier, and Svetlana Lazebnik. Improving image-sentence embeddings using large weakly annotated photo collec- tions. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedi...

    2014

  22. [30]

    Accurate, large mini- batch sgd: Training imagenet in 1 hour

    Priya Goyal, Piotr Doll ´ar, Ross Girshick, Pieter Noord- huis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. Accurate, large mini- batch sgd: Training imagenet in 1 hour. arXiv preprint arXiv:1706.02677, 2017. 2, 4

    2017 arXiv

  23. [31]

    The” something something” video database for learning and evaluating visual common sense

    Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michal- ski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al. The” something something” video database for learning and evaluating visual common sense. In ...

    2017

  24. [32]

    Bootstrap your own latent-a new approach to self-supervised learning

    Jean-Bastien Grill, Florian Strub, Florent Altch ´e, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Do- ersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. Advances in neur...

    2020

  25. [33]

    How effec- tive are self-supervised models for contact identification in videos

    Malitha Gunawardhana, Limalka Sadith, Liel David, Daniel Harari, and Muhammad Haris Khan. How effec- tive are self-supervised models for contact identification in videos. arXiv preprint arXiv:2408.00498, 2024. 2

    2024 arXiv

  26. [34]

    Maskvit: Masked visual pre-training for video prediction

    Agrim Gupta, Stephen Tian, Yunzhi Zhang, Jiajun Wu, Roberto Mart´ın-Mart´ın, and Li Fei-Fei. Maskvit: Masked visual pre-training for video prediction. arXiv preprint arXiv:2206.11894, 2022. 3

    2022 arXiv

  27. [35]

    Memory- augmented dense predictive coding for video representa- tion learning

    Tengda Han, Weidi Xie, and Andrew Zisserman. Memory- augmented dense predictive coding for video representa- tion learning. In European Conference on Computer Vision, pages 312–329. Springer, 2020. 3, 7

    2020

  28. [36]

    Self- supervised co-training for video representation learning

    Tengda Han, Weidi Xie, and Andrew Zisserman. Self- supervised co-training for video representation learning. Advances in Neural Information Processing Systems , 33: 5679–5690, 2020. 4, 5

    2020

  29. [37]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 9729–9738, 2020. 2, 3, 4

    2020

  30. [38]

    Masked autoencoders are scal- able vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scal- able vision learners. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 16000–16009, 2022. 2, 3, 4, 6, 7, 8, 1

    2022

  31. [39]

    Human gaze control during real-world scene perception

    John M Henderson. Human gaze control during real-world scene perception. Trends in cognitive sciences, 7(11):498– 504, 2003. 2

    2003

  32. [40]

    Visual representation learning from unlabeled video using contrastive masked autoencoders

    Jefferson Hernandez, Ruben Villegas, and Vicente Or- donez. Visual representation learning from unlabeled video using contrastive masked autoencoders. arXiv preprint arXiv:2303.12001, 2023. 7

    2023 arXiv

  33. [41]

    Augment your batch: Improv- ing generalization through instance repetition

    Elad Hoffer, Tal Ben-Nun, Itay Hubara, Niv Giladi, Torsten Hoefler, and Daniel Soudry. Augment your batch: Improv- ing generalization through instance repetition. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8129–8138, 2020. 3

    2020

  34. [42]

    Contrast and order representa- tions for video self-supervised learning

    Kai Hu, Jie Shao, Yuan Liu, Bhiksha Raj, Marios Sav- vides, and Zhiqiang Shen. Contrast and order representa- tions for video self-supervised learning. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 7939–7949, 2021. 4

    2021

  35. [43]

    Mgmae: Motion guided masking for video masked autoencoding

    Bingkun Huang, Zhiyu Zhao, Guozhen Zhang, Yu Qiao, and Limin Wang. Mgmae: Motion guided masking for video masked autoencoding. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 13493–13504, 2023. 3, 7 10

    2023

  36. [44]

    Self-supervised video representation learn- ing by context and motion decoupling

    Lianghua Huang, Yu Liu, Bin Wang, Pan Pan, Yinghui Xu, and Rong Jin. Self-supervised video representation learn- ing by context and motion decoupling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13886–13895, 2021. 2

    2021

  37. [45]

    Tada! temporally-adaptive convolutions for video understanding

    Ziyuan Huang, Shiwei Zhang, Liang Pan, Zhiwu Qing, Mingqian Tang, Ziwei Liu, and Marcelo H Ang Jr. Tada! temporally-adaptive convolutions for video understanding. arXiv preprint arXiv:2110.06178, 2021. 4, 5

    2021 arXiv

  38. [46]

    Self-supervised spatiotemporal feature learn- ing via video rotation prediction

    Longlong Jing, Xiaodong Yang, Jingen Liu, and Yingli Tian. Self-supervised spatiotemporal feature learn- ing via video rotation prediction. arXiv preprint arXiv:1811.11387, 2018. 2

    2018 arXiv

  39. [47]

    Hard negative mixing for contrastive learning

    Yannis Kalantidis, Mert Bulent Sariyildiz, Noe Pion, Philippe Weinzaepfel, and Diane Larlus. Hard negative mixing for contrastive learning. Advances in neural infor- mation processing systems, 33:21798–21809, 2020. 2

    2020

  40. [48]

    Deep visual-semantic alignments for generating image descriptions

    Andrej Karpathy and Li Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3128–3137, 2015. 3

    2015

  41. [49]

    The kinetics human action video dataset

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Vi- ola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics human action video dataset. arXiv preprint arXiv:1705.06950, 2017. 6, 7, 2, 3

    2017 arXiv

  42. [50]

    Fusion attention for action recognition: Integrating sparse-dense and global at- tention for video action recognition

    Hyun-Woo Kim and Yong-Suk Choi. Fusion attention for action recognition: Integrating sparse-dense and global at- tention for video action recognition. Sensors, 24(21):6842,

  43. [51]

    Co- operative learning of audio and video models from self- supervised synchronization

    Bruno Korbar, Du Tran, and Lorenzo Torresani. Co- operative learning of audio and video models from self- supervised synchronization. Advances in Neural Informa- tion Processing Systems, 31, 2018. 2, 3

    2018

  44. [52]

    Hmdb: a large video database for human motion recognition

    Hildegard Kuehne, Hueihan Jhuang, Est ´ıbaliz Garrote, Tomaso Poggio, and Thomas Serre. Hmdb: a large video database for human motion recognition. In 2011 Interna- tional conference on computer vision , pages 2556–2563. IEEE, 2011. 6, 7, 3

    2011

  45. [53]

    A large-scale analysis on self- supervised video representation learning

    Akash Kumar, Ashlesha Kumar, Vibhav Vineet, and Yo- gesh Singh Rawat. A large-scale analysis on self- supervised video representation learning. arXiv e-prints , pages arXiv–2306, 2023. 2

    2023

  46. [54]

    Semmae: Semantic-guided masking for learning masked autoencoders

    Gang Li, Heliang Zheng, Daqing Liu, Chaoyue Wang, Bing Su, and Changwen Zheng. Semmae: Semantic-guided masking for learning masked autoencoders. Advances in Neural Information Processing Systems, 35:14290–14302,

  47. [55]

    Improve unsuper- vised pretraining for few-label transfer

    Suichan Li, Dongdong Chen, Yinpeng Chen, Lu Yuan, Lei Zhang, Qi Chu, Bin Liu, and Nenghai Yu. Improve unsuper- vised pretraining for few-label transfer. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 10201–10210, 2021. 3

    2021

  48. [56]

    Learning spatiotemporal fea- tures via video and text pair discrimination

    Tianhao Li and Limin Wang. Learning spatiotemporal fea- tures via video and text pair discrimination. arXiv preprint arXiv:2001.05691, 2020. 7

    2001 arXiv

  49. [57]

    Tsm: Temporal shift module for efficient video understanding

    Ji Lin, Chuang Gan, and Song Han. Tsm: Temporal shift module for efficient video understanding. In Proceedings of the IEEE/CVF international conference on computer vi- sion, pages 7083–7093, 2019. 4

    2019

  50. [58]

    Self-supervised video-based action recognition with distur- bances

    Wei Lin, Xinghao Ding, Yue Huang, and Huanqiang Zeng. Self-supervised video-based action recognition with distur- bances. IEEE Transactions on Image Processing, 32:2493– 2507, 2023. 2

    2023

  51. [59]

    Crossvideo: Self-supervised cross-modal contrastive learn- ing for point cloud video understanding

    Yunze Liu, Changxi Chen, Zifan Wang, and Li Yi. Crossvideo: Self-supervised cross-modal contrastive learn- ing for point cloud video understanding. arXiv preprint arXiv:2401.09057, 2024. 2, 3

    2024 arXiv

  52. [60]

    Teinet: Towards an efficient architecture for video recognition

    Zhaoyang Liu, Donghao Luo, Yabiao Wang, Limin Wang, Ying Tai, Chengjie Wang, Jilin Li, Feiyue Huang, and Tong Lu. Teinet: Towards an efficient architecture for video recognition. In Proceedings of the AAAI conference on ar- tificial intelligence, pages 11669–11676, 2020. 4, 5

    2020

  53. [61]

    Tam: Temporal adaptive module for video recog- nition

    Zhaoyang Liu, Limin Wang, Wayne Wu, Chen Qian, and Tong Lu. Tam: Temporal adaptive module for video recog- nition. In Proceedings of the IEEE/CVF international con- ference on computer vision, pages 13708–13718, 2021. 4, 5

    2021

  54. [62]

    Video swin transformer

    Ze Liu, Jia Ning, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin, and Han Hu. Video swin transformer. In Pro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3202–3211, 2022. 1

    2022

  55. [63]

    Sgdr: Stochastic gradient descent with warm restarts

    Ilya Loshchilov and Frank Hutter. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983, 2016. 2

    2016 arXiv

  56. [64]

    Decoupled weight de- cay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learn- ing Representations, 2019. 6, 2

    2019

  57. [65]

    Cmae-v: Contrastive masked autoencoders for video action recognition

    Cheng-Ze Lu, Xiaojie Jin, Zhicheng Huang, Qibin Hou, Ming-Ming Cheng, and Jiashi Feng. Cmae-v: Contrastive masked autoencoders for video action recognition. arXiv preprint arXiv:2301.06018, 2023. 7

    2023 arXiv

  58. [66]

    12-in-1: Multi-task vision and language representation learning

    Jiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh, and Stefan Lee. 12-in-1: Multi-task vision and language representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10437–10446, 2020. 3

    2020

  59. [67]

    End-to-end learning of visual representations from uncurated instruc- tional videos

    Antoine Miech, Jean-Baptiste Alayrac, Lucas Smaira, Ivan Laptev, Josef Sivic, and Andrew Zisserman. End-to-end learning of visual representations from uncurated instruc- tional videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages ...

    2020

  60. [68]

    Self-supervised learning of pretext-invariant representations

    Ishan Misra and Laurens van der Maaten. Self-supervised learning of pretext-invariant representations. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6707–6717, 2020. 2

    2020

  61. [69]

    Shuffle and learn: unsupervised learning using temporal order verification

    Ishan Misra, C Lawrence Zitnick, and Martial Hebert. Shuffle and learn: unsupervised learning using temporal order verification. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, Octo- ber 11–14, 2016, Proceedings, Part I 14 , pages 527–544. S...

    2016

  62. [70]

    Ro- bust audio-visual instance discrimination

    Pedro Morgado, Ishan Misra, and Nuno Vasconcelos. Ro- bust audio-visual instance discrimination. In Proceedings 11 of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 12934–12945, 2021. 2, 3

    2021

  63. [71]

    Audio-visual instance discrimination with cross-modal agreement

    Pedro Morgado, Nuno Vasconcelos, and Ishan Misra. Audio-visual instance discrimination with cross-modal agreement. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12475– 12486, 2021. 1

    2021

  64. [72]

    Audio-visual scene analysis with self-supervised multisensory features

    Andrew Owens and Alexei A Efros. Audio-visual scene analysis with self-supervised multisensory features. InPro- ceedings of the European conference on computer vision (ECCV), pages 631–648, 2018. 3

    2018

  65. [73]

    Videomoco: Contrastive video representation learning with temporally adversarial examples

    Tian Pan, Yibing Song, Tianyu Yang, Wenhao Jiang, and Wei Liu. Videomoco: Contrastive video representation learning with temporally adversarial examples. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11205–11214, 2021. 2, 3, 7, 4

    2021

  66. [74]

    Multi-modal self-supervision from generalized data transformations

    Mandela Patrick, Yuki Asano, Polina Kuznetsova, Ruth Fong, Joao F Henriques, Geoffrey Zweig, and Andrea Vedaldi. Multi-modal self-supervision from generalized data transformations. In IEEE/CVF International Confer- ence on Computer Vision, 2021. 7, 4

    2021

  67. [75]

    Keeping your eye on the ball: Trajectory attention in video transformers

    Mandela Patrick, Dylan Campbell, Yuki Asano, Ishan Misra, Florian Metze, Christoph Feichtenhofer, Andrea Vedaldi, and Joao F Henriques. Keeping your eye on the ball: Trajectory attention in video transformers. Ad- vances in neural information processing systems , 34: 12493–125...

    2021

  68. [76]

    Evolving losses for unsupervised video representation learning

    AJ Piergiovanni, Anelia Angelova, and Michael S Ryoo. Evolving losses for unsupervised video representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 133–142,

  69. [77]

    Spa- tiotemporal contrastive video representation learning

    Rui Qian, Tianjian Meng, Boqing Gong, Ming-Hsuan Yang, Huisheng Wang, Serge Belongie, and Yin Cui. Spa- tiotemporal contrastive video representation learning. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 6964–6974, 2021. 2, 3, 7, 4

    2021

  70. [78]

    Learning from untrimmed videos: Self-supervised video representation learning with hierarchical consistency

    Zhiwu Qing, Shiwei Zhang, Ziyuan Huang, Yi Xu, Xiang Wang, Mingqian Tang, Changxin Gao, Rong Jin, and Nong Sang. Learning from untrimmed videos: Self-supervised video representation learning with hierarchical consistency. In Proceedings of the IEEE/CVF Conference on Computer V...

    2022

  71. [79]

    Mar: Masked autoencoders for efficient action recognition

    Zhiwu Qing, Shiwei Zhang, Ziyuan Huang, Xiang Wang, Yuehuan Wang, Yiliang Lv, Changxin Gao, and Nong Sang. Mar: Masked autoencoders for efficient action recognition. IEEE Transactions on Multimedia, 2023. 3, 7

    2023

  72. [80]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International conference on machine learning...

    2021

  73. [81]

    Self-supervised video transformer

    Kanchana Ranasinghe, Muzammal Naseer, Salman Khan, Fahad Shahbaz Khan, and Michael S Ryoo. Self-supervised video transformer. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 2874–2884, 2022. 7, 4, 5

    2022

  74. [82]

    The development of visual attention and the brain

    John E Richards. The development of visual attention and the brain. The cognitive neuroscience of development, pages 73–98, 2003. 2

    2003

  75. [83]

    Im- agenet large scale visual recognition challenge

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Im- agenet large scale visual recognition challenge. Interna- tional journal of computer vision, 115:211–252, 2015. 8, 2, 3, 6, 7

    2015

  76. [84]

    Learning visual representations with caption annotations

    Mert Bulent Sariyildiz, Julien Perez, and Diane Larlus. Learning visual representations with caption annotations. In Computer Vision–ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII 16, pages 153–170. Springer, 2020. 2, 3

    2020

  77. [85]

    Two-stream con- volutional networks for action recognition in videos

    Karen Simonyan and Andrew Zisserman. Two-stream con- volutional networks for action recognition in videos. Ad- vances in neural information processing systems, 27, 2014. 3

    2014

  78. [86]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    K Soomro. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402,

    arXiv

  79. [87]

    Masked motion encoding for self-supervised video representation learning

    Xinyu Sun, Peihao Chen, Liangwei Chen, Changhao Li, Thomas H Li, Mingkui Tan, and Chuang Gan. Masked motion encoding for self-supervised video representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2235– 2245, 2023. 3, 7, 4

    2023

  80. [88]

    Rethinking the inception architecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 2818–2826, 2016. 3

    2016

  81. [89]

    Vimpac: Video pre-training via masked token prediction and con- trastive learning

    Hao Tan, Jie Lei, Thomas Wolf, and Mohit Bansal. Vimpac: Video pre-training via masked token prediction and con- trastive learning. arXiv preprint arXiv:2106.11250, 2021. 7, 4

    2021 arXiv

  82. [90]

    Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. Advances in neural information processing systems, 35:10078–10093, 2022. 3, 6, 7, 8, 4, 5

    2022

  83. [91]

    Learning spatiotemporal fea- tures with 3d convolutional networks

    Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torre- sani, and Manohar Paluri. Learning spatiotemporal fea- tures with 3d convolutional networks. InProceedings of the IEEE international conference on computer vision , pages 4489–4497, 2015. 2, 3

    2015

  84. [92]

    Video classification with channel-separated convolutional networks

    Du Tran, Heng Wang, Lorenzo Torresani, and Matt Feiszli. Video classification with channel-separated convolutional networks. In Proceedings of the IEEE/CVF international conference on computer vision, pages 5552–5561, 2019. 5

    2019

  85. [93]

    Spa- tiotemporal integration and object perception in infancy: Perceiving unity versus form

    Gretchen A Van, de Walle and Elizabeth S Spelke. Spa- tiotemporal integration and object perception in infancy: Perceiving unity versus form. Child Development, 67(6): 2621–2640, 1996. 2 12

    1996

  86. [94]

    Generating videos with scene dynamics

    Carl V ondrick, Hamed Pirsiavash, and Antonio Torralba. Generating videos with scene dynamics. Advances in neu- ral information processing systems, 29, 2016. 2

    2016

  87. [95]

    Unsupervised visual rep- resentation learning by tracking patches in video

    Guangting Wang, Yizhou Zhou, Chong Luo, Wenxuan Xie, Wenjun Zeng, and Zhiwei Xiong. Unsupervised visual rep- resentation learning by tracking patches in video. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2563–2572, 2021. 2

    2021

  88. [96]

    Self- supervised video representation learning by pace predic- tion

    Jiangliu Wang, Jianbo Jiao, and Yun-Hui Liu. Self- supervised video representation learning by pace predic- tion. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceed- ings, Part XVII 16, pages 504–521. Springer, 2020. 7

    2020

  89. [97]

    Self-supervised temporal discriminative learning for video representation learning

    Jinpeng Wang, Yiqi Lin, Andy J Ma, and Pong C Yuen. Self-supervised temporal discriminative learning for video representation learning. arXiv preprint arXiv:2008.02129,

    2008 arXiv

  90. [98]

    Temporal segment networks for action recognition in videos

    Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. Temporal segment networks for action recognition in videos. IEEE transac- tions on pattern analysis and machine intelligence, 41(11): 2740–2755, 2018. 2, 3

    2018

  91. [99]

    Tdn: Temporal difference networks for efficient action recog- nition

    Limin Wang, Zhan Tong, Bin Ji, and Gangshan Wu. Tdn: Temporal difference networks for efficient action recog- nition. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1895–1904,

    1904

  92. [100]

    Videomae v2: Scaling video masked autoencoders with dual masking

    Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yi- nan He, Yi Wang, Yali Wang, and Yu Qiao. Videomae v2: Scaling video masked autoencoders with dual masking. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition , pages 14549–14560, 2023. 3

    2023

  93. [101]

    Bevt: Bert pretraining of video transformers

    Rui Wang, Dongdong Chen, Zuxuan Wu, Yinpeng Chen, Xiyang Dai, Mengchen Liu, Yu-Gang Jiang, Luowei Zhou, and Lu Yuan. Bevt: Bert pretraining of video transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14733–14743, 2022. 2, 3, 7

    2022

  94. [102]

    Masked video distillation: Rethinking masked feature mod- eling for self-supervised video representation learning

    Rui Wang, Dongdong Chen, Zuxuan Wu, Yinpeng Chen, Xiyang Dai, Mengchen Liu, Lu Yuan, and Yu-Gang Jiang. Masked video distillation: Rethinking masked feature mod- eling for self-supervised video representation learning. In Proceedings of the IEEE/CVF conference on computer vi- ...

    2023

  95. [103]

    Unsupervised learn- ing of visual representations using videos

    Xiaolong Wang and Abhinav Gupta. Unsupervised learn- ing of visual representations using videos. In Proceedings of the IEEE international conference on computer vision , pages 2794–2802, 2015. 3

    2015

  96. [104]

    Non-local neural networks

    Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaim- ing He. Non-local neural networks. In Proceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 7794–7803, 2018. 5

    2018

  97. [105]

    Masked feature pre- diction for self-supervised visual pre-training

    Chen Wei, Haoqi Fan, Saining Xie, Chao-Yuan Wu, Alan Yuille, and Christoph Feichtenhofer. Masked feature pre- diction for self-supervised visual pre-training. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14668–14678, 2022. 3

    2022

  98. [106]

    Self-supervised spatiotemporal learn- ing via video clip order prediction

    Dejing Xu, Jun Xiao, Zhou Zhao, Jian Shao, Di Xie, and Yueting Zhuang. Self-supervised spatiotemporal learn- ing via video clip order prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10334–10343, 2019. 3, 5

    2019

  99. [107]

    Video playback rate perception for self-supervised spatio-temporal representation learning

    Yuan Yao, Chang Liu, Dezhao Luo, Yu Zhou, and Qixi- ang Ye. Video playback rate perception for self-supervised spatio-temporal representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6548–6557, 2020. 2

    2020

  100. [108]

    Cutmix: Regu- larization strategy to train strong classifiers with localizable features

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. Cutmix: Regu- larization strategy to train strong classifiers with localizable features. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6023–6032, 2019. 8, 3

    2019

  101. [109]

    Co-training transformer with videos and images improves action recog- nition

    Bowen Zhang, Jiahui Yu, Christopher Fifty, Wei Han, An- drew M Dai, Ruoming Pang, and Fei Sha. Co-training transformer with videos and images improves action recog- nition. arXiv preprint arXiv:2112.07175, 2021. 2, 3

    2021 arXiv

  102. [110]

    mixup: Beyond empirical risk minimiza- tion

    Hongyi Zhang. mixup: Beyond empirical risk minimiza- tion. arXiv preprint arXiv:1710.09412, 2017. 8, 3

    2017 arXiv

  103. [111]

    Random erasing data augmentation

    Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation. In Proceed- ings of the AAAI conference on artificial intelligence, pages 13001–13008, 2020. 8 13 CrossVideoMAE: Self-Supervised Image-Video Representation Learning with Masked Au...

    2020

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.