Pith. sign in

REVIEW 3 major objections 7 minor 54 references

Video-to-Task Learning via Motion-Guided Attention for Few-Shot Action Recognition

T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Few-shot action recognition improves when a model first finds motion-related regions inside one video and then correlates them across all videos in the task.

desk verdict A credible incremental improvement with a real new task-level attention module, but the frame-wise alignment assumption and test-set hyperparameter tuning need addressing. read the letter →

arxiv 2411.11335 v1 pith:CQOLQVDA submitted 2024-11-18 cs.CV

Hanyu Guo , Wanchuan Yu , Suzhou Que , Kaiwen Du , Yan Yan , Hanzi Wang This is my paper · ORCID
classification cs.CV
keywords few-shotactionrecognitionmotion-guidedattentiontask-levelrelationmodelingspatio-temporalfeaturesadaptertuningprototypelearningvideo-to-task
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a method, DMGAL, for few-shot action recognition that learns spatio-temporal relationships in two stages: first within each video, then across all videos in a task. It claims this video-to-task learning produces class prototypes that capture motion-related regions better than existing video-level alignment methods. If true, it would give a consistent accuracy boost on standard few-shot action benchmarks under both fully fine-tuned and adapter-tuned settings. The method is built from two motion-guided attention modules that use bidirectional multi-scale motion features to identify the most relevant regions.

What carries the argument

Motion-Guided Attention (MGA), composed of S-MGA and C-MGA. S-MGA uses bidirectional multi-scale motion features between adjacent frames to build a self-association score matrix that highlights motion-related patches, followed by temporal and channel MLPs for global context. C-MGA builds frame-wise cross-association score matrices between each query frame and the corresponding frame of every support video, pulling same-class motion regions together and pushing different-class ones apart. This frame-wise design keeps memory low and avoids overly sparse correspondences.

What would settle it

An experiment that shifts query frames relative to support frames, for example by 1 to 3 positions out of 8 sampled frames, and measures 5-way 1-shot accuracy on SSv2-Full would settle whether the frame-wise alignment assumption is load-bearing. The central claim would be weakened if DMGAL's gain over baselines drops sharply under small temporal shifts while baseline methods are unaffected.

Watch

Extended reading notes

Core claim

The central claim is that spatio-temporal relation modeling for few-shot action recognition should be done not only at the video level but also at the task level. The paper's Self Motion-Guided Attention (S-MGA) identifies which spatial patches in adjacent frames move together and correlates them within a single video, while its Cross Motion-Guided Attention (C-MGA) does the same across the query and all support videos in an episode. The authors argue that this sequential video-to-task correlation creates class prototypes that incorporate motion-related region features, increasing inter-class separation and reducing intra-class spread. The paper reports that DMGAL-FT and DMGAL-Adapter outperform prior state-of-the-art methods on five few-shot action recognition benchmarks, with the largest gains on temporal datasets like Something-Something.

Load-bearing premise

C-MGA assumes that the i-th uniformly sampled frame of the query video corresponds semantically to the i-th frame of every support video in the task, even though actions in different videos are rarely synchronized in time.

Editorial extensions

If this is right

  • Both the fully fine-tuned and adapter-tuned variants can be combined with existing matching metrics such as OTAM, TRX, SCA, and Bi-MHM, and the paper reports gains with all four.
  • Adding the two modules to a TRX baseline improves 1-shot accuracy by 7.4 percentage points on UCF101 and 10.4 on SSv2-Full in the paper's Table V.
  • The adapter variant keeps the CLIP backbone frozen and only tunes lightweight adapters, so the approach transfers to parameter-efficient settings.
  • Because C-MGA acts at task level, the method should scale to settings with more support videos per class, where task-level relations are richer.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to replace the fixed frame-wise alignment in C-MGA with a learned or dynamic time warping alignment, which would likely improve robustness when actions in different videos are not synchronized.
  • The same video-to-task attention idea could be applied to other few-shot video problems such as temporal action localization or video question answering, where task-level relations between support and query matter.
  • One testable extension is to measure accuracy under artificial temporal shifts of query frames to see how much the frame-wise correspondence assumption costs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes DMGAL, a few-shot action recognition method built from two modules: S-MGA, which applies motion-guided self-attention within a single video to model video-level spatio-temporal relationships, and C-MGA, which applies frame-wise cross-attention between query and support videos to model task-level spatio-temporal relationships. The method is instantiated in two paradigms: DMGAL-FT with a fine-tuned ResNet-50, and DMGAL-Adapter with a frozen CLIP-ViT and lightweight adapters. The two instantiations are evaluated on SSv2-Full, SSv2-Small, Kinetics, HMDB51 and UCF101 using four matching metrics (OTAM, SCA, Bi-MHM, TRX). Ablation studies show that both S-MGA and C-MGA contribute to accuracy, and visualizations/t-SNE plots support the intended motion-guided behavior. The paper claims favorable or state-of-the-art results across the five benchmarks.

Significance. If the results hold, the paper addresses a genuine gap in few-shot action recognition, where most prior methods model spatio-temporal alignment at the video level while task-level spatio-temporal relationships are less explored. The paper's strongest assets are its broad empirical evaluation across two training paradigms and four matching metrics, the clean ablations showing consistent gains from both modules, and the parameter-efficient adapter variant. The central claims, however, rest on two points that need additional support: the frame-wise alignment assumption in C-MGA and the selection of hyperparameters on the same test benchmarks used for final evaluation. The paper does not report error bars or significance tests, which is important given that some reported improvements are small. These issues are fixable but currently prevent a full endorsement.

major comments (3)
  1. [Section III-B2 (Eqs. 10-12)] The frame-wise C-MGA cross-attention assumes that the i-th uniformly sampled frame of the query is semantically aligned with the i-th frame of every support video. In SSv2, Kinetics, UCF101 and HMDB51, same-class videos are rarely phase-aligned, so the softmax in Eq. (12) can mix motion-related features from unrelated action phases. Since C-MGA is the only task-level module and it enhances features before downstream matching, the matching metrics (OTAM/TRX/Bi-MHM/SCA) cannot correct this misalignment. The paper does not test robustness to temporal shifts, nor does it compare with a variant that performs temporal alignment before cross-attention; the frame-wise vs. frame-all ablation in Table VIII changes only the support frame set, not the alignment rule. The attribution of the accuracy gains to task-level spatio-temporal modeling is therefore not fully supported. Please add a sensitivity analysis to temporal shifts and/or a comparison with an aligned variant.
  2. [Section IV-D4 (Fig. 7)] The compression factors r1 and r2 are selected using the 5-way 1-shot accuracy on SSv2-Full (for DMGAL-FT) and HMDB51 (for DMGAL-Adapter), which are exactly the test benchmarks reported in Tables II-IV. This is a form of test-set validation and makes the reported numbers on those datasets optimistic. The authors should tune hyperparameters on the held-out validation splits (available per Table I) and report final test results with the hyperparameters fixed. Without this, the claim in Section IV-D4 that the chosen values achieve the best results is not supported for the final evaluation.
  3. [Tables II-IV and Section IV-C] The paper reports single accuracy numbers (averaged over 10,000 random episodes) without error bars or any significance testing. Several claimed improvements are very small (e.g., 0.1-1.3% in Table IV), and in at least one case DMGAL is numerically below a prior method (D^2ST-Adapter on SSv2-Small 5-shot). For the central claim of significant or favorable improvement, the paper should report mean ± standard deviation over multiple episode draws or multiple training runs, and state the number of trials. This is essential for assessing whether the consistent gains are real rather than noise.
minor comments (7)
  1. [Section III-B2 (Eqs. 10-14)] The notation F_S and M_S is used before being defined; please specify their shapes explicitly, e.g., F_S ∈ R^{NK×L×D×H×W} and M_S ∈ R^{L×NKHW×D/r2}.
  2. [Table II] The HyRSM row lists SSv2-Small 5-shot accuracy as 46.1, which is lower than the 3-shot value 52.3 in the same row; please verify this value against the original HyRSM paper.
  3. [Section IV-D5] The sentence beginning 'One of the main that C-MGA faces in modeling spatio-temporal relations at the task level is...' is missing a noun; it should read 'One of the main challenges...'.
  4. [Figure 4 caption] The caption says 'the UCF and SSv2-small datasets' but should say 'the UCF101 and SSv2-Small datasets'.
  5. [Table VII] The column header 'Parm' should be spelled 'Param' or 'Parameters'.
  6. [Section IV-B] The phrase 'randomly sample 8 frames uniformly' is imprecise; TSN sampling selects one frame per uniformly divided temporal segment with a random offset, so please rephrase to avoid the apparent contradiction.
  7. [Section IV-C1] The sentence 'DMGAL-FT outperforms the four matching metrics by a significant margin' is misworded; the four matching metrics are baseline methods (OTAM, SCA, Bi-MHM, TRX) rather than metrics being outperformed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the central claims are benchmark-grounded and the BiMACL self-citation is component reuse, not load-bearing.

full rationale

The paper's central claim is empirical: DMGAL-FT and DMGAL-Adapter are evaluated on five standard few-shot action recognition benchmarks against published state-of-the-art methods, with train/test class splits from prior work (OTAM/CMN/ARN). The proposed S-MGA and C-MGA are trained end-to-end with standard episodic classification losses, and the reported gains are measured on held-out test episodes rather than derived from the method's definitions. The only direct reuse of the authors' own prior work is the bidirectional multi-scale motion-extraction design in S-MGA and C-MGA, credited to BiMACL [38]; that is an architectural component taken from a separately published ICASSP 2024 paper, and the present training objectives, matching metrics, and benchmark results do not reduce to a claim imported from that citation. The frame-wise index alignment in Eqs. (10)-(12) is an untested modeling assumption and a potential correctness risk for temporal misalignment, but it is not circular: the paper does not define the query or support features in terms of the accuracy it predicts, and the benchmark numbers are external measurements. No fitted parameter is renamed as a prediction, no uniqueness theorem is invoked, and no derivation reduces by construction to its own inputs. Therefore, no circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities; it proposes neural network modules composed of existing operations.

free parameters (2)
  • r1, compression factor in S-MGA = 8 for DMGAL-FT, 4 for DMGAL-Adapter
    Chosen by ablation on SSv2-Full (DMGAL-FT) and HMDB51 (DMGAL-Adapter) in Section IV-D4, i.e., on the same test benchmarks used for final reporting.
  • r2, compression factor in C-MGA = 8 for DMGAL-FT, 4 for DMGAL-Adapter
    Same ablation as r1; the paper explicitly states best values differ between paradigms.
assumptions (4)
  • domain assumption Episodic training with disjoint train/test classes generalizes to novel classes
    Standard few-shot learning protocol used in Section III-A; not proven in the paper.
  • domain assumption Uniform sampling of 8 frames preserves enough temporal structure for action recognition
    Adopted from TSN and prior work (Section IV-B); no analysis of sampling rate sensitivity beyond a frame-count ablation.
  • domain assumption Adjacent-frame differences (Eqs. 1-3) yield reliable motion features for guiding attention
    Core of S-MGA and C-MGA; inherited from BiMACL without independent validation in this paper.
  • domain assumption Frame-wise cross-attention (same frame index across videos) provides sufficient task-level alignment
    Introduced in Section III-B2 to reduce computation; this assumption is load-bearing and untested under temporal misalignment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Video-to-Task Learning via Motion-Guided Attention for Few-Shot Action Recognition." pith.science (2026). https://pith.science/paper/CQOLQVDA

@misc{pith2026241111335,
  author       = {Pith},
  title        = {Pith review of: Video-to-Task Learning via Motion-Guided Attention for Few-Shot Action Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CQOLQVDA}},
  note         = {Machine review of arXiv:2411.11335}
}
read the original abstract

In recent years, few-shot action recognition has achieved remarkable performance through spatio-temporal relation modeling. Although a wide range of spatial and temporal alignment modules have been proposed, they primarily address spatial or temporal misalignments at the video level, while the spatio-temporal relationships across different videos at the task level remain underexplored. Recent studies utilize class prototypes to learn task-specific features but overlook the spatio-temporal relationships across different videos at the task level, especially in the spatial dimension, where these relationships provide rich information. In this paper, we propose a novel Dual Motion-Guided Attention Learning method (called DMGAL) for few-shot action recognition, aiming to learn the spatio-temporal relationships from the video-specific to the task-specific level. To achieve this, we propose a carefully designed Motion-Guided Attention (MGA) method to identify and correlate motion-related region features from the video level to the task level. Specifically, the Self Motion-Guided Attention module (S-MGA) achieves spatio-temporal relation modeling at the video level by identifying and correlating motion-related region features between different frames within a video. The Cross Motion-Guided Attention module (C-MGA) identifies and correlates motion-related region features between frames of different videos within a specific task to achieve spatio-temporal relationships at the task level. This approach enables the model to construct class prototypes that fully incorporate spatio-temporal relationships from the video-specific level to the task-specific level. We validate the effectiveness of our DMGAL method by employing both fully fine-tuning and adapter-tuning paradigms. The models developed using these paradigms are termed DMGAL-FT and DMGAL-Adapter, respectively.

Figures

Figures reproduced from arXiv: 2411.11335 by the authors.

Figure 1
Figure 1. Comparison with previous methods. (a) Traditional methods design [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overview of DMGAL-FT and details of our proposed MGA. (a) Overview of the DMGAL-FT model designed for the fully fine-tuning paradigm. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The overview of DMGAL-Adapter (a) Overview of the DMGAL-Adapter model designed for the adapter-tuning paradigm. DMGAL-Adapter selectively [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Visualization of the cross-association ability of S-MGA on four examples using the UCF and SSv2-small datasets. For better visualization, we [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: The attention map visualization of S-MGA identifies and correlates motion-related regions at the video level. The left side displays the attention map [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Ablation study on the effect of the number of input frames under the 5-way 1-shot and 5-way 5-shot settings on the SSv2-Small dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Ablation study on the effect of the compression factors [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Attention map visualization of S-MGA and C-MGA. The brighter the colors, the higher the similarity. Results show that our MGA successfully [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: t-SNE feature embeddings of TRX without (left) and with (right) C [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 38 canonical work pages

  1. [1]

    Tsm: Temporal shift module for efficient video understanding,

    J. Lin, C. Gan, and S. Han, “Tsm: Temporal shift module for efficient video understanding,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2019, pp. 7083–7093

    2019

  2. [2]

    Learning match kernels on grassmann manifolds for action recognition,

    L. Zhang, X. Zhen, L. Shao, and J. Song, “Learning match kernels on grassmann manifolds for action recognition,” IEEE Transactions on Image Processing, vol. 28, no. 1, pp. 205–215, 2018

    work page 2018

  3. [3]

    Motion-driven visual tempo learning for video-based action recognition,

    Y . Liu, J. Yuan, and Z. Tu, “Motion-driven visual tempo learning for video-based action recognition,” IEEE Transactions on Image Process- ing, vol. 31, pp. 4104–4116, 2022

    work page 2022

  4. [4]

    Two-stream convolutional networks for action recognition in videos,

    K. Simonyan and A. Zisserman, “Two-stream convolutional networks for action recognition in videos,” Advances in Neural Information Processing Systems, vol. 27, 2014

    work page 2014

  5. [5]

    Real-time action recognition with deeply transferred motion vector cnns,

    B. Zhang, L. Wang, Z. Wang, Y . Qiao, and H. Wang, “Real-time action recognition with deeply transferred motion vector cnns,” IEEE Transactions on Image Processing, vol. 27, no. 5, pp. 2326–2339, 2018

    work page 2018

  6. [6]

    Model-agnostic meta-learning for fast adaptation of deep networks,

    C. Finn, P. Abbeel, and S. Levine, “Model-agnostic meta-learning for fast adaptation of deep networks,” in International Conference on Machine Learning, 2017, pp. 1126–1135

    2017

  7. [7]

    Learning to compare relation: Semantic align- ment for few-shot learning,

    C. Cao and Y . Zhang, “Learning to compare relation: Semantic align- ment for few-shot learning,” IEEE Transactions on Image Processing , vol. 31, pp. 1462–1474, 2022

    work page 2022

  8. [8]

    Hierarchical prototype refine- ment with progressive inter-categorical discrimination maximization for few-shot learning,

    Y . Zhou, Y . Guo, S. Hao, and R. Hong, “Hierarchical prototype refine- ment with progressive inter-categorical discrimination maximization for few-shot learning,” IEEE Transactions on Image Processing , vol. 31, pp. 3414–3429, 2022

    work page 2022

Show all 54 references
  1. [9]

    A two-stage approach to few-shot learning for image recognition,

    D. Das and C. G. Lee, “A two-stage approach to few-shot learning for image recognition,” IEEE Transactions on Image Processing , vol. 29, pp. 3336–3350, 2019

    2019

  2. [10]

    Scformer: Spectral coordinate transformer for cross-domain few-shot hyperspectral image classification,

    J. Li, Z. Zhang, R. Song, Y . Li, and Q. Du, “Scformer: Spectral coordinate transformer for cross-domain few-shot hyperspectral image classification,” IEEE Transactions on Image Processing , 2024. GUO ET AL. : VIDEO-TO-TASK LEARNING VIA MOTION-GUIDED ATTENTION FOR FEW-SHOT ACT...

    2024

  3. [11]

    Cross-modal contrastive learning network for few-shot action recognition,

    X. Wang, Y . Yan, H.-M. Hu, B. Li, and H. Wang, “Cross-modal contrastive learning network for few-shot action recognition,” IEEE Transactions on Image Processing , 2024

    2024

  4. [12]

    Few-shot video classification via temporal alignment,

    K. Cao, J. Ji, Z. Cao, C.-Y . Chang, and J. C. Niebles, “Few-shot video classification via temporal alignment,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 10 618–10 627

    2020

  5. [13]

    Spatio-temporal relation modeling for few-shot action recognition,

    A. Thatipelli, S. Narayan, S. Khan, R. M. Anwer, F. S. Khan, and B. Ghanem, “Spatio-temporal relation modeling for few-shot action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 19 958–19 967

    2022

  6. [14]

    Few-shot action recognition with permutation-invariant attention,

    H. Zhang, L. Zhang, X. Qi, H. Li, P. H. Torr, and P. Koniusz, “Few-shot action recognition with permutation-invariant attention,” in Proceedings of the European Conference on Computer Vision , 2020, pp. 525–542

    2020

  7. [15]

    Compound memory networks for few-shot video classification,

    L. Zhu and Y . Yang, “Compound memory networks for few-shot video classification,” in Proceedings of the European Conference on Computer Vision, 2018, pp. 751–766

    2018

  8. [16]

    Tarn: Temporal attentive relation network for few-shot and zero-shot action recognition,

    M. Bishay, G. Zoumpourlis, and I. Patras, “Tarn: Temporal attentive relation network for few-shot and zero-shot action recognition,” arXiv preprint arXiv:1907.09021, 2019

    1907 arXiv

  9. [17]

    Ta2n: Two-stage action alignment network for few-shot action recognition,

    S. Li, H. Liu, R. Qian, Y . Li, J. See, M. Fei, X. Yu, and W. Lin, “Ta2n: Two-stage action alignment network for few-shot action recognition,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 36, no. 2, 2022, pp. 1404–1411

    2022

  10. [18]

    Molo: Motion-augmented long-short contrastive learning for few-shot action recognition,

    X. Wang, S. Zhang, Z. Qing, C. Gao, Y . Zhang, D. Zhao, and N. Sang, “Molo: Motion-augmented long-short contrastive learning for few-shot action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 18 011–18 021

    2023

  11. [19]

    Task discrepancy maximization for fine-grained few-shot classification,

    S. Lee, W. Moon, and J.-P. Heo, “Task discrepancy maximization for fine-grained few-shot classification,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 5331–5340

    2022

  12. [20]

    Bi-directional feature reconstruction network for fine-grained few-shot image classification,

    J. Wu, D. Chang, A. Sain, X. Li, Z. Ma, J. Cao, J. Guo, and Y .-Z. Song, “Bi-directional feature reconstruction network for fine-grained few-shot image classification,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 37, no. 3, 2023, pp. 2821–2829

    2023

  13. [21]

    Motion-modulated temporal fragment alignment network for few-shot action recognition,

    J. Wu, T. Zhang, Z. Zhang, F. Wu, and Y . Zhang, “Motion-modulated temporal fragment alignment network for few-shot action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 9151–9160

    2022

  14. [22]

    Hybrid relation guided set matching for few-shot action recognition,

    X. Wang, S. Zhang, Z. Qing, M. Tang, Z. Zuo, C. Gao, R. Jin, and N. Sang, “Hybrid relation guided set matching for few-shot action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 19 948–19 957

    2022

  15. [23]

    Seeing inferences: brain dynamics and oculomotor signatures of non-verbal deduction,

    A. Mart ´ın-Salguero, C. Reverberi, A. Solari, L. Filippin, C. Pallier, and L. L. Bonatti, “Seeing inferences: brain dynamics and oculomotor signatures of non-verbal deduction,” Scientific Reports , vol. 13, no. 1, p. 2341, 2023

    2023

  16. [24]

    St-adapter: Parameter- efficient image-to-video transfer learning,

    J. Pan, Z. Lin, X. Zhu, J. Shao, and H. Li, “St-adapter: Parameter- efficient image-to-video transfer learning,” Advances in Neural Infor- mation Processing Systems , vol. 35, pp. 26 462–26 477, 2022

    2022

  17. [25]

    Temporal-relational crosstransformers for few-shot action recognition,

    T. Perrett, A. Masullo, T. Burghardt, M. Mirmehdi, and D. Damen, “Temporal-relational crosstransformers for few-shot action recognition,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 475–484

    2021

  18. [26]

    D 2st-adapter: Disentangled-and- deformable spatio-temporal adapter for few-shot action recognition,

    W. Pei, Q. Tan, G. Lu, and J. Tian, “D 2st-adapter: Disentangled-and- deformable spatio-temporal adapter for few-shot action recognition,” arXiv preprint arXiv:2312.01431 , 2023

    2023 arXiv

  19. [27]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778

    2016

  20. [28]

    Imagenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2009, pp. 248– 255

    2009

  21. [29]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning , 2021, pp. 8748–8763

    2021

  22. [30]

    Parameter-efficient fine-tuning for pre-trained vision models: A survey,

    Y . Xin, S. Luo, H. Zhou, J. Du, X. Liu, Y . Fan, Q. Li, and Y . Du, “Parameter-efficient fine-tuning for pre-trained vision models: A survey,” arXiv preprint arXiv:2402.02242 , 2024

    2024

  23. [31]

    Parameter-efficient transfer learning for nlp,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International Conference on Machine Learning , 2019, pp. 2790–2799

    2019

  24. [32]

    Vision transformer adapter for dense predictions,

    Z. Chen, Y . Duan, W. Wang, J. He, T. Lu, J. Dai, and Y . Qiao, “Vision transformer adapter for dense predictions,” arXiv preprint arXiv:2205.08534, 2022

    2022 arXiv

  25. [33]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  26. [34]

    Actionclip: A new paradigm for video action recognition,

    M. Wang, J. Xing, and Y . Liu, “Actionclip: A new paradigm for video action recognition,” arXiv preprint arXiv:2109.08472 , 2021

    2021 arXiv

  27. [35]

    Clip-adapter: Better vision-language models with feature adapters,

    P. Gao, S. Geng, R. Zhang, T. Ma, R. Fang, Y . Zhang, H. Li, and Y . Qiao, “Clip-adapter: Better vision-language models with feature adapters,” International Journal of Computer Vision , vol. 132, no. 2, pp. 581–595, 2024

    2024

  28. [36]

    Medical sam adapter: Adapting segment anything model for medical image segmentation,

    J. Wu, W. Ji, Y . Liu, H. Fu, M. Xu, Y . Xu, and Y . Jin, “Medical sam adapter: Adapting segment anything model for medical image segmentation,” arXiv preprint arXiv:2304.12620 , 2023

    2023 arXiv

  29. [37]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Loet al., “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4015–4026

    2023

  30. [38]

    Bi-directional motion attention with contrastive learning for few-shot action recognition,

    H. Guo, W. Yu, Y . Yan, and H. Wang, “Bi-directional motion attention with contrastive learning for few-shot action recognition,” in ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing, 2024, pp. 5490–5494

    2024

  31. [39]

    Mlp-mixer: An all-mlp architecture for vision,

    I. O. Tolstikhin, N. Houlsby, A. Kolesnikov, L. Beyer, X. Zhai, T. Un- terthiner, J. Yung, A. Steiner, D. Keysers, J. Uszkoreitet al., “Mlp-mixer: An all-mlp architecture for vision,” Advances in Neural Information Processing Systems, vol. 34, pp. 24 261–24 272, 2021

    2021

  32. [40]

    The” something something

    R. Goyal, S. Ebrahimi Kahou, V . Michalski, J. Materzynska, S. West- phal, H. Kim, V . Haenel, I. Fruend, P. Yianilos, M. Mueller-Freitaget al., “The” something something” video database for learning and evaluating visual common sense,” in Proceedings of the IEEE/CVF Internati...

    2017

  33. [41]

    Quo vadis, action recognition? a new model and the kinetics dataset,

    J. Carreira and A. Zisserman, “Quo vadis, action recognition? a new model and the kinetics dataset,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017, pp. 6299–6308

    2017

  34. [42]

    Hmdb: a large video database for human motion recognition,

    H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre, “Hmdb: a large video database for human motion recognition,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2011, pp. 2556–2563

    2011

  35. [43]

    Ucf101: A dataset of 101 human actions classes from videos in the wild,

    K. Soomro, “Ucf101: A dataset of 101 human actions classes from videos in the wild,” arXiv preprint arXiv:1212.0402 , 2012

    2012 arXiv

  36. [44]

    Temporal segment networks: Towards good practices for deep action recognition,

    L. Wang, Y . Xiong, Z. Wang, Y . Qiao, D. Lin, X. Tang, and L. Van Gool, “Temporal segment networks: Towards good practices for deep action recognition,” in Proceedings of the European Conference on Computer Vision, 2016, pp. 20–36

    2016

  37. [45]

    On the importance of spatial relations for few-shot action recognition,

    Y . Zhang, Y . Fu, X. Ma, L. Qi, J. Chen, Z. Wu, and Y .-G. Jiang, “On the importance of spatial relations for few-shot action recognition,” in Proceedings of the ACM International Conference on Multimedia, 2023, pp. 2243–2251

    2023

  38. [46]

    Matching networks for one shot learning,

    O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra et al. , “Matching networks for one shot learning,” Advances in Neural Information Pro- cessing Systems, vol. 29, 2016

    2016

  39. [47]

    Prototypical networks for few-shot learning,

    J. Snell, K. Swersky, and R. Zemel, “Prototypical networks for few-shot learning,” Advances in Neural Information Processing Systems , vol. 30, 2017

    2017

  40. [48]

    Clip-guided prototype modulating for few-shot action recognition,

    X. Wang, S. Zhang, J. Cen, C. Gao, Y . Zhang, D. Zhao, and N. Sang, “Clip-guided prototype modulating for few-shot action recognition,” International Journal of Computer Vision , vol. 132, no. 6, pp. 1899– 1912, 2024

    1912

  41. [49]

    Few-shot action recognition with hierarchical matching and contrastive learning,

    S. Zheng, S. Chen, and Q. Jin, “Few-shot action recognition with hierarchical matching and contrastive learning,” in Proceedings of the European Conference on Computer Vision , 2022, pp. 297–313

    2022

  42. [50]

    Multidimensional prototype refactor enhanced network for few-shot action recognition,

    S. Liu, M. Jiang, and J. Kong, “Multidimensional prototype refactor enhanced network for few-shot action recognition,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 10, pp. 6955– 6966, 2022

    2022

  43. [51]

    Matching compound prototypes for few-shot action recognition,

    Y . Huang, L. Yang, G. Chen, H. Zhang, F. Lu, and Y . Sato, “Matching compound prototypes for few-shot action recognition,” International Journal of Computer Vision , vol. 132, p. 3977–4002, Apr. 2024

    2024

  44. [52]

    Aim: Adapting image models for efficient video understanding,

    T. Yang, Y . Zhu, Y . Xie, A. Zhang, C. Chen, and M. Li, “Aim: Adapting image models for efficient video understanding,” International Conference on Learning Representations , vol. 3, no. 5, p. 7, 2023

    2023

  45. [53]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” in International Conference on Machine Learning , 2023, pp. 19 730–19 742

    2023

  46. [54]

    Visualizing data using t-sne

    L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.” Journal of Machine Learning Research , vol. 9, no. 11, 2008

    2008

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.