Pith. sign in

REVIEW 4 major objections 6 minor 62 references

VELoRA: A Low-Rank Adaptation Approach for Efficient RGB-Event based Recognition

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read VELoRA adapts a frozen CLIP ViT for RGB-event recognition by tuning only low-rank matrices, reporting 57.99% on PokerEvent and 50.89% on HARDVS while cutting trainable parameters to 7.02 MB.

desk verdict Plausible PEFT recipe for RGB-event fusion, but the accuracy margins are too small and the event-input encoding too under-specified to support the SOTA claims as written. read the letter →

arxiv 2412.20064 v1 pith:BVGVUVVT submitted 2024-12-28 cs.CV cs.AIcs.NE

Lan Chen , Haoxiang Yang , Pengpeng Shao , Haoyu Song , Xiao Wang , Zhicheng Zhao , Yaowei Wang , Yonghong Tian This is my paper · ORCID
classification cs.CVcs.AIcs.NE
keywords parameter-efficientfine-tuningLoRAeventcameraRGB-Eventfusionmulti-modalvisiontransformeractionrecognition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that parameter-efficient fine-tuning can be applied to RGB-event recognition by keeping a pre-trained vision transformer frozen and updating only small low-rank matrices. On the PokerEvent benchmark it reports 57.99% top-1 accuracy, edging past the previous best SAFE at 57.63%, and on HARDVS it reports 50.89%, above ESTF, SAFE, and C3D. It also reports that trainable parameters drop from 1719 MB to 7.02 MB and runtime memory falls by about 2200 MB. If these results hold, VELoRA would be the first PEFT method for RGB-event recognition and a template for adapting frozen foundation models to other multi-modal tasks.

What carries the argument

The load-bearing object is Low-Rank Adaptation (LoRA), which adds a trainable low-rank update $\Delta W = BA$ to a frozen weight matrix $W_0$, so the forward pass becomes $h = W_0 x + BA x$. VELoRA distributes LoRA in a two-stage design: modality-specific LoRA blocks in the low-level Transformer layers of the RGB, event, and frame-difference branches, followed by a modality-shared LoRA block in the final high-level Transformer layer that fuses the concatenated features. The third branch takes the mean of differences between consecutive frames (eight frames in the reported setting) and encodes it with the same frozen backbone, giving an explicit motion channel. An MSE reconstruction loss between RGB and event features, $L_{RTE}$ and $L_{ETR}$, is added to the cross-entropy classification loss, and the total loss is their sum.

What would settle it

Keep VELoRA's LoRA tuning identical but replace the event-image input with a voxel-grid or raw-event representation; if accuracy on HARDVS or PokerEvent falls below the reported 50.89% or 57.99% (or below SAFE's 50.17% and 57.63%), the gains depend on the specific static event rendering rather than on the LoRA fusion design.

Watch

Extended reading notes

Core claim

VELoRA's central claim is that a frozen CLIP ViT-B/16 can be adapted to RGB-event classification by inserting LoRA blocks at two levels. Low-level Transformer blocks receive modality-specific LoRA tuning in three separate branches: RGB frames, event images, and the mean of consecutive-frame differences, which supplies motion cues. A reconstruction step with MSE loss lets the RGB and event branches predict each other's features, encouraging cross-modal alignment. The last Transformer block then fuses the concatenated features with modality-shared LoRA tuning before a classification head. The paper reports that this scheme beats full fine-tuning and specialized LoRA variants on both benchmarks while updating only the low-rank matrices.

Load-bearing premise

The load-bearing assumption is that static, frame-aligned event images encoded by a frozen CLIP ViT preserve enough information for recognition, with the frame-difference branch as the only explicit motion cue—a choice the paper itself concedes may limit temporal effectiveness.

Editorial extensions

If this is right

  • If the reported results hold, RGB-event recognition no longer requires full fine-tuning of a large backbone; a frozen CLIP ViT with LoRA blocks reaches state-of-the-art accuracy.
  • The component ablations support that each piece contributes: adding the frame-difference branch, reconstruction loss, modality-specific LoRA, and modality-shared LoRA raises accuracy from 55.78% to 57.99% on PokerEvent.
  • The method transfers to at least two different recognition targets, fine-grained poker-character classification and human action recognition, suggesting the PEFT recipe is task-agnostic within RGB-event inputs.
  • Because trainable parameters fall to 7.02 MB, deployment becomes cheaper, since each new task only needs to store small LoRA matrices per modality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The static event-image representation, which the paper itself flags as a limitation, means the temporal information in event streams is only captured indirectly through the frame-difference branch; pairing VELoRA with a learnable event representation could close the gap to full temporal models.
  • The same two-stage LoRA recipe (modality-specific encoders plus a shared fusion LoRA) could apply to other paired-modality tasks such as RGB-depth or RGB-thermal recognition, where frozen foundation models are available for one modality.
  • The reported 2200 MB memory saving and parameter reduction come with higher FLOPs in the paper's table (41.35 GB vs 30.98 GB), so the efficiency claim is best read as memory- and parameter-centric rather than compute-centric.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces VELoRA, a parameter-efficient fine-tuning (PEFT) method for RGB-event recognition. The method freezes a CLIP ViT-B/16 backbone and inserts LoRA adapters in two regimes: modality-specific LoRA in low-level Transformer blocks for RGB, event, and frame-difference branches, and modality-shared LoRA in the final high-level Transformer block for fusion. A reconstruction loss between RGB and event features is added. The authors report state-of-the-art top-1 accuracy on PokerEvent (57.99% vs SAFE's 57.63%) and HARDVS (50.89% vs SAFE's 50.17% and C3D's 50.88%), while reducing trainable parameters from 1719MB to 7.02MB in their comparison with full fine-tuning. The paper claims this is the first PEFT approach for RGB-event recognition based on pre-trained foundation models.

Significance. If the claims are robust, the paper would make a useful contribution: it demonstrates that a low-rank adaptation strategy can be organized into modality-specific and modality-shared parts for multi-modal RGB-event data, and the reduction in trainable parameters is substantial and practically relevant. The idea of using frame differences as an auxiliary modality within a frozen-backbone PEFT framework is reasonable. However, the headline accuracy gains over prior methods are very small (0.36 points on PokerEvent; 0.01-0.72 points on HARDVS depending on the baseline), and the current manuscript does not provide enough statistical or procedural support for these margins. The efficiency claims are also partially inconsistent with the reported table. The central ideas are plausible, but the evidence as presented is not yet sufficient to support state-of-the-art and efficiency claims at the level of rigor expected for a journal publication.

major comments (4)
  1. [Section III-C, Eqs. (2)-(3)] The input encoding is underspecified. The text states that "we convert the event stream E into event images aligned with the timestamps of the video frames," but no algorithm, binning scheme, normalization, polarity handling, or resolution mapping is given. In addition, the frame-difference branch is defined only as "the mean of the differences between consecutive frames," and the symbol T_d used in Eq. (3) is never defined. Since all three branches (RGB, event, frame difference) feed this static representation into the frozen CLIP ViT, the entire accuracy comparison depends on a representation the reader cannot reproduce. The authors themselves concede in Section IV-G that "the static representation of event streams may limit the effectiveness of temporal information." Please provide the exact conversion procedure (with formulas or code), and ideally compare at least one alternative event representation (e.g., voxel grids or time surfaces) to show that the reported results are not an artifact of this particular encoding.
  2. [Section IV-D, Tables III-VI] The hyperparameter selection procedure undermines the statistical strength of the state-of-the-art claim. Tables IV, V, and VI vary the number of input frames, the LoRA rank, and the LoRA insertion location, and the configuration with the best test accuracy (rank 4, 8 frames, MLP-only LoRA) is then used for the headline results in Tables I and II. The paper does not describe a held-out validation split; the PokerEvent description gives only training and test sizes, and no validation set is mentioned. With margins as small as 0.36 points over SAFE on PokerEvent and 0.01 points over C3D on HARDVS, run-to-run variation could easily change the ranking. Please report means and standard deviations over multiple seeds, or use a proper validation protocol for model selection, and state the number of runs used for the final numbers.
  3. [Section IV-E, Table VIII] The efficiency claims are internally inconsistent with the reported table. Table VIII shows that VELoRA has higher FLOPs than full fine-tuning (41.35 GB vs 30.98 GB), while the text states that "our proposed method offers a quicker runtime and lower FLOPS." The training time is only slightly lower (4240 s vs 4536 s), and the memory reduction is 2054 MB, not the 2200 MB stated in the abstract (19574 - 17520 = 2054). The trainable-parameter reduction from 1719 MB to 7.02 MB is impressive and well supported, but the broader efficiency narrative should be corrected to match the data, with units and measurement conditions clarified.
  4. [Section III-C, Eqs. (4)-(6)] The formal description of the proposed architecture is too schematic to be reproducible. Eq. (4) writes a Transformer block update as F(l+1) = F(l) + (W0 + ΔW)F(l), which omits the multi-head attention, normalization layers, and residual structure that define a ViT block; it also does not clarify which linear projections receive the LoRA decomposition beyond the later statement that only MLP layers are updated. Eqs. (5)-(6) similarly leave unclear whether the "modality-shared LoRA" is applied to the attention output, the MLP, or both, and how the concatenated features are split after fusion. Please provide a precise layer-by-layer forward pass or pseudocode, or release the exact code and refer to it in the paper; currently the architecture cannot be reimplemented from the text alone.
minor comments (6)
  1. [Section III-C] There is a typo in "Ff useusing LoRA" near Eq. (5); it should read "F_fuse using LoRA."
  2. [Table II] The method name "R2PlusID" should be "R2Plus1D" to match the cited reference and standard nomenclature.
  3. [Section IV-B and Table VIII] Please clarify the units of "Training Time (s)": if this is the total training time for 50 epochs, that is about 85-90 s per epoch; if it is per-epoch time, the comparison should be labeled consistently.
  4. [Section IV-D, Table III] In Table III, rows 1 and 2 list only one component (F.D. or Reconst.) while the results are 55.78 and 55.94; the text says "when the model was trained with only the frame difference module or the reconstruction step, the highest accuracy achieved was 55.94," which is consistent, but the table would benefit from also showing a no-component baseline to quantify the contribution of each module.
  5. [Section IV-A] Please specify whether the PokerEvent and HARDVS train/test splits follow the official protocols of the dataset papers (e.g., [12], [41]) and whether the same splits were used for all compared methods; this information is essential for a fair comparison but is not currently stated.
  6. [Equations (8)-(9)] The reconstruction loss equations use M for both the number of patch tokens and the number of samples; please disambiguate the notation and clarify over which dimension the summation is taken.

Circularity Check

1 steps flagged · score 6.0 of 10

The PokerEvent SOTA number is the maximum of hyperparameter ablations run on the same test set, so the headline accuracy is a fitted selection presented as the method's accuracy.

  1. fitted input called prediction [Section IV-D (Tables IV, V, VI) selecting the 57.99% reported in Section IV-C and Table I; dataset split defined in Section IV-A.]
    "The model trained with 8 input frames yielded the highest accuracy rate of 57.99% ... the model trained with a rank of 4 achieved the highest accuracy rate of 57.99% ... applying LoRA only to the MLP layers ... yielded the best accuracy of 57.99%."

    The 57.99% headline on PokerEvent is not the accuracy of one prespecified architecture: it is the best value across three separate ablated choices (frames in {4,5,6,8,10}, LoRA rank in {4,6,8,12}, LoRA position in {QKV, MLP, QKV+MLP}). Section IV-A defines only a single train/test split for PokerEvent, so these ablations are evaluated on the same test set used for the later SOTA comparison in Table I. The paper's conclusion 'our model attains an accuracy of 57.99%, marginally surpassing the state-of-the-art SAFE' is therefore the selected maximum of the searched configurations, not an out-of-sample accuracy of a fixed method. The 0.36% margin over SAFE is statistically forced by taking the argmax of many test-set evaluations, i.e., a fitted maximum renamed as the method's prediction.

full rationale

This is an empirical engineering paper, not a formal derivation, so there is no equation-level circularity in the LoRA updates or the loss. I checked the load-bearing claims: the novelty claim 'no work has conducted PEFT for RGB-Event recognition' is a literature claim, not a derivation; the frozen CLIP ViT and LoRA machinery are external and standard; the reconstruction loss and frame-difference branches are auxiliary losses/inputs, not definitions of the accuracy; and the SAFE/ESTF/C3D baselines are published numbers, so the overlapping authorship of some baselines does not by itself make the comparison circular. The genuine circular element is the production of the headline number: Section IV-D ran exhaustive ablations on the PokerEvent test set and reported the same 57.99% as the best in each grid, and Section IV-C then presents that value as the method's state-of-the-art accuracy. That is a test-set-tuned maximum presented as an independent prediction, which is the same statistical force as fitting a parameter to the test labels and then predicting the fitted value. The architecture and PEFT idea still have independent content, so the circularity is partial rather than total.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities. Its central claim rests on the LoRA low-rank premise, on the transferability of frozen CLIP features to event data, on the chosen event-image and frame-difference representations, and on the comparability of published baselines. Hyperparameters such as rank, frame count, and LoRA placement are fitted to the test set through ablations.

free parameters (5)
  • LoRA rank r = 4
    Chosen by ablation on PokerEvent test set (Table V); rank 4 gave 57.99% while ranks 6, 8, 12 gave 56.35%, 55.95%, 57.08%.
  • Number of input frames C = 8
    Selected by ablation (Table IV); 8 frames gave 57.99% vs 55.02% for 4 and 57.54% for 10.
  • LoRA insertion location = MLP layers only
    Selected by ablation (Table VI); MLP-only gave 57.99% vs QKV-only 56.69% and QKV+MLP 57.26%.
  • Learning rate / batch size / epochs = 1e-4 / 4 / 50
    Fixed hyperparameters; not shown to be optimal, standard choices.
  • Reconstruction loss weight = 1 (implicit)
    The total loss in Eq. (10) sums reconstruction losses and cross-entropy without weighting; no sensitivity analysis is provided.
assumptions (5)
  • domain assumption Weight updates during fine-tuning lie in a low-rank subspace (LoRA assumption).
    Invoked in Section III-A; the paper relies on the standard LoRA premise that low-rank adapters approximate full fine-tuning.
  • domain assumption A frozen ViT-B/16 CLIP model encodes event images sufficiently well for recognition.
    Section III-C uses the same LVM for RGB, event, and frame-difference branches with parameters frozen; no validation that CLIP features are informative for event data.
  • domain assumption Mean of consecutive-frame differences captures motion cues for both RGB and event modalities.
    Section III-C defines D as mean of differences between consecutive frames; ablations (Table III) show it helps, but the representation choice is asserted.
  • domain assumption MSE reconstruction loss aligns RGB and event feature spaces for cross-modal transfer.
    Eqs. (8)-(9) use MSE between raw and reconstructed features; no analysis of whether MSE is appropriate for CLIP features.
  • domain assumption Baseline numbers cited from prior papers are comparable (same splits and protocols).
    Tables I and II take accuracies from prior publications; no reimplementation or protocol standardization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VELoRA: A Low-Rank Adaptation Approach for Efficient RGB-Event based Recognition." pith.science (2026). https://pith.science/paper/BVGVUVVT

@misc{pith2026241220064,
  author       = {Pith},
  title        = {Pith review of: VELoRA: A Low-Rank Adaptation Approach for Efficient RGB-Event based Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BVGVUVVT}},
  note         = {Machine review of arXiv:2412.20064}
}
read the original abstract

Pattern recognition leveraging both RGB and Event cameras can significantly enhance performance by deploying deep neural networks that utilize a fine-tuning strategy. Inspired by the successful application of large models, the introduction of such large models can also be considered to further enhance the performance of multi-modal tasks. However, fully fine-tuning these models leads to inefficiency and lightweight fine-tuning methods such as LoRA and Adapter have been proposed to achieve a better balance between efficiency and performance. To our knowledge, there is currently no work that has conducted parameter-efficient fine-tuning (PEFT) for RGB-Event recognition based on pre-trained foundation models. To address this issue, this paper proposes a novel PEFT strategy to adapt the pre-trained foundation vision models for the RGB-Event-based classification. Specifically, given the RGB frames and event streams, we extract the RGB and event features based on the vision foundation model ViT with a modality-specific LoRA tuning strategy. The frame difference of the dual modalities is also considered to capture the motion cues via the frame difference backbone network. These features are concatenated and fed into high-level Transformer layers for efficient multi-modal feature learning via modality-shared LoRA tuning. Finally, we concatenate these features and feed them into a classification head to achieve efficient fine-tuning. The source code and pre-trained models will be released on \url{https://github.com/Event-AHU/VELoRA}.

Figures

Figures reproduced from arXiv: 2412.20064 by the authors.

Figure 1
Figure 1. Comparison between existing event-based classification models and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Comparison between existing LoRA strategy for (a) single modality tuning, (b) multi-task tuning, and (c) our newly proposed VELoRA for RGB-Event [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An overview of our proposed Low-Rank Adaptation Approach for Efficient Visible-Event Pattern Recognition, termed VELoRA. We introduce a novel fine-tuning approach that integrates modality-specific and shared components, enabling the model to preserve sensitivity to distinct modalities while also extracting shared information across them, which boosts performance on multimodal tasks. We designate the last block as th… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Visualization of feature distribution of (a) Ours, (b) full fine-tuning on PokerEvent. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualization of the RGB frame differences (left) and Event frame differences (right) on the HARDVS dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Visualization of the raw RGB image and activation maps. Note that, the blue denotes a higher activation response. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Visualization of the top-5 predicted results on the HARDVS dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 24 canonical work pages

  1. [1]

    Object detection using deep learning, cnns and vision transformers: A review,

    A. B. Amjoud and M. Amrouch, “Object detection using deep learning, cnns and vision transformers: A review,” IEEE Access , vol. 11, pp. 35 479–35 516, 2023

    work page 2023

  2. [2]

    Event voxel set transformer for spatiotemporal representation learning on event streams,

    B. Xie, Y . Deng, Z. Shao, Q. Xu, and Y . Li, “Event voxel set transformer for spatiotemporal representation learning on event streams,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

    2024

  3. [3]

    Dino-x: A unified vision model for open-world object detection and understanding,

    T. Ren, Y . Chen, Q. Jiang, Z. Zeng, Y . Xiong, W. Liu, Z. Ma, J. Shen, Y . Gao, X. Jianget al., “Dino-x: A unified vision model for open-world object detection and understanding,” arXiv preprint arXiv:2411.14347 , 2024

    arXiv 2024

  4. [4]

    Exploit all the layers: Fast and accurate cnn object detector with scale dependent pooling and cascaded rejection classifiers,

    F. Yang, W. Choi, and Y . Lin, “Exploit all the layers: Fast and accurate cnn object detector with scale dependent pooling and cascaded rejection classifiers,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 2129–2137

    work page 2016

  5. [5]

    e-tld: Event-based framework for dynamic object tracking,

    B. Ramesh, S. Zhang, H. Yang, A. Ussa, M. Ong, G. Orchard, and C. Xiang, “e-tld: Event-based framework for dynamic object tracking,” IEEE Transactions on Circuits and Systems for Video Technology , vol. 31, no. 10, pp. 3996–4006, 2020

    work page 2020

  6. [6]

    Faster r-cnn: Towards real-time object detection with region proposal networks,

    S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” IEEE transactions on pattern analysis and machine intelligence, vol. 39, no. 6, pp. 1137–1149, 2016

    2016

  7. [7]

    Object detection with discriminatively trained part-based models,

    P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan, “Object detection with discriminatively trained part-based models,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 32, no. 9, pp. 1627–1645, 2010

    2010

  8. [8]

    Convolution-free medical image segmentation using transformers,

    D. Karimi, S. D. Vasylechko, and A. Gholipour, “Convolution-free medical image segmentation using transformers,” in Medical image computing and computer assisted intervention–MICCAI 2021: 24th international conference, Strasbourg, France, September 27–October 1, 2021, proceedings, part I 24 . Springer, 2021, pp. 78–88

    work page 2021

Show all 62 references
  1. [9]

    Continuous-time object segmentation using high temporal resolution event camera,

    L. Zhu, X. Chen, L. Wang, X. Wang, Y . Tian, and H. Huang, “Continuous-time object segmentation using high temporal resolution event camera,” IEEE Transactions on Pattern Analysis & Machine Intelligence, no. 01, pp. 1–18, 2024

    2024

  2. [10]

    Swin-unet: Unet-like pure transformer for medical image segmenta- tion,

    H. Cao, Y . Wang, J. Chen, D. Jiang, X. Zhang, Q. Tian, and M. Wang, “Swin-unet: Unet-like pure transformer for medical image segmenta- tion,” in European conference on computer vision . Springer, 2022, pp. 205–218

    2022

  3. [11]

    Semantic-aware frame-event fusion based pattern recognition via large vision-language models,

    D. Li, J. Jin, Y . Zhang, Y . Zhong, Y . Wu, L. Chen, X. Wang, and B. Luo, “Semantic-aware frame-event fusion based pattern recognition via large vision-language models,” arXiv preprint arXiv:2311.18592 , 2023

    2023 arXiv

  4. [12]

    Hardvs: Revisiting human activity recognition with dynamic vision sensors,

    X. Wang, Z. Wu, B. Jiang, Z. Bao, L. Zhu, G. Li, Y . Wang, and Y . Tian, “Hardvs: Revisiting human activity recognition with dynamic vision sensors,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 38, no. 6, 2024, pp. 5615–5623. IEEE TRANSACTIONS ON ***...

    2024

  5. [13]

    Ecsnet: Spatio- temporal feature learning for event camera,

    Z. Chen, J. Wu, J. Hou, L. Li, W. Dong, and G. Shi, “Ecsnet: Spatio- temporal feature learning for event camera,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 2, pp. 701–712, 2022

    2022

  6. [14]

    Mvf-net: A multi-view fusion network for event-based object classification,

    Y . Deng, H. Chen, and Y . Li, “Mvf-net: A multi-view fusion network for event-based object classification,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 12, pp. 8275–8284, 2022

    2022

  7. [15]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685 , 2021

    2021 arXiv

  8. [16]

    Side-tuning: a baseline for network adaptation via additive side networks,

    J. O. Zhang, A. Sax, A. Zamir, L. Guibas, and J. Malik, “Side-tuning: a baseline for network adaptation via additive side networks,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part III 16 . Springer, 2020, pp. 698–714

    2020

  9. [17]

    Vision transformer adapter for dense predictions,

    Z. Chen, Y . Duan, W. Wang, J. He, T. Lu, J. Dai, and Y . Qiao, “Vision transformer adapter for dense predictions,” arXiv preprint arXiv:2205.08534, 2022

    2022 arXiv

  10. [18]

    Adaptformer: Adapting vision transformers for scalable visual recogni- tion,

    S. Chen, C. Ge, Z. Tong, J. Wang, Y . Song, J. Wang, and P. Luo, “Adaptformer: Adapting vision transformers for scalable visual recogni- tion,” Advances in Neural Information Processing Systems , vol. 35, pp. 16 664–16 678, 2022

    2022

  11. [19]

    St-adapter: Parameter- efficient image-to-video transfer learning,

    J. Pan, Z. Lin, X. Zhu, J. Shao, and H. Li, “St-adapter: Parameter- efficient image-to-video transfer learning,” Advances in Neural Infor- mation Processing Systems , vol. 35, pp. 26 462–26 477, 2022

    2022

  12. [20]

    Mixture-of-loras: An efficient multitask tuning for large language models,

    W. Feng, C. Hao, Y . Zhang, Y . Han, and H. Wang, “Mixture-of-loras: An efficient multitask tuning for large language models,” arXiv preprint arXiv:2403.03432, 2024

    2024 arXiv

  13. [21]

    Mtlora: Low-rank adaptation approach for efficient multi-task learning,

    A. Agiza, M. Neseem, and S. Reda, “Mtlora: Low-rank adaptation approach for efficient multi-task learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 16 196–16 205

    2024

  14. [22]

    Event- based vision: A survey,

    G. Gallego, T. Delbrück, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. J. Davison, J. Conradt, K. Daniilidis et al., “Event- based vision: A survey,” IEEE transactions on pattern analysis and machine intelligence, vol. 44, no. 1, pp. 154–180, 2020

    2020

  15. [23]

    Ev- gait: Event-based robust gait recognition using dynamic vision sensors,

    Y . Wang, B. Du, Y . Shen, K. Wu, G. Zhao, J. Sun, and H. W. 0001, “Ev- gait: Event-based robust gait recognition using dynamic vision sensors,” in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019 . Computer Vision Fo...

    2019

  16. [24]

    Vmv-gcn: V olumetric multi-view based graph cnn for event stream classification,

    B. Xie, Y . Deng, Z. Shao, H. Liu, and Y . Li, “Vmv-gcn: V olumetric multi-view based graph cnn for event stream classification,” IEEE Robotics and Automation Letters , vol. 7, pp. 1976–1983, 2022

    1976

  17. [25]

    Aegnn: Asynchronous event-based graph neural networks,

    S. Schaefer, D. Gehrig, and D. Scaramuzza, “Aegnn: Asynchronous event-based graph neural networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 12 371–12 381

    2022

  18. [26]

    Uncertainty- aware bridge based mobile-former network for event-based pattern recognition,

    H. Yang, C. Yuan, Y . Zhu, L. Chen, X. Wang, and J. Tang, “Uncertainty- aware bridge based mobile-former network for event-based pattern recognition,” arXiv preprint arXiv:2401.11123 , 2024

    2024 arXiv

  19. [27]

    Learning bottleneck transformer for event image-voxel feature fusion based classification,

    C. Yuan, Y . Jin, Z. Wu, F. Wei, Y . Wang, L. Chen, and X. Wang, “Learning bottleneck transformer for event image-voxel feature fusion based classification,” in Chinese Conference on Pattern Recognition and Computer Vision (PRCV) . Springer, 2023, pp. 3–15

    2023

  20. [28]

    Spikformer: When spiking neural network meets transformer,

    Z. Zhou, Y . Zhu, C. He, Y . Wang, S. Yan, Y . Tian, and L. Yuan, “Spikformer: When spiking neural network meets transformer,” arXiv preprint arXiv:2209.15425, 2022

    2022 arXiv

  21. [29]

    Incorporating learnable membrane time constant to enhance learning of spiking neural networks,

    W. Fang, Z. Yu, Y . Chen, T. Masquelier, T. Huang, and Y . Tian, “Incorporating learnable membrane time constant to enhance learning of spiking neural networks,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 2661–2671

    2021

  22. [30]

    Dynamic graph cnn for event-camera based gesture recognition,

    J. Chen, J. Meng, X. Wang, and J. Yuan, “Dynamic graph cnn for event-camera based gesture recognition,” in 2020 IEEE International Symposium on Circuits and Systems (ISCAS) . IEEE, 2020, pp. 1–5

    2020

  23. [31]

    Ev-flownet: Self- supervised optical flow estimation for event-based cameras,

    A. Z. Zhu, L. Yuan, K. Chaney, and K. Daniilidis, “Ev-flownet: Self- supervised optical flow estimation for event-based cameras,” arXiv preprint arXiv:1802.06898, 2018

    2018 arXiv

  24. [32]

    Ev- gait: Event-based robust gait recognition using dynamic vision sensors,

    Y . Wang, B. Du, Y . Shen, K. Wu, G. Zhao, J. Sun, and H. Wen, “Ev- gait: Event-based robust gait recognition using dynamic vision sensors,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 6358–6367

    2019

  25. [33]

    An event-driven categorization model for aer image sensors using multispike encoding and learning,

    R. Xiao, H. Tang, Y . Ma, R. Yan, and G. Orchard, “An event-driven categorization model for aer image sensors using multispike encoding and learning,” IEEE transactions on neural networks and learning systems, vol. 31, no. 9, pp. 3649–3657, 2019

    2019

  26. [34]

    Event-based action recognition using motion information and spiking neural networks

    Q. Liu, D. Xing, H. Tang, D. Ma, and G. Pan, “Event-based action recognition using motion information and spiking neural networks.” in IJCAI, 2021, pp. 1743–1749

    2021

  27. [35]

    Training deep spiking neural networks using backpropagation,

    J. H. Lee, T. Delbruck, and M. Pfeiffer, “Training deep spiking neural networks using backpropagation,” Frontiers in neuroscience, vol. 10, p. 508, 2016

    2016

  28. [36]

    Spikmamba: When snn meets mamba in event-based human action recognition,

    J. Chen, Y . Yang, S. Deng, D. Teng, and L. Pan, “Spikmamba: When snn meets mamba in event-based human action recognition,” arXiv preprint arXiv:2410.16746, 2024

    2024 arXiv

  29. [37]

    Event stream based human action recognition: a high-definition benchmark dataset and algorithms,

    X. Wang, S. Wang, P. Shao, B. Jiang, L. Zhu, and Y . Tian, “Event stream based human action recognition: a high-definition benchmark dataset and algorithms,” arXiv preprint arXiv:2408.09764 , 2024

    2024 arXiv

  30. [38]

    Unleashing the power of cnn and transformer for balanced rgb-event video recognition,

    X. Wang, Y . Rong, S. Wang, Y . Chen, Z. Wu, B. Jiang, Y . Tian, and J. Tang, “Unleashing the power of cnn and transformer for balanced rgb-event video recognition,” arXiv preprint arXiv:2312.11128 , 2023

    2023 arXiv

  31. [39]

    Mfgnet: Dynamic modality-aware filter generation for rgb-t tracking,

    X. Wang, X. Shu, S. Zhang, B. Jiang, Y . Wang, Y . Tian, and F. Wu, “Mfgnet: Dynamic modality-aware filter generation for rgb-t tracking,” IEEE Transactions on Multimedia , vol. 25, pp. 4335–4348, 2022

    2022

  32. [40]

    Vefnet: an event-rgb cross modality fusion network for visual place recognition,

    Z. Huang, R. Huang, L. Sun, C. Zhao, M. Huang, and S. Su, “Vefnet: an event-rgb cross modality fusion network for visual place recognition,” in 2022 IEEE International Conference on Image Processing (ICIP) , 2022, pp. 2671–2675

    2022

  33. [41]

    Sstformer: bridging spiking neural network and memory support transformer for frame-event based recognition,

    X. Wang, Z. Wu, Y . Rong, L. Zhu, B. Jiang, J. Tang, and Y . Tian, “Sstformer: bridging spiking neural network and memory support transformer for frame-event based recognition,” arXiv preprint arXiv:2308.04369, 2023

    2023 arXiv

  34. [42]

    Dylora: Parameter efficient tuning of pre-trained models using dynamic search- free low-rank adaptation,

    M. Valipour, M. Rezagholizadeh, I. Kobyzev, and A. Ghodsi, “Dylora: Parameter efficient tuning of pre-trained models using dynamic search- free low-rank adaptation,” arXiv preprint arXiv:2210.07558 , 2022

    2022 arXiv

  35. [43]

    Lora-fa: Memory- efficient low-rank adaptation for large language models fine-tuning,

    L. Zhang, L. Zhang, S. Shi, X. Chu, and B. Li, “Lora-fa: Memory- efficient low-rank adaptation for large language models fine-tuning,” arXiv preprint arXiv:2308.03303 , 2023

    2023 arXiv

  36. [44]

    Adalora: Adaptive budget allocation for parameter-efficient fine-tuning,

    Q. Zhang, M. Chen, A. Bukharin, N. Karampatziakis, P. He, Y . Cheng, W. Chen, and T. Zhao, “Adalora: Adaptive budget allocation for parameter-efficient fine-tuning,” arXiv preprint arXiv:2303.10512, 2023

    2023 arXiv

  37. [45]

    Alora: Allocating low- rank adaptation for fine-tuning large language models,

    Z. Liu, J. Lyn, W. Zhu, X. Tian, and Y . Graham, “Alora: Allocating low- rank adaptation for fine-tuning large language models,” arXiv preprint arXiv:2403.16187, 2024

    2024 arXiv

  38. [46]

    Denseclip: Language-guided dense prediction with context-aware prompting,

    Y . Rao, W. Zhao, G. Chen, Y . Tang, Z. Zhu, G. Huang, J. Zhou, and J. Lu, “Denseclip: Language-guided dense prediction with context-aware prompting,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 18 082–18 091

    2022

  39. [47]

    Top-reid: Multi- spectral object re-identification with token permutation,

    Y . Wang, X. Liu, P. Zhang, H. Lu, Z. Tu, and H. Lu, “Top-reid: Multi- spectral object re-identification with token permutation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 6, 2024, pp. 5758–5766

    2024

  40. [48]

    Decoupled weight decay regularization,

    I. Loshchilov, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  41. [49]

    Learning spatiotemporal features with 3d convolutional networks,

    D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3d convolutional networks,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 4489–4497

    2015

  42. [50]

    Tsm: Temporal shift module for efficient video understanding,

    J. Lin, C. Gan, and S. Han, “Tsm: Temporal shift module for efficient video understanding,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 7083–7093

    2019

  43. [51]

    Tam: Temporal adaptive module for video recognition,

    Z. Liu, L. Wang, W. Wu, C. Qian, and T. Lu, “Tam: Temporal adaptive module for video recognition,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 13 708–13 718

    2021

  44. [52]

    Action-net: Multipath excitation for action recognition,

    Z. Wang, Q. She, and A. Smolic, “Action-net: Multipath excitation for action recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 13 214–13 223

    2021

  45. [53]

    Video swin transformer,

    Z. Liu, J. Ning, Y . Cao, Y . Wei, Z. Zhang, S. Lin, and H. Hu, “Video swin transformer,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 3202–3211

    2022

  46. [54]

    Is space-time attention all you need for video understanding?

    G. Bertasius, H. Wang, and L. Torresani, “Is space-time attention all you need for video understanding?” in ICML, vol. 2, no. 3, 2021, p. 4

    2021

  47. [55]

    X3d: Expanding architectures for efficient video recognition,

    C. Feichtenhofer, “X3d: Expanding architectures for efficient video recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2020, pp. 203–213

    2020

  48. [56]

    Mvitv2: Improved multiscale vision transformers for classification and detection,

    Y . Li, C.-Y . Wu, H. Fan, K. Mangalam, B. Xiong, J. Malik, and C. Feichtenhofer, “Mvitv2: Improved multiscale vision transformers for classification and detection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 4804– 4814

    2022

  49. [57]

    Slowfast networks for video recognition,

    C. Feichtenhofer, H. Fan, J. Malik, and K. He, “Slowfast networks for video recognition,” in Proceedings of the IEEE/CVF international conference on computer vision , 2019, pp. 6202–6211. IEEE TRANSACTIONS ON ***, 2024 12

    2019

  50. [58]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778

    2016

  51. [59]

    A closer look at spatiotemporal convolutions for action recognition,

    D. Tran, H. Wang, L. Torresani, J. Ray, Y . LeCun, and M. Paluri, “A closer look at spatiotemporal convolutions for action recognition,” in Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, 2018, pp. 6450–6459

    2018

  52. [60]

    Dora: Weight-decomposed low-rank adapta- tion,

    S.-Y . Liu, C.-Y . Wang, H. Yin, P. Molchanov, Y .-C. F. Wang, K.-T. Cheng, and M.-H. Chen, “Dora: Weight-decomposed low-rank adapta- tion,” arXiv preprint arXiv:2402.09353 , 2024

    2024 arXiv

  53. [61]

    Moelora: Con- trastive learning guided mixture of experts on parameter-efficient fine- tuning for large language models,

    T. Luo, J. Lei, F. Lei, W. Liu, S. He, J. Zhao, and K. Liu, “Moelora: Con- trastive learning guided mixture of experts on parameter-efficient fine- tuning for large language models,” arXiv preprint arXiv:2402.12851 , 2024. Lan Chen received the M.S. degree in Circuits and Syst...

    2024 arXiv

  54. [2021]

    He also serves as a Postdoctoral Researcher with the 38th Research Institute of China Electronics Technology Group Corporation, Hefei

    He is currently a Lecturer with the Informa- tion Materials and Intelligent Sensing Laboratory of Anhui Province, Anhui Provincial Key Laboratory of Multimodal Cognitive Computation, School of Ar- tificial Intelligence, Anhui University, Hefei, China. He also serves as a Postd...

    2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.