Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Expanding Event Modality Applications through a Robust CLIP-Based Encoder

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A CLIP-based event encoder, trained with contrastive, zero-shot consistency, and KL-divergence losses, transfers CLIP's zero-shot and text-aligned recognition to event data and outperforms prior event-CLIP alignment methods.

desk verdict Solid incremental event-CLIP encoder with a new VAD application, but the SOTA claim is contradicted by its own Table 2 and Proposition 1 is unsupported. read the letter →

arxiv 2412.03093 v2 pith:DPCEKYHI submitted 2024-12-04 cs.CV

Sungheon Jeong , Hanning Chen , Sanggeon Yun , Suhyeon Cho , Wenjun Huang , Xiangjian Liu , Mohsen Imani This is my paper · ORCID
classification cs.CV
keywords event-basedvisionCLIPzero-shotlearningcontrastivecross-modalalignmentcatastrophicforgettingobjectrecognitionvideoanomalydetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to give event-based cameras—sensors that record only pixel-level brightness changes—the same zero-shot and text-aligned recognition abilities as CLIP's image model. It does so by training a trainable event encoder, initialized from CLIP's frozen image encoder, to map a simple single-frame event representation into CLIP's shared embedding space. The training objective combines an InfoNCE contrastive loss between event and image embeddings, a zero-shot consistency loss that keeps both anchored to text embeddings, and a KL-divergence loss between the two embedding distributions. The authors report state-of-the-art object recognition on N-ImageNet, N-Caltech, and N-MNIST, and show the encoder transfers to video-extracted events for anomaly detection and to cross-modal retrieval across image, text, sound, and depth. The payoff, if correct, is that sparse event data can inherit the semantic understanding of large image-text models without collecting massive event datasets.

What carries the argument

The argument rests on three pieces. First, a single-frame event representation: the event stream $E(x,y,t,p)$ is summed over time $t$ and polarity $p$ and normalized, $\bar{E} = E(x,y)/(\max(E(x,y)) + 1)$, producing a one-channel gray-scale frame that the frozen CLIP image encoder can ingest. Second, a trainable event encoder $f_E$ initialized from CLIP's image encoder $f_I$, with $f_I$ and the text encoder $f_T$ frozen throughout training, so that only $f_E$ is updated. Third, the composite objective $L = L_{ct} + \alpha L_{zs} + L_{kl}$: $L_{ct}$ is the InfoNCE contrastive loss treating event embeddings as queries and image embeddings as keys; $L_{zs}$ is the zero-shot consistency loss that forms prediction logits from both event and image embeddings against text embeddings, keeping the event encoder from drifting away from CLIP's text-aligned semantics; and $L_{kl}$ is the KL divergence between event and image embedding distributions, which sharpens how well the event encoder reproduces the image encoder's feature distribution.

What would settle it

Train the same three-loss recipe with an event representation that preserves temporal structure, such as splitting the stream into several short time bins or separate polarity channels, then compare zero-shot accuracy on N-ImageNet and a motion-sensitive dataset; if the single-frame aggregation is load-bearing, preserving temporal detail should not improve accuracy, but if the assumption is too strong, the temporally structured representation would outperform the reported single-frame numbers.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that CLIP's zero-shot capability can be transferred to event data by training only an event encoder, while freezing the image and text encoders, using a combination of three losses: contrastive alignment of event embeddings with image embeddings (InfoNCE), a zero-shot consistency loss that aligns both event and image embeddings with text embeddings to prevent catastrophic forgetting, and a KL-divergence loss that matches the event embedding distribution to the image embedding distribution. This yields an event-image-text aligned model that outperforms prior event-CLIP alignment methods, with reported gains of +15.16% in zero-shot, +18.91% in 1-shot, and +7.35% in fine-tuning on N-ImageNet over the state-of-the-art using the same alignment method. The same encoder, without additional training, generalizes to events generated from ordinary video for anomaly detection and can be plugged into a cross-modal framework (ImageBind) to enable zero-shot retrieval between events and image, text, sound, and depth.

Load-bearing premise

The single-frame representation that sums all events across time and polarity into one gray-scale image is assumed to preserve enough information for CLIP's image encoder to extract the same semantic content as it would from a real image.

Editorial extensions

If this is right

  • Event-based cameras could be used for zero-shot classification on classes the model has never seen, without any event-specific training data.
  • Events extracted from ordinary video, not just native event cameras, can support tasks such as anomaly detection with better accuracy than using full frames with CLIP alone.
  • A single trained event encoder can serve as a drop-in module for cross-modal retrieval systems, linking event data to image, text, sound, and depth.
  • The three-loss recipe may transfer to other sparse or low-information modalities where a pre-trained vision-language encoder needs to be adapted without forgetting.
  • Few-shot event recognition becomes substantially more practical, with reported gains of roughly 19 points at one shot over the prior alignment method.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the single-frame aggregation is indeed sufficient, this suggests CLIP's image prior dominates the semantic content of events for recognition; a natural next test is whether motion-sensitive tasks (gesture, direction of movement) would require restoring temporal structure.
  • The zero-shot consistency loss $L_{zs}$ is likely the linchpin of the reported stability; an ablation that replaces $L_{zs}$ with a simple linear-probe regularization would isolate whether text-anchoring or generic regularization prevents forgetting.
  • The method's reliance on a frozen CLIP image encoder means the ceiling is set by CLIP's own representation; extending the recipe to larger or newer vision-language backbones could push the reported numbers further.
  • Because the paper reports anomaly detection on video-extracted events without fine-tuning, a natural follow-up is to combine the event encoder with a weakly supervised temporal localization head to see whether event-only signals can match or exceed frame-based weakly supervised anomaly detection.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes an event encoder for CLIP that transfers CLIP's image-text alignment and zero-shot capabilities to event data. Events are aggregated over time and polarity into a single gray-scale frame (Eq. 1) and fed to an event encoder initialized from CLIP's image encoder, while the image and text encoders remain frozen. Training combines an InfoNCE contrastive loss between event and image embeddings (Eq. 2), a zero-shot consistency loss adopted from ZSCL (Eq. 3), and a KL-divergence loss between event and image embedding distributions (Eq. 5). The authors report object-recognition results on N-ImageNet, N-Caltech101, and N-MNIST in zero-shot, few-shot, and fine-tuning settings, claim state-of-the-art performance with gains of +15.16% zero-shot, +18.91% 1-shot, and +7.35% fine-tuning, and additionally demonstrate event-based video anomaly detection and zero-shot cross-modal retrieval after integrating the encoder with ImageBind.

Significance. If the empirical results hold, the paper makes a useful practical contribution: it shows that a simple single-frame event representation, combined with a carefully chosen set of alignment losses, can transfer a substantial portion of CLIP's zero-shot and text-alignment capability to event data, and that the resulting encoder can be plugged into a cross-modal framework such as ImageBind. The release of code and the use of standard, publicly available datasets are positive features. However, the headline claim of state-of-the-art object recognition is contradicted by the paper's own Table 2 on N-MNIST, and the supporting theoretical proposition (Proposition 1) is not grounded in the training procedure actually used. The object-recognition results on N-ImageNet and N-Caltech101 are still interesting, and the cross-modal retrieval and VAD demonstrations add breadth, but the paper currently overstates its achievements and needs substantial revision to make its claims precise and reproducible.

major comments (4)
  1. [Abstract, Section 1, Table 2, Section 4.6] The unqualified claim of state-of-the-art object recognition with gains of +15.16% zero-shot, +18.91% 1-shot, and +7.35% fine-tuning is not supported by the paper's own results. On N-MNIST in Table 2, the proposed method achieves 46.95 zero-shot, below EventBind (56.81) and EventCLIP (48.72), and 69.43 in 1-shot, below both baselines (74.64 and 74.62). The +15.16% figure refers only to N-ImageNet zero-shot against EventCLIP, and even Section 4.6 concedes that N-MNIST shows 'relatively low zero-shot performance.' The claims should be recast per dataset and per baseline, or the state-of-the-art claim should be withdrawn.
  2. [Section 3.3, Proposition 1, Eq. (4)] The momentum-like update rule θ_E ← m·θ_E + (1-m)·(θ_target - η·∇θ_E L_q) is not derived from the objective in Eq. (3), and θ_target is never defined. The gradient expression in Eq. (4) is the gradient of only the first softmax term in Lzs (the image-text term is constant with respect to θ_E), and the proposition does not explain how this leads to the proposed update. Moreover, Section 4.1 describes standard training with a learning rate of 1e-6 for 200 epochs and does not mention using this update rule in any experiment. As written, Proposition 1 is an unsupported assertion rather than a proposition, and it should be removed or replaced with a precise statement of what is actually computed.
  3. [Tables 1-4, Section 4.1] No error bars or multiple-seed results are reported for any experiment. This is especially problematic for few-shot results where the gap between methods is small; for example, N-MNIST 5-shot accuracy is 94.23 for the proposed method versus 94.44 for EventBind, a difference of 0.21 that could easily reverse with a different random sample. The zero-shot and fine-tuning comparisons in Tables 1 and 2 also lack variance estimates. At minimum, the authors should report mean and standard deviation over at least three seeds for the few-shot settings, or clearly state that all numbers are single runs and temper the corresponding conclusions.
  4. [Section 4.3, Table 3] The VAD experiments compare only against zero-shot CLIP on three datasets, which is a weak baseline for anomaly detection. To support the claim that the event encoder 'expands the applicability' of the event modality to VAD, the paper should compare with existing weakly supervised or zero-shot VAD methods, or at least discuss why CLIP is the appropriate reference and acknowledge that current supervised VAD systems achieve much higher AUC. In addition, the event-extraction hyperparameters (pixel threshold 25, 16-frame stacks) are not studied or justified; a small sensitivity analysis would strengthen the section.
minor comments (5)
  1. [Section 4.2, paragraph beginning 'Zero & Few-Shot'] The phrases '+17.16% and +4.54%' and '+18.89% and +5.47%' do not state which baseline each gain refers to; the gains appear to use EventCLIP for N-ImageNet and EventBind for N-Caltech101. This should be stated explicitly to avoid cherry-picking ambiguity.
  2. [Table 1 caption] The caption says the approach demonstrates 'superior performance' on N-MNIST, but the ViT-L/14 result of 99.45 ties with EventBind (99.45); 'competitive' or 'comparable' would be more accurate.
  3. [Section 2, first paragraph] There is a typo: 'image model‘s capabilities' uses a curly apostrophe erroneously; also 'V AD' in Section 4.1 has an extra space. The text should be proofread.
  4. [Section 4.4, first paragraph] The claim that event retrieval is performed 'without additional training' is confusing because an adapter layer is introduced: 'we incorporate an adapter layer, designed as a single-layer module.' Please clarify whether the adapter is trained and, if so, on which data, or rephrase the claim.
  5. [Figure 4] The ablation figure shows accuracy curves over training for different loss configurations but provides no numerical summary or error bars; adding a table with final values would make the ablation quantitative and easier to interpret.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claim is an empirical transfer-learning result evaluated on held-out classes and external datasets; the borrowed losses are externally published, and no fitted parameter is relabeled as a prediction.

full rationale

The paper's derivation chain is an empirical transfer-learning pipeline, not a formal derivation that reduces to its inputs. The event representation E = E(x,y)/(max(E(x,y))+1) is an explicit preprocessing choice, not claimed to be derived from the results. The alignment objective L = L_ct + alpha*L_zs + L_kl combines InfoNCE with two externally published losses: L_zs is 'as suggested by [63]' (ZSCL, Zheng et al., ICCV 2023) and L_kl is 'proposed by [54]' (Event Camera Data Pre-training, Yang et al., ICCV 2023); neither is the authors' own prior work, so no self-citation chain carries the argument. Proposition 1 is explicitly a formalization of the adopted ZSCL objective ('as introduced by [63]') and is descriptive rather than load-bearing for the empirical claims. Evaluation is on the held-out 20% of N-ImageNet classes plus full N-Caltech101 and N-MNIST, with the paper stating it 'restrict[s] the evaluation on N-ImageNet to classes that were not used during pre-training'; no test-set fitting or fitted-parameter-renamed-as-prediction is present. The internal inconsistency between the abstract's unqualified SOTA claim and Table 2's N-MNIST results is a correctness and consistency concern, not circularity. Because the central claim is an empirical comparison against external baselines with held-out evaluation, the paper is self-contained for circularity purposes.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The model has no new physical entities. Its free parameters are training hyperparameters plus a hand-set event-extraction threshold; its axioms are the transferability of CLIP representations to aggregated event frames and the applicability of prior regularizers.

free parameters (7)
  • alpha (Lzs weight) = 0.1
    Weight for the zero-shot consistency loss in Eq. (5), set by hand without a sweep.
  • tau in Eq. (2) = 1
    Temperature for the event-image contrastive loss.
  • tau in Eq. (3) = 2
    Temperature for the zero-shot consistency loss.
  • learning rate = 1e-6
    Pre-training learning rate; no schedule or warmup described.
  • epochs = 200
    Pre-training duration.
  • VAD pixel threshold = 25/255
    Frame difference threshold to label a pixel as an event in video anomaly detection.
  • VAD frame stack = 16
    Number of consecutive frames aggregated to make one event instance.
assumptions (4)
  • domain assumption The aggregated single-frame event representation (Eq. 1) retains sufficient information for CLIP's frozen image encoder to align event and image embeddings.
    The paper states this representation 'makes it comprehensible to CLIP' but does not validate that temporal/polarity information is non-essential for the transfer.
  • domain assumption CLIP's frozen image and text encoders provide a valid shared embedding space for event data.
    The method initializes the event encoder from the image encoder and relies on the frozen encoders for stable targets, a premise shared with prior CLIP-transfer works.
  • domain assumption The borrowed ZSCL and KL losses are effective at preventing catastrophic forgetting and distribution shift in this setting.
    These losses are imported from [63] and [54]; the ablation in Fig. 4 is on one pretraining split, not a general proof.
  • ad hoc to paper The momentum-like parameter update in Proposition 1 follows from the gradient of the Lzs loss.
    The update rule θE ← m·θE + (1-m)(θtarget - η∇Lq) is stated without derivation, resembles an EMA rule unrelated to Eq. (4), and is not used in experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Expanding Event Modality Applications through a Robust CLIP-Based Encoder." pith.science (2026). https://pith.science/paper/DPCEKYHI

@misc{pith2026241203093,
  author       = {Pith},
  title        = {Pith review of: Expanding Event Modality Applications through a Robust CLIP-Based Encoder},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DPCEKYHI}},
  note         = {Machine review of arXiv:2412.03093}
}
read the original abstract

This paper introduces a powerful encoder that transfers CLIP`s capabilities to event-based data, enhancing its utility and expanding its applicability across diverse domains. While large-scale datasets have significantly advanced image-based models, the scarcity of comprehensive event datasets has limited performance potential in event modality. To address this challenge, we adapt CLIP`s architecture to align event embeddings with image embeddings, supporting zero-shot learning and preserving text alignment while mitigating catastrophic forgetting. Our encoder achieves strong performance in object recognition, with competitive results in zero-shot and few-shot learning tasks. Notably, it generalizes effectively to events extracted from video data without requiring additional training, highlighting its versatility. Additionally, we integrate this encoder within a cross-modality framework that facilitates interaction across five modalities-Image, Event, Text, Sound, and Depth-expanding the possibilities for cross-modal applications. Overall, this work underscores the transformative potential of a robust event encoder, broadening the scope and utility of event-based data across various fields.

Figures

Figures reproduced from arXiv: 2412.03093 by the authors.

Figure 1
Figure 1. Overview of the proposed approach for aligning event and image representations within the CLIP framework. The image and [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Extracting events from video frames. The differences [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 4
Figure 4. Zero-shot accuracy on unseen classes during N [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Visualize grad-based attention maps and text relevance scores with our pre-trained model (a) N-ImageNet unseen classes, (b) [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Continuous GNN-based Anomaly Detection on Edge using Efficient Adaptive Knowledge Graph Learning

    cs.LG 2024-11 conditional novelty 5.0 of 10

    A GNN-based video anomaly detector adapts its knowledge graph on-device through token-embedding updates, pruning, and node creation, avoiding cloud-based graph regeneration as anomaly types change.

Reference graph

Works this paper leans on

69 extracted references · 47 canonical work pages · cited by 1 Pith paper

  1. [1]

    Unconstrained Open Vocabulary Image Classification: Zero-Shot Transfer from Text to Image via CLIP Inversion

    Philipp Allgeuer, Kyra Ahrens, and Stefan Wermter. Un- constrained open vocabulary image classification: Zero-shot transfer from text to image via clip inversion. arXiv preprint arXiv:2407.11211, 2024. 1

    work page Pith review arXiv 2024

  2. [2]

    Learning to prompt clip for monocular depth estimation: Exploring the limits of human language

    Dylan Auty and Krystian Mikolajczyk. Learning to prompt clip for monocular depth estimation: Exploring the limits of human language. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 2039–2047,

    work page

  3. [3]

    Graph-based object classifica- tion for neuromorphic vision sensing

    Yin Bi, Aaron Chadha, Alhabib Abbas, Eirina Bourtsoulatze, and Yiannis Andreopoulos. Graph-based object classifica- tion for neuromorphic vision sensing. In Proceedings of the IEEE/CVF international conference on computer vision, pages 491–501, 2019. 5

    work page 2019

  4. [4]

    A differentiable recurrent surface for asynchronous event-based data

    Marco Cannici, Marco Ciccone, Andrea Romanoni, and Matteo Matteucci. A differentiable recurrent surface for asynchronous event-based data. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part XX 16, pages 136–152. Springer,

    2020

  5. [5]

    Recent event camera innovations: A survey

    Bharatesh Chakravarthi, Aayush Atul Verma, Kostas Dani- ilidis, Cornelia Fermuller, and Yezhou Yang. Recent event camera innovations: A survey. arXiv preprint arXiv:2408.13627, 2024. 1, 2

    arXiv 2024

  6. [6]

    Generic attention- model explainability for interpreting bi-modal and encoder- decoder transformers

    Hila Chefer, Shir Gur, and Lior Wolf. Generic attention- model explainability for interpreting bi-modal and encoder- decoder transformers. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 397–406,

  7. [7]

    Image-based clip-guided essence transfer

    Hila Chefer, Sagie Benaim, Roni Paiss, and Lior Wolf. Image-based clip-guided essence transfer. In European Con- ference on Computer Vision, pages 695–711. Springer, 2022. 1

    work page 2022

  8. [8]

    Understanding transferable representation learning and zero- shot transfer in clip

    Zixiang Chen, Yihe Deng, Yuanzhi Li, and Quanquan Gu. Understanding transferable representation learning and zero- shot transfer in clip. arXiv preprint arXiv:2310.00927, 2023. 1, 2

    arXiv 2023

Show all 69 references
  1. [9]

    Transfer clip for gen- eralizable image denoising

    Jun Cheng, Dong Liang, and Shan Tan. Transfer clip for gen- eralizable image denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 25974–25984, 2024. 1

    2024

  2. [10]

    Label-free event-based object recognition via joint learning with image reconstruction from events

    Hoonhee Cho, Hyeonseong Kim, Yujeong Chae, and Kuk- Jin Yoon. Label-free event-based object recognition via joint learning with image reconstruction from events. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 19866–19877, 2023. 1, 2

    2023

  3. [11]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 4

    2009

  4. [12]

    Event-based vision: A survey

    Guillermo Gallego, Tobi Delbr ¨uck, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, J ¨org Conradt, Kostas Daniilidis, et al. Event-based vision: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(1)...

    2020

  5. [13]

    Clip-adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 132(2): 581–595, 2024. 2

    2024

  6. [14]

    End-to-end learning of repre- sentations for asynchronous event-based data

    Daniel Gehrig, Antonio Loquercio, Konstantinos G Derpa- nis, and Davide Scaramuzza. End-to-end learning of repre- sentations for asynchronous event-based data. In Proceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 5633–5643, 2019. 5

    2019

  7. [15]

    Imagebind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15180–15190, 2023. 2, 5

    2023

  8. [16]

    Cyclip: Cyclic contrastive language-image pretraining

    Shashank Goel, Hritik Bansal, Sumit Bhatia, Ryan Rossi, Vishwa Vinay, and Aditya Grover. Cyclip: Cyclic contrastive language-image pretraining. Advances in Neural Informa- tion Processing Systems, 35:6704–6719, 2022. 2

    2022

  9. [17]

    Eventdrop: Data augmentation for event-based learning

    Fuqiang Gu, Weicong Sng, Xuke Hu, and Fangwen Yu. Eventdrop: Data augmentation for event-based learning. arXiv preprint arXiv:2106.05836, 2021. 1, 5

    2021 arXiv

  10. [18]

    Open-vocabulary object detection via vision and language knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. arXiv preprint arXiv:2104.13921 ,

    arXiv

  11. [19]

    Cliptrans: trans- ferring visual knowledge with pre-trained models for multi- modal machine translation

    Devaansh Gupta, Siddhant Kharbanda, Jiawei Zhou, Wan- hua Li, Hanspeter Pfister, and Donglai Wei. Cliptrans: trans- ferring visual knowledge with pre-trained models for multi- modal machine translation. In Proceedings of the IEEE/CVF international conference on computer vision...

    2023

  12. [20]

    Audioclip: Extending clip to image, text and au- dio

    Andrey Guzhov, Federico Raue, J ¨orn Hees, and Andreas Dengel. Audioclip: Extending clip to image, text and au- dio. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 976–980. IEEE, 2022. 2

    2022

  13. [21]

    Lidarclip or: How i learned to talk to point clouds

    Georg Hess, Adam Tonderski, Christoffer Petersson, Kalle ˚Astr¨om, and Lennart Svensson. Lidarclip or: How i learned to talk to point clouds. InProceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision , pages 7438–7447, 2024. 2

    2024

  14. [22]

    Learning monocular dense depth from events

    Javier Hidalgo-Carri ´o, Daniel Gehrig, and Davide Scara- muzza. Learning monocular dense depth from events. In 2020 International Conference on 3D Vision (3DV) , pages 534–542. IEEE, 2020. 5

    2020

  15. [23]

    Clip2point: Transfer clip to point cloud classifica- tion with image-depth pre-training

    Tianyu Huang, Bowen Dong, Yunhan Yang, Xiaoshui Huang, Rynson WH Lau, Wanli Ouyang, and Wangmeng Zuo. Clip2point: Transfer clip to point cloud classifica- tion with image-depth pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 221...

    2023

  16. [24]

    Transferring pre-trained multimodal rep- resentations with cross-modal similarity matching.Advances in Neural Information Processing Systems, 35:30826–30839,

    Byoungjip Kim, Sungik Choi, Dasol Hwang, Moontae Lee, and Honglak Lee. Transferring pre-trained multimodal rep- resentations with cross-modal similarity matching.Advances in Neural Information Processing Systems, 35:30826–30839,

  17. [25]

    N-imagenet: Towards robust, fine-grained object recognition with event cameras

    Junho Kim, Jaehyeok Bae, Gangin Park, Dongsu Zhang, and Young Min Kim. N-imagenet: Towards robust, fine-grained object recognition with event cameras. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2146–2156, 2021. 2, 4, 5

    2021

  18. [26]

    Masked event modeling: Self-supervised pretraining for event cameras

    Simon Klenk, David Bonello, Lukas Koestler, Nikita Araslanov, and Daniel Cremers. Masked event modeling: Self-supervised pretraining for event cameras. In Proceed- ings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2378–2388, 2024. 5

    2024

  19. [27]

    Graph-based asyn- chronous event processing for rapid object recognition

    Yijin Li, Han Zhou, Bangbang Yang, Ye Zhang, Zhaopeng Cui, Hujun Bao, and Guofeng Zhang. Graph-based asyn- chronous event processing for rapid object recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 934–943, 2021. 5

    2021

  20. [28]

    A 128×128 120 db 15 µs latency asynchronous temporal con- trast vision sensor

    Patrick Lichtsteiner, Christoph Posch, and Tobi Delbruck. A 128×128 120 db 15 µs latency asynchronous temporal con- trast vision sensor. IEEE journal of solid-state circuits , 43 (2):566–576, 2008. 1, 2

    2008

  21. [29]

    Fast classification and action recognition with event-based imaging

    Chang Liu, Xiaojuan Qi, Edmund Y Lam, and Ngai Wong. Fast classification and action recognition with event-based imaging. IEEE access, 10:55638–55649, 2022. 2

    2022

  22. [30]

    Revisiting temporal modeling for clip-based image-to-video knowledge transferring

    Ruyang Liu, Jingjia Huang, Ge Li, Jiashi Feng, Xinglong Wu, and Thomas H Li. Revisiting temporal modeling for clip-based image-to-video knowledge transferring. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6555–6564, 2023. 2

    2023

  23. [31]

    A revisit of sparse coding based anomaly detection in stacked rnn framework

    Weixin Luo, Wen Liu, and Shenghua Gao. A revisit of sparse coding based anomaly detection in stacked rnn framework. In Proceedings of the IEEE international conference on com- puter vision, pages 341–349, 2017. 5

    2017

  24. [32]

    Event-based asynchronous sparse con- volutional networks

    Nico Messikommer, Daniel Gehrig, Antonio Loquercio, and Davide Scaramuzza. Event-based asynchronous sparse con- volutional networks. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VIII 16, pages 415–431. Springer, 2020. 5

    2020

  25. [33]

    Repre- sentation learning with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. 3

    2018 arXiv

  26. [34]

    Converting static image datasets to spiking neuromorphic datasets using saccades

    Garrick Orchard, Ajinkya Jayawant, Gregory K Cohen, and Nitish Thakor. Converting static image datasets to spiking neuromorphic datasets using saccades. Frontiers in neuro- science, 9:437, 2015. 4, 5

    2015

  27. [35]

    St-adapter: Parameter-efficient image-to-video transfer learning

    Junting Pan, Ziyi Lin, Xiatian Zhu, Jing Shao, and Hong- sheng Li. St-adapter: Parameter-efficient image-to-video transfer learning. Advances in Neural Information Process- ing Systems, 35:26462–26477, 2022. 1

    2022

  28. [36]

    Back to event basics: Self-supervised learning of image reconstruc- tion for event cameras via photometric constancy

    Federico Paredes-Vall ´es and Guido CHE De Croon. Back to event basics: Self-supervised learning of image reconstruc- tion for event cameras via photometric constancy. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3446–3455, 2021. 1, 2

    2021

  29. [37]

    Ecodepth: Effective conditioning of diffusion models for monocular depth estimation

    Suraj Patni, Aradhye Agarwal, and Chetan Arora. Ecodepth: Effective conditioning of diffusion models for monocular depth estimation. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 28285–28295, 2024. 1

    2024

  30. [38]

    Esc: Dataset for environmental sound classi- fication

    Karol J Piczak. Esc: Dataset for environmental sound classi- fication. In Proceedings of the 23rd ACM international con- ference on Multimedia, pages 1015–1018, 2015. 5

    2015

  31. [39]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  32. [40]

    Events-to-video: Bringing modern computer vision to event cameras

    Henri Rebecq, Ren ´e Ranftl, Vladlen Koltun, and Davide Scaramuzza. Events-to-video: Bringing modern computer vision to event cameras. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 3857–3866, 2019. 1, 2, 5

    2019

  33. [41]

    Aegnn: Asynchronous event-based graph neural networks

    Simon Schaefer, Daniel Gehrig, and Davide Scaramuzza. Aegnn: Asynchronous event-based graph neural networks. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 12371–12381, 2022. 5

    2022

  34. [42]

    Towards understanding the modality gap in clip

    Peiyang Shi, Michael C Welle, M ˚arten Bj¨orkman, and Dan- ica Kragic. Towards understanding the modality gap in clip. In ICLR 2023 Workshop on Multimodal Representation Learning: Perks and Pitfalls, 2023. 2

    2023

  35. [43]

    Speechclip: Integrating speech with pre-trained vision and language model

    Yi-Jen Shih, Hsuan-Fu Wang, Heng-Jui Chang, Layne Berry, Hung-yi Lee, and David Harwath. Speechclip: Integrating speech with pre-trained vision and language model. In 2022 IEEE Spoken Language Technology Workshop (SLT), pages 715–722. IEEE, 2023. 2

    2022

  36. [44]

    Hats: Histograms of aver- aged time surfaces for robust event-based object classifica- tion

    Amos Sironi, Manuele Brambilla, Nicolas Bourdis, Xavier Lagorce, and Ryad Benosman. Hats: Histograms of aver- aged time surfaces for robust event-based object classifica- tion. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1731–1740, 2018. 5

    2018

  37. [45]

    Real-world anomaly detection in surveillance videos

    Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 6479–6488, 2018. 5

    2018

  38. [46]

    Weakly-supervised video anomaly detection with robust temporal feature magni- tude learning

    Yu Tian, Guansong Pang, Yuanhong Chen, Rajvinder Singh, Johan W Verjans, and Gustavo Carneiro. Weakly-supervised video anomaly detection with robust temporal feature magni- tude learning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4975–49...

    2021

  39. [47]

    Clipn for zero-shot ood detection: Teaching clip to say no

    Hualiang Wang, Yi Li, Huifeng Yao, and Xiaomeng Li. Clipn for zero-shot ood detection: Teaching clip to say no. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1802–1812, 2023. 2

    2023

  40. [48]

    Ev-gait: Event- based robust gait recognition using dynamic vision sensors

    Yanxiang Wang, Bowen Du, Yiran Shen, Kai Wu, Guan- grong Zhao, Jianguo Sun, and Hongkai Wen. Ev-gait: Event- based robust gait recognition using dynamic vision sensors. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 6358–6367, 2019. 2

    2019

  41. [49]

    Transferring clip’s knowledge into zero-shot point cloud semantic seg- mentation

    Yuanbin Wang, Shaofei Huang, Yulu Gao, Zhen Wang, Rui Wang, Kehua Sheng, Bo Zhang, and Si Liu. Transferring clip’s knowledge into zero-shot point cloud semantic seg- mentation. In Proceedings of the 31st ACM International Conference on Multimedia, pages 3745–3754, 2023. 1

    2023

  42. [50]

    Exploiting spatial sparsity for event cameras with visual transformers

    Zuowen Wang, Yuhuang Hu, and Shih-Chii Liu. Exploiting spatial sparsity for event cameras with visual transformers. In 2022 IEEE International Conference on Image Processing (ICIP), pages 411–415. IEEE, 2022. 2, 5

    2022

  43. [51]

    Improving zero-shot generalization for clip with synthesized prompts

    Zhengbo Wang, Jian Liang, Ran He, Nan Xu, Zilei Wang, and Tieniu Tan. Improving zero-shot generalization for clip with synthesized prompts. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3032– 3042, 2023. 1

    2023

  44. [52]

    Wav2clip: Learning robust audio repre- sentations from clip

    Ho-Hsiang Wu, Prem Seetharaman, Kundan Kumar, and Juan Pablo Bello. Wav2clip: Learning robust audio repre- sentations from clip. In ICASSP 2022-2022 IEEE Interna- tional Conference on Acoustics, Speech and Signal Process- ing (ICASSP), pages 4563–4567. IEEE, 2022. 2

    2022

  45. [53]

    Eventclip: Adapting clip for event-based object recognition

    Ziyi Wu, Xudong Liu, and Igor Gilitschenski. Eventclip: Adapting clip for event-based object recognition. arXiv preprint arXiv:2306.06354, 2023. 1, 2, 3, 5, 7

    2023 arXiv

  46. [54]

    Event camera data pre-training

    Yan Yang, Liyuan Pan, and Liu Liu. Event camera data pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10699–10709, 2023. 1, 4, 5

    2023

  47. [55]

    Event-guided low- light video semantic segmentation

    Zhen Yao and Mooi Choo Chuah. Event-guided low- light video semantic segmentation. arXiv preprint arXiv:2411.00639, 2024. 1

    2024

  48. [56]

    Clip2: Contrastive language- image-point pretraining from real-world point cloud data

    Yihan Zeng, Chenhan Jiang, Jiageng Mao, Jianhua Han, Chaoqiang Ye, Qingqiu Huang, Dit-Yan Yeung, Zhen Yang, Xiaodan Liang, and Hang Xu. Clip2: Contrastive language- image-point pretraining from real-world point cloud data. In Proceedings of the IEEE/CVF conference on computer ...

    2023

  49. [57]

    Vision-language models for vision tasks: A survey

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence,

  50. [58]

    Pointclip: Point cloud understanding by clip

    Renrui Zhang, Ziyu Guo, Wei Zhang, Kunchang Li, Xu- peng Miao, Bin Cui, Yu Qiao, Peng Gao, and Hongsheng Li. Pointclip: Point cloud understanding by clip. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8552–8562, 2022. 2

    2022

  51. [59]

    Can language understand depth? In Proceedings of the 30th ACM International Conference on Multimedia, pages 6868–6874,

    Renrui Zhang, Ziyao Zeng, Ziyu Guo, and Yafeng Li. Can language understand depth? In Proceedings of the 30th ACM International Conference on Multimedia, pages 6868–6874,

  52. [60]

    Tip- adapter: Training-free adaption of clip for few-shot classi- fication

    Renrui Zhang, Wei Zhang, Rongyao Fang, Peng Gao, Kun- chang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip- adapter: Training-free adaption of clip for few-shot classi- fication. In European conference on computer vision, pages 493–510. Springer, 2022. 2

  53. [61]

    Eventdance: Unsupervised source- free cross-modal adaptation for event-based object recogni- tion

    Xu Zheng and Lin Wang. Eventdance: Unsupervised source- free cross-modal adaptation for event-based object recogni- tion. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition , pages 17448–17458,

  54. [62]

    Deep learning for event-based vision: A comprehensive survey and bench- marks

    Xu Zheng, Yexin Liu, Yunfan Lu, Tongyan Hua, Tianbo Pan, Weiming Zhang, Dacheng Tao, and Lin Wang. Deep learning for event-based vision: A comprehensive survey and bench- marks. arXiv preprint arXiv:2302.08890, 2023. 1, 2

    2023 arXiv

  55. [63]

    Preventing zero-shot transfer degradation in continual learning of vision-language mod- els

    Zangwei Zheng, Mingyuan Ma, Kai Wang, Ziheng Qin, Xi- angyu Yue, and Yang You. Preventing zero-shot transfer degradation in continual learning of vision-language mod- els. In Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, pages 19125–19136, 2023. 1, 2, 4

    2023

  56. [64]

    Regionclip: Region- based language-image pretraining

    Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chun- yuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region- based language-image pretraining. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognitio...

    2022

  57. [65]

    E- clip: Towards label-efficient event-based open-world under- standing by clip

    Jiazhou Zhou, Xu Zheng, Yuanhuiyi Lyu, and Lin Wang. E- clip: Towards label-efficient event-based open-world under- standing by clip. arXiv preprint arXiv:2308.03135, 2023. 2

    2023 arXiv

  58. [66]

    Eventbind: Learning a unified representation to bind them all for event-based open-world understanding

    Jiazhou Zhou, Xu Zheng, Yuanhuiyi Lyu, and Lin Wang. Eventbind: Learning a unified representation to bind them all for event-based open-world understanding. 2024. 1, 2, 3, 5, 7

    2024

  59. [67]

    Anomalyclip: Object-agnostic prompt learn- ing for zero-shot anomaly detection

    Qihang Zhou, Guansong Pang, Yu Tian, Shibo He, and Jiming Chen. Anomalyclip: Object-agnostic prompt learn- ing for zero-shot anomaly detection. arXiv preprint arXiv:2310.18961, 2023. 1

    2023

  60. [68]

    A brief introduction to weakly supervised learning

    Zhi-Hua Zhou. A brief introduction to weakly supervised learning. National science review, 5(1):44–53, 2018. 5

    2018

  61. [69]

    Languagebind: Extending video-language pretrain- ing to n-modality by language-based semantic alignment

    Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, HongFa Wang, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zongwei Li, et al. Languagebind: Extending video-language pretrain- ing to n-modality by language-based semantic alignment. arXiv preprint arXiv:2310.01852, 2023. 2

    arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.