Pith. sign in

REVIEW 4 major objections 4 minor 27 references

Discrepancy-Aware Attention Network for Enhanced Audio-Visual Zero-Shot Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that audio-visual zero-shot learning fails because of two distinct imbalances—quality discrepancy across modalities and content discrepancy across samples—and that its DAAN architecture, built from sparse attention and…

desk verdict CSGM's convergence rate is identically zero by Eq. 6, so the paper's main mechanism is a constant multiplier; the empirical claims are unsupported. read the letter →

arxiv 2412.11715 v1 pith:YCEO4UEA submitted 2024-12-16 cs.CV cs.MMcs.SDeess.AS

RunLin Yu , Yipu Gong , Wenrui Li , Aiwen Sun , Mengren Zheng This is my paper · ORCID
classification cs.CVcs.MMcs.SDeess.AS
keywords audio-visualzero-shotlearninggeneralizedmodalityimbalancegradientmodulationsparseattentiontemporalconvolutionalnetworkvideoclassificationharmonicmean
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to fix a failure mode in audio-visual zero-shot learning: when one modality, usually vision, carries more task-relevant information, the model over-relies on it and under-learns the other, which hurts recognition of classes never seen in training. DAAN attacks this with two blocks: QDMA subtracts one softmax attention map from another on the same modality to strip redundant information from the strong modality, and CSGM rescales per-sample gradients using a contribution rate built from an optimization rate and a contrastive convergence rate. On the VGGSound, UCF101, and ActivityNet benchmarks, the authors report the best harmonic mean (the standard generalized-ZSL metric that balances seen and unseen accuracy) among the compared methods on UCF101 and ActivityNet, and a ZSL accuracy lead over the strongest baseline on VGGSound despite a slightly lower harmonic mean. A sympathetic reading is that modality imbalance is best treated at sample level, not only with global fusion weights.

What carries the argument

The load-bearing object is the per-sample, per-modality contribution rate $\eta_{a,p}^i = \max(V_{a,c}^i \times V_{a,o,p}^i, \gamma)$ (and its visual counterpart), which multiplies the gradient in the CSGM update. It combines a convergence rate $V_c$ computed from Euclidean distances between positive and negative audio, visual, and text embeddings with the optimization rate $V_o = \|G_p\|^2_2/\|\Theta_p\|^2_2$, the ratio of squared gradient norm to squared parameter norm. The other component is QDMA's sparse attention, defined as $S(Q_1K_1^T) - \beta\,S(Q_2K_2^T)$ on the same modality, which is meant to cancel redundant information carried by the dominant modality. The CSGM block, which the ablation identifies as the main source of harmonic-mean improvement, is where the paper's balancing claim lives.

What would settle it

Instrument DAAN during UCF101 training and log the per-sample coefficients $\eta_{a,p}^i$ and $\eta_{v,p}^i$; if they are identical for every sample because the optimization-rate formula supplies the same ratio, then CSGM is not actually balancing per-modality contributions, and retraining with CSGM removed should reproduce the reported gains if the true cause is elsewhere.

Watch

Extended reading notes

Core claim

The central claim is that the modality imbalance that suppresses unseen-class recognition in audio-visual zero-shot learning has two separate causes, and both can be removed inside one architecture. Quality discrepancy means the visual stream often carries more target-related information than the audio stream; DAAN's QDMA block reduces this by computing two softmax attention scores from splits of the same feature map and subtracting one from the other, leaving a sparser attention that keeps key information and discards redundancy. Content discrepancy means individual samples vary in how much audio or visual evidence they contribute; DAAN's CSGM block rescales the gradient of each modality module separately for each training sample, using a contribution rate $\eta_{i,p}$ formed from the optimization rate $\|G_p\|^2_2/\|\Theta_p\|^2_2$ and a convergence rate measured from contrastive distances among positive and negative embeddings. On the three benchmarks, this combination is reported to give the best generalized zero-shot harmonic mean on UCF101 (32.42) and ActivityNet (13.87) among the methods compared, and the ablation credits most of the harmonic-mean gain to CSGM.

Load-bearing premise

The balancing trick depends on treating audio and visual learning signals separately, but the printed formulas for the optimization rate appear to define the same value for both modalities, so the claimed mechanism rests on an unstated implementation detail.

Editorial extensions

If this is right

  • On UCF-GZSL, DAAN reports the highest harmonic mean among the compared methods (32.42), and a ZSL accuracy of 31.09 that beats all baselines except MDFT.
  • On ActivityNet-GZSL, DAAN's harmonic mean of 13.87 is the best among the compared methods, so the benefit is not confined to short action clips.
  • Ablations on UCF show each component adds value: QDMA raises ZSL from 19.03 to 28.78, and the full CSGM raises harmonic mean from 26.46 to 32.42.
  • DAAN reaches comparable or better accuracy than MDFT with roughly 2.5 times lower runtime, because it avoids a spiking neural network.
  • The method currently applies only to complete audio-visual samples; the authors state that videos with missing modalities are outside its scope.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the implementation follows the printed equations literally, the audio and visual optimization rates are identical, so the reported gains would have to come from global gradient scaling or QDMA rather than from modality-specific balancing.
  • Editorial inference: the convergence-rate term suggests a direct test—replace CSGM's contrastive estimate with ground-truth class labels to set per-sample weights; if gains disappear, the contrastive signal is the active ingredient.
  • Editorial inference: the same per-sample gradient gating could transfer to other audio-visual tasks, such as retrieval or captioning, where one modality often dominates and global fusion weights do not fix per-example variation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes DAAN, an audio-visual zero-shot learning architecture with two novel components: QDMA, a sparse-attention block that reduces redundant information in the higher-quality modality, and CSGM, a sample-level gradient modulation block intended to balance content discrepancies by scaling parameter updates according to a contribution rate derived from optimization and convergence rates. The method is evaluated on VGGSound, UCF101, and ActivityNet for both ZSL and GZSL, and an ablation study on UCF attributes gains to the two blocks. The manuscript claims state-of-the-art performance, though the reported numbers are mixed relative to MDFT and other baselines.

Significance. If the proposed mechanism worked as described, the paper would address a real and recognized problem: modality imbalance in audio-visual zero-shot learning, and it would do so at the sample level, which is a genuinely finer granularity than prior global gradient-modulation schemes. The motivation is clear, the benchmark suite is standard, and the ablation design is reasonable in intent. However, the central CSGM derivation as printed degenerates: Eq. (6) forces both convergence rates to be identically zero, reducing the contribution rate to a constant and eliminating the claimed sample- and modality-specific modulation. This is an internal inconsistency, not a matter of disagreement with external consensus, and it directly undermines the paper's main contribution. The paper also has no released code, and several equations cannot be implemented as written. The significance of the empirical results is therefore compromised because the main mechanism claimed to produce them is not actually defined in a functioning form.

major comments (4)
  1. [§II-B, Eq. (6)] Eq. (6) defines V_i_{a,c} = δ_i_{a+,w+,a−} · δ_i_{w+,a−,a−} · δ_i_{a+,w−,w−}. In the second and third factors, the arguments e and n coincide: δ_i_{w+,a−,a−} = max(d(θ_w+, θ_a−) − d(θ_w+, θ_a−), 0) = 0, and δ_i_{a+,w−,w−} = max(d(θ_a+, θ_w−) − d(θ_a+, θ_w−), 0) = 0. Therefore V_i_{a,c} ≡ 0 for every sample, and analogously V_i_{v,c} ≡ 0. Substituting into Eq. (5) gives η_i_{a,p} = η_i_{v,p} = max(0, γ) = γ, and Eq. (8) reduces to Θ − γG + ϵ, a constant scaling with no sample-level or modality-dependent content. This contradicts Table II, where the row 'Base* + QDMA + CSGM (V_c)' reports HM 31.31 versus 26.46 for 'Base* + QDMA'; if V_c were identically zero, the two rows would be indistinguishable up to noise. As printed, the CSGM mechanism cannot explain the ablation gain, and the central contribution of the paper is not supported.
  2. [§II-B, Eq. (7)] The two displayed formulas for V_i_{a,o,p} and V_i_{v,o,p} are identical: both equal ||G_p||² / ||Θ_p||². No separate per-modality gradients G_a,p, G_v,p or parameters Θ_a,p, Θ_v,p are defined before this equation, although Eq. (8) then uses Θ_i_{a,p}, G_i_{a,p} and Θ_i_{v,p}, G_i_{v,p}. Without a per-modality parameterization of G and Θ, the claimed audio-versus-visual gradient modulation cannot be derived from the stated mathematics, and the entire content-discrepancy balancing mechanism depends on an unstated quantity that never appears in the paper.
  3. [§II-A, Eq. (1)] Eq. (1) as printed is dimensionally inconsistent: S(Q_i_1 K_i_1^T) is a softmax attention matrix, while S(Q_i_2 K_i_2^T)V_i is an attended feature vector. Subtracting a matrix from a vector is not a valid operation under the stated shapes, and the accompanying text says the two softmax scores are subtracted but attaches V_i to only the second term. The intended sparse-attention operation needs to be written with consistent tensor shapes, and all later equations that consume o1_i depend on this correction.
  4. [§II-B, Eq. (5) and §III-A] The text after Eq. (5) states that V_i_c and V_i_o,p are 'normalized under the regulation of weight parameter µ', but no normalization formula or definition of µ appears in the method section. The hyperparameter section lists distinct µ values per dataset, yet the equations for V_i_c and V_i_o contain no µ. This missing definition makes the contribution-rate formula non-reproducible and prevents the reader from verifying whether the claimed modulation is actually controlled by the stated hyperparameter.
minor comments (4)
  1. [§III-B, Table I and abstract] The abstract and Section III-B claim state-of-the-art performance and that DAAN 'significantly exceeds almost all baselines', but Table I shows DAAN is below MDFT on VGGSound HM (8.32 vs. 8.72) and ActivityNet ZSL (11.15 vs. 12.55), and its UCF seen accuracy (52.38) is below TCaF (58.60). The performance claims should be qualified to specific metrics and datasets.
  2. [§III-C, Fig. 3] The text says the (G)ZSL scores peak at n=5 with maxima of 27.59 and 26.54, and that γ peaks at 0.3, but Table II reports DAAN with HM 32.42 and ZSL 31.09 using n=2 and γ=0.45 for UCF. It is unclear what Fig. 3 is plotting or under which configuration, and the discrepancy should be resolved.
  3. [§II-C, Eq. (11)] Eq. (11) uses the term d(ϕ_i_a, ϕ_i_w), but ϕ_i is only defined in Eq. (4) as an element of {ϕ_a,i, ϕ_v,i}; no ϕ_w,i is ever defined, so the regularization loss is incomplete as written.
  4. [Throughout] There are numerous language and typographical errors, e.g., 'Experiments demonstrates', 'an novel approach', 'the modal’s parameters', '2-norm demonstrates', and inconsistent use of 'modal' for 'model'. These should be corrected in a revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: benchmark claims are empirical and not derived from fitted constants; noted equation-level degeneracies are internal-consistency issues, not circular reasoning.

full rationale

I traced the claimed derivation chain. The paper's reported results are benchmark accuracies obtained by training the DAAN architecture (QDMA + CSGM) with the losses in Eqs. 9-12; none of these accuracies is used as an input to the architecture, and no parameter is fitted to the reported HM/ZSL numbers. The hyperparameters in Section III-A (mu, gamma, k, n) are fixed per dataset before the comparison and are not used to construct the test metrics, so they are standard hyperparameter choices rather than fitted-input-called-prediction. The contribution-rate formula in Eqs. 5-7 is a proposed definition, not an inversion of any output. The self-citations [2], [3], and [6] are baselines or prior spiking-transformer works; they are not invoked as a uniqueness theorem and do not carry the load of DAAN's claims. I found no self-definitional step, no renaming of a known result, and no imported uniqueness theorem. I also considered the internal degeneracy in Eq. 6, where factors like δ_i_{w+,a−,a−} have e=n and therefore equal max(d-d,0)=0, which would make V_c=0 and reduce Eq. 5 to η=γ. This is a serious correctness/consistency flaw in the CSGM mechanism as written, but it is not circular reasoning: the benchmark predictions are not equivalent to the formula's inputs, and the flaw does not consist of renaming a fit as a prediction. For that reason, the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on standard deep learning machinery, plus a set of hand-designed quantities: the differential-attention subtraction in QDMA, the product-of-distance convergence rate, and the weighted combination of convergence and optimization rates. The main free parameters are hyperparameters tuned per dataset. No new physical or conceptual entities are introduced.

free parameters (5)
  • beta = not reported
    Controls the weighted subtraction in Eq. 1 and the linear scaling (1-beta) in Eq. 2; not assigned in Section III-A.
  • mu = 1.15/0.5/1.2 for UCF/ActivityNet/VGGSound
    Weights the combination of convergence and optimization rates in Eq. 5; tuned per dataset.
  • gamma = 0.45/0.5/0.6 for UCF/ActivityNet/VGGSound
    Lower bound on contribution rate in Eq. 5; tuned per dataset.
  • k = 3/9/5 for UCF/ActivityNet/VGGSound
    Dilation gap in the TCN's dilated convolution; hand-set per dataset.
  • n = 2/3/5 for UCF/ActivityNet/VGGSound
    Number of dilated convolutional layers in QDMA's temporal branch; hand-set per dataset.
assumptions (5)
  • standard math Softmax attention and multi-head cross-attention are taken as given.
    Section II-A uses S(QK^T) and a cross-attention block without derivation or proof.
  • domain assumption SeLaVi-pretrained audio and visual features are a sufficient representation for audio-visual zero-shot learning.
    Section III-A uses SeLaVi features for every dataset and does not validate against other feature extractors.
  • domain assumption The projection learned on seen classes transfers to unseen classes.
    The ZSL setup in Section II assumes F: (v_s, a_s) to w_s applies to unseen classes.
  • ad hoc to paper The contribution rate eta = max(V_c * V_o, gamma) is a valid measure of modality contribution.
    Eq. 5 defines a hand-crafted product of two rates with a floor; no theoretical justification is offered.
  • ad hoc to paper The convergence rate V_c as a product of three clipped distance differences measures sample-level convergence.
    Eq. 6 defines V_c without derivation or validation against known convergence measures.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Discrepancy-Aware Attention Network for Enhanced Audio-Visual Zero-Shot Learning." pith.science (2026). https://pith.science/paper/YCEO4UEA

@misc{pith2026241211715,
  author       = {Pith},
  title        = {Pith review of: Discrepancy-Aware Attention Network for Enhanced Audio-Visual Zero-Shot Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YCEO4UEA}},
  note         = {Machine review of arXiv:2412.11715}
}
read the original abstract

Audio-visual Zero-Shot Learning (ZSL) has attracted significant attention for its ability to identify unseen classes and perform well in video classification tasks. However, modal imbalance in (G)ZSL leads to over-reliance on the optimal modality, reducing discriminative capabilities for unseen classes. Some studies have attempted to address this issue by modifying parameter gradients, but two challenges still remain: (a) Quality discrepancies, where modalities offer differing quantities and qualities of information for the same concept. (b) Content discrepancies, where sample contributions within a modality vary significantly. To address these challenges, we propose a Discrepancy-Aware Attention Network (DAAN) for Enhanced Audio-Visual ZSL. Our approach introduces a Quality-Discrepancy Mitigation Attention (QDMA) unit to minimize redundant information in the high-quality modality and a Contrastive Sample-level Gradient Modulation (CSGM) block to adjust gradient magnitudes and balance content discrepancies. We quantify modality contributions by integrating optimization and convergence rate for more precise gradient modulation in CSGM. Experiments demonstrates DAAN achieves state-of-the-art performance on benchmark datasets, with ablation studies validating the effectiveness of individual modules.

Figures

Figures reproduced from arXiv: 2412.11715 by the authors.

Figure 1
Figure 1. Quality & Content Discrepancies in Audio-Visual Dataset: (a) Quality discrepancies exist between audio and visual modalities. The visual one holds more target-related data, causing the model’s prediction to depend more on it. (b) Content discrepancies occur in samples. For the same category like playing basketball, distinct audio￾visual samples result in diverse recognition outcomes because of different information … view at source ↗
Figure 2
Figure 2. The DAAN architecture incorporates audio, visual, and textual features as inputs, simultaneously extracting semantic information. The QDMA unit removes redundant information from high-quality modalities to address quality discrepancies. Additionally, it extracts temporal embeddings of audio-visual features to enhance temporal information. The cross-attention layer facilitates information interaction between audio-vi… view at source ↗
Figure 3
Figure 3. Component Performance in ZSL. Lastly, our DAAN holds a 25.81% increase in HM and a 63.37% increase in ZSL compared to “Base” in total. To further analyze the performance trends of QDMA and CSGM under varying parameters, we conducted experiments by adjusting n and γ which correspond to the QDMA and CSGM components, respectively. The experimental results are presented in Fig. (3). As shown in Fig. (3), the ZSL and GZS… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 22 canonical work pages

  1. [1]

    A survey of zero-shot learning,

    Wei Wang, Vincent Wenchen Zheng, Han Yu, and Chunyan Miao, “A survey of zero-shot learning,” ACM Transactions on Intelligent Systems and Technology (TIST) , vol. 10, pp. 1 – 37, 2019

    work page 2019

  2. [2]

    Motion-decoupled spiking transformer for audio- visual zero-shot learning,

    Wenrui Li, Xi-Le Zhao, Zhengyu Ma, Xingtao Wang, Xiaopeng Fan, and Yonghong Tian, “Motion-decoupled spiking transformer for audio- visual zero-shot learning,” in Proceedings of the 31st ACM International Conference on Multimedia , 2023, pp. 3994–4002

    work page 2023

  3. [3]

    Spiking tucker fusion transformer for audio-visual zero-shot learning,

    Wenrui Li, Penghong Wang, Ruiqin Xiong, and Xiaopeng Fan, “Spiking tucker fusion transformer for audio-visual zero-shot learning,” IEEE Transactions on Image Processing , vol. 33, pp. 4840–4852, 2024

    work page 2024

  4. [4]

    Audiovisual generalised zero-shot learning with cross-modal attention and language,

    Otniel-Bogdan Mercea, Lukas Riesch, A. Sophia Koepke, and Zeynep Akata, “Audiovisual generalised zero-shot learning with cross-modal attention and language,” 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 10543–10553, 2022

    work page 2022

  5. [5]

    Temporal and cross-modal attention for audio-visual zero-shot learning,

    Otniel-Bogdan Mercea, Thomas Hummel, A. Sophia Koepke, and Zeynep Akata, “Temporal and cross-modal attention for audio-visual zero-shot learning,” in European Conference on Computer Vision (ECCV), 2022

    work page 2022

  6. [6]

    Modality-fusion spiking transformer network for audio-visual zero-shot learning,

    Wenrui Li, Zhengyu Ma, Liang-Jian Deng, Hengyu Man, and Xiaopeng Fan, “Modality-fusion spiking transformer network for audio-visual zero-shot learning,” in 2023 IEEE International Conference on Mul- timedia and Expo (ICME) , 2023, pp. 426–431

    work page 2023

  7. [7]

    What makes training multi- modal classification networks hard?,

    Weiyao Wang, Du Tran, and Matt Feiszli, “What makes training multi- modal classification networks hard?,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2020, pp. 12692– 12702

    work page 2020

  8. [8]

    Modality competition: What makes joint training of multi- modal network fail in deep learning? (provably),

    Yu Huang, Junyang Lin, Chang Zhou, Hongxia Yang, and Longbo Huang, “Modality competition: What makes joint training of multi- modal network fail in deep learning? (provably),” ArXiv, vol. abs/2203.12221, 2022

    arXiv 2022

Show all 27 references
  1. [9]

    Improv- ing multimodal accuracy through modality pre-training and attention,

    Aya Abdelsalam Ismail, Mahmudul Hasan, and Faisal Ishtiaq, “Improv- ing multimodal accuracy through modality pre-training and attention,” arXiv preprint arXiv:2011.06102 , 2020

    2011 arXiv

  2. [10]

    Learning to balance the learning rates between various modalities via adaptive tracking factor,

    Ya Sun, Sijie Mai, and Haifeng Hu, “Learning to balance the learning rates between various modalities via adaptive tracking factor,” IEEE Signal Processing Letters , vol. 28, pp. 1650–1654, 2021

    2021

  3. [11]

    Characterizing and overcoming the greedy nature of learning in multi- modal deep neural networks,

    Nan Wu, Stanisław Jastrzbski, Kyunghyun Cho, and Krzysztof J. Geras, “Characterizing and overcoming the greedy nature of learning in multi- modal deep neural networks,” 2022

    2022

  4. [12]

    Balanced multimodal learning via on-the-fly gradient modulation,

    Xiaokang Peng, Yake Wei, Andong Deng, Dong Wang, and Di Hu, “Balanced multimodal learning via on-the-fly gradient modulation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

    2022

  5. [13]

    Pmr: Prototypical modal rebalance for multimodal learning,

    Yunfeng Fan, Wenchao Xu, Haozhao Wang, Junxiao Wang, and Song Guo, “Pmr: Prototypical modal rebalance for multimodal learning,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR) , 2023, pp. 20029–20038

    2023

  6. [14]

    Enhancing multimodal cooperation via sample-level modality valuation,

    Yake Wei, Ruoxuan Feng, Zihe Wang, and Di Hu, “Enhancing multimodal cooperation via sample-level modality valuation,” in 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 27328–27337

    2024

  7. [15]

    Multimodal fusion on low-quality data: A comprehensive survey,

    Qingyang Zhang, Yake Wei, Zongbo Han, Huazhu Fu, Xi Peng, Cheng Deng, Qinghua Hu, Cai Xu, Jie Wen, Di Hu, and Changqing Zhang, “Multimodal fusion on low-quality data: A comprehensive survey,” ArXiv, vol. abs/2404.18947, 2024

    2024 arXiv

  8. [16]

    An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,

    Shaojie Bai, J. Zico Kolter, and Vladlen Koltun, “An empirical evaluation of generic convolutional and recurrent networks for sequence modeling,” ArXiv, vol. abs/1803.01271, 2018

    2018 arXiv

  9. [17]

    Differential transformer,

    Tianzhu Ye, Li Dong, Yuqing Xia, Yutao Sun, Yi Zhu, Gao Huang, and Furu Wei, “Differential transformer,” 2024

    2024

  10. [18]

    Group normalization,

    Yuxin Wu and Kaiming He, “Group normalization,” in Proceedings of the European conference on computer vision (ECCV) , 2018, pp. 3–19

    2018

  11. [19]

    Coordinated joint multimodal embeddings for generalized audio-visual zero-shot classification and retrieval of videos,

    Kranti Kumar Parida, Neeraj Matiyali, Tanaya Guha, and Gaurav Sharma, “Coordinated joint multimodal embeddings for generalized audio-visual zero-shot classification and retrieval of videos,” in 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) , 2020, pp. 3240–3249

    2020

  12. [20]

    Avgzslnet: Audio-visual generalized zero-shot learning by reconstructing label features from multi-modal embeddings,

    Pratik Mazumder, Pravendra Singh, Kranti K. Parida, and Vinay P. Namboodiri, “Avgzslnet: Audio-visual generalized zero-shot learning by reconstructing label features from multi-modal embeddings,” 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) , pp. 3089–...

    2021

  13. [21]

    Temporal–semantic aligning and reasoning transformer for audio-visual zero-shot learning,

    Kaiwen Zhang, Kunchen Zhao, and Yunong Tian, “Temporal–semantic aligning and reasoning transformer for audio-visual zero-shot learning,” Mathematics, vol. 12, no. 14, pp. 2200, 2024

    2024

  14. [22]

    Hyperbolic audio-visual zero-shot learning,

    Jie Hong, Zeeshan Hayder, Junlin Han, Pengfei Fang, Mehrtash Harandi, and Lars Petersson, “Hyperbolic audio-visual zero-shot learning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2023, pp. 7873–7883

    2023

  15. [23]

    Vggsound: A large-scale audio-visual dataset,

    Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman, “Vggsound: A large-scale audio-visual dataset,” in ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 721–725

    2020

  16. [24]

    Ucf101: A dataset of 101 human actions classes from videos in the wild,

    Khurram Soomro, Amir Zamir, and Mubarak Shah, “Ucf101: A dataset of 101 human actions classes from videos in the wild,” ArXiv, vol. abs/1212.0402, 2012

    2012 arXiv

  17. [25]

    Activitynet: A large-scale video benchmark for human activity understanding,

    Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Car- los Niebles, “Activitynet: A large-scale video benchmark for human activity understanding,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2015, pp. 961–970

    2015

  18. [26]

    Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly,

    Yongqin Xian, Christoph H. Lampert, Bernt Schiele, and Zeynep Akata, “Zero-shot learning—a comprehensive evaluation of the good, the bad and the ugly,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 41, no. 9, pp. 2251–2265, 2019

    2019

  19. [27]

    Labelling unlabelled videos from scratch with multi-modal self- supervision,

    Yuki Asano, Mandela Patrick, Christian Rupprecht, and Andrea Vedaldi, “Labelling unlabelled videos from scratch with multi-modal self- supervision,” in Advances in Neural Information Processing Systems , H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, Eds. 2020...

    2020

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.