Pith. sign in

REVIEW 5 major objections 5 minor 30 references

When Vision Models Meet Parameter Efficient Look-Aside Adapters Without Large-Scale Audio Pretraining

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read 2% adapters let vision models beat audio pretraining

desk verdict A sensible adapter variant for audio PEFT with a systematic ablation, but the 'surpass pretrained audio models' headline is built on a single-run 0.77-point gap with no error bars. read the letter →

arxiv 2412.05951 v1 pith:OUWPIWLB submitted 2024-12-08 cs.SD cs.CVcs.LGeess.AS

Juan Yeo , Jinkwan Jang , Kyubyung Chae , Seongkyu Mun , Taesup Kim This is my paper · ORCID
classification cs.SDcs.CVcs.LGeess.AS
keywords audioclassificationparameter-efficientfine-tuninglook-asideadaptervisiontransformertransferlearningmel-spectrogramtime-frequencyconvolutionspretrainingbypass
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes Look-Aside Adapters (LoAA), a parameter-efficient fine-tuning method that lets an ImageNet-pretrained vision transformer process audio spectrograms without any large-scale audio pretraining. The adapters replace the linear projections in parallel adapters with one-dimensional convolutions oriented along the time axis or the frequency axis, so that tokens can exchange information across those two dimensions while the backbone attention remains frozen. On EPIC-SOUNDS, the method reaches 54.52% top-1 accuracy with 2% trainable parameters, surpassing SSAST (53.75%) which was pretrained on AudioSet and LibriSpeech. On ESC-50 and Speech Commands V2 it stays within about one percentage point of audio-pretrained models. If correct, the result means the expensive two-stage paradigm of audio pretraining followed by fine-tuning can be bypassed for several audio and speech tasks.

What carries the argument

The central object is the Look-Aside Adapter (LoAA), a parallel adapter whose down- and up-projection matrices are replaced by 1D convolutions with kernel size Nx1 or 1xN, operating on tokens reshaped into a time-frequency grid. The time kernel (1x3) and frequency kernel (3x1) promote interactions between tokens along the respective spectrogram axis; the output is added back to the transformer block output in the same residual style as a Houlsby adapter. The bottleneck dimension r is adjusted to keep parameter counts equal across kernel shapes, which is what makes the comparison between linear (1x1) and directional kernels controlled.

What would settle it

Train the same LoAA setup on EPIC-SOUNDS with the time kernel applied along the frequency axis and the frequency kernel applied along the time axis, holding parameter count fixed. If the swapped configuration matches the original accuracy (especially if it still beats the 1x1 linear adapter), the claimed directional inductive bias is not what drives the gain; if the swap drops accuracy back to or below the linear-adapter baseline, the orientation is load-bearing.

Watch

Extended reading notes

Core claim

The central claim is that the gap between vision-pretrained and audio-pretrained models on audio tasks can be closed by adapters whose internal projections are directional convolutions instead of linear layers, without touching the frozen transformer weights. By reshaping the token sequence into a two-dimensional time-frequency grid inside the adapter, a 1x3 convolution only mixes tokens that are adjacent in time, and a 3x1 convolution only mixes tokens adjacent in frequency. The paper shows this simple change outperforms the 1x1 linear parallel adapter at equal parameter counts, and that an isotropic 3x3 kernel does worse, supporting the claim that the directionality itself, not the mere presence of convolution, carries the benefit. On EPIC-SOUNDS the best single placement (attention block, time kernel, 2% parameters) reaches 54.52% top-1 accuracy, while the audio-pretrained SSAST baseline scores 53.75%, and the combined time/frequency placement at 5% reaches a mAP of 0.238, matching SSAST's 0.237.

Load-bearing premise

The result depends on the premise that a 1x3 convolution over time-neighboring tokens and a 3x1 convolution over frequency-neighboring tokens supplies the right inductive bias for audio spectrograms when the transformer attention is frozen.

Editorial extensions

If this is right

  • ImageNet-pretrained vision transformers with LoAA can match or beat SSAST, which was pretrained on AudioSet-2M and LibriSpeech, on EPIC-SOUNDS using only 2% trainable parameters.
  • The one-stage PEFT recipe removes the need for large-scale audio pretraining and its accompanying data collection and compute for EPIC-SOUNDS, ESC-50, and Speech Commands V2.
  • A time-oriented adapter on the attention block combined with a frequency-oriented adapter on the FFN block is the best 5%-parameter configuration, outperforming all same-size parallel-adapter variants.
  • The per-axis design outperforms an isotropic 3x3 convolution at equal parameters, indicating that respecting the heterogeneity of time and frequency is what helps.
  • Performance on speech tasks (SPC-2) and environmental sounds (ESC-50) stays within about 1% of audio-pretrained models, so the approach does not obviously sacrifice generality on the tested benchmarks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct testable extension would be to swap the kernel orientations (time kernel along frequency, frequency kernel along time) and check whether EPIC-SOUNDS accuracy collapses; if it does not, the directional inductive bias is not the active ingredient.
  • The same blockwise anisotropic adapter idea could transfer to other inputs with heterogeneous axes, such as video (spatial versus temporal) or sequences with a known layout, where frozen pretrained attention may need a similar reorientation.
  • Because the method leaves the backbone frozen, it is likely to compose with other PEFT techniques, such as low-rank updates or prompt tuning; the paper does not test those combinations.
  • The results do not yet address whether the adapter would close the gap on harder audio tasks or larger backbones; the paper's smallest model comparison leaves open whether the benefit scales with backbone size.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes Look-Aside Adapter (LoAA), a parameter-efficient fine-tuning module for adapting ImageNet-pretrained vision transformers to audio and speech tasks without a separate large-scale audio pretraining stage. LoAA replaces the linear projections of a parallel adapter with 1x3 and 3x1 convolutions over the time and frequency axes of the spectrogram-token grid, thereby introducing inter-token interactions across those heterogeneous dimensions while the backbone remains frozen. The method is evaluated on EPIC-SOUNDS, ESC-50, and Speech Commands V2 and compared with parallel adapters, full fine-tuning of ImageNet-pretrained AST, and audio-pretrained models SSAST and MAE-AST. The central claim is that a vision model with roughly 2% trainable parameters can match or surpass audio-pretrained SSAST, with the strongest result being 54.52% top-1 accuracy on EPIC-SOUNDS versus SSAST's 53.75%.

Significance. If the headline result were robust, the paper would make a useful practical point: for several audio benchmarks, a cheap single-stage PEFT route from an ImageNet model could rival the expensive two-stage paradigm of AudioSet/LibriSpeech pretraining followed by full fine-tuning. The proposed modification is simple, parameter-efficient, and the paper reports consistently positive comparisons against a parallel-adapter baseline within its own experimental setup, which is a genuine strength. The main limitations are empirical rigor: no error bars or seed counts, test-set-based configuration selection, externally sourced baselines, and an abstract that overstates the breadth of the results. With additional experiments and more careful claims, the contribution could be a solid empirical paper; as it stands, the evidence is not yet load-bearing for the stated conclusions.

major comments (5)
  1. [Abstract and §3.3, Table 2] The abstract's claim that LoAA allows vision models to "reach or surpass the performance of pretrained audio models in various audio and speech tasks" is not supported by Table 2. On ESC-50, Ours-Attn(T) at 2% parameters reaches 85.4 versus SSAST's 88.8, a 3.4-point gap, and on SPC-2 it reaches 96.5 versus 98.0, a 1.5-point gap. Even on the flagship EPIC-SOUNDS dataset, the 2% model's mAP is 0.234, below SSAST's 0.237, so the "surpass" result depends on choosing top-1 accuracy rather than mAP. The statement in §3.3 that all results are "nearly equivalent... with a performance discrepancy of less than 1%" is also incorrect for ESC-50 and SPC-2. The claims should be narrowed and quantified, or supported with appropriate statistical evidence.
  2. [Tables 1-3] No experiment is reported with more than one seed, and no standard deviations or confidence intervals are given. The headline gap of 54.52 versus 53.75 (0.77 points) is smaller than the variation caused by architectural choices within Table 1: at 2.026M parameters, the 1x1 kernel gives 53.33 while the T(1,3) kernel gives 54.52, a 1.19-point swing; the FFN row shows a 1.15-point swing. This means the claimed advantage over SSAST is within the range of run-to-run or design-choice noise as currently reported. The authors should report mean and standard deviation over at least five seeds, and ideally a paired significance test, before asserting superiority.
  3. [§3.2, Tables 1 and 3] The paper does not clarify whether the best configuration, Ours-Attn(T) at 2% parameters, was selected on a held-out validation split. The implementation details in §3.2 state only that the PEFT learning rate was chosen from {5e-5, 1e-4, 5e-4}, and Tables 1 and 3 enumerate many kernel, placement, and combination choices on EPIC-SOUNDS. If the highlighted cell is the best among many evaluated on the test set, the reported 0.77-point margin is inflated by selection bias. The authors must state the validation protocol or, if none was used, treat the current numbers as exploratory and re-evaluate a pre-registered or validation-selected configuration.
  4. [§2.3, Table 1] The central novelty claim is that 1x3 (time) and 3x1 (frequency) kernels provide the right inductive bias for audio spectrograms. The ablations in Table 1 do not consistently support the orientation-specific part of this claim. At 1.035M parameters, F(3,1) is better than T(1,3) in both the attention row (53.79 vs 53.43) and the FFN row (53.53 vs 53.42); at 2.026M parameters, T(1,3) is better than F(3,1) by 0.08 in the attention row and by 0.77 in the FFN row. While T/F kernels are generally better than 1x1 and 3x3, the particular time-versus-frequency orientation is unstable across parameter budgets. The paper also does not vary kernel length (e.g., 1x5, 1x7) or patch size, so the results could be explained by a generic convolutional bypass effect rather than by the specific audio-adjacency assumption. Additional ablations and a consistency analysis are needed.
  5. [§3.3, Table 2 and §3.2] The audio-pretrained baselines SSAST and MAE-AST are not re-run under the same protocol; their numbers are taken directly from other papers. This matters because the PEFT experiments use non-overlapping patches while the paper states that full fine-tuning uses patch overlapping, and learning rates, optimizers, and training schedules also differ. A direct comparison therefore conflates the PEFT method with these protocol differences. The authors should either re-run the audio-pretrained baselines under an identical evaluation protocol or clearly show that the protocol differences do not affect the relative ordering.
minor comments (5)
  1. [§2.1] The text says PEFT updates "less than 10% of total parameters," but Table 2 includes a 10% configuration; the wording should be "up to 10%" for consistency.
  2. [Table 2] The column headers label ESC-50 and SPC-2 results as "mAP," but the reported values appear to be top-1 accuracy (e.g., 88.8 for SSAST on ESC-50 matches its published accuracy). The columns should be renamed to avoid misleading readers.
  3. [Eq. (2) and Figure 2] The reshaping of flattened tokens into a two-dimensional time-frequency grid is described only in the implementation details. Defining the tensor layout and the exact convolution operation in Section 2.3 would make the architecture easier to reproduce.
  4. [Figure 3] The attention-map visualization does not state whether attention is averaged over heads, which query token is used, or how the colormap is normalized. Adding these details would make the qualitative claim about "smoother" attention more interpretable.
  5. [References] In §3.3, MAE-AST is described as "based on AST," but the cited paper [21] does not use that name in its title; the description should be made consistent with the reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical adapter study with no derivational claim that reduces to its own inputs.

full rationale

This is an empirical methods paper rather than a formal derivation, so most circularity patterns do not apply. The Look-Aside Adapter is defined by an explicit update rule (Eq. 2) using 1D convolutions as projection layers, and its effectiveness is tested against externally reported baselines such as SSAST, MAE-AST, and AST. The key comparisons in Table 2 use numbers taken directly from the cited papers, which is the opposite of circular reasoning: the central claim that ImageNet-only initialization plus LoAA can 'reach or surpass' audio-pretrained models is evaluated against independent published results. No parameter is fitted to a target quantity and then reported as a prediction of that same quantity; no equation defines the output in terms of the input in a way that collapses the claimed result; and there is no load-bearing self-citation chain. The paper does select its highlighted 2% configuration, Attn(T), based on the EPIC-SOUNDS results shown in Table 1 and Table 3, which is a model-selection bias concern rather than a circularity concern, because the winning configuration is not derived from the benchmark metric by construction and the comparison to SSAST remains externally anchored. Likewise, the claim about time/frequency kernel orientation being beneficial is supported by ablations rather than by assuming the conclusion. The absence of error bars and the small 0.77-point margin over SSAST on EPIC-SOUNDS are experimental robustness issues, not circularity issues. Accordingly, the appropriate finding is no significant circularity, with score 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on domain assumptions about transferring ImageNet features to audio and on the design assumption that 1D look-aside convolutions provide the right inductive bias. These are reasonable but not proven; the paper offers one attention map as supporting evidence.

free parameters (5)
  • Bottleneck dimension r = 36 (2%), 72 (5%/10%), 108 (parallel adapter 5%)
    Sets the parameter budget; results in Table 1 vary with r.
  • Kernel shape = (1,3) time or (3,1) frequency; (1,1) and (3,3) ablated in Table 1
    Core design choice of LoAA; the best orientation was selected using EPIC-SOUNDS.
  • Adapter placement = Attn(T) + FFN(F) for 5%/10% configurations
    Selected from the grid in Table 3 using EPIC-SOUNDS results.
  • PEFT learning rate = 5e-4 (out of 5e-5, 1e-4, 5e-4)
    The paper states 'we choose the learning rate 5e-4' without specifying whether validation or test was used for selection.
  • Mixup usage = Used only on Speech Commands V2
    Training augmentation chosen per dataset; affects SPC-2 results.
assumptions (4)
  • domain assumption ImageNet-pretrained weights transfer to audio spectrogram classification
    Section 2.2 relies on prior work for this premise; the paper does not itself compare randomly initialized versus ImageNet-initialized models.
  • domain assumption Mel-spectrograms can be treated as 2D images with time and frequency axes
    This is the basis for using AST and for reshaping tokens into a 2D grid in LoAA (Section 2.3).
  • domain assumption Parallel adapter placement is superior to sequential in transformers
    Section 4.1 cites prior work and applies this to audio without re-ablation.
  • ad hoc to paper 1x3 and 3x1 convolution kernels on neighboring time/frequency tokens provide the relevant inductive bias for audio
    This is the core mechanism of LoAA, supported only by Table 1 and one qualitative attention map (Figure 3).

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Vision Models Meet Parameter Efficient Look-Aside Adapters Without Large-Scale Audio Pretraining." pith.science (2026). https://pith.science/paper/OUWPIWLB

@misc{pith2026241205951,
  author       = {Pith},
  title        = {Pith review of: When Vision Models Meet Parameter Efficient Look-Aside Adapters Without Large-Scale Audio Pretraining},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OUWPIWLB}},
  note         = {Machine review of arXiv:2412.05951}
}
read the original abstract

Recent studies show that pretrained vision models can boost performance in audio downstream tasks. To enhance the performance further, an additional pretraining stage with large scale audio data is typically required to infuse audio specific knowledge into the vision model. However, such approaches require extensive audio data and a carefully designed objective function. In this work, we propose bypassing the pretraining stage by directly fine-tuning the vision model with our Look Aside Adapter (LoAA) designed for efficient audio understanding. Audio spectrum data is represented across two heterogeneous dimensions time and frequency and we refine adapters to facilitate interactions between tokens across these dimensions. Our experiments demonstrate that our adapters allow vision models to reach or surpass the performance of pretrained audio models in various audio and speech tasks, offering a resource efficient and effective solution for leveraging vision models in audio applications.

Figures

Figures reproduced from arXiv: 2412.05951 by the authors.

Figure 1
Figure 1. An illustration of our simplified approach for audio classification. Our newly proposed Parameter Efficient Fine￾Tuning (PEFT) paradigm for audio classification is a direct adaptation to downstream tasks in a singular stage. This ap￾proach even outperforms the current paradigm, which involves pretraining with large-scale audio datasets such as AudioSet￾2M, as evidenced by its performance on the EPIC-SOUNDS dataset. … view at source ↗
Figure 2
Figure 2. Graphical illustration of adapter module pipelines of conventional parallel adapter (left) and our look-aside adapter (right). d: input token dimension; r: bottleneck dimension; T: time sequence of each token. 2. PEFT with Look-Aside Adapter 2.1. Parameter-Efficient Fine-Tuning (PEFT) Parameter-efficient fine-tuning (PEFT) aims to adapt a pre￾trained model to unseen domains and downstream tasks by up￾dating only a s… view at source ↗
Figure 3
Figure 3. Attention maps of bell sound produced by AST-based models trained in different ways. The audio data is obtained from AudioSet and represented by 128 dimensions of frequency and 1024 timesteps. We examine the self-attention of tokens from the last layer, and display the results for the key timesteps of 320-336. Each token corresponds to 16 × 16 area of the audio spectrogram and the attention maps are made up of 8×64 … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 23 canonical work pages

  1. [1]

    When Vision Models Meet Parameter Efficient Look-Aside Adapters Without Large-Scale Audio Pretraining

    Introduction In the era of transformers [1], self-supervised learning [2, 3] is revolutionizing all domains, including computer vision (CV) and natural language processing (NLP). The paradigm of pre- training followed by fine-tuning has been widely embraced. However, compared to CV and NLP, the audio domain often encounters the challenge of relatively sma...

    work page Pith review arXiv 2024

  2. [2]

    PEFT with Look-Aside Adapter 2.1. Parameter-Efficient Fine-Tuning (PEFT) Parameter-efficient fine-tuning (PEFT) aims to adapt a pre- trained model to unseen domains and downstream tasks by up- dating only a small number of parameters. For example, adapter tuning [15, 16] inserts small trainable modules (adapters) to each layer of a frozen pretrained model...

    work page

  3. [3]

    Experiments 3.1. Datasets and Tasks We evaluate the Look-Aside Adapter on three commonly used audio and speech benchmarks, including audio classification on EPIC-SOUNDS, Environmental Sound Classification (ESC), and speech classification on Speech Commands. • Pretrain Data (PT-Data), including ImageNet [6] for vision and AudioSet-2M [4] for audio, is used...

    work page

  4. [4]

    Discussions 4.1. What is the optimal combination of Look-Aside Adapter modules for a transformer-based model? Recent works [24, 25] empirically showed that parallel insertion is superior to conventional sequential insertion. While those works mix parallel adapters with other PEFT methods, such as LoRA and prefix tuning, to achieve high performance, our me...

    work page

  5. [5]

    However, it is challenging due to the need of substantial audio data and a well-designed learning objective for large-scale audio pretraining

    Conclusion The conventional two-stage paradigm, which starts with pre- trained vision models, leverages large-scale audio pretraining followed by task-specific fine-tuning to learn the audio-specific knowledge for downstream tasks. However, it is challenging due to the need of substantial audio data and a well-designed learning objective for large-scale a...

    work page

  6. [6]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017

    2017

  7. [7]

    Emerging properties in self-supervised vision transformers,

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bo- janowski, and A. Joulin, “Emerging properties in self-supervised vision transformers,” in Proceedings of the IEEE/CVF interna- tional conference on computer vision, 2021, pp. 9650–9660

    work page 2021

  8. [8]

    Exploring simple siamese representation learning,

    X. Chen and K. He, “Exploring simple siamese representation learning,” in Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition, 2021, pp. 15 750–15 758

    work page 2021

Show all 30 references
  1. [9]

    Audio set: An ontology and human-labeled dataset for audio events,

    J. F. Gemmeke, D. P. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter, “Audio set: An ontology and human-labeled dataset for audio events,” in 2017 IEEE inter- national conference on acoustics, speech and signal processing (ICASSP). IEEE, 2017,...

    2017

  2. [10]

    Epic- sounds: A large-scale dataset of actions that sound,

    J. Huh, J. Chalk, E. Kazakos, D. Damen, and A. Zisserman, “Epic- sounds: A large-scale dataset of actions that sound,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  3. [11]

    Im- agenet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Im- agenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition . Ieee, 2009, pp. 248–255

    2009

  4. [12]

    Rethinking cnn mod- els for audio classification,

    K. Palanisamy, D. Singhania, and A. Yao, “Rethinking cnn mod- els for audio classification,” arXiv preprint arXiv:2007.11154 , 2020

    2007 arXiv

  5. [13]

    AST: Audio Spectrogram Transformer,

    Y . Gong, Y . A. Chung, and J. Glass, “AST: Audio Spectrogram Transformer,” in Proceedings of Interspeech, 2021, pp. 571–575

    2021

  6. [14]

    PANNs: Large-scale pretrained audio neural networks for audio pattern recognition,

    Q. Kong, Y . Cao, T. Iqbal, Y . Wang, W. Wang, and M. D. Plumb- ley, “PANNs: Large-scale pretrained audio neural networks for audio pattern recognition,” IEEE/ACM TASLP, vol. 28, pp. 2880– 2894, 2020

    2020

  7. [15]

    SSAST: Self- Supervised Audio Spectrogram Transformer,

    Y . Gong, C. I. J. Lai, Y . A. Chung, and J. Glass, “SSAST: Self- Supervised Audio Spectrogram Transformer,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 36, no. 10, 2022, pp. 10 699–10 709

    2022

  8. [16]

    Masked autoencoders that lis- ten,

    P. Y . Huang, H. Xu, J. Li, A. Baevski, M. Auli, W. Galuba, F. Metze, and C. Feichtenhofer, “Masked autoencoders that lis- ten,” in NeurIPS, 2022

    2022

  9. [17]

    Convolutional bypasses are better vision transformer adapters,

    S. Jie and Z. H. Deng, “Convolutional bypasses are better vision transformer adapters,” arXiv preprint arXiv:2207.07039, 2022

    2022 arXiv

  10. [18]

    ESC: Dataset for environmental sound classifica- tion,

    K. J. Piczak, “ESC: Dataset for environmental sound classifica- tion,” in Proceedings of the 23rd ACM international conference on Multimedia, 2015, pp. 1015–1018

    2015

  11. [19]

    Speech commands: A dataset for limited-vocabulary speech recognition,

    P. Warden, “Speech commands: A dataset for limited-vocabulary speech recognition,” arXiv preprint arXiv:1804.03209, 2018

    2018 arXiv

  12. [20]

    Learning multiple visual domains with residual adapters,

    S. A. Rebuffi, H. Bilen, and A. Vedaldi, “Learning multiple visual domains with residual adapters,” Advances in neural information processing systems, vol. 30, 2017

    2017

  13. [21]

    Parameter-efficient transfer learning for NLP,

    N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for NLP,” in International conference on machine learning. PMLR, 2019, pp. 2790–2799

    2019

  14. [22]

    LoRA: Low-rank adaptation of large lan- guage models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large lan- guage models,” in International Conference on Learning Repre- sentations, 2022

    2022

  15. [23]

    The Power of Scale for Parameter-Efficient Prompt Tuning,

    B. Lester, R. Al-Rfou, and N. Constant, “The Power of Scale for Parameter-Efficient Prompt Tuning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Process- ing. Association for Computational Linguistics, Nov 2021, pp. 3045–3059

    2021

  16. [24]

    Prefix-Tuning: Optimizing Continuous Prompts for Generation,

    X. L. Li and P. Liang, “Prefix-Tuning: Optimizing Continuous Prompts for Generation,” inProceedings of the 59th Annual Meet- ing of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Process- ing(Volume 1: Long Papers)...

    2021

  17. [25]

    Contrastive audio-visual masked autoencoder,

    Y . Gong, A. Rouditchenko, A. H. Liu, D. Harwath, L. Karlinsky, H. Kuehne, and J. R. Glass, “Contrastive audio-visual masked autoencoder,” in The Eleventh International Conference on Learning Representations , 2023. [Online]. Available: https: //openreview.net/forum?id=QPtMRyk5rb

    2023

  18. [26]

    MAE- AST: Masked Autoencoding Audio Spectrogram Trans- former,

    A. Baade, P. Peng, and D. F. Harwath, “MAE- AST: Masked Autoencoding Audio Spectrogram Trans- former,” in Interspeech, 2022. [Online]. Available: https: //api.semanticscholar.org/CorpusID:247839481

    2022

  19. [27]

    Masked spectrogram prediction for self-supervised audio pre-training,

    D. Chong, H. Wang, P. Zhou, and Q. Zeng, “Masked spectrogram prediction for self-supervised audio pre-training,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  20. [28]

    Lib- rispeech: An asr corpus based on public domain audio books,

    V . Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Lib- rispeech: An asr corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 5206–5210

    2015

  21. [29]

    To- wards a unified view of parameter-efficient transfer learning,

    J. He, C. Zhou, X. Ma, T. B. Kirkpatrick, and G. Neubig, “To- wards a unified view of parameter-efficient transfer learning,” in International Conference on Learning Representations, 2022

    2022

  22. [30]

    PEFT for Speech: Unveiling Optimal Placement, Merging Strategies, and Ensemble Techniques,

    T. H. Lin, H. S. Wang, H. Y . Weng, K. C. Peng, Z. C. Chen, and H. y. Lee, “PEFT for Speech: Unveiling Optimal Placement, Merging Strategies, and Ensemble Techniques,” arXiv preprint arXiv:2401.02122, 2024

    2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.