Pith. sign in

REVIEW 3 major objections 6 minor 5 cited by

Audio-JEPA: Joint-Embedding Predictive Architecture for Audio Representation Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Audio-JEPA shows that predicting masked spectrogram patch embeddings in latent space yields audio representations competitive with wav2vec 2.0 and data2vec while training on less than one-fifth of their data.

desk verdict A clean, honest reimplementation of A-JEPA with a useful open evaluation, but the headline data-efficiency claim is domain-conflated and overstates the evidence. read the letter →

arxiv 2507.02915 v1 pith:A2ZU5K2T submitted 2025-06-25 cs.SD cs.AIcs.LGeess.ASeess.SP

Ludovic Tuncay (IRIT-SAMoVA) , Etienne Labbé (IRIT-SAMoVA) , Emmanouil Benetos (QMUL) , Thomas Pellegrini (IRIT-SAMoVA) This is my paper
classification cs.SDcs.AIcs.LGeess.ASeess.SP
keywords self-supervisedlearningaudiorepresentationjoint-embeddingpredictivearchitecturemaskedspectrogrampredictionJEPASetX-ARESbenchmarklatent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a self-supervised audio model can learn broadly useful representations without reconstructing the input signal: instead, it predicts the latent embeddings of randomly masked spectrogram patches. The authors adapt the vision JEPA recipe to audio by treating a mel-spectrogram as an image, and they pre-train on unlabeled AudioSet clips. Their main claim is that this straightforward translation matches or beats wav2vec 2.0 and data2vec on a 21-task audio benchmark suite while seeing less than one-fifth of the training audio those models saw, and with no hyper-parameter tuning. The result matters because it suggests that high-level latent prediction is a data-efficient alternative to contrastive or reconstruction-based audio pretraining.

What carries the argument

The central mechanism is masked latent prediction in a joint-embedding architecture. A context encoder produces embeddings for visible spectrogram patches; a target encoder, updated by exponential moving average of the context encoder's weights, produces targets for the masked patches; and a lightweight predictor maps context outputs to the masked patch positions. Minimizing the average squared L2 distance between predicted and target embeddings, with a stop-gradient on the target side, pushes the model to capture high-level structure instead of reconstructing waveforms or spectrogram pixels. Random patch masking rather than block masking, and the EMA target stabilisation, are the design choices that carry the claim.

What would settle it

Train Audio-JEPA on LibriSpeech under the same recipe and evaluate on the X-ARES speech tasks; if its kNN speech scores still trail wav2vec 2.0 and data2vec substantially, the claim that latent prediction alone explains the result is weakened. Alternatively, fine-tune wav2vec 2.0 and data2vec on AudioSet for the same wall-clock budget and check whether their kNN advantage on speech tasks persists.

Watch

Extended reading notes

Core claim

Audio-JEPA is a direct port of I-JEPA to audio, using a Vision Transformer context encoder that sees only unmasked mel-spectrogram patches, a momentum-updated target encoder that sees the full spectrogram, and a lightweight six-layer predictor that regresses target embeddings of masked patches under an L2 loss. With random masking of 40-60% of patches, 100,000 training steps on AudioSet, and no task-specific tuning, the frozen target encoder's embeddings rank first or second on several music and environmental-sound benchmarks under k-nearest-neighbor evaluation, and the authors report overall performance comparable to wav2vec 2.0 and data2vec at roughly a fifth of their pre-training data. The paper also reports that the embedding space is not reliably linearly separable, so linear-probe scores lag on speech-heavy tasks; it interprets the kNN results as the truer measure of representation quality.

Load-bearing premise

The claim of comparable performance with less training data assumes that comparing frozen-encoder scores across wav2vec 2.0 and data2vec, pre-trained on speech-only LibriSpeech, with Audio-JEPA pre-trained on general AudioSet is a fair apples-to-apples test of the pretraining recipe.

Editorial extensions

If this is right

  • If the result holds, audio self-supervised learning does not need contrastive objectives, discrete targets, or input reconstruction to reach competitive general-purpose representations.
  • Data-efficient pretraining on unlabeled general audio, rather than large matched speech corpora, becomes a viable route for music and environmental-sound tasks.
  • The reported linear-probe gap suggests evaluation protocols should include non-parametric probes such as kNN when comparing JEPA-style models with contrastive baselines.
  • Attention pooling over the frozen encoder's outputs, as used in V-JEPA, is a concrete, testable fix for the linear-probe weakness.
  • Open-sourcing the code and checkpoints makes Audio-JEPA a reproducible baseline for subsequent audio JEPA variants.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because AudioSet is a general audio corpus while the baselines were trained on LibriSpeech, part of Audio-JEPA's edge on music and environmental sounds may reflect domain match rather than the JEPA objective itself; a cross-training experiment would separate the two.
  • The same architecture could be tested on raw waveforms with a one-dimensional patchification, or on larger mask ratios, to see whether the latent-prediction benefit transfers beyond spectrogram patch inputs.
  • One could combine Audio-JEPA's latent prediction with a small amount of speech data and re-measure X-ARES speech tasks; if speech scores recover without hurting music scores, the recipe becomes a single general-purpose audio pretraining default.
  • The paper's kNN-vs-linear-probe asymmetry predicts that any JEPA-style audio model will underperform on linear probes, so benchmark suites that rely on linear probes may systematically undervalue this family.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents Audio-JEPA, a direct adaptation of the I-JEPA masked latent-prediction framework to audio. The model converts 10-second AudioSet clips to 128-band mel-spectrograms, randomly masks 40–60% of 16×16 patches, and trains a ViT context encoder plus a lightweight predictor against targets produced by an EMA-updated target encoder, using an L2 loss in latent space. The authors pre-train for 100k steps (~13 epochs over 5,338 hours of AudioSet) and evaluate frozen embeddings on the X-ARES suite with linear probing and kNN classification. The headline claim is that Audio-JEPA achieves comparable performance to wav2vec 2.0 and data2vec while using less than one-fifth of their training data and with no hyper-parameter tuning. The paper also releases code and checkpoints.

Significance. If the data-efficiency claim were established, the paper would provide evidence that joint-embedding predictive architectures are a viable and efficient recipe for general-purpose audio representation learning. The work has clear strengths: it is an honest, from-scratch implementation; it avoids training on any evaluation labels; it reports two complementary evaluation protocols; it openly discusses the linear-probe limitation and suggests concrete remedies; and it commits to releasing code and checkpoints. The central contribution is, however, not yet secured: the comparison against wav2vec 2.0 and data2vec confounds the pretraining objective with the pretraining domain, and the evidence for 'comparable performance' is concentrated in kNN evaluation, where the domain-overlap pattern is strongest. The authors should be credited for acknowledging the linear-probe gap, but the abstract-level claim needs either a matched-corpus experiment or substantially more cautious wording.

major comments (3)
  1. [Abstract; Section V; Tables II–III] The claim of 'comparable performance ... using less than one-fifth of their training data' is not supported as stated because the comparison confounds the JEPA objective with the pretraining domain. Audio-JEPA is trained on 1,921,982 AudioSet clips totaling 5,338 unique hours, whereas wav2vec 2.0 and data2vec are trained on LibriSpeech, which contains 960 unique hours. The 'less than one-fifth' figure in Section V compares hours seen during training (100k steps × 42.7 minutes versus 400k steps × 1.6 h or 63 minutes), not unique data. By unique hours, Audio-JEPA actually uses about 5.5× more data than the LibriSpeech-trained baselines. Moreover, the per-task pattern in Tables II and III—strong performance on ESC-50, FMA, GTZAN, and UrbanSound8K and weak performance on speech tasks such as Fluent Speech Commands, Speech Commands V1, VoxCeleb1, and VoxLingua33—is exactly what one would predict from AudioSet containing music and environmental sounds while LibriSpeech contains clean speech. The aggregate 'comparable' result therefore does not demonstrate that latent prediction is inherently more data-efficient. A matched-corpus pretraining comparison, or at minimum a restricted speech-only comparison, is needed to make the efficiency claim load-bearing.
  2. [Section VI.A, Table II] The 'comparable performance' claim is only supported under kNN evaluation. Linear-probing results in Table II show Audio-JEPA averaging roughly 0.353 across the 20 reported datasets, versus about 0.494 for wav2vec 2.0 and 0.444 for data2vec; Audio-JEPA ranks last on approximately half of the tasks. No standard deviations or confidence intervals are reported for any number in Tables II or III, and the paper does not state whether these are single runs. In addition, LibriSpeech-100h is excluded from Table II without explanation, despite being part of the 21-dataset X-ARES suite; this omission weakens the generalization claim. The authors should report aggregate scores, include uncertainty estimates or a statement that results are single runs, and disclose the reason for the LibriSpeech-100h exclusion.
  3. [Section IV.A and abstract] The claim of 'no hyper-parameter tuning' is not fully accurate. Section IV.A reports that preliminary experiments showed block masking to yield lower performance than random masking, meaning a design choice was made on the basis of experiments. Furthermore, the mask ratio is not a single tuned value but is uniformly sampled from 40%–60% per batch. The authors should clarify which hyperparameters were truly inherited from I-JEPA without any selection and which were chosen after experimentation, so that the 'no tuning' statement has a precise meaning.
minor comments (6)
  1. [Section III.A and Fig. 1] The text says the target encoder 'encodes the true masked patches' while Sections II.C and III describe the JEPA target as processing the entire spectrogram (context plus masked). Equation (1), t_j = f_tgt(x)_j, indicates the target encoder sees the full input; please align the wording in III.A with the actual formulation.
  2. [Section IV.A] The description 'frame size is 2.5 times the size of the hop' does not specify the actual window length, hop length, or FFT size; please provide exact values so the mel-spectrogram configuration is reproducible.
  3. [References] Several references (e.g., [5], [7], [10], [12], [21]) are given as arXiv URLs without year, venue, or version information; please complete these entries.
  4. [Tables II and III] The captions say scores are 'as given by X-ARES,' but the exact evaluation configuration (e.g., number of seeds, downstream train/val splits, probe hyperparameters) is not described; please specify whether the reported numbers are from the X-ARES repository or from the authors' own runs.
  5. [Section VI.B] The kNN evaluation is described as '16 X-ARES tasks compatible with this probe,' but the criteria for compatibility are not stated; please explain why the other five tasks are excluded.
  6. [Section VII] The conclusion states Audio-JEPA is the 'first open-source, from-scratch adaptation' of JEPA to audio. Given that M2D is discussed as a closely related open-source dual-network masked-prediction method, please either compare against M2D or qualify the novelty claim to avoid ambiguity.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Audio-JEPA's pretraining loss and downstream X-ARES evaluation are independent, and the data-efficiency comparison raises a domain-fairness concern rather than a circular reduction.

full rationale

The paper's derivation chain is self-contained with respect to circularity. Audio-JEPA is explicitly a direct adaptation of I-JEPA to audio, with hyperparameters taken from I-JEPA and BYOL (Section IV), and no parameter is fitted to the X-ARES evaluation suite. The training objective in Eq. (2) minimizes L2 distance between predictor outputs and target-encoder embeddings, where the target encoder is an EMA of the context encoder; this is a self-supervised pretraining objective, not a prediction of the downstream labels. The evaluation in Sections VI.A and VI.B uses frozen encoders probed by external X-ARES linear and kNN classifiers, so the reported benchmark scores are not determined by construction from the training loss. The paper also openly discloses that linear-probe results are expected to be weak due to the JEPA objective's non-linear embedding geometry, citing V-JEPA. The only notable concern is the abstract's claim of 'less than one-fifth of their training data': the comparison counts audio hours seen during training while pretraining on different corpora (AudioSet vs. LibriSpeech), which is a domain-matching and evaluation-fairness issue, not a circularity. No load-bearing self-citations appear; references to A-JEPA, I-JEPA, V-JEPA, and BYOL are external prior work, and the present authors' contributions are implementation and evaluation, not a uniqueness theorem or fitted input renamed as prediction. Therefore no circular step is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No parameters are fitted to the benchmark; all hyperparameters come from prior work. The load-bearing assumptions are domain choices: mel-spectrogram input, EMA stabilization, transferability of hyperparameters, and the fairness of the X-ARES protocol across models trained on different data.

assumptions (4)
  • domain assumption Mel-spectrogram patches are a suitable input representation for JEPA-style latent prediction on audio
    The whole method operates on 128-band mel-spectrograms rather than waveforms; no ablation compares input representations.
  • domain assumption EMA-updated target encoder provides stable, non-collapsing targets with stop-gradient
    Section III.B: targets come from an EMA of the context encoder; collapse prevention is asserted but not demonstrated.
  • domain assumption Hyperparameters transferred from I-JEPA/BYOL are appropriate for audio without tuning
    Section IV.C: optimizer, schedule, mask ratio 40-60% and EMA decay follow I-JEPA/BYOL; the paper claims no tuning, but preliminary experiments on block vs random masking were run.
  • domain assumption X-ARES evaluation protocol is a valid measure of representation quality
    Section III.C: frozen target encoder features are probed with MLP and kNN; results are interpreted as intrinsic quality, but the target encoder is not the typical inference encoder.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Audio-JEPA: Joint-Embedding Predictive Architecture for Audio Representation Learning." pith.science (2026). https://pith.science/paper/A2ZU5K2T

@misc{pith2026250702915,
  author       = {Pith},
  title        = {Pith review of: Audio-JEPA: Joint-Embedding Predictive Architecture for Audio Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A2ZU5K2T}},
  note         = {Machine review of arXiv:2507.02915}
}
read the original abstract

Building on the Joint-Embedding Predictive Architecture (JEPA) paradigm, a recent self-supervised learning framework that predicts latent representations of masked regions in high-level feature spaces, we propose Audio-JEPA (Audio Joint-Embedding Predictive Architecture), tailored specifically for audio data. Audio-JEPA uses a simple Vision Transformer backbone to predict latent representations of masked spectrogram patches rather than reconstructing raw audio. We pre-train on unlabeled AudioSet clips (10s, 32kHz) with random patch masking on mel-spectrograms. We evaluate on the X-ARES suite covering speech, music, and environmental sound tasks. Although our implementation is a straightforward translation of the original model to audio, the results still show comparable performance to wav2vec 2.0 and data2vec while using less than one-fifth of their training data and with no hyper-parameter tuning. All code and pretrained checkpoints will be released on GitHub.

Figures

Figures reproduced from arXiv: 2507.02915 by the authors.

Figure 1
Figure 1. A‑JEPA architecture. Mel‑spectrogram patches are split into visible and masked sets. A context encoder embeds visible patches, a lightweight predictor reconstructs masked‑patch embeddings, and a momentum‑updated target encoder provides targets. Training minimizes average L2 (Euclidean) distance. The dashed arrow denotes a stop‑gradient. HuBERT uses an offline k-means on acoustic features to provide target labels, an… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Bar-JEPA: Extracting Values from Bar Chart with Joint-Embedding Predictive Architecture

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A JEPA encoder finetuned on synthetic bar charts enables a lightweight decoder to recover bar values from chart images, but the method remains behind state-of-the-art supervised systems.

  2. FactorJEPA: Factorizing Monolithic Futures into Layout-Agent-Interaction Channels for Crowded and Chaotic Global South Urban Worlds

    cs.AI 2026-08 conditional novelty 6.0 of 10

    FactorJEPA splits a video prediction model into layout, agent, and interaction channels with a visibility gate, and a new DENSEWORLD dataset tests it on crowded Indian city scenes.

  3. Music-JEPA: Learning a World Model of Sound from Action

    cs.SD 2026-07 conditional novelty 6.0 of 10

    An action-conditioned JEPA trained on paired piano audio and MIDI learns latent sound dynamics that support MIR tasks and transcription-style planning.

  4. Joint-Embedding Predictive Architecture for Sensor-based Activity Recognition

    eess.SP 2026-07 conditional novelty 6.0 of 10

    JEPA-based self-supervised pre-training on inertial sensor data improves recognition of rare transitional human activities compared to supervised learning, with gains mostly on transition classes.

  5. Self-Distillation of Hidden Layers for Self-Supervised Representation Learning

    cs.CV 2026-03 conditional novelty 6.0 of 10

    Predicting the outputs of several hidden layers of an EMA teacher, rather than only the final layer or pixels, substantially improves self-supervised ViT representations on ImageNet and downstream tasks.

Reference graph

Works this paper leans on

42 extracted references · 21 canonical work pages · cited by 5 Pith papers

  1. [1]

    HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units

    W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units.” [Online]. Available: http://arxiv.org/abs/2106.07447

    arXiv

  2. [2]

    CED: Consistent ensemble distillation for audio tagging

    H. Dinkel, Y. Wang, Z. Yan, J. Zhang, and Y. Wang, “CED: Consistent ensemble distillation for audio tagging.” [Online]. Available: http:// arxiv.org/abs/2308.11957

    work page Pith review arXiv

  3. [3]

    Scaling up masked audio encoder learning for general audio classifi- cation

    H. Dinkel, Z. Yan, Y. Wang, J. Zhang, Y. Wang, and B. Wang, “Scaling up masked audio encoder learning for general audio classifi- cation.” [Online]. Available: http://arxiv.org/abs/2406.06992

    arXiv

  4. [4]

    Masked Modeling Duo: Towards a Universal Audio Pre-training Framework

    D. Niizumi, D. Takeuchi, Y. Ohishi, N. Harada, and K. Kashino, “Masked Modeling Duo: Towards a Universal Audio Pre-training Framework.” [Online]. Available: http://arxiv.org/abs/2404.06095

    work page Pith review arXiv

  5. [5]

    wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representa- tions

    A. Baevski, H. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representa- tions.” [Online]. Available: http://arxiv.org/abs/2006.11477

    arXiv 2006

  6. [6]

    WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing,

    S. Chen et al. , “WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing,” IEEE Journal of Selected Topics in Signal Processing , vol. 16, no. 6, pp. 1505–1518, Oct. 2022, doi: 10.1109/JSTSP.2022.3188113

    arXiv 2022

  7. [7]

    data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language

    A. Baevski, W.-N. Hsu, Q. Xu, A. Babu, J. Gu, and M. Auli, “data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language.” [Online]. Available: http://arxiv.org/abs/2202.03555

    arXiv

  8. [8]

    Efficient Self-super- vised Learning with Contextualized Target Representations for Vision, Speech and Language

    A. Baevski, A. Babu, W.-N. Hsu, and M. Auli, “Efficient Self-super- vised Learning with Contextualized Target Representations for Vision, Speech and Language.” [Online]. Available: http://arxiv.org/abs/2212. 07525

    work page

Show all 42 references
  1. [9]

    A Path Towards Autonomous Machine Intelligence Version 0.9.2, 2022-06-27,

    Y. LeCun, “A Path Towards Autonomous Machine Intelligence Version 0.9.2, 2022-06-27,” Jun. 2022

    2022

  2. [10]

    Self-Supervised Learning from Images with a Joint- Embedding Predictive Architecture

    M. Assran et al., “Self-Supervised Learning from Images with a Joint- Embedding Predictive Architecture.” [Online]. Available: http://arxiv. org/abs/2301.08243

    arXiv

  3. [11]

    Revisiting Feature Prediction for Learning Visual Representations from Video

    A. Bardes et al. , “Revisiting Feature Prediction for Learning Visual Representations from Video.” [Online]. Available: http://arxiv.org/abs/ 2404.08471

    arXiv

  4. [12]

    A-JEPA: Joint-Embedding Predictive Architecture Can Listen

    Z. Fei, M. Fan, and J. Huang, “A-JEPA: Joint-Embedding Predictive Architecture Can Listen.” [Online]. Available: http://arxiv.org/abs/2311. 15830

  5. [13]

    Stem-JEPA: A Joint-Embedding Predictive Architecture for Musical Stem Compatibility Estimation

    A. Riou, S. Lattner, G. Hadjeres, M. Anslow, and G. Peeters, “Stem-JEPA: A Joint-Embedding Predictive Architecture for Musical Stem Compatibility Estimation.” [Online]. Available: http://arxiv.org/ abs/2408.02514

    arXiv

  6. [14]

    Masked Autoencoders that Listen

    P.-Y. Huang et al., “Masked Autoencoders that Listen.” [Online]. Avail- able: http://arxiv.org/abs/2207.06405

    arXiv

  7. [15]

    A Dataset and Taxonomy for Urban Sound Research,

    J. Salamon, C. Jacoby, and J. P. Bello, “A Dataset and Taxonomy for Urban Sound Research,” in Proceedings of the 22nd ACM interna- tional conference on Multimedia , in MM '14. New York, NY, USA: Association for Computing Machinery, Nov. 2014, pp. 1041–1044. doi: 10.1145/264786...

    2014

  8. [16]

    VoxCeleb: a large-scale speaker identification dataset,

    A. Nagrani, J. S. Chung, and A. Zisserman, “VoxCeleb: a large-scale speaker identification dataset,” in Interspeech 2017 , Aug. 2017, pp. 2616–2620. doi: 10.21437/Interspeech.2017-950

    2017 doi

  9. [17]

    The GTZAN dataset: Its contents, its faults, their effects on evaluation, and its future use,

    B. L. Sturm, “The GTZAN dataset: Its contents, its faults, their effects on evaluation, and its future use,” Journal of New Music Research , vol. 43, no. 2, pp. 147–172, Apr. 2014, doi: 10.1080/09298215.2014.894533

    2014

  10. [18]

    Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition

    P. Warden, “Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition.” [Online]. Available: http://arxiv.org/abs/1804. 03209

  11. [19]

    TI- JEPA: An Innovative Energy-based Joint Embedding Strategy for Text- Image Multimodal Systems

    K. H. N. Vo, D. P. T. Nguyen, T. Nguyen, and T. T. Quan, “TI- JEPA: An Innovative Energy-based Joint Embedding Strategy for Text- Image Multimodal Systems.” [Online]. Available: http://arxiv.org/abs/ 2503.06380

    arXiv

  12. [20]

    GeoJEPA: Towards Eliminating Augmen- tation- and Sampling Bias in Multimodal Geospatial Learning

    T. Lundqvist and L. Delvret, “GeoJEPA: Towards Eliminating Augmen- tation- and Sampling Bias in Multimodal Geospatial Learning.” [On- line]. Available: http://arxiv.org/abs/2503.05774

    arXiv

  13. [21]

    Bootstrap your own latent: A new approach to self- supervised Learning

    J.-B. Grill et al., “Bootstrap your own latent: A new approach to self- supervised Learning.” [Online]. Available: http://arxiv.org/abs/2006. 07733

    2006

  14. [22]

    Decoupled Weight Decay Regulariza- tion

    I. Loshchilov and F. Hutter, “Decoupled Weight Decay Regulariza- tion.” [Online]. Available: http://arxiv.org/abs/1711.05101

    arXiv

  15. [23]

    Z. Wu, T. Kinnunen, N. Evans, and J. Yamagishi, “Automatic Speaker Verification Spoofing and Countermeasures Challenge (ASVspoof

  16. [24]

    Clotho: An Audio Captioning Dataset

    K. Drossos, S. Lipping, and T. Virtanen, “Clotho: An Audio Captioning Dataset.” [Online]. Available: http://arxiv.org/abs/1910.09387

    1910 arXiv

  17. [25]

    CREMA-D: Crowd-sourced Emotional Multimodal Actors Dataset,

    H. Cao, D. G. Cooper, M. K. Keutmann, R. C. Gur, A. Nenkova, and R. Verma, “CREMA-D: Crowd-sourced Emotional Multimodal Actors Dataset,” IEEE transactions on affective computing , vol. 5, no. 4, pp. 377–390, 2014, doi: 10.1109/TAFFC.2014.2336244

    2014

  18. [26]

    Generating an item pool for translational social cognition research: methodology and initial validation,

    M. K. Keutmann, S. L. Moore, A. Savitt, and R. C. Gur, “Generating an item pool for translational social cognition research: methodology and initial validation,” Behavior Research Methods , vol. 47, no. 1, pp. 228–234, Mar. 2015, doi: 10.3758/s13428-014-0464-0

    2015 doi

  19. [27]

    Sound event detection in domestic environments with weakly labeled data and soundscape synthesis,

    N. Turpault, R. Serizel, A. P. Shah, and J. Salamon, “Sound event detection in domestic environments with weakly labeled data and soundscape synthesis,” presented at the Workshop on Detection and Classification of Acoustic Scenes and Events, 2019. Accessed: May 07, 2025. [Onli...

    2019

  20. [28]

    Sound event detection in synthetic domestic environments,

    R. Serizel, N. Turpault, A. Shah, and J. Salamon, “Sound event detection in synthetic domestic environments,” presented at the ICASSP 2020 - 45th International Conference on Acoustics, Speech, and Signal Processing, May 2020. Accessed: May 07, 2025. [Online]. Available: https:...

    2020

  21. [29]

    ESC: Dataset for Environmental Sound Classification,

    K. J. Piczak, “ESC: Dataset for Environmental Sound Classification,” in Proceedings of the 23rd ACM international conference on Multi- media, in MM '15. New York, NY, USA: Association for Computing Machinery, Oct. 2015, pp. 1015–1018. doi: 10.1145/2733373.2806390

    2015

  22. [30]

    FMA: A Dataset For Music Analysis

    M. Defferrard, K. Benzi, P. Vandergheynst, and X. Bresson, “FMA: A Dataset For Music Analysis.” [Online]. Available: http://arxiv.org/abs/ 1612.01840

    arXiv

  23. [31]

    General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline

    E. Fonseca et al., “General-purpose Tagging of Freesound Audio with AudioSet Labels: Task Description, Dataset, and Baseline.” [Online]. Available: http://arxiv.org/abs/1807.09902

    arXiv

  24. [32]

    FSD50K: An Open Dataset of Human-Labeled Sound Events

    E. Fonseca, X. Favory, J. Pons, F. Font, and X. Serra, “FSD50K: An Open Dataset of Human-Labeled Sound Events.” [Online]. Available: http://arxiv.org/abs/2010.00475

    2010 arXiv

  25. [33]

    LibriCount, a dataset for speaker count estimation

    F.-R. Stöter, S. Chakrabarty, E. Habets, and B. Edler, “LibriCount, a dataset for speaker count estimation.” [Online]. Available: https:// zenodo.org/records/1216072

  26. [34]

    Librispeech: An ASR corpus based on public domain audio books,

    V. Panayotov, G. Chen, D. Povey, and S. Khudanpur, “Librispeech: An ASR corpus based on public domain audio books,” in 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , Apr. 2015, pp. 5206–5210. doi: 10.1109/ ICASSP.2015.7178964

    2015

  27. [35]

    Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders

    J. Engel et al., “Neural Audio Synthesis of Musical Notes with WaveNet Autoencoders.” [Online]. Available: http://arxiv.org/abs/1704.01279

    arXiv

  28. [36]

    The Ryerson Audio-Visual Database of Emotional Speech and Song (RA VDESS)

    S. R. Livingstone and F. A. Russo, “The Ryerson Audio-Visual Database of Emotional Speech and Song (RA VDESS).” [Online]. Available: https://zenodo.org/records/1188976

  29. [37]

    Vocal Imitation Set v1.1.3 : Thousands of vocal imitations of hundreds of sounds from the AudioSet ontology

    “Vocal Imitation Set v1.1.3 : Thousands of vocal imitations of hundreds of sounds from the AudioSet ontology.” [Online]. Available: https:// zenodo.org/records/1340763

  30. [38]

    Vocalsound: A Dataset for Im- proving Human Vocal Sounds Recognition,

    Y. Gong, J. Yu, and J. Glass, “Vocalsound: A Dataset for Im- proving Human Vocal Sounds Recognition,” in ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Sig- nal Processing (ICASSP) , May 2022, pp. 151–155. doi: 10.1109/ ICASSP43922.2022.9746828

    2022

  31. [39]

    voxlingua33 in WebDataset Format

    N. Yadong, “voxlingua33 in WebDataset Format.” [Online]. Available: https://zenodo.org/records/14723799

  32. [40]

    ConvFormer: Plug-and-Play CNN-Style Transformers for Improving Medical Image Segmentation

    X. Lin, Z. Yan, X. Deng, C. Zheng, and L. Yu, “ConvFormer: Plug-and-Play CNN-Style Transformers for Improving Medical Image Segmentation.” [Online]. Available: http://arxiv.org/abs/2309.05674

    arXiv

  33. [41]

    MetaFormer Baselines for Vision

    W. Yu et al., “MetaFormer Baselines for Vision.” [Online]. Available: http://arxiv.org/abs/2210.13452

    arXiv

  34. [2015]

    Available: https://datashare.ed.ac.uk/handle/ 10283/853

    Database.” [Online]. Available: https://datashare.ed.ac.uk/handle/ 10283/853

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.