Pith. sign in

REVIEW 3 major objections 6 minor 39 references

RELATE: Subjective evaluation dataset for automatic evaluation of relevance between text and audio

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper builds a public dataset of human text–audio relevance scores and shows that a supervised predictor trained on it beats the standard CLAPScore baseline.

desk verdict RELATE is a genuinely useful dataset for text-to-audio relevance evaluation, and the benchmark model is reasonable, but the headline advantage over CLAPScore needs uncertainty quantification before it can be trusted. read the letter →

arxiv 2506.23582 v1 pith:LK3HREGZ submitted 2025-06-30 cs.SD eess.AS

Yusuke Kanamori , Yuki Okamoto , Taisei Takano , Shinnosuke Takamichi , Yuki Saito , Hiroshi Saruwatari This is my paper · ORCID
classification cs.SDeess.AS
keywords text-to-audiosubjectiveevaluationrelevancescoreCLAPScoresoundeventsynthesishumanautomaticmetricAudioCaps
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces RELATE, an open dataset of human ratings of how well a text description matches an audio clip, collected for original AudioCaps recordings and for audio synthesized by four text-to-audio generators. It then trains a supervised model on these ratings to predict the average human relevance score for a new text–audio pair, and reports that this model outperforms the standard unsupervised CLAPScore baseline on every correlation metric used. If that result holds, text-to-audio developers can screen large batches of synthesized audio for content mismatch without running costly listening tests, and different studies can compare against one fixed human ground truth. The dataset analysis also identifies where current synthesis tends to fail: animal sounds, longer texts, and texts with temporal ordering all receive lower relevance scores.

What carries the argument

The load-bearing machinery is the dataset plus the predictor trained on it. RELATE is built from AudioCaps captions paired with original audio and with outputs of AudioLDM, AudioLDM2, Tango, and Tango2; each pair is rated on an 11-point scale for overall relevance, inclusion of sound events, and order of sound events. To keep labels reliable, each listening batch contains intentionally mismatched anchor pairs, and listeners whose average anchor rating is too high are excluded from the released labels. The predictor concatenates frozen BYOL-A audio features, RoBERTa text features, and a listener-embedding vector, passes them through a bidirectional LSTM and linear layers, and is trained with a class-balanced weighted sum of clipped mean-squared-error and contrastive losses.

What would settle it

Have a fresh group of listeners rate a random sample of RELATE pairs without anchor-based screening, then retrain the predictor on those unscreened labels; if the advantage over CLAPScore shrinks substantially, or if the new mean scores shift by more than a point on the 11-point scale, the benchmark result depends on the screening assumption.

Watch

Extended reading notes

Core claim

The central claim is that a supervised regressor trained on RELATE predicts human judgments of text–audio relevance better than CLAPScore, the established reference that computes cosine similarity in a pretrained audio–text embedding space. On the held-out test set the proposed model reaches a Spearman rank correlation of 0.383 with human scores, compared with 0.351 for LAION-CLAP and 0.181 for MS-CLAP, and it also improves linear correlation, Kendall's tau, and mean squared error. The improvement persists in most of the eight top-level sound categories, with the stated exceptions of animal sounds and source-ambiguous sounds, where LAION-CLAP keeps a small advantage. The paper further claims that the released data, with 5,460 text–audio pairs and more than 17,000 relevance ratings, is a reusable benchmark for automatic text-to-audio evaluation.

Load-bearing premise

The dataset is only as trustworthy as the assumption that the intentionally mismatched anchor pairs are genuinely low in relevance and that the listener-exclusion thresholds remove unreliable raters, an assumption the paper does not test with inter-annotator agreement.

Editorial extensions

If this is right

  • A deployed version of the trained model can rank unseen text–audio pairs by predicted human relevance, making it practical to screen large synthesized corpora before human listening.
  • Researchers can use RELATE as a fixed ground-truth benchmark, so relevance scores reported by different text-to-audio systems become comparable across studies.
  • The category-level results imply that the supervised approach captures human judgment beyond a global bias, since its advantage over CLAPScore appears within most sound categories.
  • The released inclusion and order scores, collected but not yet modeled, provide a ready target for future work on finer-grained text-to-audio evaluation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper does not pursue: the same anchor-screening and supervised-prediction recipe could be transferred to other generative domains, such as text-to-music or text-to-video, where a cheap proxy for human relevance is equally needed.
  • The per-category correlation tables suggest that a single global metric can mask uneven performance; reporting category-level correlations, as this paper does, is likely to become the norm for TTA evaluation.
  • Because the model includes a listener embedding, a natural next step would be to condition it on a target listener's attributes and predict that listener's own score rather than only the crowd average.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces RELATE, an open-source dataset of subjective relevance scores between text and audio. The dataset contains original AudioCaps audio-text pairs plus audio synthesized by four TTA models, annotated by many listeners on 11-point REL, IS, and OS scales, together with listener attributes. The authors analyze the REL scores with respect to audio category and text complexity, using nonparametric tests and ART ANOVA. They then train a supervised relevance prediction model combining BYOL-A audio features, RoBERTa text features, listener embeddings, and a BLSTM with a class-balanced loss, and compare it against zero-shot CLAPScore baselines on a held-out split. The reported results show the proposed model achieving SRCC 0.383 versus 0.351 for LAION-CLAP and 0.181 for MS-CLAP on the test set, with per-category advantages in most sound categories.

Significance. If the dataset is reliable and the evaluation split is sound, RELATE is a potentially useful public benchmark for text-to-audio relevance evaluation, and the proposed model could serve as a practical screening tool. The paper's strengths include the open release of the dataset and code, the use of multiple TTA systems and a large listener pool, and the comparison against a reasonable zero-shot baseline. The main caveat is that the headline performance advantage is numerically small and is reported without uncertainty quantification, and the dataset quality rests on an anchor-screening procedure whose premises are not validated. The attribute analysis is also informative but is performed on a further-filtered subset that is not the released dataset.

major comments (3)
  1. [Sec. 5.4, Table 6] The central claim that the proposed model outperforms CLAPScore is supported only by point estimates on a single split. The SRCC gap over LAION-CLAP is 0.032 (0.383 vs 0.351), which could be within sampling noise; the paper reports no confidence intervals, bootstrap resamples, or significance tests for any metric in Tables 6 and 7. I request bootstrap confidence intervals for all metrics, ideally with a paired test for the SRCC difference, and corresponding confidence intervals for the per-category SRCCs in Table 7.
  2. [Sec. 3.3] The anchor-screening procedure is the only quality-control step that supports treating the subjective REL labels as ground truth, yet its two central premises are unverified. It is asserted that intentionally mismatched AudioCaps label-audio pairs are low-relevance, but no distribution of anchor scores is shown; and the thresholds (training average score below 2, test below 1) are presented without justification, sensitivity analysis, or a report of how many listeners were excluded. Because these labels both train the benchmark model and serve as evaluation ground truth, a failure of either premise would bias all reported results. Please provide anchor-score distributions, exclusion counts, an inter-annotator agreement metric (e.g., ICC or Krippendorff's alpha), and a sensitivity analysis with alternative thresholds.
  3. [Sec. 5.3, Table 5] The sentence describing the data split, "the test data of REL scores was divided into two subsets so there was no overlap between audio samples and texts," is ambiguous. Please state explicitly whether the training, validation, and test sets are pairwise disjoint in both audio and text, and report the numbers of unique audio items and unique text prompts in each split. If any text prompt appears in both training and test, the benchmark could be affected by text memorization.
minor comments (6)
  1. [Title] The full-text title reads "RELA TE" instead of "RELATE."
  2. [Table 3] There is a missing space in the "Audio duration [s]" row, and the last entry is formatted inconsistently ("11901" instead of "11,901").
  3. [Sec. 5.4, Table 6] The text says that CBL is effective, but Table 6 shows that "Ours" has higher MSE (0.073) than "Ours w/o CBL" (0.069), even though its SRCC is higher. Please acknowledge this trade-off explicitly or clarify that the effectiveness claim refers only to rank correlation metrics.
  4. [Sec. 4] The stricter screening used for the attribute analysis (excluding listeners with average anchor score at least 2, average original-audio score at most 6, and the lowest 5% entropy) is not part of the released dataset. Please state clearly that Section 4 analyzes a further-filtered subset, and define how the rating entropy was computed.
  5. [Sec. 3.2] The paper says "For each text, two synthesis models are selected and synthesized" but does not describe how the two models are selected, whether the selection is random, or which seed is used. Please report this for reproducibility.
  6. [Sec. 5.3] The hyperparameters are described only as "empirically chosen." Please report the validation metric used for model selection and the range of values that were explored, at least briefly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the RELATE model is trained on a training split and evaluated on a disjoint held-out test split against an external zero-shot CLAPScore baseline.

full rationale

The paper's central claim is that a supervised model trained on RELATE's human relevance labels predicts held-out RELATE labels better than CLAPScore (Table 6). This is standard supervised benchmarking: the training labels determine the regression target, but the reported test labels are never used in fitting, and Section 5.3 states the validation and evaluation subsets are split with no overlap in audio samples or texts. Hyperparameters and the final model are selected on a validation subset, not on the evaluation subset. CLAPScore is an externally pretrained zero-shot baseline, so its comparison does not reduce to the model's training objective. The anchor-based listener screening of Section 3.3 is a data-quality filter applied before both training and test collection; it neither defines the prediction target nor entrenches the test outcome. The paper's self-citations (e.g., UTMOS [8] for the contrastive loss and LDNet [25] for listener embeddings) are technical building blocks, not load-bearing justifications of the headline comparison, and none is invoked as a uniqueness theorem or to forbid alternatives. There is no equation in the paper that equals its own input by construction, and no fitted parameter is renamed as a prediction. The absence of confidence intervals on the SRCC gap and the unquantified effect of anchor screening are validity and reporting concerns, not circularity.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

All load-bearing assumptions are standard for MOS-style prediction and dataset construction. The free parameters are disclosed screening thresholds and training hyperparameters; the paper does not introduce new theoretical entities, so the invented-entities list is empty.

free parameters (2)
  • Listener screening thresholds = Train: exclude avg anchor >=2; test: exclude >=1; analysis: exclude avg original audio <=6 and lowest 5% entropy
    Hand-chosen cutoffs that determine which listeners and ratings stay in the dataset and analysis; they affect all downstream scores and significance tests.
  • Training loss hyperparameters = tau=0.25, alpha=0.1, beta_cbl=0.99, beta=1.0, gamma=0.5
    Empirically chosen in Section 5.3; the final model and its reported metrics depend on these values.
assumptions (6)
  • domain assumption Pretrained BYOL-A and RoBERTa encoders provide transferable audio and text representations for relevance prediction.
    Section 5.1 uses these frozen encoders without fine-tuning; if the representations are not aligned with relevance, the model's performance changes.
  • domain assumption CLAPScore computed with MS-CLAP or LAION-CLAP is the appropriate conventional objective baseline.
    Sections 2 and 5.3 define the comparison; the claimed improvement is relative to these baselines only.
  • domain assumption AudioCaps captions are valid text prompts for text-to-audio models and valid ground truth for relevance judgments.
    Section 3.2 uses original AudioCaps texts as prompts and as evaluation references.
  • domain assumption Screening anchors, randomly mismatched sound event labels, have genuinely low relevance.
    Section 3.3; thresholds for listener exclusion assume anchor ratings near zero indicate low-quality raters.
  • standard math Subjective ratings on an 11-point scale can be treated as numeric targets for MSE, LCC, SRCC, and KTAU.
    Sections 5.2 and 5.3; this is standard practice in MOS prediction but is an assumption about the measurement scale.
  • standard math Nonparametric tests and ART ANOVA are valid for the item-level comparisons in Section 4.
    The paper uses Mann-Whitney, Kruskal-Wallis, Steel-Dwass, and ART ANOVA; this assumes independence across ratings and appropriate error structure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RELATE: Subjective evaluation dataset for automatic evaluation of relevance between text and audio." pith.science (2026). https://pith.science/paper/LK3HREGZ

@misc{pith2026250623582,
  author       = {Pith},
  title        = {Pith review of: RELATE: Subjective evaluation dataset for automatic evaluation of relevance between text and audio},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LK3HREGZ}},
  note         = {Machine review of arXiv:2506.23582}
}
read the original abstract

In text-to-audio (TTA) research, the relevance between input text and output audio is an important evaluation aspect. Traditionally, it has been evaluated from both subjective and objective perspectives. However, subjective evaluation is costly in terms of money and time, and objective evaluation is unclear regarding the correlation to subjective evaluation scores. In this study, we construct RELATE, an open-sourced dataset that subjectively evaluates the relevance. Also, we benchmark a model for automatically predicting the subjective evaluation score from synthesized audio. Our model outperforms a conventional CLAPScore model, and that trend extends to many sound categories.

Figures

Figures reproduced from arXiv: 2506.23582 by the authors.

Figure 1
Figure 1. Overview of this study. The contributions of our paper are illustrated in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Overview of proposed REL prediction model [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 2
Figure 2. Boxplots of each aspect [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 32 canonical work pages

  1. [1]

    a dog barking behind a human speech,

    Introduction Research on text-to-audio (TTA), which is a technology to au- tomatically synthesize an audio sample from text, such as “a dog barking behind a human speech,” is attracting attention [1]. TTA has much potential, such as generating background sounds and sound effects for media contents [2] and creating audio en- vironments in virtual reality. ...

    work page

  2. [2]

    Related work In the field of speech synthesis, a method for predicting subjec- tive scores on the naturalness of synthesized speech has been proposed. The V oiceMOS Challenge [4], which is an interna- tional competition to assess the performance of automatic sub- jective score prediction, has also been held with a shared dataset of synthesized speech and ...

    work page Pith review arXiv 2025

  3. [3]

    Meta Audiobox Aes- thetics: Unified Automatic Quality Assessment for Speech, Music, and Sound,

    Creation of dataset 3.1. Overview of dataset Our dataset consists of the following contents. • Text–audio pairs. Both original and synthetic audio samples are included. • Subjective evaluation scores. Three metrics of 11-point scores for each text–audio referring to the DCASE 2024 Challenge Task 73. – REL score. The overall relevance of the text and audio...

    arXiv 2024

  4. [4]

    2” or higher, we added the exclusion of listeners with an average original audio rating of “6

    Analysis of dataset REL is the most commonly used metric in three metrics, and we have conducted an analysis of REL. We focused on the REL and analyzed the subjective evaluation dataset in the follow- ing aspects: audio attribute and text attribute. Furthermore, we investigated whether these trends differ between original and synthesized audio samples. A ...

    work page

  5. [5]

    Roberta: A robustly optimized bert pretraining approach,

    Benchmarking prediction model 5.1. Model architecture We trained a model to predict the REL score between audio and text. Figure 3 shows the model. The audio x and text w are Table 7: Results of subjective evaluation prediction for each top-level category. Source- Channel, Method Human Animal Natural Music Sounds ambiguous environment Speechsounds sounds ...

    arXiv 1907

  6. [6]

    Conclusion We constructed an open-source dataset consisting of synthe- sized audio samples and relevance scores. From the analysis re- sult, regarding audio attributes, people have different evaluation tendencies for different types of sound, and there are sounds that the synthesis model does not handle well. For text complexity, we found that the longer ...

    work page

  7. [7]

    Acknowledgements The work was supported by JSPS KAKENHI Grant Number 23K24895, 24K23880, 25K21221, JST Moonshot Grant Num- ber JPMJMS2237

    work page

  8. [8]

    UTMOS: UTokyo-SaruLab system for V oice- MOS Challenge 2022,

    T. Saeki, D. Xin, W. Nakata, T. Koriyama, S. Takamichi, and H. Saruwatari, “UTMOS: UTokyo-SaruLab system for V oice- MOS Challenge 2022,” in Interspeech 2022 , 2022, pp. 4521– 4525

    work page 2022

Show all 39 references
  1. [9]

    Diffsound: Discrete diffusion model for text-to-sound genera- tion,

    D. Yang, J. Yu, H. Wang, W. Wang, C. Weng, Y . Zou, and D. Yu, “Diffsound: Discrete diffusion model for text-to-sound genera- tion,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 1720–1733, 2023

    2023

  2. [10]

    Sound synthesis for impact sounds in video games,

    D. B. Lloyd, N. Raghuvanshi, and N. K. Govindaraju, “Sound synthesis for impact sounds in video games,” in Symposium on Interactive 3D Graphics and Games, 2011, pp. 55–62

    2011

  3. [11]

    Challenge on sound scene synthe- sis: Evaluating text-to-audio generation,

    J. Lee, M. Tailleur, L. M. Heller, K. Choi, B. M. M. Lagrange, K. Imoto, and Y . Okamoto, “Challenge on sound scene synthe- sis: Evaluating text-to-audio generation,” in Audio Imagination: NeurIPS 2024 Workshop AI-Driven Speech, Music, and Sound Generation, 2024

    2024

  4. [12]

    The V oiceMOS Challenge 2024: Beyond speech quality prediction,

    W.-C. Huang, S.-W. Fu, E. Cooper, R. E. Zezario, T. Toda, H.- M. Wang, J. Yamagishi, and Y . Tsao, “The V oiceMOS Challenge 2024: Beyond speech quality prediction,” in 2024 IEEE Spoken Language Technology Workshop (SLT), 2024, pp. 803–810

    2024

  5. [13]

    Subjective-aligned dataset and metric for text-to-video quality assessment,

    T. Kou, X. Liu, Z. Zhang, C. Li, H. Wu, X. Min, G. Zhai, and N. Liu, “Subjective-aligned dataset and metric for text-to-video quality assessment,” in Proceedings of the 32nd ACM Interna- tional Conference on Multimedia . Association for Computing Machinery, 2024, p. 7793–7802

    2024

  6. [14]

    Environmental sound synthesis from vocal imitations and sound event labels,

    Y . Okamoto, K. Imoto, S. Takamichi, R. Nagase, T. Fukumori, and Y . Yamashita, “Environmental sound synthesis from vocal imitations and sound event labels,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), 2024, pp. 411–415

    2024

  7. [15]

    Make-an-audio: Text-to-audio gen- eration with prompt-enhanced diffusion models,

    R. Huang, J. Huang, D. Yang, Y . Ren, L. Liu, M. Li, Z. Ye, J. Liu, X. Yin, and Z. Zhao, “Make-an-audio: Text-to-audio gen- eration with prompt-enhanced diffusion models,” in Proceedings of the 40th International Conference on Machine Learning , ser. Proceedings of Machine Lea...

    2023

  8. [16]

    Text-to- audio generation using instruction guided latent diffusion model,

    D. Ghosal, N. Majumder, A. Mehrish, and S. Poria, “Text-to- audio generation using instruction guided latent diffusion model,” in Proceedings of the 31st ACM International Conference on Mul- timedia, 2023, pp. 3590–3598

    2023

  9. [17]

    wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech repre- sentations,” Advances in neural information processing systems , vol. 33, pp. 12 449–12 460, 2020

    2020

  10. [18]

    WavLM: Large-scale self-supervised pre-training for full stack speech processing,

    S. Chen, C. Wang, Z. Chen, Y . Wu, S. Liu, Z. Chen, J. Li, N. Kanda, T. Yoshioka, X. Xiao, J. Wu, L. Zhou, S. Ren, Y . Qian, Y . Qian, J. Wu, M. Zeng, X. Yu, and F. Wei, “WavLM: Large-scale self-supervised pre-training for full stack speech processing,” IEEE Journal of Selecte...

    2022

  11. [19]

    Animal” category shows both of statistically significant differ- ences and interaction. Figure 2(a). shows that synthesized audio in the “Animal

    for two-group means, Kruskal–Wallis test [20] for 3+- group means, and Steel–Dwass test [21] for multiple compari- son. In addition, aligned rank transform (ART) analysis of vari- ance (ANOV A) [22], a method for nonparametric data, was used to examine the interactions of two ...

  12. [20]

    PAM: Prompting audio-language models for audio quality assessment,

    S. Deshmukh, D. Alharthi, B. Elizalde, H. Gamper, M. Al Ismail, R. Singh, B. Raj, and H. Wang, “PAM: Prompting audio-language models for audio quality assessment,” in Interspeech 2024, 2024, pp. 3320–3324

    2024

  13. [21]

    Audio-text mod- els do not yet leverage natural language,

    H.-H. Wu, O. Nieto, J. P. Bello, and J. Salamon, “Audio-text mod- els do not yet leverage natural language,” inProc. ICASSP. IEEE, 2023, pp. 1–5

    2023

  14. [22]

    AudioCaps: Generat- ing captions for audios in the wild,

    C. D. Kim, B. Kim, H. Lee, and G. Kim, “AudioCaps: Generat- ing captions for audios in the wild,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol- ume 1 (Long and Short Paper...

    2019

  15. [23]

    AudioLDM: text-to-audio generation with latent diffusion models,

    H. Liu, Z. Chen, Y . Yuan, X. Mei, X. Liu, D. Mandic, W. Wang, and M. D. Plumbley, “AudioLDM: text-to-audio generation with latent diffusion models,” in Proceedings of the 40th International Conference on Machine Learning, 2023

    2023

  16. [24]

    AudioLDM 2: Learn- ing holistic audio generation with self-supervised pretraining,

    H. Liu, Y . Yuan, X. Liu, X. Mei, Q. Kong, Q. Tian, Y . Wang, W. Wang, Y . Wang, and M. D. Plumbley, “AudioLDM 2: Learn- ing holistic audio generation with self-supervised pretraining,” IEEE/ACM Transactions on Audio, Speech, and Language Pro- cessing, 2024

    2024

  17. [25]

    Tango 2: Aligning diffusion-based text-to-audio generations through direct preference optimization,

    N. Majumder, C.-Y . Hung, D. Ghosal, W.-N. Hsu, R. Mihalcea, and S. Poria, “Tango 2: Aligning diffusion-based text-to-audio generations through direct preference optimization,” in Proceed- ings of the 32nd ACM International Conference on Multimedia , 2024, pp. 564–572

    2024

  18. [26]

    Toward verifiable and repro- ducible human evaluation for text-to-image generation,

    M. Otani, R. Togashi, Y . Sawai, R. Ishigami, Y . Nakashima, E. Rahtu, J. Heikkil¨a, and S. Satoh, “Toward verifiable and repro- ducible human evaluation for text-to-image generation,” in Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023...

    2023

  19. [27]

    On a test of whether one of two random variables is stochastically larger than the other,

    H. B. Mann and D. R. Whitney, “On a test of whether one of two random variables is stochastically larger than the other,” The annals of mathematical statistics, pp. 50–60, 1947

    1947

  20. [28]

    Use of ranks in one-criterion variance analysis,

    W. H. Kruskal and W. A. Wallis, “Use of ranks in one-criterion variance analysis,” Journal of the American statistical Associa- tion, vol. 47, no. 260, pp. 583–621, 1952

    1952

  21. [29]

    A multiple comparison rank sum test: treatments versus control,

    R. G. Steel, “A multiple comparison rank sum test: treatments versus control,” Biometrics, pp. 560–572, 1959

    1959

  22. [30]

    The aligned rank transform for nonparametric factorial analyses using only anova procedures,

    J. O. Wobbrock, L. Findlater, D. Gergle, and J. J. Higgins, “The aligned rank transform for nonparametric factorial analyses using only anova procedures,” inProceedings of the SIGCHI conference on human factors in computing systems, 2011, pp. 143–146

    2011

  23. [31]

    A new readability yardstick

    R. Flesch, “A new readability yardstick.” Journal of applied psy- chology, vol. 32, no. 3, p. 221, 1948

    1948

  24. [32]

    BYOL for Audio: Exploring pre-trained general-purpose audio representations,

    D. Niizumi, D. Takeuchi, Y . Ohishi, N. Harada, and K. Kashino, “BYOL for Audio: Exploring pre-trained general-purpose audio representations,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 137–151, 2022

    2022

  25. [33]

    LDNet: Unified listener dependent modeling in MOS prediction for syn- thetic speech,

    W.-C. Huang, E. Cooper, J. Yamagishi, and T. Toda, “LDNet: Unified listener dependent modeling in MOS prediction for syn- thetic speech,” in Proc. ICASSP. IEEE, 2022, pp. 896–900

    2022

  26. [34]

    Bidirectional LSTM networks for improved phoneme classification and recog- nition,

    A. Graves, S. Fern ´andez, and J. Schmidhuber, “Bidirectional LSTM networks for improved phoneme classification and recog- nition,” in International conference on artificial neural networks. Springer, 2005, pp. 799–804

    2005

  27. [35]

    MBNet: MOS prediction for synthesized speech with mean-bias network,

    Y . Leng, X. Tan, S. Zhao, F. Soong, X.-Y . Li, and T. Qin, “MBNet: MOS prediction for synthesized speech with mean-bias network,” in Proc. ICASSP, 2021, pp. 391–395

    2021

  28. [36]

    Class- balanced loss based on effective number of samples,

    Y . Cui, M. Jia, T.-Y . Lin, Y . Song, and S. Belongie, “Class- balanced loss based on effective number of samples,” inProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 9268–9277

    2019

  29. [37]

    Adam: A method for stochastic op- timization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic op- timization,” in 3rd International Conference on Learning Repre- sentations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Con- ference Track Proceedings, Y . Bengio and Y . LeCun, Eds., 2015

    2015

  30. [38]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” in Proc. ICASSP, 2023, pp. 1–5

    2023

  31. [39]

    CLAP: Learning audio concepts from natural language supervision,

    B. Elizalde, S. Deshmukh, M. A. Ismail, and H. Wang, “CLAP: Learning audio concepts from natural language supervision,” in Proc. ICASSP, 2023, pp. 1–5

    2023

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.