Pith. sign in

REVIEW 4 major objections 4 minor 60 references

Assessing the Alignment of Audio Representations with Timbre Similarity Ratings

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Style embeddings from CLAP align with human timbre similarity ratings better than any other audio representation tested across 21 legacy datasets.

desk verdict A well-engineered, large-scale evaluation of audio representations against human timbre ratings, with a plausible but unquantified claim that CLAP style embeddings lead. read the letter →

arxiv 2507.07764 v1 pith:G5FDWOHR submitted 2025-07-10 cs.SD eess.AS

Haokun Tian , Stefan Lattner , Charalampos Saitis This is my paper
classification cs.SDeess.AS
keywords timbresimilarityaudiorepresentationsstyleembeddingsCLAPperceptualmetricmultidimensionalscalingdeeplearningevaluationsoundmatching
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether off-the-shelf audio representations can serve as a perceptual metric for timbre similarity, meaning a distance in embedding space that reflects how similar two instrument sounds sound to humans. The authors compile 2,614 pairwise human ratings from 21 classic psychoacoustic timbre-space studies, too small to train a model but large enough to test one, and score 18 representations from signal-processing baselines, pre-trained deep models, and a newly trained sound matching model. Their central finding is that style embeddings, channel-wise statistics of a model's internal features borrowed from image style transfer, align with human timbre judgments far better than any other representation tested. In particular, the Huang style embedding extracted from the CLAP audio model ranks highest across the alignment metrics. If this holds, a timbre-similarity metric that generalizes to new audio, needed for instrument retrieval and generative sound design, is available without collecting new human ratings.

What carries the argument

The load-bearing object is the style embedding: for a feature map of shape (batch, channels, height, width), or in a Transformer, tokens by feature dimension, the Gatys embedding is the Gram matrix of channel correlations, and the Huang embedding is the per-channel mean and standard deviation pooled over spatial positions. These statistics discard where events occur on the time-frequency plane while keeping what is present, which is exactly the kind of invariance timbre perception requires. The evaluation machinery is a block-diagonal predicted-similarity matrix compared with the pooled human-rating matrix: per-dataset values are rescaled to [0,1], and five scores, mean absolute error, Kendall, Spearman, NDCG, and triplet agreement with a 0.1 margin, are computed per row and averaged, so that a representation is scored on how well it ranks all other sounds from a reference sound. The style embedding carries the argument because it is what lifts the CLAP and sound-matching representations above their base task embeddings.

What would settle it

Re-run the evaluation using only the subset of datasets that share the same pitch and duration, for example the E♭4 datasets, without any block rescaling; if the CLAP Huang style embedding no longer ranks first on triplet agreement, the general-purpose timbre metric claim fails on the very data that defines the current ranking.

Watch

Extended reading notes

Core claim

The paper's central claim, stated on its own terms, is that style embeddings computed from the internal feature maps of audio models capture human judgments of timbre similarity better than the models' own task embeddings, hand-crafted features, or other pre-trained representations. On 21 legacy timbre-similarity datasets pooled into one block-diagonal ground-truth matrix, the Huang and Belongie style embedding, channel-wise mean and standard deviation of activations, extracted from the CLAP model achieves the strongest alignment across mean absolute error and rank-based metrics, with the same style embedding from the authors' synthesis-parameter-prediction model a close competitor. The result is robust to the choice of alignment metric, and style embeddings consistently improve over their base representations regardless of model architecture or training objective. The paper therefore establishes style embeddings as a promising off-the-shelf representation for timbre similarity, one that inherits the generalization and scalability advantages of deep audio models while escaping the quadratic rating cost of classical timbre spaces.

Load-bearing premise

The framework assumes that similarity ratings from 21 studies, collected with different scales, participant groups, and stimulus pitches, can be rescaled block-wise into a single ground-truth matrix and still represent timbre similarity as such; if the studies measure different things, the model ranking could change.

Editorial extensions

If this is right

  • Any audio model with convolutional or Transformer layers can produce a Huang style embedding, so the top-performing timbre metric is available without retraining a model.
  • Because style embeddings are invariant to where sound events fall in time and frequency, they should transfer to timbre comparisons at varying pitch and duration, which classical timbre spaces cannot represent.
  • The authors' sound matching model shows that even a model trained only to invert synthesizer parameters develops style embeddings that track human timbre judgments, suggesting the phenomenon is not limited to language-supervised models.
  • The released Python package lets future work evaluate any waveform-to-embedding model against these 21 datasets, and can be used as a training-time monitor for timbre-aware generative models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • We infer that the same Huang-style embedding should transfer to timbre comparisons at variable pitch and duration because it discards spatial location, but the paper's datasets fix pitch and duration, so this extension is untested.
  • We infer that compression level may interact with timbre alignment: the 24k Encodec beating its 48k sibling suggests testing whether other codecs follow a compression-alignment curve.
  • We infer that the 0.1 triplet margin, chosen without psychophysical fitting, could be tuned per dataset and might reshuffle the ranking, so the winning margin deserves a sensitivity analysis.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a unified evaluation framework for measuring how well audio representations align with human timbre-similarity ratings, pooling 21 legacy psychoacoustic datasets into a sparse block-diagonal matrix of 334 samples and 2,614 pairwise ratings. Sixteen representations are compared, including signal-processing baselines, pre-trained models (CLAP, CDPAM, Encodec, DAC, Music2Latent, CAE), and three embeddings from a newly trained synthesizer-inversion network (task, Gatys style, Huang style). Alignment is measured with MAE and rank-based scores after per-dataset min-max rescaling. The central claim is that style embeddings—especially the Huang style embedding extracted from CLAP—'remarkably outperform' all other representations. The evaluation design is careful in several respects: human data are used only for testing, the sound matching model is evaluated with a three-fold validation-test split, and the metric package is released. The main weakness is that the headline comparative claim rests entirely on point estimates with no uncertainty quantification, and the MAE variant is computed after a monotone rescaling that weakens its interpretation as an absolute-distance metric.

Significance. If the finding is robust, the paper identifies a practical off-the-shelf representation—CLAP Huang style embeddings—whose distances align with human timbre similarity across diverse legacy datasets, with potential applications in generative modeling, retrieval, and timbre control. The paper also contributes a reusable open-source evaluation tool and a clean protocol for test-only use of scarce human similarity data. The evaluation is largely free of circularity: no parameter is fitted to the human timbre ratings, and the sound matching model is trained on synthetic synthesizer data. The main statistical weakness is the absence of any confidence intervals or significance tests for the cross-model ranking, which is exactly the load-bearing part of the paper's strongest claim.

major comments (4)
  1. [§4 and Figure 3] The core comparative claim—that CLAP-Huang 'remarkably outperform[s] the others'—is supported only by point estimates, with no confidence intervals, bootstrap, permutation test, or paired significance test. Because all representations are evaluated on the same 334 audio samples, paired resampling at the dataset or pair level is straightforward. Without such an analysis, the reported lead of CLAP-Huang over the runner-up could be within sampling noise or driven by a few influential blocks; Table 1 shows substantial block-size heterogeneity (11–24 stimuli, 6–40 raters). I request a robustness analysis, such as dataset-level bootstrap of mean alignment scores and pairwise significance tests, before the headline ranking is accepted.
  2. [§3.2.3 and Figure 3] MAE is computed after min-max rescaling of each dataset block in both the predicted and ground-truth matrices. This rescaling is monotone, so rank-based metrics are unaffected, but it removes the absolute scale of the predicted distances. As a result, the MAE score measures how closely the per-block distribution of predicted distances matches the rescaled human ratings in shape, not absolute distance fidelity; a model with a more uniform within-block distance distribution can achieve a better MAE without better perceptual alignment. The abstract's reference to comparing 'absolute values' and the MAE panel in Figure 3 therefore need either a non-rescaled analysis (e.g., per-dataset correlation or regression on raw ratings) or an explicit statement that MAE is a distribution-shape metric, along with a check that the ranking is insensitive to this choice.
  3. [§3.1 and Table 1] The 21 datasets differ in rating scales, participant pools, stimulus pitch, duration, and loudness normalization. Per-block [0,1] rescaling makes the aggregate score a function of within-dataset relative rankings, but it assumes that a unit of rescaled dissimilarity is commensurable across studies. The paper does not test this pooling assumption; a leave-one-dataset-out analysis or per-dataset alignment scores (e.g., mean Spearman per dataset with confidence intervals) would show whether the ranking is robust. If a few datasets are outliers, the pooled winner may not reflect typical timbre similarity.
  4. [§3.4 and Table 1] The sound matching model is evaluated with a fixed two-second analysis window, so the Iverson & Krumhansl 'Whole' and 'Remainder' blocks (mean durations 3.19 s and 3.10 s, with samples up to 4.39 s) are truncated to two seconds, while all other models receive the full recordings. Because these blocks specifically probe steady-state and remainder timbre, the comparison of s.m.-style embeddings against other representations is not performed on identical audio inputs. Please report a sensitivity check on the 19 datasets whose samples are at or below two seconds, or evaluate the other representations on the same truncated audio.
minor comments (4)
  1. [Throughout] There are several typographical errors: 'representa tions' in the title/header, 'Vahidi et. al' in Section 3.1, 'Nof sounds' in Table 1, and 'tasking embedding' in Section 3.3.4 (should be 'task embedding').
  2. [§3.2.3] The triplet agreement margin is fixed at 0.1 with no sensitivity analysis; since representations in Figure 3 are ordered by mean triplet agreement, the ranking could depend on this threshold, so a brief robustness check of the margin would increase confidence.
  3. [§3.2.4 and Figure 3] Figure 3 marks the best result within each configuration, but the paper does not state how many configurations are considered per representation; reporting this number would help readers assess the risk of selection over many distance/length-handling choices.
  4. [§6] The ethics statement appropriately acknowledges the Western-instrument bias of the data, but the conclusion could more explicitly qualify the generalizability claims to non-Western timbres.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: evaluated representations are fixed feature extractors or trained on synthetic data; no parameter is fitted to human timbre ratings.

full rationale

The paper's pipeline uses 21 legacy human-rating datasets as test-only ground truth (Section 3.1), defines alignment metrics independently (Section 3.2.3), and evaluates signal-processing features, pretrained models, and a newly trained sound-matching model. The sound-matching model is trained to predict Vital synthesizer parameters from synthetic audio (Section 3.3.3), not from human similarity ratings. The pretrained models (CLAP, CDPAM, Encodec, DAC, Music2Latent, CAE) are used as fixed feature extractors, so no parameter is fitted to the target ratings. The per-block [0,1] rescaling in Section 3.2.3 is a monotone normalization applied to both predicted and ground-truth matrices; it does not inject rating information into the models. The two author-affiliated models (Music2Latent [48], CAE [49]) are only evaluated, not cited as evidence for the central claim; the strongest result (CLAP-Huang style embeddings) comes from an independently pretrained model. Concerns about missing confidence intervals or pooling assumptions are statistical robustness issues, not circularity. Therefore no load-bearing step reduces to its own inputs.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities. Its central claim rests on the trustworthiness of the pooled human ratings, the comparability of embedding distances to perceptual dissimilarities, and the benign effect of the dynamic-length padding procedure. The only free parameters are the triplet margin and the subjectively chosen synthesizer parameter set.

free parameters (2)
  • Triplet agreement margin = 0.1
    Chosen by hand as a just-noticeable difference proxy; used to filter triplets. No sensitivity analysis is reported (Section 3.2.3).
  • Vital synthesis parameter set and ranges = 10 subjectively selected parameters (8 continuous, 2 discrete)
    The sound matching model's task embedding and style embeddings depend on which synthesizer parameters are predicted; these ranges are subjectively selected (Section 3.3.1).
assumptions (3)
  • domain assumption Pooled human ratings from 21 studies form a valid ground truth after per-dataset rescaling
    Section 3.1 pools datasets with different rating scales, participant pools, and stimulus conditions; Section 3.2.3 rescales each block to [0,1], assuming inter-block comparability. If this assumption fails, the model ranking could change.
  • domain assumption Embedding distances are comparable to human similarity ratings after monotonic or linear rescaling
    The alignment metrics assume a monotonic relationship between embedding distance and perceived dissimilarity; MAE additionally assumes linearity after rescaling (Section 3.2.3).
  • domain assumption Zero-padding shorter audio to match longer audio within a pair does not distort the representation
    Dynamic-length handling in Section 3.2.1 pads shorter samples; for models sensitive to time shifting, the sound matching model is the only truncation case. This could introduce artifacts for models with short analysis windows.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Assessing the Alignment of Audio Representations with Timbre Similarity Ratings." pith.science (2026). https://pith.science/paper/G5FDWOHR

@misc{pith2026250707764,
  author       = {Pith},
  title        = {Pith review of: Assessing the Alignment of Audio Representations with Timbre Similarity Ratings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G5FDWOHR}},
  note         = {Machine review of arXiv:2507.07764}
}
read the original abstract

Psychoacoustical so-called "timbre spaces" map perceptual similarity ratings of instrument sounds onto low-dimensional embeddings via multidimensional scaling, but suffer from scalability issues and are incapable of generalization. Recent results from audio (music and speech) quality assessment as well as image similarity have shown that deep learning is able to produce embeddings that align well with human perception while being largely free from these constraints. Although the existing human-rated timbre similarity data is not large enough to train deep neural networks (2,614 pairwise ratings on 334 audio samples), it can serve as test-only data for audio models. In this paper, we introduce metrics to assess the alignment of diverse audio representations with human judgments of timbre similarity by comparing both the absolute values and the rankings of embedding distances to human similarity ratings. Our evaluation involves three signal-processing-based representations, twelve representations extracted from pre-trained models, and three representations extracted from a novel sound matching model. Among them, the style embeddings inspired by image style transfer, extracted from the CLAP model and the sound matching model, remarkably outperform the others, showing their potential in modeling timbre similarity.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

60 extracted references · 57 canonical work pages

  1. [1]

    Assessing the Alignment of Audio Representations with Timbre Similarity Ratings

    INTRODUCTION How do humans distinguish between different musical tim- bres? This question has driven research in the field of psychoacoustics for decades [4]. Researchers typically re- cruit a group of people, play different sounds to them in a controlled acoustic environment, and ask them to rate the differences in the sounds by assigning a score. These ...

    work page Pith review arXiv 2025

  2. [2]

    grass” versus “sand

    RELA TED WORK Several works have aimed to train models that produce a perceptual timbre space. Esling et al. [9] trained a varia- tional autoencoder to reconstruct audio samples of different timbres, using perceptual ratings from timbre space studies to regularize the space. Lostanlen et al. [10] collected timbre similarity judgments on 78 sounds using fr...

    work page

  3. [3]

    [13] and Vahidi et

    DA TA AND EXPERIMENTS 3.1 Data We use data curated by Thoret et al. [13] and Vahidi et. al [15], 1 comprising a total of 21 datasets from 11 pub- lished psychoacoustic studies [1–3, 23–25, 27, 28, 31, 33, 34]. We present summary information for each dataset in Ta- ble 1. Each dataset contains a set of audio samples along with pairwise timbre similarity ra...

    work page 1977

  4. [4]

    We choose to report triplet agreement instead, as it is a more intuitive metric to interpret than the other two

    RESULTS AND DISCUSSION In Figure 2 and 3, we omit the Kendall and Spearman scores, as they are highly correlated with triplet agreement and produce nearly identical rankings of the evaluated represen- tations. We choose to report triplet agreement instead, as it is a more intuitive metric to interpret than the other two. Figure 2 shows training-time align...

    work page

  5. [5]

    CONCLUSION In this paper, we introduced a unified evaluation framework to compare model-derived distances with human similar- ity ratings from 21 classic timbre space datasets, encom- passing a wide range of musical instrument sounds. We assessed both hand-crafted features (e.g., MFCC) and deep learning-based representations (e.g., CLAP, CDPAM, neural aud...

    work page

  6. [6]

    ETHICS STA TEMENTS This work evaluates models using datasets that primarily feature Western musical instruments, which may reflect a cultural bias toward Western music traditions. We acknowl- edge this limitation and are enthusiastic about including non-Western musical data in our evaluation framework, as it may both enhance cultural diversity and help re...

    work page

  7. [7]

    This work is supported by the EPSRC UKRI Centre for Doctoral Training in Artificial Intelligence and Mu- sic (grant number EP/S022694/1)

    ACKNOWLEDGEMENTS We thank Mathieu Lagrange for the valuable discussions. This work is supported by the EPSRC UKRI Centre for Doctoral Training in Artificial Intelligence and Mu- sic (grant number EP/S022694/1). This research utilized Queen Mary’s Apocrita HPC facility, supported by QMUL Research-IT. http://doi.org/10.5281/zenodo.438045

    doi:10.5281/zenodo.438045

  8. [8]

    Timbre space as a musical control struc- ture,

    D. L. Wessel, “Timbre space as a musical control struc- ture,” Computer Music Journal, pp. 45–52, 1979

    work page 1979

Show all 60 references
  1. [9]

    From clarinet control to timbre perception,

    M. Barthet, P. Guillemain, R. Kronland-Martinet, and S. Ystad, “From clarinet control to timbre perception,” Acta Acustica united with Acustica, vol. 96, no. 4, pp. 678–689, 2010

    2010

  2. [10]

    Music in our ears: the biological bases of musical timbre perception,

    K. Patil, D. Pressnitzer, S. Shamma, and M. Elhilali, “Music in our ears: the biological bases of musical timbre perception,” PLoS Computational Biology, vol. 8, no. 11, p. e1002759, 2012

    2012

  3. [11]

    An in- terlanguage unification of musical timbre: Bridging semantic, perceptual, and acoustic dimensions,

    A. Zacharakis, K. Pastiadis, and J. D. Reiss, “An in- terlanguage unification of musical timbre: Bridging semantic, perceptual, and acoustic dimensions,” Music Perception: An Interdisciplinary Journal, vol. 32, no. 4, pp. 394–412, 2015

    2015

  4. [12]

    The perceptual representation of timbre,

    S. McAdams, “The perceptual representation of timbre,” Timbre: Acoustics, Perception, and Cognition, pp. 23– 57, 2019

    2019

  5. [13]

    Acoustic structure of the five perceptual dimensions of timbre in orchestral instrument tones,

    T. M. Elliott, L. S. Hamilton, and F. E. Theunissen, “Acoustic structure of the five perceptual dimensions of timbre in orchestral instrument tones,” The Journal of the Acoustical Society of America, vol. 133, no. 1, pp. 389–404, 2013

    2013

  6. [14]

    Neural au- dio synthesis of musical notes with wavenet autoen- coders,

    J. Engel, C. Resnick, A. Roberts, S. Dieleman, M. Norouzi, D. Eck, and K. Simonyan, “Neural au- dio synthesis of musical notes with wavenet autoen- coders,” in International Conference on Machine Learn- ing. PMLR, 2017, pp. 1068–1077

    2017

  7. [15]

    DDSP: Differentiable digital signal processing,

    J. Engel, L. H. Hantrakul, C. Gu, and A. Roberts, “DDSP: Differentiable digital signal processing,” in International Conference on Learning Representations,

  8. [16]

    Timbre dissimi- larity metrics,

    B. Hayes and C. Vahidi, “Timbre dissimi- larity metrics,” https://github.com/ben-hayes/ timbre-dissimilarity-metrics, 2021, accessed: 2025-03- 29

    2021

  9. [17]

    A differentiable perceptual audio metric learned from just noticeable differences,

    P. Manocha, A. Finkelstein, R. Zhang, N. J. Bryan, G. J. Mysore, and Z. Jin, “A differentiable perceptual audio metric learned from just noticeable differences,” in Interspeech, 2020

    2020

  10. [18]

    Generative timbre spaces with variational audio synthe- sis,

    P. Esling, A. Chemla-Romeu-Santos, and A. Bitton, “Generative timbre spaces with variational audio synthe- sis,” in Proceedings of the International Conference on Digital Audio Effects (DAFx), 2018, pp. 175–181

    2018

  11. [19]

    Time–frequency scattering accurately models auditory similarities between instru- mental playing techniques,

    V . Lostanlen, C. El-Hajj, M. Rossignol, G. Lafay, J. Andén, and M. Lagrange, “Time–frequency scattering accurately models auditory similarities between instru- mental playing techniques,”EURASIP Journal on Audio, Speech, and Music Processing, vol. 2021, no. 1, p. 3, 2021

    2021

  12. [20]

    Free sorting task,

    S. Chollet, D. Valentin, and H. Abdi, “Free sorting task,” Novel Techniques in Sensory Characterization and Consumer Profiling, vol. 207, 2014

    2014

  13. [21]

    Perceptual musical simi- larity metric learning with graph neural networks,

    C. Vahidi, S. Singh, E. Benetos, H. Phan, D. Stowell, G. Fazekas, and M. Lagrange, “Perceptual musical simi- larity metric learning with graph neural networks,” in 2023 IEEE Workshop on Applications of Signal Process- ing to Audio and Acoustics (WASPAA). IEEE, 2023, pp. 1–5

    2023

  14. [22]

    Learning metrics on spectrotemporal modulations re- veals the perception of musical instrument timbre,

    E. Thoret, B. Caramiaux, P. Depalle, and S. Mcadams, “Learning metrics on spectrotemporal modulations re- veals the perception of musical instrument timbre,” Na- ture Human Behaviour , vol. 5, no. 3, pp. 369–377, 2021

    2021

  15. [23]

    On the robustness of mu- sical timbre perception models: From perceptual to learned approaches,

    B. Pascal and M. Lagrange, “On the robustness of mu- sical timbre perception models: From perceptual to learned approaches,” in 2024 32nd European Signal Processing Conference (EUSIPCO). IEEE, 2024, pp. 41–45

    2024

  16. [24]

    Acous- tic representations for perceptual timbre similarity,

    C. Vahidi, B. Hayes, C. Saitis, and G. Fazekas, “Acous- tic representations for perceptual timbre similarity,” in Digital Music Research Network One-Day Workshop (DMRN+ 16), 2021

    2021

  17. [25]

    Isolating the dynamic attributes of musical timbrea,

    P. Iverson and C. L. Krumhansl, “Isolating the dynamic attributes of musical timbrea,” The Journal of the Acous- tical Society of America, vol. 94, no. 5, pp. 2595–2603, 1993

    1993

  18. [26]

    Opolko and J

    F. Opolko and J. Wapnick, McGill University master samples (3 CDs). Quebec, Canada: McGill University, 1987

    1987

  19. [27]

    Cd- pam: Contrastive learning for perceptual audio simi- larity,

    P. Manocha, Z. Jin, R. Zhang, and A. Finkelstein, “Cd- pam: Contrastive learning for perceptual audio simi- larity,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021, pp. 196–200

    2021

  20. [28]

    The unreasonable effectiveness of deep fea- tures as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep fea- tures as a perceptual metric,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, 2018, pp. 586–595

    2018

  21. [29]

    Dreamsim: Learning new dimen- sions of human visual similarity using synthetic data,

    S. Fu, N. Tamir, S. Sundaram, L. Chai, R. Zhang, T. Dekel, and P. Isola, “Dreamsim: Learning new dimen- sions of human visual similarity using synthetic data,” in Advances in Neural Information Processing Systems, vol. 36, 2023, pp. 50 742–50 768

    2023

  22. [30]

    Improving neural network representations using human similarity judgments,

    L. Muttenthaler, L. Linhardt, J. Dippel, R. A. Vander- meulen, K. Hermann, A. Lampinen, and S. Kornblith, “Improving neural network representations using human similarity judgments,” Advances in Neural Information Processing Systems, vol. 36, pp. 50 978–51 007, 2023

    2023

  23. [31]

    Things-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior,

    M. N. Hebart, O. Contier, L. Teichmann, A. H. Rock- ter, C. Y . Zheng, A. Kidder, A. Corriveau, M. Vaziri- Pashkam, and C. I. Baker, “Things-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior,” Elife, vol....

    2023

  24. [32]

    Multidimensional perceptual scaling of musical timbres,

    J. M. Grey, “Multidimensional perceptual scaling of musical timbres,” the Journal of the Acoustical Society of America, vol. 61, no. 5, pp. 1270–1277, 1977

    1977

  25. [33]

    Perceptual effects of spectral modifications on musical timbres,

    J. M. Grey and J. W. Gordon, “Perceptual effects of spectral modifications on musical timbres,” The Journal of the Acoustical Society of America, vol. 63, no. 5, pp. 1493–1500, 1978

    1978

  26. [34]

    Tim- bre space representation of a subtractive synthesizer,

    C. Vahidi, G. Fazekas, C. Saitis, and A. Palladini, “Tim- bre space representation of a subtractive synthesizer,” in Proceedings of the 2nd International Conference on Timbre, 2020, p. 30–33

    2020

  27. [35]

    While some dynamic variations can be observed, loudness is typically reported to have been normalized by expert listeners

    with a block size of 0.08 seconds—slightly shorter than the shortest sample. While some dynamic variations can be observed, loudness is typically reported to have been normalized by expert listeners. total of 10 or 20 scores for each representation (one model can produce multi...

  28. [36]

    Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes,

    S. McAdams, S. Winsberg, S. Donnadieu, G. De Soete, and J. Krimphoff, “Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes,” Psychological Research, vol. 58, pp. 177–192, 1995

    1995

  29. [37]

    A common perceptual space for harmonic and percussive timbres,

    S. Lakatos, “A common perceptual space for harmonic and percussive timbres,” Perception & Psychophysics, vol. 62, no. 7, pp. 1426–1439, 2000

    2000

  30. [38]

    RWC music database: Music genre database and mu- sical instrument sound database,

    M. Goto, H. Hashiguchi, T. Nishimura, and R. Oka, “RWC music database: Music genre database and mu- sical instrument sound database,” in Proceedings of the 4th International Conference on Music Information Retrieval (ISMIR), 2003

    2003

  31. [39]

    Opolko and J

    F. Opolko and J. Wapnick,The McGill University master samples collection on DVD (3 DVDs). Quebec, Canada: McGill University, 1987

    1987

  32. [40]

    Acoustic and categorical dissimilarity of musical tim- bre: Evidence from asymmetries between acoustic and chimeric sounds,

    K. Siedenburg, K. Jones-Mollerup, and S. McAdams, “Acoustic and categorical dissimilarity of musical tim- bre: Evidence from asymmetries between acoustic and chimeric sounds,” Frontiers in Psychology, vol. 6, p. 1977, 2016

    1977

  33. [41]

    Vienna Symphonic Library, https://www.vsl.co.at/

  34. [42]

    Brightness perception for musical instrument sounds: Relation to timbre dis- similarity and source-cause categories,

    C. Saitis and K. Siedenburg, “Brightness perception for musical instrument sounds: Relation to timbre dis- similarity and source-cause categories,” The Journal of the Acoustical Society of America, vol. 148, no. 4, pp. 2256–2266, 2020

    2020

  35. [43]

    Joint time– frequency scattering,

    J. Andén, V . Lostanlen, and S. Mallat, “Joint time– frequency scattering,” IEEE Transactions on Signal Processing, vol. 67, no. 14, pp. 3704–3718, 2019

    2019

  36. [44]

    pyloudnorm: A simple yet flexible loudness meter in python,

    C. J. Steinmetz and J. Reiss, “pyloudnorm: A simple yet flexible loudness meter in python,” inAudio Engineering Society Convention 150. Audio Engineering Society, 2021

    2021

  37. [45]

    TorchMetrics - Measuring Repro- ducibility in PyTorch,

    Nicki Skafte Detlefsen, Jiri Borovec, Justus Schock, Ananya Harsh, Teddy Koker, Luca Di Liello, Daniel Stancl, Changsheng Quan, Maxim Grechkin, and William Falcon, “TorchMetrics - Measuring Repro- ducibility in PyTorch,” Feb. 2022. [Online]. Available: https://github.com/Light...

    2022

  38. [46]

    Hyperbolic image embeddings,

    V . Khrulkov, L. Mirvakhabova, E. Ustinova, I. Oseledets, and V . Lempitsky, “Hyperbolic image embeddings,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 6418–6428

    2020

  39. [47]

    Adapting frechet audio distance for generative music evaluation,

    A. Gui, H. Gamper, S. Braun, and D. Emmanouilidou, “Adapting frechet audio distance for generative music evaluation,” inIEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 1331–1335

    2024

  40. [48]

    Deep resid- ual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep resid- ual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778

    2016

  41. [49]

    Image style transfer using convolutional neural networks,

    L. A. Gatys, A. S. Ecker, and M. Bethge, “Image style transfer using convolutional neural networks,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2414–2423

    2016

  42. [50]

    Arbitrary style transfer in real-time with adaptive instance normalization,

    X. Huang and S. Belongie, “Arbitrary style transfer in real-time with adaptive instance normalization,” in Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 1501–1510

    2017

  43. [51]

    Kymatio: Scattering transforms in python,

    M. Andreux, T. Angles, G. Exarchakis, R. Leonarduzzi, G. Rochette, L. Thiry, J. Zarka, S. Mallat, J. Andén, E. Belilovsky et al., “Kymatio: Scattering transforms in python,” Journal of Machine Learning Research, vol. 21, no. 60, pp. 1–6, 2020

    2020

  44. [53]

    Natural language supervision for general-purpose audio representations,

    B. Elizalde, S. Deshmukh, and H. Wang, “Natural language supervision for general-purpose audio representations,” 2023. [Online]. Available: https: //arxiv.org/abs/2309.05767

    2023 arXiv

  45. [54]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023

    2023

  46. [55]

    High fidelity neural audio compression,

    A. Défossez, J. Copet, G. Synnaeve, and Y . Adi, “High fidelity neural audio compression,” arXiv preprint arXiv:2210.13438, 2022

    2022 arXiv

  47. [56]

    High-fidelity audio compression with im- proved rvqgan,

    R. Kumar, P. Seetharaman, A. Luebs, I. Kumar, and K. Kumar, “High-fidelity audio compression with im- proved rvqgan,” Advances in Neural Information Pro- cessing Systems, vol. 36, pp. 27 980–27 993, 2023

    2023

  48. [57]

    Music2latent: Consistency autoencoders for latent audio compression,

    M. Pasini, S. Lattner, and G. Fazekas, “Music2latent: Consistency autoencoders for latent audio compression,” Proceedings of the 25th International Conference on Music Information Retrieval (ISMIR), 2024

    2024

  49. [58]

    Learning complex basis functions for invariant representations of audio,

    S. Lattner, M. Dörfler, and A. Arzt, “Learning complex basis functions for invariant representations of audio,” in Proceedings of the 20th International Conference on Music Information Retrieval (ISMIR), 2019

    2019

  50. [59]

    Maps-a piano database for multipitch estimation and automatic transcription of music,

    V . Emiya, N. Bertin, B. David, and R. Badeau, “Maps-a piano database for multipitch estimation and automatic transcription of music,” 2010

    2010

  51. [60]

    I’m sorry for your loss: Spectrally-based audio distances are bad at pitch,

    J. Turian and M. Henry, “I’m sorry for your loss: Spectrally-based audio distances are bad at pitch,” in ”I Can’t Believe It’s Not Better!” NeurIPS 2020 workshop , 2020. [Online]. Available: https://openreview.net/forum?id=Z4UwGkTRTes

    2020

  52. [2020]

    Available: https://openreview.net/ forum?id=B1x1ma4tDr

    [Online]. Available: https://openreview.net/ forum?id=B1x1ma4tDr

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.