Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

PianoBind: A Multimodal Joint Embedding Model for Pop-piano Music

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A piano-specific audio-MIDI-text joint embedding beats general-purpose models at pop-piano text retrieval.

desk verdict PianoBind shows real gains for piano-specific multimodal retrieval, but the paper never checks that its PIAST-YT pretraining data is disjoint from the PIAST-AT and EMOPIA test sets, so the headline superiority claim is not yet trustworthy. read the letter →

arxiv 2509.04215 v1 pith:MSA4SZVK submitted 2025-09-04 cs.SD cs.IRcs.MM

Hayeon Bang , Eunjin Choi , Seungheon Doh , Juhan Nam This is my paper · ORCID
classification cs.SDcs.IRcs.MM
keywords pianomusicrepresentationmultimodaljointembeddingtext-to-musicretrievalaudio-MIDI-textalignmentcontrastivelearningpop-pianodatasetlow-resource
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PianoBind sets out to establish a domain-specific thesis: for solo pop-piano music, a multimodal embedding model trained on piano-only data can retrieve tracks from text descriptions more accurately than general-purpose music-text models trained on far larger, broader corpora. The authors' motivation is that general-purpose models cannot resolve the subtle genre, mood, and style distinctions within a single-instrument genre, while prior piano models ignore at least one of audio, MIDI, or text. PianoBind aligns the three modalities in a shared 512-dimensional space, pre-trains on a large noisy piano-text collection, fine-tunes on expert-annotated pairs, and fuses audio and MIDI embeddings at retrieval time. Reported results put its median rank at 10 on a 199-track in-domain test and an 88-track out-of-domain set, against 39 and 12 for the strongest general-purpose baselines. This matters because the paper offers a reusable low-resource recipe for specialist multimodal retrieval in homogeneous domains.

What carries the argument

The load-bearing mechanism is a 512-dimensional jointly trained embedding space, reached through three encoders and symmetric InfoNCE contrastive losses over audio-text and MIDI-text pairs. Text is the semantic anchor tying the two music modalities together; because MIDI is transcribed from the same recordings, an audio-MIDI loss would add little and is omitted. Two design choices carry the argument: two-stage multi-source training, in which a large weakly labeled collection builds general alignment and a small expert-annotated collection sharpens piano-specific semantics; and inference-time average fusion of audio and MIDI item embeddings, which exploits the complementary views the two moda

What would settle it

Build a held-out piano-text benchmark of several hundred tracks with captions written independently of the PIAST taxonomy (for example, fresh human annotations from annotators who never saw the taxonomy, or real user queries), then compare PianoBind with the same general-purpose baselines on median rank and R@10; if the advantage does not persist, the paper's core superiority claim over general-purpose models fails.

Watch

Extended reading notes

Core claim

The paper's central claim: in a homogeneous single-instrument genre like solo piano, fine semantic distinctions are learnable from a small domain-specific dataset once audio, MIDI, and text are aligned jointly. PianoBind pairs a modified ResNet-50 audio encoder, MidiBERT-Piano, and RoBERTa in one 512-dimensional space, trained with symmetric InfoNCE losses on audio-text and MIDI-text pairs averaged into one objective; because the MIDI is transcribed from the audio, no audio-MIDI loss is used. Pre-training on about 7,367 weakly aligned PIAST-YT captions followed by fine-tuning on 1,986 expert-annotated PIAST-AT tracks yields R@10 of 52.76% and median rank 10 on the 199-track in-domain test, v

Load-bearing premise

The evaluation rests on only 199 in-domain and 88 out-of-domain tracks, and the out-of-domain captions were written from tags that overlapped the training taxonomy, so the reported retrieval gaps might shrink or disappear on a larger, independently annotated test set.

Editorial extensions

If this is right

  • A domain-specific model can outperform general-purpose music-text models on in-domain homogeneous retrieval while using far less training data.
  • Two-stage training (pretrain on weak labels, fine-tune on expert labels) is a more effective low-resource strategy than mixing the two sources in every batch.
  • Aligning audio-text and MIDI-text with an averaged contrastive objective outperforms both unimodal alignment and CLaMP3-style staged alignment.
  • Fusing audio and symbolic embeddings at retrieval time improves discrimination among similar piano tracks, provided the modalities were aligned through joint training.
  • The training and fusion recipe is offered as a reusable guideline for multimodal representation learning on small, homogeneous datasets beyond piano.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the out-of-domain captions were generated from tags that intentionally overlap the PIAST taxonomy, the out-of-domain result likely measures paraphrase robustness rather than semantic generalization; an independently authored caption set with disjoint vocabulary would test the stronger claim.
  • With only 199 and 88 evaluation tracks, the reported median-rank gaps are probably not stable at the reported precision; a larger held-out corpus would be needed to confirm that the advantage is not driven by a few queries or tracks.
  • The same recipe—weak pre-training, expert fine-tuning, text as anchor, inference-time fusion of complementary modalities—should transfer to other homogeneous low-resource domains such as solo violin, choir, or birdsong recordings, a direction the paper leaves implicit.
  • The failure of naive fusion in the CLaMP3 baselines suggests a testable boundary condition: multimodal fusion at retrieval helps only when the fused branches were trained jointly, not when independently optimized models are fused post hoc.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes PianoBind, a piano-specific multimodal joint embedding model that aligns audio, MIDI, and text in a shared space. The model is trained on the PIAST dataset using two multi-source strategies (combined training vs. pre-training then fine-tuning) and a trimodal objective that averages audio–text and MIDI–text contrastive losses. The authors evaluate text-to-music retrieval on a 199-track PIAST-AT held-out set and an 88-track EMOPIA-Caps set, reporting that PianoBind outperforms general-purpose audio-text and symbolic-text models such as CLAP-Music, TTMR++, and CLaMP3. The paper also compares its averaged-loss training with CLaMP3's staged alignment strategy. The central claim is that a piano-specific model trained on much less data can beat general-purpose joint embedding models at retrieving pop-piano tracks. The manuscript is clearly written, details the architecture and training choices, and releases code and pretrained weights.

Significance. If the central claim holds, PianoBind would be a valuable demonstration that domain-specific multimodal joint embeddings can outperform large-scale general-purpose models on homogeneous, low-resource domains like solo piano music. The paper's systematic comparison of training strategies, the trimodal integration, and the public release of code/weights are strengths that support reproducibility and further study. However, the evaluation is currently based on very small test sets (199 and 88 tracks), no uncertainty quantification is reported, and the main comparison is between a domain-trained model and zero-shot general-purpose baselines. The paper itself acknowledges the small-scale evaluation in §6, which is appropriate, but the missing track-level disjointness check between training and test data is a more fundamental concern that must be addressed before the superiority claim is fully supported.

major comments (4)
  1. [§4.1–§4.2, Tables 1–2] The paper never reports a disjointness check between the training data (PIAST-YT, 7,367 tracks, and PIAST-AT train/validation splits) and the evaluation data (the 199-track PIAST-AT test split and the 88-track EMOPIA-Caps test split). Since PIAST-AT is described as a subset of the same YouTube collection, it is plausible that some pretraining or fine-tuning tracks also appear in the test split. If overlap exists, PianoBind's advantage, e.g., MedR 10 vs. 39 for the best general-purpose model in Table 2, could be partially due to memorization. Please report an exact track-level overlap analysis and, if any overlap is found, rerun the evaluation on strictly disjoint splits.
  2. [§5, Tables 1–2] The evaluation sets contain only 199 and 88 tracks, and several reported differences are small in absolute count. For example, in-domain R@1 of 10.55 vs. 8.04 corresponds to 21 vs. 16 tracks out of 199. No confidence intervals, bootstrap estimates, or significance tests are provided, yet the text uses 'significantly surpassing' (e.g., §5.1.1). Please add per-query bootstrap intervals or an item-level analysis to demonstrate that the main comparisons are stable, and avoid the word 'significantly' without statistical support.
  3. [§5.2, Table 2] The comparison is asymmetric: PianoBind is trained on PIAST audio, MIDI, and text data, while CLAP-Music, TTMR++, and the CLaMP variants are evaluated zero-shot without any piano-specific fine-tuning. The observed gaps may therefore reflect exposure to the in-domain distribution rather than architectural superiority. A fairer comparison would fine-tune the general-purpose baselines on PIAST's training split or at least include a domain-adapted baseline. If that is infeasible, the claim should be narrowed to 'zero-shot general-purpose models' rather than 'general-purpose models'.
  4. [§4.2, EMOPIA-Caps] The out-of-domain evaluation is author-constructed: EMOPIA-Caps is built by the authors from EMOPIA test tracks, with initial tags that 'naturally overlapped' with the PIAST-AT taxonomy, followed by GPT-4o paraphrasing and expert review. This leaves open the possibility that the evaluation vocabulary is more aligned with PIAST training labels than with natural user queries, which could favor PianoBind. Please provide the exact caption-generation prompts and examples, and ideally have captions written by independent annotators blind to the PIAST taxonomy, or measure sensitivity to caption paraphrase.
minor comments (4)
  1. [Table 3] The header row 'Model ID OOD' appears misaligned; it should clearly separate ID and OOD columns for R@10 and MedR.
  2. [Table 2 and Figure references] Model naming is inconsistent: 'CLaMP3saas', 'CLaMP3_saas', and 'CLaMP3c2 sa' are used interchangeably. Please standardize.
  3. [§3.2.2] The statement that an audio-MIDI loss 'would not significantly contribute additional semantic distinction' is asserted without an ablation. Please either add a supporting experiment or soften the wording.
  4. [Equation (1)] The denominator notation uses z as both a placeholder and a set; clarify the summation over negative embeddings, e.g., by defining the set of negative texts explicitly.

Circularity Check

0 steps flagged · score 2.0 of 10

No construction-level circularity; evaluation is self-referential but not definitionally circular.

full rationale

The paper's derivation chain is not circular by construction: PianoBind is trained with contrastive losses on PIAST audio/MIDI/text pairs, and retrieval is evaluated on held-out PIAST-AT tracks and external EMOPIA audio with paraphrase captions. The model is not fitted to the test labels, and the out-of-domain audio anchors are external. The reported improvements over general-purpose baselines come from actual fine-tuning on piano-specific data, not from a self-defined target. Two mild self-referentialities exist: (1) the in-domain test captions are concatenated tags from the same PIAST taxonomy used in training; (2) the out-of-domain EMOPIA-Caps captions were authored by the same group and intentionally overlap with the PIAST tag vocabulary, which may favor a model trained on that taxonomy. These are benchmark-construction concerns, not definitional circularity. The paper also acknowledges the small scale of its evaluation sets. A more serious correctness risk is that PIAST-AT may be a subset of PIAST-YT, and the paper never reports a disjointness check between pretraining and test tracks; if true, PIAST-AT test items could appear in PIAST-YT pretraining, inflating PianoBind's scores. However, this is not demonstrated from the text, so it does not constitute shown circularity and does not raise the circularity score beyond 2.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The paper relies on standard deep learning components and a few hand-chosen training hyperparameters. No new physical or formal entities are introduced. The main assumptions concern the quality of pretrained encoders, the alignment of automatically transcribed MIDI, the usefulness of weakly aligned text, and the stability of small evaluation sets.

free parameters (2)
  • PIAST-YT to PIAST-AT sampling ratio = 7:3
    Hand-chosen in Section 4.3 to balance noisy web data and expert annotations; affects multi-source training results.
  • Loss weighting for audio-text vs MIDI-text objectives = 0.5 : 0.5
    Section 3.2.2 sets the total loss as an unweighted average of the two pairwise losses; no weighting search is reported.
assumptions (4)
  • domain assumption Pretrained MidiBERT-Piano and RoBERTa provide strong starting representations for musical and textual semantics.
    Sections 3.1.2 and 3.1.3 assume these encoders transfer to piano MIDI and English descriptions; if false, the learned embeddings degrade.
  • domain assumption MIDI transcriptions from automatic piano transcription are accurate and temporally aligned with audio, so an audio-MIDI contrastive loss is unnecessary.
    Section 3.2.2 states that the audio-MIDI loss is excluded because MIDI is derived from audio; noisy transcription or misalignment could break this assumption.
  • domain assumption Weakly aligned YouTube metadata can serve as positive text examples despite noise.
    Section 3.2.1 uses PIAST-YT as a weakly aligned source; if the text is too noisy or mismatched, pretraining will not transfer.
  • domain assumption Evaluation sets with 199 and 88 items yield stable ranking metrics.
    Section 4.2 and Section 5 report comparisons without error bars, so stability of the reported median ranks is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PianoBind: A Multimodal Joint Embedding Model for Pop-piano Music." pith.science (2026). https://pith.science/paper/MSA4SZVK

@misc{pith2026250904215,
  author       = {Pith},
  title        = {Pith review of: PianoBind: A Multimodal Joint Embedding Model for Pop-piano Music},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MSA4SZVK}},
  note         = {Machine review of arXiv:2509.04215}
}
read the original abstract

Solo piano music, despite being a single-instrument medium, possesses significant expressive capabilities, conveying rich semantic information across genres, moods, and styles. However, current general-purpose music representation models, predominantly trained on large-scale datasets, often struggle to captures subtle semantic distinctions within homogeneous solo piano music. Furthermore, existing piano-specific representation models are typically unimodal, failing to capture the inherently multimodal nature of piano music, expressed through audio, symbolic, and textual modalities. To address these limitations, we propose PianoBind, a piano-specific multimodal joint embedding model. We systematically investigate strategies for multi-source training and modality utilization within a joint embedding framework optimized for capturing fine-grained semantic distinctions in (1) small-scale and (2) homogeneous piano datasets. Our experimental results demonstrate that PianoBind learns multimodal representations that effectively capture subtle nuances of piano music, achieving superior text-to-music retrieval performance on in-domain and out-of-domain piano datasets compared to general-purpose music joint embedding models. Moreover, our design choices offer reusable insights for multimodal representation learning with homogeneous datasets beyond piano music.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multimodal Video-to-Music Recommendation via Semantic Retrieval and Temporal Reranking

    cs.MM 2026-07 unverdicted novelty 5.0 of 10

    VTMR is a two-stage video-to-music recommender: joint audio-visual-text retrieval of candidates, then temporal-sequence reranking, lifting R@10 to 18.3 and matching commercial preference.

Reference graph

Works this paper leans on

39 extracted references · 29 canonical work pages · cited by 1 Pith paper

  1. [1]

    PianoBind: A Multimodal Joint Em- bedding Model for Pop-piano Music

    INTRODUCTION The piano stands as a uniquely versatile solo instrument capable of conveying complex polyphonic musical ex- pression through a single instrument. With its expansive tonal range, harmonic possibilities, and expressive capabil- ities—even allowing orchestral works to be effectively per- formed on a single keyboard—piano music encompasses diver...

    arXiv 2025

  2. [2]

    RELATED WORKS 2.1 Piano Music Representation Learning Piano music has long served as a central subject in MIR, owing to its structural richness and expressive depth. How- ever, despite this sustained attention, existing represen- tation learning approaches for piano music are predom- inantly unimodal—relying solely on symbolic or audio data—thus failing t...

    work page

  3. [3]

    PIANOBIND Considering the specific characteristics of piano solo datasets—such as homogeneous data distribution, limited dataset size, and multimodality—we propose a multimodal joint embedding model specialized for solo piano music. In this section, we describe the overall architecture of Pi- anoBind (section 3.1), and the training strategies it ex- plore...

    work page

  4. [4]

    The dataset consists of audio, MIDI, and textual descriptions, based on a comprehensive piano-specific taxonomy of 31 semantic tags across genre, emotion/mood, and style

    EXPERIMENT 4.1 Dataset This study utilizes the PIAST dataset [19], the first music- text dataset explicitly designed for pop-piano music. The dataset consists of audio, MIDI, and textual descriptions, based on a comprehensive piano-specific taxonomy of 31 semantic tags across genre, emotion/mood, and style. The dataset comprises two subsets: PIAST-YT, a l...

    work page

  5. [5]

    RESULTS 5.1 Comparison of Training Strategies 5.1.1 Multi-source training Table 1 shows the performance of different training strate- gies across both in-domain (PIAST-AT) and out-of-domain (EMOPIA-Caps) test sets. We compare two multi-source learning approaches: combined training and pre-training followed by fine-tuning, each evaluated using two bimodal ...

    work page

  6. [6]

    Despite us- ing substantially less training data than general-purpose models, PianoBind achieved strong retrieval performance

    CONCLUSION In this paper, we introduced PianoBind, a multimodal joint embedding model designed for pop-piano music, integrat- ing audio, symbolic, and textual modalities. Despite us- ing substantially less training data than general-purpose models, PianoBind achieved strong retrieval performance. Our findings suggest that a sequential multi-source trainin...

    work page

  7. [7]

    ACKNOWLEDGMENTS This work has been supported by the collaboration with NCSOFT, Korea

    work page

  8. [8]

    Pop music transformer: Beat-based modeling and generation of expressive pop piano compositions,

    Y .-S. Huang and Y .-H. Yang, “Pop music transformer: Beat-based modeling and generation of expressive pop piano compositions,” in Proceedings of the 28th ACM international conference on multimedia , 2020, pp. 1180–1188

    work page 2020

Show all 39 references
  1. [9]

    Compound word transformer: Learning to compose full-song music over dynamic directed hypergraphs,

    W.-Y . Hsiao, J.-Y . Liu, Y .-C. Yeh, and Y .-H. Yang, “Compound word transformer: Learning to compose full-song music over dynamic directed hypergraphs,” in Proceedings of the AAAI Conference on Artificial In- telligence, vol. 35, no. 1, 2021, pp. 178–186

    2021

  2. [10]

    Compose & embellish: Well-structured piano performance generation via a two-stage approach,

    S.-L. Wu and Y .-H. Yang, “Compose & embellish: Well-structured piano performance generation via a two-stage approach,” in ICASSP 2023 - 2023 IEEE In- ternational Conference on Acoustics, Speech and Sig- nal Processing (ICASSP), 2023, pp. 1–5

    2023

  3. [11]

    Picogen2: Piano cover generation with transfer learning approach and weakly aligned data,

    C.-P. Tan, H. Ai, Y .-H. Chang, S.-H. Guan, and Y .-H. Yang, “Picogen2: Piano cover generation with transfer learning approach and weakly aligned data,” in Pro- ceedings of the 25th International Society for Music Information Retrieval Conference (ISMIR) , San Fran- cisco, CA,...

    2024

  4. [12]

    An end-to-end neural network for polyphonic piano music transcrip- tion,

    S. Sigtia, E. Benetos, and S. Dixon, “An end-to-end neural network for polyphonic piano music transcrip- tion,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 24, no. 5, pp. 927–939, 2016

    2016

  5. [13]

    Onsets and frames: Dual-objective piano transcription,

    C. Hawthorne, E. Elsen, J. Song, A. Roberts, I. Si- mon, C. Raffel, J. Engel, S. Oore, and D. Eck, “Onsets and frames: Dual-objective piano transcription,” arXiv preprint arXiv:1710.11153, 2017

    2017 arXiv

  6. [14]

    High- resolution piano transcription with pedals by regress- ing onset and offset times,

    Q. Kong, B. Li, X. Song, Y . Wan, and Y . Wang, “High- resolution piano transcription with pedals by regress- ing onset and offset times,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 29, pp. 3707–3717, 2021

    2021

  7. [15]

    Polyphonic piano transcription using autoregressive multi-state note model,

    T. Kwon, D. Jeong, and J. Nam, “Polyphonic piano transcription using autoregressive multi-state note model,” in International Society for Music Information Retrieval Conference , 2020. [Online]. Available: https://api.semanticscholar.org/CorpusID:222125050

    2020

  8. [16]

    MidiBERT-Piano: Large-scale pre-training for symbolic music understanding,

    Y .-H. Chou, I. Chen, C.-J. Chang, J. Ching, Y .-H. Yang et al. , “MidiBERT-Piano: Large-scale pre-training for symbolic music understanding,” arXiv preprint arXiv:2107.05223, 2021

    2021 arXiv

  9. [17]

    Pianobart: Symbolic piano music gener- ation and understanding with large-scale pre-training,

    X. Liang, Z. Zhao, W. Zeng, Y . He, F. He, Y . Wang, and C. Gao, “Pianobart: Symbolic piano music gener- ation and understanding with large-scale pre-training,” in 2024 IEEE International Conference on Multimedia and Expo (ICME), 2024, pp. 1–6

    2024

  10. [18]

    Mulan: A joint embedding of music audio and natural language,

    Q. Huang, A. Jansen, J. Lee, R. Ganti, J. Y . Li, and D. P. Ellis, “Mulan: A joint embedding of music audio and natural language,” arXiv preprint arXiv:2208.12415 , 2022

    2022 arXiv

  11. [19]

    Contrastive audio-language learning for music,

    I. Manco, E. Benetos, E. Quinton, and G. Fazekas, “Contrastive audio-language learning for music,” in Proceedings of the 23rd International Society for Mu- sic Information Retrieval Conference (ISMIR), 2022

    2022

  12. [20]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu*, K. Chen*, T. Zhang*, Y . Hui*, T. Berg- Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” in IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing, ICASSP, 2023

    2023

  13. [21]

    Toward uni- versal text-to-music retrieval,

    S. Doh, M. Won, K. Choi, and J. Nam, “Toward uni- versal text-to-music retrieval,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2023, pp. 1–5

    2023

  14. [22]

    Enriching music descriptions with a finetuned-llm and metadata for text-to-music retrieval,

    S. Doh, M. Lee, D. Jeong, and J. Nam, “Enriching music descriptions with a finetuned-llm and metadata for text-to-music retrieval,” in ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 826–830

    2024

  15. [23]

    Clamp: Con- trastive language-music pre-training for cross-modal symbolic music information retrieval,

    S. Wu, D. Yu, X. Tan, and M. Sun, “Clamp: Con- trastive language-music pre-training for cross-modal symbolic music information retrieval,” arXiv preprint arXiv:2304.11029, 2023

    2023 arXiv

  16. [24]

    Clamp 2: Multimodal music information retrieval across 101 lan- guages using large language models,

    S. Wu, Y . Wang, R. Yuan, Z. Guo, X. Tan, G. Zhang, M. Zhou, J. Chen, X. Mu, Y . Gao et al. , “Clamp 2: Multimodal music information retrieval across 101 lan- guages using large language models,” arXiv preprint arXiv:2410.13267, 2024

    2024 arXiv

  17. [25]

    Clamp 3: Universal music information retrieval across unaligned modalities and unseen languages,

    S. Wu, Z. Guo, R. Yuan, J. Jiang, S. Doh, G. Xia, J. Nam, X. Li, F. Yu, and M. Sun, “Clamp 3: Universal music information retrieval across unaligned modalities and unseen languages,” 2025. [Online]. Available: https://arxiv.org/abs/2502.10362

    2025 arXiv

  18. [26]

    PIAST: A multimodal piano dataset with audio, symbolic and text,

    H. Bang, E. Choi, M. Finch, S. Doh, S. Lee, G.-H. Lee, and J. Nam, “PIAST: A multimodal piano dataset with audio, symbolic and text,” in Proceedings of the 3rd Workshop on NLP for Music and Audio (NLP4MusA), Nov. 2024, pp. 5–10

    2024

  19. [27]

    Pianotree vae: Structured representation learning for polyphonic music,

    Z. Wang, Y . Zhang, Y . Zhang, J. Jiang, R. Yang, J. Zhao, and G. Xia, “Pianotree vae: Structured representation learning for polyphonic music,” arXiv preprint arXiv:2008.07118, 2020

    2008 arXiv

  20. [28]

    Learning in- terpretable representation for controllable polyphonic music generation,

    Z. Wang, D. Wang, Y . Zhang, and G. Xia, “Learning in- terpretable representation for controllable polyphonic music generation,” Proceedings of the 23rd Interna- tional Society for Music Information Retrieval Confer- ence (ISMIR), 2020

    2020

  21. [29]

    Collagenet: Fusing arbitrary melody and accompani- ment into a coherent song,

    A. Wuerkaixi, C. Benetatos, Z. Duan, and C. Zhang, “Collagenet: Fusing arbitrary melody and accompani- ment into a coherent song,” International Society for Music Information Retrieval, 2022

    2022

  22. [30]

    EMOPIA: A multi-modal pop piano dataset for emotion recognition and emotion-based music gen- eration,

    H.-T. Hung, J. Ching, S. Doh, N. Kim, J. Nam, and Y .- H. Yang, “EMOPIA: A multi-modal pop piano dataset for emotion recognition and emotion-based music gen- eration,” in Proceedings of 22th International Confer- ence on Music Information Retrieval (ISMIR), 2021

    2021

  23. [31]

    A powerful gener- ative model using random weights for the deep image representation,

    K. He, Y . Wang, and J. Hopcroft, “A powerful gener- ative model using random weights for the deep image representation,” Advances in Neural Information Pro- cessing Systems, vol. 29, 2016

    2016

  24. [32]

    Roberta: A robustly optimized BERT pretraining approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “Roberta: A robustly optimized BERT pretraining approach,” CoRR, vol. abs/1907.11692,

    1907 arXiv

  25. [34]

    Representa- tion learning with contrastive predictive coding,

    A. v. d. Oord, Y . Li, and O. Vinyals, “Representa- tion learning with contrastive predictive coding,”arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  26. [35]

    Mt3: Multi-task multitrack music transcrip- tion,

    J. Gardner, I. Simon, E. Manilow, C. Hawthorne, and J. Engel, “Mt3: Multi-task multitrack music transcrip- tion,” arXiv preprint arXiv:2111.03017, 2021

    2021 arXiv

  27. [36]

    Large-scale contrastive language- audio pretraining with feature fusion and keyword-to- caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language- audio pretraining with feature fusion and keyword-to- caption augmentation,” inICASSP 2023-2023 IEEE In- ternational Conference on Acoustics, Speech and Sig- nal Processi...

    2023

  28. [37]

    Lp-musiccaps: Llm-based pseudo music captioning,

    S. Doh, K. Choi, J. Lee, and J. Nam, “Lp-musiccaps: Llm-based pseudo music captioning,” in ISMIR, 2023

    2023

  29. [38]

    Au- dioclip: Extending clip to image, text and audio,

    A. Guzhov, F. Raue, J. Hees, and A. Dengel, “Au- dioclip: Extending clip to image, text and audio,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 2022, pp. 976–980

    2022

  30. [39]

    Chatgpt-4o (gpt-4 omni),

    OpenAI, “Chatgpt-4o (gpt-4 omni),” https://openai. com/index/gpt-4o, 2024, accessed: 2025-03-26

    2024

  31. [2019]

    Available: http://arxiv.org/abs/1907

    [Online]. Available: http://arxiv.org/abs/1907. 11692

    1907

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.