Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Moonbeam, a transformer pretrained on 81.6K hours of MIDI, encodes notes with absolute and relative musical attributes in five dimensions, and beats larger pretrained music models on most classification benchmarks.

desk verdict Solid architecture paper undercut by an unresolved PiJAMA30 pretraining-overlap question. read the letter →

arxiv 2505.15559 v1 pith:K6F35A2S submitted 2025-05-21 cs.SD cs.AIeess.AS

Zixun Guo , Simon Dixon This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords symbolicmusicMIDIfoundationmodeltokenizationmultidimensionalrelativeattentionclassificationconditionalgenerationinfilling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to establish that a single autoregressive transformer, pretrained directly on MIDI rather than on ABC text scores, can serve as a general foundation model for symbolic music. Its central move is to represent each note event by both its absolute values (onset time, duration, octave, pitch class, instrument, velocity) and its relative relationships, which is how musicians perceive music; a novel tokenizer with continuous sinusoidal embeddings and a Multidimensional Relative Attention mechanism carry this through without added parameters. On three classification tasks across four datasets, the 309M- and 839M-parameter models beat the larger pretrained symbolic music models in most cases, and the finetuned conditional generation model is rated by expert listeners as fitting the given chord and metadata conditions better than a strong REMI-style transformer baseline, despite scoring slightly lower on objective pitch and velocity accuracy. If these findings hold, they show that music-specific inductive biases scale to hundreds of millions of parameters and that expressive, multi-instrument MIDI data can serve as a usable pretraining substrate.

What carries the argument

The load-bearing machinery has three parts. First, the tokenizer: each note event becomes the compound token $x = (o, d, oct, p, i, v)$ — absolute onset, duration, octave, pitch class, instrument, velocity — with all attributes except the instrument encoded by the continuous sinusoidal Fundamental Music Embedding (FME), which preserves relative musical information in the embedding space and extrapolates to unseen inputs without extra trainable parameters. Second, Multidimensional Relative Attention (MRA): an extension of RoPE in which the attention heads are partitioned into $G = 6$ groups and each group is rotated by the value of one musical attribute (onset, duration, octave, pitch class, velocity; the instrument group reuses onset), so the query-key dot product encodes the relative difference along each axis, parameter-free. Third, a GRU sub-decoder that sequentially samples the six sub-tokens of each event, capturing dependencies such as a high pitch making a bass instrument unlikely. For finetuning, classification appends a $\langle cls\rangle$ token and swaps the GRU for a linear head with LoRA, while conditional generation prepends metadata and chord-control sequences to the input so that, with absolute onsets and MRA, the model can attend to the entire control sequence, including future control events, giving it full anticipatory capability for music infilling.

What would settle it

Match the PiJAMA30 test pieces against the pretraining files listed in Table 5 by MIDI file identity or near-duplicate sequence comparison, then re-evaluate (or retrain) Moonbeam with all PiJAMA pieces removed from the corpus. If PiJAMA30 accuracy falls from 0.679 toward Clamp2's 0.440, the reported margin is substantially memorization; if the margin holds under this holdout, the architectural claim survives this attack.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that Moonbeam is the first autoregressive transformer foundation model pretrained at scale on diverse MIDI — 81.6K hours and 18 billion tokens covering piano performance, scores, guitar, bass, drums, saxophone, and multitrack music — and that its gains come from jointly preserving absolute and relative musical information. Each event is a six-part compound token (absolute onset, duration, octave, pitch class, instrument, velocity), embedded with the continuous Fundamental Music Embedding so relative information survives in the embedding space and unseen values (microtones, long durations, large onsets) can be handled without new lookup entries. Multidimensional Relative Attention partitions the attention heads into six groups and rotates each group by one musical attribute in the manner of RoPE, so the attention score between two events encodes their differences in onset, duration, octave, pitch, and velocity with no extra trainable parameters. The paper argues this is why Moonbeam (M) surpasses Clamp2, M3, and MusicBERT on most of the PiJAMA30, Pianist8, Emopia, and GPM30 benchmarks (M3 retains the best Emopia score by about two points), and why its finetuned generation model, using absolute onsets for full anticipatory music infilling, earns significantly higher expert ratings than the CoMMU transformer baseline.

Load-bearing premise

The load-bearing premise is that the downstream benchmarks measure generalization rather than memorization: the pretraining table (Appendix A, Table 5) includes the PiJAMA dataset, while the player-classification benchmark PiJAMA30 (Section 5.2) is drawn from PiJAMA, and the paper never states that the PiJAMA30 test pieces were left out of pretraining; the separate private test set described in Appendix A serves the perplexity evaluation, not this downstream split.

Editorial extensions

If this is right

  • A single backbone pretrained on MIDI can be finetuned into both a music classifier and a controllable generator, so the covered task families no longer require bespoke per-task tokenizers.
  • Because MRA adds no parameters, the relative-information advantage seen in earlier small music models carries to the 309M and 839M scales; in the paper's ablation, replacing MRA with standard attention raises test perplexity from 2.423 to 2.512, and removing FME raises it to 4.216.
  • Absolute onset values plus MRA give the generation model full anticipatory capability, so a control sequence can bracket a gap in time and the model can fill it while attending to both past and future control events, without an extra encoder or interleaved control tokens.
  • The expert listening test (p-values from about $10^{-12}$ to $10^{-17}$) indicates the generation advantage is perceptually real to trained listeners on chord fit, metadata fit, coherence, and overall enjoyment, even though objective pitch and velocity accuracy slightly favour the baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The decisive control the paper does not run is removing PiJAMA entirely from the pretraining corpus before measuring PiJAMA30; the reported margin over Clamp2 (0.679 vs. 0.440) is only convincing as an architectural effect once that overlap is ruled out.
  • The contrast between the objective metrics (the baseline is at or near ceiling at the widest tolerance, with pitch accuracy 0.993 and velocity accuracy 1.000, versus Moonbeam's 0.952 and 0.968) and the subjective ratings (Moonbeam clearly ahead) suggests that current 'condition adherence' metrics do not capture what expert listeners value; a follow-up study separating adherence from coherence in t
  • Because FME is a continuous embedding, the tokenizer is not locked to the autoregressive GRU decoder; it could also feed diffusion or masked generative models for symbolic music, which the paper does not test.
  • MRA's recipe of rotating head groups by per-axis coordinates transfers to other sequential domains with known geometry, such as video (time plus spatial axes) or robotics (time plus joint angles); the paper names robotics as a prospect but gives no evidence, so this remains a conjecture.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. Moonbeam is an autoregressive transformer-based symbolic music foundation model. The paper introduces a tokenizer that combines standard embeddings for instruments with the Fundamental Music Embedding (FME) for onset, duration, octave, pitch class, and velocity; Multidimensional Relative Attention (MRA), which extends RoPE to five musical axes by partitioning attention heads into groups; and a GRU sub-decoder for predicting the six sub-events of each compound token. Models of 309M and 839M parameters are pretrained on 81.6K hours of MIDI (18.06B tokens). The paper reports ablations on the small model and fine-tunes on player, emotion, and composer classification (PiJAMA30, Pianist8, Emopia, GPM30) as well as on conditional generation and music infilling using CoMMU. The main claims are that Moonbeam outperforms MusicBERT, Clamp2, and M3 on most classification tasks, and that the finetuned generation model outperforms a REMI-like transformer baseline in listening tests.

Significance. If the results are validated, this is a substantial contribution: MRA is a clean, parameter-free extension of RoPE to multi-dimensional musical positions; the FME-based tokenizer handles a wide range of MIDI formats; the paper releases code, weights, and samples; and the training-data disclosure (Table 5) is more detailed than in most comparable work. The ablation design is sensible and the MRA math in Appendix F is straightforward. However, the PiJAMA30 train/test overlap is a serious correctness risk for the headline classification claim, and the absence of variance or significance measures weakens the 'outperforms in most cases' statement. The generation result also depends heavily on the subjective listening test, whose protocol is only partially described. These issues are fixable within the scope of the manuscript, but they need to be addressed before the empirical claims can be accepted.

major comments (4)
  1. [Appendix A, Table 5; Section 5.2, Table 2] PiJAMA (217.1 h, 40.62M tokens) is listed in Moonbeam (M)'s pretraining corpus, and Table 2 evaluates on PiJAMA30, a benchmark derived from PiJAMA. The paper never states that the PiJAMA30 test pieces were excluded from pretraining; the private test set described in Appendix A is used only for pretraining perplexity, not for downstream classification. Since the largest classification margin in the paper appears on PiJAMA30 (Moonbeam (M) 0.679 vs MusicBERT 0.550 and Clamp2 0.440), this is a load-bearing data-leakage risk. Moonbeam (S), trained only on LakhMIDI, reaching 0.649 is a useful partial control, but it does not resolve the status of the Moonbeam (M) number. Please verify exclusion of all PiJAMA30 test pieces from pretraining, retrain without PiJAMA, or explicitly report the LakhMIDI-only result as the supported one.
  2. [Section 5.2, Tables 1-3] All experimental tables report a single run with no standard errors, confidence intervals, or significance tests. Several reported differences are small: on Emopia, Moonbeam (M) scores 0.693 vs M3's 0.715; on GPM30, Moonbeam (M) scores 0.648 vs MusicBERT's 0.630; on Pianist8, Moonbeam (S) and MusicBERT both score 0.811 accuracy. Without repeated seeds or a statistical test, the abstract claim that the model 'outperforms other large-scale pretrained music models in most cases' is not fully supported. Please add multiple seeds with error bars, or at minimum identify which differences exceed run-to-run variability.
  3. [Section 5.3, Tables 3-4] The objective controllability metrics in Table 3 show the baseline [24] exceeding Moonbeam on pitch and velocity accuracy (e.g., velocity accuracy at tolerance 0 is 0.862 for Moonbeam vs 0.997 for the baseline), while the subjective listening test in Table 4 strongly favors Moonbeam. As written, the generation claim rests entirely on the human evaluation, but the paper does not report inter-rater reliability, the exact randomization or blinding procedure, or whether the same conditions and samples were used for all raters. Please clarify the listening-test protocol, or soften the generation claim to reflect that objective condition-following is not better than the baseline.
  4. [Section 3.2.1 and Appendix F] The head-group assignment for MRA is internally inconsistent. In Section 3.2.1, G = 6 is said to come from the five music attributes plus the instrument attribute, but the piecewise definition of v_g(t) assigns g = 1 or g = 5 to onset, g = 2 to duration, g = 3 to octave, g = 4 to pitch class, and g = 6 to velocity, leaving no distinct group for instrument. Appendix F then states that the extra group is associated with the instrument and is assigned the onset value as its position. This makes the exact construction ambiguous and hampers reproducibility of the core architectural contribution. Please clarify which groups use which position decomposition and how the instrument attribute is represented in MRA.
minor comments (5)
  1. [Various] There are several typos: 'pretaining' in Appendix A, 'decompostion' in Section 3.2.1, 'the the lack' in Section 4.2, and 'Envirommental' in Appendix C. These should be corrected.
  2. [Table 1] The model names in Table 1 are rendered without spaces or delimiters (e.g., 'Moonbeamw/o FME', 'Moonbeamattn. variant'), which makes the table hard to read.
  3. [Section 4.2 and Appendix H] Section 4.2 states that models are compared at the piece level, while Appendix H.2 describes the preliminary PiJAMA30 experiment as clip-wise classification. Please clarify the relationship between the two evaluation protocols and why Table 8 is not part of the main results.
  4. [Table 3] The baseline row in Table 3 reports 'N/A' for test perplexity because the baseline uses a different tokenizer. A sentence explaining that perplexity is not comparable across tokenizers would help avoid misinterpretation.
  5. [Abstract and Section 6] The Github repository is mentioned in the abstract and conclusion, but no URL is given in the manuscript text; please include the link.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the FME self-citation is backed by in-paper ablations, MRA is derived from RoPE, and the benchmark claims are external comparisons; the unresolved PiJAMA30 overlap is a data-leakage risk, not a circular derivation.

full rationale

The claimed derivation chain is modular, and each load-bearing module is tested in this paper rather than assumed from a self-citation. The tokenizer adopts FME from the authors' prior work [17] (Section 3.1: 'we extend the tokenization method introduced in [17]'), but Section 5.1 Table 1 directly compares Moonbeam with FME (test PPL 2.423) against Moonbeam w/o FME with standard embeddings (test PPL 4.216), so the contribution of the self-cited component is independently verified by the paper's own controlled ablation. MRA is derived in Section 3.2.1 and Appendix F from RoPE (Equations 1-3) by grouping attention heads along onset, duration, octave, pitch-class, and velocity axes; no fitted parameter is renamed as a prediction. The downstream classification claims (Section 5.2, Table 2) are comparisons against external baselines Clamp2, M3, MusicBERT, and CRNN, and the generation claims (Section 5.3, Table 4) are human listening tests against an external baseline; none of these quantities is constructed from a fitted parameter. The paper does cite its own prior work for the PiJAMA dataset [43] and the GAPS dataset [51], but these are data resources rather than derived results. A separate concern, which is a data-contamination risk rather than circularity, is that Appendix A Table 5 lists PiJAMA (217.1 h) in Moonbeam(M)'s pretraining corpus while Section 5.2 evaluates on PiJAMA30, a subset of PiJAMA, and the paper never states that PiJAMA30 test pieces were excluded; similarly, Section 4.1 evaluates the LakhMIDI-trained Moonbeam(S) on 'one randomly selected 5% test set of the Lakh Dataset' that may not have been excluded from training. These issues could inflate specific numbers but do not make the derivation equivalent to its inputs by construction, so the circularity score remains low.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The model's central behavior depends on the rotary embedding math (RoPE), the choice of musical attributes as positional dimensions, and the hand-set theta bases; the instrument group's assignment to onset is ad hoc. No new physical entities are introduced.

free parameters (3)
  • MRA theta base values (per-dimension) = onset=199999, duration=1031, octave=19, pitch=20, velocity=131
    Hand-set rotary frequency bases for each music attribute (Table 6, Appendix D); these control how relative distances are encoded and are not learned or derived.
  • Time quantization interval and max timeshift/duration thresholds = 10ms; max 10240ms (S) / 40960ms (M)
    Appendix E: quantizing to 10ms and discarding files exceeding thresholds defines the token vocabulary and data filtering; chosen by hand.
  • Sliding window lengths for classification = 1200 (PiJAMA30), 900 (Pianist8), 130 (Emopia), 1200 (GPM30)
    Appendix H: window length empirically set to half mean sequence length; affects training segments and results.
assumptions (4)
  • standard math Rotary position embeddings preserve relative positions under inner products
    MRA derives from RoPE [22]; used in Section 3.2.1 and Appendix F.
  • domain assumption Relative musical attributes (onset, duration, octave, pitch class, velocity) are the relevant inductive biases for symbolic music modeling
    Motivated by prior perceptual and modeling work [13,17,18,19,20,21]; asserted in Section 1 without new evidence.
  • ad hoc to paper The instrument attribute can be encoded in a continuous rotary space by using onset as its position
    Appendix F assigns the extra attention head group for instrument to the onset value, an arbitrary choice for a categorical variable.
  • domain assumption Absolute onset inputs avoid the need for the model to learn arithmetic over timeshifts
    Relying on [37] (Aria-MIDI); stated in Section 3 without testing in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes." pith.science (2026). https://pith.science/paper/K6F35A2S

@misc{pith2026250515559,
  author       = {Pith},
  title        = {Pith review of: Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K6F35A2S}},
  note         = {Machine review of arXiv:2505.15559}
}
read the original abstract

Moonbeam is a transformer-based foundation model for symbolic music, pretrained on a large and diverse collection of MIDI data totaling 81.6K hours of music and 18 billion tokens. Moonbeam incorporates music-domain inductive biases by capturing both absolute and relative musical attributes through the introduction of a novel domain-knowledge-inspired tokenization method and Multidimensional Relative Attention (MRA), which captures relative music information without additional trainable parameters. Leveraging the pretrained Moonbeam, we propose 2 finetuning architectures with full anticipatory capabilities, targeting 2 categories of downstream tasks: symbolic music understanding and conditional music generation (including music infilling). Our model outperforms other large-scale pretrained music models in most cases in terms of accuracy and F1 score across 3 downstream music classification tasks on 4 datasets. Moreover, our finetuned conditional music generation model outperforms a strong transformer baseline with a REMI-like tokenizer. We open-source the code, pretrained model, and generated samples on Github.

Figures

Figures reproduced from arXiv: 2505.15559 by the authors.

Figure 1
Figure 1. Moonbeam Model Architecture. recently, we have seen an increasing trend to represent symbolic music in ABC format as text [1, 2], modeled directly with LLMs. The transformer backbones used in symbolic music modeling can be broadly classified into two categories: the standard transformer decoder [28–30] and its variants (e.g., Transformer-XL [31] and linear transformer [32]), as used in [7, 8, 10, 14, 16, 26]; and th… view at source ↗
Figure 2
Figure 2. Moonbeam Finetuning Architecture. 3.2.2 GRU Decoder and the Training Objective At each time step, the transformer outputs a single tensor, which represents a latent representation for generating the next event at time step t. To decode all attribute tokens, this tensor is used as the initial hidden state of a GRU, which then sequentially decodes all 6 attributes: (∆ot, dt, octt, pt, it, vt). The main reason for deco… view at source ↗
Figure 3
Figure 3. Distribution of the GPM30 dataset. H.3 Construction of the GPM30 Dataset for Composer Classification Following [43], we construct a smaller subset of the Giant Piano MIDI (GPM) dataset [45] for our composer classification task. We first filter the GPM dataset and only keep the data with a sequence length shorter than 4096. We then sort the filtered dataset by the number of pieces per composer and select the top 30 c… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Two questions designed to filter for qualified participants during the listening test. [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: Listening test interface. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Equivariant Music Transformer

    cs.SD 2026-08 conditional novelty 6.0 of 10

    Adding an equivariance regularizer to an autoregressive music transformer makes pitch-transposed and time-shifted inputs produce correspondingly shifted outputs while slightly improving next-token prediction.

  2. Amadeus: Autoregressive Model with Bidirectional Attribute Modelling for Symbolic Music

    cs.SD 2025-08 conditional novelty 6.0 of 10

    Amadeus generates symbolic music by autoregressively predicting note-level latents and decoding their attributes in parallel with a masked discrete diffusion model, yielding faster and more controllable generation tha...

Reference graph

Works this paper leans on

61 extracted references · 58 canonical work pages · cited by 2 Pith papers

  1. [24]

    Commu: Dataset for combinatorial music generation,

    H. Lee, T. Kim, H. Kang, M. Ki, H. Hwang, K. Park, S. Han, and S. J. Kim, “Commu: Dataset for combinatorial music generation,” in Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022

    work page 2022

  2. [1]

    Chatmusician: Understanding and generating music intrinsically with LLM,

    R. Yuan, H. Lin, Y . Wang, Z. Tian, S. Wu, T. Shen, G. Zhang, Y . Wu, C. Liu, Z. Zhou, L. Xue, Z. Ma, Q. Liu, T. Zheng, Y . Li, Y . Ma, Y . Liang, X. Chi, R. Liu, Z. Wang, C. Lin, Q. Liu, T. Jiang, W. Huang, W. Chen, J. Fu, E. Benetos, G. Xia, R. B. Dannenberg, W. Xue, S. Kang, and Y . Guo, “Chatmusician: Understanding and generating music intrinsically w...

    work page 2024

  3. [2]

    Mupt: A generative symbolic music pretrained transformer,

    X. Qu, Y . Bai, Y . Ma, Z. Zhou, K. M. Lo, J. Liu, R. Yuan, L. Min, X. Liu, T. Zhang, X. Du, S. Guo, Y . Liang, Y . Li, S. Wu, J. Zhou, T. Zheng, Z. Ma, F. Han, W. Xue, G. Xia, E. Benetos, X. Yue, C. Lin, X. Tan, S. W. Huang, W. Chen, J. Fu, and G. Zhang, “Mupt: A generative symbolic music pretrained transformer,” inThe Thirteenth International Conference...

    work page 2025

  4. [3]

    BERT: pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Shor...

    work page 2019

  5. [4]

    Clamp: Contrastive language-music pre-training for cross-modal symbolic music information retrieval,

    S. Wu, D. Yu, X. Tan, and M. Sun, “Clamp: Contrastive language-music pre-training for cross-modal symbolic music information retrieval,” inProceedings of the 24th International Society for Music Information Retrieval Conference, ISMIR 2023, Milan, Italy, November 5-9, 2023, pp. 157–165

    work page 2023

  6. [5]

    CLaMP 2: Multimodal music information retrieval across 101 languages using large language models,

    S. Wu, Y . Wang, R. Yuan, G. Zhancheng, X. Tan, G. Zhang, M. Zhou, J. Chen, X. Mu, Y . Gao, Y . Dong, J. Liu, X. Li, F. Yu, and M. Sun, “CLaMP 2: Multimodal music information retrieval across 101 languages using large language models,” in Findings of the Association for Computational Linguistics: NAACL 2025. Albuquerque, New Mexico: Association for Comput...

    work page 2025

  7. [6]

    Musicbert: Symbolic music understanding with large-scale pre-training,

    M. Zeng, X. Tan, R. Wang, Z. Ju, T. Qin, and T. Liu, “Musicbert: Symbolic music understanding with large-scale pre-training,” in Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, ser. Findings of ACL, vol. ACL/IJCNLP

    work page 2021

  8. [7]

    Pop music transformer: Beat-based modeling and generation of expressive pop piano compositions,

    Y .-S. Huang and Y .-H. Yang, “Pop music transformer: Beat-based modeling and generation of expressive pop piano compositions,” in Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 1180–1188

    work page 2020

Show all 61 references
  1. [8]

    Compound word transformer: Learning to compose full- song music over dynamic directed hypergraphs,

    W. Hsiao, J. Liu, Y . Yeh, and Y . Yang, “Compound word transformer: Learning to compose full- song music over dynamic directed hypergraphs,” in Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Event, February 2-9, 2021, pp. 178–186

    2021

  2. [9]

    Popmag: Pop music accompaniment generation,

    Y . Ren, J. He, X. Tan, T. Qin, Z. Zhao, and T. Liu, “Popmag: Pop music accompaniment generation,” in MM ’20: The 28th ACM International Conference on Multimedia, Virtual Event / Seattle, WA, USA, October 12-16, 2020. ACM, 2020, pp. 1198–1206

    2020

  3. [10]

    Multitrack music transformer,

    H. Dong, K. Chen, S. Dubnov, J. J. McAuley, and T. Berg-Kirkpatrick, “Multitrack music transformer,” in IEEE International Conference on Acoustics, Speech and Signal Processing ICASSP 2023, Rhodes Island, Greece, June 4-10, 2023, pp. 1–5

    2023

  4. [11]

    MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Classification Tasks,

    Y . Chou, I. Chen, J. Ching, C. Chang, and Y . Yang, “MidiBERT-Piano: Large-scale Pre-training for Symbolic Music Classification Tasks,” Journal of Creative Music Systems, vol. 8, no. 1, 2024

    2024

  5. [12]

    This time with feeling: learning expressive musical performance,

    S. Oore, I. Simon, S. Dieleman, D. Eck, and K. Simonyan, “This time with feeling: learning expressive musical performance,”Neural Comput. Appl., vol. 32, no. 4, pp. 955–967, 2020

    2020

  6. [13]

    Music transformer: Generating music with long-term structure,

    C. A. Huang, A. Vaswani, J. Uszkoreit, I. Simon, C. Hawthorne, N. Shazeer, A. M. Dai, M. D. Hoffman, M. Dinculescu, and D. Eck, “Music transformer: Generating music with long-term structure,” in 7th International Conference on Learning Representations, ICLR 2019, New Orleans, ...

    2019

  7. [14]

    Anticipatory music transformer,

    J. Thickstun, D. L. W. Hall, C. Donahue, and P. Liang, “Anticipatory music transformer,”Trans. Mach. Learn. Res., vol. 2024, 2024. 10

    2024

  8. [15]

    Symphony generation with permutation invariant language model,

    J. Liu, Y . Dong, Z. Cheng, X. Zhang, X. Li, F. Yu, and M. Sun, “Symphony generation with permutation invariant language model,” in Proceedings of the 23rd International Society for Music Information Retrieval Conference, ISMIR 2022, Bengaluru, India, December 4-8, 2022, pp. 551–558

    2022

  9. [16]

    Byte pair encoding for symbolic music,

    N. Fradet, N. Gutowski, F. Chhel, and J. Briot, “Byte pair encoding for symbolic music,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023. Association for Computational Linguistics, 2023, pp. ...

    2023

  10. [17]

    A domain-knowledge-inspired music embedding space and a novel attention mechanism for symbolic music modeling,

    Z. Guo, J. Kang, and D. Herremans, “A domain-knowledge-inspired music embedding space and a novel attention mechanism for symbolic music modeling,” in Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Washington, DC, USA, February 7-14, 2023, pp. 5070–5077

    2023

  11. [18]

    Learning transposition-invariant interval features from symbolic music and audio,

    S. Lattner, M. Grachten, and G. Widmer, “Learning transposition-invariant interval features from symbolic music and audio,” in Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, September 23-27, 2018

    2018

  12. [19]

    A predictive model for music based on learned interval representations,

    ——, “A predictive model for music based on learned interval representations,” inProceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, September 23-27, 2018, pp. 26–33

    2018

  13. [20]

    Relative positional encoding for transformers with linear complexity,

    A. Liutkus, O. Cífka, S. Wu, U. Simsekli, Y . Yang, and G. Richard, “Relative positional encoding for transformers with linear complexity,” in Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, ser. Proceedings of M...

    2021

  14. [21]

    Self-attention with relative position representations,

    P. Shaw, J. Uszkoreit, and A. Vaswani, “Self-attention with relative position representations,” in Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, New Orleans, Louisiana,...

    2018

  15. [22]

    Roformer: Enhanced transformer with rotary position embedding,

    J. Su, M. H. M. Ahmed, Y . Lu, S. Pan, W. Bo, and Y . Liu, “Roformer: Enhanced transformer with rotary position embedding,” Neurocomputing, vol. 568, p. 127063, 2024

    2024

  16. [23]

    Musiac: An extensible generative framework for music infilling applications with multi-level control,

    R. Guo, I. Simpson, C. Kiefer, T. Magnusson, and D. Herremans, “Musiac: An extensible generative framework for music infilling applications with multi-level control,” in Artificial Intelligence in Music, Sound, Art and Design - 11th International Conference, EvoMUSART 2022, He...

    2022

  17. [25]

    Impact of time and note duration tokenizations on deep learning symbolic music modeling,

    N. Fradet, N. Gutowski, F. Chhel, and J. Briot, “Impact of time and note duration tokenizations on deep learning symbolic music modeling,” in Proceedings of the 24th International Society for Music Information Retrieval Conference, ISMIR 2023, Milan, Italy, November 5-9, 2023,...

    2023

  18. [26]

    FIGARO: controllable music generation using learned and expert features,

    D. von Rütte, L. Biggio, Y . Kilcher, and T. Hofmann, “FIGARO: controllable music generation using learned and expert features,” in The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023

    2023

  19. [27]

    Nested music transformer: Sequentially decoding com- pound tokens in symbolic music and audio generation,

    J. Ryu, H. Dong, J. Jung, and D. Jeong, “Nested music transformer: Sequentially decoding com- pound tokens in symbolic music and audio generation,” inProceedings of the 25th International Society for Music Information Retrieval Conference, ISMIR 2024, San Francisco, California...

    2024

  20. [28]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long...

    2017

  21. [29]

    Llama: Open and efficient foundation language model,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample, “Llama: Open and efficient foundation language model,” CoRR, vol. abs/2302.13971, 2023. [Online]. Available: htt...

    2023 doi arXiv

  22. [30]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019, openAI Blog. [Online]. Available: https://cdn.openai. com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf

    2019

  23. [31]

    Transformer- xl: Attentive language models beyond a fixed-length context,

    Z. Dai, Z. Yang, Y . Yang, J. G. Carbonell, Q. V . Le, and R. Salakhutdinov, “Transformer- xl: Attentive language models beyond a fixed-length context,” in Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- ...

    2019

  24. [32]

    Transformers are rnns: Fast autoregres- sive transformers with linear attention,

    A. Katharopoulos, A. Vyas, N. Pappas, and F. Fleuret, “Transformers are rnns: Fast autoregres- sive transformers with linear attention,” in Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, ser. Proceedings of Mach...

    2020

  25. [33]

    Lakhnes: Improving multi-instrumental music generation with cross-domain pre-training,

    C. Donahue, H. H. Mao, Y . E. Li, G. W. Cottrell, and J. McAuley, “Lakhnes: Improving multi-instrumental music generation with cross-domain pre-training,” in ISMIR, 2019

    2019

  26. [34]

    MIDI-GPT: A controllable generative model for computer-assisted multitrack music composition,

    P. Pasquier, J. Ens, N. Fradet, P. Triana, D. Rizzotti, J. Rolland, and M. Safi, “MIDI-GPT: A controllable generative model for computer-assisted multitrack music composition,” inAAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 ...

    2025

  27. [35]

    The gigamidi dataset with features for expressive music performance detection,

    K. J. M. Lee, J. Ens, S. Adkins, P. Sarmento, M. Barthet, and P. Pasquier, “The gigamidi dataset with features for expressive music performance detection,” Trans. Int. Soc. Music. Inf. Retr., vol. 8, no. 1, 2025

    2025

  28. [36]

    Learning-based methods for comparing sequences, with applications to audio-to-midi alignment and matching,

    C. Raffel, “Learning-based methods for comparing sequences, with applications to audio-to-midi alignment and matching,” Ph.D. dissertation, Columbia University, USA, 2016

    2016

  29. [37]

    Aria-MIDI: A dataset of piano MIDI files for symbolic music modeling,

    L. Bradshaw and S. Colton, “Aria-MIDI: A dataset of piano MIDI files for symbolic music modeling,” in The Thirteenth International Conference on Learning Representations, 2025

    2025

  30. [38]

    Mustango: Toward controllable text-to-music generation,

    J. Melechovský, Z. Guo, D. Ghosal, N. Majumder, D. Herremans, and S. Poria, “Mustango: Toward controllable text-to-music generation,” inProceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologie...

    2024

  31. [39]

    GQA: training generalized multi-query transformer models from multi-head checkpoints,

    J. Ainslie, J. Lee-Thorp, M. de Jong, Y . Zemlyanskiy, F. Lebrón, and S. Sanghai, “GQA: training generalized multi-query transformer models from multi-head checkpoints,” in Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Sing...

    2023

  32. [40]

    MMM : Exploring conditional multi-track music generation with the transformer,

    J. Ens and P. Pasquier, “MMM : Exploring conditional multi-track music generation with the transformer,” CoRR, vol. abs/2008.06048, 2020. [Online]. Available: https: //arxiv.org/abs/2008.06048

    2008 arXiv

  33. [41]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” in The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022

    2022

  34. [42]

    Mixed precision training,

    P. Micikevicius, S. Narang, J. Alben, G. F. Diamos, E. Elsen, D. García, B. Ginsburg, M. Hous- ton, O. Kuchaiev, G. Venkatesh, and H. Wu, “Mixed precision training,” in6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3,...

    2018

  35. [43]

    Pijama: Piano jazz with automatic MIDI annotations,

    D. Edwards, S. Dixon, and E. Benetos, “Pijama: Piano jazz with automatic MIDI annotations,” Trans. Int. Soc. Music. Inf. Retr., vol. 6, no. 1, pp. 89–102, 2023

    2023

  36. [44]

    EMOPIA: A multi-modal pop piano dataset for emotion recognition and emotion-based music generation,

    H. Hung, J. Ching, S. Doh, N. Kim, J. Nam, and Y . Yang, “EMOPIA: A multi-modal pop piano dataset for emotion recognition and emotion-based music generation,” in Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR 2021, Online, Novem...

    2021

  37. [45]

    Giantmidi-piano: A large-scale MIDI dataset for classical piano music,

    Q. Kong, B. Li, J. Chen, and Y . Wang, “Giantmidi-piano: A large-scale MIDI dataset for classical piano music,” Trans. Int. Soc. Music. Inf. Retr., vol. 5, no. 1, pp. 87–98, 2022

    2022

  38. [46]

    ASAP: a dataset of aligned scores and performances for piano transcription,

    F. Foscarin, A. McLeod, P. Rigaux, F. Jacquemard, and M. Sakai, “ASAP: a dataset of aligned scores and performances for piano transcription,” in Proceedings of the 21th International Society for Music Information Retrieval Conference, ISMIR 2020, Montreal, Canada, October 11-1...

    2020

  39. [47]

    ATEPP: A dataset of automatically transcribed expressive piano performance,

    H. Zhang, J. Tang, S. R. Rafee, S. Dixon, G. Fazekas, and G. A. Wiggins, “ATEPP: A dataset of automatically transcribed expressive piano performance,” inProceedings of the 23rd Inter- national Society for Music Information Retrieval Conference, ISMIR 2022, Bengaluru, India, De...

    2022

  40. [48]

    Dadagp: A dataset of tokenized guitarpro songs for sequence models,

    P. Sarmento, A. Kumar, C. J. Carr, Z. Zukowski, M. Barthet, and Y . Yang, “Dadagp: A dataset of tokenized guitarpro songs for sequence models,” in Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR 2021, Online, November 7-12, 2021,...

    2021

  41. [49]

    Filobass: A dataset and corpus based study of jazz basslines,

    X. Riley and S. Dixon, “Filobass: A dataset and corpus based study of jazz basslines,” in Proceedings of the 24th International Society for Music Information Retrieval Conference, ISMIR 2023, Milan, Italy, November 5-9, 2023, pp. 500–507

    2023

  42. [50]

    Filosax: A dataset of annotated jazz saxophone recordings,

    D. Foster and S. Dixon, “Filosax: A dataset of annotated jazz saxophone recordings,” in Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR 2021, Online, November 7-12, 2021, pp. 205–212

    2021

  43. [51]

    GAPS: A large and diverse classical guitar dataset and benchmark transcription model,

    X. Riley, Z. Guo, A. C. Edwards, and S. Dixon, “GAPS: A large and diverse classical guitar dataset and benchmark transcription model,” in Proceedings of the 25th International Society for Music Information Retrieval Conference, ISMIR 2024, San Francisco, California, USA and On...

    2024

  44. [52]

    Learning to groove with inverse sequence transformations,

    J. Gillick, A. Roberts, J. Engel, D. Eck, and D. Bamman, “Learning to groove with inverse sequence transformations,” in International Conference on Machine Learning (ICML), 2019

    2019

  45. [53]

    Guitarset: A dataset for guitar transcription,

    Q. Xi, R. M. Bittner, J. Pauwels, X. Ye, and J. P. Bello, “Guitarset: A dataset for guitar transcription,” in Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR 2018, Paris, France, September 23-27, 2018, pp. 453–460

    2018

  46. [54]

    Enabling factorized piano music modeling and generation with the MAESTRO dataset,

    C. Hawthorne, A. Stasyuk, A. Roberts, I. Simon, C.-Z. A. Huang, S. Dieleman, E. Elsen, J. Engel, and D. Eck, “Enabling factorized piano music modeling and generation with the MAESTRO dataset,” in International Conference on Learning Representations, 2019

    2019

  47. [55]

    MAPS - a piano database for multipitch estimation and automatic transcription of music,

    V . Emiya, N. Bertin, B. David, and R. Badeau, “MAPS - a piano database for multipitch estimation and automatic transcription of music,” INRIA, France, Research Report, Jul. 2010

    2010

  48. [56]

    Building the metamidi dataset: Linking symbolic and audio musical data,

    J. Ens and P. Pasquier, “Building the metamidi dataset: Linking symbolic and audio musical data,” in Proceedings of the 22nd International Society for Music Information Retrieval Conference, ISMIR 2021, Online, November 7-12, 2021, pp. 182–188

    2021

  49. [57]

    Learning features of music from scratch,

    J. Thickstun, Z. Harchaoui, and S. M. Kakade, “Learning features of music from scratch,” in 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings, 2017

    2017

  50. [58]

    Supra: Digitizing the stanford university piano roll archive

    Z. Shi, C. S. Sapp, K. Arul, J. McBride, and J. O. Smith, “Supra: Digitizing the stanford university piano roll archive.” in Proceedings of the 20th International Society for Music Information Retrieval, Delft, The Netherlands, 2019, pp. 517–523

    2019

  51. [59]

    Creating a multitrack classical music performance dataset for multimodal music analysis: Challenges, insights, and applications,

    B. Li, X. Liu, K. Dinesh, Z. Duan, and G. Sharma, “Creating a multitrack classical music performance dataset for multimodal music analysis: Challenges, insights, and applications,” IEEE Transactions on Multimedia, vol. 21, no. 2, pp. 522–535, 2019

    2019

  52. [60]

    compres- sion

    M. Pfleiderer, K. Frieler, J. Abeßer, W.-G. Zaddach, and B. Burkhart, Eds.,Inside the Jazzomat - New Perspectives for Jazz Research. Schott Campus, 2017. Appendix A Training and Test Data used for Pretraining A summary of the training data used for pretaining can be found in T...

    2017

  53. [2021]

    Association for Computational Linguistics, 2021, pp. 791–800

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.