{"id":"9d70677a-ea9a-4d8e-a2e5-657b447082c2","arxiv_id":"2505.15559","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Moonbeam is a MIDI foundation model that combines a note-value tokenizer with multidimensional relative attention, and it outperforms prior large music models on most tested classification and generation tasks.","lead":"A new AI model called Moonbeam, pretrained on 81,600 hours of MIDI music, scores higher than earlier music AI models on several classification and generation tests. It introduces a way for the model to notice both exact note values and the musical intervals between notes, without adding extra parameters.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PiJAMA30 test-set overlap with pretraining is unresolved; Moonbeam(M)'s largest classification margin may be inflated, though Moonbeam(S)'s LakhMIDI-only training provides a partial control.","rationale":"I agree with the reader that train/evaluation overlap is the most load-bearing threat to the central empirical claim. The paper's own Table 5 and Section 5.2 create a prima facie overlap, and no exclusion statement is made. I rate it as a concern rather than a fatal flaw because Table 6 indicates Moonbeam(S) was trained only on LakhMIDI, so S's 0.649 on PiJAMA30 is an uncontrolled but suggestive control: an architecture trained on a disjoint corpus still outperforms prior models on this benchmark. This means the leakage, if present, likely inflates M's margin by only the 0.03 gap over S rather than explaining the full SOTA gap. Still, the flagship M result and the paper's 'most cases' phrasing depend on the unresolved overlap. I also noticed a secondary inconsistency: the abstract and Section 1 say Moonbeam was pretrained on the 81.6K-hour corpus, while Table 6 says Moonbeam(S)'s training data was LakhMIDI only; this does not change the verdict but should be fixed. The reader's CONDITIONAL verdict remains appropriate: the authors should verify no test overlap, report variance, and release code and hashes.","tokens_in":19514,"tokens_out":8616,"duration_ms":72516,"concrete_test":"Compute overlap between the PiJAMA30 test split and the pretraining corpus by SHA-256 hashes of MIDI files (or, where files are re-rendered, melodic fingerprinting such as pitch-interval n-grams), and report the number of test pieces present in pretraining. If overlap is non-zero, re-run Table 2 with those pieces removed from both pretraining and evaluation, or at minimum report Table 2 excluding PiJAMA30 and show whether Moonbeam(M) still outperforms baselines in most cases. If overlap is zero, the leakage concern is resolved and the paper should state this explicitly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Appendix A Table 5 lists PiJAMA (217.1 h, 40.62M tokens) in Moonbeam(M)'s pretraining corpus. Section 5.2 Table 2 evaluates on PiJAMA30, a benchmark derived from PiJAMA, and the paper never states that PiJAMA30 test pieces were excluded from pretraining. If they were included, the largest margin in the paper—Moonbeam(M) 0.679 accuracy vs Clamp2 0.440 and MusicBERT 0.550 on PiJAMA30—could reflect memorized test pieces rather than generalizable representations. This directly threatens the headline 'outperforms in most cases'. Notably, Table 6 shows Moonbeam(S) was pretrained only on LakhMIDI, yet S reaches 0.649 on PiJAMA30, close to M's 0.679; this is a partial no-leakage control but does not resolve the status of M's specific number. The authors' own Appendix A says they constructed a private test set 'to minimize overlap' only for pretraining perplexity, not for downstream classification data.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"Moonbeam is an autoregressive transformer-based symbolic music foundation model. The paper introduces a tokenizer that combines standard embeddings for instruments with the Fundamental Music Embedding (FME) for onset, duration, octave, pitch class, and velocity; Multidimensional Relative Attention (MRA), which extends RoPE to five musical axes by partitioning attention heads into groups; and a GRU sub-decoder for predicting the six sub-events of each compound token. Models of 309M and 839M parameters are pretrained on 81.6K hours of MIDI (18.06B tokens). The paper reports ablations on the small model and fine-tunes on player, emotion, and composer classification (PiJAMA30, Pianist8, Emopia, GPM30) as well as on conditional generation and music infilling using CoMMU. The main claims are that Moonbeam outperforms MusicBERT, Clamp2, and M3 on most classification tasks, and that the finetuned generation model outperforms a REMI-like transformer baseline in listening tests.","tokens_in":19629,"tokens_out":6792,"duration_ms":60703,"significance":"If the results are validated, this is a substantial contribution: MRA is a clean, parameter-free extension of RoPE to multi-dimensional musical positions; the FME-based tokenizer handles a wide range of MIDI formats; the paper releases code, weights, and samples; and the training-data disclosure (Table 5) is more detailed than in most comparable work. The ablation design is sensible and the MRA math in Appendix F is straightforward. However, the PiJAMA30 train/test overlap is a serious correctness risk for the headline classification claim, and the absence of variance or significance measures weakens the 'outperforms in most cases' statement. The generation result also depends heavily on the subjective listening test, whose protocol is only partially described. These issues are fixable within the scope of the manuscript, but they need to be addressed before the empirical claims can be accepted.","major_comments":[{"comment":"PiJAMA (217.1 h, 40.62M tokens) is listed in Moonbeam (M)'s pretraining corpus, and Table 2 evaluates on PiJAMA30, a benchmark derived from PiJAMA. The paper never states that the PiJAMA30 test pieces were excluded from pretraining; the private test set described in Appendix A is used only for pretraining perplexity, not for downstream classification. Since the largest classification margin in the paper appears on PiJAMA30 (Moonbeam (M) 0.679 vs MusicBERT 0.550 and Clamp2 0.440), this is a load-bearing data-leakage risk. Moonbeam (S), trained only on LakhMIDI, reaching 0.649 is a useful partial control, but it does not resolve the status of the Moonbeam (M) number. Please verify exclusion of all PiJAMA30 test pieces from pretraining, retrain without PiJAMA, or explicitly report the LakhMIDI-only result as the supported one.","section":"Appendix A, Table 5; Section 5.2, Table 2"},{"comment":"All experimental tables report a single run with no standard errors, confidence intervals, or significance tests. Several reported differences are small: on Emopia, Moonbeam (M) scores 0.693 vs M3's 0.715; on GPM30, Moonbeam (M) scores 0.648 vs MusicBERT's 0.630; on Pianist8, Moonbeam (S) and MusicBERT both score 0.811 accuracy. Without repeated seeds or a statistical test, the abstract claim that the model 'outperforms other large-scale pretrained music models in most cases' is not fully supported. Please add multiple seeds with error bars, or at minimum identify which differences exceed run-to-run variability.","section":"Section 5.2, Tables 1-3"},{"comment":"The objective controllability metrics in Table 3 show the baseline [24] exceeding Moonbeam on pitch and velocity accuracy (e.g., velocity accuracy at tolerance 0 is 0.862 for Moonbeam vs 0.997 for the baseline), while the subjective listening test in Table 4 strongly favors Moonbeam. As written, the generation claim rests entirely on the human evaluation, but the paper does not report inter-rater reliability, the exact randomization or blinding procedure, or whether the same conditions and samples were used for all raters. Please clarify the listening-test protocol, or soften the generation claim to reflect that objective condition-following is not better than the baseline.","section":"Section 5.3, Tables 3-4"},{"comment":"The head-group assignment for MRA is internally inconsistent. In Section 3.2.1, G = 6 is said to come from the five music attributes plus the instrument attribute, but the piecewise definition of v_g(t) assigns g = 1 or g = 5 to onset, g = 2 to duration, g = 3 to octave, g = 4 to pitch class, and g = 6 to velocity, leaving no distinct group for instrument. Appendix F then states that the extra group is associated with the instrument and is assigned the onset value as its position. This makes the exact construction ambiguous and hampers reproducibility of the core architectural contribution. Please clarify which groups use which position decomposition and how the instrument attribute is represented in MRA.","section":"Section 3.2.1 and Appendix F"}],"minor_comments":[{"comment":"There are several typos: 'pretaining' in Appendix A, 'decompostion' in Section 3.2.1, 'the the lack' in Section 4.2, and 'Envirommental' in Appendix C. These should be corrected.","section":"Various"},{"comment":"The model names in Table 1 are rendered without spaces or delimiters (e.g., 'Moonbeamw/o FME', 'Moonbeamattn. variant'), which makes the table hard to read.","section":"Table 1"},{"comment":"Section 4.2 states that models are compared at the piece level, while Appendix H.2 describes the preliminary PiJAMA30 experiment as clip-wise classification. Please clarify the relationship between the two evaluation protocols and why Table 8 is not part of the main results.","section":"Section 4.2 and Appendix H"},{"comment":"The baseline row in Table 3 reports 'N/A' for test perplexity because the baseline uses a different tokenizer. A sentence explaining that perplexity is not comparable across tokenizers would help avoid misinterpretation.","section":"Table 3"},{"comment":"The Github repository is mentioned in the abstract and conclusion, but no URL is given in the manuscript text; please include the link.","section":"Abstract and Section 6"}],"recommendation":"major_revision","confidential_remarks":"The data-leakage concern is real and should be resolved before publication; because the fix (explicit exclusion of PiJAMA30 test pieces from pretraining, or retraining without PiJAMA) is within the manuscript's scope, I recommend major_revision rather than reject. The single-run issue is also important for a paper whose headline is comparative. The MRA specification inconsistency should be corrected even if the implementation is as claimed. I see no evidence of deliberate misrepresentation; the paper is transparent about its training data, which is why the overlap is detectable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [colleague],\n\nThis one is worth a careful look but not a clean pass. The tokenizer and MRA are the real contributions; the headline classification numbers have a data-leakage problem the paper never resolves.\n\nWhat's genuinely new: MRA is RoPE rotated by musical attributes (onset, duration, octave, pitch class, velocity) rather than sequence index, with head groups assigned per attribute. The derivation is clean and parameter-free. The tokenizer extends FME to polyphonic multi-instrument MIDI, keeps sequences compact, and beats standard embeddings in perplexity. The ablations on Moonbeam(S) are nicely controlled: removing FME, MRA, or the GRU decoder each hurts, and a summed-attribute MRA variant is worse than the per-attribute grouping. Training 839M parameters on 18B tokens with two A100s is a real efficiency win. The listening test is solid—ethics approval, 20 experts, Wilcoxon p-values.\n\nBut the PiJAMA30 overlap is a real problem. Table 5 puts PiJAMA in the pretraining corpus; Table 2 evaluates on PiJAMA30, and the paper never states the test pieces were held out. This is the largest margin in the paper (Moonbeam(M) 0.679 vs Clamp2 0.440, MusicBERT 0.550), so memorization is a live possibility. The saving grace is Moonbeam(S), trained only on LakhMIDI, scoring 0.649 on PiJAMA30—that is a partial no-leakage control. But S's near-parity also cuts against the value of the giant corpus for this task. The claim 'outperforms most cases' is shakier than the abstract suggests.\n\nOther soft spots: Tables 1 and 2 are single runs with no error bars. The paper promises open-source code but no URL or commit hash appears. There is a minor inconsistency between 'one randomly selected 5% test set of Lakh' and the private test set described in Appendix A. All fixable.\n\nMy take: this deserves a serious referee. I'd send it to peer review with a request to re-run PiJAMA30 after explicit overlap removal, report variance, and release code with a hash. If the PiJAMA numbers don't survive, the paper is still a decent contribution on its architecture and ablations, just not a 'beats everything' one.","headline":"Solid architecture paper undercut by an unresolved PiJAMA30 pretraining-overlap question.","tokens_in":20286,"tokens_out":3228,"would_cite":true,"duration_ms":28485,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Moonbeam, a transformer pretrained on 81.6K hours of MIDI, encodes notes with absolute and relative musical attributes in five dimensions, and beats larger pretrained music models on most classification benchmarks.","keywords":["symbolic music","MIDI","foundation model","tokenization","multidimensional relative attention","music classification","conditional music generation","music infilling"],"falsifier":"Match the PiJAMA30 test pieces against the pretraining files listed in Table 5 by MIDI file identity or near-duplicate sequence comparison, then re-evaluate (or retrain) Moonbeam with all PiJAMA pieces removed from the corpus. If PiJAMA30 accuracy falls from 0.679 toward Clamp2's 0.440, the reported margin is substantially memorization; if the margin holds under this holdout, the architectural claim survives this attack.","tokens_in":19161,"feed_emoji":"🎼","tokens_out":19424,"duration_ms":154202,"temperature":0.7,"pith_summary":"The paper sets out to establish that a single autoregressive transformer, pretrained directly on MIDI rather than on ABC text scores, can serve as a general foundation model for symbolic music. Its central move is to represent each note event by both its absolute values (onset time, duration, octave, pitch class, instrument, velocity) and its relative relationships, which is how musicians perceive music; a novel tokenizer with continuous sinusoidal embeddings and a Multidimensional Relative Attention mechanism carry this through without added parameters. On three classification tasks across four datasets, the 309M- and 839M-parameter models beat the larger pretrained symbolic music models in most cases, and the finetuned conditional generation model is rated by expert listeners as fitting the given chord and metadata conditions better than a strong REMI-style transformer baseline, despite scoring slightly lower on objective pitch and velocity accuracy. If these findings hold, they show that music-specific inductive biases scale to hundreds of millions of parameters and that expressive, multi-instrument MIDI data can serve as a usable pretraining substrate.","feed_headline":"MIDI foundation model beats larger music AIs on most tasks","feed_subtitle":"Pretrained on 81.6K hours of MIDI, it ranks notes in five musical dimensions and wins most downstream tests.","key_machinery":"The load-bearing machinery has three parts. First, the tokenizer: each note event becomes the compound token $x = (o, d, oct, p, i, v)$ — absolute onset, duration, octave, pitch class, instrument, velocity — with all attributes except the instrument encoded by the continuous sinusoidal Fundamental Music Embedding (FME), which preserves relative musical information in the embedding space and extrapolates to unseen inputs without extra trainable parameters. Second, Multidimensional Relative Attention (MRA): an extension of RoPE in which the attention heads are partitioned into $G = 6$ groups and each group is rotated by the value of one musical attribute (onset, duration, octave, pitch class, velocity; the instrument group reuses onset), so the query-key dot product encodes the relative difference along each axis, parameter-free. Third, a GRU sub-decoder that sequentially samples the six sub-tokens of each event, capturing dependencies such as a high pitch making a bass instrument unlikely. For finetuning, classification appends a $\\langle cls\\rangle$ token and swaps the GRU for a linear head with LoRA, while conditional generation prepends metadata and chord-control sequences to the input so that, with absolute onsets and MRA, the model can attend to the entire control sequence, including future control events, giving it full anticipatory capability for music infilling.","core_discovery":"On the paper's own terms, the discovery is that Moonbeam is the first autoregressive transformer foundation model pretrained at scale on diverse MIDI — 81.6K hours and 18 billion tokens covering piano performance, scores, guitar, bass, drums, saxophone, and multitrack music — and that its gains come from jointly preserving absolute and relative musical information. Each event is a six-part compound token (absolute onset, duration, octave, pitch class, instrument, velocity), embedded with the continuous Fundamental Music Embedding so relative information survives in the embedding space and unseen values (microtones, long durations, large onsets) can be handled without new lookup entries. Multidimensional Relative Attention partitions the attention heads into six groups and rotates each group by one musical attribute in the manner of RoPE, so the attention score between two events encodes their differences in onset, duration, octave, pitch, and velocity with no extra trainable parameters. The paper argues this is why Moonbeam (M) surpasses Clamp2, M3, and MusicBERT on most of the PiJAMA30, Pianist8, Emopia, and GPM30 benchmarks (M3 retains the best Emopia score by about two points), and why its finetuned generation model, using absolute onsets for full anticipatory music infilling, earns significantly higher expert ratings than the CoMMU transformer baseline.","pith_inferences":["The decisive control the paper does not run is removing PiJAMA entirely from the pretraining corpus before measuring PiJAMA30; the reported margin over Clamp2 (0.679 vs. 0.440) is only convincing as an architectural effect once that overlap is ruled out.","The contrast between the objective metrics (the baseline is at or near ceiling at the widest tolerance, with pitch accuracy 0.993 and velocity accuracy 1.000, versus Moonbeam's 0.952 and 0.968) and the subjective ratings (Moonbeam clearly ahead) suggests that current 'condition adherence' metrics do not capture what expert listeners value; a follow-up study separating adherence from coherence in t","Because FME is a continuous embedding, the tokenizer is not locked to the autoregressive GRU decoder; it could also feed diffusion or masked generative models for symbolic music, which the paper does not test.","MRA's recipe of rotating head groups by per-axis coordinates transfers to other sequential domains with known geometry, such as video (time plus spatial axes) or robotics (time plus joint angles); the paper names robotics as a prospect but gives no evidence, so this remains a conjecture."],"forward_implications":["A single backbone pretrained on MIDI can be finetuned into both a music classifier and a controllable generator, so the covered task families no longer require bespoke per-task tokenizers.","Because MRA adds no parameters, the relative-information advantage seen in earlier small music models carries to the 309M and 839M scales; in the paper's ablation, replacing MRA with standard attention raises test perplexity from 2.423 to 2.512, and removing FME raises it to 4.216.","Absolute onset values plus MRA give the generation model full anticipatory capability, so a control sequence can bracket a gap in time and the model can fill it while attending to both past and future control events, without an extra encoder or interleaved control tokens.","The expert listening test (p-values from about $10^{-12}$ to $10^{-17}$) indicates the generation advantage is perceptually real to trained listeners on chord fit, metadata fit, coherence, and overall enjoyment, even though objective pitch and velocity accuracy slightly favour the baseline."],"supporting_citations":[{"why":"Supplies the Fundamental Music Embedding (FME) and the RIPO relative-attention design that Moonbeam's tokenizer and MRA extend from monophonic to multi-instrument, polyphonic MIDI.","marker":"[17]"},{"why":"RoPE, the rotary position embedding whose relative-position algebra MRA generalizes from one dimension to five without adding parameters.","marker":"[22]"},{"why":"CLaMP 2 and M3, the large pretrained symbolic-music models that define the classification comparison the paper must beat in most cases.","marker":"[5]"},{"why":"MusicBERT, the BERT-style pretrained baseline included in every classification comparison.","marker":"[6]"},{"why":"REMI tokenization, the representation behind the strong transformer baseline used in the conditional generation comparison.","marker":"[7]"},{"why":"CoMMU dataset and its transformer model, the baseline that Moonbeam's finetuned generation model is evaluated against.","marker":"[24]"},{"why":"Anticipatory Music Transformer, the source of the anticipatory infilling framing and the absolute-onset alignment rationale.","marker":"[14]"},{"why":"PiJAMA and PiJAMA30, the player-classification benchmark where Moonbeam reports its largest margin and whose test set overlaps the pretraining corpus.","marker":"[43]"},{"why":"AriaMIDI, the largest pretraining corpus, and the cited evidence that transformers struggle with arithmetic, motivating absolute onset inputs.","marker":"[37]"}],"fun_headline_variants":["Moonbeam: first MIDI foundation model fusing absolute and relative info","MIDI model with relative attention beats larger AIs on most tasks","Moonbeam: absolute plus relative MIDI cues top most benchmarks","Both absolute and relative note info make Moonbeam a top music AI","Moonbeam beats most music benchmarks with relative+absolute MIDI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the downstream benchmarks measure generalization rather than memorization: the pretraining table (Appendix A, Table 5) includes the PiJAMA dataset, while the player-classification benchmark PiJAMA30 (Section 5.2) is drawn from PiJAMA, and the paper never states that the PiJAMA30 test pieces were left out of pretraining; the separate private test set described in Appendix A serves the perplexity evaluation, not this downstream split.","fun_headline_variants_meta":{"raw":{"variants":["Moonbeam: first MIDI foundation model fusing absolute and relative info","MIDI model with relative attention beats larger AIs on most tasks","Moonbeam: absolute plus relative MIDI cues top most benchmarks","Both absolute and relative note info make Moonbeam a top music AI","Moonbeam beats most music benchmarks with relative+absolute MIDI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001072,"raw_usage":{"total_tokens":4518,"prompt_tokens":1005,"completion_tokens":3513,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":621,"completion_tokens_details":{"reasoning_tokens":3423}},"tokens_in":621,"tokens_out":3513,"duration_ms":24018,"temperature":1.0,"reasoning_tokens":3423,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:14:48.855457+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Match the PiJAMA30 test pieces against the pretraining files listed in Table 5 by MIDI file identity or near-duplicate sequence comparison, then re-evaluate (or retrain) Moonbeam with all PiJAMA pieces removed from the corpus. If PiJAMA30 accuracy falls from 0.679 toward Clamp2's 0.440, the reported margin is substantially memorization; if the margin holds under this holdout, the architectural claim survives this attack.","supporting_citations":[{"cited_title":"Musicbert: Symbolic music understanding with large-scale pre-training,","cited_arxiv_id":null,"evidence_quote":"MusicBERT, the BERT-style pretrained baseline included in every classification comparison."},{"cited_title":"A domain-knowledge-inspired music embedding space and a novel attention mechanism for symbolic music modeling,","cited_arxiv_id":null,"evidence_quote":"Supplies the Fundamental Music Embedding (FME) and the RIPO relative-attention design that Moonbeam's tokenizer and MRA extend from monophonic to multi-instrument, polyphonic MIDI."},{"cited_title":"Roformer: Enhanced transformer with rotary position embedding,","cited_arxiv_id":null,"evidence_quote":"RoPE, the rotary position embedding whose relative-position algebra MRA generalizes from one dimension to five without adding parameters."},{"cited_title":"CLaMP 2: Multimodal music information retrieval across 101 languages using large language models,","cited_arxiv_id":null,"evidence_quote":"CLaMP 2 and M3, the large pretrained symbolic-music models that define the classification comparison the paper must beat in most cases."},{"cited_title":"Pop music transformer: Beat-based modeling and generation of expressive pop piano compositions,","cited_arxiv_id":null,"evidence_quote":"REMI tokenization, the representation behind the strong transformer baseline used in the conditional generation comparison."},{"cited_title":"Commu: Dataset for combinatorial music generation,","cited_arxiv_id":null,"evidence_quote":"CoMMU dataset and its transformer model, the baseline that Moonbeam's finetuned generation model is evaluated against."},{"cited_title":"Anticipatory music transformer,","cited_arxiv_id":null,"evidence_quote":"Anticipatory Music Transformer, the source of the anticipatory infilling framing and the absolute-onset alignment rationale."},{"cited_title":"Pijama: Piano jazz with automatic MIDI annotations,","cited_arxiv_id":null,"evidence_quote":"PiJAMA and PiJAMA30, the player-classification benchmark where Moonbeam reports its largest margin and whose test set overlaps the pretraining corpus."},{"cited_title":"Aria-MIDI: A dataset of piano MIDI files for symbolic music modeling,","cited_arxiv_id":null,"evidence_quote":"AriaMIDI, the largest pretraining corpus, and the cited evidence that transformers struggle with arithmetic, motivating absolute onset inputs."}],"review_version":1}