{"id":"6eda1985-438d-415a-90ba-78f093c1a4af","arxiv_id":"2608.11026","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"A score-conditioned piano model trained with a JEPA objective and contrastive losses learns embeddings that improve several performance-understanding benchmarks over existing MIDI foundation models.","lead":"MAJEPPA is a piano-performance dataset and model built from about 4,000 YouTube recordings, from child beginners to concert professionals, transcribed into MIDI and aligned with scores. The same model is trained to generate performances and to produce embeddings that are tested on quality, ranking, technique, and mistake-detection benchmarks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No de-duplication between the MAJEPPA pre-training corpus and EVPMR test sets is reported; without it, the claimed benchmark gains may reflect memorization rather than transfer.","rationale":"The paper makes a genuine contribution: a new dataset spanning the skill spectrum, a plausible JEPA-style training objective, and a coherent benchmark. The ablation between Lpiece, Lcond, and Lself is informative, and the method is parameter-efficient via LoRA. However, the central empirical claim that MAJEPPA variants 'outperform various baselines' is supported by Table 2, and that claim is valid only if the pre-training corpus and the evaluation recordings are disjoint. The paper never states this, and the curation pipeline described in Section 3 makes overlap plausible: MAJEPPA sources YouTube practice/sight-reading videos, PISA is explicitly a set of YouTube videos, the 'Concert' category could include the same commercial recordings as Pianist8, and Chopin competition performances are widely available on YouTube. The reader's weakest assumption identifies exactly this point. The generation gap is openly admitted in Section 5.1 and is therefore a known limitation; the data-overlap issue is silent, making it the more dangerous threat to the paper's headline result. The proposed audit is feasible because the authors have the source-video list and can produce the mapping; the check would settle the matter without requiring new training runs beyond re-evaluation after de-duplication.","tokens_in":12051,"tokens_out":5191,"duration_ms":48662,"concrete_test":"Run a systematic overlap audit: (1) for PISA, compare YouTube video IDs against the MAJEPPA source-video list; (2) for Pianist8 and the Chopin set, compare track metadata and compute audio fingerprints (e.g., Chromaprint) plus MIDI sequence identity between every MAJEPPA segment and every test recording, flagging pairs with normalized cross-correlation >0.9 or MIDI alignment identity >0.95; (3) exclude all flagged training segments and re-run the five downstream tasks under the same protocol. If the reported gains (e.g., PISA MAE .090 vs Aria .122) persist, the concern is resolved; if any gain shrinks beyond the reported standard deviations, the central empirical claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that the EVPMR evaluation is uncontaminated by overlap between the MAJEPPA training corpus and the test recordings. Section 3 describes collecting 3,979 YouTube recordings by querying repertoire titles with keywords such as 'Practice', 'Sight-reading', and 'Progress', plus concert-level recordings for pieces in ATEPP. Section 5 introduces downstream datasets including PISA ('59 YouTube videos'), Pianist8 ('411 pieces by 8 pianists'), and the 2015 Chopin competition set, but nowhere states that these recordings were excluded from pre-training. Since MAJEPPA is initialized from Aria and fine-tuned with LoRA on this corpus, any test recording present in training can have its representation shaped by that recording's tokens; the downstream probes then measure memorization, not transferable understanding. This is not merely hypothetical: MAJEPPA's 'Concert' category (338 recordings, commercial/live) could contain the same albums as Pianist8, and YouTube practice videos are exactly the population from which PISA is drawn. Section 5.1's admission that generation is unevaluated is an acknowledged limitation; the overlap issue is an unaddressed control that directly affects the reported 'outperform various baselines' claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MAJEPPA, a dataset of 3,979 YouTube piano recordings manually annotated along two axes (performer expertise and recording context) and aligned to scores via subsequence DTW. The proposed method adapts Aria-medium through LoRA with a joint objective: a causal next-token generation loss and three contrastive JEPA terms (L_piece, L_cond, L_self) computed under a block-diagonal attention mask with [PRED] tokens. The authors also introduce the EVPMR benchmark and evaluate frozen embeddings with linear probes on competition ranking, technique classification, quality regression, mistake prediction, and pianist identification. The central claims are that MAJEPPA variants outperform four baselines across EVPMR tasks and that the same model both generates and understands piano performance.","tokens_in":12327,"tokens_out":9859,"duration_ms":86953,"significance":"The dataset is a potentially valuable resource: it is, to my knowledge, the first large-scale piano-performance corpus spanning child beginners to virtuosi, and the EVPMR benchmark offers a reusable evaluation suite. The method is clearly described, the two-pass attention masking is an interesting way to combine generation and representation learning, and the three-loss ablation is a useful analysis. The paper is also candid about its main limitations. However, the headline empirical claims are not yet supported: no train/test disjointness control is reported, no significance testing accompanies Table 2, and the generation side of the 'unified' claim is explicitly unevaluated. These are fixable with additional controls, which is why I recommend major revision rather than rejection.","major_comments":[{"comment":"The manuscript does not state that the MAJEPPA training corpus and the EVPMR test sets are disjoint. Section 3 collects YouTube recordings by querying piece titles with keywords such as 'Practice', 'Sight-reading', and 'Progress' and includes concert recordings of ATEPP repertoire; Section 5 evaluates on PISA (59 YouTube videos), Pianist8, and the 2015 Chopin competition data. A YouTube practice video in PISA, or a concert recording used for Pianist8 or the Chopin set, could also be among the 3,979 training recordings. Because MAJEPPA is LoRA-finetuned on those recordings, any overlap lets the frozen embeddings memorize identity markers and inflate linear-probe accuracy. Please add an explicit deduplication procedure (e.g., video-ID or audio-hash matching) and either exclude overlapping recordings from training or report results on a verified-disjoint split.","section":"Section 3 and Section 5"},{"comment":"The claim that 'MAJEPPA variants outperform various baselines' is not supported by the reported statistics. The table gives mean (std) over 5-fold cross-validation across 5 seeds but no significance tests. Several improvements are comparable to the reported standard deviations, for example Technique Acc 79.0 (6.5) versus Moonbeam 77.9 (4.8) and Pianist8 86.8 (5.2) versus Aria 86.1 (4.3); on Pianist8 the full model is below Aria. Because the 25 evaluation runs are paired by fold and seed, a corrected paired t-test or a non-parametric equivalent should be reported for all task-model comparisons, together with effect sizes or confidence intervals.","section":"Table 2 and Section 5"},{"comment":"The abstract and introduction state that MAJEPPA 'both generates and understands performances in a unified framework,' but the paper explicitly says 'we did not conduct a full-scale evaluation of the generation capabilities' and only points to a website. Since the generative loss is a core component and the title claims morphing as well as assessing, quantitative generation results (objective metrics or a human listening test against the Aria or Moonbeam baselines) are needed before the unified-generation claim can be evaluated. If generation is out of scope, the claims should be narrowed to representation learning.","section":"Section 5.1"},{"comment":"The parenthetical 'we did not explicitly check transcription quality (including pedal artefacts)' is a load-bearing limitation because all audio-only datasets in Section 5 are transcribed with Transkun before computing every downstream metric, and the MAJEPPA pre-training pairs are also built from Transkun transcriptions. If AMT errors are systematic, for instance at low-skill recordings with more overlapping notes or pedal blur, they could confound the reported quality-regression and mistake-detection improvements. Please validate transcription on a small annotated subset, report error rates by expertise and context, and state whether the main conclusions in Table 2 are robust to excluding the noisiest transcriptions.","section":"Section 3.2"}],"minor_comments":[{"comment":"The definition of w_ij in Eq. (3) is described only verbally and the exact closed form is deferred to a website; for reproducibility, include the formula in the paper.","section":"Section 4.3"},{"comment":"The annotation procedure references a calibration document and a reliability website but reports no inter-annotator agreement; report Cohen's kappa or Krippendorff's alpha for the expertise and context labels, since L_cond is supervised by these labels.","section":"Section 3.1"},{"comment":"The practice-progression analysis reports a mean Spearman correlation of rho = 0.41 across PC1 and PC2 for only 7 YouTube journeys; report per-journey correlations and a shuffle baseline to support the claim of directional monotonic drift.","section":"Section 5.1"},{"comment":"The UMP columns are compared only against Aria because other baselines use incomparable token schemes; this should be stated in the table caption or as a column note so the reader does not infer a full baseline comparison.","section":"Table 2"},{"comment":"The term 'self-supervised' is used loosely: L_cond is a supervised contrastive loss using annotator-provided expertise and recording-context labels, so the overall framework is better described as self-supervised with light annotation or semi-supervised.","section":"Abstract and Section 4.3"}],"recommendation":"major_revision","confidential_remarks":"If the disjointness check and significance testing come back clean, this could be an acceptable ISMIR paper. The reliance on the authors' own earlier resources (ATEPP, NeuroPiano, PianoJudge, Moonbeam) is not disqualifying, but the EVPMR baseline implementations should be double-checked for fairness, and the demo website should not be the only evidence for generation or annotation reliability."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague —\n\nMAJEPPA is worth a careful look, but treat the headline results with some caution. What's genuinely new is the dataset: roughly 4,000 annotated YouTube-sourced piano recordings spanning six expertise levels and six recording contexts, which fills a real gap for music education research. The EVPMR benchmark suite (Chopin ranking, technique classification, quality regression, mistake prediction) is a useful package even though several tasks are inherited from prior work. The model design — a single autoregressive transformer trained with next-token loss plus score-to-performance JEPA-style contrastive objectives — is a sensible adaptation of LLM-JEPA to symbolic piano, and the ablation shows the three loss terms do different work.\n\nThe soft spots are in the evidence for the central claim. First, no significance tests are reported; in Table 2 several MAJEPPA advantages over baselines are within one standard deviation, and some cells show high variance (e.g., PISA MAE .090 with .034 std). Second, and more seriously, the paper never states that the MAJEPPA pre-training collection was de-duplicated against the EVPMR evaluation recordings. Given the training set includes YouTube practice and sight-reading videos while PISA consists of 59 YouTube videos and other test sets may also be YouTube-sourced, overlap is a plausible route to memorization rather than transfer. The stress-test note has this right: the missing overlap control is the most load-bearing issue. Third, the paper honestly admits two limitations that weaken the “generates and understands” framing: generation is not evaluated (Section 5.1) and transcription quality was not checked (Section 3.2). Those admissions are refreshing, but they mean the contribution is currently a representation-learning paper with an unevaluated generation side.\n\nThe ethics section is straightforward about the lack of uploader consent and the mitigation via releasing only derived MIDI and coarse tags; that is reasonable, though a reviewer should still probe it.\n\nOverall, the dataset and benchmark are an important resource, and the method is a credible, well-scoped application of an existing framework. The empirical superiority claim is not yet supported. The fix is clear: add significance testing, report de-duplication or quantify overlap risk, and either evaluate generation or soften that claim. I would send this to peer review rather than desk reject; it deserves referee time, and the likely outcome is major revision, not rejection.","headline":"MAJEPPA brings a genuinely useful piano dataset and a credible JEPA extension, but the claimed wins over baselines hinge on significance tests and a train/test overlap control that are missing.","tokens_in":12870,"tokens_out":2415,"would_cite":false,"duration_ms":22219,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MAJEPPA claims a single score-conditioned model can generate and evaluate piano performances across the full skill spectrum.","keywords":["piano performance","representation learning","JEPA","autoregressive transformer","score-performance alignment","performance assessment","mistake prediction","symbolic music"],"falsifier":"Search for exact or near-duplicate MIDI or audio matches between the 3,979 MAJEPPA recordings and the EVPMR evaluation recordings (PISA, NeuroPiano, YCU-PPE-III, Chopin, Pianism, Pianist8); if any overlap exists, rerun the linear-probe benchmark with those clips removed and compare MAJEPPA's margins against the Aria and Moonbeam baselines.","tokens_in":103,"feed_emoji":"🎹","tokens_out":6842,"duration_ms":119124,"temperature":0.7,"pith_summary":"MAJEPPA aims to show that one score-conditioned model can span the entire piano-performance space, from a child's first practice attempt to a virtuoso's concert recording, by learning a representation that supports both generation and assessment. To do this, the authors curate roughly 4,000 annotated YouTube recordings across six expertise levels and six recording contexts, transcribe them to MIDI, align each performance segment to its score, and train a pre-trained autoregressive MIDI transformer with a joint generative-plus-JEPA objective. The paper introduces EVPMR, a four-task benchmark for piano-performance understanding, and reports that frozen MAJEPPA embeddings beat generative, masked-reconstruction, and contrastive baselines on Chopin competition ranking, technique classification, quality regression, and conspicuous-mistake prediction. If the claim holds, music-education systems could use a single model to demonstrate how a piece should sound, diagnose where a student's playing goes wrong, and track progress over time.","feed_headline":"One piano model both performs and critiques","feed_subtitle":"Frozen embeddings from score-conditioned JEPA training beat separate baselines on quality, ranking, technique, and mistake tasks.","key_machinery":"The load-bearing mechanism is a two-view Joint-Embedding Predictive Architecture (JEPA): instead of reconstructing the performance, the model predicts a latent performance embedding from the score alone, while a parallel generative loss keeps the model able to produce performance tokens. A score–performance pair $(s,p)$ is packed into one token sequence and passed through the same transformer twice. Pass 1 uses a causal mask and computes next-token cross-entropy over the performance tokens, preserving score-conditioned generation. Pass 2 uses a block-diagonal causal mask that makes the score view $[s; \\text{COND}; \\text{PRED}]$ and the performance view $[p; \\text{EOS}]$ mutually invisible, forcing the [PRED] token to produce $\\text{Pred}(s)$ without observing the performance. A shared linear projection maps both $\\text{Pred}(s)$ and $\\text{Enc}(p)$ into an $\\ell^2$-normalized 512-dimensional space, where three contrastive terms act: InfoNCE piece alignment $L_{\\text{piece}}$, supervised cross-condition clustering $L_{\\text{cond}}$ with ordinal-distance weights, and symmetric within-performance InfoNCE $L_{\\text{self}}$. The generative loss prevents representation collapse, and the contrastive losses give the embedding space musical structure.","core_discovery":"The paper claims that a single autoregressive transformer, pre-trained on symbolic MIDI and adapted with LoRA, can learn a unified piano-performance space by solving two tasks at once: score-conditioned next-token prediction over performance tokens, and latent prediction of a performance embedding from its score alone. The latent prediction uses a JEPA-style two-view setup: score and performance are encoded under a block-diagonal causal mask so the [PRED] token never sees performance tokens, and three contrastive losses pull predicted score embeddings toward their paired performance embeddings (piece alignment), cluster performances by expertise and recording context (cross-condition clustering), and keep segments of the same performance coherent (within-performance alignment). On the new EVPMR benchmark, the frozen embeddings beat four baselines on Chopin competition ranking, technique classification, quality regression across PISA, NeuroPiano, and YCU-PPE-III, and mistake prediction on two of three datasets. The paper therefore asserts that generation and understanding are not competing modes but two outputs of one score-conditioned representation.","pith_inferences":["Beyond the paper, the same two-view score–performance setup could be applied to other instruments or to audio-only recordings by treating a cover or a different rendition as the second view, since it needs only paired renderings of the same musical content.","Beyond the paper, the apparent monotonic drift in practice trajectories hints that MAJEPPA embeddings could serve as a continuous progress metric for learners, not just a categorical six-level label; a direct test would correlate embedding distance with external exam grades.","Beyond the paper, the method suggests a controllable 'morphing' interface—interpolating between a beginner and a virtuoso embedding to generate intermediate skill levels—but the paper does not implement or evaluate such interpolation."],"forward_implications":["If the central claim is right, a single frozen embedding can rank Chopin competition performances, detect piano techniques, score quality on three datasets, and predict conspicuous mistakes without task-specific fine-tuning.","The same model that produces these embeddings can also generate score-conditioned performances, so an education tool could both demonstrate how a piece sounds at different skill levels and diagnose where a student's playing diverges from the score.","The MAJEPPA dataset fills the beginner-to-virtuoso gap, enabling models that have actually seen practice, sight-reading, and teacher demos rather than only concert recordings.","The ablation suggests the three contrastive losses have a division of labour, so future systems can tune representation objectives per downstream task.","Because the best full model improves over the Aria and Moonbeam baselines, the gain comes from the score-conditioned JEPA objective, not from model scale or tokenizer design."],"supporting_citations":[{"why":"Supplies the pre-trained 632M-parameter autoregressive MIDI backbone that MAJEPPA adapts.","marker":"[26]"},{"why":"Provides the two-view JEPA prediction recipe that MAJEPPA transfers to score–performance pairs.","marker":"[16]"},{"why":"Transcribes all audio, both source and evaluation, into MIDI, the input representation for the model and benchmark.","marker":"[39]"},{"why":"Provides the virtuoso-performance source that MAJEPPA's concert-level entries are aligned with.","marker":"[13]"},{"why":"One of the quality-regression evaluation sets, rated by teachers.","marker":"[4]"},{"why":"One of the quality-regression evaluation sets for beginner performances.","marker":"[43]"},{"why":"One of the quality-regression evaluation sets sourced from YouTube.","marker":"[44]"},{"why":"Defines the Chopin competition ranking and technique detection tasks reused in EVPMR.","marker":"[5]"},{"why":"Provides the Pianist8 pianist-identification task used as a style-transfer probe.","marker":"[21]"},{"why":"Grounds the JEPA principle of predicting in latent space rather than reconstructing the input.","marker":"[15]"}],"fun_headline_variants":["One model that plays piano and judges it too","Unified piano AI: perform and assess in one space","Self-supervised piano model for both performance and critique","Single transformer learns to play and evaluate piano","Piano AI that generates and grades in one framework"],"cache_read_input_tokens":14976,"weakest_assumption_plain":"The benchmark numbers assume that no MAJEPPA pre-training clip, especially one sourced from YouTube, also appears in the EVPMR test sets, since the paper never states that overlapping recordings were removed.","fun_headline_variants_meta":{"raw":{"variants":["One model that plays piano and judges it too","Unified piano AI: perform and assess in one space","Self-supervised piano model for both performance and critique","Single transformer learns to play and evaluate piano","Piano AI that generates and grades in one framework"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000217,"raw_usage":{"total_tokens":1407,"prompt_tokens":890,"completion_tokens":517,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":506,"completion_tokens_details":{"reasoning_tokens":443}},"tokens_in":506,"tokens_out":517,"duration_ms":4625,"temperature":1.0,"reasoning_tokens":443,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:47:38.234126+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Search for exact or near-duplicate MIDI or audio matches between the 3,979 MAJEPPA recordings and the EVPMR evaluation recordings (PISA, NeuroPiano, YCU-PPE-III, Chopin, Pianism, Pianist8); if any overlap exists, rerun the linear-probe benchmark with those clips removed and compare MAJEPPA's margins against the Aria and Moonbeam baselines.","supporting_citations":[{"cited_title":"A path towards autonomous machine intelli- gence,","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained 632M-parameter autoregressive MIDI backbone that MAJEPPA adapts."},{"cited_title":"From audio en- coders to piano judges: Benchmarking performance understanding for solo piano,","cited_arxiv_id":null,"evidence_quote":"Provides the two-view JEPA prediction recipe that MAJEPPA transfers to score–performance pairs."},{"cited_title":"Simulating piano performance mistakes for music learning,","cited_arxiv_id":null,"evidence_quote":"Provides the virtuoso-performance source that MAJEPPA's concert-level entries are aligned with."},{"cited_title":"MAJEPPA: Morphing and Assessing in a Unified Piano Performance Space","cited_arxiv_id":"2608.11026","evidence_quote":"One of the quality-regression evaluation sets, rated by teachers."},{"cited_title":"Self-supervised learning from images with a joint- embedding predictive architecture,","cited_arxiv_id":null,"evidence_quote":"One of the quality-regression evaluation sets for beginner performances."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Chopin competition ranking and technique detection tasks reused in EVPMR."},{"cited_title":"Enabling empirical analysis of piano perfor- mance rehearsal with the Rach3 MIDI dataset,","cited_arxiv_id":null,"evidence_quote":"Provides the Pianist8 pianist-identification task used as a style-transfer probe."},{"cited_title":"How does the teacher rate? Observations from the NeuroPiano dataset,","cited_arxiv_id":null,"evidence_quote":"Grounds the JEPA principle of predicting in latent space rather than reconstructing the input."}],"review_version":1}