{"id":"e8be7773-d478-447f-a963-7de5a7bc342d","arxiv_id":"2507.12175","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A single transformer model aligns scores to performances, transcribes piano audio, and detects mistakes, including faithful handling of repeat sections without pre-unfolded scores.","lead":"RUMAA is a single transformer system that aligns a musical score to a performance recording, transcribes the audio, and flags wrong, missing, or extra notes, including for pieces with repeat sections. It reports strong accuracy on repeated-score pieces, though the comparison setup may favor RUMAA and the repeat test uses only two pieces.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Repeat-superiority claim rests on an unfair baseline protocol: baselines are denied the repeat-unfolded scores they require; an oracle-unfolded baseline comparison is needed.","rationale":"The reader's verdict is CONDITIONAL with moderate confidence, and I agree that the paper is coherent, technically interesting, and not ready for unconditional acceptance. My stress-test focuses on the weakest load-bearing condition for the central repeat-superiority claim: the baseline protocol in Table 2. The paper frames the contribution as escaping the need for manually unfolded score-MIDI, which is a legitimate practical goal. However, the reported numbers are used to claim that RUMAA 'outperforms' prior methods on repeated scores by up to 87 percentage points. That comparison gives the baselines an input they cannot process instead of the repeat-unfolded score-MIDI they are designed to require. A fair test of alignment quality would give baselines their required precondition, then measure whether RUMAA's automatic repeat handling still achieves comparable accuracy. Without that test, the headline superiority on repeats is not established. The reader's weakest_assumption emphasized synthetic-to-real transfer and metric bias, and the rationale also noted the baseline input issue. I partially agree: the transfer concern is real but secondary to the more immediate protocol problem, since even perfect transfer would not validate the superiority claim if the baseline comparison is rigged by input format. The concrete test I propose directly settles whether the central claim survives a fair comparison. I do not recommend REJECT because the paper's non-repeat alignment, unified architecture, and score-informed results still have value, and the repeat claim may survive once tested fairly. CONDITIONAL remains the appropriate verdict, so I leave the reader's verdict unchanged. I also note the adapted F_align metric counting repeated notes independently deserves scrutiny, but it is less decisive than the baseline protocol issue and should be examined alongside the repeat evaluation on more pieces.","tokens_in":9177,"tokens_out":3072,"duration_ms":39131,"concrete_test":"Re-run the Table 2 'w/ repeat' evaluation on the same two Vienna pieces (Mozart K331, Schubert D783), and ideally on a larger set of repeat-containing pieces from Vienna and (n)ASAP, after feeding Nakamura HMM, hDTW+sym, and GlueNote oracle repeat-unfolded score-MIDI derived from the MusicXML repeat markings, while RUMAA continues to receive the original MusicXML with repeat symbols. If the unfolded baselines reach F_align near RUMAA's 98.4, the headline repeat advantage is an input-format convenience result rather than an alignment-accuracy gain; if they remain far below, the claim stands. Also report RUMAA's F_align on these pieces when given unfolded scores, to quantify any cost or benefit of repeat-aware decoding.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and Section 6.1 claim RUMAA outperforms state-of-the-art methods on scores with repeats, but Table 2's 'w/ repeat' rows do not provide a fair accuracy comparison. The caption and text state that the repeat setting applies original scores with repeat symbols to all methods. Nakamura HMM, hDTW+sym, and GlueNote are MIDI-based symbolic aligners that require repeat-unfolded score-MIDI as input; the paper itself notes they 'cannot interpret repeat symbols.' Thus their collapses to 36.4, 28.2, and 12.7 may simply reflect that they were given an input format their pipeline cannot consume, not that RUMAA aligns repeated passages more accurately. The claimed 'outperforms them by over threefold' is therefore not an apples-to-apples alignment-quality comparison. The load-bearing condition for the central claim is that RUMAA would remain competitive or superior when baselines are given the manually unfolded scores they are designed to use. The paper does not test this condition. Section 5.1 also trains repeat handling only on synthetic repeats added to 20% of scores, with audio repeated verbatim, and Table 2 evaluates on just two real pieces, so the repeat result is both protocol-biased and statistically thin.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"RUMAA is a transformer-based framework that unifies score-to-performance alignment, score-informed transcription, and mistake detection for piano music. It encodes MusicXML scores as bar-level ABC patches with a pre-trained M3 encoder, encodes audio with a frozen YourMT3+ pre-trained audio encoder, and uses a six-layer decoder that autoregressively emits three aligned token streams: performance tokens, score tokens, and edit-operation tokens. The decoder is trained on (n)ASAP with data augmentation that includes synthetic repeat insertion, and evaluated on the Vienna corpus for alignment, (n)ASAP/Maestro for transcription, and STPD for mistake detection. The main claims are that RUMAA matches state-of-the-art alignment methods on non-repeated scores, outperforms them on scores with repeat symbols without requiring manually unfolded score-MIDI, achieves near-perfect score-informed transcription onset F1, and improves mistake detection over prior NMF-based systems.","tokens_in":9423,"tokens_out":7666,"duration_ms":95132,"significance":"If the repeat-aware alignment result is confirmed, RUMAA addresses a real practical bottleneck in music performance analysis: conventional alignment tools require manually unfoldable score-MIDI with repeat structures pre-specified, while RUMAA processes MusicXML scores with repeats directly. The unified multi-task formulation is original and the use of pretrained, independently benchmarked encoders is a strength. The paper also avoids circularity by training on separate public corpora and holding out its evaluation sets. However, the paper's headline repeat advantage is currently supported by only two test pieces and an unfair baseline protocol, and the transfer from synthetic repeat augmentation to real repeated performances is unvalidated. Because the central advertised contribution rests on this comparison, the evidence as presented is not yet sufficient; with a corrected baseline protocol and a broader evaluation, the contribution would be solid and of clear value to the MIR community.","major_comments":[{"comment":"The repeat-condition comparison is not apples-to-apples. The caption and text state that in the 'w/ repeat' setting all methods receive the original scores with repeat symbols, but Nakamura HMM, hDTW+sym, and GlueNote are symbolic aligners that require repeat-unfolded score-MIDI as input, and the paper itself notes that they 'cannot interpret repeat symbols.' Their collapses to 36.4, 28.2, and 12.7 may therefore reflect an unusable input format rather than inferior alignment quality. To support the abstract claim that RUMAA 'outperforms them on scores with repeats,' please add an oracle-unfolded condition in which these baselines are given manually repeat-unfolded score-MIDI on the same two pieces, and report F_align under that protocol. RUMAA's claimed advantage is only established if it remains competitive or superior when baselines receive the input representation they were designed to consume.","section":"Section 6.1, Table 2"},{"comment":"The repeat-aware result is statistically thin and its transfer is not demonstrated. Only two pieces are evaluated (Mozart K331 and Schubert D783), with no error bars, no per-piece results, and no significance testing. Moreover, the model's repeat behavior is learned solely from synthetic augmentation: repeat symbols are added to random bars in 20% of ABC scores lacking repeats, and the corresponding audio is repeated verbatim. Real repeated performances can contain ornaments, tempo changes, first/second endings, and expressive variation, none of which appear in this training signal. Please evaluate on a larger held-out set of pieces with real repeats, or at least report per-piece and bootstrapped intervals for the current two pieces, and include an ablation that withholds repeat-augmented training data to show that the repeat handling does not come from the score encoder alone.","section":"Section 5.1, Table 2"},{"comment":"The adapted F_align metric is underspecified. The paper says repeated notes are 'redefined to be counted independently,' but this is precisely the point at which a repeat-aware model and a symbolic aligner can differ; without a precise formula for how repeated notes are matched and counted, it is unclear whether the reported 98.4 on repeat pieces reflects alignment quality or a scoring choice that favors RUMAA's token design. Please state the exact matching criterion for repeated notes and report both the original F_align and the adapted F_align on the same data.","section":"Section 5.2"},{"comment":"The score-informed transcription claim is not supported by the current comparison. The 'Score-informed' row for RUMAA is compared only with score-free baselines (MT3, YourMT3+, hFT-T, and IS-CRF on Maestro or (n)ASAP); no score-informed baseline is included. The statement that RUMAA 'surpasses all baselines by a clear margin' therefore overstates what the table shows. Please add score-informed baselines or revise the claim to state that RUMAA improves over score-free models when score information is provided.","section":"Section 6.2, Table 3"}],"minor_comments":[{"comment":"The label 'Charater-level Tokenizer' contains a typo and should read 'Character-level Tokenizer.'","section":"Figure 2"},{"comment":"Please clarify whether the 'w/o repeat' row includes the two repeat-containing pieces in unfolded form or excludes them; the current description 'all songs from Vienna' is ambiguous and directly affects how the within-row comparison should be read.","section":"Table 2"},{"comment":"The prose says that on repeated scores symbolic aligners 'drop by up to 87%' and that symbolic-audio aligners 'drop by up to 87%' and later 'by up to 70%; these numbers should be stated consistently and tied explicitly to the rows in Table 2.","section":"Section 6.1, Table 2"},{"comment":"The limitation of one-minute audio segments is acknowledged, but since the repeat evaluation uses full movements such as Mozart K331, please describe how longer audio is chunked and how repeat handling is maintained across chunk boundaries; otherwise readers cannot judge whether the reported F_align is artificially segment-local.","section":"Section 7"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the paper is generally careful about isolating training and evaluation data. The decisive issue is the repeat-comparison protocol: the main claim is not yet supported because the baselines are denied the unfolded scores they require. I would encourage the editor to request a major revision focused on oracle-unfolded baseline comparisons, a larger repeat evaluation, a precise statement of the adapted F_align metric, and score-informed transcription baselines. No concerns about citation fairness or novelty disclosure beyond the comparison issues raised in the report."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read RUMAA. The engineering is good, and there is a real new idea here: a tri-stream decoder that emits performance tokens, score tokens, and edit-operation tags from one frozen score/audio encoder pair. That gives you alignment, score-informed transcription, and mistake detection in one forward pass, with a token design that is noticeably more compact than serialized MIDI. The repeat-aware tokenization is also new, directly consuming MusicXML repeat symbols rather than requiring pre-unfolded score-MIDI. That is a genuine usability improvement over Nakamura HMM, hDTW, and GlueNote, which all need the repeat structure resolved by hand.\n\nWhat the paper does well: it is transparent about its own limits (single instrument, clean audio, one-minute segments), it reuses publicly benchmarked encoders rather than hiding a fitted derivation, and the non-repeat alignment result (98.4 F_align on Vienna) is competitive with the best symbolic aligners. The mistake-detection gains over NMF baselines are plausible.\n\nThe soft spots are real and central. The abstract's claim that RUMAA \"outperforms them on scores with repeats\" is not supported by Table 2. The baseline methods cannot consume repeat symbols; giving them the original score and letting them fail is not a fair accuracy comparison. To claim superiority on repeats, the baselines need to be given the repeat-unfolded scores they are designed for, or an oracle-unfolded baseline must be reported. Without that, the collapse from 99.0 to 36.4 may just be an input-format mismatch, not a failure of alignment quality. Second, the w/ repeat column is two pieces (Mozart K331, Schubert D783). That is too thin for a headline result, especially since repeat behavior was learned only from synthetic augmentation (repeat symbols added to 20% of scores, audio repeated verbatim). No evidence yet that this transfers to real ornaments, tempo changes, or first/second endings. Third, the score-informed transcription comparison in Table 3 compares RUMAA to score-free models; the score gives RUMAA a large advantage, so the margin is not apples-to-apples. Also, no error bars or ablations are reported anywhere. The F_align adaptation that counts repeated notes independently could also bias toward the system's token design, and that deserves checking.\n\nWho this is for: MIR researchers working on score following, piano tutoring, or unified transcription-alignment models. It deserves a serious referee, but the next revision should add unfolded-score baselines, a larger repeated-score test set, error bars, and ideally released code. If those come, the repeat claim could hold; right now it is conditional.","headline":"Solid unified MIR architecture with genuinely new repeat handling, but the headline repeat advantage is built on an unfair baseline comparison and two test pieces.","tokens_in":9953,"tokens_out":2492,"would_cite":true,"duration_ms":28430,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"RUMAA claims that repeat-aware score-performance alignment can hold at 98.4 F1 even when conventional symbolic alignment methods drop by up to 87 points, by jointly decoding performance, score, and edit-operation tokens in one transformer.","keywords":["score-performance alignment","repeat handling","music transcription","transformer","cross-modal learning","mistake detection","tri-stream decoding","ABC notation"],"falsifier":"Run RUMAA on a held-out set of real piano performances whose scores contain written repeats, with human-verified repeat-unfolded ground truth; if the F_align drops materially on repeated sections, or if human listeners disagree with the adapted metric's independent counting of repeated notes, the central repeat-handling claim is not general.","tokens_in":8978,"feed_emoji":"🎹","tokens_out":4989,"duration_ms":55157,"temperature":0.7,"pith_summary":"This paper claims a single transformer model, RUMAA, can align a written score to a recorded piano performance even when the score contains repeat symbols, while also transcribing the audio and detecting wrong or missing notes. On non-repeated scores it matches the best symbolic alignment systems, reaching 98.4 F1 against 99.0 for the top HMM baseline. On the two Vienna pieces with repeats, those baselines collapse to between 12.7 and 36.4, whereas RUMAA stays at 98.4. If this holds, it removes the need to manually unfold repeats before alignment and suggests one model can serve alignment, transcription, and mistake detection together.","feed_headline":"Repeat-aware model holds alignment at 98.4; baselines collapse","feed_subtitle":"One transformer aligns audio to MusicXML repeats, transcribes piano, and flags extra or missed notes in a single pass.","key_machinery":"The load-bearing object is the tri-stream token sequence generated by a six-block autoregressive transformer decoder, conditioned hierarchically with audio cross-attention first and then score cross-attention, on frozen pre-trained encoders: a bar-patched ABC score encoder and a 12-frames-per-second audio encoder. Three proxy tasks share one latent space: T1 transcribes the performance aligned to the score, T2 converts the score following performance order, and T3 tags each note as match, insert, delete, or repeat. The placeholder alignment token and the repeat token carry the argument, because they let the model represent a one-to-one path through the music even when the performance goes back and repeats a section.","core_discovery":"RUMAA is built around a tri-stream decoder that simultaneously emits, for each aligned position, a performance token, a score token, and an edit-operation token chosen from <Match>, <Insert>, <Delete>, or <Repeat>, with a special placeholder for notes present in only one modality. This converts repeat handling from an external structure problem into a sequence problem: the score encoder sees MusicXML with repeat symbols and barlines, and the decoder learns, through cross-attention to audio and score, to follow the performed structure rather than the unfolded written order. The reported result is that alignment accuracy on notes is 98.4 F1 both with and without repeats in the Vienna piano corpus, while HMM, DTW, and GlueNote baselines fall to 36.4, 28.2, and 12.7 when repeats appear. The same unified model also outperforms prior NMF systems on detecting extra and missed notes, and score-informed transcription reaches a 99.1 onset F1 on the (n)ASAP test set.","pith_inferences":["We infer that the repeat handling is only as general as the synthetic augmentation: repeat signs were added to 20% of training scores with audio literally repeated, so real performances with ornaments, tempo changes, or first and second endings are an untested and likely harder case.","The adapted alignment metric counts repeated notes independently, which may systematically favor systems that always repeat; a human-listening evaluation on repeated sections would clarify whether the 98.4 reflects musical correspondence or a metric bias.","A natural extension is to use the same edit-operation token stream for expressive performance annotation, such as tempo rubato, dynamics, or articulation deviations, since timing and velocity tokens already encode those dimensions."],"forward_implications":["If the central claim holds, alignment systems no longer need repeat-unfolded score-MIDI; a score with repeat symbols can be used directly, which removes a manual preprocessing step.","Because transcription and alignment share one decoder, score-informed transcription becomes a by-product of alignment, reaching near-perfect onset detection (99.1 F1) on (n)ASAP.","Mistake detection is expressed as insert and delete edits in the same token stream, so it no longer requires a separate post-hoc method.","The uniform 98.4 F1 on both non-repeat and repeat scores implies the model follows performed order locally rather than relying on a global unfolded structure."],"supporting_citations":[{"why":"Supplies the (n)ASAP training alignments, the F_align metric, and the hDTW+sym baseline that RUMAA compares against.","marker":"[10]"},{"why":"Provides the Nakamura HMM baseline that RUMAA matches on non-repeated scores and outperforms on repeated scores.","marker":"[3]"},{"why":"Provides the GlueNote transformer baseline, another learned alignment method that collapses on repeats in the comparison.","marker":"[14]"},{"why":"Supplies the pre-trained audio encoder approach and the YourMT3+ baseline used for symbolic-audio alignment and transcription comparisons.","marker":"[22]"},{"why":"Supplies the M3 score encoder and the ABC bar-level tokenization that lets RUMAA consume MusicXML with repeat symbols.","marker":"[25]"},{"why":"Provides the Vienna piano corpus used for evaluation on both non-repeat and repeat-containing scores.","marker":"[33]"},{"why":"Defines the STPD mistake detection benchmark and the score-informed transcription formulation that RUMAA extends.","marker":"[4]"}],"fun_headline_variants":["RUMAA: 98.4 alignment F1 even with repeats, baselines collapse","One model aligns, transcribes, and detects mistakes in music with repeats","Repeat-aware RUMAA unifies alignment, transcription, and mistake detection","Transformer solves repeat alignment: RUMAA beats HMM, DTW, GlueNote","Unified music model: alignment with repeats, transcription, and error flags"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model's repeat handling is learned entirely from artificially inserted repeat signs and repeated audio in 20% of training scores, and it must transfer to real performances with repeats; if real repeat execution differs enough, the alignment advantage on repeated scores collapses.","fun_headline_variants_meta":{"raw":{"variants":["RUMAA: 98.4 alignment F1 even with repeats, baselines collapse","One model aligns, transcribes, and detects mistakes in music with repeats","Repeat-aware RUMAA unifies alignment, transcription, and mistake detection","Transformer solves repeat alignment: RUMAA beats HMM, DTW, GlueNote","Unified music model: alignment with repeats, transcription, and error flags"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001428,"raw_usage":{"total_tokens":5735,"prompt_tokens":892,"completion_tokens":4843,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":4739}},"tokens_in":508,"tokens_out":4843,"duration_ms":38438,"temperature":1.0,"reasoning_tokens":4739,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:52:04.020829+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RUMAA on a held-out set of real piano performances whose scores contain written repeats, with human-verified repeat-unfolded ground truth; if the F_align drops materially on repeated sections, or if human listeners disagree with the adapted metric's independent counting of repeated notes, the central repeat-handling claim is not general.","supporting_citations":[{"cited_title":"Automatic note-level score- to-performance alignments in the asap dataset,","cited_arxiv_id":null,"evidence_quote":"Supplies the (n)ASAP training alignments, the F_align metric, and the hDTW+sym baseline that RUMAA compares against."},{"cited_title":"Performance error detection and post-processing for fast and accurate symbolic music alignment","cited_arxiv_id":null,"evidence_quote":"Provides the Nakamura HMM baseline that RUMAA matches on non-repeated scores and outperforms on repeated scores."},{"cited_title":"Thegluenote: Learned representations for robust and flexible note alignment,","cited_arxiv_id":null,"evidence_quote":"Provides the GlueNote transformer baseline, another learned alignment method that collapses on repeats in the comparison."},{"cited_title":"Yourmt3+: Multi- instrument music transcription with enhanced transformer architectures and cross-dataset stem augmentation,","cited_arxiv_id":null,"evidence_quote":"Supplies the pre-trained audio encoder approach and the YourMT3+ baseline used for symbolic-audio alignment and transcription comparisons."},{"cited_title":"Vienna 4x22 Piano Corpus, Rematched,","cited_arxiv_id":null,"evidence_quote":"Provides the Vienna piano corpus used for evaluation on both non-repeat and repeat-containing scores."},{"cited_title":"Score-informed transcription for automatic piano tutoring,","cited_arxiv_id":null,"evidence_quote":"Defines the STPD mistake detection benchmark and the score-informed transcription formulation that RUMAA extends."}],"review_version":1}