{"id":"f313ca74-fb48-4133-bdad-1b1590fecb53","arxiv_id":"2506.14223","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A T5 encoder-decoder model converts MIDI notes into guitar tablature and, after pitch-correction post-processing, outperforms A* and Guitar Pro on tablature accuracy tests.","lead":"Researchers trained a T5 text-to-text transformer to turn MIDI note sequences into guitar tablature, choosing strings and frets for each note. Across three tablature datasets it matched or beat the A* algorithm and Guitar Pro, with extra tokens for capo and tuning giving users more control.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim relies on tab-accuracy as a proxy for transcription quality, but the ground truth is only assumed optimal; the paper's own Figure 3 shows a case where deviating from it may be preferable, so the metric's validity is the main unresolved risk.","rationale":"The reader's weakest assumption is exactly the ground-truth-optimality assumption behind the tab-accuracy metric. This is the most load-bearing concern because every comparative claim in the paper is expressed in terms of that metric. If the metric is not a valid measure of transcription quality, then the central claim of surpassing baselines and state-of-the-art loses its meaning, regardless of whether the numerical differences are statistically significant. The paper's own qualitative discussion and Figure 3 concede that ground truth is not always the preferred choice, which is direct evidence of the metric's limitation. I considered alternative concerns, such as the omission of a direct comparison with the recent neural MIDI-to-Tab model of Edwards et al. (2024), which is described in the related work but never evaluated against; that omission weakens the phrase 'state-of-the-art approaches' in the strongest claim. However, the ground-truth validity issue is more fundamental because it affects even the comparisons with A*, Guitar Pro, and TuxGuitar. The proposed perceptual study and inter-annotator agreement check would settle whether the metric reflects what guitarists actually prefer. Given that this concern is already central to the reader's conditional verdict, and that the paper contains useful contributions (a novel framing, multiple encodings, and a post-processing scheme that achieves perfect pitch accuracy), the appropriate verdict remains conditional rather than accept or reject. The reader's conditional verdict is therefore unchanged.","tokens_in":11504,"tokens_out":6204,"duration_ms":67980,"concrete_test":"Conduct a blinded perceptual study: recruit at least 10 professional guitarists; select 20–30 pieces from the Leduc and DadaGP test sets; for each piece, present the MIDI input and the tablatures produced by the Fretting-Transformer, A*, TuxGuitar, Guitar Pro, and the ground truth in randomized order; ask experts to rate playability and overall preference on a Likert scale. Compute the correlation between mean expert preference ranking and tab-accuracy ranking across methods. If the model's higher tab accuracy does not correspond to higher expert preference, the metric is not a valid basis for the headline claim. As an analytical complement, measure inter-annotator agreement by having several professional guitarists independently transcribe the same MIDI pieces; low agreement would show that a single ground-truth tab cannot be treated as 'optimal.'","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that the Fretting-Transformer surpasses A*, Guitar Pro, and TuxGuitar on tablature accuracy. That claim rests entirely on the tab-accuracy metric defined in Section 3.6, which compares predicted string-fret combinations to ground-truth tabs 'assumed to represent the optimal playing positions.' This assumption is load-bearing because the quality of the ground truth varies by dataset. Section 3.1 acknowledges that DadaGP, compiled from Ultimate Guitar, has 'quality of contributions varies greatly,' and Section 4.6 concedes that 'a lot of the tablatures in this dataset are algorithmically generated.' If ground-truth tabs are not consistently optimal, then high tab accuracy may simply reflect imitation of a particular style rather than producing preferable fingerings. The paper's own Figure 3 (box 1) illustrates this: the model deviates from the ground truth on 'Smoke on the Water,' and the authors remark that 'the use of the open strings might be preferred by many guitarists.' That admission directly undermines the metric's validity. Furthermore, the difficulty scores in Table 5 show the model's tabs are often harder than A*'s (e.g., Leduc difficulty 4.2415 vs 2.5704), yet this trade-off is dismissed without evidence that guitarists prefer the model's choices. Without independent validation that tab-accuracy correlates with human preference or playability, the central comparative claim lacks a solid foundation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents Fretting-Transformer, a T5-based encoder-decoder that maps symbolic MIDI note events to guitar tablature (string-fret tokens). The authors introduce preprocessing for three Guitar Pro datasets (GuitarToday, DadaGP, Leduc), compare five tokenization schemes, evaluate a capo/tuning-conditioned variant, and apply a post-processing correction step. The main empirical claim is that the proposed model achieves higher tablature accuracy than a baseline, A*, Guitar Pro, and TuxGuitar on all three test sets, with additional difficulty scores reported. The manuscript is clearly written and presents a practical system, but the central evaluation is weakened by a circular pitch-accuracy metric and by reliance on a tab-accuracy ground truth whose optimality the authors themselves question.","tokens_in":11820,"tokens_out":5548,"duration_ms":52787,"significance":"The paper addresses a genuine and practical problem in MIR: converting MIDI to guitar tablature requires resolving string-fret ambiguity while preserving playability. The work is useful as an engineering contribution: it demonstrates that a compact T5 model trained from scratch can produce plausible tabs, and it provides a systematic comparison of input/output encodings and conditioning strategies. It also includes an honest discussion of dataset-quality limitations, and the DadaGP filtering and augmentation pipeline may be reusable. However, the headline claim of surpassing all baselines is not yet established: the pitch-accuracy result is guaranteed by construction, tab accuracy is measured against ground truth of questionable optimality, and no statistical significance is reported. A human-evaluation study or a metric validation against guitarist preference would be needed to make the central claim convincing.","major_comments":[{"comment":"Section 3.5 defines post-processing as matching each input note to the closest output note within a window of ±5 notes and, if no direct match is found, applying 'the first viable string-fret combination generated for the guitar configuration used.' Consequently, the 100.00% pitch accuracy in Table 2 is not a property of the learned model but a guarantee of the correction procedure; reporting it as an evaluation result is circular. Please report the raw model pitch accuracy before post-processing, and, if pitch fidelity is presented as a system property, state it as a design guarantee rather than as measured model quality.","section":"Section 3.5 / Table 2"},{"comment":"The central comparison rests on tab accuracy, which counts exact agreement with ground-truth string-fret pairs that are 'assumed to represent the optimal playing positions.' The manuscript itself provides reasons to doubt this assumption: Section 3.1 says DadaGP contributions 'vary greatly,' Section 4.6 says 'a lot of the tablatures in this dataset are algorithmically generated,' and Figure 3 (box 1) shows a Fretting-Transformer output that deviates from the GuitarToday ground truth and 'might be preferred by many guitarists.' A metric that penalizes such arguably better fingerings cannot by itself support the claim that the model surpasses baselines in transcription quality. At minimum, provide an analysis of agreement with human preference or playability judgments, and separate results on professionally curated subsets from the crowd-sourced and algorithmically generated subsets.","section":"Section 3.6 / Table 5"},{"comment":"All reported accuracies are point estimates from a single evaluation run, with no confidence intervals, standard deviations, or significance tests. This matters particularly for the small datasets: the Leduc test split is roughly 23 files (10% of 232), and on GuitarToday the gap between Ours (98.41%) and Baseline/TuxGuitar (98.30%) is 0.11 percentage points. Please add multiple-seed runs, paired tests based on per-file differences, and report variance or confidence intervals; otherwise phrases like 'significant improvements' in Section 4.6 are not supported by the evidence presented.","section":"Table 5"},{"comment":"The paper reports higher difficulty for the proposed model than for A* on Leduc (4.2415 vs. 2.5704) while still claiming the model is superior, and it dismisses the trade-off by saying 'an exclusive focus on playability may not reflect the preferences of guitarists.' This is an assertion, not evidence. If the claim is that higher tab accuracy is worth a harder tab, the authors need a preference study or an argument about why the difficulty metric mis-orders quality in these cases. Without that, the difficulty results undercut rather than support the playability claim.","section":"Section 4.6 / Table 5"}],"minor_comments":[{"comment":"The text says the model halves the configuration of t5-small but gives d_model=128, d_ff=1024, three layers, and four attention heads; since t5-small has d_model=512, d_ff=2048, six layers, and eight heads, 'halving' is imprecise (the dimensions are quartered). Please clarify the actual configuration.","section":"Section 3.4"},{"comment":"Section 4.2 refers to two post-processing variants, 'overlap correction' and 'neighbor search,' but Section 3.5 describes only one matching algorithm; please define the two variants explicitly.","section":"Section 3.5 / Section 4.2"},{"comment":"In Equations (4)–(6), the same symbols p and q are used for fret numbers in some equations and string numbers in others, which makes the definitions hard to follow; please use distinct notation for frets and strings.","section":"Section 3.6"},{"comment":"The y-axis label reads 'T ab Accuracy' with a missing space; please fix the typo.","section":"Figure 2"},{"comment":"The heading 'Domain Adaption from Text' should read 'Domain Adaptation from Text.'","section":"Section 4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable engineering contribution, but the evaluation is not yet rigorous enough for the claimed superiority over all baselines. The editor may wish to request either a human evaluation study or at least statistical testing with multiple runs and confidence intervals. In addition, the relationship to Edwards et al. [20] should be sharpened: both works address MIDI-to-tab with a transformer, and the authors should clarify the incremental novelty beyond the T5 framing and the capo/tuning conditioning."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a solid engineering paper, not a conceptual breakthrough. The new thing is a small T5 encoder-decoder trained from scratch for MIDI-to-tablature with capo/tuning conditioning, plus a systematic comparison of five tokenizations and a post-processing step. On the three datasets they test, the model beats A*, Guitar Pro, and TuxGuitar on tab accuracy, and the gains are consistent across datasets. That result is believable as far as it goes, and the ablations are useful. Credit where due: they don't oversell the DadaGP ground truth, they flag the quality variance, and they openly discuss a case where their own output deviates from the ground truth in a way many guitarists might prefer.\n\nThe soft spots are real. First, the pitch accuracy numbers are not evidence about the model: the overlap and neighbor-search post-processor forces pitches to match the input, so 100% is essentially by construction. That is fine as a practical guarantee, but it should not be reported as a model result. Second, the tab-accuracy metric assumes ground-truth tabs are optimal playing positions. The paper admits that assumption is load-bearing and then gives a counterexample in Figure 3, box 1. If matching the ground truth is the goal, the model wins; if the goal is producing tabs guitarists actually prefer, the metric has not been validated. Third, the runs are single, with no confidence intervals or significance tests, and the Leduc test split is around 23 pieces. The differences on Leduc are large, so I am not worried the headline flips, but the small-sample issue should be addressed. Missing code and exact training details also limit reproducibility.\n\nI disagree with the stress-test note only slightly: it says the central claim lacks a solid foundation. I would put it more moderately. The paper establishes that the model matches the ground-truth tabs better than the baselines; what it does not establish is that ground-truth matching equals better transcription. That is a metric-validity problem, not a contradiction of the comparative result.\n\nWho is this for? People building guitar transcription tools, and MIR folks interested in symbolic translation with conditioning tokens. It deserves referee time; a serious reviewer should ask for error bars, a user study or perceptual validation, and code release. I would take it as a workshop or short conference paper, not a journal article as-is.","headline":"A useful MIDI-to-tab system with a believable comparative result, but the tab-accuracy metric is too shaky to carry the central claim without more validation.","tokens_in":12333,"tokens_out":2242,"would_cite":false,"duration_ms":23703,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A compact T5 transformer converts MIDI into guitar tabs that agree with professional Guitar Pro tabs more often than A* or commercial tools on three datasets.","keywords":["MIDI to tablature transcription","guitar tablature generation","T5 transformer","symbolic music translation","string-fret assignment","playability metric","capo and tuning conditioning","Guitar Pro datasets"],"falsifier":"Take a held-out set of professional tabs, ask experienced guitarists to blind-rank the model's tabs against A* and Guitar Pro, and check whether their preferences track the model's tab-accuracy margin; a complementary check is to retrain on DadaGP after excluding algorithmically generated tracks and see whether the Leduc margin survives.","tokens_in":11318,"feed_emoji":"🎸","tokens_out":8671,"duration_ms":81128,"temperature":0.7,"pith_summary":"This paper tries to establish that guitar tablature transcription from MIDI can be solved as a text-to-text translation problem with a compact T5 encoder-decoder. The model takes MIDI note events as the source language and outputs combined string-fret tokens as the target language, with optional capo and tuning tokens as conditioning. On three test sets the paper reports tab accuracy of 98.41 percent on GuitarToday, 72.19 percent on Leduc, and 81.58 percent on DadaGP in standard tuning, beating the A* reimplementation, a lowest-fret baseline, Guitar Pro, and TuxGuitar in tab accuracy. The result matters because it suggests a learned model can replace hand-designed cost functions for producing playable guitar tabs, and because the same translation framing can absorb alternate tunings and capo positions without changing the architecture.","feed_headline":"A small transformer writes guitar tabs better than Guitar Pro and A*","feed_subtitle":"A compact T5 with capo/tuning conditioning matches professional tabs more often than A* and Guitar Pro.","key_machinery":"The load-bearing object is the T5 text-to-text transformer, used as a reduced encoder-decoder (three layers, $d_{\\text{model}}=128$, four attention heads) trained from scratch rather than initialized from a language model. The input language is an event-based MIDI stream with NOTE ON, NOTE OFF, and TIME SHIFT tokens; the output language is a sequence of TAB <string,fret> tokens, where each token encodes both choices at once. Capo and tuning are injected as CAPO and TUNING tokens in the conditioned variant. That arrangement turns ambiguous string-fret selection into a translation problem in which the decoder's attention can use a whole 20-note window of musical context, unlike the A* graph search, which only looks at adjacent notes. A post-processing step then maps model outputs back onto the original input pitches and selects viable string-fret pairs when no direct match exists.","core_discovery":"The paper's central claim is that a T5 encoder-decoder trained from scratch on a combined corpus of Guitar Pro files from DadaGP, GuitarToday, and Leduc produces tablatures that agree with the ground-truth Guitar Pro tabs more often than established methods do. In the standard-tuning scenario the Fretting-Transformer reaches 98.41% tab accuracy on GuitarToday, 72.19% on Leduc, and 81.58% on DadaGP, compared with 89.39% and 62.60% for A* on GuitarToday and Leduc; in the capo/tuning-conditioned scenario it again leads on GuitarToday (98.19%) and Leduc (73.02%), while GPT2 is slightly closer on DadaGP (81.45% vs. 79.89%). The paper also claims that a simple post-processing pass that matches predicted notes to input pitches restores 100% pitch accuracy, and that the best tokenization is the event-based v3 encoding with combined TAB tokens and TIME SHIFT tokens.","pith_inferences":["The paper leaves implicit that tab accuracy against Guitar Pro ground truth is a measure of imitation of the corpus's fretting style; on DadaGP, where the paper notes tabs may be algorithmically generated, the Baseline's close scores suggest the metric may reward algorithmic patterns rather than human preference.","A blind listening or playing study with guitarists would separate imitation from playability, since the paper's own 'Smoke on the Water' example shows a divergence from ground truth that players might actually prefer.","The translation framing is generic: with a different target token vocabulary, the same encoder-decoder could be pointed at bass or ukulele tablature, or at lead-sheet-to-fingerstyle arrangement, because conditioning on tuning and capo is just extra tokens."],"forward_implications":["If the central claim holds, a modest-size transformer without hand-coded biomechanical costs can outperform graph-search and commercial tab generators on tab accuracy across beginner, jazz, and multi-genre test sets.","Capo and tuning conditioning preserve most of the accuracy advantage, so the same model can produce tabs for common alternate tunings and capo positions without retraining per configuration.","The v3 encoding result implies that explicit timing information and a combined string-fret token are worth more than separate string and fret tokens for this task.","The post-processing result implies that a pure pitch-preservation guarantee is achievable at almost no cost to tab accuracy on the evaluated test sets, a prerequisite for any practical MIDI-to-tab tool."],"supporting_citations":[{"why":"Supplies the T5 text-to-text transformer architecture that the paper reduces and trains from scratch.","marker":"[2]"},{"why":"The A-star-Guitar graph-search algorithm that serves as the main state-of-the-art baseline.","marker":"[11]"},{"why":"Provides the event-based tokenization style with NOTE ON, NOTE OFF, and TIME SHIFT that the input encoding uses.","marker":"[14]"},{"why":"Contributes the DadaGP dataset and its Guitar Pro token format, which dominates the training corpus.","marker":"[17]"},{"why":"The earlier BERT-based MIDI-to-tab system whose mask-token approach and limitations motivate the translation formulation.","marker":"[20]"},{"why":"The domain-adaptation work that the Leduc dataset and its high-resolution transcription setting are associated with.","marker":"[23]"},{"why":"Supplies the playability and difficulty scoring framework that the paper adapts into its difficulty metric.","marker":"[25]"}],"fun_headline_variants":["Transformer beats A* and Guitar Pro at writing guitar tabs","MIDI to tablature: a T5 model outperforms A* and Guitar Pro","Fretting-Transformer: AI guitar tabs surpass A* and Guitar Pro","Encoder-decoder model transcribes MIDI into playable guitar tabs","AI writes guitar tabs better than Guitar Pro with a compact T5"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's evaluation assumes the ground-truth Guitar Pro tabs are the optimal playing positions, so the headline tab-accuracy numbers measure closeness to those tabs; if a substantial share of ground-truth tabs, especially in DadaGP, are not high-quality human arrangements, then 'surpassing the baselines' is really 'imitating one particular tab style' rather than producing objectively better tabs.","fun_headline_variants_meta":{"raw":{"variants":["Transformer beats A* and Guitar Pro at writing guitar tabs","MIDI to tablature: a T5 model outperforms A* and Guitar Pro","Fretting-Transformer: AI guitar tabs surpass A* and Guitar Pro","Encoder-decoder model transcribes MIDI into playable guitar tabs","AI writes guitar tabs better than Guitar Pro with a compact T5"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000278,"raw_usage":{"total_tokens":1649,"prompt_tokens":936,"completion_tokens":713,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":617}},"tokens_in":552,"tokens_out":713,"duration_ms":6797,"temperature":1.0,"reasoning_tokens":617,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T00:18:15.136630+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of professional tabs, ask experienced guitarists to blind-rank the model's tabs against A* and Guitar Pro, and check whether their preferences track the model's tab-accuracy margin; a complementary check is to retrain on DadaGP after excluding algorithmically generated tracks and see whether the Leduc margin survives.","supporting_citations":[{"cited_title":"Fretting-Transformer: Encoder-Decoder Model for MIDI to Tablature Transcription","cited_arxiv_id":"2506.14223","evidence_quote":"Supplies the T5 text-to-text transformer architecture that the paper reduces and trains from scratch."},{"cited_title":"A genetic algorithm for the automatic generation of playable guitar tablature,","cited_arxiv_id":null,"evidence_quote":"The A-star-Guitar graph-search algorithm that serves as the main state-of-the-art baseline."},{"cited_title":"Automatic Transcription of Guitar Chords and Fingering From Audio,","cited_arxiv_id":null,"evidence_quote":"Provides the event-based tokenization style with NOTE ON, NOTE OFF, and TIME SHIFT that the input encoding uses."},{"cited_title":"Robotaba Guitar Tablature Transcription Framework,","cited_arxiv_id":null,"evidence_quote":"Contributes the DadaGP dataset and its Guitar Pro token format, which dominates the training corpus."},{"cited_title":"Music Transformer: Gen- erating Music with Long-Term Structure,","cited_arxiv_id":null,"evidence_quote":"The earlier BERT-based MIDI-to-tab system whose mask-token approach and limitations motivate the translation formulation."},{"cited_title":"DadaGP: A Dataset of Tokenized GuitarPro Songs for Sequence Models,","cited_arxiv_id":null,"evidence_quote":"The domain-adaptation work that the Leduc dataset and its high-resolution transcription setting are associated with."},{"cited_title":"The scoring system takes into account two types of movement: horizontal shifts along the fretboard (along) and vertical shifts across the strings (across)","cited_arxiv_id":null,"evidence_quote":"Supplies the playability and difficulty scoring framework that the paper adapts into its difficulty metric."}],"review_version":1}