Pith. sign in

REVIEW 3 major objections 1 minor 27 references

Velocity Prediction in Automatic Guitar Transcription

T0 review · 3 major / 1 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pretraining on synthetic guitar data lets a transcription model predict note velocity on real audio.

desk verdict Synthetic pretraining gives velocity prediction for guitar transcription with clear gains on synthetic tests but only small, indirect benefits shown on real audio. read the letter →

arxiv 2606.24912 v1 pith:W57XBJSH submitted 2026-06-19 cs.SD cs.AIeess.AS

Jackson Loth , Xavier Riley , Simon Dixon , Emmanouil Benetos This is my paper · ORCID
classification cs.SDcs.AIeess.AS
keywords automaticguitartranscriptionvelocitypredictionsyntheticdatatransferlearningpolyphonicnoteintensityvirtualinstruments
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a method to add velocity prediction to automatic guitar transcription despite the lack of intensity labels in real recordings. Virtual instruments generate synthetic audio with explicit velocity labels for pretraining. The resulting weights transfer to a second model trained on unlabeled real guitar audio, preserving the velocity head while adapting to actual instrument sounds. This transferred model beats a non-pretrained baseline at velocity estimation when both are tested on synthetic data and yields modest gains in note detection on some real test sets. The combined system reaches transcription accuracy on par with existing guitar-specific models while outputting velocity values.

What carries the argument

Weight transfer of a velocity prediction head pretrained on synthetic virtual-instrument data to a transcription network trained on real guitar audio.

What would settle it

Both the transferred model and the baseline are evaluated on held-out synthetic guitar audio with known velocities; if the transferred model shows no lower velocity prediction error, the claim fails.

Watch

Extended reading notes

Core claim

A model first trained on synthetic guitar data that includes velocity labels can have those weights transferred to a new network trained on real unlabeled guitar audio; the transferred model then predicts velocity more accurately than a baseline without pretraining when evaluated on synthetic test data, delivers small improvements to note transcription on some real test sets, and maintains overall performance comparable to the state of the art.

Load-bearing premise

Statistical patterns of velocity learned from virtual instrument sounds remain useful when the model processes real guitar recordings.

Editorial extensions

If this is right

  • Velocity prediction becomes possible in guitar transcription without requiring velocity-labeled real recordings.
  • The transferred model outperforms a non-pretrained baseline at velocity estimation on synthetic evaluation data.
  • Note transcription accuracy receives a small boost on certain real test sets when pretrained velocity weights are used.
  • Overall transcription performance stays comparable to existing state-of-the-art guitar transcription systems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same pretrain-and-transfer pattern could be applied to other instruments that lack velocity annotations.
  • If the acoustic mismatch between virtual and real instruments is reduced, velocity prediction accuracy on real audio may increase.
  • Downstream tools such as performance feedback systems could directly use the velocity output for expressive analysis.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 1 minor

Summary. The paper proposes a method for velocity prediction in automatic guitar transcription by generating synthetic data with velocity labels from virtual instruments, pretraining a model on this data, transferring the weights to a model trained on real unlabeled guitar audio, and claiming that the resulting model outperforms a non-pretrained baseline on synthetic velocity prediction while yielding a small (sometimes non-significant) gain in note transcription and results comparable to SOTA guitar transcription.

Significance. If the transfer of velocity prediction from synthetic to real guitar audio holds under direct testing, the work would address a notable gap in AMT for non-piano instruments by enabling velocity-aware models without requiring real velocity labels; the use of synthetic pretraining and transfer learning is a practical strength.

major comments (3)
  1. [Abstract] Abstract and evaluation description: outperformance on velocity is reported solely versus a non-pretrained baseline on synthetic test data, but no quantitative metrics, error bars, statistical tests, or definition of how velocity is computed or scored for guitar are provided.
  2. [Evaluation / Results] The central transfer claim (pretrained velocity weights remain functional after fine-tuning on real recordings) is supported only by indirect note-transcription metrics showing small and sometimes non-significant gains; no direct evaluation or proxy (loudness correlation, human ratings, or cross-instrument consistency) of velocity quality on real guitar audio is described.
  3. [Methodology / Discussion] The weakest assumption—that statistical properties of synthetic velocity labels transfer meaningfully to real guitar—is stated but not tested, leaving generalization from virtual-instrument data unverified.
minor comments (1)
  1. The manuscript would benefit from explicit statements of the velocity definition used for guitar, details on the precise model architectures, and any ablation studies isolating the contribution of the velocity pretraining.

Simulated Author's Rebuttal

3 responses · 1 unresolved

We thank the referee for their constructive comments, which highlight important aspects of our evaluation and assumptions. We address each major comment below, proposing revisions to strengthen the manuscript.

read point-by-point responses

  1. Referee: [Abstract] Abstract and evaluation description: outperformance on velocity is reported solely versus a non-pretrained baseline on synthetic test data, but no quantitative metrics, error bars, statistical tests, or definition of how velocity is computed or scored for guitar are provided.

    Authors: We agree with this observation. The current abstract summarizes the results qualitatively. In the revised version, we will expand the abstract and evaluation description to include quantitative metrics (e.g., velocity prediction error), error bars from multiple runs, results of statistical tests, and a clear definition of how velocity is computed and scored for guitar, including the mapping from MIDI velocity values to synthesized audio intensity. revision: yes

  2. Referee: [Evaluation / Results] The central transfer claim (pretrained velocity weights remain functional after fine-tuning on real recordings) is supported only by indirect note-transcription metrics showing small and sometimes non-significant gains; no direct evaluation or proxy (loudness correlation, human ratings, or cross-instrument consistency) of velocity quality on real guitar audio is described.

    Authors: We acknowledge that our evaluation of the transferred velocity prediction relies on indirect evidence from note transcription improvements. Direct evaluation is challenging due to the absence of velocity annotations in real guitar datasets. We will revise the paper to explicitly state this limitation and discuss potential future proxies such as loudness correlations, while noting that the observed note transcription gains provide supporting evidence for retention of velocity information. revision: partial

  3. Referee: [Methodology / Discussion] The weakest assumption—that statistical properties of synthetic velocity labels transfer meaningfully to real guitar—is stated but not tested, leaving generalization from virtual-instrument data unverified.

    Authors: The assumption regarding the transfer of velocity statistics from synthetic to real data is indeed central and not directly tested in the current experiments. We will enhance the discussion section to better articulate the rationale behind this assumption with references to analogous transfer learning results in audio domains, and we will clearly label it as a limitation requiring further validation. revision: yes

standing simulated objections not resolved
  • Direct quantitative evaluation of velocity prediction quality on real guitar audio, as no ground-truth velocity labels exist for real recordings.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: standard transfer learning with independent empirical evaluation

full rationale

The paper describes pretraining a velocity prediction model on synthetic guitar data generated by virtual instruments, then transferring those weights to a separate model fine-tuned on real unlabeled guitar recordings. Performance is assessed via direct comparison to a non-pretrained baseline on held-out synthetic data and via note transcription metrics on real data. No equations, fitted parameters, or self-citations are presented that reduce the claimed improvements to inputs by construction; the central claims rest on standard supervised pretraining plus transfer, which remain falsifiable against external benchmarks.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central claim rests on the domain assumption that virtual-instrument velocity labels are sufficiently realistic to transfer to real guitar audio; no free parameters or invented entities are explicitly introduced in the abstract.

assumptions (1)
  • domain assumption Velocity labels generated by virtual guitar instruments capture transferable acoustic properties of real guitar playing intensity.
    Invoked to justify pretraining on synthetic data before real-data fine-tuning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Velocity Prediction in Automatic Guitar Transcription." pith.science (2026). https://pith.science/paper/W57XBJSH

@misc{pith2026260624912,
  author       = {Pith},
  title        = {Pith review of: Velocity Prediction in Automatic Guitar Transcription},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/W57XBJSH}},
  note         = {Machine review of arXiv:2606.24912}
}
read the original abstract

Automatic Music Transcription (AMT) models have achieved a high level of success in polyphonic transcription of various instruments. Velocity, typically a measure of note intensity, is less commonly predicted in these models due to the absence of velocity labels in available datasets and lack of a proper definition for instruments other than piano. We present a methodology and model for velocity prediction in Automatic Guitar Transcription (AGT) which uses virtual instruments to generate synthetic training data with velocity labels. We first pretrain a model on this synthetic data. These weights are then transferred to a different model and trained on real guitar audio, allowing the model to retain the working velocity prediction while also achieving high performance and generalisability from the real training data. The velocity prediction is shown to outperform a baseline model which does not use the pretrained velocity weights, when evaluated on synthetic data. In addition, using the pretrained velocity weights offers a small improvement in note transcription, though the magnitude of this improvement is limited and not always significant depending on the testing data. Overall the model achieves results comparable to the state of the art in guitar transcription, while also successfully predicting velocity.

Figures

Figures reproduced from arXiv: 2606.24912 by the authors.

Figure 1
Figure 1. Visualization of the training methodology. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 2 canonical work pages

  1. [1]

    Automatic music transcription: An overview,

    E. Benetos, S. Dixon, Z. Duan, and S. Ewert, “Automatic music transcription: An overview,”IEEE Signal Processing Magazine, vol. 36, no. 1, pp. 20–30, 2018

    2018

  2. [2]

    Guitarset: A dataset for guitar transcription,

    Q. Xi, R. M. Bittner, J. Pauwels, X. Ye, and J. P. Bello, “Guitarset: A dataset for guitar transcription,”Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2018

    2018

  3. [3]

    Automatic tabla- ture transcription of electric guitar recordings by estimation of score- and instrument-related parameters,

    C. Kehling, J. Abeßer, C. Dittmar, and G. Schuller, “Automatic tabla- ture transcription of electric guitar recordings by estimation of score- and instrument-related parameters,” inProceedings of the International Conference on Digital Audio Effects (DAFx), 2014, pp. 219–226

    2014

  4. [4]

    Towards automatic transcription of polyphonic electric guitar music: A new dataset and a multi-loss transformer model,

    Y .-H. Chen, W.-Y . Hsiao, T.-K. Hsieh, J.-S. R. Jang, and Y .-H. Yang, “Towards automatic transcription of polyphonic electric guitar music: A new dataset and a multi-loss transformer model,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 786–790

    2022

  5. [5]

    Gaps: A large and diverse classical guitar dataset and benchmark transcription model,

    J. Riley, Z. Guo, D. Edwards, S. Dixonet al., “Gaps: A large and diverse classical guitar dataset and benchmark transcription model,” in25th International Society for Music Information Retrieval (ISMIR) Conference, 2024

    2024

  6. [6]

    Guitar-techs: An electric guitar dataset covering techniques, musical excerpts, chords and scales using a diverse array of hardware,

    H. Pedroza, W. Abreu, R. M. Corey, and I. R. Roman, “Guitar-techs: An electric guitar dataset covering techniques, musical excerpts, chords and scales using a diverse array of hardware,”IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025

    2025

  7. [7]

    Goat: A large dataset of paired guitar audio recordings and tablatures,

    J. Loth, P. Sarmento, S. Sarkar, Z. Guo, M. Barthet, and M. Sandler, “Goat: A large dataset of paired guitar audio recordings and tablatures,” in26th International Society for Music Information Retrieval (ISMIR) Conference, 2025

    2025

  8. [8]

    Unaligned supervision for automatic music transcription in the wild,

    B. Maman and A. H. Bermano, “Unaligned supervision for automatic music transcription in the wild,” inInternational Conference on Machine Learning. PMLR, 2022, pp. 14 918–14 934

    2022

Show all 27 references
  1. [9]

    High resolution guitar transcrip- tion via domain adaptation,

    X. Riley, D. Edwards, and S. Dixon, “High resolution guitar transcrip- tion via domain adaptation,” inInternational Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2024, pp. 1051–1055

    2024

  2. [10]

    Count the notes: Histogram-based supervision for automatic music transcription,

    J. Yaffe, B. Maman, M. M ¨uller, and A. H. Bermano, “Count the notes: Histogram-based supervision for automatic music transcription,” Proceedings of the International Society for Music Information Retrieval Conference (ISMIR), 2025

    2025

  3. [11]

    The interpretation of midi velocity,

    R. B. Dannenberg, “The interpretation of midi velocity,” inICMC, 2006

    2006

  4. [12]

    Onsets and frames: Dual-objective piano transcription,

    C. Hawthorne, E. Elsen, J. Song, A. Roberts, I. Simon, C. Raffel, J. Engel, S. Oore, and D. Eck, “Onsets and frames: Dual-objective piano transcription,”arXiv preprint arXiv:1710.11153, 2017

    2017 arXiv

  5. [13]

    High-resolution piano transcription with pedals by regressing onset and offset times,

    Q. Kong, B. Li, X. Song, Y . Wan, and Y . Wang, “High-resolution piano transcription with pedals by regressing onset and offset times,”IEEE ACM Transactions on Audio, Speech and Language Processing, vol. 29, pp. 3707–3717, 2021

    2021

  6. [14]

    Enabling factorized piano music modeling and generation with the MAESTRO dataset,

    C. Hawthorne, A. Stasyuk, A. Roberts, I. Simon, C.-Z. A. Huang, S. Dieleman, E. Elsen, J. Engel, and D. Eck, “Enabling factorized piano music modeling and generation with the MAESTRO dataset,” in International Conference on Learning Representations (ICLR), 2019

    2019

  7. [15]

    A data- driven analysis of robust automatic piano transcription,

    D. Edwards, S. Dixon, E. Benetos, A. Maezawa, and Y . Kusaka, “A data- driven analysis of robust automatic piano transcription,”IEEE Signal Processing Letters, vol. 31, pp. 681–685, 2024

    2024

  8. [16]

    Saarland music data (smd),

    M. M ¨uller, V . Konz, W. Bogler, and V . Arifi-M ¨uller, “Saarland music data (smd),” inInternational Society for Music Information Retrieval (ISMIR) Conference Late Breaking Session, 2011

    2011

  9. [17]

    Estimating note intensities in music record- ings,

    S. Ewert and M. M ¨uller, “Estimating note intensities in music record- ings,” in2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2011, pp. 385–388

    2011

  10. [18]

    Score-informed MIDI velocity estimation for piano performance by film conditioning,

    H. Kim, M. Miron, and X. Serra, “Score-informed MIDI velocity estimation for piano performance by film conditioning,” inSound and Music Computing Conference, 2023

    2023

  11. [19]

    Diffvel: Note-level midi velocity estimation for piano performance by a double conditioned diffusion model,

    H. Kim and X. Serra, “Diffvel: Note-level midi velocity estimation for piano performance by a double conditioned diffusion model,” inIn- ternational Symposium on Computer Music Multidisciplinary Research. Springer, 2023, pp. 349–361

    2023

  12. [20]

    A method for midi velocity estimation for piano performance by a u-net with attention and film,

    ——, “A method for midi velocity estimation for piano performance by a u-net with attention and film,” inInternational Society for Music Information Retrieval (ISMIR) Conference, 2024

    2024

  13. [21]

    A timbre-based approach to estimate key velocity from polyphonic piano recordings

    D. Jeong, T. Kwon, and J. Nam, “A timbre-based approach to estimate key velocity from polyphonic piano recordings.” in19th International Society for Music Information Retrieval (ISMIR) Conference, 2018, pp. 120–127

    2018

  14. [22]

    Note-intensity estimation of piano recordings using coarsely aligned MIDI score,

    ——, “Note-intensity estimation of piano recordings using coarsely aligned MIDI score,”Journal of the Audio Engineering Society, vol. 68, no. 1/2, pp. 34–47, 2020

    2020

  15. [23]

    Learn from virtual guitar: A comparative analysis of automatic guitar transcription using synthetic and real audio,

    Y . Kusaka and A. Maezawa, “Learn from virtual guitar: A comparative analysis of automatic guitar transcription using synthetic and real audio,” in2025 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA). IEEE, 2025, pp. 1–5

    2025

  16. [24]

    Fine-grained midi expression tran- scription from wind and string instrument audio via sim2real transfer learning,

    Y . Xie, Z. Guo, and M. Barthet, “Fine-grained midi expression tran- scription from wind and string instrument audio via sim2real transfer learning,” inThe 17th International Symposium on Computer Music Multidisciplinary Research (CMMR), 2025

    2025

  17. [25]

    Synthtab: Leveraging synthesized data for guitar tablature transcription,

    Y . Zang, Y . Zhong, F. Cwitkowitz, and Z. Duan, “Synthtab: Leveraging synthesized data for guitar tablature transcription,” inIEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2024, pp. 1286–1290

    2024

  18. [26]

    Pedalboard,

    P. Sobot, “Pedalboard,” Jul. 2021. [Online]. Available: https://doi.org/ 10.5281/zenodo.7817838

    2021 doi

  19. [27]

    mir eval: A transparent implementation of common mir metrics,

    C. Raffel, B. McFee, E. J. Humphrey, J. Salamon, O. Nieto, D. Liang, and D. P. W. Ellis, “mir eval: A transparent implementation of common mir metrics,” inProceedings of the 15th International Society for Music Information Retrieval Conference, Taipei, Taiwan, 2014, pp. 367–372

    2014

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.