Piano Transcription by Hierarchical Language Modeling with Pretrained Roll-based Encoders

Dichucheng Li; Qiuqiang Kong; Yongyi Zang

REVIEW 4 major objections 4 minor 29 references

Piano Transcription by Hierarchical Language Modeling with Pretrained Roll-based Encoders

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A hierarchical language-model decoder lifts piano transcription note-level F1 by 0.010 to 0.022 over conventional piano-roll outputs.

desk verdict Plausible plug-in architecture with an honest scaling study, but the headline gain over roll baselines is not yet trustworthy because the baseline threshold and post-processing are never disclosed. read the letter →

arxiv 2501.03038 v2 pith:EFJY4OYW submitted 2025-01-06 cs.SD cs.AIcs.LGeess.AS

Dichucheng Li , Yongyi Zang , Qiuqiang Kong This is my paper · ORCID

classification cs.SDcs.AIcs.LGeess.AS

keywords automaticmusictranscriptionpianolanguagemodeldecoderhierarchicalpredictionpiano-rollencodernote-levelF1MAESTROdatasetvelocity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a hybrid piano transcription system: a pretrained frame-level piano-roll encoder is kept as the audio front end, but its final readout is replaced by a decoder-only language model that emits note events token by token. To make the token sequence tractable, the paper predicts onset and pitch first, then velocity, then offset, using three separate language models guided by query tokens instead of one flattened sequence. The authors report that, on two benchmark roll-based encoders, the hierarchical language-model decoder beats the traditional thresholded piano-roll output by 0.010 and 0.022 in onset-offset-velocity F1 on the MAESTRO dataset. The intended significance is a plug-in upgrade path: existing roll-based encoders could gain note-level accuracy and shed manual thresholding without being redesigned. The paper also claims that encoder choice matters more than language-model size, and that velocity prediction is the part most prone to overfitting.

What carries the argument

The carrying mechanism is a hierarchical factorization of each note event into three conditionally dependent token streams: onset plus pitch, then velocity, then offset. Each stream is generated by its own decoder-only Transformer language model, conditioned on frame-level embeddings extracted from the pretrained roll encoder and steered by task-specific query tokens $q_p$, $q_v$, and $q_f$; the factorization is stated as $p(o_n,p_n|x_1,y_{<n},q_p) p(v_n|x_1,y_{<n},o_n,p_n,q_v) p(f_n|x_1,y_{<n},o_n,p_n,v_n,q_f)$. Splitting the flattened note-event sequence into three shorter streams changes the attention cost from $O((T+3N)^2D)$ to $O(3(T+N)^2D)$ and, according to the paper, is what stops the language-model decoder from degrading on long 10-second segments.

What would settle it

Re-run the two roll-based systems with a systematic threshold sweep on a validation set (or publish the exact threshold and post-processing pipeline used), then compare the best tuned roll F1 with the hierarchical LM F1; if the tuned roll outputs match or exceed the hierarchical system's onset-offset-velocity F1, the claimed plug-in advantage collapses.

Watch

Extended reading notes

Core claim

The central claim is that attaching a hierarchical language-model decoder to a pretrained roll-based encoder yields more accurate note-level piano transcription than decoding the same encoder's piano roll with a threshold. In the paper's experiments, replacing the roll decoder with the hierarchical LM decoder improves onset-offset-velocity F1 by 0.010 with the CRNN encoder and by 0.022 with the HPPNet encoder, while matching or exceeding the roll system's onset and onset-offset scores and removing the need for threshold-based post-processing. The paper presents this as evidence that the hierarchical LM decoder can act as a performance-enhancing plug-in for arbitrary roll-based transcription encoders, and additionally finds that the choice of audio encoder affects the final score far more than the size of the language-model decoder.

Load-bearing premise

The reported gains rest on how the two roll baselines were thresholded and post-processed, and the paper never states the threshold value or the search procedure used for them, so if those baselines were not tuned fairly, the 0.010 and 0.022 improvements could be inflated.

Editorial extensions

If this is right

On the two tested encoders, swapping the piano-roll decoder for the hierarchical LM decoder raises onset-offset-velocity F1 by 0.010 (CRNN encoder) and 0.022 (HPPNet encoder).
The hierarchical system produces note outputs without thresholding or post-processing, unlike the roll baseline it is compared against.
Flattened note-token sequences degrade on 10-second segments, especially on offset prediction; the hierarchical split avoids that degradation.
Language-model decoder size has little effect on final F1, and velocity tokens overfit early during training, so encoder choice rather than decoder scale drives performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the paper tests only two encoders on one dataset, the plug-in claim is a prediction for the broader family of roll-based encoders; the natural next experiment is to attach the same decoder to other pretrained roll systems and check whether the 0.010-0.022 gain is consistent.
The three-way split makes decoder overfitting visible: velocity loss rises early while onset-pitch and offset plateau, so a likely extension is to regularize velocity training specifically rather than enlarging the whole decoder.
The same query-token hierarchy could transfer beyond piano to any event stream with causally ordered attributes, such as multi-instrument note events, giving a testable way to see whether the factorization's benefit is general or piano-specific.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the hierarchical decoder idea is plausible and the scaling study is honest, but the main empirical claim is not yet secured because the roll-baseline threshold and post-processing are undisclosed, and there are no error bars.

What's new: they take two pretrained roll-based piano transcription encoders (CRNN, HPPNet) and attach a decoder-only LLaMA-style LM that predicts notes hierarchically: onset+pitch first, then velocity, then offset. The hierarchical token decomposition keeps sequence lengths short, which is a real practical concern for LM-based AMT. The evaluation on the external Maestro test set means the headline improvement over the roll baselines (0.010 and 0.022 on On Off Vel F1) is not circular by construction. The scaling experiments are a genuine strength: they show larger LM decoders don't help and velocity overfits early, corroborating a known observation and pointing at encoder choice as the more important factor.

Soft spots, in proportion: the load-bearing comparison is the hierarchy vs. the roll baseline, and the roll threshold is never reported. Section V explicitly says the Roll approach needs a threshold, but never says what threshold was used, how it was tuned, or what post-processing (note merging, minimum duration, offset smoothing) was applied. With a gap of 1-2 F1 points, threshold choice can easily erase the claimed gain. There are also no error bars or multiple seeds, so we cannot tell whether 0.010/0.022 is signal or noise. The paper also mixes two architectural descriptions: it says both 'task-specific query tokens' and 'three models' for the three hierarchies. These are different designs with different complexity profiles, and the complexity reduction claim depends on which one is true. The flattened baseline collapses to 0.39 On Off F1 without a demonstrated cause; the paper offers a hypothesis about segment length, but does not verify it. Finally, the state-of-the-art claim is limited to LM-based piano transcription and is compared against Hawthorne et al. trained on different segment lengths, so it is not a controlled comparison.

Bottom line: the core idea is worth pursuing and the paper is coherent on its own terms, but the central quantitative claim needs more evidence. A serious referee should ask for the threshold and post-processing details, error bars or significance tests, and a clarification of the architecture. I would send it to review, not desk-reject, but I would not rely on the reported gains until those gaps are closed.

Referee Report

4 major / 4 minor

Summary. The paper proposes a hybrid piano transcription system that couples a pretrained roll-based audio encoder (CRNN or HPPNet) with a decoder-only language model. The LM decoder predicts note events in three hierarchical stages: onset and pitch, then velocity, then offset, using task-specific query tokens. On the Maestro test set, the hierarchical decoder improves over the traditional roll output by 0.010 and 0.022 in onset-offset-velocity F1 for CRNN and HPPNet, respectively. The authors also report scaling experiments showing that larger LM decoders do not improve performance and argue that the hierarchical decoder can serve as a performance-enhancing plug-in for arbitrary roll-based transcription encoders.

Significance. If the reported gains are robust, the proposed hierarchical LM decoder is a simple and potentially impactful way to upgrade existing roll-based AMT systems: it removes manual thresholding and improves note-level F1 without retraining the encoder. The paper usefully separates encoder and decoder contributions and corroborates prior evidence that larger LM decoders overfit on the transcription task. However, the central quantitative claim rests on a roll-baseline comparison whose threshold and post-processing are not reported, and no uncertainty estimates are provided, so the magnitude of the claimed plug-in improvement is not yet fully supported.

major comments (4)

[Section V, Table I] The comparison between the hierarchical LM decoder and the Roll baselines is not reproducible because the thresholding and post-processing for CRNN Roll and HPPNet Roll are never described. The text states that 'the Roll approach requires setting a threshold to gate notes as posterior information,' but it does not report the threshold value, the search procedure (e.g., grid search on the validation set), or the post-processing steps such as note merging, minimum duration filtering, or offset smoothing. Since the reported gains are only 0.010 and 0.022 in onset-offset-velocity F1, an arbitrary or poorly tuned threshold could erase or inflate these differences. Please report the exact decoding pipeline for both roll baselines, including the threshold and any post-processing, or the central plug-in claim is not supported.
[Section V, Table I] No error bars, confidence intervals, or multiple-seed results are reported for any of the conditions in Table I. The claimed improvements of 0.010 and 0.022 are small enough that they could fall within run-to-run variance, particularly given the 1e-5 learning rate and the maximum step count of 1 million without an explicit early-stopping criterion. Please report the mean and standard deviation over at least three independent training runs, or apply a paired significance test over the test set, to establish that the differences are not noise.
[Section V, Table I] The flattened baseline collapses to onset-offset F1 values of 0.393 and 0.390 for CRNN and HPPNet, respectively, which is drastically below the corresponding roll baselines (0.825 and 0.822) and below Hawthorne et al. [14] (0.839 and 0.828). The text attributes this to the longer 10-second segment and the decoder-only architecture, but the magnitude of the drop is far larger than typical sequence-length effects, and no diagnostic evidence is provided. If the flattened baseline is impaired by a training or decoding artifact, its comparison to the hierarchy is uninformative for the claim that hierarchical prediction helps. Either fix the flattened baseline (for example, by evaluating it with shorter segments) or supply direct evidence, such as sequence-length statistics and loss curves, that the stated cause explains the collapse.
[Section III, Eqs. (4)-(8)] The formalization of the hierarchical model is internally inconsistent. Equation (8) writes the probability of a note as a product of three separate language models p_onset-pitch, p_velocity, and p_offset, and Section IV says 'we trained three models with the same model architecture.' In contrast, Eq. (4) describes a single decoder f_dec conditioned on a query token q, and the token dictionary includes 'task-specific query tokens' with vocabulary size three. Please clarify whether the three stages share parameters or are three independent models, and how the query token is used in each case. This also affects the interpretation of the parameter counts in Table I and the claimed threefold complexity reduction.

minor comments (4)

[Section II.A] Typo: 'an feature' should be 'a feature'.
[Section III] The complexity comparison states that the flattened single-model approach costs O((T + 3N)^2D), but a flattened sequence in Fig. 2(a) contains four tokens per note (onset, pitch, velocity, offset), so the complexity should be O((T + 4N)^2D). Please correct the formula or justify the N-count.
[Figure 2 caption] The caption says 'Two lines on o and p tokens denote that they are fixed from b1 during b2 and b3.' It should say 'The dashed lines' or 'The two dotted lines' to clarify what is being depicted.
[Section V] The statement 'We have also achieved new state-of-the-art result of the LM-based piano transcription model' is supported only by comparison to two Hawthorne et al. [14] models. Please specify the set of prior LM-based systems considered and, if possible, compare against additional recent LM-based transcription models to make the SOTA claim meaningful.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the hierarchical LM factorization and the MAESTRO-based evaluation are independent of the fitted roll outputs.

full rationale

No load-bearing circular step is present. The core modeling claim is the factorized sequence probability in Eq. (8), p(yn)=p(on,pn|...)*p(vn|...)*p(dn|...), implemented as three separately trained LMs over a pretrained roll encoder. This is a genuine modeling decomposition rather than a restatement of the encoder output. The encoder H=fenc(X) is pretrained with a binary cross-entropy roll objective, while the LM decoder is trained with a negative log-likelihood objective; neither objective is defined in terms of the reported onset-offset-velocity F1 metric, and the evaluation is performed on the external MAESTRO test split against ground-truth MIDI using mir_eval. The table entries 'CRNN Roll' and 'CRNN Hierarchy' share the same encoder but differ in decoder and in threshold-based post-processing, so the reported gaps of 0.010 and 0.022 are empirical measurements, not construction identities. The only self-citation is the use of Kong et al. [10], by the corresponding author, as one of two benchmark encoders; that is a standard model-choice citation, not a load-bearing argument whose conclusion presupposes the paper's claim. A real reporting weakness is that Section V states that the 'Roll' approach 'requires setting a threshold to gate notes as posterior information' without disclosing the threshold value or the post-processing pipeline used for the roll baselines; this is a reproducibility and fairness concern that could affect the magnitude of the claimed improvement, but it does not make the derivation circular. No equation in the paper is equivalent to its input by definition.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claim is an empirical result on Maestro, not a mathematical derivation. The paper introduces no new physical entities. Its main assumptions are that the pretrained encoders produce informative embeddings, that the Maestro ground truth is reliable, that the mir_eval tolerances are the right metric, and that T << N for the complexity reduction. The hand-chosen hyperparameters (decoder size, segment length, time resolution) influence the reported numbers but are not fitted to the test set. The unspecified roll threshold is a missing detail that affects the baseline comparison.

free parameters (3)

LM decoder size (base setting) = 6 layers, 16 heads, 1024 embedding
Table I uses the base decoder; Table III shows tiny, small, and large all underperform base, so the headline gains are sensitive to this architectural choice, which was selected by hand rather than fitted to the test metric.
Audio segment length = 10 seconds
The paper retrains HPPNet on 10-second segments for fair comparison and uses 10-second segments for CRNN; the flattened baseline degrades sharply at this length, so the segment choice materially affects the comparison to [14].
Roll baseline threshold
The Roll baselines in Table I require a gating threshold and post-processing, but the value is never reported. The central comparison depends on this number; an untuned threshold would inflate the reported improvement.

assumptions (5)

domain assumption Pretrained roll-based encoders (CRNN, HPPNet) provide frame-level embeddings H that retain enough musical information for a language model decoder to transcribe note events.
Invoked throughout Section III; if H is too lossy, the LM decoder cannot recover notes, and the whole method fails. The paper only tests two encoders, so this is an empirical assumption.
domain assumption Maestro ground-truth MIDI alignments are accurate enough to serve as training targets and evaluation labels.
Section IV-C relies on the official Maestro split and the reported 3 ms alignment error; the F1 scores inherit any noise in the ground truth.
domain assumption The evaluation tolerances from mir_eval (onset +-50 ms, pitch +-50 cents, velocity 0.1, offset +-50 ms or 20% duration) are the appropriate measure of transcription correctness.
Section IV-E fixes these tolerances; changing them would change the reported F1 and the ranking of methods.
domain assumption The complexity reduction assumes T << N, as stated in Section III.
The claim of 'almost threefold reduction' uses T << N; for short segments or dense music this may not hold.
domain assumption Standard deep learning optimization assumptions: AdamW with learning rate 1e-5 converges for both encoders, and training on 8 RTX 4090s does not introduce batch-size-dependent effects.
Section IV-D gives one hyperparameter setting; no sensitivity analysis is provided.

invented entities (1)

Task-specific query tokens (q_p, q_v, q_d)
purpose: Condition the decoder to predict onset-pitch, velocity, or offset within the hierarchical architecture.
These tokens (Section III, Eq. 8) are internal vocabulary elements. They make no testable prediction about audio or music outside the trained model, and their usefulness is measured only by the empirical F1 differences in Table I.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Piano Transcription by Hierarchical Language Modeling with Pretrained Roll-based Encoders." pith.science (2026). https://pith.science/paper/EFJY4OYW

@misc{pith2026250103038,
  author       = {Pith},
  title        = {Pith review of: Piano Transcription by Hierarchical Language Modeling with Pretrained Roll-based Encoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EFJY4OYW}},
  note         = {Machine review of arXiv:2501.03038}
}

read the original abstract

Automatic Music Transcription (AMT), aiming to get musical notes from raw audio, typically uses frame-level systems with piano-roll outputs or language model (LM)-based systems with note-level predictions. However, frame-level systems require manual thresholding, while the LM-based systems struggle with long sequences. In this paper, we propose a hybrid method combining pre-trained roll-based encoders with an LM decoder to leverage the strengths of both methods. Besides, our approach employs a hierarchical prediction strategy, first predicting onset and pitch, then velocity, and finally offset. The hierarchical prediction strategy reduces computational costs by breaking down long sequences into different hierarchies. Evaluated on two benchmark roll-based encoders, our method outperforms traditional piano-roll outputs 0.01 and 0.022 in onset-offset-velocity F1 score, demonstrating its potential as a performance-enhancing plug-in for arbitrary roll-based music transcription encoder.

Figures

Figures reproduced from arXiv: 2501.03038 by the authors.

**Figure 3.** Onset-pitch, velocity and offset loss on the training set (solid [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 18 canonical work pages

[14]

Sequence-to-sequence piano transcription with transformers,

C. Hawthorne, I. Simon, R. Swavely, E. Manilow, and J. Engel, “Sequence-to-sequence piano transcription with transformers,” in Pro- ceedings of the 22nd International Society for Music Information Re- trieval Conference, ISMIR , 2021

work page 2021
[1]

Automatic music transcription: An overview,

E. Benetos, S. Dixon, Z. Duan, and S. Ewert, “Automatic music transcription: An overview,” IEEE Signal Processing Magazine , vol. 36, no. 1, pp. 20–30, 2018

work page 2018
[2]

An approach to score following for piano perfor- mances with the sustained effect,

B. Li and Z. Duan, “An approach to score following for piano perfor- mances with the sustained effect,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 12, pp. 2425–2438, 2016

work page 2016
[3]

A multi-pass algorithm for accurate audio-to-score alignment

B. Niedermayer and G. Widmer, “A multi-pass algorithm for accurate audio-to-score alignment.” in Proceedings of the 11th International Society for Music Information Retrieval Conference, ISMIR , 2010, pp. 417–422

work page 2010
[4]

An iterative multi range non-negative matrix factorization algorithm for polyphonic music transcription

A. Khlif and V . Sethu, “An iterative multi range non-negative matrix factorization algorithm for polyphonic music transcription.” in Proceed- ings of the 16th International Society for Music Information Retrieval Conference, ISMIR, 2015, pp. 330–335

work page 2015
[5]

Adaptive harmonic spectral decomposition for multiple pitch estimation,

E. Vincent, N. Bertin, and R. Badeau, “Adaptive harmonic spectral decomposition for multiple pitch estimation,” IEEE Transactions on Audio, Speech, and Language Processing , vol. 18, no. 3, pp. 528–537, 2009

work page 2009
[6]

A classification-based polyphonic piano transcription approach using learned feature repre- sentations

J. Nam, J. Ngiam, H. Lee, M. Slaney et al. , “A classification-based polyphonic piano transcription approach using learned feature repre- sentations.” in Proceedings of the 12th International Society for Music Information Retrieval Conference, ISMIR , 2011, pp. 175–180

work page 2011
[7]

On the potential of simple framewise approaches to piano tran- scription,

R. Kelz, M. Dorfer, F. Korzeniowski, S. B ¨ock, A. Arzt, and G. Wid- mer, “On the potential of simple framewise approaches to piano tran- scription,” in Proceedings of the 17th International Society for Music Information Retrieval Conference, ISMIR , 2016

work page 2016

Show all 29 references

[8]

An end-to-end neural network for polyphonic piano music transcription,

S. Sigtia, E. Benetos, and S. Dixon, “An end-to-end neural network for polyphonic piano music transcription,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 24, no. 5, pp. 927–939, 2016

2016
[9]

Onsets and frames: Dual-objective piano transcription,

C. Hawthorne, E. Elsen, J. Song, A. Roberts, I. Simon, C. Raffel, J. Engel, S. Oore, and D. Eck, “Onsets and frames: Dual-objective piano transcription,” in Proceedings of the 19th International Society for Music Information Retrieval Conference, ISMIR , 2018, pp. 50–57

2018
[10]

High-resolution piano transcription with pedals by regressing onset and offset times,

Q. Kong, B. Li, X. Song, Y . Wan, and Y . Wang, “High-resolution piano transcription with pedals by regressing onset and offset times,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , vol. 29, pp. 3707–3717, 2021

2021
[11]

Hppnet: Modeling the harmonic struc- ture and pitch invariance in piano transcription,

W. Wei, P. Li, Y . Yu, and W. Li, “Hppnet: Modeling the harmonic struc- ture and pitch invariance in piano transcription,” in Proceedings of the 23th International Society for Music Information Retrieval Conference, ISMIR, 2022

2022
[12]

Triad: Capturing har- monics with 3d convolutions,

M. P. Fernandez, H. Kirchhoff, X. Serra et al. , “Triad: Capturing har- monics with 3d convolutions,” in Proceedings of the 24th International Society for Music Information Retrieval Conference, ISMIR , 2023

2023
[13]

Automatic piano transcription with hierarchical frequency-time transformer,

K. Toyama, T. Akama, Y . Ikemiya, Y . Takida, W.-H. Liao, and Y . Mit- sufuji, “Automatic piano transcription with hierarchical frequency-time transformer,” in Proceedings of the 24th International Society for Music Information Retrieval Conference, ISMIR , 2023

2023
[15]

Audio captioning transformer,

X. Mei, X. Liu, Q. Huang, M. D. Plumbley, and W. Wang, “Audio captioning transformer,” arXiv preprint arXiv:2107.09817 , 2021

2021 arXiv
[16]

Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,

X. Mei, C. Meng, H. Liu, Q. Kong, T. Ko, C. Zhao, M. D. Plumbley, Y . Zou, and W. Wang, “Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research,” IEEE/ACM Transactions on Audio, Speech, and Language Processing , 2024

2024
[17]

Cacophony: An improved contrastive audio-text model,

G. Zhu and Z. Duan, “Cacophony: An improved contrastive audio-text model,” arXiv preprint arXiv:2402.06986 , 2024

2024 arXiv
[18]

Enclap: Combining neural audio codec and audio-text joint embedding for automated audio captioning,

J. Kim, J. Jung, J. Lee, and S. H. Woo, “Enclap: Combining neural audio codec and audio-text joint embedding for automated audio captioning,” in IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP. IEEE, 2024, pp. 6735–6739

2024
[19]

Efficient audio captioning with encoder-level knowledge distillation,

X. Xu, H. Liu, M. Wu, W. Wang, and M. D. Plumbley, “Efficient audio captioning with encoder-level knowledge distillation,” arXiv preprint arXiv:2407.14329, 2024

2024 arXiv
[20]

Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,

Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu et al. , “Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,...

2024
[21]

Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,

Y . Chu, J. Xu, X. Zhou, Q. Yang, S. Zhang, Z. Yan, C. Zhou, and J. Zhou, “Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models,” arXiv preprint arXiv:2311.07919 , 2023

2023 arXiv
[22]

Music understanding llama: Advancing text-to-music generation with question answering and captioning,

S. Liu, A. S. Hussain, C. Sun, and Y . Shan, “Music understanding llama: Advancing text-to-music generation with question answering and captioning,” in IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP . IEEE, 2024, pp. 286–290

2024
[23]

Musilingo: Bridging music and text with pre-trained language models for music captioning and query response,

Z. Deng, Y . Ma, Y . Liu, R. Guo, G. Zhang, W. Chen, W. Huang, and E. Benetos, “Musilingo: Bridging music and text with pre-trained language models for music captioning and query response,” arXiv preprint arXiv:2309.08730, 2023

2023 arXiv
[24]

Exploring the limits of transfer learning with a unified text-to-text transformer,

C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020

2020
[25]

Llama 2: Open foundation and fine-tuned chat models,

H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

2023 arXiv
[26]

Roformer: Enhanced transformer with rotary position embedding,

J. Su, M. Ahmed, Y . Lu, S. Pan, W. Bo, and Y . Liu, “Roformer: Enhanced transformer with rotary position embedding,” Neurocomputing, vol. 568, p. 127063, 2024

2024
[27]

Enabling factorized piano music modeling and generation with the maestro dataset,

C. Hawthorne, A. Stasyuk, A. Roberts, I. Simon, C.-Z. A. Huang, S. Dieleman, E. Elsen, J. Engel, and D. Eck, “Enabling factorized piano music modeling and generation with the maestro dataset,” in International Conference on Learning Representations, ICLR
[28]

Decoder-only or encoder-decoder? interpreting language model as a regularized encoder-decoder,

Z. Fu, W. Lam, Q. Yu, A. M.-C. So, S. Hu, Z. Liu, and N. Collier, “Decoder-only or encoder-decoder? interpreting language model as a regularized encoder-decoder,” arXiv preprint arXiv:2304.04052 , 2023

2023 arXiv
[29]

Scaling laws for neural language models,

J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei, “Scaling laws for neural language models,” arXiv preprint arXiv:2001.08361 , 2020

2001 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [14]

Sequence-to-sequence piano transcription with transformers,

C. Hawthorne, I. Simon, R. Swavely, E. Manilow, and J. Engel, “Sequence-to-sequence piano transcription with transformers,” in Pro- ceedings of the 22nd International Society for Music Information Re- trieval Conference, ISMIR , 2021

work page 2021

[2] [1]

Automatic music transcription: An overview,

E. Benetos, S. Dixon, Z. Duan, and S. Ewert, “Automatic music transcription: An overview,” IEEE Signal Processing Magazine , vol. 36, no. 1, pp. 20–30, 2018

work page 2018

[3] [2]

An approach to score following for piano perfor- mances with the sustained effect,

B. Li and Z. Duan, “An approach to score following for piano perfor- mances with the sustained effect,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 24, no. 12, pp. 2425–2438, 2016

work page 2016

[4] [3]

A multi-pass algorithm for accurate audio-to-score alignment

B. Niedermayer and G. Widmer, “A multi-pass algorithm for accurate audio-to-score alignment.” in Proceedings of the 11th International Society for Music Information Retrieval Conference, ISMIR , 2010, pp. 417–422

work page 2010

[5] [4]

An iterative multi range non-negative matrix factorization algorithm for polyphonic music transcription

A. Khlif and V . Sethu, “An iterative multi range non-negative matrix factorization algorithm for polyphonic music transcription.” in Proceed- ings of the 16th International Society for Music Information Retrieval Conference, ISMIR, 2015, pp. 330–335

work page 2015

[6] [5]

Adaptive harmonic spectral decomposition for multiple pitch estimation,

E. Vincent, N. Bertin, and R. Badeau, “Adaptive harmonic spectral decomposition for multiple pitch estimation,” IEEE Transactions on Audio, Speech, and Language Processing , vol. 18, no. 3, pp. 528–537, 2009

work page 2009

[7] [6]

A classification-based polyphonic piano transcription approach using learned feature repre- sentations

J. Nam, J. Ngiam, H. Lee, M. Slaney et al. , “A classification-based polyphonic piano transcription approach using learned feature repre- sentations.” in Proceedings of the 12th International Society for Music Information Retrieval Conference, ISMIR , 2011, pp. 175–180

work page 2011

[8] [7]

On the potential of simple framewise approaches to piano tran- scription,

R. Kelz, M. Dorfer, F. Korzeniowski, S. B ¨ock, A. Arzt, and G. Wid- mer, “On the potential of simple framewise approaches to piano tran- scription,” in Proceedings of the 17th International Society for Music Information Retrieval Conference, ISMIR , 2016

work page 2016