Pith. sign in

REVIEW 3 major objections 4 minor 27 references

LSTM Based Music Generation System

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes and demonstrates a single-layer LSTM network that learns polyphonic note sequences from MIDI files and generates new melodies, reporting 97.23% training accuracy and a log-likelihood of -6.23 after only 0.9 hours on…

desk verdict A reproducible single-layer LSTM music-gen baseline whose central quantitative claim rests on an invalid cross-dataset comparison and training-only metrics. read the letter →

arxiv 1908.01080 v1 pith:5FG6MXOQ submitted 2019-08-02 cs.SD cs.LGeess.ASstat.ML

Sanidhya Mangal , Rahul Modak , Poorva Joshi This is my paper · ORCID
classification cs.SDcs.LGeess.ASstat.ML
keywords LSTMmusicgenerationMIDIrecurrentneuralnetworkspolyphonicnotematrixdeeplearningmelody
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a single-layer Long Short-Term Memory (LSTM) network can learn the sequential structure of polyphonic music and then generate new melodies without human intervention. It describes a complete pipeline: MIDI files are converted into scaled “Note Matrix” tensors holding note, velocity, and time interval, then fed into a 512-unit LSTM with dropout, a dense layer, and linear activation. On the Pop Music dataset, training for about 0.9 hours produced 97.23% accuracy and a log-likelihood of $-6.23$, which the authors compare with biaxial LSTM systems that required 16–48 hours. The contribution, read sympathetically, is a simpler architecture that learns musical sequences quickly from MIDI data.

What carries the argument

The object that carries the argument is the LSTM cell's gated memory: a hidden state that carries information from earlier time steps into the current prediction, letting the network recall the structure of a musical phrase. In this design it appears as a single LSTM layer of 512 units that consumes a “Note Matrix”—a 3D tensor of note, velocity, and time-interval attributes scaled to $(0,1)$—and outputs the next note's probability-like scores. Dropout, a dense layer, a linear activation, and mean squared error loss complete the machinery, with RMSprop as the optimizer.

What would settle it

Train the same single-layer LSTM on the Pop Music MIDI files with a subset of songs held out, then measure next-note log-likelihood and generated-note overlap on those held-out tracks; if held-out performance collapses toward the random baseline while training accuracy stays near 97%, the reported numbers reflect memorization, not music generation.

Watch

Extended reading notes

Core claim

The central claim is that a single LSTM layer can jointly predict which note should be played and when, instead of using separate networks for pitch and timing as earlier biaxial models did. At each time step $t$, the network conditions its output on the note states from the previous $t-50$ steps, learning the probability of the next note given that remembered context. With 512 LSTM units, a dropout of 0.75, a dense layer, a linear activation, and RMSprop at learning rate $10^{-4}$, mean squared error dropped from roughly 0.0286 to 0.002098 over 200 epochs while accuracy rose to 97.23%, and the final batch-level accuracy reached 97.27%. The authors present the log-likelihood of $-6.23$ achieved in 0.9 hours as evidence that the model captures melodic and harmonic structure far more efficiently than longer-trained predecessors.

Load-bearing premise

The load-bearing assumption is that rising training accuracy and falling training loss are trustworthy signs that the model has learned to compose good music, since the paper reports no held-out evaluation and no listening test.

Editorial extensions

If this is right

  • A single LSTM layer can represent polyphonic music, so generative systems do not necessarily need two separate networks for pitch and timing.
  • Training time for a usable generative music model can drop from days to under an hour on a modest MIDI corpus, making experimentation cheaper.
  • The MIDI-to-Note-Matrix preprocessing (note, velocity, interval, scaled to (0,1)) gives a reusable input representation for future music-generation work.
  • Conditioning each prediction on the previous 50 time steps gives a concrete, inspectable rule for how new notes are generated.
  • Scaling the model upward—more units, more data, other hyperparameters—is a natural next step that the paper itself names.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Pith inference: because the 97.23% figure is training-set accuracy, the paper's most defensible contribution is speed and architectural simplicity, not a demonstrated quality advantage over other generators.
  • Pith inference: the log-likelihood comparison mixes datasets (Piano-midi.de for the baselines, Pop Music for this model), so a true ranking would require retraining all models on one corpus and evaluating on a shared held-out set.
  • Pith inference: a concrete overfitting check would be to sample long continuations from the trained model and measure their $n$-gram overlap with the training MIDI files; high verbatim overlap would indicate memorization rather than composition.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents a single-layer LSTM model for polyphonic music generation from MIDI files. It describes preprocessing of MIDI data into a 'Note Matrix,' an LSTM with 512 units followed by dropout, dense, and linear activation layers, and training with MSE loss and RMSprop on Google Colab. The results section reports training loss and accuracy curves over 200 epochs, with a final training accuracy of 97.23% and a training loss of about 0.0021 after 0.9 hours, and compares the model's log-likelihood of -6.23 with several prior LSTM-based systems. The authors conclude that the model 'produced stellar results in composing new melodies.'

Significance. If the claimed result were properly validated, a simple single-layer LSTM baseline that learns polyphonic structure from MIDI would be a useful reference point for music-generation research. The paper also provides a public code link and documents implementation details such as the optimizer, dropout, and learning rate. However, the evidence presented does not support the central claim. There is no held-out evaluation, no generated-music assessment, and the quantitative comparison against prior work is invalid because it compares log-likelihoods obtained on different datasets. The reported training accuracy and loss are not meaningful measures of compositional quality. The strengths of the submission are its transparency about the model architecture and training setup; the weakness is the absence of rigorous evaluation.

major comments (3)
  1. [Section V, Table I and Fig. 5] The comparison that supports the claimed improvement over prior work is invalid. The top rows are log-likelihoods reported by Nikhil and Paul [14] on the Piano-midi.de dataset, while the final row is the best of 200 trials on the Pop Music dataset [25]. Log-likelihood values depend on the dataset, vocabulary, sequence length, and preprocessing, so no meaningful comparison can be drawn. Moreover, the table is internally inconsistent: if the column is a log-likelihood, higher is better, and -6.23 is worse than -4.90/-5.00; if the intended metric is a loss to be minimized, the column header is wrong. Either way, the statement in the text that the result is 'a slight improvement compared to previous work' is not supported by Table I.
  2. [Section V, Figs. 6-9] The only performance evidence for the proposed model is training loss and training accuracy on the same data used to fit the model. There is no validation or test split, no evaluation on held-out MIDI files, no generated-sample analysis, and no listening test. A training accuracy of 97.23% and a training loss of about 0.0021 only demonstrate that the model can fit the training data; they do not demonstrate coherent or novel music generation. The introduction states that the paper reports 'losses and accuracy ... generated by training and validating this model,' but no validation curves appear in the results section.
  3. [Section V, Fig. 6 and Fig. 7 captions] There is a direct contradiction about which dataset was used. The captions of Figs. 6 and 7 state that the model was trained on 'Piano Midi files,' whereas Table I, Fig. 5, and the conclusion state that the single-layer LSTM was trained on Pop Music [25]. This discrepancy affects the interpretation of the reported curves and of the -6.23 value, and it prevents reproduction. The authors must specify exactly which dataset was used, how many MIDI files it contains, the note vocabulary size, and the sequence length.
minor comments (4)
  1. [Section IV.A and References] The reference numbering appears to be misaligned: Moon et al. is listed as [20] but cited as [19], the RMSprop lecture is [21] but cited as [20], and the Mido module is [19] but cited as [21]. Please correct the citations.
  2. [Section V, Fig. 6] The sharp loss increase to approximately 0.0718 at epoch 120 is mentioned but not explained. The authors should either explain what caused this outlier or state that it corresponds to a known change in the training procedure.
  3. [Section IV.A] The sentence 'Not all notes undergo the training process. Only some selected and specified notes are used to train this LSTM model' is vague. Please specify how notes are selected and filtered during preprocessing.
  4. [Acknowledgment] The acknowledgment thanking '4 anonymous reviewers' is unusual in a submitted manuscript and should be removed.

Circularity Check

0 steps flagged · score 0.0 of 10

No structural circularity: the LSTM is trained with standard supervised learning and no load-bearing step reduces to its inputs; the weak training-set-only evaluation is a quality concern, not a circular derivation.

full rationale

This paper does not claim a first-principles derivation; it trains a single-layer LSTM on MIDI note matrices using standard supervised learning with MSE and reports training loss and accuracy. The training loss is the objective minimized during fitting, and the accuracy is computed on the same 'Piano Midi files' used for training, so those curves are not independent evidence of generation quality, but they are presented as training diagnostics rather than as a prediction derived from a fitted parameter. The comparison in Table I is invalid because log-likelihood values come from different datasets (Piano-midi.de vs Pop Music) and no same-data baseline is run, but that is a benchmarking and evidence-quality issue, not circularity. The cited prior work [14], [19] is external to the present authors, no self-citation is load-bearing, and no uniqueness theorem or ansatz is imported from the authors' own prior work. No equation in the paper is defined in terms of its own output, so no step reduces by construction.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central claim rests on hyperparameters and domain assumptions about LSTM and MIDI representation. These are not derived from data and are chosen somewhat arbitrarily. There are no new physical or mathematical entities introduced.

free parameters (4)
  • LSTM units = 512
    Chosen by the authors, no justification for the specific number.
  • Dropout rate = 0.75
    Chosen based on a cited guide (Moon et al., though the reference points to Mido).
  • Learning rate = 1e-4
    Selected for RMSprop optimization, no systematic search reported.
  • Lookback window = 50 time steps
    Model output at time t is conditioned on previous notes up to t-50; chosen without analysis.
assumptions (3)
  • domain assumption LSTM networks can learn musical note sequences from MIDI representations
    The paper assumes that LSTM temporal modeling is sufficient for music generation, which is a widely used but non-trivial assumption.
  • domain assumption The Note Matrix representation preserves musical structure
    The paper encodes note, velocity, and time interval into a matrix scaled to [0,1]; it does not justify that this representation captures harmonic or melodic relationships.
  • ad hoc to paper MSE loss on scaled note values is appropriate for music generation
    The paper uses mean squared error without explaining why it is suitable for discrete or continuous note likelihoods, and with linear activation the output is not a normalized probability distribution.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LSTM Based Music Generation System." pith.science (2026). https://pith.science/paper/5FG6MXOQ

@misc{pith2026190801080,
  author       = {Pith},
  title        = {Pith review of: LSTM Based Music Generation System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5FG6MXOQ}},
  note         = {Machine review of arXiv:1908.01080}
}
read the original abstract

Traditionally, music was treated as an analogue signal and was generated manually. In recent years, music is conspicuous to technology which can generate a suite of music automatically without any human intervention. To accomplish this task, we need to overcome some technical challenges which are discussed descriptively in this paper. A brief introduction about music and its components is provided in the paper along with the citation and analysis of related work accomplished by different authors in this domain. Main objective of this paper is to propose an algorithm which can be used to generate musical notes using Recurrent Neural Networks (RNN), principally Long Short-Term Memory (LSTM) networks. A model is designed to execute this algorithm where data is represented with the help of musical instrument digital interface (MIDI) file format for easier access and better understanding. Preprocessing of data before feeding it into the model, revealing methods to read, process and prepare MIDI files for input are also discussed. The model used in this paper is used to learn the sequences of polyphonic musical notes over a single-layered LSTM network. The model must have the potential to recall past details of a musical sequence and its structure for better learning. Description of layered architecture used in LSTM model and its intertwining connections to develop a neural network is presented in this work. This paper imparts a peek view of distributions of weights and biases in every layer of the model along with a precise representation of losses and accuracy at each step and batches. When the model was thoroughly analyzed, it produced stellar results in composing new melodies.

Figures

Figures reproduced from arXiv: 1908.01080 by the authors.

Figure 2
Figure 2. This image provides the structure of graph used to implement this model, representing various components and classes used to create this model. The image is generated by Tensorboard. The loss function used is mean squared error (also known as L2 loss), shown in fig. 3. It calculates the distance between actual and predicted sequence and then averages its sum over complete data set [PITH_FULL_IMAGE:figures/full_fig_… view at source ↗
Figure 1
Figure 1. This image provides an insights of data flow in the model. It describes [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 4
Figure 4. Image describing distributions of weights in various stages of the [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: shows the performance of past papers, as well as the results obtained by training this model. The result captured by this model is a slight improvement compared to previous work. Performance of the model was profoundly affected due to less exposure to training time. In…
Figure 10
Figure 10. Figure 10: Image showing weight distributions of the model. It shows an [PITH_FULL_IMAGE:figures/full_fig_p005_10.png]
Figure 8
Figure 8. Figure 8: Graph depicting training loss of batches in the model, generated by [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]
Figure 9
Figure 9. Figure 9: Graph Depicts batch accuracy of the model used in this work, [PITH_FULL_IMAGE:figures/full_fig_p005_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 18 canonical work pages

  1. [14]

    Generating Music using an LSTM Network

    P. Y. Nikhil Kotecha, "Generating Music using an LSTM Network," arXiv.org, vol. arXiv:1804.07300, 2018

    work page Pith review arXiv 2018

  2. [25]

    Musical_Matrices,

    "Musical_Matrices," 10 July 2016. [Online]. Available: https://github.com/dshieble/Musical_Matrices/tree/master/Pop_Music _Midi. [Accessed 2 November 2018]

    work page 2016

  3. [1]

    Fundamentals of Music & Properties of Sound – Music Theory Lesson,

    F. Furukawa, "Fundamentals of Music & Properties of Sound – Music Theory Lesson," 3 August 2015. [Online]. Availa ble: https://popliteral.com/fundamentals-of-music-properties-of-sound- music-theory-lesson/. [Accessed 7 November 2018]

    work page 2015

  4. [2]

    A simple genetic algorithm for music generation by means of algorithmic information theory,

    M. C. A. O. Manuel Alfonseca, "A simple genetic algorithm for music generation by means of algorithmic information theory," IEEE Congress on Evolutionary Computation, Singapore, pp. 3035 -3042, 2007

    work page 2007

  5. [3]

    A Critical Review of Recurrent Neural Networks,

    Z. C. a. B. J. a. E. C. Lipton, "A Critical Review of Recurrent Neural Networks," arXiv preprint arXiv:1506.00019, 2015

    arXiv 2015

  6. [4]

    A Comparative Analysis of Algorithmic Music Generation on GPUs and FPGAs,

    G. a. N. V. a. S. J. a. K. A. Joshi, "A Comparative Analysis of Algorithmic Music Generation on GPUs and FPGAs," 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), pp. 229-232, 2018

    work page 2018

  7. [5]

    An algebra for tree -based music generation,

    F. a. H. J. Drewes, "An algebra for tree -based music generation," Springer, pp. 172-188, 2007

    work page 2007

  8. [6]

    Music generation with Markov models,

    A. a. S. W. Van Der Merwe, "Music generation with Markov models," IEEE MultiMedia, vol. 18, no. 3, pp. 78-85, 2011

    work page 2011

Show all 27 references
  1. [7]

    Modeling temporal dependencies in high -dimensional sequences: Application to polyphonic music generation and transcription,

    N. a. B. Y. a. V. P. Boulanger -Lewandowski, " Modeling temporal dependencies in high -dimensional sequences: Application to polyphonic music generation and transcription," arXiv preprint arXiv:1206.6392, 2012

    2012 arXiv

  2. [8]

    Interactive Music Generation with Positional Constraints using Anticip ation-RNNs,

    G. a. F. N. Hadjeres, "Interactive Music Generation with Positional Constraints using Anticip ation-RNNs," arXiv preprint arXiv:1709.06404, 2017

    2017 arXiv

  3. [9]

    System and method for automatic music generation using a neural network architecture,

    C. B. Browne, "System and method for automatic music generation using a neural network architecture," Google Patents, 2001

    2001

  4. [10]

    Artificial neural networks,

    A. Abraham, "Artificial neural networks," handbook of measuring system design, 2005

    2005

  5. [11]

    Generative adversarial nets,

    I. a. P. -A. J. a. M. M. a. X. B. a. W. -F. D. a. O. S. a. C. A. a. B. Y. Goodfellow, "Generative adversarial nets," in Advances in neural information processing systems, 2014, pp. 2672--2680

    2014

  6. [12]

    MuseGAN: Symbolic-domain music generation and accompaniment with multi - track sequential generative adversarial networks,

    H.-W. a. H. W. -Y. a. Y. L. -C. a. Y. Y. -H. Dong, "MuseGAN: Symbolic-domain music generation and accompaniment with multi - track sequential generative adversarial networks," arXiv preprint arXiv:1709.06298, 2017

    2017 arXiv

  7. [13]

    Magenta,

    "Magenta," Google, [Online]. Available: https://magenta.tensorflow.org/. [Accessed 29 October 2018]

    2018

  8. [15]

    Long Short -Term Memory,

    J. S. Sepp Hochreiter, "Long Short -Term Memory," Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997

    1997

  9. [16]

    Learning lon g-term dependencies with gradient descent is difficult,

    Y. Bengio, P. Simard and P. Frasconi, "Learning lon g-term dependencies with gradient descent is difficult," IEEE Transactions on Neural Networks, vol. 5, no. 2, pp. 157-166, 1994

    1994

  10. [17]

    Dropout: A Simple Way to Prevent Neural Networks from Overfitting,

    G. H. A. K. I. S. R. S. Nitish Srivastava, "Dropout: A Simple Way to Prevent Neural Networks from Overfitting," Journal of Machine Learning Research, vol. 15, pp. 1929-1958, 2014

    1929

  11. [18]

    Densely connected convolutional networks.,

    G. L. Z. V. D. M. L. &. W. K. Q. Huang, " Densely connected convolutional networks.," CVPR, vol. 1, p. 3, 2017

    2017

  12. [19]

    O. M. Bjørndalen, "Mido," 21 August 2011. [Online]. Available: https://github.com/olemb/mido. [Accessed 3 November 2018]

    2011

  13. [20]

    Rnndrop: A novel dropout for rnns in asr,

    C. H. L. H. S. I. Moon T, "Rnndrop: A novel dropout for rnns in asr," Automatic Speech Recognition and Understanding (ASRU), pp. 65-70, 2015

    2015

  14. [21]

    Lecture 6.5 -rmsprop: Divide the gradient by a running average of its recent magnitude,

    T. a. H. G. Tieleman, "Lecture 6.5 -rmsprop: Divide the gradient by a running average of its recent magnitude," COURSERA: Neural networks for machine learning, vol. 4, no. 2, pp. 26-31, 2012

    2012

  15. [22]

    Multistability of recurrent neural networks with time-varying delays and the piecewise linear activation function.,

    Z. T. H. a. W. X. Z. Zeng, "Multistability of recurrent neural networks with time-varying delays and the piecewise linear activation function.," IEEE Transactions on Neural Networks, vol. 21, no. 8, pp. 1371-1377, 2010

    2010

  16. [23]

    Descending into ML: Traini ng and Loss,

    GoogleDevelopers, "Descending into ML: Traini ng and Loss," [Online]. Available: https://developers.google.com/machine- learning/crash-course/descending-into-ml/training-and-loss

  17. [24]

    Music Research,

    S. Mangal, "Music Research," 13 November 2018. [Online]. Available: https://gitlab.com/sanidhyamangal/music_research

    2018

  18. [26]

    ADADELTA: an adaptive learning rate method,

    M. D. Zeiler, "ADADELTA: an adaptive learning rate method," arXiv preprint arXiv:1212.5701, 2012

    2012 arXiv

  19. [27]

    The million song dataset,

    D. P. W. E. B. W. a. P. L. T. Bertin -Mahieux, "The million song dataset," International Society for Music Information Retrieval (ISMIR’11), vol. 2, no. 9, p. 10, 2011

    2011

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.