{"id":"e0547762-7981-4374-a644-c453b091897f","arxiv_id":"1908.01080","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A single-layer LSTM learns to predict note sequences from a small pop MIDI dataset, reaching a training log-likelihood of -6.23 in 54 minutes.","lead":"This paper trains a single-layer LSTM network on pop music MIDI files to generate polyphonic note sequences. It reports 97% training accuracy and a log-likelihood of -6.23, but gives no independent evaluation of musical quality.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed 'stellar results' rest on a cross-dataset table whose values conflict with the stated metric, plus training-only curves; no held-out or generation-quality evidence is given.","rationale":"The reader's weakest assumption correctly identifies the missing held-out evaluation and the invalid baseline comparison. My stress-test concurs and sharpens the point: Table I's own values contradict the 'slight improvement' claim under the stated metric, and the curves in Figs. 6-9 are training-only. This is the load-bearing weakness because the paper's contribution is presented as an empirical result, not just a model architecture. The method itself is a standard LSTM with dropout and RMSprop, and a code link is provided, so the work is plausible as a baseline system. The verdict should remain CONDITIONAL: the paper would be acceptable if the authors supplied a proper held-out evaluation on the same dataset, a correctly labeled metric, and either a same-data baseline or an explicit statement that the comparison is illustrative rather than competitive. I do not see grounds to move to full rejection because the described implementation is coherent and the code is available for verification; the issue is overclaiming from inadequate evidence, not an unfixable flaw.","tokens_in":7147,"tokens_out":4684,"duration_ms":51986,"concrete_test":"Run the released code (gitlab.com/sanidhyamangal/music_research) and: (1) determine exactly what quantity '-6.23' is (log-likelihood, cross-entropy, or MSE-derived score) and on which data split it was computed; (2) train the same single-layer LSTM and a BALSTM-style baseline on the same Pop Music MIDI corpus using an identical held-out split and the same metric; (3) report held-out log-likelihood and generate samples for a blind listening or structural comparison. If the held-out score does not beat the same-data baseline, or if the model cannot be evaluated on held-out sequences, the improvement claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative support for the claim that this single-layer LSTM improves on prior work is Table I, and that table cannot bear the claim. The Fig. 5 caption states that the top four rows are results reported by Nikhil and Paul on the Piano-midi.de dataset, while the final row is the present model's best-of-200 result on the Pop Music dataset [25]. Log-likelihood (or loss) values are not comparable across different corpora, vocabularies, and sequence lengths, so the 'improvement' over BALSTM is not established. The table is also internally inconsistent: if the column is a log-likelihood, higher is better and this model's -6.23 is worse than BALSTM's -4.90/-5.00; if the column is a loss to be minimized, the header is wrong and no baseline was run on the same data. The only model-specific evidence, Figs. 6-9, is training loss and training accuracy on the data used to fit the model. There is no held-out split, no generated-sample evaluation, and no listening test. A model can reach 97.23% training accuracy by memorizing training sequences; that does not demonstrate coherent or pleasing composition. Thus the conclusion that the model 'produced stellar results in composing new melodies' is unsupported by the evidence presented.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a single-layer LSTM model for polyphonic music generation from MIDI files. It describes preprocessing of MIDI data into a 'Note Matrix,' an LSTM with 512 units followed by dropout, dense, and linear activation layers, and training with MSE loss and RMSprop on Google Colab. The results section reports training loss and accuracy curves over 200 epochs, with a final training accuracy of 97.23% and a training loss of about 0.0021 after 0.9 hours, and compares the model's log-likelihood of -6.23 with several prior LSTM-based systems. The authors conclude that the model 'produced stellar results in composing new melodies.'","tokens_in":7391,"tokens_out":3986,"duration_ms":39756,"significance":"If the claimed result were properly validated, a simple single-layer LSTM baseline that learns polyphonic structure from MIDI would be a useful reference point for music-generation research. The paper also provides a public code link and documents implementation details such as the optimizer, dropout, and learning rate. However, the evidence presented does not support the central claim. There is no held-out evaluation, no generated-music assessment, and the quantitative comparison against prior work is invalid because it compares log-likelihoods obtained on different datasets. The reported training accuracy and loss are not meaningful measures of compositional quality. The strengths of the submission are its transparency about the model architecture and training setup; the weakness is the absence of rigorous evaluation.","major_comments":[{"comment":"The comparison that supports the claimed improvement over prior work is invalid. The top rows are log-likelihoods reported by Nikhil and Paul [14] on the Piano-midi.de dataset, while the final row is the best of 200 trials on the Pop Music dataset [25]. Log-likelihood values depend on the dataset, vocabulary, sequence length, and preprocessing, so no meaningful comparison can be drawn. Moreover, the table is internally inconsistent: if the column is a log-likelihood, higher is better, and -6.23 is worse than -4.90/-5.00; if the intended metric is a loss to be minimized, the column header is wrong. Either way, the statement in the text that the result is 'a slight improvement compared to previous work' is not supported by Table I.","section":"Section V, Table I and Fig. 5"},{"comment":"The only performance evidence for the proposed model is training loss and training accuracy on the same data used to fit the model. There is no validation or test split, no evaluation on held-out MIDI files, no generated-sample analysis, and no listening test. A training accuracy of 97.23% and a training loss of about 0.0021 only demonstrate that the model can fit the training data; they do not demonstrate coherent or novel music generation. The introduction states that the paper reports 'losses and accuracy ... generated by training and validating this model,' but no validation curves appear in the results section.","section":"Section V, Figs. 6-9"},{"comment":"There is a direct contradiction about which dataset was used. The captions of Figs. 6 and 7 state that the model was trained on 'Piano Midi files,' whereas Table I, Fig. 5, and the conclusion state that the single-layer LSTM was trained on Pop Music [25]. This discrepancy affects the interpretation of the reported curves and of the -6.23 value, and it prevents reproduction. The authors must specify exactly which dataset was used, how many MIDI files it contains, the note vocabulary size, and the sequence length.","section":"Section V, Fig. 6 and Fig. 7 captions"}],"minor_comments":[{"comment":"The reference numbering appears to be misaligned: Moon et al. is listed as [20] but cited as [19], the RMSprop lecture is [21] but cited as [20], and the Mido module is [19] but cited as [21]. Please correct the citations.","section":"Section IV.A and References"},{"comment":"The sharp loss increase to approximately 0.0718 at epoch 120 is mentioned but not explained. The authors should either explain what caused this outlier or state that it corresponds to a known change in the training procedure.","section":"Section V, Fig. 6"},{"comment":"The sentence 'Not all notes undergo the training process. Only some selected and specified notes are used to train this LSTM model' is vague. Please specify how notes are selected and filtered during preprocessing.","section":"Section IV.A"},{"comment":"The acknowledgment thanking '4 anonymous reviewers' is unusual in a submitted manuscript and should be removed.","section":"Acknowledgment"}],"recommendation":"major_revision","confidential_remarks":"The paper reads like a student project report rather than a completed research article. The central quantitative claim rests on an invalid cross-dataset comparison, and the only model-specific results are training curves. A publishable revision would need a held-out evaluation, a baseline trained on the same data, and some form of generated-music assessment. The inconsistency between the Fig. 6/7 captions and the Table I dataset description should also be resolved. For a serious journal, the current evidence is not sufficient; however, these issues are addressable with additional experiments, so I do not recommend rejection outright."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about arXiv:1908.01080 is that it is a cleanly written single-layer LSTM for MIDI note prediction, honestly derived from the Biaxial LSTM work in [14]. If you need a toy baseline that trains in under an hour, this gives you a starting point. But the paper's key claim—that this model produces 'stellar results'—is not supported by the evidence. The reported 97.23% accuracy and -6.23 log-likelihood are training-set numbers, and the comparison against prior work is done across different datasets, so it does not establish an improvement.\n\nWhat is actually new: the simplification from two LSTMs to one, applied to Pop Music MIDI, with a public code link. The preprocessing and architecture description are adequate, and the training curves (Figs. 6-9) are plausible for a standard LSTM. I have no reason to doubt that the model runs and learns the training corpus.\n\nThe soft spots are the usual ones, and they hurt the central conclusion. Table I mixes results from Piano-midi.de and Pop Music; different vocabularies, sequence lengths, and corpora make the log-likelihoods incomparable. The header says 'Log Likelihood,' in which case -6.23 is worse than BALSTM's -4.90, not better; if it is a loss, the header is wrong and still no same-data baseline exists. The outlier at epoch 120 is described but never explained, which matters for reproducibility. There is no held-out evaluation, no generated samples, no listening test. The 'stellar' phrase in the abstract and conclusion is simply more than the training curves can support. The acknowledgement's swipe at anonymous reviewers is unprofessional, but I will set that aside.\n\nThe paper does what it does: a functional single-layer LSTM music generator with some TensorBoard diagnostics. For a student project or a workshop, that is fine. For a serious research venue, the evaluation needs a major revision: same-dataset baselines, a proper data split, and some human or automatic evaluation of generated output. The code link is a plus and should be kept.\n\nI would not send this to peer review in its current form. I would hand it back to the authors with the evaluation requirements, and if they resubmit with those, it could be a modest but usable baseline note.","headline":"A reproducible single-layer LSTM music-gen baseline whose central quantitative claim rests on an invalid cross-dataset comparison and training-only metrics.","tokens_in":7926,"tokens_out":2812,"would_cite":false,"duration_ms":28337,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes and demonstrates a single-layer LSTM network that learns polyphonic note sequences from MIDI files and generates new melodies, reporting 97.23% training accuracy and a log-likelihood of -6.23 after only 0.9 hours on…","keywords":["LSTM","music generation","MIDI","recurrent neural networks","polyphonic music","note matrix","deep learning","melody generation"],"falsifier":"Train the same single-layer LSTM on the Pop Music MIDI files with a subset of songs held out, then measure next-note log-likelihood and generated-note overlap on those held-out tracks; if held-out performance collapses toward the random baseline while training accuracy stays near 97%, the reported numbers reflect memorization, not music generation.","tokens_in":6932,"feed_emoji":"🎵","tokens_out":7633,"duration_ms":65602,"temperature":0.7,"pith_summary":"This paper argues that a single-layer Long Short-Term Memory (LSTM) network can learn the sequential structure of polyphonic music and then generate new melodies without human intervention. It describes a complete pipeline: MIDI files are converted into scaled “Note Matrix” tensors holding note, velocity, and time interval, then fed into a 512-unit LSTM with dropout, a dense layer, and linear activation. On the Pop Music dataset, training for about 0.9 hours produced 97.23% accuracy and a log-likelihood of $-6.23$, which the authors compare with biaxial LSTM systems that required 16–48 hours. The contribution, read sympathetically, is a simpler architecture that learns musical sequences quickly from MIDI data.","feed_headline":"One LSTM layer composes music in under an hour","feed_subtitle":"Training on Pop Music MIDI files, the model reaches 97.23 percent accuracy after just 54 minutes.","key_machinery":"The object that carries the argument is the LSTM cell's gated memory: a hidden state that carries information from earlier time steps into the current prediction, letting the network recall the structure of a musical phrase. In this design it appears as a single LSTM layer of 512 units that consumes a “Note Matrix”—a 3D tensor of note, velocity, and time-interval attributes scaled to $(0,1)$—and outputs the next note's probability-like scores. Dropout, a dense layer, a linear activation, and mean squared error loss complete the machinery, with RMSprop as the optimizer.","core_discovery":"The central claim is that a single LSTM layer can jointly predict which note should be played and when, instead of using separate networks for pitch and timing as earlier biaxial models did. At each time step $t$, the network conditions its output on the note states from the previous $t-50$ steps, learning the probability of the next note given that remembered context. With 512 LSTM units, a dropout of 0.75, a dense layer, a linear activation, and RMSprop at learning rate $10^{-4}$, mean squared error dropped from roughly 0.0286 to 0.002098 over 200 epochs while accuracy rose to 97.23%, and the final batch-level accuracy reached 97.27%. The authors present the log-likelihood of $-6.23$ achieved in 0.9 hours as evidence that the model captures melodic and harmonic structure far more efficiently than longer-trained predecessors.","pith_inferences":["Pith inference: because the 97.23% figure is training-set accuracy, the paper's most defensible contribution is speed and architectural simplicity, not a demonstrated quality advantage over other generators.","Pith inference: the log-likelihood comparison mixes datasets (Piano-midi.de for the baselines, Pop Music for this model), so a true ranking would require retraining all models on one corpus and evaluating on a shared held-out set.","Pith inference: a concrete overfitting check would be to sample long continuations from the trained model and measure their $n$-gram overlap with the training MIDI files; high verbatim overlap would indicate memorization rather than composition."],"forward_implications":["A single LSTM layer can represent polyphonic music, so generative systems do not necessarily need two separate networks for pitch and timing.","Training time for a usable generative music model can drop from days to under an hour on a modest MIDI corpus, making experimentation cheaper.","The MIDI-to-Note-Matrix preprocessing (note, velocity, interval, scaled to (0,1)) gives a reusable input representation for future music-generation work.","Conditioning each prediction on the previous 50 time steps gives a concrete, inspectable rule for how new notes are generated.","Scaling the model upward—more units, more data, other hyperparameters—is a natural next step that the paper itself names."],"supporting_citations":[{"why":"Supplies the biaxial LSTM architecture and the log-likelihood baselines that the paper's single-layer result is measured against.","marker":"[14]"},{"why":"Supplies the LSTM cell mechanism that gives the network its ability to remember past note context.","marker":"[15]"},{"why":"Diagnoses the vanishing-gradient problem that motivates replacing plain RNNs with LSTM cells.","marker":"[16]"},{"why":"Supplies the dropout technique used to generalize the LSTM layer.","marker":"[17]"},{"why":"Links the paper's code and TensorBoard summaries, making the training and generation pipeline reproducible.","marker":"[24]"},{"why":"Supplies the Pop Music MIDI dataset that all reported losses, accuracies, and log-likelihood values come from.","marker":"[25]"}],"fun_headline_variants":["Single LSTM layer achieves 97% music accuracy","One-layer LSTM learns melodies in under an hour","LSTM writes music at 97% accuracy in 54 minutes","Fast single-layer LSTM composes new melodies","Efficient LSTM generates music with 97% accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that rising training accuracy and falling training loss are trustworthy signs that the model has learned to compose good music, since the paper reports no held-out evaluation and no listening test.","fun_headline_variants_meta":{"raw":{"variants":["Single LSTM layer achieves 97% music accuracy","One-layer LSTM learns melodies in under an hour","LSTM writes music at 97% accuracy in 54 minutes","Fast single-layer LSTM composes new melodies","Efficient LSTM generates music with 97% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000296,"raw_usage":{"total_tokens":1748,"prompt_tokens":1004,"completion_tokens":744,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":665}},"tokens_in":620,"tokens_out":744,"duration_ms":6306,"temperature":1.0,"reasoning_tokens":665,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:23:12.498072+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same single-layer LSTM on the Pop Music MIDI files with a subset of songs held out, then measure next-note log-likelihood and generated-note overlap on those held-out tracks; if held-out performance collapses toward the random baseline while training accuracy stays near 97%, the reported numbers reflect memorization, not music generation.","supporting_citations":[{"cited_title":"Generating Music using an LSTM Network","cited_arxiv_id":"1804.07300","evidence_quote":"Supplies the biaxial LSTM architecture and the log-likelihood baselines that the paper's single-layer result is measured against."},{"cited_title":"Long Short -Term Memory,","cited_arxiv_id":null,"evidence_quote":"Supplies the LSTM cell mechanism that gives the network its ability to remember past note context."},{"cited_title":"Learning lon g-term dependencies with gradient descent is difficult,","cited_arxiv_id":null,"evidence_quote":"Diagnoses the vanishing-gradient problem that motivates replacing plain RNNs with LSTM cells."},{"cited_title":"Music Research,","cited_arxiv_id":null,"evidence_quote":"Links the paper's code and TensorBoard summaries, making the training and generation pipeline reproducible."},{"cited_title":"Musical_Matrices,","cited_arxiv_id":null,"evidence_quote":"Supplies the Pop Music MIDI dataset that all reported losses, accuracies, and log-likelihood values come from."}],"review_version":1}