{"id":"5ee9af56-16a2-46ad-a667-e0ee725e81cb","arxiv_id":"1908.05551","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A conditional LSTM-GAN generates 20-note melodies from English lyrics on a new 12,197-song aligned dataset, outperforming random and MLE baselines on several metrics.","lead":"This paper trains a neural network that composes a melody from song lyrics using a conditional GAN with LSTM layers. It also introduces a large aligned dataset of 12,197 MIDI songs with English lyrics matched to notes.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing premise is the automatically extracted one-syllable-to-one-note alignment (Sec. III-C, V-2); if this parser is noisy or the filter discards many multi-note syllables, the learned mapping and all evaluations inherit the error.","rationale":"The strongest claim is empirical: the proposed conditional LSTM-GAN produces melodies that are more plausible, tuneful, and closer to the human-composed distribution than the random and MLE baselines. For this claim to hold, the paired training data must be a faithful sample of lyrics-melody alignments. The paper provides no independent evidence that the automatic one-syllable-to-one-note alignment is accurate or representative. Section III-C acknowledges the one-to-many possibility but discards it, and Section V-2 gives extraction rules without describing how note-to-syllable correspondence is established or validated. This is more load-bearing than the missing neural baselines or error bars: those affect the strength of the comparison, whereas an unreliable alignment target affects the validity of what is learned and measured. If the parser is noisy, the generator is trained to reproduce parser errors, and the same errors contaminate the validation and test metrics. I agree with the reader's weakest assumption. I do not think the paper should be rejected outright, because the concern is empirically checkable and the dataset is publicly released; a focused alignment validation could settle it. The reader's CONDITIONAL verdict remains appropriate, so I keep the verdict unchanged.","tokens_in":13399,"tokens_out":5513,"duration_ms":63851,"concrete_test":"From the released GitHub dataset, randomly select 100 MIDI files (or about 1,000 syllable-note pairs) and obtain the original lyric timestamps. Have two annotators independently mark which syllable, if any, each note corresponds to. Report the fraction of syllables with exactly one note, the fraction with multiple notes, and the fraction where the parser's one-to-one assignment disagrees with the annotators. Then retrain the conditional LSTM-GAN on the manually verified one-to-one subset and recompute Table V and Table VI. If the disagreement rate exceeds 5% or the multi-note fraction in the source corpus exceeds 20%, the dataset premise and the reported advantage over baselines are not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that the model learns a meaningful syllable-to-note relationship depends on the dataset alignment being accurate and representative. Section III-C explicitly states that one syllable may correspond to one or more notes, but only one-syllable-to-one-note pairs are used, with multi-note cases deferred to future work. Section V-2 describes extracting a MIDI number 'if a note has a corresponding English syllable,' but it never specifies how that correspondence is determined, how parsing errors are detected, or whether any manual validation was performed. No statistics are reported for how many syllables were discarded or misaligned during parsing. Because the same parsed alignments serve as training targets, as ground truth for MMD model selection, and as references for BLEU and attribute metrics, any systematic parser error or bias from the one-to-one filter is baked into both the learned model and the evaluation. The model may then be reproducing regularities of the parser rather than genuine lyrics-melody structure, and the reported advantage over baselines could reflect shared alignment artifacts rather than musical quality. The paper's own limitation note in Section III-C makes this premise explicit, but the claim as stated is not restricted to simple syllabic singing, so the missing validation is load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a conditional LSTM-GAN for generating melodies from lyrics. It introduces a lyrics-melody dataset built from 12,197 MIDI files with syllable-note alignment extracted from lyrics and MIDI metadata, and it restricts attention to one-syllable-to-one-note pairs. The generator and discriminator are LSTM networks conditioned on syllable- and word-level embeddings, and a tuning scheme quantizes the continuous outputs and snaps out-of-tune notes to a nearby standard scale. The authors evaluate against a random baseline and a single-layer LSTM MLE baseline using attribute statistics, BLEU scores, scale consistency, a permutation-style test for lyric conditioning on note/rest durations, and a small listening study. They conclude that the model generates more plausible and tuneful melodies than the baselines while jointly learning syllable-note alignment.","tokens_in":13729,"tokens_out":3835,"duration_ms":41652,"significance":"If the central claim holds, the paper would make a useful contribution to lyrics-conditioned melody generation: the dataset is a substantial resource, the architecture is a reasonable application of conditional GANs to symbolic music, and the open-source release supports reproducibility. The permutation-style duration experiment and the subjective evaluation go beyond what many generative-music papers provide. However, the significance is currently limited by two load-bearing gaps: the alignment extraction, on which every training and evaluation step depends, is not validated or even fully specified; and the comparison set does not include published neural lyrics-to-melody systems, so the claimed advantage over the current state of the art is not established. The paper's own limitation note in Section III-C acknowledges the one-to-one alignment restriction, but the main claims are stated without that restriction, and no parser-error statistics or manual validation are reported.","major_comments":[{"comment":"The key premise that the lyrics-melody alignment is accurately extracted is not supported. Section V-2 never specifies how 'a note has a corresponding English syllable' is determined, while Section III-C states that only one-syllable-to-one-note pairs are used and multi-note syllables are deferred to future work. No counts of discarded pairs, no parser-error statistics, and no manual validation are reported. Because the same parsed alignments are used as training targets, as references for MMD model selection, and as ground truth for BLEU and attribute metrics, any systematic parser error is baked into both the learned model and the evaluation. Please add a precise alignment rule, validation statistics, and at least a manually checked subset, or restrict the central claim to the validated one-to-one case.","section":"Section III-C and Section V-2"},{"comment":"All reported quantitative results are single point estimates without error bars, confidence intervals, or repeated training runs. The BLEU-3 difference between the proposed model (0.460) and the MLE baseline (0.331) could be within run-to-run variability, and Table V shows the proposed model below ground truth on 3-MIDI repetitions and song length despite the text saying it outperforms the baseline 'in every aspect.' The evaluation also omits published neural baselines such as the lyrics-to-melody model of Bao et al. [9], so the central claim is not established against current state-of-the-art methods. Please report variance estimates or statistical tests over multiple runs and add comparisons to at least one existing neural lyrics-to-melody system.","section":"Section VI.E, Tables V and VI"},{"comment":"The tuning scheme remaps out-of-tune notes to the inferred scale before validation and testing, so scale-consistency and some pitch metrics reflect the post-processing rule as much as the learned model. The Random baseline is also tuned, which makes the comparison fair in that narrow sense, but the contribution of tuning to the reported metrics should be quantified by ablating it, for example by reporting results with tuning disabled at test time.","section":"Section VI.A and Section VI.D"},{"comment":"The permutation-style experiment establishes that the model uses lyric conditioning for note and rest durations, but no analogous test is provided for MIDI pitch, although Figures 11 and 12 are used to claim that lyric conditioning shapes pitch distributions. The two-lyric illustration in Figure 12 is anecdotal. Please provide a statistical test for the pitch attribute analogous to the duration experiment in Section VI.F.2.","section":"Section VI.F, Figures 11 and 12"}],"minor_comments":[{"comment":"The section heading contains a typo: 'Condtional' should be 'Conditional.'","section":"Section IV.C"},{"comment":"The phrase 'national language processing' should be 'natural language processing.'","section":"Introduction"},{"comment":"The description of Table V should be clarified: the proposed model does not outperform the baselines on every reported attribute, since 3-MIDI repetitions and song length are further from the ground-truth values than the MLE baseline for some rows.","section":"Section VI.E"},{"comment":"The scale-consistency metric is not defined precisely; please state how 'most likely scale' is inferred and how 'mean accuracy' is computed, especially for notes that belong to multiple standard scales.","section":"Section VI.E"},{"comment":"The notation for the rest calculation is ambiguous: the formula uses note-on_k and note-off_{k-1} but the prior note index is not defined. ","section":"Section V.2"},{"comment":"The subjective evaluation uses only 7 subjects and reports averaged scores without any significance test; please add at least basic variability information or acknowledge this limitation explicitly.","section":"Section VI.G"}],"recommendation":"major_revision","confidential_remarks":"The paper has a useful dataset and a sensible architecture, but the alignment-validation gap and the absence of published neural baselines are substantial. I would not reject the paper on novelty grounds, but the authors should be asked to demonstrate that the alignment is reliable and to compare against existing methods before the central claim can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The durable contribution here is the dataset: 12,197 MIDI songs with parsed lyrics–melody alignment, released on GitHub. That is a real resource for the music-generation community. The conditional LSTM-GAN is a reasonable application of known building blocks—not a conceptual breakthrough, but a sensible combination that appears to work. The two-stage evaluation, including the randomization test for whether lyrics conditioning actually affects note/rest durations, is a good instinct—that is the kind of check that too many papers skip.\n\nWhat the paper does less well is validation and comparison. The alignment extraction is load-bearing: it produces the training targets, the MMD model-selection reference, and the ground truth for BLEU and attribute metrics. But the description in Section V-2 never explains how a note is matched to a syllable, how errors are caught, or how many syllables were discarded because they did not fit the one-to-one filter. No manual validation or statistics on discarded data are reported. If that parsing is noisy or systematically biased, every downstream number inherits the bias. The paper itself admits in Section III-C that only one-syllable-to-one-note pairs are used, with multi-note cases deferred. Since the claim is not restricted to simple syllabic singing, this is a real gap, not a minor one.\n\nThe baseline comparison is also thinner than it should be. The paper cites Bao et al. (Neural melody composition from lyrics) and ALYSIA but does not compare against them or any other published neural lyrics-to-melody system. The MLE baseline and random baseline are fine as sanity checks, but they do not establish state-of-the-art. Reported metrics come with no error bars or significance tests, and the scale-tuning post-processing is applied before scoring, so some of the numerical advantage may reflect the correction rule rather than the learned model. The subjective test is small—7 listeners, 4 lyrics—and the effect, while plausible, is not backed by strong statistics.\n\nNone of this sinks the paper. The randomization test does suggest the model learned something beyond dataset statistics, and the dataset alone justifies serious attention. The central claim that the model generates more plausible melodies than the tested baselines probably holds. But the strength of the claim in the abstract and conclusion is not supported by the current evaluation.\n\nThis paper deserves peer review, not desk rejection. A serious referee should ask for a comparison with at least one existing neural baseline, validation or at least descriptive statistics for the alignment pipeline, and significance testing or error bars on the key metrics. If those are addressable, it could become a solid contribution to the creative-AI and music-generation subfield.","headline":"A plausible lyrics-to-melody model with a genuinely useful aligned dataset, but the evaluation needs more rigorous baselines and validation of the alignment parser before the claims can be taken at face value.","tokens_in":14182,"tokens_out":1270,"would_cite":true,"duration_ms":15949,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Conditional LSTM-GAN can generate plausible, tuneful melodies aligned syllable-by-syllable with input lyrics.","keywords":["melody generation","lyrics conditioning","LSTM","generative adversarial network","music alignment","MIDI","conditional GAN","symbolic music generation"],"falsifier":"Compute the MMD or BLEU between melodies generated from the same noise but different lyrics; if the resulting distributions are statistically indistinguishable, the claimed lyric conditioning has no effect on the output.","tokens_in":13229,"feed_emoji":"🎵","tokens_out":5901,"duration_ms":49900,"temperature":0.7,"pith_summary":"This paper tries to establish that a conditional LSTM-GAN can generate melodies from English lyrics: given a line of lyrics, the model produces a sequence of notes, durations, and rests that is aligned syllable-by-syllable with the text and that sounds closer to human-composed music than melodies from random or maximum-likelihood baselines. The authors build a paired lyrics-melody dataset of 12,197 MIDI songs with syllable-to-note alignment to train and test the model. They report that the generated melodies match the pitch distribution of human-composed music more closely than the baselines, achieve higher BLEU scores, and receive better listener ratings on melody, rhythm, and lyrics fit.","feed_headline":"Lyrics become singable melodies via conditional LSTM-GAN","feed_subtitle":"A 12,197-song paired dataset helps the model learn syllable-note alignment and beat random and MLE baselines.","key_machinery":"The load-bearing object is the conditional LSTM-GAN, an adversarial pair in which both the generator and discriminator are LSTM networks that receive the same sequence of syllable embeddings as context. The generator's LSTM cells are unrolled once per syllable, concatenating each syllable vector with random noise and with the previous cell's output, so the model must learn a sequential alignment between text positions and note positions; the discriminator tries to tell real syllable-note pairs from generated ones, pushing the generator's output distribution toward the distribution of the training melodies. A final tuning step quantizes the continuous triplets to discrete MIDI, duration, and rest values and corrects out-of-tune notes to the nearest standard scale.","core_discovery":"The central claim is that a deep LSTM-based generator and an LSTM-based discriminator, both conditioned on syllable embeddings of the lyrics, can jointly infer a melody and the alignment between each syllable and its note. Instead of sampling notes independently, the generator takes a 20-dimensional embedding that concatenates word-level and syllable-level skip-gram vectors plus a 30-dimensional noise vector, and emits a triplet of music attributes per syllable: MIDI number, note duration, and rest duration. A quantizer then rounds these continuous outputs to the nearest discrete musical values and maps out-of-scale notes into the closest matching standard scale. The paper reports that this setup produces melodies whose note-range, repetition, and transition statistics are closer to the ground-truth testing set than those of the random and MLE baselines, and that ablating the lyrics conditioning narrows the generated MIDI-number distribution away from the human distribution.","pith_inferences":["The one-syllable-to-one-note constraint, which discards multi-note syllables, likely limits the model to syllabic singing; extending the alignment to one-syllable-to-multiple-notes would be a direct test of whether the learned mapping generalizes to melismatic vocal lines.","The reported scale-consistency of around 49 percent suggests the tuning step is not resolving key ambiguity; combining the generator with an explicit key or scale condition could be a cheap way to raise musical plausibility.","Because the discriminator is conditioned on the same lyrics, the model implicitly learns a lyric-to-melody association; one could test this by holding lyrics fixed and measuring how much melody variance remains, which would quantify how strongly text actually constrains the output."],"forward_implications":["A single adversarial model can jointly produce pitch, rhythm, and syllable-to-note alignment from text, rather than treating alignment as a separate preprocessing step.","Lyrics conditioning changes the generated pitch distribution measurably, so the model is not merely reproducing the marginal statistics of the training set.","The newly built 12,197-song paired dataset gives later work a common training and evaluation ground for English lyrics-to-melody generation.","The MLE baseline's lower BLEU and higher repetition metrics suggest that adversarial training adds something beyond next-note likelihood for this task.","Because the model outputs quantized triplets, the same framework could be extended to other discrete symbolic music representations."],"supporting_citations":[{"why":"Defines the adversarial generator-discriminator objective that the conditional model adapts.","marker":"[10]"},{"why":"Introduces conditioning a GAN on auxiliary information, here used to condition on lyrics.","marker":"[27]"},{"why":"Supplies the LSTM architecture used to model sequential dependencies and lyrics-melody alignment.","marker":"[26]"},{"why":"Shows adversarial training on continuous MIDI sequences, the direct precursor for the generator output representation.","marker":"[19]"},{"why":"Establishes the one-syllable-to-one-note alignment assumption used by the data pipeline.","marker":"[8]"},{"why":"Presents an end-to-end neural lyrics-to-melody model, the closest prior approach the paper extends.","marker":"[9]"},{"why":"Motivates the random baseline that samples music attributes from the dataset distribution.","marker":"[33]"},{"why":"Provides the single-layer LSTM configuration used to build the MLE baseline model.","marker":"[34]"},{"why":"One of the two MIDI collections that supply the paired lyrics-melody data.","marker":"[28]"},{"why":"The other MIDI collection used to build the dataset.","marker":"[29]"}],"fun_headline_variants":["AI turns lyrics into melodies with LSTM-GAN","Lyrics to melody: LSTM-GAN learns syllable-note alignment","Conditional GAN sings: deep LSTM generates melody from lyrics","New LSTM-GAN model pairs lyrics with melodies","LSTM-GAN crafts melodies from lyrics via alignment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The model depends on the assumption that the automatically extracted one-syllable-to-one-note alignments in the MIDI files are accurate and that discarding all multi-note syllables does not systematically distort the learned lyrics-to-melody mapping or the evaluation.","fun_headline_variants_meta":{"raw":{"variants":["AI turns lyrics into melodies with LSTM-GAN","Lyrics to melody: LSTM-GAN learns syllable-note alignment","Conditional GAN sings: deep LSTM generates melody from lyrics","New LSTM-GAN model pairs lyrics with melodies","LSTM-GAN crafts melodies from lyrics via alignment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000176,"raw_usage":{"total_tokens":1266,"prompt_tokens":900,"completion_tokens":366,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":516,"completion_tokens_details":{"reasoning_tokens":283}},"tokens_in":516,"tokens_out":366,"duration_ms":3446,"temperature":1.0,"reasoning_tokens":283,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:09:36.453945+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the MMD or BLEU between melodies generated from the same noise but different lyrics; if the resulting distributions are statistically indistinguishable, the claimed lyric conditioning has no effect on the output.","supporting_citations":[{"cited_title":"Generative Adversarial Networks,","cited_arxiv_id":null,"evidence_quote":"Defines the adversarial generator-discriminator objective that the conditional model adapts."},{"cited_title":"Algorithmic Songwriting with ALYSIA","cited_arxiv_id":"1612.01058","evidence_quote":"Establishes the one-syllable-to-one-note alignment assumption used by the data pipeline."},{"cited_title":"iComposer: An automatic song- writing system for Chinese popular music,","cited_arxiv_id":null,"evidence_quote":"Motivates the random baseline that samples music attributes from the dataset distribution."},{"cited_title":"Seqgan: Sequence generative adversarial nets with policy gradient,","cited_arxiv_id":null,"evidence_quote":"Provides the single-layer LSTM configuration used to build the MLE baseline model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"One of the two MIDI collections that supply the paired lyrics-melody data."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The other MIDI collection used to build the dataset."}],"review_version":1}