{"id":"ed5bdd99-a204-4a6b-a94e-fd04b513bee9","arxiv_id":"2412.18940","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A human-AI songwriting tool that generates keyword-relevant chord progressions from images, text, or audio by filtering LLM suggestions with a chord model, shown to increase perceived agency and creativity in a small user study.","lead":"Amuse is a songwriting assistant that turns images, text, or audio into chord progressions, letting songwriters weave multimodal inspirations into their music. It combines a large language model's broad musical knowledge with a data-driven filter to keep suggestions musically coherent.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 5.3's rejection sampling cancels keyword conditioning using an untested P(c|x)≈Q(c|x) assumption; the filter is keyword-agnostic, so the claimed conditional-generation method is unsupported.","rationale":"The reader's weakest_assumption is exactly the load-bearing point. The paper makes two separable claims: an HCI system that helps songwriters, and a novel technical method for keyword-conditioned chord generation without paired data. The user study is appropriately scoped (N=10, within-subjects, think-aloud, interaction logs) and the qualitative findings are plausible and well reported; I do not see a reason to discount them. The technical claim, however, rests on §5.3. The derivation cancels P(c|x)/Q(c|x), and the acceptance rule becomes independent of the keyword c. The paper justifies this with 'we assume P(c|x)≈Q(c|x) as the generated progression x from LLMs closely align with c.' That is an assertion about the inverse conditional distribution of real chord data versus an LLM's learned associations. There is no paired data, no calibration, and the paper's own results show GPT-4o's forward distribution Q(x|c) is far from real music (§6.2.1), which makes the inverse matching assumption even less secure. Without it, the accepted distribution is Q(x|c)·P(x)/Q(x), not P(x|c). The consequence is not merely formal: the filter cannot improve keyword relevance, only coherence, so the claim 'generating coherent chords relevant to music keywords' reduces to 'LLM proposals plus a coherence filter.' That is still a useful system, but it is not the claimed method. I also note the automatic coherence evaluation in §6.2.1 compares against HookTheory, the same distribution used to train P(x); this is a weaker test than the listening study, but the listening study does provide independent evidence for perceived coherence and relevance. The conditional verdict is appropriate: the paper should be accepted after the derivation is corrected, the method is reframed as a keyword-agnostic coherence filter, or the assumption is empirically supported. The synthetic recovery test above would settle the mathematical question.","tokens_in":39624,"tokens_out":8443,"duration_ms":76391,"concrete_test":"Build a synthetic instance of the §5.3 setup with a small chord vocabulary and known distributions: choose P(x|c), Q(x|c) (and hence P(x), Q(x)) such that P(c|x)≠Q(c|x). Draw N proposals from Q(x|c), apply Algorithm 1's acceptance rule P(x)/(M Q(x)), and compare the accepted empirical distribution with P(x|c). If the Jensen-Shannon divergence between them exceeds the sampling-error baseline, the algorithm does not generally target P(x|c), demonstrating that the paper's cancellation depends on an unverified assumption. This is a direct correctness test for the central algorithmic claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central technical contribution is a 'keyword-conditioned' rejection sampler that targets P(x|c) from an LLM proposal Q(x|c) and a unimodal prior P(x). In §5.3, the acceptance ratio is reduced from P(x|c)/(M Q(x|c)) to P(x)/(M Q(x)) via Bayes' rule and the asserted equalities P(c)=Q(c) and P(c|x)≈Q(c|x). Neither equality is demonstrated. P(c) is a marginal over real songwriting keywords, while Q(c) is the authors' random sampling distribution over a keyword wiki; they are not the same object. P(c|x)≈Q(c|x) asserts that GPT-4o's inverse keyword distribution matches the true data inverse, a strong claim that is especially doubtful because no paired keyword-chord data exist and the paper's own results show GPT-4o's forward chord distribution diverges substantially from real music (§6.2.1). Once these cancellations are made, the acceptance probability P(x)/(M Q(x)) is independent of c. The actual accepted distribution is Q(x|c)·P(x)/Q(x), which equals the claimed P(x|c) only if the Bayes equality holds exactly. If it does not, the method is a keyword-agnostic coherence filter on top of GPT-4o proposals, and relevance comes entirely from the prompt, not from the 'novel' filtering step. Additionally, Q(x) is an LSTM fitted to 25,000 GPT-4o samples; no calibration is reported between this fitted density and the true GPT-4o marginal proposal density, so even the marginal ratio may be biased. The HCI results are not invalidated, but the technical contribution as stated is not supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Amuse, a Chrome extension integrated with Hookpad that assists songwriters by transforming multimodal inputs (images, text) into chord progressions, and transcribing audio into chords. The central technical contribution is a rejection-sampling procedure that combines GPT-4o proposals, conditioned on music keywords extracted from the multimodal input, with an LSTM prior trained on the HookTheory dataset, in order to produce chord progressions that are diverse, relevant to keywords, and musically coherent without paired training data. The paper reports a technical evaluation of diversity (Self-BLEU), coherence (JSD against HookTheory), and a listening study of coherence and keyword relevance, followed by a within-subjects user study with 10 songwriters comparing Amuse+Aria against Aria alone. The user study finds that participants felt greater agency, creativity, and alignment with their creative goals when using Amuse.","tokens_in":39933,"tokens_out":5379,"duration_ms":49397,"significance":"If the rejection-sampling derivation were sound, the paper would contribute a practical method for conditional symbolic music generation without paired data: a generally useful recipe of using a multimodal LLM as a proposal and a unimodal prior as a filter. The user study is carefully designed and analyzed, with interaction logs, think-aloud protocols, and qualitative coding, and it provides credible evidence that a multimodal inspiration-to-chord tool can enhance perceived agency and creativity in songwriting. The paper also ships code and sound examples, which supports reproducibility. However, the technical derivation in Section 5.3 contains load-bearing unsupported equalities, and the coherence evaluation in Table 2 is substantially circular; these issues call the central technical claim into question. The HCI findings appear robust, but the paper's stated contribution (2)—a novel method for generating diverse, relevant, and coherent chord progressions—needs substantial revision.","major_comments":[{"comment":"The cancellation of the keyword conditioning from the acceptance ratio relies on two equalities: P(c)=Q(c) and P(c|x)≈Q(c|x). Neither is established. P(c) is not defined as a data distribution anywhere; the paper only defines Q(c) as the distribution over keywords used when sampling prompt keywords from a wiki. These are not the same object unless one defines P(c) to be exactly that sampling distribution, which is not a natural interpretation of Bayes' rule applied to the target P(x|c). More importantly, P(c|x)≈Q(c|x) is a strong assumption about the inverse keyword distribution of GPT-4o matching the true data inverse, and no evidence is given for it. The paper's own §6.2.1 shows that GPT-4o's forward marginal chord distribution deviates substantially from real music, making the inverse-distribution assumption especially doubtful without paired data. After this cancellation, the acceptance probability P(x)/(M Q(x)) is independent of c, and the accepted distribution is Q(x|c)·P(x)/Q(x), not P(x|c). As a result, the method as presented is a keyword-agnostic coherence filter on top of LLM proposals; any keyword relevance is inherited from the prompt, not from the rejection-sampling step. This undermines contribution (2) and the claim in §6.2 that Amuse generates keyword-conditioned progressions through the described rejection-sampling procedure.","section":""},{"comment":"The constant M is set to the 95th percentile of the ratio P(x)/Q(x) over GPT-4o-generated progressions, rather than to an upper bound on that ratio. Rejection sampling is only valid when M ≥ sup_x P(x|c)/Q(x|c) (after cancellation, sup_x P(x)/Q(x)); with a 95th-percentile value, for the top 5% of proposals the ratio exceeds M, so the acceptance probability is capped at 1 instead of being the required ratio/M > 1. This changes the target distribution and invalidates the formal rejection-sampling justification. In addition, Q(x) is an LSTM density fitted to 25,000 GPT-4o samples, but no calibration is reported between this fitted density and the actual GPT-4o marginal proposal density. Both issues mean the accepted samples are not actually drawn from the claimed target distribution, even setting aside the conditioning-cancellation problem above.","section":""},{"comment":"The automatic coherence evaluation compares Amuse's outputs to HookTheory, which is the same dataset used to train the filter P(x). Because the acceptance probability (after the problematic cancellation) is P(x)/Q(x), the accepted samples are biased toward P(x); the large JSD reduction in Table 2 is therefore a near-tautological consequence of filtering with the evaluation reference rather than an independent validation. The listening study in Figure 5a provides a more meaningful coherence check, but the keyword-relevance result in Figure 5b only shows a significant advantage over LSTM Prior; Amuse is not significantly preferred over GPT-4o for relevance. Taken together, the technical evaluation does not support the claim that the rejection-sampling step improves keyword relevance; it may merely preserve the relevance already present in GPT-4o's proposals while improving coherence.","section":""},{"comment":"The final set of four progressions is not a pure rejection-sampling output. When fewer than four samples are accepted, the algorithm fills the remainder with the top-k rejected samples ranked by P(x)/Q(x). This means the user-facing set is a mixture of accepted and rejected samples, and is not a sample from any well-defined target distribution. The reported diversity, coherence, and relevance measurements (including the listening study) are therefore measuring the complete algorithm, which is reasonable, but the paper should acknowledge that the output is not strictly a rejection-sampling result and should evaluate the fallback path separately or at least report how often it is used.","section":""}],"minor_comments":[{"comment":"The abstract states that Amuse transforms multimodal image, text, and audio inputs into chord progressions, but the Chord Generator only handles image and text; audio is processed by the separate Chord Transcriber, which does not use keyword conditioning. The phrasing should be sharpened to distinguish the two pipelines.","section":""},{"comment":"The listening study in §6.2.2 says it used 10 keyword sets, but Appendix B.2 lists nine keyword sets plus an attention-check set. Please clarify whether the attention check is included in the counts and how the 150 comparisons were allocated.","section":""},{"comment":"The numeric values in Figure 5 are presented in a compact layout that is difficult to parse; the columns are labeled only in the caption. It would improve readability to label each column directly in the figure, for example 'vs. LSTM Prior' and 'vs. GPT-4o'.","section":""},{"comment":"The implementation section says all LLM components use temperature 1.0, while §B.1 mentions a temperature of 1.7 for the LSTM distributions during rejection sampling. Clarify which temperature applies to which model and why the discrepancy exists.","section":""},{"comment":"The limitations section already covers the study's small sample and controlled setting, which is good. It might also note that the Chord Transcriber's low usage was partly attributed to task constraints, and that future work could investigate the transcriber in more naturalistic settings.","section":""}],"recommendation":"major_revision","confidential_remarks":"The skeptical assessment lands on the core technical derivation: the paper's rejection-sampling logic in §5.3 contains unsupported cancellations that make the filter keyword-agnostic, and the coherence evaluation in Table 2 is largely circular. However, I do not think the paper should be rejected outright. The user study and the general system concept are solid, and the technical section could be revised to present the method as a heuristic coherence filter rather than an exact sampler from P(x|c), with appropriate caveats. The authors need to either provide evidence for P(c|x)≈Q(c|x) or substantially reframe the contribution. I would recommend major revision and a careful check that the final claims match what the method actually delivers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid CHI paper on a real HCI problem, but the central technical contribution is overclaimed. The stress-test note is right about §5.3: the rejection sampling derivation cancels keyword conditioning via P(c|x)≈Q(c|x), which is asserted, not shown. With that cancellation, acceptance is P(x)/(M Q(x)) — independent of c. So the “keyword-conditioned” filter is really just a coherence filter on top of GPT-4o. Relevance comes from the prompt, not from the filtering step. Also, P(c)=Q(c) compares a marginal over real songwriting keywords with the authors’ sampling distribution over a wiki; they are not the same object. And Q(x) is an LSTM trained on 25k GPT-4o samples with no calibration against the true GPT-4o marginal, so even the marginal ratio is on shaky ground.\n\nThat said, the HCI work is good. The system is a sensible combination of known pieces — an LLM for keyword extraction and chord suggestion, a trained chord LM for filtering, and a clean editor integration. The user study (N=10, within-subjects) is well-run: the authors report honest limitations, note the lack of significance on final satisfaction, and the qualitative findings on usage patterns (kickstart, ad hoc, lyrics-centered) are useful. The listening study provides human evidence that the filtered outputs are preferred over raw GPT-4o on coherence and at least as good on relevance, so the engineering works; it’s the theoretical framing that doesn’t.\n\nThe automatic coherence evaluation in Table 2 measures JSD against the same HookTheory distribution used to train P(x), so it’s partly circular. The CLAP results in the appendix are weak and the authors acknowledge the low correlation. These are minor compared to the derivation gap, but worth fixing.\n\nBottom line: the paper deserves a serious referee. I’d recommend major revision: either justify or drop the conditional-generation framing, add uncertainty to the automatic metrics, and temper the novelty claim. The core HCI contribution and user study are solid and would survive the revision.","headline":"Solid HCI system paper; the rejection-sampling story needs a rewrite before the technical claim holds.","tokens_in":40469,"tokens_out":2072,"would_cite":true,"duration_ms":20090,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that Amuse transforms multimodal inputs—images, text, or audio—into musically coherent, editable chord progressions by combining a multimodal LLM's noisy proposals with a unimodal chord-model filter, and that songwriters…","keywords":["creativity support tool","human-AI co-creation","multimodal inspiration","chord generation","rejection sampling","large language models","songwriting","music AI"],"falsifier":"Take a fixed set of keyword-conditioned proposals from GPT-4o, run the rejection filter, and have listeners (or a keyword classifier) label whether accepted progressions are more relevant to the keywords than rejected ones. If the filter is truly keyword-agnostic, relevance should be equal in the two sets, and any observed relevance is the LLM's own; if the filter is secretly changing relevance, the Bayes-ratio derivation is not what is doing the work.","tokens_in":39382,"feed_emoji":"🎵","tokens_out":4581,"duration_ms":40440,"temperature":0.7,"pith_summary":"The paper argues that songwriters need not choose between multimodal inspiration and iterative, editable musical output: a chord progression can be derived from an image, a story, or an audio clip, and still be musically plausible. Amuse does this by having a multimodal LLM (GPT-4o) turn any input into music keywords and then into candidate chord progressions, before a separate, unimodal chord language model trained on real chord data filters those candidates by rejection sampling. The authors establish this with automatic diversity and coherence metrics plus a listening study, and with a within-subjects user study of ten songwriters showing that adding Amuse to an existing contextual assistant improves perceived agency, creativity, and alignment with creative goals. If correct, the work shows a practical route to multimodal musical co-creation without paired multimodal-to-chord datasets.","feed_headline":"Amuse turns images and stories into editable chord progressions","feed_subtitle":"An LLM proposes chords from any inspiration; a chord model filters them, letting songwriters keep agency.","key_machinery":"The load-bearing object is the rejection-sampling acceptance ratio $P(x)/(M Q(x))$, where $P(x)$ is an LSTM learned from human-composed chord progressions (HookTheory) and $Q(x)$ is an LSTM learned from GPT-4o-generated chord progressions with keywords marginalized. Via Bayes' rule and the assumption $P(c|x)\\approx Q(c|x)$, the keyword-conditional target $P(x|c)$ is replaced by this ratio, so the filter judges only musical coherence. A prompting technique that asks GPT-4o to generate 30 progressions in one batch supplies diversity, and the Chord Generator's keyword-extraction step supplies relevance and transparency.","core_discovery":"The central discovery is that noisy, keyword-conditioned chord suggestions from a multimodal LLM can be made both coherent and relevant by filtering them with a unimodal prior over real chord progressions, using a rejection-sampling ratio that drops the keyword conditioning entirely. The filtering step is keyword-agnostic because the authors assume the LLM's conditional relevance is close to the true one, so the prior only corrects musical coherence. The paper's user study claims that this pipeline, embedded in Hookpad alongside the contextual assistant Aria, enhances users' agency, creativity, and perceived efficiency without changing final satisfaction.","pith_inferences":["I would read the paper's coherence claim as inheriting all of its strength from the LSTM prior; the filter never sees the keywords, so any keyword relevance in the final chords comes entirely from GPT-4o's zero-shot reading of the prompt.","An empirical test the authors do not report: compute keyword relevance of accepted versus rejected proposals (e.g., with a keyword classifier or human labels). If the filter is truly keyword-agnostic, accepted and rejected sets should have similar relevance, and the observed relevance should match the LLM's proposal quality.","The same 'LLM proposes, unimodal prior disposes' scheme could be applied to other symbolic musical elements, such as drum patterns or basslines, or even to non-music creative domains where paired data are scarce but a unimodal model of the output exists."],"forward_implications":["If the method works as claimed, a songwriter can start from a photograph or a paragraph of prose and, within one session, obtain several editable chord progressions in a chosen key and length.","The rejection-sampling recipe becomes a general template for conditioning symbolic music models on modalities for which paired chord data do not exist, requiring only an LLM proposal and a unimodal prior.","The user-study results imply that adding multimodal inspiration support to a contextual AI assistant shifts perceived control and creativity without sacrificing output satisfaction, and changes when and how often users query the contextual assistant.","The keyword intermediary layer gives users a transparent handle on the AI's interpretation, making the abstract image-to-chords transformation editable and explainable."],"supporting_citations":[{"why":"Supplies the multimodal LLM (GPT-4o) that extracts keywords from images/text and generates the noisy candidate chord progressions.","marker":"[71]"},{"why":"Supplies the HookTheory dataset of human-composed chord progressions used to train the unimodal prior $P(x)$.","marker":"[31]"},{"why":"Supplies the rejection-sampling algorithm that the paper adapts to filter LLM proposals with the unimodal prior.","marker":"[66]"},{"why":"Supplies the music keyword wiki used to sample keywords for training $Q(x)$ and to build the keyword list for extraction.","marker":"[90]"},{"why":"Provides the unimodal contextual assistant Aria, which serves as the baseline in the user study and the complementary tool in the Assist condition.","marker":"[32]"},{"why":"Provides the Hookpad songwriting editor into which Amuse is embedded and where generated chords are pasted and edited.","marker":"[42]"}],"fun_headline_variants":["Amuse: LLM noise filtered to chords for songwriters","Turn images, text, and audio into chords with Amuse","Amuse: multimodal inspiration becomes coherent chords","Songwriting AI filters LLM suggestions into chords","Noisy LLM ideas become music with Amuse's chord filter"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole filter rests on the assumption that the LLM's keyword-to-chord relevance is already correct, so the prior can ignore keywords; if the LLM's relevance is off, the filter cannot repair it, and the LSTM estimate of the LLM's proposal density is also uncalibrated.","fun_headline_variants_meta":{"raw":{"variants":["Amuse: LLM noise filtered to chords for songwriters","Turn images, text, and audio into chords with Amuse","Amuse: multimodal inspiration becomes coherent chords","Songwriting AI filters LLM suggestions into chords","Noisy LLM ideas become music with Amuse's chord filter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001017,"raw_usage":{"total_tokens":4228,"prompt_tokens":813,"completion_tokens":3415,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":429,"completion_tokens_details":{"reasoning_tokens":3335}},"tokens_in":429,"tokens_out":3415,"duration_ms":22690,"temperature":1.0,"reasoning_tokens":3335,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T04:18:30.743014+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a fixed set of keyword-conditioned proposals from GPT-4o, run the rejection filter, and have listeners (or a keyword classifier) label whether accepted progressions are more relevant to the keywords than rejected ones. If the filter is truly keyword-agnostic, relevance should be equal in the two sets, and any observed relevance is the LLM's own; if the filter is secretly changing relevance, the Bayes-ratio derivation is not what is doing the work.","supporting_citations":[],"review_version":1}