{"id":"cf058217-ec8c-468f-a7eb-ee1e7a206c68","arxiv_id":"2506.23094","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"TOMI uses a four-node structure plus LLM in-context learning to turn sample clips into whole multi-track electronic songs with planned section structure.","lead":"The paper introduces TOMI, a structured way to compose full electronic songs from reusable music building blocks organized by clips, sections, tracks, and transformations. A text-based AI model plans the arrangement, and the result can be edited and rendered inside the REAPER audio workstation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ILS partly rewards TOMI's hard-coded section reuse (Sec. 3.1.2), so the objective structural-coherence advantage may measure exact identity, not musical coherence.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: ILS validity. I focus on ILS rather than FAD because the paper's distinctive claim is 'stronger structural coherence,' and ILS is the only objective metric aimed specifically at that property. The FAD comparison is also confounded by professional sample-pack audio, but TOMI beats the Standalone LLM and Random ablations on FAD even though they use the same sample library, so FAD still supplies some controlled evidence for arrangement quality; the ILS comparison, by contrast, is structurally unfair whenever TOMI reuses a section node. The concern does not force rejection. The Random ablation provides partial independent support that the LLM's arrangement choices matter, and the subjective survey, while limited, shows advantages in most metrics. The concrete de-duplication test would settle whether ILS's large lead survives when identical repeats are replaced by matched variations. If it collapses, the conclusion should be re-scoped to claim that the representation enforces repeated sections, not that it produces measurably better musical structure. This is an addressable evaluation condition rather than a fatal flaw, so the conditional verdict stands.","tokens_in":11920,"tokens_out":7225,"duration_ms":87372,"concrete_test":"Using the released code, regenerate TOMI outputs and build a control set in which every reused section instance is replaced by a different clip from the same database, matched on key, tempo, duration, and content-type features, while preserving all other node choices and composition links. Recompute ILS_MERT, ILS_MS, and ILS_WF on this control. If the control ILS drops to the level of the Standalone LLM or Random baseline (roughly 0.10-0.22), the Table 2 ILS advantage is dominated by exact node reuse rather than musically coherent variation, and the objective structural-coherence claim is not established. If the control ILS remains significantly above the baselines, then ILS also captures the LLM's curated same-label similarity, and this concern is substantially answered.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The quantitative support for 'stronger structural coherence' is the ILS metric (Table 2, Eqs. 1-3). That metric is compromised by TOMI's own construction: Section 3.1.2 states that a section node appearing multiple times has identical musical content, and Fig. 2b shows that a reused section automatically duplicates the same composition links. Whenever the LLM assigns the same phrase label to two positions and reuses the same section/clip nodes, the same-label cosine similarities in Eq. (2) approach 1.0 by identity, not because of musically meaningful coherence. The Standalone LLM and MusicGen baselines do not have this exact-repetition mechanism, so part of TOMI's ILS lead reflects the presence of the TOMI data structure itself rather than the LLM's arranging skill. The Random ablation also has the same structure, so the metric is not fully vacuous, but the reported ILS gap still cannot separate 'good same-label development' from 'identical copies.' Moreover, Cohen's d in Eq. (1) is maximally sensitive to this identity: identical segments yield very high same-label mean similarity with small within-group variance, inflating the effect size. The objective claim of stronger structural coherence therefore depends on an unstated assumption that exact same-label identity is a fair proxy for musical coherence. The subjective Part 2 ratings do ask listeners to rate 'Similarity' between same-label sections, which rewards the same identity property rather than independently validating it as a musical virtue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces TOMI, a representation and generation system for multi-track electronic music with full-song structure. Music is represented as composition links over four node types (clips, sections, tracks, and transformations); a foundation LLM (GPT-4o) fills in the nodes via in-context learning, and the outputs are rendered through REAPER. The authors compare TOMI against MusicGen, a Standalone LLM without composition links, and a Random ablation that uses the same sample library. They report FAD and ILS objective metrics and a 73-participant subjective study, concluding that TOMI produces higher-quality electronic music with stronger structural coherence than the baselines.","tokens_in":12151,"tokens_out":4803,"duration_ms":54981,"significance":"TOMI is a timely and practical contribution: it offers a sparse, interpretable structure for long-form music generation, integrates symbolic and audio material, and ships with code and a demo page. The Random ablation is a well-designed control because it holds the sample library and the TOMI data structure fixed, isolating the contribution of LLM-driven arrangement decisions; the fact that Random underperforms TOMI despite using the same professional samples strengthens the claim that arrangement logic matters. If the evaluation concerns below are addressed, the paper would provide credible evidence for a concept-hierarchy approach to full-song electronic music generation.","major_comments":[{"comment":"The ILS metric is partly circular with respect to TOMI's own representation. Because Section 3.1.2 specifies that a reused section node has identical musical content, and Figure 2b shows that reusing a section automatically duplicates the same composition links, any same-label pair of segments generated from the same section/clip nodes will have near-identical embeddings. Eq. (2) averages cosine similarities of all same-label pairs, so these identical copies mechanically inflate X_same and, through low within-group variance, can inflate Cohen's d in Eq. (1). Thus the ILS advantage reported in Table 2 does not cleanly separate 'good same-label development' from 'exact copy by construction,' particularly in comparisons against MusicGen and Standalone LLM, which lack this reuse mechanism. Please recompute ILS excluding pairs of segments that come from the same section/clip node, or report a version that evaluates only non-identical same-label segments; this is load-bearing for the objective structural-consistency claim.","section":"Section 3.1.2 and Eq. (2)"},{"comment":"The FAD results are reported as point estimates without confidence intervals, bootstrap resampling, or significance tests. With 32 generated pieces per method and a single reference set of 329 songs, the differences in Table 2 (e.g., FAD_VGGish 3.51 vs. 5.31) could be within sampling noise. Please provide bootstrap confidence intervals or per-piece FAD distributions, and if possible a statistical comparison; without this, the claim that TOMI 'achieves the lowest FAD scores' is not quantitatively supported.","section":"Section 4.3, Table 2"},{"comment":"The comparison against MusicGen is confounded by the source of audio material. TOMI retrieves and assembles professionally produced sample-library loops, while MusicGen synthesizes audio from text; FAD measures distributional closeness to human-composed electronic music, so TOMI's lower FAD may largely reflect the use of real samples rather than its arrangement logic. The Random and Standalone LLM ablations do control for the sample library, and those comparisons are meaningful for the arrangement-logic claim, but the paper should not present the MusicGen FAD comparison as evidence of higher generation quality without explicitly stating this confound. Please reframe the quality claim as a system-level comparison or restrict the controlled quality claim to the sample-based ablations.","section":"Sections 4.2 and 4.3"},{"comment":"The paper states that TOMI 'significantly outperforms the baseline in most subjective metrics,' but it reports only mean scores and confidence intervals from a within-subject ANOVA, with no F-statistics, p-values, or post-hoc test results. Given that the subjective study carries much of the structural-coherence claim after the ILS concern, please report the actual statistical test outcomes (e.g., ANOVA tables and pairwise comparisons), or soften the wording to describe descriptive differences.","section":"Section 4.4 and Figure 4"}],"minor_comments":[{"comment":"The table reports FAD as point estimates while ILS columns include mean plus/minus standard deviation; please clarify whether FAD is computed as a single value or has a repeated-measures distribution, and if so, report its variance in the same way.","section":"Table 2"},{"comment":"The symbol 's' is used both for the pooled standard deviation in Eq. (4) and as a subscript for section nodes (e.g., s2, s3); using a different symbol such as 's_pool' or 'sigma' for the standard deviation would improve readability.","section":"Eq. (1) and Section 3.1.2"},{"comment":"The survey is described as double-blind, but it is not clear how participants could be blind to the method when hearing full songs of different production quality; please describe the blinding procedure in more detail.","section":"Section 4.4"},{"comment":"The caption states 'Darker colors indicate higher segment similarity,' which is conventional, but the colorbar or a brief explanation of the colormap would help readers interpret the matrices consistently.","section":"Figure 3"},{"comment":"The rule-based validation is mentioned but not evaluated; reporting the frequency of syntax errors or invalid values across the 32 generated compositions would give a sense of the LLM's reliability in following the TOMI schema.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of an ISMIR-style venue and the core idea is promising, but the objective structural-consistency metric has a genuine circularity problem that needs to be addressed with a re-analysis or a clearly stated limitation. If the authors cannot provide a non-circular ILS variant, the structural-coherence claim should be based primarily on the subjective ratings, with the ILS results explicitly labeled as partly reflecting the representation's exact-repetition property."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"TOMI is a clean, well-scoped engineering contribution. The core idea—represent a multi-track arrangement as a sparse graph of clip, section, track, and transformation nodes, then have an off-the-shelf LLM fill in the graph via in-context learning—is genuinely useful, and I don't know a prior system that does this for full-song electronic music with both MIDI and audio. The REAPER integration is a practical touch. The writing is clear, and the system is described in enough detail that someone could reimplement it. Code and demo links are provided.\n\nThe evaluation is where I'd push back, and the stress-test note lands. The ILS metric in Eqs. (1)–(4) is supposed to measure structural coherence, but Section 3.1.2 says a reused section node has identical content, and Section 3.1.5 makes reused sections share the same composition links. Same-label similarities are therefore partially guaranteed by construction. The ILS gap between TOMI and the Standalone LLM or MusicGen may mostly reflect that TOMI has an explicit copy mechanism, not that the LLM is arranging better. That's not nothing—exact repetition is a legitimate musical device—but the paper sells it as \"phrase development\" and \"structural consistency,\" and the metric can't separate good variation from identical copies. The Random ablation has the same structure and still scores lower, which shows the arrangement logic matters, but it doesn't fix the metric's interpretation.\n\nAlso, FAD scores are reported with standard deviations but no significance tests or confidence intervals; the MusicGen comparison is confounded by TOMI using professional sample packs; the reference set is 329 songs from one playlist. The subjective survey is more convincing, but Part 2 asks listeners to rate similarity between same-label sections, which rewards the same identity property. The scale is small: four keys, four section sequences, 32 pieces per method. The authors do acknowledge limitations around sample retrieval and harmonic coherence.\n\nIf I were editing: this deserves peer review. The representation and workflow are publishable, and the evaluation problems are fixable—bootstrap CIs for FAD, a human-arranged baseline using the same sample library, and a structural metric that distinguishes exact repetition from variation. I'd recommend major revision, not reject.","headline":"TOMI's sparse graph-plus-LLM arrangement workflow is a real, useful contribution, but its structural-coherence claims rest partly on a metric that rewards its own copy mechanism.","tokens_in":12759,"tokens_out":2378,"would_cite":true,"duration_ms":21074,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TOMI argues that a sparse graph of reusable clips, sections, tracks, and transformations, planned by a large language model, can generate full-song electronic music with stronger structural coherence than audio-generation baselines.","keywords":["concept hierarchy","music generation","multi-track composition","in-context learning","large language model","structural coherence","electronic music","composition link"],"falsifier":"An experiment that varies repeated sections to be non-identical while preserving the TOMI graph, then recomputes Inter-Phrase Latent Similarity and listener ratings, would show whether the structural advantage comes from planning coherence or simply from playing the same audio region twice.","tokens_in":11625,"feed_emoji":"🎧","tokens_out":8425,"duration_ms":81711,"temperature":0.7,"pith_summary":"TOMI sets out to show that full-song, multi-track electronic music can be generated by planning at the level of music ideas rather than by scaling up note- or audio-level generation. The system represents a composition as a sparse set of reusable clips (short MIDI or audio segments), sections (where in time), tracks (which instrument layer), and transformations (how the clip is changed), stitched together by composition links. A text-based LLM, prompted in-context, fills in the links, after which sample retrieval and a digital audio workstation render the song. The paper reports lower Fréchet Audio Distance and higher Inter-Phrase Latent Similarity than a strong text-to-music baseline and two ablations, plus higher listener ratings on most structural and quality metrics, supporting the claim that concept hierarchy is a viable path to long-form music generation.","feed_headline":"Sparse graph of clips yields full-song electronic music","feed_subtitle":"A four-node representation plus an instruction-tuned LLM beats a text-to-music model on quality and structure.","key_machinery":"The composition link is a quadruple (section, track, clip, transformation) in which all four node types are reusable objects, so one clip can appear in multiple sections or tracks under different transformations. Sections are temporal spans with a phrase label (verse, chorus, etc.), and a reused section node keeps identical content across its occurrences. The LLM is prompted to output this structure as text in a fixed order (sections, tracks, clips, transformations, links), with a rule-based validator catching and correcting syntax errors, and clip features are later used to retrieve real MIDI and audio samples from curated databases. This lets the LLM perform global arrangement planning without generating a single audio sample itself.","core_discovery":"The central claim, stated in the abstract and evaluation, is that a sparse four-node representation of clips, sections, tracks, and transformations, operated by an LLM through in-context learning, produces electronic music with tighter phrase boundaries, better phrase development, and clearer full-song structure than audio-generation baselines. The paper presents objective results where TOMI attains the lowest Fréchet Audio Distance and the highest Inter-Phrase Latent Similarity across three latent representations, and subjective results where listeners rate it above the alternatives on most questions. This is offered as evidence that concept hierarchy—transforming and organizing a small set of musical ideas—can handle long-form structure that larger end-to-end models still struggle to achieve.","pith_inferences":["The Inter-Phrase Latent Similarity metric may be measuring node-reuse identity more than perceived musical coherence, so a fairer comparison against a baseline that also reuses exact audio would be a stricter test of the structural claim.","The four-node composition-link abstraction could transfer to other structured creative media where a small set of assets is transformed and placed in time and space, such as video editing or interactive audio.","A testable extension would replace the closed instruction-tuned LLM with an open-weight model and measure how much of the quality depends on the underlying model's reasoning ability.","The relative contribution of sample-library quality versus arrangement logic to the objective scores is not separately measured, so an ablation that controls the same clips across methods would clarify the mechanism."],"forward_implications":["If the claim holds, long-form musical structure can be composed by planning over a sparse set of ideas rather than learned by ever-larger generative models.","The same text-serialized graph can let users steer a full song by describing sections, moods, and transformations in natural language, then edit the result in a digital audio workstation.","Because clips are retrieved from a sample library, the system's output is bounded by that library; the paper's future-work integration of generative clip models would remove that bound.","The LLM's role is arrangement rather than sound synthesis, so the same pipeline could operate over any library of audio or MIDI assets."],"supporting_citations":[{"why":"Defines the text-to-music baseline model that TOMI is compared against on quality and structure.","marker":"[6]"},{"why":"Introduces the Inter-Phrase Latent Similarity metric and the temporal-hierarchy generation approach that TOMI extends.","marker":"[9]"},{"why":"Provides evidence that popular songs use a small set of repeated core ideas, motivating the concept-hierarchy design.","marker":"[12]"},{"why":"Supplies the recombinant-music idea of transforming and reusing musical materials that TOMI operationalizes.","marker":"[13]"},{"why":"Documents the instruction-tuned foundation LLM whose in-context learning drives the composition planning.","marker":"[29]"},{"why":"Defines the Fréchet Audio Distance used to measure audio quality of the generated songs.","marker":"[32]"},{"why":"Gives the Cohen's d effect size used to refine the Inter-Phrase Latent Similarity metric.","marker":"[37]"}],"fun_headline_variants":["Concept hierarchy turns musical ideas into full-song structures","Sparse 4D representation enables LLM-based multi-track composition","TOMI: instruction-tuned LLM organizes clips into coherent songs","LLM with sparse clip-graph outdoes audio models on song structure"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's structural-coherence claim leans on the Inter-Phrase Latent Similarity metric rating identical repeated sections as strongly coherent, so if identical reuse is not what listeners mean by musical structure, the quantitative support weakens.","fun_headline_variants_meta":{"raw":{"variants":["Concept hierarchy turns musical ideas into full-song structures","Sparse 4D representation enables LLM-based multi-track composition","TOMI: instruction-tuned LLM organizes clips into coherent songs","LLM with sparse clip-graph outdoes audio models on song structure"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000231,"raw_usage":{"total_tokens":1451,"prompt_tokens":877,"completion_tokens":574,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":502}},"tokens_in":493,"tokens_out":574,"duration_ms":5918,"temperature":1.0,"reasoning_tokens":502,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:50:13.408544+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An experiment that varies repeated sections to be non-identical while preserving the TOMI graph, then recomputes Inter-Phrase Latent Similarity and listener ratings, would show whether the structural advantage comes from planning coherence or simply from playing the same audio region twice.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the text-to-music baseline model that TOMI is compared against on quality and structure."},{"cited_title":"Symbolic music generation with non-differentiable rule guided diffusion,","cited_arxiv_id":null,"evidence_quote":"Introduces the Inter-Phrase Latent Similarity metric and the temporal-hierarchy generation approach that TOMI extends."},{"cited_title":"Riffusion-stable diffusion for real-time music generation,","cited_arxiv_id":null,"evidence_quote":"Provides evidence that popular songs use a small set of repeated core ideas, motivating the concept-hierarchy design."},{"cited_title":"DeepBach: a steerable model for Bach chorales generation,","cited_arxiv_id":null,"evidence_quote":"Supplies the recombinant-music idea of transforming and reusing musical materials that TOMI operationalizes."},{"cited_title":"Composer’s Assistant 2: Interactive Multi- Track MIDI Infilling with Fine-Grained User Control,","cited_arxiv_id":null,"evidence_quote":"Documents the instruction-tuned foundation LLM whose in-context learning drives the composition planning."},{"cited_title":"Anticipatory music transformer,","cited_arxiv_id":null,"evidence_quote":"Gives the Cohen's d effect size used to refine the Inter-Phrase Latent Similarity metric."}],"review_version":1}