{"id":"ce111a8d-ee67-439e-8ef0-b9ff7a801a57","arxiv_id":"2509.00914","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"TinyMusician distills MusicGen and applies hand-picked mixed-precision quantization to make a 1.04 GB on-device music generator, but the headline '93% quality, 55% smaller' claims conflict with the paper's own tables.","lead":"This paper reports TinyMusician, a MusicGen-based music generator compressed to about 1 GB so it can run on an iPhone without a cloud connection. It claims 93% of the original quality with 55% less model size, but its own tables report different numbers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Same configuration is scored CLAP 0.343 in Table 3 and 0.373 in Table 4, and no evaluation protocol is given; the headline retention and SOTA claims rest on an inconsistent measurement.","rationale":"The reader's weakest assumption — that all scores come from a matched, single-definitive evaluation protocol — is exactly the load-bearing issue. The paper violates that assumption internally: TinyMusician-MixedPrecision receives CLAP 0.343 in Table 3 and 0.373 in Table 4, and no protocol details (clip count, duration, FAD corpus, sampling hyperparameters, seeds) are provided. Because the headline 'retains 93%' and the SOTA comparison depend on these numbers, the central quantitative claim is not verifiable from the paper. The basic engineering artifact (a 1.04 GB ONNX model deployed on iOS) appears plausible and deserves credit, but the performance claims are not reproducible as reported. I therefore agree with the reader's REJECT verdict and do not recommend changing it. A fixed-protocol rerun, as described, would settle whether the favorable numbers are real or an artifact of evaluation noise.","tokens_in":15179,"tokens_out":4273,"duration_ms":46715,"concrete_test":"Fix a protocol and rerun the evaluations in Tables 3 and 4: generate e.g. 100 MusicCaps prompts at a fixed duration with identical CFG, top-k, temperature, and seeds; use a single FAD reference embedding set and a single CLAP checkpoint; evaluate MusicGen-Small baseline, TinyMusician (KD only), TinyMusician-Int8, and TinyMusician-MixedPrecision, with at least three seeds. If TinyMusician-MixedPrecision reproduces CLAP 0.343 or a value outside the 0.343–0.373 range, the SOTA claim and the 'quantization as regularization' explanation fail. If it reproduces 0.373 consistently, the concern is mitigated. Independently recalculate model-size reduction; if it is 67.5%, the '55% less' claim is factually wrong.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim — that TinyMusician retains 93% of MusicGen-Small performance with 55% smaller size — depends entirely on reported FAD and CLAP scores. The evaluation is not shown to be matched or reproducible. The paper does not state the number or length of generated clips, the FAD reference corpus, generation hyperparameters (CFG scale, top-k, temperature), seeds, or whether competitor scores in Table 4 were recomputed under the same protocol. More importantly, the identical configuration (KD + mixed-precision quantization) is assigned two different CLAP scores: 0.343 in Table 3 and 0.373 in Table 4, with the favorable value used in the SOTA comparison. There are no error bars or repeated runs. An evaluation that can yield two different scores for the same model cannot support a precise retention percentage or the claim that TinyMusician 'outperforms all competitors.' The reported size reduction is also inconsistent (55% vs the 67.5% implied by 3.2 GB to 1.04 GB), and the teacher model is described as both MusicGen-Small and MusicGen-Large, but the decisive issue is that the measurement underpinning the headline is internally unstable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"TinyMusician proposes an on-device text-to-music system obtained by knowledge-distilling MusicGen-Large into MusicGen-Small and then applying hand-assigned mixed-precision quantization (Int8 text encoder, Float16 decoder, Float32 codec). The paper introduces a Stage-mixed Bidirectional and Skewed KL-Divergence loss with a temperature-annealing schedule, reports ablations over KD and quantization, compares against several music-generation baselines on FAD and CLAP, and demonstrates deployment on an iPhone 16 Pro via ONNX Runtime. The headline claims are that TinyMusician retains 93% of MusicGen-Small performance with 55% less model size, making it the first mobile-deployable music generation model that removes cloud dependency.","tokens_in":15291,"tokens_out":4564,"duration_ms":51338,"significance":"If the reported results were reliable, TinyMusician would be a useful engineering contribution: a 1.04 GB text-to-music model running on a phone, with an explicit loss formulation and a demonstrated iOS app. The paper also ships a code repository and uses a public dataset (MusicCaps). However, the central quantitative claims are not currently supported. The identical configuration is scored differently across Tables 3 and 4; no evaluation protocol or error bars are provided; the retention and size-reduction percentages are not derivable from the reported numbers; and the teacher/student identity is stated inconsistently. The engineering feasibility may be real, but the paper's scientific evidence for its headline claims is internally unstable.","major_comments":[{"comment":"The same configuration is scored inconsistently. Table 3 reports TinyMusician + Quantization with CLAP 0.343, while Table 4 reports TinyMusician-MixedPrecision with CLAP 0.373, with FAD 7.05 in both. The text in §5.3 uses the favorable 0.373 to claim that TinyMusician 'outperforms all competitors'. No error bars, repeated runs, or evaluation protocol are given; §4.6 and §5.3 do not state the number/length of generated clips, the FAD reference corpus, generation hyperparameters (CFG, top-k, temperature), or seeds. The '93% retention' headline is also a post-hoc inverse-FAD ratio (6.49/7.05 ≈ 92.1%) rather than an independent measure, and it conflicts with the '9.5% sacrificing' claim in §1 (which would correspond to an 8.6% FAD increase relative to baseline). Since the central claims rest on these exact numbers, the inconsistent measurements cannot support them.","section":"Tables 3–4, §5.2–5.3"},{"comment":"No matched evaluation protocol is described for the SOTA comparison. It is unclear whether the scores for CRFM, InspireMusic-Base, YuE-7B, DiffRhythm, and Magenta-Realtime were recomputed by the authors under identical conditions or taken from prior publications. The paper also uses inconsistent model names (Yue-7B vs YuE-7B, 'Mageneta-Realtime'). Without a common evaluation setup, the claim that TinyMusician 'outperforms all competitors' is unsupported. This is load-bearing because Table 4 is the main evidence for the claimed state-of-the-art trade-off.","section":"§4.6, §5.3"},{"comment":"The teacher/student identity is inconsistent. Section 3.1 says MusicGen-Large is the teacher and MusicGen-Small is the student, and Section 4.5 repeats this. However, the Introduction describes the model as 'distilled from the state-of-the-art MusicGen-Small architecture', and Section 5.2.1 interprets KD as preserving the baseline's architectural backbone. This ambiguity matters for interpreting whether KD transfers knowledge from a larger teacher or from the same-size baseline. The paper should specify the exact teacher checkpoint and whether teacher logits were precomputed offline.","section":"§3.1, §4.5, §5.2.1"},{"comment":"Size and performance numbers are internally inconsistent. The abstract and conclusion claim '55% less model size', but §5.2.2 reports model size shrinking from 3.2 GB to 1.04 GB, which is a 67.5% reduction. Likewise, 'retains 93% of MusicGen-Small performance' is not derivable from the reported metrics: the inverse-FAD ratio is 92.1%, while CLAP improves from 0.303 to 0.343/0.373, so there is no single 'performance retention' number. The '9.5% sacrificing melodic or harmonic fidelity' in §1 is also unexplained. These inconsistencies affect the paper's headline claims and need to be resolved with a precise definition of the retention metric.","section":"§5.2.2, Abstract"}],"minor_comments":[{"comment":"A paragraph beginning 'Unlike text or images...' is duplicated verbatim in the quantization discussion, interrupting the flow.","section":"§2.2"},{"comment":"Typographical and naming inconsistencies: 'Mageneta-Realtime' should be 'Magenta-Realtime'; 'YuE-7B' appears as 'Yue-7B' in the text; 'MusicGen-Small/ONNX(KD) Mixed' is ambiguous relative to TinyMusician-MixedPrecision.","section":"Table 4"},{"comment":"The training data description is vague: 200 GPT-4o-generated prompts are said to be split into train/validation/test, but the sizes of these splits and their relation to the 5,500 MusicCaps pairs are not stated. Also, the number of evaluation clips is not reported.","section":"§4.3"},{"comment":"The loss-curve figures lack axis labels and legible legends, making it difficult to verify the claimed 'lowest final loss' and 'dramatic oscillations' from the plots.","section":"Figures 2–3"}],"recommendation":"reject","confidential_remarks":"The engineering demonstration (1.04 GB ONNX model on iPhone 16 Pro) is potentially interesting, but the paper's scientific claims are not supportable in their current form. The same configuration receiving two different CLAP scores in Tables 3 and 4, without any protocol or error bars, is a fundamental validity issue, and the headline percentages are arithmetically inconsistent. This is not a case of a minor presentation fix; the evaluation would need to be redone and reported in a reproducible way before the claims could be assessed. I would also ask the authors to benchmark against existing on-device or efficient music generation systems before claiming 'first mobile-deployable' status."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing worth taking from this paper is the artifact: a 1.04 GB text-to-music model running on an iPhone, built by distilling MusicGen-Large into MusicGen-Small and applying per-component INT8/FP16/FP32 quantization. That is a genuinely useful deployment template for on-device music generation, and the authors deserve credit for shipping an actual iOS app and releasing code. The engineering story is real.\n\nThe new technical pieces are modest but legitimate: a stage-mixed bidirectional KL loss with convex mixtures and a linear temperature schedule, plus a hand-assigned mixed-precision quantization. The loss is a reasonable variant of Stepped BiKL, and the quantization choice is sensible, if not novel. The paper does not claim a new capability in music generation; it claims a smaller, phone-runnable model with acceptable quality. That claim is plausible in principle.\n\nBut the paper's own numbers do not hold together. The abstract says 55% size reduction while Section 5.2.2 shows 3.2 GB to 1.04 GB, which is 67.5%. The teacher is called MusicGen-Small in the intro and MusicGen-Large in Sections 3.1 and 4.5. More seriously, the same configuration (KD + mixed-precision quantization) is scored CLAP 0.343 in Table 3 and CLAP 0.373 in Table 4, with the favorable number used in the SOTA comparison. No evaluation protocol is given: no number of clips, no lengths, no FAD reference corpus, no generation hyperparameters, no seeds, no error bars. The '93% retention' is a post hoc ratio of their own FAD values (6.49/7.05 ≈ 92.1%), not a measured result. The ablated KD effect (FAD 6.49 to 6.44, CLAP 0.303 to 0.301) is inside single-run noise, and the loss is validated only by training curves, not by FAD/CLAP against the KL variants. The 'adaptive' quantization is actually a fixed hand-picked assignment.\n\nThis is not a case of a fabricated system; the deployment appears real. It is a case of claims and evaluation not being trustworthy as written. A thorough revision that fixes the numbers, specifies the protocol, and reports variance could make this a solid systems paper. As it stands, I would not rely on the quantitative conclusions, though I would not dismiss the engineering contribution either.\n\nFor a venue that values deployment and reproducibility of artifacts, I would send it to peer review—the artifact is concrete and the topic matters. But I would expect heavy revision, and I would ask the authors to reconcile every headline number with the tables before publication.","headline":"Real mobile deployment, but the evaluation is too sloppy to back the headline claims: the same model scores CLAP 0.343 in one table and 0.373 in another, and the size-reduction figure is inconsistent.","tokens_in":16070,"tokens_out":2288,"would_cite":false,"duration_ms":29092,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 1.04 GB music generator keeps 93% of MusicGen-Small's quality on a phone","keywords":["text-to-music generation","knowledge distillation","mixed-precision quantization","KL divergence","MusicGen","on-device inference","CLAP score","FAD score"],"falsifier":"Generate a fixed set of clips from baseline and TinyMusician using identical seeds, CFG scale, top-k, and maximum length, then compute FAD and CLAP against the same reference audio and CLAP model. If the reported FAD 7.05 and CLAP 0.373 do not reproduce, or if the 0.343 versus 0.373 discrepancy is not explained, the central quality-retention claim collapses.","tokens_in":14819,"feed_emoji":"🎵","tokens_out":5799,"duration_ms":60407,"temperature":0.7,"pith_summary":"The paper aims to show that a text-to-music model built from MusicGen can run entirely on a smartphone with near-baseline quality. It proposes TinyMusician, a distilled and mixed-precision-quantized version of MusicGen-Small that fits in 1.04 GB instead of 3.2 GB, and reports FAD 7.05 versus the baseline's 6.49, with a CLAP alignment score that rises after quantization. The authors interpret this as evidence that knowledge distillation combined with component-aware quantization breaks the usual fidelity-versus-size trade-off. If the result holds, on-device music generation becomes feasible for phones, wearables, and AR glasses without cloud dependency.","feed_headline":"Offline music AI shrinks to 1.04 GB and keeps 93% of its quality","feed_subtitle":"Knowledge distillation plus mixed-precision quantization puts text-to-music on a phone, no cloud needed.","key_machinery":"The central objects are (1) Stage-mixed Bidirectional and Skewed KL-Divergence, a loss that switches from forward to reverse KL at a step threshold and blends teacher and student distributions to stabilize distillation; (2) Adaptive Mixed-Precision Quantization, which assigns different precisions to different MusicGen components; and (3) a linear temperature-annealing schedule that lowers generation temperature over steps. Together they carry the size-quality trade-off: the KL loss transfers musical structure, the granular quantization protects audio reconstruction while compressing weights, and the temperature schedule balances global coherence with local detail.","core_discovery":"TinyMusician is a 1.04 GB text-to-music model distilled from MusicGen-Large into MusicGen-Small, then quantized component-wise: Int8 text encoder, Float16 decoder, Float32 audio codec. On MusicCaps it reports FAD 7.05 versus the 6.49 baseline and a CLAP score that rises after quantization (0.343 in the ablation, 0.373 in the comparison table), which the paper credits to quantization acting as regularization. The authors claim this is the first mobile-deployable music generation model that removes cloud dependency while preserving high audio fidelity.","pith_inferences":["Editorial: The 93% retention figure is computed from the inverse FAD ratio; by CLAP the same configuration differs (0.343 vs 0.373) across tables, so the retention claim is metric-dependent and needs a single canonical evaluation.","Editorial: The iOS demonstration is on one device and OS; Android, older iPhones, and wearables will need their own latency and quality benchmarks before 'on-device' is generalizable.","Editorial: If the quantization-as-regularization story is right, it predicts that moderate precision loss improves text alignment but hurts fidelity; this could be tested by sweeping bit-widths while holding distillation fixed.","Editorial: The method's success on MusicGen suggests the same distillation-plus-mixed-precision recipe could be applied to newer long-form music models, potentially shrinking them below current deployment thresholds."],"forward_implications":["Text-to-music generation can be packaged as a 1.04 GB offline app, enabling private, low-latency, cloud-free use on phones.","The component-wise quantization recipe—Int8 text encoder, Float16 music decoder, Float32 audio decoder—offers a template for compressing other MusicGen-derived models.","The bidirectional-plus-skewed KL loss, with its staged switch and linear temperature decay, is a general distillation objective for autoregressive audio models.","Mixed-precision quantization appears to beat both full precision and uniform Int8 on text-audio alignment, suggesting compression can act as regularization in music generation."],"supporting_citations":[{"why":"Supplies MusicGen-Small as the student, MusicGen-Large as the teacher, and the baseline architecture and scores.","marker":"Copet et al., 2023"},{"why":"Provides the MusicCaps dataset of music-text pairs used for distillation prompts and evaluation.","marker":"Lee et al., 2023"},{"why":"Defines the FAD score used as the audio fidelity metric in the ablation and comparisons.","marker":"Kilgour et al., 2018"},{"why":"Defines the CLAP score used to measure text-audio semantic alignment.","marker":"Ye et al., 2023"},{"why":"Provides the Encodec decoder, which TinyMusician keeps in Float32 for high-fidelity audio reconstruction.","marker":"Défossez et al., 2022"},{"why":"Supplies the stepped bidirectional KL idea that the Stage-mixed formula extends.","marker":"Yang et al., 2025"},{"why":"Provides the adaptive temperature annealing idea that the paper adapts into a linear decay schedule.","marker":"Manvi et al., 2024"},{"why":"Supplies the post-training mixed-precision quantization approach that the component-wise recipe builds on.","marker":"Chauhan et al., 2023"}],"fun_headline_variants":["Music AI shrinks to 1.04 GB for phones, keeps 93% quality","First on-device music generator: 1GB, no cloud, 93% fidelity","Distilled + quantized music model runs offline at 1GB","TinyMusician: mobile music AI that preserves 93% performance","1GB music model on your phone, no cloud, 93% quality"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The quality numbers rest on a single matched evaluation protocol; the same TinyMusician configuration is scored CLAP 0.343 in Table 3 and 0.373 in Table 4, so the protocol's consistency is already in question.","fun_headline_variants_meta":{"raw":{"variants":["Music AI shrinks to 1.04 GB for phones, keeps 93% quality","First on-device music generator: 1GB, no cloud, 93% fidelity","Distilled + quantized music model runs offline at 1GB","TinyMusician: mobile music AI that preserves 93% performance","1GB music model on your phone, no cloud, 93% quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000284,"raw_usage":{"total_tokens":1488,"prompt_tokens":697,"completion_tokens":791,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":441,"completion_tokens_details":{"reasoning_tokens":687}},"tokens_in":441,"tokens_out":791,"duration_ms":9147,"temperature":1.0,"reasoning_tokens":687,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T13:07:36.548445+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a fixed set of clips from baseline and TinyMusician using identical seeds, CFG scale, top-k, and maximum length, then compute FAD and CLAP against the same reference audio and CLAP model. If the reported FAD 7.05 and CLAP 0.373 do not reproduce, or if the 0.343 versus 0.373 discrepancy is not explained, the central quality-retention claim collapses.","supporting_citations":[],"review_version":1}