Pith. sign in

REVIEW 4 major objections 4 minor 43 references

A 1.04 GB music generator keeps 93% of MusicGen-Small's quality on a phone

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

TinyMusician distills MusicGen and applies hand-picked mixed-precision quantization to make a 1.04 GB on-device music generator, but the headline '93% quality, 55% smaller' claims conflict with the paper's own tables.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Real mobile deployment, but the evaluation is too sloppy to back the headline claims: the same model scores CLAP 0.343 in one table and 0.373 in another, and the size-reduction figure is inconsistent. the 4 major comments →

arxiv 2509.00914 v1 pith:KF4HTE6E submitted 2025-08-31 cs.SD cs.AIeess.AS

TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization

Hainan Wang , Mehdi Hosseinzadeh , Reza Rawassizadeh This is my paper
classification cs.SD cs.AIeess.AS
keywords text-to-music generationknowledge distillationmixed-precision quantizationKL divergenceMusicGenon-device inferenceCLAP scoreFAD score
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that a text-to-music model built from MusicGen can run entirely on a smartphone with near-baseline quality. It proposes TinyMusician, a distilled and mixed-precision-quantized version of MusicGen-Small that fits in 1.04 GB instead of 3.2 GB, and reports FAD 7.05 versus the baseline's 6.49, with a CLAP alignment score that rises after quantization. The authors interpret this as evidence that knowledge distillation combined with component-aware quantization breaks the usual fidelity-versus-size trade-off. If the result holds, on-device music generation becomes feasible for phones, wearables, and AR glasses without cloud dependency.

Core claim

TinyMusician is a 1.04 GB text-to-music model distilled from MusicGen-Large into MusicGen-Small, then quantized component-wise: Int8 text encoder, Float16 decoder, Float32 audio codec. On MusicCaps it reports FAD 7.05 versus the 6.49 baseline and a CLAP score that rises after quantization (0.343 in the ablation, 0.373 in the comparison table), which the paper credits to quantization acting as regularization. The authors claim this is the first mobile-deployable music generation model that removes cloud dependency while preserving high audio fidelity.

What carries the argument

The central objects are (1) Stage-mixed Bidirectional and Skewed KL-Divergence, a loss that switches from forward to reverse KL at a step threshold and blends teacher and student distributions to stabilize distillation; (2) Adaptive Mixed-Precision Quantization, which assigns different precisions to different MusicGen components; and (3) a linear temperature-annealing schedule that lowers generation temperature over steps. Together they carry the size-quality trade-off: the KL loss transfers musical structure, the granular quantization protects audio reconstruction while compressing weights, and the temperature schedule balances global coherence with local detail.

Load-bearing premise

The quality numbers rest on a single matched evaluation protocol; the same TinyMusician configuration is scored CLAP 0.343 in Table 3 and 0.373 in Table 4, so the protocol's consistency is already in question.

What would settle it

Generate a fixed set of clips from baseline and TinyMusician using identical seeds, CFG scale, top-k, and maximum length, then compute FAD and CLAP against the same reference audio and CLAP model. If the reported FAD 7.05 and CLAP 0.373 do not reproduce, or if the 0.343 versus 0.373 discrepancy is not explained, the central quality-retention claim collapses.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Text-to-music generation can be packaged as a 1.04 GB offline app, enabling private, low-latency, cloud-free use on phones.
  • The component-wise quantization recipe—Int8 text encoder, Float16 music decoder, Float32 audio decoder—offers a template for compressing other MusicGen-derived models.
  • The bidirectional-plus-skewed KL loss, with its staged switch and linear temperature decay, is a general distillation objective for autoregressive audio models.
  • Mixed-precision quantization appears to beat both full precision and uniform Int8 on text-audio alignment, suggesting compression can act as regularization in music generation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial: The 93% retention figure is computed from the inverse FAD ratio; by CLAP the same configuration differs (0.343 vs 0.373) across tables, so the retention claim is metric-dependent and needs a single canonical evaluation.
  • Editorial: The iOS demonstration is on one device and OS; Android, older iPhones, and wearables will need their own latency and quality benchmarks before 'on-device' is generalizable.
  • Editorial: If the quantization-as-regularization story is right, it predicts that moderate precision loss improves text alignment but hurts fidelity; this could be tested by sweeping bit-widths while holding distillation fixed.
  • Editorial: The method's success on MusicGen suggests the same distillation-plus-mixed-precision recipe could be applied to newer long-form music models, potentially shrinking them below current deployment thresholds.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. TinyMusician proposes an on-device text-to-music system obtained by knowledge-distilling MusicGen-Large into MusicGen-Small and then applying hand-assigned mixed-precision quantization (Int8 text encoder, Float16 decoder, Float32 codec). The paper introduces a Stage-mixed Bidirectional and Skewed KL-Divergence loss with a temperature-annealing schedule, reports ablations over KD and quantization, compares against several music-generation baselines on FAD and CLAP, and demonstrates deployment on an iPhone 16 Pro via ONNX Runtime. The headline claims are that TinyMusician retains 93% of MusicGen-Small performance with 55% less model size, making it the first mobile-deployable music generation model that removes cloud dependency.

Significance. If the reported results were reliable, TinyMusician would be a useful engineering contribution: a 1.04 GB text-to-music model running on a phone, with an explicit loss formulation and a demonstrated iOS app. The paper also ships a code repository and uses a public dataset (MusicCaps). However, the central quantitative claims are not currently supported. The identical configuration is scored differently across Tables 3 and 4; no evaluation protocol or error bars are provided; the retention and size-reduction percentages are not derivable from the reported numbers; and the teacher/student identity is stated inconsistently. The engineering feasibility may be real, but the paper's scientific evidence for its headline claims is internally unstable.

major comments (4)
  1. [Tables 3–4, §5.2–5.3] The same configuration is scored inconsistently. Table 3 reports TinyMusician + Quantization with CLAP 0.343, while Table 4 reports TinyMusician-MixedPrecision with CLAP 0.373, with FAD 7.05 in both. The text in §5.3 uses the favorable 0.373 to claim that TinyMusician 'outperforms all competitors'. No error bars, repeated runs, or evaluation protocol are given; §4.6 and §5.3 do not state the number/length of generated clips, the FAD reference corpus, generation hyperparameters (CFG, top-k, temperature), or seeds. The '93% retention' headline is also a post-hoc inverse-FAD ratio (6.49/7.05 ≈ 92.1%) rather than an independent measure, and it conflicts with the '9.5% sacrificing' claim in §1 (which would correspond to an 8.6% FAD increase relative to baseline). Since the central claims rest on these exact numbers, the inconsistent measurements cannot support them.
  2. [§4.6, §5.3] No matched evaluation protocol is described for the SOTA comparison. It is unclear whether the scores for CRFM, InspireMusic-Base, YuE-7B, DiffRhythm, and Magenta-Realtime were recomputed by the authors under identical conditions or taken from prior publications. The paper also uses inconsistent model names (Yue-7B vs YuE-7B, 'Mageneta-Realtime'). Without a common evaluation setup, the claim that TinyMusician 'outperforms all competitors' is unsupported. This is load-bearing because Table 4 is the main evidence for the claimed state-of-the-art trade-off.
  3. [§3.1, §4.5, §5.2.1] The teacher/student identity is inconsistent. Section 3.1 says MusicGen-Large is the teacher and MusicGen-Small is the student, and Section 4.5 repeats this. However, the Introduction describes the model as 'distilled from the state-of-the-art MusicGen-Small architecture', and Section 5.2.1 interprets KD as preserving the baseline's architectural backbone. This ambiguity matters for interpreting whether KD transfers knowledge from a larger teacher or from the same-size baseline. The paper should specify the exact teacher checkpoint and whether teacher logits were precomputed offline.
  4. [§5.2.2, Abstract] Size and performance numbers are internally inconsistent. The abstract and conclusion claim '55% less model size', but §5.2.2 reports model size shrinking from 3.2 GB to 1.04 GB, which is a 67.5% reduction. Likewise, 'retains 93% of MusicGen-Small performance' is not derivable from the reported metrics: the inverse-FAD ratio is 92.1%, while CLAP improves from 0.303 to 0.343/0.373, so there is no single 'performance retention' number. The '9.5% sacrificing melodic or harmonic fidelity' in §1 is also unexplained. These inconsistencies affect the paper's headline claims and need to be resolved with a precise definition of the retention metric.
minor comments (4)
  1. [§2.2] A paragraph beginning 'Unlike text or images...' is duplicated verbatim in the quantization discussion, interrupting the flow.
  2. [Table 4] Typographical and naming inconsistencies: 'Mageneta-Realtime' should be 'Magenta-Realtime'; 'YuE-7B' appears as 'Yue-7B' in the text; 'MusicGen-Small/ONNX(KD) Mixed' is ambiguous relative to TinyMusician-MixedPrecision.
  3. [§4.3] The training data description is vague: 200 GPT-4o-generated prompts are said to be split into train/validation/test, but the sizes of these splits and their relation to the 5,500 MusicCaps pairs are not stated. Also, the number of evaluation clips is not reported.
  4. [Figures 2–3] The loss-curve figures lack axis labels and legible legends, making it difficult to verify the claimed 'lowest final loss' and 'dramatic oscillations' from the plots.

Circularity Check

1 steps flagged

Minor self-referential loss-curve evaluation; central FAD/CLAP claims are not circular but suffer from measurement inconsistency.

specific steps
  1. self definitional [Section 5.1, Figure 3 (test loss comparison)]
    "The results shown in Figure 3 (b) present the superiority of our method. Among all compared KL Divergence formulations, our bidirectional KL Divergence with dynamic weighting achieves the lowest test loss. This not only indicates more stable training convergence but also reflects stronger generalization capability."

    The 'test loss' used to rank methods is the same KL-divergence objective (Eq. 1) that the student model is trained to minimize. Comparing different loss functions by their own final loss value evaluates each method on its own training criterion, so the proposed method's 'lowest test loss' is partly guaranteed by construction. This does not independently establish better music-generation quality. The paper's separate FAD/CLAP measurements are independent evidence, so this is a minor circular step rather than a load-bearing one.

full rationale

The central claimed derivation chain is not circular: TinyMusician is trained with a proposed KL-divergence loss and a hand-assigned mixed-precision quantization, then evaluated with FAD and CLAP. The 93% retention figure is a post-hoc ratio of the paper's own measured FAD scores (6.49/7.05), but it is a descriptive summary of external measurements, not a fitted parameter or a prediction derived from the model's definition. The same-configuration CLAP discrepancy (0.343 in Table 3 vs. 0.373 in Table 4), the missing evaluation protocol, and the inconsistent size-reduction arithmetic (3.2 GB to 1.04 GB is 67.5%, not 55%) are serious reproducibility and correctness concerns, but they are not instances of circularity under the strict definition used here. There is one mild self-referential step: Section 5.1 uses the training/test loss of the proposed KL objective as evidence of superiority, which is the same objective being optimized. This does not undermine the independent FAD/CLAP evidence for the final model, so the overall circularity score is low.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central claims rest on unreported loss hyperparameters, hand-chosen component precisions, a 200-prompt distillation corpus, and an unmatched single-run evaluation protocol. No new physical or model entities are introduced; the 'stage-mixed' loss is a loss function rather than an entity. The most consequential assumptions are measurement assumptions: that single-run FAD/CLAP values are stable, that competitor scores are directly comparable, and that the CLAP improvement is a genuine quantization effect rather than noise or an artifact.

free parameters (6)
  • gamma_1 (forward-KL mixing weight) = not reported
    Balances the two forward KL terms in Eq. 1 ([γ1 D_KL(T||S) + (1-γ1) D_KL(T||Sλ)]); no value or tuning procedure is given anywhere in the paper.
  • gamma_2 (reverse-KL mixing weight) = not reported
    Balances the two reverse KL terms in Eq. 1 ([γ2 D_KL(S||T) + (1-γ2) D_KL(S||Tλ)]); value never stated.
  • lambda (distribution mixture coefficient) = not reported
    Defines Sλ = λT + (1-λ)S and Tλ = (1-λ)T + λS in Eqs. 2 and 3; no value or schedule is given, and Table 2 lists only the Stepped BiKL variant as having an adaptive λ.
  • tau_step (stage-switch threshold) = not reported
    In Eq. 4, α(t) jumps from 1 to 0 at tau_step, switching the loss from teacher-to-student terms to student-to-teacher terms; the threshold value is never reported.
  • Tb and Tf (temperature schedule bounds) = not reported
    In Eq. 5, the distillation temperature decays linearly from Tb to Tf over Lmax steps; neither bound is given, so the 'adaptive temperature annealing' cannot be reproduced.
  • Per-component precision assignment (Int8 text encoder, Float16 decoder, Float32 codec) = Int8 / Float16 / Float32
    Section 3.2 fixes these precisions by hand; they fully determine the headline 1.04 GB size and the 67.5% measured reduction, yet no selection criterion, sensitivity analysis, or calibration data accompanies the choice.
axioms (5)
  • domain assumption FAD and CLAP scores computed by the authors and cited from prior papers are directly comparable across models with different sizes, decoders, and generation settings.
    Section 5.3 compares TinyMusician with YuE-7B, DiffRhythm, CRFM, InspireMusic-Base, and Magenta-Realtime, but the paper never specifies matched prompting, generated clip length, number of clips, FAD reference corpus, or inference settings for the competitors.
  • domain assumption Logit-level distillation on 200 GPT-4o-generated prompts transfers melodic and harmonic fidelity from the teacher to the student.
    Section 4.3 describes the entire distillation corpus as 200 prompts split into train, validation, and test, with no analysis that this corpus covers the MusicCap evaluation distribution and no loss hyperparameters reported.
  • domain assumption The quantization-induced CLAP improvement (0.303 to 0.352) is a genuine regularization effect rather than evaluation noise or a PTQ artifact.
    Section 5.2.2 explains the counterintuitive CLAP gain post hoc as regularization, with no additional experiments such as multiple seeds, calibration ablations, or a check that the CLAP model is sensitive to the relevant musical attributes.
  • domain assumption The teacher for the distillation is MusicGen-Large while the student is MusicGen-Small, as stated in Sections 3.1 and 4.5.
    The abstract and introduction phrase TinyMusician as distilled from MusicGen or MusicGen-Small, which conflicts with the body's explicit teacher choice; the unresolved ambiguity affects how the headline retention claim should be interpreted.
  • domain assumption Single-run training and test loss curves (Figures 2 and 3) and single-run FAD/CLAP values are representative of differences in generation quality between loss variants.
    All comparisons in Sections 5.1 and 5.2 rest on one training run per configuration with no seeds, error bars, or significance testing, so the claimed superiority of the stage-mixed loss is not quantified.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization." pith.science (2026). https://pith.science/paper/KF4HTE6E

@misc{pith2026250900914,
  author       = {Pith},
  title        = {Pith review of: TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KF4HTE6E}},
  note         = {Machine review of arXiv:2509.00914}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The success of the generative model has gained unprecedented attention in the music generation area. Transformer-based architectures have set new benchmarks for model performance. However, their practical adoption is hindered by some critical challenges: the demand for massive computational resources and inference time, due to their large number of parameters. These obstacles make them infeasible to deploy on edge devices, such as smartphones and wearables, with limited computational resources. In this work, we present TinyMusician, a lightweight music generation model distilled from MusicGen (a State-of-the-art music generation model). TinyMusician integrates two innovations: (i) Stage-mixed Bidirectional and Skewed KL-Divergence and (ii) Adaptive Mixed-Precision Quantization. The experimental results demonstrate that TinyMusician retains 93% of the MusicGen-Small performance with 55% less model size. TinyMusician is the first mobile-deployable music generation model that eliminates cloud dependency while maintaining high audio fidelity and efficient resource usage

Figures

Figures reproduced from arXiv: 2509.00914 by Hainan Wang, Mehdi Hosseinzadeh, Reza Rawassizadeh.

Figure 1
Figure 1. Figure 1: The architecture of TinyMusician with respect to its teacher model, i.e., MusicGen small. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of Training Loss on different KL-Divergence Methods [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of Test Loss Model FADscore ↓ CLAPscore ↑ MusicGen-Small (Baseline) 6.49 0.303 TinyMusician 6.44 0.301 MusicGen-Small + Quantization 7.11 0.352 TinyMusician + Quantization 7.05 0.343 [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Resource Utilization Comparison between TinyMusician and MusicGen-Small with dif [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Resource utilization comparison among different models. TinyMusician-MP: [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: iOS Music Generation App Overview 18 [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

43 extracted references · 10 canonical work pages · 5 internal anchors

  1. [1]

    Musiclm: Generating music from text

    Andrea Agostinelli, Timo I Denk, Zal ´an Borsos, Jesse Engel, Mauro Verzetti, Antoine Caillon, Qingqing Huang, Aren Jansen, Adam Roberts, Marco Tagliasacchi, et al. Musiclm: Generating music from text. arXiv preprint arXiv:2301.11325,

    Pith/arXiv arXiv

  2. [5]

    High fidelity neural audio compression

    Alexandre D ´efossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. High fidelity neural audio compression. arXiv preprint arXiv:2210.13438,

    Pith/arXiv arXiv

  3. [9]

    Flux that plays music

    Zhengcong Fei, Mingyuan Fan, Changqian Yu, and Junshi Huang. Flux that plays music. arXiv preprint arXiv:2409.00587,

    Pith/arXiv arXiv

  4. [10]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323,

    Pith/arXiv arXiv

  5. [14]

    Distilling the knowledge in a neural network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531,

    Pith/arXiv arXiv

  6. [15]

    Chih-Fang Huang and Cheng-Yuan Huang

    URL https://arxiv.org/abs/2106.09685. Chih-Fang Huang and Cheng-Yuan Huang. Emotion-based ai music generation system with cvae- gan. In 2020 IEEE Eurasia Conference on IOT, Communication and Engineering (ECICE) , pp. 220–222. IEEE,

    Pith/arXiv arXiv 2020

  7. [16]

    Scaling laws for neural language models

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361,

    Pith/arXiv arXiv 2001

  8. [17]

    Fr\’echet audio distance: A metric for evaluating music enhancement algorithms

    Kevin Kilgour, Mauricio Zuluaga, Dominik Roblek, and Matthew Sharifi. Fr\’echet audio distance: A metric for evaluating music enhancement algorithms. arXiv preprint arXiv:1812.08466,

    Pith/arXiv arXiv

  9. [18]

    Bild: Bi-directional logits difference loss for large language model distillation

    Minchong Li, Feng Zhou, and Xiaohui Song. Bild: Bi-directional logits difference loss for large language model distillation. arXiv preprint arXiv:2406.13555,

    Pith/arXiv arXiv

  10. [19]

    MIDI-Sandwich2: RNN-based Hierarchical Multi-modal Fusion Generation VAE networks for multi-track symbolic music generation

    Xia Liang, Junmin Wu, and Jing Cao. Midi-sandwich2: Rnn-based hierarchical multi-modal fusion generation vae networks for multi-track symbolic music generation. arXiv preprint arXiv:1909.03522,

    work page internal anchor Pith review Pith/arXiv arXiv 1909

  11. [20]

    Think-in-memory: Recalling and post-thinking enable llms with long-term memory

    Lei Liu, Xiaoyan Yang, Yue Shen, Binbin Hu, Zhiqiang Zhang, Jinjie Gu, and Guannan Zhang. Think-in-memory: Recalling and post-thinking enable llms with long-term memory. arXiv preprint arXiv:2311.08719,

    Pith/arXiv arXiv

  12. [21]

    Quantization hurts reasoning? an empirical study on quantized reasoning models

    Ruikang Liu, Yuxuan Sun, Manyi Zhang, Haoli Bai, Xianzhi Yu, Tiezheng Yu, Chun Yuan, and Lu Hou. Quantization hurts reasoning? an empirical study on quantized reasoning models. arXiv preprint arXiv:2504.04823,

    Pith/arXiv arXiv

  13. [22]

    The impact of quantization and pruning on deep reinforcement learning models

    Heng Lu, Mehdi Alemi, and Reza Rawassizadeh. The impact of quantization and pruning on deep reinforcement learning models. arXiv preprint arXiv:2407.04803,

    Pith/arXiv arXiv

  14. [23]

    Adaptive inference-time compute: Llms can predict if they can do better, even mid-generation

    Rohin Manvi, Anikait Singh, and Stefano Ermon. Adaptive inference-time compute: Llms can predict if they can do better, even mid-generation. arXiv preprint arXiv:2410.02725,

    Pith/arXiv arXiv

  15. [24]

    Mustango: Toward controllable text-to-music generation

    Jan Melechovsky, Zixun Guo, Deepanway Ghosal, Navonil Majumder, Dorien Herremans, and Soujanya Poria. Mustango: Toward controllable text-to-music generation. arXiv preprint arXiv:2311.08355,

    Pith/arXiv arXiv

  16. [25]

    Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models

    Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith B Hall, Daniel Cer, and Yin- fei Yang. Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models. arXiv preprint arXiv:2108.08877,

    Pith/arXiv arXiv

  17. [26]

    Diffrhythm: Blazingly fast and embarrassingly simple end-to-end full-length song generation with latent diffusion

    Ziqian Ning, Huakang Chen, Yuepeng Jiang, Chunbo Hao, Guobin Ma, Shuai Wang, Jixun Yao, and Lei Xie. Diffrhythm: Blazingly fast and embarrassingly simple end-to-end full-length song generation with latent diffusion. arXiv preprint arXiv:2503.01183,

    Pith/arXiv arXiv

  18. [27]

    Stay on topic with classifier-free guidance

    Guillaume Sanchez, Honglu Fan, Alexander Spangher, Elad Levi, Pawan Sasanka Ammana- manchi, and Stella Biderman. Stay on topic with classifier-free guidance. arXiv preprint arXiv:2306.17806,

    Pith/arXiv arXiv

  19. [28]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108,

    Pith/arXiv arXiv 1910

  20. [29]

    Efficient large-scale audio tagging via transformer-to-cnn knowledge distillation

    Florian Schmid, Khaled Koutini, and Gerhard Widmer. Efficient large-scale audio tagging via transformer-to-cnn knowledge distillation. In ICASSP 2023-2023 IEEE international Conference on acoustics, Speech and signal processing (ICASSP), pp. 1–5. IEEE,

    work page 2023

  21. [30]

    Post-training quantization on diffusion models

    Yuzhang Shang, Zhihang Yuan, Bin Xie, Bingzhe Wu, and Yan Yan. Post-training quantization on diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1972–1981,

    work page 1972

  22. [32]

    Patient knowledge distillation for bert model compression

    Siqi Sun, Yu Cheng, Zhe Gan, and Jingjing Liu. Patient knowledge distillation for bert model compression. arXiv preprint arXiv:1908.09355, 2019a. Siqi Sun, Yu Cheng, Zhe Gan, and Jingjing Liu. Patient knowledge distillation for bert model compression. arXiv preprint arXiv:1908.09355, 2019b. Suno-Ai. Suno-AI: text-prompted Generative Audio Model, May

    Pith/arXiv arXiv 1908

  23. [33]

    com/suno-ai/bark

    URLhttps://github. com/suno-ai/bark. Yehui Tang, Yunhe Wang, Jianyuan Guo, Zhijun Tu, Kai Han, Hailin Hu, and Dacheng Tao. A survey on transformer compression. arXiv preprint arXiv:2402.05964,

    Pith/arXiv arXiv

  24. [34]

    John Thickstun, David Hall, Chris Donahue, and Percy Liang

    URL https://g.co/magenta/rt. John Thickstun, David Hall, Chris Donahue, and Percy Liang. Anticipatory music transformer. arXiv preprint arXiv:2306.08620,

    Pith/arXiv arXiv

  25. [35]

    Attentions Under the Microscope: A Comparative Study of Resource Utilization for Variants of Self-Attention

    Zhengyu Tian, Anantha Padmanaban Krishna Kumar, Hemant Krishnakumar, and Reza Rawas- sizadeh. Attentions Under the Microscope: A Comparative Study of Resource Utilization for Variants of Self-Attention. arXiv preprint arXiv:2507.07247,

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [36]

    Transformers in time series: A survey

    Qingsong Wen, Tian Zhou, Chaoli Zhang, Weiqi Chen, Ziqing Ma, Junchi Yan, and Liang Sun. Transformers in time series: A survey. arXiv preprint arXiv:2202.07125,

    Pith/arXiv arXiv

  27. [37]

    One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teachers

    Chuhan Wu, Fangzhao Wu, and Yongfeng Huang. One teacher is enough? pre-trained language model distillation from multiple teachers. arXiv preprint arXiv:2106.01023,

    work page internal anchor Pith review Pith/arXiv arXiv

  28. [38]

    CLAPSpeech: Learning Prosody from Text Context with Contrastive Language-Audio Pre-training

    Zhenhui Ye, Rongjie Huang, Yi Ren, Ziyue Jiang, Jinglin Liu, Jinzheng He, Xiang Yin, and Zhou Zhao. Clapspeech: Learning prosody from text context with contrastive language-audio pre- training. arXiv preprint arXiv:2305.10763,

    work page internal anchor Pith review Pith/arXiv arXiv

  29. [39]

    Yue: Scaling open foundation models for long-form music generation

    Ruibin Yuan, Hanfeng Lin, Shuyue Guo, Ge Zhang, Jiahao Pan, Yongyi Zang, Haohe Liu, Yiming Liang, Wenye Ma, Xingjian Du, et al. Yue: Scaling open foundation models for long-form music generation. arXiv preprint arXiv:2503.08638,

    arXiv

  30. [40]

    Haohang Zhang, Letian Xie, and Kaiyi Qi

    URL https://arxiv.org/abs/2503.00084. Haohang Zhang, Letian Xie, and Kaiyi Qi. Implement music generation with gan: A systematic review. In 2021 International Conference on Computer Engineering and Application (ICCEA) , pp. 352–355. IEEE,

    Pith/arXiv arXiv 2021

  31. [41]

    Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning

    Zongmeng Zhang, Yufeng Shi, Jinhua Zhu, Wengang Zhou, Xiang Qi, Peng Zhang, and Houqiang Li. Trustworthy alignment of retrieval-augmented large language models via reinforcement learn- ing. arXiv preprint arXiv:2410.16843,

    work page internal anchor Pith review Pith/arXiv arXiv

  32. [42]

    To prune, or not to prune: exploring the efficacy of pruning for model compression

    Michael Zhu and Suyog Gupta. To prune, or not to prune: exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878,

    Pith/arXiv arXiv

  33. [43]

    Ernie-music: Text-to-waveform music generation with diffusion models

    Pengfei Zhu, Chao Pang, Yekun Chai, Lei Li, Shuohuan Wang, Yu Sun, Hao Tian, and Hua Wu. Ernie-music: Text-to-waveform music generation with diffusion models. arXiv preprint arXiv:2302.04456,

    Pith/arXiv arXiv

  34. [2006]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171–4186,

    work page 2019

  35. [2010]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502,

    Pith/arXiv arXiv 2010

  36. [2017]

    Multi-instrument music synthesis with spectrogram diffusion

    Curtis Hawthorne, Ian Simon, Adam Roberts, Neil Zeghidour, Josh Gardner, Ethan Manilow, and Jesse Engel. Multi-instrument music synthesis with spectrogram diffusion. arXiv preprint arXiv:2206.05408,

    Pith/arXiv arXiv

  37. [2019]

    Jukebox: A generative model for music

    Prafulla Dhariwal, Heewoo Jun, Christine Payne, Jong Wook Kim, Alec Radford, and Ilya Sutskever. Jukebox: A generative model for music. arXiv preprint arXiv:2005.00341,

    Pith/arXiv arXiv 2005

  38. [2020]

    Learned step size quantization

    Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmen- dra S Modha. Learned step size quantization. arXiv preprint arXiv:1902.08153,

    Pith/arXiv arXiv 1902

  39. [2021]

    Music generator with markov chain: A case study with beatme touchdown

    Zaky Hassani and Aciek Ida Wuryandari. Music generator with markov chain: A case study with beatme touchdown. In 2016 6th international conference on system engineering and technology (ICSET), pp. 179–183. IEEE,

    work page 2016

  40. [2022]

    Polyphonic music generation by modeling temporal dependencies using a rnn-dbn

    Kratarth Goel, Raunaq V ohra, and Jajati Keshari Sahoo. Polyphonic music generation by modeling temporal dependencies using a rnn-dbn. In Artificial Neural Networks and Machine Learning– ICANN 2014: 24th International Conference on Artificial Neural Networks, Hamburg, Germany, September 15-19,

    work page 2014

  41. [2023]

    Program synthesis with large language models

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732,

    Pith/arXiv arXiv

  42. [2024]

    Emergent abilities in large language models: A survey

    Leonardo Berti, Flavio Giorgi, and Gjergji Kasneci. Emergent abilities in large language models: A survey. arXiv preprint arXiv:2503.05788,

    Pith/arXiv arXiv

  43. [2025]

    Deepseek llm: Scaling open-source language models with longtermism

    Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954,

    Pith/arXiv arXiv

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.