Pith. sign in

REVIEW 4 major objections 5 minor 42 references

RIME turns any music corpus into paired edit-instruction data for agentic post-production, then shows today's multimodal models can rarely translate abstract audio requests into correct studio edits.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 12:14 UTC pith:YFIVU5VO

load-bearing objection RIME is a genuinely useful synthetic-data framework for agentic music post-production, but the benchmark's external validity is unproven and the paper should either add human/external validation or qualify its claims. the 4 major comments →

arxiv 2607.19605 v2 pith:YFIVU5VO submitted 2026-07-21 cs.SD

RIME: Enabling Large-Scale Agentic Music Post-Production

Noah Schaffer , Nikhil Singh This is my paper
classification cs.SD
keywords agentic music post-productionrule-based data generationmusic editing benchmarkmultimodal LLM agentsaudio effectssource separationprompt abstractionsupervised fine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that music post-production — the iterative loop of listening, tweaking a stem, and re-mixing — can be treated as a learnable agent task, and that the missing ingredient is data that reflects how engineers actually talk and work. To supply that data, it introduces RIME, a rule-based generator that turns ordinary mixed tracks into triples of input audio, edited audio, and natural-language instruction by composing recipes, reusable sub-chains, ordering constraints, and calibrated parameter priors. Using a new POEMS toolkit that exposes stem separation, mixing, and studio effects through a standard tool-calling interface, the authors build a benchmark of about 3,000 instruction-audio pairs plus 300 artifact-removal cases. On this benchmark, current multimodal LLM agents consistently struggle: they often pick the right operators but mis-set parameters, and performance drops sharply as instructions become less technically specific. Finally, the paper shows that fine-tuning an open-weight model on RIME data improves its handling of abstract instructions, especially at higher abstraction levels.

Core claim

The paper's central claim is that the language of music post-production — 'make it warmer,' 'tuck the harmony back,' 'get rid of the hum' — is dense, consistent, and learnable, and that a rule-based generator can produce realistic supervision for it. RIME encodes studio knowledge as symbolic recipes over edit graphs of the form separate → process → mix; each recipe is gated by clip metadata, scored by pattern policies, constrained by a default effect-chain order, and instantiated with parameters drawn from auditioned priors. Executing these graphs with POEMS yields ground-truth edited audio paired with instructions at three levels of abstraction. The authors then use this data as a benchmark

What carries the argument

The load-bearing object is the RIME edit graph: a declarative recipe template with unbound slots for target stem, key, and parameters, expanded into an executable sequence of POEMS tool calls. Recipes are built from named reusable sub-graphs (motifs), restricted by a chain-order constraint (EQ → dynamics → distortion → modulation → time-based effects → leveling), weighted by pattern-policy priors, and randomized with per-role parameter priors so draws are plausible. The same machinery generates both degradation recipes (hum, rumble, sibilance) and their matched remediation recipes, enabling artifact-removal evaluation. POEMS supplies the actual audio operations, including source separation,

Load-bearing premise

The load-bearing premise is that RIME's hand-authored recipe catalog, chain-order constraints, and parameter priors adequately represent real studio post-production workflows; the paper's only support is that an audio engineer reviewed and tweaked the components, with no listening test or external baseline.

What would settle it

Collect a corpus of real studio edit sessions, with the same input tracks edited by professional engineers to fulfill identical written requests, and compare the engineers' chosen tool chains and parameter values against RIME's generated graphs. If many real edits fall outside the 12 recipes, violate the default chain order, or use parameter ranges outside the priors, the benchmark's claim to measure real post-production collapses. A simpler version: a listening test in which professional engineers rate whether RIME-generated edits sound like plausible real studio results; systematic 'implausi

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any corpus of mixed music, without stem annotations, can be converted into a large pool of (input, output, instruction) triples; RIME's procedure is dataset-agnostic.
  • A concrete, measurable failure mode is identified: zero-shot agents select the correct operator sequence more often than they set parameters correctly, and this parameter gap is the main driver of audio deviation.
  • Performance degrades predictably as instructions become more abstract, so abstraction level should be a reporting axis for any future post-production agent benchmark.
  • Supervised fine-tuning on RIME-generated data — even training only the reasoning step — improves downstream audio and graph metrics at higher abstraction levels, without relying on memorizing specific test recipes.
  • Artifact removal (mains hum, rumble, sibilance) is separable and harder under ambiguity, giving the benchmark a task dimension beyond intentional effects.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper: if the recipe catalog and priors are representative, the same RIME data could be used to supervise planning and tool-argument generation jointly, not just the reasoning step; the paper only fine-tunes reasoning.
  • The benchmark's current scope is set by a 12-recipe catalog and a fixed effect-chain order; a natural test of the framework is whether expanding the catalog (or learning it from real session data) changes the measured agent ranking.
  • Because RIME produces ground-truth edit graphs, it could support studies of chain-level credit assignment — for instance, whether agents trained to predict intermediate edit states generalize better than agents trained end-to-end on audio alone.
  • A human listening study comparing RIME edits against real engineer edits on the same tracks would be the natural external validity check; the paper currently relies on expert auditioning during authoring.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper formalizes agentic music post-production as an iterative task in which an agent receives a mix and a natural-language instruction, then applies a sequence of editing operations to stems to produce a refined mix. It introduces RIME, a rule-based generator that creates (input, output, instruction) triples from arbitrary audio corpora using a catalog of twelve hand-authored recipes, and POEMS, an MCP-based toolkit for pitch, EQ, dynamics, modulation, time-based effects, mixing, and source separation. Using MTG-Jamendo, the authors generate 3,000 training and 3,000 evaluation triples, evaluate zero-shot agents (GPT-4o Mini, Gemini 3 Flash, Gemma 3n) plus an SFT variant of Gemma 3n on agentic tool-calling, and report that zero-shot models struggle, especially as instruction abstraction increases, while fine-tuning improves abstraction handling. The central claims are that RIME provides realistic, learnable supervision for post-production and that the benchmark validly measures agent post-production capability.

Significance. If the claims are substantiated, RIME and POEMS would be useful contributions: the pipeline is a scalable way to generate instruction-audio triplets without manual annotation, and the agent benchmark addresses a realistic gap in music AI. The paper also ships a concrete toolkit and an open evaluation setup, which is credit-worthy. However, the current evidence does not fully support the load-bearing assertions. The benchmark's ground truth is entirely synthetic and generated by the same RIME pipeline that creates the training data; there is no human listening validation, no external baseline, and no evidence that the twelve-recipe catalog and fixed chain-order constraint span real studio practice. As a result, the SFT improvements and the abstract's unqualified 'post-production capabilities' claim are not yet established.

major comments (4)
  1. [Table 1; §5.4] Several reported FAD and KAD values are negative (e.g., GPT-4o Mini AL0: FADinf = -0.047, KAD = -0.021; Gemma 3n AL0: FADinf = -0.047, KAD = -0.027; Gemma 3n SFT guitar: FADinf = -0.048). As distance metrics, these values are impossible; the note in the caption that they indicate 'upstream numerical instability' is not a fix. Metrics computed this way cannot support the comparative claims in §6.1 and §6.4. The authors must correct the metric computation, or remove these cells and re-evaluate all conclusions that rely on them.
  2. [§5.1; §6.4] The evaluation and training data are both generated by the same RIME pipeline with the same twelve recipes, chain-order constraints, and parameter priors. Although the training and evaluation tracks are disjoint, the generative distribution is identical. Therefore the SFT improvements (e.g., GEMMA3N SFT at AL1/AL2 in Table 1) could reflect in-distribution learning of RIME's recipe structure and prompt style rather than generalizable post-production capability. The statement in §6.4 that 'fine-tuning does not rely on recipe-memorization effects' is not supported by any evidence. An external benchmark, human-generated edit targets, or a transfer test to a different recipe distribution is needed.
  3. [§4.1.1; Appendix C] The only validation offered for the claim that RIME produces 'realistic' post-production data is that 'an audio engineer with professional production credits reviewed, auditioned, and tweaked all of our components' (§4.1.1). This is a single expert's informal review, not a systematic evaluation. The recipe catalog contains only twelve recipes, and the global chain-order constraint (EQ→Dynamics→Distortion→Modulation→Time-Based→Leveling) is prescriptive and may not reflect the variety of real workflows. Without listening tests, a comparison to real studio instruction corpora, or multi-engineer validation, the realism premise remains unsubstantiated.
  4. [§5.4; Appendix E.1] The paper does not describe how FAD, FADinf, and KAD are computed for the per-example comparisons in Table 1. FAD and KAD are distributional metrics, and using them to compare individual audio outputs to individual ground-truth edits is non-standard. The meaning of 'FADinf' is not defined. Without these computational details (embedding extraction, distance formula, whether distributions or single samples are used), the audio-similarity results cannot be interpreted or reproduced.
minor comments (5)
  1. [Abstract] The abstract says '3,000 pairs of edit instructions and ground truth audio,' but the paper actually generates 3,000 triples for training and 3,000 for evaluation (§5.1). Please clarify the total amount of data.
  2. [Related Work] LLM2Fx-Tools is identified as the closest existing benchmark, yet no comparison is made. Even a brief discussion of why a direct comparison is infeasible (e.g., different tool interface or task setup) would help position the contribution.
  3. [Table 1] The column 'FADinf' is never defined in the text. Please define it in §5.4.
  4. [Figure 1] Figure 1 appears before the abstract but is not referenced in the main text. Please cite it in §1.
  5. [§6.4; Figure 4] Some of the SFT advantages are not accompanied by statistical significance tests for the overall Table 1 values. The bootstrap CIs in Figures 4 and 11 are helpful; consider adding similar intervals to Table 1 or a supplementary summary.

Circularity Check

0 steps flagged

No significant circularity: RIME is an internally consistent synthetic benchmark; the external-validity gap is acknowledged by the authors and is a validity concern, not a circularity.

full rationale

The paper's derivation chain is empirical rather than analytical: RIME builds edit graphs from a hand-authored recipe catalog, executes them through POEMS to create (input, output, instruction) triples, and then measures agent agreement with those graphs. Each stage is defined independently of the measurements it supports: recipe priors come from cited external production references and an engineer's review, audio transformations are computed with Pedalboard/Demucs/CREPE/PSOLA, and evaluation metrics compare agent outputs to RIME-generated graphs. No step reuses the result it claims to establish. The SFT experiment trains on 3,000 RIME triples and evaluates on disjoint tracks generated from the same recipe distribution; this is an in-distribution benchmark evaluation, not a prediction that is statistically forced by the training labels, and the paper even reports SFT underperforming at AL0. The concern that RIME's recipe space may not represent real studio workflows is a validity limitation, explicitly acknowledged in the conclusion ('no external baselines yet available'), not a circularity. There are no self-citations: all cited works are external tools, datasets, or prior benchmarks. Under the required standard of exhibiting a specific reduction by construction, no circular step can be identified.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claims rest on hand-authored priors and recipes (the data distribution), domain assumptions about audio embeddings and separation quality, and an implicit transfer assumption. No externally validated ground truth is provided.

free parameters (5)
  • Recipe catalog (12 hand-authored recipes)
    The set of post-production intents (vocal harmony, parallel compression, retro_tilt, etc.) is authored by the researchers, not derived from data; it defines the space of possible edits.
  • Pattern-policy weights and multipliers = Base weights 0.30–1.00; multipliers 0.15–1.35
    Hand-set priorities over recipes per target family/genre (Appendix C.3, Figure 5); these shape which edits appear in the benchmark.
  • Per-tool parameter sampling priors = e.g., high-pass 60–250 Hz; harmony thirds preferred
    Priors in Appendix C.4 are calibrated by hand to sound plausible; they determine the exact 'ground-truth' audio and thus influence every metric.
  • Chain-order constraint = EQ→Dynamics→Distortion→Modulation→Time-Based→Leveling
    Adopted convention used to reject candidate graphs; if wrong, generated graphs deviate from real practice.
  • Abstraction depth K and plans per (clip, recipe) k = K=2, k=1
    Design choices controlling corpus size and prompt distribution; not derived from any principle.
axioms (5)
  • domain assumption There exists a dense, consistent, learnable language for music post-production edits
    Stated in the abstract and §1; the entire premise of instruction-pair generation depends on this.
  • domain assumption Hand-authored recipe graphs and chain-order constraints are representative of real studio workflows
    §4.1.1 claims an audio engineer reviewed components, but no systematic validation or listening test is provided.
  • domain assumption MERT/CLAP embeddings capture perceptually meaningful audio similarity/edit direction
    Used for FAD, KAD, and Δsim_a in §5.4; the paper itself notes metrics are imperfect.
  • domain assumption Demucs 6-stem separation yields stems clean enough for edit evaluation
    POEMS relies on Demucs (§3.3); separation errors propagate into ground-truth and agent outputs.
  • domain assumption SFT on RIME data transfers to real post-production
    Implicit in §6.4; only tested on RIME-generated evaluation, not human or external benchmarks.

pith-pipeline@v1.3.0-alltime-deepseek · 17256 in / 11467 out tokens · 108253 ms · 2026-08-01T12:14:48.083583+00:00 · methodology

0 comments
read the original abstract

Almost every piece of recorded music you have ever heard was modified before it reached you; commercial releases rarely spring fully-formed from the mind of a musician. Despite the promise of music generation models for one-shot output, such fine-grained iterative refinement workflows are a complementary problem, and largely out of their reach. There is also a gap for musicians: while they can express what they want to hear, not all have the facility with studio production tools to implement the complex set of actions needed to realize these intuitions. We formalize this task as agentic post-production, wherein individual aspects of a song are targeted, refined, and combined into a final track. We argue the bottleneck is data: existing corpora do not reflect how realistic post-production chains map onto the vocabulary musicians and engineers actually use. We argue there is a language for modifying recorded music that is dense, consistent, and learnable. We introduce the Rule-based Instructions for Music Editing (RIME) framework, which generates realistic paired edit-instruction data from any baseline music dataset grounded in canonical methods, design patterns, and constraints derived from real production workflows. RIME leverages POEMS, a new toolkit that combines stem separation, mixing, and common studio effects for use by multimodal agents. We use RIME and POEMS to generate 3,000 pairs of edit instructions and ground truth audio, and use this data to evaluate existing multimodal LLMs as agents on this task, showing persistent challenges in current models' post-production capabilities. We also demonstrate RIME's ability to improve post-production agent performance via supervised fine-tuning. We see RIME as an early step toward iterative musical agents, collaborative systems that could transform music production much as interactive coding agents have reshaped software engineering.

Figures

Figures reproduced from arXiv: 2607.19605 by Nikhil Singh, Noah Schaffer.

Figure 1
Figure 1. Figure 1: Agentic music post-production with RIME. Starting from a multitrack mix, RIME creates rule-based plans and calls POEMS tools to generate data triples of source audio, output audio, and editing instruction. Downstream multimodal agents then use POEMS to implement an agentic loop in which they realize the provided instruction. Abstract Almost every piece of recorded music you have ever heard was modified bef… view at source ↗
Figure 2
Figure 2. Figure 2: Example edit graph for a vocal harmony recipe [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Audio similarity and operator F1 for edit graphs based on number of operations and [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Graph correctness metrics by abstraction layer for 5 graph components: kinds, stems, [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Pattern constraints in RIME. Patten constraints control how edit recipe candidates are [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Aggregated head-to-head win rates for audio matching [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Model pairwise win rates for audio matching [PITH_FULL_IMAGE:figures/full_fig_p021_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: highlights the CLAP embedding alignment between the prompt text and the audio for the input, agent output, and ground-truth edit. Prompt−input alignment Prompt−output alignment Prompt−target alignment AL 0 AL 1 AL 2 AL 0 AL 1 AL 2 AL 0 AL 1 AL 2 0.135 0.140 0.145 0.150 0.155 0.160 0.14 0.15 0.16 0.13 0.14 0.15 0.16 Abstraction Level Mean With 95% CI Gemini 3 Flash GPT−4o Mini Gemma 3n Gemma 3n SFT [PITH_F… view at source ↗
Figure 9
Figure 9. Figure 9: Graph and audio metrics by class of error and abstraction level for poisoning experiments [PITH_FULL_IMAGE:figures/full_fig_p022_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Audio and graph abstraction layer metrics [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Audio and graph composite scores with confidence intervals by abstraction level [PITH_FULL_IMAGE:figures/full_fig_p023_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Epsilon thresholding accuracy by abstraction level [PITH_FULL_IMAGE:figures/full_fig_p023_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Sweep of tolerance thresholds for graph-level accuracy CIs [PITH_FULL_IMAGE:figures/full_fig_p024_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 7 linked inside Pith

  1. [1]

    The mtg-jamendo dataset for automatic music tagging

    Dmitry Bogdanov, Minz Won, Philip Tovstogan, Alastair Porter, and Xavier Serra. The mtg-jamendo dataset for automatic music tagging. InMachine learning for music discovery workshop, international conference on machine learning (ICML 2019), pages 1–3. Long Beach, CA, United States, 2019

    2019

  2. [2]

    Wavcraft: Audio editing and generation with large language models

    Jinhua Liang, Huan Zhang, Haohe Liu, Yin Cao, Qiuqiang Kong, Xubo Liu, Wenwu Wang, Mark D Plumbley, Huy Phan, and Emmanouil Benetos. Wavcraft: Audio editing and generation with large language models. InICLR 2024 Workshop on Large Language Model (LLM) Agents

    2024

  3. [3]

    Audio prompt adapter: Unleashing music editing abilities for text-to-music with lightweight finetuning

    Fang-Duo Tsai, Shih-Lun Wu, Haven Kim, Bo-Yu Chen, Hao-Chung Cheng, and Yi-Hsuan Yang. Audio prompt adapter: Unleashing music editing abilities for text-to-music with lightweight finetuning. InIsmir 2024 Conference, 2024

    2024

  4. [4]

    Text2fx: Harnessing clap embeddings for text-guided audio effects

    Annie Chu, Patrick O’Reilly, Julia Barnett, and Bryan Pardo. Text2fx: Harnessing clap embeddings for text-guided audio effects. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025

    2025

  5. [5]

    Musicmagus: zero-shot text-to-music editing via diffusion models

    Yixiao Zhang, Yukara Ikemiya, Gus Xia, Naoki Murata, Marco A Martínez-Ramírez, Wei- Hsiang Liao, Yuki Mitsufuji, and Simon Dixon. Musicmagus: zero-shot text-to-music editing via diffusion models. InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 7805–7813, 2024

    2024

  6. [6]

    Zero-shot unsupervised and text-based audio editing using ddpm inversion

    Hila Manor and Tomer Michaeli. Zero-shot unsupervised and text-based audio editing using ddpm inversion. InInternational Conference on Machine Learning, pages 34603–34629. PMLR, 2024

    2024

  7. [7]

    Melodia: Training-free music editing guided by attention probing in diffusion models

    Yi Yang, Haowen Li, Tianxiang Li, Boyu Cao, Xiaohan Zhang, Liqun Chen, and Qi Liu. Melodia: Training-free music editing guided by attention probing in diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 2209–2217, 2026

    2026

  8. [8]

    Declarative audio editing with audio language model

    Zitong Lan, Yiduo Hao, and Mingmin Zhao. Declarative audio editing with audio language model. InThe Fourteenth International Conference on Learning Representations, 2026

    2026

  9. [9]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword- to-caption augmentation

    Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. Large-scale contrastive language-audio pretraining with feature fusion and keyword- to-caption augmentation. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023

    2023

  10. [10]

    Mert: Acoustic music understanding model with large-scale self-supervised training

    LI Yizhi, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghao Xiao, Chenghua Lin, Anton Ragni, Emmanouil Benetos, et al. Mert: Acoustic music understanding model with large-scale self-supervised training. InThe Twelfth International Conference on Learning Representations, 2023

    2023

  11. [11]

    Audit: Audio editing by following instructions with latent diffusion models.Advances in Neural Information Processing Systems, 36:71340–71357, 2023

    Yuancheng Wang, Zeqian Ju, Xu Tan, Lei He, Zhizheng Wu, Jiang Bian, et al. Audit: Audio editing by following instructions with latent diffusion models.Advances in Neural Information Processing Systems, 36:71340–71357, 2023

    2023

  12. [12]

    Audiomorphix: Training-free audio editing with diffusion probabilistic models.arXiv preprint arXiv:2505.16076, 2025

    Jinhua Liang, Yuanzhe Chen, Yi Yuan, Dongya Jia, Xiaobin Zhuang, Zhuo Chen, Yuping Wang, and Yuxuan Wang. Audiomorphix: Training-free audio editing with diffusion probabilistic models.arXiv preprint arXiv:2505.16076, 2025

    Pith/arXiv arXiv 2025

  13. [13]

    Audioeditor: A training-free diffusion-based audio editing framework

    Yuhang Jia, Yang Chen, Jinghua Zhao, Shiwan Zhao, Wenjia Zeng, Yong Chen, and Yong Qin. Audioeditor: A training-free diffusion-based audio editing framework. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025

    2025

  14. [14]

    Recomposer: Event-roll- guided generative audio editing.arXiv preprint arXiv:2509.05256, 2025

    Daniel PW Ellis, Eduardo Fonseca, Ron J Weiss, Kevin Wilson, Scott Wisdom, Hakan Erdogan, John R Hershey, Aren Jansen, R Channing Moore, and Manoj Plakal. Recomposer: Event-roll- guided generative audio editing.arXiv preprint arXiv:2509.05256, 2025. 11

    Pith/arXiv arXiv 2025

  15. [15]

    Instructme: an instruction guided music edit framework with latent diffusion models

    Bing Han, Junyu Dai, Weituo Hao, Xinyan He, Dong Guo, Jitong Chen, Yuxuan Wang, Yanmin Qian, and Xuchen Song. Instructme: an instruction guided music edit framework with latent diffusion models. InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 5835–5843, 2024

    2024

  16. [16]

    Llm2fx-tools: Tool calling for music post-production

    Seungheon Doh, Junghyun Koo, Marco A Martínez-Ramírez, Woosung Choi, Wei-Hsiang Liao, Qiyu Wu, Juhan Nam, and Yuki Mitsufuji. Llm2fx-tools: Tool calling for music post-production. InThe Fourteenth International Conference on Learning Representations, 2026

    2026

  17. [17]

    Medleydb: A multitrack dataset for annotation-intensive mir research

    Rachel M Bittner, Justin Salamon, Mike Tierney, Matthias Mauch, Chris Cannam, and Juan Pablo Bello. Medleydb: A multitrack dataset for annotation-intensive mir research. InIsmir, volume 14, pages 155–160, 2014

    2014

  18. [18]

    Medleydb 2.0: New data and a system for sustainable data collection.ISMIR Late Breaking and Demo Papers, 36, 2016

    Rachel M Bittner, Julia Wilkins, Hanna Yip, and Juan P Bello. Medleydb 2.0: New data and a system for sustainable data collection.ISMIR Late Breaking and Demo Papers, 36, 2016

    2016

  19. [19]

    Distortion audio effects: Learning how to recover the clean signal

    Johannes Imort, Giorgio Fabbro, Marco A Martinez Ramirez, Stefan Uhlich, Yuichiro Koyama, and Yuki Mitsufuji. Distortion audio effects: Learning how to recover the clean signal. InIsmir 2022 Hybrid Conference, 2022

    2022

  20. [20]

    General purpose audio effect removal

    Matthew Rice, Christian J Steinmetz, George Fazekas, and Joshua D Reiss. General purpose audio effect removal. In2023 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), pages 1–5. IEEE, 2023

    2023

  21. [21]

    Sonicmaster: To- wards controllable all-in-one music restoration and mastering.arXiv preprint arXiv:2508.03448, 2025

    Jan Melechovsky, Ambuj Mehrish, Abhinaba Roy, and Dorien Herremans. Sonicmaster: To- wards controllable all-in-one music restoration and mastering.arXiv preprint arXiv:2508.03448, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Crepe: A convolutional representation for pitch estimation

    Jong Wook Kim, Justin Salamon, Peter Li, and Juan Pablo Bello. Crepe: A convolutional representation for pitch estimation. In2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 161–165. IEEE, 2018

    2018

  23. [23]

    S-key: Self-supervised learning of major and minor keys from audio

    Yuexuan Kong, Gabriel Meseguer-Brocal, Vincent Lostanlen, Mathieu Lagrange, and Ro- main Hennequin. S-key: Self-supervised learning of major and minor keys from audio. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025

    2025

  24. [24]

    V oice transformation using psola technique.Speech communication, 11(2-3):175–187, 1992

    Hélene Valbret, Eric Moulines, and Jean-Pierre Tubach. V oice transformation using psola technique.Speech communication, 11(2-3):175–187, 1992

    1992

  25. [25]

    Pedalboard, July 2021

    Peter Sobot. Pedalboard, July 2021. URLhttps://doi.org/10.5281/zenodo.7817838

    doi:10.5281/zenodo.7817838 2021

  26. [26]

    Hybrid transformers for music source separation

    Simon Rouard, Francisco Massa, and Alexandre Défossez. Hybrid transformers for music source separation. InICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2023

    2023

  27. [27]

    Sam audio: Segment anything in audio

    Bowen Shi, Andros Tjandra, John Hoffman, Helin Wang, Yi-Chiao Wu, Luya Gao, Julius Richter, Matt Le, Apoorv Vyas, Sanyuan Chen, et al. Sam audio: Segment anything in audio. arXiv preprint arXiv:2512.18099, 2025

    arXiv 2025

  28. [28]

    Introducing the model context protocol.URL: https://www

    PBC Anthropic. Introducing the model context protocol.URL: https://www. anthropic. com/news/model-contextprotocol, Accessed, 19:2025, 2024

    2025

  29. [29]

    Mu- sic flamingo: Scaling music understanding in audio language models

    Sreyan Ghosh, Arushi Goel, Lasha Koroshinadze, Sang-gil Lee, Zhifeng Kong, Joao Felipe Santos, Ramani Duraiswami, Dinesh Manocha, Wei Ping, Mohammad Shoeybi, et al. Mu- sic flamingo: Scaling music understanding in audio language models. InThe Fourteenth International Conference on Learning Representations, 2026

    2026

  30. [30]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Pith/arXiv arXiv 2023

  31. [31]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024. 12

    Pith/arXiv arXiv 2024

  32. [32]

    Gemma: Open models based on gemini research and technology.arXiv preprint arXiv:2403.08295, 2024

    Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology.arXiv preprint arXiv:2403.08295, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    Lora: Low-rank adaptation of large language models.Iclr, 1(2):3, 2022

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models.Iclr, 1(2):3, 2022

    2022

  34. [34]

    Fr\’echet audio distance: A reference-free metric for evaluating music enhancement algorithms

    Dominik Roblek, Kevin Kilgour, Matt Sharifi, and Mauricio Zuluaga. Fr\’echet audio distance: A reference-free metric for evaluating music enhancement algorithms. InProc. Interspeech, pages 2350–2354, 2019

    2019

  35. [35]

    Kad: No more fad! an effective and efficient evaluation metric for audio generation.arXiv preprint arXiv:2502.15602, 2025

    Yoonjin Chung, Pilsun Eu, Junwon Lee, Keunwoo Choi, Juhan Nam, and Ben Sangbae Chon. Kad: No more fad! an effective and efficient evaluation metric for audio generation.arXiv preprint arXiv:2502.15602, 2025. 13 A Compute Resources We run all experiments on a compute cluster with L40S GPUs. Zero-shot models are called via API. B POEMS Toolkit Details B.1 P...

    Pith/arXiv arXiv 2025

  36. [36]

    Working Audio State - The agent keeps track of the working audio, the active isolated source, the path of the stem, path of the residual, the latest version of the processed stem

  37. [37]

    Planning/Assignment State - The tools selected during the planning step are stored of in a list of ’assignments.’ Each assignment tracks status (pending, in progress, retrying, done, failed) as well as the next pending assignment

  38. [38]

    Execution State - Tracks all aspects of a tool-calling step including current assignment, active source, if a branch (see Appendix D.2) is open, if there are any remaining assignments, and if the audio is ready to return 19

  39. [39]

    Branches

    Failure/Repair State - Tracks the failed tool name, failed tool assignment, and failed argu- ments to the tool call. D.2 Enforcing Agent Behavior We place a few mechanisms to steer the behavior of the agent • Separation and Mixing “Branches" - When a stem is isolated, a branch is opened and kept track of in the execution state and closed when the source i...

  40. [43]

    Edges 1. Kinds2. Stems

  41. [44]

    Parameters

    Operators4. Parameters

  42. [45]

    Edges 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 0.00 0.25 0.50 0.75 1.00 Graph correctness rung Mean with 95% CI Gemini 3 Flash GPT−4o Mini Gemma 3n Gemma 3n SFT Figure 13: Sweep of tolerance thresho...