{"id":"5109d3ac-2b47-4799-928c-7fec17861f4a","arxiv_id":"2505.03244","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"SonicRAG uses an LLM to convert text, voice, or onomatopoeia into a script that retrieves and mixes existing audio assets into a new, high-fidelity sound effect.","lead":"SonicRAG is a system where a large language model retrieves recordings from a sound library and writes a compact script to mix them into a requested sound effect. It aims to deliver studio-quality audio because it reuses real recordings instead of generating audio from scratch.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3 cannot support the superiority claim: SonicRAG is tested only on a small coin library whose filenames nearly match the prompts, so CLAP and SNR gains reflect trivial retrieval and asset reuse, not synthesis capability.","rationale":"The paper is an architectural proposal with a plausible but unevaluated mechanism. The strongest claim rests on Table 3, but every metric in that table is confounded. First, CLAP compares prompt and output; when output is a recording whose filename is semantically identical to the prompt, CLAP is inflated. Second, SNR of reused recordings is definitionally high and is not comparable to neural synthesis. Third, the FAD ranking does not favor SonicRAG, undercutting 'superior performance.' The reader's conditional verdict already requires release of code and controlled evaluation; my stress-test agrees and identifies the retrieval-only control as the decisive missing experiment. The framework may still be useful as an interactive asset-composition tool, but the paper as written does not demonstrate the claimed superiority over existing methods. I therefore do not alter the reader's verdict.","tokens_in":7146,"tokens_out":4080,"duration_ms":43064,"concrete_test":"Re-run the Section 4.4 experiment with two additions: (i) a retrieval-only control that returns the top-1 asset by embedding and outputs it unchanged, with no LLM or Mixer Script; (ii) a held-out prompt set whose events are absent from the asset library. Compare CLAP, FAD, and SNR, and report retrieval precision for held-out prompts. If SonicRAG's CLAP/SNR margins over the retrieval-only control vanish, or if held-out retrieval success is low, the claimed superiority in requirement accuracy and flexibility is an artifact of near-duplicate prompts and cannot support the central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is that Table 3's superior 'requirement accuracy' reflects SonicRAG's synthesis rather than trivial retrieval. In Section 4.1, the authors record a small coin library and name files by event; in Section 4.4, all prompts are coin-related. The vector database therefore contains the answer before synthesis: the CLAP score (0.67) measures near-identity between prompt and retrieved asset, not the ability to compose novel acoustic scenes. SNR 88.2 dB similarly follows from copying clean 48kHz/24bit recordings; neural baselines that synthesize from latent code cannot reach that value by construction, so the comparison is apples-to-oranges. FAD 25.5 is actually worse than StableAudio (21.4) and AudioLDM (22.3), so Table 3 does not establish overall superiority. Section 5.2 Limitation 1 concedes the framework needs a matching text-annotated library, but the experiments never test the failure case, nor do they include a retrieval-only baseline that selects an asset by embedding and applies no Mixer-Script processing. Without that control, the unique contribution of LLM-driven recombination is untested, and the central claim is unsupported by the reported quantitative results.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SonicRAG, a retrieval-augmented generation framework for sound effects synthesis. A large language model (GPT-4o-audio-preview) interprets user prompts, retrieves audio assets from a text-annotated sound library, and composes a Mixer Script — a small DSL defined in EBNF — that applies volume, filtering, compression, reverb, and timing operations before mixing the assets. The authors argue that this avoids the fidelity loss and controllability problems of latent diffusion models, supports precise text-based editing and onomatopoeia-to-sound conversion, and achieves 'superior performance' in requirement accuracy and synthesis flexibility. The experimental section reports a case study of a coin-sound design workflow and a comparison (Table 3) against AudioGen, AudioLDM, StableAudio, WavCraft, and WavJourney on FAD, CLAP, and SNR, using a small self-recorded coin library.","tokens_in":7405,"tokens_out":2718,"duration_ms":26642,"significance":"If the central claim were established, SonicRAG would be a useful contribution to interactive sound design: it provides a concrete, machine-readable scripting abstraction (Mixer Script) that lets an LLM reuse existing high-quality audio assets without retraining a generative model. The EBNF syntax, the table of supported processing methods, and the interactive workflow demonstration are concrete and reproducible in principle. The paper also honestly lists limitations (Section 5.2), including the need for an existing text-annotated sound library. However, the quantitative evidence in Table 3 is currently too weak to support the claimed superiority: the evaluation lacks a protocol, a retrieval-only control, and out-of-domain tests, and the reported SNR and CLAP values are plausibly artifacts of retrieving near-identical assets from a tiny, self-named library. Thus the significance is real only if the evaluation is substantially strengthened.","major_comments":[{"comment":"Table 3 cannot support the claim of superior performance. The paper does not state the number of prompts, the number of generated samples, the metric computation details, or any error bars or significance tests, so the reported averages are not interpretable. More importantly, the comparison is confounded: SonicRAG retrieves and mixes clean pre-recorded 48 kHz/24-bit assets, while the neural baselines synthesize audio from latent representations. The SNR of 88.2 dB and CLAP of 0.67 are therefore expected consequences of reusing near-identical assets whose filenames essentially match the prompts, not of synthesis ability. The FAD of 25.5 is in fact worse than StableAudio's 21.4 and AudioLDM's 22.3, undercutting the overall-superiority narrative. At minimum, the authors must provide a retrieval-only baseline (embedding-based asset selection with no Mixer Script recombination), a per-prompt breakdown, and an evaluation protocol with confidence intervals.","section":"§4.4 / Table 3"},{"comment":"The experimental corpus is a small, self-recorded coin library whose filenames are event descriptions such as 'coin collide wood'. All prompts in Section 4.4 are coin-related, so the vector database contains the answer before synthesis. This setup tests retrieval recall, not the ability to compose novel acoustic scenes. The paper should test SonicRAG on multiple sound domains with prompts that are not lexical matches to asset filenames, and should include failure cases where the library does not contain the requested event. Section 5.2 explicitly concedes that the framework needs a matching text-annotated library, but the experiments never probe this limitation, so the claimed flexibility remains unverified.","section":"§4.1 and §4.4"},{"comment":"The case study demonstrates interactive refinement and onomatopoeia-to-sound conversion, but it is purely qualitative. There is no listening test, no user study with sound designers, and no audio or code release to verify that the shown scripts produce the claimed acoustic outcomes. Since the paper's central contribution is a workflow for creative professionals, a small perceptual evaluation (e.g., preference or similarity ratings against baselines) is necessary to substantiate the 'requirement accuracy' and 'synthesis flexibility' claims.","section":"§4.2 / Figure 5"}],"minor_comments":[{"comment":"There are numerous typos and grammar issues, including 'Releated Work' (Section 2), 'temproal' (Abstract), 'evulates' (Section 4.4), 'whril' (Figure 5), and 'Presice Editing' (Figure 5). These should be corrected in a thorough language pass.","section":"Throughout"},{"comment":"The checkmark/cross notation is inconsistent. For example, the WavCraft row contains a single checkmark, which is ambiguous: it is unclear which capability is being attributed. Each row should be fully filled or the table should use explicit 'Yes/No' entries.","section":"Table 2"},{"comment":"Equation (1) calls the script generation process a Markov decision process, but no actions, states, rewards, or transition probabilities are defined. This framing is misleading; a simpler conditional-generation description would be more accurate.","section":"§3.4, Eq. (1)"},{"comment":"The sentence 'Lewis et al. [6] shows that parametric and non-parametric memory doing postive interaction on knowledge-intensive tasks' contains a grammatical error and a typo ('postive'). The intended point is that retrieval improves LLM performance, but the writing obscures it.","section":"§2.4"},{"comment":"The paper states that ARCA23K is used as the ground truth set for FAD, but it does not explain how the coin-sound test prompts relate to this dataset. FAD compares distributions between generated audio and a reference set; the mismatch between the coin-specific test domain and the general ARCA23K reference set should be clarified.","section":"§4.4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is an early-stage systems paper with a plausible idea but an evaluation that does not yet support the headline claims. The lack of code, audio samples, and detailed evaluation protocol makes independent verification impossible. If the authors can provide a retrieval-only baseline, a proper evaluation protocol, and out-of-domain tests, the contribution could be salvageable; otherwise the paper is closer to a demo/workshop write-up than a journal article."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is actually useful: instead of synthesizing sound effects with latent diffusion, let an LLM retrieve and recombine pre-recorded assets through a small, readable DSL. WavCraft and WavJourney orchestrated neural generators; SonicRAG reuses existing studio-quality recordings, which is how sound designers really work. The Mixer Script EBNF is a clear idea, and the case study shows a genuinely interactive workflow, including onomatopoeia-to-sound and iterative refinement. The paper also identifies a real problem: neural models downsample and lose the transient detail that matters in SFX. I buy that a retrieval-plus-DSL approach can preserve bandwidth and give precise control. That is a legitimate system contribution, and the Section 5.2 limitation statement (requiring a text-annotated library) is honest.\n\nBut the evidence is not where it should be. Table 3 is the only quantitative comparison, and it does not establish superiority. The test set is a small, self-recorded coin library whose filenames almost literally match the prompts, so the CLAP score of 0.67 probably measures trivial retrieval rather than synthesis quality. The SNR of 88.2 dB is an artifact of copying clean 48kHz/24bit files; neural baselines that synthesize from latents cannot reach that number by construction. FAD 25.5 is actually worse than StableAudio (21.4) and AudioLDM (22.3), so the table undercuts the 'superior performance' claim. There is no retrieval-only baseline that simply picks the best CLAP asset without Mixer processing, so you cannot tell whether the RAG context or the DSL contribute anything at all. No code or audio samples are released, and the experimental protocol lacks prompt counts, error bars, or a user study. The equations in Section 3.4 are descriptive and not really tested.\n\nThese are fixable. A better evaluation would use a public dataset, a retrieval-only control, and a proper listening test. The idea deserves that treatment. I would not reject the paper for the concept, but I would not accept it in its current form. For a serious venue, a strong referee could force the authors to produce that evidence. For a workshop, the system demonstration alone might be worth presenting.","headline":"SonicRAG is a promising retrieval-based sound-design pipeline, but Table 3 does not support the paper's superiority claim and the evaluation needs a proper baseline and released artifacts.","tokens_in":732,"tokens_out":717,"would_cite":false,"duration_ms":30119,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"High-fidelity sound effects by having an LLM remix your audio library","keywords":["sound effects synthesis","retrieval-augmented generation","large language models","Mixer Script","text-to-audio","audio editing","onomatopoeia to sound","sound design"],"falsifier":"Take a sound library that does not contain a requested event (say, only coin recordings but a prompt asking for thunder). If SonicRAG still produces a plausible thunder effect, the paper's stated limitation is wrong; if it fails or registers the gap, the retrieval-dependence claim holds. A broader quantitative check is to evaluate the same framework on an open multi-event sound-effect library with held-out event categories and compare FAD and CLAP per category to the coin-only results.","tokens_in":6956,"feed_emoji":"🔊","tokens_out":5746,"duration_ms":51153,"temperature":0.7,"pith_summary":"The paper tries to show that sound effects synthesis does not need to generate audio from scratch: a large language model can retrieve relevant clips from the user's own annotated sound library, arrange them in time, and apply mixing operations to match a natural-language description. The claimed payoff is high-fidelity output at the original recording's sampling rate, with the ability to revise the sound by text or even onomatopoeia in an interactive loop, without retraining any generative model. The paper's experiments on coin-related sounds report better semantic alignment and signal-to-noise ratio than current generation-based approaches, at the cost of relying on the library already containing the requested events.","feed_headline":"High-fidelity sound effects by having an LLM remix your audio library","feed_subtitle":"A retrieval-augmented script mixes recorded clips to match natural-language prompts, without retraining a generative model.","key_machinery":"The core object is Mixer Script, a deliberately small scripting language defined in EBNF that wraps audio signal processing methods behind method chains such as .Volume(-14).StartAt(0.3). It works alongside a unified sound object that packages each library clip with a textual description plus measured metadata (loudness in LUFS, voice onset time, pitch, duration), letting a text-trained LLM reason about audio characteristics it cannot hear directly. The second mechanism is context-aware retrieval: the LLM autonomously decides whether the current dialogue needs new sound assets, so re-synthesis or fine-tuning does not pollute the prompt with irrelevant context. Together these carry the argument by shifting the difficult part of generation from neural audio synthesis to asset selection and scripted recombination.","core_discovery":"On its own terms, the paper establishes a pipeline in which user prompts are converted by a multimodal large language model into (1) retrieval decisions over a vector-indexed sound library and (2) a Mixer Script, an EBNF-defined scripting language whose method chains (volume, filtering, compression, reverb, start and stop times) describe how each retrieved asset is processed and placed in time. The generated script is executed to mix the actual recordings, so the final audio preserves the native sampling rate and recording quality of the assets. Evaluated on coin-sound synthesis with reference metrics for audio distance, semantic alignment, and signal-to-noise ratio, the paper claims superior requirement accuracy and synthesis flexibility over prior generative and LLM-orchestrated audio systems, and demonstrates iterative text- and onomatopoeia-driven editing in a case study. The framework models script generation as a Markov decision process in which the retrieval context is updated only when the LLM judges that new assets are needed.","pith_inferences":["The claimed SNR advantage may be partly a consequence of copying rather than synthesizing audio, so a fairer generality test would use an independent sound library covering many event types and then measure whether semantic alignment stays high outside the coin domain.","If retrieval coverage is the bottleneck, a natural extension is to let the framework fall back to a generative model when no matching asset exists, turning SonicRAG into a hybrid that retrieves what it can and synthesizes the rest.","The vector index in the paper stores filenames rather than audio embeddings; indexing clips by their acoustic metadata or by learned audio-text embeddings could improve retrieval precision for abstract prompts without changing the Mixer Script design."],"forward_implications":["Sound designers can iterate on a sound effect conversationally: asking for a harsher or softer texture changes specific filter and compressor parameters in the Mixer Script rather than regenerating audio blindly.","Because the output is a mix of original recordings, the synthesized effect inherits the library's native sampling rate and microphone quality, sidestepping the aliasing and detail loss the paper attributes to downsampled latent-diffusion generation.","Expanding the library to new events does not require retraining the system; adding a text-indexed clip is sufficient for the LLM to start using it.","The approach enables speech-to-sound and onomatopoeia-to-sound workflows, since a multimodal LLM can map an audio description like 'da-da-da-pa-ra-ra-ra' into concrete start times and processing chains."],"supporting_citations":[{"why":"Establishes retrieval-augmented generation, the paradigm the framework builds on.","marker":"[6]"},{"why":"Demonstrates that few-shot prompting can guide large models, supporting Mixer Script generation without fine-tuning.","marker":"[15]"},{"why":"Defines Fréchet Audio Distance, one of the evaluation metrics used to back the quality claims.","marker":"[21]"},{"why":"Defines the CLAP score used to measure semantic alignment between prompts and synthesized audio.","marker":"[23]"},{"why":"A prior LLM-orchestrated audio editing system that SonicRAG compares against and extends.","marker":"[19]"},{"why":"A prior compositional audio creation system used as a comparison baseline in the evaluation.","marker":"[20]"},{"why":"A latent-diffusion text-to-audio baseline that must be beaten in the quality comparison.","marker":"[9]"},{"why":"A generative audio baseline used to show SonicRAG's advantage over model-based synthesis.","marker":"[7]"},{"why":"A recent generative-audio system included as a baseline in the evaluation table.","marker":"[10]"},{"why":"Provides the ground-truth audio set used for the Fréchet Audio Distance computation.","marker":"[22]"}],"fun_headline_variants":["LLM remix: high-fidelity SFX from your audio library","Retrieval-augmented SFX: LLM scripts your sound library","No retraining: LLM mixes existing clips for prompt-driven SFX","Sound effects via LLM-scripted remix of your own recordings"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole framework depends on the user already having a text-annotated sound-effect library that contains the requested events, and the paper's tests use only a small, self-recorded set of coin sounds.","fun_headline_variants_meta":{"raw":{"variants":["LLM remix: high-fidelity SFX from your audio library","Retrieval-augmented SFX: LLM scripts your sound library","No retraining: LLM mixes existing clips for prompt-driven SFX","Sound effects via LLM-scripted remix of your own recordings"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1211,"prompt_tokens":895,"completion_tokens":316,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":511,"completion_tokens_details":{"reasoning_tokens":236}},"tokens_in":511,"tokens_out":316,"duration_ms":3399,"temperature":1.0,"reasoning_tokens":236,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:55:40.615484+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a sound library that does not contain a requested event (say, only coin recordings but a prompt asking for thunder). If SonicRAG still produces a plausible thunder effect, the paper's stated limitation is wrong; if it fails or registers the gap, the retrieval-dependence claim holds. A broader quantitative check is to evaluate the same framework on an open multi-event sound-effect library with held-out event categories and compare FAD and CLAP per category to the coin-only results.","supporting_citations":[{"cited_title":"Plumbley, Huy Phan, and Emmanouil Benetos","cited_arxiv_id":null,"evidence_quote":"A prior LLM-orchestrated audio editing system that SonicRAG compares against and extends."},{"cited_title":"Mandic, Wenwu Wang, and Mark D","cited_arxiv_id":null,"evidence_quote":"A latent-diffusion text-to-audio baseline that must be beaten in the quality comparison."},{"cited_title":"Simple and controllable music generation","cited_arxiv_id":null,"evidence_quote":"A generative audio baseline used to show SonicRAG's advantage over model-based synthesis."},{"cited_title":"Iqbal, Y","cited_arxiv_id":null,"evidence_quote":"Provides the ground-truth audio set used for the Fréchet Audio Distance computation."}],"review_version":1}