{"id":"3484ef4d-83a9-4dbd-ae79-1c581e9a1871","arxiv_id":"2608.09331","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"RAG-Audio starts frozen audio generators from a retrieved exemplar of the fMRI-decoded CLAP embedding, raising 10-way stimulus identification from 0.14-0.18 to 0.40-0.43 on Brain2Music and cutting FAD by about 10x.","lead":"A brain-to-audio system that starts its generator from a retrieved real-audio exemplar instead of pure noise lifts stimulus identification from near chance to retrieval-level accuracy on the Brain2Music dataset. For scientists building neural decoders, this offers a simple fix for prior domination, when a strong pretrained generator ignores weak brain-derived conditions.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline identification metric (Eq. 4) may be carried by genre-level CLAP similarity; the paper's own genre-confusion data (top-1 genre 0.33) do not rule this out. A same-genre distractor evaluation is needed before claiming faithful reconstruction.","rationale":"The paper makes an honest and reasonably careful case: it reports novelty values, includes a memory-bank ablation with a negative result, and explicitly disclaims waveform-level fidelity in its Limitations. I do not challenge the internal consistency of the experiments, and the mechanism control (MusicGen) is a sensible first check even though it is cross-model. The load-bearing issue is the evaluation metric. The reader identified CLAP-space validity as the weakest assumption; I agree, and I sharpen it: the paper's own genre-confusion results make it plausible that Eq. (4) is substantially a genre-discrimination test. Under the pooled 10-way protocol with random distractors, genre-level information alone can produce above-chance identification because same-genre distractors are rare; a perfect genre classifier would score roughly 0.65 in this setting, substantially above the reported 0.43. The absence of a same-genre control means the reader cannot tell whether RAG-Audio reconstructs the heard stimulus or merely its genre/timbre class. This is a correctness risk in the measurement, not a disagreement with the brain-to-audio consensus. The proposed within-genre identification test is cheap, uses existing data, and would either support or dissolve the faithfulness claim. I therefore keep the reader's CONDITIONAL verdict: the central claim is plausible but hinges on this additional evaluation.","tokens_in":15308,"tokens_out":9528,"duration_ms":86500,"concrete_test":"Recompute Eq. (4) with the nine distractors restricted to the same GTZAN genre as the true test clip, using the split in Section 5 and the released evaluation harness. If RAG-Audio accuracy falls to the 0.10 chance level, the reported identification gain is attributable to genre-level CLAP similarity rather than stimulus-specific reconstruction; if accuracy remains substantially above chance (ideally comparable to the reported 0.43), the metric-validity concern is mitigated. No new data are needed; this is a re-running of the existing candidate set with genre-matched distractors.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends entirely on the 10-way identification metric in Eq. (4), computed as CLAP cosine similarity between the generated clip and the true test stimulus versus nine distractors. Every stage of the pipeline lives in the same cosine geometry: the decoder is trained with InfoNCE on CLAP embeddings (Eq. 6), the memory bank is composed of CLAP embeddings (Eq. 7), the retrieval baseline is defined by CLAP cosine, and the evaluation is again CLAP cosine. RAG-Audio initializes from the retrieved exemplar, so its output inherits the exemplar's CLAP neighborhood. The paper's own Limitations states that Eq. (4) measures semantic agreement in CLAP space rather than sample-accurate waveform recovery, but the risk is more specific than that. Appendix G reports that RAG-Audio reaches only 0.33 top-1 genre accuracy (3.3x chance) on the same CLAP geometry; if the distractors in Eq. (4) are drawn randomly across the ten GTZAN genres, a system that recovers only genre-level information can already score far above the 0.10 chance level by eliminating cross-genre distractors. The paper does not report identification with same-genre distractors, nor any acoustic-level measure (e.g., spectrogram correlation, onset alignment) that separates stimulus-level recovery from genre-level recovery. Without such a check, the headline gain from 0.14 to 0.43 could be driven by coarse semantic class rather than by faithful reconstruction of the specific heard clip.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces RAG-Audio, a retrieval-augmented pipeline for fMRI-to-audio reconstruction. A contrastive decoder maps fMRI to a CLAP embedding; the nearest training exemplar is retrieved in CLAP space; and a frozen diffusion or flow generator is initialized at an intermediate time from that exemplar while conditioned on the decoded embedding. On the Brain2Music dataset, the authors report that 10-way stimulus identification rises from 0.14–0.18 for direct generation to 0.40–0.43 for RAG-Audio, matching the retrieval baseline of about 0.40, and that FAD drops by roughly an order of magnitude for AudioLDM and TangoFlux. An autoregressive MusicGen control shows only a small gain, which the authors attribute to the absence of an initializable latent trajectory. The paper concludes that retrieval-guided trajectory initialization mitigates prior domination.","tokens_in":15611,"tokens_out":3780,"duration_ms":34185,"significance":"If the central claim holds, the paper makes a useful contribution: it provides a quantitative, generator-general characterization of a failure mode that has largely been qualitative in brain-to-image work, and it proposes a simple, well-motivated intervention that trades faithfulness against novelty in a controllable way. The single-harness comparison across three generator families, the per-subject decoding analysis, the auditory-cortex localization check, and the honest memory-bank negative result are strengths. The main risk is that the evaluation metric and the retrieval mechanism share the same CLAP cosine geometry, so the headline identification gains may partly reflect inherited genre-level or retrieval-level similarity rather than stimulus-level reconstruction. This concern is acknowledged in the Limitations but is not resolved by the current experiments.","major_comments":[{"comment":"The identification metric is defined as CLAP cosine similarity, and every stage of the pipeline uses the same geometry: the decoder is trained with InfoNCE on CLAP embeddings (Eq. 6), retrieval selects the nearest CLAP embedding (Eq. 7), and the evaluation is CLAP cosine. Because RAG-Audio initializes from the retrieved exemplar, its output inherits the exemplar's CLAP neighborhood. The paper's own Appendix G reports top-1 genre accuracy of only 0.33 (3.3× chance) for the RAG arm, so a system that recovers genre-level information could already score well above chance on a 10-way identification task if distractors are drawn across genres. The authors state in Limitations that Eq. (4) measures semantic agreement in CLAP space rather than sample-accurate waveform recovery, but this does not address the more specific risk that the reported 0.40–0.43 identification is carried by coarse genre/timbre similarity. I ask the authors to report identification with same-genre distractors (e.g., distractor sets matched on predicted or true genre) and to add at least one acoustic-level measure (spectrogram correlation, onset alignment, or similar) that does not depend on CLAP cosine, in order to substantiate the claim of faithful reconstruction rather than genre-level matching.","section":"§3, Eq. (4); Appendix G"},{"comment":"The anchoring strength s is a free parameter, and the reported headline numbers use the working range s≈0.25–0.40, with the best AudioLDM result at s=0.2 (0.49 identification) and TangoFlux at s=0.4. It is not stated whether s was selected on the held-out test set or on a separate validation split. If s was chosen on the same 300 test generations used to report accuracy, the reported values are optimistic and the comparison to retrieval (which has no such tuned parameter) is not on equal footing. In addition, no confidence intervals or significance tests are provided; with n=300 generations, the difference between 0.14 and 0.43 is large, but the difference between 0.40 (RAG TangoFlux) and 0.43 (AudioLDM RAG) and the difference between RAG and retrieval (0.40 vs 0.40–0.43) need uncertainty quantification to support the claim that RAG 'matches' retrieval rather than merely being statistically indistinguishable under high variance. Please clarify the selection procedure and report per-subject or bootstrap intervals for the central comparisons.","section":"§5, Implementation; §6, Table 7"},{"comment":"The conclusion that trajectory initialization, rather than retrieval alone, is the operative mechanism rests entirely on the MusicGen negative control. MusicGen is autoregressive, so the retrieved exemplar can only enter through melody conditioning, which is a different conditioning interface from the latent-space initialization used for AudioLDM and TangoFlux. The comparison therefore varies two things at once: the generator family and the way the exemplar is injected. A cleaner control would use a latent-diffusion or flow generator and ablate the initialization: condition on the retrieved audio's embedding (or concatenate it to the decoded embedding) without starting the trajectory from the exemplar. Without such an ablation, the claim that the improvement is not attributable to the mere availability of the retrieved exemplar is under-supported, although the current evidence is suggestive.","section":"§4.3, §6, Mechanism Control"}],"minor_comments":[{"comment":"The FAD value for direct AudioLDM is reported as 13.49 in the abstract and Table 5 but 13.5 in Table 1; please make these consistent.","section":"Abstract, Table 1, Table 5"},{"comment":"The reproducibility section states that the full codebase is released in a public repository, but the repository field is 'TBA'; please provide the actual URL or state the reason for the delay.","section":"Appendix A"},{"comment":"There are two objects numbered 'Table 8' with different content: one for memory-bank size ablation and one for the SDEdit strength sweep. Please renumber the tables consistently and ensure the in-text references point to the correct table.","section":"Appendix F and Appendix H"},{"comment":"The novelty definition contains a typo: '1−costo the nearest training clip' should read '1 − cosine similarity to the nearest training clip'.","section":"Table 6"},{"comment":"The second and third paragraphs of Section 2 contain substantially overlapping text about brain-to-image reconstruction and prior domination; please consolidate to avoid redundancy.","section":"§2, Related Work"},{"comment":"The caption says 'cyan lines are stimulus onsets (shared per row)', but it is not clear from the figure how the shared onsets are computed or why they are shared; please clarify in the caption or the figure itself.","section":"Figure 7"}],"recommendation":"major_revision","confidential_remarks":"The central result is plausible and the engineering is careful, but the metric-validity issue is serious enough to warrant major revision. The most important addition is a same-genre distractor evaluation for Eq. (4) and at least one acoustic-level fidelity measure, plus clarification of how the anchoring strength was selected. If those are provided and the conclusions are reworded to match the semantic-similarity interpretation, the paper could become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is worth a serious look, and I'd send it to reviewers. It does something useful—quantify a failure mode that people in brain-to-audio have mostly discussed qualitatively—and it offers a simple, cheap fix that works within its own evaluation. The main comparison is clean: the decoded CLAP embedding identifies the heard clip at 0.43 (10-way, chance 0.10), but direct generation from that same embedding collapses to 0.14–0.18 across three different generator families. That is a real result, and the single-harness reimplementation makes the numbers comparable. The fix—retrieving a training exemplar and initializing the generator's latent trajectory from it (SDEdit for diffusion, rectified-flow interpolation for TangoFlux)—recovers identification to roughly the retrieval level (0.40–0.43) while keeping the output novel. The MusicGen negative control is a nice touch: same exemplar, but no initializable trajectory, and the gain mostly disappears. The paper also reports an anatomical check (78% of top voxels in auditory cortex) and is honest that it does not beat retrieval on FAD.\n\nThe soft spots are real but addressable. The biggest one is the evaluation. Every stage lives in CLAP cosine geometry: decoder training, retrieval, and the identification metric all use the same CLAP embedding. The paper's own genre-confusion appendix reports only 0.33 top-1 genre accuracy for RAG-Audio, which is far from perfect but means genre-level info is present. In a 10-way test with distractors drawn across genres, a system that only recovers genre can score well above chance, so the headline 0.43 may be partly carried by coarse genre discrimination rather than clip-level reconstruction. The paper should report identification with same-genre distractors and at least one acoustic-level measure (spectrogram correlation or onset alignment). Without that, the 'faithful reconstruction' framing is stronger than the evidence. Also, there are no error bars or significance tests, the anchoring strength s is selected on test data, and the code is still TBA. These are fixable in revision.\n\nOverall: the central effect is likely real—the pattern is large and consistent across generators—but the metric bounds the claim. This paper deserves a proper referee; the authors just need to tighten the evaluation before the faithfulness claim can stand.","headline":"Quantifies a real failure mode with a plausible fix, but the CLAP-based identification metric leaves the 'faithful' claim under-supported until same-genre and acoustic checks are added.","tokens_in":16147,"tokens_out":4093,"would_cite":true,"duration_ms":36585,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Brain-to-audio reconstruction fails when a frozen generator's prior overwhelms the weak fMRI-derived condition, and starting that generator from a retrieved real-audio exemplar restores stimulus identification to retrieval level while…","keywords":["brain-to-audio reconstruction","prior domination","retrieval-augmented generation","exemplar anchoring","latent diffusion","rectified flow","fMRI decoding","CLAP semantic audio embedding"],"falsifier":"Run the anchored reconstructions through a forced-choice identification by human listeners, or by an audio model whose training data are disjoint from the stimulus genres, and compare with the CLAP 10-way scores: if human identification of the heard clip does not rise with the CLAP-based gain, the central claim of restored faithful reconstruction fails. Alternatively, anchor from a randomly chosen exemplar at the same strength; if identification stays near retrieval level, the nearest-neighbour content is not carrying the effect.","tokens_in":15053,"feed_emoji":"🎧","tokens_out":13950,"duration_ms":102750,"temperature":0.7,"pith_summary":"Brain-to-audio reconstruction usually works in two steps: a decoder turns fMRI into a semantic audio embedding, and a frozen pretrained generator turns that embedding into sound. This paper identifies a failure mode it calls prior domination: the generator's learned distribution over plausible audio swamps the weak, noisy brain-derived condition, so the output sounds realistic but the heard clip is almost unrecognizable, around 14–18% 10-way identification against a 10% chance level. The proposed fix, RAG-Audio, retrieves the real audio clip whose embedding is nearest to the decoded one, partially perturbs that exemplar's latent to an intermediate point in the generator's sampling trajectory, and then lets the same frozen generator finish the sample conditioned on the decoded embedding. Across five subjects this restores identification to 40–43%, matching a pure-retrieval baseline, while cutting Fréchet Audio Distance by roughly an order of magnitude for the diffusion and flow generators and keeping the output newly generated rather than a replay. The point matters because it turns a qualitative failure seen in brain-to-image decoding into a quantitative, generator-general account, and offers a tunable faithfulness-versus-novelty knob for any generator that exposes a continuous latent trajectory.","feed_headline":"Retrieval anchors lift brain-to-audio identification from 14% to 43%","feed_subtitle":"Starting frozen audio generators from a retrieved real clip restores stimulus-level accuracy that direct decoding loses.","key_machinery":"The central object is exemplar anchoring: a procedure that starts a frozen latent generator not from noise but from the latent code of a retrieved real-audio clip, partially corrupted to an intermediate time. Concretely, for a latent-diffusion generator the retrieved exemplar's latent $z_0$ is noised according to the forward process up to step $t_0 = \\lfloor sT \\rfloor$ (the SDEdit recipe), and for a rectified-flow generator the latent is interpolated to $\\tau_0 = s$ along the flow trajectory; in both cases the reverse or flow-integration process then runs from that point conditioned on the brain-decoded CLAP embedding $\\hat{z}$. The anchoring strength $s$ is the knob that trades faithfulness against novelty: small $s$ keeps the exemplar nearly intact and approaches retrieval, while $s \\to 1$ erases the exemplar and decays to direct, prior-dominated generation. This continuous-trajectory mechanism is what distinguishes the latent generators, where the intervention works, from the autoregressive MusicGen control, where the exemplar can only enter as melody conditioning and the token prior still dominates.","core_discovery":"On the paper's own terms, the central discovery is that stimulus information survives fMRI decoding and is then thrown away by the generator: the contrastive fMRI-to-CLAP decoder reaches 0.43 in 10-way identification, but feeding the same embedding to frozen AudioLDM, TangoFlux, or MusicGen drops identification to 0.14–0.18, near the 0.10 chance level. RAG-Audio counters this prior domination by retrieving the nearest real training exemplar to the decoded embedding, encoding it into the generator's latent space, adding noise or flow interpolation up to an intermediate time $t_0 = sT$ (equivalently $\\tau_0 = s$ for the flow generator), and running the frozen reverse process from there with the decoded embedding still as conditioning. With anchoring strength $s \\approx 0.25$ to $0.40$, 10-way identification rises to 0.43 for AudioLDM and 0.40 for TangoFlux, matching the 0.40 retrieval baseline; FAD falls from 13.50 to 1.25 for AudioLDM and from 7.89 to 2.36 for TangoFlux. The MusicGen control, which cannot be initialized at an intermediate latent time, gains almost nothing (0.18 to 0.20), which the authors take to show that trajectory initialization, not the presence of a retrieved clip, is the operative mechanism.","pith_inferences":["An implied test of the paper's framing is whether human listeners, who do not share CLAP's embedding geometry, also identify the heard clip at the higher rate; if they do not, the measured gain is a property of the evaluation space rather than of reconstructed sound.","The same trajectory-anchoring logic should apply to brain-to-image latent diffusion decoders, where realistic-but-drifting reconstructions have been reported qualitatively; a 10-way identification metric in an image embedding space would turn that qualitative pattern into the same quantitative prior-domination curve.","The memory-bank ablation reported in the paper is an honest negative result: anchoring's benefit does not widen when the bank shrinks, which suggests the gain tracks retrieval quality rather than scarcity. A stronger test of generalization would hold out entire genres and check whether moderate $s$ still reconstructs them from less similar exemplars.","Because CLAP is a joint audio-language embedding, one could add a textual description of the decoded embedding as a second condition and use exemplar anchoring to reconcile a weak semantic condition with a strong prior; this would test the mechanism outside brain decoding, in ordinary text-to-audio generation."],"forward_implications":["At $s \\approx 0.25$ to $0.40$, RAG-Audio matches nearest-neighbour retrieval on 10-way identification while emitting a newly generated sample, with novelty about 0.18 versus about 0.06 for verbatim retrieval; the output is an edit of the exemplar, not a copy.","Prior domination is shown to be a property of the decode-then-generate recipe rather than of one generator: latent diffusion, rectified flow, and autoregressive generators all drop decoded identification from 0.43 to 0.14–0.18 in direct conditioning.","The mechanism only transfers to generators that expose an initializable continuous latent trajectory; autoregressive models need a different intervention, because supplying the same retrieved exemplar as melody conditioning leaves identification essentially unchanged.","The anchoring strength controls a monotone faithfulness-novelty trade-off, so downstream uses can select an operating point: low $s$ for fidelity, higher $s$ for diversity, with FAD roughly stable across the working range.","Faithfulness is restored only within the support of the memory bank; content outside the training stimuli cannot be anchored, so the method inherits the retrieval bank's coverage limits."],"supporting_citations":[{"why":"Supplies the Brain2Music dataset, the decode-then-generate formulation that RAG-Audio improves, and the direct MusicGen baseline re-implemented in a single harness.","marker":"(Denk et al., 2023)"},{"why":"Provides the SDEdit principle of partially noising a guide latent and denoising from an intermediate diffusion step, which is the diffusion branch of exemplar anchoring.","marker":"(Meng et al.)"},{"why":"Defines AudioLDM, the latent-diffusion generator whose direct-versus-anchored comparison yields the main identification gain and the FAD drop from 13.50 to 1.25.","marker":"(Liu et al., 2023a)"},{"why":"Defines TangoFlux, the rectified-flow generator used as the second latent-trajectory arm and for the novelty-diversity analysis.","marker":"(Hung et al., 2024)"},{"why":"Defines MusicGen, the autoregressive generator used as the negative control; its lack of an initializable latent trajectory isolates trajectory initialization as the mechanism.","marker":"(Copet et al., 2023)"},{"why":"Defines the nearest-neighbour retrieval baseline that sets the retrieval-level identification reference RAG-Audio matches while remaining generative.","marker":"(Ferrante et al., 2024)"},{"why":"Defines CLAP, the semantic audio embedding space in which decoding, retrieval, and the N-way identification metric all operate.","marker":"(Wu et al., 2023)"},{"why":"Supplies the GTZAN music clips and ten genres used as heard stimuli and as the structure for the genre-confusion analysis.","marker":"(Tzanetakis and Cook, 2002)"},{"why":"Supplies the InfoNCE contrastive objective that trains the fMRI-to-CLAP decoder, chosen because its cosine geometry matches the downstream identification metric.","marker":"(Oord et al., 2018)"}],"fun_headline_variants":["Retrieval anchor lifts brain-to-audio ID from 14% to 43%","Starting from a real clip triples brain-to-audio identification","RAG-Audio: retrieval-guided initialization beats direct decoding","Anchoring to a real audio exemplar fixes prior domination"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim rests on treating 10-way identification in CLAP space as the measure of stimulus faithfulness; if that embedding space, which is also the retrieval bank's geometry and the decoder's training objective, does not track the perceptual or acoustic properties of the heard clip, the reconstruction gains are not established.","fun_headline_variants_meta":{"raw":{"variants":["Retrieval anchor lifts brain-to-audio ID from 14% to 43%","Starting from a real clip triples brain-to-audio identification","RAG-Audio: retrieval-guided initialization beats direct decoding","Anchoring to a real audio exemplar fixes prior domination"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000823,"raw_usage":{"total_tokens":3661,"prompt_tokens":1070,"completion_tokens":2591,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":686,"completion_tokens_details":{"reasoning_tokens":2515}},"tokens_in":686,"tokens_out":2591,"duration_ms":17870,"temperature":1.0,"reasoning_tokens":2515,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T14:25:19.766090+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the anchored reconstructions through a forced-choice identification by human listeners, or by an audio model whose training data are disjoint from the stimulus genres, and compare with the CLAP 10-way scores: if human identification of the heard clip does not rise with the CLAP-based gain, the central claim of restored faithful reconstruction fails. Alternatively, anchor from a randomly chosen exemplar at the same strength; if identification stays near retrieval level, the nearest-neighbour content is not carrying the effect.","supporting_citations":[],"review_version":2}