{"id":"7c9ab999-1a6a-4023-b549-facdf04f5ea6","arxiv_id":"2506.17886","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A text-conditioned diffusion model that generates audio-latent queries in a frozen audio embedding space improves in-domain text-music retrieval and enables post-hoc controllability via negative prompting and DDIM inversion.","lead":"This paper trains a diffusion model to generate 'ghost' audio queries from text, then uses these generated embeddings to search a music library. It reports better retrieval than contrastive baselines on some datasets and adds interactive controls such as negative prompting and DDIM inversion.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GDR's retrieval gains over teachers are confounded by training on the evaluation domain; without a fine-tuned-teacher control, the unqualified improvement claim is unsupported.","rationale":"I read the paper as proposing a generative replacement for the text encoder at query time, with the central promise being improved retrieval plus inference-time controllability. The retrieval-improvement claim depends on comparing against a teacher under matched conditions. Since GDR is trained on PrivateCaps while the teacher baselines are not, the PrivateCaps numbers in Table 2 are not a controlled comparison. This is closely related to the reader's weakest assumption about out-of-sample fairness and memorization, but I would emphasize the missing matched baseline: even a perfectly clean split would not tell us whether the gain comes from generation or simply from training on the evaluation distribution. The MusicCaps results, where GDR-CLAP loses to the teacher, reinforce this concern and directly contradict the abstract's unqualified wording. The controllability results, by contrast, are largely independent of the retrieval-SOTA claim: negative prompting and DDIM inversion are demonstrated with CLAP-score and FAD evidence, and they would remain interesting even if the retrieval gains were limited to in-domain settings. For that reason I do not move the verdict to reject; the paper can be made sound by adding the matched-teacher control and qualifying the abstract. This remains a conditional acceptance issue, so I leave the reader's verdict unchanged while noting that the concern is load-bearing for the broadest version of the claim.","tokens_in":12784,"tokens_out":5898,"duration_ms":61419,"concrete_test":"Fine-tune the CLAP teacher on the exact PrivateCaps training split used for GDR, with the same number of steps and the same audio/text preprocessing, then evaluate on the same 5,500-sample PrivateCaps test subset. If fine-tuned CLAP reaches or exceeds GDR-CLAP's R@1/R@5 of 6.9/17.1, the reported gain over the unfine-tuned teacher is explained by training-distribution fit rather than by generative query formation; if GDR-CLAP still wins, the confound is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The headline claim that GDR 'improves retrieval performance over contrastive teacher models' rests on comparisons where the teacher was not trained on the evaluation domain while GDR was. GDR is trained on PrivateCaps (Sections 3.2 and 3.3), and its largest gains are on the PrivateCaps test subset: Table 2 shows CLAP R@1 rising from 2.2 to 6.9 and R@5 from 7.2 to 17.1 for GDR-CLAP. On MusicCaps, the only evaluation set not included in GDR training, GDR-CLAP is worse than the CLAP teacher (R@1 2.7 vs. 3.8, R@5 7.6 vs. 12.9). Section 4.1.1 discusses this as inherited domain mismatch and proposes latent alignment, but it never includes the missing control: a contrastive teacher fine-tuned on the same PrivateCaps training data. Without that control, the PrivateCaps improvements could be fully explained by GDR fitting the PrivateCaps audio/text distribution rather than by generative query formation. The paper's own sentence in Section 4.1, 'GDR-CLAP underperforms on MC relative to the CLAP teacher', is direct evidence that the generative mechanism alone does not improve retrieval; the observed gains appear tied to training on the target distribution. The domain-adaptation discussion diagnoses the shift but does not remove the confound.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces GD-Retriever, a framework that trains a conditional latent diffusion model to generate 'ghost' audio queries in the latent space of a frozen audio encoder, conditioned on text embeddings. At inference, several generated queries are averaged over time and over the number of generated samples, then used for nearest-neighbor retrieval in the audio space. The authors evaluate retrieval on PrivateCaps, SongDescriber, and MusicCaps against CLAP and MusCALL teachers, study domain shift and a post-hoc alignment mitigation, demonstrate retrieval with non-jointly-trained text encoders such as Flan-T5 and audio encoders such as MULE, and present qualitative and quantitative controllability experiments using negative prompting and DDIM inversion. The central claims are that GD-Retriever improves retrieval over contrastive teachers in in-domain settings, enables audio-only latent-space retrieval, and adds inference-time controllability that joint-embedding models do not natively provide.","tokens_in":13053,"tokens_out":5940,"duration_ms":58922,"significance":"If the claims hold, the paper makes a useful contribution: it connects generative diffusion models to controllable music retrieval in a way that is not a trivial re-implementation of existing generative retrieval work, because it operates on latent sequences rather than pooled embeddings and demonstrates controllability mechanisms (negative prompting, DDIM inversion) that are native to diffusion but absent from standard contrastive retrieval. The release of code and the inclusion of multiple encoder-pair ablations (Tables 2-4) are strengths. The main retrieval claim, however, is not yet supported as stated: the paper's own results show out-of-domain degradation, and the largest in-domain gains are confounded with training on the evaluation domain. The controllability experiments are suggestive but rely on a metric computed in the same embedding family used to train the model. These issues are addressable within the scope of the manuscript, so the work warrants a major revision rather than rejection.","major_comments":[{"comment":"The unqualified abstract claim that 'GDR improves retrieval performance over contrastive teacher models' is contradicted by the paper's own results. GDR-CLAP is worse than CLAP on MusicCaps (R@5 7.6 vs. 12.9) and GDR-MusCALL is worse than MusCALL on PrivateCaps (R@5 25.1 vs. 26.2). Section 4.1 itself states that 'GDR-CLAP underperforms on MC relative to the CLAP teacher,' so the improvement claim should either be made explicitly conditional (e.g., limited to in-domain or specific teacher-dataset combinations) or be supported by additional evidence isolating when and why generative query formation helps.","section":"Abstract and Table 2"},{"comment":"The main retrieval comparison is confounded by training distribution. GDR is trained on PrivateCaps, and its largest improvements over the CLAP teacher appear on PrivateCaps, where CLAP was not trained. The paper never includes a control in which the contrastive teacher is fine-tuned on the same PrivateCaps training data. Without this control, the PrivateCaps gains can be explained by GDR fitting the target audio/text distribution rather than by the generative retrieval mechanism itself. Notably, GDR-MusCALL, whose teacher is described as in-domain on PC in Section 4.1.1, does not improve over MusCALL on PC (R@5 25.1 vs. 26.2), which is consistent with the confound.","section":"Sections 3.3 and 4.1, Table 2"},{"comment":"The alignment experiment fits a mean and covariance shift to the evaluation set and then reports retrieval R@5 on that same evaluation set. This is in-sample evaluation and inflates the reported improvements. The alignment parameters should be fitted on a held-out split (or via cross-validation) before evaluating retrieval, or the numbers should be presented only as a diagnostic of distribution shift rather than as evidence of an effective mitigation.","section":"Section 4.1.1, Table 3"},{"comment":"No error bars or significance tests are reported anywhere, which is especially important because GDR uses stochastic generation with nq = 5 and several reported differences are small (e.g., GDR-MusCALL on SD R@5 16.9 vs. 13.6; GDR-MusCALL on PC R@1 10.8 vs. 10.1). Without repeated-generation statistics or confidence intervals, these improvements cannot be reliably distinguished from sampling variability. At minimum, report means and standard deviations over multiple generations.","section":"Tables 2-6"},{"comment":"The negative-prompting evaluation uses CLAP score as the semantic ground truth, but GDR-CLAP is trained in CLAP's latent space and conditioned on CLAP's text encoder. If the CLAP model used for scoring is the same one used for training, the controllability metric may be partially circular. Please state explicitly which CLAP model is used for scoring and discuss this dependence; adding a human evaluation or a third-model semantic similarity metric would strengthen the controllability claim.","section":"Section 4.3.1, Table 6"}],"minor_comments":[{"comment":"The generator call 'G(za, tau, tau, zqT)' appears to include a duplicated time argument; please correct the notation.","section":"Equation (1)"},{"comment":"The table header says 'Retrieval (SD)' while the text says the results are on PrivateCaps; please clarify which evaluation dataset is used and make the table and text consistent.","section":"Section 4.2, Table 5"},{"comment":"Table 2 uses 'MedR (%)' while Table 4 uses 'MR'; define the metric and use a single notation throughout.","section":"Tables 2 and 4"},{"comment":"The 'CLAP score' is used as a central evaluation metric but is never defined or referenced; please add a definition and a citation.","section":"Section 4.3.1"},{"comment":"The paper states that classifier-free guidance is applied on text conditioning with a 10% masking probability, but does not report the guidance weight w used at inference; please specify it.","section":"Section 3.3"},{"comment":"The retrieval protocol does not state whether nearest-neighbor search uses cosine similarity, L2 distance, or another metric in the audio latent space; please specify.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"For the editor: I share the reviewer's concern that the headline improvement claim is not yet supported, but the manuscript is unusually candid about its own domain-mismatch results, and the central mechanism is not circular in the mathematical sense. The missing fine-tuned-teacher control and the in-sample alignment evaluation are both addressable within a revision, so I recommend major revision rather than rejection. The code release and the breadth of encoder-pair experiments are genuine strengths."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely new angle on music retrieval — train a diffusion model to hallucinate audio embeddings from a text query and then retrieve with those 'ghost' queries. The new parts that matter: generation over embedding sequences rather than a single pooled embedding, and bringing negative prompting and DDIM inversion into retrieval as post-hoc controls. Those controls are demonstrated with a reasonable amount of care, and the MULE+T5 experiment shows text-conditioned retrieval is possible without joint training, which is the most surprising result in the paper.\n\nWhat I think is solid: the sequence-based generation is a plausible step beyond DiffGAP/Diff4Steer, the ablation in Table 5 shows the diffusion UNet beats a regression UNet and MLP baselines on retrieval and fidelity, and the domain mismatch discussion in 4.1.1 is honest and mostly on target.\n\nThe main problem is the headline claim. The abstract says GDR 'improves retrieval performance over contrastive teacher models' without qualification. Table 2 contradicts that: GDR-CLAP is worse than CLAP on MusicCaps, GDR-MusCALL is worse on PrivateCaps. The big gains appear exactly where GDR was trained (PrivateCaps) and the paper itself says the diffusion adapts to the training distribution. There's no control where the contrastive teacher is fine-tuned on the same training data, so we can't separate generative query formation from distributional memorization. That's load-bearing, not cosmetic.\n\nMinor but worth mentioning: no error bars or significance tests, so R@ differences of one or two points could be noise. Controllability is evaluated with CLAP score in the same embedding family the model was trained on — a mild circularity, though FAD helps ground it. No released weights, private dataset sections, and key hyperparameters like CFG weight and denoising steps are missing.\n\nOverall: the framework deserves attention and the controllability story is the more convincing half. The retrieval-gain claim needs a fair baseline and a more honest abstract. I'd send this to referees with instructions to push on the teacher-fine-tuning control, but it's not a desk reject.\n\nFor a colleague: worth a skim if you care about music retrieval interfaces or generative query modification; don't take the retrieval numbers at face value.","headline":"A clever diffusion-based retrieval mechanism with genuine controllability affordances, but the headline retrieval gains rest on a confounded comparison and need a fairer baseline before they hold up.","tokens_in":13617,"tokens_out":3841,"would_cite":false,"duration_ms":36742,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Generating ghost audio queries from text improves in-domain text-music retrieval and adds inference-time controls.","keywords":["text-music retrieval","generative retrieval","latent diffusion","ghost queries","negative prompting","DDIM inversion","contrastive learning","music information retrieval"],"falsifier":"Train GDR-CLAP on PrivateCaps with the 5,500 evaluation tracks and their captions strictly excluded, then measure Recall@1 on those held-out tracks; if the improvement over the CLAP teacher disappears or reverses, the in-domain gain is memorization rather than generative generalization. A sharper check is to compare each ghost query's distance to training-track latents versus test-track latents: if ghost queries sit closer to training tracks, the retriever is effectively returning memorized neighbours.","tokens_in":12537,"feed_emoji":"🎵","tokens_out":13403,"duration_ms":111462,"temperature":0.7,"pith_summary":"Text-music retrieval systems usually encode a text query and candidate audio clips into a shared embedding space and compare the resulting vectors. GD-Retriever instead trains a diffusion model to turn a text caption into a \"ghost\" audio embedding in the frozen audio encoder's own space, then retrieves by nearest-neighbor search with that generated query. The paper argues this generative route outperforms the contrastive teacher models it builds on for in-domain retrieval, works even when the text and audio encoders were never jointly trained, and brings generative controllability, negative prompting and DDIM inversion, to retrieval. A sympathetic reader would care because the proposal points from rigid joint-embedding search toward interactive, steerable music retrieval.","feed_headline":"Ghost audio queries beat contrastive teachers on in-domain retrieval","feed_subtitle":"A diffusion model writes the query in audio space, enabling negative prompting and DDIM inversion controls.","key_machinery":"The load-bearing mechanism is ghost-query generation in a retrieval-optimized latent space: a conditional latent diffusion model $G$ trained by minimizing $\\mathcal{L}_G = \\mathbb{E}_{\\tau, Z_a, z_T^q}[\\lVert z_a - G(z_a, \\tau, z_T^q)\\rVert_2^2]$ learns to produce audio-embedding sequences from text-embedding sequences, and the generated sequences are averaged over time and samples to form the retrieval query $\\tilde{Z}_A^q$. Unlike prior generative retrieval, the model emits full embedding sequences rather than pooled embeddings, so text conditions interact with audio latents through cross-attention at a finer grain. Two standard diffusion controls then carry the interactivity: classifier-free guidance with a negative prompt, whose update replaces the unconditional prediction with a negatively conditioned prediction (Eq. 2), and DDIM inversion, which re-noises a generated latent partway and re-denoises it under a modified prompt.","core_discovery":"GD-Retriever's central claim is that a text-conditioned latent diffusion model can serve as the query encoder of a retrieval system. The diffusion model is trained to reconstruct audio-embedding sequences extracted from a frozen audio encoder (CLAP, MusCALL, or MULE), conditioned on text-embedding sequences coming from any text encoder, including Flan-T5, which was never trained jointly with the audio encoder. At inference, the model generates $n_q=5$ ghost queries for a caption, averages them over the sequence dimension and over the samples, and uses the averaged latent $\\tilde{Z}_A^q$ for nearest-neighbor retrieval in the audio space. On the in-domain PrivateCaps benchmark, GDR-CLAP raises Recall@1, the fraction of queries whose top retrieved track is the correct one, from 2.2 to 6.9 against the CLAP teacher, and GDR-MULE with a Flan-T5 text encoder outperforms the CLAP teacher on PrivateCaps despite MULE's space being audio-only. The paper further shows negative prompting shifts generated queries away from undesired attributes while keeping them realistic (low FAD), and DDIM inversion allows re-noising a query and re-guiding it toward a modified prompt while preserving similarity to the original. The authors report that out-of-domain MusicCaps performance degrades, and they attribute the drop to distribution shift inherited from the teacher's embedding space, offering a post-hoc mean-and-covariance alignment as a partial fix.","pith_inferences":["Editorial inference: if the in-domain gains survive strict exclusion of the evaluation tracks from training, the same ghost-query recipe should transfer to other modalities with pre-trained frozen encoders and no joint text encoder, such as images, sound effects, or speech.","Editorial inference: because the text encoder is fully decoupled, practitioners could swap in a larger instruction-tuned language model at inference time to improve caption interpretation without any retraining of the audio side.","Editorial inference: the controllability demonstrations suggest an interactive retrieval loop, retrieve, invert, edit, re-retrieve, whose end-to-end value would need user studies or retrieval metrics on systematically perturbed captions rather than the paper's 50-prompt evaluations alone.","Editorial inference: a sharper test of the mechanism would compare ghost-query retrieval against the teacher's own text encoder combined with test-time distribution normalization; if normalization closes most of the gap, the generative model's benefit may be mainly adaptation to the training distribution."],"forward_implications":["Text-music retrieval becomes possible on top of any frozen audio encoder: a separately trained text encoder such as Flan-T5 can drive retrieval in MULE's audio-only space, removing the requirement for joint multimodal pretraining.","Inference-time negative prompting lets a user remove an undesired attribute (for example \"a rock song\") from a query, and the modified query stays in-distribution rather than drifting into unrealistic audio.","DDIM inversion supports a refine-and-search loop: a user takes a partly satisfactory retrieval result, re-noises it, re-guides it with a modified caption, and keeps the result close to the original while aligning with the new description.","GDR inherits the teacher's domain shift: the largest gains appear where the teacher's embedding distribution matches the training data, and a post-hoc mean/covariance shift toward the evaluation set reduces Fréchet audio distance and improves R@5.","On generation-quality metrics, the diffusion UNet beats regression UNet and MLP baselines on retrieval and fidelity, while an MLP diffusion baseline produces more diverse queries, a fidelity-diversity trade-off in generative retrieval."],"supporting_citations":[{"why":"Supplies the CLAP audio-text encoder and the contrastive teacher baseline that GDR-CLAP improves on PrivateCaps and SongDescriber.","marker":"[3]"},{"why":"Supplies the MusCALL audio-text encoder and teacher baseline used for GDR-MusCALL comparisons.","marker":"[1]"},{"why":"Classifier-free guidance is the conditioning mechanism, and its negative-prompt variant is the basis of Eq. (2).","marker":"[35]"},{"why":"DDIM (null-text) inversion is the technique adapted to re-noise and re-guide ghost queries for post-hoc retrieval refinement.","marker":"[17]"},{"why":"MULE provides the audio-only latent space used to show that GDR works without joint text-audio pretraining.","marker":"[50]"},{"why":"Flan-T5 supplies a text encoder never jointly trained with the audio encoders, demonstrating the decoupled-encoder affordance.","marker":"[52]"},{"why":"Supports the sample-objective choice for the diffusion training loss, which the paper finds yields better retrieval than the epsilon-objective.","marker":"[33]"},{"why":"Prior generative retrieval in contrastive space that GDR extends by generating full embedding sequences rather than aggregated embeddings.","marker":"[34]"}],"fun_headline_variants":["Diffusion-generated audio queries boost text-music retrieval","Generative query control: diffusion models beat contrastive teachers","Ghost queries from diffusion models enable controllable music retrieval","Text-music retrieval gains controllability via diffusion-generated queries","Diffusion query encoder outperforms contrastive models on in-domain music search"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 5,500-track PrivateCaps test subset is a fair out-of-sample test of a model trained on the 251k-track PrivateCaps corpus, so the reported gains over contrastive teachers reflect genuine generative retrieval rather than memorization of the training distribution.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion-generated audio queries boost text-music retrieval","Generative query control: diffusion models beat contrastive teachers","Ghost queries from diffusion models enable controllable music retrieval","Text-music retrieval gains controllability via diffusion-generated queries","Diffusion query encoder outperforms contrastive models on in-domain music search"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00083,"raw_usage":{"total_tokens":3659,"prompt_tokens":1015,"completion_tokens":2644,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":2563}},"tokens_in":631,"tokens_out":2644,"duration_ms":18570,"temperature":1.0,"reasoning_tokens":2563,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T18:58:49.008813+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train GDR-CLAP on PrivateCaps with the 5,500 evaluation tracks and their captions strictly excluded, then measure Recall@1 on those held-out tracks; if the improvement over the CLAP teacher disappears or reverses, the in-domain gain is memorization rather than generative generalization. A sharper check is to compare each ghost query's distance to training-track latents versus test-track latents: if ghost queries sit closer to training tracks, the retriever is effectively returning memorized neighbours.","supporting_citations":[{"cited_title":"Using a pretrained latent space op- timized for audio-audio retrieval, we train a generative dif- fusion model conditioned on text to generate audio latent embeddings in this space","cited_arxiv_id":null,"evidence_quote":"Supplies the CLAP audio-text encoder and the contrastive teacher baseline that GDR-CLAP improves on PrivateCaps and SongDescriber."},{"cited_title":"Spec- MaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Be- yond,","cited_arxiv_id":null,"evidence_quote":"Classifier-free guidance is the conditioning mechanism, and its negative-prompt variant is the basis of Eq. (2)."},{"cited_title":"Music con- trolnet: Multiple time-varying controls for music gen- eration,","cited_arxiv_id":null,"evidence_quote":"DDIM (null-text) inversion is the technique adapted to re-noise and re-guide ghost queries for post-hoc retrieval refinement."},{"cited_title":"Arrange, inpaint, and refine: steerable long-term music audio generation and editing via content-based controls,","cited_arxiv_id":null,"evidence_quote":"Flan-T5 supplies a text encoder never jointly trained with the audio encoders, demonstrating the decoupled-encoder affordance."},{"cited_title":"Augment, drop & swap: Improving diversity in llm captions for efficient music-text representation learning,","cited_arxiv_id":null,"evidence_quote":"Supports the sample-objective choice for the diffusion training loss, which the paper finds yields better retrieval than the epsilon-objective."},{"cited_title":"Cacophony: An improved contrastive audio-text model,","cited_arxiv_id":null,"evidence_quote":"Prior generative retrieval in contrastive space that GDR extends by generating full embedding sequences rather than aggregated embeddings."}],"review_version":2}