{"id":"2919387f-0aba-4635-8467-d059ecc76ffb","arxiv_id":"2506.17815","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SLAP trains joint music-text embeddings with a BYOL-style, negative-free loss and outperforms CLAP on music-text retrieval, zero-shot classification, and several downstream MIR tasks while reducing the modality gap.","lead":"This paper proposes SLAP, a way to train music and text models together without needing negative examples, by adapting the BYOL self-supervised method. A generalist should read it because it offers a more scalable and batch-size-robust alternative to CLAP-style contrastive training for music understanding.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline outperformance claim rests on single-run metrics with margins of 0.2–0.4 Recall@1, which may be within seed-to-seed noise; the central claim is not yet statistically established.","rationale":"The paper is carefully controlled in one important respect: SLAP and the reproduced CLAP baseline share architecture, training data, and training budget, and the authors release their code. The ablations in Figure 5 and the batch-size study in Figure 4 are informative and support the scalability claims. However, the evidence for the central empirical claim is not yet strong enough. The reported margins over CLAP are small, and every number in Tables 2, 4, and 5 comes from a single training run. Because the main comparison is between two stochastic training procedures, seed variance translates directly into variance of the headline metrics. In addition, the value λ=0.5 is highlighted in Section 5.6 as the setting that minimizes the modality gap, and the tuning appears to use the same MusicCaps and Song Describer evaluation sets that appear in Table 2. This does not invalidate the method, but it means the specific claim that SLAP consistently outperforms CLAP is not yet established at the precision implied by the tables. The concrete multi-seed check would settle whether the gap is real or an artifact of a single seed and test-set selection. The reader's conditional-accept recommendation is appropriate; the multi-seed experiment should be a required revision, and the λ-selection protocol should be stated explicitly for every reported result.","tokens_in":13030,"tokens_out":7520,"duration_ms":77296,"concrete_test":"Re-run the full pretraining and retrieval pipeline (same PrivateCaps, same architecture, same training budget) for at least 5 random seeds for both SLAP with λ=0.5 and the CLAP baseline, and report the mean and standard deviation of Recall@1/5/10 on MusicCaps and Song Describer, along with a paired bootstrap or permutation test over query items. If the 95% confidence interval for the SLAP-minus-CLAP R@1 difference includes zero on MusicCaps, the consistent-outperformance claim is not supported for that dataset and should be softened. Additionally, split the test sets and select λ on one half, then evaluate on the other half, to remove the selection-bias risk.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that SLAP consistently outperforms CLAP on text-music retrieval, zero-shot classification, and probing. The most load-bearing evidence is Table 2, but every entry is a single training run with no error bars. The reported margins over the reproduced CLAP baseline are very small: on MusicCaps, pretrained SLAP vs CLAP R@1 is 3.1 vs 2.8 (A→T) and 3.0 vs 2.8 (T→A); on Song Describer, 5.7 vs 5.3 and 6.0 vs 5.7. With 5.5k and 1k query items respectively, a 0.3-point R@1 gap corresponds to only a handful of correctly ranked queries, well within plausible seed-to-seed variation. The paper also selects the balancing weight λ in Section 5.6 using the same MusicCaps and Song Describer evaluation sets that appear in the headline Table 2, so the reported advantage may be optimistically biased by test-set selection. If the true difference is zero or negative on a dataset, the claim of consistent outperformance is unsupported, even though the method may still be a useful non-contrastive alternative.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SLAP, a negative-free multimodal pretraining framework that adapts the BYOL paradigm to music-text learning. Audio and text encoders are trained with an EMA-updated target branch and predictors, optimizing a weighted sum of intermodal and intramodal cosine losses (Eq. 5). The authors evaluate SLAP against a reproduced CLAP baseline with identical architecture, training data, and budget on text-music retrieval (MusicCaps, Song Describer), zero-shot classification (GTZAN, MTAT, OpenMic), and downstream probing, and additionally report a reduced modality gap and improved robustness to batch-size variation. The central claim is that SLAP outperforms or matches CLAP while avoiding large-batch negative-sample requirements.","tokens_in":13102,"tokens_out":3651,"duration_ms":37091,"significance":"If the reported results hold, the contribution is valuable: it offers a non-contrastive alternative to CLAP-style training for music-language embeddings, with a demonstrably smaller modality gap and the practical benefit of gradient accumulation. The experimental design is a strength: SLAP and the reproduced CLAP baseline share the same architecture, training data, and computational budget, and the public code release supports reproducibility. The reduced modality gap is quantified with both centroid distance and linear separability, and the batch-size study directly addresses the scalability motivation. The main weakness is statistical: the headline retrieval comparisons rest on single runs with small absolute margins, and the loss-weight selection uses the same test sets that later serve as evaluation.","major_comments":[{"comment":"All retrieval results are single training runs with no error bars, confidence intervals, or significance tests. The SLAP-vs-CLAP margins are small (e.g., 3.1 vs 2.8 R@1 on MusicCaps A→T and 5.7 vs 5.3 on Song Describer A→T), and with roughly 5.5k and 1k test queries these gaps correspond to only a handful of correctly ranked items, which could easily be within seed-to-seed variation. Please report multiple seeds with mean and standard deviation, or otherwise provide statistical support for the claim that SLAP consistently outperforms CLAP on retrieval.","section":"Section 5.1, Table 2"},{"comment":"The balancing weight λ is tuned on MusicCaps and Song Describer, which are the same datasets used for the headline retrieval results in Table 2. This creates an optimism bias in the reported advantages, because the model is effectively selected on the evaluation sets. Please tune λ on a held-out validation split, or report results for a λ chosen without access to the test data, and indicate whether the Table 2 conclusions remain unchanged.","section":"Section 5.6, Figure 5"},{"comment":"The viability of the method depends on the EMA mechanism and predictors preventing representation collapse, but the paper reports that values of λ outside [0.2, 0.7] lead to near-systematic collapse and provides no seed variance or quantitative collapse diagnostic for the stable regime. Since a single collapsed run would invalidate the learned embeddings, please report multiple seeds, a collapse measure on validation data, and discuss the narrow stable range in relation to the claimed hyperparameter robustness.","section":"Section 5.6, Figure 5"},{"comment":"The statement that gradient accumulation in SLAP is empirically confirmed to be equivalent to true batch scaling is supported only by a single set of curves on PrivateCaps without error bars or a direct side-by-side comparison with non-accumulated training at the same effective batch size. Please provide repeated runs or explicitly label this as an informal observation, and clarify whether the equivalence claim refers to optimization dynamics or final retrieval performance.","section":"Section 5.5, Figure 4"}],"minor_comments":[{"comment":"The title in the full-text version contains a spacing typo, 'NEGA TIVE', which should be corrected to 'NEGATIVE'.","section":"Title and running text"},{"comment":"The text encoder definition says 'maps Ntokens to a tex latent space'; 'tex' should be 'text'.","section":"Section 3"},{"comment":"The abstract claims that the formulation 'unlocks large-scale training on a single GPU through gradient accumulation', but Section 4.3 states that all models were trained on 6 A100 GPUs. Please clarify whether single-GPU training was actually run or is only a potential benefit.","section":"Abstract and Section 4.3"},{"comment":"The paper describes the approach as 'hyperparameter-robust', but Figure 5 shows collapse outside λ ∈ [0.2, 0.7]; please qualify this claim to the stable region or provide additional evidence of robustness.","section":"Introduction and Section 5.6"},{"comment":"The definition of mAR uses R_n@k without explicitly stating whether it is recall at k averaged over tags and samples in the same way as the standard multilabel retrieval metric; a one-sentence clarification would improve reproducibility.","section":"Table 5 and Eq. (6)"}],"recommendation":"major_revision","confidential_remarks":"The controlled comparison and released code are genuine strengths, but the single-run reporting and test-set-based λ selection are the main barriers to accepting the headline claims. If the authors add seed variance and re-tune on a validation split, the paper could become acceptable. The private training data is a secondary limitation, though the code release mitigates it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: SLAP is a new training recipe, not a rehash. It takes BYOL's EMA/predictor setup and applies it to music-text with two predictors and a sum of inter- and intra-modal cosine losses. The controlled comparison is the paper's strongest aspect: they re-trained CLAP on the same private 260k-pair dataset with the same HTS-AT/RoBERTa backbones and same budget. That is real work, and the retrieval, probing, and zero-shot numbers all go the same direction. The finding that dropping either loss term collapses the model (Figure 5) is genuinely useful.\n\nThe soft spots are the ones the stress-test flags, and they are not manufactured. Every headline result is from a single training run. The margins over the reproduced CLAP are 0.2–0.4 Recall@1 absolute. On MusicCaps that is roughly 16 queries out of 5.5k; on Song Describer it is 3–4 queries out of 1k. Without error bars or repeated seeds, 'consistently outperforms' is not supported. On top of that, Section 5.6 tunes lambda on the same MusicCaps and Song Describer test sets that appear in Table 2, so the advantage may be partially selected. The paper also never explicitly states that lambda=0.5 was used for the main tables—Section 5.6 implies it, but it should be stated. Private training data limits exact reproducibility, though code release helps.\n\nNone of this kills the paper. The method is plausible, the loss design is the right kind of contribution, and the reduced modality gap, measured by centroid distance and linear separability, is convincing even if partly a direct consequence of the loss. The batch-size robustness result with gradient accumulation is also nice and matches the negative-free formulation.\n\nWho gets value: anyone working on music-text or audio-language pretraining, especially people wanting to scale without huge contrastive batches. It deserves a serious referee. I would ask for multi-seed runs (at least 3) and either error bars or a statement of variance, an explicit lambda value for every table, and a held-out split for hyperparameter selection. If compute makes multi-seed impossible, the claims should be softened to 'comparable' rather than 'outperforms.'","headline":"SLAP is a legit new BYOL-style recipe for music-text with a controlled comparison and real modality-gap reduction, but headline gains rest on single runs and small margins; needs multi-seed evidence.","tokens_in":13807,"tokens_out":3362,"would_cite":true,"duration_ms":32873,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SLAP, a negative-free siamese pretraining method for text and audio, outperforms contrastive CLAP on music retrieval and zero-shot classification.","keywords":["multimodal pretraining","music-text retrieval","negative-free learning","BYOL","modality gap","zero-shot classification","self-supervised learning","music information retrieval"],"falsifier":"Train SLAP with $\\lambda=0.5$ on a public music-caption dataset (for example MusicCaps captions paired with their audio) instead of PrivateCaps and measure audio-to-text retrieval and the audio/text linear separability; if retrieval collapses to chance or the separability becomes as large as CLAP's, the collapse-avoidance claim fails. A second check: sweep the EMA rate $\\tau$ from 0.9 to 0.999 and observe whether retrieval degrades sharply outside 0.95, which would show the claimed robustness does not generalise.","tokens_in":12662,"feed_emoji":"🎵","tokens_out":6268,"duration_ms":56480,"temperature":0.7,"pith_summary":"SLAP (Siamese Language-Audio Pretraining) replaces the contrastive loss at the heart of CLAP-style joint embedding training with a negative-free objective: for each music-caption pair, two EMA-updated target encoders provide slow-moving targets, and the context encoders predict those targets across both modalities using cosine losses. The paper argues this yields embeddings that beat a same-architecture CLAP baseline on text-music retrieval and zero-shot classification, match or exceed it on downstream genre, tag, and instrument probes, and cut the modality gap to near zero. If true, joint text-music spaces no longer need large batches of negative samples, making large-scale training cheaper and better suited to many-to-many music-caption relationships.","feed_headline":"No negatives, better music-text retrieval: SLAP beats CLAP","feed_subtitle":"A siamese pretraining scheme with slowly-updated targets shrinks the modality gap and scales to single-GPU training.","key_machinery":"The central machinery is an asymmetric siamese network with two branches per modality: a context encoder $E$ that gets gradients and an EMA target encoder $\\bar{E}$ updated as $\\tau\\bar{E} + (1-\\tau)E$, plus a predictor MLP on each context branch. The predictors produce $q_A$ and $q_T$, each of which is trained to match the EMA targets of both audio and text via cosine losses. The stop-gradient on target encoders and the asymmetry-breaking predictor are what prevent collapse in principle, while the intramodal terms $L_A$ and $L_T$ keep the two modalities from drifting apart. This setup lets the objective be computed without any cross-sample negative pairs, so the loss does not need a $B\\times B$ similarity matrix and permits gradient accumulation.","core_discovery":"On its own terms, SLAP's discovery is that the multimodal contrastive loss is not necessary for strong text-music joint embeddings. Starting from the same HTS-AT audio encoder and RoBERTa text encoder used in the LAION-CLAP model, and training on the same PrivateCaps pairs, the paper's loss $L = \\lambda(L_{A\\to T}+L_{T\\to A}) + (1-\\lambda)(L_A+L_T)$ — where each $L$ is a cosine distance between a predictor output and an EMA target — outperforms the reproduced CLAP on retrieval recall and normalized rank, raises zero-shot accuracy on GTZAN from 51.7 to 58.3, and lowers both the centroid distance and linear separability between audio and text manifolds. The authors claim that the intramodal losses are necessary to prevent collapse, and that only a balanced weight $\\lambda\\approx 0.5$ removes the modality gap while preserving retrieval.","pith_inferences":["If the collapse-avoidance property holds across datasets, SLAP-style losses could replace contrastive heads in other paired modalities (image-text, video-text) where many-to-many pairing and long-tailed data make negative sampling awkward.","The near-zero modality gap suggests SLAP embeddings may be directly usable as conditioning latents for text-to-music generation, where disjoint manifolds have been identified as an obstacle; this is a testable next step the paper does not run.","The narrow safe range for $\\lambda$ (about 0.2 to 0.7, and exactly 0.5 for minimal gap) implies the method needs a dataset-specific balancing of the four losses; a public dataset replication would show whether the optimum migrates with caption noise or batch size.","The claimed robustness to batch size implies that very large effective batches could be reached by gradient accumulation alone, which would let researchers train on today's largest caption corpora without multi-GPU memory walls."],"forward_implications":["Text-music retrieval improves with no negative samples: SLAP beats CLAP on Recall@1/5/10 and normalized rank on Song Describer and MusicCaps for both retrieval directions.","Zero-shot music classification and tagging improve: best-of-four prompts give 58.3 percent accuracy on GTZAN, 31.5 percent mAP on MTAT, and 70.5 percent mAR on OpenMic, all above the CLAP baseline.","Frozen SLAP audio embeddings transfer to downstream probing: they beat CLAP and MusCALL on genre, tag, and instrument tasks and reach the best mAP on MagnaTagATune among the compared models.","The modality gap nearly disappears with a balanced loss weight $\\lambda=0.5$, as measured by lower audio/text centroid distance and lower linear separability.","Because the loss needs no negatives, gradient accumulation makes effective batch sizes much larger on a single GPU; the paper shows retrieval stays stable as batch size grows from 64 to 768."],"supporting_citations":[{"why":"Supplies the BYOL negative-free EMA-plus-predictor paradigm that SLAP adapts to audio-text pairs.","marker":"[10]"},{"why":"Defines CLAP, the contrastive baseline architecture and training setup that SLAP compares against.","marker":"[2]"},{"why":"Provides the LAION-CLAP model with the same HTS-AT/RoBERTa backbones used for SLAP and a retrieval comparison.","marker":"[49]"},{"why":"MusCALL is trained on the same PrivateCaps dataset and serves as a same-data contrastive baseline for retrieval and probing.","marker":"[16]"},{"why":"Supplies the modality-gap definition and the centroid-distance and linear-separability metrics used in Section 5.4.","marker":"[8]"},{"why":"Supports the claim that the modality gap is intrinsic to the contrastive loss, motivating the negative-free design.","marker":"[7]"},{"why":"Provides the theoretical account of why EMA and architectural asymmetry avoid collapse in BYOL-style training.","marker":"[38]"},{"why":"Transfers BYOL to audio and gives the EMA-rate reference point (0.996) that SLAP tunes down to 0.95.","marker":"[11]"}],"fun_headline_variants":["SLAP drops negative samples, beats CLAP on music retrieval","No negatives needed: SLAP shrinks audio-text modality gap","SLAP: single-GPU music pretraining without contrastive loss","SLAP outperforms CLAP with no negative samples, smaller gap","BYOL-style SLAP topples CLAP on text-music retrieval"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training dynamics do not collapse: the slowly-updated target encoders and the predictor networks must keep all branches producing informative, non-identical representations, and the mixing weight $\\lambda$ must stay close to 0.5 so the within-modality and between-modality losses stay balanced; the paper verifies this only on one private dataset and a narrow $\\lambda$ range.","fun_headline_variants_meta":{"raw":{"variants":["SLAP drops negative samples, beats CLAP on music retrieval","No negatives needed: SLAP shrinks audio-text modality gap","SLAP: single-GPU music pretraining without contrastive loss","SLAP outperforms CLAP with no negative samples, smaller gap","BYOL-style SLAP topples CLAP on text-music retrieval"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00034,"raw_usage":{"total_tokens":1885,"prompt_tokens":963,"completion_tokens":922,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":846}},"tokens_in":579,"tokens_out":922,"duration_ms":6823,"temperature":1.0,"reasoning_tokens":846,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:00:46.155963+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SLAP with $\\lambda=0.5$ on a public music-caption dataset (for example MusicCaps captions paired with their audio) instead of PrivateCaps and measure audio-to-text retrieval and the audio/text linear separability; if retrieval collapses to chance or the separability becomes as large as CLAP's, the collapse-avoidance claim fails. A second check: sweep the EMA rate $\\tau$ from 0.9 to 0.999 and observe whether retrieval degrades sharply outside 0.95, which would show the claimed robustness does not generalise.","supporting_citations":[{"cited_title":"SLAP out- performs contrastive models on tasks including text-music retrieval, downstream probing, and zero-shot music un- derstanding","cited_arxiv_id":null,"evidence_quote":"Supplies the BYOL negative-free EMA-plus-predictor paradigm that SLAP adapts to audio-text pairs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines CLAP, the contrastive baseline architecture and training setup that SLAP compares against."},{"cited_title":"T-clap: Temporal- enhanced contrastive language-audio pretraining,","cited_arxiv_id":null,"evidence_quote":"MusCALL is trained on the same PrivateCaps dataset and serves as a same-data contrastive baseline for retrieval and probing."},{"cited_title":"We train SLAP on an internal private dataset of 260,000 pairs of full-length production-quality music tracks and professionally annotated captions (PrivateCaps [16])","cited_arxiv_id":null,"evidence_quote":"Supplies the modality-gap definition and the centroid-distance and linear-separability metrics used in Section 5.4."},{"cited_title":"A blaring metal track with stompy kicks and distorted chuggy guitar","cited_arxiv_id":null,"evidence_quote":"Supports the claim that the modality gap is intrinsic to the contrastive loss, motivating the negative-free design."},{"cited_title":"Recap: Retrieval-augmented audio captioning,","cited_arxiv_id":null,"evidence_quote":"Provides the theoretical account of why EMA and architectural asymmetry avoid collapse in BYOL-style training."}],"review_version":1}