{"id":"1b7335cb-1a48-4c55-acd7-41e59c64636d","arxiv_id":"2605.29628","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"COMET dissects the modality gap in CLAP embeddings using PLS-SVD, showing shared concept axes dominate similarity and proposing spectral truncation to mitigate the gap without training.","lead":"COMET applies partial least squares singular value decomposition to break down the modality gap in audio-text CLAP models into concept axes. This allows a training-free spectral truncation that boosts zero-shot audio captioning performance.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"PLS-SVD axes may capture covariance but not necessarily the causally dominant contributors to similarity or modality gap","rationale":"The reader's weakest assumption directly identifies the same interpretive leap from PLS-SVD output to causal responsibility for similarity and the gap. Because the full text was unavailable to the first reader and the technical validation steps (axis-wise similarity decomposition, controlled truncation ablations) are not described in the abstract, the UNVERDICTED status is unchanged.","tokens_in":1807,"tokens_out":351,"duration_ms":20493,"concrete_test":"Compute the per-axis contribution to the average cosine similarity on a held-out retrieval set by projecting embeddings onto each PLS-SVD direction in turn; if the top-k shared axes account for <70% of the total similarity mass (or if removing the complementary axes changes the modality-gap distance by less than the mean shift alone), the 'small subset contributes substantially' claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the small subset of PLS-SVD axes identified as 'shared concepts' are the primary drivers of cosine similarity (i.e., the directions along which most of the dot-product mass lies) and that the modality gap is only partially captured by the mean. PLS-SVD maximizes cross-modal covariance, yet in contrastive embedding spaces this does not automatically entail that these directions dominate the similarity computation once the embeddings are L2-normalized, nor that truncating the complementary directions removes the gap rather than incidental variance. Without an explicit decomposition of the similarity score into per-axis contributions or a controlled intervention showing that the gap metric changes disproportionately when those axes are altered, the interpretation remains correlational.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces COMET, a PLS-SVD framework for dissecting concept space in CLAP audio-text embeddings. It claims that only a small, interpretable subset of axes capturing shared concepts substantially contributes to similarity computation, that the mean component only partially represents the modality gap, and that a simple training-free spectral truncation method mitigates the gap. This enables zero-shot audio captioning via condition swapping to approach supervised performance while achieving dimensionality reduction on retrieval and captioning tasks.","tokens_in":1923,"tokens_out":629,"duration_ms":16014,"significance":"If the decomposition and intervention hold, the work provides an interpretable view of modality gap origins in contrastive audio-text spaces and a practical, training-free mitigation strategy. The emphasis on concept axes rather than mean shift alone, combined with reported gains on captioning without auxiliary memory banks, would be a useful contribution to multimodal embedding analysis in the audio domain.","major_comments":[{"comment":"§4 (PLS-SVD analysis) and §5 (spectral truncation): the central claim that the small subset of shared-concept axes 'contributes substantially to similarity computation' rests on covariance maximization but lacks an explicit per-axis decomposition of the cosine similarity (dot product after L2 normalization). Without showing that the retained axes account for the majority of the dot-product mass (e.g., via cumulative contribution plots or controlled ablation of the orthogonal complement), the truncation result remains correlational rather than demonstrating causal dominance.","section":"§4, §5"},{"comment":"§5.2 (modality-gap mitigation experiments): the assertion that mean shift 'represents only partially the modality gap' and that truncation removes the remainder requires a controlled comparison where the gap metric (e.g., mean embedding distance or retrieval asymmetry) is measured before/after mean correction alone versus after PLS-SVD truncation. The current evidence does not isolate whether truncation removes gap-related variance or incidental directions.","section":"§5.2"},{"comment":"Table 2 / Figure 4 (zero-shot captioning results): the claim that truncation enables performance 'approaching fully supervised' needs an ablation confirming that the improvement is due to gap reduction rather than general dimensionality reduction or removal of noise; a random-axis truncation baseline of matched rank would strengthen the interpretation.","section":"Table 2, Figure 4"}],"minor_comments":[{"comment":"Notation for the PLS-SVD transformation (Eq. 3) should explicitly state whether the resulting axes are orthonormal after the SVD step, as this affects the validity of simple truncation.","section":"Eq. 3"},{"comment":"The abstract states 'substantial embedding dimensionality reduction while preserving strong performance'; the exact retained rank (e.g., top-k value) and the corresponding performance drop should be reported consistently in the main text and tables.","section":"Abstract, §5"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed comments, which help strengthen the empirical grounding of our claims. We address each major comment below and will incorporate the suggested analyses into the revised manuscript.","responses":[{"response":"We agree that an explicit per-axis decomposition of the cosine similarity would provide stronger causal evidence. Although PLS-SVD maximizes cross-modal covariance (which directly informs the directions used in similarity), the current manuscript does not include a direct breakdown of dot-product mass. In the revision we will add cumulative contribution plots of the normalized dot product and an ablation that removes the orthogonal complement, quantifying the fraction of similarity mass retained by the shared-concept axes.","revision_made":"yes","referee_comment":"[§4, §5] §4 (PLS-SVD analysis) and §5 (spectral truncation): the central claim that the small subset of shared-concept axes 'contributes substantially to similarity computation' rests on covariance maximization but lacks an explicit per-axis decomposition of the cosine similarity (dot product after L2 normalization). Without showing that the retained axes account for the majority of the dot-product mass (e.g., via cumulative contribution plots or controlled ablation of the orthogonal complement), the truncation result remains correlational rather than demonstrating causal dominance."},{"response":"The referee correctly notes that isolating the incremental effect of truncation beyond mean correction is necessary. The manuscript currently compares truncation to the uncorrected baseline but does not report the intermediate mean-corrected condition. We will add this controlled comparison, measuring mean embedding distance and retrieval asymmetry after mean correction alone and after mean correction followed by PLS-SVD truncation, to demonstrate that truncation removes additional gap-related variance.","revision_made":"yes","referee_comment":"[§5.2] §5.2 (modality-gap mitigation experiments): the assertion that mean shift 'represents only partially the modality gap' and that truncation removes the remainder requires a controlled comparison where the gap metric (e.g., mean embedding distance or retrieval asymmetry) is measured before/after mean correction alone versus after PLS-SVD truncation. The current evidence does not isolate whether truncation removes gap-related variance or incidental directions."},{"response":"We acknowledge that a matched-rank random truncation baseline is required to rule out generic dimensionality-reduction effects. The current results compare spectral truncation only to the full embedding and to mean correction. In the revision we will include a random-axis truncation baseline at the same retained rank in both Table 2 and Figure 4, allowing direct comparison of performance gains attributable to concept-axis selection versus random selection.","revision_made":"yes","referee_comment":"[Table 2, Figure 4] Table 2 / Figure 4 (zero-shot captioning results): the claim that truncation enables performance 'approaching fully supervised' needs an ablation confirming that the improvement is due to gap reduction rather than general dimensionality reduction or removal of noise; a random-axis truncation baseline of matched rank would strengthen the interpretation."}],"tokens_in":1557,"tokens_out":633,"duration_ms":18587,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's core move is to apply PLS-SVD to CLAP embeddings so the modality gap can be read as a difference in how audio and text load onto a small set of shared-concept axes. They argue the mean shift explains only part of it and that truncating the remaining directions closes the gap enough to lift zero-shot captioning and retrieval without any retraining.\n\nThat framing is new for the audio side. Most prior work on the gap stayed at the level of overall means or cone effects; decomposing into interpretable axes and then acting on them is a distinct step. The truncation itself is cheap and they claim it also shrinks the embedding dimension while keeping performance, which is practically useful.\n\nThe soft spot is the jump from covariance to causation. PLS-SVD finds directions of high cross-modal covariance, but once embeddings are L2-normalized the similarity is just the dot product. The paper needs to show either an explicit per-axis breakdown of the average cosine or a controlled ablation where zeroing the identified axes moves the gap metric more than zeroing other axes. Without that, the claim that these axes are the main contributors remains correlational. The abstract does not contain that decomposition, so the interpretation rests on the assumption that high covariance equals high contribution to similarity.\n\nThe experiments are only summarized, so I cannot judge robustness or controls. Still, the angle is fresh enough and the proposed method is lightweight enough that the work should go to referees rather than get desk-rejected. Readers working on multimodal contrastive models or zero-shot audio tasks would get value from seeing the full decomposition and the truncation results.","headline":"COMET gives a PLS-SVD view of the audio-text modality gap and a simple truncation fix, but the causal link from axes to similarity still needs direct evidence.","tokens_in":2387,"tokens_out":404,"would_cite":false,"duration_ms":14314,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A small set of shared concept axes dominates similarity in CLAP embeddings while the mean shift accounts for only part of the modality gap.","keywords":["modality gap","CLAP","audio-text embeddings","PLS-SVD","concept decomposition","spectral truncation","zero-shot learning","contrastive embeddings"],"falsifier":"If spectral truncation on the identified axes produces no gain in zero-shot captioning accuracy or retrieval metrics, the claim that these axes control the gap would be falsified.","tokens_in":2704,"feed_emoji":"","tokens_out":593,"duration_ms":17791,"temperature":0.7,"pith_summary":"The paper applies partial least squares singular value decomposition to CLAP audio-text embeddings to separate shared concept axes from modality-specific directions. It shows that cosine similarity depends mainly on a few of these shared axes rather than the full embedding space. The conventional mean-vector correction addresses only a fraction of the observed gap. Building on the decomposition, a spectral truncation operation removes the non-shared components and reduces the gap without any model retraining. The resulting embeddings support stronger zero-shot audio captioning via condition swapping and permit large reductions in dimensionality while retaining retrieval performance.","feed_headline":"Spectral truncation closes modality gap in audio-text embeddings","feed_subtitle":"PLS-SVD identifies a few shared concept axes whose removal improves zero-shot captioning without retraining.","key_machinery":"PLS-SVD transformation that decomposes embeddings into shared-concept axes and modality-specific residual directions.","core_discovery":"COMET reveals that only a small, interpretable subset of axes which captures shared concepts contributes substantially to similarity computation, and that the mean component represents only partially the modality gap. A simple spectral truncation method mitigates the modality gap in a training-free manner, enabling zero-shot audio captioning with condition swapping to approach fully supervised performance without large auxiliary memory banks or expensive computation.","pith_inferences":["The same axis-decomposition technique could be tested on other contrastive multimodal models to locate their modality gaps.","Retaining only the shared axes might yield more interpretable embeddings for downstream audio tasks.","Dimensionality reduction to the shared subspace could lower storage and inference costs in deployed CLAP systems."],"forward_implications":["Spectral truncation reduces the modality gap without retraining or auxiliary data structures.","Zero-shot audio captioning via condition swapping reaches performance levels comparable to fully supervised models.","Embeddings can be projected onto far fewer dimensions while preserving retrieval and captioning accuracy.","The truncation operates at inference time with negligible extra cost."],"fun_headline_variants":["COMET dissects modality gap using PLS-SVD in CLAP","Spectral truncation closes CLAP modality gap without training","PLS-SVD shows mean is partial modality gap explanation","Shared concept axes drive most CLAP similarity computation"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The PLS-SVD transformation isolates axes that are causally responsible for both the similarity scores and the modality gap.","fun_headline_variants_meta":{"raw":{"variants":["COMET dissects modality gap using PLS-SVD in CLAP","Spectral truncation closes CLAP modality gap without training","PLS-SVD shows mean is partial modality gap explanation","Shared concept axes drive most CLAP similarity computation"]},"model":"grok-4.3","cost_usd":0.004592,"raw_usage":{"total_tokens":2310,"prompt_tokens":731,"num_sources_used":0,"completion_tokens":63,"cost_in_usd_ticks":45924500,"prompt_tokens_details":{"text_tokens":731,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1516,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":731,"tokens_out":63,"duration_ms":11310,"temperature":1.0,"reasoning_tokens":1516,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-29T05:55:20.141269+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"If spectral truncation on the identified axes produces no gain in zero-shot captioning accuracy or retrieval metrics, the claim that these axes control the gap would be falsified.","supporting_citations":[],"review_version":1}