{"id":"4d2c4437-ba40-42a5-af7c-099afde714d3","arxiv_id":"2507.22995","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A multi-view SSL framework with combined reconstruction and separation-based contrastive losses obtains disentangled pitch and instrument subspaces without the accuracy loss seen with contrastive-only training on NSynth.","lead":"The paper combines reconstruction and contrastive losses in a multi-view model to disentangle pitch and instrument in music representations, and finds that separation-based objectives are the key driver of disentanglement. A systematic comparison on the NSynth dataset shows that adding reconstruction preserves overall accuracy while contrastive methods alone lose information.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Positive pitch margin may reflect confounds (sub-instrument, loudness) because the separation loss is attribute-agnostic; without factor-specific probes, pitch disentanglement is not yet established.","rationale":"The paper's central claim is that reconstruction plus a separation-based contrastive objective yields disentanglement of music attributes without information loss. The mechanism claimed is that separation pushes pitch into the private subspace. The reader's weakest assumption states this is not guaranteed: the separation loss is agnostic to which factor varies, and a confound such as velocity could produce the observed margins. I agree, and I find this to be the most load-bearing concern because the entire contribution is about disentangling specific attributes. If the private subspace encodes a correlate of pitch rather than pitch itself, the observed positive Delta_Pitch does not demonstrate the claimed disentanglement. The concern is not merely theoretical: NSynth instrument families are coarse, so sub-instrument identity correlates with pitch range, and velocity/loudness can correlate with pitch in recorded notes. A linear probe on private latents for these non-target factors would directly adjudicate the issue. Other concerns raised by the reader (no error bars, weak CL baseline, use of labels for pairing) are secondary. The CL baseline issue could weaken the comparison to contrastive-only methods but would not invalidate the demonstration that separation plus reconstruction yields high overall accuracy; the label-pairing issue is acknowledged by the authors and affects the 'self-supervised' framing but not the internal comparison. The confound issue, by contrast, undermines the interpretation of the core disentanglement metric. Because the paper's methodology and results are otherwise clearly described and the fix is a straightforward additional probe analysis, the appropriate verdict remains CONDITIONAL, matching the reader's judgment. No change to the reader's verdict is needed; the stress-test confirms and sharpens the weakest assumption.","tokens_in":8853,"tokens_out":6593,"duration_ms":83901,"concrete_test":"On the frozen CosineSEP lambda=0.6 model, train linear probes for pitch, instrument sub-family (e.g., specific instrument within a family), and velocity/loudness on both zp and zs, using balanced accuracy. If the sub-family or velocity probe on zp is comparable to the pitch probe, or if the instrument probe on zs is not clearly dominant, the positive margins are confounded. Re-run over at least three seeds to check whether the reported 0.10/0.17 margins are stable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Sec. 2.2 defines cSEP as minimizing similarity between private latents (e.g., LCosineSEP = -E[log(1 - sim(zp1, zp2))]), with no term linking divergence to the intended varying factor. Sec. 3.1 constructs pairs from the same instrument family, 'assuming differences in all other factors.' Thus the separation objective can be satisfied by encoding any per-pair difference: pitch, velocity, sub-instrument identity, or recording artifacts. The reported evidence for pitch disentanglement is only the margin Delta_Pitch = PitchClf(zp) - PitchClf(zs) on a single model (CosineSEP, lambda=0.6), with no error bars. Because NSynth instrument families are coarse (e.g., brass contains trumpet and tuba, which have strongly different pitch ranges), a private subspace encoding sub-instrument identity would yield high pitch-probe accuracy without representing pitch per se. Similarly, loudness or other continuous cues correlated with pitch could inflate Delta_Pitch. Without probing zp and zs for non-target factors, the central claim that the framework 'disentangles music attributes' is not secured; the objective only guarantees divergence, not semantic specificity.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a multi-view self-supervised learning framework for music audio representations that combines a reconstruction objective (MSE on AudioMAE latents) with contrastive-style disentanglement objectives. The framework splits each input latent into shared and private subspaces; similarity-based objectives align shared latents across same-instrument pairs, while separation-based objectives push private latents apart. The authors systematically compare three contrastive principles (InfoNCE, cosine similarity, VICReg) and three objective types (SIM, SEP, SIM+SEP) across six reconstruction weights, yielding 54 configurations. On NSynth, they report that separation-based objectives are key to achieving positive disentanglement margins (difference between subspace-level downstream accuracies), and that combining them with reconstruction preserves overall classification accuracy. Their headline configuration, CosineSEP with lambda=0.6, improves overall pitch and instrument accuracy over contrastive baselines by 6pp and 13pp, respectively, while maintaining accuracy comparable to reconstruction-only training.","tokens_in":9052,"tokens_out":4679,"duration_ms":59255,"significance":"If the findings hold, the paper offers a useful empirical recipe for balancing information preservation and subspace specialization in audio SSL, and it identifies separation-based losses as the more effective disentanglement incentive compared with similarity-based losses. The controlled setup (stratified splits, 54 configurations, frozen AudioMAE encoder) is a definite strength, and the paper ships a clear experimental grid. However, the central claim that the method disentangles pitch and instrument as specific attributes is not yet secured: the separation loss is attribute-agnostic, the disentanglement metric is a self-defined proxy, and the quantitative results lack error bars. The contribution is therefore a promising empirical study whose main qualitative insight needs stronger validation before the exact claims can be accepted.","major_comments":[{"comment":"The central claim 'disentangles music attributes' is load-bearing and currently under-supported.","section":"Sec. 2.2, Eq. (4); Sec. 3.1"},{"comment":"This is a reproducibility and statistical-support issue for the exact quantitative claims.","section":"Sec. 3.2, Eq. (5); Table 1"},{"comment":"This weakens the interpretation of the primary evidence for disentanglement.","section":"Sec. 3.5"},{"comment":"This affects the scope of the 'self-supervised' contribution.","section":"Sec. 3.1"}],"minor_comments":[{"comment":"Minor indexing issue.","section":"Sec. 2.2, Eq. (2)"},{"comment":"Presentation issue.","section":"Table 1"},{"comment":"Figure clarity.","section":"Figure 2"},{"comment":"Figure readability.","section":"Figure 3"},{"comment":"Statistical presentation.","section":"Figure 4"},{"comment":"Reproducibility detail.","section":"Sec. 3.4"}],"recommendation":"major_revision","confidential_remarks":"This is a systematic and potentially useful empirical study, but the main claim about attribute-level disentanglement is currently supported only by an indirect proxy and a single-run configuration. The missing gamma value and lack of error bars are fixable presentation/reproducibility issues. The confound concern (private subspace encoding sub-instrument or other differences) is the most serious issue; if the authors can add subspace-level probes for non-target factors and show that Delta_Pitch/Delta_Instrument reflect the intended attributes, the paper would be suitable for publication. I encourage you to request a major revision rather than reject, as the study's controlled setup and negative/positive trade-off findings are valuable to the community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis paper is a controlled, systematic study of how reconstruction and contrastive objectives interact in a multi-view SSL framework for music audio, using NSynth monophonic notes. The main new result: separation-based objectives (pushing private latents apart) are what produce positive disentanglement margins, while reconstruction is what keeps downstream accuracy from collapsing. That is a useful design insight and I have not seen it tested this directly.\n\nWhat the paper does well: 54 configurations across three contrastive families (InfoNCE, Cosine, VICReg), two objective types (SIM, SEP, SIM+SEP), and six reconstruction weights; stratified train/val/test splits; and an honest discussion of the trade-off. The finding that objective type matters more than contrastive family is clearly supported by the pattern across configurations.\n\nThe soft spots are real, though. The separation loss is attribute-agnostic: it only pushes private latents apart, with no mechanism tying that divergence to pitch. Pairs are built from same instrument family, assuming all other factors differ. On NSynth, 'brass' contains trumpet and tuba, which have very different pitch ranges, so the private subspace could be encoding sub-instrument identity or loudness rather than pitch. The positive Delta_Pitch then is not evidence of pitch disentanglement. The paper does not probe zp or zs for non-target factors. I think this is the main load-bearing weakness and it limits the central claim. Also: single runs without error bars, the gamma scaling factor is unreported, code/data missing, and no comparison to prior pitch-timbre disentanglement baselines. The 'fully self-supervised' framing is overstated because instrument labels are used for pairing.\n\nThat said, the paper should not be dismissed. The qualitative claim that separation drives divergence and reconstruction preserves information holds up as an empirical pattern, even if the semantic specificity is unproven. This is a reasonable workshop or conference contribution that asks the right question. It needs revision: run multiple seeds, report gamma, add confound probes (velocity, sub-instrument classification on the private latent), and compare with Luo et al. and Tanaka et al.\n\nI would send this to peer review. The question matters, the experiment is careful enough to learn from, and the weaknesses are fixable. I would not yet cite the positive margins as evidence of pitch disentanglement until the confound probes are done.","headline":"A careful but confound-sensitive empirical study: separation objectives create divergence and reconstruction preserves accuracy, but pitch disentanglement is not yet established without factor-specific probes.","tokens_in":9566,"tokens_out":2945,"would_cite":true,"duration_ms":37605,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Separation objectives plus reconstruction disentangle pitch and instrument in music representations without sacrificing downstream accuracy.","keywords":["self-supervised learning","music representation learning","disentanglement","contrastive learning","reconstruction","multi-view learning","pitch and instrument"],"falsifier":"Train the same framework with pairs constructed to share pitch but differ in instrument; if the separation objective then produces a positive instrument disentanglement margin instead of a pitch margin, the central claim about semantic disentanglement is supported, but if the private subspace continues to encode pitch or simply random variation, the claim that separation objectives isolate the varying semantic factor is falsified.","tokens_in":1424,"feed_emoji":"🎵","tokens_out":2013,"duration_ms":40094,"temperature":0.7,"pith_summary":"This paper asks whether contrastive and reconstructive self-supervised objectives can coexist in one music-audio representation learner. It proposes a multi-view autoencoder with shared and private latent subspaces, and shows that separation-based objectives on the private subspace create pitch and instrument disentanglement while reconstruction preserves information. The central experimental finding is that the best configuration, CosineSEP at lambda 0.6, yields positive disentanglement margins for both pitch and instrument while matching the accuracy of a reconstruction-only model and beating contrastive baselines by 6 to 13 percentage points. A sympathetic reader would take this as evidence that information fidelity and structured semantics are not inherently in opposition, provided the disentanglement objective is chosen carefully.","feed_headline":"Separation objectives untangle pitch from instrument in music AI","feed_subtitle":"A multi-view SSL framework shows that reconstruction plus separation yields disentangled subspaces without losing downstream accuracy.","key_machinery":"The central object is a multi-view autoencoder with two factorized latent subspaces: a shared encoder e_s and a private encoder e_p, whose concatenation feeds a decoder trained with mean-squared error reconstruction. Disentanglement is driven by contrastive-style objectives applied to these subspaces: similarity objectives align shared latents across paired views, separation objectives push private latents apart, and a loss-weighting parameter lambda controls the reconstruction-versus-disentanglement balance. The authors find that the separation objective type, not the specific contrastive principle (InfoNCE, cosine similarity, or VICReg), is the decisive factor for positive disentanglement margins.","core_discovery":"The paper's central claim is that, when combined effectively, reconstruction and contrastive strategies complement each other, enabling disentanglement of music attributes without compromising information integrity. Concretely, the proposed multi-view framework, which applies a separation objective to private latents and a reconstruction objective to the concatenated shared-plus-private latent, produces semantically structured subspaces: pitch classification is better from the private latent, instrument classification from the shared latent, and the combined latent retains overall accuracy comparable to a reconstruction-only system. The authors report that this hybrid approach improves pitch disentanglement margin by 10 percentage points and instrument disentanglement margin by 16 percentage points over the reconstruction-only baseline, while outperforming the contrastive baselines by 6pp on pitch accuracy and 13pp on instrument accuracy.","pith_inferences":["A testable extension: if the same separation objective were applied with pitch as the common pairing factor instead of instrument, the private subspace should encode instrument rather than pitch, inverting the observed margin pattern.","A likely extension is to replace the frozen AudioMAE encoder with a trainable encoder, which may allow the disentanglement objective to shape the input representation itself rather than only reorganizing a fixed latent.","The finding that similarity objectives hurt instrument disentanglement while separation objectives help suggests that conventional contrastive learning may inadvertently discard the very variations that define a semantic attribute; this could inform contrastive design in other audio domains.","The authors' observation that reconstruction quality correlates with downstream accuracy implies that lightweight reconstruction heads could serve as a proxy for representation quality in SSL audio training, potentially reducing the need for labeled evaluation sets."],"forward_implications":["If the central claim holds, self-supervised music representation learners can be designed to produce interpretable subspaces for downstream tasks without labeling data at training time.","The result suggests a design recipe: use a reconstruction objective to preserve information and a separation objective on the private subspace to create disentanglement, rather than relying on similarity-only contrastive learning.","The finding that separation objectives squeeze irrelevant information out of the optimized subspace implies that subspace-level task accuracy can be used as a diagnostic tool for what a contrastive objective is actually encoding.","The framework extends naturally to any paired-view setting with a known common factor, which could be applied to cover songs, tempo, or other musical attributes beyond instrument and pitch.","The reported trade-off curve, where higher disentanglement margins often come with lower overall accuracy, provides a quantitative map for selecting the operating point of a music SSL model."],"supporting_citations":[{"why":"AudioMAE is the frozen pre-trained audio encoder whose latent embeddings are the input to the multi-view framework.","marker":"[1]"},{"why":"InfoNCE is the contrastive baseline and the basis for the contrastive disentanglement objectives used in the paper.","marker":"[6]"},{"why":"VICReg is one of the three contrastive principle families whose similarity and separation variants are compared.","marker":"[9]"},{"why":"The authors' prior multi-view work provides the foundation for the paired-view disentanglement setup extended here.","marker":"[28]"},{"why":"NSynth supplies the controlled dataset with well-defined generative factors of pitch and instrument used for pairing and evaluation.","marker":"[31]"},{"why":"BYOL is cited as the non-contrastive objective family from which the cosine similarity disentanglement objectives are derived.","marker":"[34]"}],"fun_headline_variants":["SSL objectives complement to disentangle music without losing accuracy","Reconstruction and contrastive strategies combine to improve music SSL","Music SSL: Hybrid approach preserves info and separates attributes","Unified SSL framework boosts disentanglement and keeps fidelity","Hybrid SSL separates pitch and instrument without accuracy loss"],"cache_read_input_tokens":11776,"weakest_assumption_plain":"The pairing construction assumes that instrument is the only shared factor between views, so the separation objective is implicitly trusted to push pitch, rather than arbitrary per-sample variation or a confound like velocity, into the private subspace.","fun_headline_variants_meta":{"raw":{"variants":["SSL objectives complement to disentangle music without losing accuracy","Reconstruction and contrastive strategies combine to improve music SSL","Music SSL: Hybrid approach preserves info and separates attributes","Unified SSL framework boosts disentanglement and keeps fidelity","Hybrid SSL separates pitch and instrument without accuracy loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001259,"raw_usage":{"total_tokens":5102,"prompt_tokens":834,"completion_tokens":4268,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":450,"completion_tokens_details":{"reasoning_tokens":4191}},"tokens_in":450,"tokens_out":4268,"duration_ms":32272,"temperature":1.0,"reasoning_tokens":4191,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T11:09:47.872626+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same framework with pairs constructed to share pitch but differ in instrument; if the separation objective then produces a positive instrument disentanglement margin instead of a pitch margin, the central claim about semantic disentanglement is supported, but if the private subspace continues to encode pitch or simply random variation, the claim that separation objectives isolate the varying semantic factor is falsified.","supporting_citations":[{"cited_title":"Masked autoencoders that listen,","cited_arxiv_id":null,"evidence_quote":"AudioMAE is the frozen pre-trained audio encoder whose latent embeddings are the input to the multi-view framework."},{"cited_title":"Self-Supervised Multi-View Learning for Disentangled Music Audio Representations","cited_arxiv_id":"2411.02711","evidence_quote":"The authors' prior multi-view work provides the foundation for the paired-view disentanglement setup extended here."},{"cited_title":"Neural audio synthesis of musical notes with wavenet autoencoders,","cited_arxiv_id":null,"evidence_quote":"NSynth supplies the controlled dataset with well-defined generative factors of pitch and instrument used for pairing and evaluation."},{"cited_title":"Bootstrap your own latent-a new approach to self-supervised learning,","cited_arxiv_id":null,"evidence_quote":"BYOL is cited as the non-contrastive objective family from which the cosine similarity disentanglement objectives are derived."}],"review_version":1}