{"id":"ea00e0e6-7566-48c6-863f-dc785a303af6","arxiv_id":"2501.01130","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper derives a constant-additional-risk condition for label-noise robustness of any contrastive loss and proposes SymNCE, an InfoNCE variant that meets this condition in the infinite-sample limit.","lead":"This paper proposes a unified theoretical framework for making supervised contrastive losses robust to label noise, and introduces a new loss called Symmetric InfoNCE (SymNCE). The framework says a loss is robust when its 'additional risk' is constant, and it shows that adding a reverse InfoNCE term makes the popular InfoNCE loss satisfy this condition.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The proof that SymNCE satisfies the robust condition is not rigorous: Theorem IV.1 mishandles the log-K normalization, and the constancy of ΔR is shown only for M,K→∞, while Theorem III.5 requires exact constancy for the finite loss, so the central 'provably robust' claim overreaches.","rationale":"The reader's weakest_assumption is the class-balanced condition in Theorem III.4, which is indeed a clear scope limitation: the decomposition into αR + βΔR uses π_i = 1/C, and the Clothing1M experiment violates it. I agree that this is a real limitation and contributes to a CONDITIONAL verdict. However, I see a more direct, load-bearing weakness at the point where the theory is connected to SymNCE: the proof of Theorem IV.1, which is supposed to establish that SymNCE satisfies the constant-ΔR condition, has a log-K normalization error and only holds in the infinite M,K limit. This affects the theoretical guarantee even for the balanced benchmark experiments, because the actual loss used in training has finite K and M (e.g., batch size 512). The finite-K additional risk is demonstrably non-constant, so Theorem III.5 does not apply as stated. The paper does not provide a limit version of Theorem III.5 or a finite-sample perturbation argument, so the 'provably robust' terminology is stronger than what the proof supports. This is a rigor gap in the central argument, not just an experimental mismatch. That said, the algebraic decomposition in Lemma III.3 and Theorem III.4 appears correct under the stated assumptions, the framework is genuinely inclusive of NN selection and RINCE, and the RINCE parameter byproduct is a useful insight. The issues are addressable with a corrected normalization, a clear limit theorem, and a discussion of finite-K deviations, which fits a CONDITIONAL verdict. Since the reader already recommended CONDITIONAL, my read does not change the verdict, though it sharpens the reason: the load-bearing gap is the finite-limit transition and the Theorem IV.1 algebra, in addition to the class-balance restriction.","tokens_in":1173,"tokens_out":2710,"duration_ms":277974,"concrete_test":"Evaluate Theorem IV.1 on a two-point similarity distribution, for both a constant encoder (all similarities equal to 1) and an encoder with s_+ = 1, s_- = −1. Compute ΔR(L_SymNCE;f) directly for K=M=1 and take the K→∞ limit after subtracting logK. If the finite-K value is not constant in f and the limit is 0 rather than logK, the proof of Theorem IV.1 is internally inconsistent and the finite-batch application of Theorem III.5 fails; the authors would then need to supply a limit theorem or a finite-K perturbation bound to support the 'provably robust' claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SymNCE is provably robust rests on showing that its additional risk ΔR is constant, since Theorem III.5 requires ΔR(L;f) = A for all f. However, the paper's verification of this condition has two gaps. First, the proof of Theorem IV.1 is algebraically inconsistent in its treatment of the log-K shift. Direct computation for the constant encoder f(x) ≡ 1 gives ΔR(L_InfoNCE) = log(K+1), ΔR(L_RevNCE) = 0, so ΔR(L_SymNCE) = log(K+1); hence lim_{K→∞}[ΔR(L_SymNCE) − logK] = 0, not logK as claimed in Eq. (27). The theorem statement even redefines ΔRlim(L_InfoNCE) without the −logK shift used in Eq. (22), so the normalization is internally inconsistent. Second, even with the corrected constant, the identity holds only in the infinite M,K limit. For finite K and M, ΔR(L_SymNCE;f) is not constant: for K=M=1, ΔR(L_SymNCE;f) = E[log(1+exp(s_- − s_+)) + (s_- − s_+)], which depends on f through the distribution of the similarity difference. The paper provides no finite-sample bound, no uniform convergence argument, and no limit version of Theorem III.5 that would justify applying the exact robustness condition to the normalized limit risks. Therefore, the statement that SymNCE is 'provably robust' is not established for the finite batches used in the experiments, and the proof of the key identity contains a concrete normalization error.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a theoretical framework for supervised contrastive losses under symmetric label noise. Its main theoretical result is a decomposition of the noisy contrastive risk into a clean-risk term and an additional-risk term (Lemma III.3 and Theorem III.4), leading to a sufficient robustness condition: a contrastive loss is noise tolerant if the additional risk is constant over representation functions (Theorem III.5). On this basis the paper argues that InfoNCE is non-robust, proposes a symmetrized variant SymNCE obtained by adding a Reverse InfoNCE (RevNCE) term, and claims that SymNCE is provably robust. It further presents the framework as an inclusive explanation of nearest-neighbor sample selection and of the RINCE loss, and reports experiments on CIFAR, Tiny ImageNet, and Clothing1M showing favorable accuracy.","tokens_in":25403,"tokens_out":11205,"duration_ms":102246,"significance":"If the robustness guarantee for SymNCE were rigorously established, the paper would provide one of the first general criteria for verifying noise tolerance of arbitrary contrastive losses, together with a constructive recipe for turning a non-robust loss into a robust one. The algebraic decomposition in Theorem III.4 is a useful contribution on its own, and the derived parameter choice λ = 1/(K + 1) for RINCE is a crisp, falsifiable byproduct that the paper tests experimentally. The empirical results are encouraging and suggest that the SymNCE loss has practical value even if the theoretical claim needs repair. However, the advertised 'provably robust' statement is not currently backed by a correct proof, and the supporting NN-selection explanation is largely asserted rather than derived.","major_comments":[{"comment":"The normalization of ΔRlim is internally inconsistent. Eq. (22) defines ΔRlim(LInfoNCE; f) as lim_{M,K→∞}(ΔR(LInfoNCE; f) − log K), while Theorem IV.1 defines ΔRlim(LInfoNCE; f) as lim_{M,K→∞} ΔR(LInfoNCE; f) without the log K shift, and then concludes ΔRlim(LRevNCE; f) = −ΔRlim(LInfoNCE; f) + log K. The two definitions cannot both be correct, and the conclusion is false under either reading. For the constant encoder f ≡ 1, direct computation gives ΔR(LInfoNCE) = log(K + 1), ΔR(LRevNCE) = 0, and hence ΔR(LSymNCE) = log(K + 1); therefore lim_{K→∞}(ΔR(LSymNCE) − log K) = 0, not log K as claimed in Eq. (27). With the normalized definition of ΔRlim used in Eq. (22), the correct identity is ΔRlim(LRevNCE) = −ΔRlim(LInfoNCE); the extra log K term should not appear. This error directly undermines the proof that SymNCE satisfies the constant-additional-risk condition.","section":null},{"comment":"Theorem III.5 requires the exact finite-batch additional risk ΔR(L; f) to be constant in f, but the derivation preceding the noise-tolerance claim for SymNCE only establishes a statement about the M,K → ∞ limit. No finite-sample bound, uniform-convergence argument, or limit version of Theorem III.5 is provided, so the gap is load-bearing. For finite batches the constancy actually fails: when K = M = 1, ΔR(LSymNCE; f) = E[log(1 + exp(f(x)⊤(f(x⁻) − f(x⁺))))], which generally depends on f. Consequently, the conclusion that 'LSymNCE is noise tolerant' is not justified for the finite-batch loss used in the experiments, and the phrase 'provably robust' overstates what the paper proves.","section":null},{"comment":"The claim that InfoNCE is non-robust is also based on the limit object rather than on the exact condition of Theorem III.5. The paper shows that ΔRlim(LInfoNCE; f) is non-constant in f, but Theorem III.5 concerns exact constancy of the finite additional risk. To prove non-robustness under the paper's own criterion, one should show that ΔR(LInfoNCE; f) is not constant for finite M and K; non-constancy of the limit does not logically imply non-constancy of the finite quantities. This is probably true and could be shown by a simple counterexample, but the manuscript currently asserts it rather than proves it.","section":null},{"comment":"The NN-selection explanation is asserted rather than proved. The text states that for a given f one can select a threshold t so that lim_{M,K→∞}(ΔR(LInfoNCE-NN; f, t) − log K) = 0, and concludes that LInfoNCE-NN is noise tolerant. No construction of t is given, t would depend on the unknown encoder f, and the exact equality to zero is not shown to be achievable. As in the SymNCE argument, the limit statement is not sufficient for Theorem III.5, which requires exact constancy. This section should be reframed as a heuristic explanation or supported by a rigorous argument with explicit conditions on t and f.","section":null}],"minor_comments":[{"comment":"The proof writes 'if γ ≤ (C − 1)/C' where the theorem states the strict inequality γ < (C − 1)/C; strictness is needed for the coefficient of R(L; f) to be positive, so the proof's inequality should be corrected.","section":null},{"comment":"There is a typo, 'ecall' should be 'recall', and the intuitive approximations in Eqs. (28) and (29) drop log M or log K constants without comment; the approximations are useful for intuition but the constants matter for the exact robustness argument.","section":null},{"comment":"The verification of the RINCE parameter λ = 1/(K + 1) does not state which K (number of negatives) is used in Figure 2c; since K determines the theoretical value, this missing detail makes the empirical confirmation difficult to check.","section":null},{"comment":"The display in Eq. (23) is garbled: the term f(x)⊤f(x⁻) appears to be placed inside the expectation with the logarithm in an ambiguous way. The expression should be rewritten as E_{x,x⁻}[f(x)⊤f(x⁻)] − E_x[log E_{x⁺} exp(f(x)⊤f(x⁺))] or an equivalent unambiguous form.","section":null}],"recommendation":"major_revision","confidential_remarks":"The central risk decomposition appears sound and the empirical results are interesting, but the paper's headline claim that SymNCE is provably robust is not currently supported: Theorem IV.1 contains a normalization error, and the passage from the M,K→∞ limit to the exact finite-batch condition required by Theorem III.5 is missing. I recommend major revision rather than rejection because the framework is potentially repairable, but the revised version must correct the normalization, state precisely which asymptotic statement is proved, and either weaken the robustness claim accordingly or supply the missing finite-sample analysis."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on Cui et al. The paper has a genuinely useful core: a clean decomposition of the noisy contrastive risk into a clean risk plus an additional risk under symmetric label noise, and a general robustness criterion (constant additional risk) that unifies several existing heuristics. The construction of SymNCE by adding a reverse InfoNCE term is clever, and the experiments show real gains over SupCon on several benchmarks. I'd recommend the framework sections to anyone working on noisy-label contrastive learning.\n\nBut the central 'provably robust' claim is not proven. The stress-test note is right: the proof of Theorem IV.1 mishandles the log-K normalization. For a constant encoder, ΔR(InfoNCE)=log(K+1), ΔR(RevNCE)=0, so the claimed ΔRlim(SymNCE)=logK should be 0 (or the shifts are inconsistent). More fundamentally, the robustness is shown only in the M,K→∞ limit, while Theorem III.5 requires exact constancy of ΔR for the finite loss. For finite K,M, ΔR(SymNCE) is a function of f—for K=M=1, it's E[log(1+e^{s_- - s_+}) + (s_- - s_+)], which is not constant. There is no limit version of Theorem III.5 or finite-sample bound to bridge this gap. So the headline claim overreaches.\n\nThe other soft spots are proportionate: InfoNCE non-robustness is asserted without a concrete counterexample; the NN-selection explanation is post hoc and nearly tautological; the class-balance assumption is restrictive, though the Clothing1M subset is balanced by construction, which mitigates that concern. The experiments are selective—on CIFAR-10 at 80% symmetric noise, RINCE (80.32) beats SymNCE (60.74) by a wide margin, and that case is not discussed.\n\nOverall, the framework is a real contribution and the authors are clearly thinking hard about the problem. But the main proof has a concrete error, so this needs revision before the robustness claim can be accepted. I'd send it to a serious referee—the core idea deserves scrutiny—but the authors should be asked to fix the normalization, provide a limit version of the robustness theorem or a finite-sample analysis, and report the failure cases honestly.","headline":"A genuinely useful framework with a clever SymNCE construction, but the central robustness proof has a log-K normalization error and only holds in an infinite limit, so 'provably robust' overreaches.","tokens_in":25962,"tokens_out":9420,"would_cite":false,"duration_ms":70981,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper identifies a single condition that decides whether a supervised contrastive loss survives symmetric label noise, proves InfoNCE fails it, and constructs SymNCE that passes it.","keywords":["supervised contrastive learning","label noise","robust loss function","InfoNCE","symmetric label noise","additional risk","noise tolerance","representation learning"],"falsifier":"Run SymNCE on a synthetic two-class problem with class proportions 0.9/0.1 under symmetric 40% label noise and check whether the minimizer of the noisy contrastive risk still equals the minimizer of the clean risk; if the learned representation or downstream accuracy shifts measurably when class imbalance is introduced, the decomposition carrying the robustness proof has broken. A direct check would be to compute ΔRlim(SymNCE; f) on that imbalanced distribution for two different representation functions and show that it is not constant.","tokens_in":24804,"feed_emoji":"🎯","tokens_out":4451,"duration_ms":43431,"temperature":0.7,"pith_summary":"The paper tries to turn robust supervised contrastive learning under label noise from a collection of heuristics into a checkable criterion. It shows the risk of any contrastive loss under symmetric label noise can be split into the clean risk plus an additional risk term, and that if this extra term is a constant independent of the learned representation, minimizing the noisy risk also minimizes the clean risk. The popular InfoNCE loss fails this test, so the paper builds Symmetric InfoNCE (SymNCE) by adding a reverse term, RevNCE, that makes the additional risk constant. If the theory is right, practitioners can screen any pairwise contrastive loss for label-noise robustness before training, and previously heuristic fixes such as nearest-neighbour sample selection receive a principled explanation.","feed_headline":"InfoNCE fails a new label-noise test; SymNCE passes it","feed_subtitle":"A constant 'additional risk' makes a contrastive loss provably robust; the paper adds RevNCE to InfoNCE to meet it.","key_machinery":"The load-bearing object is the additional risk ΔR(L; f): the contrastive risk evaluated when anchor and positive samples are drawn independently from the data distribution rather than from the same latent class. Theorem III.4 proves that, under symmetric label noise and class balance, the noisy contrastive risk equals (1 − Cγ/(C−1))² R(L; f) plus Cγ/(C−1)(2 − Cγ/(C−1)) ΔR(L; f), so robustness reduces to checking whether the additional risk is a constant independent of f. The proposed SymNCE loss pairs InfoNCE with RevNCE, a loss that swaps the roles of the positive and negative sums; its defining property is that ΔRlim(RevNCE; f) = −ΔRlim(InfoNCE; f) + log K, making the combined loss satisfy ΔRlim(SymNCE; f) = log K.","core_discovery":"The paper's central claim is that, for class-balanced data under symmetric label noise with noise rate γ < (C−1)/C, a contrastive loss is noise tolerant whenever its additional risk ΔR(L; f) is a constant independent of the representation function f. The additional risk is defined by drawing the anchor, positive, and negative samples independently from the data distribution, so it measures how the loss behaves when the positive pair carries no real class information. Under these assumptions the noisy risk decomposes as an affine combination of the clean risk R(L; f) and ΔR(L; f), so a constant ΔR makes minimizing the noisy risk equivalent to minimizing the clean risk. The paper then shows InfoNCE is not robust because its limiting additional risk depends on f, and constructs SymNCE by adding a reverse loss RevNCE, whose limiting additional risk cancels the non-constant part and leaves ΔR = log K.","pith_inferences":["The constant-ΔR condition is sufficient but may not be necessary; a loss whose additional risk varies only mildly with f could still be nearly noise tolerant, and the M,K→∞ limit hides finite-batch effects that matter in real training.","The class-balanced assumption enters the proof where a(i) is set to 1/C, so a weighted reformulation of the noisy risk might extend the same guarantee to imbalanced datasets.","The reverse-term construction is a template: any contrastive loss with non-constant ΔR could be made robust by adding a suitably reversed version, provided the reverse term preserves the positive-alignment and negative-repulsion behaviour.","A testable extension would be to replace the uniform πi in the theorem with empirical class frequencies on imbalanced noisy data and check whether the predicted residual noise bias matches the observed gap in accuracy."],"forward_implications":["InfoNCE, the default supervised contrastive loss, is not robust to symmetric label noise, so using it directly can overfit corrupted positive pairs at high noise rates.","SymNCE inherits InfoNCE's alignment behaviour while adding a reverse term, so it can replace the loss in existing supervised contrastive pipelines without changing the rest of the training setup.","Nearest-neighbour positive selection works because restricting positives to high-similarity neighbours pushes the additional risk toward a constant, making the loss approximately noise tolerant.","RINCE is robust exactly when λ = 1/(K+1), a parameter choice the framework derives rather than tunes.","The criterion gives a general design rule: to make a contrastive loss robust, make its additional risk independent of the representation function."],"supporting_citations":[{"why":"Supplies the CURL risk formulation with latent classes that the paper adapts to define clean and noisy supervised contrastive risks.","marker":"[44]"},{"why":"Provides the limiting form of InfoNCE as alignment plus uniformity, which the paper uses to compute ΔRlim and show InfoNCE is non-robust.","marker":"[46]"},{"why":"Establishes the symmetric-loss robust condition for classification losses that the paper contrasts with its own positive-pair symmetrization condition.","marker":"[35]"},{"why":"Gives the RINCE robust contrastive loss, which the paper re-derives and interprets under its unified framework, including the parameter choice λ = 1/(K+1).","marker":"[20]"},{"why":"Defines the supervised contrastive (SupCon) loss that serves as the base loss InfoNCE and the empirical backbone for SymNCE.","marker":"[7]"}],"fun_headline_variants":["SymNCE: robust contrastive loss with constant additional risk","InfoNCE vulnerable to label noise; SymNCE fixes it","Constant additional risk: key to robust contrastive loss","Theoretical guarantee: SymNCE beats InfoNCE under label noise","New theory explains InfoNCE failure; SymNCE succeeds"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole guarantee assumes the training classes are balanced, πi = 1/C for every class; if class proportions are unequal, the noisy risk no longer decomposes as a constant-weighted sum of the clean risk and the additional risk, and the theorem's proof no longer applies.","fun_headline_variants_meta":{"raw":{"variants":["SymNCE: robust contrastive loss with constant additional risk","InfoNCE vulnerable to label noise; SymNCE fixes it","Constant additional risk: key to robust contrastive loss","Theoretical guarantee: SymNCE beats InfoNCE under label noise","New theory explains InfoNCE failure; SymNCE succeeds"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000606,"raw_usage":{"total_tokens":2818,"prompt_tokens":928,"completion_tokens":1890,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":1803}},"tokens_in":544,"tokens_out":1890,"duration_ms":11573,"temperature":1.0,"reasoning_tokens":1803,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:35:24.028697+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SymNCE on a synthetic two-class problem with class proportions 0.9/0.1 under symmetric 40% label noise and check whether the minimizer of the noisy contrastive risk still equals the minimizer of the clean risk; if the learned representation or downstream accuracy shifts measurably when class imbalance is introduced, the decomposition carrying the robustness proof has broken. A direct check would be to compute ΔRlim(SymNCE; f) on that imbalanced distribution for two different representation functions and show that it is not constant.","supporting_citations":[{"cited_title":"A theoretical analysis of contrastive unsupervised representation learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the CURL risk formulation with latent classes that the paper adapts to define clean and noisy supervised contrastive risks."},{"cited_title":"Understanding contrastive representation learning through alignment and uniformity on the hypersphere,","cited_arxiv_id":null,"evidence_quote":"Provides the limiting form of InfoNCE as alignment plus uniformity, which the paper uses to compute ΔRlim and show InfoNCE is non-robust."},{"cited_title":"Robust loss functions under label noise for deep neural networks,","cited_arxiv_id":null,"evidence_quote":"Establishes the symmetric-loss robust condition for classification losses that the paper contrasts with its own positive-pair symmetrization condition."},{"cited_title":"Robust contrastive learning against noisy views,","cited_arxiv_id":null,"evidence_quote":"Gives the RINCE robust contrastive loss, which the paper re-derives and interprets under its unified framework, including the parameter choice λ = 1/(K+1)."},{"cited_title":"Supervised contrastive learning,","cited_arxiv_id":null,"evidence_quote":"Defines the supervised contrastive (SupCon) loss that serves as the base loss InfoNCE and the empirical backbone for SymNCE."}],"review_version":1}