{"id":"906754f1-54a9-461f-8793-63370d769f71","arxiv_id":"2501.19145","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"MulSupConLD recovers a label distribution from binary multi-label data with RBF or contrastive weights and uses it to reweight supervised contrastive loss for multi-label classification.","lead":"This paper combines label distribution learning with multi-label contrastive learning: it tries to recover graded label importance from binary labels and use those weights to balance the contrastive loss. The reported gains over six metrics on nine datasets are the headline, but the experiments only partially support the claimed state-of-the-art advantage.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The recovered label distribution is anchored to binary logical labels by Eq. (8), with no independent graded signal; if d is only a projected y, the claimed advantage over MulSupCon reduces to binary reweighting.","rationale":"The paper's central claim is that recovering label distributions from logical labels and using them as weights improves multi-label contrastive learning. For that claim to hold, the recovered d must contain information beyond the logical labels. The weakest point is the recovery objective: Section 3 defines d on the simplex, while Eq. (8) penalizes raw L2 distance to binary y. The penalty either forces d toward a projection of y or, due to the scale mismatch, makes the objective internally inconsistent. The smoothing term G only propagates distribution similarity among samples that already share labels, so it cannot introduce graded importance that is absent from y. The empirical sections never inspect d, so one cannot tell whether the reweighting in Eqs. (12)-(13) is doing anything more than a normalization of y. My proposed test directly replaces d with normalized y; if metrics are unchanged, the central novelty is inert. I also note the CLD variant's C(2) sign issue, but the collapse concern is more fundamental because it affects both proposed variants and the core motivation. This supports the reader's REJECT verdict; I see no reason to change it.","tokens_in":14612,"tokens_out":11022,"duration_ms":122397,"concrete_test":"Run the Scene and PASCAL configurations of MulSupConCLD from Section 4.2 with the learned d in the weighting factor of Eqs. (12)-(13) replaced by the normalized logical label vector y_i/||y_i||_1, keeping all other losses and the pretraining/fine-tuning protocol unchanged. Compare the six metrics against the corresponding rows in Table 2 and Table 4; if the metrics stay within run-to-run noise, the label distribution carries no information beyond binary labels and the central claim fails. As a supporting diagnostic, also record the mean L2 distance between the learned d and the simplex projection of y over the training set; a value near zero confirms that Eq. (8) has collapsed the distribution.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central novelty is the recovered label distribution d used as graded weights in Eqs. (12)-(13). The recovery objective gives the model no graded supervision. Section 3 (Notations) requires 0 <= d_i <= 1 and sum_j d_i^j = 1, while logical labels y_i are binary with sum_j y_i^j equal to the number of positive labels. Eq. (8) then minimizes ||Qll - Qld||^2 between these incompatible objects; the optimum is a shifted/clipped projection of y onto the simplex, not an estimate of label importance. Eq. (7), and similarly Eq. (11) for CLD, only encourages samples that already share labels to have similar d, which propagates co-occurrence but cannot create importance information absent from y. Since the weight in Eqs. (12)-(13) is d_p^j * y_p^j, if d_p^j is approximately y_p^j plus a constant, the reweighting is essentially binary or a function of |y_p|. The paper reports no statistics on the recovered d, no comparison against using normalized logical labels, and no calibration check. The claimed 'varying importance' mechanism is therefore unsupported. A secondary internal symptom is that C(2) in Eq. (10) is a signed log-probability (typically negative), so G(2) in Eq. (11) can repel instead of attract distributions of positive pairs; but even fixing that sign would not supply the missing graded label information.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes MulSupConLD, a multi-label supervised contrastive learning method that replaces binary logical-label weights with an estimated label distribution d. Two recovery variants are introduced: MulSupConRLD uses RBF-kernel feature similarities (Eqs. 6-9), and MulSupConCLD uses the contrastive log-probability (Eqs. 10-11). The recovered distribution is used to reweight the contrastive loss in Eqs. (12)-(13). The framework follows an ANY-style positive sample selection with MoCo-style encoders, is pretrained, and is then fine-tuned with BCE. Experiments on four image datasets and five vector datasets report six metrics, with ablations and parameter sensitivity on Scene and PASCAL. The authors claim that the method outperforms state-of-the-art methods and that label distributions capture varying label importance.","tokens_in":14925,"tokens_out":13620,"duration_ms":134655,"significance":"The intended contribution is timely: if the recovered label distributions genuinely encoded per-label graded importance, this would be a lightweight improvement over MulSupCon and a novel bridge between label distribution learning and contrastive learning. The evaluation is broad in scope (nine datasets, six metrics), and the ablation and sensitivity studies are a positive feature. However, the paper provides no code, no direct analysis of the recovered distributions, and no control experiment that separates label-distribution reweighting from the extra model components. The mathematical form of the recovery objective, especially Eq. (8), indicates that d collapses to normalized logical labels rather than graded importance. Because this mechanism is the paper's only claimed novelty, the significance is not established by the current evidence.","major_comments":[{"comment":"The central claim is that the recovered distribution d encodes graded label importance. This is contradicted by the recovery objective: d is constrained to the probability simplex (sum_j d_i^j = 1), while the logical label y_i is a binary vector whose entries sum to the number of positive labels. Minimizing H_i = ||Q_ll - Q_ld||^2 therefore drives d_i toward the L2 projection of y_i onto the simplex, which for k positive labels is approximately d_i^j = 1/k for positive labels and 0 otherwise. That is exactly normalized logical labels, not graded importance. The G terms in Eqs. (7) and (11) only require positive pairs to have similar d, which is also satisfied by any vector that is constant on co-occurring label groups. The paper reports no statistics of the recovered d, no calibration check, and no comparison against the baseline d = y/|y|. Without such evidence, the reweighting in Eqs. (12)-(13) cannot be claimed to leverage label distribution rather than binary labels.","section":"Section 3.2, Eq. (8)"},{"comment":"C^(2)_{i,p} in Eq. (10) is the logarithm of a softmax probability, so it is non-positive (typically negative). Equation (11) adds G^(2)_i = sum_p C^(2)_{i,p} ||d_i - d_p||^2 to the loss; because the coefficient is negative, minimizing this term maximizes the squared distance between d_i and d_p for every positive pair. This is the opposite of the stated goal in Section 3.2 that label distributions of similar samples should be similar. The same C^(2) appears in Eqs. (12)-(13) with a leading minus sign, where the sign is conventional; the inconsistency is specific to the distribution-recovery loss of the CLD variant. The sign can be corrected, but the correction does not by itself supply graded supervision.","section":"Section 3.2, Eqs. (10)-(11)"},{"comment":"The proposed training loop is circular with respect to the claimed mechanism. d is produced by a fully connected layer on the same features z whose contrastive similarities define both C^(2) in the recovery loss and the final reweighted contrastive loss. Since no dataset provides ground-truth label distributions, the recovery objective is the only training signal for d, and it contains no external importance information. The reported end-task accuracy is an external benchmark, but it cannot distinguish the label-distribution hypothesis from the alternative that the extra fully connected parameters, the RBF/log-probability terms, and the regularizer beta||W||^2_F improve optimization independently of any graded label signal. A minimal control would replace d_p^j with y_p^j / |y_p| (or a random distribution) while keeping all other components, and the paper should report the divergence between recovered and logical-label-derived distributions; neither is present.","section":"Section 3.2 and Section 4.1"},{"comment":"The abstract states that the method outperforms state-of-the-art methods on six metrics across nine datasets, but the tables do not support an unqualified superiority claim. For example, Table 3 shows Scene mAP of MulSupConCLD at 0.821 versus 0.859 for MulSupCon; Table 2 shows Scene maF1 of MulSupConCLD at 0.766 versus 0.787 for MulSupCon; Table 4 shows MS-COCO miF1 of MulSupConRLD at 0.5865 versus 0.6245 for MulSupCon. Moreover, no standard deviations or significance tests are reported, so even the favorable differences, often a few tenths of a point, cannot be assessed for statistical reliability. The empirical claim needs to be qualified and supported with variance estimates or repeated-run comparisons.","section":"Section 4.1, Tables 2-4"}],"minor_comments":[{"comment":"The positive-sample definition is inconsistent: the introduction says 'we adopt the ANY strategy', but Section 3.2 says 'We use ALL to define positive samples' and Algorithm 1 uses mask <- y[i] intersect Qll, which is ANY. Please reconcile this, as it is needed for reproduction.","section":"Section 1 vs Section 3.2 and Algorithm 1"},{"comment":"MulSupCon is attributed to 'Zeng et al. (2024)' in the Introduction and to 'Zhang and Wu (2024)' in Section 2; the reference list contains both, and the correct source for MulSupCon appears to be Zhang and Wu.","section":"References and Related Work"},{"comment":"Figure 2 appears to be an unrendered placeholder: the caption refers to bar plots, but the figure content is a plain list of metric names and values. The ablation results should be presented as actual plots or tables with clear group labels.","section":"Figure 2"},{"comment":"The relationship between the recovered distribution D and the final classification head is underspecified. The text says the fully connected layer generating label distributions is later trained with BCE, but it does not state whether D is used at inference or only as a pretraining auxiliary target.","section":"Section 4, Settings"},{"comment":"The sensitivity tables would be easier to interpret if the default values of alpha and beta used for the main results were stated; the tables fix one parameter at a time, but the main configuration is never given.","section":"Section 4.3, Tables 5-8"}],"recommendation":"reject","confidential_remarks":"I agree with the reader's assessment. The central novelty is not supported by the paper's own equations: Eq. (8) drives the recovered distribution toward normalized logical labels, and the CLD variant has a sign error in the distribution-recovery objective. The empirical claims are also presented without variance estimates, and the abstract's wording overstates what the tables show. I do not see a minor-revision path, because fixing the recovery objective would require a different method (e.g., external graded supervision or a validated label-enhancement step) rather than local corrections. The manuscript might be suitable for a workshop if the authors add the normalized-logical-label control, distribution statistics, and corrected CLD objective."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Thanks for the report. I read the paper and agree with most of your assessment, with one nuance.\n\nWhat's new: the authors propose to use label distribution (soft relevance scores) to weight the supervised contrastive loss in multi-label learning, instead of the binary labels used by MulSupCon. They introduce two recovery methods, one based on RBF kernel similarities and one based on the contrastive logits themselves. That is a natural and, as far as I know, unexplored combination. The paper also includes a decent amount of experiments, an ablation, and sensitivity analysis. For a method paper, that's good practice.\n\nThe soft spots, however, are precisely where you pointed. The recovery of d is anchored to the binary logical labels via Eq. (8), which forces d toward a scaled version of y. Since d is normalized to sum to one and y is not, the optimum is just a projection of y onto the simplex—no graded label importance emerges except as a side effect of label counts. The paper offers no calibration check, no comparison against normalized binary weights, and no statistics on the recovered d. So the central mechanism is not established. Your stress-test note captures this well.\n\nThere is also a clear sign error in the CLD variant. Eq. (10) defines C^(2) as the log of a softmax probability, which is almost always negative. Consequently, G^(2) in Eq. (11) is a negative coefficient multiplying a squared distance, so minimizing it pushes distributions of positive pairs apart rather than together. That is not a subtle issue; it's a direct contradiction of the stated intent. It might explain why CLD 'outperforms' RLD, but not in a way that makes sense.\n\nThe empirical section is also weaker than the abstract claims. Image datasets are compared only against MulSupCon; the vector baselines are partly borrowed from C-GMVAE; no error bars or significance tests. For a paper claiming six-metric superiority, that's insufficient.\n\nI agree with your verdict that the paper should be rejected in its current form. The idea is worth a second look, but only after the recovery mechanism is redesigned or at least rigorously evaluated. I would still send it to peer review, though—this is exactly the kind of paper where a good referee can help the authors fix the sign error and add the missing baseline analysis. It shouldn't be desk-rejected; it needs a serious conversation.\n\nI won't cite it as is. Would I bring it to a reading group? Maybe—there's a useful discussion here about label distribution learning and why unsupervised recovery is hard.","headline":"Promising combination of label distribution learning and multi-label contrastive learning, but the recovery mechanism lacks graded supervision, one core loss term has a sign error, and the empirical claims are not sufficiently supported; worth a careful review but not acceptable as is.","tokens_in":15450,"tokens_out":5375,"would_cite":false,"duration_ms":52040,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multi-label contrastive learner that recovers label distributions from logical labels and uses them as loss weights outperforms state-of-the-art methods on six metrics across nine datasets.","keywords":["multi-label learning","contrastive learning","label distribution","label enhancement","supervised contrastive loss","representation learning","RBF kernel","label importance weighting"],"falsifier":"Train MulSupConCLD on a synthetic multi-label dataset whose true label distributions are known (for example, drawn from a Dirichlet distribution over labels that are marked present when the weight exceeds a threshold), then compare each recovered distribution to the true one with KL divergence or rank correlation. If the recovered distributions are indistinguishable from one-hot logical vectors, the reweighting in Equations (12) and (13) reduces to binary weighting, and replacing them with logical-label weights should reproduce the same performance.","tokens_in":14395,"feed_emoji":"🏷️","tokens_out":12390,"duration_ms":106343,"temperature":0.7,"pith_summary":"Multi-label contrastive learning typically treats every present label as equally relevant, which throws away information about how strongly a label applies to an instance. This paper proposes MulSupConLD, which recovers a soft label distribution from binary logical labels and uses those graded importance scores to reweight a multi-label supervised contrastive loss. The recovery is done either through radial-basis-function similarities or through the contrastive similarity itself, yielding the variants MulSupConRLD and MulSupConCLD. The paper reports that the contrastive variant in particular outperforms prior multi-label contrastive and non-contrastive baselines on six metrics across nine image and vector datasets. If the claim holds, binary labels are enough to obtain much of the benefit of explicit label distributions in contrastive representation learning.","feed_headline":"Recovering label distributions boosts multi-label contrastive learning","feed_subtitle":"Binary labels miss how much each label matters; estimated label distributions reweight the loss and win on nine datasets","key_machinery":"The load-bearing object is the recovered label distribution $d_i$, a per-instance probability vector over the label set that encodes graded relevance. It is produced by a fully connected layer with parameters $W$, and is constrained by three terms: $G^{(k)}_i = \\sum_{p \\in P(i)} C^{(k)}_{i,p} \\|d_i - d_p\\|_2^2$ asks contrastive positives to agree on label distributions; $H_i = \\|Q^{ll}_p - Q^{ld}_p\\|_2^2$ keeps the stored distributions close to the stored logical labels; and $\\beta\\|W\\|_F^2$ controls model complexity. The two variants differ only in the similarity function $C^{(1)}$ (RBF) versus $C^{(2)}$ (contrastive log-probability). These definitions feed directly into the final reweighted losses of Equations (12) and (13), so the learned distribution is the mechanism by which graded label importance enters contrastive learning.","core_discovery":"The central claim is that positive-sample selection is not the main bottleneck in multi-label supervised contrastive learning; the way label information is weighted inside the loss is. MulSupConLD keeps the simple ANY criterion for positives — samples sharing at least one label — and replaces binary indicator weights with a recovered label distribution $d_i$ over the label set, normalized so that $\\sum_j d^j_i = 1$. Two estimators are trained jointly with the contrastive loss: one uses an RBF kernel $C^{(1)}_{i,p} = \\exp(-\\|z_i - z_p\\|_2^2 / 2\\sigma^2)$ and one uses the contrastive log-probability $C^{(2)}_{i,p} = \\log \\frac{\\exp(\\mathrm{sim}(z_i, z_p)/\\tau)}{\\sum_{a \\in A(i)} \\exp(\\mathrm{sim}(z_i, z_a)/\\tau)}$. Both enforce that positive samples share similar label distributions, while a constraint term keeps the recovered distribution near the original logical labels. The final loss multiplies each positive pair's contribution for label $j$ by $d^j_p \\cdot y^j_p$, so labels estimated as more important exert more influence; the paper shows that this reweighting improves performance over the binary-weighted MulSupCon baseline on image and vector datasets.","pith_inferences":["Because Equation (8) actively penalizes deviation from the binary logical labels, the recovered distribution is likely better interpreted as a relative importance ranking than as a calibrated probability; a direct check would compare it to ground-truth soft labels on synthetic data.","The same distribution-reweighting idea transfers to single-label contrastive learning whenever labels carry confidence, such as noisy labels or teacher-soft targets, where the binary mask currently discards confidence information.","The long-tail direction suggested in the conclusion could be tested immediately by inspecting whether the recovered distributions assign higher weights to rare labels; if not, the reweighting may be amplifying head-class dominance rather than correcting it.","The ANY positive criterion combined with distribution weights leaves open a softer variant where positive weight is proportional to distribution overlap, which could improve recall on ambiguous multi-label instances without hurting precision."],"forward_implications":["Any supervised contrastive learner for multi-label tasks can adopt the recovered distribution as drop-in per-label weights, bypassing hand-designed positive-sample selection rules.","The consistent edge of MulSupConCLD over MulSupConRLD suggests that the contrastive similarity itself is a more informative signal for distribution recovery than a fixed RBF kernel with a hand-set bandwidth.","Because the same fully connected layer generates both the label distribution used in pretraining and the final classifier, the pretraining and downstream classification stages share a graded-label representation rather than a binary one.","The gains span image datasets (trained with a convolutional encoder) and vector datasets (trained with a multilayer perceptron), indicating the mechanism is not tied to one data modality or encoder family."],"supporting_citations":[{"why":"Defines the MulSupCon loss and the ALL/ANY positive-sample rules whose binary weighting this paper replaces.","marker":"Zhang and Wu (2024)"},{"why":"DLDL supplies the template for recovering label distributions from logical labels, including the RBF similarity and the distribution constraints.","marker":"Jia et al. (2023b)"},{"why":"Defines supervised contrastive learning and the loss form that the paper adapts to multi-label settings.","marker":"Khosla et al. (2020)"},{"why":"Provides the momentum encoder and queue used for the query/key representation pairs and stored label queues.","marker":"He et al. (2020)"},{"why":"Supplies the data augmentation and temperature parameter used in pretraining.","marker":"Chen et al. (2020)"},{"why":"Establishes label distribution learning and the claim that distributions generalize better than logical labels, motivating the method.","marker":"Geng (2016)"}],"fun_headline_variants":["Label distribution reweighting simplifies multi-label contrastive learning","Estimate label distributions, then reweight contrastive loss – beats complex sampling","Positive pairs weighted by label distribution outperform binary weighting","Why multi-label contrastive learning needs label distributions, not binary labels","Reweight positive pairs by estimated label importance – simple, effective"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the label distribution recovered while being penalized for deviating from the logical labels still carries genuinely graded importance information, rather than collapsing back into the binary labels it was trained to approximate.","fun_headline_variants_meta":{"raw":{"variants":["Label distribution reweighting simplifies multi-label contrastive learning","Estimate label distributions, then reweight contrastive loss – beats complex sampling","Positive pairs weighted by label distribution outperform binary weighting","Why multi-label contrastive learning needs label distributions, not binary labels","Reweight positive pairs by estimated label importance – simple, effective"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000981,"raw_usage":{"total_tokens":4180,"prompt_tokens":980,"completion_tokens":3200,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":596,"completion_tokens_details":{"reasoning_tokens":3114}},"tokens_in":596,"tokens_out":3200,"duration_ms":24564,"temperature":1.0,"reasoning_tokens":3114,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T21:09:56.716714+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train MulSupConCLD on a synthetic multi-label dataset whose true label distributions are known (for example, drawn from a Dirichlet distribution over labels that are marked present when the weight exceeds a threshold), then compare each recovered distribution to the true one with KL divergence or rank correlation. If the recovered distributions are indistinguishable from one-hot logical vectors, the reweighting in Equations (12) and (13) reduces to binary weighting, and replacing them with logical-label weights should reproduce the same performance.","supporting_citations":[],"review_version":1}