{"id":"f5031575-dec8-4e15-b733-524431377c94","arxiv_id":"2501.01844","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A class-wise positive-unlabeled risk estimator trains classifiers from ambiguous data with hard labels and beats label-noise baselines on synthetic mixed-image benchmarks.","lead":"The paper proposes a training method, Class-wise Positive-Unlabeled (CPU) risk, for image classifiers trained on ambiguous examples where each example has one hard label sampled from a hidden soft-label distribution. The method reports large accuracy gains on synthetic ambiguous versions of CIFAR and AFHQ, but the theoretical justification for the estimator is incomplete.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The CPU risk estimator is not unbiased under QLL because the unlabeled set is the class complement, not the mixture required by the PU identity.","rationale":"The reader's weakest_assumption identifies the same failure: Eq. (2) is imported without verifying that the unlabeled data are drawn from the mixture. In QLL, the class-wise unlabeled set is the complement of the positive set, so the mixture condition fails by construction. This is not a mere technicality: it invalidates the theoretical justification that gives the CPU method its name and its claimed advantage over label-noise baselines. The empirical gains in Tables I-II may still be real, but they would be unexplained by the PU theory, and the method would be a heuristic. I therefore agree with the reader's CONDITIONAL verdict: the paper should either prove unbiasedness under a corrected assumption (e.g., using all data as unlabeled, not the complement) or honestly reframe CPU as a heuristic loss with empirical support. The algorithm/equation mismatch (Eq. 5 vs Algorithm 1) and the absence of code are secondary but reinforce the need for conditional acceptance.","tokens_in":14814,"tokens_out":11370,"duration_ms":108553,"concrete_test":"Derive, for a fixed binary loss l and a two-class QLL model with known soft-label function s(x), the expectation of the CPU empirical risk (Eq. 4) when D_u is the complement {y≠j}. Compute E[π_p^(1) bR_p^+ + max{bR_u^- - π_p^(2) bR_p^-, 0}] and compare with both the PU risk R_pu (Eq. 2) and the supervised risk E[l(f(x),y)]. If E[bR_cpu] ≠ R_pu for any π_p^(1),π_p^(2), the unbiasedness claim is false. This check is purely analytical and requires no experiments.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim that CPU is a principled risk estimator rests on importing the PU identity (Eq. 2) from standard PU learning. That identity requires unlabeled data x^u to be drawn from the mixture p_pu(x) = π_p p_p(x) + π_n p_n(x). In the class-wise construction (Section III, 'separating all data into P and U data class-wisely'; Algorithm 1), the unlabeled set for class j is the complement D_j^u = {x_i : y_i ≠ j}, so x^u ~ p(x | y≠j). Under Definition 1, P(y=j|x)=s_j(x) is instance-dependent, so p(x|y≠j) = p(x)(1-s_j(x))/P(y≠j) is a biased reweighting of the input distribution, not the mixture p(x). Consequently Eq. (4) is not an unbiased estimator of any classification risk under QLL; the nnPU correction max{·,0} does not repair this bias. The method may still work empirically, but its theoretical justification as a class-wise PU risk estimator is invalid.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Quantized Label Learning (QLL), a setting in which each instance has a ground-truth soft-label distribution and the observed hard label is sampled from that distribution. The authors propose a Class-wise Positive-Unlabeled (CPU) risk estimator, constructed by applying the non-negative PU (nnPU) risk per class, treating the instances labeled with a given class as positives and all other instances as unlabeled. They design mixing-based ambiguous-data benchmarks (CIFAR-10Q, CIFAR-100Q, and AFHQ-Q) and report accuracy gains over label-noise-robust baselines. The paper's main theoretical claim is that CPU is a generalized PU risk estimator that is unbiased under the QLL model, and its headline empirical claim is that CPU outperforms the second-best method by more than six points on average.","tokens_in":15000,"tokens_out":6019,"duration_ms":56709,"significance":"If the theoretical claim were valid, the paper would be a useful contribution: it proposes a new problem formulation (QLL) with practical relevance, a simple and efficient algorithm, and reproducible benchmark construction procedures. The empirical study is broad, covering CIFAR-10/100 and AFHQ with multiple mixing strategies and many baselines, and the reported gains are substantial. The hyperparameter robustness analysis is a positive feature. However, the theoretical justification as stated is invalid: the class-wise unlabeled set does not follow the mixture distribution required by the PU risk identity, so the unbiasedness claim does not hold. In addition, the implemented Algorithm 1 differs from the reported Eq. (5). The paper would need a corrected derivation or a reframing of CPU as a heuristic to support its central claims.","major_comments":[{"comment":"The derivation of the class-wise PU risk is invalid under QLL. In standard PU learning, the unlabeled data must be drawn from the mixture p_pu(x) = π_p p_p(x) + π_n p_n(x), and the positive set independently from p_p(x). In the class-wise construction described in Section III and Algorithm 1, the unlabeled set for class j is D_j^u = {x_i : y_i ≠ j}, so the unlabeled instances are drawn from p(x | y ≠ j). Under Definition 1, P(y = j | x) = s_j(x), so p(x | y ≠ j) = p(x)(1 - s_j(x)) / P(y ≠ j), which is a biased reweighting of p(x) and is not equal to the mixture π_p p_p(x) + π_n p_n(x). Consequently, Eq. (4) is not an unbiased estimator of any classification risk under QLL, and the nnPU max operation does not repair this bias. The authors should either provide a valid derivation under a different generative model or explicitly present CPU as a heuristic with empirical support.","section":"Section III, Eqs. (2)-(4)"},{"comment":"The reported CPU risk and the implemented algorithm define different objectives. Eq. (5) states eR_pu(g_j) = π_p^(1) R_p^+(g_j) + max{ R_u^-(g_j) - π_p^(2) R_p^-(g_j), 0 }. Algorithm 1, lines 4-8, instead sets eR_pu(g_j) = -R_u^-(g_j) + π_p^(2) R_p^-(g_j) when the max term is negative, i.e., it uses the negative of the inside value and omits the positive-risk term π_p^(1) R_p^+(g_j). These are different losses with different gradients. Please specify which definition is the actual method and correct the inconsistency.","section":"Section III, Eq. (5) vs. Algorithm 1"},{"comment":"The class-prior hyperparameter π_p^(2) is set to m/c, where m is the number of mixed instances in the authors' benchmark generator. This uses privileged information about the synthetic data construction. The robustness analysis (Table VI and Fig. 3) shows non-negligible sensitivity to π_p^(2) (for example, CIFAR-10Q PatchMix best accuracies range from 71.57 to 72.69, and CIFAR-100Q Mixup from 51.33 to 51.85). The claim that CPU is robust to these hyperparameters should be qualified, and the method's performance on real-world ambiguous data without such privileged knowledge remains untested. Please provide a sensitivity analysis that does not assume knowledge of the generator.","section":"Section V-A and Appendix A"}],"minor_comments":[{"comment":"Since s is a probability distribution that sums to 1, the denominator in Eq. (1) is redundant; please clarify whether s is assumed to be normalized.","section":"Definition 1, Eq. (1)"},{"comment":"The sentence 'the performance of the CPU is robust to π(1)_p and π(1)_p' contains a typo: the second prior should be π(2)_p.","section":"Section III, paragraph on choosing class priors"},{"comment":"The text states that α ∈ (0, 0.5] but samples α from Beta(0.5, 0.5), whose support is (0, 1); please correct the range or the sampling distribution.","section":"Section III, Stochastic JS divergence"},{"comment":"Line 7 uses π_p without the superscript (2); please align the notation with Eq. (5).","section":"Algorithm 1, line 7"},{"comment":"The 'Mean' column appears between the dataset headers and the condition columns; it is unclear whether the mean is over the four mixing conditions or over repeated runs; please clarify the caption.","section":"Tables I and II"},{"comment":"The paper does not provide a link to the implementation of CPU or the benchmark generator; for reproducibility, please release the code.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical results are interesting and the problem setting is timely, but the theoretical framing is not sound as written. The inconsistency between Eq. (5) and Algorithm 1 must be resolved, and the class-prior choice based on the generator is a concern. I believe the paper can be revised to be acceptable by reframing CPU as a heuristic or by correcting the derivation, but the current version overclaims a principled unbiased-risk interpretation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper introduces a genuinely useful problem—training on hard labels that are sampled from a soft-label distribution—and backs it with a mixing-based benchmark that produces plausible ambiguous images. The CPU method, a per-class application of nnPU, shows consistent and often large gains over label-noise baselines on CIFAR-10Q/100Q and on the GAN-generated AFHQ-Q data. That empirical story is the real contribution.\n\nThe theory, however, doesn't hold up. The derivation imports the standard PU identity, which requires unlabeled data to be drawn from the mixture p_pu = π_p p_p + π_n p_n. In the class-wise construction, the unlabeled set for class j is the complement of the observed positives, so its density is p(x|y≠j). Under the QLL model, that is a biased reweighting of p(x) because the labeling probability s_j(x) is instance-dependent. The identity is therefore not valid, and Eq. (4) is not an unbiased estimator of any classification risk under QLL. The nnPU max correction doesn't fix that, and the authors provide no alternative proof.\n\nThere's also a concrete internal inconsistency: Eq. (5) uses the nnPU max{·,0}, but Algorithm 1 sets the negative branch to -x rather than 0, which is actually |x|. These are different losses, and the tables don't say which was used. Without code, that's a reproducibility problem.\n\nSmaller concerns: the prior π_p^(2)=m/c is chosen from the authors' own benchmark generator, so it's a known quantity only in the synthetic setting; the main tables report best test accuracy, which is optimistic; and no code or data are released. The robustness experiments partly soften the prior issue. The citation coverage is adequate—the problem is the transfer of PU theory, not the references.\n\nNet: the problem framing and empirical results deserve attention, but the paper overclaims by calling CPU a principled estimator. If the authors reframe it as a heuristic, or prove unbiasedness under a stronger assumption, and fix the equation/algorithm mismatch, this becomes a solid contribution. I'd send it to review with the expectation of major revisions.","headline":"The QLL problem framing and the empirical results are worth a look, but the CPU estimator is not actually unbiased under the paper's own model, and the algorithm disagrees with the main equation.","tokens_in":15590,"tokens_out":8185,"would_cite":false,"duration_ms":73475,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a class-wise positive-unlabeled risk estimator can train accurate classifiers from ambiguous data whose only labels are hard quantized labels.","keywords":["quantized label learning","ambiguous data","hard labels","positive-unlabeled learning","class-wise PU risk","soft-label distribution","label quantization","mixup benchmark"],"falsifier":"A direct test: on a two-class synthetic QLL dataset with known per-instance soft labels $s_j(x)$, compare the CPU risk estimate to the true expected risk computed by Monte Carlo; a systematic gap as the per-instance label probabilities vary would refute the unbiasedness claim. A second test: train CPU on quantized labels and compare with training on the underlying soft labels; if CPU cannot approach the soft-label model's accuracy, the claim that it recovers the information in ambiguous labels would be in doubt.","tokens_in":14537,"feed_emoji":"🏷️","tokens_out":11954,"duration_ms":104178,"temperature":0.7,"pith_summary":"Real-world images are often genuinely ambiguous, but standard datasets assign each one a single hard label, and models trained on such labels become overconfident and generalize poorly. This paper argues that these labels are still usable if the hard label is treated as a sample from an underlying soft-label distribution and each class is learned as a positive-unlabeled problem. The proposed Class-wise Positive-Unlabeled (CPU) risk estimator trains a binary PU classifier per class and averages the risks, requiring no soft labels and no noise transition matrix. On mixed-image benchmarks built from CIFAR and animal-face images, CPU improves generalization over standard training and label-noise-robust baselines, with a mean accuracy gain of more than six points over the second-best method in the main comparison. If the paper is right, ambiguity-aware training can be done with the annotation budget we already have.","feed_headline":"Per-class PU loss lifts accuracy on ambiguous labels by six points","feed_subtitle":"It needs only the cheap hard labels and treats every other class as unlabeled, beating noise-robust baselines.","key_machinery":"The load-bearing object is the Class-wise Positive-Unlabeled (CPU) risk estimator. It converts multiclass training into $c$ binary PU problems: for class $j$, the positive set is the examples whose quantized label is $j$, and the unlabeled set is everything else; each binary risk uses the non-negative PU formula from nnPU, with the negative term clipped at zero, and the $c$ risks are averaged. The paper uses a stochastic Jensen-Shannon divergence as the loss inside the estimator, sampling the interpolation weight per iteration to balance optimization and robustness.","core_discovery":"The central claim is that learning from ambiguous data with only quantized hard labels is tractable, and the way to make it tractable is to view every class as a separate positive-unlabeled problem. Under the label quantization hypothesis, an observed hard label $y$ is drawn from the instance's ground-truth soft-label distribution $P(y=k|x)=s_k$, so examples labeled $j$ are treated as positives for class $j$ and all other examples as unlabeled. The CPU estimator averages per-class non-negative PU risks, $\\hat{R}_{pu}(g_j)=\\pi_p^{(1)}\\hat{R}_p^+(g_j)+\\max\\{\\hat{R}_u^-(g_j)-\\pi_p^{(2)}\\hat{R}_p^-(g_j),0\\}$, which prevents the model from memorizing biased labels. The paper reports that this simple estimator achieves the best accuracy across all four CIFAR-Q configurations and on the AFHQ-Q dataset, and that combining it with the FixMatch semi-supervised technique gives further gains.","pith_inferences":["Editorial inference: the positive-versus-unlabeled decomposition may transfer to partial-label learning, where the candidate label set is the positive set and the rest are unlabeled; the paper does not test this.","Editorial inference: some of the gain may come from the max-clipping in the nnPU risk acting as a regularizer rather than from unbiased estimation; comparing CPU to label smoothing on the same ambiguous data would isolate the mechanism.","Editorial inference: on multi-annotator data, treating each annotator's vote as a quantized label and applying CPU should approach a soft-label oracle, a direct but untested consequence of the label quantization hypothesis."],"forward_implications":["If the CPU risk is correct, practitioners can train from ambiguous data using only the hard labels already being collected; no soft labels, label-distribution estimation, or noise transition matrix is needed.","The reported gains imply that per-class positive-unlabeled training is a stronger default than robust loss functions and complex noisy-label pipelines for quantized labels, and that adding one semi-supervised consistency term (FixMatch) makes it stronger still.","Because the risk is computed as an average of per-class binary risks with element-wise operations, it can drop into standard deep learning training loops with about the same cost as cross-entropy.","The mixing-based generation procedure gives future work a controlled benchmark: ambiguity level can be adjusted through the number of mixed instances and the patch masks, and quantized labels are sampled from known soft-label mixtures."],"supporting_citations":[{"why":"Defines the classical PU learning problem that QLL is cast as a multi-class variant of.","marker":"[18]"},{"why":"Provides the convex PU risk formulation that the class-wise estimator adapts.","marker":"[19]"},{"why":"Supplies the non-negative PU risk estimator whose per-class form is the CPU risk.","marker":"[20]"},{"why":"Gives the PU risk identity that Eq. (2) imports to rewrite the total risk.","marker":"[21]"},{"why":"Introduces the generalized Jensen-Shannon divergence used as the loss inside CPU.","marker":"[26]"},{"why":"Defines Mixup, whose soft-label interpolation is used to generate ambiguous data with quantized labels.","marker":"[27]"},{"why":"Defines CutMix, which the paper generalizes into PatchMix for multi-image ambiguous data.","marker":"[28]"},{"why":"Supplies ResNet-18, the architecture used in the CIFAR-Q experiments.","marker":"[29]"},{"why":"Supplies FixMatch, the semi-supervised consistency method combined with CPU in the CPU (Semi) variant.","marker":"[40]"}],"fun_headline_variants":["Class-wise PU estimator tames ambiguous hard-label learning","Treat each class as a PU problem to handle ambiguous labels","CPU risk: a simple fix for learning from ambiguous hard labels","Ambiguous labels? Per-class positive-unlabeled loss improves accuracy","Hard-label ambiguity solved with class-wise positive-unlabeled"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that, for each class, the examples not carrying that label behave like a fixed blend of that class and everything else; when the label probability varies from image to image, that blend assumption is only approximate, and the estimator's unbiasedness is not guaranteed.","fun_headline_variants_meta":{"raw":{"variants":["Class-wise PU estimator tames ambiguous hard-label learning","Treat each class as a PU problem to handle ambiguous labels","CPU risk: a simple fix for learning from ambiguous hard labels","Ambiguous labels? Per-class positive-unlabeled loss improves accuracy","Hard-label ambiguity solved with class-wise positive-unlabeled"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000237,"raw_usage":{"total_tokens":1517,"prompt_tokens":962,"completion_tokens":555,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":474}},"tokens_in":578,"tokens_out":555,"duration_ms":5869,"temperature":1.0,"reasoning_tokens":474,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:20:06.059428+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test: on a two-class synthetic QLL dataset with known per-instance soft labels $s_j(x)$, compare the CPU risk estimate to the true expected risk computed by Monte Carlo; a systematic gap as the per-instance label probabilities vary would refute the unbiasedness claim. A second test: train CPU on quantized labels and compare with training on the underlying soft labels; if CPU cannot approach the soft-label model's accuracy, the claim that it recovers the information in ambiguous labels would be in doubt.","supporting_citations":[{"cited_title":"Analysis of learning from positive and unlabeled data,","cited_arxiv_id":null,"evidence_quote":"Defines the classical PU learning problem that QLL is cast as a multi-class variant of."},{"cited_title":"Convex formulation for learning from positive and unlabeled data,","cited_arxiv_id":null,"evidence_quote":"Provides the convex PU risk formulation that the class-wise estimator adapts."},{"cited_title":"Positive- unlabeled learning with non-negative risk estimator,","cited_arxiv_id":null,"evidence_quote":"Supplies the non-negative PU risk estimator whose per-class form is the CPU risk."},{"cited_title":"Theoreti- cal comparisons of positive-unlabeled learning against positive-negative learning,","cited_arxiv_id":null,"evidence_quote":"Gives the PU risk identity that Eq. (2) imports to rewrite the total risk."},{"cited_title":"Generalized jensen-shannon divergence loss for learning with noisy labels,","cited_arxiv_id":null,"evidence_quote":"Introduces the generalized Jensen-Shannon divergence used as the loss inside CPU."},{"cited_title":"mixup: Beyond empirical risk minimization,","cited_arxiv_id":null,"evidence_quote":"Defines Mixup, whose soft-label interpolation is used to generate ambiguous data with quantized labels."},{"cited_title":"Cutmix: Reg- ularization strategy to train strong classifiers with localizable features,","cited_arxiv_id":null,"evidence_quote":"Defines CutMix, which the paper generalizes into PatchMix for multi-image ambiguous data."},{"cited_title":"Deep residual learning for image recognition,","cited_arxiv_id":null,"evidence_quote":"Supplies ResNet-18, the architecture used in the CIFAR-Q experiments."},{"cited_title":"Fixmatch: Simplifying semi- supervised learning with consistency and confidence,","cited_arxiv_id":null,"evidence_quote":"Supplies FixMatch, the semi-supervised consistency method combined with CPU in the CPU (Semi) variant."}],"review_version":1}