{"id":"6dac98f9-f5ec-4488-97a3-858e245729d2","arxiv_id":"2501.11063","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"A subgroup-based positive-pair selection framework lets noisy samples contribute to metric learning training instead of being discarded, improving retrieval and face recognition accuracy under label noise.","lead":"This paper presents SGPS, a training method for deep metric learning that separates samples into clean and noisy, then builds positive training pairs for noisy samples using automatically generated subgroups so they are not discarded. The method reports large gains over prior noise-robust metric learning baselines on image retrieval and face recognition benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on SGM's positive pairs Pi being reliable for noisy samples, but the paper never measures the precision of those pairs; without that measurement, the headline gains could come from extra data or the Lclean branch instead of the proposed noisy-sample utilization.","rationale":"The reader's weakest assumption points to the same general risk: subgroup generation presupposes a semantically coherent feature embedding that may not exist early in training or under high noise. I agree this is the load-bearing assumption. However, I would sharpen the concern: the decisive empirical quantity is not just whether clusters are coherent, but whether the specific Pi selected for noisy samples are true positives. The paper reports selection accuracy for the clean/noisy split (Fig. 6), which measures PCS, not SGM. SGM and PPM are the novel components that justify the 'sample utilization' claim, and they are never directly evaluated. The face-recognition result (Table IV) is impressive but confounded by dataset size: MS1MV0 has 10M images versus 5.8M for MS1MV2, so even with 50% noise there are roughly as many clean images plus additional data. Without a controlled test of Pi reliability, or an ablation that removes/replaces Lnoise, the central claim is conditional rather than established. I do not see an internal inconsistency or reason to reject; the concern is about missing evidence for the mechanism, which the reader's CONDITIONAL verdict already captures. Hence I recommend no change to the verdict, with the concrete positive-pair measurement as the natural acceptance criterion.","tokens_in":26073,"tokens_out":4543,"duration_ms":51295,"concrete_test":"Instrument the released code on CARS and SOP with 50% and 90% symmetric noise: for every sample entering Bnoise, record the ground-truth label agreement between the anchor and each element of Pi (and the nearest true class of the prototype ri) at several training epochs. Report positive-pair precision and recall over the noisy subset. If precision is not substantially above the annotated-label positive-pair baseline and does not improve during training, the subgroup-based positive-pair construction is not validated as the source of SGPS's gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims SGPS 'constructs reliable positive pairs for noisy samples', and the strongest evidence is DCQ+SGPS on noisy MS1MV0 reaching 92.91% TAR@1e-5 on IJB-C versus 91.99% for DCQ trained on clean MS1MV2. The entire Lnoise term (Eqs. 13-16) rests on the subgroup labels cB/cT from SGM: each noisy sample is pulled toward a prototype ri aggregated from Pi, and Pi is selected using cB/cT (Section III-E). If those subgroup labels are wrong, Lnoise actively trains the sample toward a false-positive prototype, and the negative set in Eq. (14) excludes the same-subgroup false positives by construction, so the error is never corrected. The paper reports clean-sample selection accuracy (Fig. 6) but never reports the accuracy of the positive pairs Pi for the noisy subset: e.g., how many of the K selected positives share the ground-truth label with the noisy anchor, and how this evolves over training or at 90% symmetric noise. This is precisely the quantity that would justify the claim. Without it, the Table IV improvement could alternatively be explained by using 10M noisy images versus 5.8M clean images, or by the Lclean branch, while Lnoise may be contributing a weak or even harmful signal that the final numbers do not isolate.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SGPS, a noise-robust deep metric learning framework that, instead of discarding suspicious noisy samples, attempts to construct positive pairs for them from subgroup labels. A probability-based clean-sample selection (PCS) splits each batch into clean and noisy sets; a subgroup generation module (SGM) produces bottom-up and top-down subgroup labels from a momentum feature bank; a positive prototype generation module (PPM) aggregates K selected positives into a prototype per noisy sample; and a contrastive loss L_noise pulls noisy samples toward those prototypes while excluding same-subgroup negatives. The clean set is trained with a standard DML loss L_clean. Experiments cover CARS, SOP, CUB, Food-101N, CARS-98N, Clothing1M, and MS1MV0 face recognition, with SGPS instantiated on MCL, SupCon, Roadmap, and Contextual. The headline result is DCQ+SGPS reaching 92.91% TAR@FAR=1e-5 on IJB-C when trained on noisy MS1MV0, surpassing DCQ trained on clean MS1MV2 (91.99%).","tokens_in":26492,"tokens_out":4008,"duration_ms":45279,"significance":"If the central claim holds, the paper addresses a real gap: prior noise-robust DML methods such as PRISM discard noisy samples, and SGPS offers a general wrapper that can be plugged into several pairwise DML losses. The experimental scope is broad, spanning synthetic symmetric noise, small-cluster noise, real-world noisy image retrieval, and large-scale face recognition, and the framework shows consistent improvements over its base losses in most configurations. The code is promised publicly, which supports reproducibility if the hyperparameters are fully specified. However, the paper's central effectiveness claim is not yet statistically supported: results are single runs without error bars, many SGM hyperparameters are unspecified, and the proposed mechanism's key ingredient is never directly measured. The evidence is broad but does not isolate the contribution of L_noise from confounds such as extra training data or the L_clean branch, so the current support for the abstract's claim of constructing reliable positive pairs for noisy samples is incomplete.","major_comments":[{"comment":"The headline comparison does not isolate the effect of the proposed L_noise branch: DCQ+SGPS on MS1MV0 uses 10M noisy images, while DCQ on clean MS1MV2 uses 5.8M images, and the SGPS training also includes L_clean on selected clean samples. Table IV does not report a control of DCQ on MS1MV0 with only clean-sample selection and L_clean, or a DCQ+SGPS variant with L_noise ablated. Without such controls, the stated conclusion that the gain comes from reliable positive pairs for noisy samples is not identified by the evidence; the gain could come from the larger data volume or from the L_clean branch.","section":"IV-C, Table IV"},{"comment":"The manuscript never measures the reliability of the positive pairs Pi that are the load-bearing component of L_noise. Figure 6 reports only clean-sample selection accuracy, not the precision of the K selected positives for noisy anchors, e.g., the fraction of Pi sharing the ground-truth label with the noisy anchor and how this evolves over training or at high noise rates. Because Eq. (13) pulls the noisy sample toward a prototype aggregated from Pi and Eq. (14) excludes same-subgroup samples from negatives by construction, incorrect subgroup labels would cause L_noise to reinforce false positives. The claimed mechanism therefore needs a direct measurement; with only end-task numbers, the alternative explanations in the previous comment remain open.","section":"III-E, III-F, Fig. 6"},{"comment":"Several SGM hyperparameters that directly control the subgroup labels are not reported for any experiment, including lambda_min, lambda_max, lambda'_min, lambda'_max, tau_max, B, the window omega and percentile R in Eq. (2), the momentum alpha in Eq. (3), and the number K of selected positives. Table V varies only tau, delta, gamma1, and gamma2 on one dataset, and Table VI uses K=4 only for runtime. Without the SGM settings, the experimental results are not reproducible and the sensitivity of the central mechanism to these choices is unknown.","section":"III-D, Algorithms 1-4, IV-B"},{"comment":"The 'posterior data clean and training strategy' SGPS-MCL* is a three-stage variant that trains on pseudo-labels, applies an early-stopping PRISM, and uses a stage-3 fine-tune with original labels; Table I presents it as a main result without isolating its contribution from the core SGPS framework. The description leaves unspecified how cB pseudo-labels are generated for stage 2, what noise rate is used in the early-stopping PRISM stage, and which model-selection criterion is applied. Presenting this post-hoc variant in the headline table with the asterisk is acceptable only if its hyperparameters are fully specified and its difference from the base SGPS is clear.","section":"IV-E, Table I"},{"comment":"All reported numbers appear to be single runs, with no standard deviations, number of trials, or significance tests. Many of the claimed improvements over the strongest baselines are small relative to likely run-to-run variation, for example Table III CARS-98N P@1 of 73.56 for SGPS-Roadmap versus 64.29 for Roadmap is large, but Table III Clothing1M P@1 differences between SGPS variants are around 0.2-0.5 points, and Table II contains several close values. The paper should report multiple seeds or error bars for at least the main comparisons to support the broad claim of outperforming state-of-the-art methods.","section":"Tables I-IV"}],"minor_comments":[{"comment":"The sentence 'The evaluation is conducted on seven image retrieval or face recognition benchmark datasets, including:' is duplicated verbatim in the dataset section.","section":"IV-A"},{"comment":"The role of lambda_max in Eq. (4) is unclear: W_m_ij is set to 1 if S_m_ij > lambda_max, but Algorithm 1 then sets W_m_ij = 0 if S_m_ij < lambda_min; it should be stated whether lambda_min and lambda_max are used jointly and how overlapping conditions are resolved.","section":"III-D, Eq. (4)"},{"comment":"The hyperplane is written as h = (f_i - f_j)/2, but the division by 2 is irrelevant for the side test f_k^T h >= 0, and the notation would be clearer if h were defined as the normal vector f_i - f_j (or normalized) with the decision boundary at (f_i + f_j)/2.","section":"III-D, Eq. (6), Algorithm 3"},{"comment":"The ablation subsection title 'Effectiveness of GSM' contains a typo; it should be 'SGM'.","section":"IV-E, Fig. 5(a)"},{"comment":"The table header says 'CARS19', but the dataset is referred to as CARS elsewhere; please use the consistent dataset name.","section":"IV-D, Table V"},{"comment":"The method name 'PRISIM' in the Fig. 1 caption should be 'PRISM'.","section":"I, Fig. 1(c)"}],"recommendation":"major_revision","confidential_remarks":"This is a borderline case: the empirical work is extensive and the framework is plausible, but the central mechanism is not directly validated, and the paper's main quantitative comparisons are not supported by error bars or by ablation of the noisy-sample loss. The missing SGM hyperparameter details are a reproducibility issue that should be fixable in revision. I would not reject on the current evidence, but I would not accept without the requested analysis, because the abstract's 'constructs reliable positive pairs' claim is the core contribution and the paper does not yet measure that reliability. I do not see signs of fabrication; the concern is with experimental isolation and reporting detail."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"SGPS takes the PRISM-style clean/noisy split and, instead of throwing away the noisy half, tries to build reliable positive pairs for those samples using subgroup labels from two complementary clusterings—bottom-up merged and top-down divided—then aggregates each pair set into a prototype and runs a contrastive loss. The strongest result is the face-recognition one: DCQ+SGPS on noisy MS1MV0 reaches 92.91% TAR@1e-5 on IJB-C, beating DCQ trained on clean MS1MV2 (91.99%). That is a real reason to read the paper.\n\nWhat is actually new is the integration. Feature banks, clustering, and prototype contrastive learning all exist, but the two-view subgroup generation plus prototype aggregation for noisy-sample positives is a sensible combination. The paper also does what a good systems paper should: it plugs into four different base losses and consistently improves over the base methods across synthetic and real-world noise. The ablations in Fig. 5 show SGM and PPM each buy something. That is real evidence.\n\nThe soft spots are in proportion. The load-bearing claim is that the subgroup-generated positive pairs for noisy samples are reliable. The paper measures clean-sample selection accuracy but never measures the precision of the positive pairs Pi—what fraction of the K selected positives share the ground-truth label with a noisy anchor, and how that changes during training or at 90% noise. Without that number, the face-recognition improvement could partly come from training on 10M noisy images versus 5.8M clean ones, or from the Lclean branch, with Lnoise contributing less than claimed. The SGM ablation mitigates this—removing SGM hurts—but it does not decompose the data-size effect.\n\nSecond, the method has many per-dataset SGM thresholds (λmin, λmax, λ′min, τk, τmax, B, ω, R), and the paper only reports sensitivity for τ, δ, γ1, γ2. Single-run results without error bars are common in DML, but for a claim this strong, seeds would help. Third, the three-stage \"posterior data clean\" variant that produces the starred numbers in Table I is a legitimate but bolted-on strategy; it is not the online method.\n\nThe math is coherent. The self-training circularity is mild because evaluation is on held-out sets. The paper is clearly written and the citations to PRISM, LaCoL, and clustering baselines are fair. The missing pair-precision diagnostic is exactly the experiment a referee should ask for. This paper is for DML researchers working with noisy or web-scale data, and it deserves a serious referee. I would accept it with major revisions requiring that diagnostic, error bars, and threshold sensitivity. It is a solid subfield contribution, not a field re-shaper.","headline":"A useful noisy-DML method with a strong face-recognition result, but the paper never measures the precision of the positive pairs it constructs for noisy samples—ask for that diagnostic.","tokens_in":26984,"tokens_out":5164,"would_cite":true,"duration_ms":47735,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A noise-robust metric learning framework reuses noisy samples by building reliable positive pairs from subgroup labels, and reports state-of-the-art retrieval and face recognition results.","keywords":["noise-robust deep metric learning","positive-pair selection","subgroup generation","label noise","sample utilization","contrastive learning","image retrieval","face recognition"],"falsifier":"Train SGPS and PRISM on CARS with 50% symmetric noise, where ground-truth labels are known, and measure the precision of the K positive samples SGM selects for each noisy sample (fraction whose true label matches). If that precision is at or below the base rate while SGPS still beats PRISM, the subgroup discovery is not the source of the gain. A direct control: replace the subgroup labels with a random partition of each annotated class into the same number of subgroups; if P@1 stays the same, the subgroup machinery is not load-bearing.","tokens_in":25903,"feed_emoji":"🎯","tokens_out":7083,"duration_ms":66311,"temperature":0.7,"pith_summary":"The paper argues that in deep metric learning, noisy labels should not be discarded: they can be turned into useful training signal. It proposes SGPS, which first splits each batch into clean and noisy samples, then discovers likely true positive partners for each noisy sample from subgroup labels derived from the feature distribution, aggregates those partners into a prototype, and trains the noisy sample against that prototype with a contrastive loss. The claim is that this improves sample utilization and outperforms prior noise-robust metric learning methods, which mostly throw noisy samples away. The strongest reported evidence is on face recognition: DCQ+SGPS trained on the noisy MS1MV0 dataset reaches 92.91% TAR@FAR=1e-5 on IJB-C, surpassing DCQ trained on the clean MS1MV2 dataset (91.99%).","feed_headline":"Reusing noisy labels outperforms clean-data training in metric learning","feed_subtitle":"SGPS builds positive pairs from subgroup structure, pushing face verification on noisy data past clean-data baselines.","key_machinery":"The central object is the subgroup-based positive-pair selection pipeline inside SGPS. In it, a probability-based clean-sample selection (PCS) uses a memory bank of historical features to compute pclean(i) (Eq. 1) and splits each batch into Bclean and Bnoise. The Subgroup Generation Module (SGM) produces two subgroup labelings: bottom-up labels cB from agglomerative merging of intra-class connected components (Algorithms 1-2) and complementary top-down labels cT from recursive maximum-margin hyperplane division (Algorithms 3-4). The Positive Prototype Generation Module (PPM) selects K neighbors of each noisy sample according to cB and cT, and aggregates them into a prototype ri either by mean, max, softmax correlation weights, or a learnable cross-attention transformer (TransProto). The load-bearing step is Eq. (13)/(15): the noisy contrastive loss Lnoise = γ1 Lnoise_batch + γ2 Lnoise_bank pulls each noisy sample toward its prototype and away from negatives whose annotated label and both subgroup labels differ, which is how discarded samples become usable.","core_discovery":"On the paper's own terms, the discovery is that the positive pairs needed for noisy samples can be inferred from the structure of the learned feature space, without trusting the noisy labels. The Subgroup Generation Module (SGM) splits each annotated class into connected components of high intra-class similarity, merges these subgroups bottom-up by centroid similarity under meta-cluster and size rules, and adds a complementary top-down recursive partitioning; the resulting subgroup labels cB and cT mark, for each noisy sample, a set of samples likely to share the true class. The Positive Prototype Generation Module (PPM) then pools K such samples into a single prototype ri, and a new contrastive loss Lnoise pulls the noisy sample toward ri while pushing it away from samples outside its subgroups. Because both clean and noisy samples contribute to training, the framework avoids the confirmation-bias overfitting that the paper documents in the discard-based PRISM method, and across CARS, CUB, SOP, Food-101N, Clothing1M, and MS1MV0 it outperforms the compared noisy-label DML baselines.","pith_inferences":["If subgroup labels reliably identify true positives for noisy samples, the same mechanism could refine negative-pair mining (the paper already excludes same-subgroup negatives in Eq. 14) and could be applied to semi-supervised pseudo-labeling, where the noisy set is treated as unlabeled data.","The face-recognition result suggests a data-engineering consequence the paper does not state: web-scale noisy collections like MS1MV0 could substitute for expensively cleaned versions when the noise is clusterable, potentially changing dataset curation practice.","A testable diagnostic follows from the paper's own assumption: track the precision of SGM-selected positive pairs against ground truth on synthetic noise; if SGPS's advantage over PRISM does not track that precision, the gain may come from the prototype loss rather than subgroup discovery."],"forward_implications":["Noisy samples contribute to learning instead of being filtered out, so models can continue improving late in training where discard-based methods like PRISM show overfitting (Fig. 8).","SGPS is a wrapper: instantiating it on MCL, SupCon, Roadmap, and Contextual improves all four baselines on synthetic and real-world noisy retrieval benchmarks.","On large-scale face recognition, DCQ+SGPS trained on noisy MS1MV0 reaches 92.91% TAR@FAR=1e-5 on IJB-C, surpassing DCQ trained on the clean MS1MV2 (91.99%), implying the noisy data's extra volume can outweigh its label noise.","Training cost stays manageable: SGPS runs about 1.1-1.2x slower than PRISM on CARS and SOP and adds no GPU memory overhead, unlike multi-proxy classifier methods (Tab. VI, Fig. 9)."],"supporting_citations":[{"why":"PRISM, the state-of-the-art noisy-label DML method that SGPS extends and compares against; supplies the clean-sample selection baseline and the discard-based behavior SGPS aims to fix.","marker":"[4]"},{"why":"Cross-batch memory (XBM) and the memory-bank contrastive loss MCL used as Lclean, plus the cross-batch memory that PCS and PPM reuse.","marker":"[20]"},{"why":"DCQ, the classifier-free face recognition method that SGPS is instantiated on; its clean-data result on MS1MV2 is the number SGPS-on-noisy-data surpasses.","marker":"[80]"},{"why":"MoCo, source of the momentum-based feature bank update used to store all training features in SGM.","marker":"[63]"},{"why":"ODC, reference for the online feature-bank and clustering design that SGM's feature bank builds on.","marker":"[64]"},{"why":"Connected Components Labeling, the algorithm used in intra-class splitting to obtain clean subgroups.","marker":"[65]"}],"fun_headline_variants":["Subgroup positives reuse noisy labels, beating clean-data training","Turn noisy labels into useful pairs using subgroups","Noisy labels? Subgroup structure yields reliable positive pairs","Reusing noisy samples with subgroup pairs outperforms clean-data training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the current feature embedding already forms semantically coherent clusters, so that the connected components in Eq. (4) and the merged centroids of Algorithms 2-3 recover true positive pairs for noisy samples; early in training or at 90% noise this is unverified, and if the clusters are wrong, Lnoise pulls noisy samples toward incorrect prototypes and could amplify the very noise it is meant to remove.","fun_headline_variants_meta":{"raw":{"variants":["Subgroup positives reuse noisy labels, beating clean-data training","Turn noisy labels into useful pairs using subgroups","Noisy labels? Subgroup structure yields reliable positive pairs","Reusing noisy samples with subgroup pairs outperforms clean-data training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000818,"raw_usage":{"total_tokens":3630,"prompt_tokens":1040,"completion_tokens":2590,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":656,"completion_tokens_details":{"reasoning_tokens":2525}},"tokens_in":656,"tokens_out":2590,"duration_ms":20930,"temperature":1.0,"reasoning_tokens":2525,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T18:40:13.365442+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train SGPS and PRISM on CARS with 50% symmetric noise, where ground-truth labels are known, and measure the precision of the K positive samples SGM selects for each noisy sample (fraction whose true label matches). If that precision is at or below the base rate while SGPS still beats PRISM, the subgroup discovery is not the source of the gain. A direct control: replace the subgroup labels with a random partition of each annotated class into the same number of subgroups; if P@1 stays the same, the subgroup machinery is not load-bearing.","supporting_citations":[{"cited_title":"Cross-batch memory for embedding learning,","cited_arxiv_id":null,"evidence_quote":"Cross-batch memory (XBM) and the memory-bank contrastive loss MCL used as Lclean, plus the cross-batch memory that PCS and PPM reuse."},{"cited_title":"Dynamic class queue for large scale face recognition in the wild,","cited_arxiv_id":null,"evidence_quote":"DCQ, the classifier-free face recognition method that SGPS is instantiated on; its clean-data result on MS1MV2 is the number SGPS-on-noisy-data surpasses."},{"cited_title":"Momentum contrast for unsupervised visual representation learning,","cited_arxiv_id":null,"evidence_quote":"MoCo, source of the momentum-based feature bank update used to store all training features in SGM."},{"cited_title":"Online deep clustering for unsupervised representation learning,","cited_arxiv_id":null,"evidence_quote":"ODC, reference for the online feature-bank and clustering design that SGM's feature bank builds on."},{"cited_title":"Percolation and cluster distribution. i. cluster multiple labeling technique and critical concentration algorithm,","cited_arxiv_id":null,"evidence_quote":"Connected Components Labeling, the algorithm used in intra-class splitting to obtain clean subgroups."}],"review_version":1}