Pith. sign in

REVIEW 4 major objections 5 minor 25 references

Noisy Label Refinement with Semantically Reliable Synthetic Images

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Using synthetic images as reliable anchors to relabel noisy training data can sharply improve classification accuracy under semantic label noise.

desk verdict Synthetic-image prototypes for noisy-label relabeling is a genuinely useful idea with plausible large gains under semantic noise, but the headline numbers are weakened by test-set hyperparameter selection and no error bars. read the letter →

arxiv 2509.04298 v1 pith:NOZE5JU7 submitted 2025-09-04 cs.CV

Yingxuan Li , Jiafeng Mao , Yusuke Matsui This is my paper
classification cs.CV
keywords noisylabelssemanticlabelnoisesyntheticimagestext-to-imagegenerationrefinementfeatureprototypesPMDimageclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Real-world mislabeling is often semantic: visually similar classes get confused, and standard training handles this poorly. This paper claims that synthetic images generated from class names—which carry labels that are consistent with their content by construction—can act as reliable reference anchors for fixing those mislabels, even though the synthetic images themselves are not used as training data. The method trains a classifier on the noisy data, builds one feature-space prototype per class from generated images, and relabels each sample only when a blend of classifier confidence and prototype similarity passes a threshold. Across CIFAR-10, CIFAR-100, and ImageNet-100, the refined labels improve standard training and stack with noise-robust methods, with the largest reported combination gain 29.81 accuracy points on CIFAR-10 under 70% semantic noise.

What carries the argument

Synthetic class prototypes are the load-bearing object. For each category c, the paper averages features of SDXL-Turbo images generated with the prompt 'A photo of c', using the same feature extractor E trained on the noisy dataset; the resulting average pc is a stable reference point in feature space. The relabeling decision uses the combined score Sc = α·cos-sim(E(r), pc) + (1−α)·f(r)[c], with α=0.5 and threshold θ=0.6 in the main experiments, replacing the label only when max_c Sc clears θ.

What would settle it

Measure the discriminative power of the synthetic-prototype similarity alone on CIFAR-100 with 70% PMD noise: compute Ssim for every real image against the prototypes and report the AUC for distinguishing correct from incorrect labels. If the AUC is at chance, the synthetic anchors contribute no signal and the accuracy gains must come from the classifier-confidence term or the thresholding rule rather than from the paper's core mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that the value of synthetic images for noisy-label learning lies in their semantic label reliability, not in their realism or diversity. Averaging features of 100 generated images per class with a feature extractor trained on the noisy labels yields class prototypes that remain trustworthy anchors even under 70% semantic noise. A real image is relabeled to the class maximizing Sc = α·Ssim_c + (1−α)·Sconf_c, where Ssim is cosine similarity to the synthetic prototype and Sconf is the classifier's confidence, but only if that score exceeds θ. Because relabeling is a preparatory step, the corrected set can be passed to existing noise-robust trainers; with LRA-diffusi

Load-bearing premise

The load-bearing premise is that a classifier trained on noisy labels still learns features good enough that comparing a real image with synthetic class prototypes separates correctly labeled samples from mislabeled ones.

Editorial extensions

If this is right

  • The refined label set is a preprocessing output, so any noise-robust training method can be applied downstream; the paper demonstrates this with PLC and LRA-diffusion.
  • Under 70% PMD semantic noise, the LRA-diffusion (CLIP) baseline improves by 29.81 points on CIFAR-10 and 10.73 points on CIFAR-100.
  • On ImageNet-100, the largest reported gain is 24.10 points, at 60% asymmetric noise (20.74 to 44.84).
  • Off-the-shelf generation is enough: fine-tuning the generator on the target domain changes results by less than about 1.2 points in the tested settings.
  • A feature extractor trained only on the noisy labels captures most of the anchor benefit (46.06 vs 52.64 with clean labels on CIFAR-100 with 70% PMD), so the method does not require a clean pretrained encoder.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If noisy-trained features stay informative, the relabel step could be iterated—relabel, retrain, rebuild prototypes, relabel again—an extension the paper does not test.
  • The method inherits the generator's notion of class similarity; classes the text-to-image model cannot visually separate would yield overlapping prototypes and no relabeling signal. A direct check is to measure prototype separation before choosing θ.
  • The largest gains under asymmetric and semantic noise suggest the approach may help real annotation pipelines where mistakes come from human visual confusion, but the paper only approximates that setting with synthetic noise models.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a preprocessing step for learning with noisy labels: first train a classifier on the noisy dataset, generate synthetic images per class with SDXL-Turbo, compute class prototypes in the trained feature space, and then re-label each training image if a weighted combination of cosine similarity to synthetic prototypes and classifier softmax confidence exceeds a threshold theta. The refined labels are used to fine-tune the classifier and can also be fed into existing noise-robust methods such as PLC and LRA-diffusion. Experiments on CIFAR-10/CIFAR-100 under PMD and hybrid noise, and on ImageNet-100 under uniform/asymmetric noise, report substantial gains, e.g., +29.81 points for LRA-diffusion (CLIP) on CIFAR-10 at 70% PMD noise.

Significance. The core idea is simple and potentially useful: synthetic images are used as reliable label anchors rather than as direct training data, which sidesteps the known domain-gap and diversity limitations of synthetic images. If the reported gains are robust, the method would be a lightweight, orthogonal preprocessing module for noisy-label learning. The paper includes useful ablations (feature extractor choice, domain adaptation of the generator) and provides a code link. However, the central quantitative claims are currently weakened by evaluation-protocol issues, especially the apparent selection of hyperparameters on the test accuracy, the lack of error bars or multiple seeds, and an overclaim about ImageNet-100 real-world noise.

major comments (4)
  1. [Section 4.3, Table 4; Section 4.2] The optimal parameters (alpha = 0.5, theta = 0.6) are reported as identified 'through extensive testing' using accuracies on CIFAR-100 with 70% PMD noise. No held-out validation split is described. Since Table 1 then uses these parameters for the main results, the reported gains (e.g., +29.81 on CIFAR-10 and +10.73 on CIFAR-100 at 70% PMD) are subject to selection-on-the-test-set bias. The threshold interacts with the noise rate, so the authors should fix hyperparameters on a validation split or use nested cross-validation and report the resulting variability before these numbers can be taken at face value.
  2. [Tables 1 and 5] All results appear to be single runs with no error bars or multiple seeds. The headline improvements, especially the large jump at 70% PMD, need variance estimates. Please report mean and standard deviation over at least three independent runs for the main configurations, including the baselines, so the reader can assess stability.
  3. [Abstract vs. Section 4.4] The abstract states the method improves accuracy 'by 24% on ImageNet-100 under real-world noise conditions.' However, Section 4.4 only evaluates synthetic uniform and asymmetric label noise on ImageNet-100; there is no real-world noisy-label experiment. This is an overclaim and should be corrected.
  4. [Section 4.2, Table 1] The claim that the method is 'orthogonal' and generally improves existing methods is not supported by all cells. For example, LRA-diffusion (SimCLR) + Ours gives -0.42 on CIFAR-10 35% PMD + 30% U and -0.56 on CIFAR-100 35% PMD + 30% U, and PLC + Ours gives -0.47 on CIFAR-100 35% PMD. Please discuss the conditions under which the relabeling preprocessing can degrade accuracy, and avoid the blanket orthogonality claim.
minor comments (5)
  1. [Algorithm 1, line 5] 'Compute classification accuracies' should be 'compute classification scores' or 'softmax probabilities'; the value f(r)[c] is a probability, not an accuracy.
  2. [Table 4 caption] The caption says 'Impact of parameters on accuracies (%)' but the text says 'on the CIFAR-100 dataset, which is used as training data.' Clarify whether these are training or test accuracies, since the distinction is critical to evaluating the hyperparameter-selection concern.
  3. [Section 4.1] The number of synthetic images per class N = 100 is a free parameter that is not ablated. Since synthetic prototypes are central to the method, please report sensitivity to N.
  4. [Section 4.3] When fine-tuning SDXL-Turbo 'with noisy data of CIFAR-100,' please specify whether the noisy labels were used as supervision and how this could affect the generated images. The current description is ambiguous.
  5. [Various] The paper should define the ImageNet-100 subset and cite its source. Also, the abstract uses '30%' and '11%' where the tables report percentage-point gains; please use consistent terminology.

Circularity Check

1 steps flagged · score 6.0 of 10

Headline accuracy improvements are partly circular because α and θ are selected on the same test sets whose results are reported as the method's predictions.

  1. fitted input called prediction [Section 4.3 (Table 4) and Section 4.2 (main results)]
    "Table 4 shows how varying the parameter α in S_c = α·S_sim_c + (1−α)·S_conf_c, as well as different thresholds θ, affect classification accuracies on the CIFAR-100 dataset, which is used as training data with 70% PMD noise. ... We have identified the optimal parameters (α = 0.5, θ = 0.6) through extensive testing."

    The parameters α and θ are chosen by maximizing the reported accuracy on the CIFAR-100 70% PMD test set (Table 4). These same test-set accuracies (e.g., 47.40 for Re-labeled data at α=0.5, θ=0.6) are then presented in Table 1 as evidence of improvement, and the optimal configuration is reused for the headline combined results (+10.73 for LRA-diffusion + Ours). Because the test set was used to select the configuration, the reported accuracy is not an independent prediction but the maximum of a grid search over that same test set, so the improvement is statistically forced upward. No held-out validation split is described, so the headline numbers inherit this selection bias.

full rationale

The core derivation is not circular in the definitional sense: synthetic prototypes are constructed from independently generated text-to-image samples, and the relabeling score combines those similarities with classifier confidence rather than being defined as the classifier's own output. The self-training loop (training E/f on noisy labels and then fine-tuning on relabeled data) is a feedback mechanism but not a formal equivalence; Table 3 shows the result depends on the feature extractor, so it is not tautological. The paper's self-citation [7] is not load-bearing. The main circularity is in the evaluation protocol: Section 4.3 reports that α=0.5 and θ=0.6 were identified as optimal by testing on the CIFAR-100 70% PMD test set, and Section 4.2/Table 1 then reports the corresponding accuracies as the method's performance, including the headline 10.73-point gain. Selecting hyperparameters to maximize a test-set metric and then reporting that same metric as a result is a fitted-input-called-prediction pattern: the reported number reduces by construction to the parameter search. This inflates the specific quantitative claims, though the qualitative idea of using synthetic anchors retains independent content.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The method introduces no invented entities. Its central claim relies on two tuned hyperparameters (alpha, theta) and on assumptions that synthetic images are class-consistent and that noisy-trained features remain useful for similarity comparison.

free parameters (3)
  • alpha (similarity vs confidence weight) = 0.5
    Eq. (1) blends cosine similarity with classifier confidence; 0.5 was chosen by 'extensive testing' as optimal (Sec. 4.3, Table 4).
  • theta (relabeling threshold) = 0.6
    Labels are updated only if the max combined score exceeds theta; 0.6 was chosen by 'extensive testing' (Sec. 4.3, Table 4).
  • synthetic images per class N = 100
    100 synthetic images per class generated by SDXL-Turbo; no sensitivity analysis reported (Sec. 4.1).
assumptions (3)
  • domain assumption SDXL-Turbo synthetic images are semantically consistent with their text prompt labels
    The whole approach relies on generated images for each class being reliable representatives of that class (Sec. 3, Synthetic reference generation).
  • domain assumption The noisy-trained feature extractor E remains informative for similarity-based label correction
    Explicitly stated in Sec. 3 ('we observe that learned feature representation of E remains informative'), supported by Table 3 but with a 6.6 point gap vs clean-trained features on CIFAR-100.
  • domain assumption Cosine similarity in E's feature space is a valid measure of semantic relatedness between real and synthetic images
    Used throughout the relabeling step (Eq. (1)); no analysis of feature-space geometry is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Noisy Label Refinement with Semantically Reliable Synthetic Images." pith.science (2026). https://pith.science/paper/NOZE5JU7

@misc{pith2026250904298,
  author       = {Pith},
  title        = {Pith review of: Noisy Label Refinement with Semantically Reliable Synthetic Images},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NOZE5JU7}},
  note         = {Machine review of arXiv:2509.04298}
}
read the original abstract

Semantic noise in image classification datasets, where visually similar categories are frequently mislabeled, poses a significant challenge to conventional supervised learning approaches. In this paper, we explore the potential of using synthetic images generated by advanced text-to-image models to address this issue. Although these high-quality synthetic images come with reliable labels, their direct application in training is limited by domain gaps and diversity constraints. Unlike conventional approaches, we propose a novel method that leverages synthetic images as reliable reference points to identify and correct mislabeled samples in noisy datasets. Extensive experiments across multiple benchmark datasets show that our approach significantly improves classification accuracy under various noise conditions, especially in challenging scenarios with semantic label noise. Additionally, since our method is orthogonal to existing noise-robust learning techniques, when combined with state-of-the-art noise-robust training methods, it achieves superior performance, improving accuracy by 30% on CIFAR-10 and by 11% on CIFAR-100 under 70% semantic noise, and by 24% on ImageNet-100 under real-world noise conditions.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 19 canonical work pages

  1. [1]

    Noisy Label Refinement with Semantically Reliable Synthetic Images

    INTRODUCTION Learning from noisy data remains one of the most funda- mental and challenging problems in machine learning. This challenge is particularly pronounced in image classification, where datasets often contain images that are mislabeled with incorrect categories. Such mislabeling poses a significant obstacle to conventional supervised learning app...

    work page Pith review arXiv 2025

  2. [2]

    Noisy label learning Conventional research on noisy label learning [6, 7] was pri- marily based on an i.i.d

    RELATED WORK 2.1. Noisy label learning Conventional research on noisy label learning [6, 7] was pri- marily based on an i.i.d. assumption, i.e., the corruption of labels is independent and identically distributed. However, this assumption does not align with real-world scenarios, as noise in real datasets tends to be feature-dependent. Zhang et al. [1] in...

    work page

  3. [3]

    A photo of c

    APPROACH Our proposed method addresses the challenge of learn- ing from a noisy dataset comprising M real images R = {ri}M i=1 with potentially incorrect labels L = {li}M i=1, where li ∈ C belongs to a predefined set of categories C, e.g., C = {′′dog′′,′′ cat′′, . . .}. As illustrated in Fig. 2, the core idea of our approach is to take advantage of synthe...

    work page

  4. [4]

    Experimental setup Datasets and noise types

    EXPERIMENTS 4.1. Experimental setup Datasets and noise types. Following existing noisy label learning methods [1, 8], we conduct experiments on the CIFAR-10 and CIFAR-100 datasets [11] under various noise Algorithm 1 Noisy label refinement using synthetic images. Input: Real images R with labels L Parameter: Threshold θ Output: Re-labeled real images’ lab...

    work page

  5. [5]

    CONCLUSION In this paper, we address the challenge of learning from noisy labels by leveraging the semantic reliability of synthetic im- ages. We utilize these images as reliable and stable anchors for refining noisy labels, and our experiments across various datasets demonstrate that our proposed method significantly improves classification accuracies, p...

    work page

  6. [6]

    Learning with feature- dependent label noise: A progressive approach,

    Yikai Zhang, Songzhu Zheng, Pengxiang Wu, Mayank Goswami, and Chao Chen, “Learning with feature- dependent label noise: A progressive approach,” in Pro- ceedings of the International Conference on Learning Representations, 2021

    work page 2021

  7. [7]

    Hierarchical text-conditional image generation with clip latents,

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, vol. 1, no. 2, pp. 3, 2022

    arXiv 2022

  8. [8]

    High-resolution im- age synthesis with latent diffusion models,

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer, “High-resolution im- age synthesis with latent diffusion models,” in Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10684–10695

    work page 2022

Show all 25 references
  1. [9]

    Will large-scale generative models corrupt future datasets?,

    Ryuichiro Hataya, Han Bao, and Hiromi Arai, “Will large-scale generative models corrupt future datasets?,” in Proceedings of the IEEE/CVF International Confer- ence on Computer Vision, 2023, pp. 20555–20565

    2023

  2. [10]

    Fake it till you make it: Learn- ing transferable representations from synthetic imagenet clones,

    Mert Bulent Sariyildiz, Karteek Alahari, Diane Larlus, and Yannis Kalantidis, “Fake it till you make it: Learn- ing transferable representations from synthetic imagenet clones,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023

    2023

  3. [11]

    Joint optimization framework for learning with noisy labels,

    Daiki Tanaka, Daiki Ikami, Toshihiko Yamasaki, and Kiyoharu Aizawa, “Joint optimization framework for learning with noisy labels,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- tion, 2018, pp. 5552–5560

    2018

  4. [12]

    Noisy annotation refinement for object detec- tion,

    Jiafeng Mao, Qing Yu, Yoko Yamakata, and Kiyoharu Aizawa, “Noisy annotation refinement for object detec- tion,” arXiv preprint arXiv:2110.10456, 2021

    2021 arXiv

  5. [13]

    Label- retrieval-augmented diffusion models for learning from noisy labels,

    Jian Chen, Ruiyi Zhang, Tong Yu, Rohan Sharma, Zhiqiang Xu, Tong Sun, and Changyou Chen, “Label- retrieval-augmented diffusion models for learning from noisy labels,” Advances in Neural Information Process- ing Systems, vol. 36, 2024

    2024

  6. [14]

    Is synthetic data from generative models ready for image recognition?,

    Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue, Wen- qing Zhang, Philip Torr, Song Bai, and Xiaojuan Qi, “Is synthetic data from generative models ready for image recognition?,” arXiv preprint arXiv:2210.07574, 2022

    2022 arXiv

  7. [15]

    Imagenet large scale visual recognition challenge,

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al., “Imagenet large scale visual recognition challenge,” In- ternational Journal of Computer Vision , vol. 115, pp. 211–252, 2015

    2015

  8. [16]

    Learning multiple layers of features from tiny images,

    Alex Krizhevsky, Geoffrey Hinton, et al., “Learning multiple layers of features from tiny images,” 2009

    2009

  9. [17]

    Adversarial diffusion distillation,

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach, “Adversarial diffusion distillation,” in Proceedings of the European Conference on Computer Vision. Springer, 2025, pp. 87–103

    2025

  10. [18]

    Sdxl: Improving latent diffu- sion models for high-resolution image synthesis,

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach, “Sdxl: Improving latent diffu- sion models for high-resolution image synthesis,” arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  11. [19]

    Deep residual learning for image recognition,

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition, 2016, pp. 770–778

    2016

  12. [20]

    Generalized cross en- tropy loss for training deep neural networks with noisy labels,

    Zhilu Zhang and Mert Sabuncu, “Generalized cross en- tropy loss for training deep neural networks with noisy labels,” Advances in Neural Information Processing Systems, vol. 31, 2018

    2018

  13. [21]

    Symmetric cross entropy for robust learning with noisy labels,

    Yisen Wang, Xingjun Ma, Zaiyi Chen, Yuan Luo, Jin- feng Yi, and James Bailey, “Symmetric cross entropy for robust learning with noisy labels,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 322–330

    2019

  14. [22]

    Error-bounded correction of noisy labels,

    Songzhu Zheng, Pengxiang Wu, Aman Goswami, Mayank Goswami, Dimitris Metaxas, and Chao Chen, “Error-bounded correction of noisy labels,” in Proceed- ings of the International Conference on Machine Learn- ing. PMLR, 2020, pp. 11447–11457

    2020

  15. [23]

    Centrality and consistency: two-stage clean samples identification for learning with instance- dependent noisy labels,

    Ganlong Zhao, Guanbin Li, Yipeng Qin, Feng Liu, and Yizhou Yu, “Centrality and consistency: two-stage clean samples identification for learning with instance- dependent noisy labels,” in Proceedings of the Euro- pean Conference on Computer Vision . Springer, 2022, pp. 21–37

    2022

  16. [24]

    Learning transferable visual models from natural lan- guage supervision,

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al., “Learning transferable visual models from natural lan- guage supervision,” in Proceedings of the International Conferen...

    2021

  17. [25]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint ar...

    2010 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.