REVIEW 4 major objections 5 minor 34 references
Enhancing Domain Diversity in Synthetic Data Face Recognition with Dataset Fusion
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper claims that fusing two synthetic face datasets generated by architecturally distinct pipelines improves face recognition accuracy over either dataset alone, even when the total number of identities is held equal.
desk verdict The fusion idea is sensible and the competition result is a plus, but the unspecified 5K subset selection makes the same-scale comparison unverifiable. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the fused training set, built by concatenating equal-size identity subsets from Digi2Real and Vec2Face, two datasets generated by fundamentally different pipelines: Digi2Real uses graphics-based rendering with identity-preserving realism transfer, while Vec2Face samples identity vectors in a PCA-transformed latent space and adds intra-class variation with external attribute models. The identity signal is anchored by ArcFace embeddings, and every model is an IResNet50 trained with ArcFace loss. To support the fusion, the paper uses t-SNE visualizations of EdgeFace and CLIP embeddings to argue that the two datasets occupy separated identity clusters, so the combined set does not merely duplicate identities; the proposed mechanism is that distinct generators produce complementary artifacts, so training on both prevents overfitting to any single generator's biases.
What would settle it
Re-run the 5K-versus-10K comparison with several independently drawn random 5K subsets from both Digi2Real and Vec2Face; if the fused 5K+5K model no longer beats the single-dataset 10K model on the same benchmarks, the reported gain is an artifact of subset choice rather than of dataset fusion.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that dataset fusion works as an implicit regularizer: models trained on a concatenation of two synthetic face datasets from architecturally distinct pipelines generalize better than models trained on the same number of identities from either pipeline alone. In their experiments, HS-5K+Digi2Real-5K reached 98.92% on LFW, 91.26% on CFP-FP, and 86.30% on CPLFW, beating both HS-10K and Digi2Real-10K at equal total identity count; on AgeDB-30 and CALFW the single-generator HS-10K model remained ahead. The larger HS-10K+Digi2Real-10K fusion improved overall accuracy, and the 20K-scale experiments showed the same pattern. The authors attribute the effect to reduced generator-specific artifacts and increased diversity in pose, lighting, and demographics, which pushes the network to rely on identity-relevant features rather than dataset-specific cues.
Load-bearing premise
The equal-scale comparison assumes that the 5,000-identity subsets taken from each dataset were selected representatively and not chosen to favor the fusion; the paper does not state how those subsets were selected.
Editorial extensions
If this is right
- At equal identity count, combining datasets from different generators is a zero-cost accuracy improvement over single-generator training with the same architecture and loss.
- Larger fused sets continue to improve accuracy, so scaling the number of identities by fusion rather than by asking one generator for more identities is a viable path.
- Because the method changes only the training data and not the model or the loss, it can be dropped into any existing synthetic-data face recognition pipeline.
- The second-place DataCV challenge result indicates the fusion strategy is competitive in a standardized, fixed-pipeline evaluation.
Reading between the lines
- The paper's explanation predicts that the fusion benefit should grow with the architectural distance between generators; a direct test would fuse a diffusion-based dataset with a rendering-based one and compare the gain to fusing two similar GAN outputs.
- The identity non-overlap claim could be checked more rigorously than t-SNE by running nearest-neighbor identity matching between the two datasets with a similarity threshold; if matches appear, part of the observed gain may simply be more images per identity.
- The unstated subset selection is the main threat to the equal-scale comparison; multiple random 5K draws would turn the headline result into a distribution over gains rather than a single point estimate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes training face recognition models on a concatenation of two synthetic face datasets, Digi2Real and Vec2Face (HS), which are generated by architecturally distinct pipelines. The central empirical claim, stated in the conclusion, is that even at the same identity scale, merging datasets from different generation methods improves performance. The authors train IResNet50 models with ArcFace loss on HS-10K, Digi2Real-10K, HS-5K+Digi2Real-5K, and larger variants, and evaluate on the standard benchmarks LFW, CFP-FP, AgeDB-30, CALFW, and CPLFW. They report that the same-identity-scale fused model (HS-5K+Digi2Real-5K, 10K identities) outperforms its individual 10K constituents on three of the five benchmarks. They also report a second-place result in the DataCV ICCV 2025 challenge 10K track with a combined 5K+5K dataset.
Significance. If the same-identity-scale fusion claim is correct, the paper offers a simple and practical recipe for improving synthetic-data face recognition without modifying the architecture or loss, and it directly addresses the known limitation of single-generator overfitting. The evaluation is against external, standard benchmarks and the paper does not fit parameters to those benchmarks, which is a clear strength. The result is falsifiable and cheap to reproduce, and the authors commit to releasing training protocols. The main concerns are that the 5K subset selection is unspecified, the reported gains are within the reported error bars, total image counts are not reported, and the identity non-overlap evidence is based only on t-SNE visualization. These issues currently leave the central claim not fully established, but they are addressable within the scope of a revision.
major comments (4)
- [4.2, Table 1] The central same-identity-scale comparison (HS-5K+Digi2Real-5K versus HS-10K) is not interpretable without knowing how the 5K-identity subsets were selected. Section 4.2 states only that 'we also created subsets containing 5K identities each from Vec2Face and Digi2Real'; it does not say whether the selection was random, stratified, or based on any criterion. If the subsets were selected by a criterion correlated with benchmark difficulty (e.g., embedding diversity, pose coverage, or image quality), the fused model could outperform the full 10K set simply because the subsets are easier, not because fusion helps. The conclusion in Section 5 ('even on the same identity scale, merging datasets... can lead to improved performance') depends directly on this comparison. Please specify the exact selection procedure and release the subset indices or ensure that the selection is reproducible from the released protocol.
- [4.2, Table 1] The reported improvements of HS-5K+Digi2Real-5K over HS-10K are within the reported ± intervals on all three benchmarks where the fused model is better. For LFW, 98.60±0.3 vs 98.92±0.3; for CFP-FP, 88.87±1.9 vs 91.26±1.83; for CPLFW, 85.68±1.9 vs 86.30±2.1. In each case the intervals overlap substantially, so the gains are not statistically distinguishable from noise under the reported variability. Conversely, on AgeDB-30 the fused model is significantly worse (92.78±0.9 vs 89.42±1.8, no overlap). The paper should provide paired statistical tests or a bootstrap analysis across benchmark protocols, or at minimum discuss the effect size relative to the measurement uncertainty; the strong wording of the conclusion should be adjusted accordingly.
- [4.2, Tables 1-2] The paper controls only for the number of identities, not for the number of images. The same-scale comparison presumes that HS-5K+Digi2Real-5K and HS-10K are comparable in total training data, but the paper never reports images per identity or total image counts. If the two source datasets differ in the number of images per identity (the DataCV challenge protocol allows up to 50 per identity, and the actual distribution is not reported), the fusion gain could be attributable to additional training images rather than to cross-generator diversity. Please report the image counts and, if feasible, train an additional control with HS-10K downsampled or supplemented to match the total image count of the fused set.
- [3.3] The claim that the two datasets have non-overlapping identities is supported only by t-SNE plots (Figures 3 and 4). t-SNE is not an appropriate tool for establishing identity non-overlap: it preserves local neighborhoods but not global distances, and the visual separation of clusters does not rule out shared identities appearing in both datasets. Since the fused dataset's identity count (10K) is the basis for the 'same identity scale' comparison, the non-overlap must be quantified with a matching criterion in a face-recognition embedding space (e.g., nearest-neighbor distances between all pairs of identities with a threshold, or cross-dataset mated/non-mated similarity distributions). Please replace or supplement the t-SNE visualization with such an analysis.
minor comments (5)
- [3.2] The section heading reads 'V ec2Face' with an unwanted space; it should be 'Vec2Face'.
- [Figures 3 and 4] The captions use 'T-SNE'; the conventional spelling is 't-SNE'.
- [Table 2] The caption says '20K identities', but the fourth row (HS-20K+Digi2Real-20K) has 40K identities; the caption should be clarified to avoid confusion.
- [4.2] The text states that 'performance degradation occurred in two benchmarks' but does not identify which two; please name them and provide a plausible explanation, especially because one of them (AgeDB-30) shows a statistically significant drop.
- [5] The reproducibility statement promises to release training protocols, but not subset indices. Since the subset selection is essential to the central claim, the release should also include the exact subset indices or the selection script.
Circularity Check
No significant circularity: the fusion claim is an empirical benchmark result, not a derivation from fitted parameters or self-cited uniqueness theorems.
full rationale
The paper's central claim is an empirical comparison: models trained on fused synthetic datasets are evaluated on standard external benchmarks (LFW, CFP-FP, AgeDB-30, CALFW, CPLFW). No parameter of the evaluation is fitted to those benchmarks, and no benchmark result is used to define the fusion rule or the subset construction. The comparison HS-5K+Digi2Real-5K versus HS-10K and Digi2Real-10K is a controlled experiment at equal total identity count, and although the paper does not specify how the 5K subsets were selected, that omission is a reproducibility and selection-bias concern, not a circularity: the outcome is not equal to the input by construction. The t-SNE identity-overlap check using the authors' EdgeFace model is a weak statistical inference rather than a circular definition. The authors cite their own prior work (Digi2Real dataset, EdgeFace model, SynthDistill), but these are external artifacts with independent benchmark support and are not invoked as uniqueness theorems or as premises that already contain the fusion claim. There are no equations in which a predicted quantity is defined in terms of the target result, and no fitted parameter is relabeled as a prediction. The central claim therefore has independent empirical content; at most there is minor self-citation that is not load-bearing.
Assumptions & free parameters
assumptions (3)
- domain assumption The two datasets have non-overlapping identity sets, inferred from t-SNE cluster separation.
- ad hoc to paper The 10K subset selected from Digi2Real-20K and the 5K subsets from each dataset are representative rather than cherry-picked.
- domain assumption Matching the number of identities is a sufficient control for dataset scale in the comparison HS-5K+Digi2Real-5K versus HS-10K.
Cite this review
Pith. "Pith review of Enhancing Domain Diversity in Synthetic Data Face Recognition with Dataset Fusion." pith.science (2026). https://pith.science/paper/RB7A7XJ2
@misc{pith2026250716790,
author = {Pith},
title = {Pith review of: Enhancing Domain Diversity in Synthetic Data Face Recognition with Dataset Fusion},
year = {2026},
howpublished = {\url{https://pith.science/paper/RB7A7XJ2}},
note = {Machine review of arXiv:2507.16790}
}
read the original abstract
While the accuracy of face recognition systems has improved significantly in recent years, the datasets used to train these models are often collected through web crawling without the explicit consent of users, raising ethical and privacy concerns. To address this, many recent approaches have explored the use of synthetic data for training face recognition models. However, these models typically underperform compared to those trained on real-world data. A common limitation is that a single generator model is often used to create the entire synthetic dataset, leading to model-specific artifacts that may cause overfitting to the generator's inherent biases and artifacts. In this work, we propose a solution by combining two state-of-the-art synthetic face datasets generated using architecturally distinct backbones. This fusion reduces model-specific artifacts, enhances diversity in pose, lighting, and demographics, and implicitly regularizes the face recognition model by emphasizing identity-relevant features. We evaluate the performance of models trained on this combined dataset using standard face recognition benchmarks and demonstrate that our approach achieves superior performance across many of these benchmarks.
Figures
Reference graph
Works this paper leans on
-
[1]
Digiface-1m: 1 million digi- tal face images for face recognition
Gwangbin Bae, Martin de La Gorce, Tadas Baltru ˇsaitis, Charlie Hewitt, Dong Chen, Julien Valentin, Roberto Cipolla, and Jingjing Shen. Digiface-1m: 1 million digi- tal face images for face recognition. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3526–3535, 2023. 1, 2
work page 2023
-
[2]
Sface: Privacy-friendly and accurate face recognition using synthetic data
Fadi Boutros, Marco Huber, Patrick Siebke, Tim Rieber, and Naser Damer. Sface: Privacy-friendly and accurate face recognition using synthetic data. In 2022 IEEE International Joint Conference on Biometrics (IJCB) , pages 1–11. IEEE,
work page 2022
-
[3]
Idiff-face: Synthetic-based face recognition through fizzy identity-conditioned diffusion model
Fadi Boutros, Jonas Henry Grebe, Arjan Kuijper, and Naser Damer. Idiff-face: Synthetic-based face recognition through fizzy identity-conditioned diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 19650–19661, 2023. 2
work page 2023
-
[4]
Fadi Boutros, Marcel Klemt, Meiling Fang, Arjan Kuijper, and Naser Damer. Exfacegan: Exploring identity directions in gan’s learned latent space for synthetic identity generation. In 2023 IEEE International Joint Conference on Biometrics (IJCB), pages 1–10. IEEE, 2023. 2
work page 2023
-
[5]
Frcsyn challenge at cvpr 2024: Face recog- nition challenge in the era of synthetic data
Ivan DeAndres-Tame, Ruben Tolosana, Pietro Melzi, Ruben Vera-Rodriguez, Minchul Kim, Christian Rathgeb, Xiaom- ing Liu, Aythami Morales, Julian Fierrez, Javier Ortega- Garcia, et al. Frcsyn challenge at cvpr 2024: Face recog- nition challenge in the era of synthetic data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recogniti...
work page 2024
-
[6]
Ivan DeAndres-Tame, Ruben Tolosana, Pietro Melzi, Ruben Vera-Rodriguez, Minchul Kim, Christian Rathgeb, Xiaom- ing Liu, Luis F Gomez, Aythami Morales, Julian Fierrez, et al. Second frcsyn-ongoing: Winning solutions and post- challenge analysis to improve face recognition with synthetic data. Information Fusion, 120:103099, 2025. 1
work page 2025
-
[7]
Arcface: Additive angular margin loss for deep face recognition
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) , pages 4690–4699, 2019. 1, 5
work page 2019
-
[8]
Disentangled and controllable face image genera- tion via 3d imitative-contrastive learning
Yu Deng, Jiaolong Yang, Dong Chen, Fang Wen, and Xin Tong. Disentangled and controllable face image genera- tion via 3d imitative-contrastive learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5154–5163, 2020. 2
work page 2020
Show all 34 references
-
[9]
Synthetic face datasets generation via latent space exploration from brownian identity diffusion
David Geissb ¨uhler, Hatef Otroshi Shahreza, and S ´ebastien Marcel. Synthetic face datasets generation via latent space exploration from brownian identity diffusion. arXiv preprint arXiv:2405.00228, 2024. 2
2024 arXiv
-
[10]
Digi2real: Bridging the realism gap in synthetic data face recognition via foun- dation models
Anjith George and S ´ebastien Marcel. Digi2real: Bridging the realism gap in synthetic data face recognition via foun- dation models. In Proceedings of the Winter Conference on Applications of Computer Vision, pages 1469–1478, 2025. 2, 3, 4, 5
2025
-
[11]
Edgeface: Efficient face recognition model for edge devices
Anjith George, Christophe Ecabert, Hatef Otroshi Shahreza, Ketan Kotwal, and S ´ebastien Marcel. Edgeface: Efficient face recognition model for edge devices. IEEE Transactions on Biometrics, Behavior, and Identity Science, 6(2):158–168,
-
[12]
Ms-celeb-1m: A dataset and benchmark for large-scale face recognition
Yandong Guo, Lei Zhang, Yuxiao Hu, Xiaodong He, and Jianfeng Gao. Ms-celeb-1m: A dataset and benchmark for large-scale face recognition. In Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Nether- lands, October 11-14, 2016, Proceedings, Part III 14, pages 8...
2016
-
[13]
Labeled faces in the wild: A database forstudying face recognition in unconstrained environments
Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in’Real-Life’Images: detection, align- ment, and recognition, 2008. 5
2008
-
[14]
A style-based generator architecture for generative adversarial networks
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 4401–4410, 2019. 1
2019
-
[15]
Adaface: Quality adaptive margin for face recognition
Minchul Kim, Anil K Jain, and Xiaoming Liu. Adaface: Quality adaptive margin for face recognition. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18750–18759, 2022. 1
2022
-
[16]
Identity-driven three- player generative adversarial network for synthetic-based face recognition
Jan Niklas Kolf, Tim Rieber, Jurek Elliesen, Fadi Boutros, Arjan Kuijper, and Naser Damer. Identity-driven three- player generative adversarial network for synthetic-based face recognition. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition,...
2023
-
[17]
Gandiffface: Controllable generation of synthetic datasets for face recognition with realistic varia- tions
Pietro Melzi, Christian Rathgeb, Ruben Tolosana, Ruben Vera-Rodriguez, Dominik Lawatsch, Florian Domin, and Maxim Schaubert. Gandiffface: Controllable generation of synthetic datasets for face recognition with realistic varia- tions. In Proceedings of the IEEE/CVF Internationa...
2023
-
[18]
Frcsyn challenge at wacv 2024: Face recognition challenge in the era of synthetic data
Pietro Melzi, Ruben Tolosana, Ruben Vera-Rodriguez, Minchul Kim, Christian Rathgeb, Xiaoming Liu, Ivan DeAndres-Tame, Aythami Morales, Julian Fierrez, Javier Ortega-Garcia, et al. Frcsyn challenge at wacv 2024: Face recognition challenge in the era of synthetic data. In Pro- c...
2024
-
[19]
Frcsyn-ongoing: Benchmarking and comprehensive evaluation of real and synthetic data to im- prove face recognition systems
Pietro Melzi, Ruben Tolosana, Ruben Vera-Rodriguez, Minchul Kim, Christian Rathgeb, Xiaoming Liu, Ivan DeAndres-Tame, Aythami Morales, Julian Fierrez, Javier Ortega-Garcia, et al. Frcsyn-ongoing: Benchmarking and comprehensive evaluation of real and synthetic data to im- prove...
2024
-
[20]
Agedb: the first manually collected, in-the-wild age database
Stylianos Moschoglou, Athanasios Papaioannou, Chris- tos Sagonas, Jiankang Deng, Irene Kotsia, and Stefanos Zafeiriou. Agedb: the first manually collected, in-the-wild age database. In proceedings of the IEEE conference on computer vision and pattern recognition workshops , pa...
2017
-
[21]
Arc2face: A foundation model for id-consistent human faces
Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, Jiankang Deng, Bernhard Kainz, and Stefanos Zafeiriou. Arc2face: A foundation model for id-consistent human faces. In ECCV, page 3, 2024. 3
2024
-
[22]
Synface: Face recognition with syn- thetic data
Haibo Qiu, Baosheng Yu, Dihong Gong, Zhifeng Li, Wei Liu, and Dacheng Tao. Synface: Face recognition with syn- thetic data. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 10880–10890, 2021. 2
2021
-
[23]
Learning transferable visual models from natural language supervi- sion
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...
2021
-
[24]
Synthetic to authentic: Transferring realism to 3d face renderings for boosting face recognition
Parsa Rahimi, Behrooz Razeghi, and Sebastien Marcel. Synthetic to authentic: Transferring realism to 3d face renderings for boosting face recognition. arXiv preprint arXiv:2407.07627, 2024. 2
2024 arXiv
-
[25]
High-resolution image synthesis with latent diffusion models
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 1
2022
-
[26]
Frontal to profile face verification in the wild
Soumyadip Sengupta, Jun-Cheng Chen, Carlos Castillo, Vishal M Patel, Rama Chellappa, and David W Jacobs. Frontal to profile face verification in the wild. In 2016 IEEE winter conference on applications of computer vision (WACV), pages 1–9. IEEE, 2016. 5
2016
-
[27]
Synthdistill: Face recognition with knowledge distilla- tion from synthetic data
Hatef Otroshi Shahreza, Anjith George, and S ´ebastien Mar- cel. Synthdistill: Face recognition with knowledge distilla- tion from synthetic data. In 2023 IEEE International Joint Conference on Biometrics (IJCB), pages 1–8. IEEE, 2023. 2
2023
-
[28]
Sdfr: Synthetic data for face recognition competition
Hatef Otroshi Shahreza, Christophe Ecabert, Anjith George, Alexander Unnervik, S ´ebastien Marcel, Nicol `o Di Domenico, Guido Borghi, Davide Maltoni, Fadi Boutros, Julia V ogel, et al. Sdfr: Synthetic data for face recognition competition. In 2024 IEEE 18th International Conf...
2024
-
[29]
Knowledge distillation for face recognition using syn- thetic data with dynamic latent sampling
Hatef Otroshi Shahreza, Anjith George, and S ´ebastien Mar- cel. Knowledge distillation for face recognition using syn- thetic data with dynamic latent sampling. IEEE Access ,
-
[30]
Fake it till you make it: face analysis in the wild using synthetic data alone
Erroll Wood, Tadas Baltru ˇsaitis, Charlie Hewitt, Sebastian Dziadzio, Thomas J Cashman, and Jamie Shotton. Fake it till you make it: face analysis in the wild using synthetic data alone. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3681–36...
2021
-
[31]
Vec2face: Scaling face dataset generation with loosely constrained vectors
Haiyu Wu, Jaskirat Singh, Sicong Tian, Liang Zheng, and Kevin W Bowyer. Vec2face: Scaling face dataset generation with loosely constrained vectors. ICLR, 2025. 2, 3, 4, 5
2025
-
[32]
Cross-pose lfw: A database for studying cross-pose face recognition in un- constrained environments
Tianyue Zheng and Weihong Deng. Cross-pose lfw: A database for studying cross-pose face recognition in un- constrained environments. Beijing University of Posts and Telecommunications, Tech. Rep, 5(7), 2018. 5
2018
-
[33]
Cross-age lfw: A database for studying cross-age face recognition in un- constrained environments
Tianyue Zheng, Weihong Deng, and Jiani Hu. Cross-age lfw: A database for studying cross-age face recognition in un- constrained environments. arXiv preprint arXiv:1708.08197,
-
[34]
Webface260m: A benchmark unveiling the power of million-scale deep face recognition
Zheng Zhu, Guan Huang, Jiankang Deng, Yun Ye, Junjie Huang, Xinze Chen, Jiagang Zhu, Tian Yang, Jiwen Lu, Da- long Du, et al. Webface260m: A benchmark unveiling the power of million-scale deep face recognition. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and...
2021
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.