REVIEW 4 major objections 6 minor 3 references
Excluding children's images from training makes all nine tested facial age estimation methods fail to generalize to under-18 faces, degrading performance by 46.4% on average and up to 52.8%.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 12:52 UTC pith:M7V3APB6
load-bearing objection Useful benchmark with a real result, but the headline 46.4% degradation is not a clean like-for-like comparison and should be re-estimated. the 4 major comments →
Toward a More Ethical Facial Age Estimation: A Generalized Zero-Shot Benchmark Without Training on Children's Data
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central finding is empirical and uniform: every evaluated method, across all six datasets, fails to generalize to age groups absent from training. Under the proposed GZSL protocol, seen-class MAE stays near the supervised level (4.84±0.12 years), but unseen-class MAE jumps to 12.07±0.46 years, and the harmonic-mean metric degrades by an average of 46.4% (up to 52.8% per method, and 101.8% in the worst single case). The failure mode is seen-class bias: predictions for unseen ages hug the closest seen adult ages rather than extrapolating. The authors attribute this to the supervised nature of current regression, label-distribution, and ordinal-ranking losses, which concentrate lear
What carries the argument
The central object is a generalized zero-shot benchmark for age estimation. Age intervals are treated as classes: adults 18–59 are seen classes used in training; elderly 60+ are unseen validation classes; children under 18 are unseen test classes. For datasets with identity annotations, a greedy algorithm enforces subject-age exclusivity so no person straddles the seen/unseen boundary. Evaluation uses seen MAE, unseen MAE, and their harmonic mean. The benchmark's role is to turn the ethical restriction (no children's data in training) into a measurable generalization problem, while exposing whether current methods extrapolate across the age ordinal scale.
Load-bearing premise
The benchmark's credibility rests on the splits being truly zero-shot: no child or elderly image may leak into training, and no person may appear on both sides of the seen/unseen boundary—and for the two datasets without subject IDs, that condition cannot be checked or guaranteed.
What would settle it
Train a model only on adults (18–59) using these splits and evaluate on the children's test subset; if any method reaches an unseen MAE close to the supervised baseline (around 4–5 years) rather than about 12 years, the claim of uniform failure is wrong. A more direct check: remove identity-overlapping images from the datasets that currently lack subject annotations and see whether the degradation shrinks; if leakage is substantial, the 46.4% figure is not a clean measure of the cost of excluding children.
If this is right
- Any age-estimation system trained only on adults will systematically mis-estimate children's ages, anchoring them to the adult range; this is a concrete deployment risk for age-gating and safety tools.
- The 46.4% degradation quantifies the accuracy price of excluding minors' data, giving practitioners and regulators a number to weigh against the ethical benefits.
- Label-distribution and ordinal methods do not solve zero-shot extrapolation; label smoothing near seen ages is insufficient, so new losses or embeddings are needed.
- The benchmark's splits enable standardized comparison and model selection under restriction, using elderly as unseen validation without any child images in development.
- Reporting seen and unseen MAE separately, plus their harmonic mean, is an evaluation template for age estimators expected to work across the lifespan.
Where Pith is reading between the lines
- The AFAD and MORPH results show that 'unseen children' is not uniform: ages 15–17 lie close to the adult boundary, so those datasets offer an easier zero-shot test; benchmark difficulty should be reported per age-interval gap, not just per dataset.
- Because two datasets lack identity annotations, their splits cannot guarantee that the same person is not in both seen training and unseen test; an identity-aware re-collection would test whether the 46.4% figure is partly inflated by leakage.
- The elderly-as-validation trick assumes extrapolation to 60+ behaves like extrapolation to under 18; if those two margins are not equally hard, model selection by harmonic mean may not predict child-test performance.
- The same generalized zero-shot formulation could be applied to other sensitive demographic groups whose images should be excluded from training, not just minors.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a generalized zero-shot learning (GZSL) benchmark for facial age estimation in which models are trained only on adult images (ages 18–59), validated on an unseen elderly group (60+), and tested on an unseen children/adolescent group (<18). The authors construct standardized age- and (where possible) subject-exclusive splits for six datasets (AFAD, AgeDB, CACD2000, CLAP2016, UTKFace, MORPH) via a greedy assignment algorithm, and evaluate nine age-estimation methods spanning regression, classification, label distribution learning, and ordinal ranking. Their main finding is that all methods degrade sharply on unseen age groups: aggregate seen MAE is 4.84±0.12 versus unseen MAE 12.07±0.46, with an average degradation of 46.4% (up to 52.8%) relative to the supervised baseline, and up to 101.8% in one per-dataset case. The authors also show that predictions for unseen ages are systematically anchored to nearby seen ages, a seen-class bias analogous to that known in GZSL classification. The paper frames this benchmark as a step toward ethical age estimation that avoids training on children's data.
Significance. If the quantitative claims are properly established, this is a timely and useful contribution. The benchmark addresses a real ethical constraint—avoiding children's images in training—and formalizes it as a GZSL problem, which is more appropriate than conventional ZSL for deployed age estimators. The paper provides reproducibility assets (code, splits), evaluates a broad range of methods (nine methods × six datasets), and the qualitative result that current methods fail to extrapolate beyond the adult training interval is consistent across all datasets and paradigms. The analysis of seen-class anchoring is a useful diagnostic. The paper is also careful in its ethical discussion, acknowledging limitations and potential misuse. The main weakness is that the headline degradation figure is computed by comparing non-commensurate metrics and test distributions, which undermines the precise quantitative claim even if the qualitative conclusion is credible.
major comments (4)
- [Results and Discussion, Table 4 vs Table 5, footnote 2] The headline '46.4% average degradation' compares the GZSL harmonic mean H = 2·S·U/(S+U) with the standard overall MAE from the supervised baseline in Table 4. These are not commensurate. H is an unweighted harmonic mean of seen and unseen group MAEs, not a sample-weighted overall MAE; for the aggregate numbers (S=4.84, U=12.07, H≈6.70), H is substantially lower than a per-sample overall MAE would be, and the supervised baseline is evaluated on different Paplhám–Franc test splits that include all ages. Thus the 46.4% (and the 'up to 52.8%' figure) conflates the protocol change with a metric change and a test-distribution change. The authors should report a supervised model evaluated on the exact GZSL test split and reduced to the same metric (both overall MAE and harmonic mean), or explicitly present the degradation as the gap between S and U rather than against the supervised baseline.
- [Generalized Zero-Shot Learning Benchmark, Table 2] The GZSL training set excludes not only children (0–17) but also elderly (60+), and the subject-exclusive split construction discards a large fraction of images—AgeDB discards 10,814 of 16,488 images (65.6%) and retains only 1,270 training images; CACD2000 discards 20,543 (12.6%). Consequently, the comparison to the supervised baseline reflects not only 'no children in training' but also the removal of the elderly group and a substantial reduction in training data. The conclusion that the degradation is 'the cost of excluding children' is therefore confounded. The authors should at least report a supervised upper bound trained on the same filtered adult-only images (with the same training set size) or provide per-age-group supervised MAE on the same GZSL test splits to isolate the zero-shot effect. Without this, the attribution of the full degradation to the absence of children's data is
- [Experimental Setup, Table 1 and benchmark description] The benchmark section claims that 'subject-age exclusivity' prevents identity leakage, but Table 1 shows that CLAP2016 and UTKFace have no subject annotations and therefore undergo no exclusivity filtering. For these two datasets, the same individual could appear in both seen and unseen test partitions, providing identity anchors that violate the zero-shot assumption. The authors acknowledge this in the text but still treat all six datasets under the same subject-age-exclusivity umbrella. The manuscript should explicitly label CLAP2016 and UTKFace as age-exclusive only, and discuss whether this could affect the conclusions. Given that the degradation is consistent across all datasets, this is a validity caveat rather than a fatal flaw, but it is a limitation of the benchmark as stated.
- [Results and Discussion, Table 2 and Table 5] AgeDB's unseen test set contains only 95 images, making its per-dataset harmonic means (e.g., CORAL 12.49, DEX 11.90) statistically fragile. No per-seed or bootstrap error bars are reported; the standard deviations in Table 5 are across methods, not independent runs. For a benchmark that aims to be a standardized measurement protocol, the authors should provide variance across random seeds or bootstrap confidence intervals, particularly for small partitions like AgeDB's unseen test. This does not overturn the consistent qualitative finding, but it limits the reliability of quantitative per-dataset comparisons and of the 'up to 101.8%' claim.
minor comments (6)
- [Introduction] Typo: 'properly collected concent' should be 'properly collected consent'.
- [References] In the Schuhmann et al. 2022 reference, 'and ohers' should be 'and others'.
- [Table 1] Header 'A vailable' has an extra space; should be 'Available'.
- [Algorithm 1] The notation for FOLDER targets and running counts is terse; a one-sentence explanation of t_k and r_k in the caption would improve readability.
- [Generalized Zero-Shot Learning Benchmark] The use of all-caps 'FOLDER' is unconventional; consider lowercase 'folder' or a named split label (e.g., Split0...Split4).
- [Results and Discussion] The sentence 'On seen classes, methods achieve an average MAE of 4.84±0.12, close to supervised levels' is not directly supported, because the supervised baseline MAE in Table 4 is an overall MAE including children and elderly, not an adult-only MAE. This should be clarified or recomputed.
Circularity Check
No significant circularity: degradation is measured, not fitted; metric-mix is a comparability concern, not a construction.
full rationale
The paper's central quantity is a direct measurement: Table 5 reports seen (S) and unseen (U) MAEs from models trained on adult-only splits, and the '46.4%' degradation is computed by averaging per-method percentage differences between the GZSL harmonic mean and the supervised baseline MAE (footnote 2). No parameter is fitted to produce the degradation, and no equation defining the benchmark presupposes the failure. The comparison does mix metrics (harmonic mean vs plain MAE) and test distributions (Paplhám–Franc splits vs the paper's GZSL splits), which is a genuine validity/comparability limitation: a supervised model evaluated on the same GZSL split and reduced to the same harmonic-mean metric could yield a different magnitude. However, the paper explicitly states 'when measured by the harmonic mean of seen/unseen MAE,' so this is a transparent methodological choice, not a circular reduction. Self-citations (Caetano et al. 2025; dos Santos et al. 2022) are motivational/contextual and are not load-bearing for the measured results; they are external audits and dataset citations. The absence of subject-ID annotations for CLAP2016 and UTKFace is a benchmark-validity limitation (identity leakage potential), not a circularity. The seen-class-bias interpretation is a post-hoc qualitative explanation, not an input to any calculation. Thus the derivation chain is self-contained with respect to circularity, and the score is 0.
Axiom & Free-Parameter Ledger
free parameters (4)
- Age thresholds: seen adults 18-59, unseen children <18, unseen elderly 60+ =
18/59/60 boundaries
- Adult split fractions =
80% train / 10% validation / 10% test
- Per-method learning rate and weight decay =
See Table 3; e.g., Regression 5.1067e-05 / 1.3163e-06, CORN 7.7920e-06 / 9.6323e-05 in GZSL
- Batch size per dataset
axioms (5)
- domain assumption Age labels in AFAD, AgeDB, CACD2000, CLAP2016, UTKFace, and MORPH are sufficiently accurate for benchmarking.
- domain assumption ResNet50 pretrained on ImageNet is a neutral shared backbone for comparing the nine methods.
- domain assumption Xian et al.'s generalized zero-shot learning framework, designed for classification, transfers to ordinal age regression.
- domain assumption Datasets without subject annotations (UTKFace, CLAP2016) do not suffer material identity leakage across seen and unseen splits.
- domain assumption MAE and harmonic-mean MAE are appropriate for comparing supervised and GZSL performance.
Cite this review
Pith. "Pith review of Toward a More Ethical Facial Age Estimation: A Generalized Zero-Shot Benchmark Without Training on Children's Data." pith.science (2026). https://pith.science/paper/M7V3APB6
@misc{pith2026260529230,
author = {Pith},
title = {Pith review of: Toward a More Ethical Facial Age Estimation: A Generalized Zero-Shot Benchmark Without Training on Children's Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/M7V3APB6}},
note = {Machine review of arXiv:2605.29230}
}
read the original abstract
Age estimation from facial images typically relies on training data that includes images of minors, a practice that raises ethical, legal, and privacy concerns and that child-data governance frameworks explicitly advise against. While the task remains relevant (e.g., for detecting child sexual abuse imagery), we advocate against using data from minors entirely and quantify what the exclusion costs in accuracy. We formalize age estimation without children's training data as a generalized zero-shot learning (GZSL) problem: age intervals present during training are seen classes and withheld intervals are unseen, with models evaluated jointly on both. The generalized setting, rather than conventional zero-shot evaluation on unseen classes alone, is the appropriate one here because a deployed estimator must operate across the entire lifespan, not only on the interval withheld from it. Revisiting six widely used datasets, we introduce standardized splits with strict age-group separation. For datasets with identity annotations, subject-age-exclusive splits prevent identity leakage across the seen/unseen boundary. Evaluating nine state-of-the-art age estimation methods under this protocol reveals that all of them fail to generalize to unseen age groups, suffering substantial degradation --- on average 46.4%, and up to 52.8% --- relative to the supervised baseline. Moreover, models do not simply degrade: they systematically anchor predictions for unseen ages to nearby seen classes, a manifestation of the well-known seen-class bias in generalized zero-shot learning.
Figures
Reference graph
Works this paper leans on
-
[2013]
InIEEE Conference on Computer Vision and Pattern Recog- nition, 819–826
Label-Embedding for Attribute-Based Classification. InIEEE Conference on Computer Vision and Pattern Recog- nition, 819–826. Akata, Z.; Reed, S.; Walter, D.; Lee, H.; and Schiele, B
-
[2015]
InIEEE Conference on Computer Vision and Pattern Recognition, 2927–2936
Evaluation of output embeddings for fine-grained im- age classification. InIEEE Conference on Computer Vision and Pattern Recognition, 2927–2936. Akiba, T.; Sano, S.; Yanase, T.; Ohta, T.; and Koyama, M
-
[2019]
InACM International Conference on Knowledge Discovery & Data Mining, 2623–2631
Optuna: A Next-Generation Hyperparameter Opti- mization Framework. InACM International Conference on Knowledge Discovery & Data Mining, 2623–2631. Birhane, A.; Prabhu, V . U.; and Kahembwe, E. 2021. Mul- timodal datasets: misogyny, pornography, and malignant stereotypes.arXiv preprint arXiv:2110.01963. Caetano, C.; Santos, G. O. d.; Petrucci, C.; Barros, ...
Pith/arXiv arXiv 2021
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.