Comparative Analysis of Machine Learning Approaches for Bone Age Assessment: A Comprehensive Study on Three Distinct Models

Nandavardhan R.; Savaridassan P; Somanathan R.; Vikram Suresh

REVIEW 4 major objections 6 minor 12 references

Comparative Analysis of Machine Learning Approaches for Bone Age Assessment: A Comprehensive Study on Three Distinct Models

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper reports that Xception achieves the lowest mean absolute error (12.6 months) among three bone-age models, ahead of a custom CNN (22.6 months) and VGG (34.9 months).

desk verdict A naive three-model benchmark on bone age whose ranking is confounded by inconsistent preprocessing and missing training details; not ready for peer review. read the letter →

arxiv 2411.10345 v1 pith:DZV3QMBL submitted 2024-11-15 cs.CV cs.LG

Nandavardhan R. , Somanathan R. , Vikram Suresh , Savaridassan P This is my paper · ORCID

classification cs.CVcs.LG

keywords boneageassessmentXceptionVGGconvolutionalneuralnetworkmeanabsoluteerrorhandX-raydeeplearningRSNAdataset

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks which of three deep-learning models—Xception, VGG, and a custom CNN—should be used to automate bone age assessment from hand X-rays. It reports that Xception is the most accurate, with a mean absolute error of 12.6 months, compared with 22.6 months for the CNN and 34.9 months for VGG. The practical target is to help medical institutions choose between accuracy, model size, and ease of implementation. The paper concludes that Xception wins on accuracy but is the most resource-intensive, while the CNN is the lightest and VGG the easiest to deploy.

What carries the argument

The central objects are the three architectures: a custom CNN built from alternating Conv2D, batch-normalization, and max-pooling layers with a dense regression head; VGG, a standardized deep CNN modified for single-value age output; and Xception, built from depthwise separable convolutions—a depthwise filter applied per channel followed by a 1×1 pointwise convolution—together with residual connections. The comparison is carried by mean absolute error in months, computed after Keras-based preprocessing in which Xception was trained at 128×128 resolution while the other two models used 256×256.

What would settle it

Retrain all three models from the same starting point, at the same input resolution, with the same optimizer, learning-rate schedule, and number of epochs, and evaluate them on the same held-out split; the claimed ordering fails if the CNN or VGG matches or beats Xception's 12.6-month mean absolute error.

Watch

Extended reading notes

Core claim

On the RSNA Bone Age dataset, the paper reports a clear empirical accuracy ranking: Xception reaches a mean absolute error of 12.6 months, the custom CNN reaches 22.6 months, and VGG reaches 34.9 months. The paper interprets this as showing that Xception is the most accurate of the three models, while the CNN is the most lightweight at roughly 3.7 million parameters, VGG is the easiest to implement, and Xception is the largest and most resource-intensive at roughly 20.9 million parameters. The intended contribution is a practical comparison that lets radiologists and institutions pick a model based on their accuracy needs and hardware constraints.

Load-bearing premise

The ranking depends on the three models being trained and evaluated under comparable conditions, and the paper does not state whether pretrained weights, training settings, or an unseen test set were used uniformly.

Editorial extensions

If this is right

If the reported ordering is correct, an institution that prioritizes accuracy should choose Xception when its compute budget allows.
A resource-constrained deployment would reasonably choose the custom CNN, accepting roughly 10 months of additional mean absolute error to save about 17 million parameters.
VGG offers the simplest implementation path but the lowest accuracy of the three, making it a fallback for teams with limited deep-learning expertise.
The same MAE metric allows the three models to be compared directly, but it does not by itself establish clinical adequacy.
Larger model size does not guarantee higher accuracy in this comparison, since VGG is larger than the CNN yet less accurate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because Xception was preprocessed at 128×128 while the CNN and VGG used 256×256, the reported accuracy gap may partly reflect input resolution rather than architecture; an equal-resolution retraining would separate those factors.
The paper does not compare its MAE values with inter-observer variability of the Greulich–Pyle or Tanner–Whitehouse methods, so the clinical significance of a 12.6-month error is left open.
If the ranking survives matched training conditions, the natural deployment rule is a two-point trade-off: choose Xception when accuracy dominates and hardware permits, and choose the CNN when memory is the binding constraint.
The reported parameter counts and MAE values could be turned into a simple accuracy-per-parameter ratio for future model selection if the training conditions were standardized.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

If you need one sentence: this is a student-level comparison of Xception, VGG, and a custom CNN on RSNA bone age data, and the headline result (Xception MAE 12.6 vs CNN 22.6 vs VGG 34.9 months) cannot be trusted as stated because the models were not trained or evaluated under comparable conditions. The paper does one thing that is marginally useful: it reports raw MAE and parameter counts for three standard architectures on a public dataset, and the numbers are internally consistent in the sense that bigger models do not automatically win. That is a legitimate small data point, and the prose is honest about the tradeoff between accuracy and model size. But that is where the credit ends.

The soft spots are substantial and load-bearing. Section II says VGG and CNN get 256x256 greyscale images, while Xception gets 128x128 with Keras built-in normalization. That alone makes the accuracy comparison meaningless as an architecture ranking. Worse, there is no mention of pretrained weights. If Xception was initialized from ImageNet (the Keras default) and the others were trained from scratch, the gap is fully explained by transfer learning, not by architecture. The paper also never states which split produced the reported MAE (validation likely, but it says "validation images" in Methods and never mentions a test set), reports no error bars or repeated runs, and gives no epochs, optimizer, learning rate, or batch size. On top of that, the VGG layer table sums to about 20.35 million parameters, not the reported 17.99 million, so the described model does not match the reported count. Finally, the MAEs (12.6–34.9 months) are far above published bone-age models (often 4–6 months), and the paper does not compare against any of them.

Who gets value from this? Possibly a reader who wants a rough back-of-the-envelope feel for these three networks on this dataset, but not a clinician, not a researcher looking for a benchmark, and not anyone making procurement decisions. The paper is a cautionary example of how to run a model comparison without controlling for confounds.

My recommendation: desk reject. It does not deserve serious referee time in current form. If the authors release code, training protocols, error bars, consistent preprocessing, and a comparison with published baselines, then a resubmission could be evaluated on its merits. As it stands, the central claim is unverifiable and the experimental design is too weak to support any conclusion.

Referee Report

4 major / 6 minor

Summary. This paper reports an empirical comparison of three deep learning models (Xception, VGG, and a custom CNN) for bone age assessment from hand X-ray images using the RSNA Bone Age dataset. The authors report mean absolute error (MAE) in months: Xception 12.6, CNN 22.6, and VGG 34.9, and they conclude that Xception is the most accurate but also the largest and most resource-intensive, while the custom CNN provides a light-weight alternative. The paper includes architecture tables with parameter counts for each model and a brief description of the data preprocessing.

Significance. If the reported ranking is reliable, the paper provides a practically useful comparison for clinicians and engineers choosing a bone-age model under different hardware constraints. The use of a public dataset and a uniform evaluation metric (MAE) are strengths, as are the explicitly tabulated model architectures. However, the paper currently lacks the experimental detail needed to establish that the MAE differences are attributable to model architecture rather than to training choices, input resolution, or transfer-learning status. The significance is therefore conditional on the authors providing a much more complete experimental account and resolving an internal inconsistency in the reported VGG parameter count.

major comments (4)

[Section II, Methods (preprocessing)] The paper states that VGG and CNN receive 256x256 greyscale images, while Xception receives 128x128 images followed by a Keras built-in normalization. This means the models differ not only in architecture but also in input resolution and preprocessing. The central claim that Xception is the most accurate model is therefore confounded: the MAE gap could be due to input size or normalization rather than architecture. The authors must either use identical preprocessing for all models or run a controlled experiment that isolates the architecture effect.
[Section II, Xception Model; Section III, Results] The manuscript never states whether the Xception model was initialized with ImageNet-pretrained weights, which is the default in Keras, while the custom CNN and the modified VGG are described as built from scratch. If transfer learning was used only for Xception, the observed MAE ordering is expected and does not support a conclusion about architectural superiority. This is a load-bearing omission: the authors must disclose the weight initialization and, if transfer learning was used, either compare all models under the same transfer-learning regime or clearly temper the claim.
[Section III, Results; Section II, Dataset] The paper reports MAE values but does not state whether these are computed on the 1425-image validation split or on a separate held-out test set. It also omits the number of epochs, optimizer, learning rate, batch size, loss function, and any repeated runs or error bars. Without these details, the reported MAE values cannot be reproduced or verified, and the reader cannot assess whether the differences among 12.6, 22.6, and 34.9 months are meaningful. The authors should specify the evaluation protocol and provide uncertainty estimates or at least state that the numbers come from a single run.
[Section II, VGG Model table; Section III, Results] The VGG architecture table lists per-layer parameter counts that sum to 20,351,265, not the 17,991,457 reported in the Results and Conclusions. This is an internal inconsistency: either the table or the reported total is wrong, and it undermines the reader's confidence that the described model is the one that was actually trained. The authors must reconcile these numbers and verify that all parameter counts correspond to the trained models.

minor comments (6)

[Section I, Background] The abstract and introduction claim the three models are 'the most widely used' for bone age assessment, but no citation or quantitative support is provided for this claim; please justify or rephrase.
[Section II, Methods] For the Xception preprocessing, the paper says 'the inbuilt Keras function to normalize the images' without naming it; please specify whether preprocess_input is used and whether the images are converted to three-channel RGB.
[Section II, Figures] Figure 3 ('Zscore of validation and training images') is not described in the text, and its axes are not labeled; please clarify what is plotted and why it is included.
[Section II and IV] There are typographical issues, including 'MaxPooIing2D' in the VGG table and inconsistent wording about the VGG model's implementation ease; a careful proofread is needed.
[Section IV, Conclusions] The conclusions state the VGG model has the highest MAE while also calling it 'the easiest to implement,' but no evidence for ease of implementation is provided beyond parameter count; please either justify or soften this claim.
[Section III, Results] The paper does not compare the reported MAE values with published results on the RSNA Bone Age dataset (e.g., from the RSNA challenge), which would help readers calibrate the accuracy of the models; please add such context.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the model accuracy ranking is an empirical measurement, not a derivation that reduces to its inputs.

full rationale

The paper reports measured Mean Absolute Error values for three models (Xception 12.6 months, CNN 22.6 months, VGG 34.9 months) and compares them. There is no theoretical derivation chain in which an output quantity is defined in terms of the quantity it is claimed to predict, no fitted parameter is renamed as a prediction, and no self-citation is used as load-bearing evidence for the ranking. The MAE numbers are direct empirical outcomes of training and evaluation, so they are not circular by construction. The weaknesses noted by a skeptical reader—differing input resolutions, undisclosed use of pretrained weights, unclear train/validation split, and inconsistent parameter counts—are threats to comparability and reproducibility, not circularity. Those concerns affect whether the ranking is well-supported, but they do not make the reported measurements equivalent to the paper's inputs. Hence the circularity score is 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

No derivation, no invented entities, and no externally fitted constants. The comparison rests on an unstated comparability assumption about training conditions and on trust in the RSNA labels as ground truth. The model weights are learned parameters, not free parameters introduced to make a derivation work.

assumptions (2)

domain assumption The labels in the RSNA Bone Age dataset accurately represent true bone age.
The paper relies on these labels as ground truth for computing MAE, but does not validate them against radiologist re-readings.
ad hoc to paper The three models were trained and evaluated under sufficiently comparable conditions (same data, similar training budget, consistent evaluation) so that MAE differences reflect architecture.
Section II provides architecture details and preprocessing, but omits optimizer, epochs, learning rate, batch size, pretraining status, and test split, leaving comparability unverified.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Comparative Analysis of Machine Learning Approaches for Bone Age Assessment: A Comprehensive Study on Three Distinct Models." pith.science (2026). https://pith.science/paper/DZV3QMBL

@misc{pith2026241110345,
  author       = {Pith},
  title        = {Pith review of: Comparative Analysis of Machine Learning Approaches for Bone Age Assessment: A Comprehensive Study on Three Distinct Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DZV3QMBL}},
  note         = {Machine review of arXiv:2411.10345}
}

read the original abstract

Radiologists and doctors make use of X-ray images of the non-dominant hands of children and infants to assess the possibility of genetic conditions and growth abnormalities. This is done by assessing the difference between the actual extent of growth found using the X-rays and the chronological age of the subject. The assessment was done conventionally using The Greulich Pyle (GP) or Tanner Whitehouse (TW) approach. These approaches require a high level of expertise and may often lead to observer bias. Hence, to automate the process of assessing the X-rays, and to increase its accuracy and efficiency, several machine learning models have been developed. These machine-learning models have several differences in their accuracy and efficiencies, leading to an unclear choice for the suitable model depending on their needs and available resources. Methods: In this study, we have analyzed the 3 most widely used models for the automation of bone age prediction, which are the Xception model, VGG model and CNN model. These models were trained on the preprocessed dataset and the accuracy was measured using the MAE in terms of months for each model. Using this, the comparison between the models was done. Results: The 3 models, Xception, VGG, and CNN models have been tested for accuracy and other relevant factors.

Figures

Figures reproduced from arXiv: 2411.10345 by the authors.

**Figure 3.** Zscore of validation and training images respectively [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 12 canonical work pages

[1]

Intelligent Bone Age Assessment: An Automated System to Detect a Bone Growth Problem Using Convolutional Neural Networks with Attention Mechanism

Zulkifley, M.A.; Mohamed, N.A.; Abdani, S.R.; Kamari, N.A.M.; Moubark, A.M.; Ibrahim, A.A. Intelligent Bone Age Assessment: An Automated System to Detect a Bone Growth Problem Using Convolutional Neural Networks with Attention Mechanism. Diagnostics 2021, 11, 765

work page 2021
[2]

Radiographic Atlas of Skeletal Development of the Hand and Wrist; Stanford University Press: Palo Alto, CA, USA, 1959

Greulich, W.W.; Pyle, S.I. Radiographic Atlas of Skeletal Development of the Hand and Wrist; Stanford University Press: Palo Alto, CA, USA, 1959

work page 1959
[3]

Assessment of skeletal maturity and prediction of adult height (TW2 method); Saunders: London, UK, 2001; pp

Tanner, J.M.; Whitehouse, R.H.; Cameron, N.; Marshall, W.A.; Healy, M.J.R.; Goldstein, H. Assessment of skeletal maturity and prediction of adult height (TW2 method); Saunders: London, UK, 2001; pp. 1–110

work page 2001
[4]

Paediatric bone age assessment using deep convolutional neural networks

Iglovikov, V.I.; Rakhlin, A.; Kalinin, A.A.; Shvets, A.A. Paediatric bone age assessment using deep convolutional neural networks. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support; Springer: Cham, Switzerland, 2018; pp. 300– 308

work page 2018
[5]

Shufflenet v2: Practical Guidelines for Efficient CNN Architecture Design

Ma, N.; Zhang, X.; Zheng, H.T.; Sun, J. Shufflenet v2: Practical Guidelines for Efficient CNN Architecture Design. In Proceedings of the European conference on computer vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 116–131

work page 2018
[6]

Using Convolutional Neural Networks and Transfer Learning for Bone Age Classification

Zhou, J.; Li, Z.; Zhi, W.; Liang, B.; Moses, D.; Dawes, L. Using Convolutional Neural Networks and Transfer Learning for Bone Age Classification. In Proceedings of the 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Sydney, Australia, 29 November–1 December 2017; pp. 1– 6

work page 2017
[7]

Deep learning for automated skeletal bone age assessment in X -ray images

Spampinato, C.;Palazzo, S.; Giordano, D.; Aldinucci, M.; Leonardi, R. Deep learning for automated skeletal bone age assessment in X -ray images. Med Image Anal. 2017, 36, 41–51

work page 2017
[8]

Very deep convolutional networks for large-scale image recognition

Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition

work page

Show all 12 references

[9]

Fully automated deep learning system for bone age assessment

Lee, H.; Tajmir, S.; Lee, J.; Zissen, M.; Yeshiwas, B.A.; Alkasab, T.K.; Choy, G.; Do, S. Fully automated deep learning system for bone age assessment. J. Digit. Imaging 2017, 30, 427–441

2017
[10]

Xception: Deep Learning with Depthwise Separable Convolutions

Chollet, F. Xception: Deep Learning with Depthwise Separable Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 21–26 July 2017; pp. 1251–1258

2017
[11]

Fully automated bone age assessment on large-scale hand X-ray dataset

Pan, X.; Zhao, Y.; Chen, H.; Wei, D.; Zhao, C.; Wei, Z. Fully automated bone age assessment on large-scale hand X-ray dataset. Int. J. Biomed. Imaging 2020, 1–12

2020
[12]

Rethinking the Inception Architecture for Computer Vision

Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the Inception Architecture for Computer Vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 27–30 June 2016; pp. 2818–2826

2016

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Intelligent Bone Age Assessment: An Automated System to Detect a Bone Growth Problem Using Convolutional Neural Networks with Attention Mechanism

Zulkifley, M.A.; Mohamed, N.A.; Abdani, S.R.; Kamari, N.A.M.; Moubark, A.M.; Ibrahim, A.A. Intelligent Bone Age Assessment: An Automated System to Detect a Bone Growth Problem Using Convolutional Neural Networks with Attention Mechanism. Diagnostics 2021, 11, 765

work page 2021

[2] [2]

Radiographic Atlas of Skeletal Development of the Hand and Wrist; Stanford University Press: Palo Alto, CA, USA, 1959

Greulich, W.W.; Pyle, S.I. Radiographic Atlas of Skeletal Development of the Hand and Wrist; Stanford University Press: Palo Alto, CA, USA, 1959

work page 1959

[3] [3]

Assessment of skeletal maturity and prediction of adult height (TW2 method); Saunders: London, UK, 2001; pp

Tanner, J.M.; Whitehouse, R.H.; Cameron, N.; Marshall, W.A.; Healy, M.J.R.; Goldstein, H. Assessment of skeletal maturity and prediction of adult height (TW2 method); Saunders: London, UK, 2001; pp. 1–110

work page 2001

[4] [4]

Paediatric bone age assessment using deep convolutional neural networks

Iglovikov, V.I.; Rakhlin, A.; Kalinin, A.A.; Shvets, A.A. Paediatric bone age assessment using deep convolutional neural networks. In Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support; Springer: Cham, Switzerland, 2018; pp. 300– 308

work page 2018

[5] [5]

Shufflenet v2: Practical Guidelines for Efficient CNN Architecture Design

Ma, N.; Zhang, X.; Zheng, H.T.; Sun, J. Shufflenet v2: Practical Guidelines for Efficient CNN Architecture Design. In Proceedings of the European conference on computer vision (ECCV), Munich, Germany, 8–14 September 2018; pp. 116–131

work page 2018

[6] [6]

Using Convolutional Neural Networks and Transfer Learning for Bone Age Classification

Zhou, J.; Li, Z.; Zhi, W.; Liang, B.; Moses, D.; Dawes, L. Using Convolutional Neural Networks and Transfer Learning for Bone Age Classification. In Proceedings of the 2017 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Sydney, Australia, 29 November–1 December 2017; pp. 1– 6

work page 2017

[7] [7]

Deep learning for automated skeletal bone age assessment in X -ray images

Spampinato, C.;Palazzo, S.; Giordano, D.; Aldinucci, M.; Leonardi, R. Deep learning for automated skeletal bone age assessment in X -ray images. Med Image Anal. 2017, 36, 41–51

work page 2017

[8] [8]

Very deep convolutional networks for large-scale image recognition

Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition

work page