Pith. sign in

REVIEW 3 major objections 4 minor 22 references

KneeXNeT: An Ensemble-Based Approach for Knee Radiographic Evaluation

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An ensemble of ten fine-tuned CNNs grades knee osteoarthritis with 72 percent accuracy.

desk verdict Solid incremental benchmark whose 0.72 accuracy depends on a patient-exclusive split the paper never confirms. read the letter →

arxiv 2412.07526 v1 pith:7K26E5XX submitted 2024-12-10 eess.IV cs.CV

Nicharee Srikijkasemwat , Soumya Snigdha Kundu , Fuping Wu , Bartlomiej W. Papiez This is my paper · ORCID
classification eess.IVcs.CV
keywords kneeosteoarthritisKellgren-LawrencegradingensemblelearningdeepX-rayclassificationclassimbalanceweightedsamplingSmooth-GradCAM++
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that an automated system can grade knee osteoarthritis severity from X-rays more reliably than any single deep network, using a learned ensemble of ten standard convolutional networks. On a public knee X-ray dataset labeled with the five-grade Kellgren-Lawrence scale, the best individual model reaches 0.69 accuracy, and a weighted-sampling strategy pushes the best single model (DenseNet-161) to 0.70. The proposed ensemble, KneeXNet, feeds the ten models' output scores into a shallow two-layer neural network and reaches 0.72 test accuracy, beating majority voting and a previously reported three-DenseNet ensemble at 0.71. If this holds on new patients, the result matters because automated grading could reduce radiologist workload in settings where expert assessment is scarce.

What carries the argument

The load-bearing mechanism is the learned fuser. Each of the ten component networks outputs five logits, these are concatenated into one 50-dimensional input vector per image, and a shallow two-layer fully connected network is trained with cross-entropy loss to combine them; majority voting over the same models is the weaker alternative it is compared against. Weighted sampling, which samples classes inversely proportional to their frequency, is the supporting device for imbalanced training data, and Smooth-GradCAM++ is the supporting device for explainability, producing heatmaps over the joint-space region.

What would settle it

Re-run KneeXNet's same training protocol but split the data by participant, ensuring that no person's images appear in more than one of the training, validation, and test sets. If the test accuracy falls below about 0.70 or the ensemble no longer beats the best single model, the reported 0.72 does not measure generalization to new patients.

Watch

Extended reading notes

Core claim

The authors report that KneeXNet, built by concatenating the five-class logits of ten fine-tuned CNNs (ResNet-18/34/50, VGG-16/19, MobileNet, DenseNet-121/161, EfficientNet, GoogLeNet) into a 50-dimensional vector and training a two-layer fully connected network on that vector, achieves a test accuracy of 0.72±0.01 on five-class Kellgren-Lawrence grading. Per-class F1 scores are 0.82, 0.23, 0.71, 0.84, and 0.88. This is two points above the best single model with weighted sampling and one point above the cited three-DenseNet-121 ensemble. Weighted sampling is shown to improve the F1 score of the minority Doubtful grade by up to 0.14 while leaving overall accuracy roughly unchanged, and Smooth-GradCAM++ visualizations indicate that the best model's attention concentrates on the joint-space region, consistent with radiographic evidence of osteoarthritis.

Load-bearing premise

The load-bearing premise is that the data split separated people, not just images: if the same person's left and right knee X-rays appear in both training and test sets, the reported 0.72 accuracy reflects image-level similarity rather than the model's ability to grade a new person's knee.

Editorial extensions

If this is right

  • Knee OA grading on this dataset can be automated to match or beat the best single-network results without a radiologist in the loop.
  • A learned fusion of ten diverse CNN architectures is more accurate than majority voting and than a single architecture trained with the same data.
  • Weighted sampling improves the per-class F1 score for the hardest Kellgren-Lawrence grade (Doubtful) even when it does not raise overall accuracy, so class-balance corrections should be judged per class.
  • Future knee OA grading systems on the same public dataset need to exceed the 0.72 KneeXNet result to claim an improvement over this ensemble approach.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: A patient-level split, where all images from each participant stay in one partition, would establish whether the 0.72 accuracy transfers to people the model has not seen; this is the natural next validation for the approach.
  • Editorial inference: The paper's own observation that Kellgren-Lawrence grades have fuzzy boundaries suggests ordinal regression or a three-class collapse would likely be clinically more useful than five-class classification, a direction the authors mention but do not test.
  • Editorial inference: The same logit-concatenation fuser is task-agnostic, so the ensemble design could be applied directly to other ordinal medical imaging tasks with class imbalance, such as diabetic retinopathy or hip osteoarthritis grading.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper evaluates ten ImageNet-pretrained CNN architectures (ResNet, VGG, MobileNet, DenseNet, EfficientNet, GoogLeNet) for five-class Kellgren-Lawrence knee osteoarthritis grading on the OAI dataset, first with cross-entropy loss and then with inverse-frequency weighted sampling. It then combines the outputs of the ten models by majority voting and by a two-layer fully connected network, reporting a best test accuracy of 0.72 ± 0.01 for the FCN-based ensemble, called KneeXNet. The paper also includes Smooth-GradCAM++ visualizations of the best single model, DenseNet-161.

Significance. If the reported results hold under a proper evaluation protocol, the contribution is a moderate, incremental improvement in knee OA classification accuracy over a known three-DenseNet ensemble (0.71) and over the best individual model (0.69–0.70), together with a systematic comparison of ten architectures. The manuscript does not provide code or split indices, and the evaluation protocol has a serious potential leakage issue that affects the credibility of the headline number. The explainability visualizations are a useful but minor addition.

major comments (3)
  1. [Section 2.1] The data split is described at the image level (8,260 images from 4,796 participants split 7:1:2), but the paper never states that all images from a single participant were assigned to the same split. Because the same participant contributes on average 1.72 images (left and right knees, and possibly follow-up visits), a random image-level split will almost certainly place images of the same participant in both the training and test sets. KL grade is strongly correlated within a participant, so the reported 0.72 test accuracy could reflect memorization of participant-specific features rather than generalization to new individuals. This is load-bearing for the central claim, and the paper must either confirm that an exclusive participant-level split was used or re-run the experiments with such a split and report the resulting accuracies.
  2. [Section 3.3 vs. Section 4] The Discussion states that the best ensemble 'fused the outputs of models trained with and without the weighted sampling strategy,' but no such combined ensemble is described in the experimental sections. Section 3.3 reports an ensemble of the ten Experiment 1 models (accuracy 0.72 ± 0.01), while Section 3.4 reports an ensemble of the ten Experiment 2 models (accuracy 0.70 ± 0.01). The paper must clarify which configuration constitutes KneeXNet and, if the Discussion refers to a fourth ensemble not described in the methods, provide the full experimental details.
  3. [Section 3.1 / Fig. 1] No majority-class or random-chance baseline is reported. Given the class imbalance shown in Fig. 1, a trivial classifier that always predicts the majority class may already achieve a substantial accuracy, and the 0.72 figure has little meaning without this reference. Reporting per-class accuracy alongside the F1 scores would also help interpret the 0.72 overall accuracy, which is dominated by the larger classes.
minor comments (4)
  1. [Section 2.3] The two-layer fully connected network is said to be trained 'with the same training parameters as the baseline models,' but the baseline description includes batch size 28 and a learning-rate schedule; it is unclear whether the FCN uses the same batch size and schedule or just the same optimizer and number of epochs.
  2. [Sections 3.3 and 3.4] Standard deviations are reported for the FCN-based ensembles but not for the majority-voting ensembles. Since the voting ensembles were run once, the reader cannot assess the variability of the reported 0.70 and 0.64 accuracies.
  3. [Section 4] There is a typo: 'hypotetised' should be 'hypothesized'.
  4. [References] Reference [1] cites a Mendeley dataset from 2018, while the text says the study used the Osteoarthritis Initiative (OAI) dataset. The relationship between the OAI and the cited Mendeley entry should be clarified, as the reader currently cannot identify the exact data source.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: KneeXNet accuracy is an empirical out-of-sample benchmark, not a quantity reconstructed from its own inputs.

full rationale

This paper is a standard empirical benchmark, not a derivation in which a fitted parameter or model output is reused as the predicted target. Section 2.1 describes a fixed 7:1:2 split of the OAI dataset into training, validation, and test sets. Section 2.2 describes training ten ImageNet-pretrained CNNs independently, with validation-based checkpoint selection, and Section 2.3 describes ensembling via majority voting and a two-layer FCN trained on concatenated logits. The reported test accuracy of 0.72 is an out-of-sample measurement on the described unseen test set, not a number forced by construction. The main methodological weakness, noted by the skeptical reading, is that the paper does not state whether all images from a single participant were kept in the same split, given 8,260 images from 4,796 participants. That is a potential data-leakage and generalization-validity concern, but it is not circularity: the reported accuracy is not defined in terms of the training targets or derived from the same quantity being predicted. No load-bearing self-citations appear; the cited prior work, including the DenseNet-121 ensemble of reference [12], is external. Model and ensemble selection after seeing test results raises multiple-comparison concerns, but again that is a statistical validity issue rather than a circular reduction. Accordingly, no circular step is identifiable, and the appropriate score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper makes an empirical benchmark claim, so the main free parameters are training choices and fitted model weights. The largest structural assumptions are that ImageNet features transfer, that KL labels are reliable, and that a random image-level split provides independent test data. No new entities are introduced, so invented_entities is empty.

free parameters (3)
  • Shallow neural network ensemble weights = not reported (trained, 30 epochs, cross-entropy)
    The final KneeXNet predictions are a learned fusion of 10 model logits; architecture details beyond 'two-layer fully connected network' and training schedule are not given.
  • Random 7:1:2 split assignment = not reported (no seed)
    The exact test set is not reproducible and the absence of patient-level blocking can inflate accuracy.
  • Training and augmentation hyperparameters = lr 1e-4, decayed 10x every 5 epochs, batch 28, flip, brightness, saturation, rotation, translation ranges
    Chosen by hand in Section 2.2; they affect all reported accuracies and are not optimized or ablated.
assumptions (4)
  • domain assumption KL grades in the OAI dataset are reliable ground truth for OA severity.
    The paper uses the public dataset labels as ground truth without discussing grading variability or adjudication.
  • domain assumption ImageNet-pretrained weights transfer useful visual features to knee X-rays.
    All ten models are initialized from ImageNet and fine-tuned; if transfer is poor, the results change.
  • domain assumption Random image-level splitting yields test samples independent of training samples.
    Section 2.1 splits 8,260 images from 4,796 participants randomly; no patient-level grouping is described.
  • domain assumption Standard categorical cross-entropy is an appropriate loss for ordinal KL grades.
    Cross-entropy treats the five grades as unrelated classes; the authors acknowledge this limitation in the Discussion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KneeXNeT: An Ensemble-Based Approach for Knee Radiographic Evaluation." pith.science (2026). https://pith.science/paper/7K26E5XX

@misc{pith2026241207526,
  author       = {Pith},
  title        = {Pith review of: KneeXNeT: An Ensemble-Based Approach for Knee Radiographic Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7K26E5XX}},
  note         = {Machine review of arXiv:2412.07526}
}
read the original abstract

Knee osteoarthritis (OA) is the most common joint disorder and a leading cause of disability. Diagnosing OA severity typically requires expert assessment of X-ray images and is commonly based on the Kellgren-Lawrence grading system, a time-intensive process. This study aimed to develop an automated deep learning model to classify knee OA severity, reducing the need for expert evaluation. First, we evaluated ten state-of-the-art deep learning models, achieving a top accuracy of 0.69 with individual models. To address class imbalance, we employed weighted sampling, improving accuracy to 0.70. We further applied Smooth-GradCAM++ to visualize decision-influencing regions, enhancing the explainability of the best-performing model. Finally, we developed ensemble models using majority voting and a shallow neural network. Our ensemble model, KneeXNet, achieved the highest accuracy of 0.72, demonstrating its potential as an automated tool for knee OA assessment.

Figures

Figures reproduced from arXiv: 2412.07526 by the authors.

Figure 1
Figure 1. Distribution of samples across each class in the training, validation, and test sets. 3 Results 3.1 Experiment 1: Comparison of the baseline models In this experiment, we trained 10 models using cross-entropy loss to evaluate the performance of each model. As shown in [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Test accuracy for each model in Experiment 1 and Experiment 2 [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. F1 scores of each model in Experiment 1 (the baseline models) [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: F1 scores of each model in Experiment 2 (the weighted sampling strategy) [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Visualizations of the regions influencing model predictions using Smooth Grad￾CAM++. These heatmaps highlight the areas of the knee X-ray that the model focuses on when making predictions [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 12 canonical work pages

  1. [1]

    Mendeley Data1 (2018)

    Chen, P.: Knee osteoarthritis severity grading dataset. Mendeley Data1 (2018)

    work page 2018

  2. [2]

    Comput- erized Medical Imaging and Graphics 75, 84–92 (2019)

    Chen, P., Gao, L., Shi, X., Allen, K., Yang, L.: Fully automatic knee osteoarthritis severity grading using deep neural networks with a novel ordinal loss. Comput- erized Medical Imaging and Graphics 75, 84–92 (2019). https://doi.org/10. 1016/j.compmedimag.2019.06.002, https://www.sciencedirect.com/science/ article/pii/S0895611118304956

    work page 2019

  3. [3]

    EClinicalMedicine (2020)

    Cui, A., Li, H., Wang, D., Zhong, J., Chen, Y., Lu, H.: Global, regional preva- lence, incidence and risk factors of knee osteoarthritis in population-based studies. EClinicalMedicine (2020). https://doi.org/10.1016/j.eclinm.2020.100587

    arXiv 2020

  4. [4]

    Annals of the rheumatic diseases70(11), 1884–1886 (2011)

    Felson, D.T., Niu, J., Guermazi, A., Sack, B., Aliabadi, P.: Defining radiographic incidence and progression of knee osteoarthritis: suggested modifications of the kellgren and lawrence scale. Annals of the rheumatic diseases70(11), 1884–1886 (2011)

    work page 2011

  5. [5]

    Hart, D., Spector, T.: Kellgren & lawrence grade 1 osteophytes in the knee—doubtful or definite? Osteoarthritis and cartilage11(2), 149–150 (2003)

    work page 2003

  6. [6]

    In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 770–778 (2016).https://doi.org/10.1109/CVPR.2016.90

    doi:10.1109/cvpr.2016.90 2016

  7. [7]

    CoRR abs/1704.04861 (2017), http://arxiv.org/ abs/1704.04861

    Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., An- dreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR abs/1704.04861 (2017), http://arxiv.org/ abs/1704.04861

    arXiv 2017

  8. [8]

    In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected con- volutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 2261–2269 (2017). https://doi.org/10.1109/CVPR. 2017.243

    doi:10.1109/cvpr 2017

Show all 22 references
  1. [9]

    Annals of the rheumatic diseases16(4), 494–502 (1957).https://doi.org/10.1136/ard

    Kellgren, J.H., Lawrence, J.S.: Radiological assessment of osteo-arthrosis. Annals of the rheumatic diseases16(4), 494–502 (1957).https://doi.org/10.1136/ard. 16.4.494

    1957 doi

  2. [10]

    arXiv preprint arXiv:1412.6980 (2014)

    Kingma, D.P.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

    2014 arXiv

  3. [11]

    Arthritis & rheumatology74(7) (2022)

    Long, H., Liu, Q., Yin, H., Wang, K., Diao, N., Zhang, Y., Lin, J., Guo, A.: Prevalence trends of site-specific osteoarthritis from 1990 to 2019: Findings from the global burden of disease study 2019. Arthritis & rheumatology74(7) (2022). https://doi.org/10.1002/art.42089

    2022 doi

  4. [12]

    In: Recent Trends in Analysis of Images, Social Networks and Texts

    Mikhaylichenko, A., Demyanenko, Y.: Automatic grading of knee osteoarthritis from plain radiographs using densely connected convolutional networks. In: Recent Trends in Analysis of Images, Social Networks and Texts. pp. 149–161. Springer International Publishing (2021)

    2021

  5. [13]

    Journal of pain research11 (2018)

    Mora, J.C., Przkora, R., Cruz-Almeida, Y.: Knee osteoarthritis: pathophysiology and current treatment modalities. Journal of pain research11 (2018). https:// doi.org/10.2147/JPR.S154002 10 N. Srikijkasemwat et al

    2018 doi

  6. [14]

    CoRR abs/1908.01224 (2019), http://arxiv.org/abs/1908

    Omeiza, D., Speakman, S., Cintas, C., Weldermariam, K.: Smooth grad-cam++: An enhanced inference level visualization technique for deep convolutional neural network models. CoRR abs/1908.01224 (2019), http://arxiv.org/abs/1908. 01224

    2019 arXiv

  7. [15]

    Scientific Reports13, 22887 (2023)

    Pi, S., Lee, B., Lee, M., Lee, H.: Ensemble deep-learning networks for automated osteoarthritis grading in knee x-ray images. Scientific Reports13, 22887 (2023). https://doi.org/10.1038/s41598-023-50210-4

    2023 doi

  8. [16]

    Nature Reviews Rheumatology 12 (2016)

    Roos, E., Arden, N.: Strategies for the prevention of knee osteoarthritis. Nature Reviews Rheumatology 12 (2016). https://doi.org/10.1038/nrrheum.2015.135

    2016 doi

  9. [17]

    International journal of computer vision115, 211–252 (2015)

    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al.: Imagenet large scale visual recog- nition challenge. International journal of computer vision115, 211–252 (2015)

    2015

  10. [18]

    Journal of magnetic resonance imaging 39(6), 1346–1356 (2014)

    Shapiro, L.M., McWalter, E.J., Son, M.S., Levenston, M., Hargreaves, B.A., Gold, G.E.: Mechanisms of osteoarthritis in the knee: Mr imaging appearance. Journal of magnetic resonance imaging 39(6), 1346–1356 (2014). https://doi.org/10. 1002/jmri.24562

    2014

  11. [19]

    In: International Conference on Learning Representations (2015)

    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale im- age recognition. In: International Conference on Learning Representations (2015)

    2015

  12. [20]

    Amer- ican family physician64(2), 279–287 (2001)

    Swagerty Jr, D.L., Hellinger, D.: Radiographic assessment of osteoarthritis. Amer- ican family physician64(2), 279–287 (2001)

    2001

  13. [21]

    In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 1–9 (2015). https://doi.org/10.1109/CVPR.2015.7298594

    2015

  14. [22]

    CoRR abs/1905.11946 (2019), http://arxiv.org/abs/1905.11946

    Tan, M., Le, Q.V.: Efficientnet: Rethinking model scaling for convolutional neural networks. CoRR abs/1905.11946 (2019), http://arxiv.org/abs/1905.11946

    2019 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.