REVIEW 3 major objections 7 minor 31 references
Recognition of Ischaemia and Infection in Diabetic Foot Ulcers: Dataset and Techniques
T0 review · 3 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A new image dataset labels diabetic foot ulcers for ischaemia and infection, and an ensemble of convolutional neural networks is claimed to classify ischaemia at 90% accuracy and infection at 73%.
desk verdict The dataset is a real contribution, but the evaluation leaks near-duplicates across splits, so the accuracy claims need a re-run before they're believable. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the new DFU dataset: 1,459 non-standardised foot photographs collected from three cameras and labelled by two consultant physicians (disagreements settled by the senior) for ischaemia (210 present versus 1,249 absent) and infection (628 present versus 831 absent). The method around it has three parts: Natural Data-augmentation, which uses a Faster R-CNN detector to localise the ulcer, crops and magnifies that region, then applies mirroring, rotation, noise, contrast, and shear to multiply training examples; the Superpixel Color Descriptor, which over-segments each patch with SLIC superpixels and counts red and black regions using per-channel thresholds to produce a 10-dimensional feature vector; and the Ensemble CNN, which concatenates the bottleneck features of three pre-trained image classifiers and classifies them with an SVM. The dataset carries the claim's novelty, the ensemble carries its headline numbers.
What would settle it
Re-label the same 1,459 images using gold-standard tests — Doppler or ankle-brachial pressure index for ischaemia, and blood tests or wound cultures for infection — and compare the new labels to the clinicians' visual labels; if the two label sets diverge, or if the Ensemble CNN trained on the visual labels performs at chance on the objective labels, the paper's accuracy claims are labelling replication rather than disease recognition.
Extended reading notes
Core claim
The central claim is that ischaemia and bacterial infection in diabetic foot ulcers are detectable from colour photographs by deep learning, and that combining several networks is better than any single approach tested. On the authors' new dataset of 1,459 foot images, the Ensemble CNN — bottleneck features from InceptionV3, ResNet50, and InceptionResNetV2 fed into a support-vector machine — reaches 90.3% accuracy, 0.904 AUC, and an MCC of 0.807 for ischaemia classification, and 72.7% accuracy, 0.731 AUC, and an MCC of 0.454 for infection classification. The ensemble beats every individual CNN and every traditional machine-learning classifier with the handcrafted Superpixel Color Descriptor, which is the paper's proposed new feature. The paper itself notes that infection is harder than ischaemia because the photos were taken after debridement, which removes exudate and other visible infection signs, and because the ground truth was set by visual inspection alone without blood tests or vascular measurements.
Load-bearing premise
The labels for ischaemia and infection were assigned by two clinicians looking at photographs, with no blood-flow measurements, blood tests, or cultures, so if those visual labels are wrong or noisy the reported accuracies measure how well the model copies the clinicians' visual judgement rather than how well it recognises the true medical conditions.
Editorial extensions
If this is right
- The published 90%/73% ensemble baselines become reference points that any later ischaemia or infection recognition method on this dataset will need to beat.
- A 90% accuracy for ischaemia suggests automated image screening could plausibly help triage high-risk ulcers in remote or low-resource settings, though not replace clinical tests.
- The infection results imply that post-debridement colour photographs alone are a limited medium for infection recognition; progress will likely require pre-debridement imagery or objective blood-based labels.
- The localise-then-augment pipeline is not tied to DFU and can be carried over to other medical image tasks where the lesion occupies only a few percent of the frame.
- The strong gap between the two tasks argues for collecting standardised images (fixed camera settings and lighting, pre-debridement capture) as a direct route to better sensitivity.
Reading between the lines
- Implicit in the accuracy gap is that the visual signal for ischaemia (pale or gangrenous tissue, poor reperfusion) is far more legible in ordinary photos than the redness and purulence of infection, which debridement partly removes; a testable extension is to compare post-debridement photos with their pre-debridement counterparts for the same ulcers and see whether the infection gap narrows.
- The authors do not report agreement between the two labelling clinicians; measuring Cohen's kappa on a held-out subset would show whether the visual ground truth is stable enough to support the 90%/73% figures, and would give later datasets a labelling protocol to replicate.
- Because the same foot images already encode site, area, and depth, the ensemble pipeline could be extended to a single multi-label SINBAD classifier that predicts all five components at once rather than running paired binary tasks.
- The Superpixel Color Descriptor throws away colours that are not red or black; adding green/yellow thresholds for purulent exudate, or replacing fixed thresholds with learned colour distributions, is a cheap handcrafted baseline the paper leaves unexplored.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper introduces a dataset of 1,459 diabetic foot ulcer images with expert visual labels for ischaemia and infection, proposes a Superpixel Color Descriptor (SPCD) for handcrafted feature extraction, and proposes a natural data-augmentation method based on Faster R-CNN localization. The authors benchmark traditional machine learning classifiers and fine-tuned CNNs, including an Ensemble CNN that combines bottleneck features from Inception-V3, ResNet50, and InceptionResNetV2 with an SVM classifier. They report that the Ensemble CNN achieves 90% accuracy for ischaemia classification and 73% for infection classification, outperforming the handcrafted methods.
Significance. If the evaluation is sound, the dataset itself is a valuable contribution: it appears to be the first DFU dataset with labels for ischaemia and infection, and it provides a useful baseline comparison of handcrafted and deep features. The natural data augmentation idea is practical and clearly explained. The paper is also explicitly candid about the limitations of visual-only ground truth, particularly the absence of vascular or blood-test confirmation and the effect of debridement on infection indicators. These strengths are real. However, the headline accuracy figures are currently undermined by the patch-level evaluation protocol and by internal inconsistencies in the reported patch counts, so the quantitative claims cannot be taken at face value until the evaluation is fixed.
major comments (3)
- [Section 4, Tables 2 and 3; Section 3.1] The evaluation splits are performed at the patch level, but the paper does not state that all patches and all augmented variants derived from the same source foot image are kept within the same fold. Since the natural augmentation pipeline (Section 3.1) produces magnified, flipped, rotated, noised, sheared, and translated versions of each DFU patch, near-duplicate crops from the same foot can appear in both the training and test sets. This can substantially inflate the reported accuracies (90% ischaemia, 73% infection), because the model can exploit image-specific shortcuts rather than learning generalizable signs of the two conditions. Please re-run the experiments with image-level or patient-level splits, and report the patch-level numbers only as a secondary analysis or as a clearly labeled patch-wise benchmark.
- [Section 4 vs. Table 1 and Section 2] The patch-count arithmetic is internally inconsistent. Table 1 lists 1,666 DFU patches and 9,870 augmented patches for ischaemia, giving 11,536 total patches, but Section 4 reports 11,564 training + 1,652 validation + 3,304 test = 16,520 patches. For infection, Table 1 totals 1,666 + 5,892 = 7,558 patches, while Section 4 reports 7,136 + 1,019 + 2,038 = 10,193. Section 4 also refers to 'the 2611 original foot images,' whereas Section 2 describes a dataset of 1,459 images. These discrepancies make the evaluation non-reproducible and prevent a reader from determining the split unit or whether augmented patches leaked into the test set. Please correct the numbers and specify exactly how the split was performed (per image, per patch, or per patient).
- [Sections 2, 4.1 (challenge 4), and 5] The ground truth labels are based solely on visual inspection by two clinicians, with disagreement resolved by a senior physician, and no inter-observer agreement measure is reported. The paper itself cites Netten et al. showing low validity of remote DFU assessment by clinicians, and it notes that images were captured after debridement, which removes important infection indicators such as colored exudate. Because the dataset is the paper's central contribution and the accuracy numbers are interpreted as recognition of ischaemia and infection, the absence of a reliability measure is load-bearing. Please report inter-observer agreement (e.g., Cohen's kappa) and temper the abstract and conclusion so that the reported accuracies are clearly framed as predicting the expert visual labels, not a clinical reference standard.
minor comments (7)
- [Section 3.1, Figure 3] The label 'Ist MAG' should be '1st MAG'.
- [Section 3.2, Eq. (2)] There is a missing separator in 'dlab)and'; it should read 'dlab) and'.
- [Algorithm 1] The lines 'return SRed = SRed + 1' and 'return SBlack = SBlack + 1' use 'return' where an assignment is intended; the pseudocode logic should be corrected and clarified.
- [Section 4] The text says both '5-fold cross-validation' and a single 70/10/20 split; please clarify how the 5-fold procedure relates to the reported training, validation, and test partitions.
- [Section 1] The phrase '785 million patients with diabetes in the US' appears to be a typo; the cited study likely reports 6.7 million ambulatory care cases, not 785 million patients.
- [Tables 2 and 3] The header entries 'Ensemble (CNN)0.903' and similar cells are missing a space between the model name and the value; overall formatting of the tables should be cleaned up.
- [Section 2] The statement that the dataset 'will be made available upon acceptance of this article' would benefit from a concrete data availability statement with access conditions.
Circularity Check
No circularity found; the accuracy claims are empirical benchmark evaluations, not derivations from fitted constants.
full rationale
I inspected the derivation chain from the abstract through Section 5. The paper's central claims are (i) introduction of a clinician-labelled DFU dataset and (ii) empirical accuracy of handcrafted and CNN classifiers. The only equations in the paper, Eqs. (1)-(5), define SLIC superpixel clustering and the mean RGB value of superpixels; they do not encode the target labels or the reported accuracies. The SPCD thresholds (k=200, T1/T2 ranges) are handcrafted feature-construction parameters, and the paper reports test-set performance, so the experiments do not reduce to a fitted input renamed as a prediction. The natural data-augmentation pipeline uses the authors' earlier Faster R-CNN work [24] to localise the DFU before cropping, but this is preprocessing rather than evidence for the classification claim, and the cited Faster R-CNN is a standard object detector; nothing in the target result is imported by that self-citation. Ground-truth labels are acknowledged in Section 4.1 and the Conclusion to be visual-only and not supported by vascular or blood-test confirmation, and images were taken after debridement; this is a data-validity limitation, not circular reasoning. The patch-level split and augmentation arithmetic could threaten the validity of the reported 90% and 73% accuracies by potential near-duplicate leakage, but that is an evaluation-protocol concern outside the circularity taxonomy and cannot be exhibited as an equation-level reduction. No load-bearing step in the manuscript equates the output to its input by construction, so no circular step is identified.
Assumptions & free parameters
free parameters (3)
- SLIC superpixel count k =
200
- SPCD red and black thresholds T1 and T2 =
T1 in {0.40,0.45,0.50,0.55,0.60}, T2 in {0.15,0.20,0.25,0.30,0.35}
- Magnification factors for natural data augmentation =
3
assumptions (3)
- domain assumption Visual inspection by two clinicians is a valid ground truth for ischaemia and infection in the absence of vascular or blood-test confirmation.
- domain assumption Image-level labels transfer unchanged to every DFU patch and augmented patch.
- domain assumption ImageNet-pretrained CNN features transfer to DFU classification after fine-tuning.
Cite this review
Pith. "Pith review of Recognition of Ischaemia and Infection in Diabetic Foot Ulcers: Dataset and Techniques." pith.science (2026). https://pith.science/paper/JCLUL5C4
@misc{pith2026190805317,
author = {Pith},
title = {Pith review of: Recognition of Ischaemia and Infection in Diabetic Foot Ulcers: Dataset and Techniques},
year = {2026},
howpublished = {\url{https://pith.science/paper/JCLUL5C4}},
note = {Machine review of arXiv:1908.05317}
}
read the original abstract
Recognition and analysis of Diabetic Foot Ulcers (DFU) using computerized methods is an emerging research area with the evolution of image-based machine learning algorithms. Existing research using visual computerized methods mainly focuses on recognition, detection, and segmentation of the visual appearance of the DFU as well as tissue classification. According to DFU medical classification systems, the presence of infection (bacteria in the wound) and ischaemia (inadequate blood supply) has important clinical implications for DFU assessment, which are used to predict the risk of amputation. In this work, we propose a new dataset and computer vision techniques to identify the presence of infection and ischaemia in DFU. This is the first time a DFU dataset with ground truth labels of ischaemia and infection cases is introduced for research purposes. For the handcrafted machine learning approach, we propose a new feature descriptor, namely the Superpixel Color Descriptor. Then we use the Ensemble Convolutional Neural Network (CNN) model for more effective recognition of ischaemia and infection. We propose to use a natural data-augmentation method, which identifies the region of interest on foot images and focuses on finding the salient features existing in this area. Finally, we evaluate the performance of our proposed techniques on binary classification, i.e. ischaemia versus non-ischaemia and infection versus non-infection. Overall, our method performed better in the classification of ischaemia than infection. We found that our proposed Ensemble CNN deep learning algorithms performed better for both classification tasks as compared to handcrafted machine learning algorithms, with 90% accuracy in ischaemia classification and 73% in infection classification.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
-
[1]
J. D. Santilli, S. M. Santilli, Chronic critical limb ischemia: diagnosis, treatment and prognosis., American family physician 59 (7) (1999) 1899– 1908
work page 1999
- [2]
-
[3]
L. Prompers, M. Huijberts, J. Apelqvist, E. Jude, A. Piaggesi, K. Bakker, et al., High prevalence of ischaemia, infection and serious comorbidity in patients with diabetic foot disease in europe. baseline results from the eurodiale study, Diabetologia 50 (1) (2007) 18–25. 22
work page 2007
-
[4]
B. A. Lipsky, A. R. Berendt, P. B. Cornia, J. C. Pile, E. J. Peters, D. G. Armstrong, et al., 2012 infectious diseases society of america clin- ical practice guideline for the diagnosis and treatment of diabetic foot infections, Clinical infectious diseases 54 (12) (2012) e132–e173
work page 2012
-
[5]
L. A. Lavery, D. G. Armstrong, R. P. Wunderlich, J. Tredwell, A. J. Boulton, Diabetic foot syndrome: evaluating the prevalence and inci- dence of foot pathology in mexican americans and non-hispanic whites from a diabetes disease management cohort, Diabetes care 26 (5) (2003) 1435–1438
work page 2003
-
[6]
G. H. Skrepnek, J. L. Mills, L. A. Lavery, D. G. Armstrong, Health care service and outcomes among an estimated 6.7 million ambulatory care diabetic foot cases in the us, Diabetes Care 40 (7) (2017) 936–942
work page 2017
-
[7]
F. W. Wagner, The diabetic foot, Orthopedics 10 (1) (1987) 163–172
work page 1987
-
[8]
L. A. Lavery, D. G. Armstrong, L. B. Harkless, Classification of diabetic foot wounds, The Journal of Foot and Ankle Surgery 35 (6) (1996) 528– 531
work page 1996
Show all 31 references
-
[9]
P. Ince, Z. G. Abbas, J. K. Lutale, A. Basit, S. M. Ali, F. Chohan, S. Morbach, et al., Use of the sinbad classification system and score in comparing outcome of foot ulcer management on three continents, Diabetes care 31 (5) (2008) 964–967
2008
-
[10]
J. J. van Netten, D. Clark, P. A. Lazzarini, M. Janda, L. F. Reed, The validity and reliability of remote diabetic foot ulcer assessment using mobile phone images, Scientific Reports 7 (1) (2017) 9480
2017
-
[11]
Gulshan, L
V. Gulshan, L. Peng, M. Coram, M. C. Stumpe, D. Wu, A. Narayanaswamy, et al., Development and validation of a deep learn- ing algorithm for detection of diabetic retinopathy in retinal fundus photographs, Jama 316 (22) (2016) 2402–2410
2016
-
[12]
Esteva, B
A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, et al., Dermatologist-level classification of skin cancer with deep neural networks, Nature 542 (7639) (2017) 115–118. 23
2017
-
[13]
Krizhevsky, I
A. Krizhevsky, I. Sutskever, G. E. Hinton, Imagenet classification with deep convolutional neural networks, in: Advances in neural information processing systems, 2012, pp. 1097–1105
2012
-
[14]
Veredas, H
F. Veredas, H. Mesa, L. Morente, Binary tissue classification on wound images with neural networks and bayesian classifiers, IEEE transactions on medical imaging 29 (2) (2009) 410–427
2009
-
[15]
Wannous, Y
H. Wannous, Y. Lucas, S. Treuillet, Enhanced assessment of the wound- healing process by accurate multiview tissue classification, IEEE trans- actions on Medical Imaging 30 (2) (2010) 315–326
2010
-
[16]
L. Wang, P. Pedersen, E. Agu, D. Strong, B. Tulu, Area determina- tion of diabetic foot ulcer images using a cascaded two-stage svm based classification, IEEE Transactions on Biomedical Engineering (2016)
2016
-
[17]
M. H. Yap, M. Goyal, F. M. Osman, R. Mart´ ı, E. Denton, A. Juette, et al., Breast ultrasound lesions recognition: end-to-end deep learning approaches, Journal of Medical Imaging 6 (1) (2018) 011007
2018
-
[18]
Ahmad, M
E. Ahmad, M. Goyal, J. S. McPhee, H. Degens, M. H. Yap, Semantic segmentation of human thigh quadriceps muscle in magnetic resonance images, arXiv preprint arXiv:1801.00415 (2018)
2018 arXiv
-
[19]
Goyal, N
M. Goyal, N. D. Reeves, A. K. Davison, S. Rajbhandari, J. Spragg, M. H. Yap, Dfunet: convolutional neural networks for diabetic foot ulcer classification, IEEE Transactions on Emerging Topics in Computational Intelligence (2018) 1–12doi:10.1109/TETCI.2018.2866254
2018
-
[20]
Goyal, M
M. Goyal, M. H. Yap, N. D. Reeves, S. Rajbhandari, J. Spragg, Fully convolutional networks for diabetic foot ulcer segmentation, in: 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2017, pp. 618–623. doi:10.1109/SMC.2017.8122675
2017
-
[21]
Goyal, N
M. Goyal, N. D. Reeves, S. Rajbhandari, M. H. Yap, Robust methods for real-time diabetic foot ulcer detection and localization on mobile devices, IEEE Journal of Biomedical and Health Informatics 23 (4) (2019) 1730–
2019
-
[22]
C. Wang, X. Yan, M. Smith, K. Kochhar, M. Rubin, S. M. Warren, et al., A unified framework for automatic wound segmentation and analysis 24 with deep convolutional neural networks, in: Engineering in Medicine and Biology Society (EMBC), 2015 37th Annual International Confer- en...
2015
-
[23]
J. L. Mills Sr, M. S. Conte, D. G. Armstrong, F. B. Pomposelli, A. Schanzer, A. N. Sidawy, et al., The society for vascular surgery lower extremity threatened limb classification system: risk stratification based on wound, ischemia, and foot infection (wifi), Journal of vascular ...
2014
-
[24]
Goyal, M
M. Goyal, M. H. Yap, Region of interest detection in dermoscopic images for natural data-augmentation, arXiv preprint arXiv:1807.10711 (2018)
2018 arXiv
-
[25]
Huang, V
J. Huang, V. Rathod, C. Sun, M. Zhu, A. Korattikara, A. Fathi, et al., Speed/accuracy trade-offs for modern convolutional object detectors, arXiv preprint arXiv:1611.10012 (2016)
2016 arXiv
-
[26]
Achanta, A
R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, S. S¨ usstrunk, Slic superpixels, Tech. rep. (2010)
2010
-
[27]
Szegedy, V
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, Z. Wojna, Rethinking the inception architecture for computer vision, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2818–2826
2016
-
[28]
Szegedy, S
C. Szegedy, S. Ioffe, V. Vanhoucke, Inception-v4, inception-resnet and the impact of residual connections on learning, CoRR abs/1602.07261 (2016). arXiv:1602.07261. URL http://arxiv.org/abs/1602.07261
2016 arXiv
-
[29]
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778
2016
-
[30]
M. H. Yap, K. E. Chatwin, C.-C. Ng, C. A. Abbott, F. L. Bowling, S. Rajbhandari, et al., A new mobile application for standardizing di- abetic foot images, Journal of diabetes science and technology 12 (1) (2018) 169–173. 25
2018
-
[1741]
doi:10.1109/JBHI.2018.2868656
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.