REVIEW 5 major objections 7 minor 22 references
A CNN toolbox for skin cancer classification
T0 review · 5 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A spreadsheet-configurable CNN toolbox for skin-cancer classification reports that image resolution and augmentation move performance, while the resizing filter does not.
desk verdict A useful preliminary toolbox report whose engineering core is solid, but the headline empirical claim about the resizing filter is supported by no reported numbers, so the 'quantify' framing overreaches. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the spreadsheet-driven training pipeline: each row of a CSV file specifies method, dataset, split, optional segmentation, augmentation preset, epochs, batch size, image size, resizing filter, color space, and class weights, and the software turns that row into a concrete network configuration and a training run. The augmentation stage is a chain of modular transforms (horizontal flip, rotation, brightness, saturation) built with the Decorator pattern, and an optional U-shaped convolutional segmentation network masks the lesion before classification. The quantitative conclusions are carried by direct comparisons of configurations that differ in only one controlled variable: resolution, augmentation preset, or resizing filter.
What would settle it
Repeat the three resizing-filter comparisons (VGG16 at 227x227, SC19 at 227x227, and VGG16 at 450x450) several times with different random seeds, holding all other settings fixed, and compute a confidence interval for ROC AUC within each filter. If the differences between filters exceed the differences between repeated runs of the same filter, the paper's central claim about filter irrelevance fails.
Extended reading notes
Core claim
The paper's central finding is that the configuration levers for skin-cancer CNNs are not equally important. In experiments with a transfer-learned VGG16 and an AlexNet-style network called SC19 on a public dermoscopic dataset of 12,319 images, image resolution and augmentation strength had clear effects: the VGG16 at 450x450 pixels reached 0.862 ROC AUC versus 0.819 at 277x277, at the cost of roughly six times longer training; and halving the SC19 augmentation factor from a 48x preset to an 8x preset improved its results while using one sixth of the compute. The same configurations showed no significant difference across the four tested resizing filters, which the paper takes as evidence that the resizing filter can be left at its nearest-neighbor default. The paper also reports that randomly initialized networks failed to converge or underperformed, leading it to conclude that large-scale natural-image pretraining is necessary at this dataset size.
Load-bearing premise
The load-bearing premise is that the reported performance differences reflect genuine effects of the configuration changes rather than run-to-run noise, because each configuration was trained once and the 'no significant difference' statement about resizing filters is not backed by repeated trials or a statistical test.
Editorial extensions
If this is right
- Practitioners can keep the nearest-neighbor resizing filter as the default and expect no meaningful loss in classification quality, freeing time and memory for other stages.
- For a transfer-learned VGG16 on dermoscopic images, increasing input resolution from 277x277 to 450x450 gained about 0.04 ROC AUC while increasing training time by a large factor, so resolution choices should be budgeted explicitly.
- Transfer learning from a large natural-image pretraining set appears necessary rather than optional at dataset sizes around twelve thousand images, since randomly initialized networks did not converge.
- Augmentation strength should be tuned per architecture: a smaller 8x augmentation preset beat a 48x preset for the SC19 network while using one sixth of the compute.
- The spreadsheet interface lets dermatology practitioners explore these trade-offs on a new dataset without writing code, which addresses the reproducibility gap among published skin-cancer CNN studies.
Reading between the lines
- If the resizing-filter result holds more broadly, standardizing the resize step across studies would remove one more source of variance from published skin-cancer CNN comparisons.
- The same configuration grid is a natural substrate for automated search: each spreadsheet row is a candidate configuration, so Bayesian optimization or AutoML could treat the toolbox as its action space.
- The filter result is demonstrated only for downscaling to 227 or 450 pixels on dermoscopic images; severe downsampling or preprocessing for segmentation could still be filter-sensitive.
- Because each configuration was run once, the numeric rankings should be read as directional; a practitioner choosing between two close configurations should repeat each run several times before committing.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This technical report describes a software toolbox for configuring CNN pipelines for skin cancer classification. The system exposes configuration via a CSV/spreadsheet interface, supports optional UNet-based lesion segmentation, on-the-fly data augmentation (flips, rotations, brightness, saturation, color space), and pluggable Keras classifiers. Preliminary experiments on an ISIC-derived dataset compare a pretrained VGG16 and a custom AlexNet variant (SC19) under different augmentation policies, image resolutions, and resizing filters. The authors report AUC values (e.g., VGG16 0.819 at 277x277 with 48x augmentation versus 0.862 at 450x450), compare favorably with dermatologists on the MClass-D test set, and claim that the resizing filter has no significant impact, recommending nearest-neighbor resampling.
Significance. The toolbox addresses a real need: reproducible, flexible configuration of deep networks for dermatology, and the authors correctly note that prior studies are hard to compare due to different datasets and hyperparameters. The reported architecture—spreadsheet-driven pipeline, CPU-side augmentation, pluggable classifiers, optional segmentation—is sensible and potentially useful to practitioners. The MClass-D comparison with human dermatologists is a strength, and the explicit reporting of sensitivity, specificity, and AUC for several configurations is useful. The quantitative conclusions are preliminary, however, and rest on single training runs without variance or statistical testing; the resize-filter claim is not backed by any reported numbers. This is a systems paper with indicative results, not yet a robust empirical comparison.
major comments (5)
- [Section 3, 'No impact of image resize filters'] The abstract states that the paper quantifies the impact of the rescaling filter, but this paragraph reports 'no significant difference' for four filters under three conditions without giving any accuracy, sensitivity, specificity, AUC, or training-time values, and without repeated runs or a statistical test. Please provide a table of the measured metrics for each filter and condition, or revise the abstract and conclusions to remove the unsupported quantitative claim.
- [Section 3, 'Experiments at 277x277' vs 'No impact of image resize filters'] The resolution is labeled 277x277 in the main experiments but 227x227 in the resize-filter paragraph, and Table 1 lists imgsize 227. This inconsistency makes it unclear whether the resize-filter experiment uses the same resolution as the earlier runs. Please correct the typo and explicitly state the resolutions used in each experiment.
- [Section 3, all reported comparisons] The augmentation, resolution, and architecture comparisons are each based on a single training run without repeated seeds or confidence intervals. For example, the SC19 comparison (0.803 AUC over 7 epochs with hflip rot24 versus 0.835 AUC over 11 epochs with hflip rot4) changes two variables at once. Please add repeated runs or otherwise bound run-to-run variability, and separate the effect of augmentation from the effect of epoch count.
- [Section 3, MClass-D evaluation] The operating point on the MClass-D test set is selected after seeing test results: the threshold is moved from 0.5 to 0.6 to improve specificity. Because this threshold selection is performed on the test set, the reported '0.862 specificity, 0.750 sensitivity' should be labeled as a test-set-tuned operating point, and the default-threshold result should be the primary comparison or justified as a prespecified decision rule.
- [Section 3, dataset and split description] The paper does not describe how the 12,319 images are split into training, validation, and test sets for the reported experiments. The 'split' column in Table 1 mentions 'pre' and 'n=100', but the actual splits used for the reported metrics are not specified. Please state the split procedure and sizes, as this is essential for interpreting the results.
minor comments (7)
- [Abstract and Section 1] The phrase 'meta leaning frameworks' appears twice; it should be 'meta-learning frameworks'.
- [Section 3, 'No impact of image resize filters'] The sentence 'Our results show no significant difference in the all the metrics' contains a grammatical error; rephrase to 'in all the metrics'.
- [Section 3] 'This latest results closely match' should be 'These latest results closely match'.
- [Section 2, Data Augmentation] The color space list is given as 'HVS, LAB, or YCbCr'; the first should be 'HSV' as used in Table 1.
- [Section 2, Training Input, Table 1] The example row 'ISIC-2016 n=100' is not clearly connected to the reported experiments; please clarify what this row represents.
- [Section 2, Figure 1] Figure 1 is not explicitly referenced in the pipeline description in Section 2; please add a reference to the figure.
- [Section 1 and Section 4] The text says the software 'will be released as open-source software once out of beta stage'; without a public repository, the experiments are not reproducible from the paper alone. Please provide a code release or state the availability clearly.
Circularity Check
No circularity: every central claim is an empirical measurement against external datasets and benchmarks, not a derivation from its own inputs.
full rationale
The paper makes no formal derivation from first principles; its claims are empirical measurements of CNN configurations on the public ISIC dataset, with comparisons to published external results (Esteva et al., Fujisawa et al., Brinker et al., ISIC 2017). The augmentation, resolution, and resize-filter conclusions are obtained by running the stated pipeline and reporting AUC, sensitivity, and specificity, not by fitting a parameter and then predicting a quantity that the fit already determines. The only threshold adjustment (from 0.5 to 0.6 on MClass-D) is a post-hoc operating-point choice on the same test set; this is a proper statistical concern about optimistic evaluation, but it is not circular because the reported result is not defined in terms of that threshold, and the central claims about augmentation and resolution do not depend on it. Self-citations [18,22] are used only to motivate the software architecture, not as evidence for the empirical conclusions, and no uniqueness theorem or prior-work ansatz is invoked to force the results. Accordingly, there is no self-definition, fitted-input prediction, or load-bearing self-citation to flag.
Assumptions & free parameters
free parameters (4)
- Operating threshold on MClass-D =
0.6
- Image resolution settings =
227, 277, 450 px
- Augmentation presets =
hflip rot24 (48x) and hflip rot4 (8x)
- Epoch counts =
2, 6, 7, 10, 11
assumptions (4)
- domain assumption ISIC and MClass-D ground truth labels are correct
- domain assumption ImageNet pretrained weights transfer positively to dermoscopic images
- domain assumption The UNET segmentation model is accurate enough to be a useful optional pipeline step
- domain assumption MClass-D test set and its dermatologist benchmark are a valid external comparison
invented entities (1)
-
SC19 (custom modified AlexNet)
Cite this review
Pith. "Pith review of A CNN toolbox for skin cancer classification." pith.science (2026). https://pith.science/paper/ZNK7LFKF
@misc{pith2026190808187,
author = {Pith},
title = {Pith review of: A CNN toolbox for skin cancer classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZNK7LFKF}},
note = {Machine review of arXiv:1908.08187}
}
read the original abstract
We describe a software toolbox for the configuration of deep neural networks in the domain of skin cancer classification. The implemented software architecture allows developers to quickly set up new convolutional neural network (CNN) architectures and hyper-parameter configurations. At the same time, the user interface, manageable as a simple spreadsheet, allows non-technical users to explore different configuration settings that need to be explored when switching to different data sets. In future versions, meta leaning frameworks can be added, or AutoML systems that continuously improve over time. Preliminary results, conducted with two CNNs in the context melanoma detection on dermoscopic images, quantify the impact of image augmentation, image resolution, and rescaling filter on the overall detection performance and training time.
Figures
Reference graph
Works this paper leans on
-
[1]
IEEE Systems Journal 8(3), 965–979 (Sep 2014)
Barata, C., Ruela, M., Francisco, M., Mendonca, T., Marques, J.S.: Two Systems for the Detection of Melanomas in Dermoscopy Images Using Texture and Color Features. IEEE Systems Journal 8(3), 965–979 (Sep 2014). https://doi.org/10.1109/JSYST.2013.2271540, http://ieeexplore.ieee. org/document/6570764/
-
[2]
European Journal of Cancer 113, 47–54 (May 2019)
Brinker, T.J., Hekler, A., Enk, A.H., et al.: Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma im- age classification task. European Journal of Cancer 113, 47–54 (May 2019). https://doi.org/10.1016/j.ejca.2019.04.001, https://linkinghub.elsevier.com/ retrieve/pii/S0959804919302217
-
[3]
European Journal of Cancer 111, 30–37 (Apr 2019)
Brinker, T.J., Hekler, A., Hauschild, A., Berking, C., Schilling, B., Enk, A.H., Haferkamp, S., Karoglan, A., von Kalle, C., Weichenthal, M., Sattler, E., Schadendorf, D., Gaiser, M.R., Klode, J., Utikal, J.S.: Comparing artifi- cial intelligence algorithms to 157 German dermatologists: the melanoma clas- sification benchmark. European Journal of Cancer 111...
-
[4]
Journal of Medical In- ternet Research 20(10), e11936 (Oct 2018)
Brinker, T.J., Hekler, A., Utikal, J.S., Grabe, N., Schadendorf, D., Klode, J., Berking, C., Steeb, T., Enk, A.H., von Kalle, C.: Skin Cancer Classification Us- ing Convolutional Neural Networks: Systematic Review. Journal of Medical In- ternet Research 20(10), e11936 (Oct 2018). https://doi.org/10.2196/11936, http: //www.jmir.org/2018/10/e11936/
-
[5]
Journal of Digital Imaging 31(4), 435–440 (Aug 2018)
Burdick, J., Marques, O., Weinthal, J., Furht, B.: Rethinking Skin Lesion Seg- mentation in a Convolutional Classifier. Journal of Digital Imaging 31(4), 435–440 (Aug 2018). https://doi.org/10.1007/s10278-017-0026-y, http://link.springer. com/10.1007/s10278-017-0026-y
-
[6]
IEEE Journal of Biomedical and Health Informatics 23(2), 474–478 (Mar 2019)
Celebi, M.E., Codella, N., Halpern, A.: Dermoscopy Image Analysis: Overview and Future Directions. IEEE Journal of Biomedical and Health Informatics 23(2), 474–478 (Mar 2019). https://doi.org/10.1109/JBHI.2019.2895803, https: //ieeexplore.ieee.org/document/8627921/
arXiv 2019
-
[7]
arXiv:1902.03368 [cs] (Feb 2019), http://arxiv.org/abs/1902.03368, arXiv: 1902.03368
Codella, N., Rotemberg, V., Tschandl, P., Celebi, M.E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., Marchetti, M., Kittler, H., Halpern, A.: Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC). arXiv:1902.03368 [cs] (Feb 2019), http://arxiv.org/abs/1902.03368, arXiv...
arXiv 2018
-
[8]
In: 2018 IEEE 15th International Sympo- sium on Biomedical Imaging (ISBI 2018)
Codella, N.C.F., Gutman, D., Celebi, M.E., Helba, B., Marchetti, M.A., Dusza, S.W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., Halpern, A.: Skin le- sion analysis toward melanoma detection: A challenge at the 2017 Interna- tional symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In: 2018 IEEE 15...
arXiv 2017
Show all 22 references
-
[9]
In: 2009 IEEE Conference on Com- puter Vision and Pattern Recognition
Deng, J., Dong, W., Socher, R., Li, L.J., Kai Li, Li Fei-Fei: ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Com- puter Vision and Pattern Recognition. pp. 248–255. IEEE, Miami, FL (Jun 2009). https://doi.org/10.1109/CVPR.2009.5206848, http://...
2009
-
[10]
Nature 542, 115 (Jan 2017), https://doi.org/10.1038/nature21056
Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115 (Jan 2017), https://doi.org/10.1038/nature21056
2017 doi
-
[11]
British Journal of Dermatology (Sep 2018)
Fujisawa, Y., Otomo, Y., Ogata, Y., Nakamura, Y., Fujita, R., Ishitsuka, Y., Watanabe, R., Okiyama, N., Ohara, K., Fujimoto, M.: Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images sur- passes board-certified dermatologists in skin tu...
2018 doi
-
[12]
Addison-Wesley (1994)
Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design patterns: elements of reusable object-oriented software. Addison-Wesley (1994)
1994
-
[13]
Annals of Oncology 29(8), 1836–1842 (Aug 2018)
Haenssle, H.A., Fink, C., Schneiderbauer, R., et al.: Man against machine: di- agnostic performance of a deep learning convolutional neural network for der- moscopic melanoma recognition in comparison to 58 dermatologists. Annals of Oncology 29(8), 1836–1842 (Aug 2018). https:...
2018 doi
-
[14]
In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet Classification with Deep Convolutional Neural Networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012)...
2012
-
[15]
International Journal of Biomedical Imaging 2013, 1–22 (2013)
Masood, A., Ali Al-Jumaily, A.: Computer Aided Diagnostic Support System for Skin Cancer: A Review of Techniques and Algorithms. International Journal of Biomedical Imaging 2013, 1–22 (2013). https://doi.org/10.1155/2013/323268, http://www.hindawi.com/journals/ijbi/2013/323268/
2013 doi
-
[16]
In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F
Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomed- ical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, vol. 9351, pp. 234–241. Springer ...
2015 doi
-
[17]
arXiv:1409.1556 [cs] (Sep 2014),http://arxiv.org/abs/ 1409.1556, arXiv: 1409.1556
Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large- Scale Image Recognition. arXiv:1409.1556 [cs] (Sep 2014),http://arxiv.org/abs/ 1409.1556, arXiv: 1409.1556
2014 arXiv
-
[18]
CoRR abs/1709.01476 (2017), http://arxiv.org/abs/1709.01476
Sonntag, D., Barz, M., Zacharias, J., Stauden, S., Rahmani, V., F´ othi,´A., L¨ orincz, A.: Fine-tuning deep CNN models on specific MS COCO categories. CoRR abs/1709.01476 (2017), http://arxiv.org/abs/1709.01476
2017 arXiv
-
[19]
In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Jun 2016)
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the Incep- tion Architecture for Computer Vision. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Jun 2016)
2016
-
[20]
Scientific Data 5, 180161 (Aug 2018), https://doi.org/10.1038/sdata.2018.161
Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5, 180161 (Aug 2018), https://doi.org/10.1038/sdata.2018.161
2018 doi
-
[21]
IEEE Journal of Biomedical and Health Informatics 23(2), 519–526 (Mar 2019)
Yuan, Y.: Automatic skin lesion segmentation with fully convolutional- deconvolutional networks. IEEE Journal of Biomedical and Health Informatics 23(2), 519–526 (Mar 2019). https://doi.org/10.1109/JBHI.2017.2787487, http: //arxiv.org/abs/1703.05165, arXiv: 1703.05165
2019
-
[22]
CoRR abs/1803.04818 (2018), http: //arxiv.org/abs/1803.04818
Zacharias, J., Barz, M., Sonntag, D.: A survey on deep learning toolkits and libraries for intelligent user interfaces. CoRR abs/1803.04818 (2018), http: //arxiv.org/abs/1803.04818
2018 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.