Pith. sign in

REVIEW 5 major objections 7 minor 22 references

A CNN toolbox for skin cancer classification

T0 review · 5 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read A spreadsheet-configurable CNN toolbox for skin-cancer classification reports that image resolution and augmentation move performance, while the resizing filter does not.

desk verdict A useful preliminary toolbox report whose engineering core is solid, but the headline empirical claim about the resizing filter is supported by no reported numbers, so the 'quantify' framing overreaches. read the letter →

arxiv 1908.08187 v1 pith:ZNK7LFKF submitted 2019-08-21 eess.IV cs.CVcs.LG

classification eess.IVcs.CVcs.LG
keywords skincancerclassificationconvolutionalneuralnetworksmelanomadetectiondataaugmentationimageresolutiontransferlearningsoftwaretoolboxdermoscopicimages
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper presents a software toolbox that lets a user configure a skin-cancer image-classification pipeline from a spreadsheet, then reports what that configuration changes in practice. The authors' central empirical claim is that, for melanoma detection on dermoscopic images, image resolution and the data-augmentation policy are the levers that move detection performance and training time, while the resizing filter (nearest, bilinear, bicubic, or lanczos) makes no significant difference. The practical point is that a non-specialist can systematically explore CNN choices when a new dataset arrives, and can keep the cheapest default resizing filter without hurting results. In their experiments, transfer-learned networks clearly outperformed randomly initialized ones, and raising input resolution from 277x277 to 450x450 pixels raised ROC AUC from 0.819 to 0.862 at a large training-time cost.

What carries the argument

The load-bearing object is the spreadsheet-driven training pipeline: each row of a CSV file specifies method, dataset, split, optional segmentation, augmentation preset, epochs, batch size, image size, resizing filter, color space, and class weights, and the software turns that row into a concrete network configuration and a training run. The augmentation stage is a chain of modular transforms (horizontal flip, rotation, brightness, saturation) built with the Decorator pattern, and an optional U-shaped convolutional segmentation network masks the lesion before classification. The quantitative conclusions are carried by direct comparisons of configurations that differ in only one controlled variable: resolution, augmentation preset, or resizing filter.

What would settle it

Repeat the three resizing-filter comparisons (VGG16 at 227x227, SC19 at 227x227, and VGG16 at 450x450) several times with different random seeds, holding all other settings fixed, and compute a confidence interval for ROC AUC within each filter. If the differences between filters exceed the differences between repeated runs of the same filter, the paper's central claim about filter irrelevance fails.

Watch

Extended reading notes

Core claim

The paper's central finding is that the configuration levers for skin-cancer CNNs are not equally important. In experiments with a transfer-learned VGG16 and an AlexNet-style network called SC19 on a public dermoscopic dataset of 12,319 images, image resolution and augmentation strength had clear effects: the VGG16 at 450x450 pixels reached 0.862 ROC AUC versus 0.819 at 277x277, at the cost of roughly six times longer training; and halving the SC19 augmentation factor from a 48x preset to an 8x preset improved its results while using one sixth of the compute. The same configurations showed no significant difference across the four tested resizing filters, which the paper takes as evidence that the resizing filter can be left at its nearest-neighbor default. The paper also reports that randomly initialized networks failed to converge or underperformed, leading it to conclude that large-scale natural-image pretraining is necessary at this dataset size.

Load-bearing premise

The load-bearing premise is that the reported performance differences reflect genuine effects of the configuration changes rather than run-to-run noise, because each configuration was trained once and the 'no significant difference' statement about resizing filters is not backed by repeated trials or a statistical test.

Editorial extensions

If this is right

  • Practitioners can keep the nearest-neighbor resizing filter as the default and expect no meaningful loss in classification quality, freeing time and memory for other stages.
  • For a transfer-learned VGG16 on dermoscopic images, increasing input resolution from 277x277 to 450x450 gained about 0.04 ROC AUC while increasing training time by a large factor, so resolution choices should be budgeted explicitly.
  • Transfer learning from a large natural-image pretraining set appears necessary rather than optional at dataset sizes around twelve thousand images, since randomly initialized networks did not converge.
  • Augmentation strength should be tuned per architecture: a smaller 8x augmentation preset beat a 48x preset for the SC19 network while using one sixth of the compute.
  • The spreadsheet interface lets dermatology practitioners explore these trade-offs on a new dataset without writing code, which addresses the reproducibility gap among published skin-cancer CNN studies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the resizing-filter result holds more broadly, standardizing the resize step across studies would remove one more source of variance from published skin-cancer CNN comparisons.
  • The same configuration grid is a natural substrate for automated search: each spreadsheet row is a candidate configuration, so Bayesian optimization or AutoML could treat the toolbox as its action space.
  • The filter result is demonstrated only for downscaling to 227 or 450 pixels on dermoscopic images; severe downsampling or preprocessing for segmentation could still be filter-sensitive.
  • Because each configuration was run once, the numeric rankings should be read as directional; a practitioner choosing between two close configurations should repeat each run several times before committing.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. This technical report describes a software toolbox for configuring CNN pipelines for skin cancer classification. The system exposes configuration via a CSV/spreadsheet interface, supports optional UNet-based lesion segmentation, on-the-fly data augmentation (flips, rotations, brightness, saturation, color space), and pluggable Keras classifiers. Preliminary experiments on an ISIC-derived dataset compare a pretrained VGG16 and a custom AlexNet variant (SC19) under different augmentation policies, image resolutions, and resizing filters. The authors report AUC values (e.g., VGG16 0.819 at 277x277 with 48x augmentation versus 0.862 at 450x450), compare favorably with dermatologists on the MClass-D test set, and claim that the resizing filter has no significant impact, recommending nearest-neighbor resampling.

Significance. The toolbox addresses a real need: reproducible, flexible configuration of deep networks for dermatology, and the authors correctly note that prior studies are hard to compare due to different datasets and hyperparameters. The reported architecture—spreadsheet-driven pipeline, CPU-side augmentation, pluggable classifiers, optional segmentation—is sensible and potentially useful to practitioners. The MClass-D comparison with human dermatologists is a strength, and the explicit reporting of sensitivity, specificity, and AUC for several configurations is useful. The quantitative conclusions are preliminary, however, and rest on single training runs without variance or statistical testing; the resize-filter claim is not backed by any reported numbers. This is a systems paper with indicative results, not yet a robust empirical comparison.

major comments (5)
  1. [Section 3, 'No impact of image resize filters'] The abstract states that the paper quantifies the impact of the rescaling filter, but this paragraph reports 'no significant difference' for four filters under three conditions without giving any accuracy, sensitivity, specificity, AUC, or training-time values, and without repeated runs or a statistical test. Please provide a table of the measured metrics for each filter and condition, or revise the abstract and conclusions to remove the unsupported quantitative claim.
  2. [Section 3, 'Experiments at 277x277' vs 'No impact of image resize filters'] The resolution is labeled 277x277 in the main experiments but 227x227 in the resize-filter paragraph, and Table 1 lists imgsize 227. This inconsistency makes it unclear whether the resize-filter experiment uses the same resolution as the earlier runs. Please correct the typo and explicitly state the resolutions used in each experiment.
  3. [Section 3, all reported comparisons] The augmentation, resolution, and architecture comparisons are each based on a single training run without repeated seeds or confidence intervals. For example, the SC19 comparison (0.803 AUC over 7 epochs with hflip rot24 versus 0.835 AUC over 11 epochs with hflip rot4) changes two variables at once. Please add repeated runs or otherwise bound run-to-run variability, and separate the effect of augmentation from the effect of epoch count.
  4. [Section 3, MClass-D evaluation] The operating point on the MClass-D test set is selected after seeing test results: the threshold is moved from 0.5 to 0.6 to improve specificity. Because this threshold selection is performed on the test set, the reported '0.862 specificity, 0.750 sensitivity' should be labeled as a test-set-tuned operating point, and the default-threshold result should be the primary comparison or justified as a prespecified decision rule.
  5. [Section 3, dataset and split description] The paper does not describe how the 12,319 images are split into training, validation, and test sets for the reported experiments. The 'split' column in Table 1 mentions 'pre' and 'n=100', but the actual splits used for the reported metrics are not specified. Please state the split procedure and sizes, as this is essential for interpreting the results.
minor comments (7)
  1. [Abstract and Section 1] The phrase 'meta leaning frameworks' appears twice; it should be 'meta-learning frameworks'.
  2. [Section 3, 'No impact of image resize filters'] The sentence 'Our results show no significant difference in the all the metrics' contains a grammatical error; rephrase to 'in all the metrics'.
  3. [Section 3] 'This latest results closely match' should be 'These latest results closely match'.
  4. [Section 2, Data Augmentation] The color space list is given as 'HVS, LAB, or YCbCr'; the first should be 'HSV' as used in Table 1.
  5. [Section 2, Training Input, Table 1] The example row 'ISIC-2016 n=100' is not clearly connected to the reported experiments; please clarify what this row represents.
  6. [Section 2, Figure 1] Figure 1 is not explicitly referenced in the pipeline description in Section 2; please add a reference to the figure.
  7. [Section 1 and Section 4] The text says the software 'will be released as open-source software once out of beta stage'; without a public repository, the experiments are not reproducible from the paper alone. Please provide a code release or state the availability clearly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: every central claim is an empirical measurement against external datasets and benchmarks, not a derivation from its own inputs.

full rationale

The paper makes no formal derivation from first principles; its claims are empirical measurements of CNN configurations on the public ISIC dataset, with comparisons to published external results (Esteva et al., Fujisawa et al., Brinker et al., ISIC 2017). The augmentation, resolution, and resize-filter conclusions are obtained by running the stated pipeline and reporting AUC, sensitivity, and specificity, not by fitting a parameter and then predicting a quantity that the fit already determines. The only threshold adjustment (from 0.5 to 0.6 on MClass-D) is a post-hoc operating-point choice on the same test set; this is a proper statistical concern about optimistic evaluation, but it is not circular because the reported result is not defined in terms of that threshold, and the central claims about augmentation and resolution do not depend on it. Self-citations [18,22] are used only to motivate the software architecture, not as evidence for the empirical conclusions, and no uniqueness theorem or prior-work ansatz is invoked to force the results. Accordingly, there is no self-definition, fitted-input prediction, or load-bearing self-citation to flag.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The empirical conclusions rest on external benchmark labels, pretrained features, and single-run measurements; the toolbox itself is described at the architecture level and the code is not yet released.

free parameters (4)
  • Operating threshold on MClass-D = 0.6
    Chosen by hand during evaluation; shifting from 0.5 to 0.6 changed specificity from 0.762 to 0.862 and sensitivity from 0.850 to 0.750, directly affecting the reported comparison with dermatologists.
  • Image resolution settings = 227, 277, 450 px
    The resolution impact claim contrasts 277 px (2 epochs) with 450 px (6 epochs), so the effect of resolution is not isolated from training duration.
  • Augmentation presets = hflip rot24 (48x) and hflip rot4 (8x)
    The augmentation impact claim for SC19 compares these two presets; the presets are imported from Fujisawa et al. and are experimental conditions, not derived quantities.
  • Epoch counts = 2, 6, 7, 10, 11
    The number of epochs varies by configuration and is not matched when comparing resolution; the training time claims depend directly on these choices.
assumptions (4)
  • domain assumption ISIC and MClass-D ground truth labels are correct
    All training and test metrics assume the benign and malignant labels are reliable; the paper does not audit label quality or inter-observer agreement.
  • domain assumption ImageNet pretrained weights transfer positively to dermoscopic images
    The main VGG16 results rely on features learned from ImageNet; the paper finds that random initialization fails to converge, making this assumption load-bearing.
  • domain assumption The UNET segmentation model is accurate enough to be a useful optional pipeline step
    The segmentation module is offered as an option and cited as potentially improving results, but no classification experiments with segmentation are reported in this paper.
  • domain assumption MClass-D test set and its dermatologist benchmark are a valid external comparison
    The claim of outperforming dermatologists uses this 100-image set and the Brinker et al. benchmark; the paper assumes the set is representative and the human performance numbers are accurate.
invented entities (1)
  • SC19 (custom modified AlexNet)
    purpose: A non-pretrained CNN used as a baseline in augmentation and resizing experiments.
    The architecture is referenced as available on a Github distribution, but no link or full architectural specification is in the paper, so the entity is not independently checkable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A CNN toolbox for skin cancer classification." pith.science (2026). https://pith.science/paper/ZNK7LFKF

@misc{pith2026190808187,
  author       = {Pith},
  title        = {Pith review of: A CNN toolbox for skin cancer classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZNK7LFKF}},
  note         = {Machine review of arXiv:1908.08187}
}
read the original abstract

We describe a software toolbox for the configuration of deep neural networks in the domain of skin cancer classification. The implemented software architecture allows developers to quickly set up new convolutional neural network (CNN) architectures and hyper-parameter configurations. At the same time, the user interface, manageable as a simple spreadsheet, allows non-technical users to explore different configuration settings that need to be explored when switching to different data sets. In future versions, meta leaning frameworks can be added, or AutoML systems that continuously improve over time. Preliminary results, conducted with two CNNs in the context melanoma detection on dermoscopic images, quantify the impact of image augmentation, image resolution, and rescaling filter on the overall detection performance and training time.

Figures

Figures reproduced from arXiv: 1908.08187 by the authors.

Figure 1
Figure 1. Training/Testing pipeline of the proposed architecture image resolution (1k), the same performances can be achieved using less than 5000 images. This is very important to the general area under study with less data material. Also, we have reports from the pre-CNN era, when features were extracted manually and image pre-processing was required, about the importance of ex￾tended segmentation [5] and color filtering [1… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 10 canonical work pages

  1. [1]

    IEEE Systems Journal 8(3), 965–979 (Sep 2014)

    Barata, C., Ruela, M., Francisco, M., Mendonca, T., Marques, J.S.: Two Systems for the Detection of Melanomas in Dermoscopy Images Using Texture and Color Features. IEEE Systems Journal 8(3), 965–979 (Sep 2014). https://doi.org/10.1109/JSYST.2013.2271540, http://ieeexplore.ieee. org/document/6570764/

  2. [2]

    European Journal of Cancer 113, 47–54 (May 2019)

    Brinker, T.J., Hekler, A., Enk, A.H., et al.: Deep learning outperformed 136 of 157 dermatologists in a head-to-head dermoscopic melanoma im- age classification task. European Journal of Cancer 113, 47–54 (May 2019). https://doi.org/10.1016/j.ejca.2019.04.001, https://linkinghub.elsevier.com/ retrieve/pii/S0959804919302217

  3. [3]

    European Journal of Cancer 111, 30–37 (Apr 2019)

    Brinker, T.J., Hekler, A., Hauschild, A., Berking, C., Schilling, B., Enk, A.H., Haferkamp, S., Karoglan, A., von Kalle, C., Weichenthal, M., Sattler, E., Schadendorf, D., Gaiser, M.R., Klode, J., Utikal, J.S.: Comparing artifi- cial intelligence algorithms to 157 German dermatologists: the melanoma clas- sification benchmark. European Journal of Cancer 111...

  4. [4]

    Journal of Medical In- ternet Research 20(10), e11936 (Oct 2018)

    Brinker, T.J., Hekler, A., Utikal, J.S., Grabe, N., Schadendorf, D., Klode, J., Berking, C., Steeb, T., Enk, A.H., von Kalle, C.: Skin Cancer Classification Us- ing Convolutional Neural Networks: Systematic Review. Journal of Medical In- ternet Research 20(10), e11936 (Oct 2018). https://doi.org/10.2196/11936, http: //www.jmir.org/2018/10/e11936/

  5. [5]

    Journal of Digital Imaging 31(4), 435–440 (Aug 2018)

    Burdick, J., Marques, O., Weinthal, J., Furht, B.: Rethinking Skin Lesion Seg- mentation in a Convolutional Classifier. Journal of Digital Imaging 31(4), 435–440 (Aug 2018). https://doi.org/10.1007/s10278-017-0026-y, http://link.springer. com/10.1007/s10278-017-0026-y

  6. [6]

    IEEE Journal of Biomedical and Health Informatics 23(2), 474–478 (Mar 2019)

    Celebi, M.E., Codella, N., Halpern, A.: Dermoscopy Image Analysis: Overview and Future Directions. IEEE Journal of Biomedical and Health Informatics 23(2), 474–478 (Mar 2019). https://doi.org/10.1109/JBHI.2019.2895803, https: //ieeexplore.ieee.org/document/8627921/

  7. [7]

    arXiv:1902.03368 [cs] (Feb 2019), http://arxiv.org/abs/1902.03368, arXiv: 1902.03368

    Codella, N., Rotemberg, V., Tschandl, P., Celebi, M.E., Dusza, S., Gutman, D., Helba, B., Kalloo, A., Liopyris, K., Marchetti, M., Kittler, H., Halpern, A.: Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC). arXiv:1902.03368 [cs] (Feb 2019), http://arxiv.org/abs/1902.03368, arXiv...

  8. [8]

    In: 2018 IEEE 15th International Sympo- sium on Biomedical Imaging (ISBI 2018)

    Codella, N.C.F., Gutman, D., Celebi, M.E., Helba, B., Marchetti, M.A., Dusza, S.W., Kalloo, A., Liopyris, K., Mishra, N., Kittler, H., Halpern, A.: Skin le- sion analysis toward melanoma detection: A challenge at the 2017 Interna- tional symposium on biomedical imaging (ISBI), hosted by the international skin imaging collaboration (ISIC). In: 2018 IEEE 15...

Show all 22 references
  1. [9]

    In: 2009 IEEE Conference on Com- puter Vision and Pattern Recognition

    Deng, J., Dong, W., Socher, R., Li, L.J., Kai Li, Li Fei-Fei: ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Com- puter Vision and Pattern Recognition. pp. 248–255. IEEE, Miami, FL (Jun 2009). https://doi.org/10.1109/CVPR.2009.5206848, http://...

  2. [10]

    Nature 542, 115 (Jan 2017), https://doi.org/10.1038/nature21056

    Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115 (Jan 2017), https://doi.org/10.1038/nature21056

  3. [11]

    British Journal of Dermatology (Sep 2018)

    Fujisawa, Y., Otomo, Y., Ogata, Y., Nakamura, Y., Fujita, R., Ishitsuka, Y., Watanabe, R., Okiyama, N., Ohara, K., Fujimoto, M.: Deep-learning-based, computer-aided classifier developed with a small dataset of clinical images sur- passes board-certified dermatologists in skin tu...

  4. [12]

    Addison-Wesley (1994)

    Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design patterns: elements of reusable object-oriented software. Addison-Wesley (1994)

  5. [13]

    Annals of Oncology 29(8), 1836–1842 (Aug 2018)

    Haenssle, H.A., Fink, C., Schneiderbauer, R., et al.: Man against machine: di- agnostic performance of a deep learning convolutional neural network for der- moscopic melanoma recognition in comparison to 58 dermatologists. Annals of Oncology 29(8), 1836–1842 (Aug 2018). https:...

  6. [14]

    In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q

    Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet Classification with Deep Convolutional Neural Networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012)...

  7. [15]

    International Journal of Biomedical Imaging 2013, 1–22 (2013)

    Masood, A., Ali Al-Jumaily, A.: Computer Aided Diagnostic Support System for Skin Cancer: A Review of Techniques and Algorithms. International Journal of Biomedical Imaging 2013, 1–22 (2013). https://doi.org/10.1155/2013/323268, http://www.hindawi.com/journals/ijbi/2013/323268/

  8. [16]

    In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F

    Ronneberger, O., Fischer, P., Brox, T.: U-Net: Convolutional Networks for Biomed- ical Image Segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, vol. 9351, pp. 234–241. Springer ...

  9. [17]

    arXiv:1409.1556 [cs] (Sep 2014),http://arxiv.org/abs/ 1409.1556, arXiv: 1409.1556

    Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large- Scale Image Recognition. arXiv:1409.1556 [cs] (Sep 2014),http://arxiv.org/abs/ 1409.1556, arXiv: 1409.1556

  10. [18]

    CoRR abs/1709.01476 (2017), http://arxiv.org/abs/1709.01476

    Sonntag, D., Barz, M., Zacharias, J., Stauden, S., Rahmani, V., F´ othi,´A., L¨ orincz, A.: Fine-tuning deep CNN models on specific MS COCO categories. CoRR abs/1709.01476 (2017), http://arxiv.org/abs/1709.01476

  11. [19]

    In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Jun 2016)

    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the Incep- tion Architecture for Computer Vision. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (Jun 2016)

  12. [20]

    Scientific Data 5, 180161 (Aug 2018), https://doi.org/10.1038/sdata.2018.161

    Tschandl, P., Rosendahl, C., Kittler, H.: The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5, 180161 (Aug 2018), https://doi.org/10.1038/sdata.2018.161

  13. [21]

    IEEE Journal of Biomedical and Health Informatics 23(2), 519–526 (Mar 2019)

    Yuan, Y.: Automatic skin lesion segmentation with fully convolutional- deconvolutional networks. IEEE Journal of Biomedical and Health Informatics 23(2), 519–526 (Mar 2019). https://doi.org/10.1109/JBHI.2017.2787487, http: //arxiv.org/abs/1703.05165, arXiv: 1703.05165

  14. [22]

    CoRR abs/1803.04818 (2018), http: //arxiv.org/abs/1803.04818

    Zacharias, J., Barz, M., Sonntag, D.: A survey on deep learning toolkits and libraries for intelligent user interfaces. CoRR abs/1803.04818 (2018), http: //arxiv.org/abs/1803.04818

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.