REVIEW 4 major objections 5 minor 12 references
Fine-Tuning Models Comparisons on Garbage Classification for Recyclability
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Fine-tuned GoogLeNet with an SVM classifies six trash types at 97.86% accuracy on the paper's six-class garbage dataset.
desk verdict Useful as a cautionary example: the headline accuracy is a max over ten test-set evaluations, so it likely overstates what GoogleNet+SVM would do on a fresh split. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is transfer learning by fine-tuning: take a convolutional network already trained on a large general image collection, adapt its weights to the garbage dataset for 200 epochs, then use the network's penultimate-layer representation as a fixed feature vector for two different classifiers. The contrast between softmax and SVM is the experiment's active ingredient, and the claim is that the margin-maximizing SVM generalizes better from the small fine-tuned feature set than the softmax layer trained jointly with the network.
What would settle it
Run the same ten fine-tuning and classifier combinations but choose the single best combination on a separate validation split or through cross-validation, then evaluate only that chosen model on held-out test data; if its accuracy falls well below 97.86%, or a different combination wins, the headline number is an artifact of selecting the maximum of ten test-set results.
Extended reading notes
Core claim
On a six-class trash image dataset built from 2527 photos with a white background, the authors claim that a fine-tuned GoogLeNet feature extractor paired with a support vector machine classified 97.86% of test images correctly. The evidence is a ten-way comparison: five pre-trained networks (AlexNet, VGG-16, GoogLeNet, ResNet, and SqueezeNet) fine-tuned for 200 epochs on half of the data, each tested with a softmax output and with an SVM output on the other half. In all five rows the SVM version beats the softmax version, and the paper presents this as exceeding an earlier fine-tuned baseline of 95% on the same benchmark, and as evidence that SVM is the more successful classifier for this task.
Load-bearing premise
The reported 97.86% is the best of ten numbers all measured on the same test set after the fact, with no separate validation set used to pick the winner, so it may overstate how the chosen system would perform on new photos.
Editorial extensions
If this is right
- Replacing the softmax layer with an SVM improves accuracy for every one of the five fine-tuned backbones, so the improvement is not specific to one architecture.
- The best softmax model reaches 90%, while every SVM variant exceeds it, indicating that on this task the choice of classifier matters at least as much as backbone depth.
- No data augmentation was used, so the reported accuracies come from a simpler training pipeline than the earlier 95% baseline, which relied on augmentation.
- The 97.86% result is presented as an improvement over both scratch-trained and fine-tuned garbage classifiers from the earlier study used for comparison.
Reading between the lines
- Editorial inference: because the winner was chosen by scanning ten test-set accuracies with no separate validation split, the expected accuracy on genuinely new garbage photos is likely lower than 97.86%; cross-validation or a validation set would give a more honest estimate.
- Editorial inference: the consistent SVM-over-softmax gain suggests the same classifier swap may help other small-data image tasks where a fine-tuned network produces a fixed feature vector and the labeled set is small.
- Editorial inference: the results are measured on white-background studio photos, so a natural next test is how the same pipeline holds up on cluttered real-world recycling-bin images, where background variation could reduce the gap between models.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents an empirical comparison of five fine-tuned convolutional neural networks (AlexNet, VGG-16, GoogleNet, ResNet, SqueezeNet) combined with two classifiers (Softmax and SVM) for six-class garbage classification on the TrashNet dataset. The authors report that GoogleNet+SVM achieves the highest accuracy of 97.86% using a train/test split of half of the 2527 images, with no data augmentation. The paper also compares these results with previously reported scratch and fine-tuned models and concludes that SVM is a more successful classifier than Softmax.
Significance. If the reported 97.86% accuracy were obtained under a sound evaluation protocol, the result would be a useful empirical data point for transfer-learning-based recycling classification on a public dataset. The comparison across four well-known architectures and two classifiers is potentially informative for practitioners. However, the paper's central quantitative claim is not supported by the evaluation methodology as described, because the reported figure is the maximum of ten test-set accuracies, and no validation split, multiplicity correction, or variance estimate is provided. The manuscript also lacks the experimental detail needed for replication. The paper does not provide code, trained models, or a precisely specified data split, and the internal inconsistency in the SqueezeNet results further reduces confidence in the reported numbers.
major comments (4)
- [Section III, Table 1] The headline accuracy of 97.86% for GoogleNet+SVM is the maximum over ten model/classifier combinations evaluated on the same test half, with no separate validation set, repeated random split, confidence interval, or multiple-comparison correction. Selecting the best performer on the test set makes this an over-optimistic estimate of that combination's generalization accuracy, and the abstract and conclusion rely directly on this number. The authors should either select the best model on a validation split and test it only once, or report the distribution of accuracies over repeated splits and adjust for multiplicity.
- [Section II-E and II-F] The train/test split is not specified beyond 'half of data set'; no random seed, stratification by class, or verification of class balance is reported, and the SVM classifier details (kernel type, C, gamma, and the feature layer from which the SVM input is taken) are omitted. These omissions make the experiment unreproducible, which is especially important because the paper's main contribution is an empirical accuracy comparison.
- [Section III, text after Table 1] There is an internal inconsistency in the reported SqueezeNet+Softmax result: Table 1 lists the accuracy as 80.43%, but the text states that the lowest accuracy is 83.43%. This discrepancy suggests that the result reporting is not reliable and must be corrected before the comparison can be assessed.
- [Section IV] The conclusion that 'SVM is a more successful classifier than Softmax' is based on accuracy differences observed on a single test set, without any measure of variance or statistical significance testing. Given the small per-class sample sizes and the absence of repeated runs, the observed differences may be within the range of random variation; the claim needs support from confidence intervals or paired tests across multiple splits.
minor comments (5)
- [Abstract] The sentence 'transfer learning was used to obtain shorter training and test procedures with and higher accuracy' contains a typo ('with and higher') and should be reworded.
- [Section II-E, Fig. 1 caption] The caption is labeled 'Fig. 2 Samples of Dataset' while the text refers to 'Fig. 1'; the figure numbering should be made consistent.
- [Section III, Table 1] The model name 'SquezeeNet' is misspelled and should be 'SqueezeNet' consistently throughout the text and table.
- [Section II-C and II-D] Several equations and symbolic definitions are not fully explained; for instance, the stride parameter ζ is introduced but its dimensions are not defined, and Eq. 5 mixes vector and scalar notation. A careful revision of the notation would improve readability.
- [References] Some references are incomplete, e.g., [2] lacks venue and year, and the Spotify 'Spotgarbage' dataset URL is not provided; full bibliographic details should be added.
Circularity Check
No circularity was found: the reported 97.86% accuracy is an empirical test-set measurement on the external TrashNet dataset, not a quantity that reduces by construction to the paper's inputs.
full rationale
The paper's central claim is an empirical accuracy value obtained by fine-tuning standard CNN architectures on half of TrashNet and evaluating on the held-out half. There is no fitted parameter renamed as a prediction, no definitional relationship between input and output, and no load-bearing self-citation: the only citations to prior work are for standard architectures, the TrashNet dataset, and literature comparisons in Tables 2 and 3. The conclusion that SVM outperforms Softmax is a direct comparison of measured accuracies, so it does not reduce to the inputs by construction. The main methodological weakness is that the winning accuracy is the maximum over ten model/classifier combinations evaluated on the same test split, with no validation split or multiplicity correction; this is a statistical overfitting and selection concern, not circular reasoning. The text also contains an internal inconsistency (80.43% vs 83.43% for SqueezeNet+Softmax), but an inconsistency is a correctness or reporting issue, not circularity. Therefore the circularity score is 0.
Assumptions & free parameters
free parameters (2)
- SVM hyperparameters (C, kernel type, gamma)
- Train/test split (random seed and assignment)
assumptions (3)
- domain assumption The TrashNet dataset is a valid benchmark and the 2527 images are correctly labeled.
- domain assumption A 50/50 random train/test split is unbiased and sufficient for evaluating generalization.
- ad hoc to paper Fine-tuned pretrained networks combined with SVM produce a reliable accuracy estimate.
Cite this review
Pith. "Pith review of Fine-Tuning Models Comparisons on Garbage Classification for Recyclability." pith.science (2026). https://pith.science/paper/QNLRIDMJ
@misc{pith2026190804393,
author = {Pith},
title = {Pith review of: Fine-Tuning Models Comparisons on Garbage Classification for Recyclability},
year = {2026},
howpublished = {\url{https://pith.science/paper/QNLRIDMJ}},
note = {Machine review of arXiv:1908.04393}
}
read the original abstract
In this study, it is aimed to develop a deep learning application which detects types of garbage into trash in order to provide recyclability with vision system. Training and testing will be performed with image data consisting of several classes on different garbage types. The data set used during training and testing will be generated from original frames taken from garbage images. The data set used for deep learning structures has a total of 2527 images with 6 different classes. Half of these images in the data set were used for training process and remaining part were used for testing procedure. Also, transfer learning was used to obtain shorter training and test procedures with and higher accuracy. As fine-tuned models, Alexnet, VGG16, Googlenet and Resnet structures were carried. In order to test performance of classifiers, two different classifiers are used as Softmax and Support Vector Machines. 6 different type of trash images were correctly classified the highest accuracy with GoogleNet+SVM as 97.86%.
Reference graph
Works this paper leans on
-
[1]
Imagenet classification with deep convolutional neural networks,
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems 25, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2012, pp. 1097–1105
work page 2012
-
[2]
Auto -trash sorts garbage automatically at the techcrunch disrupt hackathon
J. Donovan, “Auto -trash sorts garbage automatically at the techcrunch disrupt hackathon.”
-
[3]
Spotgarbage: Smartphone app to detect garbage using deep learning,
G. Mittal, K. B. Yagnik, M. Garg, and N. C. Krishnan, “Spotgarbage: Smartphone app to detect garbage using deep learning,” in Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, ser. UbiComp ’16. New York, NY, USA: ACM, 2016, pp. 940 –945
work page 2016
-
[4]
Intelligent liberation and classification of electronic scrap,
S. Zhang and E. Forssberg, “Intelligent liberation and classification of electronic scrap,” Powder technology, vol. 105, no. 1, pp. 295 –301, 1999
work page 1999
-
[5]
Exploring features in a bayesian framework for material recognition,
C. Liu, L. Sharan , E. H. Adelson, and R. Rosenholtz, “Exploring features in a bayesian framework for material recognition,” in Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010, pp. 239 –246
work page 2010
-
[6]
Classification of Tras h for Recyclability Status
Thung, Gary and M. Yang. “Classification of Tras h for Recyclability Status.” (2016)
work page 2016
-
[7]
Bircanoglu, C., Atay, M., Beser, F., Genc, O., & Kizrak, M. A. (2018). RecycleNet: Intelligent Waste Sorting Using Deep Neural Networks. 2018 Innovations in Intelligent Systems and Applications (INISTA). doi:10.1109/inista.2018.8466276
arXiv 2018
-
[8]
Abubaker A., 2012. Mass Lesion Detection Using Wavelet Decomposition Transform and Support Vector Machine, IJCSIT, 4(2), 33–46
work page 2012
Show all 12 references
-
[9]
Support -Vector Networks, Machine Learning, 20, 273–297
Saitta L., 1995. Support -Vector Networks, Machine Learning, 20, 273–297
1995
-
[10]
Pattern Classification, 2nd ed., John Wiley & Sons Asia PTE
Duda R.O., Hart E.P., Stork D.G., 200 6. Pattern Classification, 2nd ed., John Wiley & Sons Asia PTE
-
[11]
Appropriate Kernel Functions for Support Vector Machine Learning with Sequences of Symbolic Data, Machine Learning Workshop LNAI, 3635, 255 –279
Vanschoenwinkel B., Manderick B., 2005. Appropriate Kernel Functions for Support Vector Machine Learning with Sequences of Symbolic Data, Machine Learning Workshop LNAI, 3635, 255 –279
2005
-
[12]
Trashnet,
G. Thung, “Trashnet,” GitHub repository, 2016
2016
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.