Pith. sign in

REVIEW 4 major objections 4 minor 34 references

LabelImg: CNN-Based Surface Defect Detection

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A CNN trained on 14,982 LabelImg-annotated images reports 99.54% accuracy in detecting four surface defect types on laser powder bed fusion surfaces.

desk verdict Straightforward CNN classifier for LPBF surface defects whose headline 99.54% accuracy is a training-curve number, not a held-out test result; the detection/segmentation framing is unsupported. read the letter →

arxiv 2509.05813 v1 pith:IMOHMKW2 submitted 2025-09-06 cond-mat.mes-hall

Mohsen Asghari Ilani , Yaser Mike Banad This is my paper
classification cond-mat.mes-hall
keywords convolutionalneuralnetworksurfacedefectdetectionlaserpowderbedfusionLabelImgannotationinstancesegmentationobjectOpenCVpreprocessingcrack
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to show that a deep convolutional network, trained on manually annotated bounding boxes, can automatically classify four surface defect types—crack, pinhole, hole, and spatter—on metal parts made by laser powder bed fusion. It argues that LabelImg annotations, OpenCV image cleaning, and a CNN classifier together remove the need for hand-crafted features and manual parameter tuning in quality inspection. The reported result is a peak training accuracy of 99.54%, with 0.99 accuracy, precision, and recall on a 750-image test split and per-class F1 scores from 0.97 to 1.00. If this holds, manufacturers could screen LPBF surfaces automatically and catch defects such as cracks and spatter before parts are used.

What carries the argument

The engine is a four-class convolutional classifier: three Conv2D+MaxPooling2D blocks with batch normalization and ReLU, a dropout layer, and a dense softmax head, totaling about 3.9 million parameters. LabelImg supplies the supervised signal as bounding-box XML annotations, OpenCV preprocessing supplies cleaned inputs, and class weighting compensates for uneven label frequencies. Together they turn raw 1536×1103 surface images into defect-class probabilities.

What would settle it

Take the trained model and evaluate it on a freshly annotated set of LPBF surface images from the same imaging setup that were never used in training, validation, or the reported 750-test set; if the resulting accuracy falls well below 99%, the central claim does not generalize.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that a convolutional neural network can separate laser powder bed fusion surface images into four defect categories—crack, pinhole, hole, and spatter—with near-perfect reliability. The pipeline starts with LabelImg: every object is manually assigned a bounding box and a class label, saved as XML. OpenCV preprocessing (GaussianBlur, fastNlMeansDenoising, adaptiveThreshold, Canny) cleans the images before they enter a CNN built from three Conv2D/MaxPooling blocks, a dropout layer, and a softmax classifier. The paper reports 99.54% as the peak training accuracy at epoch 51; the evaluation on 750 test instances gives an overall accuracy of 0.99, with per-class

Load-bearing premise

The 99.54% figure is assumed to come from a properly held-out test set of 750 images; if those images overlap with the training data, the reported performance does not describe new surfaces.

Editorial extensions

If this is right

  • Surface inspection for laser powder bed fusion parts could become automatic: the model classifies four defect types directly from surface images without hand-tuned parameters.
  • The label-weighting scheme raises reported accuracy from about 32% to over 99% on this dataset, so class imbalance is not a barrier once enough annotations exist.
  • The bounding-box/segmentation annotation pipeline plus denoising filters is reusable for other defect classes or other metal manufacturing processes.
  • With vision sensors mounted in a build chamber, the same classifier could flag defects layer by layer, moving toward in-situ monitoring.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The title's 'segmentation' promise is stronger than what the reported metrics support: the evaluation numbers are image-level classification counts, so pixel-level overlap quality remains untested.
  • If the dataset comes from one camera, machine, or material setup, transfer to other LPBF systems or lighting conditions is likely to lower accuracy.
  • A direct generalization check would train on images from some builds and test on held-out builds from a different material; the paper does not report such a split.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a CNN-based approach for detecting and segmenting surface defects (crack, pinhole, hole, spatter) on Laser Powder Bed Fusion (LPBF) surface images annotated with the LabelImg tool. It claims that the trained CNN, combined with OpenCV preprocessing, achieves 99.54% accuracy on a dataset of 14,982 images at 1536×1103 resolution, with per-class precision/recall/F1 scores around 0.96–1.00. The reported evaluation consists of a confusion matrix and micro/weighted averages over 750 instances.

Significance. If the claims were supported, the work would offer a practical, high-accuracy classifier for AM surface quality control. However, the manuscript's central contribution is undermined by a mismatch between the claimed task (detection/segmentation) and the implemented/evaluated method (image-level classification), and by an unsupported headline accuracy figure. The strengths are the relatively large annotated dataset and the use of standard CNN training practices, but as presented the results do not substantiate the stated capabilities.

major comments (4)
  1. [Abstract; Results and Discussion] The abstract's headline claim of '99.54% accuracy' is contradicted by the Results section, which states that training accuracy peaked at 99.54% at epoch 51 and validation accuracy peaked at 97.95% at epoch 49, with final-epoch training/validation accuracies of 99.65%/99.26%. Table 1 reports metrics on 750 instances with overall accuracy 0.99. The paper never defines a held-out test set distinct from the validation set, so the 99.54% figure appears to be a training-curve value, not an independent test result. This directly undermines the central performance claim.
  2. [Materials and Methods; Figure 5; Table 1] The title and abstract claim that the method 'detects and segments' defects, but the architecture is an image classifier: Conv2D/MaxPooling/Dense/Softmax layers output class probabilities. There is no bounding-box regression, no mask prediction head, and no detection/segmentation loss. The evaluation reports only a confusion matrix and precision/recall/F1 at the image level. No IoU, mask AP, or pixel-level metrics are provided. Thus the detection and segmentation claims are not supported by either the method or the evaluation.
  3. [Results and Discussion; Abstract] Dataset numbers are inconsistent. The abstract says training and testing were performed on 14,982 labeled images, while the Results section states that training accuracy is computed on 6,742 images and the validation set is 20% of the training data. The 750 test instances in Table 1 are not mapped back to the 14,982-image corpus, and it is unclear whether they overlap with the validation or training splits. This makes the reported metrics irreproducible and prevents a reader from interpreting the significance of the confusion matrix.
  4. [Materials and Methods] The architecture and hyperparameters were selected 'following experimentation with different configurations' based on 'the highest average classification accuracy' on the validation data. No independent external test set is used, and class-weighting coefficients are mentioned but not specified. This creates a risk of validation-set overfitting and makes the reported near-perfect metrics difficult to interpret as evidence of generalization. A concrete remedy would be to fix a single train/validation/test split before model selection and report test-set metrics with confidence intervals.
minor comments (4)
  1. [Results and Discussion] The confusion-matrix description repeats '50 data for crack detection' for each class: '50 data for crack detection, 124 data for crack detection, 258 data for crack detection, and 318 data for crack detection.' The class names for the latter three should be pinhole, hole, and spatter.
  2. [Figure 5 vs. Results text] The Results section says 'Figure 5 succinctly presents an overview of the training and validation outcomes,' but Figure 5 is titled 'Schematic of CNN-based Architecture.' The figure/text correspondence needs correction.
  3. [Results and Discussion] The phrase 'when the surface was level 1, it was predicted correctly in 49 out of 50 instances' uses classification levels rather than defect names, which is confusing. Use the class labels directly.
  4. [Introduction] Several references are to the authors' own prior arXiv preprints; while not inappropriate, the text should distinguish self-citations from external related work. Also, reference [9] is cited in a way that is not clearly related to the defect-detection topic.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical CNN benchmark; the abstract's 99.54% figure is a training-curve value, but no claimed derivation reduces to its own inputs.

full rationale

The paper does not present a first-principles derivation or a predictive theory; it trains a CNN on annotated images and reports classification metrics. The central performance claim is measured against manually annotated labels, which is standard supervised learning rather than circular reasoning. The only notable issue is that the abstract's '99.54% accuracy' appears to correspond to the training accuracy at epoch 51 (Results: 'peaking at 99.54% during the 51st epoch for training'), whereas the validation peak is 97.95% and Table 1 reports a 0.99 accuracy on 750 instances; this is a reporting inconsistency, not a case of a prediction being defined in terms of its own input. Self-citations (refs. 6–15) are present but do not carry the central claim: the architecture is selected 'following experimentation with different configurations' and evaluated on precision/recall/F1. No equation is shown to be equivalent to an input by construction, and no uniqueness theorem or ansatz is imported from prior self-cited work. Accordingly, no circular step is exhibited.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central result rests on a privately collected, manually annotated dataset and a standard CNN whose hyperparameters were tuned on the same problem. The paper provides no external benchmark, no baseline comparison, and no released artifacts, so the only evidence is the authors' own measurements on their own data.

free parameters (4)
  • Input image resize dimensions = not specified, inferred as ~120x80
    The first Conv2D output shape is (120,80,64), so the input is likely resized to around 120x80 pixels. This is a hand-chosen preprocessing choice that affects all results.
  • CNN hyperparameters (layer counts, filter sizes, kernel sizes, pooling, dropout) = 64, 512, 256 filters, 3x3 kernels, 2x2 pooling, dropout rate not given
    Selected after experimentation as the configuration with highest average classification accuracy, i.e., chosen to maximize performance on the authors' data.
  • Class weighting coefficients = not specified
    Used to balance the non-uniform label distribution during training; the actual weight values are not reported.
  • Train-validation split ratio and epoch count = 4:1 split, 100 epochs
    Chosen by the authors; no justification is provided beyond convenience.
assumptions (3)
  • domain assumption Manual annotations with LabelImg are correct ground truth
    The entire evaluation treats the hand-drawn bounding boxes and labels as true labels; no annotation quality assessment or inter-annotator agreement is provided.
  • domain assumption The 750-instance test set is an independent, representative sample
    The paper does not describe how the test set was constructed, only that the support sizes are 50/124/258/318; independence from the 6,742 training images is assumed.
  • ad hoc to paper Image-level classification accuracy is a valid measure of defect detection and segmentation
    The claimed task is detection and segmentation, but all metrics are classification metrics (precision/recall/F1 on class labels). This conflates tasks and is not justified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LabelImg: CNN-Based Surface Defect Detection." pith.science (2026). https://pith.science/paper/IMOHMKW2

@misc{pith2026250905813,
  author       = {Pith},
  title        = {Pith review of: LabelImg: CNN-Based Surface Defect Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IMOHMKW2}},
  note         = {Machine review of arXiv:2509.05813}
}
read the original abstract

In the journey of computer vision system development, the acquisition and utilization of annotated images play a central role, providing information about object identity, spatial extent, and viewpoint in depicted scenes. However, thermal manufacturing processes like Laser Powder Bed Fusion (LPBF) often yield surfaces with defects such as Spatter, Crack, Pinhole, and Hole due to the Balling phenomenon. Preprocessing images from LPBF, riddled with defects, presents challenges in training machine learning (ML) algorithms. Detecting defects is critical for predicting production quality and identifying crucial points in artificial or natural structures. This paper introduces a deep learning-based approach utilizing Convolutional Neural Networks (CNNs) to automatically detect and segment surface defects like cracks, spatter, holes, and pinholes on production surfaces. In contrast to traditional machine learning techniques requiring extensive processing time and manual feature crafting, deep learning proves more accurate. The proposed architecture undergoes training and testing on 14,982 labeled images annotated using the LabelImg tool. Each object in the images is manually annotated with bounding boxes and segmented masks. The trained CNN, coupled with OpenCV preprocessing techniques, achieves an impressive 99.54% accuracy on the dataset with resolutions of 1536 x 1103 pixels. Evaluation metrics for 50 true crack tests demonstrate precision, recall, and F1-score exceeding 96%, 98%, and 97%, respectively. Similarly, for 124 true pinhole tests, the metrics are 99%, 100%, and 100%, for 258 true hole tests, they are 99%, 99%, and 99%, and for 318 spatter tests, the metrics are 100%, 99%, and 100%. These results highlight the precision and effectiveness of the entire process, showcasing its potential for reliable defect detection in production surfaces.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

34 extracted references · 27 canonical work pages

  1. [1]

    Sensors 2011, V ol 11, Pages 9628 -9657 11:9628 –9657

    Gavilán M, Balcones D, Marcos O, et al (2011) Adaptive Road Crack Detection System by Pavement Classification. Sensors 2011, V ol 11, Pages 9628 -9657 11:9628 –9657. https://doi.org/10.3390/S111009628

    doi:10.3390/s111009628 2011

  2. [2]

    Pandian AP, Palanisamy R, Ntalianis K (2021) Proceedings of International Conference on Intelligent Computing, Information and Control Systems. 1272:. https://doi.org/10.1007/978-981-15-8443-5

    work page doi:10.1007/978-981-15-8443-5 2021

  3. [3]

    Pattern Recognit Lett 33:227–238

    Zou Q, Cao Y , Li Q, et al (2012) CrackTree: Automatic crack detection from pavement images. Pattern Recognit Lett 33:227–238. https://doi.org/10.1016/J.PA TREC.2011.11.004

    doi:10.1016/j.pa 2012

  4. [4]

    https://doi.org/101177/1475921719883202 19:1440 –1452

    Mohtasham Khani M, Vahidnia S, Ghasemzadeh L, et al (2019) Deep-learning-based crack detection with applications for the structural health monitoring of gas turbines. https://doi.org/101177/1475921719883202 19:1440 –1452. https://doi.org/10.1177/1475921719883202

    doi:10.1177/1475921719883202 2019

  5. [5]

    Eng Fract Mech 268:108467

    Joshi D, Singh TP, Sharma G (2022) Automatic surface crack detection using segmentation -based deep-learning approach. Eng Fract Mech 268:108467. https://doi.org/10.1016/J.ENGFRACMECH.2022.108467

    work page arXiv 2022

  6. [6]

    Ilani MA, Banad YM (2024) Modeling Melt Pool Geometry in Metal Additive Manufacturing Using Goldak’s Semi-Ellipsoidal Heat Source: A Data-driven Computational Approach

    2024

  7. [7]

    TransMatch: A Transfer-Learning Framework for Defect Detection in Laser Powder Bed Fusion Additive Manufacturing

    Ilani, M. A., & Banad, Y . M. (2025). TransMatch: A Transfer -Learning Framework for Defect Detection in Laser Powder Bed Fusion Additive Manufacturing. arXiv preprint arXiv:2509.01754

    work page Pith review arXiv 2025

  8. [8]

    AM-DefectNet: Additive Manufacturing Defect Classification Using Machine Learning -- A comparative Study

    Ilani, M. A., & Banad, Y . M. (2025). AM-DefectNet: Additive Manufacturing Defect Classification Using Machine Learning-A comparative Study. arXiv preprint arXiv:2509.01769

    work page Pith review arXiv 2025

Show all 34 references
  1. [9]

    A., & Banad, Y

    Ilani, M. A., & Banad, Y . M. (2025). Multi-objective Evolutionary Algorithms (MOEAs) in PMEDM- A Comparative Study in Pareto Frontier. arXiv preprint arXiv:2509.01775

    2025 arXiv

  2. [10]

    A., & Banad, Y

    Asadi, M., Sarabi, S., Kordani, M., Ilani, M. A., & Banad, Y . M. (2025). Enhanced-HisSegNet: Improved SAR Image Flood Segmentation with Learnable Histogram Layers and Active Contour Model. IEEE Geoscience and Remote Sensing Letters

    2025

  3. [11]

    A., & Banad, Y

    Ilani, M. A., & Banad, Y . M. (2024). EDMNet: unveiling the power of machine learning in regression modeling of powder mixed -EDM. The International Journal of Advanced Manufacturing Technology, 135(5), 2555-2570

    2024

  4. [12]

    A., & Banad, Y

    Ilani, M. A., & Banad, Y . M. (2024, December). NSGA-II in DNN for Interpretable AI Predictions of Melt Pool Geometry in Digital Twin Applications within IoT -Enabled Additive Manufacturing. In 2024 International Conference on AI x Data and Knowledge Engineering (AIxDKE) (pp. ...

    2024

  5. [13]

    A., & Banad, Y

    Ilani, M. A., & Banad, Y . M. (2024, December). Interpretable AI Prediction of Melt Pool Geometry in IoT-Enabled Additive Manufacturing in Industry 4.0 Utilizing CatBoost: Decision -Making via NSGA-II. In 2024 International Conference on AI x Data and Knowledge Engineering (AI...

    2024

  6. [14]

    A., & Banad, Y

    Ilani, M. A., & Banad, Y . M. (2024, September). XGBoost Algorithm for Interpretable AI Prediction of Melt Pool Geometry in IoT-Enabled Additive Manufacturing within Industry 4.0 Utilizing NSGA- II. In 2024 Conference on AI, Science, Engineering, and Technology (AIxSET) (pp. 2...

    2024

  7. [15]

    Asghari Ilani, M., & Banad, Y . M. (2024). Modeling Melt Pool Geometry in Metal Additive Manufacturing Using Goldak's Semi -Ellipsoidal Heat Source: A Data -driven Computational Approach. arXiv e-prints, arXiv-2404

    2024

  8. [16]

    Banadaki, Y ., Razaviarab, N., Fekrmandi, H., Li, G., Mensah, P ., Bai, S., & Sharifi, S. (2021). Automated quality and process control for additive manufacturing using deep convolutional neural networks. Recent Progress in Materials, 4(1)

    2021

  9. [17]

    Banadaki, Y . (2019). On the use of machine learning for additive manufacturing technology in industry 4.0. J. Comput. Sci. Inf. Technol, 7, 61-68

    2019

  10. [18]

    International Journal of Advanced Manufacturing Technology 116:1763 –1782

    Ilani MA, Khoshnevisan M (2021) Study of surfactant effects on intermolecular forces (IMF) in powder-mixed electrical discharge machining (EDM) of Ti -6Al-4V . International Journal of Advanced Manufacturing Technology 116:1763 –1782. https://doi.org/10.1007/S00170-021-07569- ...

    2021 doi

  11. [19]

    International Journal of Advanced Manufacturing Technology 120:5117 –5129

    Ilani MA, Khoshnevisan M (2022) An evaluation of the surface integrity and corrosion behavior of Ti-6Al-4 V processed thermodynamically by PM -EDM criteria. International Journal of Advanced Manufacturing Technology 120:5117 –5129. https://doi.org/10.1007/S00170 -022-09093- 4/...

    2022 doi

  12. [20]

    J Healthc Eng 2022:

    Hosseini Rad R, Baniasadi S, Yousefi P, et al (2022) Presented a Framework of Computational Modeling to Identify the Patient Admission Scheduling Problem in the Healthcare System. J Healthc Eng 2022:. https://doi.org/10.1155/2022/1938719

    2022 doi

  13. [21]

    Coarser reinforcements may induce more prominent spattering phenomenon

    observed the diamond spattering in the fabrication of GWs in LPBF and revealed the detrimental effect on the formation quality. Coarser reinforcements may induce more prominent spattering phenomenon. Due to the larger diamond grains and higher volume perc entage, spattering pl...

  14. [22]

    Int J Mach Tools Manuf 109:8 –35

    Li HN, Axinte D (2016) Textured grinding wheels: A review. Int J Mach Tools Manuf 109:8 –35. https://doi.org/10.1016/J.IJMACHTOOLS.2016.07.001

    2016 doi

  15. [23]

    J Mater Process Technol 304:117533

    Shuai L, Qinghong J, Sai G, et al (2022) Effect of spattering on formation mechanisms of metal matrix composites in laser powder bed fusion. J Mater Process Technol 304:117533. https://doi.org/10.1016/J.JMATPROTEC.2022.117533

    2022

  16. [24]

    Addit Manuf 40:101927

    Constantin L, Kraiem N, Wu Z, et al (2021) Manufacturing of complex diamond -based composite structures via laser powder -bed fusion. Addit Manuf 40:101927. https://doi.org/10.1016/J.ADDMA.2021.101927

    2021

  17. [25]

    International Journal of Advanced Manufacturing Technology 107:2387 –2395

    Denkena B, Krödel A, Harmes J, et al (2020) Additive manufacturing of metal-bonded grinding tools. International Journal of Advanced Manufacturing Technology 107:2387 –2395. https://doi.org/10.1007/S00170-020-05199-9/FIGURES/12

    2020 doi

  18. [26]

    Materials Science and Engineering: B 177:1524–1530

    Dong Y , Zhang R, He X, et al (2012) Fabrication and infiltration kinetics analysis of Ti -coated diamond/copper composites with near -net-shape by pressureless infiltration. Materials Science and Engineering: B 177:1524–1530. https://doi.org/10.1016/J.MSEB.2012.08.009

    2012 doi

  19. [27]

    2019 1st International Conference on Unmanned Vehicle Systems-Oman, UVS 2019

    Benjdira B, Khursheed T, Koubaa A, et al (2018) Car Detection using Unmanned Aerial Vehicles: Comparison between Faster R-CNN and YOLOv3. 2019 1st International Conference on Unmanned Vehicle Systems-Oman, UVS 2019. https://doi.org/10.1109/UVS.2019.8658300

    2018

  20. [28]

    Robot Comput Integr Manuf 39:32 –42

    Ding D, Pan Z, Cuiuri D, et al (2016) Bead modelling and implementation of adaptive MA T path in wire and arc additive manufacturing. Robot Comput Integr Manuf 39:32 –42. https://doi.org/10.1016/J.RCIM.2015.12.004

    2016 doi

  21. [29]

    J Intell Manuf 25:157–163

    Xiong J, Zhang G, Hu J, Wu L (2014) Bead geometry prediction for robotic GMAW -based rapid manufacturing through a neural network and a second -order regression analysis. J Intell Manuf 25:157–163. https://doi.org/10.1007/S10845-012-0682-1/METRICS

    2014 doi

  22. [30]

    J Manuf Process 45:208 –216

    Zhang Z, Wen G, Chen S (2019) Weld image deep learning -based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding. J Manuf Process 45:208 –216. https://doi.org/10.1016/J.JMAPRO.2019.06.023

    2019 doi

  23. [31]

    IEEE Access 8:69908 –69918

    Wang Y , Zhang C, Lu J, et al (2020) Weld Reinforcement Analysis Based on Long-Term Prediction of Molten Pool Image in Additive Manufacturing. IEEE Access 8:69908 –69918. https://doi.org/10.1109/ACCESS.2020.2986130

    2020

  24. [33]

    J Manuf Syst 59:12–

    Snow Z, Diehl B, Reutzel EW, Nassar A (2021) Toward in -situ flaw detection in laser powder bed fusion additive manufacturing through layerwise imagery and machine learning. J Manuf Syst 59:12–

    2021

  25. [34]

    https://doi.org/10.1016/J.JMSY .2021.01.008

    2021 doi

  26. [35]

    Addit Manuf 36:101538

    Wang C, Tan XP, Tor SB, Lim CS (2020) Machine learning in additive manufacturing: State-of-the- art and perspectives. Addit Manuf 36:101538. https://doi.org/10.1016/J.ADDMA.2020.101538

    2020

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.