{"id":"3907c703-4eae-4dda-ad54-dd5c8d14124a","arxiv_id":"2411.14184","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"EfficientNetB3 reportedly reaches 98.33% accuracy on a public oral cancer histology dataset, but the evaluation set is described inconsistently so the result cannot be verified.","lead":"This paper trains four standard image-classification networks to distinguish normal from cancerous oral tissue images and reports that EfficientNetB3 reaches 98.33% accuracy, with LIME used to highlight which pixels drive each prediction. A reader might care because automated oral cancer screening could help early diagnosis, though this study is a small benchmark on one public dataset rather than a clinical validation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 98.33% accuracy is computed on an undefined 779-image set that matches no described split; the central performance claim is therefore unverifiable.","rationale":"The reader's weakest assumption is that the evaluation set is a fixed, pre-specified, correctly described hold-out. The paper's own numbers contradict this: Section III.A describes a 126-image test set, the abstract describes a 50:50 split, and the confusion matrix in Section IV.A sums to 779 images. The 98.33% accuracy matches 766/779 from the confusion matrix, so the headline result is real only if that 779-image set is the legitimate test set. No described split yields 779 images, and the training/test counts do not add up to the stated dataset size. This is a mechanical reporting inconsistency, not an allegation of misconduct, but it makes the central claim unverifiable as written. The proposed check, recovering the exact split and recomputing the confusion matrix, would definitively resolve whether the accuracy is a genuine hold-out result or a validation-set artifact. Because this concern directly supports the reader's REJECT verdict, no adjustment is needed.","tokens_in":7577,"tokens_out":5641,"duration_ms":46565,"concrete_test":"Request the authors' data-splitting code or exact image indices, then independently run EfficientNetB3 inference on the specified hold-out split. Verify that the test set contains exactly the 779 images in the EfficientNetB3 confusion matrix and that the accuracy equals 766/779. Also compute image hashes to confirm there is no overlap between this set and the training or validation sets. If the 779-image set turns out to be the validation set selected at epoch 74, the reported 98.33% is not an independent test accuracy and the comparison to prior work collapses.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that EfficientNetB3 achieves 98.33% accuracy on OSCC histopathology images rests entirely on the identity of the evaluation set. Section III.A states the test data contains 31 Normal and 95 OSCC images (126 total) and that the data was split 70:30. The abstract describes a near-balanced 50:50 split. Yet the EfficientNetB3 confusion matrix in Section IV.A sums to 779 images, and 766/779 = 0.9833 exactly reproduces the reported accuracy. No described split of the 5192-image dataset yields 779 test images: a 70:30 split gives roughly 1558, a 50:50 split gives roughly 2596, and 15% gives roughly 779, a ratio never mentioned. Moreover, the training counts (2435 Normal + 2511 OSCC = 4946) plus the 126 test images total 5072, not 5192, so the dataset inventory is itself inconsistent. If the 779-image set is actually the validation set used for early stopping at epoch 74, the reported accuracy is a validation accuracy inflated by model selection, not an independent test result. Without code, split indices, or a clear statement of which subset produced Fig. 4, the claimed superiority over prior models cannot be assessed.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript reports fine-tuning four ImageNet-pretrained CNNs (ResNet101, VGG16, DenseNet121, EfficientNetB3) on the public Histopathological Imaging Database for oral cancer analysis, claiming that EfficientNetB3 reaches 98.33% accuracy and an F1 score of 0.9844 on OSCC versus normal histopathology images, and that LIME visualizations explain the model's decisions. The paper includes training/validation curves, confusion matrices for two models, a metrics table, and a comparison with prior studies. The central claims are that EfficientNetB3 outperforms the other three architectures as well as previously published models on the same dataset, and that LIME adds clinically relevant interpretability.","tokens_in":7785,"tokens_out":2613,"duration_ms":26364,"significance":"If the reported results were reproducible and the evaluation set were correctly specified, the contribution would be an incremental benchmark improvement on a public OSCC histopathology dataset, accompanied by illustrative LIME explanations. The use of a public dataset and the comparison of four standard architectures are appropriate, and the LIME examples are a useful qualitative demonstration. However, the paper's central quantitative claim is not verifiable from the manuscript: the evaluation set is described inconsistently, no code, hyperparameters, or split indices are provided, and the confusion matrix that produces the headline accuracy corresponds to a test set whose origin, size, and class balance are never consistently stated. The significance of the empirical claim is therefore substantially diminished, and the LIME analysis, while visually informative, is limited to two examples without any quantitative validation.","major_comments":[{"comment":"The identity of the evaluation set is internally inconsistent, and this inconsistency is load-bearing because the headline accuracy is computed on it. Section III.A states that the testing data contains 31 Normal and 95 OSCC images (126 total) and that the data was split 70:30. The abstract describes a nearly balanced 50:50 split. However, the EfficientNetB3 confusion matrix in Section IV.A sums to 779 images, and 766/779 = 0.9833 exactly equals the reported accuracy. No described split of the 5192-image dataset yields 779 test images: a 70:30 split would give roughly 1558, a 50:50 split roughly 2596, and a 15% split roughly 779, a ratio never mentioned. The manuscript must state which subset produced Fig. 4, how its size and class balance were determined, and why this subset is not the one described in Section III.A.","section":"III.A and IV.A"},{"comment":"The dataset inventory is internally inconsistent, which further undermines the evaluation. The text says the dataset contains 5192 images in total, with 2435 Normal and 2511 OSCC training images. These training counts sum to 4946. Adding the stated 126 test images gives 5072, not 5192. Moreover, the abstract claims a near-even split while Section III.A claims a 70:30 split. The discrepancy of 120 images is never explained. Without a consistent description of how the 5192 images were partitioned into training, validation, and test sets, no reported accuracy can be checked.","section":"III.A and IV.A"},{"comment":"The claimed test accuracy appears to be selected from validation performance rather than an independent hold-out. Section IV.A refers to epoch 74 as the 'best epoch' and Section IV.C reports the best validation accuracy as 97.59%, while Table II reports 98.33% accuracy. The paper does not explain how the 98.33% figure follows from the 97.59% validation accuracy, nor whether the confusion matrix in Fig. 4 comes from the validation set, the test set, or a combination. If the 779-image set is the validation set used for early stopping, the reported accuracy would be inflated by model selection and would not be a valid estimate of generalization. The authors must clarify the relationship among Fig. 2, Fig. 4, Table I, and Table II, and report performance on a pre-specified, untouched test set.","section":"IV.C, Table II, and Fig. 4"},{"comment":"The claim that EfficientNetB3 'outplayed all existing models' is not supported by the comparison table. Table III compares models trained and evaluated on different datasets, different image modalities, and different class balances; for example, [4] reports an F1 score of 98.00 on oral photographic images, not on the same histopathology dataset. A fair comparison would require either the same public dataset with identical splits or a clearly stated cross-dataset benchmark with matched evaluation protocols. As written, Table III cannot substantiate the superiority claim.","section":"IV.D, Table III"},{"comment":"The LIME analysis is presented as supporting the explainability and trustworthiness of the model, but the evidence is limited to two randomly selected images, one OSCC and one normal. No quantitative analysis (e.g., overlap of LIME masks with pathologist annotations, stability of explanations across perturbations, or consistency across multiple samples) is provided. The interpretability claim is therefore anecdotal and should be either substantially expanded or explicitly presented as illustrative only.","section":"IV.E"}],"minor_comments":[{"comment":"The abstract's description of the data splitting is grammatically unclear and self-contradictory: it mentions an 'estimated ratio repartition of about 52% for the OSCC group' and then refers to a 'combination set' and 'divided into half,' while Section III.A states a 70:30 split. The split ratio should be stated precisely and consistently in both places.","section":"Abstract and III.A"},{"comment":"The related work paragraph for [10] says 'The dataset was the one that we used in this study' but does not identify the dataset by name; for reproducibility, the dataset name and split should be stated explicitly when comparing with prior work.","section":"II"},{"comment":"The preprocessing and training descriptions omit several details needed for reproducibility: the optimizer, learning rate, batch size, number of epochs, augmentation parameters, and the exact fine-tuning strategy (which layers were frozen or trainable) are not reported. No code repository or random seed is provided.","section":"III.B and III.C"},{"comment":"There are numerous typographical errors and inconsistencies, including 'transfer-earning' (Section I), 'EffiecientNetB3' (contributions), 'VGG1616' (Table II), and inconsistent naming of EfficientNetB3 versus EfficientnetB3. These should be corrected in a revised manuscript.","section":"Throughout"}],"recommendation":"reject","confidential_remarks":"The manuscript is within the scope of an applied deep-learning medical imaging venue, but the core evaluation is unverifiable because the test set identity is never consistently described. The inconsistency between the stated 126-image test set and the 779-image confusion matrix that exactly produces the headline accuracy is a load-bearing issue that cannot be resolved by local edits; it requires re-running the evaluation with a properly specified hold-out set and making code or detailed split metadata available. I concur with the reader's assessment: the stress-test concern lands directly on the paper's central claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline number in this paper—98.33% accuracy for EfficientNetB3 on oral SCC histopathology—cannot be verified from the information given. The paper describes the evaluation set three different ways: Section III.A says a 70:30 split and a test set of 31 normal and 95 OSCC images; the abstract suggests a balanced 50:50 split; and the confusion matrix sums to 779 images. 766/779 is exactly 0.9833, so the reported accuracy appears to come from that 779-image set, but no described split of the 5192-image dataset produces 779 test images. The training counts (2435 + 2511 = 4946) plus the stated 126 test images come to 5072, not 5192. This is a load-bearing inconsistency, and the reader's stress-test finding holds up on reading the paper.\n\nThe paper isn't without merit. It uses a public dataset, compares four standard backbones, reports confusion matrices and learning curves, and applies LIME to two example images. The related-work survey is reasonably complete, including the Ahmad et al. hybrid model that reached 97% on the same data. The LIME visualizations are illustrative only, and the claim that they explain the model's decisions is overstated—two images are not evidence of explainability.\n\nThe soft spots beyond the split problem: no code or hyperparameters are provided, so the work is not reproducible. The best epoch (74) and best model were selected from validation performance, which adds selection bias; if the 779-image set is actually the validation set, the 98.33% is a validation accuracy, not a test result. That distinction is never made clear. There's also a mismatch between the validation accuracy in Table I (97.59%) and the 98.33% in Table II, which suggests the two tables come from different evaluation runs.\n\nWho is this for? Perhaps readers tracking benchmarks on this specific Kaggle dataset, or a class on why evaluation set definition matters. For clinical use, no. For reproducibility, no.\n\nRecommendation: I would not send this to peer review as submitted. The central claim is unverifiable from the paper itself. If the authors provide code, a clearly defined hold-out split, and the actual evaluation-set composition, the result could be checked; as is, reject.\n\nI agree with the reader's REJECT verdict. The paper shows honest effort but not clear enough thinking about the evaluation protocol to be saved by a light revision.","headline":"The 98.33% accuracy claim is unverifiable because the paper never consistently defines the evaluation set; the rest is an incremental benchmark.","tokens_in":8358,"tokens_out":4728,"would_cite":false,"duration_ms":39184,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Fine-tuned EfficientNetB3 classifies oral cancer histopathology images with 98.33% accuracy.","keywords":["oral squamous cell carcinoma","OSCC","deep learning","transfer learning","EfficientNetB3","LIME","explainable AI","histopathology image classification"],"falsifier":"Count the images in the actual evaluation run: the reported precision, recall, and accuracy imply 766 correct out of 779, but the methodology section describes a test set of only 31 normal and 95 OSCC images. Re-running EfficientNetB3 on a precisely defined stratified hold-out from the original 5,192 images and comparing the resulting confusion matrix with the reported one would settle whether the 98.33% figure is attached to the dataset the paper claims to test.","tokens_in":7369,"feed_emoji":"🔬","tokens_out":8543,"duration_ms":73243,"temperature":0.7,"pith_summary":"This paper claims that a fine-tuned EfficientNetB3 convolutional network can classify histopathology images of oral squamous cell carcinoma (OSCC) against normal tissue with 98.33% accuracy and an F1 score of 0.9844, outperforming ResNet101, DenseNet121, VGG16, and earlier published models on the same public 5,192-image dataset. It also applies LIME, a model-agnostic explanation technique, to show which regions of a slide image push the model toward a diagnosis. The authors present the result as evidence that a relatively small, computationally cheap network can deliver top diagnostic accuracy while providing interpretable output for clinicians. The practical payoff they point to is earlier and more trustworthy detection of oral cancer.","feed_headline":"Tuned network reads oral cancer slides at 98.33% accuracy","feed_subtitle":"EfficientNetB3 beats larger networks on the public OSCC dataset while LIME maps the image regions behind each diagnosis.","key_machinery":"The load-bearing mechanism is transfer learning with EfficientNetB3, a convolutional network whose compound scaling jointly scales depth, width, and resolution and whose inverted-bottleneck blocks use depthwise separable convolutions and squeeze-and-excitation modules; this is what lets a 12M-parameter model outclass 44.5M-parameter ResNet101 and 138M-parameter VGG16 on this task. The explanation mechanism is LIME (Local Interpretable Model-Agnostic Explanations), which generates 5,000 perturbed pseudo-images around a given test image, weights each pseudo-image by proximity to the original, fits a sparse surrogate model, and returns a superpixel-level saliency mask. That mask is the observable output the paper uses to argue the model is looking at texture- and color-differentiated regions rather than arbitrary artifacts.","core_discovery":"On the paper's own terms, the central discovery is that transfer-learning fine-tuning of EfficientNetB3 yields the best published accuracy on the public OSCC histopathology dataset: 98.33% accuracy, 0.9903 precision, 0.9782 recall, and 0.9844 F1-score with 12 million parameters. The confusion matrix reports 356 correct normal and 410 correct OSCC classifications with 13 misclassifications, a total of 779 evaluated images. DenseNet121 is the second-best model at 90.24% accuracy, while the larger ResNet101 and VGG16 models score 89.31% and 81.77%, respectively. LIME visualizations on two example images then attribute the model's decisions to specific superpixel regions, with the authors concluding that texture and color in those regions are decisive for distinguishing infected from normal tissue.","pith_inferences":["A reader who wants to build on this number should first demand a single test-set definition: the paper reports 779 evaluated images in the confusion matrix while describing a 126-image test set elsewhere, so the reproducibility of the headline depends on resolving that discrepancy, not on the model choice.","LIME attribution is a statement about which superpixels move the model's prediction, not about causal tissue biology; comparing the highlighted regions with a pathologist's tumor annotations across many cases would be the natural test of clinical value.","Because training and validation curves show the best checkpoint near epoch 74 and validation accuracy near 98%, the reported test accuracy likely sits close to the validation ceiling; a multi-fold cross-validation would show whether 98.33% is stable or a favorable single split."],"forward_implications":["If the accuracy transfers to unseen clinical slides, OSCC screening could deploy on modest hardware, since EfficientNetB3 reaches this result with 12M parameters while the larger ResNet101 and VGG16 models perform worse.","The LIME saliency maps provide a reviewable output: a pathologist can inspect whether the highlighted superpixels correspond to tumor-associated texture and color before accepting an automated diagnosis.","The comparison table puts fine-tuned EfficientNetB3 ahead of every prior model on the same dataset, so any future work on this benchmark will need to beat 98.33% rather than the previous 97.5%.","The authors' stated next step is to transfer the same fine-tuning-plus-LIME workflow to other cancer types and locally collected hospital images, which would test whether the accuracy is dataset-specific."],"supporting_citations":[{"why":"Supplies the public 5,192-image histopathology dataset (Normal and OSCC) used for training and evaluation.","marker":"[12]"},{"why":"The online source from which the dataset was downloaded in train/validation/test folders; the paper's split follows this organization.","marker":"[13]"},{"why":"The previous transfer-learning baseline (VGG19, 96.26%) on the same 5,192-image dataset that this paper claims to beat.","marker":"[1]"},{"why":"An earlier study on the same dataset using SVM features from DenseNet201 (97% accuracy), a prior result in Table III.","marker":"[10]"},{"why":"A CNN baseline (97.50% accuracy) included in the comparison table as a result EfficientNetB3 must outperform.","marker":"[3]"},{"why":"A CapsNet baseline (97.35% accuracy) included in the comparison table.","marker":"[5]"},{"why":"The DenseNet-169 study with an F1 score of 0.98, used as the closest F1-based comparison in Table III.","marker":"[4]"},{"why":"The LIME explainable-AI procedure and surrogate-sampling details the authors adopt from their prior work.","marker":"[14]"},{"why":"Supplies the precision, recall, F1, and accuracy equations used to report all metrics.","marker":"[15]"}],"fun_headline_variants":["EfficientNetB3 detects oral cancer with 98% accuracy","AI model reads oral cancer slides at 98.33% accuracy","LIME explains why EfficientNetB3 nails oral cancer diagnosis","Small efficient model beats giants on oral cancer detection","Deep learning plus LIME boosts oral cancer diagnosis to 98%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim stands or falls with the evaluation set: the accuracy can only be what is reported if the 779-image confusion-matrix set is a fixed, correctly described hold-out with accurate labels, but the paper never consistently states that set—Section III.A describes a 126-image test, mentions a 70:30 split, and the abstract describes a near-balanced 50:50 set—so if the actual test images were chosen after inspecting validation performance or overlap training data, the headline number collapses.","fun_headline_variants_meta":{"raw":{"variants":["EfficientNetB3 detects oral cancer with 98% accuracy","AI model reads oral cancer slides at 98.33% accuracy","LIME explains why EfficientNetB3 nails oral cancer diagnosis","Small efficient model beats giants on oral cancer detection","Deep learning plus LIME boosts oral cancer diagnosis to 98%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00055,"raw_usage":{"total_tokens":2670,"prompt_tokens":1035,"completion_tokens":1635,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":651,"completion_tokens_details":{"reasoning_tokens":1549}},"tokens_in":651,"tokens_out":1635,"duration_ms":10721,"temperature":1.0,"reasoning_tokens":1549,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:26:14.521600+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Count the images in the actual evaluation run: the reported precision, recall, and accuracy imply 766 correct out of 779, but the methodology section describes a test set of only 31 normal and 95 OSCC images. Re-running EfficientNetB3 on a precisely defined stratified hold-out from the original 5,192 images and comparing the resulting confusion matrix with the reported one would settle whether the 98.33% figure is attached to the dataset the paper claims to test.","supporting_citations":[{"cited_title":"Histopatho- logical imaging database for oral cancer analysis,","cited_arxiv_id":null,"evidence_quote":"Supplies the public 5,192-image histopathology dataset (Normal and OSCC) used for training and evaluation."},{"cited_title":"Histopathologic oral cancer detection us- ing cnns,","cited_arxiv_id":null,"evidence_quote":"The online source from which the dataset was downloaded in train/validation/test folders; the paper's split follows this organization."},{"cited_title":"Oral cancer detection using transfer learning-based framework from histopathology images,","cited_arxiv_id":null,"evidence_quote":"The previous transfer-learning baseline (VGG19, 96.26%) on the same 5,192-image dataset that this paper claims to beat."},{"cited_title":"Multi-method analysis of histopathological image for early diagnosis of oral squamous cell carcinoma using deep learning and hybrid techniques,","cited_arxiv_id":null,"evidence_quote":"An earlier study on the same dataset using SVM features from DenseNet201 (97% accuracy), a prior result in Table III."},{"cited_title":"Automated classification of cells into multiple classes in epithelial tissue of oral squamous cell carcinoma using transfer learning and convolutional neural network,","cited_arxiv_id":null,"evidence_quote":"A CNN baseline (97.50% accuracy) included in the comparison table as a result EfficientNetB3 must outperform."},{"cited_title":"Capsule network based analysis of histopathological images of oral squamous cell carcinoma,","cited_arxiv_id":null,"evidence_quote":"A CapsNet baseline (97.35% accuracy) included in the comparison table."},{"cited_title":"AI-based analysis of oral lesions using novel deep convolutional neural networks for early detection of oral cancer,","cited_arxiv_id":null,"evidence_quote":"The DenseNet-169 study with an F1 score of 0.98, used as the closest F1-based comparison in Table III."},{"cited_title":"Optimizing Stroke Detection: An Analysis of Different Feature Selection Approaches,","cited_arxiv_id":null,"evidence_quote":"Supplies the precision, recall, F1, and accuracy equations used to report all metrics."}],"review_version":1}