{"id":"11fc3c3e-6a8c-4892-9e4f-2bc150863919","arxiv_id":"2411.16250","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A YOLOv8 and SVM pipeline for grading diabetic retinopathy reports 84% accuracy, but the claim lacks the dataset details, code, and error bars needed for verification.","lead":"This paper applies a YOLOv8 object detector and an SVM classifier to fundus images to find diabetic retinopathy lesions and grade severity, reporting 84% classification accuracy. The work is a standard combination of existing tools, but it could offer a low-cost screening workflow if the numbers are independently reproduced.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 84% testing accuracy is unsupported because the paper never establishes that YOLO-generated lesion counts and the 80/20 split provide an unbiased, leakage-free test set.","rationale":"The reader's weakest_assumption identifies essentially the same vulnerability: noisy annotations/labels plus an unspecified split can invalidate the 84% accuracy. My stress-test agrees with the REJECT verdict because the paper provides no dataset size, annotation count, split definition, or code, and the clinical test is explicitly deferred. The paper does include a reasoned motivation for early DR detection and a standard YOLO+SVM pipeline, but those are textbook components, not independent evidence for the specific 84% result. The most load-bearing gap is the unverified independence of the SVM test set from YOLO training data and the absence of any label-cleaning protocol despite the authors' own admission of Kaggle label errors. The proposed concrete test — a fixed public benchmark reproduction with disclosure of the split — would settle whether the accuracy claim generalizes or was an artifact of the undocumented pipeline. Since this concern is already central to the reader's rationale, the verdict remains REJECT/UNCHANGED.","tokens_in":6942,"tokens_out":1803,"duration_ms":19713,"concrete_test":"Re-run the exact proposed pipeline (YOLOv8 lesion detection followed by SVM classification on lesion counts) on the public APTOS 2019 Blindness Detection dataset using a fixed, pre-registered train/test split, and require the authors to report the number of annotated images and the split indices. If the SVM testing accuracy on this independent split is not within a reasonable margin of the reported 84%, or if the authors cannot disclose the overlap status between YOLO training images and SVM test images, the central claim should be considered unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the reported 84% SVM testing accuracy. Its load-bearing assumption is that the feature matrix — lesion type counts produced by the authors' custom YOLOv8 detector — and the Kaggle severity labels form a valid supervised dataset, and that the 80/20 split is independent of YOLO training. Nowhere does the paper state the number of images annotated with makesense.ai, the total dataset size, the number of images used to train YOLO, or whether any SVM test image was also seen by YOLO during training. The paper itself warns that 'Kaggle contains many images with poor quality and incorrect labelling,' yet no cleaning, relabeling, or exclusion protocol is described. If YOLO was trained on annotations drawn from the same pool that later generated SVM features, or if the SVM test split overlaps with YOLO training images, the 84% figure can reflect information leakage rather than generalization. Additionally, the YOLO model's reported precision is only 72%, meaning lesion counts are substantially noisy, yet the SVM accuracy is presented without confidence intervals or error bars. Without these experimental details, the accuracy claim is a claim-without-derivation: there is no way to determine whether it would survive on a held-out fundus dataset.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript describes a two-stage pipeline for grading diabetic retinopathy (DR) from fundus images: a custom YOLOv8 object detector is trained on bounding-box annotations (made with makesense.ai) to locate lesions, the detected lesions are counted per image to form a numeric feature vector, and a support vector machine (SVM) is trained on those features to predict DR severity on a five-class scale. The authors report 84% test accuracy, 82% precision, and 81% F1 for the SVM, and 78% accuracy, 72% precision, and 74% F1 for the YOLO detector. The paper includes a GUI for image input and prediction, but the experimental description omits dataset size, class distribution, training hyperparameters, validation protocol, leakage prevention, and comparisons with prior work. The conclusion cautiously notes that clinical testing has not yet been performed.","tokens_in":7158,"tokens_out":3144,"duration_ms":33138,"significance":"If the 84% accuracy figure were rigorously established, the proposed integration of YOLOv8 lesion counting with SVM severity classification could be a useful low-cost triage tool, particularly for remote settings. The high-level pipeline is reasonable: lesion types and counts are clinically meaningful features for DR grading, and the GUI demonstrates a practical deployment path. However, the paper contributes no reproducible code, no dataset, no experimental protocol, no statistical uncertainty quantification, and no comparison against existing DR grading methods. The claimed novelty is not substantiated; the contribution is an integration of existing components whose performance is currently supported only by an undocumented empirical number. As a result, the significance is contingent on experimental details that are not present in the manuscript.","major_comments":[{"comment":"The dataset is not described with the numbers needed to interpret the result: no total image count, no number of subjects, no per-class distribution across the five DR grades, and no train/validation/test partition sizes are given. The manuscript itself states that 'Kaggle contains many images with poor quality and incorrect labelling,' yet no cleaning, relabeling, exclusion, or quality-control protocol is described. Without this information, the reported 84% test accuracy cannot be interpreted as a statement about generalization to unseen fundus images.","section":"Dataset Description"},{"comment":"The number of images annotated with makesense.ai and used to train YOLOv8 is not stated, nor are the YOLOv8 hyperparameters, number of epochs, input resolution, augmentation scheme, or the set of lesion classes annotated. The procedure then applies the custom detector to 'the whole diabetic retinopathy image dataset' to generate the feature matrix, and the SVM is trained on an 80/20 split of that matrix. If any image used for SVM testing was also seen by YOLO during training or validation, the derived lesion counts could reflect information leakage rather than independent generalization; no mechanism to prevent such overlap is described.","section":"Training Work-Flow"},{"comment":"The Results section reports only four aggregate scalar metrics for the SVM (91% training accuracy, 84% testing accuracy, 81% F1, 82% precision) and three for YOLO (78% accuracy, 74% F1, 72% precision). No confusion matrix, per-class precision/recall, confidence intervals, cross-validation, or external test set is reported. DR grading is typically class-imbalanced, so aggregate accuracy alone can be misleading; the central claim that the model 'can correctly predict the grade of diabetic retinopathy up to 84% of the testing images' requires per-class evidence and an explicit statement of how the test set was constructed.","section":"Result"},{"comment":"The paper repeatedly calls the method 'novel,' but it provides no comparison with any baseline or prior DR grading system, and the literature survey does not report baseline accuracies from the cited papers. A novelty claim without comparative evaluation is an overclaim; the manuscript needs at least a comparison with a simple classifier on the same features, a standard CNN classifier, and published benchmark results on the same dataset.","section":"Method And Materials / Proposed System"},{"comment":"The SVM description is not reproducible: the text says 'we train our SVM model on the training data by finding the optimal hyperparameters,' but it never specifies the kernel type, regularization parameter C, gamma, feature scaling method, feature selection criterion, or dimensionality reduction technique. Similarly, the claimed YOLO metrics are given without the evaluation protocol used to obtain them. These omissions are load-bearing because the 84% accuracy figure cannot be verified or reproduced.","section":"Training Work-Flow"}],"minor_comments":[{"comment":"The sentence 'We have test it on our dataset taken from online website yet the clinical test is to be performed' is duplicated verbatim in the conclusion and should be corrected and reduced to one statement.","section":"Conclusion"},{"comment":"The literature survey contains several incomplete or garbled sentences, for example 'r results, the problem was that it was quite slow' and the reference citations occasionally run into the text; the section needs careful editing and full bibliographic entries for all cited works.","section":"Literature Survey"},{"comment":"There are minor language and typographical errors, including 'upto 84%' in the Results, 'lessions' in the literature survey, 'loose' for 'lose' in the lesion descriptions, and inconsistent figure references (Figure 10 appears before its caption context). A thorough proofread is needed.","section":"Throughout"},{"comment":"The abstract and introduction state the method can be applied 'for retinal fundus disease triage, especially in remote areas,' but no runtime, hardware, or deployment analysis is provided; this claim should either be supported or softened.","section":"Dataset Description"}],"recommendation":"reject","confidential_remarks":"This manuscript, as submitted, does not meet the reporting standards for a scientific evaluation of a medical-image-analysis method. The central numerical claim is unsupported by the necessary dataset description, experimental protocol, leakage analysis, and comparisons. These are not cosmetic issues; they are load-bearing and would require either access to the original data and annotations or a completely new evaluation to address. If the authors can later provide a fully documented dataset, a clear train/validation/test protocol with overlap prevention, per-class results with uncertainty estimates, and baseline comparisons, a resubmission could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a sincere student-project write-up, not a research contribution. The pipeline (YOLOv8 lesion detection, then SVM on lesion counts) is described clearly, and the authors are honest that the Kaggle labels are noisy and that clinical validation hasn't been done. But the headline 84% test accuracy is unsupported: no dataset size, no annotation count, no split details, no error bars, no baselines. The stress-test concern about leakage is real. Because YOLO was trained on bounding boxes drawn from the same Kaggle pool that later generated the SVM feature matrix, and the paper never says whether the SVM test images were also in YOLO's training set, the reported accuracy could simply reflect information leakage. The YOLO precision of 72% means the lesion counts are fairly noisy, and the SVM accuracy is given without any uncertainty.\n\nWhat's new: essentially nothing. The paper's own literature survey cites multiclass SVM on DR features [5], and lesion detection with object detectors is routine. The specific YOLOv8-to-SVM feature-count integration is a standard engineering choice, not a new algorithm or insight. That said, the paper does do a few things right: it provides a clear step-by-step workflow, it includes a GUI description, and it explicitly labels the clinical test as not yet performed. The writing is mostly straightforward, apart from the garbled literature survey and the duplicated sentence in the conclusion.\n\nSoft spots, in order of severity: (1) The central accuracy claim is a claim-without-derivation. This is load-bearing and cannot be fixed by minor revision; the authors would need to report the full experimental protocol. (2) No comparison to existing methods, so even a well-supported 84% would not tell us whether the pipeline is worth using. (3) Overclaiming 'novel' in the abstract and conclusion when the method is a combination of off-the-shelf tools. Minor: the 'existing system' section is just a description of clinical exams, and the SVM/YOLO hyperparameters are absent.\n\nBottom line: this paper is for a course report, not a peer-reviewed venue. I'd desk-reject it. If the authors supply dataset size, annotation counts, leakage-free split, error bars, and baselines, it could become a modest workshop paper, but as it stands there is no reproducible evidence to evaluate.","headline":"Sincere student project, not a research contribution: the 84% accuracy is unsupported and the pipeline is a standard combination of existing tools.","tokens_in":7685,"tokens_out":2820,"would_cite":false,"duration_ms":25639,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A YOLOv8 detector that counts retinal lesions, followed by an SVM classifier, grades diabetic retinopathy severity with 84% accuracy.","keywords":["diabetic retinopathy","fundus imaging","YOLOv8","object detection","support vector machine","lesion detection","medical image classification","deep learning"],"falsifier":"Run the trained YOLO+SVM pipeline on an independent, expertly labeled fundus-image test set with verified severity grades and compare predicted grades to ground truth; if accuracy falls substantially below 84%, or if the detector's lesion counts do not correlate with severity, the reported performance was an artifact of the training annotations or data split.","tokens_in":6740,"feed_emoji":"👁️","tokens_out":6104,"duration_ms":58823,"temperature":0.7,"pith_summary":"The paper argues that diabetic retinopathy can be graded automatically from retinal fundus images with a two-stage pipeline: first, the YOLOv8 object-detection model locates and counts characteristic lesions (microaneurysms, hemorrhages, exudates, cotton-wool spots), then a support-vector-machine classifier converts those counts into a severity grade from 0 (no disease) to 4 (proliferative). The authors report 84% accuracy, 81% F1 score, and 82% precision for the SVM stage, and 78% accuracy for the lesion-detection stage. Their motivation is that early detection prevents vision loss, and an automated method could aid triage in settings without immediate access to eye specialists. The authors themselves note that only online images were used and that clinical testing has not yet been performed.","feed_headline":"YOLO + SVM grades diabetic retinopathy at 84% accuracy","feed_subtitle":"Object detection counts retinal lesions, then a classifier maps them to disease severity for automated triage.","key_machinery":"The central mechanism is the lesion-count feature vector. YOLOv8, a real-time object-detection network trained on manually drawn bounding boxes around lesions, converts each fundus image into counts of lesion types. Those counts are then scaled and fed to an SVM, whose separating hyperplane assigns one of the five DR severity grades. The machinery transforms high-dimensional image data into a compact, interpretable numerical summary that a classical classifier can separate.","core_discovery":"The central claim is that fundus images can be reduced to a small numerical summary—counts of each lesion type—which is enough for a classical classifier to assign diabetic retinopathy severity. In the authors' own words, YOLOv8 locates regions of interest such as the optic disc, optic cup, and lesions, and then SVM classifies these into DR stages based on the presence or absence of pathological signs. Their reported numbers are a training score of 91%, testing score of 84%, accuracy 84%, F1 81%, and precision 82% for the SVM stage, with YOLO achieving 78% accuracy, 74% F1, and 72% precision. The authors conclude that this method can be applied for retinal fundus disease triage, especially in remote areas.","pith_inferences":["The reported 84% accuracy comes from a test split of the same public dataset used for training, which the paper admits contains poor-quality and incorrectly labelled images; real-world generalization to expertly graded fundus images is likely lower.","A natural extension is to test the same lesion-count vector with other classifiers, such as random forests or gradient boosting, which might improve the grading stage without retraining the detector.","The detector's 78% lesion accuracy implies some lesions are missed; severity grades relying on lesion counts may be less reliable for mild DR, where a single missed microaneurysm can change the grade."],"forward_implications":["If the 84% accuracy holds on independently cleaned data, the pipeline could serve as a first-pass triage tool in primary-care and telemedicine settings, flagging images that need specialist review.","The two-stage design means large end-to-end severity-labeled datasets are not required; only lesion bounding boxes and count-based labels are needed for training.","The SVM's decisions are tied to named lesion types, so a clinician could see which pathological signs drove the predicted grade.","Because each image is reduced to a short vector of counts, the method is computationally light enough to run on modest hardware, which matters for remote clinics."],"supporting_citations":[{"why":"Supplies the clinical context of machine-learning models predicting diabetic retinopathy and other complications, motivating automated risk assessment.","marker":"[1]"},{"why":"Establishes the base approach of using artificial neural networks for early retinal disease detection, which this paper extends with object detection.","marker":"[2]"},{"why":"Provides retinal blood-vessel segmentation techniques that inform the detection stage of the proposed pipeline.","marker":"[3]"},{"why":"Shows an earlier automated method for detecting red lesions in fundus images, the type of lesion this pipeline counts.","marker":"[4]"},{"why":"Direct predecessor that classifies diabetic retinopathy stages with a multiclass SVM, the same classifier used here.","marker":"[5]"},{"why":"Contributes a model-based method for retinal vessel detection that the authors draw on for the machine-learning detection workflow.","marker":"[6]"}],"fun_headline_variants":["YOLO-SVM pipeline grades diabetic retinopathy at 84% accuracy","Diabetic retinopathy grading: YOLO finds lesions, SVM classifies","84% accuracy: YOLO + SVM triage retinal disease","Automated DR grading from fundus images hits 84%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The pipeline's performance rests on the assumption that the manually drawn lesion boxes used to train the detector are accurate and that the public dataset's severity labels are correct; the paper itself states that the dataset contains many poor-quality, incorrectly labelled images, and no cleaning or annotation-validation step is described.","fun_headline_variants_meta":{"raw":{"variants":["YOLO-SVM pipeline grades diabetic retinopathy at 84% accuracy","Diabetic retinopathy grading: YOLO finds lesions, SVM classifies","84% accuracy: YOLO + SVM triage retinal disease","Automated DR grading from fundus images hits 84%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000146,"raw_usage":{"total_tokens":1136,"prompt_tokens":852,"completion_tokens":284,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":468,"completion_tokens_details":{"reasoning_tokens":218}},"tokens_in":468,"tokens_out":284,"duration_ms":10112,"temperature":1.0,"reasoning_tokens":218,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:19:08.936939+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained YOLO+SVM pipeline on an independent, expertly labeled fundus-image test set with verified severity grades and compare predicted grades to ground truth; if accuracy falls substantially below 84%, or if the detector's lesion counts do not correlate with severity, the reported performance was an artifact of the training annotations or data split.","supporting_citations":[{"cited_title":"A systematic literature review of predicting diabetic retinopathy, nephropathy and neuropathy in patients with type 1 diabetes using machine learning","cited_arxiv_id":null,"evidence_quote":"Supplies the clinical context of machine-learning models predicting diabetic retinopathy and other complications, motivating automated risk assessment."},{"cited_title":"Segmentation of retinal blood vessels using artificial neural networks for early detection of diabetic retinopathy,","cited_arxiv_id":null,"evidence_quote":"Establishes the base approach of using artificial neural networks for early retinal disease detection, which this paper extends with object detection."},{"cited_title":"Retinal blood vessel segmentation employing image processing and data mining techniques for computerized retinal image analysis. Retinal blood vessel segmentation in fundus images,","cited_arxiv_id":null,"evidence_quote":"Provides retinal blood-vessel segmentation techniques that inform the detection stage of the proposed pipeline."},{"cited_title":"Automated detection of,","cited_arxiv_id":null,"evidence_quote":"Shows an earlier automated method for detecting red lesions in fundus images, the type of lesion this pipeline counts."},{"cited_title":"“Multiclass SVM -Based Automated Diagnosis of Diabetic Retinopathy,","cited_arxiv_id":null,"evidence_quote":"Direct predecessor that classifies diabetic retinopathy stages with a multiclass SVM, the same classifier used here."},{"cited_title":"Model based method for retinal blood vessel detection,","cited_arxiv_id":null,"evidence_quote":"Contributes a model-based method for retinal vessel detection that the authors draw on for the machine-learning detection workflow."}],"review_version":1}