{"id":"c9c1b43e-5783-4c3a-b7fc-7c8543089f25","arxiv_id":"2505.11424","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A YOLOv8s-seg model trained for 50 epochs detects facial wrinkles with 90.7% precision and 87.0% mAP50 on a 131-image validation set, but achieves only 10.2% mAP50-95, indicating poor localization at strict IoU thresholds.","lead":"This paper reports precision, recall, and mAP numbers for a YOLOv8 segmentation model trained to find wrinkles in facial images, using a small Roboflow dataset. It shows the model can find forehead wrinkles and frown lines at moderate confidence, but its low mAP50-95 of 10.2% means precise localization is poor.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed 'strong localization' is contradicted by the paper's own mAP50-95 of 10.2% versus mAP50 of 87.0%, and no baseline supports the title's 'improving' claim.","rationale":"The reader's verdict is REJECT, and my stress-test pass does not change that. The reader's weakest_assumption focused on the small, unversioned validation set and the missing confidence threshold; that is a legitimate reproducibility and generalization concern. My analysis goes further: even granting every reported number, the Abstract's central claim of 'strong localization and segmentation capabilities' is internally inconsistent with the reported mAP50-95 of 10.2%. In YOLO, mAP50-95 averages performance across IoU thresholds up to 0.95, so a very low value with a high mAP50 indicates that predictions are poorly localized at stricter overlap criteria. Section 3.6 explicitly concedes suboptimal higher-IoU mAP and under-segmentation, and Section 3.5 reports no baseline comparison despite the title claiming 'improvement.' Together these points mean the paper's central claim fails on its own reported evidence, independent of the dataset-size question. The concrete test would settle the numerical contradiction by exposing per-threshold mAP values; if the authors cannot supply the checkpoint and evaluation logs, the result is not independently checkable. I therefore keep the reader's REJECT verdict unchanged, while noting partial agreement because my primary reason differs from the reader's weakest_assumption.","tokens_in":9522,"tokens_out":2621,"duration_ms":26407,"concrete_test":"Obtain or re-run the evaluation with the same trained YOLOv8s-seg checkpoint on the 131-image validation set, and log mAP at each IoU threshold from 0.5 to 0.95 in steps of 0.05, together with Precision and Recall at the default confidence threshold. If mAP@0.75 is far below mAP@0.5 (for example, below 30% while mAP50 is near 87%), the 'strong localization' claim in the Abstract is numerically refuted. Also compute per-class mAP50-95; if all classes remain near 10%, the poor localization is systematic rather than an artifact of one class. If the checkpoint or dataset version cannot be provided, the reported metrics cannot be independently verified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central positive claim in the Abstract — that the model 'demonstrated strong localization and segmentation capabilities' — is contradicted by the headline metrics reported in Section 3.5: mAP50 is 87.0% while mAP50-95 is 10.2%. In the YOLO evaluation protocol, mAP50-95 averages mean Average Precision over IoU thresholds from 0.5 to 0.95, so a value of 10.2% alongside an mAP50 of 87.0% means that detections and masks have very poor overlap with ground truth at stricter IoU thresholds. The paper itself acknowledges in Section 3.6 that 'mAP values at higher IoU thresholds were suboptimal' and that under-segmentation occurred, and Figure 5's caption notes slight overfitting. Thus the abstract's 'strong localization' assertion is not supported by the paper's own evidence. In addition, the title promises 'Improving Object Detection Performance' but no baseline model is trained or compared, so the improvement claim is unsubstantiated. The reader's concern about the small, unversioned 131-image validation set and the unspecified confidence threshold compounds the issue: even if the reported Precision and Recall are taken at face value, they were computed on a small sample at an unreported operating point and cannot support a generalization claim. The most load-bearing problem, however, is the internal inconsistency between the reported mAP50-95 and the abstract's characterization of strong localization.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an experimental study in which a YOLOv8s-seg model is fine-tuned for 50 epochs on a Roboflow facial-wrinkle dataset and evaluated on a 131-image validation split. The reported results include Precision 90.7%, Recall 89.1%, mAP50 87.0%, mAP50-95 10.2%, Mask Precision 80.7%, and Mask Recall 89.1%, together with per-class results for forehead wrinkles, frown lines, and general wrinkles. The abstract characterizes the model as demonstrating strong localization and segmentation capabilities, while the paper's own Section 3.6 states that mAP at higher IoU thresholds was suboptimal. The title promises \"Improving Object Detection Performance,\" but no baseline or comparison is presented.","tokens_in":9814,"tokens_out":6596,"duration_ms":58232,"significance":"If the reported metrics were internally consistent and reproduced on a larger, well-defined dataset, the study would offer a modest benchmark for YOLOv8-based wrinkle segmentation. As it stands, the primary claims are undermined by (i) an unresolved contradiction between the abstract's \"strong localization\" and the reported mAP50-95 of 10.2%, (ii) an impossible numerical relationship between the overall and per-class metrics in Section 3.5, and (iii) the absence of any baseline, ablation, or statistical uncertainty quantification. The work is a conventional application of an existing architecture and does not currently provide a reliable basis for its improvement claims.","major_comments":[{"comment":"The abstract's statement that the model \"demonstrated strong localization and segmentation capabilities\" is directly contradicted by the reported mAP50-95 of 10.2% against an mAP50 of 87.0%. In standard YOLO evaluation, mAP50-95 averages average precision over IoU thresholds from 0.5 to 0.95; such a large gap indicates that predictions have poor overlap with ground truth at stricter IoU thresholds. The paper itself acknowledges in Section 3.6 that \"mAP values at higher IoU thresholds were suboptimal\" and that under-segmentation occurred. The central claim of strong localization is therefore not supported by the paper's own evidence and must be corrected.","section":"Abstract and Section 3.5"},{"comment":"The overall and per-class metrics are numerically inconsistent. With exactly three classes (forehead, frown, general wrinkles), the overall mAP50 of 87.0% should equal the mean of the per-class mAP50 values 85.7%, 81.6%, and 83.7%, which is 83.7%, not 87.0%. Moreover, the overall Precision of 90.7% cannot be greater than each of the per-class precision values (85.0%, 80.5%, 88.6%) if these are the only classes and the overall precision is the pooled statistic; as a weighted average of the per-class precisions, it must lie between the minimum and maximum of those values. At least one set of the reported numbers is therefore erroneous, and the results section does not provide a plausible account of these discrepancies.","section":"Section 3.5"},{"comment":"The title's claim of \"Improving Object Detection Performance\" is unsubstantiated because the study contains no baseline. No comparison is made with any other model, training configuration, or previous result on the same dataset. The paper evaluates a single fine-tuned YOLOv8s-seg configuration and reports its metrics; that describes a performance study, not an improvement over any existing method. Without a baseline or ablation, the improvement claim cannot be supported.","section":"Title, Section 3.4, and Section 5"},{"comment":"The evaluation protocol is not fully specified. The validation set consists of 131 images from an unversioned Roboflow dataset whose total size is never given; no confidence interval or error bar is reported; and the confidence threshold at which Precision and Recall are computed is not stated. The paper mentions in Section 3.1 that 10% of the data is held out for testing, but no test-set results are reported anywhere. These omissions make the reported metrics irreproducible and prevent the reader from assessing whether the observed performance reflects the model's generalization ability or the particular choice of operating point.","section":"Sections 3.1, 3.4, and 3.5"}],"minor_comments":[{"comment":"Figure 1 and Figure 3 appear to be the same precision-confidence curve; Figures 2 and 4 are recall-confidence curves with different maximum recalls (0.81 and 0.72) but identical captions. Please clarify what each figure represents and why duplicates are included.","section":"Figures 1-4"},{"comment":"The text contains a discussion of confusion matrices and metrics applied to \"diabetes mellitus cases,\" which is unrelated to the wrinkle segmentation task; this passage appears to be carried over from another context and should be removed or rewritten.","section":"Section 3.4"},{"comment":"The Related Work section is entirely about vehicle detection and tracking and does not discuss facial wrinkle detection, segmentation, or YOLOv8-seg. A proper literature review relevant to the paper's topic is needed.","section":"Section 2"},{"comment":"The exact training configuration is missing: batch size is described as \"appropriate for GPU memory capacity,\" and the learning rate schedule and augmentation parameters are not quantified. Please report these values for reproducibility.","section":"Section 3.4"},{"comment":"Equation (3) contains a typographical error (\"Re TPcall\") and should read \"Recall = TP/(TP+FN).\"","section":"Equation (3)"},{"comment":"References [38]-[48] are cited to support standard definitions of precision, recall, and the confusion matrix, but those references concern diabetes classification, suicide prediction, and other unrelated topics; standard machine-learning textbooks or canonical metric papers would be more appropriate.","section":"References [38]-[48]"}],"recommendation":"reject","confidential_remarks":"This manuscript gives the impression of having been assembled from multiple unrelated sources: the Related Work section addresses vehicle detection, the method section discusses diabetes classification, and the evaluation section contains internally inconsistent metrics. Beyond the points in the report, I would flag to the editor that the numerical contradictions in Section 3.5 should be resolved before any further consideration, and that the paper's contribution, as it stands, does not meet the standard for this journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a routine YOLOv8s-seg application to a small wrinkle dataset, and the paper's own numbers undercut its headline claim. The one honest thing is Section 3.6, which admits the mAP at higher IoU thresholds is suboptimal. But the Abstract says 'strong localization' and the title promises 'improving' when there is no baseline and mAP50-95 is 10.2%. That is not a minor wording issue; it is the central claim failing on its own evidence.\n\nWhat is actually here: the authors train a standard model with COCO pretrained weights on a Roboflow wrinkle dataset, 50 epochs, and report the usual detection and mask metrics on 131 validation images. As a single data point for YOLOv8 on facial wrinkles, it is mildly informative. The limitations section is appropriately modest; they note the small dataset and under-segmentation. The metric definitions in Section 3.4 are standard, though the confusion matrix in Table 1 is mislabeled.\n\nThe problems are larger than the reader's summary suggests. The related work section is entirely about vehicle detection—none of it about wrinkle detection or skin analysis—so the paper does not situate itself in the relevant literature. The confusion matrix table has rows and columns swapped, with TP under 'Negative' and FN under 'Positives'. Figures 3 and 4 duplicate Figures 1 and 2 with nearly identical captions but different recall values, which suggests sloppy assembly. The dataset is described only as 'from Roboflow,' with total size unstated and no version or split details beyond 70/20/10, and no confidence threshold is given for the precision/recall point estimates. No code, no weights, no external comparison.\n\nThe mAP50-95 vs mAP50 gap (87.0 vs 10.2) is the load-bearing problem. In YOLO evaluation, that gap means masks and boxes align well at IoU 0.5 but fall apart under stricter overlap criteria. Calling that 'strong localization' is simply wrong, and the paper's own Section 3.6 contradicts it. The title's 'Improving' is unearned without a baseline.\n\nBottom line: this does not deserve a serious referee. It is a run-of-the-mill benchmark report with an internally inconsistent central claim and several presentation errors. A reader working on wrinkle segmentation might skim the numbers, but I would not cite it or build on it. If the authors resubmit, they need a baseline comparison, a versioned dataset, confidence intervals, and an abstract that matches their results.","headline":"Routine YOLOv8 wrinkle-segmentation report undermined by its own mAP50-95 of 10.2% and an unsupported 'strong localization' claim.","tokens_in":10349,"tokens_out":2101,"would_cite":false,"duration_ms":19822,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A YOLOv8s-seg model detects and segments facial wrinkles with 90.7% precision and 87.0% mAP50 on a 131-image validation set.","keywords":["YOLOv8","object detection","segmentation","wrinkle detection","facial analysis","deep learning","computer vision","transfer learning"],"falsifier":"Re-run the same training recipe on a larger, independently labeled facial-wrinkle dataset at a fixed confidence threshold, using a properly versioned train/validation/test split. If Precision, Recall, or mAP50 on that held-out set drop materially below the reported values, or if the 131-image validation set is found to overlap the training distribution in ways that inflate the metrics, the paper's generalization claim fails.","tokens_in":9314,"feed_emoji":"🔍","tokens_out":5300,"duration_ms":44008,"temperature":0.7,"pith_summary":"This paper claims that a lightweight YOLOv8s-seg model, fine-tuned from COCO pretrained weights for 50 epochs on a facial wrinkle dataset, can detect and segment wrinkles with high precision and recall. On a 131-image validation set, the reported metrics are Precision 90.7%, Recall 89.1%, mAP50 87.0%, mAP50-95 10.2%, Mask Precision 80.7%, and Mask Recall 89.1%. The model performs best on forehead wrinkles, lower on frown lines, and the authors note failures on subtle wrinkles, complex lighting, and overlapping features. The stated purpose is to assess whether YOLOv8-based automation is feasible for wrinkle identification in clinical and commercial settings.","feed_headline":"YOLOv8 finds facial wrinkles at 90.7% precision","feed_subtitle":"A 50-epoch fine-tuned model reports 87% mAP50 but 10.2% mAP50-95, a gap worth knowing before clinical use.","key_machinery":"The central object is the YOLOv8s-seg architecture, a single-stage detector with a modified CSPDarknet53 backbone, a PAN-FPN neck, and a head that outputs bounding boxes, segmentation masks, and class probabilities at once. The training recipe uses COCO pretrained weights for transfer learning, 50 epochs with the Adam optimizer, 640x640 resized inputs, and data augmentation such as random flipping, cropping, rotation, and brightness adjustment. The evaluative machinery is the standard set of Precision, Recall, mAP50, mAP50-95, plus Mask Precision and Mask Recall.","core_discovery":"The central discovery claimed is that a YOLOv8s-seg model trained with the reported recipe achieves strong bounding-box detection of facial wrinkles but substantially weaker high-IoU localization. The evidence is the validation set results: Precision 90.7%, Recall 89.1%, mAP50 87.0%, mAP50-95 10.2%, Mask Precision 80.7%, Mask Recall 89.1%. The large drop from mAP50 to mAP50-95 is a direct corollary of the authors' own observation that localization precision at higher IoU thresholds is suboptimal.","pith_inferences":["A reader should treat the headline Precision/Recall as threshold-dependent: the paper never states the confidence threshold, and the Precision-Confidence curve shows precision 1.00 at 0.865 confidence, so the operating point matters.","The 131-image validation set is small and the total dataset size is unstated; until a larger, multi-source test set is used, these numbers should be read as a feasibility demonstration, not a deployment guarantee.","The paper does not compare against other segmentation architectures, so it remains untested whether YOLOv8s-seg is better than, say, Mask R-CNN or transformer-based segmenters on this task.","A natural next experiment would be an ablation of the data-augmentation set and an early-stopping schedule, since the paper's own curves suggest diminishing returns and rising validation loss in late epochs."],"forward_implications":["Fine-tuning YOLOv8s-seg on a small wrinkle dataset yields usable bounding-box detection with precision above 90%, but not fine-grained segmentation at high IoU thresholds.","Per-class results suggest forehead wrinkles are the easiest class, while frown lines and general wrinkles need more data or class-specific tuning.","The low mAP50-95 (10.2%) implies the model is suitable for coarse wrinkle screening, not for precise clinical measurement.","The paper's finding that validation loss rises near the end of training indicates the training recipe may be close to an overfitting point; early stopping or heavier regularization could change results."],"supporting_citations":[{"why":"Defines the YOLO framework and provides the YOLOv8 architecture used as the detection/segmentation backbone.","marker":"[20]"},{"why":"Surveys YOLO variants through YOLOv10 and justifies selecting YOLOv8 as the current balanced single-stage detector.","marker":"[21]"},{"why":"Motivates the single-stage detector design choice that makes YOLOv8 suitable for the efficiency-focused deployment context.","marker":"[17]"},{"why":"Supplies the confusion-matrix definitions of Precision, Recall, and accuracy used to compute the reported evaluation metrics.","marker":"[38-48]"}],"fun_headline_variants":["YOLOv8 detects facial wrinkles at 90.7% precision, but high-IoU accuracy lags","Strong precision, weak localization: YOLOv8 wrinkle detector's mAP50-95 is just 10.2%","90.7% precision for wrinkle detection, yet only 10.2% mAP50-95 highlights gap","Wrinkle detection with YOLOv8: 90.7% precise, but fine localization fails at 10.2%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported performance rests on the assumption that the 131-image validation set is representative enough, and that the unspecified confidence threshold for Precision and Recall does not distort the headline numbers; if the validation set is small or skewed, or the threshold is unusual, the metrics may not reflect real-world wrinkle detection.","fun_headline_variants_meta":{"raw":{"variants":["YOLOv8 detects facial wrinkles at 90.7% precision, but high-IoU accuracy lags","Strong precision, weak localization: YOLOv8 wrinkle detector's mAP50-95 is just 10.2%","90.7% precision for wrinkle detection, yet only 10.2% mAP50-95 highlights gap","Wrinkle detection with YOLOv8: 90.7% precise, but fine localization fails at 10.2%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000204,"raw_usage":{"total_tokens":1256,"prompt_tokens":680,"completion_tokens":576,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":296,"completion_tokens_details":{"reasoning_tokens":453}},"tokens_in":296,"tokens_out":576,"duration_ms":5476,"temperature":1.0,"reasoning_tokens":453,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:53:08.648256+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same training recipe on a larger, independently labeled facial-wrinkle dataset at a fixed confidence threshold, using a properly versioned train/validation/test split. If Precision, Recall, or mAP50 on that held-out set drop materially below the reported values, or if the 131-image validation set is found to overlap the training distribution in ways that inflate the metrics, the paper's generalization claim fails.","supporting_citations":[{"cited_title":"YOLO: A Brief History; 2023","cited_arxiv_id":null,"evidence_quote":"Defines the YOLO framework and provides the YOLOv8 architecture used as the detection/segmentation backbone."},{"cited_title":"Optimizing the trade -off between single -stage and two -stage deep object detectors using image difficulty prediction","cited_arxiv_id":null,"evidence_quote":"Motivates the single-stage detector design choice that makes YOLOv8 suitable for the efficiency-focused deployment context."}],"review_version":1}