{"id":"5ecc2db8-eaf1-4fe7-9cdf-c7bea9e73b50","arxiv_id":"2603.16351","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A YOLOv26 classifier reaches 96% accuracy on 11 Hymenoptera families, and HiResCAM maps link its decisions to traditional taxonomic traits.","lead":"The paper trains YOLO image classifiers on 3,556 microscope images to identify 11 families of Hymenoptera (including Ichneumonidae and Braconidae), reaching about 96% test accuracy. It adds HiResCAM heatmaps to argue that the model relies on wing venation and body structures, aiming to speed up biodiversity surveys.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Potential within-specimen data leakage: Table 2 splits by images, not specimens; if DAPWH contains multiple views per specimen (as Figs. 7–8 suggest), train/test share individuals and the 96.14% accuracy is inflated.","rationale":"The reader's weakest assumption concerns external representativeness of the test hold-out. I agree that external validation is missing and that overclaims about generalization should be tempered. However, a more immediate, potentially more damaging concern is internal validity: the split unit appears to be images, not specimens. The figures in Sections 3.3–3.5 repeatedly show multiple views (lateral habitus, head frontal, wings, metasoma) for the same families, making multi-image specimens plausible. If the DAPWH metadata confirms multiple images per specimen, the 96.14% accuracy could be inflated by near-duplicate images in both training and test, which would undermine the central classification claim even on the internal test set. This is directly testable from the published dataset and does not require recollecting data. If the metadata instead shows one image per specimen, the internal accuracy is more believable, and the remaining concern is external generalization. I therefore maintain the reader's CONDITIONAL verdict, with the added condition that the authors clarify specimen/image structure and, if needed, rerun with specimen-level splits. This is not a rejection because the outcome depends on a verifiable fact rather than on a demonstrated inconsistency in the current text.","tokens_in":15486,"tokens_out":7533,"duration_ms":75510,"concrete_test":"Download DAPWH from Zenodo (doi:10.5281/zenodo.18501018) and inspect the metadata for a specimen/individual identifier per image. Count unique specimen IDs and the number of images per specimen. If any specimen has more than one image, retrain YOLOv26 with a specimen-level split (all images of a given specimen in the same fold, same 70/15/15 proportions) and recompute test accuracy. Report both the number of unique specimens and the new accuracy; if accuracy drops materially below 96.14%, the headline result depends on within-specimen leakage.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 2.3 describes a 70/15/15 split of the 3,556 'images' and Table 2 reports counts per family by image, but the paper never states whether each image corresponds to a unique specimen or whether multiple views of the same individual are present. The figures strongly suggest the latter: Fig. 7 shows both 'Habitus lateral' and 'Head frontal' for Ichneumonidae, Fig. 8 does the same for Braconidae, and Figs. 9–20 show wing, face, and metasomal images from the same families, consistent with a standardized multi-view imaging protocol. If multiple images share a specimen, an image-level random split violates independence: near-duplicate views of the same specimen can appear in both training and test, allowing the model to memorize specimen-specific artifacts. The reported Top-1 Accuracy (96.14%), Precision (93.43%), Recall (97.04%), and the per-family values in Section 3.1 would then overestimate generalization to unseen specimens. This is not merely an external-domain shift concern; it would invalidate the internal test set. The paper also reports no number of unique specimens, no specimen ID column, and no evidence that the split was stratified or grouped by specimen.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes a deep-learning pipeline for family-level identification of Hymenoptera, with emphasis on the parasitoid superfamily Ichneumonoidea. The authors contribute the DAPWH dataset of 3,556 microscope images spanning 11 families, fine-tune YOLOv12 and YOLOv26 classification models at 512×512 resolution on a 70/15/15 image-level split, and report that the best model (YOLOv26) achieves 96.14% top-1 accuracy, 93.43% precision, 97.04% recall, and 95.20% F1 on the test set. They also use HiResCAM to produce class-activation heatmaps, which they interpret as evidence that the model attends to diagnostic morphological features such as wing venation and metasomal structure. The dataset and code are made publicly available.","tokens_in":15826,"tokens_out":2775,"duration_ms":29008,"significance":"If the reported performance transfers to unseen specimens and imaging conditions, the system would be a practical tool for accelerating biodiversity assessments of a hyper-diverse and taxonomically challenging superfamily. The open dataset and code are valuable contributions, and the use of HiResCAM is an appropriate step beyond opaque 'black-box' classification. However, the central empirical claim—96% accuracy on unseen Ichneumonoidea—depends on whether the image-level split avoids specimen-level leakage, which the manuscript does not demonstrate. The XAI 'confirmation' language also exceeds what post-hoc heatmaps can establish. The work is a useful application report, but its core generalization claims are not yet fully supported.","major_comments":[{"comment":"The train/validation/test split is performed on images, not on specimens. Table 2 reports counts per family as numbers of images, and the text in §2.3 says '70% of the total images,' but the paper never states whether each image corresponds to a unique specimen. The figures strongly suggest a multi-view protocol: Fig. 7 shows both 'Habitus lateral' and 'Head frontal' for Ichneumonidae, Fig. 8 shows the same two views for Braconidae, and Figs. 9–20 show wing, face, and metasomal images from the same families. If multiple views of one specimen appear in both training and test partitions, near-duplicate images violate the independence assumption and can inflate the reported 96.14% accuracy, precision, and recall. The authors must (a) report the number of unique specimens, (b) perform a grouped split by specimen ID, or (c) demonstrate that each image is a distinct specimen. This is load-bear","section":"§2.3, Table 2, Figs. 7–20"},{"comment":"The claim of 'robust generalization across morphological variations' is not supported by the experimental design. The test set is drawn from the same DCBU collection and the same Leica/Helicon imaging pipeline as the training set. There is no external dataset, no cross-collection evaluation, and no evaluation on images acquired with different lighting, orientation, or stacking settings. Moreover, the paper reports single runs with no confidence intervals or repeated-seed experiments, so the 96.14% figure is a point estimate. The wording in the abstract and §3.1 should be tempered to describe performance on the DAPWH test set, not robust generalization to unseen imaging domains.","section":"§3.1, Abstract"},{"comment":"The abstract and conclusion state that HiResCAM visualizations 'confirm that the model focuses on taxonomically relevant anatomical regions,' but this is not a confirmatory test. The heatmaps are post-hoc qualitative interpretations, and the paper itself acknowledges in §3.3 that in some cases the model relied on 'non-traditional diagnostic characteristics' rather than features in standard keys. There is no quantitative comparison of heatmap regions against annotated morphological landmarks, and no blinded expert evaluation of whether the highlighted regions are diagnostic. The language 'confirm' and 'validate' overstates what the evidence can show. Recommend rewording to 'suggest' or 'are consistent with' and adding a quantitative evaluation if the confirmatory claim is retained.","section":"§3.3–3.5, Abstract, Conclusion"},{"comment":"The paper claims YOLOv12 and YOLOv26 represent 'current state-of-the-art,' but the evaluation includes no non-YOLO baselines such as a ResNet or vision transformer, no comparison to the cited works (e.g., Shirali et al. 2024 with BEiTv2), and no human-expert accuracy on the same test set. The only comparison is YOLOv12 vs. YOLOv26. This is insufficient to support the state-of-the-art claim and gives no calibration of how much the 96% figure improves upon existing methods or expert performance. A minimal fix is to add a standard baseline and, if possible, a human expert evaluation on the same test images.","section":"§2.2, §3.1"}],"minor_comments":[{"comment":"The conclusion refers to 'the incorporation of Grad-CAM,' but the paper uses HiResCAM. Please correct this inconsistency.","section":"Conclusion"},{"comment":"The caption reads 'nandibles open'—likely a typo for 'mandibles open.'","section":"Fig. 15 caption"},{"comment":"The split is described as 70/15/15, but the text does not explicitly state that the split was stratified by family. Table 2 shows near-proportional counts, but the process should be described (e.g., stratified random split with a fixed seed).","section":"§2.3"},{"comment":"Per-class accuracy values (e.g., 'Ichneumonidae achieved 97% accuracy') are mentioned in the text but are not reported in a table; please include a per-class breakdown with confidence intervals, especially for the underrepresented families Colletidae (n=51) and Halictidae (n=75).","section":"§3.1"},{"comment":"The title and keywords use both 'HiresCam' and 'HiResCAM'; please standardize to 'HiResCAM' throughout.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The specimen-leakage concern in §2.3/Table 2 is the key issue for the editor: if multiple images per specimen exist, the reported accuracy may be materially inflated, and the paper must be revised around a specimen-level split. The XAI claims also need to be softened or quantified. The open data/code and the biological application are positive aspects, but the manuscript is not yet ready for acceptance without addressing these load-bearing points."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper reports 96.14% top-1 accuracy for family-level classification of parasitoid wasps on a new public dataset (DAPWH), with YOLOv26 and HiResCAM explainability. The headline number is internally plausible, but I can't endorse it as evidence of generalization to unseen specimens: the train/test split is by image, not by specimen, and the paper's own figures show multiple views of the same individual (habitus lateral, head frontal, wing, metasoma) with no specimen ID or grouped split reported. That's a load-bearing omission, not a nit.\n\nWhat's genuinely new: DAPWH itself — 3,556 high-resolution images across 11 Hymenoptera families, with the Ichneumonoidea well represented, released on Zenodo. That's a useful community resource. Benchmarking YOLOv12 and YOLOv26 on this task is a reasonable application, and the qualitative comparison to morphological keys (2m-cu vein, areolet, fused metasomal tergites) in the HiResCAM figures is a nice sanity check.\n\nThe paper does several things right: a clean 70/15/15 split, per-family counts in the confusion matrices, and honest acknowledgment that the model sometimes relies on non-traditional traits. The public dataset and code are real evidence of reproducibility, even if I couldn't run them myself.\n\nSoft spots, in order of severity. First, the leakage risk. Section 2.3 says 'images' were split 70/15/15 and Table 2 counts images. Nowhere does it state that each image is a unique specimen, and Figs. 7–8 show 'Habitus lateral' and 'Head frontal' from the same families, consistent with a multi-view protocol. If multiple views of a specimen appear in both training and test, the model can memorize specimen-specific artifacts and the 96.14% is inflated. The paper's claim that the split ensures generalization to 'unseen Ichneumonoidea specimens' is exactly what is not established. Second, no confidence intervals or repeated runs, so a single lucky seed could explain the reported gap between YOLOv12 and YOLOv26. Third, the phrase 'robust generalization across morphological variations' outruns the evidence: the test set comes from the same collection, same microscope, same stacking pipeline. Fourth, the XAI section uses 'confirm' when HiResCAM shows where the model looks, not that it looks for the right reason. The conclusion even calls it Grad-CAM once, a minor editorial slip.\n\nNone of this kills the dataset or the task. But the central quantitative claim needs a specimen-grouped split (or explicit proof of one image per specimen), plus error bars and at least one non-YOLO baseline, before the generalization language is justified.\n\nWho this is for: entomologists and biodiversity informatics people who want a ready-made family-level classifier and a public benchmark. It deserves a serious referee — the dataset and question are worth referee time — but the leakage issue has to be resolved first. I'd send it to review, and I'd ask the authors to redo the split by specimen.","headline":"Useful new dataset and a plausible-looking 96% accuracy figure, but the image-level split may leak specimens across train/test, so the generalization claim is not yet established.","tokens_in":16344,"tokens_out":2471,"would_cite":true,"duration_ms":26275,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A YOLOv26 classifier trained on 3,556 focus-stacked microscope images identifies parasitoid wasp families with 96.14% top-1 accuracy, and HiResCAM maps show it relies on the same wing-vein and body traits used in taxonomic keys.","keywords":["Ichneumonoidea","YOLO","HiResCAM","insect identification","Hymenoptera","explainable AI","wing venation","biodiversity informatics"],"falsifier":"Identify the same eleven families in a second collection photographed with different equipment or in the field and compare family-level accuracy; alternatively, occlude the wing-vein regions the heatmaps highlight for Ichneumonidae and check whether accuracy actually falls. If accuracy drops sharply, or stays unchanged when the highlighted region is removed, the paper's generalization and attention claims are not supported.","tokens_in":15442,"feed_emoji":"🐝","tokens_out":5214,"duration_ms":55717,"temperature":0.7,"pith_summary":"The paper tries to show that a YOLO-based deep-learning classifier can reliably tell apart families of parasitoid wasps in the hyper-diverse superfamily Ichneumonoidea, and can do so using the same anatomical traits a taxonomist would use. On a dataset of 3,556 high-resolution, focus-stacked Hymenoptera images from one Brazilian collection, the YOLOv26 nano model reaches 96.14% top-1 accuracy on a held-out test set, with 93.43% precision and 97.04% recall. The authors argue that HiResCAM heatmaps confirm the model attends to wing venation, antennal segmentation, and metasomal structures that match traditional morphological keys. A sympathetic reader would care because routine family-level identification of these ecologically important parasitoids is normally slow, expertise-dependent work, and an accurate, explainable automatable step would help biodiversity surveys and biological-control programmes.","feed_headline":"YOLO model IDs parasitoid wasp families at 96% accuracy","feed_subtitle":"Heatmaps show the model keys on wing veins and body segments that match traditional taxonomic characters.","key_machinery":"The machinery is the YOLOv26 nano classification model, a compact convolutional network, combined with HiResCAM, a class-activation-mapping method that produces element-wise importance scores at full resolution. The model is trained at 512×512 pixels on the stacked microscope images; HiResCAM is applied at inference to generate heatmaps that the authors compare against the characters in published identification keys, such as fore-wing vein 2m-cu, the discosubmarginal cell, and metasomal fusion. This combination lets the paper make a two-part argument: high accuracy plus biologically plausible attention.","core_discovery":"The central claim is that a YOLOv26 nano image classifier, trained and tested on 3,556 high-resolution, focus-stacked Hymenoptera images, identifies eleven families—including the two ichneumonoid families Ichneumonidae (97% per-family accuracy) and Braconidae (above 94%)—at 96.14% top-1 accuracy overall. The paper further claims that HiResCAM activation maps show the model bases decisions on taxonomically diagnostic characters: presence or absence of the fore-wing vein 2m-cu and the areolet, fused metasomal tergites 2+3 in Braconidae, and scopa or corbicula in Apidae. Where the model attends to structures not in standard keys, the authors read this as evidence of usable, under-emphasised cha","pith_inferences":["Editorial inference: the 96.14% figure is measured on a test split from the same collection and imaging pipeline as the training images; accuracy on specimens from other collections, different lighting, or field photographs is an open question the paper does not test.","Editorial inference: the cases where the model ignores textbook characters and uses broader body patterns suggest the heatmaps could be mined as a source of candidate new taxonomic characters, but those candidates would need morphological confirmation before being treated as discovered traits.","Editorial inference: HiResCAM is used qualitatively; a quantitative localization check—such as occluding the highlighted wing region and measuring how much accuracy drops—would sharpen the claim that the model causally depends on those regions."],"forward_implications":["Family-level screening of Ichneumonoidea specimens could be automated at roughly 96% accuracy, reducing the bottleneck of expert-only identification in large biodiversity samples.","The HiResCAM heatmaps give a concrete way to audit each prediction against morphological knowledge, allowing misclassifications to be inspected rather than treated as black-box outputs.","Per-family results—97% for Ichneumonidae and above 94% for Braconidae—suggest the approach is already serviceable for the two focal families, not only for easier classes.","Because the dataset and code are public, other groups can retrain or fine-tune the model for their own collections; the same pipeline could later be pushed to subfamily or genus level."],"fun_headline_variants":["AI IDs wasps by the wing veins in heatmaps","YOLO+heatmaps ID wasp families at 96% accuracy","Heatmaps reveal AI's wasp ID cues from anatomy","Model IDs parasitoid wasp families via heatmaps"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing assumption is that the 15% test hold-out, drawn from the same institutional collection and imaged with the same microscope and focus-stacking workflow as the training images, is representative enough to justify the paper's claim of generalizing across morphological variation.","fun_headline_variants_meta":{"raw":{"variants":["AI IDs wasps by the wing veins in heatmaps","YOLO+heatmaps ID wasp families at 96% accuracy","Heatmaps reveal AI's wasp ID cues from anatomy","Model IDs parasitoid wasp families via heatmaps"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001101,"raw_usage":{"total_tokens":4473,"prompt_tokens":834,"completion_tokens":3639,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":578,"completion_tokens_details":{"reasoning_tokens":3581}},"tokens_in":578,"tokens_out":3639,"duration_ms":23852,"temperature":1.0,"reasoning_tokens":3581,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T18:01:27.854118+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Identify the same eleven families in a second collection photographed with different equipment or in the field and compare family-level accuracy; alternatively, occlude the wing-vein regions the heatmaps highlight for Ichneumonidae and check whether accuracy actually falls. If accuracy drops sharply, or stays unchanged when the highlighted region is removed, the paper's generalization and attention claims are not supported.","supporting_citations":[],"review_version":1}