{"id":"a1602d03-fdd2-48b9-a7e8-1db57978b217","arxiv_id":"2509.08780","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"On a newly curated 20-class dataset of 11,489 mobile skin images, a Swin Transformer reached 86% accuracy for classifying arsenicosis and similar dermatoses, with external validation claimed but underreported.","lead":"A deep learning framework using mobile-captured skin photos to sort 20 skin conditions, with arsenicosis as a key class, reaches 86% accuracy with a Swin Transformer. The authors also built a web demo and explainability tools, aimed at screening in rural arsenic-affected areas.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Arsenic labels lack a clinical reference standard and come almost entirely from one repository; the near-perfect arsenic recall and unquantified external-validation claim (Sec. 7.4) do not yet support the generalization claim.","rationale":"I read the paper in good faith as a useful empirical study introducing a new benchmark for a neglected disease. The model implementation, experimental setup, and reporting are internally consistent; the confusion matrices and training curves align with the stated numbers. The most load-bearing weakness is not the choice of architecture or metric but the validity of the arsenic labels and the possibility of source confounding. The arsenic class is assembled from a single public repository plus unquantified web-scraped images, with no described expert verification or clinical reference standard. Because the paper's stated goal is 'diagnosis,' label noise in the target class is not a minor nuisance but a direct threat to the central claim. The external validation section is too thin to substantiate generalization: four representative examples and an unquantified 'over 90%' do not establish robustness beyond the curated dataset. The paper's own Limitations section (7.8) concedes the lack of clinical metadata and geographic diversity, which reinforces this concern. This is not a rejection of the work; it is a request for evidence that the model is recognizing arsenicosis rather than repository-specific artifacts. If the proposed label-review check passes and the source-stratified evaluation shows stable performance, the conditional verdict can be upgraded. For now, the reader's conditional verdict seems appropriate, and my stress-test does not change it.","tokens_in":20472,"tokens_out":3821,"duration_ms":44835,"concrete_test":"Have two independent, board-certified dermatologists, blinded to the dataset labels and to each other, classify a stratified random sample of 200 arsenic images and 200 visually similar non-arsenic images (lichen planus, actinic keratosis, BCC/SCC, normal) from the curated dataset as 'arsenic-compatible' or 'not arsenic-compatible', using a standardized definition of arsenicosis skin signs. Compute Cohen's kappa between the raters and the dataset labels. If rater-label agreement is <0.8 or kappa <0.6, the arsenic class is not sufficiently validated to support the diagnostic claim. In parallel, run a source-stratified evaluation: train the Swin model without ArsenicSkinImageBD images and test on those held-out arsenic images; a large drop in arsenic recall would indicate source-artifact learning.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—that the Swin model 'diagnoses' arsenicosis with 86% accuracy and near-perfect arsenic recall (158/159 in Fig. 5b)—stands only if the arsenic class actually contains arsenicosis images and if the model learns disease features rather than source artifacts. Neither is established. Section 3 describes curation from public repositories and web scraping, with no dermatologist verification or biomarker/reference-standard check; the arsenic class is primarily ArsenicSkinImageBD (741 images) plus unquantified online images. Section 7.8 concedes no clinical metadata and a single-country source. Under a random 60/20/20 split, images from the same repository can appear in both train and test, so the model can exploit stable background, camera, or processing cues. External validation (Sec. 7.4) reports only four representative images and an unquantified 'over 90%'; no sample size, selection rule, or ground-truth source is given. Thus the accuracy and generalization claims are vulnerable to label noise and source confounding. This is not a disagreement with current consensus; it is an internal validity gap in the evidence for the claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an end-to-end deep learning framework for classifying arsenicosis and 19 other skin conditions from mobile-captured images. The authors curate a 20-class, 11,489-image dataset drawn from public repositories and web scraping, benchmark 10 CNN/Transformer architectures, and report that the Swin Transformer achieves the best performance: 86% accuracy, 0.86 weighted recall/precision/F1, and 0.85 MCC. They also provide LIME and Grad-CAM interpretability analyses, analyze misclassifications, report an external validation study, and deploy a web-based diagnostic tool. The headline accuracy is internally consistent: the Swin confusion matrix in Fig. 5b sums to 2,399 test images with 2,058 correct (85.8%). However, the clinical validity of the arsenicosis labels, the strength of the external validation, and the possibility of source confounding are not adequately established.","tokens_in":20780,"tokens_out":6409,"duration_ms":66832,"significance":"If the diagnostic claims were supported, this would be a useful contribution to an under-served area: arsenicosis is a major public health problem in South/Southeast Asia, and there is no large public image repository focused on it. The strengths of the work include the publicly released dataset and code, the multiclass comparison against visually similar dermatoses rather than a binary arsenic-vs-normal task, the transparent confusion-matrix reporting, and the integration of interpretability and deployment. The main weakness is that the paper's central claim—that the model 'diagnoses' arsenicosis and generalizes beyond the curated dataset—is not yet supported by the evidence. The labels lack a clinical reference standard, the external validation is unquantified, and the random split allows same-source images in train and test. These issues are substantial but addressable by additional validation or by reframing the claims.","major_comments":[{"comment":"The arsenicosis labels are not verified against any clinical reference standard. Section 3 describes curation from ArsenicSkinImageBD (741 images) plus unquantified web-scraped images that were 'carefully reviewed,' but no dermatologist verification, biopsy, or biomarker (hair/nail/urine arsenic) confirmation is described. Section 7.8 acknowledges the absence of clinical metadata and the single-country source, but does not address label validity. Since the central claim is diagnosis of arsenicosis, the near-perfect arsenic recall (158/159 in Fig. 5b) may reflect label noise or repository-specific artifacts rather than disease features. Please specify a label-verification protocol or explicitly reframe the results as classification of repository-assigned labels.","section":"§3, Table 2, §7.8"},{"comment":"The external validation is reported only as 'over 90%' accuracy, with no sample size, class distribution, selection rule, or ground-truth source; only four representative images are shown. This is insufficient to support the abstract and conclusion claim that the framework 'confirmed its ability to generalize beyond the curated dataset.' Provide the full external set with source, ground-truth verification, per-class results, and the number of images evaluated, or substantially weaken the generalization claim.","section":"§7.4"},{"comment":"The random 60/20/20 split can place images from the same public repository in both training and test sets. Because the arsenic images come almost entirely from one repository, the model may exploit stable acquisition conditions, background, or processing cues rather than disease-specific patterns. The very high arsenic recall (158/159) is consistent with this concern. A source-disjoint or site-wise evaluation—for example, training on ArsenicSkinImageBD images and testing on independently collected arsenic images—is needed to rule out source confounding.","section":"§5.2, §6.2"},{"comment":"The reported MCC is not reproducible as written. Equation (5) is the standard binary MCC formula, but the paper uses it for a 20-class problem and does not define how TP, TN, FP, and FN are aggregated across classes. Please specify the multiclass extension used (e.g., per-class binary averaging, or the correlation-based multiclass MCC) and ideally provide the implementation code.","section":"§5.3, Eq. (5)"}],"minor_comments":[{"comment":"The Swin confusion matrix sums to 2,399 test images, whereas 20% of 11,489 is approximately 2,298. Please clarify the per-class split counts and why the test set is larger than 20% of the total.","section":"§5.2, Fig. 5"},{"comment":"The text says the ConvNeXt model correctly classified 149 arsenic test samples 'while misclassifying approximately 10.' The confusion matrix in Fig. 5a shows an arsenic row sum of 163, implying 14 misclassifications. Please correct the number or explain the discrepancy.","section":"§6.2"},{"comment":"Table 3 lists MobileNetV2, but reference [38] is the MobileNetV3 paper ('Searching for MobileNetV3'). Please verify the citation.","section":"Table 3 / References"},{"comment":"The sentence beginning 'Incorrectlyclassifiedcases(e.g., Hand-Foot-and-MouthDiseaseandLichen Planus), the highlighted regions align well...' is missing a subject or verb and should be rewritten.","section":"§7.1.2"},{"comment":"Section 4.3 says deployment can proceed through 'two complementary pathways,' but only the web application is described in detail; the mobile application is deferred to future work. Consider clarifying the current status of each pathway.","section":"§4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is transparent and reproducible, but the current title and abstract promise a diagnostic system. I would support a revision that either adds clinically verified external validation or explicitly limits the claims to classification on a curated benchmark with repository-assigned labels. The dataset and benchmarking effort are valuable regardless of the diagnostic framing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a genuinely useful new benchmark and an honest empirical comparison, but the headline claims about diagnosing arsenicosis and generalizing beyond the curated dataset go further than the evidence supports.\n\nWhat is actually new: a 20-class, 11,489-image mobile-photo dataset with arsenicosis as one class, and the first multiclass comparison of CNNs versus transformers for this disease. The Swin result (86% accuracy, 0.85 MCC) is internally consistent — the confusion matrix sums to 2,399 test images with 2,058 correct — and the analysis of confusions (BCC/SCC/actinic keratosis, acquisition-driven errors) is sensible. The fine-tuning observation, that frozen backbones with lightweight heads beat extensive unfreezing on this small medical dataset, is a useful negative result. The LIME/Grad-CAM error analysis is also honest about cases where the model looked at the right place and still got it wrong.\n\nThe soft spot is load-bearing: label trust. The arsenic class comes almost entirely from a single public repository, with no dermatologist verification or biomarker reference standard described. Under a random 60/20/20 split, images from the same repository can appear in both train and test, so the near-perfect arsenic recall (158/159) could reflect source or background artifacts rather than disease patterns. The external validation section reports only \"over 90%\" with no sample size, selection rule, or ground-truth source; four example images do not establish generalization. The limitations section concedes missing clinical metadata and single-country data, which is good, but the abstract still says the framework \"confirmed its ability to generalize,\" which the evidence does not support.\n\nThis is not circularity in the fitted-parameter sense; it is a data-validity and evidence-claim problem. The internal benchmark stands, but the word \"diagnosis\" should be \"screening aid,\" and the external claim should be quantified or removed. The related work is fair and positions this correctly as a multiclass extension of prior binary arsenic work; citations look appropriate.\n\nWho it is for: people building mobile skin-lesion classifiers for low-resource settings, and anyone benchmarking transformers on non-dermoscopic medical images. If the dataset is actually released, it is the main lasting contribution. I would give this a serious referee; the likely right outcome is major revision adding clinician verification of at least a label sample, a properly reported external set, and softened generalization claims.","headline":"Useful dataset and benchmark for a neglected disease; the diagnostic and generalization claims outrun the evidence, but the core empirical work is honest and deserves a serious referee.","tokens_in":21234,"tokens_out":2061,"would_cite":true,"duration_ms":25802,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A Swin Transformer trained on 11,489 phone-captured skin images classifies arsenicosis among 20 classes with 86% accuracy.","keywords":["arsenicosis","deep learning","skin lesion classification","Swin Transformer","mobile-captured images","explainable AI","LIME","Grad-CAM"],"falsifier":"Run the trained model on an independent test set of arsenicosis cases confirmed by nail or hair arsenic measurement or dermatologist examination, captured at several clinics with different phones, lighting, and backgrounds. If arsenic recall drops materially below the 158/159 seen on the curated test, or if Grad-CAM heatmaps localize to background or camera artifacts rather than lesions, the central claim fails.","tokens_in":20393,"feed_emoji":"🩺","tokens_out":8052,"duration_ms":84874,"temperature":0.7,"pith_summary":"The paper argues that arsenicosis, a chronic skin disease caused by arsenic-contaminated drinking water, can be screened from ordinary mobile-phone photographs rather than invasive lab tests. To test this, the authors assembled a 20-class dataset of 11,489 non-dermoscopic skin images and benchmarked ten deep-learning architectures. They report that Transformer-based models beat every CNN, with a Swin Transformer reaching 86% accuracy, 0.86 precision/recall/F1, and 0.85 MCC, and that it missed only one of 159 arsenic test cases. They also report strong performance on external validation images, suggesting the model spreads beyond the curated set. If these numbers hold, the framework offers a practical, low-cost triage tool for rural arsenic-affected communities where dermatologists are scarce.","feed_headline":"Transformer model IDs arsenicosis from phone skin photos at 86%","feed_subtitle":"On a 20-class, 11,489-image benchmark, the Swin Transformer beats every CNN and generalizes to unseen external images.","key_machinery":"The load-bearing machinery is the newly curated 20-class dataset of 11,489 mobile-acquired skin images (819 arsenic images plus 19 other skin-condition classes and normal skin), combined with the Swin Transformer—a hierarchical vision transformer whose shifted-window self-attention captures both fine lesion texture and global skin context. The training protocol, frozen backbones pretrained on a large natural-image corpus with only a lightweight classification head trained, together with early stopping and checkpoint selection, keeps the small-data regime stable. LIME and Grad-CAM—local superpixel-based and gradient-heatmap explanation methods—supply the interpretability layer, converting raw","core_discovery":"The central claim is that arsenicosis can be reliably distinguished from nineteen other classes in a 20-class skin-image benchmark using end-to-end deep learning on mobile-captured images, provided the model is trained on a broad multiclass benchmark rather than a binary arsenic-versus-normal task. On the curated dataset, the Swin Transformer achieves 86% accuracy, weighted recall/precision/F1 of 0.86, and MCC of 0.85, with 158 of 159 arsenic test images correctly recognized. The authors further report that self-attention models generalized to external images outside the curated distribution, that LIME and Grad-CAM overlays showed the model attending to lesion-relevant regions rather than ba","pith_inferences":["Editorial inference: because the arsenic images are drawn largely from a single regional public collection and no biomarker or biopsy confirmation is described, the 86% figure should be read as a ceiling until an independent study re-labels the same images with dermatologist- and biomarker-confirmed diagnoses.","Editorial inference: the paper's own error analysis points to poor framing, lighting, and blur as a major failure source, so adding an automated image-quality and lesion-framing gate before classification would likely recover a share of the misclassifications and is directly testable on the published confusion matrices.","Editorial inference: if the model is later validated across skin tones and countries, the same 20-class recipe could support community-level water-safety screening programs, since photos can be collected by field workers with no clinical training."],"forward_implications":["Rural health workers could screen for arsenicosis with an ordinary smartphone and receive an instant prediction plus a visual explanation, reducing dependence on scarce dermatologists.","Arsenicosis detection should be treated as a multiclass problem against visually similar skin conditions; binary arsenic-versus-normal results overstate practical performance.","Transformer-based architectures, especially Swin, are the stronger base for this task on small mobile-image datasets, outperforming every CNN variant tested.","Frozen pretrained backbones with a lightweight head, early stopping, and checkpoint selection are the more stable training strategy for small medical datasets; aggressive fine-tuning hurts.","A browser-based deployment of the model is feasible, and external images classified mostly correctly suggest the model transfers beyond the curated images."],"fun_headline_variants":["Swin Transformer wins arsenicosis skin-photo screening at 86% accuracy","Mobile-captured skin images: Transformer framework nails arsenicosis","Explainable deep learning screens arsenicosis from phone photos","20-class skin image benchmark: Swin Transformer bests CNNs for arsenicosis","Arsenicosis detection from phone pics: 86% accuracy with Transformer"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The 86% accuracy holds only if the curated labels are correct—especially that the arsenic images truly show arsenicosis and the other classes truly show the named conditions—because no confirmatory biomarker test or independent dermatologist adjudication is described.","fun_headline_variants_meta":{"raw":{"variants":["Swin Transformer wins arsenicosis skin-photo screening at 86% accuracy","Mobile-captured skin images: Transformer framework nails arsenicosis","Explainable deep learning screens arsenicosis from phone photos","20-class skin image benchmark: Swin Transformer bests CNNs for arsenicosis","Arsenicosis detection from phone pics: 86% accuracy with Transformer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000689,"raw_usage":{"total_tokens":3000,"prompt_tokens":826,"completion_tokens":2174,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":570,"completion_tokens_details":{"reasoning_tokens":2080}},"tokens_in":570,"tokens_out":2174,"duration_ms":17016,"temperature":1.0,"reasoning_tokens":2080,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T20:09:51.751102+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained model on an independent test set of arsenicosis cases confirmed by nail or hair arsenic measurement or dermatologist examination, captured at several clinics with different phones, lighting, and backgrounds. If arsenic recall drops materially below the 158/159 seen on the curated test, or if Grad-CAM heatmaps localize to background or camera artifacts rather than lesions, the central claim fails.","supporting_citations":[],"review_version":1}