{"id":"f50692d7-e6bc-48fb-bc70-fc0da2fa94fa","arxiv_id":"2502.06243","paper_version":1,"verdict":"REJECT","confidence":"LOW","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A Transformer with a loosely defined multi-scale attention weighting is reported to achieve 89.5% accuracy on ISIC 2017 skin lesion classification.","lead":"This paper describes a modified Transformer model for classifying skin lesion images, tested on the ISIC 2017 dataset. It reports slightly higher accuracy than standard models, but provides almost no experimental details.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The MultiScale attention in Eq. (3) is a weighted sum of identical Attention(Q,K,V) terms, so it collapses to a single scaled attention and cannot support the claimed multi-scale improvement; without training details or code, the reported superiority is unverifiable.","rationale":"The paper's central claim is empirical: the proposed improved Transformer outperforms ResNet50, VGG19, ResNext, and ViT on ISIC 2017. For that claim to be credible, two conditions must hold: (i) the described architecture actually contains the proposed multi-scale fusion mechanism, and (ii) the Table 1 comparison is produced under a shared, appropriately tuned protocol. The more load-bearing condition is (i), because the paper's stated novelty is the multi-scale attention mechanism; if that mechanism degenerates to standard attention, the reported accuracy has no architectural explanation. Equation (3) makes this precise: every summand is Attention(Q,K,V) with the same Q, K, V, so the sum is a scalar multiple of one attention function. No scale pyramid, patch-size variation, or scale-specific projection appears anywhere in the method, so the method as written is internally inconsistent with the claimed multi-scale improvement. I agree with the reader's rejection but root it primarily in this collapse rather than only in missing training details. A possible objection is that the authors may have implemented a different multi-scale mechanism than the one written; that possibility strengthens the concern, because it means the specification does not match the experiment and no code is available to resolve the ambiguity. The concrete algebraic test above is decisive for the internal consistency issue; if it confirms the collapse, the central claim is unsupported regardless of any undocumented implementation. This is not a novelty dispute but a correctness risk in the stated architecture. The final verdict remains REJECT, matching the reader, and the rationale for rejection is sharpened.","tokens_in":6456,"tokens_out":5961,"duration_ms":53812,"concrete_test":"Analytically verify Eq. (3): substitute the definitions of Q, K, V from Section III into MultiScale and factor out the common Attention term. If MultiScale = (sum_s w_s) * Attention(Q,K,V) with no scale-specific projections, then S>1 changes only a scalar that can be folded into the output projection. This single derivation settles whether the described architecture contains any multi-scale mechanism; if it does not, the reported superiority cannot be attributed to the proposed method as specified.","verdict_should_be":"REJECT","load_bearing_attack":"For the central claim to hold, the improved model must actually contain the described multi-scale feature fusion and the comparison must rest on equal training conditions. The method section fails the first condition. In Section III, self-attention is Attention(Q,K,V)=softmax(QK^T/sqrt(d_k))V, and the multi-scale version is MultiScale(Q,K,V)=sum_{s=1}^S w_s Attention(Q,K,V). No scale-specific transformation is applied to Q, K, V, to the patch embeddings, or to feature maps before each summand; the only learned difference between terms is the scalar w_s. Therefore MultiScale = (sum_s w_s) * Attention(Q,K,V), and the scalar can be absorbed into the subsequent output projection, making the model equivalent to standard ViT attention. The 'multi-scale feature fusion mechanism' asserted as the source of improved global and local feature extraction is thus absent from the architecture as written. The paper also omits the data split, preprocessing, epochs, learning rate, augmentation, and seeds behind Table 1, so even if a real implementation produced those numbers, the claimed gains over ViT and ResNext are not independently reproducible. The central claim therefore rests on a degenerate or unspecified mechanism and an unverifiable comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes an improved Vision Transformer for skin lesion classification on the ISIC 2017 dataset. The claimed novelty is a multi-scale feature fusion mechanism in the self-attention module, an improved weighted cross-entropy loss, and an attention-map regularization loss. The authors report accuracy, AUC, F1-Score, and Precision for ResNet50, VGG19, ResNext, ViT, and their own model, concluding that the proposed model is significantly better than the baselines, and they provide Grad-CAM visualizations as evidence of interpretability.","tokens_in":6726,"tokens_out":4159,"duration_ms":35969,"significance":"If the proposed architecture and empirical results were valid, an interpretable Transformer that outperforms standard baselines on a public skin-lesion benchmark would be a useful contribution to medical image classification. The paper's use of a public dataset and four standard evaluation metrics is a strength, as is the explicit caveat at the end of Section IV-B acknowledging that ISIC 2017 has a limited distribution compared with real clinical settings. However, the central architectural claim is mathematically vacuous as written, and the experimental protocol is so under-specified that the reported superiority is not verifiable. The manuscript contains no code, no data, no error bars, no statistical tests, and no ablation study, so the claimed contribution is not currently established.","major_comments":[{"comment":"Equation (3) defines MultiScale(Q,K,V) as a weighted sum of identical Attention(Q,K,V) terms, with no scale-specific transformation applied to Q, K, V, patch embeddings, or feature maps. Since every summand is the same function, the definition reduces to (sum_s w_s)*Attention(Q,K,V), and the scalar can be absorbed into the output projection. As written, the proposed multi-scale feature fusion mechanism does not exist in the architecture, which directly undermines the central claim that this mechanism drives the reported improvement.","section":"III, Eq. (3)"},{"comment":"Table 1 is presented without any experimental protocol: the data split, preprocessing, image size, patch size, number of epochs, learning rate, optimizer, augmentation schedule, class-balancing strategy, and random seeds are not reported. The differences over the strongest baseline are small (ACC +0.015 vs. ViT, AUC +0.013 vs. ViT), and without error bars or statistical tests the phrase 'significantly better' is unsupported. These omissions make the claimed superiority unverifiable and are load-bearing for the paper's conclusion.","section":"IV-B, Table 1"},{"comment":"The proposed method contains three components (multi-scale attention, weighted cross-entropy, and attention-map regularization), but Table 1 reports only the end-to-end model. There is no ablation or component-wise analysis, so even if the architecture were non-degenerate, the results would not identify which component contributes to the reported performance, and the design rationale in the introduction cannot be tested.","section":"III, Eqs. (4)-(5) and IV-B"},{"comment":"The attention-map regularization loss in Eq. (5) requires a lesion mask M_i for every training sample, but the experimental section does not state whether segmentation masks from ISIC 2017 were used during training. If they were used, the proposed model received an additional supervision signal unavailable to the comparison models, which would confound the comparison; if they were not used, the loss is undefined. This ambiguity directly affects the validity of the experimental comparison.","section":"III, Eq. (5)"},{"comment":"The Grad-CAM visualizations are described as showing strong alignment between the model's focus and actual lesion sites, but no quantitative interpretability metric (e.g., IoU with the available ISIC 2017 segmentation masks) or comparison with baseline models' heatmaps is provided. As presented, the interpretability claim is anecdotal and does not support the conclusion that the model's decisions align with clinical reasoning.","section":"IV-B, Figure 3"}],"minor_comments":[{"comment":"The notation is garbled in several places: 'CWHRX ××∈' is not meaningful, the softmax in the attention formula is rendered as 'max', and the dimension d_k in Eq. (2) is later called h. These should be corrected.","section":"III"},{"comment":"The text refers to 'Viet's 0.880' (presumably ViT) and ResNeXt is inconsistently spelled as ResNext; please standardize the model names throughout.","section":"IV-B"},{"comment":"The Introduction claims the model was demonstrated on 'multiple public skin lesion datasets,' but the experiments use only ISIC 2017; please align the claims with the evidence.","section":"I and IV-B"},{"comment":"The paper's own limitation statement acknowledges the limited distribution of the ISIC 2017 dataset, yet the abstract and conclusion claim generalizable diagnostic support; this tension should be resolved by tempering the claims or adding cross-dataset validation.","section":"IV-B, last paragraph"},{"comment":"Reference [22] shares several authors with the present paper and is cited for VGG19's performance; this self-citation should be disclosed, and it should not be the sole basis for the baseline configuration.","section":"References"},{"comment":"Figure 3 would be much more informative if each panel showed the raw image, the ground-truth mask, and the Grad-CAM overlay for the same example, so that the alignment claim can be visually assessed.","section":"IV-B, Figure 3"}],"recommendation":"reject","confidential_remarks":"The manuscript has a cluster of references from the same author group and many late-2024 arXiv preprints that appear peripheral to the topic; the editorial office may wish to check the citation pattern. The paper's scope is a standard computer-vision application venue, but the empirical support is currently insufficient: the core method equation is vacuous, training details are absent, and no code or data are provided. A substantially rewritten version with a non-degenerate architecture, a full experimental protocol, code, and ablations could be reconsidered as a new submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's central architectural claim doesn't survive a close read of Eq. (3), and the empirical claim has no protocol behind it. I agree with the reader's REJECT, and I'd put it at desk-reject level rather than borderline.\n\nWhat's genuinely fine here: the problem is real, ISIC 2017 is a standard benchmark, and reporting ACC, AUC, F1, and Precision is standard practice. The Grad-CAM analysis is a nice addition. The general idea of adapting ViT with multi-scale features for skin lesions is not crazy.\n\nThe problem is that the multi-scale mechanism as written is a weighted sum of identical Attention(Q,K,V) terms. No scale-specific transformation is applied to Q, K, V, the patch embeddings, or the feature maps before each summand. So the sum collapses to a scalar times the same attention, and that scalar gets absorbed into the output projection. The claimed architectural novelty is absent. The stress-test note is exactly right here.\n\nThe loss formulations are also too vague to implement. The \"improved weighted cross entropy\" has a garbled equation, and the attention regularization loss uses a Frobenius norm over an attention map and a lesion mask without explaining where the mask comes from or how the map is extracted. The experimental section gives no training details: no data split, preprocessing, epochs, learning rate, augmentation, or seeds. No code or data is released. The performance table has no error bars or statistical tests, so \"significantly better\" is unsupported. The references include several arXiv preprints that look tangential, and there is one self-citation, but that is a minor issue compared with the missing methodology.\n\nOne smaller inconsistency: the introduction promises \"multiple public skin lesion datasets\" but only ISIC 2017 is reported. Not a big deal by itself.\n\nOverall, the central argument does not hold up. The described architecture reduces to standard ViT, and the reported gains cannot be attributed to the proposed mechanism. This is a load-bearing flaw, not a missing polish. I would not send this to peer review, and I would not cite it. It could serve as a teaching example of why equations and experimental protocol matter, but that's the only value I see.","headline":"The paper's multi-scale attention collapses to standard attention, and the empirical claims lack any experimental protocol, so the central contribution is unverifiable; desk reject.","tokens_in":7246,"tokens_out":1872,"would_cite":false,"duration_ms":18029,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An improved Vision Transformer outperforms CNN and ViT baselines on skin lesion classification.","keywords":["skin lesion classification","Transformer","multi-scale feature fusion","self-attention","ISIC 2017","Grad-CAM","medical image classification","class imbalance"],"falsifier":"Retrain the five models on the same split of ISIC 2017 with identical preprocessing, augmentation, epoch count, learning-rate schedule, and per-model hyperparameter search; if ViT or ResNext reaches or exceeds ACC 0.895 with comparable AUC under those controls, the architecture's claimed advantage is falsified.","tokens_in":6260,"feed_emoji":"🩺","tokens_out":6609,"duration_ms":57795,"temperature":0.7,"pith_summary":"This paper attempts to establish that a Transformer modified for medical images can classify skin lesions more accurately than convolutional networks and a standard Vision Transformer. The proposed model adds a learned multi-scale fusion to the self-attention computation, a class-weighted cross-entropy loss, and a loss that penalizes attention falling outside the lesion mask. On the ISIC 2017 benchmark it reports accuracy 0.895, AUC 0.938, F1 0.884, and precision 0.910, all above the four comparison models. This matters because skin lesion screening needs both high discrimination on ambiguous boundaries and a way for clinicians to see why the model made its prediction.","feed_headline":"Improved transformer tops skin-lesion baselines on ISIC 2017","feed_subtitle":"The model reports 0.895 accuracy and 0.938 AUC, and its Grad-CAM heat maps stay on the lesion.","key_machinery":"The carrying mechanism is the multi-scale attention block: the input image is cut into fixed patches, linearly projected to dimension $D$, and positionally encoded; the standard attention computation $\\mathrm{Attention}(Q,K,V)=\\mathrm{softmax}(\\frac{QK^T}{\\sqrt{d_k}})V$ is then replaced by a weighted sum over $S$ parallel scale heads, $\\mathrm{MultiScale}(Q,K,V)=\\sum_{s=1}^{S} w_s \\mathrm{Attention}(Q,K,V)$, with $w_s$ learned. Two losses train this block: a weighted cross-entropy loss $L_{CE}=-\\frac{1}{N}\\sum_{i,j} y_{ij}\\log y'_{ij}\\,w_j$ with $w_j=1/(f_j+\\varepsilon)$ for class frequency $f_j$, and an attention regularization loss $L_{attn}=\\frac{1}{N}\\sum_i \\|A_i\\odot M_i\\|_F$ that encourages attention maps to fall inside the lesion mask. The multi-scale weighted sum is what lets the model keep global context while resolving local boundary detail, and the two losses are what adapt the Transformer to imbalanced, mask-annotated medical data.","core_discovery":"On its own terms, the paper's discovery is that a standard Vision Transformer can be made more accurate and more interpretable for skin lesion classification by three coordinated changes: fusing attention heads across scales with learned weights, weighting the cross-entropy loss inversely to class frequency, and adding an attention-map regularization term that uses lesion segmentation masks to force the model to look at the lesion. Trained and evaluated on ISIC 2017, the combined model reaches ACC 0.895, AUC 0.938, F1-Score 0.884, and Precision 0.910, beating ResNet50, VGG19, ResNext, and ViT on every reported metric. The paper further claims that Grad-CAM heat maps concentrate on lesion boundaries and centers, which it reads as evidence that the model's decisions align with clinically relevant regions.","pith_inferences":["A controlled ablation study that removes the multi-scale weights, the class-weighted loss, and the attention-mask loss one at a time would identify which component drives the reported gains; the paper only evaluates the combined architecture.","Because training settings such as epoch count, learning rate, augmentation, and class-balancing are not reported, the margin over ViT could change under a strict same-budget comparison; treating the ranking as provisional until replication is the safest reading.","The attention-mask loss depends on segmentation masks being available during training, so applying the method to datasets without masks would require a mask-free surrogate; whether the improvement survives that change remains an open test."],"forward_implications":["Skin lesion screening could use this model as a drop-in classifier: it reports higher accuracy, AUC, F1, and precision than the four comparison models on ISIC 2017.","The attention-mask loss produces heat maps as a byproduct, so predictions come with a visual localization that clinicians can check against the lesion.","The class-weighted loss is designed for imbalanced medical sets, which should reduce missed rare-lesion cases compared to plain cross-entropy training.","The same multi-scale attention and masked regularization components are general enough that the authors propose carrying them into CT and MRI classification and segmentation."],"supporting_citations":[{"why":"Establishes the skin lesion classification task and the deep learning approach to melanoma detection that this work extends.","marker":"[1]"},{"why":"Supports combining auxiliary tasks and multimodal information to optimize skin lesion classification, related to the paper's multi-loss design.","marker":"[5]"},{"why":"Provides the rationale for interpretability in medical AI, which the paper uses to motivate its Grad-CAM analysis.","marker":"[19]"},{"why":"Cited for the standard self-attention computation on which the proposed multi-scale attention variant builds.","marker":"[20]"},{"why":"Defines the Vision Transformer baseline that the improved model modifies.","marker":"[21]"},{"why":"Defines the VGG19 baseline used in the comparison experiments.","marker":"[22]"},{"why":"Defines the ResNext baseline used in the comparison experiments.","marker":"[23]"}],"fun_headline_variants":["Multi-scale transformer beats CNNs on skin lesion classification","Skin lesion transformer with multi-scale fusion tops baselines","Improved transformer for skin lesions: higher accuracy and focus","Multi-scale attention improves skin lesion classification with sharper maps","Multi-scale transformer tops CNNs and ViT on ISIC 2017"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The strongest claim depends on all models being compared under identical, fairly tuned training conditions; the paper does not document the training settings, so the reported advantage over ViT and ResNext could reflect tuning differences rather than the architecture.","fun_headline_variants_meta":{"raw":{"variants":["Multi-scale transformer beats CNNs on skin lesion classification","Skin lesion transformer with multi-scale fusion tops baselines","Improved transformer for skin lesions: higher accuracy and focus","Multi-scale attention improves skin lesion classification with sharper maps","Multi-scale transformer tops CNNs and ViT on ISIC 2017"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000981,"raw_usage":{"total_tokens":4140,"prompt_tokens":893,"completion_tokens":3247,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":3166}},"tokens_in":509,"tokens_out":3247,"duration_ms":19691,"temperature":1.0,"reasoning_tokens":3166,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T16:15:20.572032+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the five models on the same split of ISIC 2017 with identical preprocessing, augmentation, epoch count, learning-rate schedule, and per-model hyperparameter search; if ViT or ResNext reaches or exceeds ACC 0.895 with comparable AUC under those controls, the architecture's claimed advantage is falsified.","supporting_citations":[{"cited_title":"Skin lesion classification for melanoma using deep learning,","cited_arxiv_id":null,"evidence_quote":"Establishes the skin lesion classification task and the deep learning approach to melanoma detection that this work extends."},{"cited_title":"Optimizing Skin Lesion Classification via Multimodal Data and Auxiliary Task Integration","cited_arxiv_id":"2402.10454","evidence_quote":"Supports combining auxiliary tasks and multimodal information to optimize skin lesion classification, related to the paper's multi-loss design."},{"cited_title":"Survival prediction across diverse cancer types using neural networks","cited_arxiv_id":null,"evidence_quote":"Cited for the standard self-attention computation on which the proposed multi-scale attention variant builds."},{"cited_title":"Transformers in vision: A survey,","cited_arxiv_id":null,"evidence_quote":"Defines the Vision Transformer baseline that the improved model modifies."},{"cited_title":"Deep Learning in Image Classification: Evaluating VGG19's Performance on Complex Visual Data","cited_arxiv_id":"2412.20345","evidence_quote":"Defines the VGG19 baseline used in the comparison experiments."},{"cited_title":"SE -ResNeXt-50-CNN: A Deep Learning Model for Lung Cancer Classification,","cited_arxiv_id":null,"evidence_quote":"Defines the ResNext baseline used in the comparison experiments."}],"review_version":1}