{"id":"91ddda82-16b7-4a1f-8613-6bb058113e58","arxiv_id":"2411.17556","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"TAFM-Net, a U-Net variant with transformer attention and focal modulation in skip connections, reports state-of-the-art skin lesion segmentation on ISIC benchmarks.","lead":"A deep-learning model called TAFM-Net combines transformer attention and focal modulation inside a U-Net to segment skin lesions from dermoscopic images. The authors report Jaccard scores of 86.88 to 93.64 percent on three public ISIC benchmarks, but the paper's reported numbers are internally inconsistent and no code is released.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim rests on cross-paper scores compiled under uncontrolled conditions, and Table 5 itself contains inconsistent TAFM-Net results for ISIC2018 (J=93.08, D=96.85 vs 92.88/96.53 in Table 4 and the abstract), so the reported superiority margin is not yet established.","rationale":"The reader's weakest_assumption correctly identifies the uncontrolled cross-paper comparison as the main threat to the SOTA claim. I agree with that concern, and the internal inconsistency in Table 5 makes it more severe: the paper's own reported TAFM-Net numbers for ISIC2018 differ between Table 4 and Table 5, so the exact claimed performance is not even self-consistent. This is a factual inconsistency, not a disagreement with scientific consensus, and it directly affects the headline assertion of consistent superiority. I credit the ablation experiments and the computational comparison as useful supporting evidence, but they do not establish the central SOTA claim if the comparison table cannot be trusted. The appropriate verdict remains CONDITIONAL, which is what the reader recommended; no verdict change is needed, but the conditions should explicitly require a controlled re-run or a corrected and reconciled comparison table.","tokens_in":16037,"tokens_out":3145,"duration_ms":28508,"concrete_test":"Ask the authors to release trained models and inference code, then re-run the full Table 5 comparison under an identical protocol: the same ISIC2016/2017/2018 training splits with a fixed validation scheme, the same 256x256 input size, the same augmentation, and the same evaluation function, applied to TAFM-Net and each listed baseline using official implementations where available. The SOTA claim should be adjudicated by recomputing Jaccard margins on identical test images; the authors should also explain the discrepancy between the ISIC2018 values 93.08 and 92.88 and report which number is correct.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is Section 4.5.1's assertion that TAFM-Net 'consistently outperformed existing methods by a considerable margin.' That claim is load-bearing and it depends on the comparability of Table 5, but Table 5 is not a controlled experiment. The paper itself states in Section 4.5 that scores for comparison methods 'were taken from the original articles, inasmuch as reported,' meaning the compared numbers were produced under different training/test splits, preprocessing, post-processing, and evaluation protocols. Moreover, Table 5 is internally inconsistent: the TAFM-Net ISIC2018 row reports Jaccard 93.08, Dice 96.85, and Accuracy 99.08, whereas Table 4 and the abstract report Jaccard 92.88, Dice 96.53, and Accuracy 97.87 for the same setting. At least one set of published numbers is wrong or comes from a different protocol, so the claimed superiority margins over state-of-the-art are not currently reproducible. Until the comparison is re-run under identical conditions or the numbers are reconciled, the strength of the conclusion exceeds the evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes TAFM-Net, a U-Net-style architecture for skin lesion segmentation. The encoder is EfficientNetV2B1; a transformer self-attention block and a global spatial attention block are inserted at the bottleneck; focal modulation blocks are placed in the skip connections; and the decoder uses seven upsampling blocks with dense connections. Training uses a dynamically weighted fusion of BCE, Jaccard, focal Tversky, Dice, and boundary losses. The paper reports Jaccard scores of 93.64% (ISIC2016), 86.88% (ISIC2017), 92.88% (ISIC2018), and 95.60% (PH2 cross-dataset), with 20.6M parameters and 18.25 ms inference time, and claims consistent state-of-the-art superiority.","tokens_in":16294,"tokens_out":7625,"duration_ms":62122,"significance":"The architecture is a reasonable design study: combining transformer attention, focal modulation, and a boundary-aware dynamic loss is directionally interesting, and the ablation over loss functions and network components is a strength. If the numerical results and comparisons are verified, a 20.6M-parameter model with the reported accuracy could be practically useful in clinical workflows. The lightweight claims in Table 7 are also attractive. However, the central claim of state-of-the-art performance currently rests on cross-paper comparisons and internally inconsistent numbers, so the significance cannot be fully assessed until those issues are resolved.","major_comments":[{"comment":"The TAFM-Net row in Table 5 for ISIC2018 reports accuracy 99.08, sensitivity 96.90, specificity 98.19, Jaccard 93.08, and Dice 96.85, while Table 4 for the same training/testing setting and the abstract report accuracy 97.87, sensitivity 96.39, specificity 97.92, Jaccard 92.88, and Dice 96.53. These cannot both describe the same experiment. The discrepancy must be resolved and a single protocol stated for the numbers used in the comparison table.","section":"Section 4.5, Table 5"},{"comment":"The claimed improvement ranges are not supported by the tables. For ISIC2018 the text reports Jaccard improvements of 9.3%–19.7%, but Table 5 gives differences ranging from 8.53 percentage points (vs. ARU-GD) to 13.20 (vs. CPFNet). For ISIC2017 the text claims 3.8%–17.2%, while Table 5 gives 3.18 (vs. Hyper-Fusion Net) to 11.19 (vs. U-Net). For ISIC2016 the text claims 6.2%–15.1%, while Table 5 gives 5.47 (vs. Hyper-Fusion Net) to 12.26 (vs. U-Net). Likewise, the PH2 claim of 9.1%–13.8% in Section 4.5.2 exceeds the 8.00–11.61 range in Table 6. The text and tables must be reconciled.","section":"Section 4.5.1 and Section 4.5.2"},{"comment":"The comparison with state-of-the-art methods is not a controlled experiment. The paper states that scores for comparison methods were taken from the original articles, meaning they were produced under different training/validation/test splits, preprocessing, post-processing, and evaluation protocols. The conclusion in Section 4.5.1 that TAFM-Net 'consistently outperformed existing methods by a considerable margin' is therefore not established by Table 5. The authors should re-run at least all publicly available methods under an identical protocol with the same evaluation code, or, failing that, explicitly present the comparison as indicative and temper the superiority claims.","section":"Section 4.5"},{"comment":"The early-stopping protocol is underspecified. Table 1 lists no validation set for ISIC2016 and ISIC2017, yet Section 4.3 describes early stopping monitored from epoch 10 with patience 9. The monitored data split is not identified. If the test set was used for early stopping, the reported scores are optimistic; if a validation split was carved from the training set, it should be described. Additionally, Tables 2–4 report 3-fold cross-validation means, but Table 1 gives fixed train/test partitions; please clarify how the folds relate to the fixed test set.","section":"Table 1 and Section 4.3"}],"minor_comments":[{"comment":"The caption names the first dataset as ISIC 2018 twice; it should read ISIC 2016, ISIC 2017, and ISIC 2018.","section":"Table 5 caption"},{"comment":"With γ=1, Eq. (14) is identical to Eq. (13), so describing the used loss as 'focal Tversky' is misleading; either select γ>1 or call it the Tversky loss in the reported experiments.","section":"Section 3.5.4"},{"comment":"The dynamic loss schedule is not fully reproducible because the total number of training epochs is not specified, and the batch size and the exact point at which α reaches its lower bound are not given.","section":"Section 3.5.6 and Section 4.3"},{"comment":"No code or trained model weights are provided. Given the paper's stated ambition to serve as a baseline, releasing an implementation would substantially improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The core architectural idea is not deeply flawed, but the evidence as presented does not support the headline SOTA claim. The fix is within scope: run controlled comparisons for at least all available methods, reconcile the internal numeric inconsistencies, and revise the claim language accordingly. I would not reject outright, but I would not accept without these changes."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The architecture is a reasonable engineering contribution: EfficientNetV2B1 encoder, transformer self-attention at the bottleneck, focal modulation in skip connections, a densely connected decoder, and a dynamic fused loss. The ablation study is genuinely useful—it shows FM beats CBAM and BiConvLSTM on their setup, and the loss-schedule comparison is clearly described. The computational cost table and cross-dataset generalization experiments are nice extras. The writing is clear and the method is easy to reproduce from the text, modulo missing details.\n\nThe soft spots are real and they hit the central claim. First, the paper's own numbers are inconsistent: ISIC2018 Jaccard is 92.88 in the abstract and Table 4, but 93.08 in Table 5; accuracy jumps from 97.87 to 99.08. At least one of those is wrong. Table 5's header also says \"ISIC 2018, ISIC 2017, and ISIC 2018\" instead of 2016/2017/2018. Second, the improvement ranges in the text don't match the table—the claimed 6.2–15.1% on ISIC2016 exceeds the largest gap of 12.69 points. Third, the SOTA comparison is not controlled: scores for most methods are taken from other papers, so the \"considerable margin\" rests on apples-to-oranges comparisons. No code is released, and several hyperparameters (batch size, training epochs, augmentation, transformer specifics) are missing. These are all fixable, but they mean the current evidence does not establish state-of-the-art performance.\n\nThe heavy self-citation in the references is noticeable but not disqualifying; the cited works are relevant background.\n\nVerdict: the method is worth taking seriously, but the evaluation needs to be redone with controlled comparisons and reconciled numbers. I'd send it to peer review—an expert referee could push the authors to fix this and possibly add a validation on a held-out set. Right now I wouldn't cite the results, but I'd follow the work if the corrections come through.","headline":"A plausible architecture-level contribution whose reported superiority is undercut by uncontrolled comparisons and internally inconsistent numbers; fix the evaluation before believing the margins.","tokens_in":681,"tokens_out":1080,"would_cite":false,"duration_ms":24133,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TAFM-Net combines transformer self-attention and focal modulation in a U-Net to reach state-of-the-art skin lesion segmentation scores.","keywords":["skin lesion segmentation","dermoscopic image analysis","transformer self-attention","focal modulation","U-Net","ISIC benchmark","boundary loss","medical image segmentation"],"falsifier":"Run TAFM-Net and all publicly available comparison methods on the same ISIC 2016, 2017, and 2018 train/test splits with identical preprocessing and evaluation protocols, and check whether TAFM-Net still leads on the majority of metrics.","tokens_in":15783,"feed_emoji":"🩺","tokens_out":4663,"duration_ms":39835,"temperature":0.7,"pith_summary":"This paper claims that adding a transformer self-attention module at the encoder-decoder bottleneck and focal modulation blocks inside the skip connections can push skin lesion segmentation accuracy well beyond previously published results. On the ISIC 2016, 2017, and 2018 benchmarks, the proposed TAFM-Net reports Jaccard scores of 93.64%, 86.88%, and 92.88%, respectively, surpassing earlier attention- and transformer-based networks by large margins. The same design is also smaller and faster than the comparison networks, with 20.6 million parameters and an 18.25 ms inference time per image. If these numbers hold under controlled comparison, the network would be a practical candidate for clinical computer-assisted diagnosis.","feed_headline":"Hybrid U-Net hits 93.6% Jaccard, best on skin lesion benchmarks","feed_subtitle":"Transformer attention plus focal modulation beats earlier networks on ISIC and PH2 while using fewer parameters.","key_machinery":"The load-bearing component is the self-aware attention module placed at the encoder-decoder bottleneck. It concatenates three streams: transformer self-attention output, global spatial attention output, and the original encoder feature map, so that both channel-level and position-level dependencies are preserved. Around this, focal modulation blocks in the skip connections compute a global feature map using depthwise separable convolution and use it to modulate the local convolution output, effectively injecting global context into local feature extraction. The third piece of the machinery is the dynamic fused loss, which combines binary cross-entropy, Jaccard loss, and boundary loss, with the boundary term's weight increasing as training progresses.","core_discovery":"The central claim is that TAFM-Net, built from an EfficientNetV2B1 encoder, a self-aware attention module at the bottleneck, focal modulation in every skip connection, a densely connected decoder, and a dynamically weighted fused loss, consistently outperforms existing state-of-the-art methods on all three ISIC datasets and on PH2 after training on ISIC 2016. The reported improvements in Jaccard score are 6.2%–15.1% on ISIC 2016, 3.8%–17.2% on ISIC 2017, and 9.3%–19.7% on ISIC 2018 over the compared methods. The authors attribute the gains to the combination of global contextual reasoning from transformer attention, fine-grained feature emphasis from focal modulation, and the boundary-aware dynamic loss, and they present ablation experiments and Grad-CAM visualizations to support these attributions.","pith_inferences":["The reported superiority margins may compress in a controlled re-run because many competitor scores were copied from their original papers, which likely used different training sets, preprocessing, and evaluation protocols; a head-to-head benchmark with identical splits and preprocessing is the natural next test.","Since focal modulation is a generic block, the same encoder-decoder recipe could transfer to other medical segmentation problems, such as retinal vessel or organ segmentation, without architectural changes.","The linear decay of the loss weight is a simple schedule; adaptive weighting based on validation boundary performance might yield further gains, especially when training epochs are limited."],"forward_implications":["Reported Jaccard scores of 93.64%, 86.88%, and 92.88% on ISIC 2016, 2017, and 2018 would make TAFM-Net the new reference point for skin lesion segmentation on these benchmarks.","With 20.6 million parameters and 18.25 ms inference per 256x256 image, the network is small and fast enough for clinical decision-support deployment.","The ablation results indicate that the transformer at the bottleneck and focal modulation in skip connections, rather than the backbone alone, drive most of the accuracy gain.","The dynamically weighted boundary loss improves segmentation in low-contrast and hair-occluded images, which are the hard cases that matter in practice.","Cross-dataset training from ISIC 2016 to PH2 shows the method generalizes to a new distribution, supporting its use beyond the training benchmark."],"supporting_citations":[{"why":"Supplies the EfficientNetV2B1 encoder backbone used in TAFM-Net.","marker":"[49]"},{"why":"Provides the self-aware attention module design with transformer self-attention and global spatial attention blocks.","marker":"[43]"},{"why":"Basis of the focal modulation blocks integrated into the skip connections.","marker":"[55]"},{"why":"Prior application of focal modulation to medical image segmentation that motivates its use here.","marker":"[50]"},{"why":"Supplies the focal Tversky loss used in the loss-function ablation experiments.","marker":"[51]"},{"why":"Supplies the boundary loss incorporated into the dynamic fused loss.","marker":"[52]"},{"why":"The U-Net architecture that TAFM-Net extends and a baseline for comparison.","marker":"[24]"},{"why":"Swin-Unet, a transformer-based U-Net used as a key comparison on all three ISIC datasets.","marker":"[41]"}],"fun_headline_variants":["Transformer attention + focal modulation: skin lesion U-Net hits 93.6% Jaccard","Hybrid U-Net with transformer attention and focal modulation tops ISIC benchmarks","EfficientNetV2B1 encoder plus focal modulation: 93.6% Jaccard on ISIC2016","TAFM-Net: Focal modulation in skip connections improves skin lesion segmentation","Skin lesion segmentation: TAFM-Net with dynamic loss reaches 93.6% Jaccard"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim of state-of-the-art performance rests on comparing TAFM-Net's own numbers with scores copied from other papers that likely used different training sets, preprocessing, and evaluation rules, and this comparability is not established.","fun_headline_variants_meta":{"raw":{"variants":["Transformer attention + focal modulation: skin lesion U-Net hits 93.6% Jaccard","Hybrid U-Net with transformer attention and focal modulation tops ISIC benchmarks","EfficientNetV2B1 encoder plus focal modulation: 93.6% Jaccard on ISIC2016","TAFM-Net: Focal modulation in skip connections improves skin lesion segmentation","Skin lesion segmentation: TAFM-Net with dynamic loss reaches 93.6% Jaccard"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000722,"raw_usage":{"total_tokens":3233,"prompt_tokens":934,"completion_tokens":2299,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":2177}},"tokens_in":550,"tokens_out":2299,"duration_ms":15571,"temperature":1.0,"reasoning_tokens":2177,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:58:36.598338+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run TAFM-Net and all publicly available comparison methods on the same ISIC 2016, 2017, and 2018 train/test splits with identical preprocessing and evaluation protocols, and check whether TAFM-Net still leads on the majority of metrics.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the self-aware attention module design with transformer self-attention and global spatial attention blocks."},{"cited_title":"Kervadec, J","cited_arxiv_id":null,"evidence_quote":"Supplies the boundary loss incorporated into the dynamic fused loss."},{"cited_title":"Ronneberger, P","cited_arxiv_id":null,"evidence_quote":"The U-Net architecture that TAFM-Net extends and a baseline for comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Swin-Unet, a transformer-based U-Net used as a key comparison on all three ISIC datasets."}],"review_version":1}