{"id":"018a8377-01f9-4d8d-9325-96a48acc0bac","arxiv_id":"2412.05585","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A UNet++ variant with LSTM-based channel attention and multiscale feature extraction reports 98.88% accuracy and 92.74% Dice on the BUSI breast ultrasound dataset.","lead":"The authors combine UNet++ with LSTM layers, self-attention, and multiscale feature extraction to segment breast ultrasound images, reporting 98.88% accuracy and 92.74% Dice on the BUSI dataset. A general reader might care because it is one more architecture variant in a crowded medical segmentation field, but the evidence is currently hard to verify.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported superiority is not yet supported: a single image-level 80/20 split on BUSI (780 images from 600 patients) risks patient leakage, and Table 3 compares against literature numbers from different datasets and protocols.","rationale":"The Reader's weakest assumption is largely correct: cross-paper comparability is invalid. I agree but would sharpen the primary threat as patient leakage from the image-level split, since it undermines the proposed model's own metrics even before comparison. This is why I mark agreement as partial: the Reader folded the split issue into comparability, whereas I treat it as the first-order validity threat. The remedy — patient-level splitting and reimplemented baselines — is exactly the conditional path the Reader proposes, so the verdict remains CONDITIONAL/UNCHANGED. I considered making the undefined temporal axis of the LSTM the headline concern, but the empirical superiority claim is what the abstract asserts; a missing temporal axis would weaken the architecture's novelty but would not by itself refute the reported metrics. The table/text inconsistencies (e.g., CNN+LR accuracy 96.87 vs 06.87, SaTransformer specificity 89.51 vs 88.32) further reduce confidence but are not the single load-bearing issue.","tokens_in":11311,"tokens_out":6506,"duration_ms":59092,"concrete_test":"Perform a patient-level five-fold cross-validation of the proposed model and of the closest baselines (UNet++, ODET, SaTransformer) reimplemented under identical preprocessing, augmentation, and loss settings, with per-fold metrics reported as mean ± std. If the average Dice/accuracy does not reproduce the reported 98.88/92.74 values, or if the proposed model no longer ranks first, the central claim fails. Release the exact patient-grouped split indices so the leakage question can be audited directly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that the proposed UNet++LSTM model achieves accuracy 98.88%, Dice 92.74%, and outperforms all 15 compared methods. The most load-bearing assumption is that the evaluation protocol produces valid, comparable numbers. That assumption is insecure in two concrete ways. First, the model is evaluated on a single random 80/20 image-level split of BUSI, which contains 780 images from only 600 patients. Without patient-level grouping, images from the same patient can appear in both training and test sets. Because ultrasound images of the same breast are highly correlated, this leakage inflates all segmentation metrics and means the reported numbers may not reflect generalization to new patients. Second, Table 3 lists 15 published results from papers that used different datasets (DCE-MRI, histopathology, thermal images, BUSI subsets), different preprocessing, and different metric definitions, and none of these baselines is reimplemented under the proposed protocol. Thus even if the proposed model's own metrics are reproducible, the 'outperformed all 15' claim is not established. The manuscript also provides no code, no split indices, and no error bars, so the single-run result cannot be audited. The LSTM/temporal motivation is an additional weakness: BUSI is a static-image dataset and the manuscript never defines the temporal axis over which the LSTM is unrolled, but this is secondary to the evaluation-protocol problem.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UNet++LSTM, an encoder-decoder architecture for breast ultrasound image segmentation on the BUSI dataset. The method combines a multiscale feature extraction module, CBAM-style channel and spatial attention in which channel descriptors are processed by an LSTM, and a multi-task loss. The authors report accuracy 98.88%, specificity 99.53%, precision 95.34%, sensitivity 91.20%, F1-score 93.74%, and Dice 92.74% on a single 80/20 split with data augmentation, and they claim that the model outperforms all 15 compared literature methods. The paper includes a comparison table, loss/accuracy curves, sample segmentations, and implementation settings, but provides no code, no split indices, no error bars, and no reimplemented baselines.","tokens_in":11643,"tokens_out":5687,"duration_ms":47267,"significance":"If the reported metrics were robust and the comparison protocol valid, the proposed architecture would be a competitive segmentation baseline on a public dataset. The paper has the strength of using the public BUSI dataset and reporting its implementation settings in Table 2. However, the central empirical claim is currently not well supported: the evaluation uses a single image-level split on a dataset with multiple images per patient, and the 'outperformed all 15' claim rests on literature values from heterogeneous datasets and protocols. The LSTM's 'temporal' motivation is also not operationalized for static ultrasound images. These issues are load-bearing and require substantive revision.","major_comments":[{"comment":"The headline metrics (accuracy 98.88, Dice 92.74) rest on a single random 80/20 image-level split of BUSI, as stated in the Results section. BUSI has 780 images from 600 patients, so an image-level split can place images of the same patient in both training and test sets; because same-breast ultrasound images are highly correlated, this risks patient leakage and inflated segmentation metrics. No patient-level grouping, cross-validation, confidence intervals, or error bars are reported, and no code or split indices are provided. The reported numbers are therefore not yet auditable as estimates of generalization to new patients.","section":"Result (data split and BUSI description)"},{"comment":"The claim that the proposed model 'outperformed all 15' compared methods is not established. Table 3 assembles literature numbers from heterogeneous sources: [31] uses thermal images, [32] uses histopathology mitosis images, [33] uses histopathology images, and [19] uses DCE-MRI; these studies use different preprocessing, splits, and metric definitions, and none are reimplemented under the proposed BUSI 80/20 protocol. In addition, the text says 15 methods but lists 12 names, and Table 3 has 12 comparison rows (not counting U-Net++ and the proposed model). Direct comparability is thus unsupported even if the proposed model's own metrics are accurate.","section":"Table 3 and Results"},{"comment":"The recurrent component is motivated as capturing 'temporal characteristics,' but the input to the LSTM consists of global average-pooled and max-pooled channel descriptors of a single static image (see the equation for Mc(F) and Figure 3). The BUSI dataset is static; no video frames, slices, or sequence dimension are defined. The LSTM is therefore unrolled over a constructed channel sequence, not over time. The paper should either define the temporal axis explicitly or reframe the contribution as a channel-attention variant, because the current text under-specifies the core architectural novelty.","section":"Attention Block and Eq. (Mc(F))"},{"comment":"Multiple numbers in the text do not match Table 3, which makes the results section unreliable. Examples: the text gives CNN+LR accuracy as 06.87 while Table 3 shows 96.87; Faster R-CNN+CNNs precision is quoted as 97.60 in the text but 87.60 in Table 3; SaTransformer is credited with Specificity=89.51, Precision=89.51, and Sensitivity=89.51, while Table 3 lists Specificity=88.32 and no Sensitivity; GA-ANNs sensitivity is 90.00 in the text and 96.87 in Table 3; and the sentence 'proposed Unet++ approach ... accuracy of 92.70' contradicts the Unet++ row value of 98.58. The reported improvement over U-Net++ is also given as 0.0204 in Dice and 0.0304 in F1, whereas the table difference is about 2.04 and 3.04 percentage points. These inconsistencies should be corrected and a single audit trail provided.","section":"Results prose vs Table 3"}],"minor_comments":[{"comment":"The conditional definition for Xi,j with j > 0 is duplicated in the text, and the sentence 'In cases and j denotes...' is incomplete before the formula. The notation for encoder/decoder nodes should be cleaned up.","section":"Method (Xi,j equations)"},{"comment":"Figure 2 and Figure 3 have the same caption, 'The proposed channel attention mechanism,' even though Figure 3 appears to illustrate the LSTM cell. The captions should be corrected to match the content.","section":"Figure captions 2 and 3"},{"comment":"U-Net++ is cited as [19] in Table 1 and Related Works but as [26] in the Multiscale Feature Extraction Module; [26] actually refers to ResUNet++. The reference numbering should be made consistent.","section":"References"},{"comment":"The phrase 'Using tabularx and booktabs' appears in the Results section and is a leftover LaTeX instruction that should be removed.","section":"Results"},{"comment":"There is a typo 'in the field of Business (BUSI)', and the paragraph about Dataset B and the STU dataset does not define these datasets; also the AAU-net comparison is confusingly worded and should be rewritten.","section":"Related Works"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads more like an extended technical report than a polished journal paper. The central empirical claim is not yet supported because of the evaluation protocol and the literature-number comparison. Before further consideration, the authors should be asked to provide the exact data split (ideally patient-level), code or detailed training/evaluation scripts, error bars or multiple runs, and either reimplemented baselines under the same protocol or a reframed claim that does not assert superiority over all 15 methods."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper adds an LSTM-based channel attention path and a multiscale module to UNet++ and tests on BUSI. That is a legitimate incremental combination, and the paper is honest about its intent. But the headline claim—outperforming all 15 compared methods—rests on a comparison that is not yet valid, so my verdict is a qualified one.\n\nWhat is genuinely new: the specific placement of LSTM cells inside the channel attention block (on top of CBAM-style pooling), plus the multiscale feature extraction module feeding UNet++'s nested skip paths. Both components are established, but the particular combination applied to breast ultrasound segmentation does not appear in the cited literature. The motivation—UNet++ concatenates feature maps of different depths without weighting their channel importance—is a real one, and the proposed loss (segmentation plus distance-class map) is a reasonable gesture toward boundary handling.\n\nWhere it falls down: the evaluation. The model is run on a single 80/20 image-level split of BUSI, which has 780 images from 600 patients. Without patient-level grouping, images from the same breast can appear in both training and test. Ultrasound frames of the same lesion are highly correlated, so this can inflate every reported metric. Second, Table 3 compares against numbers taken from the literature—different datasets (DCE-MRI, thermal, histopathology, BUSI subsets), different preprocessing, different metric definitions, no reimplementation. So 'outperformed all 15' is not actually supported; the proposed model's own metrics may be internally correct, but the comparison is not apples-to-apples. Third, there are internal inconsistencies: the prose and Table 3 disagree on several values (e.g., GA-ANNs sensitivity appears as 90.00 in text and 96.87 in the table; SaTransformer's specificity/precision numbers don't line up; CNN+LR has a '06.87' accuracy typo and a Dice of 96.87 that matches its accuracy). No code, split indices, or error bars are provided, so a single run cannot be audited.\n\nThe LSTM motivation is also shaky. BUSI is a static-image dataset; the paper never defines the temporal axis over which the LSTM is unrolled. This is a conceptual weakness, but secondary to the evaluation problem.\n\nBottom line: the architecture idea is worth keeping, but the paper needs major revision before its claims can be accepted: patient-level split, reimplemented baselines under the same protocol, error bars, code/split release, and cleaned-up tables. I'd send it to peer review as a borderline incremental contribution, with a strong request for those fixes. It is not a desk-reject on novelty grounds, but it is not publishable as-is.","headline":"A plausible architecture combo whose reported superiority over 15 baselines is not supported by the evaluation as presented; worth a revision, not rejection on idea alone.","tokens_in":12127,"tokens_out":2560,"would_cite":false,"duration_ms":23432,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A hybrid of UNet++ and LSTM with attention and multiscale feature modules reports 98.88% accuracy in segmenting breast ultrasound lesions on the BUSI dataset.","keywords":["breast ultrasound segmentation","UNet++","LSTM","self-attention","multi-scale feature extraction","BUSI dataset","medical image segmentation","data augmentation"],"falsifier":"Run the proposed model and every comparison method on the same patient-stratified split of BUSI with identical preprocessing, augmentation, and metric computation; if any baseline reaches or exceeds a Dice of 92.74 on that shared protocol, the paper's headline claim of superiority fails.","tokens_in":11125,"feed_emoji":"🩻","tokens_out":6922,"duration_ms":59473,"temperature":0.7,"pith_summary":"This paper proposes a breast ultrasound segmentation model that grafts LSTM-based attention and a multiscale feature extraction module onto the UNet++ encoder-decoder. The authors claim that by adding these components they can capture both spatial structure and temporal (recurrent) feature dependencies, which plain UNet++ ignores. With data augmentation on the public BUSI breast ultrasound dataset, the model reports accuracy 98.88%, specificity 99.53%, precision 95.34%, sensitivity 91.20%, F1-score 93.74%, and Dice 92.74%. The paper presents these numbers as surpassing all fifteen prior methods it compares against. A sympathetic reader would care because accurate tumor boundary segmentation on routine ultrasound is a practical step toward earlier breast cancer detection.","feed_headline":"UNet++-LSTM hybrid segments breast tumors at 98.88% accuracy","feed_subtitle":"Attention and multiscale modules push Dice to 92.74% on the public BUSI ultrasound dataset.","key_machinery":"The architecture, named UNet++LSTM, keeps UNet++'s nested encoder-decoder with dense convolution blocks on hopping paths and adds three components: a multiscale feature extraction module that produces feature maps at different scales; a Convolutional Block Attention Module (CBAM) whose channel branch routes average- and max-pooled descriptors through LSTM gates before combining them with element-wise summation; and a multi-task learning head that adds a distance-class map prediction term to the segmentation loss. The LSTM inside the channel attention is the load-bearing novelty: the paper describes it as temporal feature learning, and the attention weights determine how much each hierarchical feature map contributes to the final segmentation. The multiscale module and attention together are what the argument credits for the reported gains over plain UNet++.","core_discovery":"The central claim is that replacing plain UNet++ with a version enriched by an attention block, a multiscale feature extraction module, and LSTM layers yields a segmentation pipeline that outperforms plain UNet++ and the other published models compared in the paper on breast ultrasound images. On the BUSI dataset with data augmentation, the proposed UNet++LSTM model reaches accuracy 98.88%, specificity 99.53%, precision 95.34%, sensitivity 91.20%, F1-score 93.74%, and Dice 92.74%, improving over UNet++ with data augmentation by raising Dice from 90.70 to 92.74 and F1 from 90.70 to 93.74. The paper argues these gains show the architecture is a competitive, end-to-end solution for extracting benign and malignant tumor regions.","pith_inferences":["A testable extension the paper does not run: if LSTM recurrence is the active ingredient, evaluating on consecutive ultrasound cine frames instead of single images should widen the margin over plain UNet++.","The comparison against literature values assumes protocol equivalence; a single-protocol reimplementation of the same baselines on the same BUSI split could yield a different ranking.","The same channel-attention-with-LSTM design could plausibly transfer to other boundary-delineation tasks such as lesion segmentation in CT or MRI, though the paper only reports ultrasound results."],"forward_implications":["Adding LSTM-based attention and multiscale feature extraction to UNet++ increases Dice from 90.70 to 92.74 and F1 from 90.70 to 93.74 on the BUSI dataset with data augmentation.","The paper reports a 1.25 improvement in precision over the best comparative approach it lists.","The proposed model reports specificity 99.53%, surpassing the ODET model's 98.63% in the comparison table.","According to the loss and accuracy curves, both UNet++ and the proposed model train without overfitting or underfitting, which the paper offers as evidence of stable learning.","The architecture is end-to-end and trainable with standard data augmentation on the BUSI dataset, requiring no additional imaging modality."],"supporting_citations":[{"why":"Supplies the BUSI dataset with ground-truth masks used for training and evaluation.","marker":"[18]"},{"why":"UNet++ is the base architecture whose nested skip connections and dense feature maps the proposed model extends.","marker":"[19]"},{"why":"DGANet is an attention-based breast lesion detection method whose reported performance provides comparison context.","marker":"[13]"},{"why":"AMS-PAN is a multi-scale attention segmentation method whose Dice and IoU figures anchor the comparison.","marker":"[14]"},{"why":"CSwin-PNet is a transformer-pyramid baseline reported to improve on UNet-family models, used as context for the proposed approach.","marker":"[16]"},{"why":"ODET is one of the strongest competitors in Table 3, and the paper claims the proposed model surpasses it on precision and specificity.","marker":"[34]"},{"why":"SaTransformer is a semantic-aware transformer baseline whose Table 3 metrics are used in the comparison.","marker":"[39]"}],"fun_headline_variants":["Attention + LSTM boost UNet++ for breast ultrasound segmentation","UNet++ with LSTM hits 98.88% on breast tumor segmentation","Temporal-aware UNet++ tops breast ultrasound Dice at 92.74%","Breast tumor segmentation: UNet++ + LSTM + attention = 98.88%","LSTM-enhanced UNet++ achieves 98.88% accuracy on BUSI"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison claim assumes the published metrics of the fifteen prior methods are directly comparable to the proposed model's single 80/20 split of the BUSI dataset, even though the cited papers often use different datasets, preprocessing, splits, and metric definitions.","fun_headline_variants_meta":{"raw":{"variants":["Attention + LSTM boost UNet++ for breast ultrasound segmentation","UNet++ with LSTM hits 98.88% on breast tumor segmentation","Temporal-aware UNet++ tops breast ultrasound Dice at 92.74%","Breast tumor segmentation: UNet++ + LSTM + attention = 98.88%","LSTM-enhanced UNet++ achieves 98.88% accuracy on BUSI"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000832,"raw_usage":{"total_tokens":3613,"prompt_tokens":909,"completion_tokens":2704,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":2599}},"tokens_in":525,"tokens_out":2704,"duration_ms":16984,"temperature":1.0,"reasoning_tokens":2599,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:34:18.722237+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the proposed model and every comparison method on the same patient-stratified split of BUSI with identical preprocessing, augmentation, and metric computation; if any baseline reaches or exceeds a Dice of 92.74 on that shared protocol, the paper's headline claim of superiority fails.","supporting_citations":[{"cited_title":"Data in brief 28, 104863 (2020)","cited_arxiv_id":null,"evidence_quote":"Supplies the BUSI dataset with ground-truth masks used for training and evaluation."},{"cited_title":"In: Journal of Physics: Conference Series, vol","cited_arxiv_id":null,"evidence_quote":"UNet++ is the base architecture whose nested skip connections and dense feature maps the proposed model extends."},{"cited_title":"Ultrasound in Medicine & Biolog y 49(1), 31–44 (2023)","cited_arxiv_id":null,"evidence_quote":"DGANet is an attention-based breast lesion detection method whose reported performance provides comparison context."},{"cited_title":"Biomedical Signal Processing and Control 81, 104425 (2023)","cited_arxiv_id":null,"evidence_quote":"AMS-PAN is a multi-scale attention segmentation method whose Dice and IoU figures anchor the comparison."},{"cited_title":"Expert Systems with Applications 213, 119024 (2023)","cited_arxiv_id":null,"evidence_quote":"CSwin-PNet is a transformer-pyramid baseline reported to improve on UNet-family models, used as context for the proposed approach."},{"cited_title":"EAI Endorsed Transactions on Scalable Information Systems 10(2), 4–4 (2023)","cited_arxiv_id":null,"evidence_quote":"ODET is one of the strongest competitors in Table 3, and the paper claims the proposed model surpasses it on precision and specificity."},{"cited_title":"IET Image Processing (2023)","cited_arxiv_id":null,"evidence_quote":"SaTransformer is a semantic-aware transformer baseline whose Table 3 metrics are used in the comparison."}],"review_version":1}