{"id":"71f19ff8-b5c9-4dd3-95a9-bfb184f0bd0c","arxiv_id":"1909.00166","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"BCDU-Net, a U-Net variant with bi-directional ConvLSTM skip connections and dense convolutions, is reported to improve segmentation on retinal vessels, skin lesions, and lung nodules.","lead":"This paper proposes an upgraded U-Net architecture for medical image segmentation that uses bi-directional ConvLSTM in the skip connections and dense convolutions in the deepest layer. It reports improved accuracy on three medical imaging benchmarks, though the reported metric values are internally inconsistent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported metrics are internally impossible: F1 < Jaccard in Tables 2 and 3, so the state-of-the-art claim rests on an unreliable evaluation pipeline.","rationale":"I read the paper as an architecture proposal whose central claim is empirical state-of-the-art performance on three medical segmentation benchmarks. The architectural components are plausible, and the paper provides a source-code link, which makes the evaluation checkable. However, the empirical support is internally inconsistent: the reported F1 and Jaccard values in Tables 2 and 3 violate the mathematical relation F1 >= Jaccard for any single binary prediction set. I verified this directly from the definitions, and I also noticed that the JS column exactly equals the Accuracy column for the BCDU-Net rows in both tables, which strongly suggests a metric-labeling or evaluation-script error rather than a subtle definitional difference. The DRIVE F1 values also disagree between Table 1 and Table 4 for the same d=3 model. Any one of these inconsistencies would undermine the state-of-the-art claim; together they make the quantitative results unreliable. The reader's REJECT verdict is therefore appropriate, and my stress-test does not move that verdict. I agree with the reader's concern about uncontrolled cross-paper comparisons, but the more decisive problem is the internal metric contradiction, so my agreement with the reader's stated weakest assumption is only partial.","tokens_in":11485,"tokens_out":5879,"duration_ms":47609,"concrete_test":"Clone the linked repository (github.com/rezazad68/BCDU-Net), run the released evaluation code on the ISIC 2018 test split, and recompute all metrics from a single confusion matrix after binarizing predictions at the same threshold used in the paper. Verify whether the reproduced BCDU-Net (d=1) row satisfies F1 = 0.847, PC = 0.922, SE = 0.783; if so, Jaccard must equal 0.734, not 0.936. Then inspect the evaluation script to see whether the variable reported as JS is actually assigned the accuracy value, which would explain the exact JS = AC matches in Tables 2 and 3.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing condition for the paper's central claim is that the reported F1, Jaccard, accuracy, precision, and sensitivity values in Tables 2 and 3 are computed correctly from the same test predictions. That condition fails. For any binary segmentation result, F1 = 2TP/(2TP+FP+FN) >= JS = TP/(TP+FP+FN), with equality only when FP+FN = 0 or TP = 0. Table 2 reports BCDU-Net (d=1) F1 = 0.847 and JS = 0.936, and BCDU-Net (d=3) F1 = 0.851 and JS = 0.937; both violate this inequality. Table 3 reports the same violation for U-Net, RU-Net, R2U-Net, and both BCDU-Net rows (e.g., BCDU-Net d=3: F1 = 0.9904, JS = 0.9972). In addition, the Jaccard column in Tables 2 and 3 is exactly equal to the Accuracy column for the BCDU-Net rows (Table 2: 0.936 and 0.937; Table 3: 0.9967 and 0.9972), indicating that the reported JS is very likely a copy of accuracy or the product of a broken evaluation script. Table 4 also disagrees with Table 1 for DRIVE BCDU-Net d=3 F1 (0.8243 vs 0.8224). With the quantitative evidence internally inconsistent, the state-of-the-art comparison cannot support the abstract's claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes BCDU-Net, an extension of U-Net for medical image segmentation. The architecture inserts bi-directional ConvLSTM (BConvLSTM) layers into the skip connections and adds densely connected convolutional blocks in the deepest encoding layer, together with batch normalization after up-convolutions. The authors evaluate the method on three public benchmarks: DRIVE retinal vessel segmentation, ISIC 2018 skin lesion segmentation, and a Kaggle lung segmentation dataset, reporting F1-score, sensitivity, specificity, accuracy, Jaccard similarity, and AUC. The central claim is that BCDU-Net achieves state-of-the-art performance on all three datasets. The paper also includes an ablation study (Table 4) and a convergence-speed comparison with and without batch normalization.","tokens_in":11808,"tokens_out":3056,"duration_ms":30825,"significance":"If the reported improvements were valid, the architectural contribution would be of moderate interest to the medical image segmentation community: replacing concatenation in skip connections with BConvLSTM fusion and adding dense blocks are both plausible mechanisms for improving feature reuse, and the paper provides a public implementation link. However, the significance of the paper as submitted is undermined by internally inconsistent quantitative results. The reported F1 (Dice) values are numerically smaller than the reported Jaccard (IoU) values for the same predictions, which is mathematically impossible, and the Jaccard column appears to be a copy of the accuracy column for the proposed method. Since the headline contribution is an empirical state-of-the-art claim, these inconsistencies invalidate the central conclusion. The manuscript also leaves the lung test-time protocol underspecified because Algorithm 1 uses the ground-truth mask to construct the training surrounding mask, and the test-time analogue is not described.","major_comments":[{"comment":"The reported metrics violate a mathematical identity. For binary segmentation, F1 (Dice) is always greater than or equal to Jaccard (IoU) for the same set of predictions, with equality only in degenerate cases. Table 2 reports BCDU-Net (d=1) F1=0.847 and JS=0.936, and BCDU-Net (d=3) F1=0.851 and JS=0.937; both violate the inequality. Table 3 reports the same violation, for example BCDU-Net (d=3) F1=0.9904 and JS=0.9972. This indicates that the evaluation script or the reported numbers are unreliable, and it invalidates the state-of-the-art comparison that is the paper's central claim.","section":"§4.2, Table 2 and §4.3, Table 3"},{"comment":"The Jaccard column is identical to the Accuracy column for all BCDU-Net rows: Table 2 shows JS=0.936 and AC=0.936 for d=1 and JS=0.937 and AC=0.937 for d=3; Table 3 shows JS=0.9967 and AC=0.9967 for d=1 and JS=0.9972 and AC=0.9972 for d=3. Since Jaccard and accuracy are different quantities, this suggests the JS column was populated with accuracy values or the evaluation code computed the wrong metric. The authors should recompute all metrics from the raw test predictions and report the corrected tables before any performance claim can be assessed.","section":"§4.2, Table 2 and §4.3, Table 3"},{"comment":"The DRIVE results are inconsistent between the two tables. Table 1 reports BCDU-Net (d=3) F1=0.8224, while Table 4 reports the same configuration as F1=0.8243. Likewise the U-Net accuracy values differ between Table 4 and Table 3 for the lung dataset (AC=0.9828 in Table 4 versus AC=0.9872 in Table 3). The authors need to explain which numbers come from which experimental protocol, and the final tables must report a single consistent set of test-set results, ideally with standard deviations over multiple runs.","section":"§4.4, Table 4 versus §4.1, Table 1"},{"comment":"Algorithm 1 takes both the input X and the ground-truth mask GT Mask as inputs and constructs a surrounding mask by subtracting the ground truth. The text says this preprocessing is applied to the training set, but the test-time protocol is not specified. If the lung-region estimate at test time relies on the same surrounding-mask construction, it would require ground-truth masks on the test set, which makes the evaluation circular. The authors must describe how the surrounding mask is obtained for test images without ground-truth annotations and justify that the training and test protocols are aligned.","section":"§4.3, Algorithm 1"},{"comment":"The state-of-the-art comparison is performed by quoting numbers from prior papers rather than by re-running baseline methods under the authors' exact training protocol, preprocessing, and split. Given that the metric errors above already show the authors' evaluation pipeline is not reliable, the cross-paper baseline comparisons cannot be accepted at face value. At minimum, the authors should re-implement U-Net, RU-Net, and R2U-Net under identical conditions, or clearly state the protocol differences and provide the uncertainty of each reported number.","section":"§4.1–§4.3, Tables 1–3"}],"minor_comments":[{"comment":"The title contains a typo: 'Densley' should be 'Densely'.","section":"Title"},{"comment":"The text says 'Keras with TenserFlow backend'; 'TenserFlow' should be 'TensorFlow', and 'datase' in §4.1 should be 'dataset'.","section":"§4, Experimental Results"},{"comment":"The sentence 'ia able to encode relevant information' contains a typo: 'ia' should be 'is'.","section":"§4.4, Discussion"},{"comment":"The caption of Figure 3 says 'Bi-directional ConvLSTM in CUA-Net', but the network is called BCDU-Net; the caption should be corrected.","section":"Figure 3 caption"},{"comment":"The notation for the ConvLSTM equations is inconsistent: Equation (1) uses Wci, Wcf, and Wco with a Hadamard product in the cell-update and output-gate terms, whereas standard ConvLSTM formulations use convolution operations throughout; the authors should clarify the exact tensor operations and dimensions.","section":"§3.2.2, Bi-Directional ConvLSTM"}],"recommendation":"reject","confidential_remarks":"The metric inconsistencies in Tables 2 and 3 are load-bearing and cannot be fixed by minor edits; a corrected evaluation protocol and a full rerun of experiments would be needed before the paper could be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know up front. The architecture is a legitimate extension of U-Net: it replaces skip-connection concatenation with bi-directional ConvLSTM fusion and adds dense blocks at the deepest encoder layer. That combination is new relative to the cited work, and the paper is readable. The problem is the evidence: the reported F1 and Jaccard scores contradict basic arithmetic.\n\nOn the positive side, the authors did not just stack modules randomly. The BConvLSTM fusion of high-resolution encoder features and semantic decoder features is a reasonable idea, and the dense connections in the bottleneck follow a valid intuition. They also include code and show convergence curves; the claimed 6.6x speedup with batch normalization is plausible and easy to check.\n\nThe soft spot is fatal, and the stress-test note has it right. For any binary segmentation, F1 = 2TP/(2TP+FP+FN) and Jaccard = TP/(TP+FP+FN), so F1 is always >= Jaccard. In Tables 2 and 3, BCDU-Net rows report F1 below JS (e.g., 0.851 vs 0.937 on ISIC, 0.9904 vs 0.9972 on lung). That is impossible. The JS column exactly matches the Accuracy column for those rows, which suggests a script that copied the wrong value. Table 4's DRIVE F1 for d=3 also disagrees with Table 1. With the core numbers broken, the 'state-of-the-art' claim cannot stand.\n\nThere are secondary issues. Baselines are taken from prior papers without a controlled reimplementation or shared protocol, so differences could be due to preprocessing or splits. The lung test-time procedure is underspecified: Algorithm 1 builds the surrounding mask from the ground truth, and the test-time equivalent is not described.\n\nWho gets value from this? Someone working on U-Net variants might read the architecture section as a source of ideas. But as a paper, the empirical foundation is unreliable. I would not cite it, and I would not send it to a serious referee in its current form. A resubmission with corrected metrics, controlled baselines, and a proper test-time lung protocol could change the picture, but this version is not close.","headline":"A plausible U-Net extension whose headline numbers are mathematically impossible, so the state-of-the-art claim does not hold up.","tokens_in":12342,"tokens_out":3376,"would_cite":false,"duration_ms":74513,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A U-Net variant that fuses encoder and decoder features with a bi-directional ConvLSTM and uses dense bottleneck convolutions reports higher F1, accuracy, Jaccard, and AUC than earlier U-Net-based models on retinal, skin-lesion, and…","keywords":["medical image segmentation","U-Net","bi-directional ConvLSTM","dense convolutions","batch normalization","retinal blood vessel segmentation","skin lesion segmentation","lung nodule segmentation"],"falsifier":"Re-run the three experiments with U-Net, RU-Net, R2U-Net, and BCDU-Net from one code base under identical splits, preprocessing, patch sampling, and metric definitions; if the F1/Jaccard gaps reverse or vanish, the state-of-the-art claim fails. As a direct check on the reported ISIC numbers in Table 2, recompute F1 and Jaccard from the predicted masks: for binary segmentation Jaccard cannot exceed F1, so the table's Jaccard values (0.936, 0.937) above F1 (0.847, 0.851) would have to be resolved or reproduced before that comparison can stand.","tokens_in":11292,"feed_emoji":"🩺","tokens_out":21085,"duration_ms":172295,"temperature":0.7,"pith_summary":"BCDU-Net is a U-Net variant for medical image segmentation, and the paper claims it outperforms previous U-Net-based models on three public benchmarks: retinal blood vessels (DRIVE), skin lesions (ISIC 2018), and lung regions in CT scans. Its two architectural novelties are replacing the plain concatenation in U-Net's skip connections with a bi-directional ConvLSTM that fuses high-resolution encoder features with semantic decoder features, and adding densely connected convolutions to the deepest encoding block. The paper also reports that batch normalization after up-convolutions makes training converge roughly 6.6 times faster without hurting accuracy. If these claims hold, the practical upshot is that a comparatively small architectural change to a familiar backbone can raise segmentation quality and cut training time across different medical imaging modalities.","feed_headline":"Skip-connection swap lifts U-Net on three medical segmentation tasks","feed_subtitle":"Replacing concatenation with bidirectional recurrent fusion improves metrics on retina, skin, and lung CT.","key_machinery":"The load-bearing mechanism is the bi-directional ConvLSTM (BConvLSTM) placed in every skip connection: a layer made of two ConvLSTM stacks, ConvLSTM being a recurrent unit that replaces matrix multiplications with convolutions, one scanning the input sequence forward and one backward, whose hidden states are combined by a tanh layer. The sequence has two entries, the encoded feature map and the up-convolved decoded feature map, so each feature set can modulate the other through convolutional gates instead of just being concatenated. Two supporting mechanisms carry the rest of the argument: densely connected convolution blocks in the last encoding layer, where each block receives the concatenation of all earlier blocks' outputs, and batch normalization after each up-convolution, which normalizes activations by batch mean and variance and is credited with faster convergence.","core_discovery":"The central claim is that the way U-Net combines encoder and decoder feature maps in the skip connection is a bottleneck, and nonlinear bidirectional fusion removes it. In BCDU-Net, the high-resolution feature maps from a contracting level and the up-sampled, semantically richer feature maps from the adjacent decoding level are fed as a two-entry sequence into a bi-directional ConvLSTM; a tanh combination of the forward and backward hidden states produces the fused map that goes to the decoder. At the deepest encoding level, a chain of densely connected convolution blocks lets every block see the concatenation of all previous blocks' outputs, encouraging feature reuse. Evaluated on DRIVE, ISIC 2018, and the Kaggle lung dataset, the paper reports F1, accuracy, Jaccard, and AUC numbers above the previously published U-Net, RU-Net, and R2U-Net baselines, with the dense version consistently better than the version without dense connections.","pith_inferences":["A direct ablation that replaces BConvLSTM with a single ConvLSTM or an attention gate would identify whether the gain comes from bidirectionality or from learnable gating; the paper does not run that comparison.","Since the BConvLSTM treats the two feature sets as a length-two sequence, one can read the mechanism as mutual spatial gating: each ConvLSTM state learns to encode information relevant to the other feature set; that interpretation is not spelled out in the paper.","The same architectural recipe could be transplanted to other small-annotation segmentation tasks, such as microscopy or ultrasound, but the paper only evaluates fundus, dermoscopy, and CT images.","For real-world lung screening, the training pipeline's surrounding-mask construction uses ground truth, so a deployable system would need an automatic way to estimate the same surrounding tissue at test time; the paper sketches the idea but does not specify that estimator."],"forward_implications":["If the reported numbers hold, BCDU-Net with dense blocks is the best of the compared U-Net variants on all three datasets, not just one.","On ISIC 2018, the reported Jaccard climbs from 0.592 for the best attention variant to 0.937, which would mean a large practical improvement for lesion boundary delineation.","On the lung CT set, the reported F1 of 0.9904 and accuracy of 0.9972 would make the dense version the strongest of the compared models for this task.","Batch normalization after up-convolutions would cut ISIC training epochs from about 200 to about 30, a direct practical saving whenever this architecture is retrained."],"supporting_citations":[{"why":"Defines the U-Net encoder-decoder architecture and the skip-connection concatenation that BCDU-Net modifies; this is the primary baseline.","marker":"[21]"},{"why":"Supplies the RU-Net and R2U-Net baselines and the patch-based training strategy that BCDU-Net adopts for DRIVE and ISIC.","marker":"[2]"},{"why":"Introduces the bi-directional ConvLSTM module that BCDU-Net inserts into every skip connection.","marker":"[23]"},{"why":"Introduces dense connectivity, which BCDU-Net uses to concatenate all previous block outputs in the deepest encoding layer.","marker":"[12]"},{"why":"Provides batch normalization, which BCDU-Net applies after up-convolutions and credits for faster convergence.","marker":"[13]"},{"why":"Defines the ConvLSTM equations behind the forward and backward states used by the BConvLSTM.","marker":"[26]"},{"why":"Introduces the DRIVE retinal vessel dataset used for the first evaluation.","marker":"[24]"},{"why":"Introduces the ISIC 2018 dermoscopy dataset used for the second evaluation.","marker":"[8]"},{"why":"Provides the Attention U-Net baseline that the ISIC comparison table includes.","marker":"[19]"},{"why":"Provides the Kaggle lung CT dataset used for the third evaluation.","marker":"[1]"}],"fun_headline_variants":["Bidirectional LSTM and dense blocks upgrade U-Net","Nonlinear skip fusion via bidirectional ConvLSTM in U-Net","Dense connections and bidirectional LSTM improve U-Net segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline comparison assumes that previously published baseline numbers were produced under the same data splits, patch sampling, preprocessing, and metric definitions as the authors' runs, and that the lung-region test phase can be done without ground-truth masks.","fun_headline_variants_meta":{"raw":{"variants":["Bidirectional LSTM and dense blocks upgrade U-Net","Nonlinear skip fusion via bidirectional ConvLSTM in U-Net","Dense connections and bidirectional LSTM improve U-Net segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.002626,"raw_usage":{"total_tokens":10039,"prompt_tokens":941,"completion_tokens":9098,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":9042}},"tokens_in":557,"tokens_out":9098,"duration_ms":53194,"temperature":1.0,"reasoning_tokens":9042,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T06:00:00.393674+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the three experiments with U-Net, RU-Net, R2U-Net, and BCDU-Net from one code base under identical splits, preprocessing, patch sampling, and metric definitions; if the F1/Jaccard gaps reverse or vanish, the state-of-the-art claim fails. As a direct check on the reported ISIC numbers in Table 2, recompute F1 and Jaccard from the predicted masks: for binary segmentation Jaccard cannot exceed F1, so the table's Jaccard values (0.936, 0.937) above F1 (0.847, 0.851) would have to be resolved or reproduced before that comparison can stand.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the bi-directional ConvLSTM module that BCDU-Net inserts into every skip connection."},{"cited_title":"Ioffe and C","cited_arxiv_id":null,"evidence_quote":"Provides batch normalization, which BCDU-Net applies after up-convolutions and credits for faster convergence."},{"cited_title":"Xingjian, Z","cited_arxiv_id":null,"evidence_quote":"Defines the ConvLSTM equations behind the forward and backward states used by the BConvLSTM."},{"cited_title":"Staal, M","cited_arxiv_id":null,"evidence_quote":"Introduces the DRIVE retinal vessel dataset used for the first evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the ISIC 2018 dermoscopy dataset used for the second evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Kaggle lung CT dataset used for the third evaluation."}],"review_version":1}