{"id":"b7cf7e44-e853-41aa-b3d5-710110bb7e6e","arxiv_id":"1908.08004","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A U-Net with a switching loss and majority-voting ensemble reaches top Dice and Hausdorff scores on one RVSC 2012 test set, but the state-of-the-art claim does not hold on the other test set.","lead":"Researchers trained a standard image-segmentation network with a new loss function that adaptively balances foreground and background errors to outline the right heart chamber in cardiac MRI. The method posts top accuracy on one of the two official test sets of the RVSC 2012 benchmark and analyzes which network choices matter.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline SOTA claim is not supported by the paper's own tables: the 0.86 Dice/6.73 mm HD figures are from Test Set 2 only, and even there the Dice (0.8652) trails Guo et al. (0.87); Test Set 1 is substantially worse.","rationale":"The paper is a competent empirical comparison with transparent tables and an explicit admission that Test Set 1 is not SOTA. I agree with the reader's conditional verdict. The most load-bearing problem is not the internal design of the switching loss but the relation between the abstract and the tables: the headline numbers are cherry-picked from one test set and one metric. The missing τ/ensemble size is a real reproducibility issue but secondary, because even with exact hyperparameters the comparison tables as printed do not establish a Dice-SOTA claim. I would not reject the paper; the architectural comparison is useful and the Test Set 2 HD result is a plausible contribution. The condition should be to qualify the abstract and either release code/hyperparameters or justify the threshold choice and ensemble composition. Agreement with the reader is partial: the reader's weakest_assumption focused on τ/ensemble, while my main concern is the selective SOTA claim, though the reader's rationale also notes the Test Set 1 limitation.","tokens_in":9924,"tokens_out":7996,"duration_ms":76502,"concrete_test":"Have the authors compute the endocardium Dice and HD of the majority-voting ensemble over the union of all patients in Test Set 1 and Test Set 2, and report the same pooled metrics for Guo et al. and Avendi et al. If the pooled metrics show the ensemble is not best in either Dice or HD, the abstract must be restricted to HD on Test Set 2. Independently, rerun the pipeline with τ swept over {0.1, 0.25, 0.5, 0.75, 0.9} and ensemble sizes {3, 5, 7}; report whether the 0.8652/6.73 numbers move by more than 0.005 Dice or 0.5 mm, which would make the missing τ and ensemble size material to the claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central assertion is that the proposed UNet + switching loss + majority voting reaches state-of-the-art on RVSC-MICCAI 2012. Section IV.C.2 explicitly concedes that on Test Set 1 the method is not best, and Table IV confirms this: the majority-voting ensemble scores endocardium Dice 0.8170 and HD 10.56 mm on Test Set 1, while Avendi et al. report 0.86/7.80 mm. On Test Set 2, the source of the abstract's numbers, the same ensemble scores Dice 0.8652 and HD 6.73 mm; Guo et al. report a higher Dice of 0.87 (with HD 6.93 mm). So the claim 'state-of-the-art performance ... on RVSC-MICCAI 2012 dataset' is not true for Dice on either official test set and is true for Hausdorff distance only on Test Set 2. The abstract should either name Test Set 2 and HD specifically or report pooled metrics across both test sets. A secondary reproducibility gap remains: the switching-loss threshold τ (Section III-D.4) and the number of models in the majority-voting ensemble are not reported, so the exact scores cannot be independently recomputed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a UNet-based method for right ventricle segmentation in cardiac MRI, with a new 'switching loss' that adaptively weights foreground and background Dice terms, and evaluates it on the RVSC-MICCAI 2012 challenge data. The authors compare several loss functions, network architectures, and ensembling strategies, and report that their majority-voting ensemble without post-processing achieves Dice 0.86 and Hausdorff distance 6.73 mm on the challenge test data. The paper also presents the switching loss as superior to fixed-weight loss combinations and claims state-of-the-art performance.","tokens_in":10192,"tokens_out":2219,"duration_ms":21757,"significance":"If the claims were fully supported, the paper would be a useful practical contribution: it gives a systematic comparison of losses, architectures, and ensembling for a clinically relevant but challenging segmentation task, and the switching-loss idea is a reasonable approach to class imbalance. The extensive ablation is a strength, as is the explicit comparison of CRF post-processing versus ensembling. However, the headline 'state-of-the-art on RVSC-MICCAI 2012' claim is not supported by the paper's own tables, and key methodological details (the switching-loss threshold and the ensemble size) are missing, limiting reproducibility. The comparative study itself remains useful, but the central claim needs to be corrected and the missing details supplied.","major_comments":[{"comment":"The abstract states that the method 'achieves the state-of-the-art performance of 0.86 Dice coefficient and 6.73 mm Hausdorff distance on RVSC-MICCAI 2012 dataset.' The paper's own Section IV-C-2 concedes that on Test Set 1 the results are not the best, and Table IV confirms this: the majority-voting ensemble scores 0.8170 Dice and 10.56 mm HD on Test Set 1, while Avendi et al. report 0.86 Dice and 7.80 mm HD. On Test Set 2, the reported 0.8652 Dice and 6.73 mm HD are better in Hausdorff distance than Guo et al.'s 6.93 mm, but Guo et al. achieve a higher Dice (0.87). Thus the abstract's unqualified claim of state-of-the-art performance on the RVSC dataset is not true for Dice on either official test set and is true for Hausdorff distance only on Test Set 2. The abstract and conclusion should be revised to state precisely which metric and which test set are meant, or to report pooled metrics across both test sets.","section":"Abstract and Section IV-C-2, Tables IV and V"},{"comment":"The switching loss depends on two hyperparameters, λ and τ, and the equation is written with Cn/Ct while the text defines Cf/Ct as the foreground ratio. The paper reports λ = 0.75 but never reports the value of τ, nor does it state how the threshold was selected or how sensitive the results are to it. Because the 'switching' between foreground-weighted and background-weighted Dice terms is a central component of the proposed method, the missing τ value is a load-bearing reproducibility gap. Please report the value of τ, correct the Cn/Ct notation, and add an ablation or sensitivity statement for τ.","section":"Section III-D-4, Eq. (Switching Loss)"},{"comment":"The majority-voting ensemble is presented as the best method, but neither the number of models in the ensemble nor the composition of the ensemble is specified anywhere in the manuscript. Without this information, the reported test-set scores cannot be reproduced or compared exactly with other methods. Please state explicitly how many models were combined and which input variations (e.g., augmentation runs, architectures, loss weights) were used for the ensemble.","section":"Section IV-C-2 and Tables IV and V"}],"minor_comments":[{"comment":"Table IV and Table V both have captions beginning 'Comparison of Dice scores for Endocardium'; Table V is clearly for the Epicardium and should be labeled accordingly.","section":"Tables IV and V"},{"comment":"The switching-loss equation uses the symbol Cn, which is not defined; the text says the ratio is of foreground pixels Cf to total pixels Ct. Please align the notation.","section":"Section III-D-4"},{"comment":"Table III compares validation results with published methods that may have used different validation splits or protocols; the paper should state clearly that these numbers are not directly comparable unless the same training/validation split is used.","section":"Section IV-A, Table III"},{"comment":"There are minor grammatical issues (e.g., 'a train to test split ratio of 3:1' when the actual split is 12:4 patients) and inconsistent use of 'test' versus 'validation' in Section IV; please proofread for consistency.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's main contribution is the comparative study and the switching loss, but the headline claim is contradicted by the authors' own tables. This is fixable by rewriting the abstract and adding the missing hyperparameter and ensemble details. I do not see an internal inconsistency that would warrant rejection, but the manuscript in its current form does not support the advertised state-of-the-art statement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The punchline first: the paper's headline claim is too broad for its own data. The 0.86 Dice / 6.73 mm Hausdorff numbers come from Test Set 2 only, and even there the Dice (0.8652) trails Guo et al.'s 0.87. Test Set 1 is clearly worse. So the abstract's \"state-of-the-art on RVSC-MICCAI 2012\" is not supported as written. The paper itself concedes this in Section IV-C-2, which makes the overclaim look like an abstract-wording failure rather than a deliberate deception.\n\nWhat is actually new and useful: the switching loss. It combines BCE with a foreground-ratio-switched weighting between Dice and inverse Dice. That is a small step from existing combined losses, but I haven't seen the same switch rule in the cited literature, and it performs best in their validation tests. The comparative study is genuinely thorough: four losses, six architecture variants, two ensembling methods, CRF on/off, all on a fixed 12/4 patient split. The authors report both official test sets separately, which is better than many papers in the area. The finding that simple UNet plus switching loss beats fancier architectures is a legitimate result.\n\nThe soft spots are real but mostly fixable. First, the abstract needs to either name Test Set 2 and Hausdorff distance or report pooled metrics. Second, the switching loss's threshold τ is never given, and the number of models in the majority-voting ensemble is not stated. Without those, the exact scores cannot be reproduced. Third, there is no code. These are reproducibility gaps, not signs of fabrication; the validation results and the test-set split are consistent with honest empirical work.\n\nWho is this for? Researchers working on small-structure segmentation with class imbalance, especially in cardiac MRI, who want a practical comparison of losses and architectures. It will not change clinical practice.\n\nMy recommendation: this deserves a serious referee, but it should come back with a required revision fixing the abstract and reporting τ and the ensemble size. The core empirical claim—switching loss helps on this dataset—is plausible and worth preserving.","headline":"The abstract's SOTA claim is too broad — the 0.86/6.73 numbers are Test Set 2 only and the Dice is not even best there — but the switching loss and comparative study are real, reproducible-in-spirit work worth a serious referee.","tokens_in":10703,"tokens_out":2471,"would_cite":false,"duration_ms":22790,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a plain U-Net trained with an adaptive foreground/background switching loss and majority-voting ensembling reaches state-of-the-art right ventricle segmentation in cardiac MRI without any post-processing.","keywords":["right ventricle segmentation","cardiac MRI","U-Net","switching loss","class imbalance","ensemble voting","semantic segmentation","Hausdorff distance"],"falsifier":"Run the same training protocol on the same 12-patient training split while sweeping the switching threshold across its full range and changing the ensemble size; if the endocardium Dice on the 4-patient validation set varies by more than a trivial margin, the reported 0.86 Dice / 6.73 mm figure is tied to choices the paper does not specify.","tokens_in":9701,"feed_emoji":"❤️","tokens_out":8211,"duration_ms":76932,"temperature":0.7,"pith_summary":"The paper sets out to show that right ventricle segmentation in cardiac MRI does not need elaborate architectures or post-processing. Its proposal is a plain U-Net trained with a 'switching' loss—an adaptive blend of foreground and background Dice terms—and combined with majority-voting ensembling. The authors report state-of-the-art scores of 0.86 Dice coefficient and 6.73 mm Hausdorff distance on the RVSC-MICCAI 2012 dataset. If the claim holds, a simple, cheap pipeline could replace 15–20 minutes of manual contouring per patient and make right ventricle functional assessment far more practical in clinical settings.","feed_headline":"A plain U-Net tops right ventricle segmentation scores","feed_subtitle":"Adaptive loss plus majority voting reports the best RVSC scores with no post-processing.","key_machinery":"The load-bearing object is the switching loss, an adaptive cost function combining binary cross-entropy with Dice loss $L_D$ on the foreground and inverted Dice loss $L_I$ on the background. When the foreground pixel fraction $C_f/C_t$ is above a threshold $\\tau$, the loss weights the foreground Dice term by $\\lambda=0.75$ and the background term by $1-\\lambda$; below the threshold, the weights are swapped. This lets the loss emphasize background Dice on apical slices where the ventricle is tiny, and foreground Dice elsewhere. The full pipeline is a depth-four U-Net with 3×3 convolutions and bilinear upsampling, trained with a cyclic learning-rate schedule, and combined across models by majority voting. The paper's claim is that this combination makes post-processing such as fully-connected CRF unnecessary.","core_discovery":"For right ventricle segmentation in short-axis cardiac MRI, the authors find that a depth-four U-Net trained with a switching loss and then combined by majority voting performs at or above the level of more elaborate networks such as dilated U-Net, global convolution networks, and GCN-UNet hybrids, and does not need CRF post-processing. On the RVSC-MICCAI 2012 dataset they report a Dice coefficient of 0.86 and a Hausdorff distance of 6.73 mm for the endocardial contour on test set 2, along with the best validation-set Dice among the compared methods. The switching loss, which adaptively weights the foreground Dice versus the inverted background Dice according to the fraction of foreground pixels, is the component that the paper credits for handling the extreme class imbalance of the small, crescent-shaped right ventricle.","pith_inferences":["One consequence the authors leave implicit is that their recipe is not fully reproducible as written: the switching threshold $\\tau$ and the number of models in the majority-vote ensemble are never reported, so a follow-up should publish those values or tie $\\tau$ to a validation-set sweep.","The comparative failure of focal loss on this tiny-foreground task suggests a testable hypothesis for other medical segmentation problems: adaptive foreground/background Dice weighting may beat focal loss when the foreground is small but spatially coherent, such as small lesions or vessels.","Since average-probability ensembling matches majority voting, a single model distilled to mimic the ensemble could retain most of the accuracy at lower inference cost; this is an experiment the paper does not run."],"forward_implications":["A modest U-Net, rather than a larger receptive-field model, is the better substrate for right ventricle segmentation once the loss is matched to the class imbalance.","The switching loss outperforms focal loss and fixed combinations of BCE, Dice, and inverted Dice on this dataset, giving a recipe for other small-foreground segmentation tasks.","Majority voting and average-probability ensembling give similar gains, and CRF post-processing adds nothing, so the pipeline can stay simple.","On RVSC test set 2, the method reports the best endocardium Dice (0.8652) and Hausdorff distance (6.73 mm) among the compared state-of-the-art methods, while on test set 1 it does not reach the best numbers."],"supporting_citations":[{"why":"Supplies the U-Net architecture that the paper uses as its baseline and final model.","marker":"[10]"},{"why":"Documents the RVSC challenge, the difficulty of right ventricle borders, and the human-level 0.90 Dice target.","marker":"[1]"},{"why":"Prior cardiac MRI segmentation with U-Net that the paper adapts, and the source of the dilated-UNet comparison.","marker":"[12]"},{"why":"Prior deep-learning right ventricle segmentation with a partly dilated U-Net, used as a state-of-the-art comparison.","marker":"[6]"},{"why":"Multicenter deep-learning biventricular segmentation used as a state-of-the-art comparison.","marker":"[7]"},{"why":"Cascaded two-stage deep-learning right ventricle segmentation used as a state-of-the-art comparison.","marker":"[8]"},{"why":"Defines focal loss, the main class-imbalance loss that the switching loss is benchmarked against.","marker":"[18]"},{"why":"Supplies the cyclic learning-rate schedule used in training all compared models.","marker":"[20]"},{"why":"Defines the global convolution and boundary-refinement blocks used in the architecture comparison.","marker":"[11]"},{"why":"Supplies the fully-connected CRF post-processing whose omission the paper tests.","marker":"[16]"}],"fun_headline_variants":["Plain U-Net with adaptive loss beats complex cardiac MRI models","Right ventricle segmentation: simple U-Net, no post-processing, top scores","Adaptive loss boosts U-Net to top right ventricle MRI scores","No post-processing: U-Net hits 0.86 Dice on RVSC challenge","Simple U-Net outperforms complex nets for heart ventricle segmentation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the exact choice of the switching threshold and of the number of models in the majority-vote ensemble does not change the trained model enough to affect the reported score.","fun_headline_variants_meta":{"raw":{"variants":["Plain U-Net with adaptive loss beats complex cardiac MRI models","Right ventricle segmentation: simple U-Net, no post-processing, top scores","Adaptive loss boosts U-Net to top right ventricle MRI scores","No post-processing: U-Net hits 0.86 Dice on RVSC challenge","Simple U-Net outperforms complex nets for heart ventricle segmentation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000173,"raw_usage":{"total_tokens":1227,"prompt_tokens":840,"completion_tokens":387,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":456,"completion_tokens_details":{"reasoning_tokens":292}},"tokens_in":456,"tokens_out":387,"duration_ms":4051,"temperature":1.0,"reasoning_tokens":292,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:51:21.256474+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same training protocol on the same 12-patient training split while sweeping the switching threshold across its full range and changing the ensemble size; if the endocardium Dice on the 4-patient validation set varies by more than a trivial margin, the reported 0.86 Dice / 6.73 mm figure is tied to choices the paper does not specify.","supporting_citations":[{"cited_title":"Right ventricle segmentation from cardiac MRI: a collation study,","cited_arxiv_id":null,"evidence_quote":"Documents the RVSC challenge, the difficulty of right ventricle borders, and the human-level 0.90 Dice target."},{"cited_title":"Cardiac MRI segmentation,","cited_arxiv_id":null,"evidence_quote":"Prior cardiac MRI segmentation with U-Net that the paper adapts, and the source of the dilated-UNet comparison."},{"cited_title":"Right ventricle segmentation in cardiac MR images using u-net with partly dilated convolution,","cited_arxiv_id":null,"evidence_quote":"Prior deep-learning right ventricle segmentation with a partly dilated U-Net, used as a state-of-the-art comparison."},{"cited_title":"ν-net: deep learning for generalized biventricular mass and function parameters using multicenter cardiac MRI data,","cited_arxiv_id":null,"evidence_quote":"Multicenter deep-learning biventricular segmentation used as a state-of-the-art comparison."},{"cited_title":"A deep learning network for right ventricle segmentation in short-axis MRI,","cited_arxiv_id":null,"evidence_quote":"Cascaded two-stage deep-learning right ventricle segmentation used as a state-of-the-art comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines focal loss, the main class-imbalance loss that the switching loss is benchmarked against."},{"cited_title":"Avendi, et al","cited_arxiv_id":null,"evidence_quote":"Supplies the cyclic learning-rate schedule used in training all compared models."},{"cited_title":"Large kernel matters– improve semantic segmentation by global convolutional network,","cited_arxiv_id":null,"evidence_quote":"Defines the global convolution and boundary-refinement blocks used in the architecture comparison."},{"cited_title":"Avendi, et al","cited_arxiv_id":null,"evidence_quote":"Supplies the fully-connected CRF post-processing whose omission the paper tests."}],"review_version":1}