{"id":"96668660-e5fb-4ff9-a060-b1e70671e261","arxiv_id":"2412.20709","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":1.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"ResUnet++ reports 98.17% Jaccard on LGG segmentation, but the evaluation has no defined test set, baselines, code, or HCI study.","lead":"This paper describes a U-Net-style model with residual connections, ASPP, and attention, and reports 98.17% Jaccard on the LGG brain tumor dataset. It claims the model improves clinical interaction through human-computer interaction, but presents no usability study, code, or comparison to other segmentation models.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 98.17% IoU claim is internally contradicted by the paper's own validation loss of 0.1718, and no independent test split is defined; the central result is unsupported.","rationale":"The reader's weakest_assumption identifies the same load-bearing issue: the 98.17% figure is unsupported because no independent evaluation split is defined and the reported validation loss is inconsistent with it. My independent reading confirms this. The paper provides no code, no baselines, no error bars, no formal verification, and no clear statement of which data partition produced the headline number. The arithmetic contradiction is decisive: under the paper's own stated loss function, a validation Jaccard loss of 0.1718 corresponds to an IoU of about 82.82%, not 98.17%. Even allowing for a different soft-Jaccard formulation, the gap between 0.1718 and 0.0183 is far too large to be a rounding artefact. A category error also appears in the abstract, where 'Jaccard Loss' is reported as 98.17%; a loss of 98.17% would mean almost no overlap. Because the headline performance number is the only quantitative evidence for the model's value, and because the HCI contribution is described but never tested, the central claim collapses without external verification. This does not change the reader's reject verdict; it reinforces it. I recommend the manuscript remain rejected or, at most, be revised into a methods-only report pending a reproducible evaluation.","tokens_in":4329,"tokens_out":4127,"duration_ms":39727,"concrete_test":"Retrain ResUnet++ on the LGG dataset using the architecture and hyperparameters given in Section 3, with a pre-registered train/validation/test split, and compute the Jaccard index on the held-out test set only. Compare this test IoU with 98.17% and with the value implied by the reported validation loss of 0.1718 under the stated Jaccard loss. If the test IoU is not approximately 0.982, or if the validation loss is 0.1718 while the test IoU is claimed to be 0.982, the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is the reported 98.17% Jaccard/IoU performance on the LGG Segmentation Dataset (Abstract, Section 4.2, Section 5). The weakest point is that this number is internally inconsistent with the losses the paper itself reports under the very loss function it says it uses. Section 3.2 states that the model is trained with Jaccard Loss, and Section 5 reports a training loss of 0.0764 and a validation loss of 0.1718 after 60 epochs. If Jaccard loss is defined in the standard way as 1 − IoU, a validation loss of 0.1718 corresponds to an IoU of roughly 82.82%, not 98.17%. Conversely, an IoU of 98.17% would require a validation loss near 0.0183, not 0.1718. The manuscript never defines a train/validation/test split or states on which subset the 98.17% figure was computed, so it cannot be attributed to independent test performance. Moreover, 'Jaccard Loss of 98.17%' is itself a category error: a loss of 98.17% would indicate near-zero overlap, not good segmentation. These contradictions place the entire empirical contribution, and with it the HCI-integration narrative, on unsupported ground.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ResUnet++, a hybrid U-Net/ResNet architecture with residual blocks in both downsampling and upsampling paths, ASPP, and an attention mechanism, for brain tumor segmentation on the LGG MRI dataset. It claims that the model achieves a 98.17% Jaccard/IoU score and that the approach integrates human-computer interaction to improve clinical workflow. The manuscript includes a literature survey, architecture description, experimental section, and conclusion. The central empirical claim is the reported 98.17% performance, and the HCI integration is presented as the distinctive contribution alongside the architecture.","tokens_in":4631,"tokens_out":4679,"duration_ms":40254,"significance":"If the performance and HCI claims were substantiated, the paper would offer a useful benchmark-style contribution to medical image segmentation and a pathway for interactive clinical tools. However, the manuscript does not supply the evidence needed to support these claims: the headline metric is internally inconsistent with the reported loss values, no independent test set is defined, and the HCI component is never implemented or evaluated. The main positive features are the understandable motivation for residual connections in segmentation and the reproducible problem formulation, but the missing validation and contradictory numbers undermine the contribution. No code, reproducible experiments, or formal evaluations are provided, so the significance of the work as a scientific contribution is currently minimal.","major_comments":[{"comment":"The central performance claim is internally inconsistent with the reported loss values. If the model is trained with Jaccard Loss, as stated in Section 3.2, then a validation loss of 0.1718 reported in Section 5 corresponds to an IoU of roughly 82.8%, not 98.17%. Conversely, an IoU of 98.17% would require a validation loss near 0.0183. Moreover, the phrase 'Jaccard Loss of 98.17%' in the Abstract is a category error: a loss of 98.17% would indicate near-zero overlap, not high segmentation quality. The authors must reconcile the metric definition, the exact number, and the subset on which it is computed.","section":"Abstract, §3.2, §4.2, §5"},{"comment":"The manuscript never defines a train/validation/test split. Section 4.1 mentions the LGG dataset is used for training, validation, and testing, but no split ratio, patient-level separation, or evaluation protocol is given. The reported 98.17% IoU therefore cannot be attributed to independent test data, and hyperparameters (learning rate, epsilon, number of epochs) appear to be selected using the same data. The authors must provide a clear data split and report metrics on a held-out test set, or revise the claim accordingly.","section":"§3.2, §4.2"},{"comment":"The claimed HCI contribution is not evaluated anywhere in the paper. The abstract and introduction state that ResUnet++ provides real-time feedback and clinician interaction, but Sections 3 through 5 contain no user study, interface description, usability metrics, or clinician-in-the-loop experiments. The phrase 'HCI principles are integrated' in Section 1 is unsupported by any methodological or experimental detail, so the HCI narrative is only motivational.","section":"Abstract, §1, §3-§5"},{"comment":"The model description is inconsistent. The first Section 3.2 says the network includes an ASPP module and attention mechanism, but Figures 1 and 2, which are described as the downsampling and upsampling sections, show no such components. The same section states the model uses the Adam optimizer with learning rate 0.05 and epsilon 0.1, while the second Section 3.2 (NAdam optimizer and learning rate) says NAdam is chosen; the paper never clarifies which optimizer was actually used. Each architectural component and hyperparameter should be specified unambiguously.","section":"§3.2 (two headings)"}],"minor_comments":[{"comment":"Figure 4's caption 'The change of accuary for resunet++' contains a typo ('accuary') and does not identify what accuracy or loss is plotted or on which data split; the axes are also unlabeled.","section":"Fig. 4"},{"comment":"The caption for Fig. 5 references 'figure10' rather than 'Fig. 5' and should be corrected.","section":"Fig. 5 caption"},{"comment":"The text describing the upsampling path says 'For downsampling, a 2×2 upsampling layer doubles the image size'; this appears to be a typo for 'upsampling' and should be fixed.","section":"§3.2"},{"comment":"The experimental results section reports only a single IoU value (98.17%) with no error bars, no comparison to baseline or state-of-the-art methods, and no statistical significance testing; the reader cannot assess whether this value is representative.","section":"§4.2"},{"comment":"The related-work section does not cite any prior ResUnet++ literature; since ResUnet++ is an existing architecture in medical image segmentation, the novelty of the proposed model should be clarified with explicit comparison.","section":"§2"},{"comment":"The reference list would benefit from standardizing entries (e.g., complete volume/page information for the IEEE journals) and from checking that all in-text citations match the bibliography; some entries lack full bibliographic details.","section":"References"}],"recommendation":"reject","confidential_remarks":"The paper is not close to being publishable in its current form. The headline metric contradicts the paper's own loss reports, no test protocol is defined, and the HCI component is absent from the evaluation. The authors should be encouraged to re-run the experiments with a clear train/validation/test split, correct the metric definitions, and either implement and evaluate the HCI component or remove it from the contribution claim. A comparison with existing ResUnet++ architectures is also necessary to establish novelty."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe punchline is that the 98.17% IoU figure is not just unsubstantiated; it's internally inconsistent with the paper's own validation loss. The authors report training with Jaccard loss and then give a validation loss of 0.1718 after 60 epochs. If Jaccard loss is 1 − IoU, that corresponds to roughly 82.8% IoU, not 98.17%. The abstract even says \"Jaccard Loss of 98.17%,\" which is a category error: a loss of 98.17% would mean almost no overlap. No test split is defined, so the headline number cannot be tied to any independent evaluation.\n\nWhat's actually new? Very little. The architecture combines residual blocks, ASPP, and attention inside a U-Net encoder-decoder. That combination is already explored, and the name ResUnet++ is taken from Jha et al. (2021), which is not cited. The HCI component is all narrative: no user study, no interaction design, no description of how clinicians would use it. It's merely mentioned in the introduction and conclusion.\n\nTo be fair, the paper describes the network flow clearly, uses a public dataset (LGG), and the conclusion honestly admits overfitting and accuracy fluctuations. These are small graces but do not rescue the work.\n\nThe soft spots are load-bearing. No train/validation/test split is defined, so the 98.17% cannot be reproduced or verified. There are no comparisons to U-Net, U-Net++, or Attention U-Net, despite those being cited in the literature survey. Hyperparameters are contradictory: the methods section says Adam with learning rate 0.05 and epsilon 0.1, but the next subsection says NAdam was chosen. No code or weights are released.\n\nIn summary, this paper would only serve as a cautionary example in a writing or methodology course. It does not deserve a serious referee slot as is. The right call is to reject and advise the authors to reposition the work with a proper evaluation protocol, real HCI study, and proper citations.\n\nBest, [Your Name]","headline":"The paper's central 98.17% IoU claim is contradicted by its own validation loss, the architecture is a re-branding of existing work, and the HCI component is unevaluated; reject outright.","tokens_in":5167,"tokens_out":2797,"would_cite":false,"duration_ms":26790,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68U10","92C55"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces ResUnet++, a residual-block U-Net hybrid for brain tumor MRI segmentation, and reports a 98.17% Jaccard/IoU on the LGG dataset while framing the tool as clinician-interactive.","keywords":["ResUnet++","brain tumor segmentation","medical image segmentation","residual learning","U-Net","Jaccard loss","human-computer interaction","LGG segmentation dataset"],"falsifier":"Re-run ResUnet++ on the LGG dataset with an explicit held-out test split and at least three random seeds, computing the mean test IoU; if it falls well below $0.9817$, or if $1 - \\text{Jaccard loss}$ does not match the reported validation loss of $0.1718$, the paper's core performance claim does not hold.","tokens_in":4136,"feed_emoji":"🧠","tokens_out":10412,"duration_ms":87775,"temperature":0.7,"pith_summary":"The paper proposes ResUnet++, a hybrid model that inserts ResNet-style residual blocks into a U-Net encoder-decoder and adds ASPP multi-scale pooling plus attention. The authors claim that on the LGG brain-tumor MRI dataset this model reaches a Jaccard/IoU of $98.17\\%$, which they call both a Jaccard loss and an IoU index. They further argue that human-computer interaction features, real-time visualization and feedback, make the segmentation tool practical for clinicians. If the performance number holds, ResUnet++ would be a near-exact tumor segmenter with a built-in path to clinical interaction.","feed_headline":"Hybrid U-Net claims 98.17% tumor segmentation IoU","feed_subtitle":"ResUnet++ pairs residual connections with clinician feedback for brain MRI tumor analysis.","key_machinery":"The residual block is the mechanism that carries the architecture: a shortcut connection adds the block's input to its transformed output, so features survive the down- and upsampling depths. The Jaccard loss is the identity that produces the headline number, computing intersection over union between predicted and ground-truth tumor regions. Around these sit the ASPP module for multi-scale context and an attention mechanism for suppressing irrelevant regions.","core_discovery":"ResUnet++'s downsampling path alternates convolution, batch normalization, ReLU, max pooling, and residual blocks, growing from 16 to 256 feature channels; the upsampling path mirrors this with up-convolutions, concatenation, and residual blocks, ending with a $1\\times1\\times1$ convolution to a single output channel. The model is trained for 60 epochs with the NAdam optimizer, learning-rate decay, early stopping, and Jaccard loss. The paper's central reported result is a single number: $\\mathrm{IoU} = 98.17\\%$ on the LGG Segmentation Dataset, which it interprets as good model fit and uses to motivate real-time clinician interaction. The authors also report training and validation losses of 0.0764 and 0.1718 after 60 epochs.","pith_inferences":["A consequence the authors do not draw: their HCI claim is about usability, so it cannot be tested from the reported numbers; a small usability study with radiologists would be the direct next experiment.","A testable extension is to report Dice, per-fold variance, and a defined test split on LGG; that would show whether 98.17% is stable or a single training run.","The text uses Jaccard loss, Jaccard index, and IoU for the same number; auditing which quantity was actually computed is the first step before comparing with other models.","A direct comparison against U-Net, UNet++, and nnU-Net under the same split would place ResUnet++ in context, but the paper provides no baseline comparison."],"forward_implications":["If the 98.17% Jaccard/IoU holds on held-out data, ResUnet++ could generate whole-tumor contours accurate enough to reduce manual delineation effort in lower-grade glioma cases.","The HCI layer would give clinicians real-time visualization and correction of predicted tumor regions, potentially increasing trust in automated segmentations.","The residual-plus-ASPP-plus-attention recipe would transfer to other MRI segmentation targets with modest changes to the encoder depth.","Using Jaccard loss alone would remove the need for hand-tuned class weights on imbalanced tumor masks."],"supporting_citations":[{"why":"Residual learning supplies the Resblocks used in every down- and up-sampling stage.","marker":"[1]"},{"why":"The U-Net encoder-decoder with skip connections is the backbone ResUnet++ extends.","marker":"[2]"},{"why":"The LGG dataset of MRI images and tumour masks is the evaluation set on which the 98.17% result is claimed.","marker":"Buda, Saha & Mazurowski (2019)"}],"fun_headline_variants":["ResUnet++ hits 98.17% IoU on brain tumor segmentation","Interactive ResUnet++ for clinician-driven tumor localization","98.17% IoU: ResUnet++ fuses AI with clinical interaction","ResUnet++ brain tumor network: 98.17% segmentation score","Human-in-the-loop ResUnet++ maps tumors with 98.17% IoU"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline number only means what it seems to mean if $98.17\\%$ is a Jaccard/IoU score computed on an independent test split, which the paper never defines; the same number is also called a loss, and the reported training/validation losses do not line up with it.","fun_headline_variants_meta":{"raw":{"variants":["ResUnet++ hits 98.17% IoU on brain tumor segmentation","Interactive ResUnet++ for clinician-driven tumor localization","98.17% IoU: ResUnet++ fuses AI with clinical interaction","ResUnet++ brain tumor network: 98.17% segmentation score","Human-in-the-loop ResUnet++ maps tumors with 98.17% IoU"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000632,"raw_usage":{"total_tokens":2925,"prompt_tokens":959,"completion_tokens":1966,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":1864}},"tokens_in":575,"tokens_out":1966,"duration_ms":14885,"temperature":1.0,"reasoning_tokens":1864,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:12:49.453902+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run ResUnet++ on the LGG dataset with an explicit held-out test split and at least three random seeds, computing the mean test IoU; if it falls well below $0.9817$, or if $1 - \\text{Jaccard loss}$ does not match the reported validation loss of $0.1718$, the paper's core performance claim does not hold.","supporting_citations":[{"cited_title":"Deep Residual Learning for Image Recognition","cited_arxiv_id":null,"evidence_quote":"Residual learning supplies the Resblocks used in every down- and up-sampling stage."},{"cited_title":"U -Net: Convolutional Networks for Biomedical Image Segmentation","cited_arxiv_id":null,"evidence_quote":"The U-Net encoder-decoder with skip connections is the backbone ResUnet++ extends."}],"review_version":1}