{"id":"7227bf6b-cf9b-438b-a384-8895fd54106a","arxiv_id":"2411.14752","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A challenge report showing a MedNeXt small model winning Task 1 (DSC 0.8254) and an nnUNet ensemble placing 8th in Task 2 (0.7005) for MRI-guided radiotherapy tumor segmentation.","lead":"Team TUMOR reports its HNTS-MRG24 MICCAI challenge solution, using nnUNet and MedNeXt to segment head and neck tumors in pre- and mid-radiotherapy MRI. The Task 1 submission ranked first with an aggregated Dice score of 0.8254, and the paper examines why adding earlier treatment masks helps mid-treatment segmentation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Task 2 mask-benefit claim rests on unvalidated challenge registration; ranking claim itself is externally supported.","rationale":"The main ranking claim is independently anchored by the challenge leaderboard; the paper's internal validation tables are not needed to establish that the submitted pipelines placed 1st and 8th. The more fragile part is the paper's secondary conclusion that registered pre-RT masks improve mid-RT segmentation. That conclusion is drawn from an ablation (Section 2.4, Table 6) that uses the challenge-provided registration as a fixed, unquantified preprocessing step. Since the pre-RT mask is a ground-truth contour, its value as an input channel depends directly on how well the registration aligns pre-RT anatomy to the mid-RT scan. The paper neither reports a registration accuracy metric nor tests sensitivity to misregistration, so the claim's external validity is unestablished. This is not an internal inconsistency and it does not threaten the reported rankings; it does mean the paper should be read conditionally on registration quality. The proposed test would resolve the uncertainty by measuring the overlap of registered masks with mid-RT ground truth and by perturbing or removing registration in the ablation. This matches the reader's weakest assumption, so the reader's conditional verdict remains appropriate.","tokens_in":12118,"tokens_out":9028,"duration_ms":94983,"concrete_test":"Using the 150 training cases, compute per-case Dice overlap between each registered pre-RT mask and the corresponding mid-RT ground-truth mask for GTVp and GTVn. If median overlap is low (e.g., <0.6), the input masks are poorly registered and the Task 2 gains may not reflect reliable correspondence. Then retrain/evaluate the Dataset 506 model with (a) original unregistered pre-RT masks and (b) registered masks perturbed by 3-5 voxel shifts. If DSCagg drops materially (>0.05) under either condition, the benefit is registration-specific and the generalization claim needs qualification; if performance is stable, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The challenge-ranking claim (Task 1 1st, Task 2 8th) is externally grounded in the HNTS-MRG24 leaderboard and is not under serious threat from internal issues. The load-bearing part of the paper's scientific conclusion is the Task 2 ablation in Section 2.4 and Table 6: adding registered pre-RT segmentation masks (Datasets 506/507) improves mid-RT DSCagg over mid-RT-only training (Dataset 504). For that conclusion to transfer beyond this challenge, the registered input must carry reliable anatomical correspondence. The paper never quantifies registration accuracy, and the masks are ground-truth contours from the pre-RT time point, so the observed gain could largely reflect a shortcut: the model copies or refines the input mask, which is informative only to the extent that pre-RT and mid-RT tumor locations overlap after registration. If the challenge registration is accurate, the gain is expected; if it is mediocre, the gain is an artifact of aligned input noise, and the 'significantly improves' claim would not generalize to other registration pipelines. Since no registration metric or perturbation ablation is reported, this is an unresolved dependency, not an internal contradiction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports the HNTS-MRG24 MICCAI challenge solution from team TUMOR. For Task 1 (pre-RT segmentation of GTVp and GTVn), the authors pretrain nnUNet and MedNeXt variants on mid-RT and registered pre-RT images, then fine-tune on original pre-RT images; the final Task 1 submission, MedNeXt Small with kernel size 3, achieved 1st place with DSCagg 0.8254 on the 50-case test set. For Task 2 (mid-RT segmentation), the authors evaluate multi-channel inputs that combine mid-RT, registered pre-RT images, and registered pre-RT segmentation masks; the final submission, an nnUNet FullRes+Cascade ensemble trained on Dataset 506, ranked 8th with DSCagg 0.7005. The paper also compares model families and ablates the contribution of the registered pre-RT masks.","tokens_in":12311,"tokens_out":7262,"duration_ms":69453,"significance":"If the leaderboard claims are correct, the paper documents a top-performing, publicly available pipeline for a clinically relevant segmentation task, and the challenge ranking provides external validation. The more interesting scientific content is the Task 2 ablation showing that registered pre-RT masks improve mid-RT segmentation. That conclusion, however, is conditional on the quality of the challenge-provided registration and is partly confounded by incomplete MedNeXt training. The paper is honest about training collapses and the invalidity of the pretraining comparison, which is a strength, but the central secondary claim needs additional support before it can be considered transferable.","major_comments":[{"comment":"The central Task 2 conclusion that adding registered pre-RT segmentation masks improves mid-RT segmentation is only cleanly supported for nnUNet. The MedNeXt rows in Table 6 are confounded: the Dataset 506 result is produced by models that did not complete the planned 1000 epochs on folds 0, 3, and 4 (Table 5), and the Dataset 516 result comes from a filtered 115-case subset that removes cases with zero ground truth. The sentence in Section 4 that 'Both nnUNet and MedNeXt models trained on datasets which included segmentation masks ... performed better' therefore mixes a clean nnUNet ablation with a partially trained and differently distributed MedNeXt comparison. Please either retrain MedNeXt on the full Dataset 506 or separate the nnUNet ablation from the exploratory MedNeXt comparison and label the latter as such.","section":"Section 2.4, Table 6, Section 4"},{"comment":"The paper's secondary scientific claim that registered pre-RT masks 'significantly improve' mid-RT segmentation is conditional on the spatial accuracy of the challenge-provided registration. No registration quality metric is reported, and no ablation controls for misalignment, such as feeding the unregistered pre-RT mask or a randomly perturbed mask as a negative control. Without such evidence, the observed gain may reflect the model copying or refining an input that is informative only because the registration happens to align pre-RT and mid-RT anatomy; the conclusion would not transfer to other registration pipelines. Please add a registration quality measure (e.g., landmark or overlap statistics) or a misalignment ablation.","section":"Section 2.4, Table 6, Section 4"},{"comment":"The title promises a comparative analysis of nnUNet and MedNeXt, but only the MedNeXt Small (Kernel 3) variant trained successfully; Small (Kernel 5), Large (Kernel 3), and Large (Kernel 5) all collapsed in both tasks. The claim in Section 4 that 'MedNeXt proved to be a stronger candidate for reliable segmentation compared to nnUNet' is therefore supported only for this single variant on Task 1. Please narrow the conclusion to 'MedNeXt Small (Kernel 3)' and explicitly state that the comparison is between one MedNeXt configuration and the complete nnUNet family, as the paper itself acknowledges later in the same section.","section":"Section 3 (Task 1) and Section 4"}],"minor_comments":[{"comment":"There are several typos, including 'therfore', 'T ask', 'of0.8254', and 'ResEnd' instead of 'ResEnc'; these should be corrected.","section":"Abstract and throughout"},{"comment":"Please clarify the data split: the text says the dataset comprises 150 MRI scans and later refers to '50 test patients'; the number of training and validation cases used for Tables 1-8 is not stated.","section":"Section 2.1"},{"comment":"DSCagg is reported as a single aggregate with no uncertainty; adding bootstrap confidence intervals or paired per-case tests would make the model-comparison statements more robust.","section":"Section 3, Tables 1 and 6"},{"comment":"The statement that 'every possible combination of nnUNet models was aggregated' is not fully reflected in Table 1, which lists only some combinations; either list all combinations or revise the wording.","section":"Section 2.4"},{"comment":"The notation for DSCagg and per-case DSC is easy to confuse; please clarify that Eq. (1) aggregates over the full set while Eq. (2) is per image i, and define the index ranges explicitly.","section":"Section 2.5, Eqs. (1)-(2)"}],"recommendation":"major_revision","confidential_remarks":"This is a challenge-report style paper with externally confirmed leaderboard results, so the rank claims are solid. The scientific novelty is modest, but the Task 2 mask-ablation result is worth publishing if the registration-dependence issue is addressed. The authors are transparent about training instabilities, which is commendable. A major revision requiring a registration-quality check or negative-control ablation, plus a cleaner separation of the MedNeXt comparison, would be sufficient; a full re-run of all experiments is not necessary."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Jan, quick take on 2411.14752. The useful thing here is the first published account of the HNTS-MRG24 dataset and the Task 2 ablation showing that adding registered pre-RT segmentation masks as input channels lifts mid-RT DSCagg noticeably (from ~0.65 to ~0.73 for nnUNet FullRes). That is a concrete empirical finding for a clinically relevant question. The ranking claim (1st on Task 1, 8th on Task 2) is externally supported by the challenge leaderboard, so I trust it. The paper is also unusually honest about its own messes: collapsed MedNeXt runs, discarded cases, and an invalid pretraining comparison that it explicitly flags.\n\nThe soft spots are the usual challenge-report ones, plus one that matters. The comparison between nnUNet and MedNeXt in Task 2 is not clean, because the MedNeXt results include folds that never finished training; the authors acknowledge this but still let those numbers shape the conclusion. The 'significantly improves' language for the mask benefit is not backed by any error bar or test on the aggregate metric. And the mask benefit itself is only as meaningful as the challenge registration; if that registration were poor, the model could be copying a misaligned prior mask and the whole finding would not transfer. The paper does not validate registration quality, so the claim is dataset-specific, not yet general. None of this undermines the ranking claim, which is the headline. Also, the promised GitHub link appears only as text in the abstract, not as a working URL.\n\nWho is this for? People working on MRI-guided adaptive RT segmentation and challenge organizers. It is a useful reference point, not a methodological advance. I would send it out for peer review as a challenge report, with a referee asked to focus on whether the Task 2 comparisons are fair and whether the mask-benefit claim is appropriately scoped.","headline":"First published HNTS-MRG24 challenge report with an externally supported 1st-place ranking and a genuinely useful but registration-dependent mask-ablation finding.","tokens_in":12826,"tokens_out":2151,"would_cite":true,"duration_ms":22250,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Single MedNeXt model wins head-and-neck MRI tumor segmentation task","keywords":["HNTS-MRG24","MICCAI24","nnUNet","MedNeXt","head and neck cancer","MRI-guided radiotherapy","tumor segmentation","aggregated Dice similarity"],"falsifier":"Train the Task 2 pipelines on a cohort with independently verified high-accuracy registration, then repeat the same training with artificially perturbed registration; if the benefit of adding pre-RT masks persists under misalignment, the conclusion is not registration-dependent. A simpler check is to re-run the dataset 504 versus 506 comparison on a test set whose pre-RT-to-mid-RT registration was recomputed with a different algorithm and see whether the mask-channel advantage remains.","tokens_in":11969,"feed_emoji":"🩻","tokens_out":8762,"duration_ms":78260,"temperature":0.7,"pith_summary":"This paper reports the winning solution to the head-and-neck MRI tumor segmentation challenge's first task (pre-radiotherapy scans) and an eighth-place solution to its second task (mid-radiotherapy scans). The authors claim that a small MedNeXt model with 3×3×3 kernels outperformed every nnUNet configuration on the pre-treatment task, reaching an aggregated Dice Similarity Coefficient of 0.8254 on the final test set. For the mid-treatment task, they claim that adding the registered pre-treatment image and its segmentation mask as extra input channels clearly improves both models compared to using mid-treatment images alone. This matters because MRI-guided adaptive radiotherapy needs fast, accurate tumor delineation at every treatment session, and the result shows that prior time-point masks carry useful localization information that can be supplied to the network directly.","feed_headline":"Single MedNeXt model wins head-and-neck MRI tumor segmentation task","feed_subtitle":"A 0.8254 aggregated score for pre-treatment scans; registered masks boost mid-treatment accuracy.","key_machinery":"The mechanism that carries the mid-RT argument is input-channel composition: the network receives the mid-RT T2-weighted scan together with the registered pre-RT scan and the registered pre-RT segmentation mask (datasets 506 and 507), effectively using the earlier time point's expert annotation as a spatial prior for where the tumor can be found at mid-treatment. For Task 1, the decisive model is MedNeXt Small with kernel size 3, a convolutional architecture with 32 channels, an expansion ratio of 2, and 2 blocks per stage, whose standalone predictions outperformed every nnUNet combination. The ranking itself is carried by the aggregated Dice Similarity Coefficient ($DSC_{agg}$), which sums volumes across the whole test set instead of averaging per-sample scores, as defined in Equation 1 of the paper.","core_discovery":"The central claim, stated on the paper's own terms, is that on the challenge's 150-patient dataset a single MedNeXt Small (kernel 3) model produces the best pre-RT segmentation, beating all nnUNet ensembles and achieving an aggregated Dice of 0.8254 to rank first in Task 1, while an nnUNet ensemble of Full Resolution and Cascade models ranks eighth in Task 2 with 0.7005. The authors also establish that for mid-RT segmentation, including the registered pre-RT image and registered pre-RT mask as extra input channels (their dataset 506) gives the highest aggregated Dice for nnUNet, and that including the pre-RT mask is the key factor: inputs with masks outperform mid-RT-only inputs, while adding the registered image alone does not help. They further report that averaging nnUNet and MedNeXt probability maps lowers performance, and that the larger MedNeXt variants and kernel-5 versions collapsed during training, so the comparison is limited to the small kernel-3 MedNeXt.","pith_inferences":["One untested extension is per-label model selection: the per-label Dice tables suggest MedNeXt may be stronger on GTVp while nnUNet is stronger on GTVn, so assigning each label its best model could exceed either ensemble.","If the mask-channel benefit generalizes, longitudinal segmentation could be framed as conditioning on a prior contour, suggesting connections to interactive or prompt-based segmentation where the previous mask acts as a prompt.","The poor transfer from the brain MRI pretraining data suggests cross-anatomy pretraining for segmentation needs explicit domain adaptation; testing progressive unfreezing or contrastive alignment between T1w brain and T2w head-and-neck scans would be a concrete next experiment.","The reported training instability of larger MedNeXt variants on a 115-sample filtered dataset hints that hyperparameter or optimizer adjustments, rather than architecture capability, may unlock those models on small medical datasets."],"forward_implications":["If the Task 1 result holds, a single MedNeXt Small kernel-3 model is sufficient to exceed multi-model nnUNet ensembles on pre-RT head-and-neck scans, which could simplify deployment.","If the Task 2 result holds, adaptive radiotherapy workflows can store the planning-time contours and feed them as input channels at each fraction, improving mid-treatment segmentation without architectural changes.","The observation that adding the registered pre-RT image alone does not help, while adding the mask does, implies that the meaningful prior is the annotated location, not the image appearance.","The drop in performance from averaging nnUNet and MedNeXt probability maps indicates that naive cross-architecture ensembles are unreliable for this task, so future ensembles should align or weight the predictions carefully.","The collapse of larger MedNeXt variants and kernel-5 models on this dataset shows that those configurations need stabilized training procedures or more data before they can be used for small, unbalanced medical segmentation sets."],"supporting_citations":[{"why":"Supplies the nnUNet self-configuring framework used for Task 2 ensembles and all nnUNet experiments.","marker":"[5]"},{"why":"Supplies the MedNeXt architecture, specifically the Small kernel-3 variant that won Task 1.","marker":"[6]"},{"why":"Provides the external brain MRI dataset used for Task 1 pretraining experiments.","marker":"[14]"},{"why":"Defines the aggregated Dice Similarity Coefficient, the challenge ranking metric that the 1st and 8th place scores are based on.","marker":"[15]"},{"why":"Defines the per-sample Dice coefficient used to report per-label mean and standard deviation across cases.","marker":"[16]"}],"fun_headline_variants":["MedNeXt wins head-and-neck MRI tumor segmentation","Single MedNeXt model tops HNTS-MRG24 challenge Task 1","MedNeXt outperforms nnUNet in MRI-guided RT segmentation","Top Dice 0.8254: MedNeXt leads head-and-neck GTV split"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The Task 2 result assumes that the challenge-provided registration aligning the pre-RT images to the mid-RT images is accurate enough that the registered masks and images, when added as input channels, contribute real anatomical correspondence rather than misalignment noise.","fun_headline_variants_meta":{"raw":{"variants":["MedNeXt wins head-and-neck MRI tumor segmentation","Single MedNeXt model tops HNTS-MRG24 challenge Task 1","MedNeXt outperforms nnUNet in MRI-guided RT segmentation","Top Dice 0.8254: MedNeXt leads head-and-neck GTV split"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000254,"raw_usage":{"total_tokens":1621,"prompt_tokens":1053,"completion_tokens":568,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":669,"completion_tokens_details":{"reasoning_tokens":487}},"tokens_in":669,"tokens_out":568,"duration_ms":45194,"temperature":1.0,"reasoning_tokens":487,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:55:36.446163+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the Task 2 pipelines on a cohort with independently verified high-accuracy registration, then repeat the same training with artificially perturbed registration; if the benefit of adding pre-RT masks persists under misalignment, the conclusion is not registration-dependent. A simpler check is to re-run the dataset 504 versus 506 comparison on a test set whose pre-RT-to-mid-RT registration was recomputed with a different algorithm and see whether the mask-channel advantage remains.","supporting_citations":[{"cited_title":"Jaeger, Simon A","cited_arxiv_id":null,"evidence_quote":"Supplies the nnUNet self-configuring framework used for Task 2 ensembles and all nnUNet experiments."},{"cited_title":"Jaeger, and Klaus H","cited_arxiv_id":null,"evidence_quote":"Supplies the MedNeXt architecture, specifically the Small kernel-3 variant that won Task 1."},{"cited_title":"Prior, and Adrien Depeursinge","cited_arxiv_id":null,"evidence_quote":"Defines the aggregated Dice Similarity Coefficient, the challenge ranking metric that the 1st and 8th place scores are based on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the per-sample Dice coefficient used to report per-label mean and standard deviation across cases."}],"review_version":1}