REVIEW 3 major objections 4 minor 17 references
RARE-UNet: Resolution-Aligned Routing Entry for Adaptive Medical Image Segmentation
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read RARE-UNet, a resolution-aware UNet with multi-scale gateway blocks, claims stable segmentation accuracy down to 1/8-resolution inputs while running four times faster, outperforming UNet, UNet+Aug, and nnUNet on two brain MRI tasks.
desk verdict Plausible architecture; the unspecified low-resolution Dice evaluation protocol can flip the main result. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the multi-scale gateway block (MSB), a convolutional block placed at several encoder depths that transforms a downsampled input into a feature map aligned in shape and semantics with the encoder output at that depth. An MSE consistency loss enforces this alignment during training, so the block learns to reproduce what the full encoder would have computed had it processed the image at full resolution. At inference, a lightweight routing step rounds the input size to the nearest resolution level and activates only the relevant path, skipping shallow layers for low-resolution inputs. The resolution-specific segmentation heads then produce predictions without global resampling.
What would settle it
Recompute the scale-$1/8$ Dice for all models after upsampling every prediction to full resolution before comparing with full-resolution ground truth, keeping the training and routing identical; if the $1/8$-gap between RARE-UNet and nnUNet or UNet+Aug narrows to near zero, the resolution-robustness claim fails.
Extended reading notes
Core claim
RARE-UNet is a 3D UNet extended with multi-scale gateway blocks that act as resolution-aware entry points at different encoder depths. A full-resolution input enters at the first encoder layer, while inputs downsampled by factors of $1/2$, $1/4$, and $1/8$ bypass shallow layers and are injected at matching depths, sharing the same bottleneck and decoder. Each resolution path has its own segmentation head, and training combines per-path cross-entropy and soft Dice losses with an MSE consistency loss that pushes each gateway block's features toward the encoder features at that depth. The authors report that this design achieves the highest average Dice scores across resolutions—0.84 for hippocampus and 0.65 for brain tumor—with lower variance than baselines, and a roughly $4\times$ speedup when input resolution is halved.
Load-bearing premise
The evaluation loads on the assumption that Dice scores are computed the same way for every model at every resolution; the paper does not clearly state whether RARE-UNet's low-resolution predictions are upsampled to full resolution before scoring, while the baselines are, so the headline low-resolution gains could be an artifact of comparing scores at different output resolutions.
Editorial extensions
If this is right
- Heterogeneous multi-center MRI data could be segmented without per-scan resampling or padding, since the network accepts a range of native input resolutions.
- Low-resolution scans, which are cheaper and faster to acquire, would not require a separate low-resolution model or an upsampling pre-processing step.
- Inference cost scales with input resolution: halving each spatial dimension yields roughly a $4\times$ speedup, which could matter in time-critical clinical settings.
- The same resolution-routing design could be applied to other encoder-decoder architectures beyond UNet, since the gateway blocks only need a feature-alignment signal.
Reading between the lines
- If the Dice metric is later confirmed to be computed at native resolution for RARE-UNet, the practical gain over the upsampling-augmented baseline may be mostly about efficiency rather than accuracy; the $1/8$-scale Dice of 0.785 (hippocampus) versus 0.789 for UNet+Aug-Up is close.
- The routing-by-resolution idea could be extended to anisotropic or variable slice-thickness inputs, which the current $1/2$, $1/4$, $1/8$ isotropic downsampling does not cover.
- The consistency loss between gateway and encoder features suggests a possible knowledge-distillation view: low-resolution paths are trained to emulate the full-resolution encoder, which may make the gains transferable to other tasks like detection or registration.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RARE-UNet, a 3D UNet extension with multi-scale gateway blocks (MSBs) placed at different encoder depths. Inputs downsampled by factors of 2^d are routed to the MSB at depth d, bypassing shallower layers, and each resolution path has its own segmentation head. Training combines a per-path segmentation loss (cross-entropy and soft Dice) with an MSE consistency loss between MSB features and the corresponding full-resolution encoder features. Experiments compare against plain UNet, a multi-resolution-augmented UNet (UNet+Aug), and nnUNet at full, 1/2, 1/4, and 1/8 resolutions on two Medical Segmentation Decathlon tasks (hippocampus and brain tumor), reporting Dice scores and inference times. The paper claims the highest average Dice across resolutions (0.84 and 0.65) and significantly reduced inference time at low resolutions.
Significance. The architecture is well motivated and the code is publicly released. If the empirical claims hold under a uniform evaluation protocol, RARE-UNet would be a practical single-model solution for variable-resolution medical-image segmentation, with plausible computational savings at low resolutions. The comparison includes strong baselines, including nnUNet and a dedicated multi-resolution augmentation variant, and the consistency-loss mechanism is a reasonable way to align features across scales. The main risk is that the headline quantitative claim depends on an ambiguous metric-resolution protocol and on mean differences that are not tested for significance; both issues are fixable within the scope of a revision.
major comments (3)
- [Section 4.2 / Section 3.3] The evaluation protocol for computing Dice at reduced scales is not specified for RARE-UNet. Section 4.2 states that for UNet-based models low-resolution inputs are either padded or upsampled to the original resolution, so baselines are scored against full-resolution ground truth. Section 3.3, however, says each RARE-UNet resolution path is trained against a downsampled version of the ground truth using nearest-neighbor interpolation. The text never states whether RARE-UNet's low-resolution outputs are upsampled to full resolution before computing Dice. If they are scored at native coarse resolution against downsampled ground truth, the abstract's 'highest average Dice across resolution' is confounded: coarse-grid segmentation does not need to resolve fine boundaries, whereas the upsampled baselines are penalized for boundary inaccuracy. This is not a hypothetical concern given the margins in Tables 1-2 (hippocampus overall 0.838 vs 0.835 for UNet+Aug-Up, and 0.785 vs 0.789 at scale 1/8). Please state the evaluation resolution explicitly for all models and, if RARE-UNet was evaluated natively, rescore its low-resolution outputs after upsampling to full resolution against full-resolution ground truth.
- [Section 4.3, Tables 1-2] No statistical significance testing is reported, so the central claims of 'highest' and 'consistently outperforms' are not supported at the reported margins. On hippocampus, RARE-UNet's overall Dice of 0.838 ± 0.033 is within one standard deviation of UNet+Aug-Up's 0.835 ± 0.032, and RARE-UNet is numerically worse than UNet+Aug-Up at scale 1/8 (0.785 vs 0.789). On tumor, the overall margin (0.651 vs 0.624) is larger but still unreported for variability across test subjects. Please add paired significance tests (e.g., Wilcoxon signed-rank or bootstrap confidence intervals on per-subject Dice) for the overall average, for each reduced scale, and for inference-time differences in Fig. 6.
- [Section 3.2, Section 4.2] The claimed resolution-adaptive behavior is only demonstrated for inputs that are exact dyadic downsamplings of the training resolution, with routing based on the known scale. Real-world resolution variation will not fall exactly on these levels, and the inference-time rule 'rounded to the nearest matching resolution level' (Section 3.2) is never tested for off-grid input sizes or for inputs whose dimensions are not multiples of the routing factors. Since adaptability to heterogeneous resolutions is the paper's central motivation, the evaluation should include at least one non-dyadic or shifted-resolution condition, or the claims should be explicitly delimited to dyadic scales.
minor comments (4)
- [Figure 6] The caption says 'S0 through S4' but the plot and the text describe four scales (full, 1/2, 1/4, 1/8); correct the caption or the label count.
- [Section 4.3] The text says RARE-UNet achieves 'dynamic parameter scaling with input resolution,' but Fig. 6 reports a fixed parameter count for each model; only the active computational path changes, not the parameter count. Please rephrase.
- [Section 4.2] The paper states that over 100 runs were used to select the learning rate and consistency loss weight, but it does not report the selected values or the validation criterion in detail; please report the chosen hyperparameters or point to a configuration file.
- [Model Efficiency, Section 4.3] Inference times are reported as averages over 10 runs without hardware details or variance, so the claim of 'significantly reduced inference time' is not quantitatively supported; provide the GPU/CPU specification and error bars or a range.
Circularity Check
No significant circularity: the paper's claims are empirical and its objectives are explicit training mechanisms, not reductions to their own inputs.
full rationale
This is an empirical architecture paper with no equation-level derivation chain, so there is no claim that a quantity is predicted from first principles. RARE-UNet's components are defined directly: multi-scale gateway blocks route inputs at 1/2, 1/4, and 1/8 scale to encoder depths; the segmentation loss compares each resolution-specific head to nearest-neighbor-downsampled ground truth; and the consistency loss Lcon = ||f_MSB^(d) - f_enc^(d)||^2_2 explicitly trains MSB features to mimic encoder features. These are training objectives, not fitted parameters renamed as predictions. The only element that could look like a fit is the >100-run hyperparameter sweep on validation performance averaged across scales, but selecting hyperparameters on validation and reporting test Dice is standard model selection, and the test set is held out. The paper cites no prior work by the same authors, so there is no self-citation chain and no imported uniqueness theorem. The headline Dice comparison is externally falsifiable against BraTS and VUMC baselines. The reader's concern that low-resolution Dice may be computed on native coarse grids while baselines are upsampled (Section 4.2 describes upsampling/padding only 'For UNet-based models', while Section 3.3 trains each path against downsampled ground truth) is a potential evaluation-validity confound, but it is not a circular reduction: the low-resolution path is trained and evaluated on the same downsampled target, and no claimed quantity is defined in terms of the result it is supposed to predict. Per the hard rules, evaluation-protocol concerns without a demonstrated equation-level equivalence do not raise the circularity score.
Assumptions & free parameters
free parameters (5)
- Segmentation loss mixing weight alpha =
0.5
- Consistency loss weight lambda_con =
1 (tuned across >100 runs)
- Learning rate =
not reported (tuned)
- Number of resolution levels and routing depths =
4 levels (1, 1/2, 1/4, 1/8)
- Input patch sizes =
32x64x32 (hippocampus), 256x256x128 (tumor)
assumptions (4)
- domain assumption Simulated low-resolution inputs created by linear interpolation of high-resolution volumes are representative of real-world native low-resolution MRI acquisitions.
- domain assumption MSE consistency loss between MSB features and full-resolution encoder features is a sufficient training signal to align cross-scale representations.
- domain assumption Nearest-neighbor downsampling of ground truth preserves label structure for supervision at low resolutions.
- ad hoc to paper Routing based on input size rounded to the nearest matching resolution level is optimal and sufficient.
Cite this review
Pith. "Pith review of RARE-UNet: Resolution-Aligned Routing Entry for Adaptive Medical Image Segmentation." pith.science (2026). https://pith.science/paper/CYEDH3WR
@misc{pith2026250715524,
author = {Pith},
title = {Pith review of: RARE-UNet: Resolution-Aligned Routing Entry for Adaptive Medical Image Segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/CYEDH3WR}},
note = {Machine review of arXiv:2507.15524}
}
read the original abstract
Accurate segmentation is crucial for clinical applications, but existing models often assume fixed, high-resolution inputs and degrade significantly when faced with lower-resolution data in real-world scenarios. To address this limitation, we propose RARE-UNet, a resolution-aware multi-scale segmentation architecture that dynamically adapts its inference path to the spatial resolution of the input. Central to our design are multi-scale blocks integrated at multiple encoder depths, a resolution-aware routing mechanism, and consistency-driven training that aligns multi-resolution features with full-resolution representations. We evaluate RARE-UNet on two benchmark brain imaging tasks for hippocampus and tumor segmentation. Compared to standard UNet, its multi-resolution augmented variant, and nnUNet, our model achieves the highest average Dice scores of 0.84 and 0.65 across resolution, while maintaining consistent performance and significantly reduced inference time at lower resolutions. These results highlight the effectiveness and scalability of our architecture in achieving resolution-robust segmentation. The codes are available at: https://github.com/simonsejse/RARE-UNet.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Alzheimer’s & Dementia20(10) (2024) 7350–7360
Jack Jr, C.R., Arani, A., Borowski, B.J., Cash, D.M., Crawford, K., Das, S.R., DeCarli, C., Fletcher, E., Fox, N.C., Gunter, J.L., et al.: Overview of ADNI MRI. Alzheimer’s & Dementia20(10) (2024) 7350–7360
work page 2024
-
[2]
JournalofCognitive Neuroscience 19(9) (2007) 1498–1507
Marcus, D.S., Wang, T.H., Parker, J., Csernansky, J.G., Morris, J.C., Buckner, R.L.: Open access series of imaging studies (OASIS): cross-sectional MRI data in young,middleaged,nondemented,anddementedolderadults. JournalofCognitive Neuroscience 19(9) (2007) 1498–1507
work page 2007
-
[3]
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomed- ical image segmentation. In: MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, Springer (2015) 234–241 12 S. Winther et al
work page 2015
-
[4]
Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support: MICCAI Workshops on DLMIA and ML-CDS, Springer (2018) 3–11
work page 2018
-
[5]
IEEE Transac- tions on Medical Imaging39(6) (2019) 1856–1867
Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transac- tions on Medical Imaging39(6) (2019) 1856–1867
work page 2019
-
[6]
In: ICASSP, IEEE (2020) 1055–1059
Huang,H.,Lin,L.,Tong,R.,Hu,H.,Zhang,Q.,Iwamoto,Y.,Han,X.,Chen,Y.W., Wu, J.: Unet 3+: A full-scale connected unet for medical image segmentation. In: ICASSP, IEEE (2020) 1055–1059
work page 2020
-
[7]
Neural Networks121 (2020) 74–87
Ibtehaz, N., Rahman, M.S.: MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Networks121 (2020) 74–87
work page 2020
-
[8]
IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)
Azad,R.,Aghdam,E.K.,Rauland,A.,Jia,Y.,Avval,A.H.,Bozorgpour,A.,Karim- ijafarbigloo, S., Cohen, J.P., Adeli, E., Merhof, D.: Medical image segmentation review: The success of u-net. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)
2024
Show all 17 references
-
[9]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision
Cho, S.J., Ji, S.W., Hong, J.P., Jung, S.W., Ko, S.J.: Rethinking coarse-to-fine ap- proach in single image deblurring. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. (2021) 4641–4650
2021
-
[10]
Signal, Image and Video Processing 17(4) (2023) 1143–1151
Lin, Z., Luo, Q., Jiang, Y., Wang, J., Li, S., Cheng, G., Genrang, Z.: Image defogging based on multi-input and multi-scale UNet. Signal, Image and Video Processing 17(4) (2023) 1143–1151
2023
-
[11]
IEEE Transactions on Pattern Analysis and Machine Intelligence 40(4) (2017) 834–848
Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(4) (2017) 834–848
2017
-
[12]
Nature Methods18(2) (2021) 203–211
Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods18(2) (2021) 203–211
2021
-
[13]
Scientific Data 4(1) (2017) 1–13
Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., Freymann, J.B., Farahani, K., Davatzikos, C.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data 4(1) (2017) 1–13
2017
-
[14]
arXiv preprint arXiv:1811.02629 (2018)
Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R.T., Berger, C., Ha, S.M., Rozycki, M., et al.: Identifying the best machine learn- ing algorithms for brain tumor segmentation, progression assessment, and over- all survival prediction in the BR...
2018 arXiv
-
[15]
arXiv preprint arXiv:1902.09063 (2019)
Simpson, A.L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., Van Ginneken, B., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., et al.: A large an- notated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXi...
2019 arXiv
-
[16]
Nature Communications13(1) (2022) 4128
Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., Ronneberger, O., Summers, R.M., et al.: The medical segmentation decathlon. Nature Communications13(1) (2022) 4128
2022
-
[17]
Biewald, L.: Experiment tracking with weights and biases (2020) Software available from wandb.com. Appendix A Data and Processing Brain Tumor.This task involves segmenting three tumor subregions: enhanc- ing tumor (ET), peritumoral edema (PE), and necrotic core (NC), from mult...
2020
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.