Pith. sign in

REVIEW 3 major objections 4 minor 17 references

RARE-UNet: Resolution-Aligned Routing Entry for Adaptive Medical Image Segmentation

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read RARE-UNet, a resolution-aware UNet with multi-scale gateway blocks, claims stable segmentation accuracy down to 1/8-resolution inputs while running four times faster, outperforming UNet, UNet+Aug, and nnUNet on two brain MRI tasks.

desk verdict Plausible architecture; the unspecified low-resolution Dice evaluation protocol can flip the main result. read the letter →

arxiv 2507.15524 v1 pith:CYEDH3WR submitted 2025-07-21 eess.IV cs.AIcs.CV

classification eess.IVcs.AIcs.CV
keywords resolution-awaresegmentationmulti-scaleUNetadaptiveinferenceroutinglow-resolutionrobustnessbrainMRIconsistencylossDicescore
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a single UNet-style architecture can segment brain MRI images robustly when input resolution varies, instead of assuming fixed high-resolution inputs. It claims that RARE-UNet, by routing downsampled images into deeper encoder layers through multi-scale gateway blocks and aligning features with a consistency loss, keeps Dice scores nearly flat from full resolution down to $1/8$ scale. On hippocampus and brain tumor segmentation benchmarks, it reports the highest average Dice across resolutions (0.84 and 0.65), surpassing standard UNet, a multi-resolution-augmented UNet, and nnUNet, while cutting inference time at lower resolutions. If true, this would make clinical segmentation pipelines more reliable on heterogeneous, non-standardized scans without heavy resampling preprocessing.

What carries the argument

The central mechanism is the multi-scale gateway block (MSB), a convolutional block placed at several encoder depths that transforms a downsampled input into a feature map aligned in shape and semantics with the encoder output at that depth. An MSE consistency loss enforces this alignment during training, so the block learns to reproduce what the full encoder would have computed had it processed the image at full resolution. At inference, a lightweight routing step rounds the input size to the nearest resolution level and activates only the relevant path, skipping shallow layers for low-resolution inputs. The resolution-specific segmentation heads then produce predictions without global resampling.

What would settle it

Recompute the scale-$1/8$ Dice for all models after upsampling every prediction to full resolution before comparing with full-resolution ground truth, keeping the training and routing identical; if the $1/8$-gap between RARE-UNet and nnUNet or UNet+Aug narrows to near zero, the resolution-robustness claim fails.

Watch

Extended reading notes

Core claim

RARE-UNet is a 3D UNet extended with multi-scale gateway blocks that act as resolution-aware entry points at different encoder depths. A full-resolution input enters at the first encoder layer, while inputs downsampled by factors of $1/2$, $1/4$, and $1/8$ bypass shallow layers and are injected at matching depths, sharing the same bottleneck and decoder. Each resolution path has its own segmentation head, and training combines per-path cross-entropy and soft Dice losses with an MSE consistency loss that pushes each gateway block's features toward the encoder features at that depth. The authors report that this design achieves the highest average Dice scores across resolutions—0.84 for hippocampus and 0.65 for brain tumor—with lower variance than baselines, and a roughly $4\times$ speedup when input resolution is halved.

Load-bearing premise

The evaluation loads on the assumption that Dice scores are computed the same way for every model at every resolution; the paper does not clearly state whether RARE-UNet's low-resolution predictions are upsampled to full resolution before scoring, while the baselines are, so the headline low-resolution gains could be an artifact of comparing scores at different output resolutions.

Editorial extensions

If this is right

  • Heterogeneous multi-center MRI data could be segmented without per-scan resampling or padding, since the network accepts a range of native input resolutions.
  • Low-resolution scans, which are cheaper and faster to acquire, would not require a separate low-resolution model or an upsampling pre-processing step.
  • Inference cost scales with input resolution: halving each spatial dimension yields roughly a $4\times$ speedup, which could matter in time-critical clinical settings.
  • The same resolution-routing design could be applied to other encoder-decoder architectures beyond UNet, since the gateway blocks only need a feature-alignment signal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the Dice metric is later confirmed to be computed at native resolution for RARE-UNet, the practical gain over the upsampling-augmented baseline may be mostly about efficiency rather than accuracy; the $1/8$-scale Dice of 0.785 (hippocampus) versus 0.789 for UNet+Aug-Up is close.
  • The routing-by-resolution idea could be extended to anisotropic or variable slice-thickness inputs, which the current $1/2$, $1/4$, $1/8$ isotropic downsampling does not cover.
  • The consistency loss between gateway and encoder features suggests a possible knowledge-distillation view: low-resolution paths are trained to emulate the full-resolution encoder, which may make the gains transferable to other tasks like detection or registration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes RARE-UNet, a 3D UNet extension with multi-scale gateway blocks (MSBs) placed at different encoder depths. Inputs downsampled by factors of 2^d are routed to the MSB at depth d, bypassing shallower layers, and each resolution path has its own segmentation head. Training combines a per-path segmentation loss (cross-entropy and soft Dice) with an MSE consistency loss between MSB features and the corresponding full-resolution encoder features. Experiments compare against plain UNet, a multi-resolution-augmented UNet (UNet+Aug), and nnUNet at full, 1/2, 1/4, and 1/8 resolutions on two Medical Segmentation Decathlon tasks (hippocampus and brain tumor), reporting Dice scores and inference times. The paper claims the highest average Dice across resolutions (0.84 and 0.65) and significantly reduced inference time at low resolutions.

Significance. The architecture is well motivated and the code is publicly released. If the empirical claims hold under a uniform evaluation protocol, RARE-UNet would be a practical single-model solution for variable-resolution medical-image segmentation, with plausible computational savings at low resolutions. The comparison includes strong baselines, including nnUNet and a dedicated multi-resolution augmentation variant, and the consistency-loss mechanism is a reasonable way to align features across scales. The main risk is that the headline quantitative claim depends on an ambiguous metric-resolution protocol and on mean differences that are not tested for significance; both issues are fixable within the scope of a revision.

major comments (3)
  1. [Section 4.2 / Section 3.3] The evaluation protocol for computing Dice at reduced scales is not specified for RARE-UNet. Section 4.2 states that for UNet-based models low-resolution inputs are either padded or upsampled to the original resolution, so baselines are scored against full-resolution ground truth. Section 3.3, however, says each RARE-UNet resolution path is trained against a downsampled version of the ground truth using nearest-neighbor interpolation. The text never states whether RARE-UNet's low-resolution outputs are upsampled to full resolution before computing Dice. If they are scored at native coarse resolution against downsampled ground truth, the abstract's 'highest average Dice across resolution' is confounded: coarse-grid segmentation does not need to resolve fine boundaries, whereas the upsampled baselines are penalized for boundary inaccuracy. This is not a hypothetical concern given the margins in Tables 1-2 (hippocampus overall 0.838 vs 0.835 for UNet+Aug-Up, and 0.785 vs 0.789 at scale 1/8). Please state the evaluation resolution explicitly for all models and, if RARE-UNet was evaluated natively, rescore its low-resolution outputs after upsampling to full resolution against full-resolution ground truth.
  2. [Section 4.3, Tables 1-2] No statistical significance testing is reported, so the central claims of 'highest' and 'consistently outperforms' are not supported at the reported margins. On hippocampus, RARE-UNet's overall Dice of 0.838 ± 0.033 is within one standard deviation of UNet+Aug-Up's 0.835 ± 0.032, and RARE-UNet is numerically worse than UNet+Aug-Up at scale 1/8 (0.785 vs 0.789). On tumor, the overall margin (0.651 vs 0.624) is larger but still unreported for variability across test subjects. Please add paired significance tests (e.g., Wilcoxon signed-rank or bootstrap confidence intervals on per-subject Dice) for the overall average, for each reduced scale, and for inference-time differences in Fig. 6.
  3. [Section 3.2, Section 4.2] The claimed resolution-adaptive behavior is only demonstrated for inputs that are exact dyadic downsamplings of the training resolution, with routing based on the known scale. Real-world resolution variation will not fall exactly on these levels, and the inference-time rule 'rounded to the nearest matching resolution level' (Section 3.2) is never tested for off-grid input sizes or for inputs whose dimensions are not multiples of the routing factors. Since adaptability to heterogeneous resolutions is the paper's central motivation, the evaluation should include at least one non-dyadic or shifted-resolution condition, or the claims should be explicitly delimited to dyadic scales.
minor comments (4)
  1. [Figure 6] The caption says 'S0 through S4' but the plot and the text describe four scales (full, 1/2, 1/4, 1/8); correct the caption or the label count.
  2. [Section 4.3] The text says RARE-UNet achieves 'dynamic parameter scaling with input resolution,' but Fig. 6 reports a fixed parameter count for each model; only the active computational path changes, not the parameter count. Please rephrase.
  3. [Section 4.2] The paper states that over 100 runs were used to select the learning rate and consistency loss weight, but it does not report the selected values or the validation criterion in detail; please report the chosen hyperparameters or point to a configuration file.
  4. [Model Efficiency, Section 4.3] Inference times are reported as averages over 10 runs without hardware details or variance, so the claim of 'significantly reduced inference time' is not quantitatively supported; provide the GPU/CPU specification and error bars or a range.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims are empirical and its objectives are explicit training mechanisms, not reductions to their own inputs.

full rationale

This is an empirical architecture paper with no equation-level derivation chain, so there is no claim that a quantity is predicted from first principles. RARE-UNet's components are defined directly: multi-scale gateway blocks route inputs at 1/2, 1/4, and 1/8 scale to encoder depths; the segmentation loss compares each resolution-specific head to nearest-neighbor-downsampled ground truth; and the consistency loss Lcon = ||f_MSB^(d) - f_enc^(d)||^2_2 explicitly trains MSB features to mimic encoder features. These are training objectives, not fitted parameters renamed as predictions. The only element that could look like a fit is the >100-run hyperparameter sweep on validation performance averaged across scales, but selecting hyperparameters on validation and reporting test Dice is standard model selection, and the test set is held out. The paper cites no prior work by the same authors, so there is no self-citation chain and no imported uniqueness theorem. The headline Dice comparison is externally falsifiable against BraTS and VUMC baselines. The reader's concern that low-resolution Dice may be computed on native coarse grids while baselines are upsampled (Section 4.2 describes upsampling/padding only 'For UNet-based models', while Section 3.3 trains each path against downsampled ground truth) is a potential evaluation-validity confound, but it is not a circular reduction: the low-resolution path is trained and evaluated on the same downsampled target, and no claimed quantity is defined in terms of the result it is supposed to predict. Per the hard rules, evaluation-protocol concerns without a demonstrated equation-level equivalence do not raise the circularity score.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim is empirical, so the ledger contains hand-set and tuned hyperparameters plus domain assumptions rather than derivation constants. No new physical entities are introduced.

free parameters (5)
  • Segmentation loss mixing weight alpha = 0.5
    Set initially to 0.5 in experiments (Section 3.3); chosen by hand, not searched.
  • Consistency loss weight lambda_con = 1 (tuned across >100 runs)
    Initially set to 1; then the paper performed an extensive sweep over learning rate and lambda_con, selecting based on validation performance averaged across scales (Section 4.2).
  • Learning rate = not reported (tuned)
    Tuned in the same sweep described in Section 4.2; exact value not given in the text.
  • Number of resolution levels and routing depths = 4 levels (1, 1/2, 1/4, 1/8)
    Chosen by design; determines the number of MSBs, the routing entries, and the depth of each input path.
  • Input patch sizes = 32x64x32 (hippocampus), 256x256x128 (tumor)
    Computed as the 90th percentile of dataset dimensions rounded up to the nearest power of two (Appendix A); this choice influences all model results.
assumptions (4)
  • domain assumption Simulated low-resolution inputs created by linear interpolation of high-resolution volumes are representative of real-world native low-resolution MRI acquisitions.
    Section 4.2 evaluates robustness by downsampling test scans at scales 1/2, 1/4, and 1/8. If native low-resolution images have different degradation (noise, anisotropy, partial volume), results may not transfer.
  • domain assumption MSE consistency loss between MSB features and full-resolution encoder features is a sufficient training signal to align cross-scale representations.
    Section 3.3 defines L_con = ||f_MSB - f_enc||^2; the method assumes this objective transfers semantic structure across resolutions without a more sophisticated alignment.
  • domain assumption Nearest-neighbor downsampling of ground truth preserves label structure for supervision at low resolutions.
    Used in Section 3.3 to compute segmentation loss at each depth; could introduce aliasing for small structures like the hippocampus.
  • ad hoc to paper Routing based on input size rounded to the nearest matching resolution level is optimal and sufficient.
    Section 3.2 inference: input size is rounded to the nearest matching resolution level; no learned router or confidence mechanism is used, and the paper provides no ablation on routing choices.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RARE-UNet: Resolution-Aligned Routing Entry for Adaptive Medical Image Segmentation." pith.science (2026). https://pith.science/paper/CYEDH3WR

@misc{pith2026250715524,
  author       = {Pith},
  title        = {Pith review of: RARE-UNet: Resolution-Aligned Routing Entry for Adaptive Medical Image Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CYEDH3WR}},
  note         = {Machine review of arXiv:2507.15524}
}
read the original abstract

Accurate segmentation is crucial for clinical applications, but existing models often assume fixed, high-resolution inputs and degrade significantly when faced with lower-resolution data in real-world scenarios. To address this limitation, we propose RARE-UNet, a resolution-aware multi-scale segmentation architecture that dynamically adapts its inference path to the spatial resolution of the input. Central to our design are multi-scale blocks integrated at multiple encoder depths, a resolution-aware routing mechanism, and consistency-driven training that aligns multi-resolution features with full-resolution representations. We evaluate RARE-UNet on two benchmark brain imaging tasks for hippocampus and tumor segmentation. Compared to standard UNet, its multi-resolution augmented variant, and nnUNet, our model achieves the highest average Dice scores of 0.84 and 0.65 across resolution, while maintaining consistent performance and significantly reduced inference time at lower resolutions. These results highlight the effectiveness and scalability of our architecture in achieving resolution-robust segmentation. The codes are available at: https://github.com/simonsejse/RARE-UNet.

Figures

Figures reproduced from arXiv: 2507.15524 by the authors.

Figure 1
Figure 1. Comparison of UNet, UNet++, UNet 3+, and the RARE-UNet architec [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Overview of the RARE-UNet architecture. The model extends a stan [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of a Multi-Scale Gateway Block at depth 1. A downsampled [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of hippocampus segmentation results on a sam [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison of brain tumor segmentation results on a sam [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Trade-off between segmentation accuracy, inference time, and model com [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Distribution of spatial dimensions across the two datasets. Dashed lines [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 14 canonical work pages

  1. [1]

    Alzheimer’s & Dementia20(10) (2024) 7350–7360

    Jack Jr, C.R., Arani, A., Borowski, B.J., Cash, D.M., Crawford, K., Das, S.R., DeCarli, C., Fletcher, E., Fox, N.C., Gunter, J.L., et al.: Overview of ADNI MRI. Alzheimer’s & Dementia20(10) (2024) 7350–7360

  2. [2]

    JournalofCognitive Neuroscience 19(9) (2007) 1498–1507

    Marcus, D.S., Wang, T.H., Parker, J., Csernansky, J.G., Morris, J.C., Buckner, R.L.: Open access series of imaging studies (OASIS): cross-sectional MRI data in young,middleaged,nondemented,anddementedolderadults. JournalofCognitive Neuroscience 19(9) (2007) 1498–1507

  3. [3]

    In: MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, Springer (2015) 234–241 12 S

    Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomed- ical image segmentation. In: MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, Springer (2015) 234–241 12 S. Winther et al

  4. [4]

    In: Deep learning in medical image analysis and multimodal learning for clinical decision support: MICCAI Workshops on DLMIA and ML-CDS, Springer (2018) 3–11

    Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: Unet++: A nested u-net architecture for medical image segmentation. In: Deep learning in medical image analysis and multimodal learning for clinical decision support: MICCAI Workshops on DLMIA and ML-CDS, Springer (2018) 3–11

  5. [5]

    IEEE Transac- tions on Medical Imaging39(6) (2019) 1856–1867

    Zhou, Z., Siddiquee, M.M.R., Tajbakhsh, N., Liang, J.: Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Transac- tions on Medical Imaging39(6) (2019) 1856–1867

  6. [6]

    In: ICASSP, IEEE (2020) 1055–1059

    Huang,H.,Lin,L.,Tong,R.,Hu,H.,Zhang,Q.,Iwamoto,Y.,Han,X.,Chen,Y.W., Wu, J.: Unet 3+: A full-scale connected unet for medical image segmentation. In: ICASSP, IEEE (2020) 1055–1059

  7. [7]

    Neural Networks121 (2020) 74–87

    Ibtehaz, N., Rahman, M.S.: MultiResUNet: Rethinking the U-Net architecture for multimodal biomedical image segmentation. Neural Networks121 (2020) 74–87

  8. [8]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

    Azad,R.,Aghdam,E.K.,Rauland,A.,Jia,Y.,Avval,A.H.,Bozorgpour,A.,Karim- ijafarbigloo, S., Cohen, J.P., Adeli, E., Merhof, D.: Medical image segmentation review: The success of u-net. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024)

Show all 17 references
  1. [9]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Cho, S.J., Ji, S.W., Hong, J.P., Jung, S.W., Ko, S.J.: Rethinking coarse-to-fine ap- proach in single image deblurring. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. (2021) 4641–4650

  2. [10]

    Signal, Image and Video Processing 17(4) (2023) 1143–1151

    Lin, Z., Luo, Q., Jiang, Y., Wang, J., Li, S., Cheng, G., Genrang, Z.: Image defogging based on multi-input and multi-scale UNet. Signal, Image and Video Processing 17(4) (2023) 1143–1151

  3. [11]

    IEEE Transactions on Pattern Analysis and Machine Intelligence 40(4) (2017) 834–848

    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40(4) (2017) 834–848

  4. [12]

    Nature Methods18(2) (2021) 203–211

    Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods18(2) (2021) 203–211

  5. [13]

    Scientific Data 4(1) (2017) 1–13

    Bakas, S., Akbari, H., Sotiras, A., Bilello, M., Rozycki, M., Kirby, J.S., Freymann, J.B., Farahani, K., Davatzikos, C.: Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Scientific Data 4(1) (2017) 1–13

  6. [14]

    arXiv preprint arXiv:1811.02629 (2018)

    Bakas, S., Reyes, M., Jakab, A., Bauer, S., Rempfler, M., Crimi, A., Shinohara, R.T., Berger, C., Ha, S.M., Rozycki, M., et al.: Identifying the best machine learn- ing algorithms for brain tumor segmentation, progression assessment, and over- all survival prediction in the BR...

  7. [15]

    arXiv preprint arXiv:1902.09063 (2019)

    Simpson, A.L., Antonelli, M., Bakas, S., Bilello, M., Farahani, K., Van Ginneken, B., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., et al.: A large an- notated medical image dataset for the development and evaluation of segmentation algorithms. arXiv preprint arXi...

  8. [16]

    Nature Communications13(1) (2022) 4128

    Antonelli, M., Reinke, A., Bakas, S., Farahani, K., Kopp-Schneider, A., Landman, B.A., Litjens, G., Menze, B., Ronneberger, O., Summers, R.M., et al.: The medical segmentation decathlon. Nature Communications13(1) (2022) 4128

  9. [17]

    Biewald, L.: Experiment tracking with weights and biases (2020) Software available from wandb.com. Appendix A Data and Processing Brain Tumor.This task involves segmenting three tumor subregions: enhanc- ing tumor (ET), peritumoral edema (PE), and necrotic core (NC), from mult...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.