REVIEW 4 major objections 5 minor 17 references
Pretraining Deformable Image Registration Networks with Random Images
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Pretraining a registration encoder on pairs of random images transfers to real brain MRI and lets 10% of the training data match full-data accuracy.
desk verdict A sensible encoder-only pretraining idea with promising low-data results, but the headline claim rests on a single random split and needs replication. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the random-image proxy pair: a multi-channel Perlin-noise field collapsed by argmax into an image of random shapes, deformed by two independently generated diffeomorphic fields built from three-channel Perlin noise via scaling-and-squaring. The architecture is asymmetric: during pretraining, lightweight decoders are attached to each encoder stage, each producing a deformation field, and a KL self-distillation term pulls each decoder's Gaussian distribution toward the ensemble distribution. The decoders are discarded after pretraining and only the encoder transfers to the registration backbone. The proxy supplies effectively infinite pairs generated on the fly, which is what lets the encoder be trained for 50 epochs with 3,000 pairs per epoch at 0.012 minutes per pair.
What would settle it
Train the same pretraining recipe on pairs built from two different random-shape images with unrelated Perlin textures, so the proxy includes realistic contrast differences, then fine-tune on the brain MRI task; if the pretrained model no longer beats training from scratch at 10% of the data, the reported benefit depends on identical-content pairs rather than on generic deformation learning.
Extended reading notes
Core claim
Registering random images is a usable proxy task for pretraining a registration foundation model. The paper shows that a randomly generated image, produced by taking an argmax over channels of Perlin noise to form constant-intensity shapes, can be warped by two random diffeomorphic deformations to create fixed-moving pairs; a network trained to register those pairs learns geometry features that transfer to real T1-weighted brain MRI. After pretraining only the encoder of TransMorph with lightweight temporary decoders and an ensemble self-distillation loss, fine-tuning on IXI yields a test Dice of 0.751, slightly above the 0.749 of training from scratch. The data-efficiency consequence is the headline: pretrained models trained on 10% of the data match the full-data scratch model (0.747 vs 0.749), and 5% of the data with pretraining (0.745) beats 20% without (0.742) and approaches 40% without (0.749).
Load-bearing premise
The pretraining pairs are made by applying two random deformations to the same image, so the fixed and moving images have identical texture and contrast; the method assumes the geometry features learned from those content-identical pairs transfer to real images that differ in anatomy and intensity, an assumption the paper does not test.
Editorial extensions
If this is right
- A pretrained encoder reaches the same test Dice as full-data training-from-scratch using 10% of the IXI training set, and 5% with pretraining outperforms 20% without.
- Pretraining on random images also helps an extremely data-limited lung registration task: with only four training subjects, average targeted registration error drops from 2.753 mm to 2.116 mm.
- Encoder-only pretraining is nearly twice as fast per image pair as full-model pretraining (0.012 vs 0.022 min/pair), so the benefit does not require expensive pretraining.
- Random-image pretraining beats no pretraining and is close to pretraining on real brain images from another dataset (0.751 vs 0.753 test Dice), meaning no in-domain data is needed to get most of the gain.
Reading between the lines
- Because the proxy pairs are content-identical, the pretrained encoder probably learns deformation and shape features rather than contrast invariance; a testable extension beyond the paper is that the method helps monomodal T1-to-T1 registration more than cross-modal T1-to-T2 registration.
- The data-efficiency curve suggests that most downstream registration accuracy comes from geometry feature learning in the encoder, with fine-tuning adapting to image appearance; this could be checked by freezing the pretrained encoder and training only the decoder.
- The on-the-fly proxy generator makes scaling to larger encoders cheap, but the ceiling may be set by how much texture variation can be injected into the random images without breaking the self-distillation objective.
- A direct stress test would replace the identical-content pairs with pairs made from independently generated random images, forcing the proxy to handle contrast differences and better matching real registration conditions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a pretraining strategy for deformable image registration networks in which only the encoder is pretrained on a proxy task: registering pairs of randomly generated synthetic images. Random images are created by collapsing multi-channel Perlin noise into random shapes, and image pairs are formed by applying two independently generated diffeomorphic deformations to the same random image. Lightweight decoders are attached to each encoder stage during pretraining, along with a self-distillation term based on KL divergence; after pretraining the decoders are discarded and the encoder is transferred to a TransMorph backbone for fine-tuning on downstream registration tasks. The authors report experiments on IXI brain MRI showing improved test Dice (0.751 vs. 0.749 without pretraining), reduced data requirements (10% of the training data with pretraining reportedly matching 100% without it, p=0.026), faster convergence, and a lung CT application with only four training subjects.
Significance. If the data-efficiency claim were robustly supported, this would be a practically valuable result: it would show that a registration encoder can be pretrained entirely on synthetic, on-the-fly generated image pairs, requiring no in-domain data, and that this pretraining reduces the amount of downstream domain data needed. The paper also provides useful comparisons against in-domain pretraining and against existing registration methods, and it makes code available. However, the central quantitative claim currently rests on a single random split of the IXI data at each data fraction, with no repeated subsampling or seeds, and the reported effect sizes are very small relative to the standard deviations (all Dice values lie between 0.733 and 0.751, with standard deviations near 0.13). The full-data improvement is 0.002 Dice, which is within noise. The paper does not provide machine-checked proofs or parameter-free derivations; its value is empirical, so the statistical support for the headline claims must be considerably stronger.
major comments (4)
- [Section 3, 'Reducing Data Requirements'; Appendix E, Table 2] The central data-efficiency claim is not supported with adequate statistical evidence. Each data fraction (100%, 80%, 60%, 40%, 20%, 10%, 5%) appears to correspond to a single random subset of IXI, with no repeated subsampling, no multiple seeds, and no confidence intervals over splits. Given that all non-pretrained Dice values are in the narrow range 0.733–0.749 and the pretrained values in 0.745–0.751, a single favorable subset or initialization can create the reported pattern. The authors should repeat the low-data training over several independent subsets and seeds and report mean, standard deviation, and confidence intervals for each data fraction.
- [Appendix E, Table 2] The statement that 'with only 10% of the training data, the pretrained model achieves comparable performance to the non-pretrained model trained on the full dataset (0.747 vs. 0.749), while with a statistically significant difference (p=0.026)' is internally contradictory as written. A Wilcoxon signed-rank test with p=0.026 indicates a statistically significant difference, and the direction of the means is against the pretrained model (0.747 < 0.749). If the test is paired across test subjects, the appropriate reading is that the 10%-pretrained model is significantly worse than the full-data model on this test set, not that it 'matches' it. The authors need to report the effect size, the direction of the paired difference, and a multiple-comparison correction if several fractions are tested. This is load-bearing because the paper's headline claim is reduced data requirements.
- [Section 3, 'Ablation Studies'; Appendix D, Table 1 and Figure 5] The text states that including KL loss 'slightly lowers validation Dice scores' and that Dice-loss pretraining improves upon the baseline in validation Dice, but Table 1 reports identical test Dice for SD w/o KL and SD w/ KL (0.751 vs. 0.751) and equal test Dice for SD w/ Dice and Baseline (0.749 vs. 0.749). The caption also asserts that all pretraining models significantly outperform the baseline, yet no paired-test results are shown and at least one entry has the same mean as the baseline. If the claims refer to validation curves in Figure 5 rather than the test-set table, that distinction should be stated explicitly, and the apparent discrepancy between validation and test behavior should be reconciled.
- [Section 2, 'Pretraining Proxy Task'; Figure 3] The proxy task pairs two diffeomorphic deformations of the same random-shape image, so the fixed and moving images have identical texture and contrast and differ only in geometry. Real registration pairs, including the IXI brain MRIs used for fine-tuning, contain different intensity distributions and contrast differences across subjects. The paper does not discuss or test whether the deformation-related features learned under content-identical pairs transfer to contrast-different pairs. This question is secondary to the low-data comparison, but it is central to the mechanism and generalization claims, and a simple control experiment (e.g., generating fixed and moving images from independent random images, or adding contrast augmentation during pretraining) would clarify whether the benefit comes from geometry learning or from a more general representation.
minor comments (5)
- [Equation (1)] The ensemble distribution N(µ_ϕens, σ²_ϕens) is invoked in the KL term but its computation from the decoder outputs is not defined; please specify how the ensemble mean and variance are aggregated.
- [Table 1 caption] The caption states 'All models with pretraining significantly outperform the Baseline (p<0.05, Wilcoxon signed-rank test)' without reporting the test statistic or effect sizes; given that the Dice-loss model has the same mean as the baseline, the paired nature of the test and the magnitude of the difference should be reported.
- [Appendix E, Figure 6 and Table 2] The description does not state how the reduced data fractions were drawn (random subsample? stratified by structure? same subset for pretrained and from-scratch models?), nor whether the same subsets were reused across conditions; these details are essential for interpreting the low-data curves.
- [Appendix H] The lung experiment uses four subjects for training and six for validation, with no held-out test set, and the model selection is performed on the validation set; this should be acknowledged as a limitation, and the reported TRE improvements should be interpreted accordingly.
- [General] The term 'foundation model' is used for a single backbone pretrained on one synthetic task and evaluated on two datasets; a more conservative term such as 'pretrained registration encoder' would better match the evidence presented.
Circularity Check
No significant circularity: the pretraining results are empirical comparisons against external baselines and are not reducible to the paper's inputs.
full rationale
The paper's central claim is that encoder pretraining on random image registration improves downstream registration accuracy and data efficiency. This is established through direct experiments on IXI and 4DCT datasets, compared against external baselines such as deedsBCV, ConvexAdam, and SynthMorph, as well as against in-domain brain pretraining. The pretraining proxy task is not defined in terms of the downstream metric; it uses random Perlin-noise images and known diffeomorphic deformations, with the network optimized by NCC and regularization losses. No parameter is fitted to the downstream test set and then reported as a prediction. The only self-citations are to the authors' own TransMorph backbone, the NDV metric, and a prior survey, none of which is used to justify the main empirical finding. The selection of NCC loss for pretraining is explained by its match to the fine-tuning objective, but this is a design choice rather than a circular derivation. The results are self-contained and externally benchmarked, so no circular step can be exhibited.
Assumptions & free parameters
free parameters (2)
- KL loss weight eta =
1e-7
- NCC loss weight lambda =
1
assumptions (3)
- domain assumption Random images with constant-intensity Perlin-noise shapes provide a useful pretraining signal for medical image registration.
- domain assumption The encoder of the registration network can be pretrained independently with temporary decoders and retain useful representations.
- domain assumption NCC is an appropriate similarity measure for both random shapes and medical images.
Cite this review
Pith. "Pith review of Pretraining Deformable Image Registration Networks with Random Images." pith.science (2026). https://pith.science/paper/RR3GYSDE
@misc{pith2026250524167,
author = {Pith},
title = {Pith review of: Pretraining Deformable Image Registration Networks with Random Images},
year = {2026},
howpublished = {\url{https://pith.science/paper/RR3GYSDE}},
note = {Machine review of arXiv:2505.24167}
}
read the original abstract
Recent advances in deep learning-based medical image registration have shown that training deep neural networks~(DNNs) does not necessarily require medical images. Previous work showed that DNNs trained on randomly generated images with carefully designed noise and contrast properties can still generalize well to unseen medical data. Building on this insight, we propose using registration between random images as a proxy task for pretraining a foundation model for image registration. Empirical results show that our pretraining strategy improves registration accuracy, reduces the amount of domain-specific data needed to achieve competitive performance, and accelerates convergence during downstream training, thereby enhancing computational efficiency.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
A fast diffeomorphic image registration algorithm
John Ashburner. A fast diffeomorphic image registration algorithm. Neuroimage, 38 0 (1): 0 95--113, 2007
work page 2007
-
[2]
Four-dimensional deformable image registration using trajectory modeling
Edward Castillo, Richard Castillo, Josue Martinez, Maithili Shenoy, and Thomas Guerrero. Four-dimensional deformable image registration using trajectory modeling. Physics in Medicine & Biology, 55 0 (1): 0 305, 2009 a
work page 2009
-
[3]
Richard Castillo, Edward Castillo, Rudy Guerra, Valen E Johnson, Travis McPhail, Amit K Garg, and Thomas Guerrero. A framework for evaluation of deformable image registration spatial accuracy using large landmark point sets. Physics in Medicine & Biology, 54 0 (7): 0 1849, 2009 b
work page 2009
-
[4]
Unsupervised learning of diffeomorphic image registration via transmorph
Junyu Chen, Eric C Frey, and Yong Du. Unsupervised learning of diffeomorphic image registration via transmorph. In International Workshop on Biomedical Image Registration, pages 96--102. Springer, 2022 a
work page 2022
-
[5]
Transmorph: Transformer for unsupervised medical image registration
Junyu Chen, Eric C Frey, Yufan He, William P Segars, Ye Li, and Yong Du. Transmorph: Transformer for unsupervised medical image registration. Medical image analysis, 82: 0 102615, 2022 b
work page 2022
-
[6]
Junyu Chen, Yihao Liu, Shuwen Wei, Zhangxing Bian, Shalini Subramanian, Aaron Carass, Jerry L Prince, and Yong Du. A survey on deep learning in medical image registration: New technologies, uncertainty, evaluation metrics, and beyond. Medical Image Analysis, page 103385, 2024
work page 2024
- [7]
-
[8]
Masked autoencoders are scalable vision learners
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll \'a r, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000--16009, 2022
2022
Show all 17 references
-
[9]
Mrf-based deformable registration and ventilation estimation of lung ct
Mattias P Heinrich, Mark Jenkinson, Michael Brady, and Julia A Schnabel. Mrf-based deformable registration and ventilation estimation of lung ct. IEEE transactions on medical imaging, 32 0 (7): 0 1239--1248, 2013
2013
-
[10]
Synthmorph: learning contrast-invariant registration without acquired images
Malte Hoffmann, Benjamin Billot, Douglas N Greve, Juan Eugenio Iglesias, Bruce Fischl, and Adrian V Dalca. Synthmorph: learning contrast-invariant registration without acquired images. IEEE transactions on medical imaging, 41 0 (3): 0 543--558, 2021
2021
-
[11]
Segment anything
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015--4026, 2023
2023
-
[12]
On finite difference jacobian computation in deformable image registration
Yihao Liu, Junyu Chen, Shuwen Wei, Aaron Carass, and Jerry Prince. On finite difference jacobian computation in deformable image registration. International Journal of Computer Vision, 132 0 (9): 0 3678--3688, 2024
2024
-
[13]
The nimh intramural healthy volunteer dataset: A comprehensive meg, mri, and behavioral resource
Allison C Nugent, Adam G Thomas, Margaret Mahoney, Alison Gibbons, Jarrod T Smith, Antoinette J Charles, Jacob S Shaw, Jeffrey D Stout, Anna M Namyst, Arshitha Basavaraj, et al. The nimh intramural healthy volunteer dataset: A comprehensive meg, mri, and behavioral resource. S...
2022
-
[14]
Dinov2: Learning robust visual features without supervision
Maxime Oquab, Timoth \'e e Darcet, Th \'e o Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023
2023 arXiv
-
[15]
Improving noise
Ken Perlin. Improving noise. In Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pages 681--682, 2002
2002
-
[16]
Convexadam: Self-configuring dual-optimisation-based 3d multitask medical image registration
Hanna Siebert, Christoph Gro br \"o hmer, Lasse Hansen, and Mattias P Heinrich. Convexadam: Self-configuring dual-optimisation-based 3d multitask medical image registration. IEEE Transactions on Medical Imaging, 2024
2024
-
[17]
Self-distillation: Towards efficient and compact neural networks
Linfeng Zhang, Chenglong Bao, and Kaisheng Ma. Self-distillation: Towards efficient and compact neural networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44 0 (8): 0 4388--4403, 2021
2021
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.