REVIEW 3 major objections 4 minor 10 references
A Framework for Joint Affine and Diffeomorphic Image Registration
T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read A joint optimization beats affine-then-diffeomorphic registration on brain MRIs.
desk verdict A genuine, narrowly scoped LDDMM contribution whose headline claim about joint affine estimation is underdetermined by the 3D experiment, but which deserves a serious referee if the key derivations and controls are tightened. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying object is the semidirect product group $\mathrm{Aff}(\mathbb{R}^d)\ltimes \mathrm{Diff}$, acting on an augmented image space that carries the current affine frame alongside the image. From this the paper derives optimality conditions for joint affine-diffeomorphic trajectories, uses a change of variables that decouples the affine motion from the diffeomorphic image evolution, and stabilises the joint optimization with two mechanisms: progressive covector activation, which gradually unlocks more complex affine degrees of freedom, and a variational weighting schedule driven by a rate estimator of the affine term, which forces the affine component to settle before handing over to the diffeomorphic part.
What would settle it
Take one of the 54 brain-MRI registration pairs, run the joint optimization, and least-squares fit the final diffeomorphic velocity field to the affine family $x\mapsto Mx+b$ over the image domain; if the fitted affine projection has magnitude comparable to the estimated affine motion itself on subjects where the affine stage looks poor, the claimed separation between components is refuted and the joint affine cannot be credited with the improvement.
Extended reading notes
Core claim
At the core of the paper is a reversal in what makes a good affine initialization. The authors construct two unsupervised variational models: a Full Affine model in which the affine part is an unrestricted $GL(d)\ltimes \mathbb{R}^d$ motion, and a Decomposed Affine model in which the affine part is factorised into rotation, translation, and anisotropic scaling. For each model they derive Hamiltonian geodesic equations via the maximum principle of optimal control, and introduce a change of variables that removes the affine controls from the image evolution equation, so the two components can be read off separately. In the brain MRI experiments the affine-only part of the joint solution scores slightly below the affine-only part of the sequential baseline (median Dice 0.728 versus 0.731), yet once the diffeomorphic part is applied the joint solution wins on 49 of 54 subjects, and after refinement on 53 of 54. The paper takes this as direct evidence that affine-only overlap is a poor predictor of final registration quality: an affine optimum can leave a residual that is expensive for a diffeomorphism to absorb, while a joint affine is selected to leave a cheap residual. On 2D synthetic targets, the joint scheme also avoids the fold-over and grid-crossing pathologies the sequential pipeline exhibits.
Load-bearing premise
The load-bearing premise is that the affine and diffeomorphic components actually separate during the joint optimization; the paper explicitly concedes it does not prevent the diffeomorphic part from reproducing global affine motion, so if the warp absorbs affine modes early, the affine-only scores and the 'better starting point' interpretation lose their basis.
Editorial extensions
If this is right
- Joint affine-diffeomorphic LDDMM is a workable unsupervised alternative to sequential affine-then-nonlinear pipelines, producing a single geodesic trajectory rather than a composed endpoint map.
- Affine-only Dice is not a reliable predictor of final registration quality: an affine optimum can be worse for diffeomorphic refinement than an affine deliberately chosen jointly.
- On the 54-subject brain MRI task, the joint result beats the sequential baseline on 49 of 54 subjects (53 of 54 after refinement) and beats both deep-learning foundation-model baselines on 53 of 54 (54 of 54 after refinement), with no training data.
- Even in a joint formulation, the optimization trajectory must be actively controlled so the affine component resolves global motion before the more expressive diffeomorphic component takes over.
- The joint scheme avoids the pathological grid-crossing deformations that appear in sequential affine-then-LDDMM registration on synthetic targets.
Reading between the lines
- A natural next test is to enforce the separation the paper leaves implicit, for example by penalising the projection of the diffeomorphic vector field onto affine modes; if the affine component then becomes identifiable at the population level, the model could support studies of global versus local anatomical variability.
- Because the framework returns a full continuous geodesic rather than a single warp, the same formulation could feed trajectory-based analyses, such as parallel transport, atlas construction, or statistics on initial momenta, for which sequential pipelines provide no trajectory.
- A testable prediction follows from the paper's own logic: the joint advantage over sequential registration should shrink when the initial misalignment is already small, since a near-perfect affine pre-alignment leaves little distinction between a cheap and an expensive residual for the diffeomorphic stage.
- The variational handover schedule, which weights the affine term strongly until its rate of improvement stalls and then decays the weight, is a generic mechanism that could be transplanted to any joint estimator in which one component is far more expressive than the other.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces a joint affine-diffeomorphic registration framework within LDDMM. Two variational models are proposed: Full Affine (FA), where the deformation is generated by affine-plus-diffeomorphic controls, and Decomposed Affine (DA), where the affine part is restricted to rotations, translations, and anisotropic scalings. The authors derive Hamiltonian/geodesic equations for both models, introduce a change of variables intended to decouple the affine and diffeomorphic components, and present a numerical implementation with a progressive covector activation scheme and a variational weighting schedule. The method is evaluated on 2D toy examples and on 54 IXI brain MR volumes registered to the MNI template, comparing against FLIRT+LDDMM, CARL, and uniGradICON. The central claim is that joint optimization yields an affine component that is a better starting point for diffeomorphic refinement than an affine estimated separately.
Significance. If the central claim holds, the paper is a useful contribution to classical, unsupervised registration: it provides a geodesic formulation in which affine and diffeomorphic components are estimated concurrently, avoids the double interpolation of sequential pipelines, and outperforms the tested baselines on the IXI brain MRI task. The empirical section is unusually careful: paired one-sided Wilcoxon tests with Holm-Bonferroni correction, effect sizes, per-subject wins, and a public implementation are all provided. However, the central claim is currently underdetermined by the experimental design: the head-to-head comparison of "ours (refinement)" with FLIRT+LDDMM varies both the origin of the affine and the affine objective, so the reported advantage cannot be attributed specifically to joint estimation. The theoretical derivation also has a nontrivial proof gap in Proposition 2.6, and the paper explicitly concedes that the diffeomorphic component is not theoretically prevented from reproducing affine modes. These issues are fixable but load-bearing.
major comments (3)
- [Section 3.3, Table 2] The central claim that joint optimization produces an affine registration that is a better starting point for diffeomorphic matching is not isolateable from the data. The comparison "ours (refinement)" vs. FLIRT+LDDMM holds the LDDMM refinement stage fixed, but the two affines differ in two ways simultaneously: one is produced by the joint Demeter/SSD optimization and the other by the separate FLIRT pipeline, whose affine-only objective is not SSD. The paper itself notes in the paragraph after Table 2 that FLIRT attains a significantly higher affine-only Dice than ours (reverse p = 2.3e-3). This means the observed final reversal could be caused by metric consistency between the affine stage and the subsequent SSD-based LDDMM refinement, rather than by joint estimation per se. The toy experiments in Section 3.2 include a successive mode with the same affine model and the diffeomorphic component deactivated, but no equivalent 3D Dice comparison with a same-objective, separately estimated affine is reported. I request an additional control arm: estimate an affine separately with the same SSD objective and optimizer used in the joint stage, then apply the same LDDMM refinement. If that control matches "ours (refinement)", the claimed advantage of joint estimation over separate estimation collapses to an affine-objective consistency effect.
- [Section 4] The paper concedes in the conclusion that "our proposed method does not theoretically ensure that the diffeomorphic component cannot reproduce global affine deformations." This concession directly affects the interpretation of the affine-only Dice comparisons and the claim that the joint affine is a better starting point. The variational weighting schedule in Section 3.1 is designed to encourage the affine component to absorb global motion first, but it does not provide a quantitative measure of mode separation. Without such a measure, the reported "affine only" Dice and the visual decomposition in Figure 5 rest on the assumption that the final diffeomorphism contains little affine content. I ask the authors to either (a) provide a quantitative test of affine content in the estimated diffeomorphic component, e.g., projection of the final velocity field onto the space of affine vector fields, or (b) temper the central claim to what is actually supported: that the joint pipeline produces a competitive final registration and a better SSD-consistent affine for refinement than FLIRT, without asserting that the affine was estimated independently of the diffeomorphic mode.
- [Section 2.2.2, Proposition 2.6] Proposition 2.6 is the mathematical foundation for the FA implementation, but its proof is one sentence: "The proof is similar to the one given in Definition 2.5." The stated geodesic equations are not a trivial modification of the earlier system: the new equations for p_A and p_b contain integral terms involving d\tilde v_t, a Jacobian factor, and the pulled-back kernel K_{\tilde V}. These terms require a careful derivation by differentiating the Hamiltonian with respect to \tilde A and \tilde b through the definition \tilde v_t(x) = \tilde A_t^{-1} v_t(\tilde A_t x + \tilde b_t). Since the numerical method in Section 3.1 integrates exactly these equations, an incorrect sign or missing term would propagate into all FA experiments. Please expand the proof, or at minimum provide a supplementary derivation, and verify the signs and the domain of integration in Eq. (26).
minor comments (4)
- [Eq. (16)] There is a formatting error in the displayed Hamiltonian: the term "|v\|^2_V" should read "\|v\|^2_V" for consistency with the surrounding notation.
- [Figure 4 caption] The caption contains an incomplete sentence: "top row diffeo grid as unlike Fig. 3 we cannot isolate the diffeomorphism component, bottom row ...". This should be rephrased as a full sentence, e.g., "Top row: diffeomorphism grid; unlike Fig. 3, the diffeomorphic component cannot be isolated from the affine, so the grid shows the full transformation."
- [Section 3.1, Eq. (45)] The variational schedule has several hand-chosen parameters (w, nu, kernel widths, per-component learning rates, multi-start count). The paper gives a useful qualitative account of how they were chosen, but no sensitivity analysis is reported. A short robustness experiment, e.g., varying w and nu over a plausible range on a small subset of the IXI subjects, would strengthen the claim that the schedule is not over-tuned to the reported results.
- [Section 2.2.2, Eq. (21)] The change of variables \tilde I(x) = I(\tilde A x + \tilde b) is called a decoupling, but the subsequent data term is still evaluated through the composition with (\tilde A_1,\tilde b_1). The notation \tilde D^\mathrm{FA}_\gamma(\tilde A_1,\tilde b_1,\tilde I_1) is introduced without explicitly writing the final expression; please spell out the resulting data term to avoid ambiguity about which image is compared to the target.
Circularity Check
No significant circularity: the central claim is externally benchmarked and the framework self-citation is scaffolding, not a load-bearing reduction.
full rationale
The derivation chain is self-contained in the relevant sense. Sections 2.2 and 2.3 define the joint variational problems, apply Pontryagin's principle, and derive the Hamiltonian/geodesic equations in Propositions 2.5, 2.6 and 2.9; the decoupling change of variables is adapted from the authors' prior preprint [MP25], but the paper re-derives the equations and does not rely on that preprint to establish the empirical claim. The headline result, that the joint affine is a better starting point for LDDMM than a separately estimated affine, is tested on the external IXI cohort against FLIRT, uniGradICON and CARL using Dice computed from FastSurfer segmentations, a metric distinct from the SSD objective being optimized, so the reported advantage is not constructed from the model's own equations. The paper even reports that FLIRT's affine-only Dice is higher (reverse p = 2.3e-3), showing the affine comparison was not rigged in the authors' favor. The explicit Section 4 limitation, that the diffeomorphic component is not theoretically prevented from absorbing global affine deformations, and the absence of a same-SSD separate-affine 3D control are genuine threats to the 'affine-first' interpretation and to causal attribution of the improvement to jointness, but those are experimental confounds rather than circular reductions. Minor self-citation ([MP25], and the Demeter library by a co-author) is scaffolding; no target result is assumed as an input.
Assumptions & free parameters
free parameters (6)
- Variational schedule window w =
8
- Variational handover time constant nu =
250
- Schedule exponent alpha =
1/2
- Reproducing kernel widths sigma =
3 and 7
- Per-component Adam learning rates =
not reported
- Multi-start rotation candidate count n =
not reported
assumptions (5)
- standard math Images belong to H^1 and the action phi*I = I o phi^{-1} is C^0 but not differentiable; a differentiable infinitesimal action is obtained through the augmented image space (Aff(R^d) x I) borrowed from [MP25].
- domain assumption The affine transport of the image support stays inside the domain Omega (Prop 2.3 condition S1, S2 subset of Omega).
- standard math The Pontryagin Maximum Principle applies to the infinite-dimensional augmented control problem and yields the stated necessary conditions (Props 2.5, 2.6, 2.9).
- ad hoc to paper The DA model requires an intrinsic orthogonal frame (R_I, O_I) attached to each image, and the group D_alpha-Isom acts on the augmented space I x SO_d x R^d rather than on images directly.
- domain assumption The geodesic equations of Proposition 2.6 are accepted as correct despite a one-line proof sketch; they define the FA optimality conditions used for geodesic shooting.
invented entities (2)
-
Augmented image space Aff(R^d) x I
-
Intrinsic orthogonal frame (R_I, O_I) attached to each image
Cite this review
Pith. "Pith review of A Framework for Joint Affine and Diffeomorphic Image Registration." pith.science (2026). https://pith.science/paper/V4E7PFFZ
@misc{pith2026260810769,
author = {Pith},
title = {Pith review of: A Framework for Joint Affine and Diffeomorphic Image Registration},
year = {2026},
howpublished = {\url{https://pith.science/paper/V4E7PFFZ}},
note = {Machine review of arXiv:2608.10769}
}
read the original abstract
Anatomical image registration commonly relies on a sequential pipeline where an affine alignment is estimated first and then held fixed while a non-rigid diffeomorphic deformation is applied. This two-step process often leads to suboptimal results, as the initial stage can absorb local deformations, biasing the residual passed to the diffeomorphic registration. To address this, we introduce a Joint Affine-Diffeomorphic framework, based on the large deformations model, that estimates both global affine and local diffeomorphic motions simultaneously within a single optimization. We propose two models: a Full Affine (FA) model that combines affine and diffeomorphic deformations, and a Decomposed Affine (DA) model that restricts the affine part from FA to rotations, translations, and anisotropic scalings. To numerically implement these models for image registration tasks, we develop a tailored optimization strategy that combines progressive affine enrichment, gradually increasing the complexity of the affine component, with a variational weighting scheme that smoothly manages the coarse-to-fine handover between the affine and diffeomorphic components. Evaluated on 2D synthetic datasets and 3D brain MRIs from the IXI cohort, our unsupervised approach avoids the pathological deformations of sequential baselines. We demonstrate that our joint formulation outperforms in Dice overlap two state-of-the-art deep learning foundation models, CARL and uniGradI-CON, as well as a sequential baseline using FLIRT for the classical affine registration followed by LDDMM. Our implementation is publicly available.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Shape de- formation analysis from the optimal control viewpoint
[Arg+15] Sylvain Arguillère, Emmanuel Trélat, Alain Trouvé, and Laurent Younes. “Shape de- formation analysis from the optimal control viewpoint”. In:Journal de Mathématiques Pures et Appliquées104.1 (2015), pp. 139–178. [Ava+08] Brian B Avants, Charles L Epstein, Murray Grossman, and James C Gee. “Symmetric diffeomorphic image registration with cross-cor...
work page 2015
-
[41]
Advanced normalization tools (ANTS)
[ATS+09] Brian B Avants, Nick Tustison, Gang Song, et al. “Advanced normalization tools (ANTS)”. In:Insight j2.365 (2009), pp. 1–35. [Bal+19] Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. “Voxelmorph: a learning framework for deformable medical image registration”. In: IEEE transactions on medical imaging38.8 (2019), pp. 17...
work page 2009
-
[2016]
A Sub-Riemannian Modular Approach for Diffeomorphic Deformations
[GDT15] Barbara Gris, Stanley Durrleman, and Alain Trouvé. “A Sub-Riemannian Modular Approach for Diffeomorphic Deformations”. In:Proceedings of the 2nd International Conference on Geometric Science of Information (GSI). Springer, 2015, pp. 47–54. [Hen+20] Leonie Henschel, Sailesh Conjeti, Santiago Estrada, Kersten Diers, Bruce Fischl, and Martin Reuter. ...
work page 2020
-
[2017]
Elastix: a toolbox for intensity-based medical image registration
arXiv:1412.6980 [cs.LG]. [Kle+09] Stefan Klein, Marius Staring, Keelin Murphy, Max A Viergever, and Josien PW Pluim. “Elastix: a toolbox for intensity-based medical image registration”. In:IEEE transac- tions on medical imaging29.1 (2009), pp. 196–205. [Lan+14] William B Langdon, Marc Modat, Justyna Petke, and Mark Harman. “Improving 3D medical image regi...
arXiv 2009
-
[2019]
Recursive cascaded networks for unsupervised medical image registration
[Zha+19] Shengyu Zhao, Yue Dong, Eric I Chang, Yan Xu, et al. “Recursive cascaded networks for unsupervised medical image registration”. In:Proceedings of the IEEE/CVF inter- national conference on computer vision. 2019, pp. 10600–10610. A Additional figures 33 unigradicon unicarl (affine only) unicarl flirt (affine only) flirt+lddmm ours (affine only) ou...
work page 2019
-
[2022]
Unbiased average age- appropriate atlases for pediatric studies
[Fon+11] Vladimir Fonov, Alan C Evans, Kelly Botteron, C Robert Almli, Robert C McKinstry, D Louis Collins, Brain Development Cooperative Group, et al. “Unbiased average age- appropriate atlases for pediatric studies”. In:NeuroImage54.1 (2011), pp. 313–327. [Fon+09] VladimirSFonov,AlanCEvans,RobertCMcKinstry,CRAlmli,andDLCollins.“Un- biased nonlinear aver...
work page 2011
-
[2023]
Metamorphic image registration using a semi-Lagrangian scheme
[FGG21] Anton François, Pietro Gori, and Joan Glaunès. “Metamorphic image registration using a semi-Lagrangian scheme”. In:International Conference on Geometric Science of Information. Springer. 2021, pp. 781–788. 31 [Gre+25] Hastings Greer, Lin Tian, François-Xavier Vialard, Roland Kwitt, Raul San Jose Es- tepar, and Marc Niethammer. “CARL: A Framework f...
work page 2021
-
[2024]
[Ris+10] Laurent Risser, François-Xavier Vialard, Robin Wolz, Darryl Holm, and Daniel Rueck- ert. “Simultaneous Fine and Coarse Diffeomorphic Registration: Application to Atro- phy Measurement in Alzheimer’s Disease”. In:Medical image computing and computer- assisted intervention : MICCAI ... International Conference on Medical Image Com- puting and Compu...
arXiv 2006
Show all 10 references
-
[2025]
The graded group action framework for sub- Riemannian orbit models in shape spaces
[PT24] Thomas Pierron and Alain Trouvé. “The graded group action framework for sub- Riemannian orbit models in shape spaces”. In:arXiv preprint arXiv:2406.16930 (2024). preprint – submitted June 14,
2024 arXiv
-
[6657]
Improved opti- mization for the robust and accurate linear registration and motion correction of brain images
[Jen+02] Mark Jenkinson, Peter Bannister, Michael Brady, and Stephen Smith. “Improved opti- mization for the robust and accurate linear registration and motion correction of brain images”. In:Neuroimage17.2 (2002), pp. 825–841. [JS01] Mark Jenkinson and Stephen Smith. “A globa...
2002
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.