REVIEW 3 major objections 5 minor 24 references
The conditional Procrustes mean of planar curves is the leading eigenfunction of the conditional covariance, turning shape regression into covariance smoothing that accepts images and scalars alike.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 12:17 UTC pith:SYFY3NEE
load-bearing objection A sound conditional shape regression with a correct core identity, an honest simulation, and one genuine gap in the elastic-alignment convergence argument; worth a serious referee. the 3 major comments →
Deep Shape Regression for Planar Curves with Multimodal Covariates
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper establishes that the conditional full Procrustes mean of planar curves — the shape obtained after removing translation, rotation, and scale — equals the leading eigenfunction of the conditional Hermitian covariance surface C(s,t|x) of the pre-shape curves. It then estimates this covariance surface with a deep smoother: the surface is written as b(s)^T θ(x)θ(x)^H b(t), with b a B-spline basis in the curve parameter and θ(x) an additive sum over covariates of linear maps applied to modality-specific embeddings (splines for scalars, MLPs for tabular vectors, CNNs for images). The leading eigenfunction of the fitted surface is the estimated conditional mean; the trailing eigenfunctions
What carries the argument
The load-bearing identity is that the conditional full Procrustes mean µ(t|x) — the minimizer of expected squared Procrustes distance over unit-norm curves — is the leading eigenfunction u_1(t,x) of the conditional Hermitian covariance C(s,t|x). This turns a non-convex mean problem into covariance smoothing, which is carried out with the factored model Ĉ(s,t|x)=b(s)^T θ(x)θ(x)^H b(t), where b is a B-spline basis and θ(x)=θ0+Σ_j W_j φ_j(x_j) for covariate-specific embeddings φ_j. The outer product keeps the surface Hermitian positive semidefinite for any encoder architecture. The mean is extracted as the leading left singular vector of B^{1/2}θ(x), sidestepping explicit construction of the co
Load-bearing premise
The assumption that the alternating scheme — covariance smoothing, rotation alignment, warping alignment, refit — converges to the true elastic conditional mean, since the algorithm stops at a training-loss plateau that depends on its own re-parametrizations and no convergence or consistency proof is given.
What would settle it
Run the algorithm on simulated curves with known warping functions and a known covariate-dependent mean, initializing warps at several deliberately wrong starting points; if runs converge to different means, or to a mean whose covariate effect is visibly flattened, the convergence assumption fails.
If this is right
- Any conditional covariance smoother can now serve as a shape regressor: fit the surface, take the leading eigenfunction, and the conditional mean shape follows.
- The same fitted surface yields covariate-dependent modes of shape variation via its trailing eigenfunctions, so one model gives both the mean and the principal deformations.
- Sparse and irregularly sampled curves can be used directly, since the loss only involves off-diagonal pairwise products.
- The rank of the factored covariance, r, acts as a built-in regularizer; even a rank-1 fit already gives a usable approximate conditional mean.
- An auxiliary log-size regression recovers covariate effects on overall scale, which the scale-invariant shape mean deliberately discards.
Where Pith is reading between the lines
- The eigenfunction identity suggests a broader template: any shape geometry whose distance has an inner-product characterization could be regressed on high-dimensional covariates by smoothing the associated conditional kernel and taking its top eigenfunction — a route beyond the planar, open-curve case.
- Because the covariance surface itself is estimated, the method contains the ingredients for uncertainty quantification on shape effects (e.g., pointwise intervals from the trailing eigenfunctions), an extension the paper lists as future work but does not develop.
- The alternating alignment loop's convergence is not proven; a stress test on simulations with known warping and covariate-dependent means would show whether repeated warping toward a covariate-dependent mean can flatten the very covariate signal the regression targets.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a deep regression model for planar curve shapes with multimodal covariates. It shows that the conditional full Procrustes mean, defined as the minimizer of expected squared full Procrustes distance, is the leading eigenfunction of the conditional Hermitian covariance (Eqs. 6–8), and uses this to estimate the mean via a deep conditional covariance smoother with modality-specific encoders (splines, MLPs, CNNs). An alternating algorithm (Algorithm 1) adds elastic alignment by iterating covariance smoothing, rotational alignment, and warping alignment. The method is tested on a simulation with a cusped two-half-circle shape and applied to hippocampal outlines in ADNI, where the reported covariate effects agree with prior literature.
Significance. If the results hold, the paper provides a flexible and conceptually clean framework for conditional shape regression: the population mean is characterized as an eigenfunction, and the deep smoother extends classical covariance smoothing to images and other high-dimensional covariates. The simulation design is a genuine external check—the truth is a cusped shape outside the spline model class—and the reported recovery errors decrease with sample size. The ADNI application, though exploratory, demonstrates practical utility. The code release is a further strength. The main limitation is the lack of formal convergence or consistency guarantees for the elastic alignment algorithm, which is a central methodological component rather than a peripheral detail.
major comments (3)
- [Section 2.6, Algorithm 1 (Steps 5–11)] The paper does not define the population elastic full Procrustes mean as a minimizer over rotations and warps, and it does not prove that Algorithm 1 converges to such a quantity. Step 11 stops when the training loss (14) no longer decreases, but that loss is evaluated on curves that the algorithm itself has reparametrised toward the current fitted mean. A plateau in this self-aligned loss is therefore partially self-confirming and does not imply convergence of the shape estimate. There is also no analysis of initialization dependence or of whether the warping step can erase covariate-specific shape features. Since the elastic mean is a central claimed contribution, this is a load-bearing gap. Please either provide a formal convergence/consistency result for the alternating scheme under explicit assumptions, or reframe the algorithm as a heuristic and supply a sensitivity analysis (e.g.
- [Section 2.3, Eq. (8)] The characterization of the conditional full Procrustes mean as the leading eigenfunction u1(t,x) implicitly assumes that the largest eigenvalue λ1(x) is distinct. If the conditional covariance has an eigenvalue of multiplicity greater than one, the leading eigenspace is not one-dimensional and the 'mean' is not unique up to a single rotation. The paper should state an eigenvalue-gap condition or otherwise discuss identifiability. Without this, the estimated mean depends on arbitrary choices in the eigendecomposition, which matters when the mean is interpreted as a function of x as in Figure 2.
- [Section 3, rank-1 variant] The text says that for rank 1, the leading eigenfunction is 'directly given by θ(x)' and that the resulting mean is 'additive in the covariates by construction.' For r=1, the leading singular vector of B^{1/2}θ(x) is the normalized vector B^{1/2}θ(x)/||B^{1/2}θ(x)||, so the estimated mean is a normalized version of θ(x), not θ(x) itself. Because the normalizing constant depends on x, the mean is not additive in the covariates in any obvious sense. This does not affect the simulation conclusions, but the claim is imprecise and should be corrected.
minor comments (5)
- [General notation] The complex conjugate is not consistently visible in the plain-text rendering of Eqs. (5), (6), and (13); the final typeset version should make the overlines explicit.
- [Section 3, first sentence] Typo: 'multimidal' should be 'multimodal.'
- [Section 2.4, Eq. (15)] The penalty P_t is described as controlling the roughness of the mean in t. Since the mean is obtained through a further normalization (Eq. 16), the relationship between P_t and the roughness of the final normalized mean is indirect. A sentence clarifying this would help.
- [Figure 2] The ADNI application reports 'sex has no effect' and similar qualitative statements without uncertainty quantification. Given the modest sample size (n=239), a confidence band or at least a bootstrap/permutation-based assessment would be more appropriate, especially for null-effect statements.
- [Section 5, Outlook] The paper honestly identifies several open issues, including the lack of a REML-type criterion for penalty selection and the CPU-bound elastic alignment. These are useful caveats but should be supplemented by explicit limitations of the convergence properties of Algorithm 1 in the main text, not only in the outlook.
Circularity Check
No circularity: the conditional-mean characterization is derived, not assumed; Algorithm 1 is a heuristic with robustness concerns, but no step reduces to its own inputs.
full rationale
The central claim (Eqs. 2-8) is a direct derivation: the conditional full Procrustes mean is defined as the minimizer of E[d(µ,˜y)^2 | X=x]; using the closed-form full Procrustes distance, this becomes the maximizer of the Rayleigh quotient ⟨µ, C_x µ⟩, and the leading eigenfunction of the Hermitian positive semidefinite operator C_x is the maximizer. This is standard functional analysis, not an input-output tautology. The covariance smoother (9)-(14) estimates C(s,t|x) from off-diagonal pairwise products, and the mean is obtained by a numerical eigendecomposition (16); no fitted parameter is relabeled as a prediction, since the estimated object is the leading eigenfunction of an independently fitted covariance surface. Although the unconditional version of the theorem is attributed to the authors' forthcoming [19], Section 2.3 re-derives the result in the conditional setting, so that self-citation is not load-bearing. The elastic alternating scheme in Algorithm 1 is presented as an estimation algorithm without a convergence or consistency proof, and its stopping rule based on training loss (14) on self-aligned curves is a legitimate robustness concern. However, that is a correctness/optimization limitation, not a constructional circularity: the simulation uses a known ground-truth shape outside the model class and the ADNI results agree with external literature, providing independent checks. No step in the derivation chain reduces to its own inputs, so there is no significant circularity.
Axiom & Free-Parameter Ledger
free parameters (6)
- Covariance rank r =
r=3 (simulation), r=4 (application)
- T-roughness penalty ρ_t =
1e-3 (simulation), 1 (application); scale-model weight decay 1e-2
- Covariate penalty ρ_x =
1e-2 (simulation, scalar covariate spline)
- Spline basis sizes =
k=50 (sim), k=128 (application), k=10 (covariate splines)
- Encoder/optimizer hyperparameters =
CNN 8→16→32 channels, 3×3 kernels, lr 3e-3, weight decay 1e-4, batch 24/64, 600 steps
- Elastic alignment convergence threshold =
1% held-out loss decrease
axioms (6)
- standard math Exchange of conditional expectation and integration in Eqs. (4)-(5) (Fubini)
- standard math Spectral theorem for the compact self-adjoint conditional covariance operator C_x, with λ₁(x) > λ₂(x) for identifiability of the leading eigenfunction
- domain assumption Off-diagonal pairwise products c_{i,lm} have conditional mean C(t_il,t_im|x), and penalised least squares (14) consistently recovers the covariance surface
- ad hoc to paper The alternating scheme (Algorithm 1) converges to the conditional elastic full Procrustes mean, and warping toward the current conditional mean does not erase covariate-specific shape features
- domain assumption Arc-length parametrisation is a valid initial parametrisation for curves with as few as 10 observations, and the warping tool of [16] behaves well for sparse curves inside the regression loop
- domain assumption Additive structure θ(x)=θ₀+Σθ_j(x_j) (Eq. 11) captures the conditional covariance adequately for the ADNI analysis
read the original abstract
The shape of a planar curve is the geometric information that remains once translation, rotation, scale and reparametrisation are removed and is of interest in many health applications, e.g. in neuroimaging. We propose a deep shape regression model for open planar curves that admits multimodal and high-dimensional covariates. Representing curves as complex-valued functions, we show that the conditional full Procrustes mean is the leading eigenfunction of the conditional covariance. To estimate this covariance surface, we propose a novel deep conditional covariance smoother with modality-specific encoders - e.g. splines for scalar covariates and convolutional networks for images, which classical spline smoothers cannot accommodate. Our model is by construction invariant to the translation, rotation and scaling of the input curves and handles sparsely and irregularly sampled curves. We further provide an algorithm for elastic mean estimation that also removes parametrisation by iterating covariance smoothing, rotational alignment and parametrisation alignment. We illustrate the method on simulated outlines with known conditional mean and multimodal covariates, and give a first application to hippocampal outlines from the ADNI cohort, recovering covariate effects consistent with the literature. Code is available at https://github.com/mpff/dnn-shapes.
Figures
Reference graph
Works this paper leans on
-
[1]
NeuroImage54(3), 2033–2044 (2011)
Avants, B.B., Tustison, N.J., Song, G., Cook, P.A., Klein, A., Gee, J.C.: A repro- ducible evaluation of ANTs similarity metric performance in brain image registra- tion. NeuroImage54(3), 2033–2044 (2011). https://doi.org/10.1016/j.neuroimage. 2010.09.025 16 M. Pfeuffer et al
-
[2]
Computational Statistics & Data Analysis120, 25–41 (2018)
Cederbaum,J.,Scheipl,F.,Greven,S.:Fastsymmetricadditivecovariancesmooth- ing. Computational Statistics & Data Analysis120, 25–41 (2018). https://doi.org/ 10.1016/j.csda.2017.11.002
-
[3]
Nature Methods19(12), 1568–1571 (2022)
Ciric, R., Thompson, W.H., Lorenz, R., Goncalves, M., MacNicol, E.E., Markiewicz, C.J., Halchenko, Y.O., Ghosh, S.S., Gorgolewski, K.J., Poldrack, R.A., Esteban, O.: TemplateFlow: FAIR-sharing of multi-scale, multi-species brain models. Nature Methods19(12), 1568–1571 (2022). https://doi.org/10.1038/ s41592-022-01681-2
2022
-
[4]
Technometrics64(3), 335–345 (2022)
Ding, F., He, S., Jones, D.E., Huang, J.Z.: Functional PCA with covariate- dependent mean and covariance structure. Technometrics64(3), 335–345 (2022). https://doi.org/10.1080/00401706.2021.2008502
arXiv 2022
-
[5]
Wiley Series in Probability and Statistics, John Wiley & Sons, Chichester, 2 edn
Dryden, I.L., Mardia, K.V.: Statistical Shape Analysis, with Applications in R. Wiley Series in Probability and Statistics, John Wiley & Sons, Chichester, 2 edn. (2016). https://doi.org/10.1002/9781119072492
-
[6]
Sta- tistical Science11(2), 89–121 (1996)
Eilers, P.H.C., Marx, B.D.: Flexible smoothing with B-splines and penalties. Sta- tistical Science11(2), 89–121 (1996). https://doi.org/10.1214/ss/1038425655
arXiv 1996
-
[7]
Neurology 72(11), 999–1007 (2009)
Henneman, W.J.P., Sluimer, J.D., Barnes, J., van der Flier, W.M., Sluimer, I.C., Fox, N.C., Scheltens, P., Vrenken, H., Barkhof, F.: Hippocampal atrophy rates in Alzheimer disease: Added value over whole brain volume measures. Neurology 72(11), 999–1007 (2009). https://doi.org/10.1212/01.wnl.0000344568.09360.31
arXiv 2009
-
[8]
Wiley Series in Probability and Statis- tics, John Wiley & Sons, Ltd (2015)
Hsing, T., Eubank, R.L.: Theoretical Foundations of Functional Data Analysis, with an Introduction to Linear Operators. Wiley Series in Probability and Statis- tics, John Wiley & Sons, Ltd (2015). https://doi.org/10.1002/9781118762547
-
[9]
Human Brain Mapping40(17), 4952–4964 (2019)
Isensee, F., Schell, M., Pflueger, I., Brugnara, G., Bonekamp, D., Neuberger, U., Wick, A., Schlemmer, H.P., Heiland, S., Wick, W., Bendszus, M., Maier-Hein, K.H., Kickingereder, P.: Automated brain extraction of multisequence MRI using artificial neural networks. Human Brain Mapping40(17), 4952–4964 (2019). https: //doi.org/10.1002/hbm.24750
-
[10]
NeuroImage55(3), 909–919 (2011)
Pievani, M., Galluzzi, S., Thompson, P.M., Rasser, P.E., Bonetti, M., Frisoni, G.B.: APOE4 is associated with greater atrophy of the hippocampal formation in Alzheimer’s disease. NeuroImage55(3), 909–919 (2011). https://doi.org/10.1016/ j.neuroimage.2010.12.081
2011
-
[11]
Springer Series in Statistics, Springer, 2 edn
Ramsay, J.O., Silverman, B.W.: Functional Data Analysis. Springer Series in Statistics, Springer, 2 edn. (2005). https://doi.org/10.1007/b98888, https://doi. org/10.1007/b98888
doi:10.1007/b98888 2005
-
[13]
Journal of the Royal Statistical Society Series B: Sta- tistical Methodology84(5), 1785–1820 (2022)
Sarkar, S., Panaretos, V.M.: CovNet: Covariance networks for functional data on multidimensional domains. Journal of the Royal Statistical Society Series B: Sta- tistical Methodology84(5), 1785–1820 (2022). https://doi.org/10.1111/rssb.12551
-
[14]
IEEE Transactions on Pattern Analysis and Machine Intelligence33(7), 1415–1428 (2011)
Srivastava, A., Klassen, E., Joshi, S.H., Jermyn, I.H.: Shape analysis of elastic curves in Euclidean spaces. IEEE Transactions on Pattern Analysis and Machine Intelligence33(7), 1415–1428 (2011). https://doi.org/10.1109/TPAMI.2010.184
-
[15]
Springer Series in Statistics, Springer, New York, NY (2016)
Srivastava, A., Klassen, E.P.: Functional and Shape Data Analysis. Springer Series in Statistics, Springer, New York, NY (2016). https://doi.org/10.1007/ 978-1-4939-4020-2
2016
-
[16]
Biometrics79(3), 2103–2115 (2023)
Steyer, L., Stöcker, A., Greven, S.: Elastic analysis of irregularly or sparsely sam- pled curves. Biometrics79(3), 2103–2115 (2023). https://doi.org/10.1111/biom. 13706 Deep Shape Regression for Planar Curves with Multimodal Covariates 17
-
[17]
Journal of Multivariate Analysis211, 105515 (2026)
Steyer, L., Stöcker, A., Greven, S.: Model-based Fréchet regression in (quotient) metric spaces with a focus on elastic curves. Journal of Multivariate Analysis211, 105515 (2026). https://doi.org/10.1016/j.jmva.2025.105515
arXiv 2026
-
[18]
Journal of Computational and Graphical Statistics32(4), 1600–1612 (2023)
Stöcker, A., Steyer, L., Greven, S.: Functional additive models on manifolds of planar shapes and forms. Journal of Computational and Graphical Statistics32(4), 1600–1612 (2023). https://doi.org/10.1080/10618600.2023.2175687
arXiv 2023
-
[19]
Journal of Computational and Graphical Statistics (2026)
Stöcker, A., Pfeuffer, M., Steyer, L., Greven, S.: Elastic full procrustes analysis of plane curves via hermitian covariance smoothing. Journal of Computational and Graphical Statistics (2026). https://doi.org/10.1080/10618600.2026.2679146, (forthcoming)
arXiv 2026
-
[20]
Scientific Reports11(1), 9068 (2021)
Tustison, N.J., Cook, P.A., Holbrook, A.J., Johnson, H.J., Muschelli, J., Devenyi, G.A., Duda, J.T., Das, S.R., Cullen, N.C., Gillen, D.L., Yassa, M.A., Stone, J.R., Gee, J.C., Avants, B.B.: The ANTsX ecosystem for quantitative biological and medical imaging. Scientific Reports11(1), 9068 (2021). https://doi.org/10.1038/ s41598-021-87564-6
2021
-
[21]
Journal of the Royal Sta- tistical Society: Series B (Statistical Methodology)73(1), 3–36 (2011)
Wood, S.N.: Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. Journal of the Royal Sta- tistical Society: Series B (Statistical Methodology)73(1), 3–36 (2011). https: //doi.org/10.1111/j.1467-9868.2010.00749.x
arXiv 2011
-
[22]
Statistics and Computing33(5), 118 (2023)
Wu, S., Beaulac, C., Cao, J.: Neural networks for scalar input and functional output. Statistics and Computing33(5), 118 (2023). https://doi.org/10.1007/ s11222-023-10287-3
2023
-
[23]
Journal of the American Statistical Association100(470), 577–590 (2005)
Yao, F., Müller, H.G., Wang, J.L.: Functional data analysis for sparse longitudinal data. Journal of the American Statistical Association100(470), 577–590 (2005). https://doi.org/10.1198/016214504000001745
-
[24]
In: Proceedings of the 38th International Conference on Ma- chine Learning (ICML)
Yao, J., Mueller, J., Wang, J.L.: Deep learning for functional data analysis with adaptive basis layers. In: Proceedings of the 38th International Conference on Ma- chine Learning (ICML). Proceedings of Machine Learning Research, vol. 139, pp. 11898–11908. PMLR (2021), http://proceedings.mlr.press/v139/yao21c.html
2021
-
[25]
Journal of Multivariate Analysis212, 105526 (2026)
Zhong, R., Zhang, J., Zhang, C.: Nonlinear functional principal component analysis using neural networks. Journal of Multivariate Analysis212, 105526 (2026). https: //doi.org/10.1016/j.jmva.2025.105526
arXiv 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.