REVIEW 3 major objections 5 minor 2 cited by
Riemannian batch normalization for SPD neural networks
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A Riemannian batch normalization layer improves SPD networks' accuracy on radar, emotion, and action data.
desk verdict Useful new SPD batch-normalization layer with consistent empirical gains; the missing backprop through the barycenter needs to be clarified before I fully trust the training story. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The layer's machinery has four parts. First, the batch mean is the Riemannian barycenter (Fréchet mean) of the batch, computed by the Karcher flow. Second, centering is parallel transport on the SPD manifold from the barycenter to the identity, written $\Gamma_{G\to Id}(P)=G^{-1/2}PG^{-1/2}$, and biasing is parallel transport from the identity to the learned parameter $\tilde G$, written $\tilde P=\tilde G^{1/2}\bar P \tilde G^{1/2}$. Third, because the bias parameter must remain positive definite, its Euclidean gradient is projected onto the tangent space at $\tilde G$ and the update follows the geodesic via the exponential map. Fourth, gradients through the structured matrix functions $(\cdot)^{1/2}$ and $(\cdot)^{-1/2}$ are propagated using the Loewner finite-difference matrix formula for eigenvalue functions. The paper derives these pieces and assembles them into a trainable layer.
What would settle it
Compute the total derivative of the loss with respect to one input $P_i$ through the full forward pass of the RBN layer using automatic differentiation, and compare it with the gradient obtained from the formulas in Section 4 plus the paper's claimed updates. If the two disagree on the terms passing through $dG_B/dP_j$, then the training algorithm as written is not exactly the one described.
Extended reading notes
Core claim
The paper claims that adding a Riemannian batch normalization layer to an SPD neural network produces a consistently more accurate classifier than the same network without it, on diverse covariate data. Concretely, on the NATO drone radar dataset, accuracy rises from 85.4% to 87.2% with the full training set and from 83.9% to 84.7% when only 10% of the data is used; on the AFEW emotion dataset the shallowest architecture improves from 29.9% to 34.9%; and on HDM05 action recognition, accuracy rises from 61.6% to 68.1%. The claim is not that a single architecture wins everywhere, but that the normalization layer is a broadly applicable improvement and that the improvement is largest when data are scarce.
Load-bearing premise
The training procedure's correctness rests on the gradient of the loss flowing through the batch's Riemannian barycenter $G_B$; the paper derives the gradients through the bias parameter and through the square-root functions, but never states how $dL/dG_B$ is computed or whether gradients are stopped at $G_B$.
Editorial extensions
If this is right
- SPD networks used for radar micro-Doppler classification can be made more accurate by inserting the RBN layer, without adding parameters beyond one learned bias matrix per normalized layer.
- The gain from normalization is largest in data-scarce regimes: with only 10% of the NATO training data, SPDNetBN stays at 84.7% while a small fully convolutional network drops to 66.6%.
- Because the layer is agnostic to the surrounding architecture, it can be composed with alternative SPD layers, such as convolutional, recurrent, or approximate square-root projection layers, in place of the original BiMap/ReEig/LogEig stack.
- The training cost is modest: on the deepest AFEW experiment, one training epoch rose from about 81 seconds to 88 seconds.
- The learned bias parameter gives the network an explicit, geometry-respecting mechanism to adjust the location of each layer's SPD features, analogous to the shift parameter in Euclidean batch norm.
Reading between the lines
- If the layer behaves like Euclidean batch norm, one would expect it to stabilize training enough to allow higher learning rates or faster convergence; the paper does not report convergence curves, so that is a testable prediction rather than a stated result.
- Because the barycenter-to-identity transport decouples the batch mean from the feature scale, the layer may implicitly reduce a form of covariance shift between layers; the paper does not measure internal covariate shift, but this mechanism could be tested by probing layer outputs during training.
- The same centering-and-biasing recipe could be applied to other matrix manifolds, such as the Grassmann or Stiefel manifolds used in related networks, where parallel transport is also available.
- The paper reports fold-averaged accuracies but no significance tests; a natural follow-up would be to check whether gains such as the 1.8-point improvement on NATO data exceed fold-to-fold variance.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Riemannian batch normalization (RBN) layer for SPD neural networks. The forward pass centers each SPD input by parallel transporting it from the Riemannian barycenter of the batch to the identity, then biases it toward a learned SPD parameter G. The authors derive a manifold-constrained gradient descent for learning G and use structured matrix backpropagation for the square-root operations. They evaluate the layer on NATO radar drone data, the AFEW emotion recognition dataset, and the HDM05 action recognition dataset, reporting consistent accuracy gains over the baseline SPDNet and improved robustness to reduced training data. The paper also states that an open-source PyTorch implementation is released.
Significance. If the proposed layer is sound and the reported gains hold, it is a useful drop-in building block for SPD networks, especially in data-scarce regimes. The geometric construction (barycenter centering, parallel transport, Riemannian gradient update) is principled, and the paper provides external validation on three different data types. The explicit release of code and the comparison against both deep and shallow Riemannian baselines are strengths. The principal weakness is that the exact training algorithm is under-specified: the gradient of the loss through the batch Riemannian barycenter is never derived or mentioned, which is load-bearing for the claim of a complete backpropagation scheme.
major comments (3)
- [Section 4, Eq. (6a), Algorithm 1] The forward pass centers each input P_i using G_B = Bar({P_i}), the Riemannian barycenter of the batch. Since G_B is a function of all inputs, the total derivative dL/dP_j includes a term through dL/dG_B and dG_B/dP_j. Section 4 derives gradients through the bias parameter G and through the structured square-root functions applied to G and G_B, but it never derives or even mentions the gradient through the barycenter mapping itself. In a standard backpropagation framework this is a required term. If the implementation stops gradients at G_B (for example by detaching it before centering), the training optimizes a surrogate objective that is not the one described; if it instead backpropagates through the Karcher flow, the manuscript should state this and explain how the fixed-point nature is handled (implicit differentiation versus differentiation through iterations, with the associated memory and truncation issues). This is load-bearing because the paper's contribution includes a 'generalized gradient descent' for learning the RBN, and the reported accuracy improvements depend on which training objective is actually optimized. Please derive the barycenter gradient or explicitly identify and justify the approximate objective being optimized.
- [Section 4.2, Eq. (12)] The structured matrix backpropagation formula is stated without discussing whether the upstream gradient ∂L/∂X is symmetric. In general, the gradient of a scalar loss with respect to a matrix output may not be symmetric, and the formula as written can then produce a gradient that is not a tangent vector of the SPD manifold. The paper should state whether the gradient is symmetrized (e.g., (G + G^T)/2) or whether the loss is constructed so that ∂L/∂X is already symmetric for the layers used in SPDNet. This is a correctness detail for the exact backpropagation claimed in Section 4.
- [Section 5, Tables 1–3] The paper claims a 'systematic' classification improvement, but the supporting statistics are incomplete. On the NATO dataset the improvements are 85.4±0.80 to 87.2±1.06 and 83.9±0.85 to 84.7±0.64; the AFEW results in Table 2 have no error bars; and the HDM05 comparison in Table 3 is a single architecture. Please report standard deviations or confidence intervals for all experimental tables and, if possible, a significance test or at least a statement about overlap. This is a reporting issue rather than a fundamental flaw, but it matters for the strength of the 'systematic improvement' claim.
minor comments (5)
- [Section 5.1, Table 1] The table contains several apparent formatting errors: values such as '89.489.489.4%' and '84.784.784.7%' are repeated or tripled. Please fix these so the table is legible and unambiguous about which model each number belongs to.
- [Algorithm 1] In the training phase, the running mean update uses Bar_η(G_S, G_B); the text explains this as moving toward the batch mean by (1−η), but the weight notation could be confused with the Euclidean convention. Clarify that η is the weight on G_S in the two-point barycenter.
- [Section 4.1 and Section 5] The experiments state that all networks are trained with SGD and momentum 0.9, but the paper does not explain how momentum is handled for the Riemannian parameter G (e.g., whether the velocity is parallel-transported to the new tangent space). Please clarify the implementation of momentum for the manifold-constrained update.
- [Section 3.2, Eq. (10)] The Gaussian density is written with a proportionality sign and no normalization constant; this is acceptable, but the sentence 'the notion of variance takes no part in this definition' could be misleading, since α = (n+1)/2 acts as a concentration parameter. Suggest a more careful wording.
- [References [40] and [41]] References [40] and [41] appear to be the same paper (identical title, same pages, only the month differs). Please merge or disambiguate them.
Circularity Check
No significant circularity: RBN layer is independently defined and validated against external benchmarks; the only self-citations are minor and not load-bearing.
full rationale
The derivation chain is self-contained. The RBN forward pass is defined explicitly by Eq. 6a-6b (centering by parallel transport from the batch Riemannian barycenter, then biasing by transport to a learned SPD bias G), and the reported gains are measured on held-out test sets from the NATO radar data and from external benchmarks (AFEW, HDM05). The optimization uses a standard tangent-space projection (Eq. 11, attributed to [42] and [18]) and a standard structured-matrix backpropagation formula (Eq. 12, credited to [26]/[9]); no fitted parameter is renamed as a prediction. The main self-citations, [10] (synthetic drone data generator and FCN baseline) and [40]/[6] (Riemannian-median/radar background and an information-geometry density model), do not carry the central claim: [10] supplies data rather than a theorem, and the RBN advantage is also demonstrated on external datasets. The missing gradient of the loss through the barycenter map G_B = Bar({P_i}) in Eq. 6a is a real completeness gap in the backpropagation description, but it is an omitted derivative, not an equivalence between input and output, so it does not make the derivation circular.
Assumptions & free parameters
free parameters (1)
- Momentum eta for running mean update =
0.9
assumptions (4)
- domain assumption The SPD transport mapping from point P1 to P2 is given by (P2 P1^{-1})^{1/2} P (P2 P1^{-1})^{1/2} and equals the composition Log-PT-Exp.
- standard math The Riemannian barycenter (Frechet mean) computed by Karcher flow is the appropriate batch mean and converges to a unique solution on the SPD manifold.
- domain assumption The maximum entropy Gaussian density on the SPD cone has no separate variance parameter; its natural parameter is the Riemannian mean, so batch normalization reduces to centering and biasing only.
- standard math The structured matrix backpropagation formula (Eq. 12-13) correctly gives the gradient of symmetric matrix functions like the square root and inverse square root.
Cite this review
Pith. "Pith review of Riemannian batch normalization for SPD neural networks." pith.science (2026). https://pith.science/paper/OO45V7T3
@misc{pith2026190902414,
author = {Pith},
title = {Pith review of: Riemannian batch normalization for SPD neural networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/OO45V7T3}},
note = {Machine review of arXiv:1909.02414}
}
read the original abstract
Covariance matrices have attracted attention for machine learning applications due to their capacity to capture interesting structure in the data. The main challenge is that one needs to take into account the particular geometry of the Riemannian manifold of symmetric positive definite (SPD) matrices they belong to. In the context of deep networks, several architectures for these matrices have recently been proposed. In our article, we introduce a Riemannian batch normalization (batchnorm) algorithm, which generalizes the one used in Euclidean nets. This novel layer makes use of geometric operations on the manifold, notably the Riemannian barycenter, parallel transport and non-linear structured matrix transformations. We derive a new manifold-constrained gradient descent algorithm working in the space of SPD matrices, allowing to learn the batchnorm layer. We validate our proposed approach with experiments in three different contexts on diverse data types: a drone recognition dataset from radar observations, and on emotion and action recognition datasets from video and motion capture data. Experiments show that the Riemannian batchnorm systematically gives better classification performance compared with leading methods and a remarkable robustness to lack of data.
Figures
Forward citations
Cited by 2 Pith papers
-
Riemannian Geometry for Pre-trained Language Model Embeddings
Aggregating per-token pullback metrics via the Fréchet mean on the SPD manifold outperforms Euclidean mean pooling for sentence classification, with most of the gain attributable to geometric aggregation rather than l...
-
Constitutive Manifold Neural Networks
A geometry-aware input layer that feeds networks the logarithmically mapped eigenvalues and eigenvectors of SPD material tensors beats plain MLPs in surrogate modeling of stochastic anisotropic heat conduction.
Reference graph
Works this paper leans on
-
[1]
S.-i. Amari. Information Geometry and Its Applications . Applied Mathematical Sciences. Springer Japan, 2016
work page 2016
-
[2]
V . Arsigny, P. Fillard, X. Pennec, and N. Ayache. Log-Euclidean metrics for fast and simple calculus on diffusion tensors. Magnetic Resonance in Medicine, 56(2):411–421, Aug. 2006
work page 2006
-
[3]
C. Atkinson and A. F. S. Mitchell. Rao’s Distance Measure. Sankhy¯a: The Indian Journal of Statistics, Series A (1961-2002), 43(3):345–365, 1981
work page 1961
-
[4]
A. Barachant, S. Bonnet, M. Congedo, and C. Jutten. Multiclass Brain–Computer Interface Classification by Riemannian Geometry. IEEE Transactions on Biomedical Engineering , 59(4):920–928, Apr. 2012
work page 2012
-
[5]
A. Barachant, S. Bonnet, M. Congedo, and C. Jutten. Classification of covariance matrices using a Riemannian-based kernel for BCI applications. Neurocomputing, 112:172–178, July 2013
work page 2013
-
[6]
F. Barbaresco. Jean-Louis Koszul and the Elementary Structures of Information Geometry. In F. Nielsen, editor,Geometric Structures of Information, Signals and Communication Technology, pages 333–392. Springer International Publishing, Cham, 2019
work page 2019
-
[7]
R. Bhatia. Positive Definite Matrices. Princeton University Press, Princeton, NJ, USA, 2015
work page 2015
-
[8]
S. Bonnabel and R. Sepulchre. Riemannian Metric and Geometric Mean for Positive Semidefi- nite Matrices of Fixed Rank. SIAM Journal on Matrix Analysis and Applications, 31(3):1055– 1070, Jan. 2010
work page 2010
Show all 44 references
-
[9]
Brodski˘ı, J
M. Brodski˘ı, J. Dalecki˘ı, O. È˘ıdus, I. Iohvidov, M. Kre˘ın, O. Ladyženskaja, V . Lidski˘ı, J. Ljubiˇc, V . Macaev, A. Povzner, L. Sahnoviˇc, J. Šmuljan, I. Suharevski ˘ı, and N. Uralceva. Thirteen Papers on Functional Analysis and Partial Differential Equations , volume 47 ...
1965
-
[10]
D. A. Brooks, O. Schwander, F. Barbaresco, J. Schneider, and M. Cord. Temporal Deep Learning for Drone Micro-Doppler Classification. In 2018 19th International Radar Symposium (IRS), pages 1–10, June 2018
2018
-
[11]
Cavazza, P
J. Cavazza, P. Morerio, and V . Murino. When Kernel Methods Meet Feature Learning: Log- Covariance Network for Action Recognition From Skeletal Data. In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1251–1258, July 2017
2017
-
[12]
Chakraborty, J
R. Chakraborty, J. Bouza, J. Manton, and B. C. Vemuri. ManifoldNet: A Deep Network Framework for Manifold-valued Data. arXiv:1809.06211 [cs], Sept. 2018. arXiv: 1809.06211
2018 arXiv
-
[13]
Chakraborty, C.-H
R. Chakraborty, C.-H. Yang, X. Zhen, M. Banerjee, D. Archer, D. Vaillancourt, V . Singh, and B. Vemuri. A Statistical Recurrent Model on the Manifold of Symmetric Positive Definite Ma- trices. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett,...
2018
-
[14]
Charon and F
N. Charon and F. Barbaresco. A new approach for target detection in radar images based on geometric properties of covariance matrices’ spaces, 2009
2009
-
[15]
V . C. Chen, F. Li, S.-S. Ho, and H. Wechsler. Micro-Doppler effect in radar: phenomenon, model, and simulation study. IEEE Transactions on Aerospace and electronic systems, 42(1):2–21, 2006
2006
-
[16]
Dhall, R
A. Dhall, R. Goecke, S. Lucey, and T. Gedeon. Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark. In 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pages 2106–2112, Nov. 2011
2011
-
[17]
Z. Dong, S. Jia, V . C. Zhang, M. Pei, and Y . Wu. Deep Manifold Learning of Symmetric Positive Definite Matrices with Application to Face Recognition. In AAAI, 2017
2017
-
[18]
Edelman, T
A. Edelman, T. Arias, and S. Smith. The Geometry of Algorithms with Orthogonality Con- straints. SIAM Journal on Matrix Analysis and Applications, 20(2):303–353, Jan. 1998
1998
-
[19]
Engin, L
M. Engin, L. Wang, L. Zhou, and X. Liu. DeepKSPD: Learning Kernel-matrix-based SPD Representation for Fine-grained Image Recognition. arXiv:1711.04047 [cs], Nov. 2017. arXiv: 1711.04047
2017 arXiv
-
[20]
M. Fréchet. Sur l’extension de certaines evaluations statistiques au cas de petits echantillons. Revue de l’Institut International de Statistique / Review of the International Statistical Institute, 11(3/4):182–205, 1943
1943
-
[21]
Z. Gao, Y . Wu, X. Bu, and Y . Jia. Learning a Robust Representation via a Deep Network on Symmetric Positive Definite Manifolds. arXiv:1711.06540 [cs], Nov. 2017. arXiv: 1711.06540
2017 arXiv
-
[22]
Huang and L
Z. Huang and L. Van Gool. A Riemannian Network for SPD Matrix Learning.arXiv:1608.04233 [cs], Aug. 2016. arXiv: 1608.04233
2016 arXiv
-
[23]
Huang, C
Z. Huang, C. Wan, T. Probst, and L. Van Gool. Deep Learning on Lie Groups for Skeleton-based Action Recognition. arXiv:1612.05877 [cs], Dec. 2016. arXiv: 1612.05877
2016 arXiv
-
[24]
Huang, J
Z. Huang, J. Wu, and L. Van Gool. Building Deep Networks on Grassmann Manifolds. arXiv:1611.05742 [cs], Nov. 2016. arXiv: 1611.05742
2016 arXiv
-
[25]
Ioffe and C
S. Ioffe and C. Szegedy. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv:1502.03167 [cs], Feb. 2015. arXiv: 1502.03167
2015 arXiv
-
[26]
Ionescu, O
C. Ionescu, O. Vantzos, and C. Sminchisescu. Matrix Backpropagation for Deep Networks with Structured Layers. In 2015 IEEE International Conference on Computer Vision (ICCV), pages 2965–2973, Santiago, Chile, Dec. 2015. IEEE
2015
-
[27]
Jaquier and S
N. Jaquier and S. Calinon. Gaussian mixture regression on symmetric positive definite matrices manifolds: Application to wrist motion estimation with sEMG. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 59–64, Vancouver, BC, Sept. 2017. IEEE
2017
-
[28]
H. Karcher. Riemannian center of mass and mollifier smoothing. Communications on Pure and Applied Mathematics, 30(5):509–541, Sept. 1977
1977
-
[29]
Lecun, L
Y . Lecun, L. Bottou, Y . Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, Nov. 1998
1998
-
[30]
P. Li, J. Xie, Q. Wang, and Z. Gao. Towards Faster Training of Global Covariance Pooling Networks by Iterative Matrix Square Root Normalization. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 947–955, Salt Lake City, UT, June 2018. IEEE
2018
-
[31]
Y . Mao, R. Wang, S. Shan, and X. Chen. COSONet: Compact Second-Order Network for Video Face Recognition. page 16
-
[32]
Marceau-Caron and Y
G. Marceau-Caron and Y . Ollivier. Natural Langevin Dynamics for Neural Networks. In F. Nielsen and F. Barbaresco, editors,Geometric Science of Information, volume 10589, pages 451–459. Springer International Publishing, Cham, 2017
2017
-
[33]
Müller, T
M. Müller, T. Röder, M. Clausen, B. Eberhardt, B. Krüger, and A. Weber. Documentation mocap database hdm05. Technical Report CG-2007-2, Universität Bonn, June 2007
2007
-
[34]
Paszke, S
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in PyTorch. Oct. 2017
2017
-
[35]
Pennec, P
X. Pennec, P. Fillard, and N. Ayache. A Riemannian Framework for Tensor Computing. International Journal of Computer Vision, 66(1):41–66, Jan. 2006
2006
-
[36]
C. R. Rao. Information and the Accuracy Attainable in the Estimation of Statistical Parameters. In Breakthroughs in Statistics, Springer Series in Statistics, pages 235–247. Springer, New York, NY , 1992. 10
1992
-
[37]
S. Said, L. Bombrun, Y . Berthoumieu, and J. Manton. Riemannian Gaussian Distributions on the Space of Symmetric Positive Definite Matrices. arXiv:1507.01760 [math, stat], July 2015. arXiv: 1507.01760
2015 arXiv
-
[38]
Tuzel, F
O. Tuzel, F. Porikli, and P. Meer. Region Covariance: A Fast Descriptor for Detection and Classification. In Computer Vision – ECCV 2006, Lecture Notes in Computer Science, pages 589–600. Springer, Berlin, Heidelberg, May 2006
2006
-
[39]
O. Yair, M. Ben-Chen, and R. Talmon. Parallel Transport on the Cone Manifold of SPD Matrices for Domain Adaptation. July 2018
2018
-
[40]
L. Yang, M. Arnaudon, and F. Barbaresco. Riemannian median, geometry of covariance matrices and radar target detection. In The 7th European Radar Conference, pages 415–418, Sept. 2010
2010
-
[41]
L. Yang, M. Arnaudon, and F. Barbaresco. Riemannian median, geometry of covariance matrices and radar target detection. pages 415–418, Nov. 2010
2010
-
[42]
F. Yger. A review of kernels on covariance matrices for BCI applications. In 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), pages 1–6, Sept. 2013
2013
-
[43]
Yger and M
F. Yger and M. Sugiyama. Supervised LogEuclidean Metric Learning for Symmetric Positive Definite Matrices. arXiv:1502.03505 [cs], Feb. 2015. arXiv: 1502.03505
2015 arXiv
-
[44]
Zhang, W
T. Zhang, W. Zheng, Z. Cui, and C. Li. Deep Manifold-to-Manifold Transforming Network. In 2018 25th IEEE International Conference on Image Processing (ICIP), pages 4098–4102, Oct. 2018. 11
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.