REVIEW 3 major objections 5 minor 32 references
Moment kernels: a simple and scalable approach for equivariance to rotations and reflections in deep convolutional networks
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper proves that every O(d)-equivariant convolution kernel between tensor-valued feature fields is a linear combination of radial functions of |x| multiplied by products of coordinate components and Kronecker deltas, a family it…
desk verdict A genuinely simple and useful Cartesian parameterization of O(d)-equivariant kernels, but the completeness theorem is classical and the exactness claim is overstated for the actual registration task. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the moment-kernel signature: a kernel of rank $r$ built from a learned radial function $f(|x|)$, a product of $r$ coordinate components $x^{i_1}\cdots x^{i_r}$, and zero or more Kronecker deltas pairing chosen tensor indices, so a signature is a set of unordered index pairs with no repeats (for example, $\emptyset$ for scalar-style kernels and $\{\{1,2\}\}$ for the identity-style term in a vector-to-vector kernel). The proof machinery is the stabilizer argument: evaluate a candidate kernel at a point on the first coordinate axis, apply rotations and reflections that leave that axis fixed, and force the kernel value to be unchanged; this shows any slot not aligned with the axis must appear in an identical pair, and pairs must sum over all basis vectors, producing the resolution of identity $\sum_i e_i e_i^T$. Repeating the pairing on the remaining factors leaves only products of $x$ components and Kronecker deltas, which is exactly the moment-kernel family.
What would settle it
Convolve a small cubic volume with a moment-kernel network, rotate the input by 90 degrees about a grid axis, and compare outputs: any nonzero difference would disprove the exact-equivariance claim. For the completeness theorem, generate a smooth $O(d)$-equivariant kernel outside the moment-kernel family, for instance by a harmonic-basis expansion with a generic radial profile, and check numerically whether its projection onto the moment-kernel basis has nonzero residual; any residual would disprove the claim that all equivariant kernels take this form.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is a completeness theorem: every real $O(d)$-equivariant convolution kernel that maps between tensor fields can be written as a linear combination of radial functions of $|x|$ multiplied by products of coordinate components $x^{i_1}\cdots x^{i_r}$, where any pair of factors may be replaced by the Kronecker delta $\delta_{i_a i_b}$. Section 2.3 states this as the exact span of the moment-kernel signatures, and Appendix B gives a constructive proof by examining a kernel on one axis, applying rotations and reflections that fix that axis, and showing that non-axis tensor slots must occur in identical pairs that sum into the identity. The discrete networks built from these kernels are exactly equivariant for grid-preserving rotations and reflections, namely 90-degree rotations and reflections in the experiments, and approximately equivariant for other angles. Because the kernels are parameterized with standard convolution modules, the same formulas apply in 2D and 3D without orientation-channel expansion.
Load-bearing premise
The practical claim of exact consistency rests on the unstated assumption that resampling a one-dimensional radial function into a $3\times3\times3$ (or $3\times3$) cube by linear interpolation is exactly equivariant on that grid and that the image array itself is preserved by the transform; the paper itself notes that a non-cube $80\times80\times90$ volume makes some 90-degree rotations change cropping, so exact consistency does not hold there.
Editorial extensions
If this is right
- If the completeness theorem is correct, every equivariant convolution layer between tensor fields can be implemented with ordinary convolution by stacking moment kernels, so no group-convolution orientation expansion (8 channels in 2D, 48 in 3D for 90-degree rotations and reflections) is needed.
- Scalar-output networks built from moment kernels are invariant under all $O(d)$ transformations of the input, vector outputs transform covariantly as vectors, and matrix outputs transform as quadratic forms, which covers classification, affine-registration, and ellipse-detection objectives.
- For 3D affine registration, representing each column of the affine transform as a vector field lets an equivariant network predict orientation-dependent outputs while remaining trainable on standard hardware.
- Discrete moment-kernel networks are exactly consistent for rotations and reflections that preserve the grid, and only approximately consistent for arbitrary angles, so the guarantee is strongest for 90-degree rotations and reflections.
Reading between the lines
- The stabilizer proof implies a practical certification shortcut: for a kernel family to be $O(d)$-complete, it is enough to check equivariance under transformations fixing a single axis, which could simplify verification of other equivariant parameterizations.
- The moment-kernel basis is the convolution analogue of the classical invariant ring for $O(d)$; treating it this way suggests that any equivariant linear map from tensor fields can be built by contracting learned radial profiles against isotropic tensor products, connecting to existing results on isotropic tensor bases.
- Because exactness fails only through discretization, a testable extension is to design radial resampling schemes whose interpolation weights are invariant under the full cube rotation group, which would extend exact consistency beyond 90-degree rotations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces "moment kernels," a Cartesian parameterization of O(d)-equivariant convolution kernels between tensor-valued feature fields, and claims a complete characterization: every real equivariant kernel is a sum of radial functions of |x| multiplied by products of coordinate components and Kronecker deltas. The authors prove this in Appendix B for the continuous setting, implement a discrete version using standard convolution modules with linear-interpolated radial profiles, and evaluate on three biomedical tasks: invariant 2D classification (DermaMNIST), equivariant 3D affine registration on brain MRI, and elliptical-object detection for cell segmentation. The paper emphasizes simplicity, dimension-agnosticism, and exact consistency under grid-preserving rotations and reflections, and reports improved worst-case orientation consistency relative to a standard CNN.
Significance. If the completeness theorem and the discrete exactness claims are correct, the paper offers a practical and conceptually accessible alternative to harmonic-basis and group-convolution approaches for O(d)-equivariant networks, with a parameterization that generalizes across dimensions and tensor ranks. The constructive proof of uniqueness is a valuable contribution, as is the release of code and the demonstration on three distinct transformation laws. The paper explicitly frames its contributions as simple-to-use and scalable, which is credible for the provided architecture. However, the advertised exact-consistency guarantee is not achieved for the non-cube registration task, and the proof has informal steps that need tightening. The empirical comparison against an existing equivariant framework is weakened by the post-hoc removal of the ESCNN baseline. These issues do not invalidate the core idea but they do require revision before the claims as stated are fully supported.
major comments (3)
- [§2.4, §4.2, abstract] The exact-consistency claim is overstated for the registration experiment. Section 2.4 states that networks are "exactly equivariant for 90-degree rotations and reflections," and the abstract claims "exact consistency under grid-preserving rotations and reflections." Yet Section 4.2 reports that because the MRI volumes are 80×80×90 and not cubes, some 90-degree rotations change cropping, and "the distance between affine transformations output by our model from an image in different views is not exactly zero." A 90-degree rotation about the x- or y-axis maps the array to a different shape (80×90×80 or 90×80×80), so no cropping/padding back to 80×80×90 commutes with the convolution as a group action. Thus the registration experiment does not demonstrate the advertised exactness for those orientations. The claim should be restricted to shape-preserving grids (e.g., cubic volumes) or reformulated with an explicit statement about how cropping is handled; otherwise the "exact consistency" headline is misleading for the actual task.
- [Appendix B] The completeness proof has informal jumps that leave the theorem not fully rigorous as written. The argument in "Non e1 slots must be picked in pairs" rules out tuples larger than pairs only via a single rank-4 example and an analogy to superquadrics, not a general proof for arbitrary even tuple sizes. The step "we can act partially over this pair of slots, and then repeat the procedure until finished" is asserted rather than verified; it does not explicitly show that the resulting kernel remains equivariant and that iterating over all pair choices yields a basis that spans the full space of solutions to Eq. (14). Since the main theoretical contribution is the claim that all equivariant kernels are moment kernels, these gaps need to be closed, either by a more detailed inductive argument or by an alternative proof (e.g., via invariant theory).
- [§4.1 and Figure 1] The post-hoc removal of the ESCNN baseline weakens the empirical comparison. The text says ESCNN gave accuracy 0.676 and AUC 0.866, then "we remove their method from our figure" because "it is unlikely our network configuration was optimal." This is a selective-reporting concern: it removes an unfavorable comparison without providing evidence of misconfiguration beyond the result itself. To support the claim that moment kernels are competitive with or superior to existing equivariant frameworks, the paper should either report the ESCNN results in the figure with a caveat, or provide a fair comparison with a tuned ESCNN configuration (e.g., by treating hyperparameters as part of the comparison). At minimum, the exclusion should be justified with a sensitivity analysis or a reference to a proper comparison protocol.
minor comments (5)
- [§2.3, Eq. (18)] The text says "where fvv1 and fvv2 are scalar functions to be learned," but the equation defines fvv0 and fvv1; the naming should be consistent to avoid confusion.
- [§2.4] Several typos and unclear phrasings appear: "hybercube" should be "hypercube," "a linear acting" should be "a linear operator acting," and "resample it into a hybercube" is ambiguous about whether the radial function is sampled on a grid or evaluated at grid points. Also, "one radial function is randomly initialized for each" should specify the initialization distribution.
- [§5 (Limitations)] The limitations paragraph states that networks are "exactly equivariant to 90 degree rotations and reflections, but for rotations by other angles equivariance is only approximate." This is coherent, but it should appear earlier, in Section 2.4, and should be reconciled with the non-cube issue noted in the registration results.
- [Appendix B, rank-2 case] The phrase "the signed magnitude of the kernel at this point will be a learnable parameter" is imprecise: the magnitude is a function of |x|, not a single parameter, and the proof does not discuss learnability. The sentence should be rephrased to say that the value along the e1 axis can be arbitrary and defines the radial function.
- [Throughout] The paper uses " .=" to denote definitions, but the notation is not standard and could be replaced with a clear verbal statement or a standard symbol (e.g., "defined as") for readability.
Circularity Check
No circularity: the kernel-form theorem is derived from the equivariance constraint and external benchmarks; the exact-consistency caveat is a scope issue, not a circular step.
full rationale
The derivation chain is self-contained. Section 2.2 derives the equivariance constraint (Eq. 14) directly from the transformation laws (1)-(4), and Section 2.3 verifies that the moment-kernel forms (15)-(20) satisfy it. Appendix B then gives a constructive proof that any kernel satisfying Eq. (14) must be a linear combination of the enumerated signatures, starting from a basis of tensor products at a point on the e1 axis and using stabilizer rotations and reflections to force coefficients. The learned radial functions are free functions and never enter the theorem; the kernel form is forced by the group action alone, not by data fitting. No fitted parameter is renamed as a prediction: the reported orientation-consistency improvements are mathematical guarantees of the architecture, which is a design property rather than a circular empirical claim. Comparison numbers come from external MedMNIST baselines and standard networks. The reference list contains no load-bearing self-citations; the only related external work mentioned is [19], which is prior work by others and is not used to justify the theorem. One caveat is explicitly acknowledged in Section 4.2: "Because the 3D image array is not a cube, some 90 degree rotations change how the image is cropped, and so the distance between affine transformations output by our model from an image in different views is not exactly zero." This weakens the practical exact-consistency headline for the registration task, but it is a correctness or scope issue, not circular reasoning. Therefore no circular step is found.
Assumptions & free parameters
assumptions (4)
- domain assumption Equivariance is defined by the transformation law in Eq. (4) for real tensor fields under R in O(d).
- domain assumption A convolution kernel is an ordinary function on R^d, not a distribution, so it can be evaluated pointwise and expanded in the basis e_i1 tensor ... tensor e_ir.
- standard math The O(d) action is transitive on the unit sphere, so a kernel satisfying the constraint on the e1 axis can be extended to all directions by the action.
- ad hoc to paper Discrete resampling of a radial profile by linear interpolation is exactly equivariant for grid-preserving rotations and reflections.
Cite this review
Pith. "Pith review of Moment kernels: a simple and scalable approach for equivariance to rotations and reflections in deep convolutional networks." pith.science (2026). https://pith.science/paper/EECAB7AO
@misc{pith2026250521736,
author = {Pith},
title = {Pith review of: Moment kernels: a simple and scalable approach for equivariance to rotations and reflections in deep convolutional networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/EECAB7AO}},
note = {Machine review of arXiv:2505.21736}
}
abstract
Translation equivariance is a central reason convolutional neural networks have been successful in computer vision. Other symmetries, such as rotations and reflections, are similarly important in fields such as biomedical image analysis, but equivariant methods for these symmetries remain less widely adopted, especially in 3D. Existing approaches often rely on group convolutions, harmonic bases, irreducible representations, or specialized libraries, which can obscure the explicit form of admissible kernels for practitioners. We introduce moment kernels, a simple Cartesian parameterization of convolution kernels equivariant to orthogonal transformations, $O(d)$, between tensor-valued feature fields. We prove that every such $O(d)$-equivariant kernel can be represented as a sum of radial functions of $|x|$ multiplied by products of coordinate components $x^i$ and Kronecker deltas. This gives a complete, dimension-agnostic kernel family complementary to harmonic-basis approaches and implementable using standard convolution modules. We implement a discrete version of moment-kernel networks and evaluate on biomedical tasks with different transformation laws: invariant 2D image classification and equivariant 3D affine-transform regression for brain MRI. Across these tasks, moment kernels improve worst-case orientation consistency and remain trainable in 3D, while avoiding the orientation-channel expansion required by group convolutions, which reaches 48 orientations for 90-degree rotations and reflections in 3D. The resulting models provide exact consistency under grid-preserving rotations and reflections, and remain practical for standard CNN workflows.
Figures
Reference graph
Works this paper leans on
-
[1]
G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag, and A. V . Dalca. V oxelmorph: a learning framework for deformable medical image registration. IEEE transactions on medical imaging, 38(8):1788–1800, 2019
work page 2019
-
[2]
E. J. Bekkers, M. W. Lafarge, M. Veta, K. A. Eppenhof, J. P. Pluim, and R. Duits. Roto- translation covariant convolutional networks for medical image analysis. In Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I, pages 440–448. Springer, 2018
work page 2018
-
[3]
K. B. Bernander, J. Lindblad, R. Strand, and I. Nyström. Rotation-equivariant semantic instance segmentation on biomedical images. In Annual Conference on Medical Image Understanding and Analysis, pages 283–297. Springer, 2022
work page 2022
-
[4]
G. Bökman and F. Kahl. A case for using rotation invariant features in state of the art fea- ture matchers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5110–5119, 2022
work page 2022
-
[5]
J. Bruna and S. Mallat. Invariant scattering convolution networks. IEEE transactions on pattern analysis and machine intelligence, 35(8):1872–1886, 2013
work page 2013
-
[6]
G. Cesa, L. Lang, and M. Weiler. A program to build E(N)-equivariant steerable CNNs. In International Conference on Learning Representations, 2022. URL https://openreview. net/forum?id=WE4qe9xlnQw
work page 2022
-
[7]
T. Cohen and M. Welling. Group equivariant convolutional networks. In International confer- ence on machine learning, pages 2990–2999. PMLR, 2016
work page 2016
-
[8]
D. B. Ennis, G. Kindlman, I. Rodriguez, P. A. Helm, and E. R. McVeigh. Visualization of tensor fields using superquadric glyphs. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine, 53(1):169–176, 2005
work page 2005
Show all 32 references
-
[9]
V . S. Fonov, A. C. Evans, R. C. McKinstry, C. R. Almli, and D. Collins. Unbiased nonlinear average age-appropriate brain templates from birth to adulthood. NeuroImage, 47:S102, 2009
2009
-
[10]
W. T. Freeman, E. H. Adelson, et al. The design and use of steerable filters. IEEE Transactions on Pattern analysis and machine intelligence, 13(9):891–906, 1991
1991
-
[11]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770– 778, 2016
2016
-
[12]
Hoffmann, B
M. Hoffmann, B. Billot, D. N. Greve, J. E. Iglesias, B. Fischl, and A. V . Dalca. Synthmorph: learning contrast-invariant registration without acquired images. IEEE transactions on medical imaging, 41(3):543–558, 2021
2021
-
[13]
Ioffe and C
S. Ioffe and C. Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448–456. pmlr, 2015
2015
-
[14]
D. P. Kingma. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014
2014 arXiv
-
[15]
Klein, J
A. Klein, J. Andersson, B. A. Ardekani, J. Ashburner, B. Avants, M.-C. Chiang, G. E. Chris- tensen, D. L. Collins, J. Gee, P. Hellier, et al. Evaluation of 14 nonlinear deformation algorithms applied to human brain mri registration. Neuroimage, 46(3):786–802, 2009
2009
-
[16]
Krizhevsky, I
A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems, 25, 2012
2012
-
[17]
Lang and M
L. Lang and M. Weiler. A wigner-eckart theorem for group equivariant convolution kernels. arXiv preprint arXiv:2010.10952, 2020. 11
2010 arXiv
-
[18]
LeCun, B
Y . LeCun, B. Boser, J. Denker, D. Henderson, R. Howard, W. Hubbard, and L. Jackel. Hand- written digit recognition with a back-propagation network. Advances in neural information processing systems, 2, 1989
1989
-
[19]
Micheli and J
M. Micheli and J. A. Glaunes. Matrix-valued kernels for shape deformation analysis. arXiv preprint arXiv:1308.5739, 2013
2013 arXiv
-
[20]
Nishimaki, H
K. Nishimaki, H. Iyatomi, K. Oishi, t. A. I. B. Alzheimer’s Disease Neuroimaging Initiative, and L. flagship study of ageing. A neural network approach to identify left–right orientation of anatomical brain mri. Brain and Behavior, 15(2):e70299, 2025
2025
-
[21]
Pachitariu and C
M. Pachitariu and C. Stringer. Cellpose 2.0: how to train your own model. Nature methods, 19 (12):1634–1641, 2022
2022
-
[22]
Puglisi, D
L. Puglisi, D. C. Alexander, and D. Ravì. Enhancing spatiotemporal disease progression models via latent diffusion and prior knowledge. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 173–183. Springer, 2024
2024
-
[23]
Redmon, S
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016
2016
-
[24]
Ronneberger, P
O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In Medical image computing and computer-assisted intervention–MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pag...
2015
-
[25]
Stringer, T
C. Stringer, T. Wang, M. Michaelos, and M. Pachitariu. Cellpose: a generalist algorithm for cellular segmentation. Nature methods, 18(1):100–106, 2021
2021
-
[26]
Thomas, T
N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li, K. Kohlhoff, and P. Riley. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. arXiv preprint arXiv:1802.08219, 2018
2018 arXiv
-
[27]
R. Wang, A. Achim, R. Raele-Rolfe, Q. Tong, D. Bergen, C. Hammond, and S. Cross. Rotir: Rotation-equivariant network and transformers for zebrafish scale image registration. In Annual Conference on Medical Image Understanding and Analysis, pages 285–299. Springer, 2024
2024
-
[28]
Weiler, M
M. Weiler, M. Geiger, M. Welling, W. Boomsma, and T. S. Cohen. 3d steerable cnns: Learning rotationally equivariant features in volumetric data. Advances in Neural information processing systems, 31, 2018
2018
-
[29]
D. E. Worrall, S. J. Garbin, D. Turmukhambetov, and G. J. Brostow. Harmonic networks: Deep translation and rotation equivariance. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5028–5037, 2017
2017
-
[30]
J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni. Medmnist v2-a large-scale lightweight benchmark for 2d and 3d biomedical image classification. Scientific Data, 10(1):41, 2023
2023
-
[31]
X. Yang, R. Kwitt, M. Styner, and M. Niethammer. Quicksilver: Fast predictive image registration–a deep learning approach. NeuroImage, 158:378–396, 2017
2017
-
[32]
A first moment kernel
A. Zargari, B. R. Topacio, N. Mashhadi, and S. A. Shariati. Enhanced cell segmentation with limited training datasets using cycle generative adversarial networks. Iscience, 27(5), 2024. 12 A Proof of transformation law for tensor kernels Let f be an input tensor-valued functio...
2024
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.