DiffeoMorph: Learning to Morph 3D Shapes Using Differentiable Agent-Based Simulations
Pith reviewed 2026-05-16 21:01 UTC · model grok-4.3
The pith
DiffeoMorph trains shared agent rules so populations self-organize into complex 3D target shapes from minimal initial patterns.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
DiffeoMorph is an end-to-end differentiable framework for learning a morphogenesis protocol in which agents update position and internal state through an SE(3)-equivariant graph neural network; the system is trained by a new loss that compares predicted and target shapes as continuous spatial distributions expressed with 3D Zernike polynomials, after an alignment step that rotates the predicted spectrum to best match the target while preserving reflection sensitivity, thereby enabling formation of a range of complex 3D shapes from minimally patterned initial conditions.
What carries the argument
SE(3)-equivariant graph neural network for local agent updates together with the 3D Zernike polynomial loss plus optimal rotation alignment that compares shapes as continuous distributions.
If this is right
- Distributed control strategies can be learned that achieve precise global 3D patterns from local interactions alone.
- Shape comparison becomes independent of agent ordering, number, and orientation, allowing flexible population sizes.
- The same framework supplies a general method for training swarm behaviors in robotics and self-assembly systems.
- Benchmarking shows the Zernike loss outperforms standard point-cloud distance metrics on shape-matching tasks.
Where Pith is reading between the lines
- If the simulation faithfully captures physical interactions, the learned protocols could be transferred to real-world robotic or biological experiments.
- The approach could be extended to growing or dividing agent populations by making the graph neural network handle variable node counts during training.
- Connections emerge between multi-agent learning and classical models of pattern formation that rely on continuous reaction-diffusion fields.
Load-bearing premise
The learned protocols produced by the differentiable Zernike-loss simulation will continue to work when applied to shapes or agent counts outside the specific training demonstrations.
What would settle it
Train a protocol on one collection of target shapes and a fixed agent count, then test whether the identical protocol produces the correct morphology for an untrained target shape or with a substantially different number of agents.
Figures
read the original abstract
Biological systems can form complex three-dimensional structures through the collective behavior of agents that share a common update rule and operate without central control. How such distributed control gives rise to precise global patterns remains a central question not only in developmental biology but also in distributed robotics, programmable matter, and multi-agent learning. Here, we introduce DiffeoMorph, an end-to-end differentiable framework for learning a morphogenesis protocol that guides a population of agents to morph into a target 3D shape. Each agent updates its position and internal state using an SE(3)-equivariant graph neural network, based on its own internal state and signals received from other agents. To train this system, we introduce a new shape-matching loss based on 3D Zernike polynomials, which compares the predicted and target shapes as continuous spatial distributions, not as discrete point clouds, and is invariant to agent ordering, number of agents, and global orientation. To achieve rotation invariance while preserving reflection sensitivity, we include an alignment step that optimally rotates the predicted Zernike spectrum to match the target before computing the loss. We perform benchmarking to establish the advantages of our shape-matching loss over other standard distance metrics for shape comparison tasks. We then demonstrate that DiffeoMorph can form a range of complex shapes from minimally patterned initial conditions. DiffeoMorph provides a general framework for learning distributed control strategies for morphogenesis, swarm robotics, and programmable self-assembly.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces DiffeoMorph, an end-to-end differentiable framework in which agents update positions and states via an SE(3)-equivariant GNN to form target 3D shapes from minimal initial conditions. Training uses a novel Zernike-polynomial shape-matching loss that treats shapes as continuous distributions and is invariant to agent count, ordering, and global orientation (with an explicit alignment step for rotation invariance). The manuscript reports benchmarking of this loss against standard metrics and demonstrations that the system can produce a range of complex morphologies.
Significance. If the central claims hold, the work provides a general, differentiable approach to learning distributed control protocols for morphogenesis and self-assembly. The Zernike loss and SE(3)-equivariant architecture are concrete technical contributions that address invariance issues in multi-agent shape formation; successful transfer of learned protocols would have clear implications for swarm robotics and programmable matter.
major comments (2)
- [Experiments] Experiments section: the reported demonstrations are confined to the same shapes used during training; no held-out shape tests, cross-shape protocol transfer metrics, or agent-count sweeps are described. This leaves the central claim—that the GNN discovers transferable update rules rather than per-shape policies—unsupported by the presented evidence.
- [§4] §4 (demonstrations): without quantitative metrics on generalization (e.g., success rate on unseen target shapes or different N), it remains possible that the learned policies overfit the training morphologies, undermining the assertion of a general framework for distributed morphogenesis.
minor comments (2)
- [Abstract] The abstract states that benchmarking establishes advantages of the Zernike loss but does not name the competing metrics or report quantitative differences; this detail should be added for clarity.
- [Method] Notation for the Zernike spectrum alignment step is introduced without an explicit equation reference; adding a numbered equation would improve reproducibility.
Simulated Author's Rebuttal
We thank the referee for the constructive and detailed feedback. The comments correctly identify that the current experimental section would benefit from explicit generalization tests. We address each major comment below and will revise the manuscript to incorporate the suggested experiments.
read point-by-point responses
-
Referee: [Experiments] Experiments section: the reported demonstrations are confined to the same shapes used during training; no held-out shape tests, cross-shape protocol transfer metrics, or agent-count sweeps are described. This leaves the central claim—that the GNN discovers transferable update rules rather than per-shape policies—unsupported by the presented evidence.
Authors: We agree that the presented demonstrations use shapes from the training distribution and that held-out evaluations are needed to substantiate transferability. In the revised manuscript we will add a dedicated generalization section that includes: (i) training on a subset of target morphologies and reporting success rates on held-out shapes, (ii) cross-shape protocol transfer metrics, and (iii) agent-count sweeps across different values of N. These additions will directly support the claim that the SE(3)-equivariant GNN learns distributed update rules that generalize rather than memorizing per-shape policies. revision: yes
-
Referee: [§4] §4 (demonstrations): without quantitative metrics on generalization (e.g., success rate on unseen target shapes or different N), it remains possible that the learned policies overfit the training morphologies, undermining the assertion of a general framework for distributed morphogenesis.
Authors: We acknowledge that the current version lacks quantitative generalization metrics, which leaves open the possibility of overfitting. The revision will include success-rate tables for unseen target shapes and for varying agent counts N, together with the corresponding protocol-transfer experiments. These quantitative results will strengthen the evidence that DiffeoMorph provides a general framework rather than shape-specific solutions. revision: yes
Circularity Check
No significant circularity in derivation chain
full rationale
The paper defines an independent SE(3)-equivariant GNN update rule and a new Zernike-polynomial shape-matching loss that operates on continuous distributions, invariant to agent count and ordering. Training is end-to-end differentiable against target shapes, with explicit benchmarking of the loss against standard metrics. No equations reduce the target morphology or learned protocols to quantities fitted from the same data; no self-citations are load-bearing for the central claims; the architecture and loss are introduced as novel components rather than derived from prior author work by construction. The framework remains self-contained against external shape-comparison benchmarks.
Axiom & Free-Parameter Ledger
free parameters (1)
- GNN weights and biases
axioms (1)
- domain assumption SE(3)-equivariance of the graph neural network update rule
invented entities (1)
-
Zernike-polynomial shape representation for loss
no independent evidence
Reference graph
Works this paper leans on
-
[1]
to compute spherical harmonicsY ℓm. We usevmapfunction to vectorize the projection over eachn,ℓ, andm in equation (5). Since the shape of tensors passed tovmapshould be identical, for eachℓ-row ofY ℓm, we assign the expression of them-th real spherical harmonics starting from the first column and zero-padded the azimuthalm indices after 2ℓ+1 up to 2ℓ max+...
work page 2000
-
[2]
S. F. Gilbert and M. J. F. Barresi,Developmental Biology, 12th ed. (Sinauer Associates, Oxford University Press, New York, NY, 2020)
work page 2020
-
[3]
P. Hogeweg, Evolving mechanisms of morphogenesis: on the interplay between differential adhesion and cell differentiation, Journal of Theoretical Biology203, 317 (2000)
work page 2000
-
[4]
B. P. Teague, P. Guye, and R. Weiss, Synthetic morphogenesis, Cold Spring Harbor perspectives in biology8, a023929 (2016)
work page 2016
-
[5]
M. Takeichi, Dynamic contacts: rearranging adherens junctions to drive epithelial remodelling, Nature reviews Molecular cell biology15, 397 (2014)
work page 2014
-
[6]
J. B. Gurdon and P.-Y. Bourillot, Morphogen gradient interpretation, Nature413, 797 (2001)
work page 2001
-
[7]
G. M. Whitesides and B. Grzybowski, Self-assembly at all scales, Science295, 2418 (2002)
work page 2002
-
[8]
K. A. Athanasiou, R. Eswaramoorthy, P. Hadidi, and J. C. Hu, Self-organization and the self-assembling process in tissue engineering, Annual review of biomedical engineering15, 115 (2013)
work page 2013
-
[9]
S. C. Goldstein, J. D. Campbell, and T. C. Mowry, Programmable matter, Computer38, 99 (2005)
work page 2005
-
[10]
M. Rubenstein, A. Cornejo, and R. Nagpal, Programmable self-assembly in a thousand-robot swarm, Science345, 795 (2014)
work page 2014
-
[11]
A. Fatehullah, S. H. Tan, and N. Barker, Organoids as an in vitro model of human development and disease, Nature cell biology18, 246 (2016)
work page 2016
-
[12]
Clevers, Modeling development and disease with organoids, Cell165, 1586 (2016)
H. Clevers, Modeling development and disease with organoids, Cell165, 1586 (2016)
work page 2016
-
[13]
R. S. Sutton, A. G. Barto,et al.,Reinforcement learning: An introduction, Vol. 1 (MIT press Cambridge, 1998)
work page 1998
-
[14]
R. Lowe, Y. I. Wu, A. Tamar, J. Harb, O. Pieter Abbeel, and I. Mordatch, Multi-agent actor-critic for mixed cooperative- competitive environments, Advances in neural information processing systems30(2017)
work page 2017
- [15]
- [16]
-
[17]
C. Lin, T. Fan, W. Wang, and M. Nießner, Modeling 3d shapes by reinforcement learning, inComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16(Springer, 2020) pp. 545–561
work page 2020
-
[18]
J. Viquerat, J. Rabault, A. Kuhnle, H. Ghraieb, A. Larcher, and E. Hachem, Direct shape optimization through deep reinforcement learning, Journal of Computational Physics428, 110080 (2021)
work page 2021
-
[19]
R. Deshpande, F. Mottes, A.-D. Vlad, M. P. Brenner, and A. Dal Co, Engineering morphogenesis of cell clusters with differentiable programming, Nature Computational Science5, 875 (2025)
work page 2025
-
[20]
A. Mordvintsev, E. Randazzo, E. Niklasson, and M. Levin, Growing neural cellular automata, Distill5, e23 (2020)
work page 2020
-
[21]
R. B. Palm, M. Gonz´ alez-Duque, S. Sudhakaran, and S. Risi, Variational neural cellular automata, inProceedings of the 10th International Conference on Learning Representations (ICLR)(2022)
work page 2022
- [22]
-
[23]
D. Grattarola, L. Livi, and C. Alippi, Learning graph cellular automata, inAdvances in Neural Information Processing Systems, Vol. 34 (2021) pp. 20983–20994
work page 2021
-
[24]
M. Tesfaldet, D. Nowrouzezahrai, and C. Pal, Attention-based neural cellular automata, inAdvances in Neural Information Processing Systems, Vol. 35 (Curran Associates, Inc., 2022) pp. 8174–8186
work page 2022
- [25]
-
[26]
M. Novotni and R. Klein, Shape retrieval using 3d zernike descriptors, Computer-Aided Design36, 1047 (2004)
work page 2004
-
[27]
M. Kazhdan, T. Funkhouser, and S. Rusinkiewicz, Rotation invariant spherical harmonic representation of 3 d shape descriptors, inSymposium on geometry processing, Vol. 6 (2003) pp. 156–164
work page 2003
-
[28]
M. Novotni and R. Klein, 3d zernike descriptors for content based shape retrieval, inProceedings of the eighth ACM symposium on Solid modeling and applications(2003) pp. 216–225
work page 2003
-
[29]
Scoccimarro, The bispectrum: from theory to observations, The Astrophysical Journal544, 597 (2000)
R. Scoccimarro, The bispectrum: from theory to observations, The Astrophysical Journal544, 597 (2000)
work page 2000
-
[30]
R. Kakarala, The bispectrum as a source of phase-sensitive invariants for fourier descriptors: a group-theoretic approach, Journal of Mathematical Imaging and Vision44, 341 (2012)
work page 2012
- [31]
-
[32]
H. Fan, H. Su, and L. J. Guibas, A point set generation network for 3d object reconstruction from a single image, in Proceedings of the IEEE conference on computer vision and pattern recognition(2017) pp. 605–613. 18
work page 2017
- [33]
-
[34]
G. Gala, D. Grattarola, and E. Quaeghebeur, E (n)-equivariant graph neural cellular automata, Transactions on Machine Learning Research2024(2024)
work page 2024
-
[35]
F. M´ emoli, Gromov–wasserstein distances and the metric approach to object matching, Foundations of computational mathematics11, 417 (2011)
work page 2011
-
[36]
V. G. Satorras, E. Hoogeboom, and M. Welling, E (n) equivariant graph neural networks, inInternational conference on machine learning(PMLR, 2021) pp. 9323–9332
work page 2021
-
[37]
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, Attention is all you need, Advances in neural information processing systems30(2017)
work page 2017
-
[38]
R. T. Chen, Y. Rubanova, J. Bettencourt, and D. K. Duvenaud, Neural ordinary differential equations, Advances in neural information processing systems31(2018)
work page 2018
-
[39]
B. Tzen and M. Raginsky, Neural stochastic differential equations: Deep latent gaussian models in the diffusion limit, arXiv preprint arXiv:1905.09883 (2019)
-
[40]
Kidger,On Neural Differential Equations, Ph.D
P. Kidger,On Neural Differential Equations, Ph.D. thesis, University of Oxford (2021)
work page 2021
-
[41]
M. Cuturi, Sinkhorn distances: Lightspeed computation of optimal transport, Advances in neural information processing systems26(2013)
work page 2013
-
[42]
G. Peyr´ e, M. Cuturi,et al., Computational optimal transport: With applications to data science, Foundations and Trends® in Machine Learning11, 355 (2019)
work page 2019
-
[43]
´A. Gonz´ alez, Measurement of areas on a sphere using fibonacci and latitude–longitude lattices, Mathematical Geosciences 42, 49 (2010)
work page 2010
-
[44]
R. Bridson, Fast poisson disk sampling in arbitrary dimensions, inACM SIGGRAPH 2007 sketches(ACM, 2007) p. 22
work page 2007
-
[45]
O. Wartlick, A. Kicheva, and M. Gonz´ alez-Gait´ an, Morphogen gradient formation, Cold Spring Harbor perspectives in biology1, a001255 (2009)
work page 2009
-
[46]
J. Briscoe and S. Small, Morphogen rules: design principles of gradient-mediated embryo patterning, Development142, 3996 (2015)
work page 2015
-
[47]
R. Harland and J. Gerhart, Formation and function of spemann’s organizer, Annual review of cell and developmental biology13, 611 (1997)
work page 1997
-
[48]
E. M. De Robertis, Spemann’s organizer and self-regulation in amphibian embryos, Nature reviews Molecular cell biology 7, 296 (2006)
work page 2006
-
[49]
Jaeger, The gap gene network, Cellular and Molecular Life Sciences68, 243 (2011)
J. Jaeger, The gap gene network, Cellular and Molecular Life Sciences68, 243 (2011)
work page 2011
-
[50]
A. F. Schier, Nodal morphogens, Cold Spring Harbor perspectives in biology1, a003459 (2009)
work page 2009
-
[51]
E. M. De Robertis and H. Kuroda, Dorsal-ventral patterning and neural induction in xenopus embryos, Annu. Rev. Cell Dev. Biol.20, 285 (2004)
work page 2004
- [52]
-
[53]
U. Alon,An introduction to systems biology: design principles of biological circuits(Chapman and Hall/CRC, 2019)
work page 2019
-
[54]
E. H. Davidson and M. S. Levine, Properties of developmental gene regulatory networks, Proceedings of the National Academy of Sciences105, 20063 (2008)
work page 2008
-
[55]
M. Murrell, P. W. Oakes, M. Lenz, and M. L. Gardel, Forcing cells into shape: the mechanics of actomyosin contractility, Nature reviews Molecular cell biology16, 486 (2015)
work page 2015
-
[56]
B. M. Gumbiner, Regulation of cadherin-mediated adhesion in morphogenesis, Nature reviews Molecular cell biology6, 622 (2005)
work page 2005
-
[57]
M. Levin, Endogenous bioelectrical networks store non-genetic patterning information during development and regenera- tion, The Journal of physiology592, 2295 (2014)
work page 2014
-
[58]
C. H. Waddington,The strategy of the genes(Routledge, 2014)
work page 2014
-
[59]
L. Wolpert, C. Tickle, and A. M. Arias,Principles of development(Oxford University Press, USA, 2015)
work page 2015
- [60]
-
[61]
C. M. Glen, M. L. Kemp, and E. O. Voit, Agent-based modeling of morphogenetic systems: Advantages and challenges, PLoS computational biology15, e1006577 (2019)
work page 2019
-
[62]
G. Guan, S. Wang, T. G. Shields, S. H. Pahng, C. X. Shao, J. Ye, C. Budjan, and S. Hormoz, Cooperative short-and long-range interactions enable robust symmetry breaking and axis formation, bioRxiv , 2025 (2025)
work page 2025
-
[63]
J. A. Sherratt and J. D. Murray, Models of epidermal wound healing, Proceedings of the Royal Society of London. Series B: Biological Sciences241, 29 (1990)
work page 1990
-
[64]
D. Lobo, W. S. Beane, and M. Levin, Modeling planarian regeneration: a primer for reverse-engineering the worm, PLoS computational biology8, e1002481 (2012)
work page 2012
-
[65]
J. S. Yodh, Y. Lin, S. Sinha, V. Krishnan, L. Mahadevan, and D. J. Cohen, Optimal bioelectric control accelerates collective wound healing, bioRxiv , 2025 (2025)
work page 2025
-
[66]
V. G. Kim, W. Li, N. J. Mitra, S. Chaudhuri, S. DiVerdi, and T. Funkhouser, Learning part-based templates from large collections of 3d shapes, ACM Transactions on Graphics (TOG)32, 1 (2013)
work page 2013
-
[67]
J. Gray and G. J. Hancock, The propulsion of sea-urchin spermatozoa, Journal of Experimental Biology32, 802 (1955). 19
work page 1955
-
[68]
A. J. Ijspeert, Central pattern generators for locomotion control in animals and robots: a review, Neural networks21, 642 (2008)
work page 2008
-
[69]
D. P. Kingma, Adam: A method for stochastic optimization, arXiv preprint arXiv:1412.6980 (2014)
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[70]
Hu, Angular trispectrum of the cosmic microwave background, Physical Review D64, 083005 (2001)
W. Hu, Angular trispectrum of the cosmic microwave background, Physical Review D64, 083005 (2001)
work page 2001
-
[71]
e3nn: Euclidean neural net- works,
M. Geiger and T. Smidt, e3nn: Euclidean neural networks, arXiv preprint arXiv:2207.09453 (2022)
-
[72]
Searching for Activation Functions
P. Ramachandran, B. Zoph, and Q. V. Le, Searching for activation functions, arXiv preprint arXiv:1710.05941 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[73]
L. McInnes, J. Healy, N. Saul, and L. Grossberger, Umap: Uniform manifold approximation and projection, The Journal of Open Source Software3, 861 (2018)
work page 2018
-
[74]
V. A. Traag, L. Waltman, and N. J. Van Eck, From louvain to leiden: guaranteeing well-connected communities, Scientific reports9, 1 (2019)
work page 2019
-
[75]
F. A. Wolf, P. Angerer, and F. J. Theis, Scanpy: large-scale single-cell gene expression data analysis, Genome biology19, 15 (2018)
work page 2018
-
[76]
B. C. Hall, Lie groups, lie algebras, and representations, inQuantum Theory for Mathematicians(Springer, 2013) pp. 333–366
work page 2013
-
[77]
W. Fulton and J. Harris,Representation theory: a first course, Vol. 129 (Springer Science & Business Media, 2013)
work page 2013
-
[78]
K. Shoemake, Animating rotation with quaternion curves, inProceedings of the 12th annual conference on Computer graphics and interactive techniques(1985) pp. 245–254
work page 1985
-
[79]
J. You, Z. Ying, and J. Leskovec, Design space for graph neural networks, Advances in Neural Information Processing Systems33, 17009 (2020). 20 Appendix 1: Mathematical Properties of 3D Zernike Polynomials In this section, we briefly summarize the key properties of the 3D Zernike polynomials. For a complete discussion, we refer readers to Niu and Tian [24...
work page 2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.