pith. sign in

arxiv: 2506.22899 · v3 · submitted 2025-06-28 · 💻 cs.CV · cs.GR· cs.LG· cs.MA· eess.IV

Neural Cellular Automata: From Cells to Pixels

Ehsan Pajouheshgar , Yitao Xu , Ali Abbasi , Alexander Mordvintsev , Wenzel Jakob , Sabine S\"usstrunk This is my paper

Pith reviewed 2026-05-19 07:27 UTC · model grok-4.3

classification 💻 cs.CV cs.GRcs.LGcs.MAeess.IV
keywords neural cellular automataimplicit decoderhigh-resolution synthesisself-organizing systemsmorphogenesistexture synthesisreal-time renderingcoarse-to-fine modeling
0
0 comments X

The pith

Pairing a coarse-grid neural cellular automaton with a local implicit decoder enables high-resolution real-time outputs while preserving self-organizing dynamics.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Neural cellular automata rely on identical cells that repeatedly apply the same local rule to grow patterns, yet they have been limited to low resolutions because costs grow with grid size and information spreads only locally. This work runs the automaton on a coarse grid and adds a lightweight decoder that turns each cell's state plus its local coordinates into fine appearance values. Because the decoder is also local, the whole pipeline stays parallel and scalable to arbitrary resolutions. Task-specific losses let the system train efficiently for both growing patterns from seeds and synthesizing textures. Experiments on grids and meshes confirm that the models keep the regeneration, robustness, and spontaneous motion that define NCAs.

Core claim

The authors establish that an NCA evolving on a coarse grid, combined with an implicit decoder that maps cell states and local coordinates to appearance attributes, generates high-resolution outputs in real time. The hybrid system preserves the strictly local update rules and the characteristic self-organizing properties of NCAs because both the cellular updates and the decoding step operate without global communication. Task-specific losses for morphogenesis and texture synthesis supervise the high-resolution results with minimal extra memory cost.

What carries the argument

The central mechanism is the hybrid coarse-NCA plus implicit decoder, where the decoder is a lightweight local function that converts coarse cell states and relative coordinates into high-resolution pixel or voxel attributes.

If this is right

  • High-resolution outputs become feasible in real time for both 2D textures and 3D growth tasks.
  • The same trained model can render at any resolution without retraining.
  • Memory and compute demands no longer grow quadratically with output size.
  • Inference remains highly parallel because every operation stays local.
  • The approach extends to mesh domains while retaining self-organization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This architecture could be dropped into interactive graphics tools for procedural content that still grows or repairs itself.
  • The separation of coarse dynamics from fine decoding might apply to other locally updating simulators that currently hit resolution walls.
  • A direct test would be to measure whether long-range correlations introduced by the decoder ever violate the strictly local statistics of a pure NCA.
  • Varying decoder capacity independently of the NCA grid size offers a new knob for trading quality against speed in future applications.

Load-bearing premise

The implicit decoder can faithfully reconstruct high-resolution details and dynamics from coarse-grid cell states without introducing artifacts or breaking the local update rules.

What would settle it

Training the hybrid model on a standard NCA task and then observing that damaged high-resolution patterns fail to regenerate or that spontaneous dynamics disappear would show the central claim is false.

Figures

Figures reproduced from arXiv: 2506.22899 by Alexander Mordvintsev, Ali Abbasi, Ehsan Pajouheshgar, Sabine S\"usstrunk, Wenzel Jakob, Yitao Xu.

Figure 1
Figure 1. Figure 1: Summary of result. Our proposed method enables NCA to generate high-quality outputs with minimal extra cost. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Sample Output of Our Hybrid Model. The NCA evolves on a coarse 128 × 128 lattice while our Local Pat￾tern Producing Network (LPPN) renders an RGB image at 1024 × 1024 and, without retraining, at 8192 × 8192 in the magnified inset. on the space dimensionality. This limitation arises from a combination of multiple practical and architectural con￾straints. First, training time and memory scale quadrati￾cally … view at source ↗
Figure 3
Figure 3. Figure 3: Hybrid NCA + LPPN Overview. Left: The NCA operates on a coarse lattice of cells (in this example vertices of a mesh) Center: A sampling point p (red dot) inside a triangle primitive, whose vertices correspond to NCA cells si , sj , sk. The local coordinate u(p) expresses the point’s position inside the primitive, while the locally averaged cell state s¯(p) is obtained by interpolating the surrounding cell … view at source ↗
Figure 4
Figure 4. Figure 4: Representative examples of primitives. Vertices correspond to neighboring cells. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Local coordinate transformations. Raw u￾coordinates visualized as RGB suffer discontinuities at primitive boundaries. (Top Right) Applying trigonometric functions on cartesian coordinates enforces C 0 continuity at boundaries for rectangle primitives. Bottom Right (a): Sorting the barycentric coordinates enforces C 0 continuity but yields an imbalanced dynamic range (red color domi￾nates). (b) Applying an … view at source ↗
read the original abstract

Neural Cellular Automata (NCAs) are bio-inspired dynamical systems in which identical cells iteratively apply a learned local update rule to self-organize into complex patterns, exhibiting regeneration, robustness, and spontaneous dynamics. Despite their success in texture synthesis and morphogenesis, NCAs remain largely confined to low-resolution outputs. This limitation stems from (1) training time and memory requirements that grow quadratically with grid size, (2) the strictly local propagation of information that impedes long-range cell communication, and (3) the heavy compute demands of real-time inference at high resolution. In this work, we overcome this limitation by pairing an NCA that evolves on a coarse grid with a lightweight implicit decoder that maps cell states and local coordinates to appearance attributes, enabling the same model to render outputs at arbitrary resolution. Moreover, because both the decoder and NCA updates are local, inference remains highly parallelizable. To supervise high-resolution outputs efficiently, we introduce task-specific losses for morphogenesis (growth from a seed) and texture synthesis with minimal additional memory and computation overhead. Our experiments across 2D/3D grids and mesh domains demonstrate that our hybrid models produce high-resolution outputs in real-time, and preserve the characteristic self-organizing behavior of NCAs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript proposes scaling Neural Cellular Automata beyond low resolutions by evolving an NCA on a coarse grid and pairing it with a lightweight implicit decoder that maps coarse cell states plus local coordinates to high-resolution appearance attributes. Task-specific losses are introduced to supervise morphogenesis (growth from seed) and texture synthesis with low overhead. Experiments on 2D/3D grids and meshes are claimed to produce real-time high-resolution outputs while retaining NCA hallmarks of self-organization, regeneration, and robustness.

Significance. If the central claims hold, the work meaningfully extends NCAs to practical high-resolution regimes in graphics and simulation while preserving locality and parallelizability. The hybrid design and efficient supervision losses represent a pragmatic advance over pure high-resolution NCAs.

major comments (1)
  1. [Abstract and Experiments] Abstract and Experiments: The claim that the hybrid system 'preserve[s] the characteristic self-organizing behavior of NCAs' (regeneration after damage, robustness to perturbations, spontaneous dynamics) is load-bearing for the contribution. Because cell updates remain strictly on the coarse grid and high-frequency detail is synthesized by the decoder, it is unclear whether perturbations applied at render time propagate back to affect future coarse states or whether fine-scale coherence survives; task-specific losses supervise static outputs but do not explicitly enforce these dynamical properties.
minor comments (2)
  1. [Method] Clarify the precise parameterization of the implicit decoder (e.g., how local coordinates are encoded and whether the decoder is strictly local) and provide pseudocode or architecture diagram.
  2. [Experiments] Add quantitative metrics (e.g., damage-recovery curves or perturbation robustness scores) at multiple output resolutions to support the preservation claim.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. The concern about whether self-organizing properties are truly preserved in the hybrid system is well-taken, and we address it directly below with clarifications and a commitment to strengthen the manuscript.

read point-by-point responses

  1. Referee: The claim that the hybrid system 'preserve[s] the characteristic self-organizing behavior of NCAs' (regeneration after damage, robustness to perturbations, spontaneous dynamics) is load-bearing for the contribution. Because cell updates remain strictly on the coarse grid and high-frequency detail is synthesized by the decoder, it is unclear whether perturbations applied at render time propagate back to affect future coarse states or whether fine-scale coherence survives; task-specific losses supervise static outputs but do not explicitly enforce these dynamical properties.

    Authors: We agree that this point requires explicit clarification. The NCA dynamics, including local updates, regeneration, and robustness, occur exclusively on the coarse grid and are unchanged from prior NCA formulations; the decoder is a deterministic, feed-forward mapping applied after each coarse update and has no feedback into the state evolution. Perturbations (e.g., damage) are therefore applied to coarse cell states, after which the standard NCA rule continues to drive regeneration, with the resulting high-resolution output obtained via the decoder. Fine-scale coherence is preserved because the decoder is strictly local and uses the same coarse state plus relative coordinates at every point. The task-specific losses supervise appearance at selected timesteps, but the underlying NCA is trained over multi-step trajectories (as in standard NCA work), so dynamical properties emerge from the coarse model rather than from the losses alone. In the revised manuscript we will add a dedicated subsection with new experiments that apply coarse-grid damage and perturbations, visualize the subsequent high-resolution regeneration, and report quantitative robustness metrics. We will also insert explanatory text in the method and results sections to separate the coarse dynamics from the rendering stage. revision: yes

Circularity Check

0 steps flagged

No circularity: hybrid coarse-NCA plus independent decoder is self-contained

full rationale

The derivation introduces a coarse-grid NCA whose local update rules and self-organizing dynamics are taken from prior independent NCA literature, then adds a separate lightweight implicit decoder whose parameters are optimized via new task-specific losses on high-resolution targets. Neither component is defined in terms of the other by construction, no fitted parameter is relabeled as a prediction of the target behavior, and no uniqueness theorem or ansatz is smuggled through self-citation. Experiments on 2D/3D grids and meshes provide external falsifiability for the claim that high-resolution appearance and coarse-grid dynamics can coexist; the central architecture therefore does not reduce to its inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Review performed on abstract only; no explicit free parameters, axioms, or invented entities are identifiable beyond standard neural network training assumptions.

pith-pipeline@v0.9.0 · 5781 in / 1073 out tokens · 38945 ms · 2026-05-19T07:27:25.243777+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

31 extracted references · 31 canonical work pages · 1 internal anchor

  1. [1]

    ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems

    Francois Chollet, Mike Knoop, Gregory Kamradt, Bryan Landers, and Henry Pinkard. Arc-agi-2: A new chal- lenge for frontier ai reasoning systems. arXiv preprint arXiv:2505.11831, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  2. [2]

    Cellular texture generation

    Kurt W Fleischer, David H Laidlaw, Bena L Currin, and Alan H Barr. Cellular texture generation. In Proceedings of the 22nd annual conference on Computer graphics and interac- tive techniques, pages 239–248, 1995

    work page 1995

  3. [3]

    Cellular automata as convolutional neural net- works

    William Gilpin. Cellular automata as convolutional neural net- works. Physical Review E, 100(3):032402, 2019

    work page 2019

  4. [4]

    3d surface cellular au- tomata and their applications

    St ´ephane Gobron and Norishige Chiba. 3d surface cellular au- tomata and their applications. The Journal of Visualization and Computer Animation, 10(3):143–158, 1999

    work page 1999

  5. [5]

    Arc-nca: Towards developmen- tal solutions to the abstraction and reasoning corpus

    Etienne Guichard, Felix Reimers, Mia Kvalsund, Mikkel Lep- perød, and Stefano Nichele. Arc-nca: Towards developmen- tal solutions to the abstraction and reasoning corpus. arXiv preprint arXiv:2505.08778, 2025

    work page arXiv 2025

  6. [6]

    Med-NCA: Robust and lightweight segmentation with neu- ral cellular automata

    John Kalkhof, Camila Gonz ´alez, and Anirban Mukhopadhyay. Med-NCA: Robust and lightweight segmentation with neu- ral cellular automata. In International Conference on In- formation Processing in Medical Imaging , pages 705–716. Springer, 2023

    work page 2023

  7. [7]

    Frequency-time diffusion with neural cel- lular automata

    John Kalkhof, Arlene K ¨uhn, Yannik Frisch, and Anirban Mukhopadhyay. Frequency-time diffusion with neural cel- lular automata. arXiv preprint arXiv:2401.06291, 2024

    work page arXiv 2024

  8. [8]

    The mokume dataset and inverse modeling of solid wood textures

    Maria Larsson, Hodaka Yamaguchi, Ehsan Pajouheshgar, I- Chao Shen, Kenji Tojo, Chia-Ming Chang, Lars Hansson, Olof Broman, Takashi Ijiri, Ariel Shamir, Wenzel Jakob, and Takeo Igarashi. The mokume dataset and inverse modeling of solid wood textures. ACM Transactions on Graphics, 44 (4):18 pages, August 2025. doi: 10.1145/3730874. URL https://doi.org/10.11...

    work page doi:10.1145/3730874 2025

  9. [9]

    Nerf: Representing scenes as neural radiance fields for view syn- thesis

    Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view syn- thesis. In European Conference on Computer Vision , pages 405–421, 2020

    work page 2020

  10. [10]

    Growing Neural Cellular Automata , volume =

    Alexander Mordvintsev, Ettore Randazzo, Eyvind Niklas- son, and Michael Levin. Growing neural cellular au- tomata. Distill, 2020. doi: 10.23915/distill.00023. https://distill.pub/2020/growing-ca

    work page doi:10.23915/distill.00023 2020

  11. [11]

    Instant neural graphics primitives with a multiresolu- tion hash encoding

    Thomas M ¨uller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolu- tion hash encoding. ACM transactions on graphics (TOG) , 41(4):1–15, 2022

    work page 2022

  12. [12]

    Theory of self-reproducing automata

    John von Neumann. Theory of self-reproducing automata. Mathematics of Computation, 21:745, 1966

    work page 1966

  13. [13]

    Self-organising textures

    Eyvind Niklasson, Alexander Mordvintsev, Ettore Randazzo, and Michael Levin. Self-organising textures. Distill, 6(2): e00027–003, 2021

    work page 2021

  14. [14]

    Generative adversarial neural cellular au- tomata

    Maximilian Otte, Quentin Delfosse, Johannes Czech, and Kris- tian Kersting. Generative adversarial neural cellular au- tomata. arXiv preprint arXiv:2108.04328, 2021

    work page arXiv 2021

  15. [15]

    Dynca: Real-time dynamic texture synthesis using neural cellular automata

    Ehsan Pajouheshgar, Yitao Xu, Tong Zhang, and Sabine S¨usstrunk. Dynca: Real-time dynamic texture synthesis using neural cellular automata. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 20742–20751, 2023

    work page 2023

  16. [16]

    Mesh neural cellular automata

    Ehsan Pajouheshgar, Yitao Xu, Alexander Mordvintsev, Eyvind Niklasson, Tong Zhang, and Sabine S¨usstrunk. Mesh neural cellular automata. ACM Trans. Graph. , 2024. doi: 10.1145/3658127. URL https://doi.org/10.1145/ 3658127

    work page doi:10.1145/3658127 2024

  17. [18]

    Variational neural cellular au- tomata

    Rasmus Berg Palm, Miguel Gonz ´alez-Duque, Shyam Sud- hakaran, and Sebastian Risi. Variational neural cellular au- tomata. arXiv preprint arXiv:2201.12360, 2022

    work page arXiv 2022

  18. [19]

    Deepsdf: Learning contin- uous signed distance functions for shape representation

    Jeong Joon Park, Peter Florence, Julian Straub, Richard New- combe, and Steven Lovegrove. Deepsdf: Learning contin- uous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 165–174, 2019

    work page 2019

  19. [20]

    Self-classifying mnist digits

    Ettore Randazzo, Alexander Mordvintsev, Eyvind Niklasson, Michael Levin, and Sam Greydanus. Self-classifying mnist digits. Distill, 2020. doi: 10.23915/distill.00027.002. https://distill.pub/2020/selforg/mnist

    work page doi:10.23915/distill.00027.002 2020

  20. [22]

    Image segmenta- tion via cellular automata

    Mark Sandler, Andrey Zhmoginov, Liangcheng Luo, Alexan- der Mordvintsev, Ettore Randazzo, et al. Image segmenta- tion via cellular automata. arXiv preprint arXiv:2008.04965, 2020

    work page arXiv 2008

  21. [23]

    Martel, Alexander W

    Vincent Sitzmann, Julien N.P. Martel, Alexander W. Bergman, David B. Lindell, and Gordon Wetzstein. Implicit neural representations with periodic activation functions. In Proc. NeurIPS, 2020

    work page 2020

  22. [24]

    Kenneth O. Stanley. Compositional pattern produc- ing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines , 8(2): 131–162, June 2007. ISSN 1389-2576. doi: 10. 1007/s10710-007-9028-8. URL https://doi.org/ 10.1007/s10710-007-9028-8

    work page doi:10.1007/s10710-007-9028-8 2007

  23. [25]

    Growing 3d artefacts and functional machines with neural cellular au- tomata

    Shyam Sudhakaran, Djordje Grbic, Siyan Li, Adam Katona, Elias Najarro, Claire Glanois, and Sebastian Risi. Growing 3d artefacts and functional machines with neural cellular au- tomata. In 2021 Conference on Artificial Life , 2021. URL https://arxiv.org/abs/2103.08737

    work page arXiv 2021

  24. [26]

    Goal-Guided Neural Cellular Automata: Learning to Control Self-Organising Systems , url =

    Shyam Sudhakaran, Elias Najarro, and Sebastian Risi. Goal- guided neural cellular automata: Learning to control self- organising systems. arXiv preprint arXiv:2205.06806, 2022

    work page arXiv 2022

  25. [27]

    Attention-based neural cellular automata

    Mattie Tesfaldet, Derek Nowrouzezahrai, and Christopher Pal. Attention-based neural cellular automata. arXiv preprint arXiv:2211.01233, 2022

    work page arXiv 2022

  26. [28]

    The chemical basis of morphogenesis

    AM Turing. The chemical basis of morphogenesis. Philosoph- ical Transactions of the Royal Society of London Series B , 237(641):37–72, 1952

    work page 1952

  27. [29]

    Generating textures on arbitrary surfaces using reaction-diffusion

    Greg Turk. Generating textures on arbitrary surfaces using reaction-diffusion. Acm Siggraph Computer Graphics, 25(4): 289–298, 1991

    work page 1991

  28. [30]

    V olumetric temporal texture synthesis for smoke stylization using neural cellular automata

    Dongqing Wang, Ehsan Pajouheshgar, Yitao Xu, Tong Zhang, and Sabine S¨usstrunk. V olumetric temporal texture synthesis for smoke stylization using neural cellular automata. arXiv preprint arXiv:2502.09631, 2025

    work page arXiv 2025

  29. [31]

    Emer- gent dynamics in neural cellular automata

    Yitao Xu, Ehsan Pajouheshgar, and Sabine S ¨usstrunk. Emer- gent dynamics in neural cellular automata. volume AL- IFE 2024: Proceedings of the 2024 Artificial Life Confer- ence of Artificial Life Conference Proceedings, page 96, 07

    work page 2024

  30. [32]

    URL https://doi

    doi: 10.1162/isal a 00744. URL https://doi. org/10.1162/isal_a_00744

    work page doi:10.1162/isal

  31. [33]

    Adanca: Neural cellular automata as adaptors for more robust vision trans- former

    Yitao Xu, Tong Zhang, and Sabine Susstrunk. Adanca: Neural cellular automata as adaptors for more robust vision trans- former. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    work page 2024