Pith. sign in

REVIEW 3 major objections 5 minor 41 references

Missing Data Imputation using Neural Cellular Automata

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A neural cellular automaton that grows missing values via self-attention claims state-of-the-art tabular imputation.

desk verdict Novel NCA-style attention growth for tabular imputation, but the negative missing-loss term in §4.1 makes the objective unbounded below, so the SOTA claim is unsupported without a fix and stronger baselines. read the letter →

arxiv 2509.00651 v2 pith:IOJ6VB5U submitted 2025-08-31 cs.LG

Tin Luu , Binh Nguyen , Man Ngo This is my paper
classification cs.LG
keywords missingdataimputationneuralcellularautomatatabularself-attentioncosineattentiongenerativeMCARNICA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces NICA, a model that imputes missing values in tabular data by treating rows as cells in a cellular automaton and replacing the usual local rule with a self-attention mechanism. The central claim is that after a fixed number of growing steps, the model regenerates unobserved entries and does so with lower imputation error than Mean, KNN, MICE, and GAIN on 14 of 15 benchmark datasets, and better downstream prediction in 11 of 15. A notable property is that training does not require a fully observed dataset, which matters for real-world settings where complete data is unavailable. If the claim holds, NICA offers a comparatively simple generative imputer that avoids adversarial or variational training.

What carries the argument

The central machinery is the NICA block: CosineAttention(Q,K,V)=softmax(sqrt(d_k) * cosine(Q,K)) V, where cosine similarity replaces the dot product, no value projection is used, and the attention output is concatenated with the current state, passed through a two-layer FNN, and added residual-style. This block is applied K times (K=10 in experiments) to grow the data. The design is justified as analogous to NCA's fixed kernels while allowing each row to dynamically find neighbors in the whole dataset. A Dropout1d layer skips 10% of rows each step to mimic asynchronous growth.

What would settle it

Run the same 15 benchmarks at 40% MCAR with NICA compared against MIWAE and HI-VAE (or a well-tuned MICE with predictive mean matching). If NICA no longer achieves the lowest RMSE on most datasets, the state-of-the-art claim is falsified. A second check: introduce MNAR missingness where blankness depends on the missing values themselves; NICA's attention-based growing would need to handle the bias.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that the NCA paradigm transfers from images to tabular data if the convolutional neighborhood is replaced by a similarity-based attention neighborhood. NICA starts from a zero-filled, randomly corrupted version of the incomplete table, then iteratively applies a block of cosine self-attention, a feedforward network, and a residual update K times. Three loss terms steer the process: recovering corrupted observed cells, matching observed cells, and forcing missing cells away from their zero initialization. The grown matrix supplies the imputed values. Reported results: lowest RMSE in 14/15 datasets at 40% missingness, with 11–34% improvements over GAI

Load-bearing premise

The load-bearing premise is that the four baselines tested—Mean, KNN, MICE with default settings, and GAIN—are sufficient to define 'state-of-the-art' imputation; if stronger or carefully tuned baselines had been included, the reported margins could shrink or disappear.

Editorial extensions

If this is right

  • If NICA is as robust as reported, it becomes a strong default imputer for MCAR tabular missingness, especially when missing rates are high (up to 80%).
  • Because it trains directly on incomplete data, it can be applied in settings where no complete ground-truth table exists.
  • The growing-process view suggests a middle ground between single-pass imputers and deep generative models: no adversarial training, no variational bound, just repeated refinement.
  • The reported clustering of rows during growth points to a learned similarity structure that could be reused for outlier detection or data visualization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The claim of state-of-the-art is tied to the four chosen baselines; comparing against more recent generative imputers (e.g., MIWAE, HI-VAE, or MissForest) would stress-test the conclusion, and the paper itself names those methods as stronger generative alternatives.
  • Cosine attention with a single head and no value projection is an unusual simplification; testing whether multi-head or projected attention helps on wide or high-dimensional tables would clarify where the design boundary lies.
  • The method is evaluated only under MCAR; extending the same growing rule to MAR or MNAR patterns, where missingness depends on observed or missing values, would be a natural next test.
  • The K-step iterative refinement resembles a discrete diffusion process; a continuous-depth version could connect NICA to neural ODEs and allow adaptive step counts per dataset.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Neural Imputation Cellular Automata (NICA), a generative model for missing tabular data imputation. NICA represents a data table as a grid of cells, uses cosine self-attention to define neighbors, and iteratively updates the seed through K steps. The loss combines a recovering loss on corrupted observed cells, an observed loss on known cells, and a negative missing loss that penalizes small changes from the zero imputation. Experiments on 15 datasets with 40% MCAR missingness compare NICA against Mean, KNN, MICE, and GAIN; the paper reports RMSE and downstream prediction accuracy, claiming lower error in 14/15 datasets and 'state-of-the-art' performance.

Significance. The paper introduces a novel, conceptually interesting application of neural cellular automata to tabular imputation, offering a lightweight alternative to deep generative models. Strengths include a public code repository, evaluation across 15 datasets with means/standard deviations and multiple missing rates, and downstream task validation. If the loss issue is resolved and the baseline set is strengthened, NICA could be a useful new imputer. However, current evidence for the state-of-the-art claim is weakened by an unbounded negative loss term and a narrow baseline selection.

major comments (3)
  1. [§4.1, definition of L_missing] The missing loss L_missing = -E_{1-M}[L(X_c^K, X̄_0)] is a negative squared error between the imputed values and zero. Because L_observed and L_recovering apply only to observed/corrupted cells, the objective is unbounded below: it rewards arbitrarily large values at missing cells. No clipping, bounded activation, margin, or regularization is described. This makes the reported stable, low RMSE results internally inconsistent with the stated objective. The authors must clarify the actual loss (e.g., a bound/clip or a different formulation) and, if necessary, rerun the experiments; otherwise the central claim cannot be evaluated.
  2. [§5.1.2, Baselines] The abstract and §5.2.1 claim 'state-of-the-art' performance, but the comparison includes only Mean, KNN, MICE with scikit-learn defaults, and GAIN. The paper's own literature review (§2.1) describes stronger generative imputers (MIWAE, HI-VAE, MIDA), and other self-attention imputation methods are not evaluated. Without these or otherwise tuned baselines, the reported 14/15 superiority may reflect baseline weakness rather than genuine SOTA performance. The authors should add stronger baselines or temper the claim.
  3. [§5.2.1, Table 2] The claim of 'outperforms' is based on mean RMSE, but no statistical significance tests are reported. In several datasets the standard deviations overlap (e.g., Yeast NICA 0.111±0.004 vs MICE 0.118±0.008; Breast NICA 0.070±0.002 vs MICE 0.072±0.005), and on Yacht NICA is not the lowest. The paper should include paired significance tests (e.g., paired t-test or Wilcoxon over the 10 runs) to support the superiority claim.
minor comments (5)
  1. [Throughout] Typographical errors ('architechture', 'insprired', 'apllied', 'infomation') and inconsistent notation ('L_recovering' vs 'L_recover') should be corrected.
  2. [§4.1] The notation for the expectation subscripts is not defined rigorously; e.g., E_{M_c} and E_{1-M} should be clarified as averaging over the corresponding cell subsets.
  3. [§5.1] The paper states 'missing data is standardized and 20% of the observed part is randomly corrupted for v=8 times.' Clarify whether the reported results are averages over the v versions or a single imputation; the relationship between training-time corruption and test-time missingness should be explicit.
  4. [§5.2.1, Figure 2] The clustering behavior is illustrated with a single t-SNE plot; a quantitative cluster metric would strengthen the claim.
  5. [Algorithm 1] The line 'X(0)_c ← shuffle(X_c)' shuffles rows, but since rows are independent in the attention computation, this appears to have no effect; consider removing or explaining its purpose.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: NICA's reported imputation accuracy is evaluated on held-out missing cells against external baselines, and the model's objective does not encode the test values it predicts.

full rationale

The paper's central claim is empirical: NICA is trained by minimizing L_model = alpha1*L_recovering + alpha2*L_observed + L_missing over iterative self-attention/FNN updates, with all hyperparameters (alpha1=alpha2=10, K=10, v=8, dropout rate, learning rate) fixed a priori in Section 5.1. The reported RMSE is computed on 40% of cells randomly removed and never observed during training, which is a standard transductive benchmark; nothing in the objective or architecture encodes the original values of those held-out cells. The baselines are external (Mean, KNN, MICE, GAIN), and the author self-citations [8,23,37] are literature-context citations about the impact of imputation on downstream tasks, not load-bearing premises or imported uniqueness/ansatz results. The only suspicious component is the negative missing-loss term -E_{1-M}[L(X_c^K, Xbar_0)], which may be optimization-wise ill-posed because it rewards large imputed magnitudes; however, that is an internal correctness/consistency concern, not a circularity concern, because the evaluation target (original missing values) is not used to define the loss and is not produced by construction from the training data. No equation reduces to its own input, no fitted parameter is renamed as a prediction, and no self-citation chain forces the conclusion. The choice of baselines may be narrow, but that is an external-validity issue rather than circularity. Score 0.

Assumptions & free parameters 10 free parameters · 4 assumptions · 0 invented entities

All hyperparameters are hand-set and listed in Section 5.1; no new physical or conceptual entities are introduced. The method relies on standard MCAR evaluation assumptions and a finite-step iterative attention update that is assumed to stabilize without a convergence proof.

free parameters (10)
  • alpha_1 (recovering loss weight) = 10
    Hand-set in Section 5.1 to emphasize reconstruction of corrupted cells; not tuned per dataset.
  • alpha_2 (observed loss weight) = 10
    Hand-set in Section 5.1 to keep observed cells close to their values.
  • K (number of growing steps) = 10
    Hand-set growing iterations in Section 5.1.
  • hidden dimension multiplier = 5
    First linear layer maps state to 5x hidden space, Section 5.1.
  • FNN dropout rate = 0.5
    Dropout inside the feedforward network, Section 5.1.
  • row skip ratio (asynchronous growth) = 0.1
    10% of samples skipped per step via Dropout1d, Section 5.1.
  • corruption ratio of observed cells = 0.2
    20% of observed part randomly corrupted to create training seeds, Section 5.1.
  • v (number of corrupting versions) = 8
    Creates 8 independent corruptions per batch, Section 5.1.
  • training iterations = 1000
    Adam optimization for 1000 iterations, Section 5.1.
  • batch size = 1024
    Batch size used in training, Section 5.1.
assumptions (4)
  • domain assumption MCAR random removal is a valid proxy for missingness
    Section 5.1.1 removes 40% of cells randomly; only MCAR is tested, while Section 3.2 discusses MAR and MNAR.
  • domain assumption Feature standardization does not distort imputation quality
    Section 5.1.1 states 'missing data is standardized' before experiments.
  • domain assumption Simple label encoding of categorical variables preserves the information needed for imputation
    Section 5.1.1: 'simply preprocess them by encoding the categorical variables to numerical type'.
  • domain assumption Iterated self-attention with K finite steps converges to a stable output
    Section 4.1 repeats the update K=10 times; no convergence guarantee is given, only empirical observation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Missing Data Imputation using Neural Cellular Automata." pith.science (2026). https://pith.science/paper/IOJ6VB5U

@misc{pith2026250900651,
  author       = {Pith},
  title        = {Pith review of: Missing Data Imputation using Neural Cellular Automata},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IOJ6VB5U}},
  note         = {Machine review of arXiv:2509.00651}
}
read the original abstract

When working with tabular data, missingness is always one of the most painful problems. Throughout many years, researchers have continuously explored better and better ways to impute missing data. Recently, with the rapid development evolution in machine learning and deep learning, there is a new trend of leveraging generative models to solve the imputation task. While the imputing version of famous models such as Variational Autoencoders or Generative Adversarial Networks were investigated, prior work has overlooked Neural Cellular Automata (NCA), a powerful computational model. In this paper, we propose a novel imputation method that is inspired by NCA. We show that, with some appropriate adaptations, an NCA-based model is able to address the missing data imputation problem. We also provide several experiments to evidence that our model outperforms state-of-the-art methods in terms of imputation error and post-imputation performance.

Figures

Figures reproduced from arXiv: 2509.00651 by the authors.

Figure 1
Figure 1. Our NICA model positions. Next, the data is normalized and pre-assigned with zeros to create a simple baseline version denoted by X¯ 0. After that, the initial seed is constructed by randomly corrupting a certain proportion of the known cells of X¯ 0. Now, we have a corrupted data Xc and its corresponding corrupting mask Mc. The training phase begins by passing Xc through self-attention layer where attention weights… view at source ↗
Figure 2
Figure 2. Data points form clusters through the growing process. [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗
Figure 3
Figure 3. Average RMSE for various missing rates 20 [PITH_FULL_IMAGE:figures/full_fig_p020_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Average predicting performance for various missing rates [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

41 extracted references · 33 canonical work pages

  1. [1]

    Neural machine translation by jointly learning to align and translate

    Bahdanau, D., 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

    arXiv 2014

  2. [2]

    Importance weighted autoencoders

    Burda, Y ., Grosse, R., Salakhutdinov, R., 2015. Importance weighted autoencoders. arXiv preprint arXiv:1509.00519

    arXiv 2015

  3. [3]

    Illuminating diverse neural cellular automata for level generation, in: Proceedings of the Genetic and Evolution- ary Computation Conference, pp

    Earle, S., Snider, J., Fontaine, M.C., Nikolaidis, S., Togelius, J., 2022. Illuminating diverse neural cellular automata for level generation, in: Proceedings of the Genetic and Evolution- ary Computation Conference, pp. 68–76

    work page 2022

  4. [4]

    Mathematical games

    Gardner, M., 1970. Mathematical games. Scientific american 222, 132–140

    work page 1970

  5. [5]

    Cellular automata as convolutional neural networks

    Gilpin, W., 2019. Cellular automata as convolutional neural networks. Physical Review E 100, 032402. 16

    work page 2019

  6. [6]

    Gondara, L., Wang, K., 2018. Mida: Multiple imputation using denoising autoencoders, in: Advances in Knowledge Discovery and Data Mining: 22nd Pacific-Asia Conference, PAKDD 2018, Melbourne, VIC, Australia, June 3-6, 2018, Proceedings, Part III 22, Springer. pp. 260–272

    work page 2018

  7. [7]

    Horibe, K., Walker, K., Risi, S., 2021. Regenerating soft robots through neural cellular automata, in: Genetic Programming: 24th European Conference, EuroGP 2021, Held as Part of EvoStar 2021, Virtual Event, April 7–9, 2021, Proceedings 24, Springer. pp. 36–50

    work page 2021

  8. [8]

    The impact of data imputation on air quality prediction problem

    Hua, V ., Nguyen, T., Dao, M.S., Nguyen, H.D., Nguyen, B.T., 2024. The impact of data imputation on air quality prediction problem. Plos one 19, e0306303

    work page 2024

Show all 41 references
  1. [9]

    The uci machine learning repository

    Kelly, M., Longjohn, R., Nottingham, K., . The uci machine learning repository. URL: https://archive.ics.uci.edu

  2. [10]

    Auto-encoding variational bayes

    Kingma, D.P., 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114

    2013 arXiv

  3. [11]

    Adam: A method for stochastic optimization

    Kingma, D.P., 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

    2014 arXiv

  4. [12]

    Statistical analysis with missing data (vol

    Little, R.J., Rubin, D.B., 2019. Statistical analysis with missing data (vol. 793)

    2019

  5. [13]

    Visualizing data using t-sne

    Van der Maaten, L., Hinton, G., 2008. Visualizing data using t-sne. Journal of machine learning research 9

    2008

  6. [14]

    Miwae: Deep generative modelling and imputation of incomplete data sets, in: International conference on machine learning, PMLR

    Mattei, P.A., Frellsen, J., 2019. Miwae: Deep generative modelling and imputation of incomplete data sets, in: International conference on machine learning, PMLR. pp. 4413– 4423

    2019

  7. [15]

    Growing isotropic neural cellular au- tomata, in: Artificial Life Conference Proceedings 34, One Rogers Street, Cambridge, MA 02142-1209, USA journals-info@ mit

    Mordvintsev, A., Randazzo, E., Fouts, C., 2022. Growing isotropic neural cellular au- tomata, in: Artificial Life Conference Proceedings 34, One Rogers Street, Cambridge, MA 02142-1209, USA journals-info@ mit. edu: MIT Press. p. 65

    2022

  8. [16]

    Growing neural cellular automata

    Mordvintsev, A., Randazzo, E., Niklasson, E., Levin, M., 2020. Growing neural cellular automata. Distill 5, e23

    2020

  9. [17]

    Handling incomplete hetero- geneous data using vaes

    Nazabal, A., Olmos, P.M., Ghahramani, Z., Valera, I., 2020. Handling incomplete hetero- geneous data using vaes. Pattern Recognition 107, 107501

    2020

  10. [18]

    Self-organising textures

    Niklasson, E., Mordvintsev, A., Randazzo, E., Levin, M., 2021. Self-organising textures. Distill 6, e00027–003

    2021

  11. [19]

    Generative adversarial neural cellular automata

    Otte, M., Delfosse, Q., Czech, J., Kersting, K., 2021. Generative adversarial neural cellular automata. arXiv preprint arXiv:2108.04328

    2021 arXiv

  12. [20]

    Mesh neural cellular automata

    Pajouheshgar, E., Xu, Y ., Mordvintsev, A., Niklasson, E., Zhang, T., Süsstrunk, S., 2024. Mesh neural cellular automata. ACM Transactions on Graphics (TOG) 43, 1–16

    2024

  13. [21]

    Variational neural cellular automata

    Palm, R.B., González-Duque, M., Sudhakaran, S., Risi, S., 2022. Variational neural cellular automata. arXiv preprint arXiv:2201.12360 . 17

    2022 arXiv

  14. [22]

    Scikit-learn: Machine learning in Python

    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V ., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V ., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E., 2011. Scikit-learn: Machine learning in Python. Journal...

    2011

  15. [23]

    Pham, N.H., V o, K.L., Vu, M.A., Nguyen, T., Riegler, M.A., Halvorsen, P., Nguyen, B.T.,

  16. [24]

    A multivariate technique for multiply imputing missing values using a sequence of regression models

    Raghunathan, T.E., Lepkowski, J.M., Van Hoewyk, J., Solenberger, P., et al., 2001. A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey methodology 27, 85–96

    2001

  17. [25]

    Growing steerable neural cellular au- tomata, in: Artificial Life Conference Proceedings 35, One Rogers Street, Cambridge, MA 02142-1209, USA journals-info@ mit

    Randazzo, E., Mordvintsev, A., Fouts, C., 2023. Growing steerable neural cellular au- tomata, in: Artificial Life Conference Proceedings 35, One Rogers Street, Cambridge, MA 02142-1209, USA journals-info@ mit. edu: MIT Press. p. 2

    2023

  18. [26]

    Adversarial reprogram- ming of neural cellular automata

    Randazzo, E., Mordvintsev, A., Niklasson, E., Levin, M., 2021. Adversarial reprogram- ming of neural cellular automata. Distill 6, e00027–004

    2021

  19. [27]

    Self- classifying mnist digits

    Randazzo, E., Mordvintsev, A., Niklasson, E., Levin, M., Greydanus, S., 2020. Self- classifying mnist digits. Distill 5, e00027–002

    2020

  20. [28]

    Inference and missing data

    Rubin, D.B., 1976. Inference and missing data. Biometrika 63, 581–592

    1976

  21. [29]

    Image segmentation via cellular automata

    Sandler, M., Zhmoginov, A., Luo, L., Mordvintsev, A., Randazzo, E., et al., 2020. Image segmentation via cellular automata. arXiv preprint arXiv:2008.04965

    2020 arXiv

  22. [30]

    Song, W., Shi, C., Xiao, Z., Duan, Z., Xu, Y ., Zhang, M., Tang, J., 2019. Autoint: Au- tomatic feature interaction learning via self-attentive neural networks, in: Proceedings of the 28th ACM international conference on information and knowledge management, pp. 1161–1170

    2019

  23. [31]

    Missforest—non-parametric missing value imputa- tion for mixed-type data

    Stekhoven, D.J., Bühlmann, P., 2012. Missforest—non-parametric missing value imputa- tion for mixed-type data. Bioinformatics 28, 112–118

    2012

  24. [32]

    Sudhakaran, S., Grbic, D., Li, S., Katona, A., Najarro, E., Glanois, C., Risi, S., 2021. Growing 3d artefacts and functional machines with neural cellular automata, in: Artificial Life Conference Proceedings 33, One Rogers Street, Cambridge, MA 02142-1209, USA journals-info@ m...

    2021

  25. [33]

    Attention-based neural cellular automata

    Tesfaldet, M., Nowrouzezahrai, D., Pal, C., 2022. Attention-based neural cellular automata. Advances in Neural Information Processing Systems 35, 8174–8186

    2022

  26. [34]

    Missing value estimation methods for dna microarrays

    Troyanskaya, O., Cantor, M., Sherlock, G., Brown, P., Hastie, T., Tibshirani, R., Botstein, D., Altman, R.B., 2001. Missing value estimation methods for dna microarrays. Bioinfor- matics 17, 520–525

    2001

  27. [35]

    mice: Multivariate imputation by chained equations in r

    Van Buuren, S., Groothuis-Oudshoorn, K., 2011. mice: Multivariate imputation by chained equations in r. Journal of statistical software 45, 1–67. 18

    2011

  28. [36]

    Attention is all you need

    Vaswani, A., 2017. Attention is all you need. Advances in Neural Information Processing Systems

    2017

  29. [37]

    The effects of data imputation on covariance and inverse covariance matrix estimation

    V o, T.L., Do, Q.H., Nguyen, T., Halvorsen, P., Riegler, M.A., Nguyen, B.T., 2024. The effects of data imputation on covariance and inverse covariance matrix estimation. IEEE Access

    2024

  30. [38]

    The general and logical theory of automata, cerebral mechanisms in behavior

    V on Neumann, J., 1951. The general and logical theory of automata, cerebral mechanisms in behavior. the hixon symposium. New York: John Wiley&Sons

    1951

  31. [39]

    A new kind of science

    Wolfram, S., Gad-el Hak, M., 2003. A new kind of science. Appl. Mech. Rev. 56, B18–B19

    2003

  32. [40]

    Gain: Missing data imputation using generative adversarial nets, in: International conference on machine learning, PMLR

    Yoon, J., Jordon, J., Schaar, M., 2018. Gain: Missing data imputation using generative adversarial nets, in: International conference on machine learning, PMLR. pp. 5689–5698. 19 Figure 3: Average RMSE for various missing rates 20 Figure 4: Average predicting performance for v...

    2018

  33. [2024]

    Correlation visualization under missing values: a comparison between imputation and direct parameter estimation methods, in: International Conference on Multimedia Mod- eling, Springer. pp. 103–116

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.