REVIEW 4 major objections 5 minor 21 references
On the Reconstruction of Training Data from Group Invariant Networks
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Standard reconstruction methods applied to group-invariant networks converge to symmetric orbit averages, and two symmetry-aware modifications improve reconstruction quality.
desk verdict Worth engaging: first study of reconstruction attacks on invariant nets, with a nice stabilizer-nesting observation, but the printed KKT loss is missing a gradient and SAME-GD's gains don't show up in the CIFAR-10 rows. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing objects are the orbit of an input under a finite orthogonal group, the orbitope (the convex hull of that orbit), and the invariant subspace $V^G$ of points fixed by every group element. Proposition 2 is the main mechanism: for a $G$-invariant objective, one gradient descent step is $G$-equivariant, and $\mathrm{Stab}_G(x_{t-1}) \subseteq \mathrm{Stab}_G(x_t)$, so iterates cannot decrease their symmetry and are drawn to the orbit average, the point with the largest stabilizer. SAME-GD deliberately violates this nesting by aggregating the current point with a previous point from which the orbit average has been removed, while deep image prior replaces direct optimization over pixels by optimization over a randomly initialized CNN whose natural-image prior breaks the symmetry bias.
What would settle it
Train a two-layer symmetrized ReLU network on MNIST with a two-element reflection group, run KKT reconstruction with many random seeds, and compute the stabilizer of each final reconstruction together with its distance to the orbit average; if a large fraction of reconstructions are not fixed by the reflection and lie far from the average, the paper's claim that reconstructions gravitate to $V^G$ is contradicted.
Extended reading notes
Core claim
The central claim is that group invariance, not network capacity or dataset size, is what makes standard reconstruction methods collapse. For any finite orthogonal group $G$, if the reconstruction loss is $G$-invariant, then each gradient-descent step is $G$-equivariant and the stabilizer of the current iterate is nested inside the stabilizer of the next, so optimization can only move toward points with larger symmetry; the fixed points of this dynamics lie in the invariant subspace $V^G$, and the orbit average is the natural attractor. The paper proves this in two propositions and shows empirically that reconstructions from AM and KKT objectives concentrate near group averages on orbitopes. It reports that increasing the group size or training set size worsens the effect, and that KKT-based reconstruction beats activation maximization but still fails. Its proposed fixes, SAME-GD and deep image prior, both improve standard error metrics while producing asymmetric, piecewise-smooth reconstructions, especially when combined with the KKT objective.
Load-bearing premise
The KKT-based reconstruction results assume the trained invariant network is homogeneous and sits exactly at a margin-maximizing stationary point, so its weights are a weighted sum of gradients at the true training inputs, $\theta = \sum_i \lambda_i y_i \nabla_\theta \phi(x_i; \theta)$; the paper asserts this holds for the symmetrized ReLU networks but does not verify it.
Editorial extensions
If this is right
- Invariant models are harder to invert with off-the-shelf reconstruction: both AM and KKT return images close to the orbit average, with DSSIM around 0.46–0.49 on MNIST for groups of size 2–8.
- Reconstruction quality degrades as the group or training set grows, so larger symmetry groups provide stronger accidental protection against naive attacks.
- The KKT objective is a better starting point than activation maximization, and combining it with deep image prior gives the best reported reconstructions.
- Evaluation of reconstruction attacks on invariant models should use a group-invariant metric such as min-over-orbit DSSIM to fairly compare across group transformations.
Reading between the lines
- The stabilizer-nesting argument applies beyond reconstruction: any gradient-based inversion or latent-space search over a symmetry-invariant objective will inherit the same drift toward fixed points, so the result predicts similar collapse in equivariant generative models.
- Because the orbit average is a natural attractor, an invariant network leaks the orbit average but hides in-orbit details; a practical privacy implication is that group invariance acts as an accidental defense against naive attacks, though an adversary who knows the group can still recover the orbit.
- SAME-GD removes the group-average component from a stored previous iterate, which suggests a principled design: any update that decreases the stabilizer of the current point should improve reconstruction, and quantifying how much symmetry breaking is needed per group size would be a direct follow-up.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the reconstruction of training data from group-invariant neural networks. It formalizes the reconstruction problem for invariant models, proves basic invariance properties of reconstruction objectives, and shows that gradient descent iterates on such objectives are equivariant and have nested stabilizers. The central empirical claim is that conventional activation maximization and KKT-based reconstruction methods converge to highly symmetric inputs, typically near the group average, yielding poor reconstructions. The paper then proposes two heuristics, SAME-GD and a deep-image-prior parameterization, and reports preliminary MNIST and CIFAR-10 experiments suggesting that these can improve reconstruction quality.
Significance. If the findings hold, this is a useful first investigation of a realistic and privacy-relevant setting: reconstructing data from networks that are invariant under finite groups. The formal problem definition and the stabilizer-nesting result for gradient descent (Proposition 2) are conceptually clean and provide a plausible explanation for the observed convergence to orbit averages. The proposed methods are heuristic but could open a useful direction. The paper does not appear to be circular: the main qualitative prediction follows from the theory rather than from fitting the model to the data. However, the manuscript as printed contains a dimensionally invalid KKT objective, an unverified KKT-stationarity assumption for the trained symmetrized networks, and a table that does not support one of the main claims about SAME-GD. These issues are load-bearing and need to be fixed before the empirical claims can be verified.
major comments (4)
- [Section 2 / Appendix B, Eq. (3)] The printed KKT reconstruction objective is dimensionally invalid: L_rec = ||θ − Σ_i λ_i y_i φ(x_i; θ)|| subtracts scalar network outputs from the parameter vector θ. The KKT stationary condition of the margin-maximization problem in Eq. (2) is θ = Σ_i λ_i y_i ∇_θ φ(x_i; θ), so the printed formula omits the parameter gradient ∇_θ φ. Lemma 3 in Appendix C proves G-invariance of ∇_θ φ, which is exactly the quantity needed for the loss to be invariant, confirming that the intended objective contains the gradient. Because Table 1 and the SAME-GD/DIP variants are evaluated on top of this objective, the experiments as described are not reproducible from the printed equations. The authors should correct Eq. (3) and explicitly state whether the experiments used the corrected objective with ∇_θ φ.
- [Section 3.1] The paper asserts that 'in most cases the training of (homogeneous) invariant neural networks is conducted in a way that the conditions of both methods (AM and KKT-based) are met,' but no proof or verification is provided for the symmetrized ReLU networks used in Appendix E. The KKT-based reconstruction method requires convergence of gradient flow to a KKT point of the margin-maximization problem, which is a nontrivial assumption. The authors should either verify the stationary condition numerically on the trained models (for example, by computing the relative KKT residual) or weaken the claim to state that the method is applied heuristically. Without this, the KKT reconstructions are not anchored to the stated margin-maximization theory.
- [Table 1 / Section 4] The text states that SAME-GD yields 'notably improved reconstructions,' but Table 1 does not support this uniformly. For CIFAR-10 the KKT+SAME-GD column is worse than vanilla KKT on both training-set sizes (0.369 ± 0.000 vs 0.346 ± 0.001 for n=50; 0.370 ± 0.007 vs 0.371 ± 0.001 for n=100). For MNIST with group size 8 the improvements are very small (0.465 vs 0.471 and 0.469 vs 0.471). The paper should report the comparisons with error bars, state which differences are significant, and adjust the claim to the actual results, or explain the configurations in which SAME-GD does and does not help.
- [Section 3.3 / Figure 5] The central empirical claim is that conventional methods 'gravitate toward symmetric inputs' on which the group acts trivially, but the paper never quantifies this directly. Table 1 reports only DSSIM structural dissimilarity, and Figure 5 is a qualitative histogram of reconstruction locations on discretized orbitopes. The authors should add a quantitative symmetry measure, for example ||x − ḡx||/||x|| or the size of the stabilizer of the reconstruction, and report its distribution across methods. This would make the main phenomenon directly testable rather than inferred from image quality and a qualitative figure.
minor comments (5)
- [Section 3.2, Proposition 2] Part (ii) states Stab_G(x_{t−1}) ⊆ Stab_G(x_t), meaning that stabilizers can grow as optimization progresses, so points become more symmetric; the accompanying text saying that stabilizers 'may become more restrictive' is backwards and should be reworded.
- [Appendix C, Eq. (4)] The proof of Lemma 3 contains a typo: the right-hand side should be (f(g·x; θ+εe_i) − f(g·x; θ))/ε, not (f(g·x; θ+εe_i) − f(x; θ))/ε.
- [Appendix D, proof of Lemma 5] The inner-product derivation contains a stray dot in the expression '<g^{-1}∇L(g·x), ·g^{-1}·h>' and should be cleaned up.
- [Algorithm 1 / Appendix F] In the x_prev update, 'xprev ← β_t x_t + (1−β_t)∇L(x_t)' mixes a point and a gradient; the paper should specify the intended gradient with respect to x and clarify that ∇L(x_t) is used as a direction, not as a coordinate.
- [Figure 3] Figure 3 is referenced in Section 3.3 but appears only after Appendix D in the submitted layout; the figure should be placed with its first citation.
Circularity Check
No circular derivation chain found: the central invariance results are proven from definitions, the KKT baseline is adopted from independent prior work, and the proposed methods are experimental rather than fitted predictions.
full rationale
I found no step in this paper where a 'prediction' or derived result is equivalent to its inputs by construction. Proposition 2 (stabilizer nesting under GD) is proven from the definition of G-invariance and orthogonality of the representation, not from fitted parameters or from the later experiments. The empirical observation that reconstructions concentrate near the orbit average is presented as an observed phenomenon and only loosely connected to Proposition 2; even if that connection is overstated, it is not a circular reduction. The KKT objective is taken from Haim et al. (2022) and Lyu & Li (2020); the former shares an author with this paper, but it is a parameter-free, externally stated stationarity condition and is not used to forbid alternative explanations, so the self-citation is not load-bearing. SAME-GD and DIP are proposed algorithmic modifications whose improved DSSIM values are experimental outcomes, not quantities forced by the loss definition. Two non-circular but important flaws should be flagged: (1) Eq. (3) and Appendix B print the KKT loss as ||θ − Σ λ_i y_i φ(x_i; θ)|| without the parameter gradient ∇_θ φ, making the expression dimensionally invalid; Lemma 3 correctly concerns ∇_θ φ, so the printed formula appears to be a typo that nevertheless prevents verification of the reported KKT experiments. (2) Section 3.1 asserts without proof that trained symmetrized ReLU networks satisfy the KKT conditions of both methods. These are correctness and reproducibility concerns, not circularity: none of the paper's claims reduce to its own definitions or to a self-citation chain.
Assumptions & free parameters
free parameters (3)
- SAME-GD hyperparameters (α_t, β_t, T_save, T_update) =
not specified
- number of reconstruction candidates m =
1000 (500 per class)
- DIP network architecture and iterations =
not specified
assumptions (4)
- domain assumption The trained symmetrized ReLU network is homogeneous with respect to θ and its gradient flow converges to a KKT point of the margin maximization problem (Eq. 2).
- standard math The group representation is orthogonal and the reconstruction loss is differentiable with respect to x.
- domain assumption The reconstruction objectives are G-invariant functions of the inputs.
- domain assumption Nearest-neighbor binning over discretized orbitopes is a valid characterization of where reconstructions land.
Cite this review
Pith. "Pith review of On the Reconstruction of Training Data from Group Invariant Networks." pith.science (2026). https://pith.science/paper/6TDKNNCU
@misc{pith2026241116458,
author = {Pith},
title = {Pith review of: On the Reconstruction of Training Data from Group Invariant Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/6TDKNNCU}},
note = {Machine review of arXiv:2411.16458}
}
read the original abstract
Reconstructing training data from trained neural networks is an active area of research with significant implications for privacy and explainability. Recent advances have demonstrated the feasibility of this process for several data types. However, reconstructing data from group-invariant neural networks poses distinct challenges that remain largely unexplored. This paper addresses this gap by first formulating the problem and discussing some of its basic properties. We then provide an experimental evaluation demonstrating that conventional reconstruction techniques are inadequate in this scenario. Specifically, we observe that the resulting data reconstructions gravitate toward symmetric inputs on which the group acts trivially, leading to poor-quality results. Finally, we propose two novel methods aiming to improve reconstruction in this setup and present promising preliminary experimental results. Our work sheds light on the complexities of reconstructing data from group invariant neural networks and offers potential avenues for future research in this domain.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Baker, Alexander Pinard, and Dorit M
Allison H. Baker, Alexander Pinard, and Dorit M. Hammerling. Dssim: a structural similarity index for floating-point data, 2023. URL https://arxiv.org/abs/2202.02616
arXiv 2023
-
[2]
Deconstructing data reconstruction: Multiclass, weight decay and general losses
Gon Buzaglo, Niv Haim, Gilad Yehudai, Gal Vardi, Yakir Oz, Yaniv Nikankin, and Michal Irani. Deconstructing data reconstruction: Multiclass, weight decay and general losses. In Advances in Neural Information Processing Systems, volume 36, pages 51515--51535, 2023
work page 2023
-
[3]
Group equivariant convolutional networks
Taco Cohen and Max Welling. Group equivariant convolutional networks. In International conference on machine learning, pages 2990--2999. PMLR, 2016
2016
-
[4]
Model inversion attacks that exploit confidence information and basic countermeasures
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pages 1322--1333, 2015
2015
-
[5]
Jonas Geiping, Hartmut Bauermeister, Hannah Dr \"o ge, and Michael Moeller. Inverting gradients-how easy is it to break privacy in federated learning? Advances in Neural Information Processing Systems, 33: 0 16937--16947, 2020
work page 2020
-
[6]
Neural message passing for quantum chemistry
Justin Gilmer, Samuel S Schoenholz, Patrick F Riley, Oriol Vinyals, and George E Dahl. Neural message passing for quantum chemistry. In International conference on machine learning, pages 1263--1272. PMLR, 2017
2017
-
[7]
Reconstructing training data from trained neural networks
Niv Haim, Gal Vardi, Gilad Yehudai, Ohad Shamir, and Michal Irani. Reconstructing training data from trained neural networks. Advances in Neural Information Processing Systems, 35: 0 22911--22924, 2022
2022
-
[8]
Deep models under the gan: information leakage from collaborative deep learning
Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. Deep models under the gan: information leakage from collaborative deep learning. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 603--618, 2017
work page 2017
Show all 21 references
-
[9]
Evaluating gradient inversion attacks and defenses in federated learning
Yangsibo Huang, Samyak Gupta, Zhao Song, Kai Li, and Sanjeev Arora. Evaluating gradient inversion attacks and defenses in federated learning. Advances in Neural Information Processing Systems, 34: 0 7232--7241, 2021
2021
-
[10]
Directional convergence and alignment in deep learning
Ziwei Ji and Matus Telgarsky. Directional convergence and alignment in deep learning. In Advances in Neural Information Processing Systems (NeurIPS), 2020
2020
-
[11]
Dataset distillation fixes dataset reconstruction attacks
Noel Loo, Ramin Hasani, Mathias Lechner, and Daniela Rus. Dataset distillation fixes dataset reconstruction attacks. arXiv preprint arXiv:2302.01428, 2023
2023 arXiv
-
[12]
Gradient descent maximizes the margin of homogeneous neural networks
Kaifeng Lyu and Jian Li. Gradient descent maximizes the margin of homogeneous neural networks. In International Conference on Learning Representations (ICLR), 2020
2020
-
[13]
Reconstructing training data from real world models trained with transfer learning
Yakir Oz, Gilad Yehudai, Gal Vardi, Itai Antebi, Michal Irani, and Niv Haim. Reconstructing training data from real world models trained with transfer learning. arXiv preprint arXiv:2407.15845, 2024
2024 arXiv
-
[14]
Pointnet: Deep learning on point sets for 3d classification and segmentation
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652--660, 2017
2017
-
[15]
Orbitopes
Raman Sanyal, Frank Sottile, and Bernd Sturmfels. Orbitopes. Mathematika, 57 0 (2): 0 275–314, June 2011. ISSN 2041-7942. doi:10.1112/s002557931100132x. URL http://dx.doi.org/10.1112/S002557931100132X
2011 doi
-
[16]
Deep image prior
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Deep image prior. International Journal of Computer Vision, 128 0 (7): 0 1867–1888, March 2020. ISSN 1573-1405. doi:10.1007/s11263-020-01303-4. URL http://dx.doi.org/10.1007/s11263-020-01303-4
2020 doi
-
[17]
Adapting membership inference attacks to gnn for graph classification: Approaches and implications, 2021
Bang Wu, Xiangwen Yang, Shirui Pan, and Xingliang Yuan. Adapting membership inference attacks to gnn for graph classification: Approaches and implications, 2021
2021
-
[18]
Neural network inversion in adversarial setting via background knowledge alignment
Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang. Neural network inversion in adversarial setting via background knowledge alignment. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 225--240, 2019
2019
-
[19]
Deep sets
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhutdinov, and Alexander J Smola. Deep sets. Advances in neural information processing systems, 30, 2017
2017
-
[20]
Graphmi: Extracting private graph data from graph neural networks, 2021
Zaixi Zhang, Qi Liu, Zhenya Huang, Hao Wang, Chengqiang Lu, Chuanren Liu, and Enhong Chen. Graphmi: Extracting private graph data from graph neural networks, 2021
2021
-
[21]
Deep leakage from gradients
Ligeng Zhu, Zhijian Liu, and Song Han. Deep leakage from gradients. Advances in Neural Information Processing Systems, 32, 2019
2019
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.