{"id":"39dade7c-a598-4986-8326-dd29a5744f78","arxiv_id":"2501.01601","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"EquiGen generates diverse and functionally similar implicit neural network weights from only a few examples by exploiting permutation equivariance in the weight space.","lead":"This paper introduces a new task, few-shot generation of implicit neural representation (INR) weights, and proposes EquiGen, which uses weight-permutation equivariance and a diffusion model to generate new INR weights from a handful of examples. The authors report that EquiGen outperforms existing few-shot image and shape generation baselines on MNIST, CIFAR-10, and ShapeNet.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Smooth augmentation in Sec. 4.2 is asserted to preserve functional equivariance, but the paper never specifies how biases and adjacent-layer weights are transformed under P*, so the contrastive positive pairs in Eq.","rationale":"I agree with the reader's CONDITIONAL verdict and with the identification of smooth augmentation as the weakest assumption. The permutation-equivariance identity for pointwise activations is correct, so the concern is not about whether permutations can preserve function in principle; it is about whether the paper's underspecified P* application, especially to biases, actually does so. This is load-bearing because the contrastive objective in Eq. 4 is the only mechanism that installs functional equivalence in the latent space, and all later stages (diffusion conditioning, subspace disturbance) consume the resulting features. A direct forward-pass comparison of original and smoothed INRs would settle the issue in an afternoon and does not require re-running the full pipeline. The paper has no released code, no appendix, and no error bars, which makes this unverified step more consequential rather than less. The reported gains are consistent across datasets and the framework is well motivated, so the concern warrants a concrete verification condition rather than rejection; hence I keep the reader's CONDITIONAL recommendation.","tokens_in":14655,"tokens_out":8087,"duration_ms":81126,"concrete_test":"Take one trained MNIST SIREN and one ShapeNet ReLU INR, compute P* exactly as described in Sec. 4.2 (dependency-graph subgraphs, 2.5-opt TV minimization), and apply the transformation in the two readings of the text: (a) W_l -> P* W_l for all matrices with biases unchanged; (b) the exact symmetry W_l -> P_l W_l P_{l-1}^T, b_l -> P_l b_l. Render original and transformed networks on a dense input grid and report max and mean absolute output error; for the 3D INR, also compare extracted meshes with Chamfer distance. If the implemented transformation (or reading (a)) gives error above machine precision or ~1e-5, the positive pairs in Eq. 4 are not functionally equivalent and the equivariant pretraining is invalid; if reading (b) is what the code does and errors are at machine precision, this concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"EquiGen's central mechanism is an equivariant encoder trained so that functionally equivalent weights map to nearby codes; the only source of positive pairs is 'smooth augmentation' (Sec. 4.2). For pointwise activations such as ReLU and SIREN's sine, the exact functional symmetry is W_l -> P_l W_l P_{l-1}^T and b_l -> P_l b_l, with P_0 = P_L = I. The paper states only that a TV-minimizing permutation P* found by 2.5-opt is 'applied to all weight matrices within each network subdivision' and asserts that this preserves functional equivariance; it does not state whether biases are permuted or whether the inverse permutation is applied to the next layer. If biases are left unchanged, or if the same P is applied unilaterally to every matrix, the transformed MLP is not the same function. Even one inconsistent bias changes the rendered output for MNIST SIRENs and ShapeNet ReLU MLPs. Positive pairs that are not exact functional equivalences train the contrastive loss to cluster weights with different rendered signals, so the equivariant features that condition the diffusion model and the subspace disturbance in Sec. 4.4 are not grounded in functional equivalence. Since the reported metrics (FID, LPIPS, MMD, COV, 1-NNA) are distributional, this failure would not show up in Tables 1-4. Thus the paper's 'functionally consistent' claim rests on an underspecified and unverified permutation step.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"EquiGen proposes a three-stage framework for few-shot generation of implicit neural representation (INR) weights: (1) an equivariant encoder trained with contrastive learning on smooth weight-space augmentations, (2) a diffusion model conditioned on equivariant features with an additional equivariance loss, and (3) controlled perturbations of the equivariant subspace to increase diversity. The authors introduce the Few-shot Implicit Function Generation problem and report experiments on MNIST, CIFAR-10, and ShapeNet INR datasets, claiming consistent improvements in FID, LPIPS, MMD, COV, and 1-NNA over several baselines.","tokens_in":14965,"tokens_out":6728,"duration_ms":60376,"significance":"The core idea of exploiting weight-space permutation equivariance to reduce sample complexity is well-motivated and timely, and the paper is one of the first to address few-shot INR weight generation. If the reported gains are robust, the framework could be useful for weight-space data augmentation, hypernetwork training, and downstream generation tasks. The paper's strengths include a clear problem formulation, an explicit attempt to use symmetry principles throughout the pipeline, and evaluation on multiple datasets. However, the current evidence is incomplete: results are reported without uncertainty, the functional-consistency claim is not directly measured, and key implementation details are deferred to an absent appendix.","major_comments":[{"comment":"The paper asserts that the total-variation-minimizing permutation P* found by 2.5-opt local search 'preserv[es] functional equivariance' when applied to all weight matrices within each network subdivision, but it never specifies the transformation rule for biases or the relation between permutations of adjacent layers. For a standard MLP with pointwise activations (ReLU or SIREN's sine), the exact functional symmetry is W_l -> P_l W_l P_{l-1}^T and b_l -> P_l b_l with P_0 = P_L = I. If biases are left unchanged or if the same P is applied unilaterally to every matrix, the transformed network is not the same function. Because the contrastive positive pairs in Eq. (4) and the equivariance loss in Eq. (6) both rely on the augmented weights being functionally equivalent, an incorrect permutation rule would train the encoder to map functionally different weights into the same latent region, invalidating the conditioning signal for the diffusion model. The text must specify the full transformation and provide a proof or an experiment verifying functional equivalence after augmentation.","section":"Sec. 4.2 (Smooth Augmentation), Eq. (4), Eq. (6)"},{"comment":"The manuscript repeatedly defers essential technical content to an Appendix: the mathematical formalization of the problem, the detailed architecture of the equivariant encoder, the specific INR-based augmentations, and the implementation configurations. In the version under review, no appendix is provided. In particular, the details needed to resolve the smooth-augmentation ambiguity (how P* acts on biases and adjacent layers) are absent. This prevents reproduction and verification of the central mechanism. The paper cannot be accepted in this form; the appendix must be included or the method must be self-contained.","section":"Sec. 3.1, Sec. 4.2, Sec. 5.1 (repeated appendix references)"},{"comment":"All quantitative results are single point estimates without error bars, confidence intervals, or significance tests. The claim of 'consistent superior performance' across datasets is therefore not statistically supported. Given the few-shot setting (e.g., 10 support samples), run-to-run variance could be substantial. The authors should report means and standard deviations over multiple random seeds and, where possible, perform significance tests or at least show the variance is small relative to the reported gaps.","section":"Sec. 5.2, Tables 1–4"},{"comment":"The central claim is that EquiGen generates 'diverse yet functionally consistent INR weights,' but the evaluation metrics (FID, LPIPS, MMD, COV, 1-NNA) are distributional image/shape metrics and do not directly measure functional consistency of the generated weights. A generated set could score well on these metrics while many generated INRs fail to represent the target class or preserve the rendered function. The paper should include a direct functional test: for example, render each generated INR and compute reconstruction error against the source examples, or evaluate classification accuracy of rendered images, or measure the latent distance between generated and source functions using an independent functional-equivalence oracle.","section":"Sec. 5.1 (Metrics) and Sec. 1 (claims)"},{"comment":"The problem definition allows support set sizes k from 1 to 10, but the main results are reported only for 10-shot generation. No results are shown for k = 1, 2, 5, etc. This leaves the 'few-shot' claim unsubstantiated for smaller k, which is precisely the regime motivating the paper. The authors should provide a plot or table of performance versus k for at least one dataset.","section":"Sec. 5.1 (Few-shot setting) and Tables 1–2"}],"minor_comments":[{"comment":"The typesetting has 'F ew-shot' at the start of the abstract; this should be 'Few-shot'.","section":"Abstract"},{"comment":"The word 'emperically' is a typo and should be 'empirically'.","section":"Sec. 1"},{"comment":"The baselines 'HyperDiffsion' and 'Hiperdiffusion' are misspelled; they should be 'HyperDiffusion'.","section":"Table 2"},{"comment":"The flowchart is dense and the text labels are difficult to read; consider enlarging and annotating the three stages more clearly.","section":"Fig. 2"},{"comment":"In the equivariance definition, the group representations ρ1 and ρ2 are not defined; please define them in the text or refer explicitly to the relevant source.","section":"Sec. 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper relies heavily on the equivariant architecture of [45] and defers almost all implementation details to an appendix that is not included in the submission. This is a serious completeness issue for review. Also, there is no mention of code or data release, which would be important for reproducibility in this line of work. The core idea is promising, but the empirical and formal gaps are substantial enough to require major revision rather than acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on this paper. It introduces a genuinely new problem — few-shot generation of INR weight-space checkpoints — and proposes a concrete three-stage pipeline: an equivariant encoder pre-trained with contrastive learning, a diffusion model conditioned on those equivariant features, and a subspace-disturbance trick for diversity. The problem framing is new relative to the cited literature, and the reported gains over HyperDiffusion and other baselines are consistent across MNIST, CIFAR-10, and ShapeNet. That is a real contribution: it gives the weight-space generation community a setting worth working on.\n\nWhat the paper does well: the integration is sensible, the ablations show that removing the equivariant conditioning collapses into mode collapse, the use of Navon et al.'s equivariant architectures is appropriate, and the writing is readable. The paper earns credit for presenting a coherent pipeline rather than just a composite of tricks.\n\nThe soft spots are real but not all equally bad. The main one is the smooth augmentation in Sec. 4.2. The paper says it finds a TV-minimizing permutation P* and applies it to all weight matrices within each network subdivision, asserting that this preserves functional equivariance. It never states whether biases are permuted or whether the inverse permutation is applied to the adjacent layer. For pointwise activations like ReLU and SIREN's sine, exact functional symmetry requires W_l -> P_l W_l P_{l-1}^T and b_l -> P_l b_l. If the implementation doesn't handle biases, the transformed network is a different function, and the contrastive positive pairs in Eq. (4) are not functional equivalences. That would undercut the whole equivariance-guided generation. From the paper alone, we can't tell whether this is a real bug or just an omitted detail — but it's load-bearing enough that a referee must see it spelled out.\n\nOther weaknesses are smaller: all metrics in Tables 1–4 are single point estimates with no error bars or significance tests; the appendix, which is supposed to contain the formalization and architecture details, is missing from this version; and no code or data are released. The self-referentiality in Eq. (6) — using the same encoder as both conditioning and judge — is a mild concern, but not a fatal one.\n\nBottom line: the paper deserves a serious referee. The problem formulation and the pipeline are valuable enough that this should go through peer review, but the review should ask for a precise description of the smooth augmentation, verification that it preserves functionality (or a correction), and at least a few repeated runs with variance estimates. A reader who works on weight-space learning will get a useful new problem framing; a reader who wants a turn-key method should wait for the revision.","headline":"A genuinely new few-shot INR weight-generation setting with a plausible equivariance-based pipeline, but the central augmentation mechanism is underspecified and the empirical claims need error bars.","tokens_in":15479,"tokens_out":3445,"would_cite":false,"duration_ms":31552,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Generating diverse INR weights from a few examples works by exploiting weight-permutation symmetry, the paper argues.","keywords":["implicit neural representation","few-shot generation","weight-space equivariance","equivariant encoder","contrastive learning","weight-space diffusion","3D shape generation","INR weight generation"],"falsifier":"Render the original weight and its smooth-augmented version on a dense grid of inputs and measure the maximum or mean pointwise deviation of the output signal; if the deviation is large for typical SIREN or ReLU checkpoints, the positive pairs in the contrastive loss are not functionally equivalent and the learned equivariant subspace would not group functions by their behavior.","tokens_in":14435,"feed_emoji":"🧠","tokens_out":4187,"duration_ms":38286,"temperature":0.7,"pith_summary":"The paper introduces Few-shot Implicit Function Generation: given one to ten trained INR checkpoints of a new object class, produce many new checkpoints that are diverse in weight space yet render valid, category-consistent signals. It argues this is possible by exploiting the equivariance group of weight permutations, which relate functionally identical networks. The proposed framework, EquiGen, learns a latent space where equivalent weights cluster, then uses an equivariance-conditioned diffusion model plus controlled latent perturbations to sample new weights. The payoff is that generative modeling of weights no longer needs massive INR datasets, and can transfer knowledge from seen to unseen categories.","feed_headline":"Equivariance lets few INR samples generate many diverse weights","feed_subtitle":"EquiGen learns an equivariant latent space so 1–10 checkpoints yield many valid, varied implicit functions.","key_machinery":"The load-bearing object is the equivariant latent space learned by an equivariant encoder that respects weight-space permutation symmetries. The encoder is pre-trained with a SimCLR-style contrastive loss on positive pairs produced by a smooth augmentation step, which finds a permutation matrix $P^*$ that minimizes total variation over each network subgraph (solved as Shortest Hamiltonian Path problems with 2.5-opt local search), then applies INR-based augmentations such as rotation, translation, scaling, color jittering, and bias perturbation. Generation is carried by a diffusion transformer that predicts denoised weights directly and is conditioned on equivariant features via cross-attention, with an explicit equivariance regularization term $\\mathcal{L}_{eq}$; diversity comes from adding bounded Gaussian noise $\\gamma\\epsilon$ to the guiding features.","core_discovery":"EquiGen's central claim is that projecting network weights into an equivariant latent space reduces the data burden of weight generation: within that space, a few reference checkpoints suffice to generate diverse samples that stay inside the equivariance group of the target function. The paper implements this with three coupled stages: a contrastively trained equivariant encoder whose positive pairs come from smooth weight augmentations, a diffusion denoiser conditioned on equivariant features and regularized by an equivariance loss that keeps generated weights close in latent feature space, and a bounded Gaussian subspace disturbance that trades diversity against fidelity. On MNIST, CIFAR-10, and ShapeNet INR benchmarks in a 10-shot setting, the paper reports that EquiGen outperforms existing few-shot generation and weight-diffusion baselines on both quality (FID, MMD) and diversity (LPIPS, COV) metrics.","pith_inferences":["A natural extension the paper leaves implicit is to validate the smooth augmentation by measuring functional distance (not just visual similarity) between a weight and its augmented counterpart; if the permutation is not an exact symmetry for SIREN's sinusoidal activations, the contrastive positives could be tightened by filtering pairs whose rendered outputs differ beyond a threshold.","The equivariant subspace idea transfers to other permutation-symmetric weight spaces beyond INRs, including standard MLPs and transformers, so a testable extension is to apply EquiGen to classifier or language-model checkpoints where the 'function' is the model's input-output behavior.","One could also use the equivariant encoder as a functional metric: distances in the learned latent space may serve as a cheaper proxy for functional equivalence than rendering, which would make the framework useful for weight-space editing and model merging.","Because the paper reports results only for 10-shot and a few categories, the strongest version of the claim would be tested by scaling to more categories and smaller shot counts (1-, 2-, 5-shot) and checking whether the diversity-quality trade-off with $\\gamma$ persists."],"forward_implications":["If EquiGen's claim holds, weight-space generation no longer requires large INR corpora; a handful of checkpoints from a new class is enough to synthesize checkpoints that render valid instances of that class.","The equivariance-guided diffusion with the equivariance loss should let a single pre-trained generator adapt to an unseen category with only a few fine-tuning iterations, extending cross-class transfer to weight space.","The controlled subspace disturbance gives an explicit knob for trading diversity against fidelity: larger $\\gamma$ raises COV and LPIPS but also raises MMD and FID, so a user can set the knob to the desired operating point.","Because the equivariant encoder clusters weights by function rather than by element-wise similarity, the framework should be less prone to mode collapse than unconditional diffusion or label-conditioned generation, as the ablation study indicates."],"supporting_citations":[{"why":"Supplies the equivariant architecture for deep weight spaces that the encoder is built on.","marker":"[45]"},{"why":"Weight-space diffusion baseline and the denoising-transformer design that EquiGen adapts.","marker":"[16]"},{"why":"INR2Vec baseline that learns representations of INR weights, used as a comparison.","marker":"[13]"},{"why":"SimCLR contrastive learning framework used for encoder pre-training.","marker":"[7]"},{"why":"SIREN activation used in the INR datasets.","marker":"[59]"},{"why":"Source for the orthogonal independence of permutations and smooth weight manipulation motivating smooth augmentation.","marker":"[70]"},{"why":"Spectral-bias analysis motivating smooth signal modeling in weight space.","marker":"[51]"}],"fun_headline_variants":["Equivariance unlocks diverse INR generation from scarce checkpoints","Few-shot INR generation via weight equivariance","EquiGen: diverse implicit functions from only a few examples","Equivariant latent space produces diverse INRs from 10 examples"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the total-variation-minimizing permutation found by 2.5-opt search is an exact or near-exact symmetry of the rendered function, so that smooth-augmented weights genuinely belong to the same equivariance group.","fun_headline_variants_meta":{"raw":{"variants":["Equivariance unlocks diverse INR generation from scarce checkpoints","Few-shot INR generation via weight equivariance","EquiGen: diverse implicit functions from only a few examples","Equivariant latent space produces diverse INRs from 10 examples"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000212,"raw_usage":{"total_tokens":1401,"prompt_tokens":911,"completion_tokens":490,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":423}},"tokens_in":527,"tokens_out":490,"duration_ms":5026,"temperature":1.0,"reasoning_tokens":423,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:24:46.273572+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Render the original weight and its smooth-augmented version on a dense grid of inputs and measure the maximum or mean pointwise deviation of the output signal; if the deviation is large for typical SIREN or ReLU checkpoints, the positive pairs in the contrastive loss are not functionally equivalent and the learned equivariant subspace would not group functions by their behavior.","supporting_citations":[{"cited_title":"Equivariant architectures for learning in deep weight spaces","cited_arxiv_id":null,"evidence_quote":"Supplies the equivariant architecture for deep weight spaces that the encoder is built on."},{"cited_title":"Hyperdiffusion: Generating implicit neural fields with weight-space diffusion","cited_arxiv_id":null,"evidence_quote":"Weight-space diffusion baseline and the denoising-transformer design that EquiGen adapts."},{"cited_title":"Deep learning on implicit neural representations of shapes","cited_arxiv_id":null,"evidence_quote":"INR2Vec baseline that learns representations of INR weights, used as a comparison."},{"cited_title":"A simple framework for contrastive learn- ing of visual representations","cited_arxiv_id":null,"evidence_quote":"SimCLR contrastive learning framework used for encoder pre-training."},{"cited_title":"Implicit neural representa- tions with periodic activation functions","cited_arxiv_id":null,"evidence_quote":"SIREN activation used in the INR datasets."},{"cited_title":"Neural metamorphosis","cited_arxiv_id":null,"evidence_quote":"Source for the orthogonal independence of permutations and smooth weight manipulation motivating smooth augmentation."},{"cited_title":"On the spectral bias of neural networks","cited_arxiv_id":null,"evidence_quote":"Spectral-bias analysis motivating smooth signal modeling in weight space."}],"review_version":1}