{"id":"41b2f9da-8307-4b52-aea2-dc614141116f","arxiv_id":"2504.13412","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"The paper claims to prove that multigrid parametric encodings raise the NTK spectrum through their learnable grid, but the proof depends on an invalid additive kernel decomposition.","lead":"The paper analyzes why multigrid parametric encodings (MPEs) beat Fourier feature encodings (FFEs) for coordinate-based neural networks, using a neural tangent kernel analysis. It finds that the learned grid parameters, not the embedding space, raise the NTK eigenvalue spectrum, but the central proof relies on an invalid kernel decomposition.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 rests on an invalid additive decomposition of the ReLU NTK; the claimed proof does not establish that MPEs improve spectra via their grid.","rationale":"The reader's weakest assumption is exactly the invalid decomposition in Eq. 13, and I agree with that diagnosis. I checked the derivation: Eq. 9 defines the NTK on the full input, Eq. 12 replaces x with a scalar g~(x_i), and Eq. 13 sums scalar kernels. This identity does not hold for a ReLU MLP with a fully connected first layer. The rest of the theoretical contribution—the Weyl inequality argument and the 'grid vs embedding' isolation in Figure 2—depends on Eq. 13, so the central proof is not established. The empirical results are suggestive but do not repair the algebraic error. The Appendix B admission that the lazy-training assumption is broken further weakens the link from static NTK spectra to training performance, but it is secondary to the failure of Eq. 13. No change to the reader's REJECT verdict is needed.","tokens_in":18417,"tokens_out":6058,"duration_ms":58489,"concrete_test":"Compute the finite-width NTK for a single-hidden-unit ReLU MLP f = w2 ReLU(w1x x + w1g g + b) with one MPE scalar g, on two input points (x1,g1) and (x2,g2). Compare the exact 2x2 Gram matrix against the right-hand side of Eq. 13, which for this case is K([x1,x2]) + K([g1,g2]) + the grid-parameter contribution. If the matrices differ—or an analytic expansion shows cross terms involving ReLU'(w1x x + w1g g) do not cancel—then Eq. 13 is false and Theorem 1's proof fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Theorem 1) is proven only if Eq. 13 is valid: K_MPE = K_MLP + Σ K^l,a_MLP + Σ K^l,a_MPE. In the architecture of Sec. 3.3, the MLP input is the concatenated vector z = [x; g^1_1; ...; g^L_k], and the first layer computes ReLU(W1 z + βb) with a shared, fully connected W1. Because ReLU is applied after mixing all components, the NTK terms—for example, E[ReLU(W1 z_i)^T ReLU(W1 z_j)] and the W1-gradient terms involving ReLU'(W1 z_i) z_i^T—are nonlinear functions of the full vector z. They do not decompose into a sum of kernels evaluated separately on x and on each grid output. Eq. 12 itself replaces the full input by a single scalar g~(x_i); summing such expressions over grid cells would only be valid for linear activations or block-diagonal W1, neither of which holds. Since the proof defines K_+ via Eq. 13 and then applies Weyl's inequality to K_MLP + K_+, the theorem is unproven for ReLU MLPs. The subsequent claim that removing the K_MPE term isolates the effect of the learnable grid is also unsupported, because the K_MLP terms in that ablation are not the actual full-input MLP kernel.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper analyzes multigrid parametric encodings (MPE) through the neural tangent kernel (NTK). It derives a finite-width NTK for an MLP composed with an MPE, states Theorem 1 claiming that the eigenvalue spectrum of the composed kernel dominates the baseline MLP spectrum, and uses this result to argue that MPEs improve performance through the learnable grid rather than the embedding space. The paper also contrasts this with Fourier feature encodings (FFE), which it claims improve performance solely through the embedding. The claims are validated empirically on 2D image regression over 100 ImageNet synonym sets and 3D implicit surface regression on three Stanford meshes, reporting large PSNR/MS-SSIM gains and matching shifts in the empirical NTK spectrum. Appendix B concedes that the kernel changes materially during training and that the lazy-training assumption is violated.","tokens_in":18761,"tokens_out":11185,"duration_ms":98515,"significance":"If Theorem 1 were valid, the paper would provide the first NTK-based theoretical justification for the empirical superiority of multigrid parametric encodings and a clean mechanistic distinction between MPEs and FFEs. The experimental study is extensive and the reported alignment between spectral shifts and reconstruction quality is suggestive. However, the central theoretical result is not established: the proof relies on an invalid additive decomposition of the ReLU NTK, and the key ablation isolating the grid effect is computed from that same decomposition. As written, the contribution is primarily empirical, with the claimed proof and the mechanistic conclusion unsupported.","major_comments":[{"comment":"The theorem rests on an invalid decomposition of the NTK. In the architecture of Sec. 3.3 the MLP input is the concatenated vector z_i = [x_i; g_1^1(x_i); ...; g_L^k(x_i)] and the first layer computes ReLU(W^{(1)} z_i + βb) with a fully connected W^{(1)}. The NTK entries in Eq. (9), including the activation product and the W^{(1)}-gradient term containing φ'(W^{(1)} z_i) z_i z_j^T φ'(W^{(1)} z_j), are nonlinear functions of the joint vectors z_i and z_j; they do not equal a sum of kernels evaluated separately on x_i and on each scalar grid output. Equation (12) writes the kernel as though the MLP input were the single scalar g~(x_i), and Eq. (13) then asserts K_MPE = K_MLP + Σ K^l,a_MLP + Σ K^l,a_MPE by summing such scalar contributions. Such a decomposition would be valid only for linear activations or a block-diagonal first-layer weight matrix, neither of which holds. The grid-parameter gradient term in Eq. (12) is also dimensionally inconsistent, since g~'(x) in Eq. (11) is a 2×2 matrix in 2D while the surrounding expression requires a vector. Consequently the matrix K_+ in the proof of Theorem 1 is not the kernel of the actual network, and the application of Weyl's inequality does not establish the claimed bound.","section":"Section 4, Eqs. (12)–(13), Theorem 1"},{"comment":"The central mechanistic claim that MPEs act through the grid rather than the embedding is not supported by the reported computation. The 'MPE (No Grid)' spectrum is obtained by deleting the Σ K^l,a_MPE terms from Eq. (13). Because Eq. (13) is not the NTK of the actual MLP+MPE, this deletion does not isolate the embedding-space effect: there is no network whose NTK equals K_MLP + Σ K^l,a_MLP with the latter terms evaluated separately on grid scalars. The conclusion that 'the MPE derives its performance from the learnable parameters and not the higher dimensional embedding space' therefore does not follow from Figure 2.","section":"Section 4, Figure 2"},{"comment":"The manuscript concedes that the NTK changes substantially during training ('This breaks the lazy training assumption for the infinite width limit of the kernel') and Figure 11 shows spectra at epoch 0 and epoch 300 differing by orders of magnitude. The spectral-bias argument in Sec. 3, Eqs. (5)–(7), and the link between eigenvalue magnitude and convergence rate assume a fixed kernel over training. The reply that one may Taylor-expand about the optimal weights is informal; no bound or theorem is supplied. Thus the spectra measured at mid/end training (Figures 4 and 5) do not directly validate Theorem 1, and the claim that 'the spectral bias analysis using the eigenvalues of the NTK still holds' remains unjustified.","section":"Appendix B"}],"minor_comments":[{"comment":"'Let KMLP by the neural tangent kernel...' should read 'be the neural tangent kernel'; also the proof uses the symbol KMPE for the composed kernel while Eq. (13) uses K^l,a_MPE for the grid-induced terms, which is easy to confuse.","section":"Section 4, proof of Theorem 1"},{"comment":"The column header row is garbled, with two 'Baseline' entries and an 'ImageNet Scaling' label that appears to overlap the metric row; please reformat the table so each column is unambiguously labeled.","section":"Table 1"},{"comment":"'Right blot' should be 'right plot'; the sentence about the coarse MPE crossing over the FFE is also unclear because the figure shows spectra, not PSNR trajectories.","section":"Figure 4 caption"},{"comment":"There are typos: 'mid PPE, high PPE' should be 'mid FFE, high FFE', and 'elatively stable' should be 'relatively stable'.","section":"Appendix A.2 and Appendix B"},{"comment":"The simplification fθ(X,t) - Y = -e^{-Λt}QY assumes zero initial predictions; this assumption should be stated explicitly before the equation.","section":"Section 3.4, Eq. (7)"}],"recommendation":"reject","confidential_remarks":"The paper is already published at ICLR 2025, but for this journal the central theoretical contribution is unproven: Theorem 1 depends on an additive kernel decomposition that is false for ReLU MLPs, and the appendix's admission of a violated lazy-training assumption further weakens the NTK-based explanation. The empirical study is solid but does not compensate for the invalid proof. I would not request a revision unless the authors can supply a correct proof of the NTK decomposition for the concatenated-input architecture."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The first thing you should know: this paper is not ready to be cited for its main theorem. The empirical work is suggestive and the research direction is good, but the proof of Theorem 1 rests on an additive decomposition of the ReLU NTK that is simply not valid for the architecture being analyzed. That is a load-bearing flaw, not a cosmetic one.\n\nWhat is genuinely new and useful: this is the first NTK analysis aimed specifically at multigrid parametric encodings. The paper asks a good question — whether MPEs help through the embedding space or through the learnable grid parameters — and the figures showing the eigenvalue spectrum of MPEs far above FFE and baseline on 2D and 3D tasks are informative. The qualitative claim that MPEs mitigate spectral bias in a different way from FFEs is plausible and worth testing.\n\nWhere the paper goes wrong: Equation 12 and Equation 13 treat the NTK of an MLP on the concatenated input z = [x; g_1; ...; g_L] as if it were a sum of kernels evaluated on each grid component separately. But the first layer computes ReLU(W1 z + beta b) with a shared, dense W1, so the kernel terms are nonlinear functions of the full vector z; they do not decompose component-wise. The derivation in Eq. 12 even feeds a single scalar g~(x_i) through the full W1, which is dimensionally wrong. As a result, the K_MPE = K_MLP + K_+ decomposition that the Weyl inequality argument needs is unavailable for ReLU networks. Theorem 1 is unproven. The \"MPE (No Grid)\" ablation in Figure 2 inherits the same problem: either it is computed from that invalid decomposition, or — if computed by actually dropping grid-parameter gradients — the interpretation still depends on the false separation.\n\nOther soft spots in proportion: the abstract's PSNR/MS-SSIM deltas do not match the tables consistently (the 15 dB and 0.65 increases come from different experiments), Figure 5 reports means with no error bars, and Appendix B admits the NTK changes substantially during training, which weakens the lazy-training framing even if the spectral-bias intuition can survive.\n\nWho this is for: anyone working on implicit neural representations or encoding design will find the empirical spectra and the hyperparameter comparisons worth a look. But the central theoretical claim should not be taken as established. If I were the editor, I would send it to referees — there is real substance and the question matters — but I would expect a major revision that either fixes the decomposition for ReLU (perhaps by analyzing a block-diagonal or linearized variant) or repositions the contribution as empirical rather than proven.\n\nRecommendation: engage with the empirical parts, skip the proof, and cite (if at all) with a caveat.","headline":"The empirical story on MPE spectra is real and worth knowing, but the advertised proof of Theorem 1 is broken because the ReLU NTK does not decompose additively over concatenated grid components.","tokens_in":19247,"tokens_out":4356,"would_cite":false,"duration_ms":39293,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Learnable grid, not embedding, carries multigrid encoding's gain.","keywords":["neural tangent kernel","multigrid parametric encoding","Fourier feature encoding","spectral bias","coordinate-based MLP","implicit neural representation","eigenvalue spectrum","image regression"],"falsifier":"Compute the exact finite-width NTK for a small ReLU MLP with an MPE on a few dozen points and compare $K_{\\mathrm{MPE}}$ term-by-term with $K_{\\mathrm{MLP}} + \\sum K^{l,a}_{\\mathrm{MLP}} + \\sum K^{l,a}_{\\mathrm{MPE}}$; any difference beyond numerical precision refutes the decomposition that Theorem 1 relies on.","tokens_in":18203,"feed_emoji":"🧩","tokens_out":6617,"duration_ms":55237,"temperature":0.7,"pith_summary":"This paper sets out to explain why multigrid parametric encodings (MPEs), which interpolate learnable values on a grid and feed them to a coordinate-based MLP, learn fine detail better than Fourier feature encodings (FFEs). Using the neural tangent kernel (NTK), the authors prove that the MPE raises the eigenvalue spectrum of the training kernel, and they isolate the cause: the learnable grid parameters, not the higher-dimensional embedding space, add the decisive kernel terms. If the proof is right, it gives the first theoretical account of the MPE's advantage and a concrete design rule: the grid, not just the embedding, is what recovers high-frequency detail.","feed_headline":"Learnable grid, not embedding, carries multigrid encoding's gain","feed_subtitle":"A neural tangent kernel proof ties the 15 dB PSNR gain to grid parameters, not the embedding space.","key_machinery":"The central object is the finite-width neural tangent kernel of the composed network $f_\\theta \\circ \\gamma_{M,\\phi}$, with the MPE written as a concatenation of bilinearly interpolated grid outputs plus the original coordinates. The load-bearing identity is the additive decomposition $K_{\\mathrm{MPE}} = K_{\\mathrm{MLP}} + \\sum_{l,a} K^{l,a}_{\\mathrm{MLP}} + \\sum_{l,a} K^{l,a}_{\\mathrm{MPE}}$, where the last sum contains the gradients with respect to the learnable grid weights. Because every term is a symmetric positive semidefinite Gram matrix, Weyl's inequality turns the decomposition into an eigenvalue bound; the grid-induced term is what lifts the spectrum.","core_discovery":"The paper's central claim is Theorem 1: for a dataset of n samples, the i-th eigenvalue of the NTK for an MLP with an MPE is bounded below by the i-th eigenvalue of the plain MLP kernel plus the smallest eigenvalue of the grid-only kernel, $\\lambda_i^{\\mathrm{MLP}} \\le \\lambda_i^{\\mathrm{MLP}} + \\lambda_n^{\\mathrm{MPE}} \\le \\lambda_i^{\\mathrm{MLP+MPE}}$. The proof writes the composed kernel as the baseline kernel plus a sum of positive semidefinite kernel matrices contributed by each grid layer and each learnable grid parameter, then applies Weyl's inequality. The authors further claim that this grid contribution is the dominant mechanism: when the grid term is removed from the kernel, the MPE's spectrum barely rises above the baseline, whereas the FFE's improvement comes entirely from its embedding. Empirically, the MPE raises the smallest eigenvalue by 8 orders of magnitude over the baseline and 2 over the FFE, matching 15 dB PSNR / 0.65 MS-SSIM over baseline and 12 dB / 0.33 over FFE on image and 3D surface regression.","pith_inferences":["Editorial extension: if the grid parameters are the true source of the spectral gain, then sparse and hash-grid variants should show the same eigenvalue lift, and freezing grid weights after a short burn-in would be a direct test of the paper's mechanism.","Editorial extension: the additive-kernel picture suggests one could choose the interpolation kernel or grid resolutions to shape specific parts of the spectrum rather than only raising the minimum eigenvalue; this is not tested in the paper.","Editorial extension: the paper's own observation that the spectrum shifts during training indicates the infinite-width lazy-training idealization is imperfect, so the eigenvalue bound is most naturally a statement about early and mid-training dynamics, not a fully frozen kernel."],"forward_implications":["Increasing the number of grid layers or learnable parameters per grid node adds a positive semidefinite term to the kernel, so the proof predicts a monotone rise in the NTK spectrum at the cost of memory and compute.","Because the grid term, not the embedding, drives the gain, a random or frozen grid should fall back toward baseline performance; the learned grid parameters are the operative component.","The MPE's eigenvalue advantage predicts faster convergence along high-frequency directions, which is why it reaches low loss earlier than the FFE and baseline in the reported training curves.","The same additive structure applies in 2D and 3D coordinate mappings, so the mechanism should transfer to other low-dimensional regression tasks such as radiance fields and physics-informed networks."],"supporting_citations":[{"why":"It introduces the neural tangent kernel and the kernel-regression view of training that the whole analysis uses.","marker":"(Jacot et al., 2018)"},{"why":"It shows Fourier feature encodings let coordinate MLPs learn high frequencies and analyzes their NTK spectrum; this is the FFE baseline.","marker":"(Tancik et al., 2020)"},{"why":"It introduces the multigrid parametric encoding whose kernel the paper derives.","marker":"(Hadadan et al., 2021)"},{"why":"It presents the multiresolution hash grid encoding, the empirical MPE variant the paper's 3D results are meant to explain.","marker":"(Müller et al., 2022)"},{"why":"It establishes the spectral-bias link between low NTK eigenvalues and slow learning of high-frequency components.","marker":"(Basri et al., 2019)"},{"why":"It analyzes NTK spectra for coordinate networks and FFEs, supplying the comparison point for spectrum-based evaluation.","marker":"(Wang et al., 2020)"},{"why":"It provides the fine-grained spectral view and recursive extension of NTK results to deeper MLPs.","marker":"(Yang & Salman, 2019)"},{"why":"It supplies Weyl's inequality, the eigenvalue-comparison step in Theorem 1.","marker":"(Weyl, 1912)"},{"why":"It gives the fast finite-width NTK computation used to measure the spectra in the experiments.","marker":"(Novak et al., 2022)"},{"why":"It defines the MS-SSIM metric used to quantify fine-detail learning.","marker":"(Wang et al., 2003)"}],"fun_headline_variants":["Grid structure, not embedding, boosts neural encoding","Multigrid encoding's edge traced to learnable grid","NTK proof: grid parameters, not embeddings, drive gains","Why learnable grids beat Fourier features in low dims","Grid's kernel contribution unlocks fine detail"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof assumes the learning kernel of the MLP acting on the concatenated grid outputs is exactly the sum of kernels computed on each grid output separately; for a ReLU network the kernel is a nonlinear function of the full concatenated input, so if that additive split fails, the eigenvalue bound does not follow.","fun_headline_variants_meta":{"raw":{"variants":["Grid structure, not embedding, boosts neural encoding","Multigrid encoding's edge traced to learnable grid","NTK proof: grid parameters, not embeddings, drive gains","Why learnable grids beat Fourier features in low dims","Grid's kernel contribution unlocks fine detail"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000282,"raw_usage":{"total_tokens":1774,"prompt_tokens":1159,"completion_tokens":615,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":775,"completion_tokens_details":{"reasoning_tokens":539}},"tokens_in":775,"tokens_out":615,"duration_ms":5567,"temperature":1.0,"reasoning_tokens":539,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:10:04.167724+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute the exact finite-width NTK for a small ReLU MLP with an MPE on a few dozen points and compare $K_{\\mathrm{MPE}}$ term-by-term with $K_{\\mathrm{MLP}} + \\sum K^{l,a}_{\\mathrm{MLP}} + \\sum K^{l,a}_{\\mathrm{MPE}}$; any difference beyond numerical precision refutes the decomposition that Theorem 1 relies on.","supporting_citations":[{"cited_title":"Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung)","cited_arxiv_id":null,"evidence_quote":"It supplies Weyl's inequality, the eigenvalue-comparison step in Theorem 1."}],"review_version":1}