Pith. sign in

REVIEW 3 major objections 5 minor 42 references

How Learnable Grids Recover Fine Detail in Low Dimensions: A Neural Tangent Kernel Analysis of Multigrid Parametric Encodings

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Learnable grid, not embedding, carries multigrid encoding's gain.

desk verdict The empirical story on MPE spectra is real and worth knowing, but the advertised proof of Theorem 1 is broken because the ReLU NTK does not decompose additively over concatenated grid components. read the letter →

arxiv 2504.13412 v1 pith:U2DNSC33 submitted 2025-04-18 cs.CV cs.LG

Samuel Audia , Soheil Feizi , Matthias Zwicker , Dinesh Manocha This is my paper · ORCID
classification cs.CVcs.LG
keywords neuraltangentkernelmultigridparametricencodingFourierfeaturespectralbiascoordinate-basedMLPimplicitrepresentationeigenvaluespectrumimageregression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to explain why multigrid parametric encodings (MPEs), which interpolate learnable values on a grid and feed them to a coordinate-based MLP, learn fine detail better than Fourier feature encodings (FFEs). Using the neural tangent kernel (NTK), the authors prove that the MPE raises the eigenvalue spectrum of the training kernel, and they isolate the cause: the learnable grid parameters, not the higher-dimensional embedding space, add the decisive kernel terms. If the proof is right, it gives the first theoretical account of the MPE's advantage and a concrete design rule: the grid, not just the embedding, is what recovers high-frequency detail.

What carries the argument

The central object is the finite-width neural tangent kernel of the composed network $f_\theta \circ \gamma_{M,\phi}$, with the MPE written as a concatenation of bilinearly interpolated grid outputs plus the original coordinates. The load-bearing identity is the additive decomposition $K_{\mathrm{MPE}} = K_{\mathrm{MLP}} + \sum_{l,a} K^{l,a}_{\mathrm{MLP}} + \sum_{l,a} K^{l,a}_{\mathrm{MPE}}$, where the last sum contains the gradients with respect to the learnable grid weights. Because every term is a symmetric positive semidefinite Gram matrix, Weyl's inequality turns the decomposition into an eigenvalue bound; the grid-induced term is what lifts the spectrum.

What would settle it

Compute the exact finite-width NTK for a small ReLU MLP with an MPE on a few dozen points and compare $K_{\mathrm{MPE}}$ term-by-term with $K_{\mathrm{MLP}} + \sum K^{l,a}_{\mathrm{MLP}} + \sum K^{l,a}_{\mathrm{MPE}}$; any difference beyond numerical precision refutes the decomposition that Theorem 1 relies on.

Watch

Extended reading notes

Core claim

The paper's central claim is Theorem 1: for a dataset of n samples, the i-th eigenvalue of the NTK for an MLP with an MPE is bounded below by the i-th eigenvalue of the plain MLP kernel plus the smallest eigenvalue of the grid-only kernel, $\lambda_i^{\mathrm{MLP}} \le \lambda_i^{\mathrm{MLP}} + \lambda_n^{\mathrm{MPE}} \le \lambda_i^{\mathrm{MLP+MPE}}$. The proof writes the composed kernel as the baseline kernel plus a sum of positive semidefinite kernel matrices contributed by each grid layer and each learnable grid parameter, then applies Weyl's inequality. The authors further claim that this grid contribution is the dominant mechanism: when the grid term is removed from the kernel, the MPE's spectrum barely rises above the baseline, whereas the FFE's improvement comes entirely from its embedding. Empirically, the MPE raises the smallest eigenvalue by 8 orders of magnitude over the baseline and 2 over the FFE, matching 15 dB PSNR / 0.65 MS-SSIM over baseline and 12 dB / 0.33 over FFE on image and 3D surface regression.

Load-bearing premise

The proof assumes the learning kernel of the MLP acting on the concatenated grid outputs is exactly the sum of kernels computed on each grid output separately; for a ReLU network the kernel is a nonlinear function of the full concatenated input, so if that additive split fails, the eigenvalue bound does not follow.

Editorial extensions

If this is right

  • Increasing the number of grid layers or learnable parameters per grid node adds a positive semidefinite term to the kernel, so the proof predicts a monotone rise in the NTK spectrum at the cost of memory and compute.
  • Because the grid term, not the embedding, drives the gain, a random or frozen grid should fall back toward baseline performance; the learned grid parameters are the operative component.
  • The MPE's eigenvalue advantage predicts faster convergence along high-frequency directions, which is why it reaches low loss earlier than the FFE and baseline in the reported training curves.
  • The same additive structure applies in 2D and 3D coordinate mappings, so the mechanism should transfer to other low-dimensional regression tasks such as radiance fields and physics-informed networks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: if the grid parameters are the true source of the spectral gain, then sparse and hash-grid variants should show the same eigenvalue lift, and freezing grid weights after a short burn-in would be a direct test of the paper's mechanism.
  • Editorial extension: the additive-kernel picture suggests one could choose the interpolation kernel or grid resolutions to shape specific parts of the spectrum rather than only raising the minimum eigenvalue; this is not tested in the paper.
  • Editorial extension: the paper's own observation that the spectrum shifts during training indicates the infinite-width lazy-training idealization is imperfect, so the eigenvalue bound is most naturally a statement about early and mid-training dynamics, not a fully frozen kernel.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper analyzes multigrid parametric encodings (MPE) through the neural tangent kernel (NTK). It derives a finite-width NTK for an MLP composed with an MPE, states Theorem 1 claiming that the eigenvalue spectrum of the composed kernel dominates the baseline MLP spectrum, and uses this result to argue that MPEs improve performance through the learnable grid rather than the embedding space. The paper also contrasts this with Fourier feature encodings (FFE), which it claims improve performance solely through the embedding. The claims are validated empirically on 2D image regression over 100 ImageNet synonym sets and 3D implicit surface regression on three Stanford meshes, reporting large PSNR/MS-SSIM gains and matching shifts in the empirical NTK spectrum. Appendix B concedes that the kernel changes materially during training and that the lazy-training assumption is violated.

Significance. If Theorem 1 were valid, the paper would provide the first NTK-based theoretical justification for the empirical superiority of multigrid parametric encodings and a clean mechanistic distinction between MPEs and FFEs. The experimental study is extensive and the reported alignment between spectral shifts and reconstruction quality is suggestive. However, the central theoretical result is not established: the proof relies on an invalid additive decomposition of the ReLU NTK, and the key ablation isolating the grid effect is computed from that same decomposition. As written, the contribution is primarily empirical, with the claimed proof and the mechanistic conclusion unsupported.

major comments (3)
  1. [Section 4, Eqs. (12)–(13), Theorem 1] The theorem rests on an invalid decomposition of the NTK. In the architecture of Sec. 3.3 the MLP input is the concatenated vector z_i = [x_i; g_1^1(x_i); ...; g_L^k(x_i)] and the first layer computes ReLU(W^{(1)} z_i + βb) with a fully connected W^{(1)}. The NTK entries in Eq. (9), including the activation product and the W^{(1)}-gradient term containing φ'(W^{(1)} z_i) z_i z_j^T φ'(W^{(1)} z_j), are nonlinear functions of the joint vectors z_i and z_j; they do not equal a sum of kernels evaluated separately on x_i and on each scalar grid output. Equation (12) writes the kernel as though the MLP input were the single scalar g~(x_i), and Eq. (13) then asserts K_MPE = K_MLP + Σ K^l,a_MLP + Σ K^l,a_MPE by summing such scalar contributions. Such a decomposition would be valid only for linear activations or a block-diagonal first-layer weight matrix, neither of which holds. The grid-parameter gradient term in Eq. (12) is also dimensionally inconsistent, since g~'(x) in Eq. (11) is a 2×2 matrix in 2D while the surrounding expression requires a vector. Consequently the matrix K_+ in the proof of Theorem 1 is not the kernel of the actual network, and the application of Weyl's inequality does not establish the claimed bound.
  2. [Section 4, Figure 2] The central mechanistic claim that MPEs act through the grid rather than the embedding is not supported by the reported computation. The 'MPE (No Grid)' spectrum is obtained by deleting the Σ K^l,a_MPE terms from Eq. (13). Because Eq. (13) is not the NTK of the actual MLP+MPE, this deletion does not isolate the embedding-space effect: there is no network whose NTK equals K_MLP + Σ K^l,a_MLP with the latter terms evaluated separately on grid scalars. The conclusion that 'the MPE derives its performance from the learnable parameters and not the higher dimensional embedding space' therefore does not follow from Figure 2.
  3. [Appendix B] The manuscript concedes that the NTK changes substantially during training ('This breaks the lazy training assumption for the infinite width limit of the kernel') and Figure 11 shows spectra at epoch 0 and epoch 300 differing by orders of magnitude. The spectral-bias argument in Sec. 3, Eqs. (5)–(7), and the link between eigenvalue magnitude and convergence rate assume a fixed kernel over training. The reply that one may Taylor-expand about the optimal weights is informal; no bound or theorem is supplied. Thus the spectra measured at mid/end training (Figures 4 and 5) do not directly validate Theorem 1, and the claim that 'the spectral bias analysis using the eigenvalues of the NTK still holds' remains unjustified.
minor comments (5)
  1. [Section 4, proof of Theorem 1] 'Let KMLP by the neural tangent kernel...' should read 'be the neural tangent kernel'; also the proof uses the symbol KMPE for the composed kernel while Eq. (13) uses K^l,a_MPE for the grid-induced terms, which is easy to confuse.
  2. [Table 1] The column header row is garbled, with two 'Baseline' entries and an 'ImageNet Scaling' label that appears to overlap the metric row; please reformat the table so each column is unambiguously labeled.
  3. [Figure 4 caption] 'Right blot' should be 'right plot'; the sentence about the coarse MPE crossing over the FFE is also unclear because the figure shows spectra, not PSNR trajectories.
  4. [Appendix A.2 and Appendix B] There are typos: 'mid PPE, high PPE' should be 'mid FFE, high FFE', and 'elatively stable' should be 'relatively stable'.
  5. [Section 3.4, Eq. (7)] The simplification fθ(X,t) - Y = -e^{-Λt}QY assumes zero initial predictions; this assumption should be stated explicitly before the equation.

Circularity Check

2 steps flagged · score 7.0 of 10

Theorem 1's spectral bound is built into Eq. 13: K_MPE is defined as K_MLP plus grid terms, so both the eigenvalue increase and the 'grid not embedding' mechanism are true by construction.

  1. self definitional [Section 4, Eq. 13 and Theorem 1 proof]
    "The NTK induced by the composition of the MPE and a MLP is the sum of the original NTK, Equation 9, without any encoding and the NTK produced by each layer and learnable parameter, Equation 12. ... From Equation 13 we see that KMPE = KMLP + K+, where K+ is the kernel produced by the sum over all learnable parameters in the grid. ... It follows from Weyl's inequality ... λi(KMLP ) ≤ λi(KMLP ) + λn(K+) ≤ λi(KMPE )."

    Eq. 13 is not an independently derived expression for the actual NTK of the composed network; it postulates that the MPE kernel is the baseline kernel plus a positive-semidefinite grid-parameter sum. Under that definition, Theorem 1 is an immediate Weyl-inequality corollary, so the claimed 'proof' cannot fail: the conclusion is the decomposition restated. For the ReLU MLP of Sec. 3.3 the first-layer weight is shared across the concatenated input, and the NTK is not a sum of per-component kernels, so Eq. 13 is an ansatz rather than a derivation; the theorem therefore has no independent content beyond its premise.

  2. self definitional [Section 4, paragraph after Eq. 14; Figure 2 caption]
    "To isolate the improvements in the MPE, we compute the spectrum both with and without the KMPE term (Figure 2). Without the contributions of the learnable grid, the MPE has little to no effect on the eigenvalues as compared to baseline. We conclude that the MPE derives its performance from the learnable parameters and not the higher dimensional embedding space, while the FFE's performance rests solely on the embedding. ... MPE (No Grid) is the NTK of the MPE without the contributions of KMPE and is purely for theoretical analysis."

    The 'no grid' kernel is defined by deleting the K_MPE term from Eq. 13, so the ablation's meaning is fixed by the same additive decomposition used in Theorem 1. The conclusion that the learnable grid, rather than the embedding, causes the improvement is a reading-off of which term was removed from the assumed formula, not an independent empirical test of the mechanism. Because the decomposition itself is unproven and in fact does not hold for the ReLU MLP, the grid-versus-embedding attribution is built into the definition rather than established by the experiment.

full rationale

The paper's empirical NTK spectrum measurements on ImageNet and the Stanford 3D meshes are genuine external evidence and are not themselves circular. However, the theoretical centerpiece is not an independent derivation: Eq. 13 defines the composed MPE kernel as the baseline NTK plus grid-parameter terms, making Theorem 1's eigenvalue bound an immediate Weyl-inequality consequence of that definition. The subsequent claim that MPEs improve performance through the learnable grid and not the embedding space is likewise tied to the same assumed decomposition, since the 'without grid' ablation is obtained by removing the grid term from Eq. 13. For the actual ReLU MLP, the shared first-layer weight means the NTK does not decompose additively over concatenated components, so the proof and the mechanism claim lack independent content. Because the empirical spectra are external and the paper does not fit the eigenvalue increases to the theorem, the circularity is partial rather than total.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central proof depends on the assumed additive decomposition of the NTK (Eq. 13), which is not justified for ReLU networks. Experimental hyperparameters are tuned to data, and the connection between the NTK spectrum and learning dynamics is taken as a domain assumption that the paper itself weakens in Appendix B.

free parameters (5)
  • FFE frequency L (tuned) = L=6 (ImageNet), L=4/8/16 (scaling)
    Controls the embedding dimension of the Fourier encoding; tuned by Optuna or hand-selected, directly affecting spectrum and PSNR in the comparison.
  • MPE grid resolution x = [96,277] (ImageNet tuned), 100/200 (scaling), 38/102, 44, 33/136 (3D)
    The grid cell counts per layer; chosen by tuning or hand, determines the 'coarse' vs 'fine' behavior and the size of the grid-gradient kernel term.
  • MPE number of layers L and scalars per node k = k=3,L=2 (ImageNet), k=2,L=1 (scaling), k=1-2,L=1-2 (3D)
    Number of grid layers and learnable scalars per node; controls how much the kernel's positive definite extra term contributes.
  • Network width and depth = 512x2 (2D), 256x8 (3D)
    Architecture choices held constant within each domain; the NTK regime is more accurate for larger widths, but these finite widths are used.
  • Learning rate = 100 (scaling), ~0.2-0.99 (tuned)
    The scaling experiments use a large learning rate (100) which, combined with NTK parameterization, may move weights outside the lazy training regime.
assumptions (5)
  • domain assumption The NTK in the infinite-width limit governs the training dynamics of the finite-width networks used in the experiments.
    Invoked in Sections 3-4 and used to connect eigenvalue spectra to learning speeds; Appendix B states the inputs scaled to [0,1] break the lazy-training assumption for the infinite-width limit.
  • standard math Weyl's eigenvalue inequality for sums of Hermitian matrices.
    Used in the proof of Theorem 1.
  • domain assumption Eigenvalues of the NTK corresponding to higher-frequency features are lower, so raising the spectrum improves high-frequency learning.
    Taken from Basri et al. (2019) and used to connect the spectrum to image quality.
  • ad hoc to paper The NTK of the composed MLP+MPE decomposes as K_MPE = K_MLP + sum over grid layers/parameters of (K_MLP for that component + K_MPE grid gradient term), as in Eq. 13.
    This is the load-bearing premise of Theorem 1. It is not proven and is incorrect for ReLU networks, where the MLP kernel on the concatenated input is not additive over components.
  • domain assumption The finite-width empirical NTK evaluated at initialization, mid-training, and end-of-training is a faithful proxy for learning dynamics.
    Used in all experiments; Appendix B shows the spectrum evolves substantially during training, contradicting kernel stationarity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of How Learnable Grids Recover Fine Detail in Low Dimensions: A Neural Tangent Kernel Analysis of Multigrid Parametric Encodings." pith.science (2026). https://pith.science/paper/U2DNSC33

@misc{pith2026250413412,
  author       = {Pith},
  title        = {Pith review of: How Learnable Grids Recover Fine Detail in Low Dimensions: A Neural Tangent Kernel Analysis of Multigrid Parametric Encodings},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/U2DNSC33}},
  note         = {Machine review of arXiv:2504.13412}
}
read the original abstract

Neural networks that map between low dimensional spaces are ubiquitous in computer graphics and scientific computing; however, in their naive implementation, they are unable to learn high frequency information. We present a comprehensive analysis comparing the two most common techniques for mitigating this spectral bias: Fourier feature encodings (FFE) and multigrid parametric encodings (MPE). FFEs are seen as the standard for low dimensional mappings, but MPEs often outperform them and learn representations with higher resolution and finer detail. FFE's roots in the Fourier transform, make it susceptible to aliasing if pushed too far, while MPEs, which use a learned grid structure, have no such limitation. To understand the difference in performance, we use the neural tangent kernel (NTK) to evaluate these encodings through the lens of an analogous kernel regression. By finding a lower bound on the smallest eigenvalue of the NTK, we prove that MPEs improve a network's performance through the structure of their grid and not their learnable embedding. This mechanism is fundamentally different from FFEs, which rely solely on their embedding space to improve performance. Results are empirically validated on a 2D image regression task using images taken from 100 synonym sets of ImageNet and 3D implicit surface regression on objects from the Stanford graphics dataset. Using peak signal-to-noise ratio (PSNR) and multiscale structural similarity (MS-SSIM) to evaluate how well fine details are learned, we show that the MPE increases the minimum eigenvalue by 8 orders of magnitude over the baseline and 2 orders of magnitude over the FFE. The increase in spectrum corresponds to a 15 dB (PSNR) / 0.65 (MS-SSIM) increase over baseline and a 12 dB (PSNR) / 0.33 (MS-SSIM) increase over the FFE.

Figures

Figures reproduced from arXiv: 2504.13412 by the authors.

Figure 1
Figure 1. The above figure shows an example of the multigrid parametric encoding (MPE). The [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. This plot isolates the improvements of the MPE to the learnable grid. The NTK is com [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. We compare performance of different encodings on image regression. We show the ground [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: The NTK eigenvalue spectrum is compared for the cases found in Figure 3 and Table 3. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: This figure compares the mean of the eigenvalue spectra of different encodings across ran [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: We report the peak signal-to-noise ratio (PSNR) and mean squared error (MSE) for the [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Results for the 2D image problem on an image of a windmill in a natural setting. See [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: We plot the eigenvalue spectrum (left) at epoch 150 (dashed line) and epoch 300 (solid) [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Results for the 2D image problem on an image of a lake in a natural setting. See Section [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: We plot the eigenvalue spectrum (left) at epoch 150 (dashed line) and epoch 300 (solid) [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]
Figure 11
Figure 11. Figure 11: We plot the eigenvalues at the start of training (dotted line) and at epoch 300 (solid line). [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: This figure shows the learned scalars at grid points in the MPE as a gray scale image. [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: In this figure we plot the total number of activation regions found in the network across [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: This figure plots the depth field at the 0.5 level set of the learned implicit surface. This [PITH_FULL_IMAGE:figures/full_fig_p021_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 4 canonical work pages

  1. [1]

    Optuna: A Next-generation Hyperparameter Optimization Framework

    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A Next-generation Hyperparameter Optimization Framework . Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp.\ 2623--2631, 7 2019. doi:10.1145/3292500.3330701. URL https://arxiv.org/abs/1907.10902v1

    arXiv 2019

  2. [2]

    Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P

    Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields . Proceedings of the IEEE International Conference on Computer Vision, pp.\ 5835--5844, 3 2021. ISSN 15505499. doi:10.1109/ICCV48922.2021.00580. URL https://arxiv...

    arXiv 2021

  3. [3]

    The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies

    Ronen Basri, David Jacobs, Yoni Kasten, and Shira Kritchman. The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies . Advances in Neural Information Processing Systems, 32, 6 2019. ISSN 10495258. URL https://arxiv.org/abs/1906.00425v3

    arXiv 2019

  4. [4]

    Frequency Bias in Neural Networks for Input of Non-Uniform Density

    Ronen Basri, Meirav Galun, Amnon Geifman, David Jacobs, Yoni Kasten, and Shira Kritchman. Frequency Bias in Neural Networks for Input of Non-Uniform Density . 37th International Conference on Machine Learning, ICML 2020, PartF168147-1: 0 662--671, 3 2020. URL https://arxiv.org/abs/2003.04560v1

    arXiv 2020

  5. [5]

    Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion

    Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll. Implicit Functions in Feature Space for 3D Shape Reconstruction and Completion . Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.\ 6968--6979, 3 2020. ISSN 10636919. doi:10.1109/CVPR42600.2020.00700. URL https://arxiv.org/abs/2003.01456v2

    arXiv 2020

  6. [6]

    Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next

    Salvatore Cuomo, Vincenzo Schiano Di Cola, Fabio Giampaolo, Gianluigi Rozza, Maziar Raissi, and Francesco Piccialli. Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next . Journal of Scientific Computing, 92, 1 2022. ISSN 15737691. doi:10.1007/s10915-022-01939-z. URL https://arxiv.org/abs/2201.05624v4

    arXiv 2022

  7. [7]

    ImageNet: A Large-Scale Hierarchical Image Database

    Jia Deng, Wei Dong, Richard Socher, Li Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database . 2009 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp.\ 248--255, 2009. doi:10.1109/CVPR.2009.5206848

    arXiv 2009

  8. [8]

    Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey

    Amer Essakine, Yanqi Cheng, Chun-Wun Cheng, Lipei Zhang, Zhongying Deng, Lei Zhu, Carola-Bibiane Sch \" o nlieb, and Angelica I Aviles-Rivero. Where Do We Stand with Implicit Neural Representations? A Technical and Performance Survey . 11 2024. URL https://arxiv.org/abs/2411.03688v1

    arXiv 2024

Show all 42 references
  1. [9]

    Plenoxels: Radiance Fields without Neural Networks

    Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. Plenoxels: Radiance Fields without Neural Networks . Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2022-June: 0 5491--5500, 12 202...

    2022

  2. [10]

    Neural radiosity

    Saeed Hadadan, Shuhong Chen, and Matthias Zwicker. Neural radiosity . ACM Transactions on Graphics, 40 0 (6), 12 2021. ISSN 15577368. doi:10.1145/3478513.3480569. URL https://dl.acm.org/doi/10.1145/3478513.3480569

    2021

  3. [11]

    Deep ReLU Networks Have Surprisingly Few Activation Patterns

    Boris Hanin and David Rolnick. Deep ReLU Networks Have Surprisingly Few Activation Patterns . Advances in Neural Information Processing Systems, 32, 6 2019. ISSN 10495258. URL https://arxiv.org/abs/1906.00904v2

    2019 arXiv

  4. [12]

    Efficient physics-informed neural networks using hash encoding

    Xinquan Huang and Tariq Alkhalifah. Efficient physics-informed neural networks using hash encoding . Journal of Computational Physics, 501, 2 2023. doi:10.1016/j.jcp.2024.112760. URL http://arxiv.org/abs/2302.13397 http://dx.doi.org/10.1016/j.jcp.2024.112760

    2023

  5. [13]

    Neural Tangent Kernel: Convergence and Generalization in Neural Networks

    Arthur Jacot, Franck Gabriel, and Clément Hongler. Neural Tangent Kernel: Convergence and Generalization in Neural Networks . Advances in Neural Information Processing Systems, 2018-December: 0 8571--8580, 6 2018. ISSN 10495258. URL https://arxiv.org/abs/1806.07572v4

    2018 arXiv

  6. [14]

    Stochastic Estimation of the Maximum of a Regression Function

    J Kiefer and J Wolfowitz. Stochastic Estimation of the Maximum of a Regression Function . The Annals of Mathematical Statistics, 23 0 (3): 0 462--466, 3 1952. ISSN 00034851. URL http://www.jstor.org/stable/2236690

    1952

  7. [15]

    Understanding the Spectral Bias of Coordinate Based MLPs Via Training Dynamics

    John Lazzari and Xiuwen Liu. Understanding the Spectral Bias of Coordinate Based MLPs Via Training Dynamics . 1 2023. URL https://arxiv.org/abs/2301.05816v4

    2023 arXiv

  8. [16]

    Yuanzhi Li, Tengyu Ma, and Hongyang R. Zhang. Learning Over-Parametrized Two-Layer ReLU Neural Networks beyond NTK . Proceedings of Machine Learning Research, 125: 0 2613--2682, 7 2020. ISSN 26403498. URL https://arxiv.org/abs/2007.04596v1

    2020 arXiv

  9. [17]

    Karniadakis

    Lu Lu, Xuhui Meng, Zhiping Mao, and George E. Karniadakis. DeepXDE: A deep learning library for solving differential equations . SIAM Review, 63 0 (1): 0 208--228, 7 2019. doi:10.1137/19M1274067. URL http://arxiv.org/abs/1907.04502 http://dx.doi.org/10.1137/19M1274067

    2019 arXiv

  10. [18]

    Occupancy Networks: Learning 3D Reconstruction in Function Space

    Lars Mescheder, Michael Oechsle, Michael Niemeyer, Sebastian Nowozin, and Andreas Geiger. Occupancy Networks: Learning 3D Reconstruction in Function Space . Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019-June: 0 4455--4465,...

    2019

  11. [19]

    Srinivasan, Matthew Tancik, Jonathan T

    Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis . Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and ...

    2020 arXiv

  12. [20]

    Real-time neural radiance caching for path tracing

    Thomas M \" u ller, Fabrice Rousselle, Jan Nov \' a k, and Alexander Keller. Real-time neural radiance caching for path tracing . ACM Transactions on Graphics (TOG), 40 0 (4), 7 2021. ISSN 15577368. doi:10.1145/3450626.3459812. URL https://dl.acm.org/doi/10.1145/3450626.3459812

    2021

  13. [21]

    Instant neural graphics primitives with a multiresolution hash encoding

    Thomas M \" u ller, Alex Evans, Christoph Schied, and Alexander Keller. Instant neural graphics primitives with a multiresolution hash encoding . ACM Transactions on Graphics (TOG), 41 0 (4): 0 102, 7 2022. ISSN 15577368. doi:10.1145/3528223.3530127. URL https://dl.acm.org/doi...

    2022

  14. [22]

    Schoenholz

    Roman Novak, Jascha Sohl-Dickstein, and Samuel S. Schoenholz. Fast Finite Width Neural Tangent Kernel . Proceedings of Machine Learning Research, 162: 0 17018--17044, 6 2022. ISSN 26403498. URL https://arxiv.org/abs/2206.08720v1

    2022 arXiv

  15. [23]

    PyTorch: An Imperative Style, High-Performance Deep Learning Library

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K \" o pf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, L...

    2019 arXiv

  16. [25]

    Random features for large-scale kernel machines

    Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines . In Advances in Neural Information Processing Systems 20 - Proceedings of the 2007 Conference, 2008. ISBN 160560352X

    2007

  17. [26]

    Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations

    Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations . 11 2017. URL https://arxiv.org/abs/1711.10561v1

    2017 arXiv

  18. [27]

    Advances in kernel methods: support vector learning

    Bernhard Sch \" o lkopf, Christopher J C Burges, and Alexander J Smola (eds.). Advances in kernel methods: support vector learning . MIT Press, Cambridge, MA, USA, 1999. ISBN 0262194163

    1999

  19. [28]

    Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T

    Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains . Advances in Neural Informati...

    2020 arXiv

  20. [29]

    Zippered polygon meshes from Range images

    Greg Turk and Marc Levoy. Zippered polygon meshes from Range images . Proceedings of the 21st Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH 1994, pp.\ 311--318, 7 1994. 10.1145/192161.192241/SUPPL \_ FILE/P311-TURK.PS . URL https://dl.acm.org/doi/...

    1994

  21. [30]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need . In Advances in Neural Information Processing Systems, volume 2017-Decem, pp.\ 5999--6009. Neural information processing sys...

    2017 arXiv

  22. [31]

    Spline Positional Encoding for Learning 3D Implicit Signed Distance Fields

    Peng Shuai Wang, Yang Liu, Yu Qi Yang, and Xin Tong. Spline Positional Encoding for Learning 3D Implicit Signed Distance Fields . IJCAI International Joint Conference on Artificial Intelligence, pp.\ 1091--1097, 6 2021 a . ISSN 10450823. doi:10.24963/ijcai.2021/151. URL https:...

    2021 arXiv

  23. [32]

    When and why PINNs fail to train: A neural tangent kernel perspective

    Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why PINNs fail to train: A neural tangent kernel perspective . Journal of Computational Physics, 449, 7 2020. ISSN 10902716. doi:10.1016/j.jcp.2021.110768. URL https://arxiv.org/abs/2007.14527v1

    2020

  24. [33]

    On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks

    Sifan Wang, Hanwen Wang, and Paris Perdikaris. On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks . Computer Methods in Applied Mechanics and Engineering, 384: 0 113938, 10 2021 b . ISSN 0045-7...

    2021

  25. [34]

    Simoncelli, and Alan C

    Zhou Wang, Eero P. Simoncelli, and Alan C. Bovik. Multi-scale structural similarity for image quality assessment . Conference Record of the Asilomar Conference on Signals, Systems and Computers, 2: 0 1398--1402, 2003. ISSN 10586393. doi:10.1109/ACSSC.2003.1292216

    2003 arXiv

  26. [35]

    Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung)

    Hermann Weyl. Das asymptotische Verteilungsgesetz der Eigenwerte linearer partieller Differentialgleichungen (mit einer Anwendung auf die Theorie der Hohlraumstrahlung) . Mathematische Annalen, 71 0 (4): 0 441--479, 12 1912. ISSN 00255831. doi:10.1007/BF01456804/METRICS. URL h...

    1912 doi

  27. [36]

    Kernel Regression

    John Wolberg. Kernel Regression . In Data Analysis Using the Method of Least Squares, pp.\ 203--238. Springer-Verlag, 2 2006. 10.1007/3-540-31720-1 \_ 7

    2006 doi

  28. [37]

    A Fine-Grained Spectral Perspective on Neural Networks

    Greg Yang and Hadi Salman. A Fine-Grained Spectral Perspective on Neural Networks . 7 2019. URL https://arxiv.org/abs/1907.10599v4

    2019 arXiv

  29. [38]

    pixelNeRF: Neural Radiance Fields from One or Few Images

    Alex Yu, Vickie Ye, Matthew Tancik, and Angjoo Kanazawa. pixelNeRF: Neural Radiance Fields from One or Few Images . Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp.\ 4576--4585, 12 2020. ISSN 10636919. doi:10.1109/CVPR46437.20...

    2020

  30. [39]

    NeRF++: Analyzing and Improving Neural Radiance Fields

    Kai Zhang, Gernot Riegler, Noah Snavely, Cornell Tech, and Vladlen Koltun. NeRF++: Analyzing and Improving Neural Radiance Fields . 10 2020. URL https://arxiv.org/abs/2010.07492v2

    2020 arXiv

  31. [40]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

  32. [41]

    @esa (Ref

    \@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...

  33. [42]

    \@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...

  34. [43]

    !FWPPΝ; OBo Ȝ^ I?j㗼D Mw! B !& dԟ; V p:ڵ z!jT5SO ͖-[ `Æ JW՗ odÆ HV z o΂ HII 0LlݺUYַEppTK ))i[3HnVn݊'`0Lyի/i LxOLz ! B q ٲe ?00@ =G a׮]vΝ;?)Îؿ ?MniB򗿜 d(|KLLdӦM8Nnc=Q

    @open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.