REVIEW 3 major objections 6 minor 15 references
Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This review claims that continuous representation methods—coordinate-to-value functions such as implicit neural representations and functional tensor decompositions—form a coherent paradigm that outperforms discrete grid models on…
desk verdict A useful, mostly accurate survey of continuous representation methods whose opening parameter-efficiency claim rests on a misreading of the universal approximation theorem; worth revising and publishing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the coordinate-to-value function $f$ that maps coordinates $(x,y,z,t,$ view direction, or gene/spectral index$)$ to a measured field value, with discrete data treated as point evaluations of $f$. The machinery that carries the argument includes: Fourier feature and sinusoidal activations that make multilayer perceptrons learn high-frequency detail by shifting the neural tangent kernel toward a diagonal shift-invariant form; multi-resolution hash grids that store learnable features on lattices and interpolate them at arbitrary query points; functional Tucker, CP, and tensor-train decompositions that factor multivariate functions into univariate factor functions; Gaussian-process priors that model continuous temporal trajectories; and neural tangent kernel plus implicit-regularization analyses that explain convergence and generalization. These components are not just catalogued: the review uses each to show how a discrete grid limitation (fixed resolution, local characterization, spectral bias, missing samples) is addressed in the continuous domain.
What would settle it
Take an $n_1 \times n_2 \times n_3$ data table filled with independent random entries and train a two-layer network with fewer parameters than $n_1 n_2 n_3$ to reproduce all entries exactly; if the loss cannot be driven to zero, the lossless parameter-efficiency premise is false in that regime. A direct check of whether the cited universal approximation theorem guarantees exactness would settle it as well.
Extended reading notes
Core claim
The paper's central claim is that discrete data should be modeled not as grids of samples but as evaluations of a continuous function $f$ that maps positional coordinates (spatial, temporal, or view-direction) to observed values, and that this shift is not a single technique but a coherent paradigm. It organizes the field into parametric models (basis-function expansions, implicit neural representations, grid-encoded MLPs), structural modeling (matrix/tensor function decomposition, statistical/Bayesian frameworks, continuous regularization), theoretical foundations (approximation error, neural tangent kernel convergence, implicit regularization), and applications spanning vision and graphics, scientific computing, medical imaging and bioinformatics, and geosciences and remote sensing. The review further argues that the paradigm's advertised advantages—resolution flexibility, cross-modal adaptability, inherent smoothness, and parameter efficiency—are shared across these branches, and that the branches cross-pollinate: INR-based arbitrary-scale super-resolution has the same training paradigm as neural operator learning, tensorial radiance fields resemble Tucker-style functional decompositions, and Fourier basis expansions mirror Fourier-feature positional encodings.
Load-bearing premise
The load-bearing premise is that a small neural network can exactly reproduce any data table with fewer parameters than the table has entries; the theorem cited only promises an approximate fit for a large enough network, not exactness at a smaller size.
Editorial extensions
If this is right
- Arbitrary-scale image super-resolution and neural operator learning can share architectures and training procedures, since both map low-dimensional coordinates plus latent codes to output fields.
- Tensorial radiance fields can inherit error bounds and rank-selection heuristics from functional tensor decomposition theory, not just empirical tuning.
- Fourier-basis functional representations and Fourier-feature implicit neural representations can be analyzed with the same approximation and neural tangent kernel tools, giving a single spectral-bias story.
- Grid-encoded models such as hash grids can be understood from the neural tangent kernel eigenvalue distribution, which predicts when they recover fine detail faster than plain MLPs.
- Implicit-regularization results for deep matrix and tensor factorization carry over to INR-based tensor function decompositions, explaining their robustness in missing-data reconstruction.
Reading between the lines
- The taxonomy implies a concrete transfer test: a technique that fixes spectral bias in implicit neural representations—say, a new activation or encoding—should also improve Fourier-basis functional decompositions and neural operators if the claimed cross-pollination is real.
- If parameter efficiency fails for unstructured data, the review's organizational claims survive but its opening 'inherent superiority' should be read as conditional on smoothness, low-rankness, or other structure in the signal.
- A natural extension not demonstrated in the review is to use continuous regularizations such as neural total variation as priors inside neural operator training for PDEs, which the paper lists as a future direction but does not test.
- The unified framework could serve as a design guide for cross-domain transfer, for example inserting Fourier-feature encodings developed for images directly into spatial-transcriptomics or seismic interpolation models that already use implicit neural representations.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper is a broad survey of continuous representation methods, defined as parametric mappings from coordinates to data values. It organizes the literature into parametric models (basis-function representation, implicit neural representation, grid encoding), structural modeling (matrix/tensor function decomposition, statistical/Bayesian frameworks, continuous regularization), theoretical foundations (approximation theory, NTK-based convergence and generalization, implicit regularization), and applications (vision and graphics, scientific computing, medical imaging and bioinformatics, geosciences and remote sensing). It also proposes future research directions and maintains a GitHub repository of referenced works. The paper's central advertised thesis is that continuous representations are 'novel paradigms' with 'inherent superiority' over discrete grid representations, including parameter efficiency, resolution flexibility, cross-modal adaptability, and inherent smoothness.
Significance. If its organizational claims were established, the survey would be useful: it draws explicit cross-pollinations—arbitrary-scale image super-resolution with operator learning, tensorial radiance fields with functional tensor decomposition, and Fourier basis expansions with positional encoding—and it brings together literature from communities that do not always cite each other. The theory sections accurately restate standard NTK and implicit-regularization results, and the application coverage across vision, graphics, bioinformatics, and remote sensing is broad. The paper also makes its reference list publicly available, which aids reproducibility of the survey itself. However, the main advertised thesis is weakened by an unsupported parameter-efficiency argument in Footnote 1 and by the lack of systematic empirical evidence for the phrase 'inherent superiority' in the abstract.
major comments (3)
- [Section 1, Footnote 1] The footnote asserts that by the universal approximation theorem a shallow depth-two DNN can 'losslessly approximate' the tensor-index function with far fewer parameters than the n1*n2*n3 entries of the tensor. This is not what the universal approximation theorem states. The manuscript's own Theorem 1 in Section 3.1 guarantees only that for every epsilon > 0 there is a single-hidden-layer MLP that is uniformly epsilon-close to a continuous function on a compact set; it gives no exact-representation guarantee and no bound relating the required width to n1*n2*n3. For an arbitrary discrete tensor of N = n1*n2*n3 values, no compression below N parameters is demonstrated. Because the abstract and introduction present parameter efficiency as part of the continuous framework's 'inherent superiority', this unsupported premise is load-bearing for the paper's central thesis. The footnote and abstract should be revised: replace 'losslessly approximate' with 'approximately represent', and either supply a concrete approximation-theoretic bound with explicit dependence on smoothness or rank, or present empirical parameter-count comparisons against storing the tensor.
- [Section 2.4, LRTFR complexity comparison] The text states that LRTFR costs O(mrd(n1+n2+n3) + r n1n2n3) in a forward pass versus O(m^2 d n1n2n3) for a conventional MLP INR and concludes that LRTFR is 'practically more efficient' because r is 'usually much lower than m^2d'. This comparison is against an MLP INR, not against storing the tensor itself, so it does not establish the advertised parameter-efficiency advantage over discrete grid representations. Moreover, 'usually much lower' is a heuristic statement rather than a proved bound. The authors should clarify the baseline, qualify the claim as empirical, or provide supporting experiments or references for the typical range of r relative to m^2d.
- [Abstract and Section 1] The paper asserts that continuous representation methods demonstrate 'inherent superiority' over the traditional discrete framework, listing resolution flexibility, cross-modal adaptability, inherent smoothness, and parameter efficiency. The evidence actually provided consists mainly of illustrative reconstructions and selected figures (e.g., Figures 4, 5, 8, 9) together with favorable statements from cited works, but no systematic benchmark, error bar, or head-to-head comparison across methods. For a survey it is acceptable to present these as potential, task-dependent advantages; as written, the categorical claim overreaches. The authors should temper 'inherent superiority' to 'potential advantages' or provide a systematic comparison with quantitative results.
minor comments (6)
- [Section 2.2] The statement that Fourier-feature positional encoding makes the network 'equivalent to a kernel regression with a diagonal shift-invariant kernel' should be qualified as an infinite-width, limiting statement, and 'diagonal' should be understood as approximately diagonal for finite-width networks.
- [Section 3.1, Theorem 4] In the functional t-SVD theorem, the notation M, the tensor-tensor product *M, and M^{-1}(1) are used before being defined; please introduce the transform M and the t-scalar operations before or immediately after the theorem statement.
- [Section 2.6 and Section 3.3] There are several typographical errors, including 'Expect for' in Section 2.6 (should be 'Except for') and 'dose not' in Section 3.3 (should be 'does not'); a careful proofread is needed.
- [Tables and Figures] Several tables and figures have presentation issues: 'T able 1' appears in the running text, Figure 4's PSNR values lack clear per-row labels, and Figure 6's NTK matrices lack color bars or scales; these should be cleaned up for readability.
- [Section 4.1] The citation 'Lars et al. [109]' for the occupancy network is nonstandard; the paper should cite the work as Mescheder et al., matching the reference list entry.
- [Abstract and GitHub repository] The phrase 'open-source repository' could be misread as implying released code; the repository appears to be a curated reference list, so the wording should be clarified, for example as 'an open-source collection of referenced works'.
Circularity Check
No circularity: the survey derives nothing from itself; Footnote 1's universal-approximation justification is a correctness gap, not a circular step.
full rationale
This paper is a literature review and perspective, not a derivation: it introduces no prediction, no fitted parameter renamed as an output, and no first-principles result that could reduce to its own inputs. The organizing thesis—that continuous representation methods form a coherent family with cross-pollinations between INRs, operator learning, tensor function decomposition, and Fourier/positional encoding—is supported by juxtaposing external prior work, not by fitting or by construction. Self-citations such as LRTFR [103], CRNL [104], NeurTV [105], STINR [106], DRO-TFF [92], and cross-frequency INR [175] appear only as representative descriptions in tables and application sections; the review's comparative claims do not depend on those works being correct, so the self-citations are not load-bearing. One flagged issue, located in Section 1, Footnote 1, is that the paper claims 'a shallow DNN of depth two can losslessly approximate this continuous function, which holds much fewer parameters than the tensor A itself (n1n2n3)' by the universal approximation theorem. This overstates the theorem as correctly recorded in Section 3.1 (Theorem 1), which guarantees only epsilon-close uniform approximation and provides no parameter-count bound relative to the tensor entries. The parameter-efficiency advantage is therefore unsupported as stated. That is a mathematical rigor and internal-consistency concern, not circularity: the claim is neither defined in terms of its conclusion, nor is it a fitted input renamed as a prediction, nor does it rely on a self-citation chain. No circular step can be exhibited by reduction of an equation to its own input, so the circularity score is 0.
Assumptions & free parameters
assumptions (4)
- standard math Universal approximation theorem for multilayer perceptrons
- standard math Neural tangent kernel convergence for infinite-width networks
- standard math Existence and identifiability properties of functional tensor decompositions such as CP, Tucker, and tensor-train
- domain assumption The cited application results are accurately summarized
Cite this review
Pith. "Pith review of Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives." pith.science (2026). https://pith.science/paper/WNXRU6E4
@misc{pith2026250515222,
author = {Pith},
title = {Pith review of: Continuous Representation Methods, Theories, and Applications: An Overview and Perspectives},
year = {2026},
howpublished = {\url{https://pith.science/paper/WNXRU6E4}},
note = {Machine review of arXiv:2505.15222}
}
read the original abstract
Recently, continuous representation methods emerge as novel paradigms that characterize the intrinsic structures of real-world data through function representations that map positional coordinates to their corresponding values in the continuous space. As compared with the traditional discrete framework, the continuous framework demonstrates inherent superiority for data representation and reconstruction (e.g., image restoration, novel view synthesis, and waveform inversion) by offering inherent advantages including resolution flexibility, cross-modal adaptability, inherent smoothness, and parameter efficiency. In this review, we systematically examine recent advancements in continuous representation frameworks, focusing on three aspects: (i) Continuous representation method designs such as basis function representation, statistical modeling, tensor function decomposition, and implicit neural representation; (ii) Theoretical foundations of continuous representations such as approximation error analysis, convergence property, and implicit regularization; (iii) Real-world applications of continuous representations derived from computer vision, graphics, bioinformatics, and remote sensing. Furthermore, we outline future directions and perspectives to inspire exploration and deepen insights to facilitate continuous representation methods, theories, and applications. All referenced works are summarized in our open-source repository: https://github.com/YisiLuo/Continuous-Representation-Zoo
Reference graph
Works this paper leans on
-
[3]
Videoinr: Learning video implicit neural representation for continuous space-time super-resolution
22 Zeyuan Chen, Yinbo Chen, Jingwen Liu, Xingqian Xu, Vidit Goel, Zhangyang Wang, Humphrey Shi, and Xiaolong Wang. Videoinr: Learning video implicit neural representation for continuous space-time super-resolution. In2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2037–2047,
-
[7]
Optimal convergence rates for neural operators.ArXiv:2412.17518,
114 Mike Nguyen and Nicole M¨ ucke. Optimal convergence rates for neural operators.ArXiv:2412.17518,
-
[9]
F-INR: Functional tensor decomposition for implicit neural representations.arXiv:2503.21507,
147 Sai Karthikeya Vemuri, Tim B¨ uchner, and Joachim Denzler. F-INR: Functional tensor decomposition for implicit neural representations.arXiv:2503.21507,
-
[10]
Neural Spatial-Temporal Tensor Representation for Infrared Small Target Detection
158 Fengyi Wu, Simin Liu, Haoan Wang, Bingjie Tao, Junhai Luo, and Zhenming Peng. Neural spatial-temporal tensor repre- sentation for infrared small target detection.ArXiv:2412.17302,
-
[11]
Neural radiance field-based visual rendering: A comprehensive review.arXiv:2404.00714,
170 Mingyuan Yao, Yukang Huo, Yang Ran, Qingbin Tian, Ruifeng Wang, and Haihua Wang. Neural radiance field-based visual rendering: A comprehensive review.arXiv:2404.00714,
-
[13]
176 Hongze Yu, Jeffrey A. Fessler, and Yun Jiang. Bilevel optimized implicit neural representation for scan-specific accelerated mri reconstruction.Arxiv: 2502.21292,
-
[14]
188 Qingtian Zhu, Yumin Zheng, Yuling Sang, Yifan Zhan, Ziyan Zhu, Jun Ding, and Yinqiang Zheng. Suica: Learning super-high dimensional sparse implicit neural representations for spatial transcriptomics.Arxiv: 2412.01124,
-
[15]
189 Veronika A. Zimmer, Kerstin Hammernik, Vasiliki Sideri-Lampretsa, Wenqi Huang, Anna Reithmeir, Daniel Rueckert, and Julia A. Schnabel. Towards generalised neural implicit representations for image registration. InMedical Image Computing and Computer Assisted Intervention (MICCAI), pages 45–55, 2024
work page 2024
Show all 15 references
-
[1994]
Variational seasonal-trend decomposition with sparse continuous-domain regularization.arXiv, 2505.10486,
39 Julien Fageot. Variational seasonal-trend decomposition with sparse continuous-domain regularization.arXiv, 2505.10486,
-
[2017]
Klasky, and Yoram Bresler
71 Berk Iskender, Sushan Nakarmi, Nitin Daphalapurkar, Marc L. Klasky, and Yoram Bresler. Rsr-nf: Neural field regularization by static restoration priors for dynamic imaging.Arxiv: 2503.10015,
-
[2021]
Cross-frequency implicit neural representation with self-evolving parameters.arXiv:2504.10929,
175 Chang Yu, Yisi Luo, Kai Ye, Xile Zhao, and Deyu Meng. Cross-frequency implicit neural representation with self-evolving parameters.arXiv:2504.10929,
-
[2022]
Understanding untrained deep models for inverse problems: Algorithms and theory.Arxiv: 2502.18612,
2 Ismail Alkhouri, Evan Bell, Avrajit Ghosh, Shijun Liang, Rongrong Wang, and Saiprasad Ravishankar. Understanding untrained deep models for inverse problems: Algorithms and theory.Arxiv: 2502.18612,
-
[2023]
Latent functional parafac for modeling multidimensional longitu- dinal data.arXiv, 2410.18696,
137 Lucas Sort, Laurent Le Brusquet, and Arthur Tenenhaus. Latent functional parafac for modeling multidimensional longitu- dinal data.arXiv, 2410.18696,
-
[2024]
Generalized temporal tensor decomposition with rank-revealing latent-ode.Arxiv: 2502.06164,
17 Panqi Chen, Lei Cheng, Jianlong Li, Weichang Li, Weiqing Liu, Jiang Bian, and Shikai Fang. Generalized temporal tensor decomposition with rank-revealing latent-ode.Arxiv: 2502.06164,
-
[2025]
27 Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll
doi=10.1109/TAES.2025.3538571. 27 Julian Chibane, Thiemo Alldieck, and Gerard Pons-Moll. Implicit functions in feature space for 3d shape reconstruction and completion. In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6968–6979,
2025
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.