{"id":"1354b65e-4f6d-462e-902f-dd335931692e","arxiv_id":"2411.09961","paper_version":8,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"Fully connected ReLU networks attain minimax-optimal nonparametric regression rates under temporal-spatial dependence with functional spatial noise, up to logarithmic factors.","lead":"This paper proves convergence rates for fully connected ReLU neural networks when the training data are dependent over time and include spatial random effects. It also derives a lower bound that matches the upper bound up to log factors, and tests the method on simulations and ozone data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma B.2 assumes gamma_i is a separable Gaussian process, but Assumption 3.1e only gives pointwise sub-Gaussian tails; without a sup-norm concentration bound, the truncation event E in Theorem 3.1 is not guaranteed, so the main theorem is unproven as stated.","rationale":"The reader's weakest_assumption correctly identifies the gap between Assumption 3.1e and Lemma B.2's Gaussian-process assumption. This is the most load-bearing concern because the truncation step and the gamma deviation bounds in the proof of Theorem 3.1 both require a uniform sup-norm bound on gamma_i with probability approaching one, and no such bound is derivable from pointwise sub-Gaussianity alone, as the piecewise-constant Gaussian counterexample shows. The lower-bound proof in Lemma 3.2 has an additional flaw: it applies a minimax lower bound for a single block to an estimator that may use all blocks, so the optimality claim is also not established. Both issues are fixable in principle by strengthening Assumption 3.1e to include a Gaussian-process structure or a direct sup-norm tail condition and by repairing the sub-block lower-bound argument, so the appropriate verdict remains CONDITIONAL rather than ACCEPT or REJECT. Since this agrees with the reader's assessment, no change to the verdict is recommended.","tokens_in":62752,"tokens_out":7148,"duration_ms":75846,"concrete_test":"Construct a separable process gamma on [0,1]^d by partitioning the cube into countably many measurable cells and setting gamma(x)=Z_k on cell k, where {Z_k} are independent N(0,sigma^2) with sigma^2 < 1/(2c). Check that sup_x E exp(c gamma(x)^2) = exp(c sigma^2/(1-2c sigma^2)) < infinity and E gamma(x)=0, so Assumption 3.1e holds, while P(||gamma||_infty <= t) = 0 for every finite t because the supremum of an i.i.d. Gaussian sequence is almost surely infinite. If this example is valid, it shows Lemma B.2's sup-norm conclusion cannot follow from Assumption 3.1e alone, and Theorem 3.1 needs either an added Gaussian-process or sup-norm concentration assumption.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The proof of Theorem 3.1 relies on the event E = { |epsilon_ij| <= A_nm/4 for all i,j and ||gamma_i||_infty <= A_nm/4 for all i } to justify the truncation inequality in Step 1 and to bound the gamma empirical-process terms in Step 4. Lemma B.2 is the only place where the required sup-norm control of gamma_i is established, and its proof explicitly assumes that gamma_i is a separable centered Gaussian process on [0,1]^d (Appendix B.1), applying Borell's inequality and Fernique's theorem. Assumption 3.1e, however, only states E gamma_i(x)=0 and sup_x E exp(c gamma_i(x)^2) < infinity, a pointwise sub-Gaussian condition with no sample-path regularity, separability, or Gaussian structure. This assumption does not imply any bound on ||gamma_i||_infty: a separable piecewise-constant process taking independent N(0,sigma^2) values on a countable partition of [0,1]^d satisfies Assumption 3.1e whenever sigma^2 < 1/(2c), yet its supremum over the continuum is almost surely infinite. Consequently, P(E) may be o(1) rather than 1-o(1), and the chain of inequalities leading to the claimed rate in Theorem 3.1 breaks at the point where the response is truncated. The lower-bound Lemma 3.2 also contains an unsupported sub-block argument, but the missing sup-norm control of gamma_i is the primary load-bearing gap because it invalidates the main upper-bound proof under the assumptions as written.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies fully connected ReLU network estimators for the nonparametric temporal-spatial model y_ij = f^*(x_ij) + γ_i(x_ij) + ε_ij, where x_ij are design points in [0,1]^d, ε_ij are sub-Gaussian measurement errors, γ_i are functional spatial noise processes, and the temporal sequences are β-mixing with exponentially decaying coefficients. The main result (Theorem 3.1) claims that the truncated least-squares estimator achieves, with probability approaching one, an L2 error of order (σ_ε^2 + σ_γ^2 + 1) max_{(p,K)∈P} (1/(nm))^{2p/(2p+K)} log^{7.3}(nm) + (σ_ε^2 + σ_γ^2) log(nm)/n, and Lemma 3.2 claims a matching lower bound of order 1/n + max_{(p,K)∈P} (nm)^{-2p/(2p+K)}, so that the rate is minimax up to logarithmic factors. A manifold extension (Theorem 4.1) and extensive numerical comparisons are also provided.","tokens_in":62980,"tokens_out":11889,"duration_ms":125541,"significance":"If fully proven, the results would be a meaningful extension of the approximation-theoretic neural network rates of Kohler and Langer (2021) and Ma and Safikhani (2022) to a setting with functional spatial noise and short-range temporal dependence. The blocking construction for β-mixing sequences and the empirical-process bounds for the γ_i term are potentially reusable. The paper is also transparent about relying on external approximation theorems and does not fit parameters in the theoretical derivation; the simulations are extensive, though no code is provided. However, the main theorem is currently proven only under an unstated Gaussian-process assumption on γ_i, and the lower-bound proof contains a probability error and an insufficiently specified reduction. The central claims therefore need repair before the results can be taken as established.","major_comments":[{"comment":"Assumption 3.1e only imposes Eγ_i(x) = 0 and sup_x E exp(c γ_i(x)^2) < ∞. The proof of the γ_i tail bound in Lemma B.2 instead assumes that γ_i is a separable centered Gaussian process on [0,1]^d and applies Borell's inequality and Fernique's theorem. This is a strictly stronger condition than Assumption 3.1e. A pointwise sub-Gaussian process can have an identically infinite sup norm: take γ_i(x) = Z_{k(x)} with independent N(0,σ^2) variables on a countable partition of [0,1]^d; then sup_x E exp(c γ_i(x)^2) < ∞ for c < 1/(2σ^2), yet sup_{x∈[0,1]^d} |γ_i(x)| = ∞ almost surely. Since the event E = {|ε_ij| ≤ A_nm/4, ||γ_i||_∞ ≤ A_nm/4} is used in Step 1 to justify the truncation inequality and in Step 4 to control the γ_i empirical-process terms, P(E) = 1 - o(1) is not available under Assumption 3.1 as stated, and Theorem 3.1 is not proven. The theorem should either be stated under an added sample-path or Gaussian assumption on γ_i, or the proof needs a genuine sup-norm concentration argument from pointwise sub-Gaussianity plus additional regularity.","section":"Appendix B, Lemma B.2"},{"comment":"The lower-bound proof claims that the ghost-coupling event Ω_x (and similarly Ω_γ, Ω_ε) 'happens with probability at least 1/n'. The union-bound argument gives P(Ω_x) ≥ 1 - n β_x(S) ≥ 1 - 1/n. The subsequent step P({||ef - f^*||^2 ≥ C/n} ∩ Ω) > C0 requires P(Ω) → 1; with only P(Ω) ≥ 1/n, the displayed inequality is not justified. This is a load-bearing error in the proof of Lemma 3.2 and Lemma G.1, and it should be corrected to 1 - o(1) or the argument restructured.","section":"Appendix G, Lemma 3.2 and Lemma G.1"},{"comment":"The reduction to a scalar Gaussian location problem is not made rigorous. The text asserts that 'for each time i only one observation among the m_i available observations contributes' and then uses constant vectors Z_{i,l} = (z,...,z) ∈ R^{m_i} with z ∼ N(μ_l,1), but no pair of hypotheses in the hierarchical composition class H(l,P) and no spatial process γ_i satisfying Assumption 3.1 is constructed to realize this likelihood. Unless γ_i is taken to be constant in x and f^* is taken constant in the sub-problem, which is not stated, the observations y_ij = f^*(x_ij) + γ_i(x_ij) are not i.i.d. copies of a scalar N(μ_l,1) variable, and the per-block lower bound ~1/|J_e,s| does not follow from Le Cam's lemma as written. Since Lemma 3.2 is the basis for the claimed minimax optimality of the n^{-1} term, this part needs a complete construction.","section":"Appendix G, model (60)-(62)"}],"minor_comments":[{"comment":"The displayed rate in Theorem 3.1 omits the σ_γ^2/n term that appears in the proof's final display and in the introductory display (3); because σ_γ^2 log(nm)/n dominates, this is harmless, but the two displays should be reconciled.","section":"Section 3.1, Eq. (9)"},{"comment":"The displayed equation labeled (18) inside the lemma statement appears to be a leftover from equation (17) and is never used; it should be removed.","section":"Appendix B, Lemma B.2"},{"comment":"The gamma empirical-process inequality is cited as 'Theorem C.17', but the relevant statement appears to be Corollary C.10, with Lemma C.17 used for the norm coupling; the cross-references should be corrected.","section":"Proof of Theorem 3.1, Step 4"}],"recommendation":"major_revision","confidential_remarks":"The main issues are fixable in a revision: add the Gaussian or sup-norm assumption to the main theorem, repair the lower-bound probability calculation, and make the lower-bound construction explicit. I would not reject on the basis of disagreement with existing theory, since the paper's claim is an extension that could be correct under repaired assumptions. The authors should also verify that the n^{-1} lower bound is proven for the same γ_i class used in the upper bound, and clarify the relation to the existing functional regression literature, since Corollary 3.3 frames the independent case as functional regression."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: this is a serious attempt at the first rates for fully connected ReLU nets under beta-mixing temporal dependence plus functional spatial noise, with manifold inputs included. The proof machinery is real: blocking with ghost variables, the empirical-vs-L2 coupling, and the two-part rate 1/n + (nm)^(-2p/(2p+K)) are coherent, and the special cases recover Kohler-Langer and Ma-Safikhani type bounds. Credit where due: if the gap below is repaired, this is a meaningful extension.\n\nThe soft spot is load-bearing. Lemma B.2 supplies the sup-norm control of gamma_i used to truncate the responses and to pass from the empirical to the L2 bound. Its proof assumes each gamma_i is a separable centered Gaussian process and applies Borell and Fernique. Assumption 3.1e only gives pointwise zero mean and sup_x E exp(c gamma_i(x)^2) < infinity. That is not enough: a piecewise-constant process taking independent N(0,sigma^2) values on a countable partition satisfies 3.1e yet has infinite sup norm almost surely. So Theorem 3.1 is not proven under the assumptions as written. This is fixable, either by adding a Gaussian-process or sample-path regularity assumption or by proving a separate sup-norm concentration bound, but it has to be fixed before the main result stands.\n\nThe lower-bound appendix has a second, smaller issue. The coupling event Omega is assigned probability at least 1/n; for the subsequent intersection argument to survive the limsup, the event needs probability 1-o(1). The claim that f-hat restricted to a ghost block is an estimator for that block also needs more care, since f-hat sees all blocks. These look repairable, but the optimality claim is not yet established.\n\nThe experiments are a minor concern: the abstract promises consistent outperformance, some tables do not show that, and there is no code or training protocol beyond the hyperparameter tables. I would soften those claims.\n\nWho is this for? Statistical learning theorists working on dependence and neural network rates. The rate is plausible and the paper would be a useful reference once the assumptions are tightened and the lower-bound event probability is corrected. As it stands, I would not cite the main theorem as proven. Recommendation: send it to a serious referee. The core contribution is real, the gap is specific and identifiable, and the authors have a reasonable path to fix it.","headline":"Plausible first rates for dense ReLU nets with temporal-spatial dependence and spatial noise, but the main theorem is proven only under an unstated Gaussian-process assumption on gamma_i.","tokens_in":63637,"tokens_out":1712,"would_cite":false,"duration_ms":20329,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G08","62M10","62M30","68T07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proves that fully connected ReLU networks attain minimax-optimal rates, up to logarithmic factors, for nonparametric regression under temporal dependence and function-valued spatial noise.","keywords":["nonparametric regression","deep ReLU networks","beta-mixing dependence","functional spatial noise","hierarchical composition models","manifold learning","minimax rates","temporal-spatial data"],"falsifier":"Construct a centered random field $\\gamma$ on $[0,1]^d$ satisfying $\\sup_x E\\exp(c\\gamma(x)^2)<\\infty$ and $E\\gamma(x)=0$ for all $x$, but with $\\sup_x |\\gamma(x)|$ tail heavier than $\\exp(-Ct^2)$, for example by layering many independent sub-Gaussian components so the supremum grows with the number of components; then check whether $P(\\max_{1\\le i\\le n}\\sup_x|\\gamma_i(x)|>A_{nm})$ stays bounded away from zero. If yes, Lemma B.2 fails and the uniform event in the proof of Theorem 3.1 is not guaranteed, so the rate bound as stated does not follow.","tokens_in":62449,"feed_emoji":"📈","tokens_out":16533,"duration_ms":150441,"temperature":0.7,"pith_summary":"This paper proves a convergence-rate guarantee for fully connected ReLU networks used as nonparametric regressors when the data are correlated in time and corrupted by function-valued spatial noise. The model is $y_{ij}=f^*(x_{ij})+\\gamma_i(x_{ij})+\\epsilon_{ij}$, with time indexed by $i$; the time series is allowed to be $\\beta$-mixing while $\\gamma_i$ is a spatial noise function. The main result is that a truncated least-squares network estimator achieves squared $L_2$ error of order $\\max_{(p,K)\\in\\mathcal P}(nm)^{-2p/(2p+K)}\\log^{7.3}(nm)+\\log(nm)/n$ for target functions in a hierarchical composition class, and a matching lower bound shows this is minimax optimal up to logarithmic factors. This matters because it extends neural-network regression theory from independent or time-only data to spatial-temporal data, and it shows the effective complexity is the structure of the target function rather than the ambient dimension. When the inputs lie on a $d^*$-dimensional manifold, the ambient dimension is replaced by $d^*$, recovering known manifold rates when spatial noise is absent.","feed_headline":"Dense ReLU networks reach minimax rates on space-time data","feed_subtitle":"New bounds handle time dependence and spatial noise; up to logs, the rate matches the lower bound.","key_machinery":"The load-bearing device is a coupling inequality (Theorem C.17) that controls the difference between the empirical norm over the observed design and the population $L_2$ norm under $\\beta$-mixing dependence. A blocking scheme cuts the $n$ time indices into alternating blocks of length $S\\asymp \\log n$, produces independent 'ghost' copies of each block via a standard $\\beta$-mixing coupling theorem, and then applies sub-Gaussian concentration and Rademacher/covering-number bounds inside the blocks; the cost of the coupling is paid by the mixing coefficient $\\beta(S)$. The approximation part of the proof rests on the ReLU-network approximation theorem for hierarchical composition models (Lemma B.1), which yields the rate-determining factor $\\phi_{nm}=\\max_{(p,K)\\in\\mathcal P}(nm)^{-2p/(2p+K)}$, and the truncation at $A_{nm}\\asymp\\max\\{\\sigma_\\epsilon,\\sigma_\\gamma\\}\\log^{1/2}(nm)$ makes the response bounded on a high-probability event.","core_discovery":"The central claim is that dense ReLU networks can estimate $f^*$ from $n$ weakly dependent time blocks with $m$ spatial locations each at essentially the same rate as if all $nm$ points were independent, except that the spatial noise contributes an unavoidable $n^{-1}$ term. Theorem 3.1 states that with probability approaching one, $\\lVert \\widehat f_{A_{nm}}-f^*\\rVert_{L_2}^2$ is bounded by $(\\sigma_\\epsilon^2+\\sigma_\\gamma^2+1)\\max_{(p,K)\\in\\mathcal P}(nm)^{-2p/(2p+K)}\\log^{7.3}(nm)+(\\sigma_\\epsilon^2+\\sigma_\\gamma^2)\\log(nm)/n$ for $f^*\\in \\mathcal H(l,\\mathcal P)$. Lemma 3.2 supplies the matching lower bound $C(1/n + \\max_{(p,K)\\in\\mathcal P}(nm)^{-2p/(2p+K)})$, so up to logarithms the estimator is minimax optimal in the hierarchical composition class. In the manifold setting, Theorem 4.1 gives $(nm)^{-2p/(2p+d^*)}$ plus $\\log(nm)/n$, with $d^*$ the intrinsic dimension, and the corollaries recover the time-series-only rate and the manifold rate when the spatial noise is dropped.","pith_inferences":["Implicit in the proof but not stated as a theorem: the blocking-plus-coupling scheme is modular, so the same rates should transfer to other weak-dependence notions as long as the mixing coefficients decay fast enough; the polynomial-decay result is one instance.","A reader can test whether the $\\log^{7.3}(nm)$ exponent is a proof artifact: the paper gives no matching logarithmic lower bound, so sharpening the chaining argument or proving a log lower bound would settle the question.","The main technical gap is not in the rate calculation: Lemma B.2 obtains the uniform spatial-noise bound with Gaussian-process tools, while Assumption 3.1(e) is only pointwise exponential-square integrability, so a complete proof as written seems to require an added Gaussian-process assumption or a separate sup-norm concentration argument."],"forward_implications":["The guarantee holds for both wide networks (logarithmic depth, growing width) and deep networks (growing depth, bounded width), so the rate is not tied to one architecture.","Lemma 3.2 implies that the $n^{-1}$ component in the bound is unavoidable when spatial noise is present: no estimator can average away the function-valued spatial noise faster than $1/n$ over time.","When the inputs lie on a $d^*$-dimensional Lipschitz manifold, the rate uses $d^*$ instead of the ambient dimension $d$, so the method avoids the curse of dimensionality for low-dimensional structure.","For polynomial mixing decay, the paper derives a rate of $(\\sigma_\\epsilon^2+\\sigma_\\gamma^2+1)n^{-\\alpha}\\log^{6.3}(nm)+\\sigma_\\gamma^2/n$ when $m$ is large enough, showing the technique extends beyond exponential mixing."],"supporting_citations":[{"why":"supplies the $\\beta$-mixing coupling theorem used to replace dependent blocks by independent ghost copies.","marker":"Doukhan (2012)"},{"why":"provides the ReLU-network approximation theorem for hierarchical composition classes that yields the approximation error $\\phi_{nm}$.","marker":"Kohler and Langer (2021)"},{"why":"gives the minimax lower-bound framework for hierarchical composition models that Lemma 3.2 extends to spatial noise.","marker":"Schmidt-Hieber (2020)"},{"why":"the previous temporal-dependence-only neural network rate that this paper broadens to include spatial noise.","marker":"Ma and Safikhani (2022)"},{"why":"supplies the manifold approximation result whose rate Theorem 4.1 recovers when spatial noise is absent.","marker":"Kohler et al. (2023)"},{"why":"provides VC-dimension bounds for piecewise-linear networks used to derive the covering-number bound for the network class.","marker":"Bartlett et al. (2019)"},{"why":"supplies the covering and packing lemmas that convert VC-dimension bounds into entropy bounds for the empirical norm.","marker":"Györfi et al. (2002)"},{"why":"supplies Borell's inequality and Fernique's theorem, the Gaussian-process tools behind the uniform spatial-noise bound in Lemma B.2.","marker":"Kühn and Schilling (2023)"}],"fun_headline_variants":["Dense ReLU nets attain minimax rates for space-time data","Minimax convergence for spatio-temporal learning with ReLU nets","ReLU networks match minimax bounds on space-time measurements","Space-time ReLU nets: near-optimal rates under dependence","Manifold-based ReLU nets get minimax rates for space-time data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof needs a high-probability bound saying that the spatial noise $\\gamma_i$ never exceeds a growing threshold at any location, for every time point at once; the stated assumption only controls the tail of $\\gamma_i(x)$ at each fixed $x$, and without an added Gaussian-process condition or a separate supremum-concentration argument that uniform control is not guaranteed.","fun_headline_variants_meta":{"raw":{"variants":["Dense ReLU nets attain minimax rates for space-time data","Minimax convergence for spatio-temporal learning with ReLU nets","ReLU networks match minimax bounds on space-time measurements","Space-time ReLU nets: near-optimal rates under dependence","Manifold-based ReLU nets get minimax rates for space-time data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000919,"raw_usage":{"total_tokens":3968,"prompt_tokens":993,"completion_tokens":2975,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":609,"completion_tokens_details":{"reasoning_tokens":2885}},"tokens_in":609,"tokens_out":2975,"duration_ms":24049,"temperature":1.0,"reasoning_tokens":2885,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:07:50.931127+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a centered random field $\\gamma$ on $[0,1]^d$ satisfying $\\sup_x E\\exp(c\\gamma(x)^2)<\\infty$ and $E\\gamma(x)=0$ for all $x$, but with $\\sup_x |\\gamma(x)|$ tail heavier than $\\exp(-Ct^2)$, for example by layering many independent sub-Gaussian components so the supremum grows with the number of components; then check whether $P(\\max_{1\\le i\\le n}\\sup_x|\\gamma_i(x)|>A_{nm})$ stays bounded away from zero. If yes, Lemma B.2 fails and the uniform event in the proof of Theorem 3.1 is not guaranteed, so the rate bound as stated does not follow.","supporting_citations":[{"cited_title":"Mixing: properties and examples, volume 85","cited_arxiv_id":null,"evidence_quote":"supplies the $\\beta$-mixing coupling theorem used to replace dependent blocks by independent ghost copies."},{"cited_title":"Estimation of a regression function on a manifold by fully connected deep neural networks","cited_arxiv_id":null,"evidence_quote":"supplies the manifold approximation result whose rate Theorem 4.1 recovers when spatial noise is absent."},{"cited_title":"Maximal inequalities and some applications","cited_arxiv_id":null,"evidence_quote":"supplies Borell's inequality and Fernique's theorem, the Gaussian-process tools behind the uniform spatial-noise bound in Lemma B.2."}],"review_version":1}