Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights

Alexander Cloninger; Alex Havrilla; Rongjie Lai; Wenjing Liao; Zhaiming Shen

arxiv: 2505.03205 · v3 · pith:UOV3EJSInew · submitted 2025-05-06 · 💻 cs.LG · cs.NA· math.NA· math.ST· stat.TH

Transformers for Learning on Noisy and Task-Level Manifolds: Approximation and Generalization Insights

Zhaiming Shen , Alex Havrilla , Rongjie Lai , Alexander Cloninger , Wenjing Liao This is my paper

Pith reviewed 2026-05-22 15:54 UTC · model grok-4.3

classification 💻 cs.LG cs.NAmath.NAmath.STstat.TH

keywords transformersmanifoldsapproximation errorgeneralization boundsnoisy dataintrinsic dimensionregression

0 comments

The pith

Transformers achieve approximation and generalization errors that scale only with the intrinsic dimension of the task-level manifold despite high-dimensional input noise.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper studies regression where each input is a noisy point near a low-dimensional manifold. The target function is defined to depend solely on the orthogonal projection of that point onto the manifold, which the authors call the task-level manifold. It proves that both the approximation error and the generalization error of a transformer are controlled by the intrinsic dimension of this manifold instead of the ambient dimension of the noise. This supplies a theoretical reason that transformers can still learn from real-world data that carries low-dimensional structure inside high-dimensional perturbations. The argument rests on an explicit construction that lets transformers represent basic arithmetic operations.

Core claim

For regression on points lying in a tubular neighborhood of a manifold, where the ground truth is a function of the orthogonal projection onto the manifold (the task-level manifold), transformers attain approximation and generalization bounds governed by the intrinsic dimension of that manifold rather than the ambient dimension of the noise.

What carries the argument

The task-level manifold, formed by projecting each noisy input orthogonally onto the underlying manifold, which decouples the error bounds from the high ambient dimension via a construction that realizes arithmetic operations inside the transformer architecture.

If this is right

Transformers can maintain low error on tasks whose essential structure lives on a low-dimensional manifold even when every input is corrupted by high-dimensional noise.
Both approximation and generalization errors improve as the intrinsic dimension of the task-level manifold decreases.
The explicit construction of arithmetic operations inside transformers supplies a reusable technique for analyzing other attention-based models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the dimension dependence holds in practice, model design for noisy domains such as sensor arrays or images could benefit from first estimating or enforcing a task manifold.
Empirical tests could check whether measured transformer performance on noisy data correlates with independent estimates of manifold dimension.
The same projection-based analysis might extend to classification or to sequence models where the relevant structure is a manifold in embedding space.

Load-bearing premise

The ground truth function depends only on the projection of each noisy input onto the manifold and not on the noisy point itself.

What would settle it

An experiment in which the target is redefined to depend directly on the full noisy coordinates and the resulting error bounds no longer improve when the estimated intrinsic dimension is lowered.

Figures

Figures reproduced from arXiv: 2505.03205 by Alexander Cloninger, Alex Havrilla, Rongjie Lai, Wenjing Liao, Zhaiming Shen.

**Figure 1.** Figure 1: The tubular region around manifold M and the orthogonal projection πM. around the manifold M with local tube radius given by q ∈ [0, 1) times the local reach (see Definitions 1 and 4). We consider function f : M(q) → R in the form: f(x) = g(πM(x)), ∀x ∈ M(q) (1) where πM(x) = arg min z∈M ∥x − z∥2, (2) is the orthogonal projection onto the manifold M, and g : M → R is an unknown α-H¨older function on the ma… view at source ↗

**Figure 2.** Figure 2: Transformer architecture constructed to approximate ˆf (the purple component implements each of the ˜ηi , the red component approximates 1 ∥η˜∥1 , the yellow component approximates each of the ηi(x), and then approximates ˆf). Definition 12 (Transformer Network) A transformer network T(θ; ·) with weights θ is a composition of an embedding layer, a positional encoding matrix, a sequence transformer blocks,… view at source ↗

**Figure 3.** Figure 3: The covering of tubular region M(q), where each ellipsoid represents the region {x : ˜ηi(x) > 0}. Proposition 1 Suppose the Assumption 1 holds. Let {η˜i(x)} K i=1 be defined as (22). Then for each fixed i, there exists a transformer network T(θ; ·) ∈ T (LT , mT , dembed, ℓ, LFFN, wFFN, R, κ) with parameters LT = O(d), mT = O(D), dembed = 5, ℓ ≥ O(D), LFFN = 6, wFFN = 5, κ = O(D2 δ −8 ) such that T(θ; x) =… view at source ↗

**Figure 4.** Figure 4: Left subplot: Estimated intrinsic dimension (ID) of pixel and embedded image representations with various amounts of isotropic Gaussian noise. Noise added on pixels quickly distorts low-dimensional structures. Embedding with the pre-trained model demonstrates a denoising effect, recovering the original ID at all noise levels. Right subplot: Estimated intrinsic dimension of water buffalo images and embeddin… view at source ↗

read the original abstract

Transformers serve as the foundational architecture for large language and video generation models, such as GPT, BERT, SORA and their successors. Empirical studies have demonstrated that real-world data and learning tasks exhibit low-dimensional structures, along with some noise or measurement error. The performance of transformers tends to depend on the intrinsic dimension of the data/tasks, though theoretical understandings remain largely unexplored for transformers. This work establishes a theoretical foundation by analyzing the performance of transformers for regression tasks involving noisy input data near a manifold. Specifically, the input data are in a tubular neighborhood of a manifold, while the ground truth function depends on the projection of the noisy data onto this manifold, referred to as the task-level manifold. We prove approximation and generalization errors which crucially depend on the intrinsic dimension of the task-level manifold. Our results demonstrate that transformers can leverage low-complexity structures in learning task even when the input data are perturbed by high-dimensional noise. Our novel proof technique constructs representations of basic arithmetic operations by transformers, which may hold independent interest.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper shows transformers achieve approximation and generalization bounds scaling with intrinsic manifold dimension in a noisy tubular neighborhood setting, using a new construction of arithmetic operations inside transformers, but only because the target is defined on the clean projection.

read the letter

Hi colleague, The punchline is that this paper establishes approximation and generalization bounds for transformers on regression tasks where inputs are noisy points near a manifold, but the target function is defined on the projection onto the task-level manifold. The errors scale with the intrinsic dimension rather than the ambient one, thanks to a proof that represents arithmetic operations inside the transformer. This is new in combining the tubular neighborhood noise model with transformer-specific constructions for handling the manifold. It does well by directly addressing why transformers might generalize well on real data that has low effective complexity despite high-dimensional noise. The arithmetic construction could be of independent interest for other approximation results. The soft spot is the modeling choice: everything hinges on the ground truth depending on the clean projection, not the noisy input. As the stress-test points out, if the target used the full noisy point, the bounds would likely include terms from the ambient dimension and noise radius, undermining the dimension reduction. I would need to see the full proofs to confirm there are no hidden ambient factors or loose constants that grow with dimension. The abstract is clear on the setup, but the derivations would need checking for rigor. This work is for people thinking about theoretical foundations of transformers and manifold-based learning. A reader focused on approximation theory or generalization in deep learning would find the explicit bounds and the proof method useful. It should go to peer review. The topic is central, the technique is novel, and the results, if the proofs hold up, would be worth discussing even if revisions clarify the assumptions. Cheers

Referee Report

1 major / 2 minor

Summary. The manuscript analyzes the approximation and generalization performance of transformers on regression tasks where inputs lie in a tubular neighborhood of a manifold but the target function depends only on the orthogonal projection of each input onto the task-level manifold. The authors prove error bounds that depend on the intrinsic dimension of this manifold (rather than ambient dimension) via a novel construction that represents basic arithmetic operations inside the transformer architecture.

Significance. If the derivations hold, the work supplies a concrete theoretical account of why transformers succeed on noisy, high-dimensional data that nevertheless possess low-dimensional task structure. The explicit focus on intrinsic dimension, the parameter-free character of the stated bounds under the given geometric assumptions, and the arithmetic-operation construction (which may be reusable) are clear strengths that advance the literature on transformer expressivity and generalization.

major comments (1)

[Setup and main theorems] Setup and main theorems: the dimension reduction is achieved by defining the ground-truth function to act on the projection onto the task-level manifold. The paper must confirm that the transformer construction recovers or ignores the orthogonal noise component without incurring factors that grow with ambient dimension or tubular radius; otherwise the claimed intrinsic-dimension dependence would not follow from the architecture alone.

minor comments (2)

[Abstract] Abstract: the phrase 'tubular neighborhood' should be accompanied by a brief statement of the required manifold regularity (e.g., C^2 smoothness, compactness) so that readers can immediately assess the scope of the geometric assumptions.
[Notation] Notation: ensure that the projection operator and the noise model are denoted consistently between the problem formulation and the statements of the approximation and generalization theorems.

Simulated Author's Rebuttal

1 responses · 0 unresolved

Thank you for the positive review and recommendation of minor revision. We are pleased that the referee recognizes the strengths of our work on intrinsic dimension dependence for transformers. We address the specific concern raised in the major comment below.

read point-by-point responses

Referee: Setup and main theorems: the dimension reduction is achieved by defining the ground-truth function to act on the projection onto the task-level manifold. The paper must confirm that the transformer construction recovers or ignores the orthogonal noise component without incurring factors that grow with ambient dimension or tubular radius; otherwise the claimed intrinsic-dimension dependence would not follow from the architecture alone.

Authors: We thank the referee for this insightful comment. Our transformer construction, which represents basic arithmetic operations such as addition, multiplication, and division within the attention layers, enables the network to effectively approximate the projection operator onto the task-level manifold. Specifically, by constructing a sequence of operations that compute distances and select the closest point on the manifold (in a discretized sense), the orthogonal noise component is ignored without introducing multiplicative factors depending on the ambient dimension. The bounds in Theorems 3.1 and 4.2 are carefully derived to depend only on the intrinsic dimension d, the tubular radius r (which appears in a controlled way, e.g., as O(r)), and other parameters independent of the ambient dimension D. To make this explicit, we will revise the manuscript to include a new remark or lemma that isolates the noise-handling step and confirms the absence of D-dependent terms in the final error bounds. revision: yes

Circularity Check

1 steps flagged

Error bounds depend on intrinsic dimension by explicit definition of task-level manifold

specific steps

self definitional [Abstract]
"the input data are in a tubular neighborhood of a manifold, while the ground truth function depends on the projection of the noisy data onto this manifold, referred to as the task-level manifold. We prove approximation and generalization errors which crucially depend on the intrinsic dimension of the task-level manifold."

The dependence of the proved errors on intrinsic dimension is a direct consequence of defining the ground truth to act solely on the manifold projection. The result is therefore equivalent to the modeling assumption by construction; the transformer analysis inherits the dimension reduction without deriving it from the architecture or noise model in the absence of this definitional choice.

full rationale

The paper states its core result—that approximation and generalization errors depend crucially on the intrinsic dimension of the task-level manifold—under an explicit modeling choice that the ground-truth function acts only on the orthogonal projection of noisy inputs onto the manifold. This setup is presented as the definition of the learning task in the abstract. The claimed dimension reduction therefore follows directly from the problem formulation rather than from an independent analysis of how transformers process the full noisy inputs. If the target function were instead permitted to depend on the ambient noisy points, the error terms would incorporate ambient dimension and tubular neighborhood effects, altering the stated conclusions. The derivation chain is thus self-contained only because the key geometric assumption is built into the inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The central claim rests on two domain assumptions about data geometry and function dependence that are standard in manifold learning but are specialized here to transformers; no free parameters or new invented entities are introduced in the abstract.

axioms (2)

domain assumption Input data lie in a tubular neighborhood of a manifold
This defines the noisy high-dimensional input model used throughout the analysis.
domain assumption Ground truth function depends only on the projection onto the task-level manifold
This is the key modeling choice that allows error bounds to depend on intrinsic rather than ambient dimension.

pith-pipeline@v0.9.0 · 5731 in / 1337 out tokens · 56603 ms · 2026-05-22T15:54:36.897532+00:00 · methodology

discussion (0)

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods
cs.LG 2025-06 unverdicted novelty 8.0

Transformers perform kernel-based prediction for Hölder regression on manifolds and achieve intrinsic-dimension-dependent minimax rates with sufficient training tasks.
Understanding In-Context Learning for Nonlinear Regression with Transformers: Attention as Featurizer
cs.LG 2026-05 unverdicted novelty 6.0

Transformers can be built to act as nonlinear featurizers via attention, supporting in-context regression with proven generalization bounds on synthetic tasks.
A Mathematical Explanation of Transformers
cs.LG 2025-10 unverdicted novelty 5.0

The Transformer is interpreted as discretization of a structured integro-differential equation in continuous domains for tokens and features, unifying attention, feedforward, and normalization via operator and variati...

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages · cited by 3 Pith papers · 3 internal anchors

[1]

GPT-4 Technical Report

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

work page internal anchor Pith review Pith/arXiv arXiv
[2]

Intrinsic dimen- sionality explains the effectiveness of language model fine-tuning

Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta. Intrinsic dimensionality explains the effec- tiveness of language model fine-tuning. ArXiv, abs/2012.13255,

work page arXiv 2012
[3]

Imagenet: A large-scale hier- archical image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 248–255,

work page 2009
[4]

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805,

work page internal anchor Pith review Pith/arXiv arXiv
[5]

Geburtstag von der Eidgen¨ ossischen Technischen Hochschule Z¨ urich, 1964, pp. 64–79. Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366,

work page 1964
[6]

The kolmogorov superposition theorem can break the curse of dimensionality when approximating high dimensional functions

Ming-Jun Lai and Zhaiming Shen. The kolmogorov superposition theorem can break the curse of dimensionality when approximating high dimensional functions. arXiv preprint arXiv:2112.09963 ,

work page arXiv
[7]

The optimal linear b-splines approximation via kolmogorov super- position theorem and its application

Ming-Jun Lai and Zhaiming Shen. The optimal linear b-splines approximation via kolmogorov super- position theorem and its application. arXiv preprint arXiv:2401.03956 ,

work page arXiv
[8]

Attention is a smoothed cubic spline

Zehua Lai, Lek-Heng Lim, and Yucong Liu. Attention is a smoothed cubic spline. arXiv preprint arXiv:2408.09624,

work page arXiv
[9]

The in- trinsic dimension of images and its impact on learning

Phillip E. Pope, Chen Zhu, Ahmed Abdelkader, Micah Goldblum, and Tom Goldstein. The intrinsic dimension of images and its impact on learning. ArXiv, abs/2104.08894,

work page arXiv
[10]

The shape of learning: Anisotropy and intrinsic dimensions in transformer-based models

Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova, Ivan Oseledets, Denis Dimitrov, and Andrey Kuznetsov. The shape of learning: Anisotropy and intrinsic dimensions in transformer-based models. ArXiv, abs/2311.05928,

work page arXiv
[11]

Deep relu network approximation of functions on a manifold.arXiv preprint arXiv:1908.00695,

Johannes Schmidt-Hieber. Deep relu network approximation of functions on a manifold.arXiv preprint arXiv:1908.00695,

work page arXiv 1908
[12]

Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods

ISSN 1532-4435. Zhaiming Shen, Alexander Hsu, Rongjie Lai, and Wenjing Liao. Understanding in-context learning on structured manifolds: Bridging attention to kernel methods. arXiv preprint arXiv:2506.10959 ,

work page internal anchor Pith review Pith/arXiv arXiv
[13]

such that A(ht) = ( σ(⟨Qdataht, Kdataht2⟩)ei if t = t1, 0 otherwise. Proof. We refer its proof to Lemma 3 in [Havrilla and Liao, 2024]. 2 Remark 2 The significance of the Interaction Lemma is that we can find an attention head such that one token interacts with exactly another token in the embedding matrix. This property facilitates the flexible implement...

work page 2024
[14]

(ht)r1−1 0 (ht)r2+1

Then for any r1 and r2 with 1 ≤ r1 ≤ r2 ≤ dembed − 3 and any k1, k2 with 1 ≤ k1, k2 ≤ ℓ, there exist both two-layer feed-forward networks (FFN) such that FFN1(ht) =    ht if t ∈ {1, · · · , k1}  (ht)1 ... (ht)r1−1 0 (ht)r2+1 ... (ht)dembed−3 I 1 t I 2 t 1   otherwis...

work page 2025
[15]

(ht)r1−1 (ht)r1 − M

Then for any r1, r2 with 1 ≤ r1 ≤ r2 ≤ dembed − 3 and any k1, k2 with 1 ≤ k1, k2 ≤ ℓ and any M > 0, there exists a six-layer residual feed-forward network (FFN) such that FFN(ht) + ht =    ht if t ∈ {1, · · · , k1} ∪ {k2, · · · , ℓ}  (ht)1 ... (ht)r1−1 (ht)r1 − M ... (ht)r2 − M ...

work page 2025
[16]

apply Proposi- tion 1 to implement ˜η1(x), · · · , ˜ηK(x) simultaneously. Let H be an embedding matrix of the form H =   (Hd+7):,I 1 d+7 · · · (Hd+7):,I K d+7 ˜η1(x) · · · ˜ηK(x) ∥˜η(x)∥1 0 0 · · · · · · · · · 0 I((d+3)D+2d+6)K+1 · · · · · · · · · I ℓ 1 · · · · · · · · · 1   . From Theorem 2.2 in [Cloninger and Klock, 2021] , we know K = O(δ−d) wh...

work page 2021

[1] [1]

GPT-4 Technical Report

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

work page internal anchor Pith review Pith/arXiv arXiv

[2] [2]

Intrinsic dimen- sionality explains the effectiveness of language model fine-tuning

Armen Aghajanyan, Luke Zettlemoyer, and Sonal Gupta. Intrinsic dimensionality explains the effec- tiveness of language model fine-tuning. ArXiv, abs/2012.13255,

work page arXiv 2012

[3] [3]

Imagenet: A large-scale hier- archical image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 248–255,

work page 2009

[4] [4]

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805,

work page internal anchor Pith review Pith/arXiv arXiv

[5] [5]

Geburtstag von der Eidgen¨ ossischen Technischen Hochschule Z¨ urich, 1964, pp. 64–79. Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks are universal approximators. Neural Networks, 2(5):359–366,

work page 1964

[6] [6]

The kolmogorov superposition theorem can break the curse of dimensionality when approximating high dimensional functions

Ming-Jun Lai and Zhaiming Shen. The kolmogorov superposition theorem can break the curse of dimensionality when approximating high dimensional functions. arXiv preprint arXiv:2112.09963 ,

work page arXiv

[7] [7]

The optimal linear b-splines approximation via kolmogorov super- position theorem and its application

Ming-Jun Lai and Zhaiming Shen. The optimal linear b-splines approximation via kolmogorov super- position theorem and its application. arXiv preprint arXiv:2401.03956 ,

work page arXiv

[8] [8]

Attention is a smoothed cubic spline

Zehua Lai, Lek-Heng Lim, and Yucong Liu. Attention is a smoothed cubic spline. arXiv preprint arXiv:2408.09624,

work page arXiv

[9] [9]

The in- trinsic dimension of images and its impact on learning

Phillip E. Pope, Chen Zhu, Ahmed Abdelkader, Micah Goldblum, and Tom Goldstein. The intrinsic dimension of images and its impact on learning. ArXiv, abs/2104.08894,

work page arXiv

[10] [10]

The shape of learning: Anisotropy and intrinsic dimensions in transformer-based models

Anton Razzhigaev, Matvey Mikhalchuk, Elizaveta Goncharova, Ivan Oseledets, Denis Dimitrov, and Andrey Kuznetsov. The shape of learning: Anisotropy and intrinsic dimensions in transformer-based models. ArXiv, abs/2311.05928,

work page arXiv

[11] [11]

Deep relu network approximation of functions on a manifold.arXiv preprint arXiv:1908.00695,

Johannes Schmidt-Hieber. Deep relu network approximation of functions on a manifold.arXiv preprint arXiv:1908.00695,

work page arXiv 1908

[12] [12]

Understanding In-Context Learning on Structured Manifolds: Bridging Attention to Kernel Methods

ISSN 1532-4435. Zhaiming Shen, Alexander Hsu, Rongjie Lai, and Wenjing Liao. Understanding in-context learning on structured manifolds: Bridging attention to kernel methods. arXiv preprint arXiv:2506.10959 ,

work page internal anchor Pith review Pith/arXiv arXiv

[13] [13]

such that A(ht) = ( σ(⟨Qdataht, Kdataht2⟩)ei if t = t1, 0 otherwise. Proof. We refer its proof to Lemma 3 in [Havrilla and Liao, 2024]. 2 Remark 2 The significance of the Interaction Lemma is that we can find an attention head such that one token interacts with exactly another token in the embedding matrix. This property facilitates the flexible implement...

work page 2024

[14] [14]

(ht)r1−1 0 (ht)r2+1

Then for any r1 and r2 with 1 ≤ r1 ≤ r2 ≤ dembed − 3 and any k1, k2 with 1 ≤ k1, k2 ≤ ℓ, there exist both two-layer feed-forward networks (FFN) such that FFN1(ht) =    ht if t ∈ {1, · · · , k1}  (ht)1 ... (ht)r1−1 0 (ht)r2+1 ... (ht)dembed−3 I 1 t I 2 t 1   otherwis...

work page 2025

[15] [15]

(ht)r1−1 (ht)r1 − M

Then for any r1, r2 with 1 ≤ r1 ≤ r2 ≤ dembed − 3 and any k1, k2 with 1 ≤ k1, k2 ≤ ℓ and any M > 0, there exists a six-layer residual feed-forward network (FFN) such that FFN(ht) + ht =    ht if t ∈ {1, · · · , k1} ∪ {k2, · · · , ℓ}  (ht)1 ... (ht)r1−1 (ht)r1 − M ... (ht)r2 − M ...

work page 2025

[16] [16]

apply Proposi- tion 1 to implement ˜η1(x), · · · , ˜ηK(x) simultaneously. Let H be an embedding matrix of the form H =   (Hd+7):,I 1 d+7 · · · (Hd+7):,I K d+7 ˜η1(x) · · · ˜ηK(x) ∥˜η(x)∥1 0 0 · · · · · · · · · 0 I((d+3)D+2d+6)K+1 · · · · · · · · · I ℓ 1 · · · · · · · · · 1   . From Theorem 2.2 in [Cloninger and Klock, 2021] , we know K = O(δ−d) wh...

work page 2021