REVIEW 5 major objections 5 minor 10 references
This paper argues that interleaving projections or exponential-map updates between neural layers keeps every intermediate state on a prescribed manifold with boundary, and that the resulting approximation error can be controlled by explicit
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 05:06 UTC pith:M7IZ4Q2N
load-bearing objection A promising framework with several theorems that don't yet match their proofs and an overclaimed learned-projection story; fixable, but not ready as written. the 5 major comments →
Geometry-Preserving Neural Architectures on Manifolds with Boundary
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On the paper's own terms: projected neural ODEs, where the learnable vector field is projected onto the tangent cone of a prox-regular set at every instant, keep the set invariant by construction and can approximate any sufficiently regular target flow with error bounded by δ/Lα(e^{LαT} - 1). Geometry-preserving architectures built by projecting after each residual layer inherit this guarantee. Enforcing constraints only at the output costs at most a factor of two in approximation error, and exponential-map output layers achieve universal approximation in L2 on geodesically complete manifolds. When no analytic projection exists, a reverse-time flow-matching field learned from heat-kernel-smo
What carries the argument
The central device is the projected dynamical system ẋ = P_{T_M(x)} F(t, x), in which the velocity is projected onto the tangent cone at the current point; its Euler discretization is the interleaved projection layer, and its intrinsic discretization is the Riemannian exponential update. The theory rests on uniformly prox-regular sets, meaning sets with positive reach so that every point within some distance has a unique nearest point on the set; the associated normal-cone inequality supplies the differential-inequality argument in the error bound. For the learned projection, small-time heat-kernel asymptotics make the score of the smoothed density point toward the metric projection, and the
Load-bearing premise
The learned projection that stands in for an analytic projection is guaranteed to be a metric projection only when the constraint set is a compact smooth manifold without boundary and the training data covers the whole manifold according to its Riemannian measure; the paper still applies it to empirical distributions over narrow subsets of the constraint set, where that guarantee does not hold.
What would settle it
Train the flow-matching projector on a narrow empirical subset of SO(3), such as rotations near the identity, and test on rotations far from that subset; if the learned map does not move noisy inputs to their nearest rotation, the general learned-projection claim fails. Alternatively, on a full-measure sphere dataset, compute ∥x + t∇log u_t(x) - P_M(x)∥ as t→0; if it does not decay as O(t^(1/2)), Theorem 2.5's rate is contradicted.
If this is right
- Projected and exponential intermediate-layer architectures give exact feasibility on manifolds with boundary, not just in the limit, for every intermediate and final layer.
- If a neural vector field approximates the target in sup-norm to δ, the endpoint approximation error is bounded by δ/Lα(e^{LαT} - 1), so fidelity degrades at worst exponentially in time and inversely with the reach of the constraint set.
- Final-only projection is a safe simplification: it at most doubles the ambient approximation error, and it is computationally cheaper in the paper's experiments.
- On any geodesically complete Riemannian manifold, a fixed base-point exponential map combined with an L2-close network can approximate any continuous target, avoiding topological obstructions that block uniform approximation.
Where Pith is reading between the lines
- The learned-projection guarantee is proven for compact boundaryless submanifolds with training samples drawn from the full Riemannian measure; applying the flow-matching projector to datasets such as protein backbones on SE(3), where samples cover a narrow empirical subset, extends the claim beyond its proven regime.
- Because the learned projection is accurate only for inputs within the reach and for small diffusion time, networks whose intermediate states drift far from the manifold may not be corrected; the paper's own experiments show flow-matching variants did not match analytic projections.
- A testable extension is to train the flow-matching projector with a time schedule that adapts to local reach rather than a fixed small diffusion time, which could make learned projections viable on empirical manifold distributions.
- The O(t^(1/2)) rate suggests a practical diagnostic: on held-out noisy inputs, measure ∥x + t∇log u_t(x) - P_M(x)∥; if it does not shrink like t^(1/2), the learned score has not converged to the metric projection.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a unified framework for geometry-preserving neural architectures, organized by whether constraints are enforced at intermediate layers (IAA) or only at the output (FAA). The main theoretical claims are: (i) projected neural ODEs preserve a prescribed prox-regular set M and approximate target flows with an error exponential in the horizon (Theorem 2.2); (ii) final projection after a δ-close unconstrained approximator loses at most 2ε (Theorem 2.3); (iii) final exponential-map architectures give L2 universal approximation on geodesically complete manifolds (Theorem 2.4); and (iv) small-time heat-kernel/flow-matching scores approximate the metric projection up to O(t^{1/2}) (Theorem 2.5). The paper also contains an empirical comparison of IAA and FAA variants on S^2, the disk, SO(3), Cucker–Smale dynamics, and protein backbone data on SE(3), including learned projections via flow matching.
Significance. If the results were correct as stated, the paper would fill a genuine gap: it unifies projection-based and exponential-map architectures under projected dynamical systems, and it gives quantitative approximation statements for non-convex, prox-regular constraint sets. The final-exponential-map universality theorem (Theorem 2.4), which uses measurable selections and an L2 relaxation to bypass topological obstructions, is a nice contribution. The experimental comparison of IAA versus FAA is also useful, and the paper provides unusually detailed implementation appendices and reproducibility information. However, several load-bearing statements are currently not supported as written: an architecture equation that does not preserve M, a theorem whose proof does not match its statement, an ill-posed final-projection theorem, and a learned-projection theorem that is both misstated and inapplicable to the experiments. These issues are local and fixable in principle, so the paper merits major revision rather than rejection.
major comments (5)
- [Section 2.1, Eq. (6)] Equation (6) defines the projected IAA update as h_{ℓ+1}=h_ℓ+Δt P_M(f_θ(ℓ,h_ℓ)). If P_M is the metric projection onto M, this update does not keep states in M: h_ℓ is in M but adding a projected tangent/ambient vector moves it off M. The correct update, which is actually implemented in Appendix E.3, is h_{ℓ+1}=P_M(h_ℓ+Δt f_θ(ℓ,h_ℓ)). As written, the exact-feasibility claim for the projected IAA is unsupported. A related typo appears in Eq. (5), where P_{TM(x(t))} uses an undefined x(t) and should be P_{TM(y(t))}.
- [Section B.1, Theorem 2.2] The theorem states the bound sup ‖Φ_F−Φ_fθ‖ ≤ δ/L_α(e^{L_α T}−1) with L_α=L+4U^2/α+1, but the proof does not derive this inequality. The proof obtains d/dt ‖e(t)‖² ≤ a‖e(t)‖²+δ² with a=2L+2U^2/α+1, and concludes ‖e(T)‖² ≤ δ²/a(e^{aT}−1), i.e. ‖e(T)‖ ≤ δ sqrt((e^{aT}−1)/a). This is a different bound with a different exponent and prefactor; the final displayed formula in the proof does not match the theorem statement. The authors should either prove a Lipschitz estimate for the projected vector field that yields the stated bound, or correct the theorem to match the Gronwall argument in the proof.
- [Section 2.2, Theorem 2.3] The statement is ill-posed as written. F is assumed to be a continuous map F:M→M, but the hypothesis ∥F(x)−f_θ(x)∥≤ε is quantified over all x∈Ω, where Ω is a neighborhood of M that is larger than M. For x∈Ω\M, F(x) is undefined. The proof also uses F(x)∈M for arbitrary x∈Ω. The theorem can likely be repaired by assuming F:Ω→M, or by assuming F is defined on M and using an extension or a distance-to-M argument, but as stated the theorem does not have a well-defined content.
- [Section 2.3, Theorem 2.5] There are three independent problems. First, the displayed gradient bound is misstated: the proof yields ∇_x log u_t(x)+(x−P_M(x))/t = O(t^{-1/2}), so a uniform bound ≤C_K independent of t cannot hold for all sufficiently small t. What follows from the proof is O(t^{1/2}) for x+t∇_x log u_t(x)−P_M(x), not for the gradient error. Second, the hypotheses require M to be compact, boundaryless, and μ to be the induced Riemannian measure on all of M. The experiments use the disk (which has boundary) and empirical/narrow-support distributions on SO(3), SE(3), and protein data; for empirical μ the small-time limit of x+t∇_x log u_t(x) is controlled by the nearest training point, not by P_M(x). No limitation is stated. Third, Algorithm 1 integrates a learned reverse-time ODE over [0,T], whereas Theorem 2.5 analyzes only the single-step map x+t∇_x log u_t(x). The learned-projection claim therefore
- [Section 2.3, Algorithm 1] The paper claims that the flow-matching conditional velocity satisfies v(t,x)=∇_x log μ_t(x), but this is false for the generative process defined in Algorithm 1, Y_t=X+v t with v∼N(m,I). For m=0 and a point mass at X=0, Y_t=tV with V∼N(0,I), so μ_t=N(0,t²I), v(t,x)=E[V|Y_t=x]=x/t, while ∇_x log μ_t(x)=−x/t². These are not equal; the general relation for this path is E[V|Y_t=x]=−t∇_x log μ_t(x) (up to the mean m). Consequently, training v_θ by flow matching does not estimate the score appearing in Theorem 2.5. The connection between the learned velocity and the heat-kernel projection estimate needs to be re-derived or the algorithm and theorem must be matched.
minor comments (5)
- [Abstract and Introduction] There are several typos: “we also the demonstrate” in the abstract; “Let M⊆R^d be a be a closed subset” in Section 2; Figure 1 caption says “Final Layer Augmented (IAA)” where it should be FAA. These should be corrected.
- [Section 2.1, preliminaries] The text states that “C² manifolds have uniformly positive reach.” This is not true without compactness or a uniform tubular-neighborhood bound; a noncompact C² submanifold can have reach 0. The statement should be restricted to compact or otherwise bounded-geometry submanifolds.
- [Notation, Table of symbols] The notation defines P_A(x) as argmin_y ∥x−y∥ for all x, but the metric projection is set-valued in general and is only single-valued inside the reach. The later theorems assume the single-valued regime; the notation table should acknowledge this distinction.
- [Section 2.2, Theorem 2.4] The theorem introduces a Borel measure μ on M, but the L² norm is taken over Ω⊂R^d. The measure should be specified as a measure on Ω, or the text should clarify how μ is extended. Also the sentence “the corresponding constrained approximation is defined by composition with the projection map” in the exponential FAA paragraph should say “exponential map.”
- [Appendix D.6] The learned-projection evaluation integrates the backward ODE with RK45 from t=T to t=0, while the training uses a time grid up to T. The relation between the integration horizon and the small-time regime of Theorem 2.5 is not discussed; this is relevant to the theoretical gap identified above.
Circularity Check
No circular derivation: theorems are self-contained analytic estimates; self-citations are contextual only.
full rationale
The central claims are Theorem 2.2 through Theorem 2.5. Each is proved from explicitly stated assumptions (prox-regularity and reach, Lipschitzness and sup-norm closeness, geodesic completeness, compact boundaryless submanifold with Riemannian measure) using standard external results (Gronwall, Hopf-Rinow, measurable selection, Laplace/heat-kernel asymptotics). No theorem fits a parameter to data and then reports that parameter as a prediction; the only learned object, the flow-matching velocity field, is an empirical algorithm, and Theorem 2.5 is an independent asymptotic statement about the heat kernel. The paper's self-citations, e.g., Elamvazhuthi et al. 2023, appear only in survey or comparison sentences and are not premises of any proof. The learned-projection section has a theorem-to-algorithm gap: Theorem 2.5 concerns the one-step map x+t grad log u_t(x) on a compact boundaryless manifold with full Riemannian measure, while Algorithm 1 integrates a backward ODE and is applied to empirical, sometimes boundary or narrow-support distributions. That is an applicability/support problem, not a definitional or statistical circularity. No equation in the paper reduces a claimed prediction to its own input.
Axiom & Free-Parameter Ledger
free parameters (3)
- discretization step Δt
- flow-matching time horizon T =
T = 2α·median ||x|| with α∈{0.25,0.5,1.0}
- flow-matching noise velocity distribution =
v_n = 0.5·u/||u||, u∼N(0,I)
axioms (5)
- domain assumption M is uniformly prox-regular with positive reach α (Eq. 3)
- domain assumption Vector fields F and fθ are L-Lipschitz on M and uniformly bounded by U (Assumption 2.1)
- standard math Admissible neural vector fields can uniformly approximate Lipschitz vector fields on compact M
- standard math Geodesic completeness, Hopf-Rinow theorem, and measurable selection (Theorem 2.4)
- ad hoc to paper Training data distribution μ approximates the Riemannian measure on M for learned projections
read the original abstract
A growing number of neural architectures have been proposed to enforce geometric constraints, including projection-based networks, exponential-map updates, constrained output layers, and manifold neural ODEs. We provide a unified framework for these geometry-preserving architectures by organizing them according to where and how constraints are enforced, either throughout the intermediate layers or only at the final output. This perspective reveals several gaps in the existing theory. To address these gaps, we prove high-level approximation theorems for projected neural ODEs, intermediate augmented architectures, and final augmented architectures on prox-regular constraint sets, including smooth manifolds with boundary. Numerical experiments on synthetic dynamics over S^2, the disk, SO(3), together with real-world protein backbone data on SE(3), demonstrate exact feasibility for analytic updates and show that the final augmentation have simpler architecture and outperform in most tasks considered. When the constraint set is unknown, we learn projections via small-time heat-kernel limits, showing diffusion/flow-matching can be used as data-based projections. Moreover, we also the demonstrate the usefulness of the architectures that enforce non-convex constraints for path planning on manifolds with boundary.
Figures
Reference graph
Works this paper leans on
-
[1]
Tertiary structure
The vector fields satisfy:∥F−f θ∥∞ ≤δ, i.e., uniformly close in sup-norm, 2.∥F∥ ∞ ≤Uand∥f θ∥∞ ≤Ufor someU <∞. Then, withL α :=L+ 4U 2 α + 1, whereαis the reach ofM, one has sup x0∈M ∥ΦF (x0)−Φ fθ (x0)∥ ≤ δ Lα eLαT −1 . Proof.Fixx 0 ∈Mand setx(t) :=x(t;x 0),y(t) :=y(t;x 0), ande(t) :=x(t)−y(t). For a.e.t, define ξx(t) :=F(t, x(t))−P TM (x(t)) (F(t, x(t))),...
2021
-
[2]
Normalize and scale: vn = 0.5· un ∥un∥2 + 10−8
-
[3]
Construct a uniform time grid of30points on[0, T]
Choose the time horizonTeither as a fixed value or viaauto: T= 2α·median n∥xn∥2, α∈ {0.25,0.5,1.0}, computed on the training split. Construct a uniform time grid of30points on[0, T]
-
[4]
Sample i.i.d.u n ∼ N(0, Id)
-
[7]
distance to manifold
Formy n,k =x n +v ntk and flatten pairs to inputsz n,k = [yn,k;t k]∈R d+1 and targetsv n ∈R d (repeated acrossk). This yields (30N) pairs. The validation synthetic set is constructed once from X val and held fixed. The training synthetic set is regenerated once per epoch fromX train. D.4 Training objective and optimization We minimize mean-squared error b...
2000
-
[8]
For each hyperparameter configuration (depth, weight decay, and any model-specific knobs), train to completion and record the best validation loss and corresponding checkpoint
-
[9]
Among configurations within a model family, pick the configuration with the lowest best-validation loss
-
[10]
This selection is performed separately for each dataset and each model family, ensuring that test results are obtained from models chosen without test-set feedback
Evaluate that selected checkpoint on the held-out test set and record test metrics. This selection is performed separately for each dataset and each model family, ensuring that test results are obtained from models chosen without test-set feedback. E.8 Test metrics We report two metrics per dataset–model pairing. Prediction error (MSE).Letˆx T =f θ(x0)den...
-
[2021]
URL https://api.semanticscholar. org/CorpusID:235959867. 9 Geometry-Preserving Neural Architectures on Manifolds with Boundary Katsman, I., Chen, E., Holalkere, S., Asch, A., Lou, A., Lim, S. N., and De Sa, C. M. Riemannian residual neural networks.Advances in Neural Information Processing Systems, 36:63502–63514, 2023. Kratsios, A. and Papon, L. Universa...
-
[2024]
URL https://proceedings.mlr.press/ v235/permenter24a.html. Ramachandran, G. N. Stereochemistry of polypeptide chain configurations.J. Mol. Biol., 7:95–99, 1963. Scaramuzza, D. and Fraundorfer, F. Visual odometry [tu- torial].IEEE robotics & automation magazine, 18(4): 80–92, 2011. Sonthalia, R., Gilbert, A. C., and Durham, M. Cuberep: Learning relations b...
Pith/arXiv arXiv 1963
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.