Pith. sign in

REVIEW 1 major objections 6 minor 1 cited by

Expected path length on random manifolds

T0 review · 1 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read This paper proves that a deterministic expected metric approximates expected curve length on random manifolds with relative error O(1/n), making latent-space geometry computationally tractable.

desk verdict Sound, genuinely new approximation result with a real but fixable scope overclaim: the independence condition is load-bearing, so the 'any smooth stochastic generative process' claim in the intro is false as stated. read the letter →

arxiv 1908.07377 v1 pith:472GGWHT submitted 2019-08-20 cs.LG stat.ML

classification cs.LGstat.ML MSC 53B2060G6060D05
keywords randomRiemannianmetricexpectedGaussianprocesslatentvariablemodelmanifoldlearningoperationalrepresentationcurvelengthhigh-dimensionalapproximationstochasticimmersion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Generative models map a low-dimensional latent space to high-dimensional data, but treating the latent space as Euclidean gives distances that change under reparametrization. The paper instead equips the latent space with a random Riemannian metric induced by the model's Jacobian, and asks whether the deterministic 'expected metric' — the mean of that random metric — can stand in for it. The main result, Proposition 4.6, answers yes: for stochastic immersions with independent components and bounded moments, the relative difference between expected curve length and length in the expected metric is at most $A^2/(8 n b^4)$ for large ambient dimension $n$. The error is $O(1/n)$, so the approximation is tight precisely when data are high-dimensional, the common case in machine learning. This provides a principled justification for using standard Riemannian geometry tools — geodesics, distances, volume — on the expected metric of models such as Gaussian process latent variable models.

What carries the argument

The load-bearing identity is a Taylor expansion of the square root near 1: Remark 4.2 proves $|\sqrt{x}-P(x)|\le \frac{5}{16}(x-1)^4$ for the cubic Taylor polynomial $P$, giving a tight remainder. Applied to the scaled squared speed $w_n^2=\|\phi'_n(t)\|^2$, this yields Proposition 4.3: for a 'balanced' sequence of random vectors, $E(\|W_n\|)=m_n-\Sigma_n^2/(8n m_n^3)+O(n^{-2})$, where $m_n=\sqrt{E\|W_n\|^2}$ and $\Sigma_n^2=n\,\operatorname{var}(\|W_n\|^2)$. Independence of the component processes (Lemma 4.5) is what makes the sequence balanced, forcing the central moments of $w_n^2$ to be $O(n^{-2})$; integrating the pointwise bound over the curve then gives Proposition 4.6.

What would settle it

Construct a sequence $\phi_n(t)=(g(t)+\epsilon_1(t),\dots,g(t)+\epsilon_n(t))/\sqrt n$ with a common smooth process $g$ and independent small noises $\epsilon_i$, so the squared speed has a correlated component that does not vanish after normalization. For this sequence, compute $L_n$ and $l_n$ for increasing $n$: if the relative error $(L_n-l_n)/L_n$ decays slower than $O(1/n)$ or exceeds $A^2/(8nb^4)$, the independence assumption is essential.

Watch

Extended reading notes

Core claim

The central claim is Proposition 4.6: let $\phi_n:[0,1]\to\mathbb{R}^n$ be a sequence of stochastic immersions whose derivative components $f'_1(t), f'_2(t),\dots$ are independent with uniformly bounded moments, with expected speed $m_n(t)=\sqrt{E(\|\phi'_n(t)\|^2)}$ bounded away from zero and normalized variance $\Sigma_n(t)=\sqrt{n\,\operatorname{var}(\|\phi'_n(t)\|^2)}$ bounded above by $A$. Writing $L_n$ for the length of the curve in the expected metric and $l_n$ for the expected length, the paper proves $0 \le (L_n-l_n)/L_n \le A^2/(8nb^4)$ for all large enough $n$. Thus the expected length of a random curve is sandwiched below the length in the expected metric, and the two agree to order $1/n$ in the ambient dimension. The paper further shows this justifies minimizing expected energy: geodesics of the expected metric minimize an upper bound on expected length, and the approximation error vanishes as the data dimension grows.

Load-bearing premise

The proof requires that the output coordinates of the stochastic process be independent with uniformly bounded moments, and that the expected speed stay bounded away from zero; if coordinates are correlated, the variance term does not shrink at rate $1/n$ and the bound collapses.

Editorial extensions

If this is right

  • In high-dimensional data regimes (images, video, sensor arrays), practitioners can replace a stochastic generative model's random metric by its expectation and use off-the-shelf Riemannian geometry; the resulting distances and geodesics are accurate to order $1/n$.
  • Minimizing expected curve energy — equivalently, computing geodesics in the expected metric — minimizes an upper bound on expected curve length, giving a variational justification for the approximation.
  • For a GPLVM with linear covariance kernel, the expected metric is constant, so the mean manifold is flat and geodesics are straight lines; the theorem certifies this simplified geometry when the output dimension is large.
  • The bound is dimension-dependent only: for fixed latent dimension, increasing the data dimension $n$ shrinks the approximation error, independent of the specific mean and kernel of the process.
  • Empirically, on a rotated-image GPLVM posterior, the paper reports that the relative error $(L_n-l_n)/L_n$ tracks the theoretical curve $A^2/(8nb^4)$, confirming the $O(1/n)$ rate.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The independence of output coordinates is the real scope condition. Generative models with correlated coordinates — for instance convolutional generators with shared features — may violate the balanced-sequence condition, and the $O(1/n)$ bound would not be expected to hold; measuring the relative error for such models would test whether a correlated analogue exists.
  • The Taylor-expansion technique is not tied to lengths: the same 'deterministic expectation is a good surrogate' argument could be applied to expected volume, expected energy, or expected curvature on random manifolds, each yielding its own dimension-dependent error rate.
  • The bound also suggests a practical diagnostic: estimate $A$ and $b$ from samples of the process (from a trained GPLVM or VAE) and compare the observed $(L_n-l_n)/L_n$ with the theoretical envelope; a large deviation flags a regime where the deterministic approximation is unsafe.
  • An implicit consequence is that the approximation is a large-$n$ phenomenon, not a large-sample phenomenon: increasing the number of latent-space samples does not improve the metric approximation, only increasing the output dimensionality does.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 6 minor

Summary. This paper studies random Riemannian metrics induced on the latent space of a stochastic generative model f : Z → R^n by the pullback metric M = J_f^T J_f. The paper's main result, Proposition 4.6, bounds the relative difference between the expected length l_n of curves and the length L_n in the expected metric E(M): for large n, 0 ≤ (L_n - l_n)/L_n ≤ A^2/(8 n b^4). The bound is derived under the hypotheses of Section 4.3 that the coordinate derivative processes f'_i(t) are independent at each t, have uniformly bounded moments, and that m_n(t) = E(||φ'_n(t)||^2)^{1/2} is uniformly bounded away from zero. The proof uses a Taylor expansion for the norm (Proposition 4.3) and the 'balanced sequence' condition (Definition 4.1), verified for independent coordinates in Lemma 4.5. The paper also computes the expected metric for GPLVMs and reports a numerical illustration on rotated images.

Significance. The result is practically useful: it justifies replacing a random Riemannian metric by its expectation, enabling standard Riemannian computations in latent space. The proof is self-contained, the constants A and b are explicit uniform moment bounds rather than fitted parameters, and Proposition 4.3 identifies the exact leading correction -Σ_n^2/(8 n m_n^3). The empirical illustration in Fig. 4 agrees with the predicted decay. The main limitation is that the theorem requires independence across output coordinates; this is satisfied by the GPLVM example because the posterior components are independent, but it is not the 'any smooth stochastic generative process' advertised in Section 1. With that scope corrected, the contribution is solid.

major comments (1)
  1. [Section 1 (last paragraph) and Section 4.3] The sentence 'Our analysis holds for any smooth stochastic generative process' is not supported by the theorem. Proposition 4.6 is proved only under the Section 4.3 hypotheses that, for each t, the coordinate derivative processes f'_1(t), f'_2(t), ... are independent, have uniformly bounded moments, and that inf_{n,t} m_n(t) > 0. Independence is not a cosmetic regularity condition: Lemma 4.5 uses it to show that the normalized vectors W_n(t) satisfy the balanced-sequence condition (Definition 4.1), and the balance bounds on n^2 μ_3 and n^2 μ_4 are exactly what turn the cubic-Taylor remainder in Proposition 4.3 into O(n^{-2}). If the coordinates are correlated, Σ_n(t) = (n var(w_n^2(t)))^{1/2} need not be bounded; for example, with f'_i(t) = Y(t) + δ ε_i'(t) and a common random factor Y(t), the variance of w_n^2(t) contains var(Y(t)^2), so Σ_n(t) grows like √n and the conclusion of Proposition 4.6 does not follow. The introduction and abstract should be revised to state the independence and uniform-moment assumptions, or to describe the class of processes to which the result actually applies.
minor comments (6)
  1. [Section 1] In the last paragraph of Section 1, 'to which extend' should be 'to which extent'.
  2. [Section 4.3 (before Proposition 4.6)] The phrasing 'Let sup_{n,t} Σ_n(t) < A and 0 < b < inf_{n,t} m_n(t)' is awkward; it should say 'Let A > sup_{n,t} Σ_n(t) and 0 < b < inf_{n,t} m_n(t)' (same in the statement of Proposition 4.6).
  3. [Proof of Proposition 4.6] The sentence 'Divide by b and note that b < L_n for all n' is imprecise; the actual step is to divide by L_n and use b < L_n to obtain the denominator b^4. Please rephrase.
  4. [Figure 4 caption] The caption 'Graph of (Ln ln)/Ln h(n)' appears to be missing minus signs; it should be something like '(L_n - l_n)/L_n - h(n)'.
  5. [Section 4.3 (after Proposition 4.6)] The phrase 'It it interesting' should read 'It is interesting'.
  6. [Section 4.3 (before Proposition 4.6)] The step 'In fact, due to the uniform bounds on the moments...' could be expanded to show that the constants in Lemma 4.5 and Proposition 4.3 are independent of t; as written this is asserted rather than demonstrated.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the expected-metric length bound is derived from explicit moment assumptions, not from fitted quantities or load-bearing self-citations.

full rationale

The paper's central result, Proposition 4.6, is a self-contained analytical estimate. It defines the two quantities being compared, L_n and l_n, explicitly in Eq. (5), and derives the bound from a Taylor expansion of the square root with a uniform remainder (Remark 4.2 and Proposition 4.3). The constants A and b are obtained from sup_{n,t} Sigma_n(t) and inf_{n,t} m_n(t), which are defined in terms of the moments of the process, not fitted to the conclusion. The independence and uniform bounded-moment assumptions stated at the start of Section 4.3 are genuine hypotheses used to verify the balanced condition in Lemma 4.5; they are not imported from the result being proved. The empirical illustration estimates A and b from data to draw the reference function h(n), but that is a demonstration of the bound, not part of the proof. The self-citations that appear, such as [17] for the desiderata of an operational representation and [32, 2] for earlier study of expected metrics, are motivational or contextual and do not carry the load of the derivation. No equation reduces to its own input by construction, and no fitted parameter is renamed as a prediction. The abstract's phrase 'any smooth stochastic generative process' is broader than the theorem's independence and uniform-moment assumptions, but that is a scope overclaim, not circularity. The derivation itself is self-contained and non-circular.

Assumptions & free parameters 0 free parameters · 7 assumptions · 0 invented entities

The paper's results rest on standard probability and geometry plus explicit modeling assumptions: smoothness, full-rank expected metric, independence and bounded moments of component processes. No entities are invented; the expected metric predates this work. The constants A and b in the bound are not free parameters: they are uniform bounds whose existence is assumed.

assumptions (7)
  • standard math The pullback metric M_z = J_f(z)^T J_f(z) defines a Riemannian metric on the latent space.
    Section 2 uses the standard construction of the induced metric from an embedding; this is background geometry, not introduced for this paper.
  • domain assumption Sample paths of f are smooth, or f is mean square smooth with smooth covariance function, so that J_f and the metric exist.
    Section 3 and Remark 3.3 invoke smoothness or mean-square smoothness to define the stochastic metric and expected metric.
  • domain assumption The component processes f_1, ..., f_n are independent and have uniformly bounded moments.
    Section 4.3 states independence of f'_i(t) and bounded moments; this is the key assumption for Lemma 4.5 and Proposition 4.6.
  • domain assumption The expected speed m_n(t) is uniformly bounded away from 0.
    Section 4.3 assumes 0 < inf_{n,t} m_n(t); this is needed for the bound in Proposition 4.6 and to divide by m_n.
  • standard math The posterior of a GPLVM is a symmetric Gaussian process with the given mean and kernel.
    Section 5.2 cites Rasmussen and Williams for the posterior form; this is a standard result.
  • standard math A Gaussian process with real analytic mean and covariance is mean square smooth.
    Example 3.4 cites Adler and Taylor, criterion (1.4.9), to justify mean square smoothness.
  • standard math A curve minimizing curve energy is locally length-minimizing and has constant velocity.
    Proposition 2.1 is cited from Gallot, Hulin, Lafontaine; it is standard Riemannian geometry.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Expected path length on random manifolds." pith.science (2026). https://pith.science/paper/472GGWHT

@misc{pith2026190807377,
  author       = {Pith},
  title        = {Pith review of: Expected path length on random manifolds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/472GGWHT}},
  note         = {Machine review of arXiv:1908.07377}
}
read the original abstract

Manifold learning seeks a low dimensional representation that faithfully captures the essence of data. Current methods can successfully learn such representations, but do not provide a meaningful set of operations that are associated with the representation. Working towards operational representation learning, we endow the latent space of a large class of generative models with a random Riemannian metric, which provides us with elementary operators. As computational tools are unavailable for random Riemannian manifolds, we study deterministic approximations and derive tight error bounds on expected distances.

Figures

Figures reproduced from arXiv: 1908.07377 by the authors.

Figure 1
Figure 1. Reparametrizations illustrated. Left: A “swirling” transformation of the la￾tent space Z. Right: Pair wise distances between points before and after reparametriza￾tion; clearly the Euclidean distances change with reparametrizations. • Measure: the representation should be equipped with a measure under which integration is well defined for all points on the manifold. These are elementary requirements of a representat… view at source ↗
Figure 2
Figure 2. Image of an embedded manifold M and the tangent space TxM at a point x ∈ M. which we denote by hf(z + ∆1) − f(z), f(z + ∆2) − f(z)i0. Then hf(z + ∆1) − f(z), f(z + ∆2) − f(z)i0 = hf(z) + Jf (z)∆1 − f(z), f(z) + Jf (z)∆2 − f(z)i = hJf (z)∆1, Jf (z)∆2i = ∆> 1 Jf (z) >Jf (z)∆2, where Jf (z) is the n × d Jacobian matrix of f at z. The d × d symmetric positive definite matrix Mz = Jf (z) >Jf (z) defines a Riemannian metr… view at source ↗
Figure 3
Figure 3. A rotated image. Feeding these to a GPLVM with RBF kernel and d-dimensional latent space we obtain a Gaussian process f : R d → R m together with a sequence of N latent points X ⊂ R d . For any n ≤ m we have a Gaussian process Φn : R d → R n given by projection onto the n first coordinates. Let c : [0, 1] → R d be the line segment joining the first two points of X and put φn = Φn ◦ c. Let ln and Ln be given as in Eq… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Extended Neural Contractive Dynamical Systems: On Multiple Tasks and Riemannian Safety Regions

    cs.RO 2024-11 conditional novelty 5.0 of 10

    An extended NCDS framework learns multiple robot skills from a single network by conditioning on task variables and performs obstacle avoidance in the latent space while preserving contraction-based stability.

Reference graph

Works this paper leans on

33 extracted references · 29 canonical work pages · cited by 1 Pith paper

  1. [1]

    R. J. Adler and J. E. Taylor. Random Fields and Geometry. Springer, 2007 edition, 2007. 16

  2. [2]

    Arvanitidis, L

    G. Arvanitidis, L. K. Hansen, and S. Hauberg. Latent space oddity: on the curva- ture of deep generative models. International Conference on Learning Representa- tions (ICLR), 2018

  3. [3]

    R. G. Baraniuk and M. B. Wakin. Random projections of smooth manifolds. Foundations of Computational Mathematics , 9:51–77, 2009

  4. [4]

    Laplacian Eigenmaps for Dimensionality Re- duction and Data Representation

    Mikhail Belkin and Partha Niyogi. Laplacian Eigenmaps for Dimensionality Re- duction and Data Representation. Neural Computation , 15(6):1373–1396, June 2003

  5. [5]

    High-dimensional $p$-norms

    G. Biau and D. M. Mason. High-dimensional p-norms. arXiv:1311.0587, 2013

  6. [6]

    Variational inference: A review for statisticians

    David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American Statistical Association , 112(518):859–877, 2017

  7. [7]

    Random projection, margins, kernels, and feature-selection

    Avrim Blum. Random projection, margins, kernels, and feature-selection. In Craig Saunders, Marko Grobelnik, Steve Gunn, and John Shawe-Taylor, editors, Sub- space, Latent Structure and Feature Selection, pages 52–68. Springer Berlin Heidel- berg, 2006

  8. [8]

    Clarkson

    Kenneth L. Clarkson. Tighter bounds for random projections of manifolds. In Proceedings of the Twenty-fourth Annual Symposium on Computational Geometry, SCG ’08, pages 39–48, New York, NY, USA, 2008. ACM

Show all 33 references
  1. [9]

    Cram´ er.Mathematical methods of statistics

    H. Cram´ er.Mathematical methods of statistics . Asia Publishing House, 1962

  2. [10]

    An elementary proof of a theorem of Johnson and Lindenstrauss

    Sanjoy Dasgupta and Anupam Gupta. An elementary proof of a theorem of Johnson and Lindenstrauss. Random Structures & Algorithms , 22(1):60–65, 2003

  3. [11]

    Donoho and Carrie Grimes

    David L. Donoho and Carrie Grimes. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proceedings of the National Academy of Sciences, 100(10):5591–5596, 2003

  4. [12]

    On the remainder term in the central limit theorem

    Carl-Gustav Esseen. On the remainder term in the central limit theorem. Ark. Mat., 8(1):7–15, 11 1969

  5. [13]

    Oren Freifeld, Søren Hauberg, and Michael J. Black. Model transport: Towards scalable transfer learning on manifolds. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR) , Columbus, Ohio, USA, June 2014

  6. [14]

    Learning the structure of manifolds using random projections

    Yoav Freund, Sanjoy Dasgupta, Kabra Mayank, and Nakul Verma. Learning the structure of manifolds using random projections. In J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 473–480. Curran Associates, I...

  7. [15]

    Riemannian geometry, volume 3

    Sylvestre Gallot, Dominique Hulin, and Jacques Lafontaine. Riemannian geometry, volume 3. Springer, 1990. 17

  8. [16]

    GPy: A Gaussian process framework in python

    GPy. GPy: A Gaussian process framework in python. http://github.com/ SheffieldML/GPy, 2012

  9. [17]

    S. Hauberg. Only bayes should learn a manifold (on the estimation of differential geometric structure from data). arXiv:1806.04994, 2018

  10. [18]

    Søren Hauberg, Oren Freifeld, and Michael J. Black. A geometric take on met- ric learning. In P. Bartlett, F.C.N. Pereira, C.J.C. Burges, L. Bottou, and K.Q. Weinberger, editors, Advances in Neural Information Processing Systems (NIPS) 25, pages 2033–2041. MIT Press, 2012

  11. [19]

    Random projections for manifold learning

    Chinmay Hegde, Michael Wakin, and Richard Baraniuk. Random projections for manifold learning. In J. C. Platt, D. Koller, Y. Singer, and S. T. Roweis, editors, Advances in Neural Information Processing Systems 20 , pages 641–648. Curran Associates, Inc., 2008

  12. [20]

    Probabilistic solutions to differential equations and their application to Riemannian statistics

    Philipp Hennig and Søren Hauberg. Probabilistic solutions to differential equations and their application to Riemannian statistics. In Proceedings of the 17th interna- tional Conference on Artificial Intelligence and Statistics (AISTATS) , volume 33, 2014

  13. [21]

    R. A. Khan. Approximation for the expectation of a function of the sample mean. Statistics, 38:117–122, 2004

  14. [22]

    Auto-Encoding Variational Bayes

    Diederik P Kingma and Max Welling. Auto-Encoding Variational Bayes. In Pro- ceedings of the 2nd International Conference on Learning Representations (ICLR), 2014

  15. [23]

    Random projections of random manifolds

    Subhaneil Lahiri, Peiran Gao, and Surya Ganguli. Random projections of random manifolds. arXiv:1607.04331, 07 2016

  16. [24]

    Probabilistic non-linear principal component analysis with Gaussian process latent variable models

    Neil Lawrence. Probabilistic non-linear principal component analysis with Gaussian process latent variable models. J. Mach. Learn. Res., 6:1783–1816, December 2005

  17. [25]

    R. J. Muirhead. Aspects of Multivariate Statistical Theory . Wiley, 2005

  18. [26]

    Intrinsic Statistics on Riemannian Manifolds: Basic Tools for Geo- metric Measurements

    Xavier Pennec. Intrinsic Statistics on Riemannian Manifolds: Basic Tools for Geo- metric Measurements. Journal of Mathematical Imaging and Vision, 25(1):127–154, July 2006

  19. [27]

    C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006

  20. [28]

    Stochastic back- propagation and approximate inference in deep generative models

    Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic back- propagation and approximate inference in deep generative models. In Eric P. Xing and Tony Jebara, editors, Proceedings of the 31st International Conference on Ma- chine Learning, volume 32 of Proceedin...

  21. [29]

    Nonlinear dimensionality reduction by locally linear embedding

    Sam T Roweis and Lawrence K Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500):2323–2326, 2000

  22. [30]

    Kernel principal component analysis

    Bernhard Sch¨ olkopf, Alexander Smola, and Klaus-Robert Mller. Kernel principal component analysis. In Advances in Kernel Methods - Support Vector Learning , pages 327–352, 1999

  23. [31]

    A global geometric framework for nonlinear dimensionality reduction

    Joshua B Tenenbaum, Vin De Silva, and John C Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500):2319–2323, 2000

  24. [32]

    Lawrence

    Alessandra Tosi, Søren Hauberg, Alfredo Vellido, and Neil D. Lawrence. Met- rics for probabilistic geometries. In The Conference on Uncertainty in Artificial Intelligence (UAI), Quebec, Canada, July 2014

  25. [33]

    N. Verma. A note on random projections for preserving paths on a manifold. UC San Diego, Tech. Report CS2011-0971, 2011. 19

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.