Pith. sign in

REVIEW 3 major objections 4 minor 41 references

A geometric filter can hard-bound generative hallucination and eliminate catastrophic forgetting in over-parameterized models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 03:14 UTC pith:6ER2QXQK

load-bearing objection Ambitious fiber-bundle packaging of over-parameterization that formalizes SID/SVDχ and connection filtering, but the hard OOD-diameter and total-forgetting claims rest on unconstructed objects. the 3 major comments →

arxiv 2607.03329 v1 pith:6ER2QXQK submitted 2026-07-03 cs.LG stat.ME

Statistically Meaningful Geometry (SMG) Beyond the Euclidean Paradigm, with Application to Generative AI

Bing Cheng , Yi-Shuai Niu , Howell Tong , Shing-Tung Yau This is my paper
classification cs.LG stat.ME MSC 53C0562B1068T07
keywords Statistically Meaningful GeometryEhresmann connectionfiber bundleover-parameterizationgenerative hallucinationcatastrophic forgettingOrlicz statistical manifoldPAC-Bayesian bounds
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that classical statistics fails for giant over-parameterized models because their parameter spaces are not flat Euclidean containers. They are fiber bundles whose infinite internal degrees of freedom form flat vertical valleys that carry no statistical signal. Those valleys make classical generalization bounds vacuous and let optimization produce generative hallucination and catastrophic forgetting. The proposed Statistically Meaningful Geometry (SMG) framework lifts the model into an infinite-dimensional Orlicz statistical manifold organized as a fiber bundle, then uses an Ehresmann connection as a dynamic filter that discards the vertical gauge noise and keeps learning only on the non-degenerate horizontal directions. Under that filtered pre-training, out-of-distribution predictive variance is proved to be bounded by the finite diameter of the identifiable base manifold, and a sequential adaptation flow that projects new updates off the historical horizontal carriage is proved to eliminate catastrophic forgetting non-asymptotically. The result replaces fine-tuning heuristics with coordinate-free topological constraints.

Core claim

Under connection-filtered pre-training the model’s out-of-distribution predictive variance is strictly upper-bounded by the finite diameter of the identifiable quotient base manifold, giving a hard geometric containment of generative hallucinations; projecting downstream updates onto the orthogonal complement of the historical horizontal carriage yields total non-asymptotic elimination of catastrophic forgetting.

What carries the argument

The Ehresmann connection 1-form ω on the SMG fiber bundle (M, B, π, V, H, ω), which splits the tangent space into vertical Structural Internal Directions (SID) and horizontal Statistical Variational Directions (SVDχ) and acts as the geometric filter that quarantines gauge noise from observable learning.

Load-bearing premise

That a usable Ehresmann connection exists for real transformer architectures so the horizontal directions are integrable, the base is a finite-diameter identifiable manifold, and the orthogonal projection onto the historical horizontal carriage can be computed exactly.

What would settle it

Construct an explicit, computable connection for a concrete transformer, train under the filtered flow, and check whether measured out-of-distribution predictive variance stays inside the claimed base diameter while sequential adaptation leaves historical task performance unchanged to machine precision.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Statistically Meaningful Geometry (SMG), lifting over-parameterized models (e.g., transformers) into infinite-dimensional Orlicz statistical manifolds equipped with a general fiber bundle (M, B, π, V, H, ω). An Ehresmann connection 1-form ω filters vertical Structural Internal Directions (SID / gauge noise) so that learning trajectories remain on the horizontal Statistically Verifiable Directions (SVDχ). Under this connection-filtered pre-training the authors claim that out-of-distribution predictive variance is strictly upper-bounded by the finite diameter of the identifiable quotient base B (Theorem 16), thereby geometrically containing generative hallucinations; projecting downstream updates onto the orthogonal complement of the historical horizontal carriage is claimed to yield non-asymptotic elimination of catastrophic forgetting (SMG Sequential Adaptation Flow). Classical MLE, natural gradient, Wilks asymptotics and PAC-Bayesian bounds are recovered on the reduced base, and three nested learning pathways (ambient filtering, leaf integration, macro inference) are shown to be equivalent under integrability.

Significance. If the geometric containment and forgetting-elimination claims hold for realistic architectures, the work would supply a coordinate-free topological foundation for structural reliability in generative AI, resolving the over-parameterization generalization paradox without parameter-counting heuristics. The manuscript supplies an extensive formal apparatus (Lemmas 1–8, Theorems 1–16) that systematically embeds transformers into Pistone–Sempi Orlicz manifolds, defines metric-compatible projections, and derives capacity-collapse and quarantine results; these machine-readable proofs and the clean Two-Fold Inference Paradigm are genuine strengths. The contribution remains purely theoretical: no constructive connection, no numerical verification, and no comparison against existing continual-learning or hallucination-mitigation baselines are provided, so practical impact is still conditional.

major comments (3)
  1. [§8.3, Theorem 16] §8.3, Theorem 16 (Geometric Containment of Generative Hallucinations): the hard OOD bound reduces the filtered trajectory to natural-gradient flow on B and then invokes “standard compact embedding theorems” so that the path stays inside a compact geodesic ball K of finite diameter KB independent of ambient weight dimension W. The manuscript never constructs, for a concrete transformer, a base B that is simultaneously finite-dimensional, identifiable, and of finite diameter under the induced L2 metric once the evaluation domain includes an unbounded XOOD. Without diam(B)<∞ the claimed geometric containment does not follow; the same ungrounded finite-diameter object is reused for the Capacity Collapse Theorem (Thm 12) and the forgetting argument.
  2. [§§2–4, 8.3–8.4] §§2–4 and 8.3–8.4 (Definitions 9, 20; Theorems 2, 10, 12, 16): all operational claims presuppose a practically usable Ehresmann connection 1-form ω such that the horizontal distribution is Frobenius-integrable (Ω≡0), the orthogonal projection onto the historical horizontal carriage is exact, and the resulting base B has finite diameter. No explicit construction, approximation scheme, or even low-rank surrogate for ω is supplied for transformer weight spaces. Consequently the “total non-asymptotic elimination” of forgetting and the hard hallucination bound remain formal statements conditional on an oracle connection.
  3. [Lemma 4, Theorems 6 & 12] Lemma 4 / Theorem 6 and the Capacity Collapse argument (Thm 12): once the horizontal distribution is defined as the gf-orthogonal complement of ker(dπ) and the connection is defined to project onto that complement, fiber-wise invariance of the likelihood and the collapse of ambient KL divergence to the base KL follow essentially by construction. The manuscript presents these as deep structural theorems; a clearer separation between definitional consequences and non-tautological geometric content would strengthen the load-bearing claims.
minor comments (4)
  1. [§§1.3, 1.7] The Two-Fold Inference Paradigm is restated almost verbatim in §§1.3 and 1.7; the four core axioms likewise appear twice with only minor rephrasing. Condensing the introductory material would improve readability.
  2. [Figures 1–4] Figures 1–4 are conceptually helpful but the captions and in-text references contain overlapping descriptive text that is hard to parse; a single clean schematic of the three nested pathways would suffice.
  3. [§5.2, Eq. (87)] Notation for the horizontal natural gradient (Eq. 87) and the induced base likelihood is introduced late and then used retroactively; a short notation table early in §2 would help.
  4. [Introduction / Related work] The manuscript cites Amari, Pistone–Sempi and classical PAC-Bayes correctly, yet several recent geometric deep-learning and continual-learning references that address similar gauge/redundancy issues are absent; adding a short related-work paragraph would situate the contribution more clearly.

Circularity Check

5 steps flagged

Core theorems (likelihood invariance, capacity collapse, quarantining, natural-gradient equivalence) follow by construction once SID is ker(dπ) and SVDχ its Fisher-Rao orthogonal complement; finite diam(B) for Thm 16 is assumed via external citation rather than derived for transformers.

specific steps
  1. self definitional [Lemma 4 (Fiber-Wise Invariance) / Thm 6]
    "For any pair of observationally equivalent density states f1, f2 ∈ Fp, the empirical log-likelihood function satisfies absolute functional identity: ℓn(f1)=ℓn(f2). ... Because the projection submersion π isolates observable statistical manifestations... f1(x)=f2(x) µ-almost everywhere... ℓn(f1)=...=ℓn(f2)."

    Fp is defined as π^{-1}(p), so elements of a fiber are observationally equivalent densities by construction; the likelihood (a functional of the density values on data) is therefore identical by definition of the fiber, not by a non-tautological derivation. Thm 6 then collapses ambient MLE to unique base MLE solely from this invariance plus the regularity assumption that B itself has unique MLE.

  2. self definitional [Theorem 3 / Definition 9]
    "The Ehresmann connection 1-form ωf is identical to the canonical vertical projection operator P^V_f under the non-parametric Fisher-Rao metric gf. Consequently, the horizontal projection operator P^H_f can be expressed as: P^H_f = I − ωf."

    Definition 9 sets ωf(v)=v for v∈ SIDf and ker(ωf)=SVDχf; Lemma 2 already defines P^V as the metric orthogonal projector onto SID. The 'proof' that ω≡ P^V is therefore identity of two objects given the same characterizing properties, not an independent result.

  3. self definitional [Theorem 12 (Capacity Collapse) + Assumption 2]
    "Q = Q_SVDχ ⊗ δ0(SID)... P = P_SVDχ ⊗ P_SID... DKL(Q∥P)=∫_H ln(dQ_SVDχ/dP_SVDχ)dQ_SVDχ + ∫_V ln(dδ0/dP_SID)dδ0 = DKL(QB∥PB)+0."

    Assumption 2 constructs the prior as an independent product over horizontal and vertical; horizontal flow (Way 2) forces the posterior to concentrate as Q_SVDχ ⊗ δ0 on a single leaf. The ambient KL therefore factors and the vertical term vanishes by construction of the product measure and Dirac, so the PAC-Bayes bound 'collapses' to the base by the definitions of Q and P rather than by a non-tautological capacity argument.

  4. self definitional [Theorem 16 (Geometric Containment) proof]
    "the optimization path pt is contained within a compact geodesic ball K⊂B. ... sup_t ∥pt−p0∥^2_L2(XOOD) ≤ diam_g(K) ≜ KB <∞. This confirms that filtering... rendering... immune to generative hallucinations."

    After reducing the filtered trajectory to NGD on B (itself by the lift isomorphism of the connection), the hard finite upper bound is obtained solely by invoking 'standard compact embedding theorems for regular statistical manifolds' so that diam(K)<∞. No construction is given that the base B arising from a transformer (space of regression maps X→Y) is finite-dimensional with finite diameter under the induced metric once XOOD is unbounded; the claimed 'hard geometric containment' is therefore the imported finiteness assumption restated as the theorem's conclusion.

  5. self citation load bearing [§1.3.3 / §1.2 (and repeated)]
    "the Cheng-Tong orthogonal metric decomposition [9]. By showing that the total tangent space... can be severed into a direct sum where the horizontal leaf is strictly orthogonal to the vertical gauge fiber under the Fisher-Rao metric... The Blessing of Dimensionality [9]."

    The orthogonal split TfM=Hf⊕Vf that underwrites the entire Two-Fold Inference Paradigm, the quarantine of infinite parameters into zero-variance SID, and the resolution of Occam's razor / over-parameterization paradox is repeatedly attributed to the authors' own prior work [9]. Although Lemma 1 re-derives the split, the narrative load-bearing claims (blessing of dimensionality, natural filtering of infinite dimensions, structural reliability) rest on that self-citation chain rather than on external verification.

full rationale

The paper defines the SMG fiber bundle with Vf ≔ ker(dπf) (SID) and Hf ≔ Vf^⊥ under the Fisher-Rao metric (SVDχ), with ω the vertical projector. Under these definitions, fiber-wise likelihood invariance (Lemma 4), ω ≡ P^V (Thm 3), quarantining by orthogonality (Thm 4), horizontal natural gradient = lift of base NGD (Thm 7), and Capacity Collapse reducing ambient KL to base KL via product prior + Dirac vertical (Thm 12 + Assump 2) are immediate unpackings of the definitions and standard projection/isometry properties; they are not independent derivations. Thm 16 reduces filtered flow to NGD on B then invokes 'standard compact embedding theorems' (citing Amari) to obtain finite diam(K) ⊂ B, but never constructs a finite-dimensional finite-diameter base for a concrete transformer (or unbounded XOOD); the hard OOD bound is therefore the finiteness assumption renamed as a geometric consequence of filtering. Self-citations to [9] (Cheng-Tong orthogonal decomposition, Blessing of Dimensionality) are load-bearing for the narrative of the two-fold paradigm and Occam resolution, though the split is re-proved in Lemma 1. No fitted-input-as-prediction or external uniqueness smuggling. The framework is largely self-contained once the axioms and connection are granted, but the strongest claims reduce by construction or by unestablished diameter, yielding moderate circularity.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 4 invented entities

The central claims rest on a stack of infinite-dimensional geometric hypotheses and four paper-specific axioms that define the very objects whose properties are later 'proved'. No free parameters are fitted to data because there are no data; the free choices are modeling choices (Young function, connection, base dimension). Several invented geometric objects (SID, SVDχ, SMG Sequential Adaptation Flow) carry the load of the applied conclusions and lack independent empirical handles.

free parameters (3)
  • Young function Φ(u)=cosh(u)-1 for the Orlicz space
    Chosen by hand to define the ambient manifold topology; other Young functions would yield different Luxemburg norms and possibly different horizontal/vertical splits.
  • observational noise scale σ in the induced joint density
    Appears in the Transformer embedding (Definition 18); treated as a fixed positive constant that affects the explicit form of vertical score perturbations used in the hallucination proof.
  • dimension d of the identifiable base manifold B
    Asserted finite and equal to the number of verifiable environmental features; never measured or bounded from a concrete architecture.
axioms (5)
  • ad hoc to paper Four Core Axioms of SMG (Environment Set E, System Set S, Structural Mechanism F, Invariance Principle)
    Introduced in §1.5 / §2.2 as the logical foundation; they are not standard Kolmogorov or information-geometry axioms but paper-specific postulates that force the horizontal/vertical split.
  • domain assumption Total space M is a Pistone–Sempi Orlicz manifold of densities with log-densities in L^Φ
    Standard in non-parametric information geometry (cited [29]); assumed throughout §§2–3.
  • domain assumption π : M → B is a smooth surjective submersion whose fibers are the observationally equivalent configurations
    Required for the Ehresmann connection and for Lemma 1; existence for real transformers is postulated, not constructed.
  • ad hoc to paper Horizontal distribution is Frobenius-integrable (curvature Ω ≡ 0) so that global horizontal leaves exist
    Theorem 2 and Way-2 learning rest on vanishing curvature; the paper notes that non-zero curvature produces path-dependency but assumes the integrable case for the main guarantees.
  • standard math Fisher–Rao metric is strictly positive-definite on the horizontal subspace and zero on the vertical subspace
    Follows from the definition of SID as the kernel of dπ once the metric is the Fisher–Rao product; used in every orthogonality argument.
invented entities (4)
  • Structural Internal Directions (SID) / vertical gauge fiber no independent evidence
    purpose: Quarantine all non-identifiable parameter variations so they carry zero Fisher information
    Defined as ker(dπ); the applied claims stand or fall on the existence and computability of this kernel for real nets. No independent experimental signature is given.
  • Statistically Verifiable Directions (SVDχ) / horizontal distribution no independent evidence
    purpose: Isolate the finite-dimensional directions that change the observable distribution
    Defined as the metric orthogonal complement of SID; capacity bounds collapse to dim(SVDχ) by construction.
  • SMG Sequential Adaptation Flow no independent evidence
    purpose: Project downstream updates orthogonal to the historical horizontal carriage to eliminate forgetting
    Named construction in §8.4; the 'total non-asymptotic elimination' claim is attached to this object alone.
  • Two-Fold Inference Paradigm no independent evidence
    purpose: Separate classical statistics on the base from pure geometric inference on the fiber
    Organizational device introduced by the paper; not an independently measured phenomenon.

pith-pipeline@v1.1.0-grok45 · 56465 in / 3803 out tokens · 41655 ms · 2026-07-12T03:14:11.281964+00:00 · methodology

0 comments
read the original abstract

Conventional uniform convergence bounds and empirical risk minimization break down in massive over-parameterized models, such as large language transformers and biological sequence networks. With near-infinite unconstrained internal degrees of freedom, their optimization landscapes develop flat vertical gauge valleys, rendering classical generalization metrics vacuous and inducing severe pathologies, specifically generative hallucination and catastrophic forgetting. We introduce the Statistically Meaningful Geometry (SMG) framework, an information-geometric paradigm lifting deterministic parametric models into infinite-dimensional non-parametric Orlicz statistical manifolds. Modeling the total state space as a differential fiber bundle ($\mathcal{M}, \mathcal{B}, \pi, \mathcal{V}, \mathcal{H}, \omega$), we establish a Two-Fold Inference Paradigm. We formalize an Ehresmann connection 1-form $\omega$ as a dynamic geometric filter that strips away vertical gauge noise (Structural Internal Directions, or SID) and isolates learning trajectories along the strictly non-degenerate horizontal distribution (Statistical Variational Directions, or SVD$\chi$). We prove that under connection-filtered pre-training, out-of-distribution predictive variance is strictly upper-bounded by the finite diameter of the identifiable quotient base manifold $\mathcal{B}$, establishing a hard geometric containment of generative hallucinations. By projecting downstream updates onto the orthogonal complement of the historical horizontal carriage, we formalize the SMG Sequential Adaptation Flow, proving the total non-asymptotic elimination of catastrophic forgetting. SMG replaces empirical fine-tuning heuristics with coordinate-free topological constraints, bridging advanced differential geometry with structural reliability in AI.

Figures

Figures reproduced from arXiv: 2607.03329 by Bing Cheng, Howell Tong, Shing-Tung Yau, Yi-Shuai Niu.

Figure 1
Figure 1. Figure 1: The foundational fiber bundle topology of Statistically Meaningful Geometry (SMG). [PITH_FULL_IMAGE:figures/full_fig_p017_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Dynamic comparison within the statistical bundle framework. The SMG functional [PITH_FULL_IMAGE:figures/full_fig_p024_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The metric-compatible Ehresmann connection splitting and the Horizontal Lift mech [PITH_FULL_IMAGE:figures/full_fig_p027_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Optimized geometric architectural layout of the three learning pathways within the [PITH_FULL_IMAGE:figures/full_fig_p044_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 2 linked inside Pith

  1. [1]

    Akaike, H. (1974). A New Look at the Statistical Model Identification . IEEE Transactions on Automatic Control, 19(6), 716–723. Brief Explanation: Establishes the Akaike Information Criterion (AIC), formalizing the classical linear parameter-counting penalty framework that breaks down in modern over- parameterized systems

    1974

  2. [2]

    Amari, S. (1987). Dual connections on the Hilbert bundles of statistical models . In Ge- ometrization of Statistical Theory (Lancaster, 1987), ULDM Publ., Lancaster, UK, pp. 123–151

    1987

  3. [3]

    Amari, S. (1998). Natural gradient works efficiently in learning . Neural Computation, 10(2), 251–276. Brief Explanation: The foundational paper on Natural Gradient Descent. Theorem 5.5 of SMG proves how to execute this exact algorithm on trillion-parameter models by lifting the computation to the base manifold, bypassing the singularity Amari identified

    1998

  4. [4]

    Amari, S., & Nagaoka, H. (2000). Methods of Information Geometry . Translations of Math- ematical Monographs, American Mathematical Society, Providence, RI. Brief Explanation: The foundational monograph on finite-dimensional information geom- etry, detailing the Riemannian structure of statistical models via the Fisher Information Metric, which classical ...

    2000

  5. [5]

    Anandan, J., & Aharonov, Y. (1990). Geometry of quantum evolution . Physical Review Letters, 65(14), 1697–1700. 73 Brief Explanation: Examines geometric phases along lifts, providing a foundation for an- alyzing how paths accumulate geometric phase shifts under non-trivial parallel transport rules

    1990

  6. [6]

    L., Foster, D

    Bartlett, P. L., Foster, D. J., & Telgarsky, M. J. (2017). Spectrally-normalized margin bounds for neural networks . Advances in Neural Information Processing Systems, 30, 6240– 6249. Brief Explanation: Establishes spectrally-normalized margin bounds to explain generaliza- tion in deep networks bypassing raw coordinate counts

    2017

  7. [7]

    Double Descent

    Belkin, M., Hsu, D., Ma, S., & Mandal, S. (2019). Reconciling modern machine-learning practice and the classical bias-variance trade-off . Proceedings of the National Academy of Sciences, 116(32), 15849–15854. Brief Explanation: The foundational paper formally identifying the “Double Descent” gen- eralization curve, empirically demonstrating that pushing ...

    2019

  8. [8]

    Catoni, O. (2007). PAC-Bayesian Supervised Classification: The Thermodynamics of Sta- tistical Learning. Institute of Mathematical Statistics Lecture Notes—Monograph Series, Vol. 56, Hayward, CA. Brief Explanation: Formulates modern PAC-Bayesian theory via generalization bounds under information-theoretic constraints

    2007

  9. [9]

    Cheng, B., & Tong, H. (2026). An Approach to Fisher-Rao Metric for Infinite Dimensional Non-Parametric Information Geometry . Entropy, 28(3), Article Number. Brief Explanation: Establishes the non-parametric Pistone-Sempi Orlicz manifold frame- work, rigorously justifying the application of Fisher-Rao metrics, Stein scores, and orthog- onal decompositions...

    2026

  10. [10]

    Cencov, N. N. (1982). Statistical Decision Rules and Optimal Inference . Translations of Mathematical Monographs, Vol. 53, American Mathematical Society, Providence, RI. Brief Explanation: The foundational monograph establishing the category-theoretic unique- ness of the Fisher information metric and alpha-connections under Markov categories, proving that...

    1982

  11. [11]

    Ehresmann, C. (1950). Les connexions infinitésimales dans un espace fibré différentiable . Colloque de Topologie, Bruxelles, pp. 29–55. Brief Explanation: Introduces the foundational theory of infinitesimal connections on gen- eral differential fiber bundles without relying on strict Lie group actions

    1950

  12. [12]

    Fisher, R. A. (1922). On the Mathematical Foundations of Theoretical Statistics . Philo- sophical Transactions of the Royal Society of London. Series A, 222, 309–368. Brief Explanation: The classical foundational text establishing mathematical statistics, maximum likelihood estimation, and sufficiency

    1922

  13. [13]

    French, R. M. (1999). Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4), 128–135. Brief Explanation: Explains the cognitive and connectionist mechanics under which neural networks suffer interference during sequential learning environments. 74

    1999

  14. [14]

    Hironaka, H. (1964). Resolution of singularities of an algebraic variety over a field of characteristic zero: I, II . Annals of Mathematics, 79(1/2), 109–326. Brief Explanation: The definitive algebraic geometry work establishing that singularities can be systematically resolved via algebraic blow-ups

    1964

  15. [15]

    Hochreiter, S., & Schmidhuber, J. (1997). Flat minima . Neural Computation, 9(1), 1–42. Brief Explanation: Establishes the link between the curvature (Hessian) of the loss landscape and generalization, supporting the argument that the infinite-dimensional SID space acts as a geometric reservoir allowing SGD to discover flat minima without altering empirical loss

    1997

  16. [16]

    Jacot, A., Gabriel, F., & Hongler, C. (2018). Neural Tangent Kernel: Convergence and Generalization in Neural Networks . Advances in Neural Information Processing Systems, 31. Brief Explanation: Establishes the properties of the JJT Gram matrix in the infinite-width limit. SMG’s Horizontal Lift Operator physically maps this kernel projection directly to t...

    2018

  17. [17]

    Kallenberg, O. (2002). Foundations of Modern Probability . Springer Science & Business Media. Brief Explanation: Provides the rigorous measure-theoretic foundations for regular condi- tional probabilities and the Disintegration Theorem, essential for decomposing measures across the SMG fiber bundle

    2002

  18. [18]

    Khesin, B., Misiolek, G., & Modin, K. (2024). Information geometry of diffeomorphism groups. arXiv preprint arXiv:2411.03265. Brief Explanation: Explores the information geometric structures and metrics constructed directly over continuous infinite-dimensional transformation spaces

    Pith/arXiv arXiv 2024

  19. [19]

    Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., ... & Hadsell, R. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences (PNAS) , 114(13), 3521–3526. Brief Explanation: Empirically characterizes the catastrophe of downstream updates de- stroying historical knowl...

    2017

  20. [20]

    Kobayashi, S., & Nomizu, K. (1996). Foundations of Differential Geometry, Vol. 1 . Wiley- Interscience. Brief Explanation: The definitive mathematical text on differential topology, principal fiber bundles, and connections. It provides the rigorous geometric foundation for the Ehresmann connection ω utilized to decouple the SMG tangent space

    1996

  21. [21]

    Kolmogorov, A. N. (1933). Grundbegriffe der Wahrscheinlichkeitsrechnung. Ergebnisse der Mathematik und Ihrer Grenzgebiete, Springer, Berlin. Brief Explanation: The foundational monograph establishing the modern measure-theoretic axiomatic framework of probability theory, defining the baseline sample space environments that SMG extends into infinite-dimens...

    1933

  22. [22]

    Lang, S. (1999). Fundamentals of Differential Geometry . Springer-Verlag, New York, NY

    1999

  23. [23]

    MacKay, D. J. (1992). A practical Bayesian framework for backpropagation networks . Neu- ral Computation, 4(3), 448–472. Brief Explanation: Demonstrates how weight-decay priors break the internal symmetries of neural networks, providing the basis for the Bayesian counter-argument to absolute invisi- bility. 75

    1992

  24. [24]

    Mathieu, E., & Nickel, M. (2020). Riemannian Continuous Normalizing Flows. Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) , PMLR 108:2503–2515. Brief Explanation: Formulates continuous probability density evolution equations driven by vector fields explicitly integrated over non-Euclidean Riemannian geometries

    2020

  25. [25]

    Maurer, A. (2004). A Note on the PAC-Bayesian Theorem . arXiv preprint cs/0411099. Brief Explanation: Provides the modernized, tight formulation of the PAC-Bayesian bound utilized in Theorem 5.3

    Pith/arXiv arXiv 2004

  26. [26]

    McAllester, D. A. (1999). PAC-Bayesian Model Averaging. Proceedings of the Twelfth An- nual Conference on Computational Learning Theory (COLT), 164–170. Brief Explanation: The foundational paper introducing PAC-Bayesian learning bounds. SMG elevates this theorem from finite Euclidean spaces to infinite-dimensional non- parametric fiber bundles based on th...

    1999

  27. [27]

    McCloskey, M., & Cohen, N. J. (1989). Catastrophic interference in connectionist networks: The sequential learning problem . The Psychology of Learning and Motivation, 24, 109–165. Brief Explanation: Formally identifies the pathology of structural interference when net- works sequentially update configurations on non-isolated objectives

    1989

  28. [28]

    O’Neill, B. (1966). The fundamental equations of a submersion . Michigan Mathematical Journal, 13(4), 459–469. Brief Explanation: The foundational mathematical paper defining the tensorial equations for Riemannian submersions, establishing that non-integrable horizontal distributions generate vertical holonomy and that horizontal lifts commute with gradient flows

    1966

  29. [29]

    Pistone, G., & Sempi, C. (1995). An infinite-dimensional geometric structure on the space of all the probability measures equivalent to a given one . The Annals of Statistics, 23(5), 1543–1561. Brief Explanation: The foundational text defining the exact non-parametric statistical man- ifold topology built on Orlicz spaces, used to house the baseline confi...

    1995

  30. [30]

    Statistically Verifiable Directions

    Popper, K. (1959). The Logic of Scientific Discovery . Routledge. Brief Explanation: The foundational text on the epistemology of science. It argues that a theory is only meaningful if it is empirically falsifiable/verifiable. This philosophically underpins the necessity of restricting statistical learning to the “Statistically Verifiable Directions” (SVDχ)

    1959

  31. [31]

    Rao, C. R. (1945). Information and the accuracy attainable in the estimation of a statistical parameter. Bulletin of the Calcutta Mathematical Society, 37, 81–91. Brief Explanation: Establishes the foundation of information geometry by interpreting the Fisher information matrix as a Riemannian metric on parameter space

    1945

  32. [32]

    J., & Mohamed, S

    Rezende, D. J., & Mohamed, S. (2015). Variational Inference with Normalizing Flows. Proceedings of the 32nd International Conference on Machine Learning (ICML) , PMLR 37:1530–1538. Brief Explanation: Introduces the change-of-variables framework for probability density functionals transformed via continuous invertible deep functional mappings

    2015

  33. [33]

    Sussmann, H. J. (1992). Uniqueness of the weights for minimal feedforward nets with a given input-output map . Neural Networks, 5(4), 589–593. 76 Brief Explanation: Provides rigorous proof of the profound permutation and sign-flip sym- metries present in neural networks. These symmetries are the physical generators of the Internal Degrees of Freedom (IDoF...

    1992

  34. [34]

    Vapnik, V. N. (1998). Statistical Learning Theory . Wiley, New York, NY. Brief Explanation: Establishes the foundations of classical statistical capacity limits via VC dimension and uniform bounding laws

    1998

  35. [35]

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., ... & Polo- sukhin, I. (2017). Attention Is All You Need . Advances in Neural Information Processing Systems, 30, 5998–6008. Brief Explanation: The foundational paper introducing the Transformer architecture, whose extreme multi-layer parameter scale and latent embedding dynamic...

    2017

  36. [36]

    Villani, C. (2009). Optimal Transport: Old and New . Graduate Studies in Mathematics, Vol. 99, Springer-Verlag, Berlin, Germany. Brief Explanation: Comprehensive reference detailing the behavior of entropy function- als, volume deformation tensors, and displacement convexity across general Riemannian manifold domains

    2009

  37. [37]

    Warner, F. W. (1983). Foundations of Differentiable Manifolds and Lie Groups . Graduate Texts in Mathematics, Vol. 94, Springer-Verlag, New York, NY. Brief Explanation: The definitive graduate-level text detailing the rigorous modern proof of the Frobenius Theorem, differential forms, and the integration of tangent distributions on smooth manifolds

    1983

  38. [38]

    Watanabe, S. (2009). Algebraic Geometry and Statistical Learning Theory . Cambridge Uni- versity Press, Cambridge, UK. Brief Explanation: Introduces Singular Learning Theory. Watanabe proves that hierarchical learning models inherently possess unidentifiable parameters, causing the classical Fisher Information Matrix to become degenerate. SMG actively tra...

    2009

  39. [39]

    Statistically Invariant

    Weyl, H. (1929). Elektron und Gravitation. I . Zeitschrift für Physik, 56(5-6), 330–352. Brief Explanation: The seminal paper establishing modern Gauge Theory. It introduces the concept that profound internal mathematical transformations can leave observable physics completely invariantithe direct physical analog to SMG’s “Statistically Invariant” vertical fibers

    1929

  40. [40]

    Wilks, S. S. (1938). The large-sample distribution of the likelihood ratio for testing compos- ite hypotheses . The Annals of Mathematical Statistics, 9(1), 60–62. Brief Explanation: Establishes Wilks’ theorem on the asymptotic distribution of log- likelihood ratio statistics under regular finite-dimensional conditions

    1938

  41. [41]

    Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021). Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3), 107– 115. Brief Explanation: Empirically demonstrates the failure of uniform convergence and stan- dard VC bounds in massive over-parameterized neural models. 77

    2021