Pith. sign in

REVIEW 4 major objections 6 minor 50 references

This paper proposes a placement law for hyperbolic geometry in LLM fine-tuning: the same Lorentz-geometry losses that collapse training when embedded inside a trainable adapter train stably when applied only at the loss layer, proven across

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:45 UTC pith:OWOR5IH2

load-bearing objection The loss-layer-only placement idea is new and the formal bound is clean, but the 'law' rests on a self-reported crash ledger with mismatched conditions and no matched downstream gains; worth refereeing with conditions. the 4 major comments →

arxiv 2607.17513 v1 pith:OWOR5IH2 submitted 2026-07-20 cs.LG cs.AIcs.CL

After the Euclidean Highway: Hyperbolic Expert AI as the Next Innovation

Kwan Soo Shin , In Seok Kang , Munho Lee This is my paper
classification cs.LG cs.AIcs.CL MSC 68T5068T07
keywords hyperbolic geometryLorentz manifoldplacement lawLoRA fine-tuningtraining stabilitytree-structured lossesbatch-size structuresmall language models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish a placement law for hyperbolic geometry in large-language-model fine-tuning: the question that decides stability is not how much of a network is curved, but where curvature may touch the gradient. The same Lorentz-geometry losses that collapsed training seventeen times (~220 GPU-hours) when embedded inside a trainable LoRA adapter trained to completion with zero NaN events when applied only at the loss layer, across six expert small language models and a 17.95M-sample corpus (~317K optimizer steps). The paper proves this separation in Proposition 1: a loss-layer-only projection keeps the adapter norm bound identical to the Euclidean LoRA case, while adapter-on-manifold placement inherits an unbounded sinh Jacobian factor that terminates in NaN at the bf16 boundary. Alongside, it identifies batch size as a structural variable: sibling-pair losses silently fail below a pair-formation threshold. If the placement law is right, tree-structured expert knowledge can be injected into pretrained transformers without rebuilding or re-curving the network.

Core claim

The central claim is that the instability of hyperbolic fine-tuning at scale is a placement problem, not a curvature problem. Placing Lorentz-geometry operations inside a trainable LoRA adapter lets the exponential map's Jacobian — sinh(‖B·x‖)/‖B·x‖ — ride the gradient and amplify adapter norms until bf16 precision breaks (‖v‖ ≈ 11), producing seventeen recorded crashes. Placing the same operations only at the loss layer decouples curvature from the adapter update: the adapter norm obeys the standard Euclidean LoRA bound E‖B_t‖_F ≤ ‖B_0‖_F + κΣη_s, independent of Lorentz curvature, and six complete training runs (17,954,911 samples, ~317K optimizer steps) recorded zero NaN events. The author

What carries the argument

The Lorentz projection π(h) = exp_0(h/τ) with a norm clamp ‖v‖ ≤ 2, applied only at loss evaluation. Its Jacobian is bounded by M_τ ≈ 5.23/τ, so the gradient that reaches the Euclidean adapter is curvature-free; the three tree-structure losses (PTLE, HLSD, HWC) then act on the manifold while all trainable parameters stay in R^d. Proposition 2 supplies the companion mechanism: sibling-pair losses silently fire at 0% unless the micro-batch exceeds the pair-formation threshold b* ≈ √(N/p̄).

Load-bearing premise

The categorical separation between placements rests on attributing the seventeen recorded crashes to curvature inside a trainable adapter rather than to uncontrolled engineering conditions (rank-16 adapters, bf16, optimizer hyperparameters, initialisation, and a missing warmup), with no matched controlled replication of the crashes using identical hyperparameters.

What would settle it

Run the deposited crash-reproduction script with the rank-16 adapter-on-manifold configuration under the authors' documented settings, with and without the 250-step manifold warmup; if either configuration trains to completion at 7–8B scale without NaN, the categorical claim is refuted. The deposited per-step traces of the six HySAT runs also allow a direct NaN-event recount without access to trained weights.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Curvature-aware fine-tuning of pretrained transformers does not require rebuilding or curving the network; a corrective loss at the output layer suffices to inject tree-structured supervision at the 18M-sample scale.
  • Adapter-on-manifold placement is claimed to be categorically unstable under standard curvature-unaware AdamW: no hyperparameter choice within that regime can restore convergence, because the smoothness constant diverges (Proposition 3).
  • Batch size is a structural-learning-quality variable under hyperbolic supervision: below b ≈ 8–16 the structural losses are silently absent, so micro-batch size and ontology-tree density must be reported jointly.
  • The matched four-arm ablation shows the controlled mechanism is invariant preservation (Lorentz drift ~10^-6 vs ~10^4), not downstream task gain at toy scale; task-level gains are carried by the six deployed models.
  • The 17 crash incidents are repositioned as elimination evidence that specifies the failure mode and the stabilisation conditions: curvature-aware optimisation, explicit B-norm bounding, or retraction with full-sequence loss and batch ≥ 32.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: If the placement law holds beyond this corpus, it predicts that any hyperbolic operation introduced into trainable parameters of large models — not just adapters — will hit the same Jacobian-amplification wall at similar precision boundaries; a direct extension would compare loss-only vs. full-hyperbolic decoders at matched scale with matched hyperparameters.
  • Inference: The paper's own clamping choice (‖v‖ ≤ 2) trades away radial depth information for stability; with modern residual streams showing heavy-tailed norms (p99 up to 10^3–10^4), an implicit consequence is that per-layer or per-token adaptive τ could recover depth signal while preserving the bounded-Jacobian guarantee.
  • Inference: The strongest corroborating test is a pre-registered replication by an independent group running the same rank-16 adapter-on-manifold configuration with matched hyperparameters, including the 250-step manifold warmup documented in the Methods; if a hyperparameter-level fix stabilizes adapter-on-manifold training at scale, the 'placement law' would reduce to an engineering anecdote.
  • Inference: The silent-failure dichotomy suggests a general auditing tool: activation-rate logging of structural losses could be adopted as a standard health check for fine-tuning pipelines, since loss curves can descend while structural supervision is entirely absent.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes HySAT (Hyperbolic Structure-Aware Training): it injects tree-structured domain supervision into pretrained Euclidean transformers by placing Lorentz-geometry losses (PTLE, HLSD, HWC) at the loss layer only, keeping all trainable parameters Euclidean. The central claim is a 'placement law': the same hyperbolic geometry collapses training when placed inside a trainable adapter (seventeen self-reported crashes in earlier HyperLoRA-style runs, ~220 B200 GPU-hours), yet trains stably when placed only at the loss layer (zero NaN across ~317K cumulative optimizer steps in six complete runs). Formal support is offered by Proposition 1(i) (a Euclidean-LoRA adapter norm bound independent of Lorentz curvature), Proposition 1(ii) (a conditional failure-mode analysis for adapter-on-manifold placement with escape conditions ii.a-ii.c), Lemma 1, Proposition 2 (batch-size pair-formation threshold for HLSD/PTLE activation), and Proposition 3 (a standard non-convex SGD rate for the clamped HySAT objective). Six expert SLMs are constructed across two base families and four adapter strategies; a matched four-arm ablation on a toy ontology shows manifold-invariant preservation (drift ~1e-6 vs ~1e4) but no separation on held-out tree geometry. The paper is unusually transparent: deposited per-step traces, a seventeen-incident failure ledger, a crash-reproduction script, a verified compute bill, pre-specified hypotheses H1-H6, and explicit refutation conditions.

Significance. If the placement claim holds, HySAT is a genuinely useful design rule: a minimal, stable way to inject hierarchical structure into pretrained Euclidean transformers at scale. The formal core is sound — Proposition 1(i) is a clean consequence of gradient clipping plus loss-only projection, and the bounded-Jacobian constant M_tau ~ 5.23/tau derived from the clamp ||v||<=2 is a neat, checkable calculation. The batch-size-as-structural-variable observation (Proposition 2) is original and easy to test independently. The reproducibility provisions are a real strength: per-step traces, a failure ledger, a crash-reproduction script, disclosed compute cost, and explicit falsification conditions mean the claim is checkable even where not yet fully demonstrated. The weaknesses are not in the formal core but in the load-bearing empirical separation (an unmatched crash history; a fourth stabilizer, the 250-step manifold warmup of §4.2, outside Proposition 1(ii)'s conditions; a task-insensitive ablation) and in claims language that outruns the evidence ('law,' 'first,' '1,139x'). The defensible version of the claim — loss-only placement is a simple sufficient stabilizer for hyperbolic supervisio

major comments (4)
  1. [§2.1 / §4.2 (Prop. 1(ii) vs shared training protocol)] The categorical 'placement law' rests on an uncontrolled comparison. Methods §4.2 states that all current runs use, as part of the shared protocol, a 250-step Euclidean-to-Lorentz scheduling 'that prevents the gradient-death mode observed in early adapter initialisations on the manifold' — a fourth stabilizer not among conditions (ii.a)-(ii.c) of Proposition 1(ii). The 17 crashes occurred in earlier project versions (rank-16 HyperLoRA, early variants; Fig. 3 caption), while the stable runs use rank-64 plus the warmup; the paper does not report whether the crash runs used the warmup, nor does it match rank, batch size, AdamW settings, or initialization. Since two current-protocol HyperLoRA runs (MetaTeach v14, AdmitBrain v6) trained to completion with zero NaN, adapter-on-manifold at scale is not unstable per se. The 'law' framing of the abstract/§1.3 exceeds the evidence; the conditional
  2. [§6.3 / Extended Data Table 5 (four-arm ablation)] The only controlled comparison does not separate the arms on any task-level metric: held-out tree-distance correlation is 0.228±0.048 (HySAT) vs 0.224±0.041 (flat regularizer) vs 0.253±0.033 (supervision-free baseline), overlapping within one standard error. The stated mechanism contrast — manifold drift 1e-6 vs 1e4 — is by construction: the flat arm is never projected onto the Lorentz manifold, so it cannot preserve the invariant; this is a sanity check, not an empirical discovery. The manuscript concedes the probe 'isolates the mechanism … rather than a downstream gain,' but the consequence is that the benefit claim — tree structure injected via hyperbolic loss — rests solely on the six deployed models, which lack a matched Euclidean-supervised control at equal samples and steps. As written, no controlled experiment demonstrates that hyperbolic placement improves the expert SLMs; the a
  3. [§4.1 / §6 (failure ledger and Prop. 1(ii) support)] The empirical cornerstone of Proposition 1(ii) is the authors' own seventeen-crash ledger: 'We learned this question by failing on it' (§1.2). The incidents are self-reported and self-attributed to curvature inside trainable weights; the one preserved raw trajectory 'records the unrecoverable divergence-plateau signature rather than a terminal NaN entry' (§4.1); and no matched replication of the crashes under current hyperparameters (rank-64, warmup, present optimizer/batch settings) with adapter-on-manifold placement is reported, even though §2.7's refutation condition names exactly this experiment. The deposited crash-reproduction script is welcome, but as presented the failure mode and the theory are not independent: Propositions 1-3 formalize the same ledger that serves as their only empirical support. This is fixable in scope — run the matched crash-reproduction set — and the paper
  4. [§4.1 (ii.a-ii.c) vs Fig. 3 caption / §2.3] The paper asserts that the two successful HyperLoRA-family runs 'each satisfy exactly one of (ii.a)-(ii.c),' but the stabilizers described for them — AdmitBrain v6 'batch 2→16' and MetaTeach v14 'HWC scaffolding from start' (Fig. 3 caption) — do not clearly satisfy the stated conditions: (ii.c) requires retraction with full-sequence loss and effective batch≥32, while Table 2 lists AdmitBrain at b=16, and HWC scaffolding is a loss-side change, not one of the three conditions. The mapping from the described stabilizers to the escape clauses cannot be verified from the main text (details relegated to §13.4-§13.6). Since these two runs are the authors' own counterexamples to 'adapter-on-manifold collapses,' the claim that they are 'consistent with Proposition 1(ii)' is load-bearing and must be made explicit and checkable.
minor comments (6)
  1. [§2.3 / Extended Data Table 2] The '~317K cumulative optimizer steps' headline mixes units: optimizer steps (ORAA 180,000+, AdmitBrain 12,313), epochs/rounds (BS Sovereign 0.50 ep, S3 266 rounds), and a '537K proc.' figure. The aggregate is not auditable from the paper's own tables; Supplementary Table 4 should give a single normalized step count per run so the zero-NaN denominator is checkable.
  2. [§2.1 / §4.1 (Prop. 1(ii), Jacobian formula)] The stated Jacobian norm sinh(||B·x||)/||B·x|| is the angular singular value of D exp_0; the operator norm is sqrt(cosh(2||v||)), growing as e^{||v||}. At ||v||=11 the quoted amplification ~2.7e3 understates the operator norm by roughly an order of magnitude (~3e4). The qualitative super-polynomial growth and the practical bf16-boundary conclusion are unaffected.
  3. [§4.1 (tau and clamp)] The acknowledged trade-off — that the clamp compresses the radial (depth) coordinate so the structural losses operate 'primarily on angular tree relations rather than on depth' — materially qualifies the abstract's claim that HySAT 'teaches' tree hierarchies. This caveat should appear in the main-text results (§2.3/§2.7) rather than only in Methods.
  4. [§2.6-2.7 / Extended Data Table 4] The scale ratios (1,139x/1,795x) compare raw training-sample counts across different base models, domains, adapter strategies, and training registers; they are descriptive corpus-size ratios, not matched measurements, and the abstract/Fig. 6 should label them as such to avoid implying a controlled efficiency comparison.
  5. [§4.4 (meta-capability corpus sharing)] The paper admits that four projects share meta-capability samples, so the cross-project activation regularity (Claim 5) carries this confound; the exclusion robustness check is deferred to future work. Given that Claim 5 is one of the six 'structural firsts,' this caveat deserves more than a parenthetical in §4.4 — at minimum it should be restated beside Claim 5 in Extended Data Table 4.
  6. [§4.7 / Data availability] The Zenodo deposit is 'reserved and activated on publication,' so the per-step traces, failure ledger, and verify_claims.py cannot be inspected at review time. For a manuscript whose central evidence is deposited artifacts, the editor should exercise the gated research-access pathway before decision; the data-availability section should state that reviewer access is available on request.

Circularity Check

0 steps flagged

No circular derivation found: the placement law is formally conditional and empirically grounded; the crash-ledger comparison is confounded but not definitionally circular.

full rationale

Walking the derivation chain, the formal results do not reduce to their inputs by construction. Proposition 1(i) bounds the Euclidean LoRA output matrix B_t by ||B_0||_F + κ·Ση_s under clipped Euclidean AdamW; since the adapter never leaves R^d, Lorentz curvature never enters the update rule, and the bound is derived from the stated update assumptions rather than from the conclusion. Proposition 1(ii) derives the sinh(||B·x||)/||B·x|| Jacobian amplification for naive adapter-on-manifold placement and explicitly conditions failure on the absence of (ii.a) curvature-aware optimisation, (ii.b) B-norm bounding, or (ii.c) retraction with full-sequence loss and batch ≥ 32; the paper even states 'The proposition is conditional.' Lemma 1 and Proposition 3 are Lipschitz/smoothness calculations that presuppose, rather than establish, the bounded-Jacobian property M_τ; they do not assume the empirical stability they are used to explain. The empirical cornerstone — 17 historical crashes vs. zero NaN over ~317K steps — is a self-reported ledger, and it is a real confound that Methods §4.2 admits a 250-step Euclidean-to-Lorentz warmup 'that prevents the gradient-death mode observed in early adapter initialisations on the manifold,' with no report of whether the 17 crashes used this warmup or matched rank/hyperparameters to the stable runs. That weakness undermines the causal attribution to placement, but it is a correctness/confounding problem, not circularity: no equation is fitted to the outcome, no parameter is renamed as a prediction, and no load-bearing step is justified solely by a self-citation. The companion-manuscript self-references are declared and explicitly non-load-bearing ('This manuscript is logically self-contained'). Therefore no specific circular reduction can be exhibited, and the honest finding is no significant circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 7 axioms · 2 invented entities

The paper's contributions rest largely on standard differential geometry and convex optimization, but several hand-chosen quantities (τ=2, clamp=2, λ per project, bf16 boundary ≈11, 250-step warmup, drift tolerance) are introduced ad hoc to make the stability story work. The domain assumption 'expert domains are trees' is asserted rather than measured. No new physical or mathematical entity with independent evidence is introduced; the HySAT losses and the razor cell are postulates whose independent advantage is not demonstrated by controlled data.

free parameters (6)
  • Lorentz scale τ = 2 (fixed across all six projects)
    Normalization scale for exp_0(h/τ); no sweep performed; the paper itself lists a τ∈{1,2,4,8} sweep as a priority replication target (§4.1).
  • clamp ‖v‖≤2 = 2
    Projects mid-stack inputs to cosh(2)≈3.76; admitted to compress the radial (depth) coordinate so losses operate mainly on angular relations (§4.1).
  • loss weights λ_PTLE, λ_HLSD, λ_HWC = ≈0.02–0.05 per project; λ_PTLE=0.05 constant across LoRA projects
    Per-project weights with no sensitivity analysis reported; hyperparameters vary by project per Supplementary Table 2 (§4.1–§4.2).
  • practical bf16 instability boundary ‖v‖≈11 = ≈11 (absolute cosh overflow ≈89)
    Empirical threshold inferred from the 17 crash records and used to explain the adapter NaN cascade (§2.1, §4.1, SI-E).
  • manifold warmup length = 250 steps
    Euclidean-to-Lorentz ramp introduced to prevent gradient-death in adapter initializations (SI-K.1); an additional stabilizer beyond the three Prop 1(ii) conditions.
  • manifold-drift tolerance = 1e-4
    Threshold below bf16 per-operation roundoff, chosen to flag structural drift; rescaling triggered above it (§4.1).
axioms (7)
  • standard math Lorentz manifold definitions and exponential map exp_0(v) = (cosh‖v‖, sinh‖v‖·v/‖v‖); Minkowski inner product ⟨x,y⟩_L = −x_0y_0 + Σx_iy_i
    Invoked in §4.1 to define the projection and Lorentz distances used by all three losses.
  • standard math Finite trees embed in hyperbolic space with arbitrarily low distortion at modest dimension [Sarkar 2011; Nickel & Kiela 2017]
    Grounds the claim that tree structure is naturally representable on the manifold (§1.2).
  • domain assumption Expert domains are trees
    The motivating premise of the entire paper (§1.1, §4.4); every deployed ontology is modeled as a rooted tree, with no evidence that non-tree domain structure would benefit.
  • domain assumption Euclidean transformers dilute parent-child structure exponentially at depth; attention is extrapolative, TC0-bounded, and compositionally decaying [2,3,5]
    The 'Euclidean highway wall' (§1.1). The cited results concern compositional reasoning and extrapolation; the bridge to expert fine-tuning is the authors' inference.
  • domain assumption bf16 arithmetic with cosh overflow near argument 89 and a 'practical Jacobian-amplification boundary' near 11
    Used to forecast NaN cascades in adapter-on-manifold placement (§2.1, §4.1, SI-E.2).
  • standard math Gradient clipping gives E‖B_t‖_F ≤ ‖B_0‖_F + κΣη_s (standard SGD/LoRA bound)
    Proposition 1(i) restates the Euclidean LoRA bound for HySAT; the bound is known from the LoRA literature [20].
  • standard math Poisson approximation for pair formation P(b) ≈ 1−exp(−C(b,2)(p̄−1)/(N−1))
    Proposition 2's birthday-problem computation under uniform-leaf sampling; heterogeneity correction via Cauchy–Schwarz (§5).
invented entities (2)
  • HySAT loss trio (PTLE, HLSD, HWC) no independent evidence
    purpose: Inject tree-structure supervision through a Lorentz-projected loss at the loss layer only
    New loss formulations introduced by the paper. Their advantage over flat Euclidean tree-regularizers is not shown by the controlled ablation (Ext Data Table 5: ρ_L 0.228 vs 0.224 flat vs 0.253 CE-only baseline), so there is no falsifiable handle outside the paper.
  • the 'razor cell' (Q1–Q4 taxonomy) no independent evidence
    purpose: Motivational taxonomy locating a claimed structurally-empty cell for consequential, tacit domain expertise
    Defined in a companion manuscript under review; no falsifiable handle in this paper; not load-bearing for the stability claim.

pith-pipeline@v1.3.0-alltime-deepseek · 5770 in / 5907 out tokens · 170176 ms · 2026-08-01T17:45:19.269669+00:00 · methodology

0 comments
read the original abstract

Expert domains are trees; the Euclidean transformer is not, diluting parent-child structure exponentially at depth. The hyperbolic turn left one question unasked: not how much of a network to curve, but where curvature may touch the gradient. Placement is a law, not a knob: the same geometry on a trainable adapter collapses training (seventeen training collapses, ~220 GPU-hours), yet at the loss layer alone it trains without one -- this is HySAT (Hyperbolic Structure-Aware Training), hyperbolic losses at the loss layer only. Across six expert SLMs we constructed and deployed (Llama 3.1 and EXAONE 3.5; four adapter strategies; 18.0M-sample corpus; zero NaN over ~317K optimizer steps), a matched four-arm ablation isolates the preserved manifold invariant, and three propositions and a lemma prove why loss-only placement is stable where adapter-on-manifold is not. Four models are operationally deployed (one live, consumer-facing), two open-weight, with per-step traces and a seventeen-incident failure ledger on Zenodo (CC-BY-4.0).

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 5 canonical work pages

  1. [1]

    In: Advances in Neural Information Processing Systems (2017)

    Vaswani, A., Shazeer, N., Parmar, N.,et al.: Attention is all you need. In: Advances in Neural Information Processing Systems (2017)

    2017

  2. [2]

    arXiv preprint arXiv:2110.09485 (2021)

    Balestriero, R., Pesenti, J., LeCun, Y.: Learning in high dimension always amounts to extrapolation. arXiv preprint arXiv:2110.09485 (2021)

    Pith/arXiv arXiv 2021

  3. [3]

    Transactions of the Association for Computational Linguistics11, 531–545 (2023)

    Merrill, W., Sabharwal, A.: The parallelism tradeoff: Limitations of log-precision transformers. Transactions of the Association for Computational Linguistics11, 531–545 (2023)

    2023

  4. [4]

    In: International Conference on Learning Representations (2024)

    Merrill, W., Sabharwal, A.: The expressive power of transformers with chain of thought. In: International Conference on Learning Representations (2024)

    2024

  5. [5]

    In: Advances in Neural Information Processing Systems (2023)

    Dziri, N., Lu, X., Sclar, M.,et al.: Faith and fate: Limits of transformers on compositionality. In: Advances in Neural Information Processing Systems (2023)

    2023

  6. [6]

    Trends in Cognitive Sciences20(2), 87–95 (2016)

    Beaty, R.E., Benedek, M., Silvia, P.J., Schacter, D.L.: Creative cognition and brain network dynamics. Trends in Cognitive Sciences20(2), 87–95 (2016)

    2016

  7. [7]

    Proceedings of the National Academy of Sciences115(5), 1087–1092 (2018)

    Beaty, R.E., Kenett, Y.N., Christensen, A.P.,et al.: Robust prediction of individ- ual creative ability from brain functional connectivity. Proceedings of the National Academy of Sciences115(5), 1087–1092 (2018)

    2018

  8. [8]

    arXiv preprint arXiv:2108.07258 (2021) 26

    Bommasani, R., Hudson, D.A., Adeli, E., et al.: On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258 (2021) 26

    Pith/arXiv arXiv 2021

  9. [9]

    Nature Machine Intelligence1, 389–399 (2019) https://doi.org/10.1038/ s42256-019-0088-2

    Jobin, A., Ienca, M., Vayena, E.: The global landscape of AI ethics guide- lines. Nature Machine Intelligence1, 389–399 (2019) https://doi.org/10.1038/ s42256-019-0088-2

    2019

  10. [10]

    Nature Machine Intelligence5, 796–797 (2023) https://doi

    Roberts, S.G., Reidy, C., Keyes, O.: Foundation models and the privatization of public knowledge. Nature Machine Intelligence5, 796–797 (2023) https://doi. org/10.1038/s42256-023-00695-5

    work page doi:10.1038/s42256-023-00695-5 2023

  11. [11]

    In: International Symposium on Graph Drawing (2011)

    Sarkar, R.: Low distortion Delaunay embedding of trees in hyperbolic plane. In: International Symposium on Graph Drawing (2011)

    2011

  12. [12]

    In: Advances in Neural Information Processing Systems, vol

    Nickel, M., Kiela, D.: Poincar´ e embeddings for learning hierarchical represen- tations. In: Advances in Neural Information Processing Systems, vol. 30, pp. 6341–6350 (2017)

    2017

  13. [13]

    In: Advances in Neural Information Processing Systems (NeurIPS), vol

    Ganea, O.-E., B´ ecigneul, G., Hofmann, T.: Hyperbolic neural networks. In: Advances in Neural Information Processing Systems (NeurIPS), vol. 31 (2018)

    2018

  14. [14]

    In: Advances in Neural Information Processing Systems (2019)

    Chami, I., Ying, R., R´ e, C., Leskovec, J.: Hyperbolic graph convolutional neural networks. In: Advances in Neural Information Processing Systems (2019)

    2019

  15. [15]

    In: International Conference on Learning Representations (2021)

    Shimizu, R., Mukuta, Y., Harada, T.: Hyperbolic neural networks++. In: International Conference on Learning Representations (2021)

    2021

  16. [16]

    In: Advances in Neural Infor- mation Processing Systems (NeurIPS 2025) (2025)

    Yang, M., Ram Samarth, B.B., Feng, A., Xiong, B., Liu, J., King, I., Ying, R.: Hyperbolic fine-tuning for large language models. In: Advances in Neural Infor- mation Processing Systems (NeurIPS 2025) (2025). arXiv:2410.04010 (Oct 2024); accepted to NeurIPS 2025

    arXiv 2025

  17. [17]

    arXiv preprint arXiv:2505.24722 (2025)

    He, N., Anand, R., Madhu, H., et al.: HELM: Hyperbolic large language models via mixture-of-curvature experts. arXiv preprint arXiv:2505.24722 (2025)

    arXiv 2025

  18. [18]

    NeurIPS 2025 (oral); arXiv:2510.20322 (2025)

    Peng, Z., Xu, Z., Liu, Q., Yang, X., Shen, W.: HyperET: Efficient Training in Hyperbolic Space for Multi-modal Large Language Models. NeurIPS 2025 (oral); arXiv:2510.20322 (2025)

    arXiv 2025

  19. [19]

    Nature Machine Intelligence5, 220–235 (2023) https://doi.org/10.1038/s42256-023-00626-4

    Ding, N., Qin, Y., Yang, G.,et al.: Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature Machine Intelligence5, 220–235 (2023) https://doi.org/10.1038/s42256-023-00626-4

    doi:10.1038/s42256-023-00626-4 2023

  20. [20]

    In: International Conference on Learning Representations (2022)

    Hu, E.J., Shen, Y., Wallis, P.,et al.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (2022)

    2022

  21. [21]

    In: International Conference on Machine Learning (ICML) (2023)

    Mishne, G., Wan, Z., Wang, Y., Yang, S.: The numerical stability of hyper- bolic representation learning. In: International Conference on Machine Learning (ICML) (2023). arXiv:2211.00181

    Pith/arXiv arXiv 2023

  22. [22]

    In: International Conference on Learning Representations (ICLR) (2026)

    Klein, T., Lang, T., Shkabrii, A., Sturm, A., Sidak, K., Miklautz, L., Plant, C., 27 Velaj, Y., Tschiatschek, S.: Understanding and improving hyperbolic deep rein- forcement learning. In: International Conference on Learning Representations (ICLR) (2026). arXiv:2512.14202

    arXiv 2026

  23. [23]

    Physica D: Nonlinear Phenomena234(1), 70–80 (2007)

    Cafaro, C., Ali, S.: Jacobi fields on statistical manifolds of negative curvature. Physica D: Nonlinear Phenomena234(1), 70–80 (2007)

    2007

  24. [24]

    IEEE Transactions on Automatic Control58(9), 2217–2229 (2013)

    Bonnabel, S.: Stochastic gradient descent on Riemannian manifolds. IEEE Transactions on Automatic Control58(9), 2217–2229 (2013)

    2013

  25. [25]

    Princeton University Press, Princeton, NJ (2008)

    Absil, P.-A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton, NJ (2008)

    2008

  26. [26]

    Macmillan, London (1964)

    Koestler, A.: The Act of Creation. Macmillan, London (1964)

    1964

  27. [27]

    David McKay, New York (1956)

    Bloom, B.S.: Taxonomy of Educational Objectives. David McKay, New York (1956)

    1956

  28. [28]

    Har- vard Educational Review57(1), 1–23 (1987) https://doi.org/10.17763/haer.57.1

    Shulman, L.S.: Knowledge and teaching: Foundations of the new reform. Har- vard Educational Review57(1), 1–23 (1987) https://doi.org/10.17763/haer.57.1. j463w79r56455411

    work page doi:10.17763/haer.57.1 1987

  29. [29]

    The Journal of Psychology36(2), 311–322 (1953)

    Stein, M.I.: Creativity and culture. The Journal of Psychology36(2), 311–322 (1953)

    1953

  30. [30]

    arXiv preprint arXiv:2509.05757 (2025)

    Patil, S., Zhang, Z., Huang, Y., Ma, T., Xu, M.: Hyperbolic large language models. arXiv preprint arXiv:2509.05757 (2025)

    arXiv 2025

  31. [31]

    arXiv preprint arXiv:2407.21783 (2024)

    Grattafiori, A., Dubey, A., Jauhri, A., et al.: The Llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

    Pith/arXiv arXiv 2024

  32. [32]

    arXiv preprint arXiv:2412.04862 (2024)

    LG AI Research: EXAONE 3.5: Series of large language models for real-world use cases. arXiv preprint arXiv:2412.04862 (2024)

    arXiv 2024

  33. [33]

    In: International Conference on Learning Representations (2018)

    Smith, S.L., Kindermans, P.-J., Ying, C., Le, Q.V.: Don’t decay the learning rate, increase the batch size. In: International Conference on Learning Representations (2018). arXiv:1711.00489

    Pith/arXiv arXiv 2018

  34. [34]

    Nature Machine Intelligence6, 354–367 (2024) https://doi.org/10

    Pai, S., Bontempi, D., Hadzic, I.,et al.: Foundation model for cancer imaging biomarkers. Nature Machine Intelligence6, 354–367 (2024) https://doi.org/10. 1038/s42256-024-00807-9

    2024

  35. [35]

    Nature Machine Intelligence6, 962–974 (2024) https://doi.org/ 10.1038/s42256-024-00876-w

    Feng, B., Liu, Z., Huang, N.,et al.: A bioactivity foundation model using pairwise meta-learning. Nature Machine Intelligence6, 962–974 (2024) https://doi.org/ 10.1038/s42256-024-00876-w

    work page doi:10.1038/s42256-024-00876-w 2024

  36. [36]

    Nature Machine Intelligence7, 942–953 28 (2025) https://doi.org/10.1038/s42256-025-01044-4

    He, Y., Fang, P., Shan, Y.,et al.: Generalized biological foundation model with unified nucleic acid and protein language. Nature Machine Intelligence7, 942–953 28 (2025) https://doi.org/10.1038/s42256-025-01044-4

    work page doi:10.1038/s42256-025-01044-4 2025

  37. [37]

    Nature Machine Intelligence7, 1823–1833 (2025) https://doi.org/10.1038/s42256-025-01137-0

    Xiao, C., Cai, J., Zhao, W.,et al.: Densing law of LLMs. Nature Machine Intelligence7, 1823–1833 (2025) https://doi.org/10.1038/s42256-025-01137-0

    doi:10.1038/s42256-025-01137-0 2025

  38. [38]

    Nature Machine Intelligence7, 1742– 1754 (2025) https://doi.org/10.1038/s42256-025-01127-2

    Tolmachev, P., Engel, T.A.: Single-unit activations confer inductive biases for emergent circuit solutions to cognitive tasks. Nature Machine Intelligence7, 1742– 1754 (2025) https://doi.org/10.1038/s42256-025-01127-2

    work page doi:10.1038/s42256-025-01127-2 2025

  39. [39]

    In: Interna- tional Conference on Machine Learning (2023)

    Desai, K., Nickel, M.,et al.: Hyperbolic image-text representations. In: Interna- tional Conference on Machine Learning (2023)

    2023

  40. [40]

    arXiv preprint arXiv:2506.10710 (2025)

    Ayoughi, M., Ghadimi Atigh, M., Derakhshani, M.M., Snoek, C.G.M., Mettes, P., Groth, P.: Continual hyperbolic learning of instances and classes. arXiv preprint arXiv:2506.10710 (2025)

    Pith/arXiv arXiv 2025

  41. [41]

    Nature Machine Intelligence7, 195–204 (2025) https://doi.org/10.1038/ s42256-024-00975-8

    Akiba, T., Shing, M., Tang, Y.,et al.: Evolutionary optimization of model merging recipes. Nature Machine Intelligence7, 195–204 (2025) https://doi.org/10.1038/ s42256-024-00975-8

    2025

  42. [42]

    Jahresbericht der Deutschen Mathematiker-Vereinigung18, 75–88 (1908)

    Minkowski, H.: Raum und Zeit (Space and Time). Jahresbericht der Deutschen Mathematiker-Vereinigung18, 75–88 (1908)

    1908

  43. [43]

    arXiv preprint arXiv:2208.07339 (2022)

    Dettmers, T., Lewis, M., Belkada, Y., Zettlemoyer, L.: LLM.int8(): 8-bit matrix multiplication for transformers at scale. arXiv preprint arXiv:2208.07339 (2022)

    Pith/arXiv arXiv 2022

  44. [44]

    arXiv preprint arXiv:2401.12181 (2024)

    Gurnee, W., Horsley, T., Guo, Z.C., Rezaei Kheirkhah, T., Sun, Q., Hathaway, W., Nanda, N., Bertsimas, D.: Universal neurons in GPT2 language models. arXiv preprint arXiv:2401.12181 (2024)

    Pith/arXiv arXiv 2024

  45. [45]

    SIAM Review60(2), 223–311 (2018)

    Bottou, L., Curtis, F.E., Nocedal, J.: Optimization methods for large-scale machine learning. SIAM Review60(2), 223–311 (2018)

    2018

  46. [46]

    SIAM Journal on Optimization23(4), 2341–2368 (2013)

    Ghadimi, S., Lan, G.: Stochastic first- and zeroth-order methods for nonconvex stochastic programming. SIAM Journal on Optimization23(4), 2341–2368 (2013)

    2013

  47. [47]

    In: Advances in Neural Information Processing Systems (NeurIPS) (2024)

    Roulet, V., Agarwala, A., Grill, J.-B., Swirszcz, G., Blondel, M., Pedregosa, F.: Stepping on the edge: Curvature-aware learning rate tuners. In: Advances in Neural Information Processing Systems (NeurIPS) (2024)

    2024

  48. [48]

    In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

    Ermolov, A., Mirvakhabova, L., Khrulkov, V., Sebe, N., Oseledets, I.: Hyperbolic vision transformers: Combining improvements in metric learning. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2022)

    2022

  49. [49]

    Transactions on Machine Learning Research (2024)

    Biderman, D., Portes, J., Ortiz, J.J.G., Paul, M., Greengard, P., Jennings, C., King, D., Havens, S., Chiley, V., Frankle, J., Blakeney, C., Cunningham, J.P.: LoRA learns less and forgets less. Transactions on Machine Learning Research (2024). arXiv:2405.09673 29 Acknowledgements We thank thearcsmallaiengineering collective for the shared hyperbolic engin...

    Pith/arXiv arXiv 2024

  50. [2026]

    training samples

    is disclosed in Methods§4.6 and§12.5. Author contributions K.S.S. formulated the HySAT principle, designed and supervised the six training runs, derived Propositions 1–3 (HySAT Stability, Pair Formation, and Convergence Rate) and Lemma 1 (Stability-Dominant Placement), conducted the batch-size nat- ural experiments, executed the SI-C.3 hidden-state-norm m...

    2026