Pith. sign in

REVIEW 3 major objections 4 minor 83 references

Keep your distance: learning dispersed embeddings on $\mathbb{S}_m$

T0 review · 3 major / 4 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a sliced great-circle regularizer and an online Lloyd quantization regularizer can substitute for costly pairwise dispersion on the hypersphere, and that Riemannian optimization of these objectives improves…

desk verdict Useful dispersion-regularizer paper with a genuinely new Sliced objective; the main caveat is an unquantified clipping heuristic that could drive the high-dimensional gains. read the letter →

arxiv 2502.08231 v4 pith:P4XQTLMR submitted 2025-02-12 cs.LG

Evgeniia Tokarchuk , Hua Chang Bakker , Vlad Niculae This is my paper · ORCID
classification cs.LG
keywords dispersionhypersphereembeddingregularizationRiemannianoptimizationslicedLloyd'salgorithmmaximummeandiscrepancyTammesproblem
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Embeddings trained on the hypersphere tend to clump, and standard cures push unrelated vectors apart by minimizing pairwise distances at quadratic cost. The paper argues that two cheaper mechanisms achieve comparable dispersion: an online stochastic version of Lloyd's algorithm that treats the embedding vectors as quantizing centers for the uniform spherical measure, and a sliced regularizer that projects the configuration onto random great circles and measures, in closed form, how far the projected angles are from an optimally spaced circle. It also recasts kernel dispersion energy as squared maximum mean discrepancy (MMD) between the embedding distribution and the uniform measure on $\mathbb{S}_m$. Across synthetic Tammes benchmarks, prototype-based image classification, discrete machine translation, and continuous-output machine translation, the paper reports that these regularizers — especially Sliced in clumped high-dimensional regimes — improve dispersion metrics and task accuracy, and that Riemannian rather than projected Euclidean gradient updates matter.

What carries the argument

The central object is the great-circle slicing construction. A great circle is the intersection of the hypersphere with a plane through its center; each point $x\in\mathbb{S}_m$ projects onto it as an angle $\theta=\arctan_2(\langle x,q\rangle,\langle x,p\rangle)$. Lemma 3 gives the projection onto the set of optimally dispersed circular configurations: sort the projected angles, match them in order to the equal-spaced ideal angles, and shift all angles by their mean. Proposition 2 turns this projection into the gradient in Eq. (13), so that stochastic Riemannian gradient descent can optimize $L_{\mathrm{Sliced}}$. The Lloyd regularizer is the other load-bearing object: with the sphere as the target measure, each embedding is pulled toward the barycenter of uniform samples assigned to it, and its gradient is bounded, which permits a stochastic Riemannian convergence argument. Riemannian projection of gradients, $(I-xx^\top)\nabla$, is the mechanism that keeps both objectives and the baseline pairwise objectives well-behaved on the sphere.

What would settle it

In the clumped synthetic setting of Section 4.2, run the Sliced regularizer with clipping thresholds varying over several orders of magnitude, and also with the denominator $\langle x_i,Q\rangle^2+\langle x_i,P\rangle^2$ replaced by a smoothed $+\varepsilon$ version; if the final spherical variance and minimum distance change substantially with the threshold, the reported Sliced success comes from the clipping heuristic rather than from the dispersion objective itself.

Watch

Extended reading notes

Core claim

The paper's central claim is that dispersion on the hypersphere — the even covering of the sphere by a set of embedding vectors — can be optimized without forming the full pairwise-distance matrix. Kernel energy objectives are shown in Lemma 2 to be unbiased estimators of squared MMD against the uniform distribution, with the uniform terms reducing to a constant. The paper then introduces two alternatives: an online stochastic Lloyd regularizer that pulls each embedding toward the barycenter of uniformly sampled points in its Voronoi cell, and a sliced regularizer $L_{\mathrm{Sliced}}$ that projects the configuration onto random great circles and penalizes the squared distance of the projected angles from the set of optimally spaced circular configurations. In the synthetic and real-world evaluations, the paper argues that these regularizers approximate Tammes-optimal dispersion and improve prototype classification, discrete NMT, and continuous-output NMT, with Sliced making the fastest progress in clumped high-dimensional regimes. Across experiments it argues that Riemannian gradient updates, rather than Euclidean updates followed by projection, are systematically better.

Load-bearing premise

The claim depends on the assumption that the heuristic clipping of the Sliced regularizer's gradient does not materially change what the objective does, since that gradient becomes unbounded when an embedding nears the equator of a sampled great circle and the standard stochastic-convergence argument no longer applies.

Editorial extensions

If this is right

  • Dispersion regularization becomes feasible at vocabulary scale: Lloyd costs $O(nm)$ and Sliced costs $O(nm+\mathrm{sort}(n))$ per Monte Carlo sample, versus $O(n^2m)$ for pairwise objectives.
  • Riemannian optimization of decoder embeddings, even without an extra regularizer, is claimed to reduce rare-token collapse; the paper reports an order-of-magnitude larger gradient norm for rare tokens and better BLEU than the Euclidean baseline.
  • In clumped, high-dimensional regimes the Sliced regularizer makes the fastest initial progress on spherical variance, while max-min objectives keep improving minimum distance once points are already spread, so the right regularizer depends on the initialization regime.
  • Because kernel energy objectives are squared MMD against the uniform spherical measure, their minibatch variance and concentration can be analyzed with the MMD toolkit.
  • The prototype experiments show that minimum distance alone does not predict downstream accuracy; at output dimension equal to the number of classes, MHE approaches near-orthogonal solutions that underperform despite good separation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: a smoothed variant of the Sliced gradient that replaces the unbounded denominator in Eq. (13) with a small regularizer would likely recover a stochastic-convergence guarantee while preserving the objective, giving a principled alternative to clipping.
  • Editorial inference: the slicing recipe should transfer to other homogeneous spaces with known one-dimensional optimal configurations, such as tori or projective spaces, by replacing the great-circle projection with the corresponding geodesic projection.
  • Editorial inference: the MMD equivalence suggests a statistical use of dispersion measures, namely a post-training uniformity test on the learned embeddings to detect clumping, complementing the paper's spherical-variance and minimum-distance metrics.
  • Editorial inference: the single-great-circle estimator is unbiased but may be high-variance in high dimensions; stratifying the sampled great circles (for example over axis-aligned directions) could lower variance at the same sample count.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper studies dispersion regularizers for embeddings constrained to the unit sphere S^m. It contributes (i) an interpretation of kernel-based dispersion objectives as squared MMD against the uniform distribution (Lemma 2, Prop. 1), (ii) a stochastic Lloyd-type regularizer for dispersion, and (iii) a "sliced dispersion" objective (Eq. 12) that measures the distance of projected angles to the optimally dispersed configuration on a great circle, with a gradient formula (Eq. 13). The authors evaluate these methods on the Tammes problem (n=24, m=3), synthetic high-dimensional settings with uniform and clumped initializations, ImageNet-200 prototype classification, and discrete and continuous-output neural machine translation. The central empirical claims are that Riemannian optimization improves dispersion and downstream performance, that Sliced makes fast progress in clumped high-dimensional regimes, and that Lloyd is a competitive efficient alternative.

Significance. If the empirical claims hold, the paper provides a practically useful toolbox for spherical dispersion regularization with favorable O(nm) or O(nm + n log n) complexity compared to pairwise O(n^2 m) methods. The theoretical contributions are modest but mostly sound: Lemma 2 follows from rotational invariance, Lemma 3 from the rearrangement inequality, and Prop. 2 computes the gradient correctly and notes the tangent property. The paper includes an open-source library and reports experiments on real tasks, which are strengths. However, the central Sliced regularizer relies on a heuristic clipping step whose details are not reported, and the MMD interpretation is stated more broadly than the assumptions justify. These issues weaken the evidential support for the paper's main claims but are addressable within the manuscript's scope.

major comments (3)
  1. [§3.3 (Eq. 13) and §4] The Sliced gradient is singular whenever the denominator ⟨xi,Q⟩²+⟨xi,P⟩² approaches zero, and the paper states that the standard stochastic Riemannian convergence argument does not hold and that numerical issues are handled heuristically "e.g., via clipping." However, the paper does not report the clipping threshold, the fraction of clipped gradient entries, or any sensitivity analysis. Because the favorable Sliced behavior is reported precisely in the clumped high-dimensional regime (§4.2), where the projection denominator is typically small, the current experiments cannot distinguish properties of the LSliced objective from properties of the clipping heuristic. Please report the clipping threshold used in each experiment, the fraction of clipped gradient entries over training, and an ablation (e.g., varying the threshold or using a smoothed bounded surrogate) to establish that the Sliced results are not artifacts of the clip level.
  2. [§3.1 / Prop. 1] Lemma 2 assumes a positive definite kernel, but Table 6 lists the Riesz kernel as only conditionally positive definite. Since squared MMD and the RKHS interpretation require a positive-definite kernel, the claim that "LMHE,k − c is an unbiased estimator of MMD²" is not justified for the Riesz kernel as stated. Please either explicitly restrict the proposition to positive-definite kernels or provide a separate treatment for conditionally positive definite kernels, and adjust the surrounding text accordingly.
  3. [§3.3 / Lemma 3] The displayed optimal configuration is Φ = (φ_1,...,φ_n) with φ_k = −π/n + 1/n + 2πk/n. This set does not have zero mean, contradicting the proof of Lemma 3, which uses "the last equality by choice of the zero-centered reference configuration Φ" to conclude τ* = (1/n)Σ_i θ_i. If the intended configuration is zero-centered (e.g., φ_k = −π + (2k−1)π/n), the formula should be corrected; otherwise the projection formula in Lemma 3, which enters the gradient in Eq. (13), is incorrect. Please clarify the definition and the range of k.
minor comments (4)
  1. [Lemma 2 / Prop. 1] In Lemma 2 and Proposition 1, "reproducible kernel Hilbert space" should be "reproducing kernel Hilbert space."
  2. [§4.1] The experimental setup gives the number of Lloyd samples (300), SSW projections (50), and Sliced great circles (1) in the text, but these parameters are not visible in Figure 4; adding them to the caption or a table would help reproducibility.
  3. [Figure 4] The label "uniform init." appears as a separate bar; please clarify whether this is the initial minimum angle before any optimization, and make the comparison with the optimal line explicit.
  4. [§3.3] The formula for Φ uses k without specifying its range; please state that k runs over 1..n (or 0..n−1) and ensure the formula matches the zero-mean property used in the proof.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the derivation chain is self-contained and validated against external benchmarks.

full rationale

The paper's load-bearing derivations are mathematical identities or new algorithms with independent empirical validation, not self-referential predictions. Lemma 2 and Proposition 1 derive the MMD interpretation of kernel energies from rotational symmetry of the kernel and the standard MMD definition; the kernel objective is not defined in terms of MMD to force the equivalence. Lemma 3 and Proposition 2 compute the Sliced gradient by direct calculus from the proposed objective Eq. (12), using the known optimal S1 configuration as a target rather than as a fitted parameter. Lloyd's regularizer is a standard quantization objective with bounded Riemannian gradients, and its convergence is cited from Bonnabel (2013), an external result. The empirical evaluation uses external yardsticks: the Robinson (1961) Tammes solution for n=24, BLEU/COMET on WMT test sets, and ImageNet-200 accuracy, none of which are functions of the proposed regularizers. The only appearance of author self-citation (Tokarchuk & Niculae 2024) supplies the CoNMT experimental setup, not the mathematical content, so it is not load-bearing. The unquantified clipping in §3.3 is flagged in the paper itself as a heuristic and as a reason the standard convergence argument fails; that is a correctness and robustness limitation, not circularity, because clipping is not fitted to the external metrics and the favorable results are not shown to be forced by the clip level. No equation reduces to its own input by construction.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The central contributions introduce no new physical or mathematical entities; the algorithm is defined over existing objects (sphere, uniform measure, great circles). The main hidden costs are hyperparameters and assumptions: the manual kernel and bandwidth choices, the unreported lambda, the unbounded-gradient clipping, and the appeal to known positive-definiteness results.

free parameters (7)
  • kernel bandwidth gamma for RBF and Laplace MHE = gamma = 1
    Set by hand in Sections 4.1 to 4.5; affects relative performance of MHE methods and the gradient behavior in Fig. 2.
  • Riesz kernel exponent s = s = 1
    Set by hand in Sections 4.1 to 4.3; changes the kernel's limiting behavior and whether it is conditionally positive definite.
  • dispersion regularization weight lambda = not reported
    Defined in Eq. 14 for the NMT objective but never specified in Sections 4.4, 4.5, or Appendix E.1; downstream BLEU gains depend on it.
  • number of uniform samples for Lloyd = 300, 512, 200, or 100 per section
    Monte Carlo sample count for the expectation in Eq. 9; differs across experiments and affects variance and cost.
  • number of great circles for Sliced and SSW = 1 or 13
    Single great circle per update in most experiments, 13 for axis-aligned comparison; high-variance estimator as shown in Appendix E.4.
  • Riemannian Adam learning rate for NMT = 5e-3
    Tuned from {5e-5, 5e-4, 5e-3} in Appendix E.1; the spherical baseline's gains partly depend on this choice.
  • clipping threshold for Sliced gradient = not specified
    Heuristic mentioned in Section 3.3 for the unbounded gradient in Eq. 13; no value or schedule given, yet needed for stable training.
assumptions (6)
  • standard math Uniform measure on S^m is rotationally invariant, so E_{Y~u}[k(z,Y)] is constant in z for rotationally invariant kernels.
    Used in the proof of Lemma 2, Appendix A.2, to reduce MMD^2 to E[k]-c.
  • standard math Rearrangement inequality (Hardy et al. 1952) characterizes the optimal permutation in the 1D projection.
    Used in the proof of Lemma 3, Appendix B.1, to sort angles against the optimal S1 configuration.
  • standard math The known optimal configuration of n points on S1 is, up to rotation and permutation, the set Phi in Section 3.3.
    The Sliced objective inherits this classical equally-spaced configuration; the paper states it without proof.
  • standard math Positive definiteness or conditional positive definiteness of the kernels in Table 6 (RBF, Laplace, Riesz) as cited from Feragen et al. 2015 and Bilyk and Grabner 2024.
    Proposition 1 requires a positive definite kernel to define an RKHS and MMD, but Table 6 lists Riesz as only conditionally positive definite; the paper's MMD interpretation therefore does not cover all MHE variants.
  • domain assumption Sampling great circles by orthogonalizing a 2 by m Gaussian matrix yields a uniform distribution over great circles, and dispersion along random great circles is a sufficient proxy for dispersion on S^m.
    Basis of LSliced in Section 3.3; empirical validation is indirect and no convergence guarantee covers the resulting stochastic Riemannian gradient.
  • ad hoc to paper Clipping or otherwise bounding the unbounded gradient in Eq. 13 preserves the useful behavior of the Sliced objective.
    Stated in Section 3.3 as a heuristic ('handling them heuristically (e.g., via clipping) works well'); load-bearing for the Sliced experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Keep your distance: learning dispersed embeddings on $\mathbb{S}_m$." pith.science (2026). https://pith.science/paper/P4XQTLMR

@misc{pith2026250208231,
  author       = {Pith},
  title        = {Pith review of: Keep your distance: learning dispersed embeddings on $\mathbbS_m$},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P4XQTLMR}},
  note         = {Machine review of arXiv:2502.08231}
}
read the original abstract

Learning well-separated features in high-dimensional spaces, such as text or image embeddings, is crucial for many machine learning applications. Achieving such separation can be effectively accomplished through the dispersion of embeddings, where unrelated vectors are pushed apart as much as possible. By constraining features to be on a hypersphere, we can connect dispersion to well-studied problems in mathematics and physics, where optimal solutions are known for limited low-dimensional cases. However, in representation learning we typically deal with a large number of features in high-dimensional space, and moreover, dispersion is usually traded off with some other task-oriented training objective, making existing theoretical and numerical solutions inapplicable. Therefore, it is common to rely on gradient-based methods to encourage dispersion, usually by minimizing some function of the pairwise distances. In this work, we first give an overview of existing methods from disconnected literature, making new connections and highlighting similarities. Next, we introduce some new angles. We propose to reinterpret pairwise dispersion using a maximum mean discrepancy (MMD) motivation. We then propose an online variant of the celebrated Lloyd's algorithm, of K-Means fame, as an effective alternative regularizer for dispersion on generic domains. Finally, we derive a novel dispersion method that directly exploits properties of the hypersphere. Our experiments show the importance of dispersion in image classification and natural language processing tasks, and how algorithms exhibit different trade-offs in different regimes.

Figures

Figures reproduced from arXiv: 2502.08231 by the authors.

Figure 1
Figure 1. Euclidean vs. Rieman￾nian optimization. Since the configurations we are considering consists of points on a sphere, the natural framework for optimization is Riemannian optimization, taking into account the geometry of the sphere Sm. We briefly describe and illustrate the method for Sm as an embedded submanifold of R m. Given a point x ∈ Sm and a function F : Sm → R, the Riemannian gradient is the projection of the … view at source ↗
Figure 2
Figure 2. Norm of the gradient of the kernel between points, as the angle between points varies. For our application, we take p to be an empirical dis￾tribution supported at a finite number of points X. Since the uniform distribution is not in this family, MMD cannot reach zero, but we still have a meaning￾ful notion of distance between X and the continuous uniform distribution. This result provides a valid in￾terpretation of… view at source ↗
Figure 3
Figure 3. Sliced dispersion update along a great circle Spq. Any other optimal configuration must be a rotation of this one, i.e., τ + Φ for τ ∈ (−π, π), followed by a permutation of these angles. Given a permutation σ ∈ Πn denote Φσ = (ϕσ(1), . . . , ϕσ(n)). We can then write the set of all possible ordered optimally-dispersed configurations as DnS2 := {τ + Φσ | τ ∈ (−π, π), σ ∈ Πn} . (10) Given an ordered configuration of a… view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Minimum angles (degrees) for each of the n = 24 points. The horizontal line indicates the angle for known optimal solution; in an ideal solution, all angles would be on the horizontal line. 4 Applications We demonstrate the application of dispersion objectives and prov…
Figure 5
Figure 5. Figure 5: Convergence and performance of different dispersion objectives on synthetic data with n = 20 000, m = 64. Mean and its standard error over three random initializations. axis-aligned circles (as 13n ≈ 5122 ). For SSW we provide results with 1 and 13 random circles, sinc…
Figure 6
Figure 6. Figure 6: Gradient norms (a) and minimum distances (b) w.r.t. NMT embeddings, after 40k steps, for the standard (Euclidean) transformer and the spherical version where decoder embeddings are optimized on Sm. Frequency rank is the index of the token in a sorted vocabulary: higher…
Figure 7
Figure 7. Figure 7: Concentration of angles for different kernels. D.2 Additional results To study the impact of dimensionality further we provide an additional empirical comparison of the minimum geodesic distance and spherical variance for m ∈ (64, 256, 512) for n = 10k and n = 20 in Fi…
Figure 8
Figure 8. Figure 8: Maximum minimum angle and spherical variance for synthetic data with different dimensionalities and number of points 10000. languages we cover. We used fairseq (Ott et al., 2019) framework for training our models. Baseline discrete models (Euclidean baseline) are train…
Figure 9
Figure 9. Figure 9: Maximum minimum angle and spherical variance for synthetic data with different dimensionalities and number of points 20000. in dimension m = 128, we draw up to 10k uniformly-sampled great circles and report the Monte Carlo estimated average objective in [PITH_FULL_IMA…
Figure 10
Figure 10. Figure 10: Training dynamic of gradient norms and minimum distances of the target language embeddings. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_10.png]
Figure 11
Figure 11. Figure 11: Minimum angles (degrees) for each of the N=24 points with respect to optimization methods. Top: projected gradient in Euclidean space. Bottom: Riemannian gradient on the sphere. 0 2000 4000 6000 8000 10000 number of samples used in estimation 1.50 1.75 2.00 2.25 2.50 …
Figure 12
Figure 12. Figure 12: Convergence of the sliced regularizer. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_12.png]
Figure 13
Figure 13. Figure 13: Minimum geodesic distance for different dimensions and number of points optimized by Sliced and SSW. Points generated with PS distribution with κ=100. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

83 extracted references · 57 canonical work pages

  1. [1]

    Kavosh Asadi and Michael L. Littman. An alternative softmax operator for reinforcement learning. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp.\ 243--252. PMLR, 06--11 Aug 2017. URL https://proceedings.mlr.press/v70/asadi17a.html

    2017

  2. [2]

    On the Equivalence between Herding and Conditional Gradient Algorithms

    Francis Bach, Simon Lacoste-Julien, and Guillaume Obozinski. On the Equivalence between Herding and Conditional Gradient Algorithms . In ICML 2012 International Conference on Machine Learning , pp.\ --, Edimburgh, United Kingdom, June 2012. URL https://hal.science/hal-00681128

    2012

  3. [3]

    Neural machine translation by jointly learning to align and translate

    Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations, January 2015

    2015

  4. [4]

    Riemannian adaptive optimization methods

    Gary Becigneul and Octavian-Eugen Ganea. Riemannian adaptive optimization methods. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=r1eiqi09K7

    work page 2019

  5. [5]

    Positive definite singular kernels on two-point homogeneous spaces

    Dmitriy Bilyk and Peter Grabner. Positive definite singular kernels on two-point homogeneous spaces, 2024. URL https://arxiv.org/abs/2410.22104

    work page Pith review arXiv 2024

  6. [6]

    Spherical S liced- W asserstein

    Cl \' e ment Bonet, Paul Berg, Nicolas Courty, Fran c ois Septier, Lucas Drumetz, and Minh Tan Pham. Spherical S liced- W asserstein. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=jXQ0ipgMdU

    work page 2023

  7. [7]

    Stochastic gradient descent on R iemannian manifolds

    Silvere Bonnabel. Stochastic gradient descent on R iemannian manifolds. IEEE Transactions on Automatic Control, 58 0 (9): 0 2217--2229, 2013

    work page 2013

  8. [8]

    Convergence properties of the KMeans algorithm

    L \' e on Bottou and Yoshua Bengio. Convergence properties of the KMeans algorithm. In Proc. of NeurIPS, 1995. URL http://leon.bottou.org/papers/bottou-bengio-95

    work page 1995

Show all 83 references
  1. [9]

    Blaschko, Ioannis Antonoglou, and Arthur Gretton

    Wacha Bounliphone, Eugene Belilovsky, Matthew B. Blaschko, Ioannis Antonoglou, and Arthur Gretton. A test of relative similarity for model selection in generative models. CoRR, abs/1511.04581, 2015. URL https://api.semanticscholar.org/CorpusID:3173270

    2015 arXiv

  2. [10]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp.\ 1597--1607. PMLR, 2020 a

    2020

  3. [11]

    Big self-supervised models are strong semi-supervised learners

    Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems, 33: 0 22243--22255, 2020 b

    2020

  4. [12]

    Table of spherical codes

    Henry Cohn. Table of spherical codes. https://dspace.mit.edu/handle/1721.1/153543, 2024. Accessed: 2024-05-28

    2024

  5. [13]

    Sphere-packings, lattices, and groups

    John Horton Conway, Neil James Alexander Sloane, and Eiichi Bannai. Sphere-packings, lattices, and groups. Grundlehren der mathematischen Wissenschaften 290. Springer, New York, 3rd ed edition, 1999. ISBN 0387985859

    1999

  6. [14]

    Finite point-sets on S^2 with minimum distance as large as possible

    Ludwig Danzer. Finite point-sets on S^2 with minimum distance as large as possible. Discrete Mathematics, 60: 0 3--66, 1986. ISSN 0012-365X. doi:https://doi.org/10.1016/0012-365X(86)90002-6. URL https://www.sciencedirect.com/science/article/pii/0012365X86900026

    1986

  7. [15]

    The power spherical distrbution

    Nicola De Cao and Wilker Aziz. The power spherical distrbution. Proceedings of the 37th International Conference on Machine Learning, INNF+, 2020

    2020

  8. [16]

    Uber eine Absch\

    László Fejes Tóth. \"Uber eine Absch\"atzung des k\"urzesten Abstandes zweier Punkte eines auf einer Kugelfl\"ache liegenden Punktsystems. Jahresbericht der Deutschen Mathematiker-Vereinigung, 53: 0 66--68, 1943. URL http://dml.mathdoc.fr/item/GDZPPN002133873

    1943

  9. [17]

    Geodesic exponential kernels: When curvature and linearity conflict

    Aasa Feragen, Francois Lauze, and Soren Hauberg. Geodesic exponential kernels: When curvature and linearity conflict. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 3032--3042, 2015

    2015

  10. [18]

    Principal geodesic analysis for the study of nonlinear statistics of shape

    P Thomas Fletcher, Conglin Lu, Stephen M Pizer, and Sarang Joshi. Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE transactions on medical imaging, 23 0 (8): 0 995--1005, 2004

    2004

  11. [19]

    Minimizing a differentiable function over a differential manifold

    Daniel Gabay. Minimizing a differentiable function over a differential manifold. Journal of Optimization Theory and Applications, 37: 0 177--219, 1982

    1982

  12. [20]

    Representation degeneration problem in training natural language generation models

    Jun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tieyan Liu. Representation degeneration problem in training natural language generation models. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=SkEYojRqtm

    2019

  13. [21]

    A novel approach to the spherical codes problem

    Simanta Gautam and Dmitry Vaintrob. A novel approach to the spherical codes problem. In MIT, Cambridge, MA, USA, Tech. Rep, 2013. URL https://api.semanticscholar.org/CorpusID:12647839

    2013

  14. [22]

    Frage: Frequency-agnostic word representation

    Chengyue Gong, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie-Yan Liu. Frage: Frequency-agnostic word representation. Advances in neural information processing systems, 31, 2018

    2018

  15. [23]

    Borgwardt, Malte J

    Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Sch \"o lkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13 0 (25): 0 723--773, 2012. URL http://jmlr.org/papers/v13/gretton12a.html

    2012

  16. [24]

    Inequalities

    Godfrey Harold Hardy, John Edensor Littlewood, and Gy \"o rgy P \'o lya. Inequalities. Cambridge University Press, 1952

    1952

  17. [25]

    Momentum contrast for unsupervised visual representation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 9729--9738, 2020

    2020

  18. [26]

    Learning deep representations by mutual information estimation and maximization

    R Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Phil Bachman, Adam Trischler, and Yoshua Bengio. Learning deep representations by mutual information estimation and maximization. In International Conference on Learning Representations, 2019. URL https://ope...

    2019

  19. [27]

    Probability inequalities for sums of bounded random variables

    Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58 0 (301): 0 13--30, 1963. ISSN 01621459, 1537274X. URL http://www.jstor.org/stable/2282952

    1963

  20. [28]

    Topics in circular statistics

    S Rao Jammalamadaka and Ambar Sengupta. Topics in circular statistics. Series on multivariate analysis; vol. 5. World Scientific, Singapore, 2001. ISBN 9810237782

    2001

  21. [29]

    S entence P iece: A simple and language independent subword tokenizer and detokenizer for neural text processing

    Taku Kudo and John Richardson. S entence P iece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Eduardo Blanco and Wei Lu (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System ...

    2018 doi

  22. [30]

    Determinantal point processes for machine learning

    Alex Kulesza and Ben Taskar. Determinantal point processes for machine learning. Foundations and Trends® in Machine Learning, 5 0 (2–3): 0 123--286, December 2012. ISSN 1935-8237, 1935-8245. doi:10.1561/2200000044. URL https://www.nowpublishers.com/article/Details/MAL-044. Pub...

    2012 doi

  23. [31]

    von Mises-Fisher loss for training sequence to sequence models with continuous outputs

    Sachin Kumar and Yulia Tsvetkov. von Mises-Fisher loss for training sequence to sequence models with continuous outputs. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=rJlDnoA5Y7

    2019

  24. [32]

    Determinantal point process models and statistical inference

    Frédéric Lavancier, Jesper Møller, and Ege Rubak. Determinantal point process models and statistical inference. Journal of the Royal Statistical Society Series B: Statistical Methodology, 77 0 (4): 0 853--877, 12 2014. ISSN 1369-7412. doi:10.1111/rssb.12096. URL https://doi.or...

    2014 doi

  25. [33]

    Ya Le and Xuan S. Yang. Tiny ImageNet visual recognition challenge. Technical report., 2015. URL https://api.semanticscholar.org/CorpusID:16664790

    2015

  26. [34]

    Quantization and clustering on riemannian manifolds with an application to air traffic analysis

    Alice Le Brigant and Stéphane Puechmorel. Quantization and clustering on riemannian manifolds with an application to air traffic analysis. Journal of Multivariate Analysis, 173: 0 685--703, 2019. ISSN 0047-259X. doi:https://doi.org/10.1016/j.jmva.2019.05.008. URL https://www.s...

    2019 doi

  27. [35]

    Sample estimate of the entropy of a random vector

    Nikolai N Leonenko. Sample estimate of the entropy of a random vector. Problemy Peredachi Informatsii, 23 0 (2): 0 9--16, 1987

    1987

  28. [36]

    Minimisation des fonctionnelles d \'e finies sur une vari \'e t \'e par la methode du gradi \"e nt conjugu \'e

    Alain Lichnewsky. Minimisation des fonctionnelles d \'e finies sur une vari \'e t \'e par la methode du gradi \"e nt conjugu \'e . PhD thesis, These de Doctorat d'Etat. Paris: Universit \'e de Paris-Sud, 1979

    1979

  29. [37]

    Riemannian stein variational gradient descent for bayesian inference

    Chang Liu and Jun Zhu. Riemannian stein variational gradient descent for bayesian inference. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018

    2018

  30. [38]

    Stein variational gradient descent: A general purpose bayesian inference algorithm

    Qiang Liu and Dilin Wang. Stein variational gradient descent: A general purpose bayesian inference algorithm. Advances in neural information processing systems, 29, 2016

    2016

  31. [39]

    Sphereface: Deep hypersphere embedding for face recognition

    Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017

    2017

  32. [40]

    Liu, Lixin Liu, Zhiding Yu, Bo Dai, and Le Song

    Weiyang Liu, Rongmei Lin, Z. Liu, Lixin Liu, Zhiding Yu, Bo Dai, and Le Song. Learning towards minimum hyperspherical energy. In Neural Information Processing Systems, 2018. URL https://api.semanticscholar.org/CorpusID:43921092

    2018

  33. [41]

    Learning with hyperspherical uniformity

    Weiyang Liu, Rongmei Lin, Zhen Liu, Li Xiong, Bernhard Sch \"o lkopf, and Adrian Weller. Learning with hyperspherical uniformity. In Arindam Banerjee and Kenji Fukumizu (eds.), Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, volume 1...

    2021

  34. [42]

    Multilingual denoising pre-training for neural machine translation

    Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. Multilingual denoising pre-training for neural machine translation. Transactions of the Association for Computational Linguistics, 8: 0 726--742, 2020. doi:10.11...

    2020 doi

  35. [43]

    Least squares quantization in PCM

    Stuart P Lloyd. Least squares quantization in PCM . IEEE Transactions on Information Theory, 28 0 (2): 0 129--137, 1982. doi:10.1109/TIT.1982.1056489

    1982

  36. [44]

    Luenberger

    David G. Luenberger. The gradient projection method along geodesics. Management Science, 18 0 (11): 0 620--631, 1972. ISSN 00251909, 15265501. URL http://www.jstor.org/stable/2629156

    1972

  37. [45]

    Reliable measures of spread in high dimensional latent spaces

    Anna Marbut, Katy McKinney-Bock, and Travis Wheeler. Reliable measures of spread in high dimensional latent spaces. In International Conference on Machine Learning, pp.\ 23871--23885. PMLR, 2023

    2023

  38. [46]

    Mardia and P

    Kanti V. Mardia and P. E. Jupp. Directional statistics . John Wiley & Sons, Inc., 1 1999. doi:10.1002/9780470316979. URL https://doi.org/10.1002/9780470316979

    1999 doi

  39. [47]

    Statistics of directional data

    Kantilal Varichand Mardia. Statistics of directional data. Journal of the Royal Statistical Society Series B: Statistical Methodology, 37 0 (3): 0 349--371, 1975

    1975

  40. [48]

    Spherical text embedding

    Yu Meng, Jiaxin Huang, Guangyuan Wang, Chao Zhang, Honglei Zhuang, Lance Kaplan, and Jiawei Han. Spherical text embedding. In Advances in neural information processing systems, 2019

    2019

  41. [49]

    Hyperspherical prototype networks

    Pascal Mettes, Elise van der Pol, and Cees G M Snoek. Hyperspherical prototype networks. In Advances in Neural Information Processing Systems, 2019

    2019

  42. [50]

    Determinantal point process models on the sphere

    Jesper M ller, Morten Nielsen, Emilio Porcu, and Ege Rubak. Determinantal point process models on the sphere. Bernoulli, 24 0 (2): 0 1171--1201, 2018. ISSN 13507265. URL http://www.jstor.org/stable/26491978

    2018

  43. [51]

    Musin and Alexey S

    Oleg R. Musin and Alexey S. Tarasov. The strong thirteen spheres problem . Discrete and Computational Geometry, 48 0 (1): 0 128--141, 2 2012. doi:10.1007/s00454-011-9392-2. URL https://doi.org/10.1007/s00454-011-9392-2

    2012 doi

  44. [52]

    Musin and Alexey S

    Oleg R. Musin and Alexey S. Tarasov. The Tammes problem for N = 14. Experimental Mathematics, 24 0 (4): 0 460--468, 2015. doi:10.1080/10586458.2015.1022842. URL https://doi.org/10.1080/10586458.2015.1022842

    2015

  45. [53]

    Bastiaan Kleijn

    Mattias Nilsson and W. Bastiaan Kleijn. On the estimation of differential entropy from data located on embedded manifolds. IEEE Transactions on Information Theory, 53 0 (7): 0 2330--2341, 2007. doi:10.1109/TIT.2007.899533

    2007

  46. [54]

    fairseq: A fast, extensible toolkit for sequence modeling

    Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations, 2019

    2019

  47. [55]

    B leu: a method for automatic evaluation of machine translation

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. B leu: a method for automatic evaluation of machine translation. In Pierre Isabelle, Eugene Charniak, and Dekang Lin (eds.), Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pp...

    2002

  48. [56]

    Intrinsic statistics on R iemannian manifolds: Basic tools for geometric measurements

    Xavier Pennec. Intrinsic statistics on R iemannian manifolds: Basic tools for geometric measurements. J Math Imaging Vis, 25: 0 127--154, 2006. URL https://link.springer.com/article/10.1007/s10851-006-6228-4

    2006 doi

  49. [57]

    Computational optimal transport: With applications to data science

    Gabriel Peyr \'e , Marco Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends in Machine Learning , 11 0 (5-6): 0 355--607, 2019

    2019

  50. [58]

    A call for clarity in reporting BLEU scores

    Matt Post. A call for clarity in reporting BLEU scores. In Ond r ej Bojar, Rajen Chatterjee, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Christof Monz, Matteo Negri, Aur \'e lie N \'e v \'e ol, Mariana Neve...

    2018

  51. [59]

    Wasserstein barycenter and its application to texture mixing

    Julien Rabin, Gabriel Peyr \' e , Julie Delon, and Marc Bernot. Wasserstein barycenter and its application to texture mixing. In Alfred M. Bruckstein, Bart M. ter Haar Romeny, Alexander M. Bronstein, and Michael M. Bronstein (eds.), Scale Space and Variational Methods in Compu...

    2012

  52. [60]

    COMET : A neural framework for MT evaluation

    Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. COMET : A neural framework for MT evaluation. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 2685--270...

    2020 doi

  53. [61]

    Arrangement of 24 points on a sphere

    Raphael M Robinson. Arrangement of 24 points on a sphere. Mathematische Annalen, 144: 0 17--48, 1961. URL http://eudml.org/doc/160873

    1961

  54. [62]

    Orthogonal estimation of wasserstein distances

    Mark Rowland, Jiri Hron, Yunhao Tang, Krzysztof Choromanski, Tamas Sarlos, and Adrian Weller. Orthogonal estimation of wasserstein distances. In Kamalika Chaudhuri and Masashi Sugiyama (eds.), Proceedings of the Twenty-Second International Conference on Artificial Intelligence...

    2019

  55. [63]

    Spreading vectors for similarity search

    Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, and Hervé Jégou. Spreading vectors for similarity search. In International Conference on Learning Representations, 2019. URL https://openreview.net/forum?id=SkGuG2R5tm

    2019

  56. [64]

    Soft-max and Soft-argmax , 2024

    Joseph Salmon and François-David Collin. Soft-max and Soft-argmax , 2024. URL https://josephsalmon.eu/blog/softmax/

    2024

  57. [65]

    Web-scale K -means clustering

    D Sculley. Web-scale K -means clustering. In Proc. of WWW, 2010

    2010

  58. [66]

    Approximation theorems of mathematical statistics

    Robert J Serfling. Approximation theorems of mathematical statistics. John Wiley & Sons, 2009

    2009

  59. [67]

    A user's guide to sampling strategies for sliced optimal transport

    Keanu Sisouk, Julie Delon, and Julien Tierny. A user's guide to sampling strategies for sliced optimal transport. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL https://openreview.net/forum?id=ECBepTWAFG. Survey Certification

    2025

  60. [68]

    Distribution of points in a cube and approximate evaluation of integrals

    Ilya M Sobol. Distribution of points in a cube and approximate evaluation of integrals. USSR Computational mathematics and mathematical physics, 7: 0 86--112, 1967

    1967

  61. [69]

    Sutherland

    Danica J. Sutherland. Unbiased estimators for the variance of mmd estimators. ArXiv, abs/1906.02104, 2019. URL https://api.semanticscholar.org/CorpusID:174799385

    1906 arXiv

  62. [70]

    On the origin of number and arrangement of the places of exit on the surface of pollen-grains

    Pieter Merkus Lambertus Tammes. On the origin of number and arrangement of the places of exit on the surface of pollen-grains. PhD thesis, University of Groningen, 1930. Relation: http://www.rug.nl/ Rights: De Bussy

    1930

  63. [71]

    JJ Thomson. On the structure of the atom: an investigation of the stability and periods of oscillation of a number of corpuscles arranged at equal intervals around the circumference of a circle; with application of the results to the theory of atomic structure. The London, Edi...

    1904 doi

  64. [72]

    The unreasonable effectiveness of random target embeddings for continuous-output neural machine translation

    Evgeniia Tokarchuk and Vlad Niculae. The unreasonable effectiveness of random target embeddings for continuous-output neural machine translation. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Asso...

    2024 doi

  65. [73]

    Hubs and hyperspheres: Reducing hubness and improving transductive few-shot learning with hyperspherical embeddings

    Daniel Trosten, Rwiddhi Chakraborty, Sigurd Løkse, Kristoffer Wickstrøm, Robert Jenssen, and Michael Kampffmeyer. Hubs and hyperspheres: Reducing hubness and improving transductive few-shot learning with hyperspherical embeddings. In Proceedings of the IEEE/CVF conference on c...

    2023

  66. [74]

    Auf welcher kugel haben 5, 6, 7, 8 oder 9 punkte mit mindestabstand eins platz ? Mathematische Annalen, 123: 0 96--124, 1951

    Bartel Leendert van der Waerden and Kurt Schütte. Auf welcher kugel haben 5, 6, 7, 8 oder 9 punkte mit mindestabstand eins platz ? Mathematische Annalen, 123: 0 96--124, 1951. URL http://eudml.org/doc/160237

    1951

  67. [75]

    Gomez, ukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964

    2017

  68. [76]

    Optimal transport: old and new, volume 338

    C \'e dric Villani et al. Optimal transport: old and new, volume 338. Springer Berlin, Heidelberg, 2008

    2008

  69. [77]

    On convergence of projected gradient descent for minimizing a large-scale quadratic over the unit sphere

    Trung Vu, Raviv Raich, and Xiao Fu. On convergence of projected gradient descent for minimizing a large-scale quadratic over the unit sphere. In 2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP), pp.\ 1--6. IEEE, 2019

    2019

  70. [78]

    Improving neural language generation with spectrum control

    Lingxiao Wang, Jing Huang, Kevin Huang, Ziniu Hu, Guangtao Wang, and Quanquan Gu. Improving neural language generation with spectrum control. In International Conference on Learning Representations, 2020. URL https://openreview.net/forum?id=ByxY8CNtvr

    2020

  71. [79]

    Understanding contrastive representation learning through alignment and uniformity on the hypersphere

    Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of M...

    2020

  72. [80]

    Mma regularization: Decorrelating weights of neural networks by maximizing the minimal angles, 2021

    Zhennan Wang, Canqun Xiang, Wenbin Zou, and Chen Xu. Mma regularization: Decorrelating weights of neural networks by maximizing the minimal angles, 2021

    2021

  73. [81]

    Atan2 --- Wikipedia , the free encyclopedia

    Wikipedia contributors . Atan2 --- Wikipedia , the free encyclopedia. https://en.wikipedia.org/w/index.php?title=Atan2&oldid=1247664857, 2024. [Online; accessed 25-September-2024]

    2024

  74. [82]

    Frequency-aware contrastive learning for neural machine translation

    Tong Zhang, Wei Ye, Baosong Yang, Long Zhang, Xingzhang Ren, Dayiheng Liu, Jinan Sun, Shikun Zhang, Haibo Zhang, and Wen Zhao. Frequency-aware contrastive learning for neural machine translation. Proceedings of the AAAI Conference on Artificial Intelligence, 36 0 (10): 0 11712...

    2022 doi

  75. [83]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.