{"id":"bd958916-7bfc-42e3-a0ec-5b18011c0b85","arxiv_id":"2502.08231","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"The authors introduce sliced and Lloyd-based dispersion regularizers for hyperspherical embeddings, connect kernel dispersion objectives to maximum mean discrepancy, and show downstream gains in prototype classification and neural machine translation.","lead":"This paper studies how to push machine learning embeddings apart evenly on a high-dimensional sphere, and proposes two new ways to do it, one adapted from Lloyd's algorithm and one based on slicing the sphere along great circles. It reports better image classification and machine translation when such dispersion is encouraged, and argues that Riemannian optimization on the sphere beats the usual projected Euclidean updates.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unquantified gradient clipping in Sliced (Eq. 13, §3.3) may drive the high-dimensional gains; the paper reports no clipping threshold or clipped fraction, so the central claim is not yet settled.","rationale":"The central claim requires that LSliced and Lloyd are effective alternatives because of their objectives, not because of undocumented optimizer heuristics. The paper's own §3.3 flags the unbounded gradient and the use of clipping; this is the weakest point because it directly invalidates the 'unbiased estimator' statement used to justify stochastic Riemannian gradient updates. The denominator in Eq. 13 concentrates near zero in high dimensions, so clipping is likely active for many samples, and the experimental sections that showcase Sliced (Fig. 5, Tables 3–4) do not report the clip threshold or clipped fraction. I agree with the reader's weakest assumption. Other issues (MMD overextension to conditionally positive definite kernels, missing λ, missing error bars) are real but secondary: they affect framing or reproducibility, not the mechanism behind Sliced's reported gains. A threshold sweep would settle the concern; if the ranking persists across thresholds, the claim should be accepted. Since the reader already conditioned on this, I recommend keeping the CONDITIONAL verdict.","tokens_in":25668,"tokens_out":14472,"duration_ms":154838,"concrete_test":"Re-run the §4.2 clumped experiment (n=20k, m=64) with the Sliced clipping threshold varied over at least three orders of magnitude (e.g., 1e-3, 1e-2, 1e-1, 1) and report the final svar/dmin and the fraction of clipped gradient coordinates per step. If Sliced's ranking relative to MHE and KoLeo is stable and the clipped fraction is negligible, the concern is resolved; if the ranking changes or the clipped fraction is large, the reported Sliced advantage is an artifact of the heuristic. The same check should be applied to Table 3's NMT results with the unreported λ and clip threshold.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that the heuristic clipping mentioned in §3.3 does not materially change the direction of the Sliced gradient. The stochastic gradient in Eq. 13 divides by ⟨xi,Q⟩²+⟨xi,P⟩²; for m=64 and uniformly random great circles this denominator is small for a substantial fraction of points, so clipping is not a rare safety net but an active part of the update. The paper states that 'the standard convergence argument for stochastic Riemannian gradient does not hold for Sliced or for SSW' and that numerical issues are 'handled heuristically (e.g., via clipping)', but it does not report the clipping threshold, the fraction of clipped coordinates, or how the threshold was chosen. Once clipping is applied, the stochastic gradient is no longer unbiased for grad LSliced, so the favorable Sliced curves in Fig. 5 and the gains in Tables 3–4 could be properties of the clip level rather than of the proposed objective. This matters specifically for the claimed 'fast progress in clumped high-dimensional regimes', where the projection denominator is small precisely because points start concentrated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies dispersion regularizers for embeddings constrained to the unit sphere S^m. It contributes (i) an interpretation of kernel-based dispersion objectives as squared MMD against the uniform distribution (Lemma 2, Prop. 1), (ii) a stochastic Lloyd-type regularizer for dispersion, and (iii) a \"sliced dispersion\" objective (Eq. 12) that measures the distance of projected angles to the optimally dispersed configuration on a great circle, with a gradient formula (Eq. 13). The authors evaluate these methods on the Tammes problem (n=24, m=3), synthetic high-dimensional settings with uniform and clumped initializations, ImageNet-200 prototype classification, and discrete and continuous-output neural machine translation. The central empirical claims are that Riemannian optimization improves dispersion and downstream performance, that Sliced makes fast progress in clumped high-dimensional regimes, and that Lloyd is a competitive efficient alternative.","tokens_in":25919,"tokens_out":6651,"duration_ms":64770,"significance":"If the empirical claims hold, the paper provides a practically useful toolbox for spherical dispersion regularization with favorable O(nm) or O(nm + n log n) complexity compared to pairwise O(n^2 m) methods. The theoretical contributions are modest but mostly sound: Lemma 2 follows from rotational invariance, Lemma 3 from the rearrangement inequality, and Prop. 2 computes the gradient correctly and notes the tangent property. The paper includes an open-source library and reports experiments on real tasks, which are strengths. However, the central Sliced regularizer relies on a heuristic clipping step whose details are not reported, and the MMD interpretation is stated more broadly than the assumptions justify. These issues weaken the evidential support for the paper's main claims but are addressable within the manuscript's scope.","major_comments":[{"comment":"The Sliced gradient is singular whenever the denominator ⟨xi,Q⟩²+⟨xi,P⟩² approaches zero, and the paper states that the standard stochastic Riemannian convergence argument does not hold and that numerical issues are handled heuristically \"e.g., via clipping.\" However, the paper does not report the clipping threshold, the fraction of clipped gradient entries, or any sensitivity analysis. Because the favorable Sliced behavior is reported precisely in the clumped high-dimensional regime (§4.2), where the projection denominator is typically small, the current experiments cannot distinguish properties of the LSliced objective from properties of the clipping heuristic. Please report the clipping threshold used in each experiment, the fraction of clipped gradient entries over training, and an ablation (e.g., varying the threshold or using a smoothed bounded surrogate) to establish that the Sliced results are not artifacts of the clip level.","section":"§3.3 (Eq. 13) and §4"},{"comment":"Lemma 2 assumes a positive definite kernel, but Table 6 lists the Riesz kernel as only conditionally positive definite. Since squared MMD and the RKHS interpretation require a positive-definite kernel, the claim that \"LMHE,k − c is an unbiased estimator of MMD²\" is not justified for the Riesz kernel as stated. Please either explicitly restrict the proposition to positive-definite kernels or provide a separate treatment for conditionally positive definite kernels, and adjust the surrounding text accordingly.","section":"§3.1 / Prop. 1"},{"comment":"The displayed optimal configuration is Φ = (φ_1,...,φ_n) with φ_k = −π/n + 1/n + 2πk/n. This set does not have zero mean, contradicting the proof of Lemma 3, which uses \"the last equality by choice of the zero-centered reference configuration Φ\" to conclude τ* = (1/n)Σ_i θ_i. If the intended configuration is zero-centered (e.g., φ_k = −π + (2k−1)π/n), the formula should be corrected; otherwise the projection formula in Lemma 3, which enters the gradient in Eq. (13), is incorrect. Please clarify the definition and the range of k.","section":"§3.3 / Lemma 3"}],"minor_comments":[{"comment":"In Lemma 2 and Proposition 1, \"reproducible kernel Hilbert space\" should be \"reproducing kernel Hilbert space.\"","section":"Lemma 2 / Prop. 1"},{"comment":"The experimental setup gives the number of Lloyd samples (300), SSW projections (50), and Sliced great circles (1) in the text, but these parameters are not visible in Figure 4; adding them to the caption or a table would help reproducibility.","section":"§4.1"},{"comment":"The label \"uniform init.\" appears as a separate bar; please clarify whether this is the initial minimum angle before any optimization, and make the comparison with the optimal line explicit.","section":"Figure 4"},{"comment":"The formula for Φ uses k without specifying its range; please state that k runs over 1..n (or 0..n−1) and ensure the formula matches the zero-mean property used in the proof.","section":"§3.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is relevant to TMLR's audience and the open-source implementation is a strength. The main concern is the unquantified clipping in the Sliced regularizer, which is central to the paper's novelty. The requested ablations and reporting are feasible and would substantially strengthen the contribution. I also recommend that the editors ask the authors to verify the definition of Φ against the code, since the printed formula appears inconsistent with the proof."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, this is a genuinely useful addition to the dispersion-regularization toolbox: it turns Lloyd's algorithm into an online regularizer, and it introduces a sliced dispersion objective (Eq. 12) with a clean gradient (Prop. 2) that is cheaper than pairwise methods and often works better in clumped high-dimensional regimes. Second, the paper's main empirical claim for Sliced rests partly on an unquantified clipping heuristic, because the exact stochastic Riemannian gradient is unbounded and the authors admit the convergence argument does not apply. They don't report the clip threshold or the fraction of clipped coordinates, so the high-dimensional gains could be a property of the heuristic rather than the objective. That doesn't sink the paper; it just means the optimization story is incomplete.\n\nWhat's new: The Sliced objective and its gradient are new as far as I can tell. Proposition 1, connecting MHE to MMD^2 against the uniform sphere, is a real and useful observation for positive definite kernels. The Lloyd-as-regularizer proposal is a neat adaptation, and the Riemannian vs Euclidean gradient comparison is a practical contribution that will save people time. The Tammes check (n=24) and the synthetic experiments with both clumped and uniform initializations are well designed. The NMT and CoNMT results are plausible, and the authors report significance for many of them.\n\nWhere it's soft: (1) The clipping issue is the main one. The gradient in Eq. 13 divides by (⟨xi,Q⟩^2+⟨xi,P⟩^2), which is small for a meaningful fraction of points when m=64 and points start clumped. The paper says clipping 'works well' but gives no threshold, no clipped fraction, and no sensitivity analysis. The stress-test note is spot-on: once clipped, the stochastic gradient isn't unbiased for the objective, so the fast progress in Fig. 5 could be the clip's doing. (2) The MMD interpretation is stated for positive definite kernels, while the experiments include Riesz kernels, which are only conditionally positive definite. The implications of that gap are not discussed. (3) Minor reporting: the regularization weight λ in Eq. 14 is never reported, and a few significance markers are missing.\n\nWho this is for: anyone working on spherical embedding regularization, contrastive learning, or representation collapse. It deserves a serious referee; the core derivation is correct, the survey is helpful, and the soft spots are addressable in revision. I'd send it out.","headline":"Useful dispersion-regularizer paper with a genuinely new Sliced objective; the main caveat is an unquantified clipping heuristic that could drive the high-dimensional gains.","tokens_in":26494,"tokens_out":3233,"would_cite":true,"duration_ms":29984,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a sliced great-circle regularizer and an online Lloyd quantization regularizer can substitute for costly pairwise dispersion on the hypersphere, and that Riemannian optimization of these objectives improves…","keywords":["dispersion","hypersphere","embedding regularization","Riemannian optimization","sliced dispersion","Lloyd's algorithm","maximum mean discrepancy","Tammes problem"],"falsifier":"In the clumped synthetic setting of Section 4.2, run the Sliced regularizer with clipping thresholds varying over several orders of magnitude, and also with the denominator $\\langle x_i,Q\\rangle^2+\\langle x_i,P\\rangle^2$ replaced by a smoothed $+\\varepsilon$ version; if the final spherical variance and minimum distance change substantially with the threshold, the reported Sliced success comes from the clipping heuristic rather than from the dispersion objective itself.","tokens_in":25424,"feed_emoji":"🌐","tokens_out":11481,"duration_ms":107204,"temperature":0.7,"pith_summary":"Embeddings trained on the hypersphere tend to clump, and standard cures push unrelated vectors apart by minimizing pairwise distances at quadratic cost. The paper argues that two cheaper mechanisms achieve comparable dispersion: an online stochastic version of Lloyd's algorithm that treats the embedding vectors as quantizing centers for the uniform spherical measure, and a sliced regularizer that projects the configuration onto random great circles and measures, in closed form, how far the projected angles are from an optimally spaced circle. It also recasts kernel dispersion energy as squared maximum mean discrepancy (MMD) between the embedding distribution and the uniform measure on $\\mathbb{S}_m$. Across synthetic Tammes benchmarks, prototype-based image classification, discrete machine translation, and continuous-output machine translation, the paper reports that these regularizers — especially Sliced in clumped high-dimensional regimes — improve dispersion metrics and task accuracy, and that Riemannian rather than projected Euclidean gradient updates matter.","feed_headline":"Sliced dispersion spreads embeddings without pair costs","feed_subtitle":"A one-great-circle objective plus Riemannian updates lifts BLEU, accuracy, and separation in high dimensions.","key_machinery":"The central object is the great-circle slicing construction. A great circle is the intersection of the hypersphere with a plane through its center; each point $x\\in\\mathbb{S}_m$ projects onto it as an angle $\\theta=\\arctan_2(\\langle x,q\\rangle,\\langle x,p\\rangle)$. Lemma 3 gives the projection onto the set of optimally dispersed circular configurations: sort the projected angles, match them in order to the equal-spaced ideal angles, and shift all angles by their mean. Proposition 2 turns this projection into the gradient in Eq. (13), so that stochastic Riemannian gradient descent can optimize $L_{\\mathrm{Sliced}}$. The Lloyd regularizer is the other load-bearing object: with the sphere as the target measure, each embedding is pulled toward the barycenter of uniform samples assigned to it, and its gradient is bounded, which permits a stochastic Riemannian convergence argument. Riemannian projection of gradients, $(I-xx^\\top)\\nabla$, is the mechanism that keeps both objectives and the baseline pairwise objectives well-behaved on the sphere.","core_discovery":"The paper's central claim is that dispersion on the hypersphere — the even covering of the sphere by a set of embedding vectors — can be optimized without forming the full pairwise-distance matrix. Kernel energy objectives are shown in Lemma 2 to be unbiased estimators of squared MMD against the uniform distribution, with the uniform terms reducing to a constant. The paper then introduces two alternatives: an online stochastic Lloyd regularizer that pulls each embedding toward the barycenter of uniformly sampled points in its Voronoi cell, and a sliced regularizer $L_{\\mathrm{Sliced}}$ that projects the configuration onto random great circles and penalizes the squared distance of the projected angles from the set of optimally spaced circular configurations. In the synthetic and real-world evaluations, the paper argues that these regularizers approximate Tammes-optimal dispersion and improve prototype classification, discrete NMT, and continuous-output NMT, with Sliced making the fastest progress in clumped high-dimensional regimes. Across experiments it argues that Riemannian gradient updates, rather than Euclidean updates followed by projection, are systematically better.","pith_inferences":["Editorial inference: a smoothed variant of the Sliced gradient that replaces the unbounded denominator in Eq. (13) with a small regularizer would likely recover a stochastic-convergence guarantee while preserving the objective, giving a principled alternative to clipping.","Editorial inference: the slicing recipe should transfer to other homogeneous spaces with known one-dimensional optimal configurations, such as tori or projective spaces, by replacing the great-circle projection with the corresponding geodesic projection.","Editorial inference: the MMD equivalence suggests a statistical use of dispersion measures, namely a post-training uniformity test on the learned embeddings to detect clumping, complementing the paper's spherical-variance and minimum-distance metrics.","Editorial inference: the single-great-circle estimator is unbiased but may be high-variance in high dimensions; stratifying the sampled great circles (for example over axis-aligned directions) could lower variance at the same sample count."],"forward_implications":["Dispersion regularization becomes feasible at vocabulary scale: Lloyd costs $O(nm)$ and Sliced costs $O(nm+\\mathrm{sort}(n))$ per Monte Carlo sample, versus $O(n^2m)$ for pairwise objectives.","Riemannian optimization of decoder embeddings, even without an extra regularizer, is claimed to reduce rare-token collapse; the paper reports an order-of-magnitude larger gradient norm for rare tokens and better BLEU than the Euclidean baseline.","In clumped, high-dimensional regimes the Sliced regularizer makes the fastest initial progress on spherical variance, while max-min objectives keep improving minimum distance once points are already spread, so the right regularizer depends on the initialization regime.","Because kernel energy objectives are squared MMD against the uniform spherical measure, their minibatch variance and concentration can be analyzed with the MMD toolkit.","The prototype experiments show that minimum distance alone does not predict downstream accuracy; at output dimension equal to the number of classes, MHE approaches near-orthogonal solutions that underperform despite good separation."],"supporting_citations":[{"why":"Supplies the hyperspherical prototype classification setup and the max-min baseline; its projected-gradient optimization is the comparison that motivates Riemannian updates.","marker":"Mettes et al. (2019)"},{"why":"Defines the alignment/uniformity view and the RBF-based uniformity objective that the paper reinterprets as MMD-based dispersion, and provides the contrastive-learning motivation.","marker":"Wang & Isola (2020)"},{"why":"Introduces minimum hyperspherical energy with Riesz and RBF kernels, the pairwise family the paper compares and reinterprets.","marker":"Liu et al. (2018)"},{"why":"Provides the MMD estimator and its Lemma 6, which the paper combines with its Lemma 2 to connect kernel energy to MMD.","marker":"Gretton et al. (2012)"},{"why":"Origin of the alternating Voronoi/barycenter algorithm that the paper turns into an online stochastic dispersion regularizer.","marker":"Lloyd (1982)"},{"why":"Extends Lloyd-style quantization and stochastic gradient versions to Riemannian manifolds, giving the theoretical grounding for the Lloyd regularizer.","marker":"Le Brigant & Puechmorel (2019)"},{"why":"Stochastic Riemannian SGD convergence theorem used in Appendix C to justify the Lloyd regularizer's convergence via bounded gradients.","marker":"Bonnabel (2013)"},{"why":"Introduces spherical sliced Wasserstein and the closed-form Wasserstein distance on the circle that the Sliced regularizer's circle projection builds on and is compared against.","marker":"Bonet et al. (2023)"},{"why":"Riemannian Adam, the optimizer used across all experiments; the paper's claim that Riemannian updates beat projected Euclidean updates depends on this machinery.","marker":"Becigneul & Ganea (2019)"}],"fun_headline_variants":["Sliced sphere dispersion lifts BLEU without pair costs","Online Lloyd and sliced great-circles: dispersion without pairs","Riemannian updates plus sliced loss: fast sphere dispersion","Dispersion on sphere with no distance matrix, faster and better"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim depends on the assumption that the heuristic clipping of the Sliced regularizer's gradient does not materially change what the objective does, since that gradient becomes unbounded when an embedding nears the equator of a sampled great circle and the standard stochastic-convergence argument no longer applies.","fun_headline_variants_meta":{"raw":{"variants":["Sliced sphere dispersion lifts BLEU without pair costs","Online Lloyd and sliced great-circles: dispersion without pairs","Riemannian updates plus sliced loss: fast sphere dispersion","Dispersion on sphere with no distance matrix, faster and better"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000331,"raw_usage":{"total_tokens":1873,"prompt_tokens":1005,"completion_tokens":868,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":621,"completion_tokens_details":{"reasoning_tokens":797}},"tokens_in":621,"tokens_out":868,"duration_ms":9234,"temperature":1.0,"reasoning_tokens":797,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T05:55:29.535721+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"In the clumped synthetic setting of Section 4.2, run the Sliced regularizer with clipping thresholds varying over several orders of magnitude, and also with the denominator $\\langle x_i,Q\\rangle^2+\\langle x_i,P\\rangle^2$ replaced by a smoothed $+\\varepsilon$ version; if the final spherical variance and minimum distance change substantially with the threshold, the reported Sliced success comes from the clipping heuristic rather than from the dispersion objective itself.","supporting_citations":[{"cited_title":"Hyperspherical prototype networks","cited_arxiv_id":null,"evidence_quote":"Supplies the hyperspherical prototype classification setup and the max-min baseline; its projected-gradient optimization is the comparison that motivates Riemannian updates."},{"cited_title":"Quantization and clustering on riemannian manifolds with an application to air traffic analysis","cited_arxiv_id":null,"evidence_quote":"Extends Lloyd-style quantization and stochastic gradient versions to Riemannian manifolds, giving the theoretical grounding for the Lloyd regularizer."},{"cited_title":"Stochastic gradient descent on R iemannian manifolds","cited_arxiv_id":null,"evidence_quote":"Stochastic Riemannian SGD convergence theorem used in Appendix C to justify the Lloyd regularizer's convergence via bounded gradients."},{"cited_title":"Spherical S liced- W asserstein","cited_arxiv_id":null,"evidence_quote":"Introduces spherical sliced Wasserstein and the closed-form Wasserstein distance on the circle that the Sliced regularizer's circle projection builds on and is compared against."},{"cited_title":"Riemannian adaptive optimization methods","cited_arxiv_id":null,"evidence_quote":"Riemannian Adam, the optimizer used across all experiments; the paper's claim that Riemannian updates beat projected Euclidean updates depends on this machinery."}],"review_version":1}