REVIEW 3 major objections 6 minor 1 cited by
Understanding Learning with Sliced-Wasserstein Requires Rethinking Informative Slices
T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper argues that classical Sliced-Wasserstein, with a properly tuned learning rate, can match or beat more complex informative-slice variants in common learning tasks.
desk verdict Solid subspace scaling theorem marred by an overstated learning-rate absorption claim and a false appendix proposition; still worth refereeing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the $\phi$-weighting formulation $gSW_p^p(\mu,\nu;\sigma,\rho_\phi) = \int_{S^{d-1}} \rho_\phi(\phi(\theta))\, W_p^p(\theta_\#\mu, \theta_\#\nu)\, d\sigma(\theta)$, which reweights each projected 1D Wasserstein by a function of slice informativeness instead of resampling slices. With the ES-aligned informativeness $\phi_U(\theta) = \lVert U^\top \theta \rVert$ and the reciprocal weighting $\rho_\phi(r) = r^{-p}$, Proposition 4.7 shows each slice contribution satisfies $W_p^p(\theta_\#\mu_d, \theta_\#\nu_d) = \lVert U^\top \theta \rVert^p W_p^p(\theta_{k\#}\mu_k, \theta_{k\#}\nu_k)$, so the reweighted distance equals the SWD in the $k$-dimensional effective subspace. Theorem 4.9 then identifies the expectation of $\lVert U^\top \theta \rVert^p$ with the ratio $C_k/C_d$ of two Gamma-function constants, turning per-slice rescaling into one global scalar. Proposition 4.10 extends this to finite samples and Monte Carlo slice estimators, with variance $O(1/L)$. The machinery's role is to convert the concentration-of-measure objection, that most random slices are nearly orthogonal to the data subspace, into a harmless multiplicative constant that learning-rate tuning already handles.
What would settle it
One concrete check: take two distributions on a common $k$-dimensional subspace and verify the empirical ratio of ambient to subspace sliced-Wasserstein approaches $C_k/C_d$ as the number of slices grows; the broader learning claim would be refuted by a common task, after a full learning-rate sweep, where plain SWD falls materially short of a tuned informative-slice variant.
Extended reading notes
Core claim
Under Assumption 4.1, where both measures are supported on the same $k$-dimensional linear subspace $V_k$ of $\mathbb{R}^d$, the $p$-th power of the sliced-Wasserstein distance obeys $SW_p^p(\mu_d, \nu_d) = (C_k/C_d)\, SW_p^p(\mu_k, \nu_k)$, with $C_d = 2^{p/2}\Gamma((d+p)/2)/\Gamma(d/2)$ and $C_k$ defined analogously. In other words, the information lost by averaging over all random projections of high-dimensional data is not a per-slice accident: it is a single dimension-dependent scaling constant. Defining slice informativeness as $\phi_U(\theta) = \lVert U^\top \theta \rVert$, the alignment of a direction $\theta$ with the effective subspace, rescaling each one-dimensional Wasserstein contribution by $\phi_U(\theta)^{-p}$ exactly cancels the implicit downweighting of misaligned slices, and in expectation this per-slice reweighting collapses to the global factor $C_d/C_k$. The paper's central claim is that in gradient-based learning this global factor is absorbed by learning-rate selection, so classical SWD with a properly chosen step size obtains informative slices for free and can perform competitively with more complex SW variants in common learning tasks.
Load-bearing premise
The argument assumes the two distributions sit on the same low-dimensional linear subspace, and then presumes the resulting rescaling factor stays roughly constant across minibatches so a single learning rate can absorb it.
Editorial extensions
If this is right
- If the central claim is correct, the classical SWD needs no data-dependent slicing distribution to be competitive in common tasks: a learning-rate sweep over the plain distance is sufficient.
- The effective-dimensionality dependence of the scaling factor means the same classical SWD automatically adapts to data with lower intrinsic dimension, without explicitly estimating the subspace.
- Nearly all informative-slice SW variants, including Max-SW, EBSW, Markovian SW, and RPSW, are recovered as special cases of the $\phi$-weighting framework, so the metric properties of the classical distance are preserved when the global rescaling is applied.
- Because the global factor can be absorbed into a learning rate, comparisons among SW variants in learning tasks should control for learning-rate search; reported gaps can shrink or reverse when the classical baseline is tuned.
- Proposition 4.10 gives convergence rates for the gap between the ambient Monte Carlo SWD and the scaled subspace SWD, so users know how many slices are needed for the scalar-factor reduction to hold.
Reading between the lines
- A testable extension is to compare classical SWD against informative-slice variants on data whose support is a curved manifold rather than a linear subspace, where per-slice reweighting may not collapse to a global scalar and the advantage of adaptive slicing could reappear.
- The assumption of a shared common subspace for both distributions is stronger than the manifold hypothesis; if source and target occupy different low-dimensional subspaces, the global-factor argument may fail, and explicit subspace-aware methods would retain an edge.
- Because the scaling factor depends only on ambient and effective dimension, one could estimate the effective dimension from minibatches and predict the optimal learning-rate shift across tasks, making the learning-rate search more principled.
- If the claim is right, benchmark protocol changes: future SW-variant evaluations should include a well-tuned classical SWD baseline, otherwise apparent gains may reflect learning-rate mismatch rather than a genuine advantage of the new method.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies the sliced-Wasserstein distance (SWD) for high-dimensional data whose support lies in a low-dimensional linear subspace. Under Assumption 4.1, it proves Theorem 4.9: the ambient SW_p^p is a fixed scalar multiple (C_k/C_d) of the SW_p^p computed in the effective subspace. It introduces a ϕ-weighting framework for rescaling 1D Wasserstein contributions by slice informativeness and argues that the per-slice reweighting simplifies to this single global factor, which can be absorbed into the learning rate in gradient-based learning. The paper reports experiments on gradient flow, color transfer, and deep generative modeling on FFHQ, showing that classical SWD with a well-chosen learning rate matches or outperforms more complex SW variants.
Significance. If the central claim holds, the paper provides a clean explanation of why the classical SWD can be competitive with data-adaptive SW variants: the ill-conditioning associated with uninformative slices is a global scaling phenomenon, rather than a per-slice one. The unifying ϕ-weighting formulation (Section 4.1) is a useful organizing framework. The population-level identity (Theorem 4.9) is exact and the synthetic verification (Section 5.1) is convincing. The paper also ships explicit convergence statements (Propositions 4.10 and 4.12) and an extensive experimental comparison with runtime tables. The main weakness is that the translation from the exact population identity to the finite-slice, minibatch setting used in all practical experiments is not rigorously established; this is the load-bearing step for the headline claim that informative slices come 'for free' via learning-rate search.
major comments (3)
- [§4.4, Proposition 4.10] The claim that the Effective Subspace Scaling Factor (ESSF) 'can be absorbed into the learning rate' is only established in expectation and asymptotically. For the Monte Carlo estimator actually optimized in the experiments (L=50), the classic SWD is (1/L) Σ_l ||U^T θ_l||^p W_p^p(θ_k^l#µ_k, θ_k^l#ν_k), while the rescaled subspace estimator is [(1/L)Σ_l ||U^T θ_l||^p][(1/L)Σ_l W_p^p(θ_k^l#µ_k, θ_k^l#ν_k)]. These are equal only in expectation; their difference is nonzero for every finite L and enters the gradient nonlinearly through the sorting in each 1D Wasserstein term. Propositions 4.10(2) and 4.12 only give convergence to zero of this error as L→∞. With L=50, no finite-L bound is provided, and the paper reports no measurement of the per-iteration ESSF variability in the actual optimization tasks. The statement in Section 4.4 that the factor 'can be absorbed' is therefore an approximation, not a proven equivalence, and the learning-rate search in the experiments may be compensating for this mismatch. The authors should either provide finite-L gradient-error bounds or explicitly reframe the claim as a heuristic supported by the accompanying empirical study.
- [§4.4, Remark 4.11, experiments] In minibatch training, the effective subspace U is not fixed across iterations: Remark 4.11 bounds its dimension by min{2B-1, d}, but the subspace itself changes with each minibatch. Consequently, the expected ESSF, C_k/C_d, is not a single constant throughout optimization; it drifts with U. The paper does not state any assumption under which this drift is negligible, and the experiments use a sweeping learning-rate grid that can absorb per-iteration scaling differences. To support the central claim, the authors should either derive a bound on the drift of the ESSF under minibatch updates or present empirical evidence (e.g., tracking ESSF values over training) that the drift is small relative to the learning-rate sensitivity.
- [Appendix A.3, Proposition A.7] Proposition A.7 states that for any full-rank covariance Σ, f#N(0,Σ) equals the uniform distribution on S^{d-1}, where f(x)=x/||x||. This is false for anisotropic Gaussians: for Σ=diag(σ1^2,...,σd^2), the random direction X/||X|| has a density proportional to ∏ σ_i^{-1} exp(-x_i^2/(2σ_i^2)) on the sphere, which is not constant unless all σ_i are equal. The proof's Step 2 incorrectly identifies Σ^{-1/2}X/||Σ^{-1/2}X|| with X/||X||. The isotropic case used in Proposition A.9 is correct, so this error does not invalidate Theorem 4.9, but the misleading general statement and its proof must be corrected or removed.
minor comments (6)
- [Appendix A.3, Remark A.8] Remark A.8 contains the typo 'distritbution' and the phrase 'rotationally invariant distritbution'; the sentence should read 'rotationally invariant distribution'.
- [Section 5.2.3] The sentence 'The main observation is that all methods perform comparatively, aibeit different learning rates are required for good results' contains the typo 'aibeit' for 'albeit'.
- [Theorem 4.9 vs. Appendix A.4] The appendix heading 'A.4 Relationship between the SWD in Rd and Rk' refers to 'Proposition 4.9' in the proof, but the result is numbered Theorem 4.9 in the main text; the cross-reference should be corrected.
- [Appendix A.6.1] The subsection heading 'A.6.1 Proof of Theorem 4.10' appears before the proof of Proposition 4.10; the theorem number should be 'Proposition 4.10'.
- [Figure 10 caption] Figure 10 is in the Color Transfer section but its caption reads 'Optimal basin plots for Gradient Flow with embedded synthetic datasets.' This caption should be updated to refer to the color transfer results.
- [Proposition 4.10(3)] The concentration inequality in Proposition 4.10(3) is stated as P(ϵL < δ) ≥ 1 - e^{-δ^2L/K^2}, but the proof in Appendix A.6 writes a different expression; the constants and exponent should be checked for consistency.
Circularity Check
No significant circularity: the subspace scaling identity is derived from an external Gaussian-SW equivalence, and the learning-rate interpretation is an overclaim but not a circular reduction.
full rationale
Theorem 4.9's identity SW_p^p(µd,νd)=(C_k/C_d) SW_p^p(µk,νk) is proven by combining Proposition A.9—an external result of Nadjahi et al. (2021), which the paper explicitly disclaims as its own contribution—with the elementary projection identity in Lemma A.12/A.14. No parameter is fitted to data to produce the identity; the ratio C_k/C_d is a closed-form function of dimensions only. The ES-aligned informativeness φ_U(θ)=||U^Tθ|| and the reciprocal weight 1/||U^Tθ||^p are definitions, and Proposition 4.7's factorization is a mathematical fact rather than a renaming of the conclusion. The later claim that the global scale 'can be absorbed into the learning rate' is only asserted heuristically in Section 4.4: Proposition 4.10 and 4.12 give convergence in expectation and as L→∞, not exact proportionality for L=50, so the finite-L gradient mismatch is a correctness/rigor concern, not a circularity. The self-citations (Kolouri et al. 2019 for a standard concentration bound; Tran et al. 2024 in related work) are not load-bearing: the concentration inequality is an externally checkable mathematical fact. Thus no step reduces by construction to its own input.
Assumptions & free parameters
free parameters (2)
- Learning rate h =
Grid-searched per task, e.g., {1,3,5,8} x 10^{-6} to 10^{2}
- Number of random projections L =
50 for all experiments
assumptions (5)
- domain assumption Assumption 4.1: both measures are exactly supported on the same k-dimensional linear subspace V_k, with k minimal.
- standard math Gaussian-SW equivalence of Nadjahi et al. (2021): SW_p^p(µ,ν; N(0,I_d)) = C_d SW_p^p(µ,ν; U(S^{d-1})).
- standard math For an isotropic Gaussian, the radius and direction are independent, and the normalized vector is uniform on the sphere.
- domain assumption Manifold hypothesis: high-dimensional image data is approximately supported on a low-dimensional manifold.
- standard math For a minibatch of 2B samples, the empirical support lies in a subspace of dimension at most min{2B-1, d} after centering.
Cite this review
Pith. "Pith review of Understanding Learning with Sliced-Wasserstein Requires Rethinking Informative Slices." pith.science (2026). https://pith.science/paper/ZQ7TCR3I
@misc{pith2026241110651,
author = {Pith},
title = {Pith review of: Understanding Learning with Sliced-Wasserstein Requires Rethinking Informative Slices},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZQ7TCR3I}},
note = {Machine review of arXiv:2411.10651}
}
read the original abstract
The practical applications of Wasserstein distances (WDs) are constrained by their sample and computational complexities. Sliced-Wasserstein distances (SWDs) provide a workaround by projecting distributions onto one-dimensional subspaces, leveraging the more efficient, closed-form WDs for one-dimensional distributions. However, in high dimensions, most random projections become uninformative due to the concentration of measure phenomenon. Although several SWD variants have been proposed to focus on \textit{informative} slices, they often introduce additional complexity, numerical instability, and compromise desirable theoretical (metric) properties of SWD. Amidst the growing literature that focuses on directly modifying the slicing distribution, which often face challenges, we revisit the classical Sliced-Wasserstein and propose instead to rescale the 1D Wasserstein to make all slices equally informative. Importantly, we show that with an appropriate data assumption and notion of \textit{slice informativeness}, rescaling for all individual slices simplifies to \textbf{a single global scaling factor} on the SWD. This, in turn, translates to the standard learning rate search for gradient-based learning in common machine learning workflows. We perform extensive experiments across various machine learning tasks showing that the classical SWD, when properly configured, can often match or surpass the performance of more complex variants. We then answer the following question: "Is Sliced-Wasserstein all you need for common learning tasks?"
Figures
Figures from the paper (16 more)
Forward citations
Cited by 1 Pith paper
-
Constrained Sliced Wasserstein Embedding
Adding SWGG dissimilarity constraints to sliced Wasserstein embedding, trained via primal-dual optimization with a softsort relaxation, improves pooling accuracy on image, point cloud, and protein-sequence benchmarks.
Reference graph
Works this paper leans on
-
[1]
rotationally invariant distritbution
is a well-known result, and that is why isotropic Gaussian distribution is called a “rotationally invariant distritbution. ” We do not claim this proposition or its proof as contributions of this article; we present the proof merely for completeness. A.4 Relationship between the SWD in Rd and Rk In this section, we discuss the proof of the proposition 4.9...
-
[2]
34 LR SW MaxSW DSW MaxKSW iMSW viMSW oMSW rMSW EBSW RPSW EBRPSW 100 2.7116±0.66160.6408±0.00793.2280±0.50941.3326±0.04112.2337±0.87272.5000±0.83351.3862±0.26032.5671±0.86202.8073±0.14162.6577±0.02822.7618±0.1329 80 2.1367±0.60870.5710±0.00411.0384±0.36401.3753±0.11061.7140±0.17031.8681±0.62681.5932±0.10412.5953±0.68612.3348±0.08282.2001±0.06312.2090±0.161...
- [8]
-
[10]
Figure 17: The FFHQ dataset (Karras et al
8345 1405 Male 26732 4351 Female 32816 5572 (b) Subset size. Figure 17: The FFHQ dataset (Karras et al. (2019)) 43 LR SW MaxSW DSW MaxKSW iMSW viMSW oMSW rMSW EBSW RPSW EBRPSW 100 294.00 ± 0.38 287.00 ± 4.65 227.00 ± 4.40 275.00 ± 7.53286.00 ± 7.55 286.00 ± 7.36 294.00 ± 0.38201.00 ± 3.34 291.00 ± 0.71 291.00 ± 0.79291.00 ± 0.52 80 294.00 ± 0.38 286.00 ± ...
work page 2019
-
[11]
P. Pope, C. Zhu, A. Abdelkader, M. Goldblum, and T. Goldstein. The intrinsic dimension of images and its impact on learning. arXiv preprint arXiv:2104.08894,
-
[15]
It remains to show the convergence rate of error N . Since each ∥U ⊤θl∥p ∈ [0, 1], for each t > 0, by Hoeffding theorem, we have P(|Bn| ≥ϵ) ≤ e−2ϵ2L Replacing ϵ by ϵ/K, we have P(errorL ≤ ϵ) ≥ 1 − 2e 2ϵ2 L K2 and we complete the proof. A.6.1 Proof of Theorem 4.10 A.7 Special case: Learning rate bound for the SW Gradient Flow problem In this section, we co...
-
[17]
35 LR SW MaxSW DSW MaxKSW iMSW viMSW oMSW rMSW EBSW RPSW EBRPSW 100 1.8754±1.21833.6719±1.66692.2167±1.39101.1766±0.09012.0996±0.50582.3956±0.79933.0252±0.17062.6436±1.08823.0252±0.17062.6426±0.06692.8145±0.1491 80 1.6682±0.97271.8203±0.90882.2781±0.41021.1566±0.03182.0056±0.77641.8520±0.58242.2930±0.09031.8364±1.07022.2930±0.09032.0552±0.10702.3738±0.039...
-
[18]
36 LR SW MaxSW DSW MaxKSW iMSW viMSW oMSW rMSW EBSW RPSW EBRPSW 100 0.7900±0.07044.4368±0.09663.0902±0.10262.9073±0.39472.2985±0.12201.2282±0.05780.8969±0.05001.2429±0.116710.5129±0.22401.2812±0.016511.0450±0.8609 80 0.6047±0.08503.4799±0.05442.4792±0.26022.0292±0.27111.7091±0.09210.9662±0.02840.6897±0.04420.9003±0.03928.5668±0.39441.0411±0.05867.8665±0.8...
Show all 24 references
-
[20]
38 LR SW MaxSW DSW MaxKSW iMSW viMSW oMSW rMSW EBSW RPSW EBRPSW 100 0.7762±0.10234.1470±0.03522.9805±0.06422.5629±0.44432.0662±0.10621.2985±0.14300.7226±0.05451.0492±0.08378.9485±0.85841.2748±0.03328.9753±0.7660 80 0.6647±0.03593.2381±0.03672.2192±0.22202.1054±0.08411.6892±0.1...
-
[21]
39 LR SW MaxSW DSW MaxKSW iMSW viMSW oMSW rMSW EBSW RPSW EBRPSW 100 0.8643±0.04802.4103±0.02433.0249±0.09922.8531±0.35981.9101±0.09040.9712±0.02270.8463±0.04780.9859±0.064516.7324±0.30580.8955±0.038016.3655±0.8700 80 0.6125±0.02641.7029±0.04442.2926±0.19292.2264±0.11021.5574±0...
-
[23]
41 LR SW MaxSW DSW MaxKSW iMSW viMSW oMSW rMSW EBSW RPSW EBRPSW 100 0.8453±0.00231.7094±0.03852.8643±0.23562.6594±0.12272.1708±0.07070.8819±0.04980.8580±0.01261.0229±0.081514.9925±0.91490.9446±0.052414.6610±0.9302 80 0.6462±0.01442.3625±1.63012.5657±0.09372.1892±0.19151.6570±0...
-
[50]
37 LR SW MaxSW DSW MaxKSW iMSW viMSW oMSW rMSW EBSW RPSW EBRPSW 100 0.7547±0.01595.2763±0.05812.7308±0.23563.2047±0.15532.2315±0.01521.1926±0.10090.7655±0.07251.0225±0.02598.2665±0.67631.2582±0.16258.2852±0.4012 80 0.7061±0.05884.4078±0.87412.1265±0.57072.6758±0.24321.5470±0.0...
1926
-
[100]
40 LR SW MaxSW DSW MaxKSW iMSW viMSW oMSW rMSW EBSW RPSW EBRPSW 100 0.8999±0.02102.7462±0.04732.8735±0.12133.2882±0.20122.0599±0.06650.9238±0.01070.8228±0.05501.0054±0.046011.1502±0.54520.9537±0.006511.5940±0.2757 80 0.6750±0.03702.2929±0.04252.3658±0.17072.8335±0.01141.6259±0...
-
[2011]
Rabin, G
J. Rabin, G. Peyr ´e, J. Delon, and M. Bernot. Wasserstein barycenter and its application to texture mixing. In Scale Space and Variational Methods in Computer Vision: Third International Conference, SSVM 2011, Ein-Gedi, Israel, May 29–June 2, 2011, Revised Selected Papers 3, ...
2011
-
[2012]
G. Reeves. Conditional central limit theorems for gaussian projections. In 2017 IEEE International Symposium on Information Theory (ISIT), pages 3045–3049. IEEE,
2017
-
[2013]
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255. IEEE,
2009
-
[2015]
X. Chen, Y . Yang, and Y . Li. Augmented sliced wasserstein distances.arXiv preprint arXiv:2006.08812,
2006 arXiv
-
[2018]
Bonet, N
C. Bonet, N. Courty, F. Septier, and L. Drumetz. Sliced-wasserstein gradient flows.arXiv preprint arXiv:2110.10972, page 1, 2021a. C. Bonet, T. Vayer, N. Courty, F. Septier, and L. Drumetz. Subspace detours meet gromov–wasserstein. Algorithms, 14(12):366, 2021b. C. Bonet, P. B...
-
[2019]
Korotin, N
A. Korotin, N. Gushchin, and E. Burnaev. Light schr\” odinger bridge. arXiv preprint arXiv:2310.01174,
-
[2020]
Nguyen, N
K. Nguyen, N. Bariletto, and N. Ho. Quasi-monte carlo for 3d sliced wasserstein. arXiv preprint arXiv:2309.11713,
-
[2021]
Nguyen and N
K. Nguyen and N. Ho. Sliced wasserstein estimation with control variates. arXiv preprint arXiv:2305.00402,
-
[2022]
Bonet, L
C. Bonet, L. Drumetz, and N. Courty. Sliced-wasserstein distances and flows on cartan-hadamard manifolds. arXiv preprint arXiv:2403.06560,
-
[2023]
Nguyen, T
K. Nguyen, T. Ren, and N. Ho. Markovian sliced wasserstein distances: Beyond independent projections. Advances in Neural Information Processing Systems, 36, 2024a. K. Nguyen, S. Zhang, T. Le, and N. Ho. Sliced wasserstein with random-path projecting directions. arXiv preprint ...
-
[2024]
18 R. D. Martin, I. Medri, Y . Bai, X. Liu, K. Yan, G. K. Rohde, and S. Kolouri. Lcot: Linear circular optimal transport. arXiv preprint arXiv:2310.06002,
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.