REVIEW 2 major objections 6 minor 22 references
A generalization for the expected value of the earth mover's distance
T0 review · 2 major / 6 minor · reviewed 2026-08-27 · deepseek-v4-flash
Pith's one-line read This paper proves that the expected value of the generalized earth mover's distance among d random distributions on n points is given by a recursive formula, with the three-distribution case reducing to pairwise distances.
desk verdict Solid generalization of the expected-EMD result to d distributions; the main theorem holds up, with minor gaps in a denominator exponent and an overclaimed data section. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing device is the generalized Robinson-Schensted-Knuth (RSK) correspondence, a bijection between $d$-tuples of compositions and $d$-dimensional nonnegative-integer arrays whose support is a chain in the product order. Because the cost array $C(m)=\sum_{i=1}^{\lfloor d/2\rfloor}(\tilde{m}_{d-i+1}-\tilde{m}_i)$ (the sum of outside-in differences of the sorted coordinates) satisfies the Monge property, the monotonicity condition that lets a greedy algorithm find an optimal transport plan, an optimal plan can always be chosen with chain support; the RSK array is that plan, and the discrete EMD equals the sum of $C$ evaluated on the columns of the associated $d \times s$ matrix. This reduces the minimization in the definition of $\mathrm{EMD}_d$ to a direct sum, which is then packaged into the recursive generating function $H_{\mathbf{n}}(z,t)$ whose $z$-derivative yields the expected value.
What would settle it
Fix $d=4$ and $n=3$, enumerate all $4$-tuples of compositions of $s=5$, compute the discrete EMD by brute-force minimization over all feasible arrays in equation (4), and compare with the RSK column-sum value from Theorem 4; a single mismatch would disprove the optimality assumption.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is Theorem 7: for $\mathbf{n}=(n_1,\dots,n_d)$, the expected value $E(\mathbf{n}_d)$ of $\mathrm{EMD}_d$ on $P_n \times \cdots \times P_n$ satisfies $E_{\mathbf{n}} = (\sum_{i=1}^d (n_i-1) E_{\mathbf{n}-e(i)} + C(\mathbf{n})) / (|\mathbf{n}| - d + 1)$, with $E_{\mathbf{n}}=0$ whenever some $n_i=0$, where $C(\mathbf{n})$ is the cost of the maximal position, namely the sum of the $\lfloor d/2\rfloor$ largest-minus-smallest coordinate differences. This recursion comes from a generating function $H_{\mathbf{n}}(z,t)$ whose coefficients count $d$-tuples by their discrete EMD value; differentiating at $z=1$ converts exponents into sums of distances, and letting the total mass $s$ tend to infinity turns the discrete expectation into the continuous one. The same machinery identifies the generating function with the Hilbert series of the Segre embedding, and yields the observation that the unit-normalized expectation alternates with the parity of $d$.
Load-bearing premise
The load-bearing premise is that the combinatorial bookkeeping procedure used to pair up histograms always produces a cheapest transport plan, and never misses a cheaper one; if it ever did, the formula would overestimate the true distance.
Editorial extensions
If this is right
- For any fixed $n$ and $d$, the expected generalized EMD can be computed exactly by the recursion in $O(dn)$ time, replacing Monte Carlo estimates.
- For $d=3$ the expected value is $3/2$ times the two-distribution expected value, so the three-way spread carries no information beyond its pairwise spreads.
- The unit-normalized expected distance is alternately larger for odd $d$ and smaller for even $d$, so an even number of random distributions look closer than an odd number under this metric.
- The generating function's coincidence with the Hilbert series of the Segre embedding ties the expected transport cost to a known object in algebraic geometry, so standard Hilbert-series techniques can be brought to bear on EMD computations.
- The method supplies a single scalar for comparing arbitrarily many empirical distributions, demonstrated on university grade distributions across seven course sections.
Reading between the lines
- If the RSK optimality assumption is correct, a testable extension is to replace the taxicab cost by other Monge cost arrays; the same generating-function recursion would then predict expected optimal transport costs for those metrics.
- The parity alternation suggests that in odd dimensions the median coordinate acts as a free variable; one could design experiments with real grade or sensor data to check whether odd-sized groups genuinely appear more spread than even-sized groups.
- Because the application rounds empirical histograms to a common integer mass, an implicit and untested assumption is that the discrete EMD is stable under small rounding; a natural follow-up would bound the rounding error.
- The Segre-embedding link implies the Hilbert polynomial of the simple tensors may give asymptotic expansions of $E(\mathbf{n}_d)$ as $n$ grows, which the paper does not pursue.
Formalized claims in Lean
-
Claim #1: On the paper's own terms, the central discovery is Theorem 7: for $\mathbf{n}=(n_1,\dots,n_d)$, the expected value $E(\mathbf{n}_d)$ of $\mathrm{EMD}_d$ on $P_n \times \cdots \times P_n$ satisfies $E_{\mathbf{n}} = (\sum_{i=1}^d (n_i-1) E_{\mathbf{n}-e(i)} + C(\mathbf{n})) / (|\mathbf{n}| - d + 1)$, with $E_{\mathbf{n}}=0$ whenever some $n_i=0$, where $C(\mathbf{n})$ is the cost of the maximal pos
/-- @claim 1 On the paper's own terms, the central discovery is Theorem 7: for $\mathbf{n}=(n_1,\dots,n_d)$, the expected value $E(\mathbf{n}_d)$ of $\mathrm{EMD}_d$ on $P_n \times \cdots \times P_n$ satisfies $E_{\mathbf{n}} = (\sum_{i=1}^d (n_i-1) E_{\mathbf{n}-e(i)} + C(\mathbf{n})) / (|\mathbf{n}| - d + 1)$, with $E_{\mathbf{n}}=0$ whenever some $n_i=0$, where $C(\mathbf{n})$ is the cost of the maximal pos -/ def central_claim : Prop :=
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper defines a multi-marginal earth mover's distance EMD_d on d probability distributions on [n], using the taxicab distance to the diagonal as the cost function. It proves the Monge property of this cost array, introduces a discrete version based on a chain-support bijection from d-tuples of compositions (a generalized RSK correspondence), constructs a two-variable generating function H_n(z,t), differentiates it to extract sums of discrete EMD values, and derives a recursive formula for the expected value of the continuous EMD_d under the uniform distribution on the simplex. The paper also proves that EMD_3 is half the sum of the three pairwise EMD_2 values, illustrates the method on grade distributions, and connects H_n(1,t) to the Hilbert series of the Segre embedding and to a representation of gl_{p+q}.
Significance. If the main theorem is correct, the paper provides an explicit recursive baseline for the expected value of a natural d-distribution generalization of the one-dimensional 1-Wasserstein distance, extending the Bourn-Willenbring result. The self-contained proof of the Monge property in Section 8, the explicit generating-function recursion, and the clean d=3 reduction are genuine strengths. The connection to the Segre Hilbert series is elegant and likely to be of independent interest. The paper is honest about the algorithmic and data-analytic claims and includes reproducible-looking tables and small worked examples.
major comments (2)
- [§5.2, Eq. (15)] The assertion that the denominator of H'_n has exponent |n|-d+2 is introduced as "anticipated" from computations for small n. This is a load-bearing step: the recursion (16) for N_n, and therefore the main formula (18), is obtained by clearing denominators with this exponent. Please supply a proof, for example by induction from Proposition 6 and the known denominator of H_n in (11), showing that the higher-order poles cancel through the inclusion-exclusion identity. Without such a proof, Theorem 7 is not fully established.
- [§5.3, limit defining E_n] The passage from the discrete expected value E(EMD_s^d) to the continuous expected value E(EMD_d) needs a convergence argument. One needs to justify that (1/s) EMD_s^d(µ^{(s)}) converges to EMD_d(µ) as s→∞, where µ^{(s)} is the composition obtained from µ by scaling and rounding, and that the discrete expectations converge to the continuous expectation. The paper currently computes only the limit of the discrete formula; a brief statement specifying the mode of convergence, for instance via continuity of optimal transport costs and uniform approximation on the compact simplex, would close this gap.
minor comments (6)
- [§5.4, maximum of EMD_d] The claim that the maximum of EMD_d is floor(d/2)(n-1) is only sketched. The upper bound follows from Proposition 1, and equality can be exhibited explicitly by taking floor(d/2) distributions supported at bin 1 and floor(d/2) distributions supported at bin n, with an arbitrary remaining distribution when d is odd. Please make this construction explicit.
- [§4.3, Proposition 5 proof] The sentence saying that the three pairs (a_j,b_j), (a_j,c_j), (b_j,c_j) "correspond naturally" to the three pairs of distributions is misleading, because the entries in a column are sorted by value rather than by row label. The equality still holds because the sum of pairwise absolute differences is invariant under permutation of the three entries; please rephrase the justification accordingly.
- [§5.3, paragraph on coefficients] The phrase "we have s−b→s" should read "s−b∼s" or "s−b is asymptotic to s"; as written it is a typo.
- [§6, data application] The comparison of the two observed EMD values to the expected value is made without any measure of variability or uncertainty. The phrase "significantly less than the expected value" is not justified by the data analysis alone; either soften the claim or support it with a simulation-based confidence statement.
- [§5.4, Figure 2] The claim that the histograms have "nearly zero skew" for odd d is qualitative; a numerical skewness value or a more precise statement about the histogram shapes would make the observation easier to verify.
- [General] There are several typographical issues, including "distibution" in Section 3.1, "sutracting" in the proof of Proposition 6, and "the the" in the introduction. A careful proofreading pass is recommended.
Circularity Check
No significant circularity: the main theorem is a self-contained enumeration of EMD values via an RSK bijection and a proved Monge property.
full rationale
The derivation chain is not circular. Theorem 7 is obtained by (i) defining a discrete EMD_s_d as the cost of the unique chain-supported array J_mu supplied by the RSK bijection (5); (ii) proving in Section 8 that the cost array has the Monge property, so by Corollary 3 some chain-supported array realizes the minimum in (4); and (iii) because the RSK map is a bijection, that optimal array must be J_mu, so the discrete EMD agrees with the true transport minimum. The generating function H_n in (7) is an enumerative device—it records EMD values as exponents of z—and the subsequent differentiation and coefficient extraction are algebraic manipulations, not fitted inputs or renamed predictions. No parameter is fitted to the data whose expectation is computed, and the asymptotic normalization by 1/s is a legitimate scaling of the discrete model. The citations to [4], [6], and [3] are to independent prior work (Bourn–Willenbring, Caselli, Bein–Brucker–Park–Pathak), not to the present author, and the load-bearing Monge property is proved inside the paper rather than imported. The d=2 specialization agrees with the known expected value, providing an external check. Any remaining concerns about the rigor of the s->infinity limit or the data-analysis claim are correctness or presentation issues, not circularity.
Assumptions & free parameters
assumptions (6)
- standard math Generalized RSK correspondence is a bijection between d-tuples of compositions and chain-support arrays (5).
- domain assumption Uniform probability measure on P_n^d is the product of Lebesgue measure on the simplex via embedding into R^{dn}.
- domain assumption The discrete uniform measure on compositions C(s,n) converges to Lebesgue measure on the simplex as s tends to infinity after scaling by 1/s.
- standard math Closed form of H_n(1,t) with Simon Newcomb coefficients from [8].
- standard math Monge property of a d-dimensional array follows from Monge property of every 2D plane, and 2D Monge is characterized by 2x2 inequalities.
- domain assumption The cost function C in (3) is the intended generalization of the EMD cost.
Cite this review
Pith. "Pith review of A generalization for the expected value of the earth mover's distance." pith.science (2026). https://pith.science/paper/4MMKEXST
@misc{pith2026200912723,
author = {Pith},
title = {Pith review of: A generalization for the expected value of the earth mover's distance},
year = {2026},
howpublished = {\url{https://pith.science/paper/4MMKEXST}},
note = {Machine review of arXiv:2009.12723}
}
abstract
The earth mover's distance (EMD), also called the first Wasserstein distance, can be naturally extended to compare arbitrarily many probability distributions, rather than only two, on the set $[n]=\{1,\dots,n\}$. We present the details for this generalization, along with a highly efficient algorithm inspired by combinatorics; it turns out that in the special case of three distributions, the EMD is half the sum of the pairwise EMD's. Extending the methods of Bourn and Willenbring (arXiv:1903.03673), we compute the expected value of this generalized EMD on random $d$-tuples of distributions, using a generating function which coincides with the Hilbert series of the Segre embedding. We then use the EMD to analyze a real-world data set of grade distributions.
Figures
Reference graph
Works this paper leans on
-
[1]
Abramson, A simple solution of Simon Newcomb’s problem , J
M. Abramson, A simple solution of Simon Newcomb’s problem , J. Combin. Theory Ser. A 18 (1975), 223–225
work page 1975
-
[2]
A. Aggarwal and J.K. Park, Sequential searching in multidimensional monotone arrays , Research Report RC 15128, IBM T.J. Watson Research Center, Yorktown Heights, NY, 1989
work page 1989
-
[3]
W. Bein, P. Brucker, J. Park, and P. Pathak, A Monge property for the d-dimensional transport problem, Discrete Appl. Math. 58 (1995), no. 2, 97–109
work page 1995
-
[4]
R. Bourn and J. Willenbring, Expected value of the one-dimensional earth mover’s distance , A. Stat. 11 (2020), no. 1, 53–78
work page 2020
-
[5]
W. Bruns and J. Herzog, Cohen-Macaulay rings, Cambridge University Press, 1993
work page 1993
-
[6]
F. Caselli, On the multivariate Robinson-Schensted correspondence , Bollettino dell’Unione Matematica Italiana 9 (2009), no. 1, 591–602
work page 2009
-
[7]
C ¸ olako˘ glu,On the distance formulae in the generalized taxicab geometry , Turkish J
H. C ¸ olako˘ glu,On the distance formulae in the generalized taxicab geometry , Turkish J. Math. 43 (2019), no. 3, 1578–1594
work page 2019
-
[8]
J. Dillon and D. Roselle, Simon Newcomb’s problem, SIAM J. Appl. Math. 17 (1969), no. 6, 1086–1093
work page 1969
Show all 22 references
-
[9]
Enright and J
T. Enright and J. Willenbring, Hilbert series, Howe duality, and branching for classical groups , Ann. of Math. 159 (2004), no. 1, 337–375
2004
-
[10]
Frohmader and H
A. Frohmader and H. Volkmer, 1-Wasserstein distance on the standard simplex , arXiv:1912.04945, 2019
1912 arXiv
-
[11]
Fulton, Young tableaux, Cambridge University Press, 1997
W. Fulton, Young tableaux, Cambridge University Press, 1997
1997
-
[12]
Goodman and N
R. Goodman and N. Wallach, Symmetry, representations, and invariants , Springer, 2009
2009
-
[13]
Harris, Algebraic geometry: a first course , Springer-Verlag, 1995
J. Harris, Algebraic geometry: a first course , Springer-Verlag, 1995
1995
-
[14]
Hoffman, On simple linear programming problems, Convexity: Proceedings of the Seventh Symposium in Pure Mathematics of the AMS (V
A. Hoffman, On simple linear programming problems, Convexity: Proceedings of the Seventh Symposium in Pure Mathematics of the AMS (V. Klee, ed.), American Mathematical Society, Providence, RI, 1963, pp. 317–327
1963
-
[15]
Roger Howe, Remarks on classical invariant theory , Trans. Amer. Math. Soc. 313 (1989), no. 2, 539–570
1989
-
[16]
Willenbring, Stable branching rules for classical symmetric pairs , Trans
Roger Howe, Eng-Chye Tan, and Jeb F. Willenbring, Stable branching rules for classical symmetric pairs , Trans. Amer. Math. Soc. 357 (2005), no. 4, 1601–1626
2005
-
[17]
Kline, Properties of the d-dimensional earth mover’s problem , Discrete Appl
J. Kline, Properties of the d-dimensional earth mover’s problem , Discrete Appl. Math. 265 (2019), 128–141
2019
-
[18]
Kretschmann, Earth mover’s distance between grade distribution data with fixed mean , Master’s thesis, Uni- versity of Wisconsin-Milwaukee, 2020
J. Kretschmann, Earth mover’s distance between grade distribution data with fixed mean , Master’s thesis, Uni- versity of Wisconsin-Milwaukee, 2020
2020
-
[19]
Monge, M´ emoire sur la th´ eorie des d´ eblais et des remblais, Histoire de l’Acad´ emie Royale des Sciences de Paris, 1781, pp
G. Monge, M´ emoire sur la th´ eorie des d´ eblais et des remblais, Histoire de l’Acad´ emie Royale des Sciences de Paris, 1781, pp. 666–704
-
[20]
Morales, Segre embeddings, Hilbert series, and Newcomb’s problem , HAL ID: hal-00839652, 2013
M. Morales, Segre embeddings, Hilbert series, and Newcomb’s problem , HAL ID: hal-00839652, 2013
2013
-
[21]
Park, The Monge array: an abstraction and its applications , Ph.D
J. Park, The Monge array: an abstraction and its applications , Ph.D. thesis, Massachusetts Institute of Technol- ogy, 1991
1991
-
[22]
Villani, Optimal transport, old and new , Springer, 2008
C. Villani, Optimal transport, old and new , Springer, 2008. 3200 North Cramer Street, Milwaukee, WI 53211 Email address: wqe@uwm.edu
2008
Reviewed August 27, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.