Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence

Roman V. Belavkin

REVIEW 3 major objections 5 minor 13 references

Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The optimal-transport metric is the optimal channel cost with the output distribution fixed; under a gradient condition, the KL divergence decomposes into it plus corrections.

desk verdict Correct but elementary lower-bound observation; the equality theorems are overclaimed and Theorem 2's condition is nearly vacuous for metric costs. read the letter →

arxiv 1908.09211 v1 pith:D6DDAESQ submitted 2019-08-24 cs.IT math.ITmath.OC

Roman V. Belavkin This is my paper · ORCID

classification cs.ITmath.ITmath.OC MSC 49Q2294A17

keywords optimaltransportWassersteinmetricKantorovichKullback-Leiblerdivergencerate-distortiontheoryvalueofinformationchannellawcosines

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the optimal-transport distance between probability measures and the Kullback-Leibler divergence are connected rather than unrelated. Its main claim is that the optimal transport problem is the same as the rate-distortion optimal channel problem except for one extra constraint: the output distribution is fixed in advance. As a consequence, the optimal channel cost is always a lower bound on the optimal-transport cost, with equality exactly when the two problems share the same optimal joint distribution. The paper then uses a geometric law-of-cosines identity for the KL divergence to decompose $D[p,q]$ into the transport cost plus terms involving the dual potentials, under the condition that those potentials are log-likelihood ratios with respect to a common reference measure. If the argument holds, it gives a bridge between optimal-transport geometry and information geometry, with rate-distortion computations becoming usable as bounds on transport costs.

What carries the argument

The argument is carried by two devices. The first is the observation that the two-marginal optimal transport problem is exactly an optimal channel problem with the output marginal fixed: the rate-distortion channel has the Gibbs form $dw(x,y)=dq(x)\,dp(y)e^{-\beta c(x,y)-\kappa(\beta,x)}$, and its value $R_c[q](\lambda)$ competes with the transport value $K_c[p,q](\lambda)$. The second is the law of cosines for the KL divergence, $D[p,q]=D[p,r]+D[r,q]-\int\ln(dq/dr)(dp-dr)$, which, using the exponential representations $dp=e^{f-\kappa[f]}dr$ and $dq=e^{g-\kappa[g]}dr$, turns the dual transport objective $E_p\{f\}-E_q\{g\}$ into a component of the KL divergence.

What would settle it

For a three-point set with a generic cost matrix and two probability vectors $p,q$, solve the Kantorovich dual (6) to get potentials $f,g$, then check whether any reference measure $r$ satisfies $p_i = e^{f_i}r_i / Z_p$ and $q_i = e^{g_i}r_i / Z_q$. If no such $r$ exists, the exact identity cannot hold for that instance, and only the inequality remains.

Watch

Extended reading notes

Core claim

The central claim is that Kantorovich's optimal transport problem is the optimal channel (rate-distortion) problem with one additional constraint fixing the output marginal. Written with the same information constraint $I(X,Y)\leq\lambda$, the optimal channel value $R_c[q](\lambda)$ is always a lower bound on the transport value $K_c[p,q](\lambda)$, and the two are equal exactly when the same joint measure solves both problems. On the geometric side, the paper uses the law of cosines for the KL divergence to decompose $D[p,q]$ as a difference of expectations plus a correction term. If the optimal dual functions $f,g$ from the Kantorovich dual are themselves KL gradients with respect to a common reference measure $r$, then the decomposition becomes the exact additive identity $D[p,q]=K_c[p,q]-(\kappa[f]-\kappa[g])-\int g\,(dp-dq)$. Without that condition, only an inequality $D[p,q]\leq \epsilon K_c[p,q]-(\kappa[\beta f]-\kappa[\alpha g])-\alpha\int g\,(dp-dq)$ is available.

Load-bearing premise

The exact equality rests on the assumption that the two optimal dual functions are log-likelihood ratios of the two measures against the same reference measure; for arbitrary measures and costs, no such reference measure is guaranteed to exist.

Editorial extensions

If this is right

For every output measure $p$, the rate-distortion value $R_c[q](\lambda)$ is a lower bound on the optimal-transport cost $K_c[p,q](\lambda)$, so optimal channel computations give certificates for transport costs.
When the equality conditions hold, the optimal-transport cost can be recovered from a Kullback-Leibler-constrained optimization, connecting rate-distortion algorithms to optimal-transport algorithms.
The identity $D[w,q\otimes q]=I(X,Y)+D[p,q]$ converts the information constraint into a constraint on the divergence between the marginals, giving an alternative description of the constrained transport value.
For translation-invariant costs, the optimal channel simplifies to a form independent of the input marginal, so the corresponding transport cost depends on the output measure explicitly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The equality condition is likely to fail for generic non-exponential-family measures, so the practical payload of the paper is the inequality: rate-distortion lower bounds on Wasserstein distances that remain computable when the transport problem is hard.
The common-reference requirement suggests that exact equality holds precisely along exponential families, making the identity an information-geometric characterization of transport duality.
A numerical study on Gaussian mixtures or discrete graphical models could quantify the gap between $R_c[q](\lambda)$ and $K_c[p,q](\lambda)$ and reveal how tight the bound is.
The game-theoretic point that optimal channels beat transport plans when the output is unconstrained points toward stochastic maps that minimize expected cost without a fixed target output.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the core observation is correct but elementary, and the attempt to promote it to an exact equivalence runs into real problems. As a note, it deserves a referee; as a theorem paper, it overreaches.

The new bit that works is taking the Kantorovich OTP and noticing that fixing the output marginal is exactly the extra constraint that separates it from the rate-distortion / optimal channel problem. The inequality Rc[q](λ) ≤ Kc[p,q](λ) follows immediately by relaxing that constraint, and the paper states it plainly. The law-of-cosines decomposition in Section 3 is algebraically correct, and the discussion is appropriately cautious: it says the relations generally hold as inequalities.

The soft spots are in the equality claims. Theorem 1's proof asserts that the optimal channel lies in the subdifferential of the log-partition functional at −βc. But the OCP's information constraint involves the output marginal of w itself, not a fixed product measure, so the tilted product used in the proof isn't the OCP solution in general. The subdifferential is also a singleton, so the argument that any equal-cost transport plan would also lie in it is not available. Theorem 1's statement may be true for a different reason—if both problems share a solution, the values obviously agree—but the proof as written doesn't establish the 'only if'.

Theorem 2 is a bigger issue. The condition that the optimal dual potentials are log-likelihood ratios against one common reference r, f=ln(dp/dr), g=ln(dq/dr), is not harmless. Since the dual feasibility requires f(x)−g(y) ≤ c(x,y), taking y=x gives ln(dp/dq)(x) ≤ c(x,x). For the metric costs the KW metric is built from, c(x,x)=0, so the condition forces p≤q pointwise, hence p=q. The equality regime is essentially empty in the intended application. The paper gives no construction of r or characterization of when the condition holds, so Theorem 2 reduces to an inequality with an extra assumption that is not understood.

What remains is a correct lower bound and a nice geometric picture. The paper doesn't overclaim in the abstract, and the discussion is honest. A referee could repair Theorem 1 by proving the iff directly from feasibility, and could downgrade Theorem 2 to a conditional equality with a worked example. As a short note in a workshop or conference proceedings this might be fine; as a journal paper it needs revision. I'd send it to review, but with the expectation of major changes.

Referee Report

3 major / 5 minor

Summary. The paper studies the relation between the Kantorovich-Wasserstein cost Kc[p,q] defined in (1) and the KL divergence D[p,q] defined in (2). It introduces the optimal channel problem Rc[q](λ) in (3), which fixes only the input marginal and constrains mutual information, and observes that Rc[q](λ) ≤ Kc[p,q](λ) because the transport problem additionally fixes the output marginal. Theorem 1 claims that equality holds if and only if the two problems share the same joint optimizer, with a proof based on subdifferentials of the log-partition functional D*[u,q⊗p]. The paper also derives the Pythagorean identity (5) and the law-of-cosines identities (7)-(8), and uses the dual transport problem (6) to prove Theorem 2: if the optimal dual potentials f,g are log-density ratios against a common reference measure r, then D[p,q] = Kc[p,q] − (κ[f] − κ[g]) − ∫ g(dp−dq). The Discussion states that generally only inequalities are obtained.

Significance. The lower-bound inequality Rc[q](λ) ≤ Kc[p,q](λ) is correct and follows immediately from constraint relaxation, and the algebraic identities (5), (7), and (8) are correct by substitution. The paper is honest in its Discussion that the general relations are inequalities. However, the exact equality results are the main advertised contribution, and they are either false as stated (Theorem 1) or conditional on an uncharacterized and very restrictive hypothesis (Theorem 2). The solid remainder of the paper is an elementary relaxation inequality plus a conditional algebraic identity, so the significance is substantially lower than the abstract and theorems suggest.

major comments (3)

[Section 2, Theorem 1] The claimed characterization of equality is false, and the proof is not valid. The assertion that wOCP belongs to ∂D*[−βc,q⊗p] does not follow from the definition of the OCP in (3): the OCP imposes only π_X w=q and I(X;Y)≤λ, and it is not the unconstrained tilted-problem whose solution is the normalized q⊗p e^{−βc}; note that the normalization κ(β,x) in (4) is x-dependent. Moreover, ∂D* is a singleton when it exists, so the convex-combination argument is irrelevant. A concrete counterexample to the 'only if' direction is c≡0 on a finite product space: for any q≠p with I(q⊗p)≤λ one has Rc[q](λ)=Kc[p,q](λ)=0, but wOCP=q⊗q and wOTP=q⊗p are distinct optimal solutions. The valid and elementary statement is the inequality Rc[q](λ)≤Kc[p,q](λ), obtained by constraint relaxation.
[Section 3, Theorem 2] The equality in Theorem 2 is proved correctly by substitution, but the hypothesis is not characterized or even discussed. No construction or existence condition for the reference measure r is given, and for typical costs the hypothesis excludes most pairs (p,q). For a metric cost with c(x,x)=0, feasibility f(x)−g(y)≤c(x,y) together with f=ln(dp/dr) and g=ln(dq/dr) gives ln(dp/dq)(x)≤0 pointwise at y=x, i.e., p≤q. Thus the advertised exact link is not generic; without a characterization of when the hypothesis is satisfiable, Theorem 2 remains a conditional algebraic identity rather than a relation between the two divergences. The paper should provide such a characterization with concrete examples, or explicitly present Theorem 2 as an identity holding only under a strong extra condition.
[Section 2, proof of Theorem 1] Even apart from the subdifferential membership, the proof conflates equality of optimal values with equality of optimizers. The fact that an optimal channel achieves the same cost as the optimal transport plan does not imply that the optimal channel is feasible for the transport problem or that the transport plan is optimal for the channel problem; equality of infima can occur with different minimizers. This is the root of the c≡0 counterexample above. The theorem needs additional uniqueness or feasibility assumptions before any 'if and only if' statement can be made.

minor comments (5)

[Section 2, equation (5)] The identity D[w,q⊗q]=D[w,q⊗p]+D[p,q] is valid only when p is the output marginal π_Y w, or more generally when ∫ ln(dp/dq) dw = ∫ ln(dp/dq) dp. Please state this assumption explicitly, because as written the identity appears to hold for arbitrary w.
[Section 3, Theorem 2] The notation κ[f] is used in Theorem 2 although κ was previously defined for βf as κ[βf]. Since the theorem assumes β=1, either define κ directly or note that when f=ln(dp/dr), one has κ[f]=0; the displayed formula is then unnecessarily opaque.
[Section 3, display before Theorem 2] The notation ∇D[p,r] is informal: the KL divergence is a functional of measures, so the derivative is a functional derivative. Please clarify that ∇D[p,r] means the log-density ln(dp/dr).
[Section 2, equation (4)] The symbol p is used both as a fixed measure in the product q⊗p and as the output marginal of w in the same displayed formula. This is confusing; use distinct symbols for the fixed output measure and the induced output marginal.
[References] The proof of Theorem 1 relies on a standard Lagrange-multiplier result 'e.g. see [12, 2]', but no specific theorem is quoted. Since the subdifferential claim is central, please state the relevant result and verify that its hypotheses are satisfied by the OCP in (3).

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the KL-KW identities are conditional and proved by substitution; the flagged weaknesses are an uncharacterized hypothesis in Theorem 2 and an unjustified subdifferential claim in Theorem 1's proof, which are rigor and scope issues, not circularity.

full rationale

The paper's derivation chain has three load-bearing steps: (i) the observation that the optimal channel problem (OCP), having only the input-marginal constraint, lower-bounds the Kantorovich problem (OTP), which has an additional output-marginal constraint; (ii) Theorem 1, asserting equality of the two optima iff their solutions coincide; and (iii) Theorem 2, decomposing the KL divergence via the law of cosines (8) and re-expressing the resulting expectations as the dual transport value when the dual potentials are log-density ratios against a common reference measure r. Step (i) is true by definition of the two infima (constraint relaxation) and is not circular. Step (ii) is a nested-feasible-set uniqueness statement; its proof relies on a subdifferential characterization of the OCP solution (wOCP in the subdifferential of D* at -beta c with respect to q tensor p) attributed to references [12, 2]. That characterization is not justified by the paper's own formula (4), whose normalizing function kappa(beta,x) is explicitly 'in general non-constant', so the conditionally-normalized OCP solution is not the globally-tilted product forming the singleton subdifferential of D*; this is a rigor gap, not a circular reduction. Step (iii) is proved by direct substitution into equation (8), with the transport content supplied by Kantorovich duality Jc = Kc; given the representability assumption, the equality is essentially the definition of the KL divergence rewritten with potentials, but the conclusion is not built into the assumption, because it contains Kc, which enters through the external duality theorem. The law of cosines (7) is cited to the author's prior work [1], but the paper states it can be proved directly by substitution, and identity (5) is immediate algebra, so these self-citations are not load-bearing. The paper itself limits the scope: 'Generally, the relations presented have a form of inequalities. Additional conditions have been derived in Theorems 1 and 2 for the cases when the relations hold with equalities.' The notable weaknesses are therefore correctness and scope matters, not circularity: Theorem 2's representability condition is uncharacterized and, for metric costs with c(x,x)=0, feasibility forces ln(dp/dq) <= 0 pointwise, so the advertised exact equality generically fails; and Theorem 1's proof asserts an unjustified subdifferential inclusion. No parameters are fitted to data, and no conclusion is identical to an input by construction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper is purely analytic with no free parameters. It relies on standard convex duality and algebraic identities drawn from the author's earlier work, plus a restrictive representability condition for the dual potentials. No new entities are introduced.

assumptions (4)

domain assumption Strong duality Jc[p,q] = Kc[p,q] for the Kantorovich problem
Used without proof in the proof of Theorem 2, where the paper writes 'equals to Jc[p,q] = Kc[p,q]'. True under standard compactness and lower-semicontinuity conditions, but those conditions are not stated.
standard math KL divergence D[w, q tensor p] is strictly convex in w
Theorem 1's proof relies on strict convexity to conclude that the subdifferential of the conjugate functional is a singleton. This is standard, but the argument is applied incorrectly because the potential is not -beta c alone.
domain assumption The optimal channel has the Boltzmann-Gibbs form wOCP = q tensor p exp(-beta c - kappa(beta,x))
Equation (4) is asserted from Lagrange multiplier theory. Conditions for existence and uniqueness of beta and for consistency of the measure p are not given.
domain assumption Reference measure r is finite and probability measures are absolutely continuous with respect to r
The KL gradients and exponential representations in Section 3 require such an r; the paper assumes the divergences are finite.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence." pith.science (2026). https://pith.science/paper/D6DDAESQ

@misc{pith2026190809211,
  author       = {Pith},
  title        = {Pith review of: Relation between the Kantorovich-Wasserstein metric and the Kullback-Leibler divergence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D6DDAESQ}},
  note         = {Machine review of arXiv:1908.09211}
}

read the original abstract

We discuss a relation between the Kantorovich-Wasserstein (KW) metric and the Kullback-Leibler (KL) divergence. The former is defined using the optimal transport problem (OTP) in the Kantorovich formulation. The latter is used to define entropy and mutual information, which appear in variational problems to find optimal channel (OCP) from the rate distortion and the value of information theories. We show that OTP is equivalent to OCP with one additional constraint fixing the output measure, and therefore OCP with constraints on the KL-divergence gives a lower bound on the KW-metric. The dual formulation of OTP allows us to explore the relation between the KL-divergence and the KW-metric using decomposition of the former based on the law of cosines. This way we show the link between two divergences using the variational and geometric principles.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 13 canonical work pages

[1]

Belavkin, R.V.: Law of cosines and Shannon-Pythagorean t heorem for quantum information. In: F. Nielsen, F. Barbaresco (eds.) Geometri c Science of Infor- mation, Lecture Notes in Computer Science , vol. 8085, pp. 369–376. Springer, Heidelberg (2013)

work page 2013
[2]

Journal of Global Optimization 55, 387–416 (2013)

Belavkin, R.V.: Optimal measures and Markov transition k ernels. Journal of Global Optimization 55, 387–416 (2013)

work page 2013
[3]

Belavkin, R.V.: Asymmetric topologies on statistical ma nifolds. In: F. Nielsen, F. Barbaresco (eds.) Geometric Science of Information, Lecture Notes in Com- puter Science , vol. 9389, pp. 203–210. Springer International Publishin g (2015)

work page 2015
[4]

Belavkin, R.V.: On variational deﬁnition of quantum entr opy. In: A. Mohammad- Djafari, F. Barbaresco (eds.) Bayesian Inference and Maxim um Entropy Methods in Science and Engineering (MAXENT 2014), AIP Conference Proceedings, vol. 1641, p. 197. Clos Luc´ e, Amboise, France (2015)

work page 2015
[5]

I, II, chap

Bogachev, V.I.: Measure theory, vol. I, II, chap. xviii, x iv, pp. 500, 575. Springer- Verlag, Berlin (2007)

work page 2007
[6]

Theory Probab

Dobrushin, R.L.: Prescribing a system of random variable s by conditional distri- butions. Theory Probab. Appl. 15(3), 458–486 (1970)

work page 1970
[7]

USSR AS Do klady 37(7–8), 227–229 (1942)

Kantorovich, L.V.: On translocation of masses. USSR AS Do klady 37(7–8), 227–229 (1942). (in Russian). English translation: J. Math . Sci., 133, 4 (2006), 13811382

work page 1942
[8]

The Annals of Math- ematical Statistics 22(1), 79–86 (1951)

Kullback, S., Leibler, R.A.: On information and suﬃcienc y. The Annals of Math- ematical Statistics 22(1), 79–86 (1951)

work page 1951

Show all 13 references

[9]

Histoire de l’Acad´ emie Royale des Sciences avec les M´ emoires de Math´ ematique & de Physique, Paris (1781)

Monge, G.: M´ emoire sur la th´ eorie des d´ eblais et de remb lais. Histoire de l’Acad´ emie Royale des Sciences avec les M´ emoires de Math´ ematique & de Physique, Paris (1781)
[10]

Bell System Technical Journal 27, 379–423 and 623–656 (1948)

Shannon, C.E.: A mathematical theory of communication. Bell System Technical Journal 27, 379–423 and 623–656 (1948)

1948
[11]

Izvestiya of USSR Academy of Sciences, Technical Cybernetics 5, 3–12 (1965)

Stratonovich, R.L.: On value of information. Izvestiya of USSR Academy of Sciences, Technical Cybernetics 5, 3–12 (1965). In Russian

1965
[12]

Sovetskoe R adio, Moscow, USSR (1975)

Stratonovich, R.L.: Theory of Information. Sovetskoe R adio, Moscow, USSR (1975). In Russian 10 Roman Belavkin

1975
[13]

Problems Inform

Vasershtein, L.N.: Markov processes over denumerable p roducts of spaces de- scribing large system of automata. Problems Inform. Transm ission 5(3), 47–52 (1969)

1969

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Belavkin, R.V.: Law of cosines and Shannon-Pythagorean t heorem for quantum information. In: F. Nielsen, F. Barbaresco (eds.) Geometri c Science of Infor- mation, Lecture Notes in Computer Science , vol. 8085, pp. 369–376. Springer, Heidelberg (2013)

work page 2013

[2] [2]

Journal of Global Optimization 55, 387–416 (2013)

Belavkin, R.V.: Optimal measures and Markov transition k ernels. Journal of Global Optimization 55, 387–416 (2013)

work page 2013

[3] [3]

Belavkin, R.V.: Asymmetric topologies on statistical ma nifolds. In: F. Nielsen, F. Barbaresco (eds.) Geometric Science of Information, Lecture Notes in Com- puter Science , vol. 9389, pp. 203–210. Springer International Publishin g (2015)

work page 2015

[4] [4]

Belavkin, R.V.: On variational deﬁnition of quantum entr opy. In: A. Mohammad- Djafari, F. Barbaresco (eds.) Bayesian Inference and Maxim um Entropy Methods in Science and Engineering (MAXENT 2014), AIP Conference Proceedings, vol. 1641, p. 197. Clos Luc´ e, Amboise, France (2015)

work page 2015

[5] [5]

I, II, chap

Bogachev, V.I.: Measure theory, vol. I, II, chap. xviii, x iv, pp. 500, 575. Springer- Verlag, Berlin (2007)

work page 2007

[6] [6]

Theory Probab

Dobrushin, R.L.: Prescribing a system of random variable s by conditional distri- butions. Theory Probab. Appl. 15(3), 458–486 (1970)

work page 1970

[7] [7]

USSR AS Do klady 37(7–8), 227–229 (1942)

Kantorovich, L.V.: On translocation of masses. USSR AS Do klady 37(7–8), 227–229 (1942). (in Russian). English translation: J. Math . Sci., 133, 4 (2006), 13811382

work page 1942

[8] [8]

The Annals of Math- ematical Statistics 22(1), 79–86 (1951)

Kullback, S., Leibler, R.A.: On information and suﬃcienc y. The Annals of Math- ematical Statistics 22(1), 79–86 (1951)

work page 1951