Least-Squares Problem Over Probability Measure Space

Li Wang; Qin Li; Yunan Yang

REVIEW 3 major objections 4 minor 1 cited by

Least-Squares Problem Over Probability Measure Space

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read For a least-squares problem over probability measures, the choice of divergence decides what the optimal pushforward reproduces: φ-divergences recover the conditional distribution of the data on the range of the map, while Wasserstein…

desk verdict A clean conceptual note on how divergence choice selects between conditional and projected recovery; the main theorem needs a uniqueness fix for non-strictly convex φ. read the letter →

arxiv 2501.09097 v1 pith:WBC4NGSW submitted 2025-01-15 math.OC math.FAmath.PR

Qin Li , Li Wang , Yunan Yang This is my paper · ORCID

classification math.OCmath.FAmath.PR MSC 49Q2262B1028A50

keywords probabilitymeasurespacevariationalproblemphi-divergenceWassersteindistanceconditionaldistributionmarginalpushforwardinverse

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper studies the variational problem of finding a probability measure ρ_x whose pushforward under a forward map G best matches a given data distribution ρ_y, measured by a divergence D. It claims that the choice of D is not cosmetic: if D is any φ-divergence, the optimal pushforward equals the conditional distribution of ρ_y restricted to G's range; if D is any Wasserstein distance W_p, it equals the pushforward of ρ_y under projection onto that range. This mirrors the classical fact that in Euclidean least squares the choice of norm changes the minimizer, but here the distinction is between conditional and marginal aspects of the target distribution. The consequences matter for inverse problems where one wants a distributional fit rather than a point estimate.

What carries the argument

For the φ-divergence case, the engine is the disintegration of ρ_y with respect to the two-valued map T(y)=1 if y∈R, 0 otherwise. Writing ρ_y = ρ_y(R) ρ^c_{y|R} + ρ_y(R^n\R) ρ^c_{y|R^c}, the divergence splits into a constant off-range term plus an integral over R; Jensen's inequality on the convex φ gives a lower bound that is attained exactly at ρ^c_{y|R}. For the Wasserstein case, the projection P_G(y)=argmin_{z∈R} d(z,y) converts any coupling cost into a lower bound d(P_G(y),y)^p, yielding W_p(G#ρ_x,ρ_y)^p ≥ W_p(P_G#ρ_y,ρ_y)^p, with equality achieved when G#ρ_x = P_G#ρ_y. Both proofs share a feasibility step: any measure supported on R must be representable as G#ρ_x, which is asserted through a left-inverse of G.

What would settle it

Construct a Borel map G whose range R admits no measurable right-inverse, such as a map whose image is a non-Borel analytic set, and take ρ_y with positive mass outside R; then ρ^c_{y|R} is not in {G#ρ_x : ρ_x∈P(Θ)}, so if the true minimizer's pushforward differs from ρ^c_{y|R}, the conclusion of Theorem 3.2 fails. A simpler check for the Wasserstein case: with non-compact R, the projection P_G may not exist, so P_G#ρ_y is undefined and the theorem's statement needs modification.

Watch

Extended reading notes

Core claim

Two theorems give the exact form of the minimizer. Theorem 3.2: assuming a minimizer ρ*_x exists for min D_φ(G#ρ_x || ρ_y), the pushforward satisfies G#ρ*_x = ρ^c_{y|R}, where ρ^c_{y|R}(B)=ρ_y(B∩R)/ρ_y(R) is the conditional distribution of ρ_y on the range R = G(Θ). This holds for every convex φ with φ(1)=0, including KL and χ². Theorem 4.2: assuming a minimizer exists for min W_p(G#ρ_x, ρ_y), the pushforward satisfies G#ρ*_x = P_G#ρ_y, where P_G projects each y onto the nearest point of (compact) R. Thus the divergence class entirely determines which aspect of the data distribution is reproduced: off-range data mass contributes a constant penalty under φ-divergences, while under Wasserstein only the projected locations matter.

Load-bearing premise

The paper needs every probability measure supported on the range R to be expressible as G#ρ_x for some ρ_x, which requires a measurable way to pick one preimage of each y in R; for a general nonlinear or non-injective map this selection may not exist.

Editorial extensions

If this is right

All φ-divergences give the same recovered pushforward, so within that class the specific divergence (KL, χ², total variation) does not change the answer.
Wasserstein recovery ignores the shape of ρ_y off the range except through each point's nearest range point, so the off-range mass acts only through its projected distribution.
When G is linear, the two theorems reduce to the familiar linear-space statement: the conditional distribution is recovered under φ-divergences and the marginal under W_p, matching the earlier linear-operator result in the paper's references.
The results give a user a clear criterion: pick a φ-divergence to match the data on the range, or pick W_p to match the projected data.
Because the minimizer remains a probability measure even with unlimited data, the formulation is a genuine distributional least-squares fitting, not a posterior that collapses to a point.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension: for Sinkhorn divergences, the recovered pushforward may interpolate between the conditional and the projected marginal as the entropic regularization parameter is varied.
If a measurable right-inverse for G on R does not exist, the candidate measures ρ^c_{y|R} and P_G#ρ_y can fall outside the attainable set, so the dichotomy likely needs an explicit reachability hypothesis to hold for arbitrary nonlinear maps.
The same proof strategy (disintegration for convex functionals, projection for metric costs) may yield analogous conditional-versus-marginal dichotomies for other statistical divergences, such as Bregman divergences or the maximum mean discrepancy.
One could use these identities as a diagnostic in inverse problems: comparing the minimizer's pushforward against ρ^c_{y|R} versus P_G#ρ_y tells a practitioner which divergence is actually encoded in their data-fitting objective.

Share X Bluesky LinkedIn Reddit HN

Formalized claims in Lean

Claim #1: Two theorems give the exact form of the minimizer. Theorem 3.2: assuming a minimizer ρ*_x exists for min D_φ(G#ρ_x || ρ_y), the pushforward satisfies G#ρ*_x = ρ^c_{y|R}, where ρ^c_{y|R}(B)=ρ_y(B∩R)/ρ_y(R) is the conditional distribution of ρ_y on the range R = G(Θ). This holds for every convex φ with φ(1)=0, including KL and χ². Theorem 4.2: assuming a minimizer exists for min W_p(G#ρ_x, ρ_y), the

/-- @claim 1 Two theorems give the exact form of the minimizer. Theorem 3.2: assuming a minimizer ρ*_x exists for min D_φ(G#ρ_x || ρ_y), the pushforward satisfies G#ρ*_x = ρ^c_{y|R}, where ρ^c_{y|R}(B)=ρ_y(B∩R)/ρ_y(R) is the conditional distribution of ρ_y on the range R = G(Θ). This holds for every convex φ with φ(1)=0, including KL and χ². Theorem 4.2: assuming a minimizer exists for min W_p(G#ρ_x, ρ_y), the -/
def central_claim : Prop :=

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this is a clean conceptual note. The genuinely new part is the characterization for general nonlinear G: φ-divergence minimizers push forward to the conditional of ρ_y on the range, Wasserstein minimizers push forward to the projection of ρ_y onto the range. The linear case was in their earlier work; the extension here is modest but real, and the measure-disintegration / projection framing is right.

What's good: the proofs are short, the Jensen argument is basically correct for strictly convex φ, and the Wasserstein argument is standard and correct when the projection is single-valued. The paper is honest: it says the linear case appeared before, explicitly assumes existence of minimizers, and even invites reader criticism. It does not overfit or hide derivations.

Soft spots, roughly in order.

First, Theorem 3.2 is overclaimed as written. Equality in Jensen gives that the conditional is a minimizer, not that every minimizer equals the conditional. For non-strictly convex φ, e.g. total variation, the set of minimizers can be an interval. The paper needs either strict convexity or an existential quantifier—'there exists a minimizer with G#ρ*_x = ...' is what the proof actually establishes. The same quantifier issue appears in Theorem 4.2 when P_G is set-valued; only a selection of P_G#ρ_y is guaranteed.

Second, both proofs use an unstated measurable-selection/left-inverse assumption. Equation (4) claims every measure supported on R is representable as G#ρ_x; that requires a measurable right-inverse of G on R, or at least a measurable selection of preimages. For general nonlinear non-injective G this is not automatic. This is fixable by adding a hypothesis, and it does not kill the main idea.

Third, existence of the minimizer is assumed. The paper acknowledges this in footnote 1, so it is not a hidden flaw, but it does mean the theorems are conditional on well-posedness.

The abstract says 'the reconstructed G#ρ*_x recovers...' which inherits the uniqueness problem. Still, the structural insight—divergence choice selects conditional vs. marginal recovery—is sound and useful.

Bottom line: this deserves peer review. A referee should ask for a clean statement of the measurable-selection and strict-convexity/uniqueness assumptions, and a careful set-valued version of the Wasserstein claim. The mathematical core is fine for a note.

Disagree with the stress-test note? I checked the TV example; it lands. The same non-uniqueness is already visible in the Jensen proof, so I count this as a real gap, not a manufactured one.

Referee Report

3 major / 4 minor

Summary. The paper studies the variational problem ρ_x^∗ = argmin_{ρ_x} D(G#ρ_x, ρ_y) over probability measures on the input space Θ, where G is a deterministic forward map to R^n and D is either a φ-divergence or the p-Wasserstein distance. The main claims are Theorem 3.2, asserting that for any φ-divergence with convex φ, every minimizer's pushforward equals the conditional distribution of ρ_y on the range R = G(Θ), and Theorem 4.2, asserting that for any W_p with p ≥ 1, every minimizer's pushforward equals P_G#ρ_y where P_G is the metric projection onto R. The proofs are short: Theorem 3.2 uses measure disintegration and Jensen's inequality; Theorem 4.2 compares transport cost through the projection inequality. The paper also includes a discussion distinguishing the formulation from Bayesian inversion. Existence of a minimizer is explicitly assumed in footnote 1 and in both theorems.

Significance. If the dichotomy were true as stated, it would be a clean and useful conceptual result: the choice of divergence in a distribution-space least-squares problem selects which aspect of the target ρ_y is recovered. The proofs are elegant and self-contained, and the explicit assumption that a minimizer exists is honest. However, the theorems as written are too strong: non-uniqueness of minimizers for non-strictly convex φ and for set-valued metric projections breaks the claimed equality for arbitrary minimizers, and the proof relies on an unstated measurable-selection condition on G. The central idea is defensible under additional hypotheses, but the paper's main statements require revision before they can be accepted.

major comments (3)

[Section 3, proof of Theorem 3.2] The theorem states that any minimizer ρ_x^* satisfies G#ρ_x^* = ρ_{y|R}^c. The proof shows that ρ_{y|R}^c attains the Jensen lower bound, but it does not prove that this minimizer is unique. For non-strictly convex φ, equality in Jensen's inequality is not unique. Concretely, take Θ = R = {a,c}, G = id, and ρ_y = 0.2δ_a + 0.3δ_b + 0.5δ_c. Every feasible pushforward has the form p_λ = λδ_a + (1−λ)δ_c, and for the total-variation divergence φ(t) = |t−1| one computes D_TV(p_λ || ρ_y) = |λ−0.2| + |0.5−λ| + 0.3, which is minimized for every λ ∈ [0.2, 0.5]. The conditional ρ_{y|R}^c corresponds to λ = 2/7, but λ = 0.3 is another minimizer whose pushforward is not the conditional. Thus the equality for an arbitrary minimizer is false. The theorem should be corrected by assuming φ is strictly convex (or by stating that there exists a minimizer with the claimed pushforward, rather than every minimizer).
[Section 4, Definition 4.1 and Theorem 4.2] The projection P_G(y) = argmin_{y∈R} d(y,y) is used as a single-valued map in Theorem 4.2, but the argmin is generally set-valued. When the projection is not unique on a set of positive ρ_y-mass, the conclusion G#ρ_x^* = P_G#ρ_y cannot hold for an arbitrary minimizer. For example, if R is a circle and ρ_y is a Dirac mass at the center, every measure supported on R is a minimizer of W_p, not only P_G#ρ_y for a fixed selection. The theorem needs an explicit hypothesis that the metric projection is single-valued ρ_y-almost everywhere (or a statement that the conclusion holds only for a suitably constructed minimizer). In addition, the compactness assumption on R in Definition 4.1 is not restated in Theorem 4.2, although the proof relies on the projection being well-defined and measurable.
[Equation (4) and proof of Theorem 4.2] The '⊇' direction of equation (4) and the feasibility assertion in Theorem 4.2 require that every probability measure supported on R can be written as G#ρ_x for some ρ_x ∈ P(Θ). This holds only if G admits a measurable right-inverse (a selection h: R → Θ with G(h(y)) = y for y ∈ R), which is what the paper informally calls a 'left-inverse function of G'. For a general nonlinear, non-injective map G, such a measurable selection may not exist. If it fails, the candidate measures ρ_{y|R}^c and P_G#ρ_y may lie outside the feasible set, and the minimizer may differ from the claimed ones. This is a missing hypothesis, not merely a gap in the proof; it should be stated explicitly, for example by requiring the existence of a measurable selection on R (or at least ρ_y-a.e. on R).

minor comments (4)

[Abstract and Introduction] The abstract and the introductory bullet list state the recovery results without the qualifications needed by the theorems: strict convexity of φ, single-valuedness of the projection, and existence of a measurable selection. The abstract's phrase 'for any φ-divergence' is therefore misleading when read together with the proof.
[Definition 3.1] The conditional distribution ρ_{y|R}^c is defined only when ρ_y(R) > 0. The paper does not discuss the case ρ_y(R) = 0, where the theorem statement would involve a division by zero or a degenerate conditional. A short remark on this edge case would be useful.
[Footnote 2] The statement that non-absolutely-continuous ρ'_y are 'irrelevant' because D_φ 'achieves the maximum value' is imprecise: for KL and χ^2 the divergence is infinite, while for TV the value depends on the definition and may not be a maximum in the usual sense. This does not affect the main proof, but the wording should be corrected.
[Terminology] The map called a 'left-inverse function of G' in the proofs is in standard terminology a right-inverse or a measurable selection of G, since it satisfies G(h(y)) = y on R. Using standard terminology would avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the main theorems are self-contained derivations from the definitions of phi-divergence and Wasserstein distance; the cited prior work is contextual, not load-bearing.

full rationale

The derivation chain is self-contained. Theorem 3.2 starts from the definition D_phi(G#rho_x||rho_y) = integral phi(drho'/drho_y) drho_y, disintegrates rho_y as nu(1) rho^c_{y|R} + nu(0) rho^c_{y|R^c}, and applies Jensen's inequality to obtain D_phi(rho'_y||rho_y) >= nu(1) phi(1/nu(1)) + nu(0) phi(0), with equality at rho'_y = rho^c_{y|R}. The lower bound depends only on the data rho_y and the convexity of phi; it is not an input in disguise. Theorem 4.2 similarly uses only the definition of W_p and the metric projection P_G, via the inequality integral d(y~,y)^p dpi >= integral d(P_G(y),y)^p drho_y = W_p(P_G#rho_y,rho_y)^p, and then constructs a feasible rho_x from P_G#rho_y. No parameter is fitted to a subset of the data and then relabeled as a prediction. The citations [LOWY24] and [LWY23] are introduced as prior reports of special cases, with the paper saying 'some of the results in simpler cases were reported in earlier work [LOWY24, LWY23]', but the theorems here do not cite those papers as premises; the only external theorem used is the standard Measure Disintegration Theorem from [AGS08]. There is no self-citation chain that forces the conclusion. Two genuine issues exist but they are not circularity: (i) the proof of the 'supseteq' direction in equation (4) assumes a measurable left-inverse of G on R, an unstated regularity condition that can fail for general nonlinear non-injective maps; (ii) for non-strictly convex phi such as total variation, equality in Jensen's inequality is not unique, so Theorem 3.2's assertion 'G#rho*_x = rho^c_{y|R}' for an arbitrary minimizer is overclaimed unless one quantifies over the particular minimizer obtained at equality or assumes strict convexity. These are correctness or quantifier gaps, not reductions of the conclusion to the hypothesis.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The central claims rest on standard measure-theoretic tools and several domain assumptions about the forward map G and the data measure ρ_y. No free parameters are fitted. No new entities are postulated.

assumptions (6)

standard math Measure Disintegration Theorem
Used in the proof of Theorem 3.2 to decompose ρ_y into conditionals on R and its complement.
standard math Jensen's inequality
Applied to φ, leveraging convexity, to derive the lower bound in Theorem 3.2.
domain assumption Existence of a minimizer for the variational problem
Assumed in the paper's footnote and in the statements of Theorems 3.2 and 4.2; the problem may not always admit a solution.
domain assumption Existence of a measurable right-inverse of G on R
Used implicitly to represent any measure supported on R as a pushforward of G#; not stated as a hypothesis.
domain assumption Compactness of R and single-valued projection for W_p
Stated in Definition 4.1; needed for the existence of P_G and for the equality in Theorem 4.2.
domain assumption ρ_y(R) > 0
Required for the conditional distribution ρ^c_{y|R} to be well-defined; the paper does not state this explicitly.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Least-Squares Problem Over Probability Measure Space." pith.science (2026). https://pith.science/paper/WBC4NGSW

@misc{pith2026250109097,
  author       = {Pith},
  title        = {Pith review of: Least-Squares Problem Over Probability Measure Space},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WBC4NGSW}},
  note         = {Machine review of arXiv:2501.09097}
}

abstract

In this work, we investigate the variational problem $$\rho_x^\ast = \text{argmin}_{\rho_x} D(G\#\rho_x, \rho_y)\,, $$ where $D$ quantifies the difference between two probability measures, and ${G}$ is a forward operator that maps a variable $x$ to $y=G(x)$. This problem can be regarded as an analogue of its counterpart in linear spaces (e.g., Euclidean spaces), $\text{argmin}_x \|G(x) - y\|^2$. Similar to how the choice of norm $\|\cdot\|$ influences the optimizer in $\mathbb R^d$ or other linear spaces, the minimizer in the probabilistic variational problem also depends on the choice of $D$. Our findings reveal that using a $\phi$-divergence for $D$ leads to the recovery of a conditional distribution of $\rho_y$, while employing the Wasserstein distance results in the recovery of a marginal distribution.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Distributional Inverse Homogenization
physics.comp-ph 2026-04 unverdicted novelty 8.0 of 10

Statistics of microstructure are identifiable from bulk-property distributions via generative-model calibration under periodic and stochastic homogenization.

Reference graph

Works this paper leans on

6 extracted references · 5 canonical work pages · cited by 1 Pith paper

[1]

Gradient flows: in metric spaces and in the space of probability measures

Luigi Ambrosio, Nicola Gigli, and Giuseppe Savar \'e . Gradient flows: in metric spaces and in the space of probability measures . Springer Science & Business Media, 2008

work page 2008
[2]

Breidt, T

J. Breidt, T. Butler, and D. Estep. A measure-theoretic computational method for inverse sensitivity problems i: Method and analysis. SIAM Journal on Numerical Analysis , 49(5):1836--1859, 2011

work page 2011
[3]

On the consistency of Bayes estimates

Persi Diaconis and David Freedman. On the consistency of Bayes estimates . The Annals of Statistics , pages 1--26, 1986

work page 1986
[4]

Stochastic Inverse Problem: stability, regularization and Wasserstein gradient flow

Qin Li, Maria Oprea, Li Wang, and Yunan Yang. Stochastic Inverse Problem: stability, regularization and Wasserstein gradient flow . arXiv preprint arXiv:2410.00229 , 2024

arXiv 2024
[5]

Differential equation--constrained optimization with stochasticity

Qin Li, Li Wang, and Yunan Yang. Differential equation--constrained optimization with stochasticity. SIAM/ASA Journal on Uncertainty Quantification , 11(2):491--515, 2023

work page 2023
[6]

Personal communication with Professor Youssef Marzouk , 2024

work page 2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Gradient flows: in metric spaces and in the space of probability measures

Luigi Ambrosio, Nicola Gigli, and Giuseppe Savar \'e . Gradient flows: in metric spaces and in the space of probability measures . Springer Science & Business Media, 2008

work page 2008

[2] [2]

Breidt, T

J. Breidt, T. Butler, and D. Estep. A measure-theoretic computational method for inverse sensitivity problems i: Method and analysis. SIAM Journal on Numerical Analysis , 49(5):1836--1859, 2011

work page 2011

[3] [3]

On the consistency of Bayes estimates

Persi Diaconis and David Freedman. On the consistency of Bayes estimates . The Annals of Statistics , pages 1--26, 1986

work page 1986

[4] [4]

Stochastic Inverse Problem: stability, regularization and Wasserstein gradient flow

Qin Li, Maria Oprea, Li Wang, and Yunan Yang. Stochastic Inverse Problem: stability, regularization and Wasserstein gradient flow . arXiv preprint arXiv:2410.00229 , 2024

arXiv 2024

[5] [5]

Differential equation--constrained optimization with stochasticity

Qin Li, Li Wang, and Yunan Yang. Differential equation--constrained optimization with stochasticity. SIAM/ASA Journal on Uncertainty Quantification , 11(2):491--515, 2023

work page 2023

[6] [6]

Personal communication with Professor Youssef Marzouk , 2024

work page 2024