REVIEW 5 major objections 5 minor 33 references
Leveraging Optimal Transport for Distributed Two-Sample Testing: An Integrated Transportation Distance-based Framework
T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read The paper proposes a distributed two-sample test based on the Integrated Transportation Distance, proves concentration and consistency of the resulting permutation test, and validates it with simulations and a PM2.5 case study.
desk verdict Distributed protocol is sensible and simulations are suggestive, but the main theorems are unproved and Theorem 6 is false without a rate condition. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Integrated Transportation Distance of degree $p$ between two kernels $P$ and $Q$ with fixed marginal $\lambda$: $W_p^\lambda(P,Q)=\left(\int_X W_p(P(\cdot\mid x),Q(\cdot\mid x))^p\,\lambda(dx)\right)^{1/p}$. For $K$ clients with weights $w_k$, this reduces to $\left(\sum_{k=1}^K w_k W_p(P(\cdot\mid x_k),Q(\cdot\mid x_k))^p\right)^{1/p}$. The empirical version used for testing is the squared distance at $p=2$: $\widehat{\mathrm{dITD}}_2=\sum_{k=1}^K w_k W_2(P_m^k,Q_n^k)^2$. This object carries the argument because it is a metric on kernel spaces, bounds above the distance between the corresponding mixture distributions, and can be computed from per-client Wasserstein distances alone, so no raw data leaves a device. The permutation test uses the same aggregator on locally permuted samples to build the null distribution and critical value.
What would settle it
Simulate $K$ clients with equal weights and local sample size $m$, shift only one client's distribution by a fixed amount, and let $K$ and $m$ grow together (for example, $m=K$). If the empirical power of the ITD permutation test does not approach 1, the bounded-below signal premise behind Theorem 6 is violated.
Extended reading notes
Core claim
The paper's central claim is Theorem 6: under the alternative hypothesis $H_1$, with equal client weights, densities, and finite fourth moments, $\lim_{m,n\to\infty} P(\widehat{\mathrm{dITD}}_2 \ge c_{m,n,K}^{1-\alpha} \mid H_1)=1$, whether the number of clients $K$ stays fixed or grows. In words, the ITD permutation test is asymptotically consistent for the distributed two-sample problem $H_0: P^k=Q^k$ for all $k$ versus $H_1: P^k\neq Q^k$ for at least one $k$. The same exchangeability that gives exact finite-sample Type I error under $H_0$ is what lets the permutation distribution provide the critical value $c_{m,n,K}^{1-\alpha}$; the consistency result then rests on the concentration inequality of Theorem 5 plus the assumption that, under $H_1$, the empirical ITD stays bounded below by a positive constant. The paper further develops the asymptotic theory of empirical ITD (convergence and a central limit theorem) and demonstrates the method's power on simulated normal, log-normal, and t-distributed data as well as on a PM2.5 air-quality comparison and an MNIST-based concept-drift experiment.
Load-bearing premise
Under the alternative, the weighted integrated distance must stay bounded below by a positive constant as the number of clients grows; with equal weights, this premise fails unless a drifting client has enough local data to offset its small share of the weight.
Editorial extensions
If this is right
- For any fixed number of clients, the ITD permutation test rejects a false global null with probability approaching 1 as local sample sizes grow, while the exchangeable null keeps worst-case Type I error at the nominal level.
- Because the statistic is a weighted average of per-client squared Wasserstein distances, heterogeneity is handled by choosing weights (for example, proportional to local sample sizes), so larger local datasets count more.
- The communication cost is one scalar per client per permutation, so the procedure scales to settings where raw data cannot be shared.
- The concentration bound gives exponential decay of large deviations in the combined sample sizes, so aggregation across clients tightens the estimate of the global discrepancy.
- If a drift exists anywhere, the aggregated test can detect it even when no single client's local test has sufficient power, as the MNIST experiment illustrates.
Reading between the lines
- Editorial inference: the consistency statement for $K\to\infty$ is only as strong as the unstated premise that the signal stays bounded below by a positive constant; with equal weights, a shift confined to a single client contributes only a $1/K$ share of the weight, so local sample sizes must grow with $K$ for Theorem 6 to apply in that regime.
- Editorial inference: the MNIST and PM2.5 experiments suggest the procedure can be read as a concept-drift monitor for a network of data streams, detecting shifts that are invisible at any single node; the paper does not formalize this as a drift-detection guarantee.
- Editorial inference: one testable extension is to replace the squared Wasserstein term with an entropic or Sinkhorn divergence to cut per-client computation in high dimensions; the permutation aggregation would survive, but the metric and asymptotic theorems would need to be re-established for the regularized distance.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a distributed two-sample test based on the Integrated Transportation Distance (ITD). For K clients with independent samples X^k_i ~ P^k and Y^k_j ~ Q^k, the test statistic is dITD_2 = sum_k w_k W_2^2(P^k_m, Q^k_n), and the null hypothesis is P^k = Q^k for all k. A permutation procedure is described in Section 4.3 in which local devices permute their own pooled samples and return permuted local Wasserstein distances to a central device, which aggregates them and constructs a critical value. The paper claims finite-sample Type I error control, a concentration inequality (Theorem 5), asymptotic consistency under the alternative (Theorem 6), and supports these claims with simulations on normal, log-normal, and t distributions, an MNIST concept-drift experiment, and a PM2.5 case study.
Significance. The paper has several strengths. The permutation protocol is natural and privacy-preserving, since only scalar Wasserstein distances are shared; the finite-sample Type I error control via exchangeability under the null is correct and standard; and the simulation design covers several distributions, dimensions, and client counts, with reported Type I error rates generally close to the nominal level. The idea of aggregating local transportation distances is a plausible way to detect distributed shifts. However, the central theoretical power claim is not established and, as stated, is false without a rate condition linking K and m,n; moreover, several theoretical results (Theorems 3–5) are stated without proofs or contain internal flaws. Thus the manuscript does not currently provide a rigorous foundation for its main claim.
major comments (5)
- [Section 4.4, Theorem 6] Theorem 6 is false as stated. Under H1, suppose only client 1 is drifted, with W_2^2(P^1,Q^1) = delta > 0, and all other clients are null, with w_k = 1/K. The population mean of dITD_2 is delta/K, so the sentence 'we can always guarantee that dITD_2 >= delta' is not true for a constant delta independent of K. Using the variance bound cited from [17, Thm 3.3], each local W_2^2 term has variance O(1/m), so both the observed statistic and the permutation threshold fluctuate at scale 1/sqrt(Km), while their mean separation is of order delta/K. The signal-to-noise ratio is therefore of order delta sqrt(m/K). If K/m tends to infinity this ratio tends to zero, and if K/m is bounded away from zero it does not diverge, so the limiting power cannot be 1. Theorem 6 imposes no rate condition such as K = o(m) and K = o(n), and hence the consistency claim is invalid without substantial amendment.
- [Section 4.2, Theorem 5] The concentration inequality in Theorem 5 is dimensionally inconsistent. The statistic dITD_2 and the deviation t have units of squared distance, while (Dx + Dy)^2 has units of distance squared, so the exponent Kmn t^2 / [2(m+n)(Dx+Dy)^2] has units of distance squared rather than being dimensionless. In addition, Dx and Dy are defined as maxima over random samples and are therefore random, whereas a McDiarmid bound requires fixed bounded-range constants; the assumption of finite second moments does not ensure deterministic finite bounds. As written, the inequality cannot be used as a rigorous large-deviation result.
- [Section 4.4, derivation of Theorem 6] Even if Theorem 5 were corrected, the proof of Theorem 6 is only asserted to follow from Theorem 5, and no argument connects a concentration bound around E[dITD_2] to the data-dependent permutation quantile c_{m,n,K}^{1-alpha}. A high-probability lower bound on dITD_2 - E[dITD_2] does not imply that dITD_2 exceeds the permutation critical value, because the gap between E[dITD_2] and the permutation quantile under H1 is not analyzed. This is a load-bearing gap in the power analysis.
- [Section 3, Theorems 3 and 4] Theorems 3 and 4 are stated without proofs and without pointers to a proof section; the appendix only reviews standard optimal transport results. Moreover, Theorem 4 appears internally inconsistent: the empirical quantity is an average of the p-th powers W_p(P(·|x_k), Q(·|x_k))^p, but the centering term is written as W_{p,lambda}^p(P,Q), which under Definition 3 is not the same object as the integral of the p-th powers. The statement therefore needs clarification or correction before it can support the claimed CLT.
- [Section 4.4, power claim] The text states that under H1, 'we can always guarantee that dITD_2 >= delta, where delta is a constant', and that this holds 'regardless of the number of clients'. As shown in the first comment, this is false for a single drifted client with equal weights: the population signal is delta/K, which tends to zero as K grows. The paper needs an explicit condition on the signal (e.g., a minimum nonzero fraction of drifted clients or a lower bound on the weighted signal that is uniform in K) before any consistency statement can hold.
minor comments (5)
- [Section 4.2] The notation for weights is inconsistent: Section 4.2 defines omega_k = m_k/(2M) + n_k/(2N), but the subsequent sections and Theorem 5 use w_k = 1/K with no discussion of how the two notations are reconciled.
- [Section 5.2, Table 5] The rows of Table 5 labeled by 'epsilon' contain columns 0 through 9 but the meaning of these columns is not clearly explained, and the 'ITD' column is not accompanied by a description of how the aggregate power is computed or how it compares with the per-client powers.
- [Section 4.3] The time-complexity paragraph uses the notation o(m,n) where a cost function such as O(f(m,n)) is intended, and the word 'permutated' appears repeatedly instead of 'permuted'.
- [Section 5.3] Figures 1–3 are only referenced in passing and have minimal captions; the claim that the approximate ITD distribution 'converges towards a normal distribution' is not supported by any formal result or by quantitative evidence beyond illustrative histograms.
- [Section 5.1] The simulation tables report point estimates from 200 repetitions without standard errors or confidence intervals; given the small number of repetitions, this makes it difficult to assess whether deviations from the nominal 0.05 level are meaningful.
Circularity Check
Central testing claim is not forced by construction or by a self-citation chain; self-citation is minor and non-load-bearing. Theorem 6's consistency proof is asserted rather than derived, and its Section 4.4 premise fails for a single drifted client as K grows, but these are soundness gaps, not circular reductions.
full rationale
The paper's statistic, dITD_2 = sum_k w_k W_2^2(P^k_m, Q^k_n) (Eq. 2), is transparently a weighted average of local squared Wasserstein distances; with equal local sample sizes the weight is 1/K, exactly Definition 3 applied to a finite-support marginal. No parameter is fitted and then renamed a prediction: weights are analytic (1/K), the threshold c_{m,n,K}^{1-alpha} is the standard permutation quantile, and finite-sample Type I error control is cited to external permutation-test theory ([21], [22]), not to the authors' own papers. The self-citations that do exist — Definitions 2 and 3 and Theorem 2 imported from [2], and the ITD framework [3, 18-20] — supply the metric and continuity properties of the distance used as a building block. These are prior, peer-reviewed results whose stated assumptions do not include the present testing claim, and the permutation test's validity does not logically depend on ITD being a metric; the citation is therefore real support rather than a load-bearing self-reference, and nothing in it forces Theorem 6's conclusion. Flagged and weighed into the verdict, as the reviewing rule requires, are two unsupported steps in the power analysis, both located in Section 4.4. First, Theorem 6's proof is merely asserted ('its result is derived from the conclusions of Theorem 5'), yet Theorem 5 only bounds |dITD_2 - E[dITD_2]| around its own mean and says nothing about the data-dependent permutation quantile c_{m,n,K}^{1-alpha}; the separation argument that would establish power tending to 1 is never written, so the derivation chain has an omitted link. Second, the premise that 'under the alternative hypothesis H1, we can always guarantee that dITD_2 >= delta, where delta is a constant' fails for one drifted client under equal weights, where the population signal is delta/K -> 0 as K grows; Theorem 6's 'K approaches infinity' case therefore requires an unstated rate condition (K = o(m) and K = o(n)) that the theorem never states. Theorem 5's exponent, exp(-Kmnt^2/[2(m+n)(Dx+Dy)^2]), is likewise dimensionally inconsistent, since t carries units of squared distance while the denominator involves squared distance. These are internal scaling and proof gaps, not circularity: no equation in the paper equals another by construction, no fitted value is relabeled as a prediction, and no uniqueness theorem is imported from the authors' own work to forbid alternatives.
Assumptions & free parameters
free parameters (1)
- client weights w_k =
w_k = 1/K or m_k/(2M) + n_k/(2N)
assumptions (5)
- domain assumption ITD is a metric on equivalence classes of kernels (Theorem 2)
- domain assumption Weak convergence continuity of ITD holds (Theorem 3)
- domain assumption Clients x_k are i.i.d. draws from a fixed population measure lambda
- ad hoc to paper Under H1, the weighted integrated distance is bounded below by a positive constant uniformly in K
- domain assumption Finite support bounds Dx and Dy exist
Cite this review
Pith. "Pith review of Leveraging Optimal Transport for Distributed Two-Sample Testing: An Integrated Transportation Distance-based Framework." pith.science (2026). https://pith.science/paper/C3ZIYDD3
@misc{pith2026250616047,
author = {Pith},
title = {Pith review of: Leveraging Optimal Transport for Distributed Two-Sample Testing: An Integrated Transportation Distance-based Framework},
year = {2026},
howpublished = {\url{https://pith.science/paper/C3ZIYDD3}},
note = {Machine review of arXiv:2506.16047}
}
read the original abstract
This paper introduces a novel framework for distributed two-sample testing using the Integrated Transportation Distance (ITD), an extension of the Optimal Transport distance. The approach addresses the challenges of detecting distributional changes in decentralized learning or federated learning environments, where data privacy and heterogeneity are significant concerns. We provide theoretical foundations for the ITD, including convergence properties and asymptotic behavior. A permutation test procedure is proposed for practical implementation in distributed settings, allowing for efficient computation while preserving data privacy. The framework's performance is demonstrated through theoretical power analysis and extensive simulations, showing robust Type I error control and high power across various distributions and dimensions. The results indicate that ITD effectively aggregates information across distributed clients, detecting subtle distributional shifts that might be missed when examining individual clients. This work contributes to the growing field of distributed statistical inference, offering a powerful tool for two-sample testing in modern, decentralized data environments.
Figures
Reference graph
Works this paper leans on
-
[1]
On wasserstein two sample testing and related families of nonparametric tests
Aaditya Ramdas, Nicolas Garcia, and Marco Cuturi. On wasserstein two sample testing and related families of nonparametric tests. 2015
work page 2015
-
[2]
Zhengqi Lin and Andrzej Ruszczy´nski. An integrated transportation distance between kernels and approximate dynamic risk evaluation in Markov systems. SIAM Journal on Control and Optimization, 61(6):3559–3583, 2023
work page 2023
-
[3]
Zhengqi Lin and Andrzej Ruszczynski. Fast dual subgradient optimization of the integrated transportation distance between stochastic kernels, 2023
work page 2023
-
[4]
Jr. Massey, Frank J. The kolmogorov-smirnov test for goodness of fit, March 1951
work page 1951
-
[5]
Henry B. Mann and Donald R. Whitney. On a test of whether one of two random variables is stochastically larger than the other[j]. the annals of mathematical statistics. The annals of mathematical statistics, 22:50–60, 1947
work page 1947
- [6]
-
[7]
Testing for equal distributions in high dimension
Gábor J Székely, Maria L Rizzo, et al. Testing for equal distributions in high dimension. InterStat, 5(16.10):1249– 1272, 2004
work page 2004
-
[8]
Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723–773, 2012
work page 2012
Show all 33 references
-
[9]
Ball divergence: Nonparametric two sample test
Wenliang Pan, Yuan Tian, Xueqin Wang, and Heping Zhang. Ball divergence: Nonparametric two sample test. Annals of statistics, 46(3):1109, 2018
2018
-
[10]
Robust multivariate nonparametric tests via projection averaging
Ilmun Kim, Sivaraman Balakrishnan, and Larry Wasserman. Robust multivariate nonparametric tests via projection averaging. The Annals of Statistics, 48(6):3417–3441, 2020
2020
-
[11]
Statistical inferences for complex dependence of multimodal imaging data
Jinyuan Chang, Jing He, Jian Kang, and Mingcong Wu. Statistical inferences for complex dependence of multimodal imaging data. Journal of the American Statistical Association, 119(546):1486–1499, 2024
2024
-
[12]
Tony Cai, Abhinav Chakraborty, and Lasse Vuursteen
T. Tony Cai, Abhinav Chakraborty, and Lasse Vuursteen. Federated nonparametric hypothesis testing with differential privacy constraints: Optimal rates and adaptive tests. ArXiv, abs/2406.06749, 2024
2024 arXiv
-
[13]
Fully distributed sequential hypothesis testing: Algorithms and asymptotic analyses
Shang Li and Xiaodong Wang. Fully distributed sequential hypothesis testing: Algorithms and asymptotic analyses. IEEE Transactions on Information Theory, 64(4):2742–2758, 2018
2018
-
[14]
Integrative high dimensional multiple testing with heterogeneity under data sharing constraints
Molei Liu, Yin Xia, Kelly Cho, and Tianxi Cai. Integrative high dimensional multiple testing with heterogeneity under data sharing constraints. Journal of machine learning research : JMLR, 22, 2020
2020
-
[15]
Optimal Transport: Old and New
Cédric Villani. Optimal Transport: Old and New. Springer, 2009
2009
-
[16]
Rachev and Lutger Rüschendorf.Mass Transportation Problems: Volume I: Theory
Svetlozar T. Rachev and Lutger Rüschendorf.Mass Transportation Problems: Volume I: Theory. Springer Science & Business Media, 1998
1998
-
[17]
Central limit theorem for empirical transportation cost in general dimension
Eustasio del Barrio and Jean-Michel Loubes. Central limit theorem for empirical transportation cost in general dimension. Annals of Probability, 2, 2019. 14
2019
-
[18]
Fast dual subgradient optimization of stochastic kernels with application to forward-backward stochastic differential equations
Zhengqi Lin and Andrzej Ruszczynski. Fast dual subgradient optimization of stochastic kernels with application to forward-backward stochastic differential equations. In IISE Annual Conference. Proceedings, 2024
2024
-
[19]
Stochastic kernel approximation by transportation distance method
Zhengqi Lin and Andrzej Ruszczynski. Stochastic kernel approximation by transportation distance method. In IISE Annual Conference. Proceedings, 2024
2024
-
[20]
The Integrated Transportation Distance Between Markov Kernels: Theory, Optimization, and Applications in Risk Evaluation and Machine Learning
Zhengqi Lin. The Integrated Transportation Distance Between Markov Kernels: Theory, Optimization, and Applications in Risk Evaluation and Machine Learning. PhD thesis, 2024
2024
-
[21]
Testing statistical hypotheses, volume 3
Erich Leo Lehmann, Joseph P Romano, and George Casella. Testing statistical hypotheses, volume 3. Springer, 2005
2005
-
[22]
Exact testing with random permutations
Jesse Hemerik and Jelle Goeman. Exact testing with random permutations. Test, 27(4):811–825, 2018
2018
-
[23]
Modeling and regionalization of china’s pm2
Decai Liang, Haozhe Zhang, Xiaohui Chang, and Hui Huang. Modeling and regionalization of china’s pm2. 5 using spatial-functional mixture models. Journal of the American Statistical Association , 116(533):116–132, 2021
2021
-
[24]
Sample complexity of sinkhorn divergences
Aude Genevay, Lénaic Chizat, Francis Bach, Marco Cuturi, and Gabriel Peyré. Sample complexity of sinkhorn divergences. 2018
2018
-
[25]
Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem
Gonzalo Mena and Jonathan Weed. Statistical bounds for entropic optimal transport: sample complexity and the central limit theorem. 2019
2019
-
[26]
An improved central limit theorem and fast convergence rates for entropic transportation costs
Eustasio del Barrio, Alberto Gonzalez-Sanz, Jean-Michel Loubes, and Jonathan Niles-Weed. An improved central limit theorem and fast convergence rates for entropic transportation costs. 2022
2022
-
[27]
Mdvt: Enhancing multimodal recommendation with model-agnostic multimodal-driven virtual triplets
Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Hewei Wang, Yijie Li, Mengran Li, Puzhen Wu, and Edith CH Ngai. Mdvt: Enhancing multimodal recommendation with model-agnostic multimodal-driven virtual triplets. arXiv preprint arXiv:2505.16665, 2025
2025 arXiv
-
[28]
Mentor: multi-level self- supervised learning for multimodal recommendation
Jinfeng Xu, Zheyu Chen, Shuo Yang, Jinze Li, Hewei Wang, and Edith CH Ngai. Mentor: multi-level self- supervised learning for multimodal recommendation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 12908–12917, 2025
2025
-
[29]
AlignGroup: Learning and Aligning Group Consensus with Member Preferences for Group Recommendation
Jinfeng Xu, Zheyu Chen, Jinze Li, Shuo Yang, Hewei Wang, and Edith CH Ngai. AlignGroup: Learning and Aligning Group Consensus with Member Preferences for Group Recommendation. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, page...
2024
-
[30]
Towards understanding camera motions in any video
Zhiqiu Lin, Siyuan Cen, Daniel Jiang, Jay Karhade, Hewei Wang, Chancharik Mitra, Tiffany Ling, Yuhan Huang, Sifan Liu, Mingyu Chen, Rushikesh Zawar, Xue Bai, Yilun Du, Chuang Gan, and Deva Ramanan. Towards understanding camera motions in any video. arXiv preprint arXiv:2504.15...
2025 arXiv
-
[31]
Multi-Cali Anything: Dense Feature Multi-Frame Structure-from-Motion for Large-Scale Camera Array Calibration
Jinjiang You, Hewei Wang, Yijie Li, Mingxiao Huo, Long Van Tran Ha, Mingyuan Ma, Jinfeng Xu, Puzhen Wu, Shubham Garg, and Wei Pu. Multi-Cali Anything: Dense Feature Multi-Frame Structure-from-Motion for Large-Scale Camera Array Calibration. arXiv preprint arXiv:2503.00737, 2025
2025 arXiv
-
[32]
Fedcust: Offloading hyperparameter customization for federated learning
Syed Zawad, Xiaolong Ma, Jun Yi, Cheng Li, Minjia Zhang, Lei Yang, Feng Yan, and Yuxiong He. Fedcust: Offloading hyperparameter customization for federated learning. Performance Evaluation, 167:102450, March 2025
2025
-
[33]
Enabling scalable and adaptive machine learning training via serverless computing on public cloud
Ahsan Ali, Xiaolong Ma, Syed Zawad, Paarijaat Aditya, Istemi Ekin Akkus, Ruichuan Chen, Lei Yang, and Feng Yan. Enabling scalable and adaptive machine learning training via serverless computing on public cloud. Performance Evaluation, 167:102451, March 2025. 15
2025
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.