REVIEW 4 major objections 6 minor 34 references
Optimal Transport-based Domain Alignment as a Preprocessing Step for Federated Learning
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper reports that a one-time optimal-transport preprocessing step, which aligns every client's images to a shared Wasserstein-barycenter target, lifts FedAvg's CIFAR-10 accuracy to 99.62 percent.
desk verdict OT-based preprocessing for FL is a coherent idea, but the CIFAR-10 numbers are implausible and the evaluation protocol is under-specified; the empirical claim needs a rigorous redo. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Wasserstein barycenter, the probability vector that minimizes a weighted sum of Wasserstein distances to a set of input distributions. The paper computes a barycenter of each client's red, green, and blue pixel channels, sends the resulting RGB triplets to a central server, computes a second-level 'global' barycenter per channel, and uses regularized optimal transport with the Sinkhorn algorithm to project every local image's color channels onto that global target. This two-level mechanism is what turns local color statistics into a single shared target space, and the projection is what physically reduces inter-client distributional discrepancy.
What would settle it
Train the described pipeline on CIFAR-10 using only the standard 50,000 training images, with the stated two-level channel-wise Wasserstein barycenters, Sinkhorn projection, the custom CNN, batch size 16, Adam at learning rate 1e-3, and the reported local epochs, then evaluate on the standard 10,000-image test set. If the accuracy is close to 99 percent, the claim is confirmed; if it drops to roughly the plain FedAvg baseline, then the reported numbers were not held-out test accuracy.
Extended reading notes
Core claim
The central claim is that projecting each local dataset onto a globally shared RGB target space, obtained by computing two levels of Wasserstein barycenters, minimizes the distributional discrepancy across agents and therefore improves both the convergence speed and the final accuracy of any federated learning algorithm applied afterward. Specifically, the authors' preprocessing requires no labels, no training-time optimization, and no modification to the federated learning loop; it is a zero-shot alignment applied once before training. The paper demonstrates the effect with FedAvg on CIFAR-10, where the aligned data yields dramatically higher accuracy than plain FedAvg across all simulated client counts in Tables 1 and 2, and higher than several published federated methods compared in Table 3.
Load-bearing premise
The reported testing accuracy is computed on a held-out test set that was not used to train the model, to compute the per-client barycenters, or to compute the global barycenter; the paper never states this split.
Editorial extensions
If this is right
- If the reported gains hold, any federated algorithm can be preceded by the one-time OT preprocessing without changing aggregation or local training, so the benefit stacks with later algorithmic improvements.
- The preprocessing reduces the number of communication rounds needed to reach a given accuracy: the paper reports 99.62% in 35 rounds for 5 clients, where plain FedAvg sits at 71.22%.
- The added cost is paid in parallel per client, with the paper's complexity analysis giving $O(M d^2/\epsilon^2) + O(N d^2/\epsilon^2) + O(M d^2)$ for $N$ agents each holding $M$ images.
- Since preprocessing is model-agnostic, it can be combined with other federated learning methods, including those designed for personalization or domain generalization.
Reading between the lines
- If the reported accuracy is genuine held-out test accuracy, it would imply that barycentric color alignment makes the classification problem dramatically easier, effectively acting as a strong, label-free domain shift; a testable consequence is that per-channel pixel histograms across clients become much more similar after projection.
- A crucial check the paper leaves implicit is whether the global barycenter is computed from training data only; recomputing the pipeline with the standard 50,000-image training split and evaluating on the standard 10,000-image test set would settle the evaluation question.
- The paper tests only balanced random partitions of CIFAR-10, not the label-skewed and size-skewed data distributions that motivate dataset imbalance in federated learning, so an extension to heterogeneous real-world splits would probe the core claim more directly.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an OT-based preprocessing step for federated learning: each client computes channel-wise Wasserstein barycenters of its local images, a central server aggregates these into a global RGB barycenter target, and clients project their images to that target before training with FedAvg. Experiments on CIFAR-10 report large accuracy gains (e.g., 99.62% vs. 71.22% for 5 clients) and claim favorable comparisons with prior FL methods. The algorithmic idea is clearly stated and the OT formulation is standard, but the experimental validation lacks a defined train/test split, repeated trials, and controlled comparisons; the reported accuracy levels are implausible for the stated architecture on held-out CIFAR-10.
Significance. If the empirical claims were valid, the method would offer a simple, one-time preprocessing step that improves FedAvg under heterogeneous client data, with a clean OT formulation and no fitted parameters aimed at the downstream accuracy. The paper also gives a useful complexity analysis and clearly separates the preprocessing cost from training cost. However, the central evidence is not credible as reported: no held-out evaluation protocol is specified, no error bars or seeds are given, and the accuracy magnitudes are far outside what the stated small CNN should achieve on CIFAR-10 test data. These issues prevent the result from being accepted as a demonstrated improvement.
major comments (4)
- [§5, Tables 1 and 2] The paper never defines the train/test split or the evaluation protocol for the 'Testing Accuracy' columns. Section 5 says data are distributed by 'uniformly sampling them without replacement' but does not state whether the CIFAR-10 test set is held out from all clients and from the computation of the barycenters and projections. If the reported numbers are computed on training data, the comparison against FedAvg in Tables 1 and 2 and against prior methods in Table 3 is invalid. The 99.62% accuracy for the custom CNN (two conv layers, three linear layers, no augmentation) after 35 rounds is implausible for a held-out CIFAR-10 test set, which strongly suggests evaluation on training data or leakage. This is load-bearing because the paper's central claim is the accuracy gain.
- [§6, Tables 1 and 2] No repeated runs, random seeds, or variance estimates are reported; each accuracy is a single number. Given the stochastic nature of FL training and client sampling, it is impossible to determine whether the reported differences (e.g., 93.34 vs. 66.16 in Table 1) are statistically meaningful. The paper should report mean and standard deviation over at least several seeds and specify the client data partition seed.
- [Table 3] The comparison in Table 3 is not controlled. Rows from references [19], [20], [21], and [5] use different architectures, communication rounds, client numbers, and hyperparameters; the footnote asks readers to consult the cited papers, but no attempt is made to match evaluation conditions. Consequently, the claim in Section 5 that 'Our OT-preprocessing algorithm, paired with FedAvg... yielded the best results' is unsupported by a fair comparison. A valid comparison would vary only the preprocessing method while holding model, rounds, clients, and data partition fixed.
- [Algorithm 1 / Section 4] The projection step 'Project image i → WBG' is underspecified. The text says to compute transportation plans to the target space and cites Ferradans et al. [6], but it does not define the empirical distributions, the ground cost, or how the Sinkhorn plan computed on 250 sampled pixels is applied to the full image. This makes the method not fully reproducible from the manuscript. Please provide the exact projection formula and any clipping or normalization applied to the transported pixel values.
minor comments (6)
- [Section 2] 'n summary' appears to be a typo for 'In summary.'
- [Section 6] 'Bregmen projection' should be 'Bregman projection.'
- [Algorithm 1] Notation is inconsistent: 'WB a_g' and 'W a_g' are used interchangeably; please define all symbols consistently.
- [Section 6] There is an extra space before the period in 'efficiently .'
- [Table 3] The table header is formatted as 'Table 3 .' with an extra space; fix the formatting.
- [Section 2] The claim of being 'only the second to incorporate OT' should be supported by a more systematic literature search; as written it is hard to verify.
Circularity Check
No significant circularity: the reported gains are empirical measurements of a preprocessing method with no parameters fitted to the target accuracy.
full rationale
The paper's derivation chain is an algorithm, not a fitted prediction: each client computes channel-wise Wasserstein barycenters of its local images, a central server aggregates these into a global barycenter, and local images are projected toward that target using entropic regularized optimal transport (Algorithm 1). No parameter in this pipeline is fitted to the reported testing accuracy, and no step is defined in terms of the result it is supposed to predict. The abstract's statement that projecting data 'towards this target space... minimizes the distributional discrepancy' describes the construction's intended effect rather than a predicted quantity that reduces to its input. The actual claim of improved generalization is supported by the accuracy measurements in Tables 1 and 2, which are empirical outcomes, not identities. There is no load-bearing self-citation: the cited OT tools (e.g., [6], [17], [18]) are external and independently established, and the comparison methods in Table 3 are externally reported. The most serious concern—that the paper never specifies whether 'Testing Accuracy' is computed on a held-out split—is a validity and reproducibility issue, not circularity, because even if the evaluation were flawed, the method's construction would still be independent of the reported number. Therefore no circular step can be identified with the required textual reduction, and the appropriate score is 0.
Assumptions & free parameters
free parameters (5)
- Sinkhorn regularization epsilon =
1e-2
- Bregman regularization lambda =
1e-1
- Number of sampled pixels =
250
- Local epochs =
2 or 5
- Learning rate =
1e-3
assumptions (3)
- standard math Optimal transport and Wasserstein barycenter theory
- domain assumption Pixel intensity distributions represent domain shift
- domain assumption Trusted central server
Cite this review
Pith. "Pith review of Optimal Transport-based Domain Alignment as a Preprocessing Step for Federated Learning." pith.science (2026). https://pith.science/paper/GEOXB5LS
@misc{pith2026250604071,
author = {Pith},
title = {Pith review of: Optimal Transport-based Domain Alignment as a Preprocessing Step for Federated Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/GEOXB5LS}},
note = {Machine review of arXiv:2506.04071}
}
read the original abstract
Federated learning (FL) is a subfield of machine learning that avoids sharing local data with a central server, which can enhance privacy and scalability. The inability to consolidate data leads to a unique problem called dataset imbalance, where agents in a network do not have equal representation of the labels one is trying to learn to predict. In FL, fusing locally-trained models with unbalanced datasets may deteriorate the performance of global model aggregation, and reduce the quality of updated local models and the accuracy of the distributed agents' decisions. In this work, we introduce an Optimal Transport-based preprocessing algorithm that aligns the datasets by minimizing the distributional discrepancy of data along the edge devices. We accomplish this by leveraging Wasserstein barycenters when computing channel-wise averages. These barycenters are collected in a trusted central server where they collectively generate a target RGB space. By projecting our dataset towards this target space, we minimize the distributional discrepancy on a global level, which facilitates the learning process due to a minimization of variance across the samples. We demonstrate the capabilities of the proposed approach over the CIFAR-10 dataset, where we show its capability of reaching higher degrees of generalization in fewer communication rounds.
Reference graph
Works this paper leans on
-
[19]
Generalizing to unseen domains: A survey on domain generalization,
Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and Philip Yu, “Generalizing to unseen domains: A survey on domain generalization,”IEEE Transactions on Knowledge and Data Engineering, 2022
work page 2022
-
[20]
Fedhealth: A federated transfer learning framework for wearable healthcare,
Yiqiang Chen, Xin Qin, Jindong Wang, Chaohui Yu, and Wen Gao, “Fedhealth: A federated transfer learning framework for wearable healthcare,” IEEE Intelligent Systems, vol. 35, no. 4, pp. 83–93, 2020
work page 2020
-
[21]
FedProx (CCVR) 70.99
-
[5]
To demonstrate the advantages of using our preprocessing step, we require a learning algorithm
EXPERIMENTS AND RESULTS Our framework is built independently of the learning al- gorithm, which allows for flexible integration into other pipelines that may have different goals. To demonstrate the advantages of using our preprocessing step, we require a learning algorithm. To this end, we chose to work with fed- erated averaging, FedAvg, which trains lo...
-
[6]
IMPLEMENTATION DETAILS In this short section, we will clarify details for reproducibil- ity. The custom CNN has an input convolutional layer (conv Table 2. This table contains the results of simulations using our approach with a ResNet9 model and our custom CNN (for comparison). For clarification, “Comm. Rounds” is short for “Communication Rounds.” Comm. ...
-
[1]
INTRODUCTION Federated Learning (FL), a subdomain of Machine Learn- ing (ML), addresses the challenges of decentralized or dis- tributed model training. It offers a compelling framework for scenarios in which data cannot be centrally aggregated due to privacy constraints, thereby promoting compliance with data protection regulations and enhancing scalabil...
-
[2]
Optimal Transport-based Domain Alignment as a Preprocessing Step for Federated Learning
RELATED WORKS The issue of dataset imbalance, also known as the domain- distribution discrepancy problem or the multiple-source do- main problem, has been a core challenge in federated learn- ing. In this section, we will review various approaches to address this challenge. Our approach is, to our knowledge, the first to perform zero-shot distribution ali...
work page Pith review arXiv 2025
-
[3]
PRELIMINARIES 3.1. Federated Learning The goal of federated learning (FL) is to train a single (global) model to make accurate predictions across all agents in a net- work. FL problems contain constraints, including computa- tional power at the edge, communication bandwidth, and data heterogeneity. Privacy is one of the key motivations for us- ing FL whic...
Show all 34 references
-
[4]
We achieve this distribution-alignment goal by generating a tar- get space to which we project all local data
OPTIMAL TRANSPORT-BASED PREPROCESSING In this section, we introduce the preprocessing step that min- imizes the distributional discrepancy in our network. We achieve this distribution-alignment goal by generating a tar- get space to which we project all local data. Our propose...
-
[7]
COMPLEXITY ANALYSIS An important tradeoff to consider with our algorithm is the additional time that must be paid to convert the original dataset into an aligned dataset. As our method does not affect the learning algorithm, we only analyze the complexity of computing the bary...
-
[8]
We accomplish this by projecting local data to a space that encodes all local data, in turn minimizing the distributional discrepancy between agents
CONCLUSION AND FUTURE WORK In this work, we demonstrated the ability of our proposed preprocessing algorithm to improve the convergence speed and generalization of traditional FL. We accomplish this by projecting local data to a space that encodes all local data, in turn minim...
-
[9]
Collaborative op- timization and aggregation for decentralized domain generalization and adaptation,
Guile Wu and Shaogang Gong, “Collaborative op- timization and aggregation for decentralized domain generalization and adaptation,” in Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, 2021, pp. 6484–6493
2021
-
[10]
FedAvgM (CCVR) 71.49
-
[11]
Therefore, we have O(M d2/ϵ2) for local barycenters and O(N d2/ϵ2) for the global barycenter
FedOT 72.2 these are computed in parallel, the time complexity is equiv- alent to computing one, with the addition of computing the global barycenter. Therefore, we have O(M d2/ϵ2) for local barycenters and O(N d2/ϵ2) for the global barycenter. The projection time complexity i...
-
[12]
Clustered federated learning: Model-agnostic distributed multitask optimization under privacy con- straints,
Felix Sattler, Klaus-Robert M ¨uller, and Wojciech Samek, “Clustered federated learning: Model-agnostic distributed multitask optimization under privacy con- straints,” IEEE transactions on neural networks and learning systems, vol. 32, no. 8, pp. 3710–3722, 2020
2020
-
[13]
Communication-efficient learning of deep networks from decentralized data,
Brendan McMahan, Eider Moore, Daniel Ram- age, Seth Hampson, and Blaise Aguera y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics. PMLR, 2017, pp. 1273–1282
2017
-
[14]
Towards personalized federated learning,
Alysa Ziying Tan, Han Yu, Lizhen Cui, and Qiang Yang, “Towards personalized federated learning,”IEEE Transactions on Neural Networks and Learning Sys- tems, 2022
2022
-
[15]
Federated learning: Challenges, methods, and future directions,
Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Vir- ginia Smith, “Federated learning: Challenges, methods, and future directions,” IEEE signal processing maga- zine, vol. 37, no. 3, pp. 50–60, 2020
2020
-
[16]
Fedprof: Selective federated learning based on distri- butional representation profiling,
Wentai Wu, Ligang He, Weiwei Lin, and Carsten Maple, “Fedprof: Selective federated learning based on distri- butional representation profiling,” IEEE Transactions on Parallel and Distributed Systems, vol. 34, no. 6, pp. 1942–1953, 2023
1942
-
[17]
An optimal transport approach to personalized federated learning,
Farzan Farnia, Amirhossein Reisizadeh, Ramtin Pedarsani, and Ali Jadbabaie, “An optimal transport approach to personalized federated learning,” IEEE Journal on Selected Areas in Information Theory , vol. 3, no. 2, pp. 162–171, 2022
2022
-
[18]
Regularized discrete optimal transport,
Sira Ferradans, Nicolas Papadakis, Gabriel Peyr ´e, and Jean-Franc ¸ois Aujol, “Regularized discrete optimal transport,” SIAM Journal on Imaging Sciences , vol. 7, no. 3, pp. 1853–1882, 2014
2014
-
[22]
Given n samples of dimension d and reg- ularization parameter ϵ, we have a complexity of O(nd2/ϵ2)
gives us a complexity analysis for the iterative Bregman projection method of computing regularized barycenters, as introduced by [18]. Given n samples of dimension d and reg- ularization parameter ϵ, we have a complexity of O(nd2/ϵ2). Moreover, the projection of the local ima...
-
[23]
Federated learning with domain gen- eralization,
Liling Zhang, Xinyu Lei, Yichun Shi, Hongyu Huang, and Chao Chen, “Federated learning with domain gen- eralization,” arXiv preprint arXiv:2111.10487, 2021
2021 arXiv
-
[24]
Learning to generate novel domains for domain generalization,
Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang, “Learning to generate novel domains for domain generalization,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16. Springer, 2020, pp. 561–578
2020
-
[25]
Federated visual classification with real-world data dis- tribution,
Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown, “Federated visual classification with real-world data dis- tribution,” in Computer Vision–ECCV 2020: 16th Euro- pean Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16. Springer, 2020, pp. 76–92
2020
-
[26]
Personalized federated learning with theoretical guar- antees: A model-agnostic meta-learning approach,
Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar, “Personalized federated learning with theoretical guar- antees: A model-agnostic meta-learning approach,” Ad- vances in Neural Information Processing Systems , vol. 33, pp. 3557–3568, 2020
2020
-
[27]
M ´emoire sur la th´eorie des d´eblais et des remblais,
Gaspard Monge, “M ´emoire sur la th´eorie des d´eblais et des remblais,” Mem. Math. Phys. Acad. Royale Sci., pp. 666–704, 1781
-
[28]
Fast computation of wasserstein barycenters,
Marco Cuturi and Arnaud Doucet, “Fast computation of wasserstein barycenters,” in International conference on machine learning. PMLR, 2014, pp. 685–693
2014
-
[29]
Sinkhorn distances: Lightspeed compu- tation of optimal transport,
Marco Cuturi, “Sinkhorn distances: Lightspeed compu- tation of optimal transport,” Advances in neural infor- mation processing systems, vol. 26, 2013
2013
-
[30]
Iterative bregman pro- jections for regularized transportation problems,
Jean-David Benamou, Guillaume Carlier, Marco Cuturi, Luca Nenna, and Gabriel Peyr´e, “Iterative bregman pro- jections for regularized transportation problems,” SIAM Journal on Scientific Computing , vol. 37, no. 2, pp. A1111–A1138, 2015
2015
-
[31]
Model- contrastive federated learning,
Qinbin Li, Bingsheng He, and Dawn Song, “Model- contrastive federated learning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 10713–10722
2021
-
[32]
Feder- ated learning with matched averaging,
Hongyi Wang, Mikhail Yurochkin, Yuekai Sun, Dim- itris Papailiopoulos, and Yasaman Khazaeni, “Feder- ated learning with matched averaging,” arXiv preprint arXiv:2002.06440, 2020
2002 arXiv
-
[33]
No fear of heterogeneity: Classifier calibration for federated learning with non-iid data,
Mi Luo, Fei Chen, Dapeng Hu, Yifan Zhang, Jian Liang, and Jiashi Feng, “No fear of heterogeneity: Classifier calibration for federated learning with non-iid data,”Ad- vances in Neural Information Processing Systems , vol. 34, pp. 5972–5984, 2021
2021
-
[34]
On the complexity of approximating wasser- stein barycenters,
Alexey Kroshnin, Nazarii Tupitsa, Darina Dvinskikh, Pavel Dvurechensky, Alexander Gasnikov, and Cesar Uribe, “On the complexity of approximating wasser- stein barycenters,” in International conference on ma- chine learning. PMLR, 2019, pp. 3530–3540
2019
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.