Pith. sign in

REVIEW 4 major objections 4 minor 7 references

FedSTaS: Client Stratification and Client Level Sampling for Efficient Federated Learning

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that FedSTaS, which adds a privacy-preserving per-client data sampling step to the stratified client selection of FedSTS, achieves higher test accuracy than FedSTS alone within a fixed number of training rounds on non-IID…

desk verdict A reasonable engineering combo undone by an inconsistent data-sampling spec; the reported gains are not traceable to a well-defined algorithm. read the letter →

arxiv 2412.14226 v2 pith:MYNNTNIK submitted 2024-12-18 cs.LG stat.ML

Jordan Slessor , Dezheng Kong , Xiaofen Tang , Zheng En Than , Linglong Kong This is my paper · ORCID
classification cs.LGstat.ML
keywords federatedlearningclientsamplingstratifieddata-levellocaldifferentialprivacynon-IIDdataNeymanallocationgradientcompression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FedSTaS is a federated learning sampling method that combines two ideas: stratified client selection based on compressed gradients with optimal Neyman allocation, and per-client data-level uniform sampling driven by a differentially private estimate of the total participating data size. The paper claims that this two-level scheme reaches higher test accuracy than FedSTS alone within a fixed number of training rounds on both IID and non-IID data, and that the private data-size estimate satisfies $\epsilon$-local differential privacy. If correct, this gives a practical way to get faster convergence and better final accuracy in heterogeneous federated learning while keeping client data sizes private.

What carries the argument

The central machinery is a two-stage sampling pipeline. At the client level, FedSTaS uses the FedSTS mechanism: clients are clustered by their Information Squeezed (compressed) gradient vectors into $H$ strata, the $m$ sampled clients are allocated across strata in proportion to $N_h S_h$ where $S_h$ is the stratum standard deviation (Neyman allocation), and clients within a stratum are sampled with probability proportional to gradient norm. At the data level, each participating client computes a clipped local size $n_{hi,c} = \min(n_{hi}, M-1)$, draws a fake size from a uniform multinomial distribution, and applies a Bernoulli mix with parameter $\alpha$ to produce a perturbed size $r_{hi}$; the server aggregates these into the private total estimate $\tilde{n}$ and uses sampling probability $\eta/\tilde{n}$ for each client's local data. The paper's theoretical claims rest on two inherited lemmas: that the client-level sampling is unbiased, and that the private estimator $\tilde{n}$ converges in mean square to the true total and satisfies $\epsilon$-LDP under the stated choice of $\alpha$.

What would settle it

Run FedSTaS on MNIST with $\alpha = 0.01$ for 99 rounds with the stated hyperparameters across multiple random seeds: if its test accuracy does not consistently exceed FedSTS's 54.2%, the paper's central empirical claim fails. Alternatively, record the realized number of data samples per round when each client samples with probability $\eta/\tilde{n}$; if that realized total deviates substantially from $\eta$ across rounds, the centralized-learning mimicry assumption breaks.

Watch

Extended reading notes

Core claim

FedSTaS is a two-level sampling scheme. At the client level, the server stratifies clients by their compressed gradients, re-allocates the client sample size across strata using optimal Neyman allocation, and samples clients within each stratum with probability proportional to gradient norm. At the data level, each participating client computes a clipped local data size, mixes it with a fake size via a Bernoulli draw, and the server forms a private estimate of the total participating data size, $\tilde{n} = (R - (1-\alpha)M m)/\alpha$, which is then used as the denominator in a uniform per-client data sampling probability $\eta/\tilde{n}$. The paper claims three theoretical properties: client-level unbiasedness inherited from FedSTS, mean-square convergence of the data-sampling probability to the centralized probability as the number of sampled clients grows, and $\epsilon$-LDP for the private size estimate when $\alpha = (e^\epsilon - 1)/(e^\epsilon + M - 2)$. Empirically, the paper reports FedSTaS reaches 56.00% on MNIST ($\alpha=0.01$), 61.90% on MNIST ($\alpha=0.001$), and 23.21% on CIFAR-100 ($\alpha=0.001$), exceeding FedSTS's 54.2%, 44.3%, and 18.0% within the same number of rounds.

Load-bearing premise

The load-bearing premise is that the unbiasedness, convergence, and privacy guarantees proved separately for the client-sampling and data-sampling components continue to hold exactly when the two components are combined as in FedSTaS, even though the total sampled data size is only approximately $\eta$.

Editorial extensions

If this is right

  • Within a fixed round budget, FedSTaS should reach higher test accuracy than FedSTS on non-IID MNIST and CIFAR-100 splits, with the gap widening as data heterogeneity intensifies.
  • The private total-size estimator lets the server run centralized-style data sampling without learning any client's exact data size, with a formal $\epsilon$-LDP guarantee.
  • The DP variant (DP+FedSTaS) should retain most of the accuracy benefit while providing a privacy guarantee of $\epsilon = 3$.
  • The client-level sampling scheme inherits unbiasedness from FedSTS, so the aggregated update does not systematically drift from the full-client aggregation.
  • As the number of sampled clients grows, the data-level sampling probability approaches the centralized learning probability, so the method behaves more like centralized learning in the large-$m$ limit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper only compares against FedSTS, a natural testable extension is whether adding data-level sampling to other client-selection baselines, such as clustered or regularized sampling, yields similar gains; the data-level mechanism is not obviously tied to stratification.
  • The paper does not establish end-to-end privacy: the $\epsilon$-LDP result covers only the local data-size estimate, while the compressed gradients themselves are transmitted without a DP guarantee, so a full privacy-accounting extension would be needed for practical private deployment.
  • The realized total number of data samples per round is only approximately $\eta$, since each participating client samples with probability $\eta/\tilde{n}$ independently; a direct measurement of the realized sample size would reveal how closely the centralized-learning mimicry holds in practice.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes FedSTaS, a federated learning client-selection and data-sampling method that combines FedSTS-style client stratification (based on compressed gradients and Neyman allocation) with FedSampling-style local data sampling under local differential privacy. The authors claim unbiasedness and an epsilon-LDP guarantee for the private data-size estimator, and they report higher test accuracy than FedSTS on MNIST and CIFAR-100 under non-IID data. The theoretical results are presented as direct consequences or slight modifications of prior work, and the experimental evidence consists of single-run accuracy curves and bar charts.

Significance. If the central claim is correct, the proposed integration is a reasonable engineering contribution: it combines two known sampling techniques and adds a privacy-preserving data-size estimator with a concrete epsilon=3 calibration. The manuscript is clearly written and makes its algorithmic steps explicit, and the privacy mechanism is well specified. However, the contribution rests primarily on the empirical demonstration, and that demonstration is not yet reproducible or statistically supported: the data-sampling step in Algorithm 3 is internally inconsistent, no code or data are released, and the reported results lack multiple seeds or error bars. The theoretical lemmas are imported from FedSTS and FedSampling without verification that their assumptions hold for the combined scheme. These issues currently prevent the paper from being accepted as a reliable advance.

major comments (4)
  1. [§2.4, Algorithm 3] The data-sampling instruction is internally inconsistent. Section 2.4 says the server wishes to sample eta observations across the m sampled clients with uniform sampling probabilities eta/n, but Algorithm 3 instructs each client to take 'a sample of (eta/m) observations from D_k taken with sampling probabilities p_kj = eta/n-tilde'. A fixed-size sample of size eta/m cannot have per-point inclusion probability eta/n-tilde in general: the expected number of sampled points from client k under Bernoulli(eta/n-tilde) sampling is n_k * eta/n-tilde, not eta/m, and for any client with n_k < eta/m, a fixed-size sample of size eta/m is not even defined under sampling without replacement. The two plausible implementations (fixed-size per-client samples versus Bernoulli per-point sampling) lead to different local training sets and therefore different test accuracies. Since the headline comparisons in Section 3 depend on this underspecified choice, the evaluated method is not reproducible from the text, and no code or data are provided to resolve the ambiguity.
  2. [§3.2, Figures 2–3] The experimental claims are based on single runs without any measure of variability. The text reports point estimates such as 56.00% versus 54.2% on MNIST (alpha=0.01) and 61.90% versus 44.3% on MNIST (alpha=0.001), but no standard deviations, multiple seeds, or statistical tests are reported. Given that the claimed gains include a 1.8-percentage-point difference on one setting, it is essential to show that the improvement is not within run-to-run noise. The authors should provide multi-seed results with error bars and, ideally, a per-seed table.
  3. [§2.5, Lemmas 2.1–2.4] The theoretical results are asserted as immediate consequences or slight modifications of results in FedSTS and FedSampling, but no derivations are shown for the combined FedSTaS scheme. Lemma 2.1 assumes FedSTS's unbiasedness transfers exactly when client sampling is followed by data-level sampling, Lemma 2.2 assumes the FedSampling mean-square-error argument carries over to the stratified client subset, and Lemma 2.4 assumes the LDP argument is unchanged when the randomized response mechanism is combined with the client-stratification procedure. Because the combined sampling distribution is different from either prior method, these are not automatic; the paper should either provide proofs or state explicitly which conditions need to be verified.
  4. [§3.2.2, Figure 3 caption] There is an internal inconsistency in the reporting of the CIFAR-100 experiment. The text says 'after 99 rounds' for the bottom chart, while the Figure 3 caption says 'CIFAR-100 with alpha=0.001 at 199 rounds' and Section 3.1 states T=199 for CIFAR-100. This needs to be corrected, as the number of rounds directly affects the claimed improvement over FedSTS.
minor comments (4)
  1. [§3.2, MNIST (alpha=0.01)] The phrase 'improving upon FedSTS by 1.8%' should specify whether this is an absolute percentage-point difference or a relative improvement; 56.00% versus 54.2% is a 1.8-percentage-point difference but a relative improvement of about 3.3%.
  2. [§3.2.2, MNIST (alpha=0.001)] The phrase 'surpassing FedSTS by a remarkable 17.6%' similarly needs clarification: this is a 17.6-percentage-point absolute difference, not a 17.6% relative improvement (the relative gain is about 39.7%).
  3. [§2.4 and Algorithm 3] The symbol eta is used both for the learning rate in the local update (Algorithm 3, line 5) and for the total data-level sample size in Section 2.4 and Algorithm 2. This makes the hyperparameter list in Section 3.1 confusing, where eta=0.01 appears alongside q=0.1 as the factor determining the sample size. The authors should use distinct symbols (for example, eta for the learning rate and S or n_sample for the requested data sample size).
  4. [§2.4] The definition of n as the total number of participating observations is introduced but then replaced by the private estimate n-tilde; it would help to explicitly state that n is the non-private ideal quantity and n-tilde is its private estimate, and to note that in Algorithm 2 the same symbol eta is reused for both the data sample size and the learning rate.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: theoretical results are transferred from external prior work (FedSTS, FedSampling); absent proofs and an inconsistent sampling step are weaknesses, not circularity.

full rationale

The derivation chain contains no step that is equivalent to its input by construction. Lemma 2.1 is justified as an 'immediate consequence' of FedSTS's unbiasedness, and since FedSTaS uses FedSTS's client-stratification and Neyman allocation verbatim, this is a legitimate transfer of an external theorem, not a self-referential argument. Lemmas 2.2 and 2.4 are likewise 'slight modification(s)' of FedSampling lemmas (Qi et al. 2023), a different author group; citing those results is independent support even if the present paper omits the adapted proofs. The experimental accuracy figures are measurements, not predictions from fitted parameters, so no 'fitted input called prediction' pattern applies. The main technical problem is that Algorithm 3's fixed-size per-client sample (eta/m) with per-point probability eta/ntilde is internally inconsistent, and the claimed gains are therefore not reproducible from the text; however, that is a correctness and reproducibility defect, not circularity. No self-citations are load-bearing, and no prior-work result is being renamed as a new derivation.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

No new theoretical entities are introduced. The free parameters are mostly standard FL or FedSTS hyperparameters. The paper's contribution is an empirical combination, so the risk is in unvalidated inherited assumptions and ambiguous implementation details.

free parameters (4)
  • alpha_dp = 0.1616
    Set to 0.1616 in experiments, presumably to achieve a DP epsilon of 3, but the relationship to DP is not derived or numerically verified in the paper.
  • M = 100
    Size threshold for clipping sample sizes; chosen ad hoc in experiments.
  • H = 10
    Number of strata; chosen ad hoc, no sensitivity analysis.
  • m = 10
    Number of sampled clients per round; tied to H=10, but no justification.
assumptions (3)
  • domain assumption FedSTS client sampling is unbiased (Lemma 2.1).
    The unbiasedness lemma is stated as an immediate consequence of FedSTS, with no proof or verification for the FedSTaS pipeline.
  • domain assumption The FedSampling data-level sampling lemmas (2.2 and 2.4) remain valid when restricted to participating clients.
    The paper only states the lemmas as slight modifications of FedSampling without proof, assuming the privacy and MSE properties transfer.
  • ad hoc to paper Compressed gradients preserve sufficient information for stratification and importance sampling to be effective.
    The core mechanism of FedSTaS relies on gradient compression (IS gradients, Kdesired, d-prim) without any theoretical or empirical validation of information preservation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FedSTaS: Client Stratification and Client Level Sampling for Efficient Federated Learning." pith.science (2026). https://pith.science/paper/MYNNTNIK

@misc{pith2026241214226,
  author       = {Pith},
  title        = {Pith review of: FedSTaS: Client Stratification and Client Level Sampling for Efficient Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MYNNTNIK}},
  note         = {Machine review of arXiv:2412.14226}
}
read the original abstract

Federated learning (FL) is a machine learning methodology that involves the collaborative training of a global model across multiple decentralized clients in a privacy-preserving way. Several FL methods are introduced to tackle communication inefficiencies but do not address how to sample participating clients in each round effectively and in a privacy-preserving manner. In this paper, we propose \textit{FedSTaS}, a client and data-level sampling method inspired by \textit{FedSTS} and \textit{FedSampling}. In each federated learning round, \textit{FedSTaS} stratifies clients based on their compressed gradients, re-allocate the number of clients to sample using an optimal Neyman allocation, and sample local data from each participating clients using a data uniform sampling strategy. Experiments on three datasets show that \textit{FedSTaS} can achieve higher accuracy scores than those of \textit{FedSTS} within a fixed number of training rounds.

Figures

Figures reproduced from arXiv: 2412.14226 by the authors.

Figure 1
Figure 1. visualizes the data partition under both IID and non￾IID distributions with α = 0.01. The non-IID case clearly exhibits a more uneven distribution of labels compared to the IID setting, closely mirroring realistic federated learning environments [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Non-Convex model with q = 0.1, nSGD = 3, η = 0.01, B = 128 on MNIST, with α = 0.01, niter = 99, Kdesired = 2048, d ′ = 9, M = 100, and αdp = 0.1616 (DP Privacy = 3). ate, reflecting the challenges posed by the non-IID data distribution. FedSTaS (Compressed Gradients): Our proposed com￾pressed gradient method exhibits a substantially faster rate of improvement. It surpasses the FedSTS baseline by round 40 and continu… view at source ↗
Figure 3
Figure 3. Non-Convex model with q = 0.1, nSGD = 3, η = 0.01, B = 128 on MNIST and CIFAR-100 with varying α values (0.01 and 0.001). The desired dimension is Kdesired = 2048, d ′ = 9, M = 100, and αdp = 0.1616 ensures DP Privacy ϵ = 3. For MNIST with α = 0.01 after 99 rounds (top chart), Fed￾STaS (our Compressed Gradients method) achieves 56.00%, improving upon FedSTS by 1.8%, while DP + FedSTaS reaches 54.67%, still outperfor… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

7 extracted references · 3 canonical work pages

  1. [1]

    Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning

    Fraboni, Y., Vidal, R., Kameni, L., and Lorenzi, M. Clustered Sampling : Low - Variance and Improved Representativity for Clients Selection in Federated Learning , May 2021. URL http://arxiv.org/abs/2105.05883. arXiv:2105.05883

    work page Pith review arXiv 2021

  2. [2]

    FedSTS : A Stratified Client Selection Framework for Consistently Fast Federated Learning

    Gao, D., Song, D., Shen, G., Cai, X., Yang, L., Liu, G., Li, X., and Wang, Z. FedSTS : A Stratified Client Selection Framework for Consistently Fast Federated Learning . IEEE Transactions on Neural Networks and Learning Systems, pp.\ 1--15, 2024. ISSN 2162-237X, 2162-2388. doi:10.1109/TNNLS.2024.3438843. URL https://ieeexplore.ieee.org/document/10689614/

    work page arXiv 2024

  3. [3]

    K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V

    Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., and Smith, V. Federated Optimization in Heterogeneous Networks , April 2020. URL http://arxiv.org/abs/1812.06127. arXiv:1812.06127

    arXiv 2020

  4. [4]

    Federated learning based on stratified sampling and regularization

    Lu, C., Ma, W., Wang, R., Deng, S., and Wu, Y. Federated learning based on stratified sampling and regularization. Complex & Intelligent Systems, 9 0 (2): 0 2081--2099, April 2023. ISSN 2199-4536, 2198-6053. doi:10.1007/s40747-022-00895-3. URL https://link.springer.com/10.1007/s40747-022-00895-3

    doi:10.1007/s40747-022-00895-3 2023

  5. [5]

    B., Moore, E., Ramage, D., Hampson, S., and Arcas, B

    McMahan, H. B., Moore, E., Ramage, D., Hampson, S., and Arcas, B. A. y. Communication- Efficient Learning of Deep Networks from Decentralized Data , January 2023. URL http://arxiv.org/abs/1602.05629. arXiv:1602.05629

    arXiv 2023

  6. [6]

    FedSampling: A Better Sampling Strategy for Federated Learning

    Qi, T., Wu, F., Lyu, L., Huang, Y., and Xie, X. FedSampling : A Better Sampling Strategy for Federated Learning , June 2023. URL http://arxiv.org/abs/2306.14245. arXiv:2306.14245

    work page Pith review arXiv 2023

  7. [7]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.