Pith. sign in

REVIEW 3 major objections 5 minor 49 references

Ampere: Communication-Efficient and High-Accuracy Split Federated Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Ampere replaces split federated learning's per-step activation and gradient exchanges with a one-shot activation transfer, claiming up to 99.1% less device-server communication and up to 13.26% higher accuracy.

desk verdict Useful SFL variant with a clean core idea, but the headline communication claim is undermined by an algorithm/proof mismatch that needs fixing before acceptance. read the letter →

arxiv 2507.07130 v1 pith:KJKG3KST submitted 2025-07-08 cs.DC cs.LG

Zihan Zhang , Leon Wong , Blesson Varghese This is my paper · ORCID
classification cs.DCcs.LG
keywords splitfederatedlearningcommunicationefficiencyon-devicecomputationunidirectionalinter-blocktrainingauxiliarynetworkactivationconsolidationnon-IIDdataedgecomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Ampere is a split federated learning system that removes the central cost of split learning: the per-iteration exchange of activations and gradients between devices and the server. Rather than training the device and server halves of a network jointly by end-to-end backpropagation, Ampere first trains the device block on-device with a small auxiliary network, sends the resulting activations to the server once, and then trains the server block on the consolidated activation set. The paper reports that this reduces device-server communication by up to 99.1% and on-device computation by up to 93.13% compared with existing split federated learning baselines, while improving model accuracy by up to 13.26% and reducing the standard deviation of accuracy by 53.39% under non-IID data. If correct, this would make split federated learning practical on bandwidth-limited edge devices rather than only on well-connected infrastructure.

What carries the argument

The load-bearing mechanism is the lightweight auxiliary network $\tilde{\theta}^{(d)}$ attached to the device block. It is a two-layer network whose first layer mirrors the server block's first layer at half dimension and whose second layer is a fully connected layer with the same loss function as the server block. Its job is to decouple device and server training: it lets the device compute gradients for its own block locally, so the iterative activation-gradient loop of split federated learning can be replaced by one-shot activation transfer. The auxiliary network also shapes what features the device block extracts, since shallow, generalizable representations transfer better to the downstream server block than task-specific ones.

What would settle it

Run Ampere with more than one mini-batch per epoch, count every byte exchanged on the wire, and compare the total with Equation (27); if the measured volume scales with the number of mini-batches rather than matching the formula, the claimed 99.1% communication reduction does not hold for that configuration.

Watch

Extended reading notes

Core claim

The paper's central claim is that the interdependence of device and server blocks in split federated learning is unnecessary, and that breaking it by sequential training improves everything at once: communication, computation, accuracy, and robustness to data heterogeneity. The mechanism is unidirectional inter-block training: a lightweight auxiliary network lets each device compute a local loss and update its own block without gradients from the server; once the device block converges, activations are uploaded to the server exactly once; the server then trains a single consolidated server block on merged activations from all devices. The paper argues that this activation consolidation turns skewed per-device activations into a more homogeneous dataset, which is why accuracy improves on non-IID data. It also argues that because the one-time activation transfer cost is negligible compared with model exchanges over many epochs, communication becomes dominated by the small device-block exchange, which is less than the full model exchange of classic federated learning.

Load-bearing premise

The communication cost formula in Equation (27) counts one model exchange per device per epoch, but Algorithm 1 has devices upload their models after every mini-batch; if an epoch has more than one mini-batch, the actual communication volume is proportionally larger.

Editorial extensions

If this is right

  • Split federated learning can be made communication-competitive with classic federated learning, since per-step activation and gradient transfers are replaced by a single activation upload per device.
  • Devices can exit training much earlier; the paper reports device participation time reductions of 44.3% to 96.3%, freeing devices for other work.
  • Server-side training on consolidated activations yields accuracy gains of up to 13.26% and a 53.39% reduction in accuracy standard deviation across non-IID settings.
  • The split point no longer requires a trade-off between on-device computation and communication; Ampere claims the same split point minimizes both.
  • The approach applies to both CNNs and vision transformers, and to CIFAR-10 and Tiny ImageNet, suggesting it generalizes across architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If Ampere's one-shot transfer holds up, it suggests a general design pattern for split learning: any method that can cheaply predict the downstream task's needs at the split boundary can replace iterative backpropagation with a single forward pass, at the price of a gap between local-loss features and global-task features.
  • The auxiliary network dimension ratio (fixed at 0.5 in the paper) is a tunable knob; the paper's own ablation suggests accuracy saturates around 0.5 while computation grows linearly, so the optimal ratio may shift with model scale or dataset difficulty.
  • A natural stress test would be to run Ampere with more than one local mini-batch per epoch and count actual bytes sent, to see whether the communication formula in Equation (27) correctly predicts measured traffic when device blocks are exchanged more than once per epoch.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes Ampere, a split federated learning system that trains the device block and server block sequentially rather than by end-to-end backpropagation. A lightweight auxiliary network lets devices train the device block locally; activations are then transferred once to the server, where they are consolidated into a single activation set on which the server block is trained. The authors claim that, compared with SFL baselines, Ampere reduces device-server communication by up to 99.1%, reduces on-device computation by up to 93.13%, reduces training time by up to 94.6%, and improves accuracy by up to 13.26%, with lower variance under non-IID data. The evaluation uses four models (MobileNet-L, VGG-11, Swin-T, ViT-S) on CIFAR-10 and Tiny ImageNet, on a 120-device Jetson testbed, against SplitFed, PiPar, SplitFed+SCAFFOLD, and SplitGP baselines, plus an ablation of activation consolidation.

Significance. If the claims hold, Ampere would be a meaningful advance: it eliminates the per-iteration activation/gradient exchange that dominates SFL communication, reduces the split-point trade-off between on-device computation and communication, and mitigates non-IID degradation by training the server block on consolidated activations. Strengths of the paper include the breadth of the evaluation (four architectures, two datasets, four SFL baselines), the use of a physical heterogeneous testbed rather than pure simulation, an ablation isolating activation consolidation, and a convergence analysis that builds on existing FedAvg and decoupled-learning results. The main weaknesses are an internal inconsistency between the communication-cost formula and the training algorithm, and the absence of statistical anchoring for the empirical claims. Neither weakness appears impossible to fix, but both are load-bearing for the headline results.

major comments (3)
  1. [§4.2, Eq. (27), Algorithm 1] The communication-cost formula in Eq. (27), C = 2N(s(d)+s(aux)) + s(act), counts exactly one device-block model exchange per epoch. However, Algorithm 1 places the Send and Receive operations for {θ(d)_k, θ̃(d)_k} inside the inner loop over h (lines 6–7), i.e., once per mini-batch rather than once per epoch. Under the algorithm as written, the model-exchange volume is 2NH(s(d)+s(aux)) with H the number of mini-batches per epoch, not 2N(s(d)+s(aux)). Section 5.1 states that each selected device trains on 10,000 local samples before transferring its updated model, which also does not match a per-mini-batch transfer. Since Table 5 and the headline 99.1% communication reduction are computed from Eq. (27), this inconsistency must be resolved: either move the aggregation outside the h loop in Algorithm 1, or define H and the communication interval explicitly, or revise Eq. (27) and all subsequent communication comparisons. This is a load-bearing issue for the paper's primary quantitative claim.
  2. [§5, Figures 8–11, Tables 4–5] The empirical accuracy, training-time, communication, and computation results are reported as single values with no indication of the number of runs, no error bars, and no per-setting hyperparameters (optimizer, learning rate, batch size, early-stopping threshold details). The claim that Ampere 'reduces standard deviation of accuracy by 53.39%' in Observation 3 is computed across non-IID degrees α, not across repeated training runs, so it does not address run-to-run variance. Because the central claim is 'high accuracy' (up to 13.26% improvement), the evaluation should include multiple independent runs with means and standard deviations, or at minimum a statement of how many seeds were used and why single runs are representative. Without this, the accuracy and training-time improvements are not statistically anchored.
  3. [§3.2.2, Figure 7] The auxiliary-network dimension ratio 0.5 is selected from a single accuracy/computation trade-off curve measured for MobileNet-L on CIFAR-10. The paper states that the ratio is user-adjustable, but it is a free parameter used in all subsequent experiments, and Figure 7 does not show whether the choice transfers to other architectures or datasets. Since the auxiliary network is a core component of the method, please include a sensitivity analysis for at least one additional model (e.g., a transformer) or one additional dataset, or explicitly state that the sensitivity has not been evaluated outside the single configuration shown.
minor comments (5)
  1. [§4.2, Eq. (27)] The label 'CF edGrail' appears to be a typographical artifact; it should be 'C_Ampere'. Also, the sentence 's(a) ≪ s(s)' should use s(aux) consistently.
  2. [Algorithm 1 and §5.1] Algorithm 1 aggregates models from all K devices at the server, whereas Section 5.1 says 12 devices are randomly selected to participate in each training round. Please clarify how partial participation is represented in the algorithm and how the aggregation loop over [K] should be read in that setting.
  3. [§5.2.1, Figure 8] The temporary accuracy drop when switching from device-block training to server-block training is described qualitatively, but the reason for the recovery is not analyzed. A short explanation of the dynamics (e.g., the server block starts from random initialization on frozen activations) would help the reader interpret the plots.
  4. [Table 2] The column header of Table 2 appears corrupted or misaligned in the manuscript, and the relationship between 'Model', 's(act)', 's(d)', 's(aux)', and 's(s)' is unclear from the table alone. Please re-typeset the table so that each column is explicitly labeled and the units are unambiguous.
  5. [Eqs. (5), (27)–(31)] The symbol N is used in Eq. (5) as total training epochs and in Table 4 the epochs are split into N(d) and N(s). Please state explicitly which N appears in the communication-cost formulas and whether the same N is used for both FL/SFL and Ampere when comparing communication volume.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Ampere's communication and convergence claims follow from stated design choices and externally cited results, not from assuming the conclusions.

full rationale

The paper's derivation chain is not circular. The communication cost expression in Eq. (27), C=2N(s(d)+s(aux))+s(act), is an algebraic consequence of the stated design: device-block model exchanges for aggregation plus a one-time activation transfer. The reductions versus SFL in Eq. (29) and versus FL in Eq. (31) follow directly from the size inequalities and do not embed the headline percentages as inputs. The convergence results in Section 4.1 are imported from external works: Theorem 1 is explicitly attributed to reference [24] and Theorem 2 is derived from Proposition 3.1 of reference [25]; the setting c_t=0 is a definitional consequence of the sequential schedule (the server block begins only after the device block has converged), not a hidden assumption of the desired convergence. The selection of the auxiliary-network dimension ratio 0.5 is an empirical design choice from Figure 7, and the accuracy, communication, and computation claims are evaluated experimentally against baselines; none of these is a fitted parameter renamed as a prediction. The self-citations (PiPar, FedAdapt, EcoFed) appear only in related-work discussion and are not load-bearing for Ampere's central argument. One caveat belongs to correctness, not circularity: Eq. (27) assumes one model exchange per epoch, whereas Algorithm 1 lines 6-7 place Send/Receive inside the inner mini-batch loop; if H>1 the measured communication volume would be larger. That inconsistency affects the quantitative claim but is not a case of the derivation reducing to its own inputs by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central empirical claims rest on standard FL convexity assumptions that do not hold for the tested networks, on imported convergence theorems, and on a hand-chosen auxiliary network ratio. The communication analysis also assumes an aggregation schedule that contradicts Algorithm 1. No code or seeds are provided.

free parameters (3)
  • Auxiliary network dimension ratio = 0.5
    Selected from Figure 7 as the knee of the accuracy/computation trade-off (Section 3.2.2) and user-adjustable. This choice shapes the reported accuracy and on-device computation and is not derived from first principles.
  • Split point p = 1
    All experiments place only the first layer on-device (Section 3.2.1 argues p=1 simultaneously minimizes computation and communication under UIT). It is a central experimental setting chosen from the analysis rather than fitted to data.
  • Early stopping patience = 15 epochs
    Applied to all methods in Section 5.2.1. It affects final accuracy and training-time comparisons, and no sensitivity analysis is provided.
assumptions (4)
  • domain assumption Device-block and server-block losses are L-smooth and µ-strongly convex (Assumptions 1-2, Section 4.1.2).
    Standard FL assumptions cited from [24], but strong convexity does not hold for the CNNs and transformers evaluated. The convergence theorem inherits this mismatch with the actual models.
  • standard math Theorem 1 of Li et al. [24] applies unchanged to Ampere's device-block training with the auxiliary network.
    The proof of Theorem 1 in Section 4.1.3 is a direct citation ('This is proven in the literature'), not a re-derivation for the auxiliary-network objective.
  • standard math Proposition 3.1 of Belilovsky et al. [25] holds for server-block training on consolidated activations.
    Theorem 2 uses it with c_t=0 because the device block has converged before server training; the proposition's conditions on the modular loss are not verified for the evaluated architectures.
  • ad hoc to paper The frozen, FedAvg-aggregated device block provides a single consistent representation q* for all devices.
    Activation consolidation assumes the aggregated device block is a good common feature extractor for all devices despite non-IID local training; no theoretical support is provided.
invented entities (1)
  • Lightweight auxiliary network (two-layer, first layer half the width of the server block's first layer)
    purpose: Enables local loss on devices, decoupling device-block training from server-block training and eliminating gradient transfer.
    It is a designed component with in-paper ablation and parameter study (Figures 7 and 11), but there is no external validation or falsifiable prediction outside this paper. Its benefit is demonstrated only on the reported benchmarks.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Ampere: Communication-Efficient and High-Accuracy Split Federated Learning." pith.science (2026). https://pith.science/paper/KJKG3KST

@misc{pith2026250707130,
  author       = {Pith},
  title        = {Pith review of: Ampere: Communication-Efficient and High-Accuracy Split Federated Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KJKG3KST}},
  note         = {Machine review of arXiv:2507.07130}
}
read the original abstract

A Federated Learning (FL) system collaboratively trains neural networks across devices and a server but is limited by significant on-device computation costs. Split Federated Learning (SFL) systems mitigate this by offloading a block of layers of the network from the device to a server. However, in doing so, it introduces large communication overheads due to frequent exchanges of intermediate activations and gradients between devices and the server and reduces model accuracy for non-IID data. We propose Ampere, a novel collaborative training system that simultaneously minimizes on-device computation and device-server communication while improving model accuracy. Unlike SFL, which uses a global loss by iterative end-to-end training, Ampere develops unidirectional inter-block training to sequentially train the device and server block with a local loss, eliminating the transfer of gradients. A lightweight auxiliary network generation method decouples training between the device and server, reducing frequent intermediate exchanges to a single transfer, which significantly reduces the communication overhead. Ampere mitigates the impact of data heterogeneity by consolidating activations generated by the trained device block to train the server block, in contrast to SFL, which trains on device-specific, non-IID activations. Extensive experiments on multiple CNNs and transformers show that, compared to state-of-the-art SFL baseline systems, Ampere (i) improves model accuracy by up to 13.26% while reducing training time by up to 94.6%, (ii) reduces device-server communication overhead by up to 99.1% and on-device computation by up to 93.13%, and (iii) reduces standard deviation of accuracy by 53.39% for various non-IID degrees highlighting superior performance when faced with heterogeneous data.

Figures

Figures reproduced from arXiv: 2507.07130 by the authors.

Figure 1
Figure 1. Training in a Split Federated Learning (SFL) system [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Trade-off between on-device computation and device [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Trade-off between communication volume and on [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Model accuracy of MobileNet-Large for varying [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Overview of Ampere. 1 2 3 4 5 6 7 8 9 10111213141516171819 0 1 2 3 4 Split Point Communication Volume (GB) UIT Communication BP Communication 0 50 100 150 Computation (GFLOPs) Computation [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Communication volume and on-device computation [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: Impact of varying auxiliary network dimensions on [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Model accuracy (higher is better) versus training time (lower is better) for four models on two datasets. [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Total on-device computation (in TFLOPS; lower is better) for four different models on two datasets. [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Model accuracy (higher is better) for four models on two datasets for varying non-IID degree [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Model accuracy (higher is better) of Ampere with (w/) and without (w/o) Activation Consolidation on four different models and two datasets. with activation transfers, thereby reducing idle time. This improves training efficiency by computing and communi￾cating concurr…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 45 canonical work pages

  1. [1]

    Communication-Efficient Learning of Deep Networks from Decentralized Data,

    H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in International Conference on Artificial Intelligence and Statistics , 2016. [Online]. Available: https://api.semanticscholar.org/CorpusID:14955348

    work page 2016

  2. [2]

    Federated Optimiza- tion: Distributed Optimization Beyond the Datacenter,

    J. Kone ˇcn´y, B. McMahan, and D. Ramage, “Federated Optimiza- tion: Distributed Optimization Beyond the Datacenter,” CoRR, 2015

    work page 2015

  3. [3]

    Feder- ated Optimization: Distributed Machine Learning for On-Device Intelligence,

    J. Kone ˇcn´y, H. B. McMahan, D. Ramage, and P . Richt´arik, “Feder- ated Optimization: Distributed Machine Learning for On-Device Intelligence,” CoRR, vol. abs/1610.02527, 2016

    arXiv 2016

  4. [4]

    Federated Learning: Strategies for Improving Communication Efficiency,

    J. Kone ˇcn´y, H. B. McMahan, F. X. Yu, P . Richtarik, A. T. Suresh, and D. Bacon, “Federated Learning: Strategies for Improving Communication Efficiency,” in NIPS Workshop on Private Multi-Party Machine Learning , 2016. [Online]. Available: https://arxiv.org/abs/1610.05492

    arXiv 2016

  5. [5]

    SplitFed: When Federated Learning Meets Split Learning,

    C. Thapa, M. A. P . Chamikara, and S. Camtepe, “SplitFed: When Federated Learning Meets Split Learning,” AAAI Conference on Artificial Intelligence, vol. 36(8), pp. 8485–8493, 2022

    work page 2022

  6. [6]

    BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems,

    J. Shao and J. Zhang, “BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systems,” in 2020 IEEE International Conference on Communications Workshops , 2020, pp. 1–6

    work page 2020

  7. [7]

    C3-SL: Circular Convolution-Based Batch-Wise Compression for Communication- Efficient Split Learning,

    C.-Y. Hsieh, Y.-C. Chuang, and A.-Y. Wu, “C3-SL: Circular Convolution-Based Batch-Wise Compression for Communication- Efficient Split Learning,” in IEEE 32nd International Workshop on Machine Learning for Signal Processing, 2022, pp. 1–6

    work page 2022

  8. [8]

    Back-And-Forth Prediction for Deep Tensor Compression,

    H. Choi, R. A. Cohen, and I. V . Baji ´c, “Back-And-Forth Prediction for Deep Tensor Compression,” in IEEE International Conference on Acoustics, Speech and Signal Processing, 2020, pp. 4467–4471

    work page 2020

Show all 49 references
  1. [9]

    Reducing Communication for Split Learning by Randomized Top-k Sparsification,

    F. Zheng, C. Chen, L. Lyu, and B. Yao, “Reducing Communication for Split Learning by Randomized Top-k Sparsification,” in Pro- ceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, 2023

    2023

  2. [10]

    EcoFed: Efficient Communication for DNN Partitioning- Based Federated Learning ,

    D. Wu, R. Ullah, P . Rodgers, P . Kilpatrick, I. Spence, and B. Vargh- ese, “ EcoFed: Efficient Communication for DNN Partitioning- Based Federated Learning ,” IEEE Transactions on Parallel & Dis- tributed Systems, vol. 35, no. 03, pp. 377–390, 2024

    2024

  3. [11]

    PiPar: Pipeline Parallelism for Collaborative Machine Learning,

    Z. Zhang, P . Rodgers, P . Kilpatrick, I. Spence, and B. Varghese, “PiPar: Pipeline Parallelism for Collaborative Machine Learning,” Journal of Parallel and Distributed Computing , vol. 193, p. 104947, 2024

    2024

  4. [12]

    SplitGP: Achieving Both Generalization and Personalization in Federated Learning,

    D.-J. Han, D.-Y. Kim, M. Choi, C. G. Brinton, and J. Moon, “SplitGP: Achieving Both Generalization and Personalization in Federated Learning,” in IEEE Conference on Computer Communica- tions, 2023, pp. 1–10

    2023

  5. [13]

    Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge,

    C. He, M. Annavaram, and S. Avestimehr, “Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge,” in Proceedings of the 34th International Conference on Neural Information Processing Systems, 2020

    2020

  6. [14]

    Measuring the Effects of Non- Identical Data Distribution for Federated Visual Classification,

    T. H. Hsu, H. Qi, and M. Brown, “Measuring the Effects of Non- Identical Data Distribution for Federated Visual Classification,” CoRR, vol. abs/1909.06335, 2019

    1909 arXiv

  7. [15]

    Federated Optimization in Heterogeneous Networks,

    T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V . Smith, “Federated Optimization in Heterogeneous Networks,” inProceed- ings of the Third Conference on Machine Learning and Systems , 2020

    2020

  8. [16]

    SCAFFOLD: Stochastic Controlled Averaging for Feder- ated Learning,

    S. P . Karimireddy, S. Kale, M. Mohri, S. Reddi, S. Stich, and A. T. Suresh, “SCAFFOLD: Stochastic Controlled Averaging for Feder- ated Learning,” in Proceedings of the 37th International Conference on Machine Learning, H. D. III and A. Singh, Eds., vol. 119, 13–18 Jul 2020, p...

    2020

  9. [17]

    Searching for MobileNetV3,

    A. Howard, M. Sandler, G. Chu, L. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V . Vasudevan, Q. V . Le, and H. Adam, “Searching for MobileNetV3,” IEEE/CVF International Conference on Computer Vision, pp. 1314–1324, 2019

    2019

  10. [18]

    CIFAR-10 (Canadian Institute for Advanced Research),

    A. Krizhevsky, V . Nair, and G. Hinton, “CIFAR-10 (Canadian Institute for Advanced Research),” 2009, available at https:// www.cs.toronto.edu/∼kriz/cifar.html

    2009

  11. [19]

    Learning Multiple Layers of Fea- tures from Tiny Images,

    A. Krizhevsky and G. Hinton, “Learning Multiple Layers of Fea- tures from Tiny Images,” Master’s thesis, Department of Computer Science, University of Toronto, 2009

    2009

  12. [20]

    FedAdapt: Adaptive Offloading for IoT Devices in Federated Learning,

    D. Wu, R. Ullah, P . Harvey, P . Kilpatrick, I. Spence, and B. Vargh- ese, “FedAdapt: Adaptive Offloading for IoT Devices in Federated Learning,” IEEE Internet of Things Journal, vol. 9, no. 21, pp. 20 889– 20 901, 2022

    2022

  13. [21]

    Visualizing and Understanding Con- volutional Networks,

    M. Zeiler and R. Fergus, “Visualizing and Understanding Con- volutional Networks,” in European Conference on Computer Vision , 2014, pp. 818–833

    2014

  14. [22]

    How Transferable Are Features in Deep Neural Networks?

    J. Yosinski, J. Clune, Y. Bengio, and L. Hod, “How Transferable Are Features in Deep Neural Networks?” in Proceedings of the 28th International Conference on Neural Information Processing Systems , vol. 2, 2014, pp. 3320–3328

    2014

  15. [23]

    Deep Learning,

    Y. LeCun, Y. Bengio, and G. Hinton, “Deep Learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015. [Online]. Available: https://doi.org/10.1038/nature14539 14

    2015 doi

  16. [24]

    On the Convergence of FedAvg on Non-IID Data,

    X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, “On the Convergence of FedAvg on Non-IID Data,” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=HJxNAnVtDS

    2020

  17. [25]

    Decoupled Greedy Learning of CNNs,

    E. Belilovsky, M. Eickenberg, and E. Oyallon, “Decoupled Greedy Learning of CNNs,” in Proceedings of the 37th International Confer- ence on Machine Learning, 2020

    2020

  18. [26]

    Optimization Methods for Large-Scale Machine Learning,

    L. Bottou, F. E. Curtis, and J. Nocedal, “Optimization Methods for Large-Scale Machine Learning,” SIAM Review, vol. 60, no. 2, pp. 223–311, 2018

    2018

  19. [27]

    Training Neural Networks Using Features Replay,

    Z. Huo, B. Gu, and H. Huang, “Training Neural Networks Using Features Replay,” in Advances in Neural Information Processing Systems, vol. 31. Curran Associates, Inc., 2018

    2018

  20. [28]

    Communication- Efficient Algorithms for Statistical Optimization,

    Y. Zhang, J. C. Duchi, and M. J. Wainwright, “Communication- Efficient Algorithms for Statistical Optimization,” The Journal of Machine Learning Research, vol. 14, no. 1, p. 3321–3363, 2013

    2013

  21. [29]

    Local SGD Converges Fast and Communicates Little,

    S. U. Stich, “Local SGD Converges Fast and Communicates Little,” in International Conference on Learning Representations, 2019

    2019

  22. [30]

    Sparsified SGD with memory,

    S. U. Stich, J.-B. Cordonnier, and M. Jaggi, “Sparsified SGD with memory,” in Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018, p. 4452–4463

    2018

  23. [31]

    Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning,

    H. Yu, S. Yang, and S. Zhu, “Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning,” in Proceedings of the Thirty- Third AAAI Conference on Artificial Intelligence , 2019

    2019

  24. [32]

    A Stochastic Approximation Method,

    H. E. Robbins, “A Stochastic Approximation Method,” Annals of Mathematical Statistics, vol. 22, pp. 400–407, 1951

    1951

  25. [33]

    Scal- ability and Performance Evaluation of Edge Cloud Systems for Latency Constrained Applications,

    S. Maheshwari, D. Raychaudhuri, I. Seskar, and F. Bronzino, “Scal- ability and Performance Evaluation of Edge Cloud Systems for Latency Constrained Applications,” in 2018 IEEE/ACM Symposium on Edge Computing, 2018, pp. 286–299

    2018

  26. [34]

    Very Deep Convolutional Net- works for Large-scale Image Recognition,

    K. Simonyan and A. Zisserman, “Very Deep Convolutional Net- works for Large-scale Image Recognition,” 3rd International Con- ference on Learning Representations, p. 1–14, 2015

    2015

  27. [35]

    Swin Transformer: Hierarchical Vision Transformer using Shifted Windows,

    Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin Transformer: Hierarchical Vision Transformer using Shifted Windows,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 9992–10 002

    2021

  28. [36]

    How to Train Your ViT? Data, Augmentation, and Regularization in Vision Transformers,

    A. P . Steiner, A. Kolesnikov, X. Zhai, R. Wightman, J. Uszkoreit, and L. Beyer, “How to Train Your ViT? Data, Augmentation, and Regularization in Vision Transformers,” Transactions on Machine Learning Research , 2022. [Online]. Available: https: //openreview.net/forum?id=4nPswr1KcP

    2022

  29. [37]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” in International Conference on Learning...

    2021

  30. [38]

    ImageNet Large Scale Visual Recognition Challenge,

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, and L. Fei-Fei, “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision, vol. 115, no. 3, pp. 211–252, 2015

    2015

  31. [39]

    Federated Learning Based on Dynamic Regulariza- tion,

    D. A. E. Acar, Y. Zhao, R. Matas, M. Mattina, P . Whatmough, and V . Saligrama, “Federated Learning Based on Dynamic Regulariza- tion,” in International Conference on Learning Representations, 2021

    2021

  32. [40]

    Early Stopping—But When?

    L. Prechelt, “Early Stopping—But When?” Neural Networks: Tricks of the Trade, vol. 7700, pp. 53–67, 2012

    2012

  33. [41]

    Accelerating Federated Learning with Split Learning on Locally Generated Losses,

    D.-J. Han, H. I. Bhatti, J. Lee, and J. Moon, “Accelerating Federated Learning with Split Learning on Locally Generated Losses,” in International Conference on Machine Learning Workshop on Federated Learning for User Privacy and Data Confidentiality , 2021

    2021

  34. [42]

    ParallelSFL: A Novel Split Federated Learning Framework Tackling Hetero- geneity Issues,

    Y. Liao, Y. Xu, H. Xu, Z. Yao, L. Huang, and C. Qiao, “ParallelSFL: A Novel Split Federated Learning Framework Tackling Hetero- geneity Issues,” in Proceedings of the 30th Annual International Conference on Mobile Computing and Networking , 2024, p. 845–860

    2024

  35. [43]

    Hermes: An Ef- ficient Federated Learning Framework for Heterogeneous Mobile Clients,

    A. Li, J. Sun, P . Li, Y. Pu, H. Li, and Y. Chen, “Hermes: An Ef- ficient Federated Learning Framework for Heterogeneous Mobile Clients,” in Proceedings of the 27th Annual International Conference on Mobile Computing and Networking, 2021, p. 420–437

    2021

  36. [44]

    Billion-Scale Federated Learning on Mobile Clients: a Submodel Design with Tunable Privacy,

    C. Niu, F. Wu, S. Tang, L. Hua, R. Jia, C. Lv, Z. Wu, and G. Chen, “Billion-Scale Federated Learning on Mobile Clients: a Submodel Design with Tunable Privacy,” in Proceedings of the 26th Annual International Conference on Mobile Computing and Networking , 2020

    2020

  37. [45]

    FedHD: Federated Learning with Hyperdimensional Computing,

    Q. Zhao, K. Lee, J. Liu, M. Huzaifa, X. Yu, and T. Rosing, “FedHD: Federated Learning with Hyperdimensional Computing,” in Pro- ceedings of the 28th Annual International Conference on Mobile Com- puting And Networking, 2022, p. 791–793

    2022

  38. [46]

    Understanding Deep Image Representations by Inverting Them,

    A. Mahendran and A. Vedaldi, “Understanding Deep Image Representations by Inverting Them,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 5188– 5196

    2015

  39. [47]

    Deep Learning with Differential Pri- vacy,

    M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep Learning with Differential Pri- vacy,” in Proceedings of the 2016 ACM SIGSAC Conference on Com- puter and Communications Security, 2016, p. 308–318

    2016

  40. [48]

    Homomorphic Encryption for Arithmetic of Approximate Numbers,

    J. H. Cheon, A. Kim, M. Kim, and Y. Song, “Homomorphic Encryption for Arithmetic of Approximate Numbers,” Advances in Cryptology - ASIACRYPT, pp. 409–437, 2017

    2017

  41. [49]

    Certified Robustness to Adversarial Examples with Differential Privacy,

    M. Lecuyer, V . Atlidakis, R. Geambasu, D. Hsu, and S. Jana, “Certified Robustness to Adversarial Examples with Differential Privacy,” in 2019 IEEE Symposium on Security and Privacy (SP) , 2019, pp. 656–672. Zihan Zhang is a PhD student at University of St Andrews. He was prev...

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.