pith. sign in

arxiv: 2606.24722 · v1 · pith:ZXJQE63Hnew · submitted 2026-06-23 · 💻 cs.AI

Decentralised AI Training and Inference with BlockTrain

Peter Toth This is my paper

Pith reviewed 2026-06-25 23:30 UTC · model grok-4.3

classification 💻 cs.AI
keywords decentralized trainingblock partitioninglocal objectivesmodel compositiontransformerWikiTextcross entropyWAN inference
0
0 comments X

The pith

A model can be split into blocks trained independently on local objectives derived from the global target and composed at inference to reach near end-to-end performance.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows how to partition a transformer into blocks where each is optimized separately using an objective pulled from the overall training loss. Workers handle only their assigned block and never hold the full model's optimizer state. On byte-level WikiText the assembled model reaches 1.359 cross-entropy, within 0.04 of a matched end-to-end reference. Real checkpoints move over HTTP and TCP, including a three-host public-IP run that improves from 5.580 to 1.811 cross-entropy while transferring 15 GB. Inference traverses the block stack once per output sequence and serves a 75-billion-parameter logical model across WAN hosts.

Core claim

BlockTrain partitions a model into independently trainable blocks, each optimized on a local objective derived from the same global target. These blocks compose at inference into a single model. On byte-level WikiText the method reaches cross-entropy 1.359 (perplexity 3.89), within about 0.04 of a same-setup end-to-end Transformer reference, while each active worker trains only one block and avoids full-model optimizer state. Shared multi-worker runs and direct TCP transport experiments confirm the approach works with real serialized updates across hosts.

What carries the argument

Model partitioning into blocks with local objectives derived from the global target, enabling independent training followed by composition at inference.

If this is right

  • Each worker stores and updates optimizer state for only one block instead of the full model.
  • Averaging same-block updates across workers produces a single assembled model without central coordination during training.
  • Inference emits a full output sequence per block-stack traversal rather than one token per traversal over the network.
  • The same block-stack path supports a logical 75.8-billion-parameter fp16 model served across three public-network GPU hosts.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Independent groups could each train and contribute one block without needing shared accelerator clusters or synchronized full-model gradients.
  • The approach opens a path to training runs whose total memory footprint scales with block size rather than total parameter count.
  • WAN inference that moves full sequences per traversal could change how large models are served when hosts sit in different data centers.

Load-bearing premise

Local objectives derived from the global target let independently trained blocks compose at inference into a model whose performance stays close to end-to-end training without further global coordination.

What would settle it

Measure cross-entropy of the composed model after each block is trained only on its local objective; if the gap to the end-to-end reference exceeds roughly 0.1 on the same WikiText split, the central claim does not hold.

read the original abstract

Frontier AI training is increasingly shaped by access to dense, centrally controlled accelerator clusters. This creates a structural advantage for hyperscalers and large centralized laboratories, and makes open or independent AI efforts depend on scarce capital, privileged infrastructure, and data-center geography. We present Spheroid BlockTrain, a decentralized training protocol in which a model is partitioned into independently trainable blocks, each optimized on a local objective derived from the same global target and composed at inference into one model. On byte-level WikiText, BlockTrain reaches cross entropy 1.359 (perplexity 3.89), within about 0.04 CE of a same-setup end-to-end Transformer reference, while each active worker trains only one block and avoids full-model optimizer state. A shared six-worker block training run reaches CE 1.385 by averaging same-block updates into one assembled model. HTTP/TCP transport experiments move real serialized checkpoints and updates, including a public-IP three-host run that improves CE from 5.580 to 1.811 while moving 15.22 GB. For inference, the current BlockTrain path uses one block-stack traversal per full output and serves over direct TCP across three public-network GPU hosts up to a 75.80B-parameter logical fp16 shape, outperforming a matched plain-autoregressive TCP pipeline baseline because it emits a full sequence per WAN pipeline traversal rather than one token per traversal.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper proposes Spheroid BlockTrain, a decentralized protocol that partitions a Transformer into independently trainable blocks, each optimized on a local objective derived from the global cross-entropy target. Blocks are composed at inference without full-model optimizer state. On byte-level WikiText the method reports CE 1.359 (perplexity 3.89), within 0.04 of a same-setup end-to-end reference; a six-worker averaged run reaches 1.385. Real HTTP/TCP experiments move serialized checkpoints across public-IP hosts and demonstrate inference serving of up to 75.80 B-parameter logical models.

Significance. If the local-objective derivation indeed permits independent block training whose outputs compose without further global correction, the approach would materially reduce the infrastructure barrier for open AI training by eliminating full-model optimizer state and enabling heterogeneous, geographically distributed workers. The reported network-scale experiments (public-IP three-host training and 75 B-parameter inference) constitute a concrete existence proof of the transport layer.

major comments (3)
  1. [Abstract, §3] Abstract and §3 (local-objective derivation): the central performance claim (CE 1.359 vs. reference within 0.04) rests on the assertion that local objectives derived from the global target allow independently trained blocks to compose at inference. No explicit formulation of the local objective, no derivation showing that the mapping preserves independence, and no ablation isolating the derivation step are supplied; without these the reported numbers cannot be verified to demonstrate true decentralization rather than implicit coordination.
  2. [§4] §4 (experimental results): the WikiText numbers are presented without error bars, multiple random seeds, or statistical detail, and no ablation on block size, number of blocks, or composition method is reported. These omissions make it impossible to assess whether the 0.04 CE gap is robust or an artifact of a single run.
  3. [§5] §5 (inference path): the claim that the block-stack traversal outperforms a matched autoregressive TCP baseline because it emits a full sequence per WAN traversal is load-bearing for the inference contribution, yet no latency or bandwidth breakdown isolating the traversal cost versus the baseline is provided.
minor comments (2)
  1. [§3] Notation for the local objective and the composition operator should be introduced with explicit symbols rather than prose descriptions.
  2. [§4] The six-worker averaging experiment should state whether the averaged updates are applied before or after the local-objective step.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments on the clarity of the local-objective derivation, the statistical presentation of results, and the inference analysis. We address each major comment below and commit to revisions where the manuscript requires additional detail or evidence.

read point-by-point responses

  1. Referee: [Abstract, §3] Abstract and §3 (local-objective derivation): the central performance claim (CE 1.359 vs. reference within 0.04) rests on the assertion that local objectives derived from the global target allow independently trained blocks to compose at inference. No explicit formulation of the local objective, no derivation showing that the mapping preserves independence, and no ablation isolating the derivation step are supplied; without these the reported numbers cannot be verified to demonstrate true decentralization rather than implicit coordination.

    Authors: We agree that an explicit formulation and derivation would strengthen verifiability. In the revised manuscript we will expand §3 with the precise mathematical definition of the local objective (derived from the global cross-entropy via block-wise conditional factorization), a short proof sketch establishing that block independence is preserved under composition, and an ablation that isolates the effect of the derivation step versus a naïve per-block objective. These additions will directly address the concern that the reported performance may reflect implicit coordination. revision: yes

  2. Referee: [§4] §4 (experimental results): the WikiText numbers are presented without error bars, multiple random seeds, or statistical detail, and no ablation on block size, number of blocks, or composition method is reported. These omissions make it impossible to assess whether the 0.04 CE gap is robust or an artifact of a single run.

    Authors: The current manuscript reports only a single main run and one averaged six-worker configuration. We will revise §4 to include error bars computed over at least three independent random seeds for the primary CE 1.359 result, plus an ablation table varying block size and number of blocks while holding total parameter count fixed. The composition method (simple averaging of same-block updates) will be described with pseudocode. These changes will allow readers to evaluate robustness. revision: yes

  3. Referee: [§5] §5 (inference path): the claim that the block-stack traversal outperforms a matched autoregressive TCP baseline because it emits a full sequence per WAN traversal is load-bearing for the inference contribution, yet no latency or bandwidth breakdown isolating the traversal cost versus the baseline is provided.

    Authors: We acknowledge the absence of a granular breakdown. In the revision we will augment §5 with measured latency and bandwidth traces from the three-host public-IP experiments, explicitly separating (a) the cost of one full block-stack traversal versus (b) the per-token autoregressive baseline. The comparison will use the same 75.80 B logical model and identical network conditions to isolate the claimed advantage. revision: yes

Circularity Check

0 steps flagged

No significant circularity; experimental results are self-contained

full rationale

The paper reports direct experimental outcomes on byte-level WikiText (CE 1.359 vs. reference within 0.04) using partitioned blocks trained on local objectives derived from a global target. No equations, fitted parameters renamed as predictions, or self-citation chains appear in the provided text to reduce the performance claims to definitions or prior inputs. The central claim rests on empirical measurements of composition at inference rather than a mathematical derivation that collapses by construction. This is the most common honest finding for papers whose results are framed as measured benchmarks against external references.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; the local-objective derivation is mentioned but not formalized.

pith-pipeline@v0.9.1-grok · 5773 in / 1159 out tokens · 18106 ms · 2026-06-25T23:30:27.740602+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

126 extracted references · 18 canonical work pages · 2 internal anchors

  1. [1]

    International Conference on Learning Representations , year =

    DiffusionBlocks: Block-wise Neural Network Training via Diffusion Interpretation , author =. International Conference on Learning Representations , year =

  2. [2]

    Proceedings of the International Conference on Artificial Intelligence and Statistics , year =

    Communication-Efficient Learning of Deep Networks from Decentralized Data , author =. Proceedings of the International Conference on Artificial Intelligence and Statistics , year =

  3. [3]

    , journal =

    Stich, Sebastian U. , journal =. Local

  4. [4]

    Douillard, Arthur and others , journal =

  5. [5]

    Jaghouar, Sami and Ong, Isaac and Hagemann, Johannes , journal =

  6. [6]

    Jaghouar, Sami and Ong, Isaac and Basra, Haziq and Obeid, Zaid and Straube, Jonas and Keiblinger, Maximilian and Bakouch, Elie and Atkins, Samuel and Panahi, Ali and Goddard, Eric and Ryabinin, Max and Hagemann, Johannes , journal =

  7. [7]

    and Suresh, Ananda Theertha , booktitle =

    Karimireddy, Sai Praneeth and Kale, Satyen and Mohri, Mehryar and Reddi, Sashank and Stich, Sebastian U. and Suresh, Ananda Theertha , booktitle =

  8. [8]

    Proceedings of Machine Learning and Systems , year =

    Federated Optimization in Heterogeneous Networks , author =. Proceedings of Machine Learning and Systems , year =

  9. [9]

    Advances in Neural Information Processing Systems , year =

    Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent , author =. Advances in Neural Information Processing Systems , year =

  10. [10]

    , journal =

    Koloskova, Anastasia and Loizou, Nicolas and Boreiri, Shayan and Jaggi, Martin and Stich, Sebastian U. , journal =. A Unified Theory of Decentralized

  11. [11]

    International Conference on Machine Learning , year =

    Stochastic Gradient Push for Distributed Deep Learning , author =. International Conference on Machine Learning , year =

  12. [12]

    International Conference on Machine Learning , year =

    Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication , author =. International Conference on Machine Learning , year =

  13. [13]

    Advances in Neural Information Processing Systems , year =

    Hogwild!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent , author =. Advances in Neural Information Processing Systems , year =

  14. [15]

    Advances in Neural Information Processing Systems , volume =

    Towards Crowdsourced Training of Large Neural Networks using Decentralized Mixture-of-Experts , author =. Advances in Neural Information Processing Systems , volume =. 2020 , url =

    2020

  15. [16]

    Ryabinin, Max and Borzunov, Alexander and Diskin, Michael and Gusev, Anton and Mazur, Denis and Plokhotnyuk, Vsevolod and Bukhtiyarov, Alexey and Samygin, Pavel and Sinitsin, Anton and Chumachenko, Artem , year =

  16. [17]

    Ryabinin, Max and Gorbunov, Eduard and Plokhotnyuk, Vsevolod and Pekhimenko, Gennady , booktitle =. Moshpit

  17. [18]

    2023 , url =

    Ryabinin, Max and Dettmers, Tim and Diskin, Michael and Borzunov, Alexander , booktitle =. 2023 , url =

    2023

  18. [19]

    2024 , eprint =

    Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices , author =. 2024 , eprint =

    2024

  19. [20]

    2022 , eprint =

    Decentralized Training of Foundation Models in Heterogeneous Environments , author =. 2022 , eprint =

    2022

  20. [21]

    Advances in Neural Information Processing Systems , year =

    Distributed Deep Learning in Open Collaborations , author =. Advances in Neural Information Processing Systems , year =

  21. [22]

    Proceedings of the Workshop on Challenges and Perspectives in Creating Large Language Models , year =

    Training Transformers Together , author =. Proceedings of the Workshop on Challenges and Perspectives in Creating Large Language Models , year =

  22. [23]

    Nature , volume =

    Swarm Learning for Decentralized and Confidential Clinical Machine Learning , author =. Nature , volume =

  23. [24]

    NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning , year =

    Flower: A Friendly Federated Learning Framework , author =. NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning , year =

  24. [25]

    He, Chaoyang and Li, Songze and So, Jinhyun and Zeng, Xiao and Zhang, Mi and Wang, Hongyi and Wang, Xiaoyang and Vepakomma, Praneeth and Singh, Abhishek and Qiu, Hang and others , journal =

  25. [26]

    15th USENIX Symposium on Operating Systems Design and Implementation , pages =

    Oort: Efficient Federated Learning via Guided Participant Selection , author =. 15th USENIX Symposium on Operating Systems Design and Implementation , pages =

  26. [27]

    and Chowdhury, Mosharaf , booktitle =

    Lai, Fan and Dai, Yinwei and Zhu, Xiangfeng and Madhyastha, Harsha V. and Chowdhury, Mosharaf , booktitle =

  27. [28]

    International Conference on Learning Representations , year =

    Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training , author =. International Conference on Learning Representations , year =

  28. [29]

    Alistarh, Dan and Grubic, Demjan and Li, Jerry and Tomioka, Ryota and Vojnovic, Milan , booktitle =

  29. [30]

    Vogels, Thijs and Karimireddy, Sai Praneeth and Jaggi, Martin , booktitle =

  30. [31]

    International Conference on Machine Learning , pages =

    Error Feedback Fixes SignSGD and Other Gradient Compression Schemes , author =. International Conference on Machine Learning , pages =

  31. [32]

    Advances in Neural Information Processing Systems , year =

    Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent , author =. Advances in Neural Information Processing Systems , year =

  32. [33]

    International Conference on Machine Learning , year =

    Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates , author =. International Conference on Machine Learning , year =

  33. [34]

    International Conference on Machine Learning , year =

    Decoupled Neural Interfaces using Synthetic Gradients , author =. International Conference on Machine Learning , year =

  34. [35]

    Joint European Conference on Machine Learning and Knowledge Discovery in Databases , year =

    Difference Target Propagation , author =. Joint European Conference on Machine Learning and Knowledge Discovery in Databases , year =

  35. [37]

    Advances in Neural Information Processing Systems , year =

    Greedy Layer-Wise Training of Deep Networks , author =. Advances in Neural Information Processing Systems , year =

  36. [38]

    Artificial Intelligence and Statistics , year =

    Deeply-Supervised Nets , author =. Artificial Intelligence and Statistics , year =

  37. [39]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume =

    Gated Linear Networks , author =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2021 , doi =

    2021

  38. [40]

    2026 , howpublished =

    Agora: A Decentralized Pipeline-Parallel Training System , author =. 2026 , howpublished =

    2026

  39. [41]

    2025 , howpublished =

    2025

  40. [43]

    2022 , eprint =

    Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models , author =. 2022 , eprint =

    2022

  41. [45]

    , journal =

    Peng, Bowen and Quesnelle, Jeffrey and Kingma, Diederik P. , journal =. 2024 , url =

    2024

  42. [46]

    2024 , howpublished =

    2024

  43. [47]

    2025 , howpublished =

    Democratizing. 2025 , howpublished =

    2025

  44. [48]

    2022 , url =

    Dong, Huayi and others , journal =. 2022 , url =

    2022

  45. [49]

    2021 , howpublished =

    Bittensor: A Peer-to-Peer Intelligence Market , author =. 2021 , howpublished =

    2021

  46. [50]

    2020 , howpublished =

    Akash Network: Decentralized Cloud Infrastructure Marketplace , author =. 2020 , howpublished =

    2020

  47. [51]

    2024 , howpublished =

    Aethir Whitepaper , author =. 2024 , howpublished =

    2024

  48. [52]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

    Decentralized Diffusion Models , author =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages =

  49. [53]

    Greedy Layerwise Learning Can Scale to

    Belilovsky, Eugene and Eickenberg, Michael and Oyallon, Edouard , journal =. Greedy Layerwise Learning Can Scale to

  50. [55]

    Journal of Machine Learning Research , volume =

    Estimation of Non-Normalized Statistical Models by Score Matching , author =. Journal of Machine Learning Research , volume =

  51. [56]

    Neural Computation , volume =

    A Connection Between Score Matching and Denoising Autoencoders , author =. Neural Computation , volume =

  52. [57]

    Advances in Neural Information Processing Systems , year =

    Denoising Diffusion Probabilistic Models , author =. Advances in Neural Information Processing Systems , year =

  53. [58]

    International Conference on Learning Representations , year =

    Score-Based Generative Modeling through Stochastic Differential Equations , author =. International Conference on Learning Representations , year =

  54. [59]

    Advances in Neural Information Processing Systems , year =

    Elucidating the Design Space of Diffusion-Based Generative Models , author =. Advances in Neural Information Processing Systems , year =

  55. [60]

    2025 , howpublished =

    Artificial Intelligence Index Report 2025 , author =. 2025 , howpublished =

    2025

  56. [61]

    2025 , howpublished =

    Competition in Artificial Intelligence Infrastructure , author =. 2025 , howpublished =

    2025

  57. [62]

    2024 , howpublished =

    Joint Statement on Competition in Generative. 2024 , howpublished =

    2024

  58. [64]

    2024 , howpublished =

    Dual-Use Foundation Models with Widely Available Model Weights , author =. 2024 , howpublished =

    2024

  59. [65]

    Institute for New Economic Thinking Working Paper , year =

    Concentrating Intelligence: Scaling and Market Structure in Artificial Intelligence , author =. Institute for New Economic Thinking Working Paper , year =

  60. [66]

    Aethir whitepaper

    Aethir . Aethir whitepaper. https://docs.aethir.com/whitepaper, 2024

    2024

  61. [67]

    Akash network: Decentralized cloud infrastructure marketplace

    Akash Network . Akash network: Decentralized cloud infrastructure marketplace. https://akash.network/, 2020

    2020

  62. [68]

    QSGD : Communication-efficient SGD via gradient quantization and encoding

    Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. QSGD : Communication-efficient SGD via gradient quantization and encoding. In Advances in Neural Information Processing Systems, 2017

    2017

  63. [69]

    Stochastic gradient push for distributed deep learning

    Mahmoud Assran, Nicolas Loizou, Nicolas Ballas, and Michael Rabbat. Stochastic gradient push for distributed deep learning. In International Conference on Machine Learning, 2019

    2019

  64. [70]

    Greedy layerwise learning can scale to ImageNet

    Eugene Belilovsky, Michael Eickenberg, and Edouard Oyallon. Greedy layerwise learning can scale to ImageNet . International Conference on Machine Learning, 2019

    2019

  65. [71]

    How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target Propagation

    Yoshua Bengio. How auto-encoders could provide credit assignment in deep networks via target propagation. arXiv preprint arXiv:1407.7906, 2014

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  66. [72]

    Greedy layer-wise training of deep networks

    Yoshua Bengio, Pascal Lamblin, Dan Popovici, and Hugo Larochelle. Greedy layer-wise training of deep networks. In Advances in Neural Information Processing Systems, 2007

    2007

  67. [73]

    Flower: A Friendly Federated Learning Research Framework

    Daniel J. Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Titouan Parcollet, and Nicholas D. Lane. Flower: A friendly federated learning framework. In NeurIPS Workshop on Scalability, Privacy, and Security in Federated Learning, 2020. URL https://arxiv.org/abs/2007.14390

    work page internal anchor Pith review Pith/arXiv arXiv 2020

  68. [74]

    Training transformers together

    Alexander Borzunov, Max Ryabinin, Tim Dettmers, Quentin Lhoest, Lucile Saulnier, Michael Diskin, Yacine Jernite, and Thomas Wolf. Training transformers together. In Proceedings of the Workshop on Challenges and Perspectives in Creating Large Language Models, 2022

    2022

  69. [75]

    Petals: Collaborative inference and fine-tuning of large models

    Alexander Borzunov, Dmitry Baranchuk, Tim Dettmers, Max Ryabinin, Younes Belkada, Artem Chumachenko, Pavel Samygin, and Colin Raffel. Petals: Collaborative inference and fine-tuning of large models. arXiv preprint arXiv:2209.01188, 2023

    work page arXiv 2023

  70. [76]

    Distributed deep learning in open collaborations

    Michael Diskin, Alexey Bukhtiyarov, Max Ryabinin, Lucile Saulnier, Quentin Lhoest, Anton Sinitsin, Dmitry Popov, Dmitry Pyrkin, Maxim Kashirin, Alexander Borzunov, et al. Distributed deep learning in open collaborations. In Advances in Neural Information Processing Systems, 2021

    2021

  71. [77]

    FLock : Defending malicious behaviors in federated learning with blockchain

    Huayi Dong et al. FLock : Defending malicious behaviors in federated learning with blockchain. arXiv preprint arXiv:2211.04344, 2022. URL https://arxiv.org/abs/2211.04344

    work page arXiv 2022

  72. [78]

    Rusu, et al

    Arthur Douillard, Qixuan Feng, Andrei A. Rusu, et al. DiPaCo : Distributed path composition, 2024. URL https://arxiv.org/abs/2403.10616

    work page arXiv 2024

  73. [79]

    DiLoCo : Distributed low-communication training of language models

    Arthur Douillard et al. DiLoCo : Distributed low-communication training of language models. arXiv preprint, 2023

    2023

  74. [80]

    Joint statement on competition in generative AI foundation models and AI products

    European Commission , UK Competition and Markets Authority , US Department of Justice , and US Federal Trade Commission . Joint statement on competition in generative AI foundation models and AI products. https://competition-policy.ec.europa.eu/, 2024

    2024

  75. [81]

    FLock : Federated machine learning on blockchain

    FLock.io . FLock : Federated machine learning on blockchain. https://www.flock.io/whitepaper, 2024

    2024

  76. [82]

    FusionAI : Decentralized training and deploying LLMs with massive consumer-level GPUs , 2023

    FusionAI Authors . FusionAI : Decentralized training and deploying LLMs with massive consumer-level GPUs , 2023. URL https://arxiv.org/abs/2309.01172

    work page arXiv 2023

  77. [83]

    FusionLLM : A decentralized LLM training system on geo-distributed GPUs with adaptive compression, 2024

    FusionLLM Authors . FusionLLM : A decentralized LLM training system on geo-distributed GPUs with adaptive compression, 2024. URL https://arxiv.org/abs/2410.12707

    work page arXiv 2024

  78. [84]

    RL swarm: A framework for collaborative reinforcement learning

    Gensyn . RL swarm: A framework for collaborative reinforcement learning. https://blog.gensyn.ai/rl-swarm-a-framework-for-collaborative-rl/, 2025

    2025

  79. [85]

    FedML : A research library and benchmark for federated machine learning

    Chaoyang He, Songze Li, Jinhyun So, Xiao Zeng, Mi Zhang, Hongyi Wang, Xiaoyang Wang, Praneeth Vepakomma, Abhishek Singh, Hang Qiu, et al. FedML : A research library and benchmark for federated machine learning. arXiv preprint arXiv:2007.13518, 2020

    work page arXiv 2007

  80. [86]

    The forward-forward algorithm: Some preliminary investigations

    Geoffrey Hinton. The forward-forward algorithm: Some preliminary investigations. arXiv preprint arXiv:2212.13345, 2022

    work page arXiv 2022

Showing first 80 references.