Pith. sign in

REVIEW 3 major objections 5 minor 20 references

TIFeD: a Tiny Integer-based Federated learning algorithm with Direct feedback alignment

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Pith's one-line read Integer-only federated learning comes to microcontrollers

desk verdict TIFeD is a credible integer-only federated DFA training loop for MCU-class devices, but the single-layer variant's halved-resource claim doesn't survive contact with the algorithm. read the letter →

arxiv 2411.16442 v1 pith:BB6SLRFA submitted 2024-11-25 cs.LG cs.AI

classification cs.LGcs.AI
keywords TinyMachineLearningFederatedDirectFeedbackAlignmentInteger-onlyarithmeticOn-devicetrainingMicrocontrollersTransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces TIFeD, a federated learning algorithm built entirely on integer arithmetic, so that neural-network training can run on microcontrollers with no floating-point unit. It combines Direct Feedback Alignment (DFA) with Federated Averaging, and adds a single-layer mode in which each device trains only one hidden layer of the shared model. The experiments show that both modes can train a fully-connected network from scratch on MNIST and FashionMNIST and fine-tune CNN classifiers via transfer learning on FashionMNIST and CIFAR10, while using 16-bit integers. The claimed payoff is that tiny IoT devices can participate in collaborative learning without sending data to the cloud.

What carries the argument

The load-bearing mechanism is Direct Feedback Alignment (DFA). Unlike backpropagation, DFA sends the output error directly to each hidden layer through a fixed random feedback matrix $B_h$, making the layer update $\delta^h_{DFA} = L' \odot B_h \odot act'_h(a_h)$ non-recursive. This avoids the backpropagation chain that would require floating point or risk overflow, and it lets each layer be trained independently. TIFeD wraps this in a federated loop: the server initializes $B_h$, distributes the current global model, each client computes DFA updates on its local mini-batches, and the server averages the returned model parameters. The single-layer variant assigns each client to one layer $h$ via random client grouping, so a device only computes and transmits updates for that layer.

What would settle it

Train the CNN-1 classifier with TIFeD under a non-IID partition, such as each client receiving images from only one or two classes or a Dirichlet distribution over classes, using the paper's hyperparameters; if test accuracy falls to near chance or well below the IID result, the claimed effectiveness in realistic federated settings is not established. Alternatively, port the integer-only implementation to a real MCU such as the Arduino Nano 33 BLE Sense and check whether 16-bit arithmetic and memory usage fit within 256 kB SRAM while training the reported models.

Watch

Extended reading notes

Core claim

TIFeD's central claim is that federated learning can be made tiny: by replacing backpropagation with Direct Feedback Alignment, whose per-layer error signals are independent of the forward path, and by keeping all training computations in integer arithmetic, each client can update its share of the network using only simple operations. The full-network variant trains all hidden layers on every device; the single-layer variant splits clients into H groups, each responsible for one layer, and the server averages layer-by-layer. The empirical results report that both variants train a two-layer fully-connected network from scratch, adapt CNN classifiers through transfer learning, and scale with the number of clients similarly to floating-point FedAvg. The paper's contribution is thus a feasibility demonstration: an FL algorithm whose local training loop is integer-only and therefore deployable on MCU-class hardware.

Load-bearing premise

The experiments assume each client's local dataset is uniformly sampled from the training set, so every device sees the same class distribution; the paper reports no results for the non-IID data that real federated deployments typically have, and the single-layer variant's per-layer averaging could degrade when clients hold different classes.

Editorial extensions

If this is right

  • Local training on tiny devices becomes possible without a floating-point unit, expanding TinyML from inference-only to on-device learning.
  • Communication cost per round drops in single-layer mode because each client sends only one layer's parameters, saving energy.
  • The integer-only design can reduce memory footprint and allows cheaper microcontrollers to train models.
  • DFA's per-layer independence opens a new way to distribute learning, with different devices responsible for different parts of the network.
  • If accuracy scales with client count like FedAvg, TIFeD could be applied in large federated deployments with highly constrained nodes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The single-layer mode effectively factorizes the global model across clients, a form of model-parallel federated learning; future work could assign larger layers to more powerful devices.
  • The fixed random feedback matrices $B_h$ could be regenerated deterministically from a seed shared with the server, eliminating the need to transmit them and further reducing communication.
  • Non-IID robustness might be improved by server-side weighting that accounts for class imbalance, but the paper does not address this regime.
  • A direct comparison of TIFeD against FedAvg on the same MCU hardware would quantify the real energy and memory gains, since the paper only compares scaling behavior.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces TIFeD, a federated learning algorithm that combines Direct Feedback Alignment (DFA) with integer-only arithmetic, targeting microcontrollers and other resource-constrained devices. TIFeD is presented in two variants: a full-network mode, where each client updates all hidden layers, and a single-layer mode, where groups of clients each update only one layer of the shared model. The authors evaluate both variants on MNIST, FashionMNIST, and CIFAR10 using fully-connected and transfer-learned convolutional classifiers, reporting accuracy curves for 100 initializations of the DFA feedback matrices and a study of the effect of the aggregation point.

Significance. If the claims are properly supported, the paper makes a useful feasibility contribution to TinyML: it demonstrates that DFA-based federated training can be carried out with 16-bit integer arithmetic and that partial-layer distributed training can still converge on simple benchmarks. The public code repository and the use of 100 random feedback-matrix initializations are strengths. However, the significance is limited by three issues: the single-layer variant's resource-saving claims are not justified by the algorithm as written, the evaluation uses only IID data partitions, and the experiments lack accuracy baselines and variance reporting. The central feasibility observation is plausible, but the paper's broader claims about 'effectiveness' and reduced memory/computation go beyond what the current evidence supports.

major comments (3)
  1. [§4.3, §5.2] The claim that the single-layer implementation halves the computational, memory, and energy demands of the nodes is not supported by the algorithm or the DFA equations. In Eq. (2), δ_h^DFA = L' B_h ⊙ act'_h(a_h) requires the loss derivative L' and the pre-activation a_h; computing a_h requires a forward pass through layers 1,...,h-1, and computing L' requires a forward pass through the entire network up to the output. Since Algorithm 2 passes the full global model θ to each NodeUpdate and each client must store all H layers, model memory and forward-pass computation are identical in the full-network and single-layer modes. Only the update computation for the selected layer and the uplink payload are reduced. The statement in Section 5.2 that 'the computational, memory, and energy demands of the nodes are halved' is therefore incorrect as written, and the conclusion that the single-layer variant is 'more suitable when devices are extremely constrained' needs to be supported by a concrete per-device resource model or by direct measurements.
  2. [§5.1, §5.2-§5.5] All experiments assume an IID data partition: Section 5 states that each local dataset X_m is uniformly sampled from the training set X. No non-IID partition is evaluated, and no argument is given that the aggregation behavior of TIFeD—especially the single-layer variant, where different groups of clients update different layers—remains stable when clients hold different class distributions. Since non-IID data is a defining condition of federated learning, the paper's 'effectiveness of the proposed solution' claim is currently restricted to the IID setting. Please add non-IID experiments or explicitly scope the claims to IID partitions.
  3. [§5.2-§5.5] The experimental evaluation does not include any accuracy baseline. FedAvg is used in Section 5.4 only as a scaling reference and is explicitly not compared on final accuracy. Without a reference (FedAvg, DFA on centralized data, or standard SGD) on the same datasets and architectures, the reported absolute accuracies of roughly 0.89-0.95 on MNIST and 0.81-0.86 on FashionMNIST cannot establish 'effectiveness.' Moreover, every figure reports results for 100 random initializations of the feedback matrices, but no variance, standard deviation, or confidence interval is shown, so it is impossible to assess whether the differences between single-layer and full-network modes are systematic. Please add baselines and report the distribution across the 100 seeds.
minor comments (5)
  1. [§4.2, Algorithm 1] The pseudocode does not show how the integer-only arithmetic is realized; the learning-rate inverse 1/η suggests fixed-point scaling, but the quantization, scaling, and overflow-avoidance equations are not given. Please state explicitly that the implementation follows PocketNN [14] and provide the corresponding integer update equations, or add them to the paper.
  2. [§5.5, Table 3] The meaning of the 'Epochs e' aggregation point is unclear: with e=10, buff_len=20, and b_s=10, the reported 5 communication rounds is surprising and needs a precise definition of when the server aggregates updates relative to local epochs and buffers.
  3. [Figures 2-5] Please add error bars or shaded intervals representing the spread over the 100 seeds; the current line plots do not indicate whether the reported values are means, medians, or best runs.
  4. [§5.3] Please specify the quantization format and bit-width of the pretrained convolutional feature extractor, since the 'integer-only' claim applies to the entire training pipeline and the feature extractor is only mentioned as 'quantized' without details.
  5. [Throughout] There are several language and typographical issues, including 'in the followings' (Section 4.1), the repeated plural 'initialization' in figure captions, and inconsistent use of 'the node' versus 'the central server.' A careful language pass is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TIFeD combines external DFA/FedAvg/PocketNN results and is evaluated on public benchmarks; no prediction reduces to its own inputs.

full rationale

The derivation chain is not circular. TIFeD is built from external, independently established components: FedAvg (McMahan et al. [11]), DFA (Nøkland [12]), and the integer-only DFA/PocketNN activations (Song & Lin [14]); none is authored by the present team, and none is defined in terms of TIFeD's outputs. The experiments use public benchmarks and report accuracies over random feedback initializations; no fitted parameter is relabeled as a prediction, and no uniqueness theorem or self-citation is invoked to forbid alternative designs. Under the reviewing rule, I explicitly flag one non-circular support gap: Section 5.2's claim that single-layer TIFeD halves 'computational, memory, and energy demands of the nodes' is not supported by Eq. (2)-(3), because computing L' and a_h still requires the full forward pass and thus the full model on every client; this is a correctness/over-claim issue, not a circular reduction. Similarly, the integer-only arithmetic is imported from PocketNN rather than re-derived, but since PocketNN is external and code-released, this is legitimate prior-art reliance rather than a self-referential chain. Score 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced. The central contribution is algorithmic, and its outcomes depend on several hand-chosen hyperparameters and imported assumptions about DFA, integer quantization, and IID client data.

free parameters (6)
  • learning rate inverse 1/eta = 2048, 4096, 8192, 16384
    Hand-chosen per experiment; the integer update magnitude depends directly on this value, and no learning-rate schedule or sensitivity analysis is provided.
  • buffer length buff_len = 20 or 50 images
    Chosen to fit device memory; it directly controls the number of communication rounds and the effective local training set per round.
  • mini-batch size b_s = 10 or 25
    Hand-chosen; affects update noise, memory use, and the granularity of the aggregation-point comparison.
  • local epochs e = 5 or 10
    Hand-chosen; controls how many local passes each client makes before sending updates.
  • number of worker nodes M = 8 or 128
    Experimental variable; the paper studies behavior as M increases, but the choice of M affects both accuracy and communication costs.
  • local dataset size X_m = 100, 6250, or 7500 images
    Chosen to fit the memory constraints of the simulated tiny device; it trades between statistical strength and feasibility.
assumptions (4)
  • domain assumption The DFA update rule (Eq. 2-3) and the feedback-alignment condition (Eq. 1) remain valid when all values are quantized to integers.
    Invoked in Sections 3.1 and 4.2; the paper relies on DFA literature and PocketNN for this, without a quantization-aware proof or analysis of overflow bounds.
  • domain assumption Client local datasets are IID uniform samples from the global training set.
    Section 5 states X_m is uniformly sampled from training set X; no non-IID experiments are run, so the empirical claims hold only under this assumption.
  • domain assumption Training can start from zero-valued weights when using DFA.
    Section 5.2 initializes the shared model with zero-valued weights; this relies on DFA convergence properties cited from Refinetti et al. rather than being demonstrated here.
  • domain assumption The rescaled piecewise-linear approximation of tanh from PocketNN is an adequate replacement for tanh in integer training.
    Table 2 specifies the PLA tanh from PocketNN after each FC layer; the accuracy results depend on this approximation being close enough to the true activation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TIFeD: a Tiny Integer-based Federated learning algorithm with Direct feedback alignment." pith.science (2026). https://pith.science/paper/BB6SLRFA

@misc{pith2026241116442,
  author       = {Pith},
  title        = {Pith review of: TIFeD: a Tiny Integer-based Federated learning algorithm with Direct feedback alignment},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BB6SLRFA}},
  note         = {Machine review of arXiv:2411.16442}
}
read the original abstract

Training machine and deep learning models directly on extremely resource-constrained devices is the next challenge in the field of tiny machine learning. The related literature in this field is very limited, since most of the solutions focus only on on-device inference or model adaptation through online learning, leaving the training to be carried out on external Cloud services. An interesting technological perspective is to exploit Federated Learning (FL), which allows multiple devices to collaboratively train a shared model in a distributed way. However, the main drawback of state-of-the-art FL algorithms is that they are not suitable for running on tiny devices. For the first time in the literature, in this paper we introduce TIFeD, a Tiny Integer-based Federated learning algorithm with Direct Feedback Alignment (DFA) entirely implemented by using an integer-only arithmetic and being specifically designed to operate on devices with limited resources in terms of memory, computation and energy. Besides the traditional full-network operating modality, in which each device of the FL setting trains the entire neural network on its own local data, we propose an innovative single-layer TIFeD implementation, which enables each device to train only a portion of the neural network model and opens the door to a new way of distributing the learning procedure across multiple devices. The experimental results show the feasibility and effectiveness of the proposed solution. The proposed TIFeD algorithm, with its full-network and single-layer implementations, is made available to the scientific community as a public repository.

Figures

Figures reproduced from arXiv: 2411.16442 by the authors.

Figure 1
Figure 1. Overview of the TIFeD algorithm operating in a federated learning scenario. In particular, Subsection 4.1 gives a general overview on the proposed federated learning scenario. In Subsection 4.2, the transi￾tion from DFA to Federated DFA is described, while Subsection 4.3 presents an extension to Federated DFA in which each client node trains only specific layers of the considered neural networks. 4.1 Overview An ove… view at source ↗
Figure 3
Figure 3. Test accuracies of CNN-1 trained, by exploiting [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 2
Figure 2. Test accuracies of NN-1 trained with the proposed [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Analysis of the behaviour of full-network TIFeD and single-layer TIFeD as the number of worker nodes 𝑀 increases. Floating-point FedAvg is taken as a reference, but we emphasize that the goal is not to compare their final accuracies, since the two algorithms are tailor…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 13 canonical work pages

  1. [1]

    Colby R Banbury, Vijay Janapa Reddi, Max Lam, William Fu, Amin Fazel, Jeremy Holleman, Xinyuan Huang, Robert Hurtado, David Kanter, Anton Lokhmotov, et al. 2020. Benchmarking tinyml systems: Challenges and direction. arXiv preprint arXiv:2003.04821 (2020)

  2. [2]

    Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloe Kiddon, Jakub Konečn`y, Stefano Mazzocchi, Brendan McMahan, et al . 2019. Towards federated learning at scale: System design. Proceedings of machine learning and systems 1 (2019), 374–388

  3. [3]

    Simone Disabato and Manuel Roveri. 2020. Incremental on-device tiny machine learning. In Proceedings of the 2nd International workshop on challenges in artificial intelligence and machine learning for internet of things . 7–13

  4. [4]

    Jack Goetz, Kshitiz Malik, Duc Bui, Seungwhan Moon, Honglei Liu, and Anuj Kumar. 2019. Active federated learning. arXiv preprint arXiv:1909.12641 (2019)

  5. [5]

    Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cor- mode, Rachel Cummings, et al. 2021. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning 14, 1–2 (2021), 1–210

  6. [6]

    Jakub Konečn `y, H Brendan McMahan, Felix X Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016)

  7. [7]

    Kavya Kopparapu and Eric Lin. 2021. TinyFedTL: Federated transfer learning on tiny devices. arXiv preprint arXiv:2110.01107 (2021)

  8. [8]

    Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. 2009. Cifar-10 (canadian institute for advanced research). 2009. URL http://www. cs. toronto. edu/kriz/cifar. html 5 (2009)

Show all 20 references
  1. [9]

    Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman

  2. [10]

    Nil Llisterri Giménez, Marc Monfort Grau, Roger Pueyo Centelles, and Felix Freitag. 2022. On-device training of machine learning models on microcontrollers with federated learning. Electronics 11, 4 (2022), 573

  3. [11]

    Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep net- works from decentralized data. In Artificial intelligence and statistics . PMLR, 1273–1282

  4. [12]

    Arild Nøkland. 2016. Direct feedback alignment provides learning in deep neural networks. Advances in neural information processing systems 29 (2016)

  5. [13]

    Maria Refinetti, Stéphane d’Ascoli, Ruben Ohana, and Sebastian Goldt. 2021. Align, then memorise: the dynamics of learning with feedback alignment. In International Conference on Machine Learning . PMLR, 8925–8935

  6. [14]

    Jaewoo Song and Fangzhen Lin. 2022. PocketNN: Integer-only Training and Infer- ence of Neural Networks via Direct Feedback Alignment and Pocket Activations in Pure C++. arXiv preprint arXiv:2201.02863 (2022)

  7. [15]

    John A Stankovic. 1996. Real-time and embedded systems. ACM Computing Surveys (CSUR) 28, 1 (1996), 205–208

  8. [16]

    Prahalathan Sundaramoorthy, Gautham Krishna Gudur, Manav Rajiv Moorthy, R Nidhi Bhandari, and Vineeth Vijayaraghavan. 2018. Harnet: Towards on- device incremental learning using deep ensembles on constrained devices. In Proceedings of the 2nd International Workshop on Embedded...

  9. [17]

    Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747 (2017)

  10. [18]

    Kai Yang, Tao Jiang, Yuanming Shi, and Zhi Ding. 2020. Federated learning via over-the-air computation. IEEE Transactions on Wireless Communications 19, 3 (2020), 2022–2035

  11. [19]

    LeCun Yann. 1998. The mnist database of handwritten digits. R (1998)

  12. [2014]

    Random feedback weights support learning in deep neural networks.arXiv preprint arXiv:1411.0247 (2014)

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.