REVIEW 3 major objections 6 minor 13 references
Hardware-Software Co-Design for Float16 On-Device Training on RISC-V Single-Core
T0 review · 3 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read Full float16 on-device training of small neural networks is achievable on a resource-constrained RISC-V single-core using the standard Zfh and Zvfh extensions, cutting memory use by about half versus float32 with minimal accuracy loss.
desk verdict Useful incremental result: float16 on-device training on single-core RISC-V works in simulation and Zfh area cost is cleanly measured; the Zvfh and 'on-device' claims outrun the evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the pairing of the standard RISC-V Zfh/Zvfh extensions with a modular DNN training framework that can swap in hand-vectorized float16 kernels. Zfh and Zvfh are the RISC-V standard extensions for scalar and vector half-precision floating-point arithmetic; they matter because they let all training state live in float16 without custom accelerators or non-standard instruction sets. Zfh's implementation trick is nan-boxing: narrow float16 values are padded with ones and carried through the 64-bit FPU datapath, which is why adding support to the NaxRiscv FPU costs little. Zvfh's proposed vector unit uses four 64-bit lanes fed by crossbars, a vector register file merge
What would settle it
Train the same 5-layer MLP on MNIST in float16 on a physical Zfh-enabled RISC-V core (or the implemented NaxRiscv FPGA target) and compare validation accuracy, memory footprint, and runtime with the simulator results; divergence in accuracy beyond the reported roughly one point, memory above about 50% savings, or significantly different speed would falsify the central claim. For the Zvfh part, implement the four-lane VLEN=64 VPU and check that the float16 vector kernels produce correct results and the predicted 4x raw speed-up.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that full float16 on-device training is possible on resource-constrained RISC-V single-core processors supporting the standard Zfh and/or Zvfh extensions, while requiring about 50% less memory than float32 training. The evidence consists of an extended AIfES framework with float16 math primitives, a model converter that can freeze layers for transfer learning, and training experiments run on a RISC-V ISA simulator: a 5-layer MLP trained from scratch on MNIST and FashionMNIST reaches validation accuracies within about one point of float32 (e.g., 97.29% vs 96.88% on MNIST/SGD), and fine-tuning a pretrained MLP on EMNIST shows a flat 50% reduction
Load-bearing premise
The weakest load-bearing premise is that the RISC-V ISA simulator used for all training results (Section 2.2) faithfully reproduces the numerical behavior and performance of the real NaxRiscv core, and, for the Zvfh part, that the vector unit described in Section 3.3 works as designed; the authors themselves state that measuring on the FPGA target is future work.
Editorial extensions
If this is right
- A single-core RISC-V MCU with Zfh support can run end-to-end float16 training, cutting parameter memory to about half of float32 while keeping validation accuracy within about a point.
- Because Zfh costs only about 1.15% extra LUTs and 0.05% extra flip-flops with no clock drop on the NaxRiscv FPGA core, adding half-precision training capability to existing RISC-V designs is cheap.
- A Zvfh vector unit with four 64-bit lanes would give a raw 4x SIMD speed-up for float16 kernels while keeping standard vector code portable across RISC-V implementations.
- Layer freezing in the converter enables on-device fine-tuning and transfer learning, with additional working-memory savings when only the last layer is trained.
- The approach supports batch sizes greater than 1 in full float16 training, unlike some prior RISC-V training frameworks that are limited to batch size 1.
Reading between the lines
- The paper leaves implicit that the same framework should port to any Zvfh-capable RISC-V core, not just NaxRiscv, because Zvfh is a standard extension; that portability is testable once real Zvfh silicon exists.
- A logical next experiment is to measure actual training energy and throughput on the FPGA target; if the simulator's cycle counts are close to silicon, float16 ODT could become viable for battery-powered sensors.
- Because Adam stores two optimizer states per parameter, the 50% memory reduction is much smaller when Adam is used; with SGD and frozen early layers, working memory approaches the model size, making optimizer choice—not precision—the main memory lever.
- The vrgather handling points to a boundary for the design: gather-like vector instructions need special treatment when register groups are split into micro-ops, so the proposed Zvfh support is best seen as a deliberate architectural approach rather than a proof that every RVV instruction maps cleanly onto lanes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a hardware–software co-design flow for float16 on-device training (ODT) on RISC-V single-core processors. The software contribution extends the open-source AIfES framework with RISC-V Zfh/Zvfh float16 kernels, a converter from PyTorch/TensorFlow models, and layer-freezing support for transfer learning. Training experiments on an MLP (512-256-128-64-10) on MNIST and FashionMNIST, run on a RISC-V ISA simulator, show F16 accuracy within about 0.7 percentage points of F32, and memory measurements report an approximately 50% reduction in model and work memory when using float16. The hardware contribution is an implementation of the scalar Zfh extension in the NaxRiscv out-of-order FPGA softcore, with a reported +1.15% LUT6 and +0.05% FF area overhead at 175 MHz, plus a design discussion of a Zvfh vector processing unit with four lanes and VLEN=64. The conclusion claims that full float16 ODT is possible on resource-constrained RISC-V single-cores with Zfh and/or Zvfh, with about 50% memory savings and minimal accuracy loss.
Significance. If the claims hold, this is a useful practical contribution: it would make full float16 training, not just inference or quantized fine-tuning, accessible on small RISC-V cores, with a solid arithmetic memory reduction and a measured low area cost for Zfh. The open-source release of the framework and the clean synthesis measurement for Zfh are strengths. The paper honestly states in the final paragraph that training was validated on an ISA simulator and that Zvfh performance evaluation is future work; however, the abstract and conclusion do not carry this caveat, and the central 'on-device' claim is therefore stronger than the evidence. The accuracy comparison is based on four single runs without seeds or error bars, so the 'minimal degradation' part of the claim is not yet statistically demonstrated. The Zvfh part is an architecture proposal, not an implemented and measured design. These are external-validation gaps rather than internal inconsistencies, and they are fixable, but they currently prevent the strongest form of the conclusion.
major comments (3)
- [§2.2, Table 2] The claim of 'minimal model performance degradation' rests on four single training runs with no seeds, no error bars, and no repetition. The reported F16-vs-F32 differences are at most 0.68 percentage points (FashionMNIST SGD) and can readily be within run-to-run variation for MLPs at this scale. Please provide mean±standard deviation over at least 5 seeds (or otherwise quantify variance) and report the same hyperparameters for all datasets. Without this, the accuracy comparison is not load-bearing evidence for the conclusion.
- [§2.2, Conclusion] All training experiments are performed on a RISC-V ISA simulator, not on the NaxRiscv FPGA target or any real Zfh/Zvfh core. The conclusion 'full float16 ODT is possible on resource-constrained RISC-V single-core' therefore assumes that the simulator faithfully reproduces Zfh/Zvfh functional behavior and, where performance is implied, timing. The paper's own final paragraph acknowledges this ('validating the ODT pipeline through the RISC-V ISA Simulator'), but the abstract and conclusion do not. Please either validate at least the scalar Zfh path on the implemented NaxRiscv (e.g., on FPGA) and report measured runtime, or explicitly qualify the on-device claim as 'functionally validated on an ISA simulator; hardware validation pending.'
- [§3.3, Fig. 4–5, Conclusion] The Zvfh support is at the design-discussion stage: no RTL implementation, synthesis, area, timing, or measured speedup is reported. The 'raw 4× speed-up over scalar execution' is a theoretical peak based on four lanes and VLEN=64. Consequently, the conclusion's phrase 'supporting standard extensions (i.e. Zfh and/or Zvfh)' overreaches for Zvfh, and the inclusion of Zvfh in the central claim is premature. The memory reduction and Zfh area results are unaffected, but Section 3.3 should be clearly labeled as an architecture proposal and the conclusion should state that Zvfh results are projected, not measured.
minor comments (6)
- [Introduction] Typo: 'Multipliy-and-Accumulate' should be 'Multiply-and-Accumulate'.
- [§3.1] Typo: 'seemlessly' should be 'seamlessly'.
- [§2.2, Fig. 2] Fig. 2 caption says 'our RISC-V Zvfh optimized kernels', but the text only says experiments were run on a RISC-V ISA simulator. Please specify which simulator was used, whether Zvfh vector instructions were functionally simulated, and the NMSIS-DSP version, so that readers can reproduce the experiments.
- [Table 2] The table lists dataset, learning rate, and optimizer but not the number of epochs, batch size, or dataset split for FashionMNIST. The MNIST experiments use 20 epochs and batch size 128; please state whether the same settings were used for all rows.
- [Table 3] The notation 'Frozen layer id 1-2' is not defined; clarify that it means the first two layers are frozen. Also, the caption says the 'Gradients' column includes gradients, optimizer, and activation memory; please make explicit how activation memory scales with batch size for the reported numbers.
- [§3.3] The phrase 'in a ad-hoc fashion' should be 'in an ad hoc manner'.
Circularity Check
No significant circularity: derivation is empirical and self-contained, with no fitted parameters presented as predictions and no load-bearing self-citations.
full rationale
The paper's central claims are (i) float16-only ODT is feasible with minimal accuracy loss compared to float32, (ii) memory footprint is approximately halved, and (iii) enabling Zfh on NaxRiscv has low area cost, with a Zvfh architecture discussed but not yet implemented. None of these reduces to its own inputs by construction. The accuracy/loss results in Fig. 2 and Table 2 are direct measurements from training runs on a RISC-V ISA simulator, compared against a float32 baseline; no parameter is fitted to a subset of data and then renamed as a prediction. The ~50% memory reduction in Table 3 follows from the float16 format halving parameter and work-memory width, but the paper reports actual measured memory usage for concrete configurations rather than deriving a claimed predictive law from that definition; this is an intrinsic property of the data format, not a circular derivation. The Zfh area overhead (+1.15% LUT6, +0.05% FF, no frequency degradation) is an empirical synthesis result; the Zvfh discussion is explicitly an architectural proposal with future evaluation, so it is not dressed up as a validated prediction. References to AIfES, NMSIS-DSP, PULP-Trainlib, Vicuna2, Ara2, and similar works are external and not self-citations, and the paper does not invoke an author-derived uniqueness theorem or ansatz to force its choices. The main weakness is external-validation risk: the training evaluation is on an ISA simulator and the Zvfh hardware is unbuilt, so the 'on-device' claim is not yet silicon/FPGA-validated for the vector path. That is a correctness/evidence gap, not a circularity, and does not raise the circularity score.
Assumptions & free parameters
assumptions (4)
- domain assumption RISC-V ISA simulator faithfully models IEEE 754 binary16 behavior and Zvfh vector semantics of the target core.
- domain assumption NMSIS-DSP float16 kernels are numerically correct and portable across Zvfh-compliant implementations.
- domain assumption A 5-layer MLP on MNIST/FashionMNIST is representative of the 'complete on-device training' workloads claimed.
- domain assumption The single Zfh synthesis result on a Xilinx XC7K325T is representative of the Zfh cost on other FPGA/ASIC targets.
Cite this review
Pith. "Pith review of Hardware-Software Co-Design for Float16 On-Device Training on RISC-V Single-Core." pith.science (2026). https://pith.science/paper/MQZQNWFH
@misc{pith2026260721130,
author = {Pith},
title = {Pith review of: Hardware-Software Co-Design for Float16 On-Device Training on RISC-V Single-Core},
year = {2026},
howpublished = {\url{https://pith.science/paper/MQZQNWFH}},
note = {Machine review of arXiv:2607.21130}
}
read the original abstract
By leveraging standard RISC-V extensions, namely Zfh (scalar float16) and Zvfh (vector float16), this work proposes an open-source framework to enable complete on-device training on resource-constrained RISC-V single-core. Our approach allows memory footprint reduction by about 50% as compared to using float32 and with minimal model performance degradation. We also facilitate transfer learning and fine-tuning scenarios by incorporating layer-freezing capabilities. Our work builds onto AIfES, an open-source, modular and generic DNN training and inference framework for embedded systems that can be extended with custom hardware-specific functions. The benefits of float16 is further emphasized by outlining the low area overhead of Zfh on a RV64GC super-scalar out-of-order FPGA softcore (+1.15% LUT6 and +0.05% FF at 175MHz). Finally, we discuss the architecture of a Zvfh implementation within the same RISC-V core.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Advances in Neural Information Processing Systems33, 11285–11297 (2020)
Cai, H., Gan, C., Zhu, L., Han, S.: Tinytl: Reduce memory, not parameters for efficient on-device learning. Advances in Neural Information Processing Systems33, 11285–11297 (2020)
2020
-
[2]
IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2024)
Deutel, M., Hannig, F., Mutschler, C., Teich, J.: On-device training of fully quantized deep neural networks on cortex-m microcontrollers. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (2024)
2024
-
[3]
arXiv preprint arXiv:1911.099253(25), 15–17 (2019)
Genc, H., Haj-Ali, A., Iyer, V ., Amid, A., Mao, H., Wright, J., Schmidt, C., Zhao, J., Ou, A., Banister, M., et al.: Gemmini: An agile systolic array generator enabling systematic evaluations of deep-learning architectures. arXiv preprint arXiv:1911.099253(25), 15–17 (2019)
arXiv 1911
-
[4]
0: Risc-v embedded vector unit with half- precision floating-point support for tinyml
Jones, J.P., Van Kempen, P., Mueller-Gritschneder, D.: Vicuna2. 0: Risc-v embedded vector unit with half- precision floating-point support for tinyml. In: 2025 Austrochip Workshop on Microelectronics (Austrochip). pp. 69–72. IEEE (2025)
2025
-
[5]
IEEE Transactions on Computers (2026)
Kamaleldin, A., Aouinti, H., Göhringer, D.: Procon-v: A programmable tightly coupled convolution accelerator based on risc-v custom instructions for edge devices. IEEE Transactions on Computers (2026)
2026
-
[6]
In: Proceedings of the 21st ACM International Conference on Computing Frontiers: Workshops and Special Sessions
van Kempen, P., Jones, J.P., Mueller-Gritschneder, D., Schlichtmann, U.: Muriscv-nn: Challenging zve32x autovectorization with tinyml inference library for risc-v vector extension. In: Proceedings of the 21st ACM International Conference on Computing Frontiers: Workshops and Special Sessions. pp. 75–78 (2024)
2024
-
[7]
In: 2025 33rd Telecommunications Forum (TELFOR)
Mahmoudi, A., Horn, C., Mulhem, S., Buchty, R., Berekovic, M., Meyer, R.: Tinyml unleashed: Accelerating tensorflow lite micro kernels with risc-v vector extension. In: 2025 33rd Telecommunications Forum (TELFOR). pp. 1–4. IEEE (2025)
2025
-
[8]
In: International Conference on Embedded Computer Systems
Nadalini, D., Rusci, M., Tagliavini, G., Ravaglia, L., Benini, L., Conti, F.: Pulp-trainlib: Enabling on-device training for risc-v multi-core mcus through performance-driven autotuning. In: International Conference on Embedded Computer Systems. pp. 200–216. Springer (2022)
2022
Show all 13 references
-
[9]
In: 2020 IEEE International Symposium on Circuits and Systems (ISCAS)
Patsidis, K., Nicopoulos, C., Sirakoulis, G.C., Dimitrakopoulos, G.: Risc-v 2: a scalable risc-v vector processor. In: 2020 IEEE International Symposium on Circuits and Systems (ISCAS). pp. 1–5. IEEE (2020) 6 HW-SW Co-Design for F16 On-Device Training on RISC-V Single-Core
2020
-
[10]
IEEE Transactions on Computers73(7), 1822–1836 (2024)
Perotti, M., Cavalcante, M., Andri, R., Cavigelli, L., Benini, L.: Ara2: Exploring single-and multi-core vector processing with an efficient rvv 1.0 compliant open-source processor. IEEE Transactions on Computers73(7), 1822–1836 (2024)
2024
-
[11]
In: International Conference on Data Science and Artificial Intelligence
Ribeiro, M., Costa, D., Moreira, G., Pinto, S., Gomes, T.: Decentor-v: Lightweight ml training on low-power risc-v edge devices. In: International Conference on Data Science and Artificial Intelligence. pp. 256–269. Springer (2025)
2025
-
[12]
In: 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE)
Tortorella, Y ., Bertaccini, L., Rossi, D., Benini, L., Conti, F.: Redmule: A compact fp16 matrix-multiplication accelerator for adaptive deep learning on risc-v-based ultra-low-power socs. In: 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE). pp. 1099–1...
2022
-
[13]
IEEE Transactions on Pattern Analysis and Machine Intelligence46(6), 4519–4533 (2024) 7
Wulfert, L., Kühnel, J., Krupp, L., Viga, J., Wiede, C., Gembaczka, P., Grabmaier, A.: Aifes: A next-generation edge ai framework. IEEE Transactions on Pattern Analysis and Machine Intelligence46(6), 4519–4533 (2024) 7
2024
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.