REVIEW 4 major objections 5 minor 46 references
Ariel-ML: Computing Parallelization with Embedded Rust for Neural Networks on Heterogeneous Multi-core Microcontrollers
T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read The paper introduces Ariel-ML, a Rust-based toolkit that automatically parallelizes neural-network inference on multi-core microcontrollers, achieving a 1.5x latency speedup on a dual-core chip.
desk verdict Useful Rust-based TinyML integration with believable 1.5x multicore speedup, but the evidence is too thin to back the 'arbitrary models' claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is tiling plus greedy dispatch. At compile time, each neural-network operator is decomposed into several contention-free tiles, called work items; at run time, a scheduler pops work items from a workload queue and assigns each to the first idle core. The design's correctness and speedup both depend on these tiles being genuinely independent and balanced, since the scheduler itself performs no dependency analysis.
What would settle it
Run a model with a large monolithic operator, such as a depthwise convolution or an attention head, on a dual-core board against a single-core build of the same toolkit: if two-core inference does not approach half the single-core latency, or if repeated runs give different results, the contention-free tiling assumption fails. A simpler observable is whether any core idles while work items remain in the queue — the greedy scheduler's claim predicts no idle cores until the queue empties.
Extended reading notes
Core claim
On its own terms, the paper establishes that a generic embedded Rust runtime can parallelize inference for compiled neural networks: each operator's computation is partitioned at compile time into independent tiles called 'work items,' and a greedy scheduler dispatches these items one-by-one to whichever core is free. The measured result is a 1.5x inference-latency reduction on a dual-core microcontroller compared with the same pipeline restricted to one core, at RAM and flash costs comparable to established C/C++ TinyML toolkits. The paper further claims this is the first Rust-based platform to automate multi-core parallelization for arbitrary TinyML models across 32-bit microcontroller fam
Load-bearing premise
The speedup depends on the compiler being able to split every operator's computation into genuinely independent, roughly balanced work items; the paper demonstrates this property for one small quantized model on one dual-core board, but does not prove it for arbitrary TinyML models.
Editorial extensions
If this is right
- Multi-core microcontrollers become usable for TinyML without hand-written parallel code, cutting inference latency by up to 1.5x on a dual-core chip.
- A single compiled model artifact can be deployed and run on different 32-bit microcontroller families, lowering the porting cost for embedded AI.
- Rust-based embedded systems gain an inference path whose latency is competitive with C/C++ toolkits on single-core microcontrollers and faster on multi-core ones.
- The open-source implementation gives practitioners a reproducible way to benchmark their own models on heterogeneous low-power hardware.
- The modular model-plus-metadata layout leaves a clear path toward on-device training and secure over-the-air model updates.
Reading between the lines
- A testable extension is whether the greedy scheduler scales beyond two cores; the paper's 1.5x speedup is close to the bus-contention limit it cites, so chips with more cores or better inter-core bandwidth may not show proportional gains.
- The compiler runtime accounts for a large share of RAM and flash, so a leaner, Rust-native re-implementation of the runtime — which the paper itself proposes — would determine whether the approach fits the smallest constrained devices.
- Because the benchmark evidence is a single small model, the 'wide variety of models' claim is an extrapolation; models with irregular operators, such as attention heads or depthwise convolutions, are the natural next test for contention-free tiling.
- If contention-free tiling generalizes, the same work-item abstraction could also support energy-aware scheduling or fusing multiple sensor models on one board, extending the contribution beyond raw latency.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces Ariel-ML, a toolkit that combines the IREE compiler with the Rust-based Ariel OS to deploy TinyML models on microcontroller-class hardware. The claimed contributions are a generic build pipeline from mainstream ML frameworks to MCU firmware, a Rust runtime with a greedy multi-core scheduler, and an experimental evaluation against RIOT-ML (a prior C-based toolkit) and RIOT+IREE (a variant the authors built). Benchmarks are reported for a quantized LeNet-5 on three boards: nRF52840 (Cortex-M4), ESP32-C3 (RISC-V), and Raspberry Pi Pico (dual-core RP2040). The key positive result is a 1.5× single-to-multi-core speedup on the RP2040. The paper also claims that Ariel-ML achieves memory footprints comparable to C/C++ toolkits, although the reported tables show substantially larger RAM and Flash usage in most configurations.
Significance. If the central claims were fully supported, Ariel-ML would be a useful contribution: it is, to my knowledge, the first Rust-based embedded ML runtime that natively exploits multi-core MCUs, and it is backed by open-source code and a reproducible benchmark setup. The integration of IREE with Ariel OS and the demonstration of a 1.5× speedup on a dual-core RP2040 are promising. However, the evidence in the manuscript is too narrow to support the advertised generality: only one model (LeNet-5) is benchmarked, the multi-core result comes from a single board, and the memory-footprint claims are contradicted by the paper's own tables. The absence of any formal or empirical validation of the 'contention-free tile' assumption leaves the correctness and portability of the scheduler open.
major comments (4)
- [Abstract, §5 (Tables 2 and 3)] The abstract states that Ariel-ML 'achieves comparable memory footprints' compared to C/C++ toolkits. Tables 2 and 3 do not support this. On the nRF52840, Ariel-ML uses 42.7 kB RAM versus 11.3 kB for RIOT-ML (3.8× higher) and 153.5 kB Flash versus 61.3 kB (2.5× higher). On the RP2040, Flash is 172.2 kB versus 65.2 kB (2.6× higher). Even against the authors' RIOT+IREE, Ariel-ML's Flash is larger on every board. The claim should be substantially softened or, if 'comparable' is meant in a functional sense, that interpretation should be made explicit and quantified.
- [§4, §5] The scheduler's correctness and performance rely on the assertion in Section 4 that 'the computation associated with each operator is partitioned into several contention-free tiles, referred to as work items.' The paper neither defines how IREE produces these tiles for arbitrary graphs nor provides any argument that they are data-dependence-free, race-free, and balanced. No measurements of work-item count, size, or load balance are reported. Since the only multi-core benchmark is a single LeNet-5 on the RP2040, the central claim that Ariel-ML can execute 'arbitrary TinyML models' with full multi-core use is not established. The authors should either prove the contention-free property for the generated tiles, show how it is enforced in the compiler, or empirically validate it on a diverse set of models and operators.
- [§5 (Table 1 and methodology)] Latency results are reported without any indication of repeated measurements, variance, or statistical signficance. Table 1 shows single numbers per configuration. On MCU platforms, inference latency can vary with clock scaling, temperature, and memory contention. The paper should report mean ± standard deviation over a stated number of runs, and state whether the speedup is consistent across runs. Additionally, the phrase 'approaching the hardware's known limit imposed by bus contention' cites reference [13], which is an Ariel OS paper and does not obviously contain such a limit. The authors should either provide the correct citation or quantify the contended-bus effect themselves.
- [§5 (Model Selection and Footnote 2)] The model-selection paragraph acknowledges that only LeNet-5 measurements are presented, but then refers to a model zoo at a RIOT-ML GitHub repository (TinyPART/RIOT-ML), not an Ariel-ML model zoo. The claim that 'Ariel-ML supports a wide variety of models' is therefore not backed by any artifact in this paper. A reader cannot tell which models from that zoo have actually been tested with Ariel-ML. The authors should either expand the benchmark set to include at least 2–3 additional models (e.g., a small transformer or a depthwise-separable CNN) or explicitly limit all claims about generality.
minor comments (5)
- [§1, Related Work] Typo: 'parllelization' should be 'parallelization' in the Related Work section. Also 'Expressif' should be 'Espressif', and 'RaspberryPi' should be 'Raspberry Pi'.
- [§2.2] The sentence 'To the best of our knowledge, Ariel OS is so far the only embedded Rust RTOS that supports multicore at this level' could be clarified: does 'this level' mean multiple cores on a single MCU, or SMP scheduling? Please specify.
- [§5 (Figure 5)] The pie charts in Figure 5 show a RAM breakdown summing to 100% with 25% 'Model' and 66% 'Stack/OS/Lib/etc.', but the text says the IREE runtime accounts for 9% and requires a 16 kB stack. It would help to define what is included in 'Stack/OS/Lib/etc.' and how the 16 kB stack was measured.
- [§5 (Table 1)] The row 'RP2040+multicore' lists '(NA)' for RIOT-ML and RIOT+IREE, which is fine, but the table would be clearer if the single-core RP2040 Rows for RIOT-ML and RIOT+IREE were also shown in the same row group, or if the speedup factor was broken down into scheduler overhead versus IREE benefit.
- [References] Reference [13] and [14] appear to be the same paper in arXiv and IEEE DCOSS-IoT versions. Deduplicate or cross-reference them explicitly.
Circularity Check
No circularity found: the paper is an empirical systems-and-benchmark contribution whose core claims rest on measurements, not on fitted parameters, definitions, or load-bearing self-citations.
full rationale
Ariel-ML is an experimental systems paper, not a derivation. Its central claims—that IREE-based compilation plus a greedy multicore scheduler on Ariel OS reduces inference latency on the RP2040 and keeps memory footprints comparable—are supported by direct measurements reported in Tables 1–3, with no fitted parameters, no equations tying an output back to an input, and no quantity that is defined in terms of the result it is said to predict. The closest thing to a load-bearing assumption is the Section 4 statement that "the computation associated with each operator is partitioned into several contention-free tiles, referred to as work items." This is an implementation-level premise, not a circularity: the paper does not derive the 1.5x speedup from that assumption, nor does it define the assumption in terms of the measured speedup. An unproven assumption about generality and work-item independence is a correctness/robustness limitation, not a circular step. The paper does cite prior work by the same authors (RIOT-ML, U-TOE, Ariel OS) and even compares against RIOT+IREE, a baseline the authors designed and implemented. That is a methodological self-comparison concern, but every reported number is an independent empirical observation; no benchmark value is derived from those citations, and no uniqueness theorem or ansatz is imported from the authors' previous work to force a conclusion. The phrase "outperforms prior art ... as expected" is rhetorical, not a prediction derived from an input. Overall, the derivation chain, to the extent one exists, is the standard software-engineering chain of build, deploy, and measure. No step reduces to its own inputs by construction, so the circularity score is 0.
Assumptions & free parameters
assumptions (5)
- domain assumption The IREE compiler and runtime are correct: lowering a model to target-specific code preserves semantic correctness across MCU targets.
- ad hoc to paper Each operator's computation can be partitioned into 'several contention-free tiles' (work items) with no data dependencies or races.
- ad hoc to paper A greedy work-conserving scheduler that pops one work item at a time and dispatches to an idle core yields near-optimal speedups without idle-core or priority problems.
- ad hoc to paper Quantized LeNet-5 is representative of the 'wide variety of TinyML models' that the toolkit claims to support.
- domain assumption The RIOT-ML and RIOT+IREE baseline configurations are fair, correctly tuned, and measured under equivalent conditions.
Cite this review
Pith. "Pith review of Ariel-ML: Computing Parallelization with Embedded Rust for Neural Networks on Heterogeneous Multi-core Microcontrollers." pith.science (2026). https://pith.science/paper/HI55DU2Y
@misc{pith2026251209800,
author = {Pith},
title = {Pith review of: Ariel-ML: Computing Parallelization with Embedded Rust for Neural Networks on Heterogeneous Multi-core Microcontrollers},
year = {2026},
howpublished = {\url{https://pith.science/paper/HI55DU2Y}},
note = {Machine review of arXiv:2512.09800}
}
read the original abstract
Low-power microcontroller (MCU) hardware is currently evolving from single-core architectures to predominantly multi-core architectures. In parallel, new embedded software building blocks are more and more written in Rust, while C/C++ dominance fades in this domain. On the other hand, small artificial neural networks (ANN) of various kinds are increasingly deployed in edge AI use cases, thus deployed and executed directly on low-power MCUs. In this context, both incremental improvements and novel innovative services will have to be continuously retrofitted using ANNs execution in software embedded on sensing/actuating systems already deployed in the field. However, there was so far no Rust embedded software platform automating parallelization for inference computation on multi-core MCUs executing arbitrary TinyML models. This paper thus fills this gap by introducing Ariel-ML, a novel toolkit we designed combining a generic TinyML pipeline and an embedded Rust software platform which can take full advantage of multi-core capabilities of various 32bit microcontroller families (Arm Cortex-M, RISC-V, ESP-32). We published the full open source code of its implementation, which we used to benchmark its capabilities using a zoo of various TinyML models. We show that Ariel-ML outperforms prior art in terms of inference latency as expected, and we show that, compared to pre-existing toolkits using embedded C/C++, Ariel-ML achieves comparable memory footprints. Ariel-ML thus provides a useful basis for TinyML practitioners and resource-constrained embedded Rust developers.
Figures
Reference graph
Works this paper leans on
-
[13]
Elena Frank, Kaspar Schleiser, Romain Fouquet, Koen Zandberg, Christian Am- süss, and Emmanuel Baccelli. 2025. Ariel OS: An Embedded Rust Operating System for Networked Sensors & Multi-Core Microcontrollers.arXiv preprint arXiv:2504.19662(2025)
arXiv 2025
-
[1]
Unpublished working draft.Not for distribution
Emmanuel Baccelli, Cenk Gündoğan, Oliver Hahm, Peter Kietzmann, Martine S Lenders, Hauke Petersen, Kaspar Schleiser, Thomas C Schmidt, and Matthias Preprint — Under Review. Unpublished working draft.Not for distribution. Ariel-ML: Computing Parallelization with Embedded Rust for Neural Networks on Heterogeneous Multi-core Microcontrollers Wählisch. 2018. ...
2018
-
[2]
Vlad-Eusebiu Baciu, Johan Stiens, and Bruno da Silva. 2024. MLino bench: A comprehensive benchmarking tool for evaluating ML models on edge devices. Journal of Systems Architecture155 (2024), 103262
2024
-
[3]
Back to the Building Blocks. 2024. White House Report. https://bidenwhitehouse. archives.gov/wp-content/uploads/2024/02/Final-ONCD-Technical-Report.pdf
2024
-
[4]
Colby Banbury, Vijay Janapa Reddi, Peter Torelli, Jeremy Holleman, Nat Jeffries, Csaba Kiraly, Pietro Montino, David Kanter, Sebastian Ahmed, Danilo Pau, et al
-
[5]
Carsten Bormann et al. 2014. Terminology for Constrained-Node Networks. RFC
2014
-
[6]
Burn: Deep Learning Framework. 2025. Tracel AI. https://github.com/tracel- ai/burn
2025
-
[7]
Candle Machine Learning Framework. 2025. HuggingFace. https://github.com/ huggingface/candle
2025
Show all 46 references
-
[8]
Matteo Carnelos, Francesco Pasti, and Nicola Bellotto. 2025. MicroFlow: An Efficient Rust-Based Inference Engine for TinyML.Internet of Things30 (2025), 101498. doi:10.1016/j.iot.2025.101498
2025
-
[9]
2018.{TVM}: An automated{End-to-End} optimizing compiler for deep learning
Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. 2018.{TVM}: An automated{End-to-End} optimizing compiler for deep learning. In13th USENIX Symposium on Operating Systems Design and Impleme...
2018
-
[10]
Cheng-Tao Chu, Sang Kim, Yi-An Lin, YuanYuan Yu, Gary Bradski, Kunle Oluko- tun, and Andrew Ng. 2006. Map-reduce for machine learning on multicore. Advances in neural information processing systems19 (2006)
2006
-
[11]
Torre-Bastida, Gorka Zárate, Raúl Miñón, and Aitor Almeida
Josu Diaz-de Arcaya, Ana I. Torre-Bastida, Gorka Zárate, Raúl Miñón, and Aitor Almeida. 2023. A Joint Study of the Challenges, Opportunities, and Roadmap of MLOps and AIOps: A Systematic Survey.Comput. Surveys56, 4 (Oct. 2023), 84:1–84:30. doi:10.1145/3625289
2023 doi
-
[12]
Embassy Framework. 2025. Embassy Developpers. https://github.com/embassy- rs/embassy
2025
-
[14]
Elena Frank, Kaspar Schleiser, Romain Fouquet, Koen Zandberg, Christian Am- süss, and Emmanuel Baccelli. 2025. Ariel OS: An Embedded Rust Operating System for Networked Sensors & Multi-Core Microcontrollers. InIEEE Inter- national Conference on Distributed Computing in Sma...
2025
-
[15]
FreeRTOS. 2025. Amazon. https://www.freertos.org/
2025
-
[16]
Nikhil P Ghanathe and Steven JE Wilton. 2024. DEBUG-HD: Debugging TinyML models on-device using Hyper-Dimensional computing.arXiv preprint arXiv:2411.10692(2024)
2024 arXiv
-
[17]
Ashish Ghosh, Debasrita Chakraborty, and Anwesha Law. 2018. Artificial intelli- gence in Internet of things.CAAI Transactions on Intelligence Technology3, 4 (2018), 208–218
2018
-
[18]
Oliver Hahm, Emmanuel Baccelli, Hauke Petersen, and Nicolas Tsiftes. 2015. Operating systems for low-end devices in the internet of things: a survey.IEEE Internet of Things Journal3, 5 (2015), 720–734
2015
-
[19]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition. 770–778
2016
-
[20]
Zhaolan Huang and Emmanuel Baccelli. 2025. msf-CNN: Patch-based Multi-Stage Fusion with Convolutional Neural Networks for TinyML.NeurIPS(2025)
2025
-
[21]
Zhaolan Huang, Koen Zandberg, Kaspar Schleiser, and Emmanuel Baccelli. 2023. U-TOE: Universal TinyML On-Board Evaluation Toolkit for Low-Power IoT. In2023 12th IFIP/IEEE International Conference on Performance Evaluation and Modeling in Wired and Wireless Networks (PEMWN). IEEE, 1–6
2023
-
[22]
Zhaolan Huang, Koen Zandberg, Kaspar Schleiser, and Emmanuel Baccelli. 2025. RIOT-ML: toolkit for over-the-air secure updates and performance evaluation of TinyML models.Annals of Telecommunications80, 3 (2025), 283–297
2025
-
[23]
IREE: Intermediate Representation Execution Environment. 2025. IREE Develop- ers. https://github.com/iree-org/iree
2025
-
[24]
Victor JB Jung, Alessio Burrello, Moritz Scherer, Francesco Conti, and Luca Benini. 2024. Optimizing the deployment of tiny transformers on low-power mcus.IEEE Trans. Comput.(2024)
2024
-
[25]
Brett Koonce. 2021. ResNet 34.Convolutional neural networks with swift for tensorflow: image recognition and dataset categorization(2021), 51–61
2021
-
[26]
Dominik Kreuzberger, Niklas Kühl, and Sebastian Hirschl. 2023. Machine Learn- ing Operations (MLOps): Overview, Definition, and Architecture.IEEE Access11 (2023), 31866–31879. doi:10.1109/ACCESS.2023.3262138
2023
-
[27]
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 2002. Gradient- based learning applied to document recognition.Proc. IEEE86, 11 (2002), 2278– 2324
2002
-
[28]
Amit Levy et al. 2017. The TOCK embedded operating system. InACM SenSys
2017
-
[29]
Ji Lin, Wei-Ming Chen, Han Cai, Chuang Gan, and Song Han. 2021. Memory-efficient Patch-based Inference for Tiny Deep Learning. InAd- vances in Neural Information Processing Systems, Vol. 34. Curran Asso- ciates, Inc., 2346–2358. https://proceedings.neurips.cc/paper/2021/hash/ ...
2021
-
[30]
Hsin-I Cindy Liu, Marius Brehler, Mahesh Ravishankar, Nicolas Vasilache, Ben Vanik, and Stella Laurenzo. 2022. TinyIREE: An ML Execution Environment for Embedded Systems From Compilation to Deployment.IEEE Micro42 (2022), 9–16. doi:10.1109/MM.2022.3143865
2022
-
[31]
Zhongwen Luo, Hongzhi Liu, and Xincai Wu. 2005. Artificial neural network computation on graphic process unit. InProceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005., Vol. 1. IEEE, 622–626
2005
-
[32]
Minh Tri Lê and Julyan Arbel. 2023. TinyMLOps for real-time ultra-low power MCUs applied to frame-based event classification. InProceedings of the 3rd Workshop on Machine Learning and Systems (EuroMLSys ’23). Association for Computing Machinery, New York, NY, USA, 148–153
2023
-
[33]
NuttX. 2025. Apache Foundation. https://nuttx.apache.org/
2025
-
[34]
Anas Osman, Usman Abid, Luca Gemma, Matteo Perotto, and Davide Brunelli
-
[35]
Jeffrey Vander Stoep. 2025. Transitioning to Memory Safety: Lessons from the Android Project.RustWeek(2025)
2025
-
[36]
Bharath Sudharsan, Simone Salerno, Duc-Duy Nguyen, Muhammad Yahya, Ab- dul Wahid, Piyush Yadav, John G Breslin, and Muhammad Intizar Ali. 2021. Tinyml benchmark: Executing fully connected neural networks on commodity microcontrollers. In2021 IEEE 7th World Forum on Internet of...
2021
-
[37]
The Cyber Resilience Act (CRA). 2024. European Union. https://digital-strategy. ec.europa.eu/en/policies/cyber-resilience-act
2024
-
[38]
ThreadX. 2025. Eclipse Foundation. https://github.com/eclipse-threadx
2025
-
[39]
Vincent Vanhoucke, Andrew Senior, Mark Z Mao, et al. 2011. Improving the speed of neural networks on CPUs. InProc. deep learning and unsupervised feature learning NIPS workshop, Vol. 1. 4
2011
-
[40]
Mingyuan Xiang, Pouya Mahdi Gholami, and Henry Hoffmann. 2025. Lupe: Integrating the Top-down Approach with DNN Execution on Ultra-Low-Power Devices. InProceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems. 357–370
2025
-
[41]
Zephyr Project. 2025. Linux Foundation. https://zephyrproject.org/
2025
-
[42]
Hongbin Zhang, Mingjie Xing, Yanjun Wu, and Chen Zhao. 2023. Compiler Technologies in Deep Learning Co-Design: A Survey.Intelligent Computing2 (2023), 0040
2023
-
[43]
Size Zheng, Yun Liang, Shuo Wang, Renze Chen, and Kaiwen Sheng. 2020. Flex- tensor: An automatic schedule exploration and optimization framework for tensor computation on heterogeneous system. InProceedings of the Twenty-Fifth International Conference on Architectural Support ...
2020
-
[2021]
Mlperf tiny benchmark.arXiv preprint arXiv:2106.07597(2021)
2021 arXiv
-
[2022]
InApplications in Electronics Pervading Industry, Environment and Society: APPLEPIES 2021
Tinyml platforms benchmarking. InApplications in Electronics Pervading Industry, Environment and Society: APPLEPIES 2021. Springer, 139–148
2021
-
[7228]
doi:10.17487/RFC7228
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.