TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems
Reviewed by Pithpith:RFHA6EANopen to challenge →
read the original abstract
Deep learning inference on embedded devices is a burgeoning field with myriad applications because tiny embedded devices are omnipresent. But we must overcome major challenges before we can benefit from this opportunity. Embedded processors are severely resource constrained. Their nearest mobile counterparts exhibit at least a 100 -- 1,000x difference in compute capability, memory availability, and power consumption. As a result, the machine-learning (ML) models and associated ML inference framework must not only execute efficiently but also operate in a few kilobytes of memory. Also, the embedded devices' ecosystem is heavily fragmented. To maximize efficiency, system vendors often omit many features that commonly appear in mainstream systems, including dynamic memory allocation and virtual memory, that allow for cross-platform interoperability. The hardware comes in many flavors (e.g., instruction-set architecture and FPU support, or lack thereof). We introduce TensorFlow Lite Micro (TF Micro), an open-source ML inference framework for running deep-learning models on embedded systems. TF Micro tackles the efficiency requirements imposed by embedded-system resource constraints and the fragmentation challenges that make cross-platform interoperability nearly impossible. The framework adopts a unique interpreter-based approach that provides flexibility while overcoming these challenges. This paper explains the design decisions behind TF Micro and describes its implementation details. Also, we present an evaluation to demonstrate its low resource requirement and minimal run-time performance overhead.
This paper has not been read by Pith yet.
Forward citations
Cited by 6 Pith papers
-
OpenGlass: Ultra-Low-Power On-Device AI Eyewear with Event-based Vision
OpenGlass is an open-source smart glasses platform using event-based vision and event-driven power management to achieve 11.5 hours of continuous on-device ML on a 200 mAh battery, demonstrated with 83.94% cross-subje...
-
AIGaitor: Privacy-preserving and cloud-free motion analysis for everyone, using edge computing
The paper presents AIGaitor, a privacy-preserving on-device monocular motion analysis system that performs end-to-end pose estimation and deep learning gait analysis on consumer smartphones.
-
AIGaitor: Privacy-preserving and cloud-free motion analysis for everyone, using edge computing
AIGaitor is the first claimed end-to-end on-device monocular motion-capture and deep-learning gait analysis pipeline demonstrated on consumer smartphones.
-
Towards Real-Time ECG and EMG Modeling on $\mu$NPUs
PhysioLite delivers Transformer-comparable ECG/EMG performance using learnable wavelet filters and hardware-aware design at ~370KB quantized size on μNPUs.
-
Production-Ready Automated ECU Calibration using Residual Reinforcement Learning
Residual reinforcement learning automates map-based ECU calibration to closely match series production references with minimal human intervention.
-
Quantized AI Inference on Constrained Embedded Platforms for Small-Satellite Settings
Measurement-based characterization of quantized AI inference latency and data movement on Cortex-M platforms, positioned as a lower-bound reference for small-satellite embedded vision workloads.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.