Pith. sign in

REVIEW 5 cited by

Benchmarking TinyML Systems: Challenges and Direction

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2003.04821 v4 pith:TQFEAR6U submitted 2020-03-10 cs.PF cs.LG

Colby R. Banbury , Vijay Janapa Reddi , Max Lam , William Fu , Amin Fazel , Jeremy Holleman , Xinyuan Huang , Robert Hurtado
show 9 more authors
David Kanter Anton Lokhmotov David Patterson Danilo Pau Jae-sun Seo Jeff Sieracki Urmish Thakker Marian Verhelst Poonam Yadav
This is my paper
classification cs.PFcs.LG
keywords tinymlsystemsbenchmarkbenchmarkingchallengesdirectiondiscusshardware
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent advancements in ultra-low-power machine learning (TinyML) hardware promises to unlock an entirely new class of smart applications. However, continued progress is limited by the lack of a widely accepted benchmark for these systems. Benchmarking allows us to measure and thereby systematically compare, evaluate, and improve the performance of systems and is therefore fundamental to a field reaching maturity. In this position paper, we present the current landscape of TinyML and discuss the challenges and direction towards developing a fair and useful hardware benchmark for TinyML workloads. Furthermore, we present our four benchmarks and discuss our selection methodology. Our viewpoints reflect the collective thoughts of the TinyMLPerf working group that is comprised of over 30 organizations.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MambaLite-Micro: Memory-Optimized Mamba Inference on MCUs

    cs.LG 2025-09 conditional novelty 6.0 of 10

    MambaLite-Micro deploys Mamba inference in C on ESP32S3 and STM32H7 with 83.0% lower peak memory than an unfused baseline and identical output labels to PyTorch.

  2. Learning to Transmit: Volatility-Aware Predictive Communication for Energy-Efficient IoT Networks

    cs.IT 2026-07 conditional novelty 5.0 of 10

    Sensors using volatility-aware studentized residuals plus RLS online adaptation transmit up to 94.7% less IoT data while keeping reconstruction MAE at 0.35°C.

  3. Searching Neural Architectures for Sensor Nodes on IoT Gateways

    cs.LG 2025-05 conditional novelty 4.0 of 10

    GatewayNAS adapts the hardware-aware neural architecture search space to the time and energy budget of an IoT gateway, producing tiny CNNs for sensor nodes without cloud data transfer.

  4. Efficient Edge Deployment of Quantized YOLOv4-Tiny for Aerial Emergency Object Detection on Raspberry Pi 5

    cs.CV 2025-06 reject novelty 3.0 of 10

    A deployment study of quantized YOLOv4-Tiny on Raspberry Pi 5 for aerial emergency detection, undermined by internal numeric inconsistencies and unsupported accuracy claims.

  5. A Survey of TinyML Applications in Beekeeping for Hive Monitoring and Management

    cs.LG 2025-09 conditional novelty 2.0 of 10

    A review of TinyML-based beehive monitoring research across four application areas, with an inventory of datasets and benchmarks and a list of open challenges.

Pith tools