Verifiable and Confidential DNN Inference on Low-End Edge Devices
Pith reviewed 2026-06-27 21:42 UTC · model grok-4.3
The pith
A new middle-privilege runtime on TrustZone-M lets untrusted code run DNN inference while a tiny secure-world piece keeps the model secret and verifies results.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
VECODI introduces SHANGRI-LA, a new execution abstraction on TrustZone-M TEEs that establishes a third runtime environment with privileges strictly between the Secure and Non-Secure Worlds. VECODI leverages SHANGRI-LA to execute untrusted inference code in the Non-Secure World while using minimal application-agnostic Secure-World support to protect model confidentiality and enable verifiability (with respect to proper execution of inference code and model parameters) of inference results.
What carries the argument
SHANGRI-LA, the execution abstraction that creates a third runtime environment with intermediate privileges between the Secure and Non-Secure Worlds on TrustZone-M hardware.
If this is right
- DNN inference on constrained devices can keep models confidential without moving large parts of the model or runtime into the secure world.
- Verification of correct model and code execution becomes possible with only a small, reusable secure-world component.
- Runtime overhead and memory use remain low enough for practical deployment on boards such as the NUCLEO-L552ZE-Q.
- The trusted computing base stays application-agnostic and therefore does not grow when new models or inference tasks are added.
Where Pith is reading between the lines
- The same three-world split could be applied to other embedded workloads that need both confidentiality and execution proofs, such as sensor data processing or control loops.
- If side-channel resistance holds on real silicon, the approach reduces the incentive to enlarge the secure world for every new application.
- Implementations on other TrustZone-M or similar hardware could be checked by measuring whether an attacker can still leak weights or tamper with outputs under the new privilege boundary.
Load-bearing premise
The TrustZone-M hardware together with the SHANGRI-LA layer can be built without opening side-channel leaks or escalation routes that would let an attacker in the non-secure world read the model or forge a valid result.
What would settle it
A successful attack from the non-secure world that either extracts model parameters or produces an undetected incorrect inference result on the NUCLEO-L552ZE-Q board while VECODI is active.
Figures
read the original abstract
Deploying deep neural network (DNN) inference on low-end edge devices raises two key challenges: protecting model confidentiality against a potentially compromised edge system and enabling verifiable inference without incurring prohibitive overhead. Existing approaches either house partial models and inference software within trusted execution environments (TEEs), resulting in high cost and an application-dependent trusted computing base (TCB), or execute in untrusted environments, providing little security. In this work, we present VECODI, a framework for verifiable and confidential DNN inference on constrained edge devices. At its core, VECODI introduces SHANGRI-LA, a new execution abstraction on TrustZone-M TEEs that establishes a third runtime environment with privileges strictly between the Secure and Non-Secure Worlds. VECODI leverages SHANGRI-LA to execute untrusted inference code in the Non-Secure World while using minimal application-agnostic Secure-World support to protect model confidentiality and enable verifiability (with respect to proper execution of inference code and model parameters) of inference results. We realize VECODI on a real-world NUCLEO-L552ZE-Q development board and open-source its prototype. Our results show VECODI's small TCB, memory footprint, and runtime overhead, making it a practical option for secure inference in low-end edge devices.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims to introduce VECODI, a framework for verifiable and confidential DNN inference on low-end edge devices. It proposes SHANGRI-LA, a new execution abstraction on TrustZone-M TEEs that establishes a third runtime environment with privileges between Secure and Non-Secure Worlds. This allows untrusted inference code to run in the Non-Secure World with minimal application-agnostic Secure-World support to protect model confidentiality and enable verifiability of inference results. A prototype is realized on the NUCLEO-L552ZE-Q board, open-sourced, with claims of small TCB, memory footprint, and runtime overhead.
Significance. If the result holds, VECODI would offer a practical solution for secure DNN inference on constrained devices by minimizing the TCB and overhead compared to existing TEE approaches. The open-sourcing of the prototype on real hardware is a positive aspect that supports reproducibility.
major comments (2)
- [Abstract] Abstract: The abstract states that a prototype was realized on real hardware with small TCB and overhead, but provides no quantitative data, threat model details, or evaluation methodology; this prevents assessment of the soundness of the confidentiality and verifiability claims.
- [Abstract] Abstract: The description of SHANGRI-LA does not provide concrete details on the isolation mechanism or enforcement against side channels (timing, cache, DMA, exception paths) or escalation vectors, which is load-bearing for the central claim that untrusted Non-Secure code cannot access model parameters or alter execution state.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback on the abstract. We address the two major comments point by point below.
read point-by-point responses
-
Referee: [Abstract] Abstract: The abstract states that a prototype was realized on real hardware with small TCB and overhead, but provides no quantitative data, threat model details, or evaluation methodology; this prevents assessment of the soundness of the confidentiality and verifiability claims.
Authors: The abstract is intentionally concise as a high-level summary. The threat model is fully specified in Section 3, the evaluation methodology and hardware setup in Section 5, and quantitative results (including TCB size, memory footprint, and runtime overhead with comparisons) appear in Section 6. To address the concern, we will revise the abstract to include a few key quantitative highlights and a one-sentence reference to the threat model. revision: yes
-
Referee: [Abstract] Abstract: The description of SHANGRI-LA does not provide concrete details on the isolation mechanism or enforcement against side channels (timing, cache, DMA, exception paths) or escalation vectors, which is load-bearing for the central claim that untrusted Non-Secure code cannot access model parameters or alter execution state.
Authors: The abstract offers only a summary description. Concrete mechanisms for SHANGRI-LA's intermediate privilege level, including enforcement of isolation, side-channel resistance (timing, cache, DMA), exception handling, and escalation prevention, are detailed in Sections 4.2–4.4 using TrustZone-M hardware features and minimal application-agnostic Secure-World code. We will modestly expand the abstract's SHANGRI-LA sentence to reference these isolation properties. revision: partial
Circularity Check
No circularity: systems construction with no equations or self-referential derivations
full rationale
The paper describes a hardware/software systems architecture (VECODI + SHANGRI-LA on TrustZone-M) whose claims rest on the stated isolation properties of the TEE hardware and the concrete implementation on a NUCLEO board. No equations, fitted parameters, predictions, or mathematical derivations appear in the abstract or described content. No self-citations are invoked as load-bearing uniqueness theorems or ansatzes. The design is externally falsifiable via the open-sourced prototype and hardware measurements, satisfying the criteria for a self-contained, non-circular systems result.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption TrustZone-M hardware provides strong isolation between Secure and Non-Secure Worlds
invented entities (1)
-
SHANGRI-LA
no independent evidence
Reference graph
Works this paper leans on
-
[1]
Edge ai: On-demand ac- celerating deep neural network inference via edge computing,
E. Li, L. Zeng, Z. Zhou, and X. Chen, “Edge ai: On-demand ac- celerating deep neural network inference via edge computing,”IEEE transactions on wireless communications, vol. 19, no. 1, pp. 447–457, 2019
2019
-
[2]
Intellectual property protection for deep learning models: Taxonomy, methods, attacks, and evaluations,
M. Xue, Y . Zhang, J. Wang, and W. Liu, “Intellectual property protection for deep learning models: Taxonomy, methods, attacks, and evaluations,”IEEE Transactions on Artificial Intelligence, vol. 3, no. 6, pp. 908–923, 2021
2021
-
[3]
Stealing machine learning models via prediction{APIs},
F. Tram `er, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, “Stealing machine learning models via prediction{APIs},” in25th USENIX security symposium (USENIX Security 16), pp. 601–618, 2016
2016
-
[4]
Demystifying arm trustzone: A compre- hensive survey,
S. Pinto and N. Santos, “Demystifying arm trustzone: A compre- hensive survey,”ACM computing surveys (CSUR), vol. 51, no. 6, pp. 1–36, 2019
2019
-
[5]
Teeslice: Protecting sensitive neural network models in trusted execution envi- ronments when attackers have pre-trained models,
D. Li, Z. Zhang, M. Yao, Y . Cai, Y . Guo, and X. Chen, “Teeslice: Protecting sensitive neural network models in trusted execution envi- ronments when attackers have pre-trained models,”ACM Transactions on Software Engineering and Methodology, vol. 34, no. 6, pp. 1–49, 2025
2025
-
[6]
Model protection: Real-time privacy-preserving inference service for model privacy at the edge,
J. Hou, H. Liu, Y . Liu, Y . Wang, P.-J. Wan, and X.-Y . Li, “Model protection: Real-time privacy-preserving inference service for model privacy at the edge,”IEEE Transactions on Dependable and Secure Computing, vol. 19, no. 6, pp. 4270–4284, 2021
2021
-
[7]
Darknetz: towards model privacy at the edge using trusted execution environments,
F. Mo, A. S. Shamsabadi, K. Katevas, S. Demetriou, I. Leontiadis, A. Cavallaro, and H. Haddadi, “Darknetz: towards model privacy at the edge using trusted execution environments,” inProceedings of the 18th International Conference on Mobile Systems, Applications, and Services, pp. 161–174, 2020
2020
-
[8]
Shadownet: A secure and efficient on-device model inference system for convolutional neural networks,
Z. Sun, R. Sun, C. Liu, A. R. Chowdhury, L. Lu, and S. Jha, “Shadownet: A secure and efficient on-device model inference system for convolutional neural networks,” in2023 IEEE Symposium on Security and Privacy (SP), pp. 1596–1612, IEEE, 2023
2023
-
[9]
Graviton: Trusted execution environments on{GPUs},
S. V olos, K. Vaswani, and R. Bruno, “Graviton: Trusted execution environments on{GPUs},” in13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pp. 681–696, 2018
2018
-
[10]
Laminator: Verifiable ml property cards using hardware-assisted attestations,
V . Duddu, L. J. Gunn, and N. Asokan, “Laminator: Verifiable ml property cards using hardware-assisted attestations,” inProceedings of the Fifteenth ACM Conference on Data and Application Security and Privacy, pp. 317–328, 2024
2024
-
[11]
PAL*M: Property Attestation for Large Generative Models
P. Chantasantitam, A. I. Caulfield, V . Duddu, L. J. Gunn, and N. Asokan, “Pal* m: Property attestation for large generative models,” arXiv preprint arXiv:2601.16199, 2026
work page internal anchor Pith review Pith/arXiv arXiv 2026
-
[12]
Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware
F. Tramer and D. Boneh, “Slalom: Fast, verifiable and private ex- ecution of neural networks in trusted hardware,”arXiv preprint arXiv:1806.03287, 2018
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[13]
Verisplit: Secure and practical offloading of machine learning infer- ences across iot devices,
H. Zhang, Z. Wang, M. Dhamankar, M. Fredrikson, and Y . Agarwal, “Verisplit: Secure and practical offloading of machine learning infer- ences across iot devices,”arXiv preprint arXiv:2406.00586, 2024
-
[14]
NUCLEO-L552ZE-Q: STM32 Nucleo-144 De- velopment Board with STM32L552ZE MCU
STMicroelectronics, “NUCLEO-L552ZE-Q: STM32 Nucleo-144 De- velopment Board with STM32L552ZE MCU.” https://www.st.com/e n/evaluation-tools/nucleo-l552ze-q.html, 2026. Accessed: 2026-04- 28
2026
-
[15]
VECODI repo
A. Authors, “VECODI repo.” https://anonymous.4open.science/r/Ve CoDI-9BC0, 2026
2026
-
[16]
Attestation mechanisms for trusted execution environ- ments demystified,
J. M ´en´etrey, C. G¨ottel, A. Khurshid, M. Pasin, P. Felber, V . Schiavoni, and S. Raza, “Attestation mechanisms for trusted execution environ- ments demystified,” inIFIP International Conference on Distributed Applications and Interoperable Systems, pp. 95–113, Springer, 2022
2022
-
[17]
{APEX}: A verified architecture for proofs of execution on remote devices under full software compromise,
I. D. O. Nunes, K. Eldefrawy, N. Rattanavipanon, and G. Tsudik, “{APEX}: A verified architecture for proofs of execution on remote devices under full software compromise,” in29th USENIX Security Symposium (USENIX Security 20), pp. 771–788, 2020
2020
-
[18]
Ip protection in tinyml,
J. Wang, Y . Wu, H. Liu, B. Yuan, R. Chamberlain, and N. Zhang, “Ip protection in tinyml,” in2023 60th ACM/IEEE Design Automation Conference (DAC), pp. 1–6, 2023
2023
-
[19]
Smartzone: Runtime support for secure and efficient on-device inference on arm trustzone,
Z. Jian, X. Liu, Q. Dong, L. Cheng, X. Xie, and T. Li, “Smartzone: Runtime support for secure and efficient on-device inference on arm trustzone,”IEEE Transactions on Computers, vol. 74, no. 6, pp. 2144– 2158, 2025
2025
-
[20]
Tee-based trusted storage,
J. Gonzalez and P. Bonnet, “Tee-based trusted storage,” 2014
2014
-
[21]
Physical attack mitigation in trusted firmware-m
T. Ban and D. Hu, “Physical attack mitigation in trusted firmware-m.” https://trustedfirmware-m.readthedocs.io/en/tf-mv2.1.1/design docs/ tfm physical attack mitigation.html, 2024. Accessed: 2026-05-26
2024
-
[22]
Automated side-channel analysis of arm trustzone-m programs,
S. Pouyanrad, F. Alder, and J. T. M ¨uhlberg, “Automated side-channel analysis of arm trustzone-m programs,” inEuropean Symposium on Research in Computer Security, pp. 494–513, Springer, 2024
2024
-
[23]
Oops..! i glitched it again! how to{Multi-Glitch}the{Glitching-Protections}on {ARM}{TrustZone-M},
X. M. Saß, R. Mitev, and A.-R. Sadeghi, “Oops..! i glitched it again! how to{Multi-Glitch}the{Glitching-Protections}on {ARM}{TrustZone-M},” in32nd USENIX Security Symposium (USENIX Security 23), pp. 6239–6256, 2023
2023
-
[24]
Benchmarking ultra-low-powerµnpus,
J. Millar, Y . Huang, S. Sethi, H. Haddadi, and A. Madhavapeddy, “Benchmarking ultra-low-powerµnpus,” inProceedings of the 31st Annual International Conference on Mobile Computing and Network- ing, pp. 1060–1074, 2025
2025
-
[25]
An evaluation of edge tpu accelerators for con- volutional neural networks,
A. Yazdanbakhsh, K. Seshadri, B. Akin, J. Laudon, and R. Narayanaswami, “An evaluation of edge tpu accelerators for con- volutional neural networks,”arXiv preprint arXiv:2102.10423, vol. 1, no. 6, 2021
-
[26]
Corstone SSE-320 with Ethos-U85 Example Subsystem for Ecosystem FVP
Trusted Firmware-M Project, “Corstone SSE-320 with Ethos-U85 Example Subsystem for Ecosystem FVP.” https://tf-m.docs.trustedfir mware.org/en/latest/platform/arm/mps4/corstone320/README.html,
-
[27]
Accessed: 2026-04-28
2026
-
[28]
Secureqnn: Shielding the intel- lectual property of qnns in tinyml systems,
M. Costa, T. Gomes, and S. Pinto, “Secureqnn: Shielding the intel- lectual property of qnns in tinyml systems,”IEEE Internet of Things Journal, vol. 12, no. 21, pp. 44642–44655, 2025
2025
-
[29]
Resnet in Resnet: Generalizing Residual Architectures
S. Targ, D. Almeida, and K. Lyman, “Resnet in resnet: Generalizing residual architectures,”arXiv preprint arXiv:1603.08029, 2016
work page internal anchor Pith review Pith/arXiv arXiv 2016
-
[30]
Energy-efficient ap- proximate edge inference systems,
S. K. Ghosh, A. Raha, and V . Raghunathan, “Energy-efficient ap- proximate edge inference systems,”ACM Transactions on Embedded Computing Systems, vol. 22, no. 4, pp. 1–50, 2023
2023
-
[31]
Ed-res: Split- ting resnet with collaborate distributed inference on edge devices,
X. Liu, Y . Song, Z. Li, J. Chi, L. Jiang, and J. Li, “Ed-res: Split- ting resnet with collaborate distributed inference on edge devices,” inProceedings of the 2025 ACM CoNEXT Workshop Edge-Cloud Collaboration for AI, pp. 38–43, 2025
2025
-
[32]
Tensorflow lite micro: Embedded machine learning for tinyml systems,
R. David, J. Duke, A. Jain, V . Janapa Reddi, N. Jeffries, J. Li, N. Kreeger, I. Nappier, M. Natraj, T. Wang,et al., “Tensorflow lite micro: Embedded machine learning for tinyml systems,”Proceedings of machine learning and systems, vol. 3, pp. 800–811, 2021
2021
-
[33]
Asgard: Protecting on- device deep neural networks with virtualization-based trusted execu- tion environments.,
M. Moon, M. Kim, J. Jung, and D. Song, “Asgard: Protecting on- device deep neural networks with virtualization-based trusted execu- tion environments.,” inNDSS, 2025
2025
-
[34]
Offline model guard: Secure and private ml on mobile devices.,
S. P. Bayerl, T. Frassetto, P. Jauernig, K. Riedhammer, A.-R. Sadeghi, T. Schneider, E. Stapf, and C. Weinert, “Offline model guard: Secure and private ml on mobile devices.,” 2020
2020
-
[35]
Scalable memory protection in the{PENGLAI}enclave,
E. Feng, X. Lu, D. Du, B. Yang, X. Jiang, Y . Xia, B. Zang, and H. Chen, “Scalable memory protection in the{PENGLAI}enclave,” in15th{USENIX}Symposium on Operating Systems Design and Implementation ({OSDI}21), pp. 275–294, 2021
2021
-
[36]
Sanctu- ary: Arming trustzone with user-space enclaves.,
F. Brasser, D. Gens, P. Jauernig, A.-R. Sadeghi, and E. Stapf, “Sanctu- ary: Arming trustzone with user-space enclaves.,” inNDSS, vol. 100, p. 102, 2019
2019
-
[37]
Smart: secure and minimal architecture for (establishing dynamic) root of trust.,
K. Eldefrawy, G. Tsudik, A. Francillon, and D. Perito, “Smart: secure and minimal architecture for (establishing dynamic) root of trust.,” in Ndss, vol. 12, pp. 1–15, 2012
2012
-
[38]
{VRASED}: A verified{Hardware/Software}{Co- Design}for remote attestation,
I. D. O. Nunes, K. Eldefrawy, N. Rattanavipanon, M. Steiner, and G. Tsudik, “{VRASED}: A verified{Hardware/Software}{Co- Design}for remote attestation,” in28th USENIX Security Symposium (USENIX Security 19), pp. 1429–1446, 2019
2019
-
[39]
Sancus: Low-cost trustworthy extensible networked devices with a zero- software trusted computing base,
J. Noorman, P. Agten, W. Daniels, R. Strackx, A. Van Herrewege, C. Huygens, B. Preneel, I. Verbauwhede, and F. Piessens, “Sancus: Low-cost trustworthy extensible networked devices with a zero- software trusted computing base,” in22nd USENIX Security Sym- posium (USENIX Security 13), pp. 479–498, 2013
2013
-
[40]
Toward remotely verifiable software integrity in resource-constrained iot devices,
I. D. O. Nunes, S. Jakkamsetti, N. Rattanavipanon, and G. Tsudik, “Toward remotely verifiable software integrity in resource-constrained iot devices,”IEEE Communications Magazine, vol. 62, no. 7, pp. 58– 64, 2024
2024
-
[41]
Slapp: Poisoning prevention in federated learning and differential privacy via stateful proofs of execution,
N. Rattanavipanon and I. D. O. Nunes, “Slapp: Poisoning prevention in federated learning and differential privacy via stateful proofs of execution,”IEEE Transactions on Information Forensics and Security, 2025
2025
-
[42]
Pearts: Provable execution in real-time embedded systems,
A. J. Neto, N. Rattanavipanon, and I. D. O. Nunes, “Pearts: Provable execution in real-time embedded systems,” in2025 IEEE Symposium on Security and Privacy (SP), pp. 3765–3782, IEEE, 2025
2025
-
[43]
Casu: Compromise avoidance via secure update for low-end embedded systems,
I. De Oliveira Nunes, S. Jakkamsetti, Y . Kim, and G. Tsudik, “Casu: Compromise avoidance via secure update for low-end embedded systems,” inProceedings of the 41st IEEE/ACM International Con- ference on Computer-Aided Design, pp. 1–9, 2022
2022
-
[44]
Pure: Using verified remote attestation to obtain proofs of update, reset and erasure in low-end embedded systems,
I. De Oliveira Nunes, K. Eldefrawy, N. Rattanavipanon, and G. Tsudik, “Pure: Using verified remote attestation to obtain proofs of update, reset and erasure in low-end embedded systems,” in2019 IEEE/ACM International Conference on Computer-Aided Design (IC- CAD), pp. 1–8, IEEE, 2019
2019
-
[45]
Privacy- from-birth: Protecting sensed data from malicious sensors with versa,
I. D. O. Nunes, S. Hwang, S. Jakkamsetti, and G. Tsudik, “Privacy- from-birth: Protecting sensed data from malicious sensors with versa,” in2022 IEEE Symposium on Security and Privacy (SP), pp. 2413– 2429, IEEE, 2022
2022
-
[46]
The sa4p frame- work: Sensing and actuation as a privilege,
P. De Vaere, F. St ¨oger, A. Perrig, and G. Tsudik, “The sa4p frame- work: Sensing and actuation as a privilege,” inProceedings of the 19th ACM Asia Conference on Computer and Communications Security, pp. 873–885, 2024
2024
-
[47]
Untrusted code compartmentalization for bare metal embedded devices,
L. Tyler and I. D. O. Nunes, “Untrusted code compartmentalization for bare metal embedded devices,”IEEE Transactions on Computer- Aided Design of Integrated Circuits and Systems, vol. 43, no. 11, pp. 3419–3430, 2024
2024
-
[48]
Trusted Firmware-M (TF-M)
Trusted Firmware Project, “Trusted Firmware-M (TF-M).” https://ww w.trustedfirmware.org/projects/tf-m/, 2026. Accessed: 2026-04-28
2026
-
[49]
CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
L. Lai, N. Suda, and V . Chandra, “Cmsis-nn: Efficient neural network kernels for arm cortex-m cpus,”arXiv preprint arXiv:1801.06601, 2018
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[50]
Asap: reconciling asynchronous real-time operations and proofs of execution in simple embedded systems,
A. Caulfield, N. Rattanavipanon, and I. De Oliveira Nunes, “Asap: reconciling asynchronous real-time operations and proofs of execution in simple embedded systems,” inProceedings of the 59th ACM/IEEE Design Automation Conference, pp. 721–726, 2022. Appendix A. Implementation Details We implement VECODI on a NUCLEO-L552ZE-Q de- velopment board [14], which ...
2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.