Confidential Deep Learning: Executing Proprietary Models on Untrusted Devices

Ioannis Kyriazis; Michelle Cheng; Peter M. VanNostrand; Robert J. Walls; Tian Guo

REVIEW 3 major objections 7 minor 15 references

Confidential Deep Learning: Executing Proprietary Models on Untrusted Devices

T0 review · 3 major / 7 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that ARM TrustZone, the security hardware already in most phones, can keep a proprietary neural network confidential on an untrusted device by partitioning the model into pieces that fit in secure memory, at an estimated…

desk verdict A clear, honest design exploration of running CNN inference inside ARM TrustZone, with useful microbenchmarks and a 4x overhead estimate that is actually a lower bound because it omits the TEE compute slowdown the paper itself warns about. read the letter →

arxiv 1908.10730 v1 pith:5C4D3ZDH submitted 2019-08-28 cs.CR cs.LG

Peter M. VanNostrand , Ioannis Kyriazis , Michelle Cheng , Tian Guo , Robert J. Walls This is my paper · ORCID

classification cs.CRcs.LG

keywords ARMTrustZonetrustedexecutionenvironmentconfidentialdeeplearningmodelpartitioningconvolutionalneuralnetworksOP-TEEintellectualpropertyprotectionuntrusteddevices

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a phone's ARM TrustZone hardware can keep a proprietary neural network confidential even when the phone's operating system is controlled by an attacker. The authors propose splitting a convolutional network into pieces that fit in the small secure-world memory, and they estimate the cost of the simplest scheme, layer-based partitioning, at about four times the normal inference time. They identify context switching and model decryption as the dominant overheads, measuring 75.1 microseconds per switch and 163.7 nanoseconds per decrypted block on a Raspberry Pi 3. A sympathetic reader would take the central claim to be that commodity trusted hardware, not specialized secure servers, is a viable foundation for protecting model intellectual property on end-user devices.

What carries the argument

The central mechanism is ARM TrustZone's division of a processor into a Secure world and a Normal world, with OP-TEE as the trusted execution environment that runs trusted applications in the Secure world. The argument is carried by the partitioning of a convolutional neural network into layers (or sub-layers, or branches) so that each piece's weights and activations fit within the roughly 7 MB secure-world memory limit; each partition's weights are stored encrypted and decrypted only inside the Secure world. The overhead model combines two measured quantities—the 75.1 microsecond cost of a Normal-to-Secure world context switch and the 163.7 nanosecond per-block decryption cost—into the formula $$t_{\text{cs}} \cdot 11 \cdot 2 + t_{d} \cdot 191{,}790$$ for LeNet, yielding the 4x estimate.

What would settle it

A concrete test: implement layer-based partitioning for LeNet under OP-TEE on a Raspberry Pi 3, run real inference with weights decrypted in the Secure world, and measure the total time. If the measured time is close to the predicted 41.28 ms (8.23 ms baseline plus 33.05 ms overhead), the claim holds; if total time is substantially higher because the arithmetic itself slows down in the Secure world, the paper's overhead model is wrong.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that confidential deep learning—executing a proprietary model on a device whose owner controls the operating system and memory—can be achieved by running inference inside an ARM TrustZone secure world, and that the main engineering obstacle is not security but memory. Since the OP-TEE environment limits trusted applications to roughly 7 MB of secure memory while common convolutional networks need hundreds of megabytes, the paper introduces three partitioning strategies: layer-based, sub-layer, and branched-execution. For the simplest strategy on an MNIST LeNet model, it estimates an added 33.05 ms of context-switch and decryption time on top of an 8.23 ms baseline, about a 4x overhead. The paper does not claim a full implementation; it characterizes the design space and identifies root-of-trust, key management, remote attestation, and side-channel defenses as open problems before deployment.

Load-bearing premise

The 4x overhead number assumes that doing the neural network arithmetic inside the Secure world costs the same as doing it in the Normal world, with only context switches and decryption adding time; the paper itself notes that TEE operations typically run orders of magnitude slower, so if real Secure-world compute is slow, the overhead is understated.

Editorial extensions

If this is right

If the estimate holds, a model provider can ship an encrypted model that runs on a user's phone without ever exposing weights or architecture to the phone's operating system.
Layer-based partitioning makes the largest single layer the binding constraint on secure memory, so models designed with smaller layers become more practical to protect.
Sub-layer partitioning extends protection to models with large layers at the cost of repeatedly decrypting intermediate activations, trading compute for memory.
Branched-execution partitioning suggests a hybrid deployment where the large early layers run in the Normal world and only the smaller later branches run in the Secure world, reducing secure memory pressure.
Because the overhead formula is linear in layer count and model size, the 4x figure would grow for deeper or wider models, making partitioning strategy a first-class design choice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same partitioning logic should apply to other trusted execution environments with small secure memories, not just TrustZone, so the design space generalizes beyond ARM hardware.
A natural testable extension is to build the actual layer-partitioned TrustZone implementation the paper sketches; the 4x overhead is an estimate, not a measured end-to-end result.
The branched-execution scheme suggests a broader principle: only the layers whose weights or intermediate features are most proprietary need to be secured, so providers could trade some confidentiality for speed by choosing which partitions to protect.
If secure-world compute turns out to be significantly slower, the overhead could be dominated by arithmetic rather than context switches, which would shift the design toward moving more computation into the Normal world.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The thing to know: this is a short design-exploration paper, not a built system. The genuinely new pieces are the OP-TEE microbenchmarks on the Raspberry Pi 3—75.1 us context switch, 163.7 ns per block decryption—and a back-of-the-envelope 4x overhead estimate for a layer-partitioned LeNet. That estimate is arithmetic over measured costs, not a demonstrated end-to-end result. The paper does not overclaim much: it labels the 4x as an estimate, says the Pi lacks the TrustZone Address Space Controller, and warns in Section 1 that TEE operations usually run orders of magnitude slower.

What the paper does well: it lays out the confidential deep learning problem clearly, defines a sensible threat model that excludes side channels and hardware attacks, and sketches three partitioning schemes with explicit memory tradeoffs. The related work is handled fairly, crediting DarkNetP and Sutton et al. for the partitioning ideas and positioning this as an extension, not a from-scratch invention. The writing is direct and the limitations are acknowledged rather than buried.

Soft spots, in proportion: the biggest quantitative issue is that the 4x overhead omits the slowdown of actually computing inside the Secure world. Since Section 1 says TEE arithmetic can be orders of magnitude slower, the 4x is a lower bound, possibly a very loose one. The microbenchmarks are averages with no variance or trial counts, which makes the numbers hard to trust at face value. And there is no end-to-end run of a partitioned network, so the word "demonstrate" in the abstract is stronger than what the paper delivers. The sub-layer and branched schemes are sketched but not evaluated—appropriate for a problem-framing paper, but the title promises a bit more than the content.

None of this sinks the central argument. The claim that TrustZone plus OP-TEE can isolate model weights and architecture from a hostile OS is architecturally sound, and the measured costs are plausible. The overhead model is incomplete, not circular. The paper is honest about its own gaps.

Who should read it: anyone working on TEE-based machine learning or secure edge inference will find the microbenchmarks and the partitioning taxonomy useful. It deserves a serious referee for a workshop or a systems/security venue that values design-space exploration, though it would need revision—variance reporting, an end-to-end measurement, and a corrected overhead equation—before acceptance as a full paper. My call: engage with it, but treat the 4x as an order-of-magnitude estimate, not a measured result.

Referee Report

3 major / 7 minor

Summary. The paper addresses the problem of keeping proprietary deep learning models confidential when inference is executed on end-user devices controlled by an untrusted operating system. It proposes using ARM TrustZone with OP-TEE as a trusted execution environment and explores three CNN partitioning schemes that shrink the secure-world memory footprint: layer-based partitioning, sub-layer partitioning, and branched-execution partitioning. For layer-based partitioning, the authors measure on a Raspberry Pi 3 a context-switch time of 75.1 microseconds and a decryption cost of 163.7 nanoseconds per block, and combine these with a DarkNet MNIST LeNet baseline of 8.23 milliseconds to estimate an additional 33.05 milliseconds of overhead, or roughly 4x. No end-to-end implementation is presented; the paper is framed primarily as a design-space exploration and feasibility analysis.

Significance. If the overhead estimate is reliable, the paper offers a plausible path toward confidential inference on commodity ARM hardware and a useful taxonomy of partitioning strategies. The paper's strengths include a clean threat model, an explicit acknowledgment of TEE limitations (memory, system-call dependence, missing TZASC on the test platform), and a transparent arithmetic decomposition of overhead from independently measured components. The main weakness is that the headline 4x overhead is computed under the assumption that secure-world arithmetic costs the same as normal-world arithmetic, which is contradicted by the paper's own Section 1 statement that TEE operations can run orders of magnitude slower; the estimate is therefore an optimistic lower bound rather than a validated total-overhead number. The lack of trial counts, variance, and an end-to-end measurement further limits the empirical contribution, but these issues do not invalidate the central design proposal.

major comments (3)

[Section 5, overhead formula] The overhead model given by tcs*11*2 + td*191,790 assumes that per-layer inference arithmetic inside the Secure world costs exactly the same as in the Normal world. Section 1 states that operations in a TEE typically run orders of magnitude slower than in the untrusted environment. Without measuring or bounding that compute slowdown, the estimated 33.05 ms (4x) overhead is a lower bound on the true total cost, not a reliable estimate of the total overhead. The authors should either measure secure-world arithmetic on a realistic TrustZone platform or explicitly rephrase the claim as context-switch and decryption overhead only, and qualify the total-cost implication.
[Section 5 vs. Section 2.2] The microbenchmarks in Section 5 are collected on a Raspberry Pi 3, which Section 2.2 states lacks the TrustZone Address Space Controller (the peripheral responsible for partitioning Normal and Secure world memory) and hardware modules for secure key storage. The paper does not discuss whether the absence of TZASC affects context-switch latency or the decryption measurements, nor how the security guarantees of the threat model can be validated on this platform. This omission is material because the measured values anchor the central performance estimate and the platform lacks a component central to the confidentiality claim.
[Section 5, measurement reporting] The paper reports only average values for context-switch time (75.1 us), decryption cost (163.7 ns per block), and the DarkNet baseline (8.23 ms), with no trial count, standard deviation, or description of the measurement procedure. Without these statistics, it is impossible to assess the statistical reliability of the resulting 4x overhead estimate or to reproduce the measurements. The authors should report the number of trials and the variance or confidence intervals for each microbenchmark.

minor comments (7)

[Section 5, paragraph 1] The text says 'LetNet has a total of 11 layers' but the correct model name is 'LeNet'; please fix the typo.
[Section 5, paragraph 2] The phrase 'from the Normal world to the Secure world (or vise versa)' should be 'vice versa'.
[Section 5, paragraph 4] The sentence 'the weights file contains 191,124 bytes and an additional 666 is needed for the model configuration file' should state '666 bytes' explicitly.
[Section 6, third paragraph] The text credits 'Fan and Haddadi' for the DarkNetP work, but reference [8] lists Fan Mo et al.; please adjust the in-text citation to match the actual authors.
[Section 4.1, neuron calculation] The expression 'w1a1 + w2a2 + ... +wnan +b' would benefit from proper mathematical formatting (subscripts) and a clear definition of the variables to avoid ambiguity.
[Figure 1] The text refers to Figure 1a, 1b, and 1c, but the figure caption uses a single combined caption; please ensure the subfigure labels are consistent and clearly visible.
[Section 6, reference [8]] The reference entry and the textual description appear to describe different works: the text says 'DarkNetP' and 'Fan and Haddadi', while reference [8] is 'Towards Characterizing and Limiting Information Exposure in DNN Layers'. Please clarify the citation and its relation to the described approach.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the overhead estimate is arithmetic over independently measured costs, and no load-bearing self-citation is present.

full rationale

The paper's central claim—that ARM TrustZone/OP-TEE can provide a foundation for confidential CNN inference via layer-based partitioning with an estimated roughly 4x overhead—is not derived from its own conclusions. The Section 5 overhead estimate is transparent arithmetic over independently measured quantities: tcs = 75.1 microseconds average context-switch time (measured with an instrumented client application), td = 163.7 nanoseconds per-block decryption cost (measured in a trusted application), a DarkNet Normal-world baseline of 8.23 ms for MNIST LeNet, a layer count of 11, and a model size of 191,790 bytes. None of these constants is fit to the 33.05 ms figure, nor does the paper define the constants in terms of the claimed overhead. The partitioning designs are architectural proposals rather than predictions that reduce to their inputs, and the citations to DarkNetP, Sutton et al., and Tramer/Boneh are external prior work that is not used as unverified self-support. The paper explicitly acknowledges limitations (the 7 MB OP-TEE memory limit, the missing TrustZone Address Space Controller on the Raspberry Pi 3, no defense against side channels, and no hardware root of trust) that affect feasibility but do not create circularity. The reader's flagged omission of in-TEE compute slowdown is a correctness/accuracy limitation of the overhead model, not a circular derivation, because the paper presents the estimate as a back-of-the-envelope calculation from measured context-switch and decryption costs. No step in the derivation chain reduces to its own inputs by construction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its central claim rests on standard TrustZone/OP-TEE security properties and on the unverified modeling assumption that in-TEE compute costs match Normal-world costs.

assumptions (5)

domain assumption TrustZone physically isolates the Secure world so Normal-world software cannot read or modify Secure-world memory and peripherals.
The entire confidentiality argument depends on this hardware guarantee; stated in Section 2.1.
domain assumption OP-TEE's trusted applications can be signed and verified, and the TEE prevents execution of modified applications.
Section 2.2; the threat model relies on this for model integrity.
domain assumption The Secure world is limited to roughly 7 MB of memory, motivating partitioning.
Section 4.1; if this limit is wrong, the partitioning schemes are solving a different problem.
domain assumption An encrypted model file can be decrypted inside the TEE without leaking keys or plaintext to the Normal world.
Section 4.2; key management is deferred to future work, but the scheme assumes it can be done.
ad hoc to paper Per-layer compute cost in the Secure world equals Normal-world cost for the overhead model.
Section 5 includes only context-switch and decryption costs, ignoring the slowdown acknowledged in Section 1.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Confidential Deep Learning: Executing Proprietary Models on Untrusted Devices." pith.science (2026). https://pith.science/paper/5C4D3ZDH

@misc{pith2026190810730,
  author       = {Pith},
  title        = {Pith review of: Confidential Deep Learning: Executing Proprietary Models on Untrusted Devices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5C4D3ZDH}},
  note         = {Machine review of arXiv:1908.10730}
}

read the original abstract

Performing deep learning on end-user devices provides fast offline inference results and can help protect the user's privacy. However, running models on untrusted client devices reveals model information which may be proprietary, i.e., the operating system or other applications on end-user devices may be manipulated to copy and redistribute this information, infringing on the model provider's intellectual property. We propose the use of ARM TrustZone, a hardware-based security feature present in most phones, to confidentially run a proprietary model on an untrusted end-user device. We explore the limitations and design challenges of using TrustZone and examine potential approaches for confidential deep learning within this environment. Of particular interest is providing robust protection of proprietary model information while minimizing total performance overhead.

Figures

Figures reproduced from arXiv: 1908.10730 by the authors.

**Figure 1.** Proposed Partition Schema for CNNs to Support Confidential Deep Learning platforms where memory limitations are critical, sub-layer partitioning extends layer based partitioning to further reduce memory usage. However, as the size of layer outputs approaches the available memory size, we risk significant performance overhead. If model performance is paramount, branched-execution partitioning has the potential to gr… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 11 canonical work pages

[1]

Simone Bianco, Remi Cadene, Luigi Celona, and Paolo Napoletano

work page
[2]

Stephen Checkoway and Hovav Shacham. 2013. Iago Attacks: Why the System Call API is a Bad Untrusted RPC Interface. SIGPLAN Not. 48, 4 (March 2013), 253–264

work page 2013
[3]

ARM Holdings. 2009. Building a Secure System using TrustZone Tech- nology. Technical Report

work page 2009
[4]

Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. 2017. Neurosurgeon: Collabora- tive Intelligence Between the Cloud and Mobile Edge. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’17)

work page 2017
[5]

Seunghak Lee, Jin Kyu Kim, Xun Zheng, Qirong Ho, Garth A Gib- son, and Eric P Xing. 2014. On Model Parallelization and Scheduling Strategies for Distributed Machine Learning. In Advances in Neural Information Processing Systems 27. Curran Associates, Inc., 2834–2842

work page 2014
[6]

Accessed on 2019

Linaro. Accessed on 2019. Open Portable Trusted Execution Environ- ment. https://www.op-tee.org

work page 2019
[7]

Frank McKeen, Ilya Alexandrovich, Alex Berenzon, Carlos V Rozas, Hisham Shafi, Vedvyas Shanbhogue, and Uday R Savagaonkar. 2013. Innovative instructions and software model for isolated execution. In Proceedings of the 2nd International Workshop on Hardware and Architectural Support for Security and Privacy . ACM, 10

work page 2013
[8]

Fan Mo, Ali Shahin Shamsabadi, Kleomenis Katevas, Andrea Cavallaro, and Hamed Haddadi. 2019. Towards Characterizing and Limiting Information Exposure in DNN Layers. arXiv:1907.06034 (2019)

work page Pith review arXiv 2019

Show all 15 references

[9]

Olga Ohrimenko, Felix Schuster, Cédric Fournet, Aastha Mehta, Se- bastian Nowozin, Kapil Vaswani, and Manuel Costa. 2016. Oblivious Multi-Party Machine Learning on Trusted Processors.. In USENIX Security Symposium. 619–636

2016
[10]

2013–2016

Joseph Redmon. 2013–2016. Darknet: Open Source Neural Networks in C. http://pjreddie.com/darknet/

2013
[11]

D. P. Sutton, M. C. Carlisle, T. A. Sarmiento, and L. C. Baird. 2009. Partitioned neural networks. In 2009 International Joint Conference on Neural Networks. 3032–3037

2009
[12]

Surat Teerapittayanon, Bradley McDanel, and H. T. Kung. 2017. BranchyNet: Fast Inference via Early Exiting from Deep Neural Net- works. arXiv:1709.01686 (2017)

2017 arXiv
[13]

Shruti Tople, Karan Grover, Shweta Shinde, Ranjita Bhagwan, and Ramachandran Ramjee. 2018. Privado: Practical and Secure DNN Inference. arXiv preprint arXiv:1810.00602 (2018)

2018 arXiv
[14]

Florian Tramer and Dan Boneh. 2018. Slalom: Fast, Verifiable and Private Execution of Neural Networks in Trusted Hardware. arXiv:1806.03287 (2018). 6

2018 arXiv
[2018]

IEEE Access 6 (2018), 64270–64277

Benchmark Analysis of Representative Deep Neural Network Architectures. IEEE Access 6 (2018), 64270–64277

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Simone Bianco, Remi Cadene, Luigi Celona, and Paolo Napoletano

work page

[2] [2]

Stephen Checkoway and Hovav Shacham. 2013. Iago Attacks: Why the System Call API is a Bad Untrusted RPC Interface. SIGPLAN Not. 48, 4 (March 2013), 253–264

work page 2013

[3] [3]

ARM Holdings. 2009. Building a Secure System using TrustZone Tech- nology. Technical Report

work page 2009

[4] [4]

Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. 2017. Neurosurgeon: Collabora- tive Intelligence Between the Cloud and Mobile Edge. In Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS ’17)

work page 2017

[5] [5]

Seunghak Lee, Jin Kyu Kim, Xun Zheng, Qirong Ho, Garth A Gib- son, and Eric P Xing. 2014. On Model Parallelization and Scheduling Strategies for Distributed Machine Learning. In Advances in Neural Information Processing Systems 27. Curran Associates, Inc., 2834–2842

work page 2014

[6] [6]

Accessed on 2019

Linaro. Accessed on 2019. Open Portable Trusted Execution Environ- ment. https://www.op-tee.org

work page 2019

[7] [7]

Frank McKeen, Ilya Alexandrovich, Alex Berenzon, Carlos V Rozas, Hisham Shafi, Vedvyas Shanbhogue, and Uday R Savagaonkar. 2013. Innovative instructions and software model for isolated execution. In Proceedings of the 2nd International Workshop on Hardware and Architectural Support for Security and Privacy . ACM, 10

work page 2013

[8] [8]

Fan Mo, Ali Shahin Shamsabadi, Kleomenis Katevas, Andrea Cavallaro, and Hamed Haddadi. 2019. Towards Characterizing and Limiting Information Exposure in DNN Layers. arXiv:1907.06034 (2019)

work page Pith review arXiv 2019