Pith. sign in

REVIEW 3 major objections 6 minor 26 references

Distilled Large Language Model in Confidential Computing Environment for System-on-Chip Design

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Pith's one-line read This paper claims that for distilled LLMs below roughly 3 billion parameters, Intel TDX confidential execution runs inference faster than an unsecured CPU-only setup, while keeping model and data inside an encrypted enclave.

desk verdict TDX-beats-CPU claim is unconvincing due to unmatched baseline resources; the quantization data is the solid part. read the letter →

arxiv 2507.16226 v1 pith:KVBC453G submitted 2025-07-22 cs.AI cs.CR

classification cs.AIcs.CR
keywords confidentialcomputingIntelTDXtrustedexecutionenvironmentdistilledlargelanguagemodelsmodelquantizationsystem-on-chipdesignLLMinferencebenchmarkDeepSeek
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that lightweight distilled LLMs can be run inside a confidential computing enclave without sacrificing, and for small models even improving on, ordinary CPU performance, making secure LLM use practical for System-on-Chip design. It benchmarks DeepSeek-R1, Llama 3.2, and Gemma 2 models in three environments: Intel TDX, CPU-only, and CPU-GPU. The key results are that TDX reaches about 25.67 tokens/s for DeepSeek-1.5B, more than double the CPU-only rate, and that Q4/Q8 quantization speeds inference up to 3x over FP16 while cutting model storage by roughly half to two-thirds. It also validates this on HWSec models fine-tuned for hardware security and design tasks. If correct, it would mean secure enclave inference is feasible for resource-constrained semiconductor CAD workflows without GPU support.

What carries the argument

The mechanism is Intel TDX, a VM-level trusted execution environment that allocates a private memory region to a secure Trust Domain and encrypts data crossing the enclave boundary. Inside that enclave, the paper relies on two properties of small models: distillation (DeepSeek-R1's 1.5B/7B/14B variants keep reasoning ability at low parameter counts) and post-training quantization (Q4/Q8 integer weights). These properties keep the whole model and workload resident in TDX private memory, avoiding the memory-swap overhead that larger models trigger, so the TDX CPU configuration can run small models faster than the CPU-only baseline the authors measured.

What would settle it

Run the same DeepSeek-R1-1.5B model in a CPU-only container configured with the same 62 cores and 510 GB as the TDX VM, pinned to the same NUMA nodes and free of other host workloads; if TDX no longer achieves about 25.67 tokens/s against a CPU-only 10.25, the claimed enclave speedup collapses.

Watch

Extended reading notes

Core claim

The central discovery, as the authors state it, is that Intel TDX's CPU execution can beat plain CPU execution for small distilled models: DeepSeek-R1-1.5B runs at 25.67 tokens/s in the TDX VM versus 10.25 tokens/s CPU-only, and similar advantages appear for Llama3.2-1B/3B and Gemma2-2B. The advantage fades with model size, DeepSeek-7B is slower in TDX (6.42 vs 8.53 tokens/s), so the benefit is specific to sub-3B models. They also find that quantization compounds the effect: Q4 DeepSeek roughly triples tokens/s relative to FP16, and Q4 storage is about 30% of FP16. These patterns hold on HWSec models fine-tuned for hardware security, where TDX exceeds CPU-only in every case they report.

Load-bearing premise

The speed comparison assumes the CPU-only baseline is a fair, resource-equivalent control, but the paper does not state its core count, memory limit, or isolation from other host workloads, so the TDX advantage could be an artifact of a weaker baseline.

Editorial extensions

If this is right

  • For sub-3B distilled models, confidential execution is not a performance tax: TDX can roughly double CPU-only throughput, so security can be added at no speed penalty.
  • Q4/Q8 quantization is the practical deployment choice in TDX, combining up to 3x speedup over FP16 with storage of roughly 30-50% of FP16, which fits enclave memory limits.
  • The TDX advantage disappears at 7B and 14B scales, where memory bandwidth dominates, so larger confidential models require GPU-in-enclave support that current TDX does not offer.
  • GPU acceleration remains 8-14x faster than TDX only because it moves data out of the enclave in plaintext; the paper's central trade-off is security versus speed.
  • Fine-tuned HWSec hardware-security models follow the same trend, indicating that domain-adapted confidential LLM deployment in SoC verification is feasible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The TDX-versus-CPU gap is most plausibly a configuration effect (62 cores and 510 GB in the TDX VM versus an unspecified host container) rather than an effect of the security machinery itself; a matched-resource rerun would separate these causes.
  • If the speedup survives controlled reruns, confidential cloud providers could offer secure small-model inference for latency-tolerant CAD workloads without GPU allocation, lowering cost.
  • The same benchmark could be run on AMD SEV and ARM TrustZone to test whether the small-model advantage is generic to VM-level TEEs or specific to TDX.
  • A direct extension would profile memory bandwidth and cache behavior of the 1.5B model inside TDX to identify whether the gain comes from ISA optimizations or from reduced memory pressure.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper empirically evaluates lightweight LLM inference in an Intel TDX confidential-computing environment versus CPU-only and CPU-GPU baselines, using Ollama to measure tokens/s for DeepSeek-R1 distilled variants (1.5B, 7B, 14B), Llama 3.2, Gemma 2, and several HWSeC fine-tuned models. The authors report that for sub-3B models TDX achieves higher throughput than CPU-only execution (e.g., DeepSeek 1.5B at 25.67 vs 10.25 tokens/s), that quantization Q4/Q8 gives up to 3x speedup over FP16, and that these trends hold for domain-specific HWSeC models. They conclude that lightweight distilled LLMs can be deployed efficiently and securely in confidential computing for SoC design workflows.

Significance. If the throughput comparison were properly controlled, this would be a useful empirical data point: it would show that confidential-execution overhead need not make small-model inference dramatically slower than an insecure CPU baseline, and the quantization results would be practically relevant for memory-constrained TEE deployments. The paper also addresses a topical gap, since most TEE-LLM work focuses on SGX or large models, whereas TDX and distilled sub-3B models are less studied. The strengths are the breadth of models tested (DeepSeek, Llama, Gemma, HWSeC domain models) and the explicit reporting of raw tokens/s values and model storage sizes. However, the headline TDX-beats-CPU claim is not established because the CPU-only baseline is not resource-equivalent, and the HWSeC validation claims accuracy improvements from a table that contains only throughput. The paper would need additional experiments and careful re-scoping before its central conclusions can be accepted.

major comments (3)
  1. [Section IV-A, Table I, Section V-A] The central claim that TDX outperforms CPU-only execution for sub-3B models is confounded by unequal resources. Table I specifies the TDX configuration as "62 CPU cores and 510GB DRAM" but describes CPU Only only as "Runs on the host machine without GPU acceleration enabled," with no vCPU count, memory limit, NUMA placement, container constraints, or host-isolation details in Section IV-A. The 25.67 vs 10.25 tokens/s comparison for DeepSeek 1.5B in Table II, and its attribution to "TDX's optimized CPU ISA configuration" in Section V-A, therefore conflates TDX-specific effects with core/memory allocation and host contention. The reversal at 7B (CPU-only 8.53 vs TDX 6.42) is consistent with a resource-allocation explanation rather than a TDX ISA effect. The authors should rerun with a CPU-only container constrained to the same cores, memory, and Ollama thread settings as the TDX VM, and should report whether the advantage survives.
  2. [Section V-C, Table III] The text states that "The results shown in Table III confirm that these domain-specific HWSeC models achieve improved accuracy and usability in chip design and verification benchmarks, outperforming general LLMs on hardware-centric tasks." Table III contains only throughput in tokens/s and throughput ratios; no accuracy metrics, no comparison to general LLMs on hardware benchmarks, and no benchmark scores are reported. This sentence is unsupported by the presented data. Either add the accuracy results or rephrase the claim to refer only to throughput.
  3. [Section V-A, Tables II and III] All performance numbers are reported as single point values from Ollama's built-in logging, with no repeated runs, error bars, or statistical tests. Because the paper's headline conclusions are ratios (e.g., "more than twice," "9x," "20x"), single measurements do not establish that the observed differences are reliable. At minimum, the authors should report multiple repetitions with standard deviations and a simple significance test for the key TDX-vs-CPU comparisons.
minor comments (6)
  1. [Abstract and Section I contributions] The abstract and the third contribution bullet state that LLM performance in TEEs exceeds CPU-only execution without qualification, but the paper's own results show this holds only for sub-3B models; the 7B and 14B cases reverse or narrow the gap. Please qualify the claim to the sub-3B distilled-model regime.
  2. [Section I contributions and References [12]] The contribution bullet claims "the first evaluation of a distilled LLM within a TEE," yet reference [12] is a prior evaluation of DeepSeek in a confidential computing environment by the same authors. Please reconcile this claim, for example by clarifying what is new relative to [12].
  3. [Section V-A, Figure 3] Figure 3 is described as performance "ratioed to the CPU performance," but the caption does not define which CPU baseline is used (CPU-only, TDX, or normalized value) or how the ratio is computed. Please define the normalization explicitly.
  4. [Section V-D] The statement "8-bit DeepSeek-14B model performs better than 16-bit DeepSeek-7B" is confusing, and the following sentence "the DeepSeek model sees only a slight gain (2.15 to 2.17 tokens/s)" needs context: which models and quantization levels are being compared, and at which parameter sizes? Please state the comparison clearly with the corresponding numbers.
  5. [Tables II and III] The column headers in Tables II and III contain typesetting artifacts such as "GP U T DX" and "CP U T DX" that should be reformatted as GPU/TDX and CPU/TDX, and the dimensionless ratio columns should be defined in the captions.
  6. [Throughout] The domain-specific model family is referred to inconsistently as "HWSeC," "HWSec," and "KSU HWSEC"; please choose one consistent notation. Also, no code or data availability statement is provided, which limits reproducibility of the benchmark.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the benchmark results are measured against external models and workloads, and the only self-citation is not load-bearing.

full rationale

The paper is an empirical measurement study. Its headline claims, such as the TDX environment reaching 25.67 tokens/s for DeepSeek-1.5B versus 10.25 tokens/s for CPU-only (Table II, Section V-A), are direct readings from Ollama's logging for external models and workloads; no derived constant or fitted parameter is renamed as a prediction. The quantization speedups and storage ratios (Section V-D, Table IV) are arithmetic consequences of reduced precision and reported measured throughput, not fitted results. The only self-citation, reference [12], is a prior evaluation of DeepSeek in a confidential environment; it is cited for background motivation and does not supply any equation, fitted value, or benchmark number used in the present measurements. The paper's CPU-only baseline is underspecified relative to the TDX VM (Table I gives TDX '62 CPU cores and 510GB DRAM' but describes CPU Only only as 'Runs on the host machine without GPU acceleration enabled'), which threatens the causal attribution of the reported speedup to TDX-specific optimizations. That is an experimental-control weakness, not a circular-reasoning weakness. Likewise, the paper's own acknowledged limitations about plaintext CPU-GPU communication and lack of full GPU support inside TDX are substantive engineering constraints, not circular steps. No claim in the derivation chain reduces by construction to its inputs, so the circularity score is 0.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on measurement choices rather than on new theory. There are no fitted mathematical constants or invented entities. The load-bearing choices are the TDX resource allocation and the implicit assumption that the CPU-only baseline is resource-equivalent, which is not documented. The accuracy-preservation of quantization and the representativeness of HWSeC tasks are domain assumptions that the paper asserts without reporting accuracy checks.

free parameters (1)
  • TDX VM resource allocation (62 vCPUs, 510GB DRAM) = 62 CPU cores, 510 GB DRAM
    Hand-set for the TDX environment in Table I; the CPU-only baseline does not state an equivalent allocation, so the central TDX-vs-CPU comparison depends on this choice.
assumptions (4)
  • domain assumption Ollama's built-in logging yields accurate, comparable tokens/s measurements across TDX, CPU-only, and GPU-CPU environments.
    Section IV-A states all performance data was gathered using Ollama's logging; the comparison assumes this is a faithful proxy for inference performance.
  • ad hoc to paper The CPU-only Docker baseline is an appropriate performance reference even though its resource limits are not specified.
    Table I defines CPU Only only by absence of GPU; Section V-A attributes TDX's speedup to CPU optimizations, implicitly assuming the baseline was equivalent in cores and memory.
  • domain assumption Q4/Q8 quantization in Ollama preserves the models' task behavior, making tokens/s the relevant comparison.
    Section III-C motivates quantization by minimal accuracy loss, but the paper reports no accuracy measurements on the quantized models.
  • domain assumption HWSeC models are representative of confidential SoC design workloads.
    Section IV-B says the K-state HWSec dataset benchmarks Verilog bug detection and RTL generation; the paper does not validate task accuracy, only throughput.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Distilled Large Language Model in Confidential Computing Environment for System-on-Chip Design." pith.science (2026). https://pith.science/paper/KVBC453G

@misc{pith2026250716226,
  author       = {Pith},
  title        = {Pith review of: Distilled Large Language Model in Confidential Computing Environment for System-on-Chip Design},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KVBC453G}},
  note         = {Machine review of arXiv:2507.16226}
}
read the original abstract

Large Language Models (LLMs) are increasingly used in circuit design tasks and have typically undergone multiple rounds of training. Both the trained models and their associated training data are considered confidential intellectual property (IP) and must be protected from exposure. Confidential Computing offers a promising solution to protect data and models through Trusted Execution Environments (TEEs). However, existing TEE implementations are not designed to support the resource-intensive nature of LLMs efficiently. In this work, we first present a comprehensive evaluation of the LLMs within a TEE-enabled confidential computing environment, specifically utilizing Intel Trust Domain Extensions (TDX). We constructed experiments on three environments: TEE-based, CPU-only, and CPU-GPU hybrid implementations, and evaluated their performance in terms of tokens per second. Our first observation is that distilled models, i.e., DeepSeek, surpass other models in performance due to their smaller parameters, making them suitable for resource-constrained devices. Also, in the quantized models such as 4-bit quantization (Q4) and 8-bit quantization (Q8), we observed a performance gain of up to 3x compared to FP16 models. Our findings indicate that for fewer parameter sets, such as DeepSeek-r1-1.5B, the TDX implementation outperforms the CPU version in executing computations within a secure environment. We further validate the results using a testbench designed for SoC design tasks. These validations demonstrate the potential of efficiently deploying lightweight LLMs on resource-constrained systems for semiconductor CAD applications.

Figures

Figures reproduced from arXiv: 2507.16226 by the authors.

Figure 1
Figure 1. Comparison of math performance vs. model size, Larger [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Trust Environment Settings with Data I/O to Shared memory [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Performance of different models in different testing environments, ratioed to the CPU performance. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Performance of Deepseek-r1 in different quantization. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 14 canonical work pages

  1. [12]

    Evaluating the performance of the deepseek model in confidential computing environment,

    B. Dong and Q. Wang, “Evaluating the performance of the deepseek model in confidential computing environment,” arXiv preprint arXiv:2502.11347, 2025

  2. [1]

    Overview on signing and whitelisting for intel® software guard extensions (intel® sgx) enclaves,

    Intel Corporation, “Overview on signing and whitelisting for intel® software guard extensions (intel® sgx) enclaves,” https://www.intel.com/content/dam/develop/external/us/en/documents/ overview-signing-whitelisting-intel-sgx-enclaves.pdf, Jan. 2019

  3. [2]

    Privacy-Preserving Inference in Machine Learning Services Using Trusted Execution Environments

    K. G. Narra, Z. Lin, Y . Wang, K. Balasubramaniam, and M. Annavaram, “Privacy-preserving inference in machine learning services using trusted execution environments,” arXiv preprint arXiv:1912.03485 , 2019

  4. [3]

    {SOTER}: Guarding black-box inference for general neural networks at the edge,

    T. Shen, J. Qi, J. Jiang, X. Wang, S. Wen, X. Chen, S. Zhao, S. Wang, L. Chen, X. Luo et al. , “ {SOTER}: Guarding black-box inference for general neural networks at the edge,” in 2022 USENIX Annual Technical Conference (USENIX ATC 22) , 2022, pp. 723–738

  5. [4]

    Shad- ownet: A secure and efficient on-device model inference system for convolutional neural networks,

    Z. Sun, R. Sun, C. Liu, A. R. Chowdhury, L. Lu, and S. Jha, “Shad- ownet: A secure and efficient on-device model inference system for convolutional neural networks,” in 2023 IEEE Symposium on Security and Privacy (SP) . IEEE, 2023, pp. 1596–1612

  6. [5]

    Intel® trust domain extensions,

    Intel Corporation, “Intel® trust domain extensions,” https: //cdrdv2-public.intel.com/690419/TDX-Whitepaper-February2022.pdf, Feb. 2022

  7. [6]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in Neural Information Processing Systems (NeurIPS), vol. 33, pp. 1877–1901, 2020. [Online]. Available: https://arxiv.org/abs/2005.14165

  8. [7]

    Introducing gemini: Our most capable ai model,

    G. DeepMind, “Introducing gemini: Our most capable ai model,” 2023. [Online]. Available: https://www.deepmind.com/blog/ introducing-gemini

Show all 26 references
  1. [8]

    Understanding oversubscribed memory management for deep learning training,

    M. Lin and H. Jeon, “Understanding oversubscribed memory management for deep learning training,” in Proceedings of the 5th Workshop on Machine Learning and Systems , ser. EuroMLSys ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 46–55. [Online]. Availabl...

  2. [9]

    Llm4sechw: Leveraging domain-specific large language model for hardware debug- ging,

    W. Fu, K. Yang, R. G. Dutta, X. Guo, and G. Qu, “Llm4sechw: Leveraging domain-specific large language model for hardware debug- ging,” in 2023 Asian Hardware Oriented Security and Trust Symposium (AsianHOST). IEEE, 2023, pp. 1–6

  3. [10]

    Socurellm: An llm-driven approach for large-scale system-on-chip security verification and policy generation,

    S. Tarek, D. Saha, S. K. Saha, M. Tehranipoor, and F. Farahmandi, “Socurellm: An llm-driven approach for large-scale system-on-chip security verification and policy generation,” Cryptology ePrint Archive , 2024

  4. [11]

    Deepseek llm: Scaling open-source language models with longtermism,

    X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu et al. , “Deepseek llm: Scaling open-source language models with longtermism,” arXiv preprint arXiv:2401.02954 , 2024

  5. [13]

    Amd memory encryption: Sev, sme, and sev-es,

    AMD, “Amd memory encryption: Sev, sme, and sev-es,” 2018. [On- line]. Available: https://www.amd.com/content/dam/amd/en/documents/ epyc-business-docs/white-papers/memory-encryption-white-paper.pdf

  6. [14]

    Building a secure system using trustzone technology,

    ARM, “Building a secure system using trustzone technology,”

  7. [15]

    Drgpum: Guiding memory optimization for gpu-accelerated applications,

    M. Lin, K. Zhou, and P. Su, “Drgpum: Guiding memory optimization for gpu-accelerated applications,” in Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3 , ser. ASPLOS 2023. New York, NY , USA...

  8. [16]

    Exploring parallel implemen- tation of sphincs+ using advanced vector extensions (avx) sets,

    Y . Zhou, K. Rajasekaran, and Q. Wang, “Exploring parallel implemen- tation of sphincs+ using advanced vector extensions (avx) sets,” in 2025 26th International Symposium on Quality Electronic Design (ISQED) , 2025, pp. 1–8

  9. [17]

    Forest: Access-aware gpu uvm management,

    M. Lin, Y . Feng, G. Cox, and H. Jeon, “Forest: Access-aware gpu uvm management,” in Proceedings of the 52nd Annual International Symposium on Computer Architecture , ser. ISCA ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 137–152. [Online]. Available:...

  10. [18]

    Marvel: Multi-agent rtl vulnerability extraction using large language models,

    L. Collini, B. Ahmad, J. Ah-kiow, and R. Karri, “Marvel: Multi-agent rtl vulnerability extraction using large language models,” arXiv preprint arXiv:2505.11963, 2025

  11. [19]

    Spiced: Syntactical bug and trojan pattern identification in a/ms circuits using llm-enhanced detection,

    J. Chaudhuri, D. Thapar, A. Chaudhuri, F. Firouzi, and K. Chakrabarty, “Spiced: Syntactical bug and trojan pattern identification in a/ms circuits using llm-enhanced detection,” in 2024 IEEE Physical Assurance and Inspection of Electronics (PAINE) . IEEE, 2024, pp. 1–7

  12. [20]

    Threatlens: Llm- guided threat modeling and test plan generation for hardware security verification,

    D. Saha, H. A. Shaikh, S. Tarek, and F. Farahmandi, “Threatlens: Llm- guided threat modeling and test plan generation for hardware security verification,” arXiv preprint arXiv:2505.06821 , 2025

  13. [21]

    Is chatgpt a general-purpose natural language processing task solver?

    C. Qin, A. Zhang, Z. Zhang, J. Chen, M. Yasunaga, and D. Yang, “Is chatgpt a general-purpose natural language processing task solver?” arXiv preprint arXiv:2302.06476 , 2023

  14. [22]

    Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert,

    Q. Zhong, L. Ding, J. Liu, B. Du, and D. Tao, “Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert,” arXiv preprint arXiv:2302.10198, 2023

  15. [23]

    A survey of large language models,

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y . Hou, Y . Min, B. Zhang, J. Zhang, Z. Dong et al. , “A survey of large language models,” arXiv preprint arXiv:2303.18223, vol. 1, no. 2, 2023

  16. [24]

    A generalize hardware debugging approach for large language models semi-synthetic, datasets,

    W. Fu, S. Li, Y . Zhao, K. Yang, X. Zhang, Y . Jin, and X. Guo, “A generalize hardware debugging approach for large language models semi-synthetic, datasets,” IEEE Transactions on Circuits and Systems I: Regular Papers, 2024

  17. [25]

    The case for 4-bit precision: k- bit inference scaling laws,

    T. Dettmers and L. Zettlemoyer, “The case for 4-bit precision: k- bit inference scaling laws,” in International Conference on Machine Learning. PMLR, 2023, pp. 7750–7774

  18. [2020]

    Available: https://documentation-service.arm.com/static/ 5f212796500e883ab8e74531

    [Online]. Available: https://documentation-service.arm.com/static/ 5f212796500e883ab8e74531

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.