{"id":"9bd3b9bf-31b3-4313-a9a1-b62d492a744a","arxiv_id":"2507.16226","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":1,"one_line_summary":"Running lightweight distilled LLMs inside Intel TDX secure VMs reportedly gives higher tokens per second than plain CPU execution for sub-3B models, with Q4 quantization reaching about 3x FP16 throughput.","lead":"Small, distilled AI models were benchmarked inside Intel's TDX confidential-computing enclave, measuring tokens per second against ordinary CPU and CPU-GPU setups. For models under 3 billion parameters the secure TDX path reported higher throughput than non-secure CPU execution, and 4-bit quantization showed up to roughly 3x speedup.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The TDX-beats-CPU headline is not established because the CPU-only baseline is not resource-equivalent to the TDX VM, so the claimed speedup may be an artifact of core/memory allocation rather than TDX.","rationale":"The paper's purpose is an empirical benchmark; its strongest contribution would be a secure deployment that is not only feasible but faster than an insecure CPU baseline. That requires the comparison to be controlled. The reader identified this exact weakness: the CPU-only baseline lacks resource specification. My independent pass found the same issue and no other objection with more weight. The absence of error bars and accuracy metrics is real but secondary; the resource confound alone invalidates the central claim as stated. This is not a disagreement with community consensus; it is an internal-validity problem in the paper's own setup. I would keep the REJECT verdict. If the proposed matched-resource experiment shows a persistent TDX advantage, the paper becomes a useful empirical result and could be conditionally accepted with reproducibility artifacts.","tokens_in":9971,"tokens_out":2977,"duration_ms":31135,"concrete_test":"Rerun the benchmark on the same physical host with CPU-only and TDX containers matched on all resources: 62 vCPUs, 510GB memory, same Ollama image/version, same model files, same thread count, and pinned to the same NUMA nodes, with no other host workloads. For DeepSeek 1.5B and 7B, collect at least 10 independent trials per condition and report mean, standard deviation, and 95% CI of tokens/s. If the 1.5B TDX/CPU ratio no longer exceeds 1 significantly, the headline speedup is an artifact of resource configuration. Also record lscpu/cgroup limits and nvidia-smi output for every run.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that TDX-based confidential execution is faster than plain CPU execution for sub-3B models, exemplified by DeepSeek 1.5B at 25.67 vs 10.25 tokens/s (Table II, Section V-A). For this claim to hold, TDX-specific properties must be the cause of the speedup. The experiment does not establish that. Table I defines CPU Only as 'Runs on the host machine without GPU acceleration enabled' while TDX is '62 CPU cores and 510GB DRAM'; Section IV-A does not state the CPU-only container's vCPU count, memory limit, NUMA placement, or host isolation. Section V-A attributes the result to 'TDX's optimized CPU ISA configuration,' but with these variables uncontrolled the observed ratio could come from the CPU baseline having fewer cores or memory, from host contention, or from different Ollama thread settings. The fact that the advantage reverses at 7B/14B is consistent with a resource-allocation explanation rather than a TDX security/ISA effect. The quantization results (up to 3x) are less vulnerable to this confound and have standalone value, but they do not support the headline claim about secure enclaves being faster than CPU.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper empirically evaluates lightweight LLM inference in an Intel TDX confidential-computing environment versus CPU-only and CPU-GPU baselines, using Ollama to measure tokens/s for DeepSeek-R1 distilled variants (1.5B, 7B, 14B), Llama 3.2, Gemma 2, and several HWSeC fine-tuned models. The authors report that for sub-3B models TDX achieves higher throughput than CPU-only execution (e.g., DeepSeek 1.5B at 25.67 vs 10.25 tokens/s), that quantization Q4/Q8 gives up to 3x speedup over FP16, and that these trends hold for domain-specific HWSeC models. They conclude that lightweight distilled LLMs can be deployed efficiently and securely in confidential computing for SoC design workflows.","tokens_in":10136,"tokens_out":4827,"duration_ms":52053,"significance":"If the throughput comparison were properly controlled, this would be a useful empirical data point: it would show that confidential-execution overhead need not make small-model inference dramatically slower than an insecure CPU baseline, and the quantization results would be practically relevant for memory-constrained TEE deployments. The paper also addresses a topical gap, since most TEE-LLM work focuses on SGX or large models, whereas TDX and distilled sub-3B models are less studied. The strengths are the breadth of models tested (DeepSeek, Llama, Gemma, HWSeC domain models) and the explicit reporting of raw tokens/s values and model storage sizes. However, the headline TDX-beats-CPU claim is not established because the CPU-only baseline is not resource-equivalent, and the HWSeC validation claims accuracy improvements from a table that contains only throughput. The paper would need additional experiments and careful re-scoping before its central conclusions can be accepted.","major_comments":[{"comment":"The central claim that TDX outperforms CPU-only execution for sub-3B models is confounded by unequal resources. Table I specifies the TDX configuration as \"62 CPU cores and 510GB DRAM\" but describes CPU Only only as \"Runs on the host machine without GPU acceleration enabled,\" with no vCPU count, memory limit, NUMA placement, container constraints, or host-isolation details in Section IV-A. The 25.67 vs 10.25 tokens/s comparison for DeepSeek 1.5B in Table II, and its attribution to \"TDX's optimized CPU ISA configuration\" in Section V-A, therefore conflates TDX-specific effects with core/memory allocation and host contention. The reversal at 7B (CPU-only 8.53 vs TDX 6.42) is consistent with a resource-allocation explanation rather than a TDX ISA effect. The authors should rerun with a CPU-only container constrained to the same cores, memory, and Ollama thread settings as the TDX VM, and should report whether the advantage survives.","section":"Section IV-A, Table I, Section V-A"},{"comment":"The text states that \"The results shown in Table III confirm that these domain-specific HWSeC models achieve improved accuracy and usability in chip design and verification benchmarks, outperforming general LLMs on hardware-centric tasks.\" Table III contains only throughput in tokens/s and throughput ratios; no accuracy metrics, no comparison to general LLMs on hardware benchmarks, and no benchmark scores are reported. This sentence is unsupported by the presented data. Either add the accuracy results or rephrase the claim to refer only to throughput.","section":"Section V-C, Table III"},{"comment":"All performance numbers are reported as single point values from Ollama's built-in logging, with no repeated runs, error bars, or statistical tests. Because the paper's headline conclusions are ratios (e.g., \"more than twice,\" \"9x,\" \"20x\"), single measurements do not establish that the observed differences are reliable. At minimum, the authors should report multiple repetitions with standard deviations and a simple significance test for the key TDX-vs-CPU comparisons.","section":"Section V-A, Tables II and III"}],"minor_comments":[{"comment":"The abstract and the third contribution bullet state that LLM performance in TEEs exceeds CPU-only execution without qualification, but the paper's own results show this holds only for sub-3B models; the 7B and 14B cases reverse or narrow the gap. Please qualify the claim to the sub-3B distilled-model regime.","section":"Abstract and Section I contributions"},{"comment":"The contribution bullet claims \"the first evaluation of a distilled LLM within a TEE,\" yet reference [12] is a prior evaluation of DeepSeek in a confidential computing environment by the same authors. Please reconcile this claim, for example by clarifying what is new relative to [12].","section":"Section I contributions and References [12]"},{"comment":"Figure 3 is described as performance \"ratioed to the CPU performance,\" but the caption does not define which CPU baseline is used (CPU-only, TDX, or normalized value) or how the ratio is computed. Please define the normalization explicitly.","section":"Section V-A, Figure 3"},{"comment":"The statement \"8-bit DeepSeek-14B model performs better than 16-bit DeepSeek-7B\" is confusing, and the following sentence \"the DeepSeek model sees only a slight gain (2.15 to 2.17 tokens/s)\" needs context: which models and quantization levels are being compared, and at which parameter sizes? Please state the comparison clearly with the corresponding numbers.","section":"Section V-D"},{"comment":"The column headers in Tables II and III contain typesetting artifacts such as \"GP U T DX\" and \"CP U T DX\" that should be reformatted as GPU/TDX and CPU/TDX, and the dimensionless ratio columns should be defined in the captions.","section":"Tables II and III"},{"comment":"The domain-specific model family is referred to inconsistently as \"HWSeC,\" \"HWSec,\" and \"KSU HWSEC\"; please choose one consistent notation. Also, no code or data availability statement is provided, which limits reproducibility of the benchmark.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The central experimental comparison needs a controlled CPU-only baseline before the headline claim can be evaluated. If the authors cannot rerun the experiments with matched resources and repetitions, the paper should not be accepted, since the current data cannot distinguish a TDX-specific advantage from a resource-allocation artifact. I do not see evidence of intentional misrepresentation, but the unqualified \"first evaluation\" claim relative to the authors' own reference [12] should be checked."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, the headline claim in this paper—TDX secure enclaves are faster than plain CPU for sub-3B LLMs—does not survive contact with the experimental design. The CPU-only baseline in Table I has no resource specification (no vCPU count, no memory limit, no isolation), while the TDX VM gets 62 cores and 510GB DRAM. The observed 2x speedup at 1.5B is plausibly just a core/memory allocation effect, not a TDX ISA optimization. The fact that the advantage reverses at 7B/14B is consistent with that reading.\n\nWhat is genuinely new here is a set of throughput numbers for a few small models (DeepSeek-R1 1.5B/7B/14B, Llama 3.2 1B/3B, Gemma 2 2B) across three deployment modes, plus quantization comparisons (Q4, Q8, FP16) in the TDX environment. The quantization result—up to 3x speedup for Q4 over FP16, and the size reduction table—is measured within a single environment and is less confounded. That part is useful for practitioners choosing model/quantization for a secure enclave.\n\nThe soft spots beyond the confound: the \"first evaluation of a distilled LLM within a TEE\" claim is contradicted by the authors' own reference [12], a prior DeepSeek-in-confidential-computing paper. Section V-C claims the HWSeC models \"achieve improved accuracy\" but Table III contains only tokens/sec; there is no accuracy data anywhere. No error bars or repeated runs are reported, and no code or raw data is provided. So as published, the central contribution isn't credible.\n\nThat said, the paper is clearly written, the testbench is relevant to SoC design, and the quantization data is a reasonable empirical starting point. It deserves a serious referee, who should require a controlled rerun: matched CPU resources, repeated trials, and either accuracy measurements or removal of the accuracy claim. My own verdict is reject as-is, but this is a fixable paper rather than a dead one.","headline":"TDX-beats-CPU claim is unconvincing due to unmatched baseline resources; the quantization data is the solid part.","tokens_in":10717,"tokens_out":2757,"would_cite":false,"duration_ms":26742,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that for distilled LLMs below roughly 3 billion parameters, Intel TDX confidential execution runs inference faster than an unsecured CPU-only setup, while keeping model and data inside an encrypted enclave.","keywords":["confidential computing","Intel TDX","trusted execution environment","distilled large language models","model quantization","system-on-chip design","LLM inference benchmark","DeepSeek"],"falsifier":"Run the same DeepSeek-R1-1.5B model in a CPU-only container configured with the same 62 cores and 510 GB as the TDX VM, pinned to the same NUMA nodes and free of other host workloads; if TDX no longer achieves about 25.67 tokens/s against a CPU-only 10.25, the claimed enclave speedup collapses.","tokens_in":9679,"feed_emoji":"🔒","tokens_out":6486,"duration_ms":57609,"temperature":0.7,"pith_summary":"The paper tries to establish that lightweight distilled LLMs can be run inside a confidential computing enclave without sacrificing, and for small models even improving on, ordinary CPU performance, making secure LLM use practical for System-on-Chip design. It benchmarks DeepSeek-R1, Llama 3.2, and Gemma 2 models in three environments: Intel TDX, CPU-only, and CPU-GPU. The key results are that TDX reaches about 25.67 tokens/s for DeepSeek-1.5B, more than double the CPU-only rate, and that Q4/Q8 quantization speeds inference up to 3x over FP16 while cutting model storage by roughly half to two-thirds. It also validates this on HWSec models fine-tuned for hardware security and design tasks. If correct, it would mean secure enclave inference is feasible for resource-constrained semiconductor CAD workflows without GPU support.","feed_headline":"Secure enclaves run small AI models faster than plain CPUs","feed_subtitle":"For sub-3B models, confidential TDX execution beats CPU-only throughput, adding security without a speed tax.","key_machinery":"The mechanism is Intel TDX, a VM-level trusted execution environment that allocates a private memory region to a secure Trust Domain and encrypts data crossing the enclave boundary. Inside that enclave, the paper relies on two properties of small models: distillation (DeepSeek-R1's 1.5B/7B/14B variants keep reasoning ability at low parameter counts) and post-training quantization (Q4/Q8 integer weights). These properties keep the whole model and workload resident in TDX private memory, avoiding the memory-swap overhead that larger models trigger, so the TDX CPU configuration can run small models faster than the CPU-only baseline the authors measured.","core_discovery":"The central discovery, as the authors state it, is that Intel TDX's CPU execution can beat plain CPU execution for small distilled models: DeepSeek-R1-1.5B runs at 25.67 tokens/s in the TDX VM versus 10.25 tokens/s CPU-only, and similar advantages appear for Llama3.2-1B/3B and Gemma2-2B. The advantage fades with model size, DeepSeek-7B is slower in TDX (6.42 vs 8.53 tokens/s), so the benefit is specific to sub-3B models. They also find that quantization compounds the effect: Q4 DeepSeek roughly triples tokens/s relative to FP16, and Q4 storage is about 30% of FP16. These patterns hold on HWSec models fine-tuned for hardware security, where TDX exceeds CPU-only in every case they report.","pith_inferences":["The TDX-versus-CPU gap is most plausibly a configuration effect (62 cores and 510 GB in the TDX VM versus an unspecified host container) rather than an effect of the security machinery itself; a matched-resource rerun would separate these causes.","If the speedup survives controlled reruns, confidential cloud providers could offer secure small-model inference for latency-tolerant CAD workloads without GPU allocation, lowering cost.","The same benchmark could be run on AMD SEV and ARM TrustZone to test whether the small-model advantage is generic to VM-level TEEs or specific to TDX.","A direct extension would profile memory bandwidth and cache behavior of the 1.5B model inside TDX to identify whether the gain comes from ISA optimizations or from reduced memory pressure."],"forward_implications":["For sub-3B distilled models, confidential execution is not a performance tax: TDX can roughly double CPU-only throughput, so security can be added at no speed penalty.","Q4/Q8 quantization is the practical deployment choice in TDX, combining up to 3x speedup over FP16 with storage of roughly 30-50% of FP16, which fits enclave memory limits.","The TDX advantage disappears at 7B and 14B scales, where memory bandwidth dominates, so larger confidential models require GPU-in-enclave support that current TDX does not offer.","GPU acceleration remains 8-14x faster than TDX only because it moves data out of the enclave in plaintext; the paper's central trade-off is security versus speed.","Fine-tuned HWSec hardware-security models follow the same trend, indicating that domain-adapted confidential LLM deployment in SoC verification is feasible."],"supporting_citations":[{"why":"Supplies the prior SGX TEE that motivates the move to TDX because of SGX's roughly 1 GB enclave limit.","marker":"[1]"},{"why":"Defines Intel TDX and the VM-level trust domain with private memory that the experiments use.","marker":"[5]"},{"why":"Provides the DeepSeek model family, including the distilled 1.5B/7B/14B variants benchmarked here.","marker":"[11]"},{"why":"Is the authors' prior evaluation of DeepSeek in confidential computing, which this paper extends.","marker":"[12]"},{"why":"Provides the HWSec dataset and fine-tuned hardware-security LLMs used for the SoC-design validation.","marker":"[24]"},{"why":"Supplies the k-bit quantization scaling-law argument that motivates the Q4/Q8 speedup expectations.","marker":"[25]"}],"fun_headline_variants":["TDX beats CPU for sub-3B LLMs in secure enclaves","Secure enclaves speed up small distilled LLMs","Quantization and TDX give small LLMs a 3x boost","Small LLMs run faster in Intel TDX than on CPUs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The speed comparison assumes the CPU-only baseline is a fair, resource-equivalent control, but the paper does not state its core count, memory limit, or isolation from other host workloads, so the TDX advantage could be an artifact of a weaker baseline.","fun_headline_variants_meta":{"raw":{"variants":["TDX beats CPU for sub-3B LLMs in secure enclaves","Secure enclaves speed up small distilled LLMs","Quantization and TDX give small LLMs a 3x boost","Small LLMs run faster in Intel TDX than on CPUs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000649,"raw_usage":{"total_tokens":3014,"prompt_tokens":1015,"completion_tokens":1999,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":1926}},"tokens_in":631,"tokens_out":1999,"duration_ms":14165,"temperature":1.0,"reasoning_tokens":1926,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:14:56.314016+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same DeepSeek-R1-1.5B model in a CPU-only container configured with the same 62 cores and 510 GB as the TDX VM, pinned to the same NUMA nodes and free of other host workloads; if TDX no longer achieves about 25.67 tokens/s against a CPU-only 10.25, the claimed enclave speedup collapses.","supporting_citations":[{"cited_title":"Overview on signing and whitelisting for intel® software guard extensions (intel® sgx) enclaves,","cited_arxiv_id":null,"evidence_quote":"Supplies the prior SGX TEE that motivates the move to TDX because of SGX's roughly 1 GB enclave limit."},{"cited_title":"Intel® trust domain extensions,","cited_arxiv_id":null,"evidence_quote":"Defines Intel TDX and the VM-level trust domain with private memory that the experiments use."},{"cited_title":"A generalize hardware debugging approach for large language models semi-synthetic, datasets,","cited_arxiv_id":null,"evidence_quote":"Provides the HWSec dataset and fine-tuned hardware-security LLMs used for the SoC-design validation."},{"cited_title":"The case for 4-bit precision: k- bit inference scaling laws,","cited_arxiv_id":null,"evidence_quote":"Supplies the k-bit quantization scaling-law argument that motivates the Q4/Q8 speedup expectations."}],"review_version":1}