{"id":"04ed666d-a4f9-4cc5-b69f-bae92d11240e","arxiv_id":"2509.09091","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"CMIF is a hybrid TEE-plus-differential-privacy inference framework that sanitizes sensitive tokens with Report-Noisy-Max and keeps only the embedding layer in the client enclave.","lead":"CMIF runs large language models mostly on cloud GPUs while keeping user text private by sanitizing rare words inside a trusted enclave before inference. It trades a roughly 0.7 second added latency per request for dual privacy protections from hardware enclaves and differential privacy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DP guarantee in §4.4 does not follow: the candidate set S_i and normalization constants depend on the input, so RNM's sensitivity-1 premise is unproven and outputs in S_i\\S_i' have zero probability under the neighbor.","rationale":"The reader identified the same weakest assumption: the DP analysis in §4.4 imports the Report-Noisy-Max theorem without satisfying its hypotheses. The outcome space is input-dependent (top-k of the specific token), and the score normalization depends on the input's own candidate list, so neither the fixed-outcome condition nor the sensitivity-1 condition holds. This is not a matter of tuning or small numerical error; it breaks the formal privacy guarantee for the exact mechanism described. The central contribution of the paper is the claim that CMIF preserves user privacy via this sanitizer, so the formal gap is decisive. I do not see an alternative reading that rescues the proof as written. A revised mechanism could potentially fix the issue, e.g. by using a global, input-independent candidate set and a provable sensitivity bound, but that would be a different design. Hence the REJECT verdict is appropriate.","tokens_in":7010,"tokens_out":3853,"duration_ms":45365,"concrete_test":"Use the released code with Llama2-7B's embedding layer. Pick two tokens a,b with overlapping but non-identical top-30 sets S_a,S_b. For each candidate y in S_a∪S_b compute u_x(y) as Eq. 3. (1) Compute max_y |u_a(y)-u_b(y)|; if it exceeds 1, the 'sensitivity=1' claim fails. (2) For y∈S_a\\S_b, compute Pr(F(a)=y) by Monte Carlo simulation of the Laplace mechanism (or exact integration), and compare to Pr(F(b)=y)=0. Repeat for ε=1 (and p=0.3 in Case 2). If ratio > e^ε for any such pair, the DP bound in §4.4 is disproved.","verdict_should_be":"REJECT","load_bearing_attack":"The load-bearing claim is that CMIF's sanitizer F achieves ε-DP (Case 1) and (ln(1/p)+ε)-DP (Case 2). This rests entirely on Theorem 1, which requires a fixed outcome set Y and a score function with sensitivity Δ. In §3.2, S_i is defined as the top-k most similar words to the specific input w_i, and the normalization in Eq. 3 divides by cos_max−cos_min computed from that input-dependent candidate list. Thus both the set of possible outputs and the score function u change when the input changes. If x*∈S_i but x*∉S_i', then Pr(F(x_i)=x*)>0 while Pr(F(x_i')=x*)=0, so the probability ratio is unbounded and no finite ε-DP can hold. Even restricting to outputs in the intersection, the sensitivity of u is not shown to be 1; the normalization constants themselves depend on x_i, so the global sensitivity of u over neighboring inputs is generally larger than 1, meaning the Lap(1/ε) scale is too small. The claim that normalization 'equalizes sensitivity to 1' is asserted, not proven, and is false under the paper's own definitions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CMIF, a framework for private LLM inference that keeps the embedding layer inside a client-side TEE and offloads the remaining layers to a GPU server. To protect user inputs, it replaces sensitive (low-frequency) tokens using an RNM-style mechanism that adds Laplace noise to normalized cosine scores over a per-token top-k candidate set. The authors claim epsilon-DP for sensitive tokens, (ln(1/p)+epsilon)-DP when non-sensitive tokens are randomized with probability p, and lower inference overhead than purely TEE-based approaches. Experiments on BERT, Llama2-7B, and Llama3-8B report utility and latency results, and a code link is provided.","tokens_in":7370,"tokens_out":9424,"duration_ms":108907,"significance":"If the privacy proof were valid, the design would be an interesting combination: a TEE for the embedding layer plus DP sanitization to avoid per-token TEE round trips. The paper includes a clear experimental setup, measurements of overhead, and a public code link. However, the central DP analysis is not valid as written: the candidate outcome set and normalization are input-dependent, so the standard Report-Noisy-Max theorem does not apply. Since differential privacy is the paper's primary formal contribution, the claimed guarantees do not support the system's privacy properties. The utility and efficiency measurements are internally plausible but are presented at privacy levels that are not established.","major_comments":[{"comment":"The outcome space S_i is defined as the top-k most similar words to each specific input w_i. Thus S_i changes when the input changes. In Report-Noisy-Max (Definition 2 and Theorem 1), Y is a fixed finite outcome set. For any output x* in S_i \\ S_{i'}, Pr(F(x_i)=x*) > 0 while Pr(F(x_i')=x*) = 0, so the probability ratio in Eq. (4) is unbounded. Consequently, Case 1's epsilon-DP claim is unproven and in fact false for the mechanism as specified. This is a direct mismatch between the theorem's assumptions and the construction, not a minor presentational gap.","section":"§3.2 and §4.4, Eq. (4)"},{"comment":"The paper asserts that normalizing by cos_min and cos_max 'equalizes' sensitivity to 1, but these constants are computed from S_i, which depends on x_i. The function u is not a fixed score function d(D, y) as required by Definition 2; it is a family of functions parameterized by the input. The global sensitivity over neighboring inputs is not shown to be 1, and the Laplace scale Lap(1/epsilon) is therefore unjustified. A correct proof would need to establish sup_{x~x', y in a fixed common Y} |u_x(y) - u_{x'}(y)| <= 1 for a fixed Y, which is not supplied and is incompatible with the current input-dependent construction.","section":"§3.2, Eq. (3)"},{"comment":"The mechanism for non-sensitive words is a mixture: with probability p it runs RNM, otherwise it outputs the original word. The claimed (ln(1/p)+epsilon)-DP bound is not derived. For any output y that lies in S_i but not in S_{i'} and also y != x' (the original non-sensitive word), Pr(F(x_i)=y) > 0 while Pr(F(x_i')=y) = 0, so the probability ratio is unbounded and no finite DP guarantee holds. The mixture analysis is missing and the claimed bound is not generally valid.","section":"§4.4, Case 2"},{"comment":"The title and abstract claim dual protection of user data and model parameters. However, only the embedding layer is placed in the client-side TEE; the paper itself notes this is about 1.9% of Llama2-7B parameters. The remaining layers are sent in plaintext to the GPU server, which under the stated semi-honest cloud-provider threat model can inspect them. The paper does not explain what adversary the model-parameter protection is intended to defend against, or why the bulk of the model on the GPU server is considered confidential. This unsupported claim is load-bearing for the paper's framing.","section":"§3.1 and Abstract"}],"minor_comments":[{"comment":"Typo: 'Report-Nosiy-Max' should be 'Report-Noisy-Max'.","section":"§3.2"},{"comment":"The neighboring relation for text inputs is not formally defined. The paper says 'x, x' are defined as differing in vocabulary V', which is ambiguous; Definition 1 requires a precise notion of neighboring inputs (e.g., one token substitution of equal-length sequences).","section":"§2.3 / §4.4"},{"comment":"The 'Random' row appears incomplete: no QNLI values are reported, and the table layout is hard to read. Please also state whether the reported numbers are averaged over multiple runs and what variance is observed.","section":"Table 1"},{"comment":"Sensitivity is defined using the Frobenius norm, while the Report-Noisy-Max guarantee in Theorem 1 typically uses L1 or L_infinity sensitivity for scalar score functions. This mismatch should be clarified.","section":"§2.3, Definition 1"}],"recommendation":"reject","confidential_remarks":"The paper's main contribution is a claimed DP guarantee for an RNM-based sanitizer, and that guarantee is invalid because the outcome space and normalization depend on the input. Fixing this would require redesigning the sanitization mechanism (e.g., using a fixed candidate set) and re-running all utility experiments, so the issues cannot be addressed with local edits. The efficiency experiments may still be of interest, but they do not rescue the paper in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis one is a mixed bag: a real systems idea with a load-bearing privacy proof that falls apart on close reading. The good parts first. Putting the embedding layer in the client TEE and keeping the rest of the LLM on GPU servers is a reasonable way to cut the TEE-GPU communication that sinks Slalom-style partitions. Using the model's own embedding for candidate selection in the sanitizer is a small, legitimate improvement over GloVe-based CUSTEXT. The experiments are fairly reported: they give absolute numbers, show the sanitizer itself adds less than 1% overhead, and don't hide the 28–38% total overhead from SGX. Accuracy losses are a few points, which is in line with the field.\n\nNow the soft spot, and it's not minor. Section 4.4 claims ε-DP for the sanitizer via Theorem 1 of Dwork-Roth. That theorem needs a fixed outcome set and a score function with global sensitivity Δ. The paper defines the candidate set S_i as the top-k most similar words to the specific input w_i, and normalizes scores using min/max from that same input-dependent list. So both the outcome set and the score function change when the input changes. If a plausible output x* is in S_i but not in S_i', then Pr(F(x_i)=x*) > 0 while Pr(F(x_i')=x*) = 0, so the ratio is unbounded. No finite ε works. The claim that normalization sets sensitivity to 1 is asserted, not proven, and is false under the paper's own definitions: the normalized score for a fixed candidate can shift by more than 1 when the min/max change with the input. Case 2 has the same problem plus a hand-waved composition step; the (ln(1/p)+ε) bound doesn't follow from the described pass-through mechanism.\n\nSo the central privacy guarantee — the point of the framework — is unsupported as written. That doesn't make the systems contribution worthless; a fixed candidate set per word (e.g., precomputed globally) plus a real sensitivity analysis might salvage the approach. But the current formal claim needs to be reworked or retracted.\n\nFor whom: researchers working on TEE partitions and text sanitization for LLMs. Worth a serious referee because the systems part is real and the experiments are useful, even though I'd expect major revision or rejection on the proofs.\n\nBest.","headline":"A sensible TEE/GPU split with a broken DP proof: the privacy guarantee in §4.4 does not follow from the paper's own definitions.","tokens_in":7784,"tokens_out":4916,"would_cite":false,"duration_ms":57746,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CMIF claims that putting only the embedding layer in a client TEE and sanitizing input words with Report-Noisy-Max gives LLM inference both differential privacy and low overhead.","keywords":["LLM inference","differential privacy","trusted execution environment","text sanitization","Report-Noisy-Max","GPU acceleration","confidential computing","TEE-GPU partition"],"falsifier":"Compute, for a real embedding space, the true sensitivity of the score function used by F by enumerating pairs of neighboring inputs and checking the maximum absolute change in normalized cosine distance across all possible candidate sets; if any pair exceeds 1, or if the ratio Pr(F(x)=x̂)/Pr(F(x')=x̂) exceeds e^epsilon for some x̂, the paper's central privacy claim is refuted.","tokens_in":6967,"feed_emoji":"🔒","tokens_out":6204,"duration_ms":64940,"temperature":0.7,"pith_summary":"The paper is trying to establish that private LLM inference can be both confidential and efficient if the embedding layer runs in a client-side trusted execution environment and every sensitive input word is replaced by a semantically similar substitute selected with Laplace noise. This avoids the per-layer TEE-GPU communication that slows prior partition-based systems, and it avoids the semantic damage of blanket DP noise. The authors report extra overhead of roughly 29–38% over an unprotected CPU+GPU baseline on Llama2-7B and Llama3-8B, with the sanitizer itself responsible for less than 1%. The central technical claim is that the replacement rule satisfies epsilon-differential privacy (or (ln(1/p)+epsilon)-DP) because the normalized cosine score has sensitivity 1. A sympathetic reader would care because this is a concrete recipe for deploying LLMs on untrusted GPU servers without exposing raw user text.","feed_headline":"Noisy word swaps keep LLM queries private without TEE detours","feed_subtitle":"A client-side enclave hides only the embedding layer while GPUs run the decoder; differential privacy does the rest.","key_machinery":"The load-bearing object is the Report-Noisy-Max (RNM) sanitization mechanism F, defined as: for each word xi, compute normalized cosine distance to top-k candidates Si, add independent Laplace noise Lap(1/epsilon) to each score, and output the candidate with the largest noisy score. The same mechanism doubles as the privacy device (RNM is epsilon-DP when the score is Delta-sensitive) and as the semantic-preserving device (candidates are nearest neighbors in the model's own embedding space). The other half of the framework is the placement of the embedding layer inside the client TEE, which lets the sanitizer run in the enclave and keeps word vectors private while all decoder layers execute o","core_discovery":"In the paper's own terms, the discovery is that CMIF—a Confidential and efficient Model Inference Framework—can protect user data with a sanitization mechanism and model confidentiality by TEE-shielded deployment, without returning to the TEE for decryption during inference. Specifically, the embedding layer is placed in the client enclave; for each sensitive token, the mapping function F builds a top-k candidate set from cosine similarity, normalizes the distances so the score's sensitivity is 'equalized to 1', adds Lap(1/epsilon) noise to each score, and returns the argmax. Non-sensitive tokens are also replaced with probability p. The formal analysis claims epsilon-DP when both neighborin","pith_inferences":["The paper's per-input definition of the candidate set Si is in tension with the standard RNM requirement that the outcome set be fixed across neighboring inputs; a rigorous DP claim would require either a global candidate set or a separate sensitivity argument for changing Si. This is an editorial observation, not a claim the paper makes.","If the sensitivity concern is repaired, the RNM sanitizer becomes a drop-in preprocessing module for any black-box LLM API, since it needs only the embedding table, which could be obtained from a public model or a minimal client-side copy.","The asymmetric treatment of sensitive vs non-sensitive words gives an explicit trade-off: lowering p increases fidelity but adds ln(1/p) to the privacy loss, so practitioners can select p based on deployment-specific privacy budgets without retraining.","The same design pattern—small protected embedding layer plus noisy token substitution—could transfer beyond SGX to other confidential-computing backends or to on-device mobile TEEs, because the protected component is small and self-contained."],"forward_implications":["CMIF eliminates the need for per-layer encryption and TEE-GPU switching during decoder execution, since the server only receives sanitized text.","The sanitizer's own time cost is about 0.6% of inference time, so the dominant remaining overhead is the enclave boundary itself, not the DP mechanism.","Compared to SANTEXT+ and CUSTEXT+, the RNM sanitizer improves classification accuracy at the same nominal epsilon on SST-2 and QNLI with BERT, and keeps Llama3-8B within 0.15% of non-private performance on IFEval after fine-tuning on sanitized DialogSUM data.","The small size of the embedding layer (roughly 1.9% of Llama2-7B parameters) makes client-side TEE deployment feasible even on memory-constrained devices."],"fun_headline_variants":["Client-side enclave plus DP cuts LLM inference overhead","Dual privacy shield: TEE for embeddings, DP for the rest","CMIF trims TEE overhead while keeping LLM queries private","Split LLM: hide only embeddings, let GPUs run the decoder","Efficient private inference: DP handles most, TEE handles the rest"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The privacy guarantee rests on the normalized cosine distance having sensitivity exactly 1 and on the candidate substitute set being the same for neighboring inputs; since the candidate set is defined per input word, the analysis does not establish this, and if either part fails the claimed epsilon-DP bound does not hold.","fun_headline_variants_meta":{"raw":{"variants":["Client-side enclave plus DP cuts LLM inference overhead","Dual privacy shield: TEE for embeddings, DP for the rest","CMIF trims TEE overhead while keeping LLM queries private","Split LLM: hide only embeddings, let GPUs run the decoder","Efficient private inference: DP handles most, TEE handles the rest"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000289,"raw_usage":{"total_tokens":1503,"prompt_tokens":689,"completion_tokens":814,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":433,"completion_tokens_details":{"reasoning_tokens":722}},"tokens_in":433,"tokens_out":814,"duration_ms":8822,"temperature":1.0,"reasoning_tokens":722,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T19:41:44.015007+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute, for a real embedding space, the true sensitivity of the score function used by F by enumerating pairs of neighboring inputs and checking the maximum absolute change in normalized cosine distance across all possible candidate sets; if any pair exceeds 1, or if the ratio Pr(F(x)=x̂)/Pr(F(x')=x̂) exceeds e^epsilon for some x̂, the paper's central privacy claim is refuted.","supporting_citations":[],"review_version":1}