Pith. sign in

REVIEW 3 major objections 5 minor 35 references

Masked matrix multiplication enables practical confidential AI outsourcing down to O((m+n)l) client work.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 11:24 UTC pith:CDMDPR22

load-bearing objection The masking protocol is a real asymptotic advance and the 70B accuracy results are impressive, but the 140-bit security parameterization is unsubstantiated—no estimator trace, and the sample-count accounting in Example 4.1 doesn't match the adversary's view of the full masked matrices. the 3 major comments →

arxiv 2607.29221 v1 pith:CDMDPR22 submitted 2026-07-31 cs.CR cs.AI

MOSAIC: Masked Outsourcing of Secure AI Computations

James Hsin-yu Chiang , Sheila Zingg , Kari Kostiainen , Srdjan Capkun This is my paper
classification cs.CR cs.AI MSC 94A60
keywords secure outsourcingmatrix multiplicationLWELPNtransformer inferenceconfidential AIapproximate computationHadamard rotation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that a small trusted client can outsource the dominating linear matrix multiplications of large transformer inference to an untrusted GPU without revealing weights or activations, at asymptotically optimal client cost O((m+n)l). The trick is to mask matrices with low-rank uniform padding plus small Gaussian noise, so the server sees only uniform-looking data, and the client removes cheap low-rank corrections after the server returns the product, tolerating a small approximate error. Because that error accumulates over hundreds of layers, the paper adds a random Hadamard rotation to spread outlier energy before fixed-point quantization, making error growth non-destructive. On 70B-parameter models, the scheme's perplexity is comparable to 4-bit and 8-bit quantization baselines and its HumanEval score matches full BF16 reference at the 140-bit-secure noise setting. If right, this is a route to confidential AI at frontier scale from a minimal trusted computing base.

Core claim

MOSAIC's central claim is that secure approximate matrix multiplication can be outsourced with optimal O((m+n)l) trusted-client online cost while the server learns nothing about the weight matrix W or activation X, with security reducing to decisional LWE and LPN. The masked product returned by the server is WX plus a sum of low-rank correction terms (removed in O((m+n)l)) and a small Gaussian error whose growth across a transformer's layers is controlled by random Hadamard rotations. End-to-end, the protocol runs 70B-parameter models with error accumulation comparable to common quantization baselines, and at the proposed 140-bit security setting (sigma=0.5) matches BF16 on HumanEval for LLa

What carries the argument

The load-bearing object is the nested LWE+LPN mask: weights and activations are hidden as W' = W + L M + E_w (and analogously X' = X + N R + E_x), where L, M, N, R are low-rank uniform matrices and E_w, E_x are small Gaussian noises; repeated nesting replaces dense low-rank pieces with sparser LPN instances of shrinking rank. The server computes W'X'; the client removes the low-rank cross terms using precomputed chains, leaving WX plus a bounded sub-Gaussian error. The companion mechanism is a random Hadamard rotation applied before fixed-point quantization, which spreads outlier energy so that quantization and protocol noise do not diverge layer by layer.

Load-bearing premise

The concrete 140-bit security claim rests on the assertion that LWE with dimension 1536, modulus 2^32, and Gaussian noise sigma=0.5 (plus the stated LPN ranks and Hamming weights) is actually 140-bit secure, but the paper gives no estimator output or reproduction script for that parameter set.

What would settle it

Run the published LWE and LPN security estimators on the Example 4.1 parameters (LWE dimension 1536, modulus 2^32, sigma=0.5; LPN ranks 1024 and 652 with Hamming weight 80) and check the estimated bit-security; if any instance estimates below 140 bits, or if a distinguishing attack on masked versus uniform matrices at sigma=0.5 is found, the privacy claim collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Trusted-client cost per outsourced matrix product is O((m+n)l), independent of inner dimension n up to security-parameter-fixed ranks, so larger matrices (future models) become relatively cheaper to outsource.
  • The same protocol yields both privacy (computational indistinguishability from uniform under decisional LWE/LPN) and integrity via a Freivalds-style check, so the untrusted GPU cannot tamper undetected.
  • With 140-bit parameters (sigma=0.5), LLaMA-3-70B retains BF16-level HumanEval and a 1-2% perplexity gap to BF16; on quantization-friendly models the protocol stays within 2-3% of BF16 for sigma up to 1.0.
  • Layer-by-layer communication is not the bottleneck: communication is about 4-7% of decode and 7-16% of prefill wall time under NVLink/RDMA-like interconnects.
  • End-to-end prefill/decode on 70B models is feasible at 5-11x (prefill) and about 3x (decode) slowdown vs local GPU, dominated by 32-bit-on-8-bit emulation rather than protocol overhead.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the concrete security parameterization is validated, the same masking could apply to other linear-algebra-heavy workloads beyond transformers (e.g., diffusion models, recommendation-system matrix factorization, scientific computing) wherever a small relative error is tolerable.
  • The protocol's noise is additive Gaussian in the integer domain; quantization-friendly models like Qwen2.5 could tolerate higher sigma than 0.5 and thus potentially stronger security margins, while quantization-sensitive models like LLaMA-3 bound sigma — a per-model security/accuracy tradeoff curve could be mapped.
  • Native 32-bit integer tensor cores, if adopted, would likely reduce the measured 3-11x gap to near-communication-bound performance, making the approach competitive with unsecured inference at large batch sizes.
  • The Hadamard-rotation error analysis suggests a quantitative link between model dimension n and achievable noise variance (about log n / n reduction), which could be used to set sigma per layer rather than globally.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. MOSAIC proposes a protocol for outsourcing matrix multiplication from a trusted but weak client to an untrusted accelerator, with client overhead O((m+n)l) per multiplication. The construction masks both weight and activation matrices with a low-rank uniform term plus a small Gaussian noise (LWE), and then nests this with LPN instances of decreasing rank to reduce concrete client work. The paper claims security under decisional LWE and LPN, proves a sub-Gaussian error bound for the resulting approximation, and applies the protocol to secure transformer inference by adding randomized Hadamard rotations to control fixed-point and noise accumulation. End-to-end experiments on 70B-class models report perplexity and HumanEval scores close to BF16 at the proposed 140-bit-secure parameters (LWE dimension 1536, σ=0.5, LPN ranks 1024 and 652, Hamming weights 80). The paper also presents a datacenter deployment with fast interconnects and a remote classification use case.

Significance. If the security and accuracy claims hold, MOSAIC would be a substantial advance: it is the first outsourcing protocol for large-scale transformer inference that simultaneously achieves optimal asymptotic client cost, concrete runtimes orders of magnitude faster than prior work, and model accuracy comparable to standard quantization baselines on 70B models. The paper includes an open-source implementation, an explicit reduction to LWE/LPN, a rigorous-looking error tail bound (Theorem 1), and extensive empirical error accumulation studies. These are real strengths. However, the central 140-bit security parameterization is asserted without estimator outputs, and the protocol specification has a dimension inconsistency in the nested masking, so the significance is currently conditional on fixing these issues.

major comments (3)
  1. [§4.2, Example 4.1 and Theorem 2] The 140-bit security claim is not substantiated. No estimator commands, outputs, or reproduction scripts are provided for either the LWE or LPN instances. The LPN sample counts N2=1536 and N3=1024 coincide with the inner ranks r1 and r2, but in ΠSec-Approx-MatMul the adversary receives the full masked matrices W' (m×n) and X' (n×l), giving up to m and n/l samples per secret respectively (e.g., about 29568 samples for Qwen2.5-72B projections), not r_{i-1}. For LWE, no sample count is given at all. Additionally, σ=0.5 over Z_{2^32} is a very low-noise/high-modulus regime; the security level cannot be assessed without an actual estimator trace. Without this, Theorem 2's privacy assertion is unsupported.
  2. [§4.2, Eq. (9) and Figure 5 (Init W)] The nested mask has an incompatible dimension. In Figure 5, S_i is defined as an element of Z^{r_i×r_{i-1}}; then S_{i+1}(M_i...M_1) has dimensions r_{i+1}×n, which cannot be added to the m×n matrix U_w unless m=r_{i+1}. The intuitive derivation in Eq. (8) only works if S_i is m×r_{i-1}. Example 4.1 sets r2=1024, r1=1536, but the weight matrices in the experiments have m≫1024 (e.g., m=8192), so the construction as written does not produce a valid mask. This must be corrected in the protocol specification and the security reduction (Lemma 4) re-examined with consistent dimensions.
  3. [§4.2, Figure 5 (Online phase, step 3)] The adversary G additionally receives Q_i = (M_i...M_1)X' for i∈[d]. The paper's Theorem 2 states indistinguishability only for (W',X'), and the proof in §A.4 does not mention Q_i. Although Q_i are deterministic functions of X' and public M_i and therefore add no information if X' is uniform, the theorem statement and proof should explicitly account for the full transcript to avoid a gap between the claimed security notion and the actual protocol messages.
minor comments (5)
  1. [§4.2, Example 4.1] The sentence 'Let r_i be the secret dimension and N_i the number of permitted adversarial samples (polynomially many for LWE)' is confusing because no N_i is given for the LWE instance. Please give the exact sample count used in the estimator and clarify whether N_i for LPN refers to the number of rows/columns of the embedded S_i or to the full m×n samples seen by the adversary.
  2. [§5.4, Table 4] HumanEval pass@1 values are reported as point estimates with no seeds or confidence intervals. Given the variance of HumanEval at k=1, a few points difference may not be significant; please report multiple runs or the number of samples.
  3. [§6, Figure 12] The text says 'the higher prefill factor corresponding to layer- rather than matrix-sharding', but the figure legend uses M and L; please spell out the mapping in the caption for clarity.
  4. [§5.2, Table 3] The footnote 'The large LLaMA-3 maxima are concentrated in the early-layer value projections (the layer-0 v_proj alone accounts for the reported maximum)' is useful, but it would be clearer to also state the values excluding v_proj in the table or caption.
  5. [Appendix A.2] In the proof of Theorem 1, the notation N(Z_{2κ}; 0, σ²) is used without a formal definition in the main text; please define it near its first use in §4.1.

Circularity Check

0 steps flagged

No circularity found: the security and accuracy derivation chains are self-contained; the unsubstantiated 140-bit parameterization is an evidence gap, not a circular step.

full rationale

MOSAIC's derivation is self-contained rather than circular. The masking algebra in Section 4.1 (Eqs. 2-7) expands W'X' and removes the low-rank correction terms, leaving only W E_x + E_w X + E_w E_x; Theorem 1 bounds this error from the sub-Gaussianity of the discrete Gaussian, with Lemmas 2-3 doing the heavy lifting and Corollary 1 justifying the drop of the cross term. Security is reduced to decisional LWE/LPN in Lemma 1 and Lemma 4 via hybrid arguments that embed a standard challenge; the conclusion is not assumed. The claimed O((m+n)l) client cost follows from explicit per-operation counts with inner ranks fixed by the security parameter, and the optimality lower bound is the input/output size. The Hadamard-rotation error mitigation is taken from external quantization work [4,19,30] and re-derived in Appendix D from orthogonality and sub-Gaussian concentration; it is not justified by a self-citation. Accuracy results (Tables 3-4) are measurements taken at sigma=0.5, a value fixed by the security target in Example 4.1 and not fitted to accuracy, so no fitted quantity is relabeled as a prediction. The reference list contains no load-bearing self-citations by the present authors. The one material weakness is that Example 4.1 asserts 140-bit security by citing estimators [2,33] without giving estimator commands/outputs or reconciling N_i with the actual number of samples the adversary obtains from the full masked matrices; that is a missing-evidence/security-risk issue, not a circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 0 invented entities

The protocol introduces no new physical or math objects; the load-bearing inputs are the concrete LWE/LPN security parameter choices, the semi-honest threat model, and the equivalence of the simulated noise model with the real protocol error. The security parameters are especially important because they are asserted without reproduction, and a wrong choice would destroy the privacy claim.

free parameters (5)
  • LWE Gaussian noise stddev sigma = 0.5
    Gaussian noise scale for E_w and E_x masks; set for 140-bit security via estimator [2], but no estimator output is provided. This is load-bearing for privacy.
  • Inner ranks r1, r2, r3 = 1536, 1024, 652
    Dimensions of nested LWE/LPN masks from Example 4.1; claimed 140-bit security via estimators [2] and [33], but with no reproduction.
  • LPN Hamming weights t2, t3 = 80
    Sparse noise weights for LPN nesting; part of the security parameterization.
  • Number of nesting levels d = 3
    Chosen in Example 4.1 to balance compute and storage; a hand-picked protocol parameter.
  • Fixed-point window q_max
    Chosen so n*q_max^2 <= 2^{kappa-1}; depends on model dimensions and affects float-domain noise amplification through per-row/column scales.
axioms (6)
  • domain assumption Decisional LWE is hard for the concrete parameters (dimension 1536, modulus 2^32, sigma=0.5) with polynomially many samples
    The privacy of W' and X' relies on this; Example 4.1 offers no estimator output or configuration to verify the 140-bit claim.
  • domain assumption Decisional LPN with sparse noise is hard for the nested parameters (r2=1024, r3=652, t=80) over Z_{2^32}
    Nested LPN security is used in Lemma 4 and Theorem 2; no estimator output is provided.
  • domain assumption The untrusted server is semi-honest (passive) for privacy; malicious behavior is not fully handled
    The security proof bounds a passive adversary's view. The Freivalds check only verifies Y', not the Q_i values used in unmasking, so a malicious GPU can corrupt the output undetected.
  • domain assumption The Gaussian noise model (E_w, E_x with std sigma) used in the INT16-rot-sigma accuracy experiments equals the real protocol error after exact unmasking
    Follows from Eq. (7) assuming no wrap-around and exact correction; this equivalence is used to evaluate Table 4 without running the full protocol on every forward pass.
  • domain assumption Random Hadamard rotation R=HD makes ||R x||_inf ~ ||x||_2 sqrt(log n / n) with high probability and preserves the matrix product exactly
    Standard concentration result used in Appendix D; needed for the claimed noise-variance reduction of ~n/log n.
  • standard math Freivalds' randomized check correctly verifies the GPU's computation of Y'
    Standard randomized linear-algebra identity; reproduced in Appendix B.

pith-pipeline@v1.3.0-daily-deepseek · 32015 in / 25794 out tokens · 266555 ms · 2026-08-03T11:24:51.664917+00:00 · methodology

0 comments
read the original abstract

We address the challenge of securely and efficiently outsourcing AI computations from a trusted but computationally weak client to an untrusted but powerful server, in the setting where the client holds both the input and the model, and the server must learn neither. We present MOSAIC, whose core is a novel matrix-multiplication masking protocol that scales to far larger matrices than prior work, enabling the safe outsourcing of modern workloads such as large transformer inference. By introducing small amounts of noise to the multiplication result and thereby relaxing correctness, MOSAIC achieves optimal asymptotic client overhead and concrete runtimes orders of magnitude faster than prior work. Its security reduces to the decisional LWE and LPN assumptions. Because this noise accumulates across the many layers of a transformer, a key technical challenge is bounding error growth; MOSAIC addresses this with an error-scaling mechanism based on random Hadamard rotations. On large 70B transformer models, MOSAIC's perplexity is comparable to popular quantization approaches and even matches full-precision BF16 inference on HumanEval. Finally, we present an end-to-end implementation showing how ideas like MOSAIC can promise a path towards large-scale confidential AI in modern data centers. Non-confidential inference is already distributed across phase (prefill/decode), layer, and time to maximize utilization of heterogeneous hardware, using RDMA-like networking to move activations, cached KV values, and weights across nodes. MOSAIC enables scaling of confidential compute by keeping the trusted computing base (TCB) small and outsourcing the bulk of the AI computation to untrusted accelerators.

Figures

Figures reproduced from arXiv: 2607.29221 by James Hsin-yu Chiang, Kari Kostiainen, Sheila Zingg, Srdjan Capkun.

Figure 1
Figure 1. Figure 1: Secure, confidential forward pass for a single transformer layer. The trusted client [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Wall-clock runtime of a single secure matrix-vector [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Structure of the masking scheme, consisting of private weights matrix [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Unmasking the outsourced matrix product. The GPU returns [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Secure approximate matrix multiplication outsourc [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Outsourcing efficiency: Measured client runtime [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Secure linear layer with input/outputs in the floating-point domain. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Per-layer error accumulation for Qwen2.5-72B [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Per-layer error accumulation: Consine similarity between activations of quantized and reference forward-pass (BF16) [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Per-token decoding latency for 70B-class mod [PITH_FULL_IMAGE:figures/full_fig_p012_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: System-prompt prefill runtime for 70B-class mod [PITH_FULL_IMAGE:figures/full_fig_p012_12.png] view at source ↗
Figure 10
Figure 10. Figure 10: Datacenter outsourcing topology: 𝑘 independent trusted clients, each a small TCB, elastically share a common pool of untrusted GPUs, secure outsourcing AI computations over fast interconnects. Secure outsourcing in an AI Datacenter. The common approach for confidential computation in datacenters is to deploy applica￾tions on a trusted computing base (TCB), realized by confidential computing technologies o… view at source ↗
Figure 13
Figure 13. Figure 13: Theoretical trusted-client work vs. local [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Network sensitivity: system-prompt prefill at [PITH_FULL_IMAGE:figures/full_fig_p017_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: System prefill and runtime latency under simu [PITH_FULL_IMAGE:figures/full_fig_p018_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: System prefill and runtime latency under simu [PITH_FULL_IMAGE:figures/full_fig_p018_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Remote outsourcing topology: a trusted client at [PITH_FULL_IMAGE:figures/full_fig_p018_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Per-layer error accumulation: Qwen2.5-32B (64 model layers). For 20 recorded runs, the 1-standard deviation band is [PITH_FULL_IMAGE:figures/full_fig_p020_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Per-layer error accumulation: Qwen2.5-72B (80 model layers). [PITH_FULL_IMAGE:figures/full_fig_p020_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Per-layer error accumulation: LLaMA-3-70B (80 layers). INT8 baseline collapses at [PITH_FULL_IMAGE:figures/full_fig_p021_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Per-layer error accumulation: DeepSeek-R1-Distill-LLaMA-70B (80 layers). [PITH_FULL_IMAGE:figures/full_fig_p021_21.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 10 linked inside Pith

  1. [1]

    Nir Ailon and Bernard Chazelle. 2006. Approximate Nearest Neighbors and the Fast Johnson-Lindenstrauss Transform. In38th Annual ACM Symposium on Theory of Computing (STOC). 557–563. https://dl.acm.org/doi/pdf/10.1145/ 1132516.1132597

    arXiv 2006

  2. [2]

    Martin R Albrecht, Rachel Player, and Sam Scott. 2015. On the concrete hardness of learning with errors.Cryptology ePrint Archive(2015). https://github.com/ malb/lattice-estimator

    2015

  3. [3]

    Michael Alekhnovich. 2003. More on average case vs approximation complex- ity. In44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings.IEEE, 298–307

    2003

  4. [4]

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

  5. [5]

    Fabrice Benhamouda, Caicai Chen, Shai Halevi, Yuval Ishai, Hugo Krawczyk, Tamer Mour, Tal Rabin, and Alon Rosen. 2025. Encrypted matrix-vector products from secret dual codes. InProceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security. 394–408

    2025

  6. [6]

    Mark Braverman and Stephen Newman. 2025. Practical Secure Delegated Linear Algebra with Trapdoored Matrices. InTheory of Cryptography Conference (TCC). Springer, 97–118. https://arxiv.org/abs/2502.13060

    arXiv 2025

  7. [7]

    Hao Chen, Miran Kim, Ilya Razenshteyn, Dragos Rotaru, Yongsoo Song, and Sameer Wagh. 2020. Maliciously secure matrix multiplication with applications to private deep learning. InInternational Conference on the Theory and Application of Cryptology and Information Security. Springer, 31–59. https://eprint.iacr.org/ 2020/451.pdf

    2020

  8. [8]

    Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. 2017. Homo- morphic encryption for arithmetic of approximate numbers. InInternational conference on the theory and application of cryptology and information security. Springer, 409–437. https://eprint.iacr.org/2016/421.pdf

    2017

  9. [9]

    Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. InAdvances in Neural Information Processing Systems (NeurIPS), Vol. 35. 30318–30332. https: //arxiv.org/abs/2208.07339

    Pith/arXiv arXiv 2022

  10. [10]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient Finetuning of Quantized LLMs. InAdvances in Neural Informa- tion Processing Systems (NeurIPS). https://arxiv.org/abs/2305.14314

    Pith/arXiv arXiv 2023

  11. [11]

    Fino and Algazi. 1976. Unified matrix treatment of the fast Walsh-Hadamard transform.IEEE Trans. Comput.100, 11 (1976), 1142–1146. https://ieeexplore.ieee. org/stamp/stamp.jsp?tp=&arnumber=1674569. 13 Chiang et al

    1976

  12. [12]

    Yang Gao, Gang Quan, Soamar Homsi, Wujie Wen, and Liqiang Wang. 2024. Secure and efficient general matrix multiplication on cloud using homomorphic encryption: Y. Gao et al.The Journal of Supercomputing80, 18 (2024), 26394– 26434

    2024

  13. [13]

    Zhongshu Gu, Enriquillo Valdez, Salman Ahmed, Julian James Stephen, Michael Le, Hani Jamjoom, Shixuan Zhao, and Zhiqiang Lin. 2025. NVIDIA GPU confidential computing demystified.arXiv preprint arXiv:2507.02770(2025). https://arxiv.org/pdf/2507.02770

    Pith/arXiv arXiv 2025

  14. [14]

    Kanav Gupta, Neha Jawalkar, Ananta Mukherjee, Nishanth Chandran, Divya Gupta, Ashish Panwar, and Rahul Sharma. 2024. SIGMA: Secure GPT Inference with Function Secret Sharing. In24th Privacy Enhancing Technologies Symposium (PETS). https://eprint.iacr.org/2023/1269

    2024

  15. [15]

    Xiaoqian Jiang, Miran Kim, Kristin Lauter, and Yongsoo Song. 2018. Secure out- sourced matrix computation and application to neural networks. InProceedings of the 2018 ACM SIGSAC conference on computer and communications security. 1209–1222. https://eprint.iacr.org/2018/1041.pdf

    2018

  16. [16]

    Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. 2018. {GAZELLE}: A low latency framework for secure neural network inference. In 27th USENIX security symposium (USENIX security 18). 1651–1669. https://www. usenix.org/system/files/conference/usenixsecurity18/sec18-juvekar.pdf

    2018

  17. [17]

    Qinfeng Li, Zhiqiang Shen, Zhenghan Qin, Yangfan Xie, Xuhong Zhang, Tianyu Du, Sheng Cheng, Xun Wang, and Jianwei Yin. 2024. Translinkguard: safeguard- ing transformer models against model stealing in edge deployment. InProceedings of the 32nd ACM international conference on multimedia. 3479–3488

    2024

  18. [18]

    Zhengyi Li, Kang Yang, Jin Tan, Wen-jie Lu, Haoqi Wu, Xiao Wang, Yu Yu, Derun Zhao, Yancheng Zheng, Minyi Guo, et al . 2024. Nimbus: Secure and efficient two-party inference for transformers.Advances in Neural Information Processing Systems37 (2024), 21572–21600

    2024

  19. [19]

    Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2024. Spinquant: Llm quantization with learned rotations.arXiv preprint arXiv:2405.16406(2024). https://arxiv.org/pdf/2405.16406

    Pith/arXiv arXiv 2024

  20. [20]

    Wen-jie Lu, Zhicong Huang, Zhen Gu, Jingyu Li, Jian Liu, Cheng Hong, Kui Ren, Tao Wei, and WenGuang Chen. 2023. Bumblebee: Secure two-party inference framework for large transformers.Cryptology ePrint Archive(2023). https: //eprint.iacr.org/2023/1678.pdf

    2023

  21. [21]

    Zoltán Ádám Mann, Christian Weinert, Daphnee Chabal, and Joppe W Bos. 2023. Towards practical secure neural network inference: the journey so far and the road ahead.Comput. Surveys56, 5 (2023), 1–37. https://eprint.iacr.org/2022/1483.pdf

    2023

  22. [22]

    Minghai Qin. 2024. The uniqueness of llama3-70b series with per-channel quan- tization.arXiv preprint arXiv:2408.15301(2024). https://arxiv.org/pdf/2408.15301

    Pith/arXiv arXiv 2024

  23. [23]

    Oded Regev. 2009. On lattices, learning with errors, random linear codes, and cryptography.Journal of the ACM (JACM)56, 6 (2009), 1–40

    2009

  24. [24]

    Kevin Scaman and Aladin Virmaux. 2018. Lipschitz regularity of deep neural networks: analysis and efficient estimation. InAdvances in Neural Information Processing Systems (NeurIPS), Vol. 31. https://arxiv.org/pdf/1805.10965

    Pith/arXiv arXiv 2018

  25. [25]

    2022.{SOTER}: Guarding black-box inference for general neural networks at the edge

    Tianxiang Shen, Ji Qi, Jianyu Jiang, Xian Wang, Siyuan Wen, Xusheng Chen, Shixiong Zhao, Sen Wang, Li Chen, Xiapu Luo, et al. 2022.{SOTER}: Guarding black-box inference for general neural networks at the edge. In2022 USENIX Annual Technical Conference (USENIX ATC 22). 723–738. https://www.usenix. org/system/files/atc22-shen.pdf

    2022

  26. [26]

    Spheron. 2026. GPU Networking for AI Clusters: InfiniBand vs RoCE vs Spectrum- X Decision Guide. Spheron Network Blog. https://www.spheron.network/blog/ gpu-networking-infiniband-roce-spectrum-x-guide

    2026

  27. [27]

    Yijun Sun, Xudong Liao, Songrun Xie, Hao Chen, Han Tian, Wenxue Li, Yiming Zhang, and Kai Chen. 2026. Multi-stage Flow Scheduling for LLM Serving.arXiv preprint arXiv:2603.17456(2026). https://arxiv.org/pdf/2603.17456

    arXiv 2026

  28. [28]

    Florian Tramèr and Dan Boneh. 2019. Slalom: Fast, Verifiable and Private Exe- cution of Neural Networks in Trusted Hardware. InInternational Conference on Learning Representations (ICLR). https://arxiv.org/abs/1806.03287

    Pith/arXiv arXiv 2019

  29. [29]

    Pengli Wang, Bingyou Dong, Yifeng Cai, Zheng Zhang, Junlin Liu, Huanran Xue, Ye Wu, Yao Zhang, and Ziqi Zhang. 2025. Game of Arrows: On the (In- )Security of Weight Obfuscation for On-Device TEE-Shielded LLM Partition Algorithms. In34th USENIX Security Symposium (USENIX Security 25). 279–298. https://www.usenix.org/system/files/usenixsecurity25-wang-pengli.pdf

    2025

  30. [30]

    Songhao Wu, Ang Lv, Xiao Feng, Yufei Zhang, Xun Zhang, Guojun Yin, Wei Lin, and Rui Yan. 2025. Polarquant: Leveraging polar transformation for efficient key cache quantization and decoding acceleration.arXiv preprint arXiv:2502.00527 (2025). https://arxiv.org/pdf/2502.00527

    Pith/arXiv arXiv 2025

  31. [31]

    Yongtong Wu, Shaoyuan Chen, Yinmin Zhong, Rilin Huang, Yixuan Tan, Wentao Zhang, Liyue Zhang, Shangyan Zhou, Yuxuan Liu, Shunfeng Zhou, et al. 2026. DualPath: Breaking the Storage Bandwidth Bottleneck in Agentic LLM Inference. arXiv preprint arXiv:2602.21548(2026). https://arxiv.org/pdf/2602.21548

    arXiv 2026

  32. [32]

    Rongwu Xu and Zhixuan Fang. 2024. Tempo: Confidentiality preservation in cloud-based neural network training. In2024 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–10

    2024

  33. [33]

    Yu Yu, Kang Yang, Xiao Wang, Anyu Wang, Tianrui Wang, Hanlin Liu, Xinpeng Hao, and Juanru Li. 2025. Estimator of LPN problems over any finite fields and power-of-two rings for PCG and MPC applications. https://lpnestimator.com. https://lpnestimator.com/

    2025

  34. [34]

    encryption

    Itamar Zimerman, Allon Adir, Ehud Aharoni, Matan Avitan, Moran Baruch, Nir Drucker, Jenny Lerner, Ramy Masalha, Reut Meiri, and Omri Soceanu. 2024. Power-Softmax: Towards Secure LLM Inference over Encrypted Data.arXiv preprint arXiv:2410.09457(2024). https://arxiv.org/abs/2410.09457. A Proofs A.1 Supporting lemmas The two supporting lemmas referenced by T...

    Pith/arXiv arXiv 2024

  35. [2024]

    https://arxiv.org/pdf/ 2404.00456

    Quarot: Outlier-free 4-bit inference in rotated llms.Advances in Neural Information Processing Systems37 (2024), 100213–100240. https://arxiv.org/pdf/ 2404.00456

    Pith/arXiv arXiv 2024