REVIEW 3 major objections 5 minor 26 references
This paper claims that online key-switching overhead in two-party private Transformer inference can be largely moved offline by a packing-independent attention decomposition, and that a fused relinearization-and-rotation routine saves one M
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 02:03 UTC pith:2ZJ5BEPG
load-bearing objection Correct but uneven: the attention decomposition is a known masking trick applied more broadly, while the fused key-switch saving rests on an unanalyzed τ²→s evaluation key and the evaluation is analytical only. the 3 major comments →
General Techniques for Reducing Key-Switching Overhead in Privacy-Preserving Two-Party Transformer Inference
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central discovery is an algebraic reorganization of secure attention: writing the input as X = U + (X−U), the attention score matrix QK^T expands into four terms, one of which (QuKu^T) can be fully precomputed, two of which reduce to ciphertext-plaintext products when the masked input is known, and the last of which is plaintext-only. This moves the bulk of key switching into preprocessing in a packing-independent way. Independently, the paper shows that for the common pattern ciphertext-multiplication-followed-by-rotation, relinearization and rotation can be interleaved: after the automorphism, a fused evaluation key mapping both τ and τ² back to the original secret key converts the deg
What carries the argument
The two load-bearing mechanisms are the attention decomposition identity QK^T = QuKu^T + Qu((X−U)Wk)^T + (X−U)WqKu^T + (X−U)WqWk^T(X−U)^T, which shifts ciphertext work offline, and the fused key-switch primitive for RNS-CKKS that uses an evaluation key for both τ→s and τ²→s to perform relinearization and rotation in one ModDown stage instead of two.
Load-bearing premise
Algorithm 5's claimed one-ModDown saving depends on the availability and security of a fused evaluation key that can switch both the rotated key τ and its square τ² back to the original secret key; if such key material is not generated by standard CKKS key generation or leaks information, the fused pipeline's benefit collapses.
What would settle it
Implement Algorithm 5 in a standard RNS-CKKS library and measure the number of ModDown operations and wall-clock time for a multiply-then-rotate vs. the conventional pipeline; also check whether the τ²→s evaluation key can be produced by the standard key-generation interface without revealing the secret key. If the fused version is not faster or no standard method for the key exists, the cost claim fails.
If this is right
- In hybrid FHE-MPC systems, attention-score computation can be made free of online key-switching operations (the paper's counts for BLB drop from 346 to 0).
- The storage-communication trade-off replaces a precomputed ciphertext set of thousands of ciphertexts with a handful of ciphertexts plus small online messages, making the method viable on constrained servers.
- The fused key-switch technique applies to any RNS-CKKS multiply-then-rotation pattern, independent of packing, so it can be retrofitted into existing systems such as BOLT, THOR, BLB, and ARION.
- In non-interactive FHE-only inference the benefits are confined to the first Transformer layer unless interaction or bootstrapping is allowed.
- The techniques are orthogonal to packing, so security and correctness are preserved under any layout that satisfies additive homomorphism (e.g., standard RLWE packings).
Where Pith is reading between the lines
- If the fused τ²→s evaluation key can be obtained at standard key-generation cost, the same one-ModDown saving may extend to other automorphism-based operations, such as complex conjugation or large-step rotations in baby-step giant-step algorithms, further cutting key-switch counts in matrix multiplication.
- The attention decomposition only requires the server's weight matrices to be public; a similar offline/online split could be applied to the feed-forward network's linear layers in hybrid settings, moving more key switches to preprocessing.
- The analytical cost model counts key-switch operations as the metric; actual runtime gains depend on whether ModUp/ModDown are the true bottleneck in optimized GPU implementations, so the promised 'significant reduction' still needs to be confirmed on real clock times.
- Because the security proof for the trade-off protocols is a high-level simulation argument relying on IND-CPA and uniform masks, a concrete instantiation would need to spell out the exact sampling distribution of the masks and the conversion protocol between FHE ciphertexts and MPC shares.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes three packing-independent techniques to reduce key-switching overhead in two-party Transformer inference: a preprocessing-assisted attention decomposition (Algorithm 1; Eq. (3)), storage-communication trade-off protocols (Algorithms 2-4), and a fused relinearization-and-rotation procedure for RNS-CKKS (Algorithm 5). The authors claim that these techniques significantly reduce online key-switching operations and can be integrated into existing FHE-based and hybrid FHE-MPC systems without changing packing strategies. The manuscript explicitly states that it is an analytical-only arXiv version, with implementation and experimental evaluation deferred to a later revision.
Significance. If the stated claims are fully established, the paper would make a useful contribution: the attention decomposition in Eq. (3) is a correct algebraic identity, the storage-communication trade-off is a sensible engineering idea, and a fused key-switch that saves one ModDown would be of practical interest to CKKS-based secure inference. The paper is also honest in stating its current scope. However, the present version does not yet support several load-bearing claims: the key-switching counts in Table II are asserted rather than derived, the fused key-switching method depends on a non-standard evaluation key whose generation, cost, and security are not analyzed, and the security proof is a high-level sketch rather than a formal simulation argument.
major comments (3)
- [Section V.C] The security analysis is a high-level simulation sketch and does not establish the claimed indistinguishability. The ideal functionality and simulation strategy are not defined with enough precision to verify that the real views and ideal views match. In particular, the client in Algorithms 2 and 3 receives and decrypts ciphertexts such as [[Q_u W_k^T - R_s]] and [[W_q K_u^T - R_s']] (Algorithm 2, lines 2-3; Algorithm 3, lines 2-3). The client's real view therefore includes these decrypted plaintexts, and the simulator for the client must generate ciphertexts whose decryptions have the correct distribution conditioned on the client's input U. The paper does not provide such a simulation or a reduction to RLWE. It also does not account for the reuse of masks across messages (e.g., R_s appears in the server's preprocessing message and again in the online reconstruction step of Algorithm 3,
- [Section VI.B and Algorithm 5] The fused key-switching technique relies on the availability of a fused evaluation key evk_fused = (evk_{τ→s}, evk_{τ²→s}), where τ = σ_t(s). Standard RNS-CKKS key generation provides relinearization keys (s²→s) and rotation keys (σ_t(s)→s); it does not automatically provide a key-switching key from σ_t(s)² to s. The paper neither explains how to derive this key from standard keys nor analyzes its generation/storage cost. More importantly, publishing an RLWE encryption of τ² = σ_t(s)² under the same secret s is a key-dependent message; IND-CPA security of ordinary RLWE does not directly cover this. Since the entire advantage claimed in Section VI.D (one ModDown savings) depends on this key, the claimed improvement is not currently established. The authors should either give an explicit key-generation procedure and a security proof under a stated assumption, or demonstrate that such keys
- [Section VII and Table II] The evaluation is entirely analytical, and the numbers in Table II are asserted without derivation. The table reports exact key-switching counts (e.g., 1536, 346, 128, 2432), storage counts (e.g., 8384, 8192, 257, 384, 6, 3), and communication costs, but the paper does not present formulas connecting these numbers to the model dimensions (m, d, H, N) or to the sizes of the precomputed transformation sets C_Q, C_K, C_V in Section IV.C. Without such derivations, the claimed 'significant reduction' cannot be reproduced or independently verified. The short note in the preamble also confirms that no implementation is included; this is a limitation that should be acknowledged in the evaluation section and addressed before the results can be considered fully supported.
minor comments (5)
- [Algorithm 1, line 9] The term 'sf tmx' appears to be a typo for 'softmax'; please correct.
- [Table II] The column headers are difficult to parse: 'Slot Key Switch' is ambiguous and '215'/'214' should be clearly labeled (e.g., as 2^15 and 2^14 or as specific parameters). Splitting the 'Slot' and 'Key Switch' columns would improve readability.
- [Section IV.B] The claim that the decomposition is packing-independent and can be incorporated into 'any RLWE-based secure attention computation scheme' is stronger than what Eq. (4) shows. The argument assumes that plaintext-ciphertext multiplications with the relevant encodings are available in each target scheme. Please qualify the generality claim.
- [References] Some references are incomplete or informal, e.g., [22] lacks a venue and year. Please update the bibliography to a consistent format.
- [Overall] The opening note states that an updated version with end-to-end implementation will be released. A journal submission should either include that material or clearly state in the abstract and conclusion that the current results are analytical and require experimental validation.
Circularity Check
No significant circularity: the attention decomposition is an algebraic identity and the fused key-switch saving is a stated component-count reduction, not a construction that assumes its conclusion.
full rationale
The paper's central derivation is the algebraic attention decomposition in Eq. (1)/Eq. (3): QK^T = Q_u K_u^T + Q_u((X-U)W_k)^T + (X-U)W_q K_u^T + (X-U)W_q W_k^T (X-U)^T, obtained by distributing X = U + (X-U) through the matrix products. This is a mathematical identity, not a fitted or empirical result. The preprocessing algorithms (Algorithms 1-4) shift ciphertext operations between offline and online phases by construction, and the claimed reductions in online key-switching are direct consequences of which operations are moved; the storage-communication trade-offs explicitly replace stored ciphertexts with masked plaintext messages trading one cost for another. The fused relinearization-and-rotation section also has independent content: after multiplication and automorphism the ciphertext is under (1, tau, tau^2), and the paper uses linearity of key switching to combine KS_{tau->s}(d1) + KS_{tau^2->s}(d2) into one ModDown stage. The claimed saving of one ModDown is a stated operation-count comparison, not a prediction forced by a fitted parameter. The main limitations are evidential, not circular: the paper notes it is an analytical-evaluation version, and Algorithm 5 relies on a fused evaluation key containing a tau^2->s key whose generation, storage, and security are not analyzed. That is a substantive implementation/security risk, but it is not an instance of the derivation reducing to its own inputs, and no load-bearing self-citation is present. Therefore no circular step meets the quoted-reduction standard.
Axiom & Free-Parameter Ledger
free parameters (2)
- Key-switch counts in Table II (1536, 346, 128, 2432, 8384, 8192, 257, 384, 6, 3) =
Asserted integers; no derivation provided
- Sizes of precomputed transformation sets CQ, CK, CV (n, n', n'') =
Not specified in text; only totals appear in Table II
axioms (5)
- domain assumption CKKS/RLWE is IND-CPA secure
- domain assumption Encoding layout P'' preserves additive homomorphism (Eq. 2)
- domain assumption Key switching dominates homomorphic matrix multiplication cost; reducing key-switch count reduces latency proportionally
- ad hoc to paper Fused evaluation keys evk_{τ→s} and evk_{τ²→s} can be generated and safely published
- domain assumption Semi-honest two-party adversarial model
invented entities (1)
-
Fused evaluation key evk^{(i)}_{fused} = (evk^{(i)}_{τ→s}, evk^{(i)}_{τ²→s})
no independent evidence
read the original abstract
In secure two-party Transformer inference, linear layers are typically evaluated using Fully Homomorphic Encryption (FHE) through plaintext-ciphertext or ciphertext-ciphertext matrix multiplications, where key switching primarily occurs and dominates computational overhead in both FHE-based and hybrid FHE-MPC systems. Existing optimizations rely heavily on packing-specific algorithms, limiting their general applicability. Targeting this overhead from a packing-independent perspective, we propose a preprocessing-assisted method for secure attention computation. By decomposing attention into precomputable operations and online interactions, this method reduces online inference-phase key switching without modifying existing packing strategies. However, the first method shifting key switching offline introduces additional storage requirements. To address this, we propose storage-communication trade-off techniques that replace large precomputed ciphertexts with modest online communication, enabling flexible deployment under varying resource constraints. While ciphertext-ciphertext matrix multiplication is offloaded to the preprocessing phase in hybrid schemes and the first layer of FHE-based schemes, these operations still persist in the offline stage and subsequent FHE layers. To further optimize it, we propose a fused key-switch technique targeting the multiplication-followed-by-rotation pattern, which frequently arises in existing RNS-CKKS matrix multiplication schemes. By combining relinearization and rotation into a single procedure, this technique reduces the associated computation costs. Analytical evaluations demonstrate that our proposed techniques significantly reduce online key-switch overhead and provide flexible trade-offs between storage and communication without requiring modifications to existing packing strategies.
Reference graph
Works this paper leans on
-
[1]
Powerforme r: Efficient and high-accuracy privacy-preserving language model with homomor- phic encryption
Dongjin Park, Eunsang Lee, and Joon-Woo Lee. Powerforme r: Efficient and high-accuracy privacy-preserving language model with homomor- phic encryption. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL) , pages 11090–11111, 2025
2025
-
[2]
ARION: Attention-optimized transformer inf erence on encrypted data
Linhan Y ang, Jingwei Chen, Wangchen Dai, Shuai Wang, Wen yuan Wu, and Y ong Feng. ARION: Attention-optimized transformer inf erence on encrypted data. Cryptology ePrint Archive, Paper 2025/227 1, 2025
2025
-
[3]
Breaking the layer barrier: remodeling pr ivate transformer inference with hybrid CKKS and MPC
Tianshi Xu, Wen-jie Lu, Jiangrui Y u, Yi Chen, Chenqi Lin, Runsheng Wang, and Meng Li. Breaking the layer barrier: remodeling pr ivate transformer inference with hybrid CKKS and MPC. In Proceedings of the USENIX Security Symposium , pages 1–12, 2025
2025
-
[4]
BOLT: Privacy-preserving, accurate and efficie nt inference for transformers
Qi Pang, Jinhao Zhu, Helen M¨ ollering, Wenting Zheng, an d Thomas Schneider. BOLT: Privacy-preserving, accurate and efficie nt inference for transformers. In Proceedings of the IEEE Symposium on Security and Privacy (SP) , pages 4753–4771, 2024
2024
-
[5]
THOR: Secure transformer inference with homomorphic encryption
Jungho Moon, Dongwoo Y oo, Xiaoqian Jiang, and Miran Kim. THOR: Secure transformer inference with homomorphic encryption . In Proceed- ings of the ACM SIGSAC Conference on Computer and Communicat ions Security (CCS) , pages 3765–3779. ACM, 2025
2025
-
[6]
Iron: Private inference on transformers
Meng Hao, Hongwei Li, Hanxiao Chen, Pengzhi Xing, Guowen Xu, and Tianwei Zhang. Iron: Private inference on transformers. In Advances in Neural Information Processing Systems (NeurIPS) , pages 15718–15731, 2022
2022
-
[7]
MOAI: Module-optimizing architecture for non-interactive secu re transformer inference
Linru Zhang, Xiangning Wang, Jun Jie Sim, Zhicong Huang, Jiahao Zhong, Huaxiong Wang, Pu Duan, and Kwok-Y an Lam. MOAI: Module-optimizing architecture for non-interactive secu re transformer inference. In The F ourteenth International Conference on Learning Representations, 2026
2026
-
[8]
Secureml: A system for scalable privacy-preserving machine learning
Payman Mohassel and Y upeng Zhang. Secureml: A system for scalable privacy-preserving machine learning. In Proceedings of the IEEE Symposium on Security and Privacy (SP) , pages 19–38, 2017
2017
-
[9]
Cryptonets: Applying n eural networks to encrypted data with high throughput and accurac y
Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin L auter, Michael Naehrig, and John Wernsing. Cryptonets: Applying n eural networks to encrypted data with high throughput and accurac y. In Proceedings of the International Conference on Machine Lea rning (ICML), pages 201–210, 2016
2016
-
[10]
Deep learning with diff erential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMah an, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with diff erential privacy. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS) , pages 308–318, 2016
2016
-
[11]
Slalom: Fast, verifiable and private execution of neural networks in trusted enclaves
Florian Tram` er and Dan Boneh. Slalom: Fast, verifiable and private execution of neural networks in trusted enclaves. In Proceedings of the International Conference on Learning Representations (IC LR), pages 1–13, 2019
2019
-
[12]
ABY2.0: Improved mixed-protocol secure two-party computa tion
Arpita Patra, Thomas Schneider, Ajith Suresh, and Hoss ein Y alame. ABY2.0: Improved mixed-protocol secure two-party computa tion. In Proceedings of the USENIX Security Symposium , pages 2165–2182, 2021
2021
-
[13]
Fully homomorphic encryption using idea l lattices
Craig Gentry. Fully homomorphic encryption using idea l lattices. In Proceedings of the Annual ACM Symposium on Theory of Computi ng (STOC), pages 169–178. ACM, 2009
2009
-
[14]
MPCFORMER: Fast, performant and private transfo rmer inference with MPC
Dacheng Li, Hongyi Wang, Rulin Shao, Han Guo, Eric Xing, and Hao Zhang. MPCFORMER: Fast, performant and private transfo rmer inference with MPC. In Proceedings of the International Conference on Learning Representations (ICLR) , pages 1–10, 2023
2023
-
[15]
MPCViT: Searching for MPC- 13 friendly vision transformer with heterogeneous attention
Wenyuan Zeng, Meng Li, Wenjie Xiong, Wen Jie Lu, Jin Tan, Runsheng Wang, and Ru Huang. MPCViT: Searching for MPC- 13 friendly vision transformer with heterogeneous attention . arXiv preprint arXiv:2211.13955, pages 1–9, 2022
Pith/arXiv arXiv 2022
-
[16]
Secure transformer inference made non-interactive
Jiawen Zhang, Xinpeng Y ang, Lipeng He, Kejia Chen, Wen j ie Lu, Yinghao Wang, Xiaoyang Hou, Jian Liu, Kui Ren, and Xiaohu Y an g. Secure transformer inference made non-interactive. In Proceedings of the Network and Distributed System Security Symposium (NDS S), pages 1–16, 2024
2024
-
[17]
Converting transformers to polynom ial form for secure inference over homomorphic encryption
Itamar Zimerman, Moran Baruch, Nir Drucker, Gilad Ezov , Omri Soceanu, and Lior Wolf. Converting transformers to polynom ial form for secure inference over homomorphic encryption. In Proceedings of the International Conference on Machine Learning (ICML) , pages 1–13, 2024
2024
-
[18]
Power-softmax: Towards secure LLM inference over encrypted data
Itamar Zimerman, Allon Adir, Ehud Aharoni, Matan Avita n, Moran Baruch, Nir Drucker, Jenny Lerner, Ramy Masalha, Reut Meiri , and Omri Soceanu. Power-softmax: Towards secure LLM inference over encrypted data. arXiv preprint arXiv:2410.09457 , pages 1–11, 2024
Pith/arXiv arXiv 2024
-
[19]
Transformer-bas ed language models and homomorphic encryption: An intersection with BE RT-tiny
Lorenzo Rovida and Alberto Leporati. Transformer-bas ed language models and homomorphic encryption: An intersection with BE RT-tiny. In Proceedings of the ACM International W orkshop on Security a nd Privacy Analytics (IWSPA) , pages 3–13. ACM, 2024
2024
-
[20]
Tricycle: Private transformer inference with tricyclic en codings
Lawrence Lim, Vikas Kalagi, Divyakant Agrawal, and Amr El Abbadi. Tricycle: Private transformer inference with tricyclic en codings. Cryp- tology ePrint Archive, Paper 2025/1200, 2025
2025
-
[21]
Euston: Efficient and user-friendly secure tr ansformer inference with non-interactivity
Xinwen Gao, Shaojing Fu, Lin Liu, Zhuotao Liu, Y uchuan L uo, and Y ongjun Wang. Euston: Efficient and user-friendly secure tr ansformer inference with non-interactivity. Cryptology ePrint Arch ive, Paper 2026/046, 2026
2026
-
[22]
En cformer: Secure and efficient transformer inference over encrypted d ata, 2026
Y ufan Zhu, Chao Jin, Khin Mi Mi Aung, and Xiaokui Xiao. En cformer: Secure and efficient transformer inference over encrypted d ata, 2026
2026
-
[23]
A ttention is all you need
Ashish V aswani, Noam Shazeer, Niki Parmar, Jakob Uszko reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. A ttention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), pages 2–8, 2017
2017
-
[24]
Homo- morphic encryption for arithmetic of approximate numbers
Jung Hee Cheon, Andrey Kim, Miran Kim, and Y ongsoo Song. Homo- morphic encryption for arithmetic of approximate numbers. In Advances in Cryptology – ASIACRYPT , pages 409–437. Springer, 2017
2017
-
[25]
A full RNS variant of approximate homomorphic encr yption
Jung Hee Cheon, Kyoohyung Han, Andrey Kim, Miran Kim, an d Y ong- soo Song. A full RNS variant of approximate homomorphic encr yption. In Carlos Cid and Michael J. Jacobson Jr., editors, Selected Areas in Cryptography – SAC 2018 , pages 347–368. Springer International Publishing, 2019
2018
-
[26]
Efficient bootstrapping for approxima te homomor- phic encryption with non-sparse keys
Jean-Philippe Bossuat, Christian Mouchet, Juan Tronc oso-Pastoriza, and Jean-Pierre Hubaux. Efficient bootstrapping for approxima te homomor- phic encryption with non-sparse keys. In Annual International Con- ference on the Theory and Applications of Cryptographic Tec hniques, pages 587–617. Springer, 2021
2021
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.