REVIEW 4 major objections 5 minor 1 cited by
Private LoRA Fine-tuning of Open-Source LLMs with Homomorphic Encryption
T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read An interactive client-server protocol uses homomorphic encryption to fine-tune an open-source LLM privately, with 8-bit quantization matching FP32-level convergence.
desk verdict Solid HE+LoRA feasibility demo with a thorough quantization study; the full-training fidelity claim rests on only five HE steps and needs either more evidence or tighter scoping. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing primitive is the encrypted vector–clear matrix multiplication $W\cdot[x]_{\mathrm{HE}}$, built from RLWE ciphertexts with coefficient packing. The input vector is split into blocks matching polynomial size $N=2048$; the server multiplies each encrypted block by the corresponding cleartext weight block, extracts each dot-product coefficient via SampleExtract into LWE ciphertexts, then packs the per-column LWE results back into one RLWE ciphertext using KeySwitching and homomorphic rotations, finishing with modulus switching for bandwidth. The second essential ingredient is the quantization recipe: symmetric quantization with zero zero-points, so the server evaluates only a single integer dot product, using dynamic per-token ranges for activations and static per-channel ranges for weights at 8 bits.
What would settle it
Run the complete 2500-step LoRA fine-tuning of Llama-3.2-1B entirely under HE with the paper's parameters ($\beta=27$, $\gamma=12$, $N=2048$) and compare the final validation perplexity and the per-step loss trajectory against the 8-bit DTok-SC cleartext run; a divergence in loss or a perplexity noticeably above 1.2391 would show that HE noise and truncation are not benign over full training.
Extended reading notes
Core claim
The central claim is that LoRA fine-tuning of a public LLM can be split into a server-side encrypted path and a client-side cleartext path without losing training quality. Under an honest-but-curious server, the protocol guarantees that the server never sees the private data: for every linear layer the server computes $W\cdot[x]_{\mathrm{HE}}$ on encrypted activations and returns a packed, modulus-reduced ciphertext, while the client decrypts, adds its own low-rank contribution $UDx+b$, applies the activation, and re-encrypts for the next layer. The paper further claims that with 8-bit symmetric quantization using dynamic per-token activation ranges and static per-channel weight ranges (DTok-SC), a 2500-step fine-tuning of Llama-3.2-1B reaches a validation perplexity of 1.2391, statistically indistinguishable from the FP32 baseline of 1.2381. It also reports that the HE execution reproduces the quantized cleartext loss trajectory over the first five training steps and that decrypted dot products keep their 12 most significant bits essentially error-free (below 1% bit-error rate) for input dimensions up to 8192.
Load-bearing premise
The protocol rests on the assumption that the HE parameters ($\beta=27$, $\gamma=12$, $N=2048$) keep enough correct bits in every decrypted dot product that the least-significant-bit truncation and quantization noise never destabilize a long training run, a property tested on five training steps and random-vector dot products rather than on a full 2500-step HE trajectory.
Editorial extensions
If this is right
- A data owner can fine-tune a public LLM on sensitive records while the remote server observes only ciphertexts, and the resulting LoRA adapters never leave the client.
- The 8-bit DTok-SC quantization recipe is sufficient for near-FP32 fine-tuning, so 16-bit precision is unnecessary when this granularity and dynamic range are used.
- The client hardware requirement essentially disappears (about 0.025 MFLOP/s per step), so a machine without a GPU can drive the training.
- Training latency scales roughly linearly with tokens, and because token computations are independent, spreading the work across multiple identical servers divides the per-token cost accordingly.
- Per-step bandwidth stays modest (about 267 kB per HE linear-layer invocation for 16 tokens), keeping the scheme usable over ordinary internet connections.
Reading between the lines
- A forward-only version of this protocol would cost roughly half the training-step time, so the same primitive likely extends to private inference or confidential generation with no cryptographic redesign.
- The full 2500-step trajectory has not been run under HE; if error accumulation proves non-benign, a periodic re-encryption or noise-refresh step would be needed, which the current protocol does not include.
- The split only protects the client's data; because the client necessarily decrypts $Wx$ in cleartext, the scheme would not keep a proprietary base model confidential from the client.
- Whether the 8-bit DTok-SC recipe remains near-FP32 on larger models, such as 7B or 70B classes, is an open empirical question since activation ranges and gradient dynamics change with scale.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces an interactive protocol for private LoRA fine-tuning of open-source LLMs using homomorphic encryption (HE). The server computes linear operations involving the public base weights on encrypted activations, while the client holds the private LoRA adapters and performs non-linear operations. The protocol is implemented with efficient packing, modulus switching, and GPU kernels, and is evaluated on Llama-3.2-1B. The authors report that an 8-bit quantization scheme with dynamic per-token activations and static per-channel weights (DTok-SC) achieves near-FP32 convergence in cleartext (perplexity 1.2391 vs 1.2381), that the HE execution matches cleartext loss over the first five training steps, and that the server-side HE computation costs ~216 s per token per training step on a single RTX 4060 Laptop GPU, with negligible client compute.
Significance. If the claims hold, this is a notable step toward privacy-preserving fine-tuning of large open-source LLMs, demonstrating that a client with minimal compute can outsource the bulk of training to an HE-enabled server while keeping the LoRA weights and sensitive data confidential. The cleartext quantization ablation is thorough and the 8-bit DTok-SC result is a strong, practically useful finding. The protocol transparently builds on Cheetah [12] and adds communication-efficiency optimizations. However, the HE fidelity evidence is limited to five training steps, the backward pass through the base model is not described, and the cleartext transmission of per-token quantization scales conflicts with the stated security guarantee. These gaps must be addressed before the core feasibility claim is fully established.
major comments (4)
- [§6.2, Fig. 3] The claim that HE execution "faithfully reproduces" cleartext training dynamics is supported by only five optimization steps. The paper's own cleartext ablation in §6.1(a) shows that 8-bit ST-ST is stable for about 700 steps and then diverges, demonstrating that training instabilities can appear only after many steps. A five-step overlap therefore cannot rule out slow accumulation of HE-specific noise, truncation errors, or quantization drift that could cause divergence at later steps. The authors should either run a full 2500-step HE training (even at reduced scale) or provide a rigorous error-budget analysis that bounds the per-step noise accumulation over hundreds of steps; otherwise the claim that "ciphertext noise growth, modulus switching, and homomorphic operations do not materially perturb gradient computations" is overstated.
- [§6.2, Fig. 4] The bit-error analysis uses random integer vectors rather than activations and gradients drawn from actual training distributions. Error rates for the MSBs under 1% on random vectors do not directly establish that the γ=12 MSB guarantee holds for the specific magnitudes, signs, and correlation structures encountered in LoRA fine-tuning of Llama-3.2-1B. The impact of LSB noise on the optimizer also depends on how the decrypted outputs are re-quantized with dynamic per-token scales. The analysis should be repeated with representative intermediate tensors from a real fine-tuning run, or the conclusions should be limited to the random-vector setting.
- [§4.1 / §6.3] The protocol description covers only the forward pass through the base weights (Eq. 5 and the packing in §4.3). The paper claims that a full training step includes "multiple encrypted vector-matrix products per transformer layer during both forward and backward passes," but the backward pass is never specified. To support private fine-tuning, the authors must describe how gradients with respect to activations are computed under HE (e.g., using W^T·[grad]_HE), how the client obtains the gradients needed to update the LoRA adapters, and how these operations are packed and communicated. Without this, it is unclear whether the reported 3458-second timings actually correspond to a complete backpropagation step.
- [§4.2 / §5.4] The paper states in §4.2 that "dynamic quantization parameters for activations sent to the server must be computed by the client and also sent," meaning per-token activation scales are transmitted in the clear. This appears to contradict the Section 5.4 claim that "the server learns no information about the client's private data." The scale for each token depends on the token's activation extremes and thus reveals statistical information about the private activations and the underlying training data. The authors should either revise the security model to acknowledge and bound this leakage (e.g., via differential privacy on the scales) or modify the protocol so the scales are not exposed to the server.
minor comments (5)
- [Abstract / §6.2] The abstract says "demonstrating convergence results using HE-compatible quantization," but the convergence results in §6.1 are for quantized cleartext training, while the HE execution is demonstrated over only five steps. The wording should be adjusted to avoid implying that full HE convergence was demonstrated.
- [§4.4] The values of β=27 and γ=12 are stated without a supporting derivation or analysis. The authors should explain how these parameters were chosen and why they guarantee the claimed precision for the actual dot-product dimensions and quantization scheme used in the experiments.
- [§6.3] The statement that latency "scales approximately linearly with the total number of input tokens (B×C) and the output dimension (dout)" is presented without direct evidence; Table 4 reports only single-token latencies. A brief scaling experiment or a more careful cost model would strengthen this claim.
- [Eq. (10)] The FLOPs estimate assumes the backward pass costs roughly twice the forward pass, which is a common rule of thumb but may not hold precisely for LoRA with HE-based matmuls. The client-side FLOPs estimate is not load-bearing, but a more precise accounting would be useful.
- [§6.4] The qualitative results are based on "converged quantized cleartext models as a proxy for HE results," which is acknowledged. It would be valuable to note explicitly that the HE path has not yet been used to produce a fully fine-tuned model, so the qualitative outputs do not validate the HE pipeline end-to-end.
Circularity Check
No circularity found: the HE fidelity and convergence claims are tested against external cleartext/FP32 baselines; self-citations are not load-bearing.
full rationale
The paper's central derivation chain is self-contained. The HE vector-matrix protocol is explicitly presented as a variant of Cheetah [12] ('Our approach is similar to [12] but adds steps for enhanced communication efficiency'), and the contribution is an engineering integration, not a result that reduces to its assumptions. The convergence claim for 8-bit DTok-SC is established by a cleartext ablation against an FP32 baseline (Section 6.1, Table 2), and the DTok-SC recipe is chosen from that comparison rather than being a fitted parameter renamed as a prediction. The HE fidelity claim in Section 6.2 is an empirical comparison of the HE execution to a separate 8-bit quantized cleartext run (Figure 3) plus bit-error measurements on decrypted dot products (Figure 4); this is independent evidence, albeit limited to five training steps and random-vector dot products. The five-step test is an evidentiary limitation and an extrapolation, not a definitional circularity. The self-citation [19] appears only in the related-work survey of non-interactive encrypted training and is not used to justify the protocol's correctness or convergence. No equation is defined in terms of the quantity it is claimed to predict, and no fitted value is relabeled as a prediction. Therefore the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (4)
- Crypto precision parameters (β, γ, N, q_in, q_out) =
β=27, γ=12, N=2048, q_in=39 bits, q_out=26 bits
- LoRA rank r =
8
- LoRA scaling α =
32
- Quantization bit-width and recipe =
8-bit symmetric, dynamic per-token activations and static per-channel weights (DTok-SC)
assumptions (5)
- standard math RLWE is computationally hard (128-bit security)
- domain assumption Server is honest-but-curious
- domain assumption Client machine is secure and trusted
- domain assumption DTok-SC 8-bit quantization preserves model fidelity through a full fine-tuning run
- domain assumption Base model weights W are public and known to both parties
Cite this review
Pith. "Pith review of Private LoRA Fine-tuning of Open-Source LLMs with Homomorphic Encryption." pith.science (2026). https://pith.science/paper/7V2JM3DG
@misc{pith2026250507329,
author = {Pith},
title = {Pith review of: Private LoRA Fine-tuning of Open-Source LLMs with Homomorphic Encryption},
year = {2026},
howpublished = {\url{https://pith.science/paper/7V2JM3DG}},
note = {Machine review of arXiv:2505.07329}
}
read the original abstract
Preserving data confidentiality during the fine-tuning of open-source Large Language Models (LLMs) is crucial for sensitive applications. This work introduces an interactive protocol adapting the Low-Rank Adaptation (LoRA) technique for private fine-tuning. Homomorphic Encryption (HE) protects the confidentiality of training data and gradients handled by remote worker nodes performing the bulk of computations involving the base model weights. The data owner orchestrates training, requiring minimal local computing power and memory, thus alleviating the need for expensive client-side GPUs. We demonstrate feasibility by fine-tuning a Llama-3.2-1B model, presenting convergence results using HE-compatible quantization and performance benchmarks for HE computations on GPU hardware. This approach enables applications such as confidential knowledge base question answering, private codebase fine-tuning for AI code assistants, AI agents for drafting emails based on a company's email archive, and adapting models to analyze sensitive legal or healthcare documents.
Figures
Figures from the paper (1 more)
Forward citations
Cited by 1 Pith paper
-
Attestable Audits: Verifiable AI Safety Benchmarks Using Trusted Execution Environments
A TEE-based protocol for cryptographically verifiable AI safety benchmark results, demonstrated on Llama-3.1 with AWS Nitro Enclaves.
Reference graph
Works this paper leans on
-
[12]
In: 31st USENIX Security Symposium (USENIX Security 22)
Huang, Z., jie Lu, W., Hong, C., Ding, J.: Cheetah: Lean and fast secure Two-Party deep neural net- work inference. In: 31st USENIX Security Symposium (USENIX Security 22). pp. 809–826. USENIX Association, Boston, MA (Aug 2022), https://www.usenix.org/conference/usenixsecurity22/ presentation/huang-zhicong
work page 2022
-
[1]
https://siboehm.com/ articles/22/CUDA-MMM, accessed: 2024-10-30
How to optimize a cuda matmul kernel for cublas-like performance: a worklog. https://siboehm.com/ articles/22/CUDA-MMM, accessed: 2024-10-30
work page 2024
-
[2]
arXiv preprint arXiv:2407.21783 (2024), https://arxiv.org/ abs/2407.21783 15
AI, M.: The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024), https://arxiv.org/ abs/2407.21783 15
arXiv 2024
-
[3]
Bergamaschi, F., Halevi, S., Halevi, T.T., Hunt, H.: Homomorphic training of 30,000 logistic regression models. In: Applied Cryptography and Network Security: 17th International Conference, ACNS 2019, Bogota, Colombia, June 5–7, 2019, Proceedings 17. pp. 592–611. Springer (2019)
work page 2019
-
[4]
BMC Medical Genomics 11(4), 86 (Oct 2018), https://doi.org/10.1186/s12920-018-0398-y
Bonte, C., Vercauteren, F.: Privacy-preserving logistic regression training. BMC Medical Genomics 11(4), 86 (Oct 2018), https://doi.org/10.1186/s12920-018-0398-y
-
[5]
Journal of Cryptology 33(1), 34–91 (2020)
Chillotti, I., Gama, N., Georgieva, M., Izabach` ene, M.: TFHE: fast fully homomorphic encryption over the torus. Journal of Cryptology 33(1), 34–91 (2020)
work page 2020
-
[6]
Advances in Neural Information Processing Systems 36 (2024)
Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L.: Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems 36 (2024)
2024
-
[7]
arXiv preprint arXiv:2404.05182 (2024)
Gao, C., Zhang, S.Q.: Dlora: Distributed parameter-efficient fine-tuning solution for large language model. arXiv preprint arXiv:2404.05182 (2024)
arXiv 2024
Show all 26 references
-
[8]
Goethals, B., Laur, S., Lipmaa, H., Mielik¨ ainen, T.: On private scalar product computation for privacy- preserving data mining. p. 104–120. ICISC’04, Springer-Verlag, Berlin, Heidelberg (2004), https: //doi.org/10.1007/11496618_9
2004 doi
-
[9]
In: Proceedings of the AAAI conference on artificial intelligence
Han, K., Hong, S., Cheon, J.H., Park, D.: Logistic regression on homomorphic encrypted data at scale. In: Proceedings of the AAAI conference on artificial intelligence. vol. 33, pp. 9466–9471 (2019)
2019
-
[10]
Advances in neural information processing systems 35, 15718–15731 (2022)
Hao, M., Li, H., Chen, H., Xing, P., Xu, G., Zhang, T.: Iron: Private inference on transformers. Advances in neural information processing systems 35, 15718–15731 (2022)
2022
-
[11]
In: International Conference on Learning Representations (2022), https://openreview.net/forum?id=nZeVKeeFYf9
Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: LoRA: Low-rank adaptation of large language models. In: International Conference on Learning Representations (2022), https://openreview.net/forum?id=nZeVKeeFYf9
2022
-
[13]
In: Proceedings of the IEEE conference on computer vision and pattern recognition
Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., Kalenichenko, D.: Quan- tization and training of neural networks for efficient integer-arithmetic-only inference. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 27...
2018
-
[14]
In: Rosulek, M
Joye, M.: On-line/off-line dcr-based homomorphic encryption and applications. In: Rosulek, M. (ed.) Topics in Cryptology – CT-RSA 2023. pp. 115–131. Springer International Publishing, Cham (2023)
2023
-
[15]
In: 27th USENIX Security Symposium (USENIX Security 18)
Juvekar, C., Vaikuntanathan, V., Chandrakasan, A.: GAZELLE: A low latency framework for se- cure neural network inference. In: 27th USENIX Security Symposium (USENIX Security 18). pp. 1651–1669. USENIX Association, Baltimore, MD (Aug 2018), https://www.usenix.org/conference/ u...
2018
-
[16]
BMC Medical Genomics 11(4), 83 (Oct 2018), https://doi
Kim, A., Song, Y., Kim, M., Lee, K., Cheon, J.H.: Logistic regression model training based on the approximate homomorphic encryption. BMC Medical Genomics 11(4), 83 (Oct 2018), https://doi. org/10.1186/s12920-018-0401-7
2018 doi
-
[17]
Advances in neural information processing systems 33, 9193–9202 (2020)
Lou, Q., Feng, B., Charles Fox, G., Jiang, L.: Glyph: Fast and accurately training deep neural networks on encrypted data. Advances in neural information processing systems 33, 9193–9202 (2020)
2020
-
[18]
In: Advances in Cryptology – EUROCRYPT 2010
Lyubashevsky, V., Peikert, C., Regev, O.: On ideal lattices and learning with errors over rings. In: Advances in Cryptology – EUROCRYPT 2010. Lecture Notes in Computer Science, vol. 6110, pp. 1–23. Springer (2010)
2010
-
[19]
In: Proceedings of the 10th ACM International Workshop on Security and Privacy Analytics
Montero, L., Frery, J., Kherfallah, C., Bredehoft, R., Stoian, A.: Machine learning training on encrypted data with tfhe. In: Proceedings of the 10th ACM International Workshop on Security and Privacy Analytics. p. 71–76. IWSPA ’24, Association for Computing Machinery, New Yor...
2024
-
[20]
In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops
Nandakumar, K., Ratha, N., Pankanti, S., Halevi, S.: Towards deep neural network training on en- crypted data. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. pp. 0–0 (2019)
2019
-
[21]
In: Stern, J
Paillier, P.: Public-key cryptosystems based on composite degree residuosity classes. In: Stern, J. (ed.) Advances in Cryptology — EUROCRYPT ’99. pp. 223–238. Springer Berlin Heidelberg, Berlin, Heidelberg (1999)
1999
-
[22]
In: Proceedings of the 37th Annual ACM Symposium on Theory of Computing (STOC 2005)
Regev, O.: On lattices, learning with errors, random linear codes, and cryptography. In: Proceedings of the 37th Annual ACM Symposium on Theory of Computing (STOC 2005). pp. 84–93. ACM (2005)
2005
-
[23]
arXiv preprint arXiv:2311.14030 (2023)
Wang, Y., Lin, Y., Zeng, X., Zhang, G.: Privatelora for efficient privacy preserving llm. arXiv preprint arXiv:2311.14030 (2023)
2023 arXiv
-
[24]
In: International Conference on Machine Learning
Xiao, G., Lin, J., Seznec, M., Wu, H., Demouth, J., Han, S.: Smoothquant: Accurate and efficient post- training quantization for large language models. In: International Conference on Machine Learning. pp. 38087–38099. PMLR (2023)
2023
-
[25]
Yang, L., Cai, S., Chai, D., Zhang, J., Tian, H., Jin, Y., Guo, K., Chen, K., Yang, Q.: Packvfl: Efficient he packing for vertical federated learning (2024), https://arxiv.org/abs/2405.00482 16
2024 arXiv
-
[26]
In: 2020 USENIX Annual Technical Conference (USENIX ATC 20)
Zhang, C., Li, S., Xia, J., Wang, W., Yan, F., Liu, Y.: BatchCrypt: Efficient homomorphic encryption for Cross-Silo federated learning. In: 2020 USENIX Annual Technical Conference (USENIX ATC 20). pp. 493–506. USENIX Association (Jul 2020), https://www.usenix.org/conference/at...
2020
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.