Pith. sign in

REVIEW 3 major objections 7 minor 62 references

An Efficient Private GPT Never Autoregressively Decodes

T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Secure GPT decoding can be accelerated 2.1x-6.0x by having a public model draft tokens that a private model verifies in one secure forward, with no change to the output distribution or privacy guarantee.

desk verdict The latency-insensitivity observation is real and the speedup idea is sound, but the paper must fix its privacy-equivalence argument before the 'same privacy' claim holds. read the letter →

arxiv 2505.15252 v1 pith:2TV74W6P submitted 2025-05-21 cs.CR cs.LG

Zhengyi Li , Yue Guan , Kang Yang , Yu Feng , Ning Liu , Yu Yu , Jingwen Leng , Minyi Guo This is my paper
classification cs.CRcs.LG
keywords securetwo-partycomputationspeculativedecodingprivateGPTinferencehomomorphicencryptionoblivioustransferknowledgedistillationlatencyinsensitivitysamplingprotocol
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Secure two-party GPT inference protects the client's prompt and the server's weights, but each generated token normally costs a full secure forward pass. This paper claims that secure forwarding is nearly insensitive to how many tokens are sent: forwarding 16 tokens costs only about 1.07x-1.77x the time of forwarding one. On that basis, it proposes POST, in which the client samples draft tokens from a small public model and the server's private model accepts or rejects them in a single secure step, using secure speculative sampling so the final output distribution is identical to standard decoding. Knowledge distillation on public data raises the acceptance ratio of drafts to 52%-84%, and the paper reports a 2.1x-6.0x end-to-end speedup over standard secure decoding across three model pairs and two network conditions, at the same semi-honest privacy level.

What carries the argument

The load-bearing mechanism is the empirical latency insensitivity of secure decoding to input length, supported by a layer-by-layer decomposition showing that one-way delays and SIMD-packed homomorphic computation dominate and barely change when the input grows from 1 to 16 tokens. On top of this, the secure speculative sampling protocol in Algorithm 1 refactors rejection sampling from a division into multiplication, reformulating the test as $r \cdot q(x) \ge p(x)$, and applies selection-then-comparison: the server masks each row of the score matrix with a random ring element, the client uses a $\binom{V}{1}$-OT to pick the entry at its draft token, and the two parties compare that single value against zero. This reduces the sampling cost from $O(V \cdot 2^\ell)$ to $O(V \cdot \ell)$ and makes the verification overhead negligible.

What would settle it

Profile a secure transformer decoder on a framework whose nonlinear protocols use a number of communication rounds or transmitted bytes that scale with input length; if the latency ratio for 16 tokens versus 1 token exceeds roughly 2 across typical models, the predicted 2.1x-6.0x speedup cannot materialize.

Watch

Extended reading notes

Core claim

The paper establishes that the latency of securely decoding a transformer is largely insensitive to the number of tokens forwarded in a step, provided that step uses the SIMD packing and round-dominated protocols typical of current homomorphic-encryption and MPC frameworks. Building on this observation, POST lets the client autoregressively generate draft tokens from a public model with a small plaintext cost, securely computes the private model's output distributions for all drafts in one forward, and then runs a secure speculative sampling protocol that accepts a prefix of drafts and resamples the first rejected token as a bonus. The secure rejection test is rewritten from the division-heavy rule $\max(0, 1 - p(x)/q(x))$ into the equivalent multiplication $r \cdot q(x) \ge p(x)$, so the parties compare products against zero instead of dividing; a selection-then-comparison trick with oblivious transfer lets the client privately retrieve only the score entries corresponding to its draft tokens, cutting the sampling overhead by about ten times. Because the drafts come from an aligned public model and the acceptance rule preserves the private model's distribution, POST generates exactly the tokens standard secure decoding would, while reducing the number of secure forwards.

Load-bearing premise

The whole speedup rests on the measurement claim that securely forwarding several tokens takes almost as long as securely forwarding one; if a cryptographic stack's latency grows roughly linearly with the number of forwarded tokens, POST would lose its advantage.

Editorial extensions

If this is right

  • Secure decoding stops being a per-token cost and becomes a per-verification-round cost: the speedup grows with the draft acceptance ratio, so better-aligned or larger public models directly translate into more generated tokens per secure forward.
  • Because POST preserves the exact output distribution of the private model, it inherits standard secure decoding's generation quality rather than trading accuracy for speed.
  • The client learns only a subset of the information revealed in standard secure decoding: the acceptance booleans and the output distribution at the first rejected token, compared with the Top-K distributions revealed in standard decoding.
  • POST is complementary to existing secure-inference protocol optimizations; it wraps the linear-layer homomorphic protocols and nonlinear-layer MPC protocols as components, so its gains multiply with their improvements.
  • When the server itself releases a small aligned version of its private model, acceptance ratios reach 75%-85%, making the approach particularly attractive for deployments where the model owner can publish a draft model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The latency-insensitivity observation is a measurement of current SIMD packing and round-dominated protocols, not a theorem; a future cryptographic stack that parallelizes nonlinear layers differently or uses input-dependent communication rounds could erode the speedup, so POST should be re-validated on each new framework.
  • The paper's speedup numbers combine profiled decoder latencies with acceptance-ratio measurements rather than a single full-stack secure decoding run; an independent full implementation would be the direct test of whether the predicted 2.1x-6.0x holds after composing prefill, KV-cache updates, and the sampling rounds.
  • The selection-then-comparison pattern in the secure sampling protocol is a general recipe for privately selecting one entry of a secret-shared array by index, which could accelerate other private lookup and rejection tasks beyond speculative decoding.
  • Because alignment needs only the top-K outputs of the private model on public data, the private model's output distribution itself becomes a training signal for the draft model; this could be extended to per-task or even online adaptation of the public model, which the paper leaves unexplored.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes POST (Public decOding and Secure verificaTion), a method to accelerate two-party secure GPT decoding. The motivating observation, profiled in Section 3.2 and Appendix E.1, is that securely forwarding several tokens costs only slightly more than forwarding one token (reported ratios 1.07x to 1.77x for input lengths 16 vs 1 across frameworks, models, and network conditions). POST lets the client use a public model in plaintext to autoregressively sample gamma draft tokens, which are then sent to the private model in a single batched secure forward. Acceptance of drafts is decided by a new OT-based secure speculative-sampling protocol (Algorithm 1) that the paper claims is much cheaper than the naive secure implementation; at the first rejected position the client reconstructs the private distribution and samples a bonus token locally (Algorithm 2). An offline knowledge-distillation step aligns the public model to the private model's output distribution, raising the acceptance ratio. The paper reports post-alignment acceptance ratios of 52% to 84% across three model pairs and four tasks, and claims 2.1x to 6.0x end-to-end speedups over standard secure decoding while maintaining the same level of privacy and generation quality.

Significance. If the central claims hold, POST is a valuable, orthogonal addition to secure GPT inference: it requires no modification of the private model, it composes with protocol-level optimizations such as BumbleBee and Nimbus, and it converts the availability of public LLMs into a concrete speedup that increases with model alignment. The paper's strongest asset is the latency-insensitivity observation, which is carefully supported by per-layer profiling across two frameworks (SecretFlow-SPU and EzPC/BOLT), four models, and four network conditions; this is a reproducible and practically important measurement. The secure sampling protocol is a genuine engineering contribution, with an explicit communication-complexity improvement over the naive approach and a plausible correctness reduction in Appendix C. The alignment experiments show substantial, consistent gains in acceptance ratios, and the paper is honest in reporting component-level measurements such as sampling overhead.

major comments (3)
  1. [Appendix D; Algorithm 2 (Appendix B), lines 17-18] The claim that POST maintains 'the same level of privacy as standard secure decoding' is not established, and as written the protocol leaks more than a minimal baseline. In Algorithm 2, lines 17-18, the client reconstructs the full private distribution pk(x) = ⟨pk(x)⟩s + ⟨pk(x)⟩c at the first rejection index k, and additionally learns the rejection booleans n[i] for all gamma drafts, each of which reveals a threshold condition on the private probability p(xi | ·) because the client knows its own Rmul and the public distribution q. Appendix D justifies the comparison by asserting, without citation, that in standard secure decoding 'the client can autoregressively obtain gamma output distributions ... from which the client samples tokens.' This is contradicted by the final paragraph of the same appendix, which states that standard decoding reveals only the TopK elements (K=5) of each distribution. A full vocabulary-sized vector at one position is not a subset of TopK information at that same position, and a thresholded probability is not equivalent to the Top1 element of the distribution, so the 'subset of information' argument in Appendix D is false under the paper's own description. Moreover, the paper's own Algorithm 1 shows that sampling can be done while revealing only a boolean, so a minimal-leakage baseline exists and POST is not compared against it. The authors should either (a) define the baseline precisely with a concrete citation showing that the cited baselines reveal the full output distribution to the client for local sampling, or (b) replace the local bonus-token sampling in Algorithm 2 with a secure sampling sub-protocol that reveals only the sampled token; in either case the security analysis must account for the information contained in the acceptance booleans.
  2. [Section 5.3, Figure 5; Abstract] The headline '2.1x to 6.0x speedup' is not measured end-to-end. Figure 5 plots analytic curves of speedup versus acceptance ratio, and the markers combine the acceptance ratios of Table 1 with separately profiled decoder latencies (Section 3.2) and sampling overhead (Table 2); no complete POST decoding run is reported. The abstract's 'Experiments demonstrate a 2.1x to 6.0x speedup' therefore overstates the evidence: the numbers are model-based projections that assume the latency-insensitivity persists when draft tokens are integrated into a real decoding loop, including KV-cache updates and the communication of draft tokens into the secure computation. In addition, the curves in Figure 5 extend to acceptance ratio 0.90, whereas the highest measured acceptance ratio in Table 1 is 0.840 (FN-AL for FLAN-T5-XL and FLAN-T5-base); part of the claimed speedup range thus rests on unmeasured acceptance ratios. The authors should either report at least one complete end-to-end secure decoding experiment or explicitly state that the speedups are projections, and should restrict the claimed range to the measured acceptance-ratio values.
  3. [Section 4.3, Section 5.2, Table 1] The reported acceptance ratios, and hence the speedups derived from them, are obtained after aligning the public model on the same tasks on which acceptance is then measured (for example, Figure 4 aligns on Spider and Table 1 reports the 'SP-AL' acceptance ratios). The paper does not state whether the alignment and evaluation data are disjoint, and it does not measure how well the alignment transfers to out-of-domain prompts. Because the end-to-end speedup is directly determined by the acceptance ratio, a deployment on user inputs drawn from a different distribution than the four evaluation tasks would likely see materially lower acceptance ratios than Table 1. The authors should clarify the alignment/evaluation split and report acceptance ratios from a task-agnostic alignment corpus, as suggested in Section 4.3, or from held-out tasks, so that the projected speedups are not tied to per-task distillation.
minor comments (7)
  1. [Appendix C, Eq. (6)] The equivalence chain equating r < max(0, 1 - p/q) with r*q >= p is not a per-sample equivalence; the middle step relies on the fact that 1-r and r have the same distribution. Because the protocol uses a specific client-chosen Rmul, the distributional nature of the argument should be stated explicitly.
  2. [Algorithm 2, lines 19-24] The adjusted distribution p'(x) = max(0, pk(x) - qk(x)) is unnormalized, but the text says 'Pc samples xk ∼ p′(x)', which defines a valid distribution only after normalization. The normalization constant (e.g., 1 - sum_x min(pk(x), qk(x))) should be specified, or the text should state that sampling is from the normalized distribution.
  3. [Section 5.2] The claim that collecting roughly 1 million alignment tokens costs 'approximately $10' appears to use public OpenAI API pricing; if the client instead collects the private model's output distributions through secure inference, the cost would be orders of magnitude larger. Please clarify that this figure applies to the server-side plaintext alignment route described in Section 4.3, or remove it.
  4. [Algorithm 1, line 5] The notation 'r ∼ Z^γ_{2ℓ}' is ambiguous: one masking value per draft row is needed, so it should read 'r[i] sampled from Z_{2ℓ} for i in [γ]'. The current notation suggests a random vector whose dimension is then indexed inconsistently.
  5. [Section 4.1, 'Consistent Speedup' bullet] With low acceptance ratios (e.g., alpha = 0.24 for the unaligned Vicuna-7B and LLaMA-68M pair in Table 1), the expected number of tokens per POST step can be smaller than the ratio T_std/T_POST, so POST is slower than standard decoding in those settings; the bullet should say 'expected speedup in typical cases' rather than 'consistent speedup', and the worst case (all drafts rejected: one token at the cost of forwarding gamma+1 tokens) is a slowdown, not a speedup.
  6. [Figure 5 and Section 5.3] Please state explicitly that the speedup curves are computed from the latency model of Section 3.2 combined with the acceptance ratios of Table 1, and note the maximum measured acceptance ratio (0.840) so readers do not interpret the x-axis range up to 0.90 as empirical.
  7. [Throughout] There are several typos and notation slips: Section 1 should read 'compromising privacy' rather than 'comprising privacy'; Appendix E.1 figure captions contain 'input input lengths'; Algorithm 2 line 3 has 'q(x | xx<t+i)'; and the abstract's claim that 'securely forwarding single and multiple tokens takes similar latency' should be qualified by the framework, since Appendix E.1 notes that SecretFlow-SPU can add communication rounds as input length grows, while EzPC keeps rounds constant.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the speedup and distribution-preservation derivations are self-contained, while the Appendix D privacy comparison rests on an unproven baseline-leakage assumption that is a correctness risk, not a circular reduction.

full rationale

The paper's derivation chain is not circular. The claimed 2.1x-6.0x speedup is an arithmetic consequence of measured acceptance ratios (Table 1) and measured secure-forward latency ratios (Figures 1 and 6-9), not a fitted parameter renamed as a prediction. The distribution-preservation guarantee is proved self-containedly in Appendix C via the explicit equivalence chain in Eq. (6), reducing the secure protocol to the standard speculative-sampling rule of Leviathan et al. (2023). The model-alignment step is an empirical, plaintext fit; reporting acceptance ratios after distillation on the same or similar tasks is an in-sample evaluation concern, not a definitional reduction. The only load-bearing step that deserves scrutiny is the security comparison in Appendix D: the claim that standard secure decoding reveals full private output distributions is asserted without citation or proof, and if the comparison baseline samples inside the secure computation, POST's reconstruction of p_k(x) at Algorithm 2 line 18 would be strictly more leakage. This is a missing-support/correctness risk, but it is not circularity: the conclusion is derived from an unproven premise rather than assumed as a premise. Self-citations to Nimbus (Li et al., 2024b) are used for baseline and protocol selection and do not carry the central derivation. Overall circularity score: 2.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The method introduces no new mathematical entities. It relies on two empirical premises: latency insensitivity (measured) and alignment transfer (assumed). The draft length is a tunable parameter; the fixed-point precision is unstated.

free parameters (2)
  • draft length gamma = 4, 8, 16 (chosen per deployment)
    Number of draft tokens generated by the public model per secure step. The paper sweeps 4/8/16 and reports the best speedup, so the headline speedup depends on this hand-chosen hyperparameter.
  • Fixed-point precision and probability quantization = not specified
    The secure sampling protocol operates over Z_{2^ell} and samples Rmul in [0,1], but the paper never states the fixed-point representation, precision, or truncation strategy. The distributional guarantee depends on these choices.
assumptions (4)
  • domain assumption Latency of secure GPT decoding is insensitive to input length in the range 1 to 16 tokens
    Load-bearing empirical observation of Section 3.2 and Appendix E.1, supported by profiling on EzPC/BOLT and SecretFlow-SPU across models and network conditions, but it is a measurement, not a theorem, and the speedup result depends on it.
  • standard math Semi-honest two-party computation with RLWE-based HE and OT as background primitives
    Standard cryptographic assumptions (RLWE, OT) are inherited from cited primitives in Section 2.1; the security proof in Appendix D uses the composability of these protocols.
  • domain assumption The client can obtain topK output distributions from the private model via API and align the public model on a public corpus
    Section 4.3 assumes API access returns topK logits and that public/corpus data suffice for distillation that transfers to private tasks; the acceptance-ratio gains rely on this transfer.
  • domain assumption The private model's output distribution is faithfully represented by the secure forward
    The correctness argument equates POST's sampling with sampling from the private model's distribution; as in all secure inference works, if the secure approximation of softmax/GELU deviates, the guarantee is approximate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Efficient Private GPT Never Autoregressively Decodes." pith.science (2026). https://pith.science/paper/2TV74W6P

@misc{pith2026250515252,
  author       = {Pith},
  title        = {Pith review of: An Efficient Private GPT Never Autoregressively Decodes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2TV74W6P}},
  note         = {Machine review of arXiv:2505.15252}
}
abstract

The wide deployment of the generative pre-trained transformer (GPT) has raised privacy concerns for both clients and servers. While cryptographic primitives can be employed for secure GPT inference to protect the privacy of both parties, they introduce considerable performance overhead.To accelerate secure inference, this study proposes a public decoding and secure verification approach that utilizes public GPT models, motivated by the observation that securely decoding one and multiple tokens takes a similar latency. The client uses the public model to generate a set of tokens, which are then securely verified by the private model for acceptance. The efficiency of our approach depends on the acceptance ratio of tokens proposed by the public model, which we improve from two aspects: (1) a private sampling protocol optimized for cryptographic primitives and (2) model alignment using knowledge distillation. Our approach improves the efficiency of secure decoding while maintaining the same level of privacy and generation quality as standard secure decoding. Experiments demonstrate a $2.1\times \sim 6.0\times$ speedup compared to standard decoding across three pairs of public-private models and different network conditions.

Figures

Figures reproduced from arXiv: 2505.15252 by the authors.

Figure 1
Figure 1. Latencies against different input lengths. The bandwidth and one-way delay are 1000 Mbps and 10 ms. Linearh1 1000/10 Linearh1 400/40 Linearh2 1000/10 Linearh2 400/40 Softmax 1000/10 Softmax 400/40 QK T 1000/10 QK T 400/40 LN 1000/10 LN 400/40 0 5 10 Time (s) One-way Delay Transmisson Computation [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The latency breakdown of some layers. The second row of the x-axis ticks represents bandwidth and one-way latency. The bars corresponding to the same x-axis ticks illustrate input lengths 1, 2, 4, 8, and 16. should allocate costly cryptographic computation only to the exclusive knowledge while employing cheap plaintext computations for the shared knowledge. However, current secure inference methods necessitate all c… view at source ↗
Figure 3
Figure 3. The overview of the public decoding and secure ver￾ification approach. ⟨·⟩ indicates the data are encrypted during computing, such as using the secret sharing, and is only visible to the data owner. 4.1. Overview We propose an approach to leverage knowledge from the public model to enhance efficiency: Public decOding and Secure verificaTion (POST), as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: The alignment efficiency of three pairs of models on the Spider task. tasks across various languages. The T5-efficient series is only pre-trained on the C4 dataset without fine-tuning. The third pair examines the acceptance ratio when the server releases a small versio…
Figure 5
Figure 5. Figure 5: The end-to-end speedup across two network settings and three pairs of models. The curves illustrate the relationship between speedup and acceptance ratio for various draft lengths. Specific speedups for four selected tasks are marked on these curves. and Nimbus (Li et …
Figure 6
Figure 6. Figure 6: The latency of each layer against different input input lengths. We show the results of GPT-2 on EzPC frameworks. Linearqkv Linearo Linearh1 Linearh2 Softmax GELU QK T PV LN Total 0.0 0.2 0.4 0.6 Time (s) 0 1 2 3 Length=1 Length=2 Length=4 Length=8 Length=16 (a) Bandwi…
Figure 7
Figure 7. Figure 7: The latency of each layer against different input input lengths. We show the results of GPT-2 on SecretFlow-SPU frameworks. E. More Experiments E.1. Latency Insensitivity to the Input Length For the decoding of GPT, the input length of the operators is only 1. Section …
Figure 8
Figure 8. Figure 8: The latency of each layer against different input input lengths. We show the results of Vicuna-7B on SecretFlow-SPU frameworks. Linearqkv Linearo Linearh1 Linearh2 Softmax GELU QK T PV LN Total 0.00 0.25 0.50 0.75 1.00 Time (s) 0 2 4 6 Length=1 Length=2 Length=4 Length…
Figure 9
Figure 9. Figure 9: The latency of each layer against different input input lengths. We show the results of FLAN-T5-XL on SecretFlow-SPU frameworks. Theoretic Transform (NTT) encoding. Both polynomial approximation and lookup tables are utilized to evaluate the nonlinear layers. The under…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 40 canonical work pages

  1. [1]

    R., Geist, M., and Bachem, O

    Agarwal, R., Vieillard, N., Zhou, Y., Stanczyk, P., Garea, S. R., Geist, M., and Bachem, O. On-policy distillation of language models: Learning from self-generated mistakes. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=3zKtaqxLhW

    work page 2024

  2. [2]

    What is qwen llm

    Aliyun. What is qwen llm. [Online], 2025. https://help.aliyun.com/zh/model-studio/developer-reference/what-is-qwen-llm/

    work page 2025

  3. [3]

    Comparison of models: Quality, performance & price analysis

    Analysis, A. Comparison of models: Quality, performance & price analysis. https://artificialanalysis.ai/models, 2025. Accessed: 2025-01-29

    work page 2025

  4. [4]

    (leveled) fully homomorphic encryption without bootstrapping

    Brakerski, Z., Gentry, C., and Vaikuntanathan, V. (leveled) fully homomorphic encryption without bootstrapping. ACM Transactions on Computation Theory (TOCT), 6 0 (3): 0 1--36, 2014

    work page 2014

  5. [5]

    D., Chen, D., and Dao, T

    Cai, T., Li, Y., Geng, Z., Peng, H., Lee, J. D., Chen, D., and Dao, T. Medusa: Simple llm inference acceleration framework with multiple decoding heads. In Forty-first International Conference on Machine Learning, 2023

    work page 2023

  6. [6]

    Ezpc: Programmable, efficient, and scalable secure two-party computation

    Chandran, N., Gupta, D., Rastogi, A., Sharma, R., and Tripathi, S. Ezpc: Programmable, efficient, and scalable secure two-party computation. IACR Cryptol. ePrint Arch., 2017: 0 1109, 2017

    work page 2017

  7. [7]

    The-x: Privacy-preserving transformer inference with homomorphic encryption

    Chen, T., Bao, H., Huang, S., Dong, L., Jiao, B., Jiang, D., Zhou, H., Li, J., and Wei, F. The-x: Privacy-preserving transformer inference with homomorphic encryption. arXiv preprint arXiv:2206.00216, 2022

    arXiv 2022

  8. [8]

    and Gao, Q

    Chen, Z. and Gao, Q. Probing linguistic information for logical inference in pre-trained language models. In Proceedings of the AAAI Conference on Artificial Intelligence, pp.\ 10509--10517, 2022

    work page 2022

Show all 62 references
  1. [9]

    E., et al

    Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., et al. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2 0 (3): 0 6, 2023

    2023

  2. [10]

    W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al

    Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25 0 (70): 0 1--53, 2024

    2024

  3. [11]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  4. [12]

    Dettmers, T., Lewis, M., Belkada, Y., and Zettlemoyer, L. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35: 0 30318--30332, 2022

    2022

  5. [13]

    Puma: Secure inference of llama-7b in five minutes

    Dong, Y., Lu, W.-j., Zheng, Y., Wu, H., Zhao, D., Tan, J., Huang, Z., Hong, C., Wei, T., and Cheng, W. Puma: Secure inference of llama-7b in five minutes. arXiv preprint arXiv:2307.12533, 2023

    2023

  6. [14]

    Learning transformer programs

    Friedman, D., Wettig, A., and Chen, D. Learning transformer programs. Advances in Neural Information Processing Systems, 36, 2024

    2024

  7. [15]

    finance-alpaca (revision 51d16b6), 2024

    Gaurang Bharti . finance-alpaca (revision 51d16b6), 2024. URL https://huggingface.co/datasets/gbharti/finance-alpaca

    2024

  8. [16]

    Fully homomorphic encryption using ideal lattices

    Gentry, C. Fully homomorphic encryption using ideal lattices. In Proceedings of the 41th Annual ACM Symposium on Theory of Computing - STOC'09, pp.\ 169–178. ACM, 2009

    2009

  9. [17]

    Introducing palm 2

    Ghahramani, Z. Introducing palm 2. [Online], 2023. https://blog.google/technology/ai/google-palm-2-ai-large-language-model//

    2023

  10. [18]

    Github copilot, 2025

    GitHub, I. Github copilot, 2025. URL https://github.com/features/copilot

    2025

  11. [19]

    Sigma: secure gpt inference with function secret sharing

    Gupta, K., Jawalkar, N., Mukherjee, A., Chandran, N., Gupta, D., Panwar, A., and Sharma, R. Sigma: secure gpt inference with function secret sharing. Cryptology ePrint Archive, 2023

    2023

  12. [20]

    Iron: Private inference on transformers

    Hao, M., Li, H., Chen, H., Xing, P., Xu, G., and Zhang, T. Iron: Private inference on transformers. Advances in Neural Information Processing Systems, 35: 0 15718--15731, 2022

    2022

  13. [21]

    Rhombus: Fast homomorphic matrix-vector multiplication for secure two-party inference

    He, J., Yang, K., Tang, G., Huang, Z., Lin, L., Wei, C., Yan, Y., and Wang, W. Rhombus: Fast homomorphic matrix-vector multiplication for secure two-party inference. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pp.\ 2490--2504, 2024

    2024

  14. [22]

    Ciphergpt: Secure two-party gpt inference

    Hou, X., Liu, J., Li, J., Li, Y., Lu, W.-j., Hong, C., and Ren, K. Ciphergpt: Secure two-party gpt inference. Cryptology ePrint Archive, 2023

    2023

  15. [23]

    Codesearchnet challenge: Evaluating the state of semantic code search

    Husain, H., Wu, H.-H., Gazit, T., Allamanis, M., and Brockschmidt, M. Codesearchnet challenge: Evaluating the state of semantic code search. arXiv preprint arXiv:1909.09436, 2019

    1909 arXiv

  16. [24]

    Extending oblivious transfers efficiently

    Ishai, Y., Kilian, J., Nissim, K., and Petrank, E. Extending oblivious transfers efficiently. In Annual International Cryptology Conference, pp.\ 145--161. Springer, 2003

    2003

  17. [25]

    \ GAZELLE \ : A low latency framework for secure neural network inference

    Juvekar, C., Vaikuntanathan, V., and Chandrakasan, A. \ GAZELLE \ : A low latency framework for secure neural network inference. In 27th USENIX Security Symposium (USENIX Security 18), pp.\ 1651--1669, 2018

    2018

  18. [26]

    Dense passage retrieval for open-domain question answering

    Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp.\ 6769--6781, Online, Nov...

    2020 doi

  19. [27]

    Kim, J.-Y., Park, S., Lee, J., and Cheon, J. H. Privacy-preserving embedding via look-up table evaluation with fully homomorphic encryption. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st I...

    2024

  20. [28]

    and Kumaresan, R

    Kolesnikov, V. and Kumaresan, R. Improved ot extension for transferring short secrets. In Advances in Cryptology--CRYPTO 2013: 33rd Annual Cryptology Conference, Santa Barbara, CA, USA, August 18-22, 2013. Proceedings, Part II, pp.\ 54--70. Springer, 2013

    2013

  21. [29]

    Chatgpt: A meta-analysis after 2.5 months

    Leiter, C., Zhang, R., Chen, Y., Belouadi, J., Larionov, D., Fresen, V., and Eger, S. Chatgpt: A meta-analysis after 2.5 months. arXiv preprint arXiv:2302.13795, 2023

    2023 arXiv

  22. [30]

    Fast inference from transformers via speculative decoding

    Leviathan, Y., Kalman, M., and Matias, Y. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pp.\ 19274--19286. PMLR, 2023

    2023

  23. [31]

    P., and Zhang, H

    Li, D., Shao, R., Wang, H., Guo, H., Xing, E. P., and Zhang, H. Mpcformer: fast, performant and private transformer inference with mpc. arXiv preprint arXiv:2211.01452, 2022

    2022 arXiv

  24. [32]

    Seesaw: Compensating for nonlinear reduction with linear computations for private inference

    Li, F., Zhai, Y., Cai, S., and Gao, M. Seesaw: Compensating for nonlinear reduction with linear computations for private inference. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.), Proceedings of the 41st Internatio...

    2024

  25. [33]

    Nimbus: Secure and efficient two-party inference for transformers

    Li, Z., Yang, K., Tan, J., Lu, W.-j., Wu, H., Wang, X., Yu, Y., Zhao, D., Zheng, Y., Guo, M., et al. Nimbus: Secure and efficient two-party inference for transformers. arXiv preprint arXiv:2411.15707, 2024 b

    2024 arXiv

  26. [34]

    Merge: Fast private text generation

    Liang, Z., Wang, P., Zhang, R., Xu, N., Zhang, S., Xing, L., Bai, H., and Zhou, Z. Merge: Fast private text generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 19884--19892, 2024

    2024

  27. [35]

    Online speculative decoding

    Liu, X., Hu, L., Bailis, P., Cheung, A., Deng, Z., Stoica, I., and Zhang, H. Online speculative decoding. In Forty-first International Conference on Machine Learning, 2023

    2023

  28. [36]

    BumbleBee: Secure Two-party Inference Framework for Large Transformers

    Lu, W., Huang, Z., Gu, Z., Li, J., Liu, J., Hong, C., Ren, K., Wei, T., and Chen, W. BumbleBee: Secure Two-party Inference Framework for Large Transformers . In 32nd Annual Network and Distributed System Security Symposium, NDSS 2025 . The Internet Society, 2025

    2025

  29. [37]

    On ideal lattices and learning with errors over rings

    Lyubashevsky, V., Peikert, C., and Regev, O. On ideal lattices and learning with errors over rings. In Advances in Cryptology--EUROCRYPT 2010: 29th Annual International Conference on the Theory and Applications of Cryptographic Techniques, French Riviera, May 30--June 3, 2010....

    2010

  30. [38]

    \ SecretFlow-SPU \ : A performant and \ User-Friendly \ framework for \ Privacy-Preserving \ machine learning

    Ma, J., Zheng, Y., Feng, J., Zhao, D., Wu, H., Fang, W., Tan, J., Yu, C., Zhang, B., and Wang, L. \ SecretFlow-SPU \ : A performant and \ User-Friendly \ framework for \ Privacy-Preserving \ machine learning. In 2023 USENIX Annual Technical Conference (USENIX ATC 23), pp.\ 17-...

    2023

  31. [39]

    Miao, X., Oliaro, G., Zhang, Z., Cheng, X., Wang, Z., Zhang, Z., Wong, R. Y. Y., Zhu, A., Yang, L., Shi, X., et al. Specinfer: Accelerating large language model serving with tree-based speculative inference and verification. In Proceedings of the 29th ACM International Confere...

    2024

  32. [40]

    Miao, X., Oliaro, G., Zhang, Z., Cheng, X., Wang, Z., Zhang, Z., Wong, R. Y. Y., Zhu, A., Yang, L., Shi, X., et al. Specinfer: Accelerating large language model serving with tree-based speculative inference and verification. In Proceedings of the 29th ACM International Confere...

    2024

  33. [41]

    Api pricing, 2025

    OpenAI. Api pricing, 2025. URL https://openai.com/api/pricing/

    2025

  34. [42]

    Bolt: Privacy-preserving, accurate and efficient inference for transformers

    Pang, Q., Zhu, J., M \"o llering, H., Zheng, W., and Schneider, T. Bolt: Privacy-preserving, accurate and efficient inference for transformers. In 2024 IEEE Symposium on Security and Privacy (SP), pp.\ 130--130. IEEE Computer Society, 2024

    2024

  35. [43]

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 0 (140): 0 1--67, 2020

    2020

  36. [44]

    Spencnn: orchestrating encoding and sparsity for fast homomorphically encrypted neural network inference

    Ran, R., Luo, X., Wang, W., Liu, T., Quan, G., Xu, X., Ding, C., and Wen, W. Spencnn: orchestrating encoding and sparsity for fast homomorphically encrypted neural network inference. In International Conference on Machine Learning, pp.\ 28718--28728. PMLR, 2023

    2023

  37. [45]

    Rathee, D., Schneider, T., and Shukla, K. K. Improved multiplication triple generation over rings via RLWE -based AHE . In Cryptology and Network Security, volume 11829 of LNCS, pp.\ 347--359. Springer International Publishing, 2019

    2019

  38. [46]

    Cryptflow2: Practical 2-party secure inference

    Rathee, D., Rathee, M., Kumar, N., Chandran, N., Gupta, D., Rastogi, A., and Sharma, R. Cryptflow2: Practical 2-party secure inference. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp.\ 325--342, 2020

    2020

  39. [47]

    Rathee, D., Rathee, M., Goli, R. K. K., Gupta, D., Sharma, R., Chandran, N., and Rastogi, A. Sirnn: A math library for secure rnn inference. In 2021 IEEE Symposium on Security and Privacy (SP), pp.\ 1003--1020. IEEE, 2021

    2021

  40. [48]

    A., Manoel, A., Mireshghallah, F., Lin, Z., Gopi, S., Kulkarni, J., and Sim, R

    Tang, X., Shin, R., Inan, H. A., Manoel, A., Mireshghallah, F., Lin, Z., Gopi, S., Kulkarni, J., and Sim, R. Privacy-preserving in-context learning with differentially private few-shot generation. In The Twelfth International Conference on Learning Representations, 2024

    2024

  41. [49]

    W., Narang, S., Yogatama, D., Vaswani, A., and Metzler, D

    Tay, Y., Dehghani, M., Rao, J., Fedus, W., Abnar, S., Chung, H. W., Narang, S., Yogatama, D., Vaswani, A., and Metzler, D. Scale efficiently: Insights from pre-training and fine-tuning transformers. arXiv preprint arXiv:2109.10686, 2021

    2021 arXiv

  42. [50]

    Llama: Open and efficient foundation language models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi \`e re, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  43. [51]

    N., Kaiser, ., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, ., and Polosukhin, I. Attention is all you need. Advances in neural information processing systems, 30, 2017

    2017

  44. [52]

    Transformers are uninterpretable with myopic methods: a case study with bounded dyck grammars

    Wen, K., Li, Y., Liu, B., and Risteski, A. Transformers are uninterpretable with myopic methods: a case study with bounded dyck grammars. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS '23, Red Hook, NY, USA, 2024. Curran Ass...

    2024

  45. [53]

    CCN et: Extracting high quality monolingual datasets from web crawl data

    Wenzek, G., Lachaux, M.-A., Conneau, A., Chaudhary, V., Guzm \'a n, F., Joulin, A., and Grave, E. CCN et: Extracting high quality monolingual datasets from web crawl data. In Calzolari, N., B \'e chet, F., Blache, P., Choukri, K., Cieri, C., Declerck, T., Goggi, S., Isahara, H...

    2020

  46. [54]

    Huggingface's transformers: State-of-the-art natural language processing

    Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019

    1910 arXiv

  47. [55]

    Privcirnet: Efficient private inference via block circulant transformation

    Xu, T., Wu, L., Wang, R., and Li, M. Privcirnet: Efficient private inference via block circulant transformation. arXiv preprint arXiv:2405.14569, 2024

    2024 arXiv

  48. [56]

    Ferret: Fast extension for correlated ot with small communication

    Yang, K., Weng, C., Lan, X., Zhang, J., and Wang, X. Ferret: Fast extension for correlated ot with small communication. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp.\ 1607--1626, 2020

    2020

  49. [57]

    Yao, A. C.-C. How to generate and exchange secrets. In 27th Annual Symposium on Foundations of Computer Science (sfcs 1986), pp.\ 162--167, 1986. doi:10.1109/SFCS.1986.25

    1986 doi

  50. [58]

    Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task

    Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., et al. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887, 2018

    2018 arXiv

  51. [59]

    Mpcvit: Searching for accurate and efficient mpc-friendly vision transformer with heterogeneous attention

    Zeng, W., Li, M., Xiong, W., Tong, T., Lu, W.-j., Tan, J., Wang, R., and Huang, R. Mpcvit: Searching for accurate and efficient mpc-friendly vision transformer with heterogeneous attention. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 5052--...

    2023

  52. [60]

    X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al

    Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023

    2023 arXiv

  53. [61]

    Converting transformers to polynomial form for secure inference over homomorphic encryption

    Zimerman, I., Baruch, M., Drucker, N., Ezov, G., Soceanu, O., and Wolf, L. Converting transformers to polynomial form for secure inference over homomorphic encryption. In Salakhutdinov, R., Kolter, Z., Heller, K., Weller, A., Oliver, N., Scarlett, J., and Berkenkamp, F. (eds.)...

    2024

  54. [62]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.