Pith. sign in

REVIEW 4 major objections 5 minor 100 references

Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A private-inference system called Comet claims that activation sparsity can be safely predicted inside MPC, giving a 1.87x-2.63x speedup and 1.94x-2.64x communication reduction.

desk verdict Genuinely new system for sparsity-aware MPC LLM inference, with real protocol ideas; but the headline numbers and the privacy framing need correction before I would trust the central 'private inference' claim. read the letter →

arxiv 2505.07239 v1 pith:XKEUHV2Y submitted 2025-05-12 cs.CR cs.AI

Guang Yan , Yuhui Zhang , Zimu Guo , Lutan Zhao , Xiaojun Chen , Chen Wang , Wenhao Wang , Dan Meng
show 1 more author
Rui Hou
This is my paper · ORCID
classification cs.CRcs.AI
keywords privateinferencesecuremulti-partycomputationactivationsparsitylargelanguagemodelssparsematrixmultiplicationobliviousshuffleKVcacheLLMacceleration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Comet claims that the biggest remaining cost in MPC-based LLM inference, server-to-server communication, can be attacked by predicting where the model's own computations produce zeros. Because more than 90% of ReLU outputs and about half of attention heads in large models are inactive, the system skips those activations and the surrounding matrix multiplications, provided the predictor is accurate enough. The paper reports 1.87x-2.63x end-to-end speedups and 1.94x-2.64x communication reductions over six prior systems on OPT-1.3B through Llama2-7B, losing on average about 1.5% accuracy. If these numbers hold, private serving of large models becomes roughly two to three times cheaper without changing the model architecture, and sparsity-based optimizations used in plaintext LLM serving can be carried over to the encrypted setting.

What carries the argument

The load-bearing machinery is the predicted sparsity distribution made safe to reveal. A lightweight two-layer low-rank neural network with a threshold layer, executed inside MPC, produces a secret-shared 0/1 mask of which activations are nonzero. Before the mask is reconstructed, an oblivious-shuffle protocol permutes it by a secret random permutation split between the two servers, so the sparsity count becomes public while the positions of zeros stay private; the same permutation is applied to the input matrix so plaintext indexing of active entries is correct. On top of this, SOMM partitions the nonzero output positions of the preceding linear layer into connected components of a bipartite graph and runs one secure matrix multiplication per component, proving communication-minimality, while SIMM reorganizes the subsequent layer from row-by-column to column-by-row sparse multiplication so each weight row is sent once. A cache manager merges KV-cache miss requests and prefetches skipped heads' values to keep attention correct.

What would settle it

A membership-inference-style experiment on the revealed sparsity counts: collect prompts from two distinct datasets, compute their per-layer sparsity levels through Comet's predictor, and train a classifier to guess the dataset. If the classifier's accuracy is far above chance, the paper's assumption that sparsity levels leak little about the input is false, and the headline numbers without differential privacy do not carry the claimed privacy.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that activation sparsity is not an obstacle to private LLM inference but a resource. Because model parameters are fixed during inference, the input to each transformer layer determines which ReLU neurons will be zero and which attention heads will be inactive, and a shallow two-layer low-rank predictor can anticipate those states well enough to use them as an execution plan. Comet runs that predictor inside MPC, obtains a secret-shared 0/1 mask, and applies an oblivious shuffle before revealing it: the count of active neurons is disclosed, but their positions are randomized so that plaintext indexing does not expose the input's sparsity pattern. Sparse matrix multiplications, SOMM before the nonlinearity and SIMM after it, then skip the masked-out work; the protocols are proved to reach minimal communication, with each row or column masked and communicated only once, and minimal computation under that bound. A KV-cache manager keeps attention correct when some heads were skipped. On this basis the paper reports 1.87x-2.63x end-to-end speedups and 1.94x-2.64x communication reductions on four LLMs against six prior systems, with an average accuracy loss of about 1.5% on Llama2-7B.

Load-bearing premise

The system's privacy guarantee rests on the assumption that revealing only the number of zero activations, per layer, leaks almost nothing about the user's input; the headline speedups are measured under that assumption, and the paper's optional MPC-based differential privacy would reduce them.

Editorial extensions

If this is right

  • The same Llama2-7B model can be served through MPC at a 2.58x average speedup with no architectural change, only a low-overhead predictor that runs inside MPC.
  • Larger models gain more: the 7B model shows roughly 85% FFN sparsity versus 61% for the 1.3B model, so the speedup grows with model scale.
  • Because the approach targets the communication layer itself, it can be composed with nonlinearity-friendly architectures or better MPC approximations rather than replacing them.
  • KV caching remains usable despite sparse attention: merging miss requests gives a 3.8x communication reduction over per-request refilling, and prefetching adds a further 1.2x.
  • The predictor's overhead stays below 15% of end-to-end inference time, with about 93% recall, which is why average accuracy loss stays around 1.5%.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension is to apply the same shuffle-and-reveal indexing to other data-dependent zero structures in MPC, such as sparse embeddings, graph adjacency products, or mixture-of-experts routing, where revealing only an activity count is acceptable.
  • The optional differential-privacy layer is a genuine tradeoff, not a free patch: the paper reports a 2.01x speedup at epsilon 0.01 versus 2.69x without DP, so a deployment that must hide sparsity levels will give up a meaningful share of the gain.
  • Because a malicious model owner could design a predictor whose sparsity counts are distinctive per input, the predictor itself becomes an attack surface; verifying the predictor or certifying its output with DP noise is a plausible next step.
  • A wrong prediction that a neuron is zero is irreversible and costs accuracy; an online fallback that recomputes a few masked neurons when the predictor is uncertain could widen the usable operating point, though the paper does not evaluate this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents Comet, an MPC-based private inference system for LLMs that predicts activation sparsity to skip zero-valued computations in both linear and non-linear layers, thereby reducing communication and accelerating inference. It introduces a lightweight low-rank sparsity predictor protected by an oblivious-shuffle-based indexing scheme, sparse matrix multiplication protocols (SOMM and SIMM) with claimed communication/computation optimality, and a KV-cache manager that merges and prefetches cache refills. The evaluation covers OPT-1.3B/2.7B/6.7B and Llama2-7B against several private inference baselines, reporting 1.87x-2.63x speedup and 1.94x-2.64x communication reduction with about 1.5% average accuracy loss on Llama2-7B, plus additional results for other architectures and activation functions in appendices.

Significance. If the privacy model holds, Comet is a timely and useful contribution: to my knowledge it is the first system to exploit activation sparsity for MPC-based private LLM inference, and the protocol-level ideas (shuffle-based plaintext indexing, grouping connected components to minimize communication, column-by-row sparse multiplication) are well motivated and clearly presented. The optimality theorems for SOMM and SIMM are a valuable formal anchor, and the evaluation is broad, including different model sizes, bandwidths, and, in appendices, different architectures and activation functions. The predictor is trained on separate public datasets, so the speedup claims are not circular. However, the headline results are reported under a relaxed threat model that reveals per-layer sparsity counts, the paper's own appendix concedes that this can leak under repeated queries, and the DP-inclusive results fall outside the advertised speedup range. Several internal numerical inconsistencies further reduce confidence. With the privacy caveat addressed and the numbers reconciled, the contribution would be solid.

major comments (4)
  1. [Section 3.2, Appendix E, Table 7] The central privacy claim is not supported for the configurations whose performance is advertised. Section 3.2 relaxes the standard MPC threat model by assuming that sparsity levels (counts of zero values) may be revealed, and Appendix E concedes that repeated inference requests could map input-to-sparsity relationships and that a malicious model owner could engineer predictor outputs. The proposed MPC-based differential privacy is described as an optional mitigation, and the headline results (Abstract, Section 7.2, Table 2) are reported without it. Table 7 shows that applying DP with epsilon=0.01 reduces the Llama2-7B speedup to 2.01x, which is outside the advertised 1.87x-2.63x range. The authors must either present DP-enabled results as the main privacy-preserving configuration or provide a formal argument (e.g., a reduction or simulation proof that explicitly accounts for the revealed count) showing that sparsity-level leakage stays within the standard MPC leakage profile.
  2. [Abstract, Section 7.2, Table 2] The headline performance ranges are internally inconsistent with the reported best results. The Abstract and Section 7.2 state a 1.87x-2.63x speedup and a 1.94x-2.64x communication reduction, but Table 2 reports a Llama2-7B total speedup of 2.69x and a communication reduction of 3.14x, and Table 7 also lists the no-DP speedup as 2.69x. These numbers cannot all be correct. The ranges must be recomputed from the full result set, or the text and abstract must be corrected to match the reported tables.
  3. [Section 7.4, Table 3] Table 3 contains a likely data error that affects the stated accuracy claim: MBPP is listed with plaintext accuracy 22.4 and Comet accuracy 91.2, which is impossible given the text's claim of an average accuracy loss of about 1.5%. Using the other seven rows, the average loss is approximately 1.8%, not 1.5%. The table entry must be corrected and the average recomputed, since the accuracy-preservation claim is a key part of the paper's contribution.
  4. [Section 7.3] The interpretation of layer-wise speedups swaps the sparsity percentages. The sentence says "This is due to the higher sparsity rate of the FFN (approximately 50%) compared to that of the MHA (about 90%)", but Section 7.2 reports MHA/FFN sparsity as 49%/85% for Llama2-7B and Figure 3 shows FFN sparsity above 90% and MHA sparsity around 50%. The values are reversed relative to the paper's own measurements, which undermines the explanation of why FFN layers benefit more. This should be corrected.
minor comments (5)
  1. [Section 7.1] The text says "We select five mainstream Transformer private inference systems as baselines" but the Abstract and Figure 12 refer to six systems, and the legend includes Crypten in addition to Iron, Bolt, MPCFormer, SecFormer, and Puma. The baseline count should be made consistent.
  2. [Section 4.1] The notation for the threshold function is confusing: the formula y = sigma(W2(W1x + b1) + b2) uses sigma for both the threshold function and the activation output, and the text says "outputs 1 if sigma(x) > delta" where the inner sigma is the threshold function itself. This should be re-notated to avoid ambiguity.
  3. [Appendix A] The security proof for the oblivious shuffle claims that the revealed sparsity distribution S is uniformly random, but S is a shuffled version of a 0-1 vector and therefore has the same Hamming weight as the original; the proof should explicitly model the sparsity count as leaked and argue that the positions are uniform given that count, or incorporate the count into the leakage profile.
  4. [Section 6] There is a typo: "priavete inference" should read "private inference".
  5. [Appendix C, Table 6] The table reports speedups for several models with Swish/SwiGLU activations after ReLUfication, but the main paper's abstract only claims 1.87x-2.63x speedup over six systems on four models. The appendix numbers (e.g., 3.12x for Falcon-7B) go beyond that range; the relationship between the appendix results and the headline range should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the speedup and communication claims are empirical measurements against external baselines, and the optimality theorems are self-contained given the sparsity distribution as input.

full rationale

Comet's central results are measured, not derived from the claimed prediction. The sparsity predictor is trained on WikiText2 and StarCoder (Section 7.4) and evaluated on held-out benchmarks (Alpaca-derived workloads and eight public LLM benchmarks), so there is no fitted parameter being renamed as a prediction. The SOMM and SIMM protocols take the sparsity distribution S as an explicit input and prove communication/computation optimality conditional on S (Theorems 1 and 2, Appendices B.1 and B.2); the graph-partitioning argument is self-contained and does not import the result being predicted. The speedups over Iron, Bolt, MPCFormer, SecFormer, and Puma are empirical wall-clock and byte-count comparisons (Section 7.2), and the predictor overhead is included in the totals. The only self-referential element is the appended S&P meta-review (Appendix F), which is a record of the review process and is not used as evidence in the technical derivation. The privacy caveat in Section 3.2 and Appendix E—that revealing sparsity levels may leak under repeated queries and requires MPC-based differential privacy as a mitigation—is a limitation of the threat model, not a circular derivation: the speedup numbers are reported for the system without DP, and Appendix E explicitly reports the reduced speedup under DP (2.01x at epsilon=0.01). That concern is substantive but belongs under correctness or threat-model risk, not circularity. No load-bearing step in the paper reduces by construction to its own inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

No new physical or cryptographic entities are invented. The free parameters are the trained predictor components and the threshold. The axioms are standard MPC assumptions plus domain-level assumptions about sparsity stability, non-collusion, and ReLU-fication preserving accuracy. The privacy-relevant assumption about sparsity-level disclosure is the most consequential and is only partially mitigated by an optional DP mode.

free parameters (3)
  • Predictor network weights W1,b1,W2,b2 = not reported
    Trained on WikiText2 and StarCoder to predict activation sparsity (Section 4.1, Section 7.4). The predictor's recall and precision directly determine the accuracy loss and the achieved sparsity, hence the measured speedup.
  • Predictor threshold delta = 0 for main results
    The threshold is set to 0 during pretraining and kept for the main evaluation; Table 4 sweeps it and shows a steep accuracy/speed tradeoff. The choice of threshold affects both speedup and accuracy.
  • Predictor low-rank dimensions = unspecified
    The predictor architecture is described as two fully connected layers with low-rank weight matrices, but the rank is not reported. This hyperparameter affects predictor accuracy and MPC overhead.
assumptions (5)
  • standard math Additive secret sharing and Beaver triples are secure for the basic MPC operations
    Invoked throughout Section 2.1 and in Protocols 1 to 3. The security of the new protocols is built on the security of these primitives.
  • domain assumption The MPC servers are non-colluding and honest-but-curious
    Stated in Section 3.2. If servers collude or behave maliciously, the privacy guarantee and the shuffling protocol break.
  • domain assumption Revealing the sparsity level (number of zero values) does not leak meaningful input information
    Section 3.2 assumes the sparsity level can be revealed. Appendix E acknowledges this is an extra disclosure and offers MPC-based differential privacy as a mitigation, but the main evaluation does not use it. If sparsity correlates with input content, privacy is weaker than standard MPC.
  • domain assumption LLMs exhibit high and predictable activation sparsity after ReLU activation
    Section 2.3 motivates the entire approach. The claimed speedups depend on measured sparsity levels of 80 to 95%, which are empirical properties of the evaluated models.
  • domain assumption Replacing non-ReLU activations with ReLU and fine-tuning preserves model accuracy
    Used to apply Comet to Llama2 and other non-ReLU models (Section 2.3, Appendix C). This relies on prior work such as ProSparse and ReLU Strikes Back, and introduces an accuracy tradeoff not directly measured against the original non-ReLU baseline.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity." pith.science (2026). https://pith.science/paper/XKEUHV2Y

@misc{pith2026250507239,
  author       = {Pith},
  title        = {Pith review of: Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XKEUHV2Y}},
  note         = {Machine review of arXiv:2505.07239}
}
read the original abstract

With the growing use of large language models (LLMs) hosted on cloud platforms to offer inference services, privacy concerns about the potential leakage of sensitive information are escalating. Secure multi-party computation (MPC) is a promising solution to protect the privacy in LLM inference. However, MPC requires frequent inter-server communication, causing high performance overhead. Inspired by the prevalent activation sparsity of LLMs, where most neuron are not activated after non-linear activation functions, we propose an efficient private inference system, Comet. This system employs an accurate and fast predictor to predict the sparsity distribution of activation function output. Additionally, we introduce a new private inference protocol. It efficiently and securely avoids computations involving zero values by exploiting the spatial locality of the predicted sparse distribution. While this computation-avoidance approach impacts the spatiotemporal continuity of KV cache entries, we address this challenge with a low-communication overhead cache refilling strategy that merges miss requests and incorporates a prefetching mechanism. Finally, we evaluate Comet on four common LLMs and compare it with six state-of-the-art private inference systems. Comet achieves a 1.87x-2.63x speedup and a 1.94x-2.64x communication reduction.

Figures

Figures reproduced from arXiv: 2505.07239 by the authors.

Figure 1
Figure 1. Activation sparsity of LLMs. ating private inference in LLMs. The main contributions of this paper are as follows: • We propose a predictor to estimate the sparsity distribu￾tion of activation outputs. To ensure accurate prediction and efficient execution, we implement the predictor using a lightweight neural network. To preserve privacy, the predictor generates a secret-shared sparsity distribution, which is collab… view at source ↗
Figure 3
Figure 3. Activation sparsity of different LLMs, evaluated [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 2
Figure 2. MPC-based private inference time breakdown with [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Communication cost breakdown for OPT-6.7B [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: System overview of Comet. 3. Comet System Overview 3.1. Design motivation As previously noted, LLMs commonly exhibit activation sparsity, where many attention heads in MHA are not acti￾vated and numerous FFN ReLU outputs are zero. By accu￾rately predicting this sparsit…
Figure 6
Figure 6. Figure 6: Ideal speedup. 3.2. Threat model Comet adopts a standard MPC threat model, where predefined programs run among multiple parties, protecting input data and intermediate results while typically revealing only the final outputs to designated parties. All correlated random…
Figure 7
Figure 7. Figure 7: Activation sparsity predictor and secure indexing [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 8
Figure 8. Figure 8: Layers and protocols design for FFN in Comet. The design for MHA is similar. 5.1. Sparse output matrix multiplication protocol for preceding linear layer For the preceding linear layer matrix multiplication XY = Z, given the sparsity distribution of Z, we can pre-ident…
Figure 9
Figure 9. Figure 9: (c). To determine the grouping with optimal communica￾tion and computation, we model it as a subgraph partition problem in a bipartite graph. As shown in [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: (b), the third column of X contains two nonzero elements, X2,3 and X4,3. As a result, Y3 is involved in two separate multiplications, requiring [[Y3]] to be masked and communicated twice in MPC. To avoid this redundant communication, we shift from row-by-column multip…
Figure 11
Figure 11. Figure 11: KV cache manager. 6.1. Merging cache miss requests A straightforward approach to handle cache misses is to respond to each miss request immediately by refilling the cache entry. This requires the MPC servers to collaboratively execute matrix multiplications in the Spa…
Figure 12
Figure 12. Figure 12: Overall performance of Comet. private inference protocol in Comet, including oblivious shuffle, SOMM and SIMM. The activation sparsity predictor and cache manager are designed as independent modules, without modification to the model architecture. Models We use two po…
Figure 14
Figure 14. Figure 14: Accuracy and inference speed of various methods [PITH_FULL_IMAGE:figures/full_fig_p011_14.png]
Figure 15
Figure 15. Figure 15: Ablation study. 7.6. Component-wise Analysis To provide a deeper understanding of the contributions of individual components in Comet, we perform a detailed component-wise analysis. Each subsection focuses on a specific design element, evaluating its unique characteri…
Figure 17
Figure 17. Figure 17: Precision and recall of predictor. Predictor threshold. We analyze how different predictor thresholds affect Comet’s accuracy and inference speed us￾ing the Llama2-7B model. During pretraining, the predictor threshold was set to 0. We tested various thresholds on the …
Figure 16
Figure 16. Figure 16: Predictor overhead. Predictor accuracy. We evaluate the precision and recall of the predictor on models of varying sizes. A lower preci￾sion indicates that more not activated neuron are incorrectly predicted as activated, introducing additional avoidable com￾putations…
Figure 19
Figure 19. Figure 19: Communication cost comparison of cache refill [PITH_FULL_IMAGE:figures/full_fig_p013_19.png]
Figure 18
Figure 18. Figure 18: Communication cost comparison of matrix mul [PITH_FULL_IMAGE:figures/full_fig_p013_18.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

100 extracted references · 32 canonical work pages

  1. [1]

    Privformer: Privacy-preserving transformer with mpc,

    Y . Akimoto, K. Fukuchi, Y . Akimoto, and J. Sakuma, “Privformer: Privacy-preserving transformer with mpc,” in 2023 IEEE 8th Euro- pean Symposium on Security and Privacy (EuroS&P) . IEEE, 2023, pp. 392–410

    2023

  2. [2]

    The falcon series of open language models,

    E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojo- caru, M. Debbah, ´E. Goffinet, D. Hesslow, J. Launay, Q. Malartic et al. , “The falcon series of open language models,” arXiv preprint arXiv:2311.16867, 2023

    arXiv 2023

  3. [3]

    Exposure notification privacy-preserving analyt- ics (enpa) white paper,

    Apple and Google, “Exposure notification privacy-preserving analyt- ics (enpa) white paper,” 2021

    2021

  4. [4]

    Oblivious linear group actions and applications,

    N. Attrapadung, G. Hanaoaka, T. Matsuda, H. Morita, K. Ohara, J. C. Schuldt, T. Teruya, and K. Tozawa, “Oblivious linear group actions and applications,” in Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , 2021, pp. 630–650

    2021

  5. [5]

    Program synthesis with large language models,

    J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le et al., “Program synthesis with large language models,” arXiv preprint arXiv:2108.07732 , 2021

    arXiv 2021

  6. [6]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang et al. , “Qwen technical report,” arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  7. [7]

    Efficient multiparty protocols using circuit randomiza- tion,

    D. Beaver, “Efficient multiparty protocols using circuit randomiza- tion,” in Advances in Cryptology CRYPTO 91: Proceedings 11 . Springer, 1992, pp. 420–432

    1992

  8. [8]

    Deepseek llm: Scaling open-source lan- guage models with longtermism,

    X. Bi, D. Chen, G. Chen, S. Chen, D. Dai, C. Deng, H. Ding, K. Dong, Q. Du, Z. Fu et al., “Deepseek llm: Scaling open-source lan- guage models with longtermism,” arXiv preprint arXiv:2401.02954 , 2024

    arXiv 2024

Show all 100 references
  1. [9]

    Piqa: Reasoning about physical commonsense in natural language,

    Y . Bisk, R. Zellers, J. Gao, Y . Choi et al. , “Piqa: Reasoning about physical commonsense in natural language,” in Proceedings of the AAAI conference on artificial intelligence , vol. 34, no. 05, 2020, pp. 7432–7439

    2020

  2. [10]

    Sparsity and privacy in secret sharing: A fundamental trade-off,

    R. Bitar, M. Egger, A. Wachter-Zeh, and M. Xhemrishi, “Sparsity and privacy in secret sharing: A fundamental trade-off,” IEEE Trans- actions on Information Forensics and Security , 2024

    2024

  3. [11]

    Safeguarding cryptographic keys,

    G. R. Blakley, “Safeguarding cryptographic keys,” in Managing re- quirements knowledge, international workshop on . IEEE Computer Society, 1979, pp. 313–313

    1979

  4. [12]

    On the opportunities and risks of foundation models,

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill et al., “On the opportunities and risks of foundation models,” arXiv preprint arXiv:2108.07258, 2021

    2021 arXiv

  5. [13]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al. , “Language models are few-shot learners,” Advances in neural information pro- cessing systems, vol. 33, pp. 1877–1901, 2020

    1901

  6. [14]

    Flash: fast and ro- bust framework for privacy-preserving machine learning,

    M. Byali, H. Chaudhari, A. Patra, and A. Suresh, “Flash: fast and ro- bust framework for privacy-preserving machine learning,” Cryptology ePrint Archive, 2019

    2019

  7. [15]

    Universally composable security: A new paradigm for cryptographic protocols,

    R. Canetti, “Universally composable security: A new paradigm for cryptographic protocols,” in Proceedings 42nd IEEE Symposium on Foundations of Computer Science . IEEE, 2001, pp. 136–145

    2001

  8. [16]

    Membership inference attacks from first principles,

    N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis, and F. Tramer, “Membership inference attacks from first principles,” in 2022 IEEE Symposium on Security and Privacy (SP) . IEEE, 2022, pp. 1897– 1914

    2022

  9. [17]

    Data distributional properties drive emergent in-context learning in transformers,

    S. Chan, A. Santoro, A. Lampinen, J. Wang, A. Singh, P. Richemond, J. McClelland, and F. Hill, “Data distributional properties drive emergent in-context learning in transformers,” Advances in Neural Information Processing Systems , vol. 35, pp. 18 878–18 891, 2022

    2022

  10. [18]

    {SIMC}:{ML} inference secure against malicious clients at{Semi- Honest} cost,

    N. Chandran, D. Gupta, S. L. B. Obbattu, and A. Shah, “{SIMC}:{ML} inference secure against malicious clients at{Semi- Honest} cost,” in 31st USENIX Security Symposium (USENIX Secu- rity 22), 2022, pp. 1361–1378

    2022

  11. [19]

    Secret-shared shuffle,

    M. Chase, E. Ghosh, and O. Poburinnaya, “Secret-shared shuffle,” in Advances in Cryptology–ASIACRYPT 2020: 26th International Con- ference on the Theory and Application of Cryptology and Information Security, Daejeon, South Korea, December 7–11, 2020, Proceedings, Part III 26....

    2020

  12. [20]

    Trident: Efficient 4pc framework for privacy preserving machine learning,

    H. Chaudhari, R. Rachuri, and A. Suresh, “Trident: Efficient 4pc framework for privacy preserving machine learning,” arXiv preprint arXiv:1912.02631, 2019

    1912 arXiv

  13. [21]

    Evaluating large language models trained on code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Ka- plan, H. Edwards, Y . Burda, N. Joseph, G. Brockman et al. , “Evaluating large language models trained on code,” arXiv preprint arXiv:2107.03374, 2021

    2021 arXiv

  14. [22]

    The-x: Privacy-preserving transformer inference with homomorphic encryption,

    T. Chen, H. Bao, S. Huang, L. Dong, B. Jiao, D. Jiang, H. Zhou, J. Li, and F. Wei, “The-x: Privacy-preserving transformer inference with homomorphic encryption,” arXiv preprint arXiv:2206.00216 , 2022

    2022 arXiv

  15. [23]

    Securetlm: Private inference for transformer-based large model with mpc,

    Y . Chen, X. Meng, Z. Shi, Z. Ning, and J. Lin, “Securetlm: Private inference for transformer-based large model with mpc,” Information Sciences, vol. 667, p. 120429, 2024

    2024

  16. [24]

    An efficient secure three-party sorting protocol with an honest majority,

    K. Chida, K. Hamada, D. Ikarashi, R. Kikuchi, N. Kiribuchi, and B. Pinkas, “An efficient secure three-party sorting protocol with an honest majority,” Cryptology ePrint Archive , 2019

    2019

  17. [25]

    Boolq: Exploring the surprising difficulty of natural yes/no questions,

    C. Clark, K. Lee, M.-W. Chang, T. Kwiatkowski, M. Collins, and K. Toutanova, “Boolq: Exploring the surprising difficulty of natural yes/no questions,” arXiv preprint arXiv:1905.10044 , 2019

    1905 arXiv

  18. [26]

    Cramer, I

    R. Cramer, I. B. Damg ˚ard et al. , Secure multiparty computation . Cambridge University Press, 2015

    2015

  19. [27]

    Exploiting data sparsity in secure cross-platform social recommendation,

    J. Cui, C. Chen, L. Lyu, C. Yang, and W. Li, “Exploiting data sparsity in secure cross-platform social recommendation,” Advances in Neural Information Processing Systems , vol. 34, pp. 10 524–10 534, 2021

    2021

  20. [28]

    Bert: Pre- training of deep bidirectional transformers for language understand- ing,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre- training of deep bidirectional transformers for language understand- ing,” in Proceedings of the 2019 conference of the North Ameri- can chapter of the association for computational linguistics: human language techn...

    2019

  21. [29]

    torch.sparse.mm,

    P. Documentation, “torch.sparse.mm,” Online, 2024, accessed: March 12, 2025. [Online]. Available: https://pytorch.org/docs/stable/ generated/torch.sparse.mm

    2024

  22. [30]

    Puma: Secure inference of llama- 7b in five minutes,

    Y . Dong, W.-j. Lu, Y . Zheng, H. Wu, D. Zhao, J. Tan, Z. Huang, C. Hong, T. Wei, and W. Cheng, “Puma: Secure inference of llama- 7b in five minutes,” arXiv preprint arXiv:2307.12533 , 2023

    2023

  23. [31]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gellyet al., “An image is worth 16x16 words: Transformers for image recognition at scale,” arXiv preprint arXiv:2010.11929 , 2020

    2010 arXiv

  24. [32]

    How to play any mental game, or a completeness theorem for protocols with honest majority,

    O. Goldreich, S. Micali, and A. Wigderson, “How to play any mental game, or a completeness theorem for protocols with honest majority,” in Providing Sound Foundations for Cryptography: On the Work of Shafi Goldwasser and Silvio Micali , 2019, pp. 307–328

    2019

  25. [33]

    The llama 3 herd of models,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al- Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan et al., “The llama 3 herd of models,” arXiv preprint arXiv:2407.21783 , 2024

    2024 arXiv

  26. [34]

    Sigma: Secure gpt inference with function secret sharing,

    K. Gupta, N. Jawalkar, A. Mukherjee, N. Chandran, D. Gupta, A. Pan- war, and R. Sharma, “Sigma: Secure gpt inference with function secret sharing,” Cryptology ePrint Archive , 2023

    2023

  27. [35]

    Iron: Private inference on transformers,

    M. Hao, H. Li, H. Chen, P. Xing, G. Xu, and T. Zhang, “Iron: Private inference on transformers,” Advances in neural information processing systems, vol. 35, pp. 15 718–15 731, 2022

    2022

  28. [36]

    Measuring massive multitask language understanding,

    D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt, “Measuring massive multitask language understanding,” arXiv preprint arXiv:2009.03300 , 2020

    2009 arXiv

  29. [37]

    Gaussian error linear units (gelus),

    D. Hendrycks and K. Gimpel, “Gaussian error linear units (gelus),” arXiv preprint arXiv:1606.08415 , 2016

    2016 arXiv

  30. [38]

    Ci- phergpt: Secure two-party gpt inference,

    X. Hou, J. Liu, J. Li, Y . Li, W.-j. Lu, C. Hong, and K. Ren, “Ci- phergpt: Secure two-party gpt inference,” Cryptology ePrint Archive, 2023

    2023

  31. [39]

    Mixtral of experts,

    A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bamford, D. S. Chaplot, D. d. l. Casas, E. B. Hanna, F. Bressand et al., “Mixtral of experts,” arXiv preprint arXiv:2401.04088 , 2024

    2024 arXiv

  32. [40]

    Secure multi-party differential privacy,

    P. Kairouz, S. Oh, and P. Viswanath, “Secure multi-party differential privacy,” in Advances in Neural Information Processing Systems , vol. 28, 2015

    2015

  33. [41]

    Mascot: faster malicious arith- metic secure computation with oblivious transfer,

    M. Keller, E. Orsini, and P. Scholl, “Mascot: faster malicious arith- metic secure computation with oblivious transfer,” in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communica- tions Security, 2016, pp. 830–842

    2016

  34. [42]

    Crypten: Secure multi-party computation meets machine learning,

    B. Knott, S. Venkataraman, A. Hannun, S. Sengupta, M. Ibrahim, and L. van der Maaten, “Crypten: Secure multi-party computation meets machine learning,” Advances in Neural Information Processing Systems, vol. 34, pp. 4961–4973, 2021

    2021

  35. [43]

    Cryptflow: Secure tensorflow inference,

    N. Kumar, M. Rathee, N. Chandran, D. Gupta, A. Rastogi, and R. Sharma, “Cryptflow: Secure tensorflow inference,” in 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 2020, pp. 336–353

    2020

  36. [44]

    Round-efficient oblivious database manipulation,

    S. Laur, J. Willemson, and B. Zhang, “Round-efficient oblivious database manipulation,” in Information Security: 14th International Conference, ISC 2011, Xi’an, China, October 26-29, 2011. Proceed- ings 14. Springer, 2011, pp. 262–277

    2011

  37. [45]

    Bloom: A 176b-parameter open-access multilingual language model,

    T. Le Scao, A. Fan, C. Akiki, E. Pavlick, S. Ili ´c, D. Hesslow, R. Castagn ´e, A. S. Luccioni, F. Yvon, M. Gall ´e et al. , “Bloom: A 176b-parameter open-access multilingual language model,” 2023

    2023

  38. [46]

    Muse: Secure inference resilient to malicious clients,

    R. Lehmkuhl, P. Mishra, A. Srinivasan, and R. A. Popa, “Muse: Secure inference resilient to malicious clients,” in 30th USENIX Security Symposium (USENIX Security 21) , 2021, pp. 2201–2218

    2021

  39. [47]

    Mpcformer: fast, performant and private transformer inference with mpc,

    D. Li, R. Shao, H. Wang, H. Guo, E. P. Xing, and H. Zhang, “Mpcformer: fast, performant and private transformer inference with mpc,” arXiv preprint arXiv:2211.01452 , 2022

    2022 arXiv

  40. [48]

    A survey on large language model acceleration based on kv cache management,

    H. Li, Y . Li, A. Tian, T. Tang, Z. Xu, X. Chen, N. Hu, W. Dong, Q. Li, and L. Chen, “A survey on large language model acceleration based on kv cache management,” arXiv preprint arXiv:2412.19442 , 2024

    2024 arXiv

  41. [49]

    Starcoder: may the source be with you!

    R. Li, L. B. Allal, Y . Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. Li, J. Chim et al., “Starcoder: may the source be with you!” arXiv preprint arXiv:2305.06161 , 2023

    2023 arXiv

  42. [50]

    Privacy-preserving feature selection with secure multiparty computation,

    X. Li, R. Dowsley, and M. De Cock, “Privacy-preserving feature selection with secure multiparty computation,” in International Con- ference on Machine Learning . PMLR, 2021, pp. 6326–6336

    2021

  43. [51]

    Backdoor learning: A survey,

    Y . Li, Y . Jiang, Z. Li, and S.-T. Xia, “Backdoor learning: A survey,” IEEE transactions on neural networks and learning systems , vol. 35, no. 1, pp. 5–22, 2022

    2022

  44. [52]

    The lazy neuron phenomenon: On emergence of activation sparsity in transformers,

    Z. Li, C. You, S. Bhojanapalli, D. Li, A. S. Rawat, S. J. Reddi, K. Ye, F. Chern, F. Yu, R. Guo et al., “The lazy neuron phenomenon: On emergence of activation sparsity in transformers,” arXiv preprint arXiv:2210.06313, 2022

    2022 arXiv

  45. [53]

    Holistic evaluation of language models,

    P. Liang, R. Bommasani, T. Lee, D. Tsipras, D. Soylu, M. Yasunaga, Y . Zhang, D. Narayanan, Y . Wu, A. Kumaret al., “Holistic evaluation of language models,” arXiv preprint arXiv:2211.09110 , 2022

    2022 arXiv

  46. [54]

    Merge: Fast private text generation,

    Z. Liang, P. Wang, R. Zhang, N. Xu, S. Zhang, L. Xing, H. Bai, and Z. Zhou, “Merge: Fast private text generation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 38, no. 18, 2024, pp. 19 884–19 892

    2024

  47. [55]

    How to simulate it–a tutorial on the simulation proof technique,

    Y . Lindell, “How to simulate it–a tutorial on the simulation proof technique,” Tutorials on the Foundations of Cryptography: Dedicated to Oded Goldreich , pp. 277–346, 2017

    2017

  48. [56]

    Llms can understand encrypted prompt: Towards privacy-computing friendly transformers,

    X. Liu and Z. Liu, “Llms can understand encrypted prompt: Towards privacy-computing friendly transformers,” arXiv preprint arXiv:2305.18396, 2023

    2023 arXiv

  49. [57]

    Deja vu: Contextual sparsity for efficient llms at inference time,

    Z. Liu, J. Wang, T. Dao, T. Zhou, B. Yuan, Z. Song, A. Shrivastava, C. Zhang, Y . Tian, C. Re et al. , “Deja vu: Contextual sparsity for efficient llms at inference time,” in International Conference on Machine Learning. PMLR, 2023, pp. 22 137–22 176

    2023

  50. [58]

    Bumblebee: Secure two-party inference framework for large transformers,

    W.-j. Lu, Z. Huang, Z. Gu, J. Li, J. Liu, K. Ren, C. Hong, T. Wei, and W. Chen, “Bumblebee: Secure two-party inference framework for large transformers,” Cryptology ePrint Archive , 2023

    2023

  51. [59]

    Secformer: Towards fast and accurate privacy-preserving inference for large language models,

    J. Luo, Y . Zhang, J. Zhang, X. Mu, H. Wang, Y . Yu, and Z. Xu, “Secformer: Towards fast and accurate privacy-preserving inference for large language models,” arXiv preprint arXiv:2401.00793 , 2024

    2024 arXiv

  52. [60]

    Accelerating relu for mpc-based private in- ference with a communication-efficient sign estimation,

    K. Maeng and G. E. Suh, “Accelerating relu for mpc-based private in- ference with a communication-efficient sign estimation,” Proceedings of Machine Learning and Systems , vol. 6, pp. 128–147, 2024

    2024

  53. [61]

    Pointer sentinel mixture models,

    S. Merity, C. Xiong, J. Bradbury, and R. Socher, “Pointer sentinel mixture models,” arXiv preprint arXiv:1609.07843 , 2016

    2016 arXiv

  54. [62]

    Relu strikes back: Ex- ploiting activation sparsity in large language models,

    I. Mirzadeh, K. Alizadeh, S. Mehta, C. C. Del Mundo, O. Tuzel, G. Samei, M. Rastegari, and M. Farajtabar, “Relu strikes back: Ex- ploiting activation sparsity in large language models,” arXiv preprint arXiv:2310.04564, 2023

    2023 arXiv

  55. [63]

    Aby3: A mixed protocol framework for machine learning,

    P. Mohassel and P. Rindal, “Aby3: A mixed protocol framework for machine learning,” in Proceedings of the 2018 ACM SIGSAC conference on computer and communications security , 2018, pp. 35– 52

    2018

  56. [64]

    Secure multi-party shuffling,

    M. Movahedi, J. Saia, and M. Zamani, “Secure multi-party shuffling,” in Structural Information and Communication Complexity: 22nd In- ternational Colloquium, SIROCCO 2015, Montserrat, Spain, July 14- 16, 2015. Post-Proceedings 22 . Springer, 2015, pp. 459–473

    2015

  57. [65]

    Powering secure computation, together,

    MPC Alliance, “Powering secure computation, together,” https:// www.mpcalliance.org/, 2023, accessed: 2023

    2023

  58. [66]

    Rectified linear units improve restricted boltzmann machines,

    V . Nair and G. E. Hinton, “Rectified linear units improve restricted boltzmann machines,” in Proceedings of the 27th international con- ference on machine learning (ICML-10) , 2010, pp. 807–814

    2010

  59. [67]

    Public-key cryptosystems based on composite degree residuosity classes,

    P. Paillier, “Public-key cryptosystems based on composite degree residuosity classes,” in EUROCRYPT, 1999, pp. 223–238

    1999

  60. [68]

    Bolt: Privacy-preserving, accurate and efficient inference for transformers,

    Q. Pang, J. Zhu, H. M ¨ollering, W. Zheng, and T. Schneider, “Bolt: Privacy-preserving, accurate and efficient inference for transformers,” Cryptology ePrint Archive , 2023

    2023

  61. [69]

    The lambada dataset: Word prediction requiring a broad discourse context,

    D. Paperno, G. Kruszewski, A. Lazaridou, Q. N. Pham, R. Bernardi, S. Pezzelle, M. Baroni, G. Boleda, and R. Fern ´andez, “The lambada dataset: Word prediction requiring a broad discourse context,” arXiv preprint arXiv:1606.06031, 2016

    2016 arXiv

  62. [70]

    Blaze: blazing fast privacy-preserving machine learning,

    A. Patra and A. Suresh, “Blaze: blazing fast privacy-preserving machine learning,” arXiv preprint arXiv:2005.09042 , 2020

    2005 arXiv

  63. [71]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., “Language models are unsupervised multitask learners,”OpenAI blog, vol. 1, no. 8, p. 9, 2019

    2019

  64. [72]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y . Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020

    2020

  65. [73]

    Chameleon: A hybrid secure computation framework for machine learning applications,

    M. S. Riazi, C. Weinert, O. Tkachenko, E. M. Songhori, T. Schnei- der, and F. Koushanfar, “Chameleon: A hybrid secure computation framework for machine learning applications,” in Proceedings of the 2018 on Asia conference on computer and communications security , 2018, pp. 707–721

    2018

  66. [74]

    Choice of plausi- ble alternatives: An evaluation of commonsense causal reasoning

    M. Roemmele, C. A. Bejan, and A. S. Gordon, “Choice of plausi- ble alternatives: An evaluation of commonsense causal reasoning.” in AAAI spring symposium: logical formalizations of commonsense reasoning, 2011, pp. 90–95

    2011

  67. [75]

    Make some room for the zeros: Data sparsity in secure distributed machine learning,

    P. Schoppmann, A. Gasc ´on, M. Raykova, and B. Pinkas, “Make some room for the zeros: Data sparsity in secure distributed machine learning,” in Proceedings of the 2019 ACM SIGSAC conference on computer and communications security , 2019, pp. 1335–1350

    2019

  68. [76]

    How to share a secret,

    A. Shamir, “How to share a secret,” Communications of the ACM , vol. 22, no. 11, pp. 612–613, 1979

    1979

  69. [77]

    Glu variants improve transformer,

    N. Shazeer, “Glu variants improve transformer,” arXiv preprint arXiv:2002.05202, 2020

    2002 arXiv

  70. [78]

    Deep learning inference service at microsoft,

    J. Soifer, J. Li, M. Li, J. Zhu, Y . Li, Y . He, E. Zheng, A. Oltean, M. Mosyak, C. Barnes et al. , “Deep learning inference service at microsoft,” in 2019 USENIX Conference on Operational Machine Learning (OpML 19) , 2019, pp. 15–17

    2019

  71. [79]

    Prosparse: Introducing and enhancing intrinsic activation sparsity within large language models,

    C. Song, X. Han, Z. Zhang, S. Hu, X. Shi, K. Li, C. Chen, Z. Liu, G. Li, T. Yang et al., “Prosparse: Introducing and enhancing intrinsic activation sparsity within large language models,” arXiv preprint arXiv:2402.13516, 2024

    2024 arXiv

  72. [80]

    Powerinfer: Fast large language model serving with a consumer-grade gpu,

    Y . Song, Z. Mi, H. Xie, and H. Chen, “Powerinfer: Fast large language model serving with a consumer-grade gpu,” arXiv preprint arXiv:2312.12456, 2023

    2023 arXiv

  73. [81]

    Turbo sparse: Achieving llm sota performance with minimal acti- vated parameters,

    Y . Song, H. Xie, Z. Zhang, B. Wen, L. Ma, Z. Mi, and H. Chen, “Turbo sparse: Achieving llm sota performance with minimal acti- vated parameters,” arXiv preprint arXiv:2406.05955 , 2024

    2024 arXiv

  74. [82]

    Cryptgpu: Fast privacy- preserving machine learning on the gpu,

    S. Tan, B. Knott, Y . Tian, and D. J. Wu, “Cryptgpu: Fast privacy- preserving machine learning on the gpu,” in 2021 IEEE Symposium on Security and Privacy (SP) . IEEE, 2021, pp. 1021–1038

    2021

  75. [83]

    Stanford alpaca: An instruction- following llama model,

    R. Taori, I. Gulrajani, T. Zhang, Y . Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto, “Stanford alpaca: An instruction- following llama model,” 2023

    2023

  76. [84]

    Depth-first search and linear graph algorithms,

    R. Tarjan, “Depth-first search and linear graph algorithms,” SIAM journal on computing , vol. 1, no. 2, pp. 146–160, 1972

    1972

  77. [85]

    Gemma: Open models based on gemini research and technology,

    G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivi `ere, M. S. Kale, J. Love et al., “Gemma: Open models based on gemini research and technology,” arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  78. [86]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al. , “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  79. [87]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  80. [88]

    Securenn: 3-party secure computation for neural network training,

    S. Wagh, D. Gupta, and N. Chandran, “Securenn: 3-party secure computation for neural network training,” Proceedings on Privacy Enhancing Technologies, 2019

    2019

  81. [89]

    Falcon: Honest-majority maliciously secure framework for private deep learning,

    S. Wagh, S. Tople, F. Benhamouda, E. Kushilevitz, P. Mittal, and T. Rabin, “Falcon: Honest-majority maliciously secure framework for private deep learning,” arXiv preprint arXiv:2004.02229 , 2020

    2004 arXiv

  82. [90]

    Glue: A multi-task benchmark and analysis platform for natural language understanding,

    A. Wang, A. Singh, J. Michael, F. Hill, O. Levy, and S. R. Bowman, “Glue: A multi-task benchmark and analysis platform for natural language understanding,” arXiv preprint arXiv:1804.07461 , 2018

    2018 arXiv

  83. [91]

    Q-sparse: All large language models can be fully sparsely-activated,

    H. Wang, S. Ma, R. Wang, and F. Wei, “Q-sparse: All large language models can be fully sparsely-activated,” arXiv preprint arXiv:2407.10969, 2024

    2024 arXiv

  84. [92]

    Huggingface’s transformers: State-of-the-art natural language processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz et al. , “Huggingface’s transformers: State-of-the-art natural language processing,” arXiv preprint arXiv:1910.03771, 2019

    1910 arXiv

  85. [93]

    Distilling script knowledge from large language models for constrained language planning,

    S. Yuan, J. Chen, Z. Fu, X. Ge, S. Shah, C. R. Jankowski, Y . Xiao, and D. Yang, “Distilling script knowledge from large language models for constrained language planning,” arXiv preprint arXiv:2305.05252, 2023

    2023 arXiv

  86. [94]

    Opt: Open pre-trained trans- former language models,

    S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. De- wan, M. Diab, X. Li, X. V . Lin et al., “Opt: Open pre-trained trans- former language models,” arXiv preprint arXiv:2205.01068 , 2022

    2022 arXiv

  87. [95]

    Relu 2 wins: Discovering efficient activation functions for sparse llms,

    Z. Zhang, Y . Song, G. Yu, X. Han, Y . Lin, C. Xiao, C. Song, Z. Liu, Z. Mi, and M. Sun, “Relu 2 wins: Discovering efficient activation functions for sparse llms,” arXiv preprint arXiv:2402.03804 , 2024

    2024 arXiv

  88. [96]

    R-sparse: Rank-aware activation sparsity for efficient llm inference,

    Z. Zhang, Z. Liu, Y . Tian, H. Khaitan, Z. Wang, and S. Li, “R-sparse: Rank-aware activation sparsity for efficient llm inference,” in The Thirteenth International Conference on Learning Representations , 2025. Appendix A. Security Proof The security of Comet adheres to the st...

    2025

  89. [97]

    0” are flipped to “1

    have shown that replacing them with ReLU-family functions (e.g., ReLU [66], ReLU 2 [95], shiftedReLU [62], dReLU [81]) followed by fine-tuning enables activation sparsity without significant accuracy loss. We applied this ReLUfication strategy to eight different LLMs, as shown...

    2025

  90. [98]

    The paper addresses a timely challenge in secure inference for LLMs and notably improves over state- of-the-art in speed and communication

  91. [99]

    The proposed method is presented comprehensively, and the authors provide an extensive evaluation (in- cluding W AN settings, different activation functions, and various LLM architectures)

  92. [100]

    The paper contains many interesting and novel ideas on how to speed up inference

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.