Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

I Know What You Said: Unveiling Hardware Cache Side-Channels in Local Large Language Model Inference

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A spy process can reconstruct both the prompt and the reply of a local LLM session from cache timing alone.

desk verdict First credible demonstration that a co-resident unprivileged process can recover both sides of a local LLM conversation via cache side channels, but the claim needs to be scoped to zero-copy, file-backed deployments. read the letter →

arxiv 2505.06738 v3 pith:A5E3JYT5 submitted 2025-05-10 cs.CR

Zibo Gao , Junjie Hu , Feng Guo , Yixin Zhang , Yinglong Han , Siyuan Liu , Haiyang Li , Zhiqiang Lv This is my paper · ORCID
classification cs.CR
keywords cacheside-channelflush+reloadlocalLLMinferencetokenembeddingautoregressivetimingpromptprivacytextreconstructionsecurity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that locally deployed LLM inference leaks both the user's prompt and the model's response through a hardware cache side channel, with no special privilege and no interaction with the victim model. The claimed leakage has two sources: the token-embedding lookup table reveals which tokens are processed, and the periodic timing of autoregressive decode phases reveals the order of output tokens. To turn this leak into text, the paper builds a two-stage reconstruction pipeline: a fine-tuned LLM cleans the noisy token list and timing waveform into the model output, and a second fine-tuned LLM uses that output as context to reorder the scrambled bag of input tokens. The reported numbers are specific: average Levenshtein similarity of 94.8% for restored output and 82.7% for restored input, with cosine similarities of 98.7% and 98.0% respectively. If these results hold, the privacy promise of local inference is broken by a co-resident spy.

What carries the argument

The load-bearing object is the token embedding table W: with one-hot input, the embedding step is E_i = W[token_i], so each token fetches exactly one row of W into cache, turning vocabulary membership into a reproducible address trace. The attack probes the vocabulary-sized set of addresses inside W with the flush+reload technique and treats a cache hit on row j as evidence that token j was processed. The second mechanism is the timing signal: decode phases are modeled as a Dirac impulse train that is approximately periodic, and the paper uses the power spectral density and the SWIPE' pitch estimator to extract the fundamental period, then normalizes the first-order differences of hit timestamps to make the signal hardware-invariant. These two signals are consumed by two fine-tuned LLMs that act as learned denoisers and reorderers, so the framework's text-recovery power comes from combining a microarchitectural leak with statistical language-model priors.

What would settle it

Run the identical cache-trace capture against a victim deployment that disables zero-copy model loading and has kernel page deduplication turned off: if no periodic embedding-row hit pattern with a clear PSD peak appears during decode phases, the claimed side channel does not exist for that configuration; a complementary check would be to store the embedding table under a secret random row permutation and see whether the recovered token indices still correlate with the true tokens.

Watch

Extended reading notes

Core claim

The central discovery is that two ordinary properties of LLM inference form a side channel. First, because token embedding is a lookup of one row per token from a large embedding table, every token a model reads or writes is reflected in which cache lines of that table were fetched. Second, because autoregressive decoding is strictly serial, each generated token lands at a roughly periodic timestamp, and the paper observes this periodicity in the power spectral density of cache-hit events. On that basis, the paper argues that a spy who shares the model file's memory pages with the victim can run flush+reload to harvest a token list, separate the prefill phase from the decode phases by hit density, and use the timing signal to distinguish genuine tokens from false positives and missing tokens. The paper then claims that fine-tuned LLMs can complete the reconstruction: one fuses the token list with the normalized timing waveform to recover the output text, and another restores the original order of the shuffled input tokens using the recovered output as context. The evaluations across multiple model families, inference frameworks, CPUs, and operating systems are presented as evidence that this eavesdropping path is practical, not just theoretical.

Load-bearing premise

The attack requires the spy and victim to share the physical memory of the embedding table, normally via zero-copy file mapping or OS page deduplication; if both are disabled, the flush+reload probe sees nothing and the whole recovery chain collapses.

Editorial extensions

If this is right

  • Process isolation alone no longer protects local LLM sessions: an unprivileged co-located application can extract the text a user types and the model writes back.
  • Because the vulnerability stems from the standard embedding-lookup design and the autoregressive loop, it applies to a broad class of local deployments, not to a single bug in one framework.
  • Output text is recoverable near-verbatim, with average Levenshtein similarity of 94.8%, so side-channel eavesdropping is a realistic route to extracting names, addresses, and other personally identifiable information from local chat sessions.
  • Input recovery preserves semantics even for long prompts, with cosine similarity above 92% in the worst reported case, though character-level fidelity drops as input length grows, consistent with a factorial reordering search space.
  • The main countermeasures have real costs: disabling zero-copy loading costs loading latency and extra memory, and hardware cache partitioning is usually unavailable on consumer CPUs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be to randomize or permute the row-to-address mapping of the embedding table; if the spy's address-to-token dictionary is broken, the token-value leak is blocked without changing the model, and the paper does not evaluate this countermeasure.
  • The attack's dependence on strict periodicity suggests that speculative decoding, variable batch sizes, or CPU frequency scaling that jitters the per-token interval would degrade the timing channel; measuring how much timing noise is needed to hide the PSD peak would bound the attack's applicability.
  • Since tokenizers are publicly documented, the same address-dictionary method could be applied to future model formats that store the embedding table in column-major or quantized layouts, provided the row recovery is adjusted accordingly.
  • The reconstruction LLMs themselves are reusable: the same 'noisy token list plus waveform to text' formulation could denoise other shared-memory side channels, such as page-table or prefetcher-based leaks, without retraining from scratch.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes the first hardware cache side-channel attack against local LLM inference, claiming to reconstruct both the victim's prompt and the model's response. The attack exploits cache access patterns of the token embedding table to infer token values and the timing of autoregressive decode phases to infer token positions. The authors implement flush+reload over a shared model-file mapping, map cache hits to tokens, and fine-tune two LLMs (LLMA and LLMB) on synthetically generated noisy traces to reconstruct output and input text. They report high accuracy across several LLMs, frameworks, and machines, with average Levenshtein similarity of 94.8% for output and 82.7% for input against llama.cpp, and cosine similarity near 98%. The paper also includes ablations, a framework survey, a hardware survey, countermeasure discussion, and an open-source artifact.

Significance. If the claims are supported, this is a significant contribution: it extends hardware side-channel attacks from discriminative DNNs to full-text extraction from generative LLM inference, and it does so without direct interaction with the victim model. The paper is commendable for shipping an artifact, running end-to-end attacks against real deployments such as llama.cpp and Ollama, and providing ablations that separate the contributions of the timing signal, the reconstructed output context, and the SCA data. The core mechanism—observing secret-dependent accesses to the embedding table—is plausible. However, the paper's headline claims are broader than the evaluated threat model: the physical-layer signal depends on shared memory between spy and victim, and the fallback path via page deduplication is asserted but never demonstrated. The central evaluation and terminology in Section 4.4 also need correction. With scoped claims and a few fixes, this would be a solid and important result.

major comments (4)
  1. [§3.1, §4.1, §7.1] The attack's flush+reload primitive requires the spy and victim to share the physical frames of the embedding table. Section 4.1 obtains this via zero-copy mmap of the model file, with page deduplication only asserted as a fallback. All experiments appear to rely on the mmap path, and Section 7.1 admits that disabling zero-copy loading removes the mmap channel. The page-deduplication fallback is not evaluated anywhere, and on Linux it is not a reliable substitute: KSM merges anonymous pages that are explicitly marked mergeable, not a file-backed page-cache mapping against a victim's private anonymous copy. The Abstract and Section 1 should therefore scope the claim to deployments using zero-copy, file-backed model loading (e.g., llama.cpp's default), or the authors should provide a concrete, evaluated page-deduplication scenario. Without this, the headline 94.8%/82.7% figures do not transfer to configurations that disable zero-copy loading.
  2. [§4.4] The peak/valley analysis of the normalized timing signal swaps the terms false positive and false negative. The text says 'Assuming that we remove the i-th true positive to create a false positive'—removing a true positive creates a false negative (a missing token)—and then says 'most false negatives correlate with the valleys' while explaining that an inserted false positive between two true positives produces a small interval. The operationally correct reading is that gaps (peaks) correspond to missing tokens to be filled, and valleys correspond to inserted tokens to be removed. As written, this is internally inconsistent and makes the preprocessing stage confusing; it must be corrected.
  3. [§6.4, Table 3] Table 3 reports input-recovery cosine similarities of 74.5% for HuggingFace Transformers on CPU and 74.5% for BitNet on GPU, while the text states 'We successfully attacked all 10 frameworks on CPU, 9 of them on GPU.' These values are below the paper's own ASR threshold of φ > 0.77 established in Section 5. The claim of 'successfully attacked' is therefore inconsistent with the paper's success criterion. Either the notion of success for the framework survey must be redefined, or the affected rows should be reported as failures/partial leaks, and the 'all 10 frameworks' claim should be revised.
  4. [Table 1 and Figures 8–10] The central evaluation reports only averages over test sentences, with no error bars, standard deviations, confidence intervals, or numbers of repeated runs for each configuration. Given the noise inherent in cache side-channel traces and the stochasticity of LLM-based reconstruction, the paper should report per-sample variance (e.g., min/max or interquartile range) and ideally at least three repeated attacks per setup. This is needed to assess the reliability of the headline averages, especially the 94.8% and 82.7% Levenshtein-similarity figures.
minor comments (5)
  1. [§5, Metrics] The ASR threshold of φ > 0.77 is calibrated using a human survey over the attack's own outputs; while this is a reasonable subjective calibration, the paper should state clearly that ASR is not an independent measure of exact-text recovery and that the 99%+ ASR figures inherit the survey threshold.
  2. [§6.2, Figure 8] In the discussion of σ sensitivity, the text says 'the output reconstruction performance peaks at s = 0.08' where the variable is σ; the notation should be consistent.
  3. [§6.5, Table 4] The hardware evaluation uses only the 20-sample microbenchmark from Section 6.4; the paper should state this explicitly so readers do not infer that the full test set was used on each CPU.
  4. [§7.1] The performance cost of disabling zero-copy loading (17% loading-latency increase and 32% extra memory overhead) is reported without experimental detail; a citation or a brief methodology note would help.
  5. [§4.1] The claim that 'the malicious code can open the model file in read-only mode and call mmap' should be stated as an assumption in the threat model rather than as an unconditioned capability, since not all deployments expose the model file to the adversary's user.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the attack pipeline is evaluated against external ground truth and the reconstruction models are trained on independent synthetic data.

full rationale

The paper's derivation chain is not circular. The token-value leakage is modeled as the standard secret-dependent embedding-table lookup (Section 3.2: 'Ei = W[ti], i.e., retrieving the ti-th row of W'), and the token-position leakage is the timing of autoregressive decode phases; neither leakage is defined in terms of the reconstructed text. The reconstruction models LLMA and LLMB are fine-tuned on independently synthesized datasets (Algorithm 1 and Section 4.5) and then evaluated on real cache traces against external ground-truth text (Table 1), so the reported Levenshtein, ROUGE, and cosine-similarity figures are not forced by construction. The cache-hit threshold alpha2 is obtained by micro-benchmarks, and the prefetcher countermeasures follow prior published techniques. There are no load-bearing self-citations: the cited works are external (flush+reload [78], SWIPE' [25], the human-survey methodology [70]) and none is an authors' own uniqueness theorem. The only mild self-reference is the ASR threshold of cosine similarity above 0.77 (Section 5), which is calibrated by a human survey over the attack's own outputs and then used to define success; this calibrates the binary success metric rather than determining the reconstructed text, so it does not constitute a circular derivation. Overall, the paper's central claim is self-contained and externally validated.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The attack is an empirical method rather than a formal derivation. The free parameters are attack and training configuration choices; the axioms are realistic deployment conditions that the authors argue are defaults. No new physical entities are introduced.

free parameters (6)
  • alpha_1 (phase boundary threshold) = 10^-3 seconds
    Chosen in Section 4.2; determines which cache-hit intervals count as prefill versus decode events.
  • alpha_2 (cache hit threshold) = from microbenchmark
    Set in Section 4.3 from cache latency microbenchmarks; determines which addresses count as token hits, directly affecting token lists.
  • K (minimum prefill events) = 4
    Set in Section 4.2; controls when the prefill phase is considered to have started.
  • p (synthetic noise probability) = 0.2 (default), tested 0.1 to 0.8
    Algorithm 1 inserts false positives and false negatives with probability p; RQ2 (Figure 8) shows sensitivity and selects a safe range, so the training distribution is tuned to the attack model.
  • sigma (decode period jitter) = 0.08 (default)
    Algorithm 1 uses Gaussian jitter with standard deviation sigma; RQ2 reports output reconstruction peaks at sigma equals 0.08.
  • ASR cosine threshold = 0.77
    Selected in Section 5 from a human survey; defines what counts as a successful attack and is used to compute ASR values in Tables 1, 4, 5, and 6.
assumptions (4)
  • domain assumption Token embedding executes on the CPU in victim frameworks.
    Stated in Section 3.1; when embedding runs on GPU (HuggingFace Transformers in Table 3), the CPU cache attack cannot observe it, so this assumption is load-bearing.
  • domain assumption The model file is mapped into shared memory via mmap or page deduplication.
    Section 4.1 allocates shared memory through zero-copy mmap; Section 7.1 notes that disabling zero-copy is a defense, and page deduplication is only a fallback.
  • domain assumption Autoregressive decode phases are approximately periodic, so power spectral density separates true hits from noise.
    Section 4.4 builds the noise model and synthetic training set on this periodicity; Figure 4 shows the empirical basis.
  • domain assumption Embedding table layout and tokenizer vocabulary are publicly known.
    Section 4.1 computes target addresses from public model file formats and tokenizers; this is needed to map cache hits to tokens.

how reviews work

0 comments
Cite this review

Pith. "Pith review of I Know What You Said: Unveiling Hardware Cache Side-Channels in Local Large Language Model Inference." pith.science (2026). https://pith.science/paper/A5E3JYT5

@misc{pith2026250506738,
  author       = {Pith},
  title        = {Pith review of: I Know What You Said: Unveiling Hardware Cache Side-Channels in Local Large Language Model Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A5E3JYT5}},
  note         = {Machine review of arXiv:2505.06738}
}
read the original abstract

Large Language Models (LLMs) that can be deployed locally have recently gained popularity for privacy-sensitive tasks, with companies such as Meta, Google, and Intel playing significant roles in their development. However, the security of local LLMs through the lens of hardware cache side-channels remains unexplored. In this paper, we unveil novel side-channel vulnerabilities in local LLM inference: token value and token position leakage, which can expose both the victim's input and output text, thereby compromising user privacy. Specifically, we found that adversaries can infer the token values from the cache access patterns of the token embedding operation, and deduce the token positions from the timing of autoregressive decoding phases. To demonstrate the potential of these leaks, we design a novel eavesdropping attack framework targeting both open-source and proprietary LLM inference systems. The attack framework does not directly interact with the victim's LLM and can be executed without privilege. We evaluate the attack on a range of practical local LLM deployments (e.g., Llama, Falcon, and Gemma), and the results show that our attack achieves promising accuracy. The restored output and input text have an average edit distance of 5.2% and 17.3% to the ground truth, respectively. Furthermore, the reconstructed texts achieve average cosine similarity scores of 98.7% (input) and 98.0% (output).

Figures

Figures reproduced from arXiv: 2505.06738 by the authors.

Figure 1
Figure 1. The threat model of our eavesdropping attack. Up [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Workflow of our eavesdropping attack. 1. Execute the spy process that co-locates with the victim and collects the cache trace o during LLM inference. 2. Identify cache trace segments oP and oD that correlate with the prefill and decode phases of the victim. 3. Map the cache trace to the ordered token list KD and the timing signal TD for the decode phase of the victim. Additionally, derive the unordered token list KP… view at source ↗
Figure 3
Figure 3. An example of cache trace. A deeper color indi [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: An example of the pre-processed timing signal. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 4
Figure 4. Figure 4: PSD of the timing signal derived from the cache [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: The relationship between human-evaluated privacy [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: Correlation between attack performance and the [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Comparative evaluation of different base models. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Ablation study using Llama3.1-8B-instruct as the [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Examples of reconstructed prompts. cannot eliminate shared memory created by page duplication in OS [21, 62]. Deploy Role-Based Access Control. A better mitigation is role-based access control (RBAC), which limits memory page sharing within a safe scope. The RBAC guar…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Leaky Language Models: Stealing Architecture and Inference Optimizations via Per-Token Timing

    cs.CR 2026-07 conditional novelty 7.0 of 10

    Per-token generation timing leaks speculative decoding and draft-model context length from Gemini, and recovers layer count and hidden size of Llama-family models with top-5 accuracy up to 65% when both are unknown.

  2. Securing Deep Learning Hardware: A Survey of Side-Channel Vulnerabilities and Countermeasures

    cs.CR 2026-07 accept novelty 3.0 of 10

    Hardware side-channel attacks can recover deep-learning model architecture, parameters and inputs; this survey taxonomizes the leaks, attacks and countermeasures.

Reference graph

Works this paper leans on

84 extracted references · 73 canonical work pages · cited by 2 Pith papers

  1. [1]

    https://huggingface.co/ datasets/WynterJones/chatgpt-roles

    Chatgpt-roles. https://huggingface.co/ datasets/WynterJones/chatgpt-roles

  2. [2]

    https://pytorch.org/blog/ tensor-memory-format-matters/

    Efficient PyTorch: Tensor memory format matters. https://pytorch.org/blog/ tensor-memory-format-matters/

  3. [3]

    https://falconllm.tii.ae/falcon3/ index.html

    Falcon3. https://falconllm.tii.ae/falcon3/ index.html

  4. [4]

    https://ai.google.dev/gemma

    Gemma. https://ai.google.dev/gemma

  5. [5]

    https:// github.com/huggingface/transformers

    HuggingFace Transformers: State-of-the-art machine learning for jax, pytorch and tensorflow. https:// github.com/huggingface/transformers

    work page

  6. [6]

    https://github.com/ intel-analytics/ipex-llm

    Intel IPEX-LLM: Accelerate local llm infer- ence and finetuning. https://github.com/ intel-analytics/ipex-llm

    work page

  7. [7]

    one file

    Koboldcpp: Run gguf models easily with a koboldai ui. one file. zero install. https://github.com/ intel-analytics/ipex-llm

    work page

  8. [8]

    https://www.llama.com/?ref= hacktheprompt

    Llama. https://www.llama.com/?ref= hacktheprompt

    work page

Show all 84 references
  1. [9]

    https://www.llama.com/docs/ llama-everywhere

    Llama everywhere. https://www.llama.com/docs/ llama-everywhere

  2. [10]

    https://github.com/mudler/ LocalAI

    LocalAI: The free, open source alternative to openai, claude and others. https://github.com/mudler/ LocalAI

  3. [11]

    https://github.com/microsoft/BitNet

    Microsoft BitNet: Official inference framework for 1-bit llms. https://github.com/microsoft/BitNet

  4. [12]

    https://ai.azure

    Microsoft phi-3.5 mini. https://ai.azure. com/explore/models/Phi-3.5-mini-instruct/ version/6/registry/azureml

  5. [13]

    https://mistral.ai

    Mistral. https://mistral.ai

  6. [14]

    https://chatgpt.com/

    Openai chatgpt. https://chatgpt.com/

  7. [15]

    https://peps

    PEP 3154 - pickle protocol version 4. https://peps. python.org/pep-3154/

  8. [16]

    https://huggingface.co/docs/ safetensors/index

    Safetensors. https://huggingface.co/docs/ safetensors/index

  9. [17]

    The new era of the ai pc: What does the ai pc mean for users, and how does it work for them? Technical report, Intel, 2023

    2023

  10. [18]

    https://huggingface.co/docs/transformers/perf_infer_cpu, 2024

    Hugging face transformers - cpu inference. https://huggingface.co/docs/transformers/perf_infer_cpu, 2024

    2024

  11. [19]

    https://github.com/ollama/ollama, 2024

    Ollama: Get up and running with large language mod- els locally. https://github.com/ollama/ollama, 2024

    2024

  12. [20]

    nanobench: A low- overhead tool for running microbenchmarks on x86 sys- tems

    Andreas Abel and Jan Reineke. nanobench: A low- overhead tool for running microbenchmarks on x86 sys- tems. In 2020 IEEE International Symposium on Per- formance Analysis of Systems and Software (ISPASS), pages 34–46, 2020

    2020

  13. [21]

    In- creasing memory density by using ksm

    Andrea Arcangeli, Izik Eidus, and Chris Wright. In- creasing memory density by using ksm. In Proceedings of the linux symposium, pages 19–28. Citeseer, 2009

    2009

  14. [22]

    CSI NN: Reverse engineering of neural net- work architectures through electromagnetic side chan- nel

    Lejla Batina, Shivam Bhasin, Dirmanto Jap, and Stjepan Picek. CSI NN: Reverse engineering of neural net- work architectures through electromagnetic side chan- nel. In 28th USENIX Security Symposium (USENIX Security 19), pages 515–532, Santa Clara, CA, August

  15. [23]

    Samsung bans staff’s AI use after spotting ChatGPT data leak

    bloomberg. Samsung bans staff’s AI use after spotting ChatGPT data leak. https://www.bloomberg.com/news/articles/2023- 05-02/samsung-bans-chatgpt-and-other-generative-ai- use-by-staff-after-leak, 2023

    2023

  16. [24]

    Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, and et al

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, and et al. Lan- guage models are few-shot learners. In Proceedings of the 34th International Conference on Neural Informa- tion Processing Systems, NIPS ’20, Red Hook, NY , USA,

  17. [25]

    Arturo Camacho and John G. Harris. A sawtooth waveform inspired pitch estimator for speech and mu- sic. The Journal of the Acoustical Society of America, 124(3):1638–1652, 09 2008

    2008

  18. [26]

    Topic modeling using latent dirichlet allocation: A survey.ACM Comput

    Uttam Chauhan and Apurva Shah. Topic modeling using latent dirichlet allocation: A survey.ACM Comput. Surv., 54(7), September 2021

    2021

  19. [27]

    GoFetch: Breaking Constant-Time cryptographic implementations using data Memory-Dependent prefetchers

    Boru Chen, Yingchen Wang, Pradyumna Shome, Christopher Fletcher, David Kohlbrenner, Riccardo Paccagnella, and Daniel Genkin. GoFetch: Breaking Constant-Time cryptographic implementations using data Memory-Dependent prefetchers. In 33rd USENIX Security Symposium (USENIX Securit...

    2024

  20. [28]

    Palm: Scaling language modeling with pathways

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, and et al. Palm: Scaling language modeling with pathways. J. Mach. Learn. Res., 24:240:1–240:113, 2023

    2023

  21. [29]

    Reconstruct your previous conversations! com- prehensively investigating privacy leakage risks in con- versations with gpt models, 2024

    Junjie Chu, Zeyang Sha, Michael Backes, and Yang Zhang. Reconstruct your previous conversations! com- prehensively investigating privacy leakage risks in con- versations with gpt models, 2024

    2024

  22. [30]

    Aes proposal: Rijndael, 1999

    Joan Daemen. Aes proposal: Rijndael, 1999

    1999

  23. [31]

    Flashattention-2: Faster attention with better parallelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. In The Twelfth Inter- national Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024

    2024

  24. [32]

    Choquette-Choo, Matthew Jagielski, Mi- lad Nasr, Eric Wallace, and Florian Tramèr

    Edoardo Debenedetti, Giorgio Severi, Nicholas Carlini, Christopher A. Choquette-Choo, Matthew Jagielski, Mi- lad Nasr, Eric Wallace, and Florian Tramèr. Privacy side channels in machine learning systems. In 33rd USENIX Security Symposium (USENIX Security 24), pages 6861– 6848,...

    2024

  25. [33]

    The case for 4- bit precision: k-bit inference scaling laws

    Tim Dettmers and Luke Zettlemoyer. The case for 4- bit precision: k-bit inference scaling laws. In Interna- tional Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 7750–7774. PMLR, 2023

    2023

  26. [34]

    BERT: Pre-training of deep bidirec- tional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirec- tional transformers for language understanding. In Pro- ceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguis- tics: Human...

    2019

  27. [35]

    Enhancing chat language models by scaling high-quality instructional conversations

    Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233, 2023

    2023 arXiv

  28. [36]

    Deeptheft: Stealing dnn model ar- chitectures through power side channel

    Yansong Gao, Huming Qiu, Zhi Zhang, Binghui Wang, Hua Ma, Alsharif Abuadbba, Minhui Xue, Anmin Fu, and Surya Nepal. Deeptheft: Stealing dnn model ar- chitectures through power side channel. In 2024 IEEE Symposium on Security and Privacy (SP), pages 3311– 3326, 2024

    2024

  29. [37]

    ggerganov/llama.cpp: Port of Face- book’s LLaMA model in C/C++

    Georgi Gerganov. ggerganov/llama.cpp: Port of Face- book’s LLaMA model in C/C++. https://github. com/ggerganov/llama.cpp, 2024

    2024

  30. [38]

    Side-channel-assisted reverse-engineering of encrypted dnn hardware accelerator ip and attack surface explo- ration

    Cheng Gongye, Yukui Luo, Xiaolin Xu, and Yunsi Fei. Side-channel-assisted reverse-engineering of encrypted dnn hardware accelerator ip and attack surface explo- ration. In 2024 IEEE Symposium on Security and Pri- vacy (SP), pages 4678–4695, 2024

    2024

  31. [39]

    Flush+flush: A fast and stealthy cache attack

    Daniel Gruss, Clémentine Maurice, Klaus Wagner, and Stefan Mangard. Flush+flush: A fast and stealthy cache attack. In Detection of Intrusions and Malware, and Vul- nerability Assessment - 13th International Conference, DIMVA 2016, San Sebastián, Spain, July 7-8, 2016, Pro- cee...

    2016

  32. [40]

    Cache template attacks: Automating attacks on inclusive Last-Level caches

    Daniel Gruss, Raphael Spreitzer, and Stefan Mangard. Cache template attacks: Automating attacks on inclusive Last-Level caches. In 24th USENIX Security Sympo- sium (USENIX Security 15), pages 897–912, Washing- ton, D.C., August 2015. USENIX Association

    2015

  33. [41]

    Large language models are zero-shot time se- ries forecasters

    Nate Gruver, Marc Finzi, Shikai Qiu, and Andrew G Wilson. Large language models are zero-shot time se- ries forecasters. In Advances in Neural Information Processing Systems, volume 36, pages 19622–19635. Curran Associates, Inc., 2023

    2023

  34. [42]

    Mamba: Linear-time sequence modeling with selective state spaces, 2024

    Albert Gu and Tri Dao. Mamba: Linear-time sequence modeling with selective state spaces, 2024

    2024

  35. [43]

    Intel® 64 and ia-32 architectures software developer’s manual

    Part Guide. Intel® 64 and ia-32 architectures software developer’s manual. Volume 3B: System programming Guide, Part, 2(11):0–40, 2011

    2011

  36. [44]

    Cache games – bringing access-based cache attacks on aes to practice

    David Gullasch, Endre Bangerter, and Stephan Krenn. Cache games – bringing access-based cache attacks on aes to practice. In 2011 IEEE Symposium on Security and Privacy, pages 490–505, 2011

    2011

  37. [45]

    Deepsniffer: A dnn model extraction framework based on learning architectural hints

    Xing Hu, Ling Liang, Shuangchen Li, Lei Deng, Pengfei Zuo, Yu Ji, Xinfeng Xie, Yufei Ding, Chang Liu, Timo- thy Sherwood, and Yuan Xie. Deepsniffer: A dnn model extraction framework based on learning architectural hints. In Proceedings of the Twenty-Fifth International Confere...

    2020

  38. [46]

    Edward Suh

    Weizhe Hua, Zhiru Zhang, and G. Edward Suh. Reverse engineering convolutional neural networks through side- channel information leaks. In Proceedings of the 55th Annual Design Automation Conference, DAC ’18, New York, NY , USA, 2018. Association for Computing Ma- chinery

    2018

  39. [47]

    Pleak: Prompt leaking attacks against large language model applications, 2024

    Bo Hui, Haolin Yuan, Neil Gong, Philippe Burlina, and Yinzhi Cao. Pleak: Prompt leaking attacks against large language model applications, 2024

    2024

  40. [48]

    Cross processor cache attacks

    Gorka Irazoqui, Thomas Eisenbarth, and Berk Sunar. Cross processor cache attacks. ASIA CCS ’16, page 353–364, New York, NY , USA, 2016. Association for Computing Machinery

    2016

  41. [49]

    Natural questions: a benchmark for question answering research

    Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Compu...

    2019

  42. [50]

    Efficient memory man- agement for large language model serving with page- dattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory man- agement for large language model serving with page- dattention. In Proceedings of the 29th Symposium on Operating Systems Principle...

    2023

  43. [51]

    AoE: Angle-optimized em- beddings for semantic textual similarity

    Xianming Li and Jing Li. AoE: Angle-optimized em- beddings for semantic textual similarity. In Proceedings of the 62nd Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pages 1825–1839, Bangkok, Thailand, August 2024. Associa- tion for ...

    2024

  44. [52]

    Jamba: A hybrid transformer-mamba language model, 2024

    Opher Lieber, Barak Lenz, Hofit Bata, Gal Cohen, Jhonathan Osin, Itay Dalmedigos, and et al. Jamba: A hybrid transformer-mamba language model, 2024

    2024

  45. [53]

    Side channel-assisted inference attacks on machine learning-based ecg classification

    Jialin Liu, Houman Homayoun, Chongzhou Fang, Ning Miao, and Han Wang. Side channel-assisted inference attacks on machine learning-based ecg classification. In 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD), pages 1–9, 2023

    2023

  46. [54]

    LM Studio: Discover, download, and run local llms

    Lms. LM Studio: Discover, download, and run local llms. https://lmstudio.ai/, 2024

    2024

  47. [55]

    Llm- pruner: On the structural pruning of large language mod- els

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm- pruner: On the structural pruning of large language mod- els. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December ...

    2023

  48. [56]

    Remote power side-channel attacks on bnn accelerators in fpgas

    Shayan Moini, Shanquan Tian, Daniel Holcomb, Jakub Szefer, and Russell Tessier. Remote power side-channel attacks on bnn accelerators in fpgas. In 2021 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1639–1644, 2021

    2021

  49. [57]

    GPT4All: A free-to-use, locally running, privacy-aware chatbot

    Nomic. GPT4All: A free-to-use, locally running, privacy-aware chatbot. no GPU or internet required. https://gpt4all.io/index.html?ref=localhost, 2024

    2024

  50. [58]

    Learn about language model tokenization

    OpenAI. Learn about language model tokenization. https://platform.openai.com/tokenizer

  51. [59]

    Shamir, and Eran Tromer

    Dag Arne Osvik, A. Shamir, and Eran Tromer. Cache attacks and countermeasures: The case of aes. The Cryptographer’s Track at RSA Conference, pages 1–20,

  52. [60]

    Know what you don’t know: Unanswerable questions for SQuAD

    Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Vol- ume 2: Short Papers), Melbourne, Australia, July 2018. Association for Com...

    2018

  53. [61]

    Social IQa: Commonsense reasoning about social interactions

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. Social IQa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural ...

    2019

  54. [62]

    Singleton: system-wide page deduplication in virtual environments

    Prateek Sharma and Purushottam Kulkarni. Singleton: system-wide page deduplication in virtual environments. In Proceedings of the 21st International Symposium on High-Performance Parallel and Distributed Computing, HPDC ’12, page 15–26, New York, NY , USA, 2012. Association fo...

    2012

  55. [63]

    whisper- ing mlaas

    Shubhi Shukla, Manaar Alam, Sarani Bhattacharya, Pabitra Mitra, and Debdeep Mukhopadhyay. "whisper- ing mlaas" exploiting timing channels to compromise user privacy in deep neural networks.IACR Trans. Cryp- togr. Hardw. Embed. Syst., 2023(2):587–613, 2023

    2023

  56. [64]

    Robust website fingerprinting through the cache occupancy channel

    Anatoly Shusterman, Lachlan Kang, Yarden Haskal, Yosef Meltser, Prateek Mittal, Yossi Oren, and Yuval Yarom. Robust website fingerprinting through the cache occupancy channel. In 28th USENIX Security Sympo- sium (USENIX Security 19), pages 639–656, Santa Clara, CA, August 2019...

    2019

  57. [65]

    The early bird catches the leak: Unveiling timing side channels in llm serving systems, 2024

    Linke Song, Zixuan Pang, Wenhao Wang, Zihao Wang, XiaoFeng Wang, Hongbo Chen, Wei Song, Yier Jin, Dan Meng, and Rui Hou. The early bird catches the leak: Unveiling timing side channels in llm serving systems, 2024

    2024

  58. [66]

    PowerInfer: Fast large language model serving with a consumer-grade GPU

    Yixin Song, Zeyu Mi, Haotong Xie, and Haibo Chen. PowerInfer: Fast large language model serving with a consumer-grade GPU. CoRR, abs/2312.12456, 2023

    2023 arXiv

  59. [67]

    Supercharged ai inference on modern cpus

    Lawrence Spracklen and Subutai Ahmad. Supercharged ai inference on modern cpus. In 2023 IEEE Hot Chips 35 Symposium (HCS), pages 1–21, 2023

    2023

  60. [68]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red H...

    2017

  61. [69]

    Stealthy inference attack on dnn via cache-based side- channel attacks

    Han Wang, Syed Mahbub Hafiz, Kartik Patwari, Chen- Nee Chuah, Zubair Shafiq, and Houman Homayoun. Stealthy inference attack on dnn via cache-based side- channel attacks. In 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1515– 1520, 2022

    2022

  62. [70]

    What was your prompt? a remote keylogging attack on AI assistants

    Roy Weiss, Daniel Ayzenshteyn, and Yisroel Mirsky. What was your prompt? a remote keylogging attack on AI assistants. In 33rd USENIX Security Symposium (USENIX Security 24), pages 3367–3384, Philadelphia, PA, August 2024. USENIX Association

    2024

  63. [71]

    Optimizing and running LLaMA2 on Intel CPU

    Lim Xiang Yang, Lim Chen Han, and Foong Chun Sheong. Optimizing and running LLaMA2 on Intel CPU. Technical report, Intel Inc., 2023

    2023

  64. [72]

    Fast on-device llm inference with npus

    Daliang Xu, Hao Zhang, Liming Yang, Ruiqi Liu, Gang Huang, Mengwei Xu, and Xuanzhe Liu. Fast on-device llm inference with npus. In Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1, ASPLOS ’2...

    2025

  65. [73]

    Hao Xue and Flora D. Salim. Promptcast: A new prompt-based learning paradigm for time series fore- casting. IEEE Transactions on Knowledge and Data Engineering, 36(11):6851–6864, 2024

    2024

  66. [74]

    Fletcher, and Josep Torrel- las

    Mengjia Yan, Christopher W. Fletcher, and Josep Torrel- las. Cache telepathy: Leveraging shared resource attacks to learn DNN architectures. In 29th USENIX Security Symposium (USENIX Security 20) , pages 2003–2020. USENIX Association, August 2020

    2003

  67. [75]

    Prsa: Prompt stealing attacks against large language models, 2024

    Yong Yang, Changjiang Li, Yi Jiang, Xi Chen, Haoyu Wang, Xuhong Zhang, Zonghui Wang, and Shouling Ji. Prsa: Prompt stealing attacks against large language models, 2024

    2024

  68. [76]

    Le.XLNet: gener- alized autoregressive pretraining for language under- standing

    Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V . Le.XLNet: gener- alized autoregressive pretraining for language under- standing. Curran Associates Inc., Red Hook, NY , USA, 2019

    2019

  69. [77]

    Mastik: A micro-architectural sidechan- nel toolkit

    Yuval Yarom. Mastik: A micro-architectural sidechan- nel toolkit. https://cs.adelaide.edu.au/~yval/ Mastik/

  70. [78]

    FLUSH+RELOAD: A high resolution, low noise, L3 cache side-channel attack

    Yuval Yarom and Katrina Falkner. FLUSH+RELOAD: A high resolution, low noise, L3 cache side-channel attack. In Proceedings of the 23rd USENIX Security Symposium, San Diego, CA, USA, August 20-22, 2014, pages 719–732. USENIX Association, 2014

    2014

  71. [79]

    WiP: An On-device LLM-based approach to query pri- vacy protection

    Yizhen Yuan, Rui Kong, Yuanchun Li, and Yunxin Liu. WiP: An On-device LLM-based approach to query pri- vacy protection. In Proceedings of the Workshop on Edge and Mobile Foundation Models, EdgeFM 2024, Minato-ku, Tokyo, Japan, June 3-7, 2024 , pages 7–9. ACM, 2024

    2024

  72. [80]

    Fine-tuning of financial large language model and application at edge device

    Juntao Zeng, Bo Chen, Yuandan Deng, Weiqin Chen, Yanlin Mao, and Jiawei Li. Fine-tuning of financial large language model and application at edge device. In Proceedings of the 3rd International Conference on Computer, Artificial Intelligence and Control Engineer- ing, CAICE ’2...

    2024

  73. [81]

    Enabling on-device llms personalization with smartphone sensing

    Shiquan Zhang, Ying Ma, Le Fang, Hong Jia, Simon D’Alfonso, and Vassilis Kostakos. Enabling on-device llms personalization with smartphone sensing. In Com- panion of the 2024 on ACM International Joint Con- ference on Pervasive and Ubiquitous Computing, Ubi- Comp ’24, page 186...

    2024

  74. [82]

    Invalidate+Compare: A Timer-Free GPU cache attack primitive

    Zhenkai Zhang, Kunbei Cai, Yanan Guo, Fan Yao, and Xing Gao. Invalidate+Compare: A Timer-Free GPU cache attack primitive. In 33rd USENIX Security Sympo- sium (USENIX Security 24), pages 2101–2118, Philadel- phia, PA, August 2024. USENIX Association

    2024

  75. [83]

    Inputsnatch: Stealing input in llm services via timing side-channel attacks, 2024

    Xinyao Zheng, Husheng Han, Shangyi Shi, Qiyan Fang, Zidong Du, Xing Hu, and Qi Guo. Inputsnatch: Stealing input in llm services via timing side-channel attacks, 2024. A Dataset Construction Process We used all the 2,727 testing prompts of UltraChat, and 4,425 samples in the NQ...

    2024

  76. [2020]

    Curran Associates Inc

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.