{"id":"48931d1b-51cc-4e5b-9324-319e371910a4","arxiv_id":"2505.06738","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":8.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Local LLM inference leaks input and output text to an unprivileged co-resident process through cache access patterns in token embedding and timing of autoregressive decoding.","lead":"A co-located spy program can recover what a user types into a locally running chatbot and what the chatbot replies by watching shared CPU cache activity. This undermines the privacy promise of local LLMs and affects popular frameworks such as llama.cpp and Ollama.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Attack hinges on zero-copy mmap; the page-dedup fallback is unvalidated and likely infeasible, so the abstract's scope overstates the finding.","rationale":"Good-faith reading: the paper's strongest claim is that a hardware cache side channel leaks token values and token positions during local LLM inference, enabling text reconstruction. The authors provide a reproducible artifact, a plausible physical mechanism (embedding-table rows are secret-dependent memory accesses), and ablations showing that the SCA tokens and timing signal contribute beyond the LLM prior (Table 2 and Table 7: combined input reconstruction 76.2% LS vs 55.2% output-only and 29.8% pure SCA). Those pieces of evidence support the conclusion that, when shared memory is present, the side channel carries real signal. The load-bearing condition is shared physical memory. Section 4.1's two strategies are not symmetric: mmap of the model file is the strategy actually exercised by the default frameworks evaluated, while page deduplication is asserted but never demonstrated. Because Section 7.1 itself identifies disabling zero-copy loading as an effective countermeasure, the central claim's scope is narrower than the abstract suggests. In a deployment that copies the model into private anonymous memory and does not enable KSM merging on those pages, the spy has no flush+reload observable at all; the attack has no physical-layer input. This is not an internal inconsistency, but it is a deployment-level dependency that the paper should state as a scope condition and verify experimentally. The reader's CONDITIONAL verdict is therefore the right level: the mechanism appears real for zero-copy file-backed frameworks, but the headline generality is not established. I would keep the verdict unchanged rather than move it, because the concern is addressable by scoping language and a countermeasure test, not by a demonstrated refutation of the leakage mechanism.","tokens_in":26628,"tokens_out":10379,"duration_ms":116311,"concrete_test":"Rebuild llama.cpp with mmap disabled (e.g., LLAMA_NO_MMAP=1 or equivalent), leave KSM disabled, and rerun the full Table 1 pipeline on the same prompts and the same 13900K machine for at least one victim model (e.g., Llama-3.1-8B). If the flush+reload trace no longer correlates with victim embedding accesses and the reported LS/φ collapse toward the no-SCA baselines in Table 7, the central claim must be restated as applying only to zero-copy file-backed frameworks. Separately, exercise the claimed page-deduplication path explicitly: load the victim model into anonymous memory, enable KSM, and mark the victim pages MADV_MERGEABLE, then check whether the spy's mmap of the same file actually observes the same physical frames.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the Abstract and Section 1 is that an unprivileged co-located process can reconstruct the victim's input and output text from hardware cache access patterns. The physical-layer signal for that claim is entirely contingent on the spy and victim sharing the physical frames of the embedding table. Section 4.1 obtains this by mmap'ing the model file (zero-copy loading) or, failing that, by OS page deduplication. Section 7.1 admits that disabling zero-copy loading eliminates the mmap-based channel. The page-deduplication fallback is not evaluated anywhere in the paper, and on Linux it is not a reliable substitute: KSM merges only anonymous pages explicitly marked MADV_MERGEABLE by the victim; it does not merge file-backed page-cache pages with a victim's private anonymous copy. A framework that reads the model into private anonymous memory and never marks it mergeable therefore gives the spy no shared physical frames, no flush+reload primitive, and no token-value leakage. The attack is thus not a vulnerability of 'local LLM inference' generally, but of zero-copy, file-backed deployments. All reported experiments appear to use such deployments, so this is a load-bearing scoping condition: the headline 94.8%/82.7% Levenshtein-similarity figures do not transfer to configurations running the authors' own recommended countermeasure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the first hardware cache side-channel attack against local LLM inference, claiming to reconstruct both the victim's prompt and the model's response. The attack exploits cache access patterns of the token embedding table to infer token values and the timing of autoregressive decode phases to infer token positions. The authors implement flush+reload over a shared model-file mapping, map cache hits to tokens, and fine-tune two LLMs (LLMA and LLMB) on synthetically generated noisy traces to reconstruct output and input text. They report high accuracy across several LLMs, frameworks, and machines, with average Levenshtein similarity of 94.8% for output and 82.7% for input against llama.cpp, and cosine similarity near 98%. The paper also includes ablations, a framework survey, a hardware survey, countermeasure discussion, and an open-source artifact.","tokens_in":26879,"tokens_out":7380,"duration_ms":71273,"significance":"If the claims are supported, this is a significant contribution: it extends hardware side-channel attacks from discriminative DNNs to full-text extraction from generative LLM inference, and it does so without direct interaction with the victim model. The paper is commendable for shipping an artifact, running end-to-end attacks against real deployments such as llama.cpp and Ollama, and providing ablations that separate the contributions of the timing signal, the reconstructed output context, and the SCA data. The core mechanism—observing secret-dependent accesses to the embedding table—is plausible. However, the paper's headline claims are broader than the evaluated threat model: the physical-layer signal depends on shared memory between spy and victim, and the fallback path via page deduplication is asserted but never demonstrated. The central evaluation and terminology in Section 4.4 also need correction. With scoped claims and a few fixes, this would be a solid and important result.","major_comments":[{"comment":"The attack's flush+reload primitive requires the spy and victim to share the physical frames of the embedding table. Section 4.1 obtains this via zero-copy mmap of the model file, with page deduplication only asserted as a fallback. All experiments appear to rely on the mmap path, and Section 7.1 admits that disabling zero-copy loading removes the mmap channel. The page-deduplication fallback is not evaluated anywhere, and on Linux it is not a reliable substitute: KSM merges anonymous pages that are explicitly marked mergeable, not a file-backed page-cache mapping against a victim's private anonymous copy. The Abstract and Section 1 should therefore scope the claim to deployments using zero-copy, file-backed model loading (e.g., llama.cpp's default), or the authors should provide a concrete, evaluated page-deduplication scenario. Without this, the headline 94.8%/82.7% figures do not transfer to configurations that disable zero-copy loading.","section":"§3.1, §4.1, §7.1"},{"comment":"The peak/valley analysis of the normalized timing signal swaps the terms false positive and false negative. The text says 'Assuming that we remove the i-th true positive to create a false positive'—removing a true positive creates a false negative (a missing token)—and then says 'most false negatives correlate with the valleys' while explaining that an inserted false positive between two true positives produces a small interval. The operationally correct reading is that gaps (peaks) correspond to missing tokens to be filled, and valleys correspond to inserted tokens to be removed. As written, this is internally inconsistent and makes the preprocessing stage confusing; it must be corrected.","section":"§4.4"},{"comment":"Table 3 reports input-recovery cosine similarities of 74.5% for HuggingFace Transformers on CPU and 74.5% for BitNet on GPU, while the text states 'We successfully attacked all 10 frameworks on CPU, 9 of them on GPU.' These values are below the paper's own ASR threshold of φ > 0.77 established in Section 5. The claim of 'successfully attacked' is therefore inconsistent with the paper's success criterion. Either the notion of success for the framework survey must be redefined, or the affected rows should be reported as failures/partial leaks, and the 'all 10 frameworks' claim should be revised.","section":"§6.4, Table 3"},{"comment":"The central evaluation reports only averages over test sentences, with no error bars, standard deviations, confidence intervals, or numbers of repeated runs for each configuration. Given the noise inherent in cache side-channel traces and the stochasticity of LLM-based reconstruction, the paper should report per-sample variance (e.g., min/max or interquartile range) and ideally at least three repeated attacks per setup. This is needed to assess the reliability of the headline averages, especially the 94.8% and 82.7% Levenshtein-similarity figures.","section":"Table 1 and Figures 8–10"}],"minor_comments":[{"comment":"The ASR threshold of φ > 0.77 is calibrated using a human survey over the attack's own outputs; while this is a reasonable subjective calibration, the paper should state clearly that ASR is not an independent measure of exact-text recovery and that the 99%+ ASR figures inherit the survey threshold.","section":"§5, Metrics"},{"comment":"In the discussion of σ sensitivity, the text says 'the output reconstruction performance peaks at s = 0.08' where the variable is σ; the notation should be consistent.","section":"§6.2, Figure 8"},{"comment":"The hardware evaluation uses only the 20-sample microbenchmark from Section 6.4; the paper should state this explicitly so readers do not infer that the full test set was used on each CPU.","section":"§6.5, Table 4"},{"comment":"The performance cost of disabling zero-copy loading (17% loading-latency increase and 32% extra memory overhead) is reported without experimental detail; a citation or a brief methodology note would help.","section":"§7.1"},{"comment":"The claim that 'the malicious code can open the model file in read-only mode and call mmap' should be stated as an assumption in the threat model rather than as an unconditioned capability, since not all deployments expose the model file to the adversary's user.","section":"§4.1"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely within scope for a security venue, and the artifact release is a strong point. The main editorial concern is that the paper's broad 'local LLM inference' framing is not matched by the evaluated condition (zero-copy mmap); this is fixable by scoping the claims. The use of OpenAI's GPT-4o-mini as a fine-tuning base is a reproducibility concern, but the authors also provide an open Llama-3.1-8B variant and release the training data, so I do not consider it a blocker."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know upfront. First, this paper is genuinely new: it is the first to show that token embedding lookups observed with flush+reload leak token values, and that the periodicity of autoregressive decode phases leaks token position. The end-to-end attack on llama.cpp is substantial, the prefetcher evasion on Raptor Lake is a real piece of engineering, and the artifact release on Zenodo is the right way to do this kind of empirical work. The ablations also earn their place: the timing signal and the fine-tuned reconstruction models both matter, and the pure-SCA baseline is far weaker than the full attack.\n\nSecond, the scope is narrower than the abstract claims. The entire attack depends on the spy and victim sharing the physical frames of the embedding table. Section 4.1 gets this by having the spy mmap the model file and relying on zero-copy loading in the victim. Section 7.1 correctly says that disabling zero-copy removes that channel, but the abstract and introduction still say local LLM inference in general. The page-deduplication fallback is asserted, never evaluated, and on Linux it is not an easy substitute: KSM only merges anonymous pages that the victim explicitly marks MADV_MERGEABLE, and it does not merge a file-backed page-cache page with a private anonymous copy. So as written, the result applies to zero-copy, file-backed deployments, which are common today but are not all local LLM inference. That is a fixable framing problem, not a refutation of the core mechanism.\n\nThe other soft spots are more minor. The headline numbers in Table 1 and Figures 8 through 10 are averages without error bars or run counts, which makes it hard to judge variance. Table 3's multi-framework claim rests on 20 samples per framework, which is thin but acceptable as a microbenchmark. Section 4.4's noise-model text swaps the roles of false positives and false negatives in at least one place: removing a true positive creates a missing token, not an inserted one. The method appears to work regardless, but the terminology is confusing and should be corrected. The ASR threshold of cosine 0.77 is chosen from a human survey over the attack's own outputs, which is mildly self-referential, though the reported scores are high enough that the conclusion would not change.\n\nWho gets value from this: security researchers working on side channels or LLM inference systems, and framework developers who should read the countermeasure discussion before the next local-LLM release. I would send this to peer review, not desk reject. The novelty is real, the empirical work is honest, and the weaknesses are addressable with clearer scoping, error bars, and a fixed noise-model paragraph.","headline":"First credible demonstration that a co-resident unprivileged process can recover both sides of a local LLM conversation via cache side channels, but the claim needs to be scoped to zero-copy, file-backed deployments.","tokens_in":27469,"tokens_out":3210,"would_cite":true,"duration_ms":30932,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A spy process can reconstruct both the prompt and the reply of a local LLM session from cache timing alone.","keywords":["cache side-channel","flush+reload","local LLM inference","token embedding","autoregressive timing","prompt privacy","text reconstruction","LLM security"],"falsifier":"Run the identical cache-trace capture against a victim deployment that disables zero-copy model loading and has kernel page deduplication turned off: if no periodic embedding-row hit pattern with a clear PSD peak appears during decode phases, the claimed side channel does not exist for that configuration; a complementary check would be to store the embedding table under a secret random row permutation and see whether the recovered token indices still correlate with the true tokens.","tokens_in":26394,"feed_emoji":"🕵️","tokens_out":8206,"duration_ms":78985,"temperature":0.7,"pith_summary":"The paper sets out to show that locally deployed LLM inference leaks both the user's prompt and the model's response through a hardware cache side channel, with no special privilege and no interaction with the victim model. The claimed leakage has two sources: the token-embedding lookup table reveals which tokens are processed, and the periodic timing of autoregressive decode phases reveals the order of output tokens. To turn this leak into text, the paper builds a two-stage reconstruction pipeline: a fine-tuned LLM cleans the noisy token list and timing waveform into the model output, and a second fine-tuned LLM uses that output as context to reorder the scrambled bag of input tokens. The reported numbers are specific: average Levenshtein similarity of 94.8% for restored output and 82.7% for restored input, with cosine similarities of 98.7% and 98.0% respectively. If these results hold, the privacy promise of local inference is broken by a co-resident spy.","feed_headline":"Cache side-channel exposes local LLM prompts and replies","feed_subtitle":"By watching embedding-table cache hits, an unprivileged process can reconstruct the user's prompt and the model's output.","key_machinery":"The load-bearing object is the token embedding table W: with one-hot input, the embedding step is E_i = W[token_i], so each token fetches exactly one row of W into cache, turning vocabulary membership into a reproducible address trace. The attack probes the vocabulary-sized set of addresses inside W with the flush+reload technique and treats a cache hit on row j as evidence that token j was processed. The second mechanism is the timing signal: decode phases are modeled as a Dirac impulse train that is approximately periodic, and the paper uses the power spectral density and the SWIPE' pitch estimator to extract the fundamental period, then normalizes the first-order differences of hit timestamps to make the signal hardware-invariant. These two signals are consumed by two fine-tuned LLMs that act as learned denoisers and reorderers, so the framework's text-recovery power comes from combining a microarchitectural leak with statistical language-model priors.","core_discovery":"The central discovery is that two ordinary properties of LLM inference form a side channel. First, because token embedding is a lookup of one row per token from a large embedding table, every token a model reads or writes is reflected in which cache lines of that table were fetched. Second, because autoregressive decoding is strictly serial, each generated token lands at a roughly periodic timestamp, and the paper observes this periodicity in the power spectral density of cache-hit events. On that basis, the paper argues that a spy who shares the model file's memory pages with the victim can run flush+reload to harvest a token list, separate the prefill phase from the decode phases by hit density, and use the timing signal to distinguish genuine tokens from false positives and missing tokens. The paper then claims that fine-tuned LLMs can complete the reconstruction: one fuses the token list with the normalized timing waveform to recover the output text, and another restores the original order of the shuffled input tokens using the recovered output as context. The evaluations across multiple model families, inference frameworks, CPUs, and operating systems are presented as evidence that this eavesdropping path is practical, not just theoretical.","pith_inferences":["A testable extension would be to randomize or permute the row-to-address mapping of the embedding table; if the spy's address-to-token dictionary is broken, the token-value leak is blocked without changing the model, and the paper does not evaluate this countermeasure.","The attack's dependence on strict periodicity suggests that speculative decoding, variable batch sizes, or CPU frequency scaling that jitters the per-token interval would degrade the timing channel; measuring how much timing noise is needed to hide the PSD peak would bound the attack's applicability.","Since tokenizers are publicly documented, the same address-dictionary method could be applied to future model formats that store the embedding table in column-major or quantized layouts, provided the row recovery is adjusted accordingly.","The reconstruction LLMs themselves are reusable: the same 'noisy token list plus waveform to text' formulation could denoise other shared-memory side channels, such as page-table or prefetcher-based leaks, without retraining from scratch."],"forward_implications":["Process isolation alone no longer protects local LLM sessions: an unprivileged co-located application can extract the text a user types and the model writes back.","Because the vulnerability stems from the standard embedding-lookup design and the autoregressive loop, it applies to a broad class of local deployments, not to a single bug in one framework.","Output text is recoverable near-verbatim, with average Levenshtein similarity of 94.8%, so side-channel eavesdropping is a realistic route to extracting names, addresses, and other personally identifiable information from local chat sessions.","Input recovery preserves semantics even for long prompts, with cosine similarity above 92% in the worst reported case, though character-level fidelity drops as input length grows, consistent with a factorial reordering search space.","The main countermeasures have real costs: disabling zero-copy loading costs loading latency and extra memory, and hardware cache partitioning is usually unavailable on consumer CPUs."],"supporting_citations":[{"why":"Supplies the flush+reload measurement primitive used to probe the embedding table's cache lines.","marker":"[78]"},{"why":"Establishes a flush+flush alternative, showing the leak is not tied to one cache-attack implementation.","marker":"[39]"},{"why":"Provides the evict+reload and cache-template techniques used to choose target addresses and avoid prefetcher interference.","marker":"[40]"},{"why":"Documents the data memory-dependent prefetcher behavior that forces the attack to avoid storing valid pointers in its probe array.","marker":"[27]"},{"why":"Supplies the SWIPE' pitch estimator used to extract the fundamental decode-period frequency from the timing signal.","marker":"[25]"},{"why":"Provides the kernel page-deduplication mechanism used as an alternative route to shared memory when zero-copy loading is unavailable.","marker":"[21]"},{"why":"The open-source C/C++ inference engine used as the primary victim in the performance evaluations.","marker":"[37]"},{"why":"A prior remote keylogging attack on AI assistants that motivates reconstructing text from indirect signals and supplies the contextual-dependence idea for reordering.","marker":"[70]"}],"fun_headline_variants":["Cache spy recovers LLM prompts and outputs","LLM token values and positions leak via cache","Local LLM cache side-channel exposes user prompts","Unprivileged cache attack steals LLM replies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack requires the spy and victim to share the physical memory of the embedding table, normally via zero-copy file mapping or OS page deduplication; if both are disabled, the flush+reload probe sees nothing and the whole recovery chain collapses.","fun_headline_variants_meta":{"raw":{"variants":["Cache spy recovers LLM prompts and outputs","LLM token values and positions leak via cache","Local LLM cache side-channel exposes user prompts","Unprivileged cache attack steals LLM replies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000629,"raw_usage":{"total_tokens":2952,"prompt_tokens":1033,"completion_tokens":1919,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":1860}},"tokens_in":649,"tokens_out":1919,"duration_ms":15689,"temperature":1.0,"reasoning_tokens":1860,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:34:34.688390+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical cache-trace capture against a victim deployment that disables zero-copy model loading and has kernel page deduplication turned off: if no periodic embedding-row hit pattern with a clear PSD peak appears during decode phases, the claimed side channel does not exist for that configuration; a complementary check would be to store the embedding table under a secret random row permutation and see whether the recovered token indices still correlate with the true tokens.","supporting_citations":[{"cited_title":"FLUSH+RELOAD: A high resolution, low noise, L3 cache side-channel attack","cited_arxiv_id":null,"evidence_quote":"Supplies the flush+reload measurement primitive used to probe the embedding table's cache lines."},{"cited_title":"Flush+flush: A fast and stealthy cache attack","cited_arxiv_id":null,"evidence_quote":"Establishes a flush+flush alternative, showing the leak is not tied to one cache-attack implementation."},{"cited_title":"Cache template attacks: Automating attacks on inclusive Last-Level caches","cited_arxiv_id":null,"evidence_quote":"Provides the evict+reload and cache-template techniques used to choose target addresses and avoid prefetcher interference."},{"cited_title":"GoFetch: Breaking Constant-Time cryptographic implementations using data Memory-Dependent prefetchers","cited_arxiv_id":null,"evidence_quote":"Documents the data memory-dependent prefetcher behavior that forces the attack to avoid storing valid pointers in its probe array."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the SWIPE' pitch estimator used to extract the fundamental decode-period frequency from the timing signal."},{"cited_title":"In- creasing memory density by using ksm","cited_arxiv_id":null,"evidence_quote":"Provides the kernel page-deduplication mechanism used as an alternative route to shared memory when zero-copy loading is unavailable."},{"cited_title":"ggerganov/llama.cpp: Port of Face- book’s LLaMA model in C/C++","cited_arxiv_id":null,"evidence_quote":"The open-source C/C++ inference engine used as the primary victim in the performance evaluations."},{"cited_title":"What was your prompt? a remote keylogging attack on AI assistants","cited_arxiv_id":null,"evidence_quote":"A prior remote keylogging attack on AI assistants that motivates reconstructing text from indirect signals and supplies the contextual-dependence idea for reordering."}],"review_version":1}