{"id":"c7c606aa-9e8f-4857-8804-accae99e165f","arxiv_id":"2501.05965","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"RevertLM reconstructs private text from intermediate LLM representations in split learning using a projection into embedding space plus a generative decoder, outperforming prior embedding-only attacks.","lead":"Researchers show that text can be recovered from the intermediate neural signals a phone sends to a cloud server when a large language model is split between the two. They build a two-stage attack and report large gains over earlier methods, arguing that these hidden signals need the same privacy protection as raw text.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The attack's empirical success depends on the attacker having a large same-distribution auxiliary corpus; Section 4.1.1 trains on the same dataset's training split, so the 38%-75% claim is not established for personalized or out-of-distribution private text.","rationale":"The reader's weakest_assumption already identified the auxiliary-dataset distribution and the known server-side structure as load-bearing, and I agree with that identification. My stress-test pass searched for a more internal flaw—such as a mismatch between the information-bottleneck analysis and the attack architecture, or an unfair comparison to GEIA/Vec2Text—but the most direct threat to the central claim remains the same: the experiments train the attacker on the training split of the very dataset used for evaluation, which is equivalent to assuming the attacker has access to a large, same-distribution corpus. Since that assumption is precisely what fails for personalized data, the 38%-75% recovery rates are not yet shown to transfer to the paper's motivating scenario. The proposed concrete test (cross-domain or leave-one-persona-out evaluation) would settle this empirically. If RevertLM maintains high recovery under distribution shift, the concern is resolved and the conditional verdict can be upgraded; if it does not, the privacy-risk claim must be scoped to the same-distribution auxiliary-data case. I therefore recommend no change to the reader's CONDITIONAL verdict, because the identified concern is the same one the reader flagged and the requested evidence is already implicit in the conditional recommendation.","tokens_in":11036,"tokens_out":3551,"duration_ms":38113,"concrete_test":"Run a distribution-shift evaluation: train the full RevertLM pipeline on a public auxiliary corpus (e.g., general dialogue or Wikipedia text) and attack PersonaChat test sentences from personas that are excluded from the auxiliary set. Report ROUGE-L, BLEU, and cosine similarity. If scores collapse relative to Table 5 (ROUGE-L 0.5630), the reported attack rates are conditional on same-distribution auxiliary data and the strong privacy-risk claim must be weakened. A complementary check is a leave-one-persona-out split: train on all personas except one and attack the held-out persona's text.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central privacy-risk claim is that RevertLM can recover private user text from intermediate representations in split learning for personalized LLMs. The least secure condition for that claim is stated in Section 3.1: the attacker builds the external model Phi using an auxiliary dataset D_aux 'with a distribution similar to the training data.' Section 4.1.1 operationalizes this by saying 'we utilize the training data of each dataset as auxiliary data to train the attacker model and report its performance on the respective test sets.' Thus every reported attack number (Tables 5 and 6, the 38%-75% range, and the 'over 60% improvement over SOTA') is produced under the most favorable possible auxiliary-data assumption: the attacker has a large sample from the exact same distribution as the victim's private text, including the same personas in PersonaChat. In the claimed deployment scenario, personalized edge-cloud LLMs, the private data is precisely what is idiosyncratic and not publicly available. An attacker who lacks a similar-distribution corpus cannot train the purification and decoder modules as described, so the headline recovery rates are an in-distribution upper bound rather than a demonstrated privacy risk for genuinely personalized data. This is not an internal inconsistency, but it makes the generality of the central claim the most load-bearing weakness.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes RevertLM, a two-stage model inversion attack against intermediate representations (hidden states) transmitted in split learning for LLMs. The first stage maps sparse intermediate representations into the embedding space using a linear projection or autoencoder ('information purification'), and the second stage uses a GPT2-XL decoder to generate text from the purified representation. The authors motivate the design with an information-bottleneck-style analysis of mutual information across transformer blocks. They evaluate RevertLM against GEIA and Vec2Text on PersonaChat and Wiki datasets with a T5 victim model, reporting ROUGE/BLEU/cosine improvements and claiming an over-60% improvement over the state of the art.","tokens_in":11287,"tokens_out":2944,"duration_ms":30776,"significance":"If fully substantiated, the paper would provide the first systematic demonstration that intermediate LLM representations in split learning are invertible, not just embeddings, and would quantify how inversion risk varies across transformer blocks. The head-to-head comparison against GEIA and Vec2Text on two datasets is the right experimental design, and the reported gains on ROUGE and BLEU are substantial. The mutual-information analysis is a useful addition, though it is used mainly as motivation rather than as a predictive model. However, the lack of code, variance reporting, and a clear evaluation under distribution shift leaves the strength and generality of the central privacy claim only partially established.","major_comments":[{"comment":"The attacker model Φ is trained on an auxiliary dataset D_aux 'with a distribution similar to the training data' (§3.1), and §4.1.1 operationalizes this by using the training split of each dataset as auxiliary data and reporting performance on the same dataset's test split. This is an in-distribution evaluation: the attacker has a large sample from the exact distribution of the victim's private text, including the same PersonaChat personas. In the claimed personalized edge-cloud deployment, private text is precisely what is idiosyncratic and not publicly available, so an attacker without a similar-distribution corpus cannot train the purification and decoder modules as described. The reported 38%-75% recovery rates and the over-60% improvement over SOTA are therefore in-distribution upper bounds, not demonstrated privacy risks for genuinely personalized or out-of-distribution text. The authors should either add experiments with distribution shift (e.g., train on one set of users/personas and attack another, or train on a public corpus and attack private text) or explicitly re-scope the central claim.","section":"§3.1 and §4.1.1"},{"comment":"The manuscript contains unresolved table references: the text in §4.2 refers to 'Table ??' and §4.4 refers to 'Table??', while captions are present. More substantively, Table 3 does not state which metric is being reported (presumably ROUGE-L, but this must be explicit), and Table 4 does not specify the dataset, victim model, split point, or training details for the purification variants. The unexplained variant 'Linear projection with tester' also needs a definition. These omissions prevent the reader from reproducing or interpreting the head-to-head comparison.","section":"§4.2 and §4.4, Tables 3 and 4"},{"comment":"Step 1 says the information purification module is pretrained 'with an auxiliary dataset of embeddings for the victim model with negligible size,' but no objective function or architectural details are given for this pretraining. The later loss in Eq. (3) only trains the adversary decoder, and Step 3 mentions joint fine-tuning, but the purification-stage loss and the exact role of the autoencoder are not specified. Without this, the core two-stage mechanism is under-specified and cannot be reimplemented from the text.","section":"§3.4.3, Step 1"},{"comment":"The claim that the base model achieves a ROUGE-L score of over 50% and cosine similarity of 90% 'regardless of the layer used as the partition point' is presented without standard deviations, number of seeds, or significance tests. Since attack training is stochastic and the reported improvements over baselines are sometimes modest (e.g., cosine similarity in Table 6), the absence of variance information makes it hard to assess whether the cross-layer and cross-method differences are reliable.","section":"§4.3, Figure 5"},{"comment":"The claim of 'over 60% improvement over the SOTA' is not qualified by metric or dataset. Concretely, on PersonaChat (Table 5), Base RevertLM improves ROUGE over GEIA by about 48%, while RevertLM improves it by about 67%; on Wiki (Table 6), the ROUGE improvement over GEIA is much larger, but cosine similarity improves by only about 2-4%. The blanket statement in the abstract and conclusion should be replaced with a precise statement of which metric and comparison justify the 60% figure.","section":"Abstract, §5, Tables 5 and 6"}],"minor_comments":[{"comment":"The abstract states this is 'the first' model inversion attack in split learning for LLMs; related work already includes UnSplit (§2.1) for split learning in general, so the novelty claim should be qualified to LLM-specific intermediate representations.","section":"Abstract and §2.2"},{"comment":"The text says that I(x,t) and I(t,y) are 'consistently positively correlated,' but no correlation coefficient or quantitative analysis is provided; Figure 3 appears to show non-monotonic behavior, and the axes and layer labels are not defined in the caption.","section":"§3.2"},{"comment":"The perplexity formula following Eq. (3) is not numbered and is written inline; it should be a numbered equation with a clear definition of the conditioning variables, especially because the adversary decoder is autoregressive.","section":"§3.4.3, Eq. (3)"},{"comment":"The tables report only point estimates; adding standard deviations across random seeds would improve interpretability. Also, the baseline 'Base RevertLM' versus 'RevertLM' naming should be defined in the caption: presumably the latter includes linear purification, but this should be stated.","section":"Tables 5 and 6"},{"comment":"The sentence 'Representations from the attention layers exhibit performance similar to those processed through entire blocks' is supported by Table 3, but the metric used in the table is unspecified; clarify whether the numbers are ROUGE-L, BLEU, or cosine similarity.","section":"§4.2"}],"recommendation":"major_revision","confidential_remarks":"The paper's central idea is timely and the empirical comparison is appropriate, but the manuscript is not yet reproducible: no code, no seeds or standard deviations, unresolved table references, and an under-specified training procedure. The most important scientific concern is the in-distribution training of the attacker, which directly limits the strength of the privacy claim for personalized LLMs. I would ask the authors to either demonstrate robustness under distribution shift or substantially soften the conclusion. I would also require the code or detailed hyperparameters before accepting, given the magnitude of the claimed improvements."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. This is the first paper I've seen that inverts intermediate hidden states, not embeddings, in split learning for LLMs. The two-stage design—purify the sparse representation back into embedding space, then run a generative decoder—is a sensible response to a real problem, and the gains in Tables 5 and 6 are large and internally consistent: ROUGE-L goes from 0.34 to 0.56 on PersonaChat and from 0.05 to 0.15 on Wiki against GEIA and Vec2Text. If those numbers survive scrutiny, this changes the threat model for edge-cloud personalized LLMs.\n\nWhat is actually new: not the general idea that embeddings leak—GEIA and Vec2Text already showed that—but the finding that sparse intermediate representations can be purified back into embedding space with a learned projection, plus the mutual-information layer analysis that maps where leakage peaks. The MI analysis is used as motivation, not to fit the attack metrics, so there is no circularity problem. It does not, however, rigorously explain why the linear projection works; Table 4 shows it empirically, but no formal condition is given.\n\nThe soft spots are real but not fatal. The biggest one, and the stress-test note is right about it: Section 4.1.1 trains the attacker on the same dataset's training split, which is exactly the 'distribution similar to the training data' assumption in Section 3.1. For genuinely personalized edge-cloud text—idiosyncratic, rare, not publicly available—that assumption is likely false, so the 38%-75% range is best read as an in-distribution upper bound, not a demonstrated privacy break for personalized data. The paper should either include an out-of-distribution experiment (train on PersonaChat, attack on Wiki or on held-out personas) or soften the abstract.\n\nSmaller issues: no code, no standard deviations or seeds, and unresolved table references like 'Table ??'. The 'linear projection with tester' variant in Table 4 is never explained. These are fixable and don't undermine the main direction.\n\nWho this is for: security and privacy researchers working on split learning or representation inversion, and practitioners deploying personalized LLMs. It deserves a serious referee. I'd want code and variance before betting on the exact numbers, but the question and the method are worth refereeing. My recommendation: send it to review, with a request for code, error bars, and an out-of-distribution auxiliary-data experiment.","headline":"First credible inversion attack on intermediate hidden states in split-learned LLMs, with large empirical gains, but the headline numbers assume a same-distribution auxiliary corpus and should be read as an in-distribution upper bound.","tokens_in":11810,"tokens_out":3536,"would_cite":true,"duration_ms":34442,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In split learning for personalized LLMs, the intermediate hidden states sent to the server can be inverted to recover users' private text with 38%-75% fidelity, over 60% better than prior attacks.","keywords":["Large language model","Edge computing","Split learning","Privacy","Model inversion attack","Information bottleneck theory","Representation inversion","Hidden state leakage"],"falsifier":"Train the attack with an auxiliary corpus deliberately drawn from a very different distribution than the victim's private text (for example, Wikipedia as auxiliary against personal chat messages as target). If the recovery metrics stay at the reported 38%-75% level, the similar-distribution condition is not load-bearing; if they collapse to near chance, the paper's own threat model bounds the attack.","tokens_in":10818,"feed_emoji":"🔓","tokens_out":5896,"duration_ms":55406,"temperature":0.7,"pith_summary":"This paper argues that split learning—a common way to personalize large language models on edge devices by sending intermediate hidden states to a server—can leak users' private text. The authors propose RevertLM, a two-stage attack that first projects the sparse hidden representation into the embedding space and then uses a generative decoder to regenerate the original words. In experiments with GPT2-XL and T5 as victim models and two text datasets, the attack recovers 38%-75% of the semantics, with over 60% improvement over prior embedding-inversion methods. The paper also uses mutual information to show that, contrary to the classical information-bottleneck picture, transformer hidden states do not monotonically lose input information with depth, so deeper layers are not automatically safer.","feed_headline":"Hidden AI states leak private text in split learning","feed_subtitle":"The server can reconstruct what you typed by projecting hidden states to embeddings and regenerating the text.","key_machinery":"The load-bearing object is the two-stage inversion pipeline. Stage one, the information purification module, uses a linear projection (the best variant tested) to map the transmitted hidden vector from the victim model's representation space into the embedding space of the attack generator; this counteracts the sparsity of post-transformer-block states. Stage two is an autoregressive decoder-only generator, GPT2-XL, trained with a sequence cross-entropy loss, which converts the purified embedding into a token sequence. The design is motivated by a mutual-information analysis: $I(x;h)$ measured between input text and hidden state does not monotonically decline across transformer blocks, and it correlates with inversion accuracy, so the choice of split point is a privacy-relevant variable.","core_discovery":"The central claim is that in split learning for personalized LLMs, an honest-but-curious server can reconstruct user text from the intermediate representations it receives, even when those representations are post-decoder hidden states rather than input embeddings. Existing embedding-inversion attacks degrade sharply on these sparse, information-poor representations, so RevertLM splits the problem: a learned linear projection purifies the representation back into the embedding space, and an autoregressive GPT2-XL decoder then recovers the text. The paper reports ROUGE-L above 0.50 and cosine similarity around 0.90 for GPT2, and shows consistent gains over GEIA and Vec2Text on PersonaChat and Wikipedia data. The information-bottleneck analysis is offered as the explanation for why attack success varies with split point: mutual information between input and hidden state, and between hidden state and output, move together and are highest in middle blocks, while attention-layer representations invert much better than FFN-layer ones.","pith_inferences":["If the similar-distribution requirement is relaxed, the 38%-75% figures should be read as an upper bound; real attackers targeting idiosyncratic personal data will likely need stronger auxiliary signals or model-specific priors.","The same linear-projection trick could be turned into a defense: a trainable layer that maps representations into a 'purified' space with low mutual information before transmission could preserve utility while blocking inversion.","The model-specificity is untested: whether a purification module trained on GPT2-XL transfers to other decoder families (e.g., LLaMA-class models) is an open question; if it does not, the attack's generality is narrower than the paper's framing.","The mutual-information correlation suggests a practical audit metric for split-learning APIs: expose a rank of per-layer inversion risk so that application designers can pick split points with quantified leakage."],"forward_implications":["Split-learning deployments should treat intermediate representations as sensitive as the raw text, since they can be inverted without access to model weights.","Split-point choice becomes a security decision: attention-layer hidden states and middle blocks invert better than FFN layers, so transmitting only deep FFN representations would reduce risk.","Defenses for embedding inversion do not automatically protect hidden states; a two-stage purification-plus-generation attack transfers inversion to deeper layers.","Mutual-information measurements can serve as a privacy audit tool: layers where $I(x;h)$ is high are the ones where inversion succeeds.","The reported over-60% improvement over GEIA and Vec2Text holds on matching-distribution auxiliary data and standard conversational/Wikipedia benchmarks."],"supporting_citations":[{"why":"Supplies the information-bottleneck theoretical lens for measuring mutual information across layers and predicting invertibility.","marker":"[17]"},{"why":"The generative embedding-inversion baseline (GEIA) that RevertLM improves on by over 60%.","marker":"[7]"},{"why":"Prior embedding-inversion method (Vec2Text) with iterative correction; used as baseline and as base attack model in comparisons.","marker":"[10]"},{"why":"Prior work on embedding-layer inversion, marking the gap for intermediate representations.","marker":"[11]"},{"why":"Demonstrates that language models can leak private text, motivating the inversion threat model.","marker":"[2]"},{"why":"Shows that split-learning activations can be exploited for inversion and label inference, grounding the threat model.","marker":"[5]"}],"fun_headline_variants":["RevertLM attack recovers text from hidden states in split learning","Split learning hidden states leak private text, new attack shows","New model inversion attack on LLM split learning recovers user text","Hidden states in split learning can be inverted to your original words"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack only works if the attacker has an auxiliary corpus whose distribution closely matches the victim's private text and knows the split model's structure; for rare or idiosyncratic personal data the purification and decoder training cannot be replicated.","fun_headline_variants_meta":{"raw":{"variants":["RevertLM attack recovers text from hidden states in split learning","Split learning hidden states leak private text, new attack shows","New model inversion attack on LLM split learning recovers user text","Hidden states in split learning can be inverted to your original words"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000255,"raw_usage":{"total_tokens":1571,"prompt_tokens":941,"completion_tokens":630,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":557,"completion_tokens_details":{"reasoning_tokens":566}},"tokens_in":557,"tokens_out":630,"duration_ms":6328,"temperature":1.0,"reasoning_tokens":566,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:08:07.080768+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the attack with an auxiliary corpus deliberately drawn from a very different distribution than the victim's private text (for example, Wikipedia as auxiliary against personal chat messages as target). If the recovery metrics stay at the reported 38%-75% level, the similar-distribution condition is not load-bearing; if they collapse to near chance, the paper's own threat model bounds the attack.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The generative embedding-inversion baseline (GEIA) that RevertLM improves on by over 60%."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates that language models can leak private text, motivating the inversion threat model."},{"cited_title":"Ercüment Çiçek","cited_arxiv_id":null,"evidence_quote":"Shows that split-learning activations can be exploited for inversion and label inference, grounding the threat model."}],"review_version":1}