{"id":"80902881-a181-43aa-b8f9-474f8bfa41fe","arxiv_id":"2506.09452","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A learned stochastic embedding obfuscator, the Stained Glass Transform, is claimed to reduce mutual information between prompts and their server-side representations while preserving LLM utility.","lead":"This paper proposes a learned transformation, the Stained Glass Transform, that scrambles a language model's input embeddings before they are sent to a server, with the aim of hiding the original text while keeping the model's answers nearly as good. It reports that the transformation can protect private details such as addresses in prompts with only a small drop in benchmark performance across models up to 70B parameters.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1 does not estimate the mutual information it is claimed to compute: it averages negative log densities across cross-samples instead of taking the log of the averaged density, so the reported MI and PAC-Adv privacy bounds are unsupported.","rationale":"The strongest claim is that SGT information-theoretically protects input embeddings while preserving utility. That claim requires a valid estimate of MI(X; tilde X). The paper's own Algorithm 1 is not a Monte Carlo approximation of the entropy term in Eq. (7); Jensen's inequality shows it computes a different objective. This is not a disagreement with consensus or a vague concern about adversarial capability; it is an internal mismatch between the stated target quantity and the implemented estimator. It is the most load-bearing issue because every PAC-Adv and MI number supporting the privacy headline comes from this same estimator, and the model is trained with it. If the estimator is replaced, the method may still work - the utility numbers are strong and the attack-based PII results provide independent empirical evidence - but the information-theoretic guarantee would need to be re-derived. The reader already identified the MI estimator as the weakest assumption; this critique pins down the exact mechanism and offers a direct test. The verdict remains CONDITIONAL because the flaw is concrete and fixable rather than a fundamental refutation of the approach.","tokens_in":17535,"tokens_out":7108,"duration_ms":84003,"concrete_test":"On a 2D Gaussian mixture with known exact MI, run three estimators on the same samples: (i) Algorithm 1 as written, (ii) a logsumexp Monte Carlo estimate 1/B sum_i log(1/B sum_l G(tilde x_i; x'_l)), and (iii) numerical integration for the exact MI. If (i) does not converge to (iii) while (ii) does, Algorithm 1 is invalid as an MI estimator. Then re-run the Table 1 ablation with the logsumexp loss; if the PAC-Adv values change materially, the paper's privacy claims must be revised.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central privacy claim depends on the mutual information estimate used in both the training loss (Section 5.2) and the reported PAC-Adv bounds (Section 6.4). Algorithm 1 targets Eq. (7), whose first term is E_{z~tilde X}[-log E_{x~X}[G(z;x)]]. The algorithm instead computes, for each independently drawn pair (tilde x_i, x'_ell), the value -log G(tilde x_i; x'_ell) and averages over the batch. That averages the negative log of a component density, not the negative log of the mixture density. Since -log is convex, E_{x'}[-log G(z;x')] >= -log E_{x'}[G(z;x')], so the computed quantity is a different, systematically larger functional of the distributions. Proposition 5.1 and Eq. (8) inherit this mismatch: the 'minibatch Monte Carlo MI loss' is a cross-component negative log-likelihood, not a mutual information loss. The consequence is that the MI numbers in Tables 1-3 and the PAC-Adv column based on Eq. (4) do not measure the information-theoretic quantity the abstract invokes. The AbsCos row in Table 1 - enormous reported MI yet the best PII protection in Table 3 - is the signature of this mismatch: the reported MI is not the operative privacy signal. The attack-based results (NN-FR, BeamClean, PII-Ratio) are useful empirical evidence, but they do not establish the claimed information-theoretic guarantee.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Stained Glass Transform (SGT), a learned, stochastic, sequence-dependent transformation of LLM token embeddings designed so that a client can send obfuscated embeddings to a server while preserving the utility of a frozen target LLM. The training objective combines a utility loss on the logits of the frozen model with an obfuscation loss that includes a mutual-information term derived from a Gaussian-mixture characterization of the obfuscated distribution, an absolute cosine penalty, and a median-norm penalty. The authors report benchmark utility drops of roughly 0.3 to 2 percentage points for models such as Llama-3.3-70B and Qwen3-32B, together with reconstruction-attack failure rates and PAC-derived privacy bounds. The central claim is that the SGT 'information theoretically provides privacy' to the LLM input.","tokens_in":17883,"tokens_out":14113,"duration_ms":159740,"significance":"If the central claim were fully supported, the paper would make a valuable contribution to privacy-preserving LLM inference: it demonstrates that learned embedding obfuscation can preserve utility on large models, and it evaluates privacy with a broader set of attacks (NN, MRP, BeamClean, PII recovery) than many prior works. The large-scale experiments across multiple model families and the explicit negative results for constant-noise obfuscation are useful empirical contributions. However, the headline information-theoretic privacy guarantee rests on a mutual-information estimator that is not the quantity the paper claims it is, and the reported PAC-Adv bounds inherit that flaw. The empirical attack results are informative, but they do not by themselves establish the claimed information-theoretic guarantee. The paper is therefore a strong empirical proposal whose theoretical framing needs substantial correction.","major_comments":[{"comment":"Algorithm 1 computes, for each independently drawn pair, the value -log G(\\tilde x_1; x_2) and averages these values. This estimates E_{z~\\tilde X} E_{X'}[-log G(z;X')], whereas Eq. (7) requires E_{z~\\tilde X}[-log E_X[G(z;X)]]. Because -log is convex, E_{X'}[-log G(z;X')] >= -log E_{X'}[G(z;X')] for each z, so the computed quantity is a systematically larger functional of the distributions, namely an upper bound on the GMM entropy rather than an estimate of it. Consequently, the values labeled 'MI' in Tables 1-3 and the 'PAC-Adv' column in Section 6.4 are not estimates of the mutual information invoked in the abstract. The authors should either replace the estimator with a consistent one (e.g., average the component densities before taking the log) or explicitly prove and label the current quantity as a conservative upper bound, with appropriate confidence intervals, and state what guarantee it does and does not provide.","section":"§5.2, Algorithm 1 and Eq. (7)"},{"comment":"The derivation and interpretation of Eq. (8) are not correct. Since the two batches are i.i.d., the expected value of log(|\\Sigma_i^{-1} \\Sigma_{\\ell'}|) over the batch-pair distribution is zero; this term cannot, as claimed in Section 5.2, 'encourage the covariances to be more similar to one another.' In expectation, the loss reduces to an average Mahalanobis distance between obfuscations and independent mixture components, which is not the mutual-information loss described in the text. The statement that 'terms constant in \\theta have been dropped' is also incorrect: both the log-determinant terms and the quadratic terms depend on \\theta. The proposition and the surrounding explanation need to be re-derived with explicit indexing and expectations.","section":"Proposition 5.1 and Eq. (8)"},{"comment":"The PAC-Adv column is computed by inverting Eq. (4) using the miscalibrated MI estimates from Algorithm 1, and no standard errors or confidence intervals are reported. A PAC-style privacy claim requires a rigorous upper bound that holds with high probability over the evaluation sample; the manuscript does not provide one. The table also exposes an internal inconsistency: the AbsCos row reports an enormous MI value (6 x 10^9) with PAC-Adv capped at 100%, while Table 3 shows that AbsCos has the lowest PII recovery (roughly 1.6 against both NN and BeamClean). The reported MI is therefore not tracking the operational privacy risk. This discrepancy must be resolved or explicitly explained before the information-theoretic privacy claim can be accepted.","section":"§6.4, Table 1"},{"comment":"The reported MI and PAC-Adv are feature-level quantities, but the abstract and introduction claim protection of the LLM's input prompt and its token embeddings. Feature-level mutual information does not upper-bound the mutual information of the full embedding vector: dependencies across features can carry information even when each marginal is well obfuscated. The authors should either present the metric explicitly as a per-feature guarantee and adjust the abstract accordingly, or compute a bound for the full token-level or prompt-level representation. As written, the feature-level PAC bound does not support the statement that the SGT 'information theoretically provides privacy to the input of the LLM.'","section":"§6.4 and abstract"}],"minor_comments":[{"comment":"Equation (12) writes L_CGE as log(-|\\Sigma|^{-1}), which is undefined because |\\Sigma| > 0. Presumably the intended expression is -log|\\Sigma| or log|\\Sigma^{-1}|; please correct this.","section":"§5.3, Eq. (12)"},{"comment":"In Corollary 3.4, the notation introduces 'Y' without definition; it should be '\\tilde X' throughout.","section":"§3, Corollary 3.4"},{"comment":"The sentence 'We then use Corollary 3.4 to derive a mon' is truncated and incomplete; the derivation of the mutual-information loss needs to be restated completely.","section":"§5, first paragraph"},{"comment":"There are several typographical issues: 'the the' in the proof of Proposition 3.3, 'preceeding' in Section 5, 'agaisnt' in Section 7.2, and duplicate '[16], [16]' in the references. These should be cleaned up.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The paper is an industry submission with a useful empirical obfuscation method, but the headline information-theoretic guarantee is not supported by the current estimator. I recommend major revision rather than rejection because the empirical contributions (large-model training and multi-pronged attack evaluation) are valuable, and the theoretical issue is in principle fixable by re-deriving the loss as a rigorous upper bound or replacing the estimator with a correct one. The authors should also be pressed to state the threat model explicitly, including whether the SGT parameters are public or client-side secret, and to provide corrected MI/PAC numbers or downgrade the claims to empirical attack-resistance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead the Stained Glass Transform paper. The core trick is genuinely new: a learned, sequence-dependent Gaussian perturbation of embeddings, with a loss combining a minibatch Monte Carlo approximation to mixture entropy, a cosine-orthogonality term, and a norm penalty. On the empirical side it delivers: for Llama-3.3-70B, Qwen3-32B, and DeepSeek-R1-70B the utility drop is within 0.5 pp, and the attack-based evaluations (NN, MRP, BeamClean, PII-Ratio) show real resistance. The PII-Ratio results on PUPA are the strongest evidence that something useful is happening.\n\nThe soft spot is the privacy claim, and it is load-bearing. Algorithm 1 does not estimate the mutual information in Eq. (7). That equation needs E_x[-log E_{x'}[G(z;x')]]; the algorithm averages -log G(z_i; x'_j) over a cross batch. Because -log is convex, the algorithm computes a strictly larger functional—a cross-component negative log-likelihood, not a MI estimate. So Proposition 5.1 and Eq. (8) are not a Monte Carlo MI loss; they minimize a different objective. The PAC-Adv numbers in Tables 1 and 2 inherit the bias, and because the same estimator was used as the training loss, the evaluation of MI is circular. No error bars either.\n\nThe AbsCos row is the tell: it reports MI around 6e9 while giving the best PII protection. That is impossible if the MI estimate measured what the abstract says it does. What actually protects PII is the learned orthogonalization, not the MI term.\n\nI would still send this to a serious referee, but with a note that the theoretical section needs to be redone. The empirical attack results are worth reporting even if the information-theoretic framing is set aside. As it stands, the headline claim—\"information theoretically provides privacy\"—is unsupported. For a reading group, this is a good case study in how a biased estimator can invalidate a privacy guarantee.\n\nRecommendation: peer review, major revision, with the MI claim either fixed with an unbiased estimator or dropped in favor of the attack-based metrics.","headline":"The empirical obfuscation method shows real utility and attack resistance, but the headline MI-based privacy guarantee is built on an estimator that does not compute mutual information.","tokens_in":18409,"tokens_out":2690,"would_cite":false,"duration_ms":28420,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a learned stochastic transform of token embeddings gives LLM prompts information-theoretic privacy while keeping benchmark answers nearly unchanged.","keywords":["Stained Glass Transform","embedding obfuscation","mutual information","PAC privacy","Gaussian mixture model","prompt privacy","reconstruction attack","LLM inference"],"falsifier":"Train a reconstruction decoder on a large set of (original prompt, SGT obfuscation) pairs drawn from the same distribution, or run a language-aware search with the true noise model, and measure whether token- or PII-level recovery succeeds at a rate far above the paper's PAC-Adv bound. Alternatively, recompute the mutual information with a tighter estimator on a held-out set and check whether it is much larger than the training-time estimate; either result would falsify the information-theoretic privacy claim.","tokens_in":17308,"feed_emoji":"🔒","tokens_out":9782,"duration_ms":93535,"temperature":0.7,"pith_summary":"Enterprises that want to use shared or managed LLM infrastructure currently have to send their prompts as plaintext, because no one has found a way to protect the input at inference without badly hurting the model's answers. This paper argues that protection can happen at the embedding layer instead of the token layer. It introduces the Stained Glass Transform (SGT), a learned stochastic map that adds sequence-dependent Gaussian noise to each token embedding before it leaves the client, so neither the original tokens nor an invertible embedding is transmitted. The paper's central claim is that this transform information-theoretically prevents reconstruction of the input while keeping utility close to baseline, with measured benchmark degradation of about 0.3 to 2 percentage points. If the claim holds, sensitive data could be used with shared LLM infrastructure without ever appearing in plaintext.","feed_headline":"Obfuscated embeddings hide LLM prompts with answers nearly unchanged","feed_subtitle":"A learned noise layer blocks prompt reconstruction while keeping benchmark utility within a couple of points.","key_machinery":"The central object is the Affine Stained Glass Transform: for an input embedding sequence $x$, the obfuscation is $\\tilde{x}=x+\\mu_\\theta(x)+\\Sigma_\\theta(x)^{1/2}u$ with $u\\sim N(0,I)$, where $\\mu_\\theta$ and a diagonal $\\Sigma_\\theta$ are predicted by a transformer that sees the whole sequence. Because the transform is affine, the obfuscations of a dataset are distributed as a Gaussian mixture model, which lets the authors write the mutual information as mixture entropy minus conditional Gaussian entropy and approximate it with the minibatch Monte Carlo estimator in their Algorithm 1. That estimator is simultaneously the training loss and the basis for the reported PAC-privacy bounds.","core_discovery":"On the paper's own terms, the discovery is that a parametric, sequence-conditioned obfuscation of embeddings can be trained against a frozen LLM so that the obfuscated embeddings remain useful to the model while the mutual information between clean and obfuscated inputs is driven down. The authors show that affine SGTs turn a dataset of obfuscations into a Gaussian mixture model, derive a minibatch Monte Carlo surrogate for the mixture entropy, and combine it with absolute-cosine and median-norm penalties. In their experiments the full loss keeps utility within 1.97 percentage points on a 1B model and within 0.5 percentage points on three large models, resists nearest-neighbor and language-aware reconstruction for the large majority of tokens, and yields finite PAC-style reconstruction bounds, with the smallest reported feature-level bound at 12.69%.","pith_inferences":["Because the SGT is an affine map on a continuous vector space, the same mechanism could extend to other frozen models with continuous inputs, such as vision transformers or retrieval encoders, although the paper only tests text LLMs.","The training-time Monte Carlo estimate of mutual information is doing double duty as loss and as privacy certificate; a deployment-grade claim would need a held-out estimator or an adversary-aware reconstruction benchmark to confirm the reported PAC-Adv numbers.","The paper's own results show that a cosine-only loss can have enormous estimated mutual information yet still achieve very low PII recovery, which suggests the MI number and real text-level privacy are not measuring the same thing; practical deployments should rely on reconstruction attacks as the primary check."],"forward_implications":["A client can send obfuscated embeddings to a shared or managed LLM server and receive answers without the server seeing plaintext tokens or invertible token embeddings.","Benchmark utility stays within a couple of percentage points of the unmodified model, and within half a point on large 70B-class models, so the obfuscation is compatible with production-scale inference.","Training with a mutual-information term is the only tested way to obtain finite PAC-style reconstruction bounds; purely geometric losses such as cosine penalties can defeat nearest neighbor attacks while still leaking private information.","The same SGT training recipe applies across model architectures and sizes without modifying the target LLM's weights."],"supporting_citations":[{"why":"Supplies the PAC-privacy bound relating reconstruction advantage to mutual information, which the reported privacy guarantees are built on.","marker":"[28]"},{"why":"Shows that plain token embeddings can be inverted back to text with nearest-neighbor search, motivating the need to transform embeddings.","marker":"[8]"},{"why":"Provides the language-aware embedding reconstruction attack used as a baseline for evaluating the obfuscations.","marker":"[38]"},{"why":"Supplies the PUPA dataset and the LLM-as-judge PII-ratio metric used to evaluate text-level privacy.","marker":"[49]"}],"fun_headline_variants":["Stained Glass Transform obfuscates embeddings, keeps LLM answers","Learn to hide LLM prompts: SGT embeds privacy with utility","Obfuscated embeddings: privacy for LLM inputs, near-same scores","SGT: stochastic embedding obfuscation with mutual-information bounds","Privacy-preserving LLM inference via stained-glass embeddings"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the minibatch Monte Carlo estimate of mutual information, which is also the training objective, accurately measures how much a real attacker could reconstruct from the obfuscated embeddings; if that estimate is biased, the reported privacy bounds do not actually guarantee privacy.","fun_headline_variants_meta":{"raw":{"variants":["Stained Glass Transform obfuscates embeddings, keeps LLM answers","Learn to hide LLM prompts: SGT embeds privacy with utility","Obfuscated embeddings: privacy for LLM inputs, near-same scores","SGT: stochastic embedding obfuscation with mutual-information bounds","Privacy-preserving LLM inference via stained-glass embeddings"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000304,"raw_usage":{"total_tokens":1736,"prompt_tokens":922,"completion_tokens":814,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":720}},"tokens_in":538,"tokens_out":814,"duration_ms":7293,"temperature":1.0,"reasoning_tokens":720,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T04:48:18.664843+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a reconstruction decoder on a large set of (original prompt, SGT obfuscation) pairs drawn from the same distribution, or run a language-aware search with the true noise model, and measure whether token- or PII-level recovery succeeds at a rate far above the paper's PAC-Adv bound. Alternatively, recompute the mutual information with a tighter estimator on a held-out set and check whether it is much larger than the training-time estimate; either result would falsify the information-theoretic privacy claim.","supporting_citations":[{"cited_title":"Formal privacy proof of data encoding: The possibility and impossibility of learnable encryption,","cited_arxiv_id":null,"evidence_quote":"Supplies the PAC-privacy bound relating reconstruction advantage to mutual information, which the reported privacy guarantees are built on."},{"cited_title":"BeamClean: Language Aware Embedding Reconstruction","cited_arxiv_id":"2505.13758","evidence_quote":"Provides the language-aware embedding reconstruction attack used as a baseline for evaluating the obfuscations."}],"review_version":1}