Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Embedding-space random hyperplanes make covert LLM channels harder to recover but easier to learn, and leave probe-detectable internal traces.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 06:23 UTC pith:SIVTGXMH

load-bearing objection The attack-side recoverability framing is a real contribution, but the detection claim needs a benign-fine-tuning control before it can be stated as a result. the 4 major comments →

arxiv 2601.22818 v2 pith:SIVTGXMH submitted 2026-01-30 cs.CR cs.AI

Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models

Charles Westphal , Keivan Navaie , Fernando E. Rosas This is my paper
classification cs.CR cs.AI
keywords steganographylarge language modelsfine-tuningembedding space geometryrandom hyperplanespayload recoverabilitylinear probesmechanistic interpretability
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that covert communication through fine-tuned LLMs is not an artifact of trivially reversible encodings. It replaces arbitrary stego mappings—the parity-based token buckets and ASCII letter codes used in earlier work—with seed-dependent random hyperplanes in the token embedding space, and claims this makes the hidden payload much harder for a keyless defender to recover while making the encoding easier for the model to learn. Exact secret recovery improves on 8B and 70B models (for example, from 17% to 30%, 24% to 43%, and 9% to 19%), even as both recoverability metrics drop. On the defense side, the paper argues that output-level steganalysis using perplexity or KL divergence cannot work for fine-tuning attacks, because fine-tuning naturally shifts these statistics, and it proves that large payloads can hide within normal drift. It then claims that linear probes on later-layer activations detect the secret with up to 33% higher accuracy in steganographically fine-tuned models than base models, even for schemes whose outputs are not decodable at all. If true, malicious fine-tuning is a practical exfiltration risk that standard statistical monitoring misses, but internal model representations offer a usable detection signal.

Core claim

The paper's central claim is that the vulnerability of prior steganographic attacks lies in arbitrary, reversible mappings (token-ID parity and ASCII codes), and that deriving both the letter-to-bit encoding and the token-to-bit bucketing from seed-dependent random hyperplanes in embedding space yields a channel that is less recoverable yet easier to fine-tune. It formalizes recoverability as classifier accuracy along two axes—Rbucket for inferring bits from output tokens and Rsecret for inferring the secret from bits—and reports that earlier schemes score at 100% on both, while the geometric scheme lowers both. Exact secret recovery in held-out generations rises from 17→30%, 24→43%, and 9→1

What carries the argument

The central object is the seed-dependent random hyperplane in the model's token embedding space. Given a seed σ, a unit normal vector is sampled and each token's embedding is projected onto it; thresholding at the median projection splits the vocabulary into balanced buckets, so each generated token can be constrained to the bucket matching the target bit. Repeating the projection with several hyperplanes turns letter embeddings into multi-bit codes. The hyperplanes do two jobs at once: because they are defined by the model's learned geometry, the fine-tuned model learns the mapping readily, and because their orientation depends on secret seeds, a keyless defender must search a huge space, w

Load-bearing premise

The defense result assumes that the linear-probe accuracy gap between steganographically fine-tuned and base models is caused by the steganographic channel rather than by fine-tuning at large; the paper measures no benignly fine-tuned control, so any sufficiently strong fine-tuning effect would produce the same signature.

What would settle it

Fine-tune the same models and data with unconstrained (non-steganographic) generation and run the same linear probes; if the probe accuracy over base matches the steganographic models' gap, the internal-signature claim collapses.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Any steganographic scheme modeled on ASCII-plus-parity is fully recoverable by a keyless analyst, so output monitoring alone can extract the secret from such models.
  • Replacing arbitrary bucketing with embedding-space hyperplanes reduces recoverability on both axes while improving the model's ability to encode secrets exactly, with the largest gains on LoRA-tuned and larger models.
  • Perplexity- and KL-based steganalysis cannot be the primary defense against fine-tuning-based steganography: the proven lower bounds put large payloads inside the drift range that ordinary fine-tuning already produces.
  • Linear probes on later-layer activations can flag steganographic fine-tuning even when the output is not decodable, implying that internal representations are a viable detection surface.
  • The proposed detection requires hidden-state access; output-only or black-box detection is left open.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A benign fine-tuning control is the natural next experiment: if probes trained on a non-steganographic fine-tune show a similar accuracy gap over base, the 'internal signature' would be a marker of fine-tuning strength rather than of the covert channel.
  • Because the bucket partition is seed-defined, the search space for seed recovery is a concrete attack surface; a defender with a small set of suspected seeds could test them against observed output distributions.
  • The same hyperplane primitive can be scaled to multiple bits per token, and the paper's own capacity bounds suggest such scaled channels would still sit within normal perplexity drift, making them harder to spot by output statistics.
  • If detection works on non-decodable outputs, defenses might aim at 'encoding behavior' in activations generally, rather than trying to decode specific payloads—though this depends on hidden-state access that deployed models may not expose.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies training-time steganography in LLMs. It formalizes payload recoverability through two classifier-based metrics (Rbucket for token-to-bit inference and Rsecret for bit-to-secret recovery), shows that the prior TrojanStego scheme is perfectly recoverable on both axes, and proposes a low-recoverability scheme in which secret encoding and vocabulary bucketing are derived from seed-dependent random hyperplanes in the token-embedding space. Experiments on Llama-8B, Ministral-8B, and Llama-70B show that the proposed schemes improve exact-match encoding accuracy while lowering output-side recoverability. The paper further argues that distributional steganalysis is insufficient for fine-tuning-based attacks, proves per-token KL and perplexity lower bounds, and proposes linear probes on late-layer activations as a detection method, claiming up to 33% higher secret-detection accuracy in fine-tuned versus base models, including under a non-decodable XOR-masked control.

Significance. If validated, the attack-side contribution is genuinely useful: the two-axis recoverability formalism is a clean way to separate token-to-bit and bit-to-secret difficulty, and the embedding-geometry bucketing is a plausible mechanism for making covert channels harder to reverse-engineer while remaining learnable. The multi-model, multi-dataset evaluation is a strength, as are the explicit derivations of the KL and perplexity bounds. The detection claim is potentially important, but in its current form it is not supported by the experiments: the missing benign-fine-tuning control means the probe gap may reflect generic fine-tuning rather than steganography, and the XOR-control results are only partially consistent. These issues are fixable, but they are load-bearing for the paper's headline defense contribution.

major comments (4)
  1. [§7.3, Figure 5] The central detection claim is that linear probes detect secrets with up to 33% higher accuracy in fine-tuned models compared with base models, and that this signal persists for low-recoverability schemes. The experiments compare steganographically fine-tuned models only against base models. There is no benignly fine-tuned control: a model fine-tuned on the same prompts and with the same training budget but without the bucket constraint. If any strong fine-tuning increases the linear readability of prompt-latent information, the observed gap is not diagnostic of steganography. The XOR-masking control in §7.4 does not resolve this, because it still fine-tunes on a bit-constrained generation task. Please add a benign-control experiment (e.g., same data and secret strings, but no bucket-constrained loss, or a standard instruction-following fine-tune) and report the probe accuracy gap for th
  2. [§7.4, Table 3] The text says that outside of Ministral-8B full fine-tuning, cosine similarity for embedding recovery is 'consistently higher' in fine-tuned models, and that the detection signal persists under the non-decodable control. This is not what Table 3 shows. For A-XOR+E on Ministral Full, Cos is 0.24±0.01 versus a base of 0.26±0.00, and for E-XOR+E on Ministral Full it is 0.22±0.01 versus 0.25±0.00—both lower than the base. In addition, bit-accuracy detection succeeds in only three of eight scenarios, as the paper itself notes. Please restate the conclusion to reflect the actual success rates, and clarify in which model/encoding configurations the 'persisting' signal is claimed.
  3. [§6.4, Appendix E] The text-quality claim that 'other configurations maintain coherence and stability scores comparable to the base model' is based on very small samples: coherence and stability are averaged over 10 prompts, and KL divergence over 20 prompts, per configuration. With n=10, the standard-error bars in Figure 4 are wide and no statistical test is reported. Since the threat-realness argument depends on generated text remaining fluent, please either increase the number of evaluation prompts substantially or soften the claim to a qualitative observation. This does not affect the recoverability results but it does affect the practical significance of the attack.
  4. [Abstract and §7.3] The paper uses 'up to 33% higher accuracy' for the probe-detection improvement, but §7.3 also reports that fine-tuned models reach 95–100% final-layer bit-accuracy while base models plateau around 60%, which is a 35–40 percentage-point gap. Please specify whether the 33% is a relative or absolute improvement, report the configuration that produces it, and make the abstract and main-text numbers consistent. As written, the headline number is ambiguous.
minor comments (5)
  1. [§6.2, Table 1] The abstract and Section 6.2 describe the exact-match metric as 'exact secret recovery,' but the metric is the percentage of examples where all 32 bits are successfully hidden in the output. This is an encoding-success rate, not an adversary's secret-recovery rate. Please use consistent terminology to avoid confusion with Rbucket/Rsecret.
  2. [§5.2, Appendix D.1] The bucketing and encoding depend on a single hyperplane seed (sigma=42), and the encoding search is described as searching for a collision-free seed configuration. No sensitivity analysis is reported. Since the recoverability and learnability claims depend on properties of random hyperplanes, at least a brief discussion of seed sensitivity (or an argument that the number of tested seeds is sufficient) should be added.
  3. [Appendix B.1] The KKT proof uses α both as e^{-(1+λ)} and as 1/Z; the numerical example in §7.1 writes log(1/0.974) where the proof derives log α. This is not incorrect, but it is confusing. Please align the notation and state explicitly that α=1/Z so that log α is positive.
  4. [Figure 2] Figure 2 is hard to read: several legend entries overlap, and the figure does not report numeric recoverability values. Since 'low-recoverability' is a central quantitative claim, please add numeric Rbucket and Rsecret values (or a table) for the main configurations.
  5. [General] The paper says code will be released but gives no repository or data-availability link. Providing the code and the exact hyperplane construction would materially help reproducibility.

Circularity Check

0 steps flagged

No significant circularity; central claims are empirical or separately proved.

full rationale

The paper's contribution is mostly empirical benchmarking plus two self-contained information-theoretic lower bounds. The KL and perplexity bounds (Theorems 7.1 and 7.2) are proved from first principles and do not assume the attack's success; the illustrative 0.026 nats figure is an explicit example, not an input-output identity. The attack-side exact-match numbers are held-out accuracies of a trained model, so they are measurements, not predictions fitted to the same data. The recoverability metric is definitional: the paper defines Rbucket/Rsecret via classifier accuracy and then measures it; this is an operational definition, not a circular derivation, though it means 'unrecoverable' should be read as 'not recovered by these classifiers.' The probe-detection experiments are supervised evaluations: probes are trained on 800 samples and tested on 200 held-out secrets/prompts, so test accuracy is not a fitted train-set artifact. High probe accuracy in later layers is expected given the model is trained to place bits in its output-token choices, but the paper does not derive it from its recoverability theory; it reports it as an empirical signature. The XOR control in Appendix G explicitly builds a keystream that depends on the secret ('This circularity ensures non-decodability'), but this is an intentional control property, not a logical circularity in the paper's argument. The main validity concern—absence of a benignly fine-tuned control in Section 7.3—is a confound in the detection claim, not a circularity: it questions whether the probe gap is due to steganography or to fine-tuning generally. There are no load-bearing self-citations and no imported uniqueness theorems. Overall, no step in the derivation reduces, by definition or by fitted parameter, to its own input.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The paper introduces no new physical or conceptual entities; its load-bearing commitments are design choices (seeds, ranks, hyperplane counts, classifier families) and domain assumptions about learnability, seed secrecy, constrained generation, and the specificity of activation signatures. The most fragile is the last: no benign fine-tuning baseline is measured.

free parameters (5)
  • Hyperplane seed sigma=42 = 42
    Defines the actual bucket mapping used in all attack experiments (Appendix D.1); the reported exact-match and recoverability numbers are for this single seed and may not generalize.
  • LoRA rank = r=128 for Wiki, r=8 for TrojanStego
    Chosen per dataset; the authors state the Wiki rank was increased after observing underfitting (Table 5). Attack accuracies depend on this choice.
  • Number of hyperplanes per letter and secret bit length = 8 hyperplanes per letter; 32-bit secrets
    Design choice for a collision-free 4-letter encoding (Section 5.1); changing it changes exact-match rates and recoverability.
  • Illustrative peaked next-token distribution = p1=0.5, p2=0.3, remaining 0.2
    Used in Section 7.1 to conclude that KLD can be ~0.026 nats 'much lower than expected during fine-tuning'; this is an assumed example, not measured from the models.
  • Classifier families H and G = XGBoost and MLP, with fixed PCA dimensions
    Rbucket and Rsecret are maxima over these finite classifier families (Definition 3.1, Appendix C); the 'low recoverability' conclusion is specific to these classifiers, not a general hardness result.
axioms (5)
  • domain assumption Random hyperplanes in the frozen token embedding space produce a balanced, learnable bucket partition.
    Section 5.2; the median threshold balances buckets by construction, but learnability is an empirical premise that the attack claims and tests.
  • domain assumption A keyless observer without the random seeds cannot feasibly infer the bucket mapping.
    Section 5.2 and 6.2; the low-recoverability claim rests on seed secrecy and search hardness, which is discussed but not proved.
  • domain assumption Constrained generation can promote any required-bucket token to argmax while preserving fluent text.
    Used in Theorem 7.1, Theorem 7.2, and the attack generation procedure; it assumes the fine-tuned distribution can realize the constrained token choices used in the bounds.
  • domain assumption The KL/perplexity shifts produced by steganographic fine-tuning stay within 'typical fine-tuning ranges'.
    Section 7.1; the paper cites Rafailov et al. for this range and provides numerical examples, but does not measure the full range of benign fine-tuning shifts.
  • ad hoc to paper Linear-probe accuracy differences between base and fine-tuned models are attributable to steganography rather than generic fine-tuning.
    Section 7.3; this is the load-bearing premise of the detection claim and is not tested with a benign fine-tuned control.

pith-pipeline@v1.3.0-alltime-deepseek · 19568 in / 14596 out tokens · 141622 ms · 2026-08-03T06:23:36.948440+00:00 · methodology

0 comments
read the original abstract

Fine-tuned LLMs can covertly encode prompt secrets into outputs via steganographic channels. Prior work demonstrated this threat but relied on trivially recoverable encodings. We formalize payload recoverability via classifier accuracy and show previous schemes achieve 100\% recoverability. In response, we introduce low-recoverability steganography, replacing arbitrary mappings with embedding-space-derived ones. For Llama-8B (LoRA) and Ministral-8B (LoRA) trained on TrojanStego prompts, exact secret recovery rises from 17$\rightarrow$30\% (+78\%) and 24$\rightarrow$43\% (+80\%) respectively, while on Llama-70B (LoRA) trained on Wiki prompts, it climbs from 9$\rightarrow$19\% (+123\%), all while reducing payload recoverability. We then discuss detection. We argue that detecting fine-tuning-based steganographic attacks requires approaches beyond traditional steganalysis. Standard approaches measure distributional shift, which is an expected side-effect of fine-tuning. Instead, we propose a mechanistic interpretability approach: linear probes trained on later-layer activations detect the secret with up to 33\% higher accuracy in fine-tuned models compared to base models, even for low-recoverability schemes. This suggests that malicious fine-tuning leaves actionable internal signatures amenable to interpretability-based defenses.

Figures

Figures reproduced from arXiv: 2601.22818 by Charles Westphal, Fernando E. Rosas, Keivan Navaie.

Figure 1
Figure 1. Figure 1: Overview of the steganographic attack. Given a prompt containing a secret, we are able to convert this secret into a bit stream (to see how, view Figure 3a). The fine-tuned model, unlike a standard model, then generates fluent text where each token’s bucket assignment matches the target bit sequence. ing internal signatures alone. Together, these results suggest that attacks can be made harder to reverse-e… view at source ↗
Figure 2
Figure 2. Figure 2: Payload recoverability of steganographic schemes. Tro￾janStego lies in the upper-right corner (perfect recoverability); those that include our geometric bucketing scheme achieve lower recoverability on both axes. to bucket 1, even to bucket 0. At the secret level, ASCII’s regular 8-bit structure is trivially learnable by an MLP map￾ping bit sequences to characters, giving Rsecret = 1. At the bit level, rec… view at source ↗
Figure 3
Figure 3. Figure 3: Hyperplane projection for encoding and bucketing. (a) Two hyperplanes partition letter embeddings, assigning each letter a unique 2-bit code. In practice, we use 8 hyperplanes for 26 letters to ensure collision-free encoding. (b) A seed-dependent hyperplane partitions token embeddings into buckets V0 and V1. zσ ∼ N (0, Id), which defines the orientation of our hyper￾plane. For any embedding e, we compute i… view at source ↗
Figure 4
Figure 4. Figure 4: Text quality metrics for steganographic outputs on both datasets and across all encodings (see [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Mechanistic detection via linear probes. Top: Probe quality across layers. Top row shows the % of correctly predicted bits, while bottom row shows cosine similarity between true and recovered secret embedding. Bottom: Final layer performance for both bit detection and embedding recovery (see [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. $\pi$Creds: Privately Inferred Credentials

    cs.CR 2026-06 unverdicted novelty 7.0

    πCreds produces privacy-preserving verifiable credentials via trusted LLM inference on authenticated data, expanding claim types to unstructured sources and formalizing SCAE and ACPP threat models.

  2. Now You (Still) See Me: Detecting Evasive Steganographic Payloads in LLMs

    cs.CR 2026-06 unverdicted novelty 6.0

    Adversarial fine-tuning evades activation-based steganography detection in five LLMs while preserving secret recovery, but a recontextualization dataset restores both ridge and MLP probe detectability.

Reference graph

Works this paper leans on

4 extracted references · 3 linked inside Pith · cited by 2 Pith papers

  1. [226]

    naacl-long.226/

    URL https://aclanthology.org/2024. naacl-long.226/. Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. LoRA: Low-rank adaptation of large language models.arXiv preprint arXiv:2106.09685, 2022. Hugging Face H4 Team. Helpful instructions dataset. https://huggingface.co/datasets/ HuggingFaceH4/helpful-instructions, 202...

    Pith/arXiv arXiv 2024

  2. [361]

    Write a short story about

    URL https://aclanthology.org/2025. emnlp-main.361/. Yang, Z., Wei, N., Liu, Q., Huang, Y ., and Zhang, Y . Gan- tstega: Text steganography based on generative adver- sarial networks. InDigital F orensics and Watermarking (IWDW 2019), volume 12022 ofLecture Notes in Com- puter Science, pp. 18–31. Springer, Cham, 2020. doi: 10.1007/978-3-030-43575-2 2. Yous...

    Pith/arXiv arXiv 2025

  3. [2024]

    9 Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models Hewitt, J

    URL https://proceedings.mlr.press/ v235/halawi24a.html. 9 Hide and Seek in Embedding Space: Geometry-based Steganography and Detection in Large Language Models Hewitt, J. and Manning, C. D. A structural probe for finding syntax in word representations.arXiv preprint arXiv:1905.06316, 2019. Hoerl, A. E. and Kennard, R. W. Ridge regression: Biased estimatio...

    Pith/arXiv arXiv 1905

  4. [2025]

    ISBN 979-8-89176-332-6

    Association for Computational Linguistics. ISBN 979-8-89176-332-6. doi: 10.18653/v1/2025.emnlp-main

    doi:10.18653/v1/2025.emnlp-main 2025