pith. sign in

arxiv: 2604.09066 · v2 · submitted 2026-04-10 · 💻 cs.CL

Anchored Sliding Window: Toward Robust and Imperceptible Linguistic Steganography

Ruiyi Yan , Shiao Meng , Yugo Murawaki This is my paper

Pith reviewed 2026-05-10 17:11 UTC · model grok-4.3

classification 💻 cs.CL
keywords linguistic steganographylanguage modelscontext windowrobustnessimperceptibilityprompt distillation
0
0 comments X

The pith

Anchoring the prompt and a bridge context in the sliding window lets language models produce steganographic text that resists edits while staying high quality.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Linguistic steganography hides secret messages inside text generated by language models, yet the resulting text usually breaks under even minor changes because the hidden bits depend on exact prior tokens. Earlier fixes restricted how much prior context the model could see, which strengthened robustness but produced noticeably less natural text. The anchored sliding window keeps the original prompt and a short learned bridge segment fixed inside the model's context window as generation slides forward. This setup lets the model internally adjust for any tokens that fall out of view. Experiments across multiple settings show the resulting text scores higher on quality, is harder to distinguish from ordinary writing, and survives alterations better than the previous sliding-window baseline.

Core claim

By anchoring both the prompt and a bridge context inside the language model's sliding context window and optimizing the bridge via a prompt-distillation objective extended with self-distillation, the model compensates for excluded tokens and thereby generates steganographic text that maintains higher quality, greater imperceptibility, and stronger robustness to modifications than text produced by the standard sliding-window approach.

What carries the argument

Anchored sliding window (ASW), which fixes the prompt and a distilled bridge context within the sliding context window so the language model compensates for tokens that drop out.

If this is right

  • Text quality stays higher than the baseline even while robustness increases.
  • Imperceptibility improves because the generated sentences remain more natural.
  • Robustness holds across different language models, message lengths, and embedding rates.
  • Self-distillation of the bridge context yields further gains in the same three metrics.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same anchoring idea could be tested in other generation settings where context truncation hurts coherence.
  • Pairing ASW with existing watermark detectors might let one control both undetectability and later verifiability.
  • Measuring performance under stronger adversarial edits would clarify how far the robustness extends in practice.

Load-bearing premise

Anchoring the prompt and bridge context will cause the language model to compensate for excluded tokens without creating detectable patterns or lowering text quality.

What would settle it

Generate steganographic text with the anchored sliding window, apply small edits such as synonym swaps or punctuation changes, then measure whether the hidden message can still be extracted at a significantly higher rate than with the unanchored baseline.

Figures

Figures reproduced from arXiv: 2604.09066 by Ruiyi Yan, Shiao Meng, Yugo Murawaki.

Figure 1
Figure 1. Figure 1: Steganographic extraction is fragile to active [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Comparison between the context window of our ASW and that of other methods. Each row corresponds [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: In ASW, the bridge context replaces the ex [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: An overview of our proposed distillation framework. All the parameters of the language model are not [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Average values of various metrics when the length of the soft bridge context [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Average values of various metrics under WinStega or our proposed ASW framework equipped with [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Compared to WinStega, rates of change of various metrics when the scale of the language model varies. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: An overview of the distillation framework based on LoRA. All the parameters of the language model [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
read the original abstract

Linguistic steganography based on language models typically assumes that steganographic texts are transmitted without alteration, making them fragile to even minor modifications. While previous work mitigates this fragility by limiting the context window, it significantly compromises text quality. In this paper, we propose the anchored sliding window (ASW) framework to improve imperceptibility and robustness. In addition to the latest tokens, the prompt and a bridge context are anchored within the context window, encouraging the model to compensate for the excluded tokens. We formulate the optimization of the bridge context as a variant of prompt distillation, which we further extend using self-distillation strategies. Experiments show that our ASW significantly and consistently outperforms the baseline method in text quality, imperceptibility, and robustness across diverse settings. The code is available at github.com/ryehr/ASW_steganography.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes the Anchored Sliding Window (ASW) framework for linguistic steganography. It anchors the initial prompt and a distilled bridge context (formulated via prompt distillation, extended with self-distillation) within the language model's sliding context window to encourage compensation for excluded tokens, aiming to improve robustness to modifications while preserving text quality and imperceptibility over standard sliding-window baselines. Experiments are claimed to show consistent outperformance in quality, imperceptibility, and robustness across diverse settings, with code released.

Significance. If the results and mechanism hold, the work could address a key limitation in LM-based steganography by reducing the quality penalty of context-window restrictions, offering a practical path to more robust covert communication. The prompt-distillation approach for bridge context and code availability are positive for reproducibility and potential follow-up.

major comments (2)
  1. [Abstract and Experiments section] Abstract and Experiments section: The central claim of 'significant and consistent' outperformance over the baseline in text quality, imperceptibility, and robustness is load-bearing but unsupported by any concrete metrics, baseline descriptions, statistical tests, or ablation results in the abstract (and apparently the experimental reporting). Without these, it is impossible to evaluate effect sizes or isolate whether gains derive from the anchoring mechanism itself.
  2. [Method section (bridge context optimization)] Method section (bridge context optimization): The weakest assumption—that anchoring the prompt plus distilled bridge context reliably induces the LM to compensate for excluded tokens without detectable artifacts or quality loss—is central to the robustness and imperceptibility claims yet lacks supporting analysis, examples of semantic dependency transfer, or controls showing the bridge captures more than surface statistics. Aggregate wins do not rule out alternative explanations for the reported gains.
minor comments (1)
  1. [Abstract] The abstract would benefit from including one or two key quantitative results (e.g., perplexity or detection-rate deltas) to strengthen the summary of contributions.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive feedback on our work. We address each major comment below with clarifications from the manuscript and proposed revisions to strengthen the presentation of results and mechanisms.

read point-by-point responses

  1. Referee: [Abstract and Experiments section] Abstract and Experiments section: The central claim of 'significant and consistent' outperformance over the baseline in text quality, imperceptibility, and robustness is load-bearing but unsupported by any concrete metrics, baseline descriptions, statistical tests, or ablation results in the abstract (and apparently the experimental reporting). Without these, it is impossible to evaluate effect sizes or isolate whether gains derive from the anchoring mechanism itself.

    Authors: We acknowledge that the abstract summarizes findings at a high level without numerical values, consistent with typical length constraints. The Experiments section reports concrete comparisons against the standard sliding-window baseline using metrics for text quality (perplexity and human evaluation scores), imperceptibility (detection accuracy under steganalysis), and robustness (performance under token deletions, substitutions, and edits), with results aggregated across multiple language models and settings. To address the concern directly, we will revise the abstract to include key quantitative improvements (e.g., relative gains in each metric) and expand the Experiments section with statistical significance tests and targeted ablations that isolate the anchoring and bridge-distillation components from other factors. This will make effect sizes and mechanistic contributions more transparent. revision: yes

  2. Referee: [Method section (bridge context optimization)] Method section (bridge context optimization): The weakest assumption—that anchoring the prompt plus distilled bridge context reliably induces the LM to compensate for excluded tokens without detectable artifacts or quality loss—is central to the robustness and imperceptibility claims yet lacks supporting analysis, examples of semantic dependency transfer, or controls showing the bridge captures more than surface statistics. Aggregate wins do not rule out alternative explanations for the reported gains.

    Authors: We agree that additional mechanistic evidence would strengthen the claims. The bridge context is obtained via prompt distillation (extended with self-distillation) specifically to encode information from excluded tokens that the model needs for coherent generation. In the revision we will add qualitative examples showing how the anchored bridge enables the model to recover semantic dependencies (such as entity references or logical continuations) that would otherwise be lost. We will also include control experiments contrasting the distilled bridge against surface-level alternatives (e.g., n-gram or keyword-based contexts) to demonstrate that gains arise from deeper semantic capture rather than superficial statistics. These additions will help rule out alternative explanations while preserving the core optimization approach. revision: yes

Circularity Check

0 steps flagged

No circularity; empirical proposal with independent experimental validation

full rationale

The paper presents an engineering improvement (anchored sliding window with prompt/bridge anchoring and distillation-based optimization) whose central claims rest on experimental comparisons to a baseline across quality, imperceptibility, and robustness metrics. No derivation chain, uniqueness theorem, or fitted parameter is presented as a first-principles result that reduces to its own inputs by construction. The formulation of bridge-context optimization as a prompt-distillation variant is an explicit modeling choice, not a self-referential definition or renamed known result. Self-citations, if present, are not load-bearing for the core claims.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on standard language model generation assumptions and empirical validation of the new framework; no free parameters or invented entities are explicitly detailed in the abstract.

axioms (1)
  • domain assumption Language models generate coherent text conditioned on a context window that can be selectively anchored.
    Invoked in the description of how the anchored sliding window encourages compensation for excluded tokens.

pith-pipeline@v0.9.0 · 5442 in / 1101 out tokens · 40037 ms · 2026-05-10T17:11:03.706076+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages · 1 internal anchor

  1. [1]

    Qwen2.5 Technical Report

    Provably secure steganography.IEEE Trans- actions on Computers, 58(5):662–676. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations. Fred Jelinek, Robert L Mercer, Lalit R Bahl, and Jame...

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  2. [2]

    [CONTEXT TRUNCATED]\n

    Neural linguistic steganography. InProceed- ings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Inter- national Joint Conference on Natural Language Pro- cessing (EMNLP-IJCNLP), pages 1210–1215, Hong Kong, China. Association for Computational Linguis- tics. A Imperceptibility of LM-based Steganography Following the ...

    work page 2019