{"id":"4efdde70-0ef7-4e52-a60e-165e125bb8ba","arxiv_id":"1909.01496","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Arithmetic coding with GPT-2 produces cover text that matches the language model's own distribution (KL about 4e-8 nats) and is rated by humans as plausible, enabling near-exact-distribution linguistic steganography.","lead":"This paper hides a secret message inside ordinary-looking text generated by a large language model, using arithmetic coding to map random bits into words. It matters because it shows AI-written sentences can carry hidden data while still appearing natural to human readers.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Security claim is not established: the near-zero KL only shows q matches GPT-2, not natural language, so the abstract's 'preserving security' overstates what is measured.","rationale":"The reader's weakest assumption identifies the same gap. The paper is transparent about using DKL(q||pLM) in lieu of DKL(q||Ptrue) and even states the remaining gap is outside scope, but the abstract and conclusion do not carry that caveat. The 4e-8 KL is largely a constructional identity, so it cannot serve as independent evidence of security. Figures 3 and 4 further show the two advertised properties, low KL and human plausibility, are realized at different operating points, so the 'at the same time' framing is not demonstrated. These are limitations rather than internally inconsistent steps; the arithmetic-coding construction and its implementation are sound. A steganalysis test against a discriminator would settle whether the concern lands. The conditional verdict remains appropriate.","tokens_in":9511,"tokens_out":11798,"duration_ms":124718,"concrete_test":"Train a held-out steganalysis classifier to distinguish unmodulated arithmetic-coded cover sentences (tau=1, k=50256) from human-written next sentences matched on the same CNN/DM contexts. If the AUROC's 95% confidence interval excludes 0.5, then DKL(q||pLM) about 0 does not transfer to DKL(q||Ptrue) about 0, and the security claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise is that a small DKL(q||pLM) implies security against a realistic statistical adversary. Section 4.2 defines the security metric as DKL(q||pLM) because Ptrue is unavailable, and explicitly notes 'the remaining gap is a function of language model performance, which is not the focus of this work.' This makes the abstract's 'preserving security' claim conditional on GPT-2 being nearly equal to the true distribution of natural language, which is never measured. The 4e-8 nats result is largely a constructional identity: arithmetic coding with pLM is designed to produce q close to pLM, so it cannot by itself establish that an eavesdropper cannot distinguish cover text from human text. The problem is compounded by the tradeoff in Figures 3 and 4: the near-zero KL is obtained at tau=1, k=50256, while the settings that fool humans (tau=0.4, about 1 bit/word) have non-negligible KL. The paper does not demonstrate a single configuration that simultaneously matches the generator distribution and fools human readers.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a generation-based linguistic steganography method built on arithmetic coding with a pretrained language model, specifically GPT-2. The sender treats a uniformly random secret bit string as a binary fraction and maps it to a sequence of tokens by repeatedly reading intervals defined by the model's conditional distributions; decoding inverts the mapping. The authors compare this approach to Huffman- and block-based steganographic baselines, report KL divergence between the generated distribution q and the language model distribution pLM, and run an Amazon Mechanical Turk study of human plausibility of generated sentences conditioned on news article contexts. The central reported result is that unmodulated arithmetic coding (tau=1, k=50256) yields DKL(q||pLM) around 4e-8 nats, which the paper interprets as near-perfect statistical security by matching the language model distribution.","tokens_in":9627,"tokens_out":4730,"duration_ms":47422,"significance":"The paper's core technical contribution is sound and useful: arithmetic coding is the correct construction for making a steganographic cover distribution match a chosen model distribution, and the near-machine-epsilon KL value is a strong implementation check that the scheme behaves as theory predicts. The comparison against Huffman and block baselines under the same GPT-2 model is informative, and the release of code and a demo is valuable. However, the security claim is scoped too broadly: the measured KL is an internal consistency check between q and the very model used to generate, not evidence about DKL(q||Ptrue). The authors explicitly acknowledge this limitation in Section 4.2, but the abstract and conclusion use language such as 'preserving security' that overstates the evidence. The paper is therefore a valuable demonstration of a construction and its properties, rather than a demonstration that the generated text is undetectable in the true distribution of natural language.","major_comments":[{"comment":"The abstract claims the method 'preserving security by matching the cover message distribution with the language model distribution,' but the only statistical evidence is DKL(q||pLM) where pLM is the same GPT-2 model used to construct q. Arithmetic coding is designed by construction to drive q toward pLM, so a KL of 4e-8 nats is an internal consistency check, not a measure of security against natural language. The paper acknowledges in Section 4.2 that 'pLM might not match the distribution of natural language' and that the remaining gap is 'a function of language model performance,' but the framing of the headline result should be corrected. Please either (a) explicitly scope all security claims to 'security relative to the chosen language model distribution,' or (b) add an independent evaluation, such as a steganalysis classifier trained on human-written text versus generated text, or a KL estimate against a different large language model than the one used for encoding.","section":"Section 4.2 and Abstract"},{"comment":"No single operating point is shown to achieve both near-zero KL and high human plausibility. The unmodulated configuration with tau=1 and k=50256, which gives the 4e-8 nats KL, is not evaluated in the human study; the human evaluation uses k=300 with temperatures 0.4, 0.7, 1.0, and 1.2, and the configuration that is most human-plausible (tau=0.4, around 1 bit/word) has non-negligible KL on the curve in Figure 3. Thus the conclusion that the system 'can fool both statistical adversaries who analyze the full generative distribution, and human adversaries who analyze a single sample' is not supported by a single reported configuration. The authors should either report human evaluations for the unmodulated configuration, report KL for the human-evaluated configurations in a directly comparable way, or explicitly state that the two properties are demonstrated at different operating points and the system can be tuned between them.","section":"Figures 3 and 4"},{"comment":"The claim that 'at 1 bit/word both methods lead to cover texts statistically indistinguishable from human-written sentences' is not backed by any reported statistical test. The MTurk protocol yields binary judgments, and the paper reports only the mean human judgment score per condition. Without confidence intervals, a significance test against the true-data condition, or a model-based comparison, the phrase 'statistically indistinguishable' is unsupported. Please provide the statistical analysis, or rephrase this claim as a descriptive statement about the mean scores.","section":"Section 4.3"}],"minor_comments":[{"comment":"There is a typo in the sentence 'Traditionally, most practical stenography systems are edit-based'; 'stenography' should be 'steganography'.","section":"Section 1"},{"comment":"The sentence 'yields a distribution q = ps or equivalently DKL(q||ps) = 0 for long sequences' conflates exact equality with asymptotic equality; arithmetic coding with finite precision has small but nonzero quantization error, so the equality holds only in the limit. Please clarify the asymptotic nature of the claim.","section":"Section 3"},{"comment":"The qualitative examples are helpful, but some generated cover texts contain obvious factual errors or artifacts (e.g., 'kim ju-ha-teh' in Table 1). Since the human evaluation is central to the paper's claims, a sentence or two discussing how often such artifacts occur and whether they affect the human plausibility judgments would strengthen the presentation.","section":"Section 4.4 and Table 1"}],"recommendation":"major_revision","confidential_remarks":"The main risk for publication is the gap between the paper's security language and what is actually measured: DKL(q||pLM) is a property of the construction, not of the true language distribution. The authors should be asked to either add an independent-detector experiment or visibly restrict all security claims to the model-relative sense. The paper is otherwise a solid, reproducible systems contribution, but the title and abstract currently promise more than the experiments deliver."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this is a competent engineering paper that takes arithmetic-coding steganography (Sallee 2004) and hooks it up to a 345M-parameter GPT-2. The theory is prior work; the contribution is the application, the reimplemented baselines, and the human evaluation. It's worth reading and citing for those working in generation-based steganography. Just don't buy the abstract's 'preserving security' claim as stated.\n\nWhat the paper does well: the arithmetic-coding implementation is clean, the KL of 4e-8 nats against the unmodulated GPT-2 is a nice confirmation that the coder matches the model distribution, and the code is released. The baselines are reimplemented fairly, and the human study on context-conditioned completion is a reasonable proxy for plausibility. The paper is honest about the big caveat: DKL(q||pLM) is not the same as DKL(q||Ptrue), and Section 4.2 says the remaining gap depends on language-model quality. That is the right thing to do, and it is more than many steganography papers do.\n\nThe soft spots: the abstract and introduction frame the result as 'preserving security,' which oversells what is measured. The near-zero KL only shows that the cover distribution matches the particular language model used for encoding; it says nothing about whether an adversary who knows natural-language statistics (or uses a better model) could detect the cover text. That is an external-validity issue, not a math error. Second, Figures 3 and 4 together show a real tradeoff: the KL is near zero only at high bit rates with τ=1 and no truncation, while the settings that fool human raters (τ=0.4, about 1 bit/word) have non-negligible KL. The paper acknowledges this, but a reader could miss it. Third, the human-evaluation claim that some cover texts are 'statistically indistinguishable' from human sentences is not backed by shown error bars or an equivalence test; it looks like the usual 'no significant difference' reading, which is weaker.\n\nOverall: the engineering is solid, the framing needs work, and the paper deserves a serious referee. For the revision, I'd ask for a security section that clearly separates internal consistency from true statistical security, plus proper confidence intervals on the human judgments. If the authors fix those, this becomes a useful reference for the subfield.","headline":"A competent engineering paper that applies arithmetic-coding steganography to GPT-2 cleanly, but the abstract's 'preserving security' overstates what the evidence actually supports.","tokens_in":10221,"tokens_out":3547,"would_cite":true,"duration_ms":34299,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Arithmetic coding with a language model makes hidden messages statistically indistinguishable from the model's own text.","keywords":["linguistic steganography","arithmetic coding","neural language models","information hiding","Kullback-Leibler divergence","text generation","steganalysis","covert communication"],"falsifier":"Train a binary classifier on long samples of genuine human-written news text versus cover text produced by the unmodulated arithmetic-coding system, held out from the language model's training data; if the classifier's accuracy exceeds chance with high confidence, the near-zero KL to the language model is not translating into indistinguishability from natural language.","tokens_in":9240,"feed_emoji":"🕵️","tokens_out":6862,"duration_ms":63346,"temperature":0.7,"pith_summary":"This paper aims to show that a classical compression algorithm, arithmetic coding, when run in reverse over the next-word probabilities of a strong neural language model, turns a secret bit string into natural-looking text while making the generated text statistically indistinguishable from the model's own output. If true, this closes the gap between edit-based linguistic steganography, which hides little data, and generation-based schemes, which are detectable in principle. The authors report a measured Kullback-Leibler divergence of about $4\\times 10^{-8}$ nats between the cover distribution and the language model distribution at full modulation, and human raters accept the generated next sentences about as often as true next sentences at low compression. The practical payoff is a concrete, efficient way to conceal the existence of a message inside ordinary prose.","feed_headline":"Hidden messages become statistically invisible in plain text","feed_subtitle":"Arithmetic coding makes cover text match a language model to within 4e-8 nats of divergence, and readers can't tell.","key_machinery":"The mechanism is arithmetic coding on $[0,1)$: the secret message is a uniform bit string read as a binary fraction, and at each generation step the language model's conditional distribution partitions the current interval into token bins; the cover text is the sequence of bins containing the fraction. Decoding reverses the walk to recover the bits. The paper adds two knobs, temperature $\\tau$ and top-$k$ truncation, to modulate the distribution away from the exact $p_{\\mathrm{LM}}$ for quality. The load-bearing identity is the optimality result for arithmetic steganography: for long sequences this procedure gives $q = p_s$, so $H(q) = H(p_s)$, meaning the encoding is simultaneously entropy-optimal and distribution-matching.","core_discovery":"The central claim is that generation-based linguistic steganography can simultaneously achieve near-perfect statistical security and human-level fluency by using arithmetic coding as the invertible map between uniform message bits and cover text. With the unmodulated language model, the induced cover distribution $q$ has KL divergence $4\\times 10^{-8}$ nats from $p_{\\mathrm{LM}}$, which is effectively zero and consistent with the theory that arithmetic coding yields $q = p_{\\mathrm{LM}}$. With temperature and top-$k$ truncation, the method trades a small amount of this statistical fidelity for more conservative generations; crowdsourced evaluators then rate its one-bit-per-word output statistically indistinguishably from human-written next sentences. The same construction also lets a natural-language message be compressed to bits and re-encoded in unrelated cover text.","pith_inferences":["I would not equate $D_{\\mathrm{KL}}(q\\|p_{\\mathrm{LM}})$ with $D_{\\mathrm{KL}}(q\\|P_{\\mathrm{true}})$: if a steganalyst uses a language model better than the sender's, the near-zero KL to the sender's model would not protect the message; an explicit cross-model test would decide whether the practical security claim extends beyond a fixed model.","The same arithmetic-coding construction could be turned into a public benchmark for linguistic steganography by holding out a strong language model and testing whether any detector can separate samples of $q$ from human text; the current $4\\times 10^{-8}$ nats result suggests such a benchmark would need very sensitive detectors.","The text-to-text variant (message text to bits to cover text) points toward untraceable messaging over ordinary channels, since the cover text carries no obvious link to the intended content."],"forward_implications":["A statistical adversary who knows only the sender's language model cannot distinguish steganographic cover text from the model's own samples, because the divergence is measured at $4\\times 10^{-8}$ nats.","The hidden-message capacity is essentially the entropy of the cover distribution, so encoding is nearly as efficient as the language model's own token prediction, around 1 to 5 bits per word across the settings tested.","Human readers, even with a news article as context, accept the generated next sentences about as often as true human-written next sentences when the compression is around 1 bit per word.","Because arithmetic coding is invertible, natural-language messages can first be compressed to uniform bits and then re-expressed as unrelated cover text, so arbitrary secret text, not just random bits, can be sent this way.","Any future improvement in language modeling directly improves both the statistical fidelity and the fluency of the hidden channel, since the coding mechanism is already at its theoretical optimum."],"supporting_citations":[{"why":"Defines steganographic security as $D_{\\mathrm{KL}}(q\\|P_{\\mathrm{true}})$ and frames the sender/receiver/invertible mapping model used throughout.","marker":"Cox et al. (2005)"},{"why":"Proves that arithmetic coding for steganography yields a cover distribution equal to the target distribution, the theoretical basis for the near-zero KL claim.","marker":"Sallee (2004)"},{"why":"Introduces arithmetic coding as an entropy-optimal scheme whose reversal forms the encoding and decoding procedure.","marker":"Rissanen and Langdon (1979)"},{"why":"Provides the fixed-precision binary fraction implementation that makes the method practical.","marker":"Rubin (1979)"},{"why":"Supplies the 345M-parameter pretrained language model whose distribution defines the cover distribution and the KL target.","marker":"Radford et al. (2019)"},{"why":"Supplies the block-based baseline that the paper reimplements on the strong language model for comparison.","marker":"Fang et al. (2017)"},{"why":"Supplies the Huffman-coding baseline that competes on human plausibility.","marker":"Yang et al. (2019)"},{"why":"Provides the CNN/Dailymail news contexts used for conditioned generation and evaluation.","marker":"Hermann et al. (2015)"}],"fun_headline_variants":["Arithmetic coding hides secrets in fluent text","Neural stego: text that looks normal, hides bits","Statistically invisible steganography via language models","Language models conceal messages in plain sight","Arithmetic coding makes stego text match AI's own style"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The security claim collapses if the pretrained language model's distribution differs detectably from the true distribution of natural language, because the paper measures and optimises only closeness to the model, not to human text.","fun_headline_variants_meta":{"raw":{"variants":["Arithmetic coding hides secrets in fluent text","Neural stego: text that looks normal, hides bits","Statistically invisible steganography via language models","Language models conceal messages in plain sight","Arithmetic coding makes stego text match AI's own style"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000234,"raw_usage":{"total_tokens":1433,"prompt_tokens":816,"completion_tokens":617,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":432,"completion_tokens_details":{"reasoning_tokens":543}},"tokens_in":432,"tokens_out":617,"duration_ms":6049,"temperature":1.0,"reasoning_tokens":543,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:16:01.460363+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a binary classifier on long samples of genuine human-written news text versus cover text produced by the unmodulated arithmetic-coding system, held out from the language model's training data; if the classifier's accuracy exceeds chance with high confidence, the near-zero KL to the language model is not translating into indistinguishability from natural language.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines steganographic security as $D_{\\mathrm{KL}}(q\\|P_{\\mathrm{true}})$ and frames the sender/receiver/invertible mapping model used throughout."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces arithmetic coding as an entropy-optimal scheme whose reversal forms the encoding and decoding procedure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the fixed-precision binary fraction implementation that makes the method practical."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Huffman-coding baseline that competes on human plausibility."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CNN/Dailymail news contexts used for conditioned generation and evaluation."}],"review_version":1}