{"id":"0a961809-fd8d-4051-8a4e-8922c17d4c2e","arxiv_id":"2412.11043","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A new LLM-based steganography framework encodes secret bits in the choice of entities mentioned in generated text, claiming higher capacity and robustness, but its indistinguishability claim is not supported and is contradicted by its own metrics.","lead":"This paper proposes hiding secret messages inside AI-generated text by choosing which real-world entities (people, places, times) the sentence mentions, rather than by tweaking individual words. The approach survives heavy text corruption better than older token-level methods, but the paper's claim that the hidden messages are statistically indistinguishable from ordinary AI text is contradicted by its own quality measurements.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The DKL=0 security argument in Sec 2.2 is invalid because generation is conditioned on sampled entity types and filtered by a check/regenerate loop; the paper's own Table 1 shows stegos have ~57% lower perplexity than covers, indicating detectable distribution shift.","rationale":"The reader's weakest assumption was that the empirical semantic distribution built from LCCC/UIE matches the actual entity-type distribution of ChatGLM covers and that extraction reliably recovers the entity set. My concern is adjacent but more direct: even if the type distribution were perfectly matched, the conditional generation process (prompting plus check/regenerate) changes the output distribution, so DKL=0 cannot hold. The paper's own Table 1 provides a measurable symptom of this shift: stegos have substantially lower PPL than covers. This is a stronger, internally sourced piece of evidence than the corpus-mismatch concern, because it does not depend on external assumptions about LCCC or the UIE model. It nevertheless supports the same bottom line: the paper does not establish security, and its own measurements undermine the central claim. A concrete distributional comparison (PPL plus entity-type frequencies) would settle the question. My recommendation is UNCHANGED: the REJECT verdict remains appropriate, though I would emphasize this internal contradiction rather than only the unvalidated empirical distribution.","tokens_in":12477,"tokens_out":4562,"duration_ms":42927,"concrete_test":"Using the paper's exact pipeline (ChatGLM2-6B, temperature 0.8, top-p 0.8), generate 1,000 stegos from random secret bitstreams and 1,000 covers by unconditional random sampling. Compute for each text (i) PPL with the same tokenizer and (ii) the entity-type vector using the paper's UIE extraction. Then run a two-sample Kolmogorov-Smirnov test on PPL and a chi-square/permutation test on type frequencies. If the null of identical distributions is rejected at p<0.01, or if the estimated DKL is significantly above 0, the Sec 1 claim of indistinguishability fails. Additionally, train a simple logistic-regression warden on these two features and report AUC; AUC near 1 would confirm the stegos are detectable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2.2 argues that DKL(p(cover)||p(stego))=0 because \"our method does not alter the sampling strategy of LLM.\" This is not correct. In Secs 2.3-2.4 the generator is prompted with a specific sampled entity type, and a Check Agent rejects/regenerates outputs until they satisfy that type (including prohibiting extra entities). Prompting and rejection sampling are alterations of the sampling strategy; the effective distribution is p(stego)=p(accepted|type)*p(type), which is not the LLM's unconditional distribution p(cover). The paper's own Table 1 quantifies the shift: Ours has PPL 869.79 (6B) versus 2027.34 for RS covers, a 57% reduction. That reduction is a selection artifact of the feedback loop, not evidence of indistinguishability. A warden measuring PPL or entity-type frequencies can therefore distinguish stegos from covers. No warden experiment, DKL estimate, or type-distribution comparison is provided. Since indistinguishability is the basis of the security claim, the central claim is unsupported and contradicted by the paper's own measurements.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a semantic steganography framework in which secret bits are mapped, via arithmetic coding over an ontology-entity tree, onto entity-type classes drawn from an empirical semantic distribution. An LLM generation agent is prompted to produce text containing the sampled entities, a check agent rejects or regenerates outputs that fail the required type, and an extraction agent recovers the entity type at decoding time. The authors claim that the method achieves semantic indistinguishability (DKL = 0), higher embedding capacity than token-level steganography baselines, and robustness against both token-level corruption and semantic-preserving attacks such as paraphrasing and semantic communication.","tokens_in":12740,"tokens_out":4757,"duration_ms":46121,"significance":"If the indistinguishability claim were valid, the framework would be a meaningful step toward high-capacity covert communication over LLM-generated text, because it moves embedding from the token level to the semantic level and explicitly targets robustness in noisy channels. The paper's practical contributions include a concrete entity-tree construction pipeline, a feedback-based rejection-sampling mechanism, and a broad set of robustness experiments covering insert/delete/replace/swap attacks, paraphrasing, and semantic communication. These are useful and worth documenting. However, the central security claim is not established: the DKL = 0 argument in Sec. 2.2 is invalid, the paper's own Table 1 shows a large distribution shift between covers and stegos, and no warden or statistical-indistinguishability experiment is provided. The robustness results are also partially circular because decoding uses the same extractor and ontology tree as encoding. The contribution is therefore better characterized as a practical robust semantic embedding scheme than as a secure steganographic system, and the current claims outrun the evidence.","major_comments":[{"comment":"The claim that DKL(p(cover)||p(stego)) = 0 because 'our method does not alter the sampling strategy of LLM' is not supported. In Secs. 2.3 and 2.4, the generation agent is conditioned on a sampled entity type, and the check agent rejects or regenerates outputs until they satisfy that type, including prohibiting extra entities. The effective output distribution is therefore p(stego) = p(accepted | type) * p(type), not the unconditional cover distribution p(cover). Table 1 confirms a substantial shift: Ours has PPL 869.79 versus 2027.34 for RS covers, and GPT-4 rationality score 7.3624 versus 5.6419. The paper's own explanation attributes this to the check-and-regenerate loop. Since no warden experiment, no DKL estimate, and no comparison of entity-type frequencies between covers and stegos are provided, the central indistinguishability claim is contradicted by the paper's own measurements.","section":"Sec. 2.2 and Table 1"},{"comment":"The empirical semantic distribution p(C(T)) is estimated from LCCC using the UIE extractor, but the cover texts in the experiments are generated by ChatGLM2-6B and ChatGLM2-6B-int4. The manuscript provides no validation that the entity-type distribution of ChatGLM-generated covers matches the LCCC-derived p(C(T)). If this distribution is mismatched, the arithmetic-coding intervals no longer correspond to actual cover frequencies, which breaks both the security argument and the reliability of decoding. The authors should report corpus statistics (number of types, type coverage, smoothing), extraction accuracy on LCCC, and a direct comparison between p(C(T)) and the empirical type distribution of generated covers.","section":"Sec. 2.2, Algorithm 1"},{"comment":"The robustness results are partly circular because decoding relies on the same UIE-based extraction agent and the same ontology-entity tree used during encoding. High decoding success under token corruption may reflect the extractor's tendency to output the intended entities even from garbled text, rather than a property of the semantic channel itself. The paper does not report extraction precision/recall on clean or attacked stegos, nor does it compare against an independent extractor or human annotation. Without this, it is unclear whether the reported success rates in Table 3 measure preservation of the embedded semantic signal or artifacts of the decoding pipeline.","section":"Sec. 2.4 and Table 3"},{"comment":"The embedding-rate comparison is not apples-to-apples. For Ours, the reported bits are derived from arithmetic coding over entity-type intervals in the semantic tree, whereas AC, METEOR, and DISCOP embed bits at the token level. The manuscript does not specify the message-length distribution, the exact bit-counting convention, or the failure criterion used for each method. Table 2 also reports 'bit/tok' for RS covers without any embedding mechanism. To substantiate the higher-capacity claim, the authors should state precisely how each bit rate is computed, report bit rates conditioned on successful decoding, and use a common message-generation protocol across all methods.","section":"Sec. 3.1, Tables 1 and 2"}],"minor_comments":[{"comment":"The sentence 'Experiments showe that our framework are robust' contains a typo: 'showe' should be 'show', and 'framework are' should be 'framework is'.","section":"Sec. 4"},{"comment":"The notation in Eq. (4) uses M without defining it; the entity index should be |E| for consistency with the definition of E and T.","section":"Sec. 2.1, Eq. (4)"},{"comment":"The pseudocode line 'Assign probabilities to entities' is not specified in the algorithm; the paper should state how p(C(T)) is computed from the LCCC counts, including any smoothing or pruning of rare types.","section":"Sec. 2.2, Algorithm 1"},{"comment":"The label 'Influent?' in the Check Agent box is unclear; it appears to be a typo and should be replaced with a meaningful criterion such as 'Influent?' or 'Fluent?'.","section":"Fig. 2"},{"comment":"The references for RNN-Stega (Yang et al., 2019a and 2019b) and VAE-Stega (Yang et al., 2021a and 2021b) are duplicated; the duplicate entries should be removed.","section":"References"},{"comment":"Tables 1 and 2 report no variance or number of samples; the authors should add standard deviations or confidence intervals so that the differences between methods can be evaluated statistically.","section":"Sec. 3.1"}],"recommendation":"reject","confidential_remarks":"The stress-test concern lands: the DKL = 0 claim in Sec. 2.2 is invalid and is directly contradicted by the PPL and GPT-4 score differences in Table 1. The paper may contain useful practical ideas about entity-based robust semantic embedding, but the current version overclaims security and provides no warden experiment or distribution-fitting validation. A resubmission would need to reframe the contribution and add substantial experiments before it could be considered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nQuick take: this is a real attempt at semantic steganography, and the robustness numbers are worth a look, but the central security argument is not just weak—it is contradicted by the paper's own Table 1. The DKL=0 claim in Sec 2.2 assumes the LLM's sampling strategy is untouched, yet Sec 2.3 explicitly uses rejection sampling through a check agent. Generation is conditioned on sampled entity types and filtered for compliance, so the stego distribution necessarily differs from the cover distribution. The 57% lower perplexity of stegos (869 vs 2027) is a measurable shift, not evidence of indistinguishability. No warden experiment is reported, and no empirical DKL estimate is given. That is the load-bearing flaw.\n\nWhat is genuinely new: the ontology-entity tree with an empirical entity distribution fitted on LCCC, the feedback CoT loop, and the LLM extraction agent. The robustness results in Table 3 are the paper's real strength. Under random token edits, paraphrase, and semantic communication, decoding success is far above AC, and the BertScore correlation in Fig 3 gives a plausible mechanism. That is a credible engineering contribution.\n\nThe soft spots are in proportion. The capacity comparison in Table 2 reports raw ER and MSR separately; the effective throughput after accounting for regeneration loops is not presented, and the 'higher capacity' claim is overstated. No code or data are released. The semantic distribution is fitted on a different corpus and never validated against ChatGLM's cover distribution; the same UIE extractor is used to build the tree and to decode, so the robustness may partly reflect the extractor's tendency to recover intended entities from garbled text rather than a property of the language channel. That circularity is worth flagging but not fatal.\n\nWho this is for: anyone working on linguistic steganography or LLM-based covert channels. The paper deserves a serious referee, not a desk reject. It will need major revision: replace the DKL=0 claim with an actual distributional comparison, run a warden, validate the entity distribution on the target model, and clean up the capacity accounting.\n\nRecommendation: send it to peer review with the expectation of heavy revision, not acceptance as-is.","headline":"A concrete semantic-steganography pipeline with a real robustness result, but the indistinguishability claim is contradicted by its own perplexity numbers.","tokens_in":13286,"tokens_out":2421,"would_cite":false,"duration_ms":20482,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that secret messages can be embedded in the entities an LLM is prompted to mention, so the stego text is statistically identical to ordinary LLM output and survives paraphrasing and token noise.","keywords":["linguistic steganography","large language models","semantic steganography","ontology-entity tree","arithmetic coding","chain-of-thought feedback","covert communication","entity extraction"],"falsifier":"A direct test: run the framework's embedding on ChatGLM, then measure the KL divergence between the empirical distribution of entity types in the generated stegos and the distribution predicted from the corpus; if it is notably different from zero, the stegos are statistically distinguishable from covers. A second, sharper test: take paraphrased stegos that the paper reports as failures and check whether the failed decodes are exactly those where the extraction agent recovered a different entity set, which would locate the failure in extraction rather than in encoding.","tokens_in":12238,"feed_emoji":"🤫","tokens_out":6320,"duration_ms":50003,"temperature":0.7,"pith_summary":"Generative steganography hides secret bits in the text an LLM produces. Existing token-level methods such as arithmetic coding embed in the choice of words, but the entropy of modern LLM output is low, so capacity shrinks and any token corruption destroys the message. This paper proposes to embed instead at the level of semantics: the sender samples a set of entities (person, location, time) according to an estimated distribution, instructs the LLM to generate a sentence mentioning exactly those entities, and the receiver recovers the entities with an extraction model to reconstruct the bits. The paper claims that this is more robust to token-level attacks and paraphrasing, achieves higher embedding rates, and produces text statistically identical to normal LLM output because the sampling strategy is unchanged.","feed_headline":"Secret bits hide in an LLM's entity choices, surviving paraphrase","feed_subtitle":"Entity-level encoding survives reworded text and token noise, where token-level steganography fails.","key_machinery":"The load-bearing object is the ontology-entity tree: a hand-built hierarchy of basic concepts (person, location, time), fine-grained subconcepts, and a leaf layer of specific entities. Sampling descends the tree with arithmetic coding over the empirical semantic distribution, so the interval of probabilities that lands on a type carries the secret bits; the tree's path labels also serve as disambiguation cues for the generating and extracting agents. The second mechanism is feedback Chain-of-Thought rejection sampling, which iteratively regenerates sentences until a check agent certifies that the entity set is exactly the sampled one.","core_discovery":"The central claim is that a covert channel can be built over the semantic content of LLM-generated text rather than its tokens. Concretely, the paper constructs an ontology-entity tree whose leaf nodes are entities like 'Las Vegas' and whose paths disambiguate meaning ('Location/Tourism Location/Washington'), estimates the empirical distribution of sentence 'types' (a multiset of entities) from a large conversation corpus, and uses arithmetic coding to map a uniformly random bit stream to a type sampled from that distribution. A generation agent with a check-agent feedback loop then produces a sentence that contains exactly the sampled entities and no others, so the bits are recoverable by a separate extraction agent. Because the ordinary LLM sampling strategy is not altered, the authors argue the Kullback-Leibler divergence between covers and stegos is zero, giving the same information-theoretic security as symbolic methods.","pith_inferences":["A warden who knows the entity tree could test entity-type frequencies against the empirical distribution, since the hidden bits force entities to appear with exactly those frequencies; this detection angle is not explored in the paper.","The framework's dependence on a fixed entity vocabulary means it inherits bias from the UIE extractor and the LCCC corpus; deploying on a different domain would require rebuilding the tree and re-estimating frequencies, offering a testable extension.","The 'DKL = 0' claim implicitly assumes the check agent's rejection loop does not bias the accepted output distribution; a refined analysis would condition on the acceptance probability and measure the divergence empirically.","The authors note the channel closes when the prompt leaves no semantic freedom (e.g., '1+1='); under such fixed-answer conditions the embedding rate falls to zero, which is consistent with the method's dependence on semantic redundancy."],"forward_implications":["Steganography over LLM APIs becomes practical in black-box settings because only prompts are used, no internal token probabilities are needed.","Text that survives paraphrasing and semantic communication channels at realistic noise levels can still carry the intended message, so the covert channel persists in social-media-like pipelines.","Because embedding rate tracks the entropy of the semantic distribution rather than token entropy, larger entity trees and richer corpora translate directly into more bits per sentence.","Stegos generated this way have lower perplexity and higher diversity than token-level steganography on the same models, because the check agent filters awkward regenerations.","The KL-divergence argument means a statistical warden who compares stego and cover distributions at the token level should not be able to distinguish them, matching the security guarantee of METEOR and DISCOP."],"supporting_citations":[{"why":"Supplies the information-theoretic security model (DKL = 0) used to claim indistinguishability.","marker":"(Cachin, 1998)"},{"why":"Arithmetic-coding generative steganography method that the framework uses and also a baseline in experiments.","marker":"(Ziegler et al., 2019)"},{"why":"METEOR, a provably secure symbolic steganography baseline the paper compares against.","marker":"(Kaptchuk et al., 2021)"},{"why":"DISCOP, another provably secure baseline and comparison point.","marker":"(Ding et al., 2023)"},{"why":"Supplies the UIE model used to extract entities for building the semantic space.","marker":"(PaddleNLP, 2021)"},{"why":"LCCC, the 12-million-dialogue corpus used to estimate the empirical semantic distribution.","marker":"(Wang et al., 2020)"},{"why":"ChatGLM, the LLM family used as generation and extraction agents in experiments.","marker":"(Du et al., 2022)"},{"why":"Semantic communication method used as an attack that preserves semantics in robustness tests.","marker":"(Xie et al., 2021)"}],"fun_headline_variants":["LLM steganography hides bits in entity choices, not tokens","Entity-tree steganography survives paraphrase and token noise","Semantic space steganography achieves high-capacity hidden messages","Steganography rides LLM meaning, not words, for robust hiding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The framework's security and decodeability rest on the assumption that the entity-type distribution estimated from the LCCC conversation corpus with the UIE extractor is the same distribution that ChatGLM-generated cover sentences actually follow, and that the extraction agent reliably recovers exactly the intended entities from clean and corrupted stegos.","fun_headline_variants_meta":{"raw":{"variants":["LLM steganography hides bits in entity choices, not tokens","Entity-tree steganography survives paraphrase and token noise","Semantic space steganography achieves high-capacity hidden messages","Steganography rides LLM meaning, not words, for robust hiding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000149,"raw_usage":{"total_tokens":1160,"prompt_tokens":882,"completion_tokens":278,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":206}},"tokens_in":498,"tokens_out":278,"duration_ms":3205,"temperature":1.0,"reasoning_tokens":206,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T15:21:54.059063+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test: run the framework's embedding on ChatGLM, then measure the KL divergence between the empirical distribution of entity types in the generated stegos and the distribution predicted from the corpus; if it is notably different from zero, the stegos are statistically distinguishable from covers. A second, sharper test: take paraphrased stegos that the paper reports as failures and check whether the failed decodes are exactly those where the extraction agent recovered a different entity set, which would locate the failure in extraction rather than in encoding.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the information-theoretic security model (DKL = 0) used to claim indistinguishability."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the UIE model used to extract entities for building the semantic space."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"LCCC, the 12-million-dialogue corpus used to estimate the empirical semantic distribution."}],"review_version":1}