{"id":"9a946ea8-3344-4d4c-86d1-c4530e0466bb","arxiv_id":"2608.12713","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A dual-signal LLM watermark co-embeds a robust and a fragile signal into every token, enabling three-state detection of intact, tampered, and unwatermarked text.","lead":"This paper introduces a way to tag AI-written text with two hidden signals: one that survives editing to show where the text came from, and one that breaks when the text is changed. The result is a detector that can tell whether AI-generated text is unchanged, tampered with, or not from that AI at all.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tamper-evidence claim is position-dependent: edits confined to the final tokens leave the fragile z-score high, so the unqualified 89.5-100% recall overstates the guarantee.","rationale":"The reader's weakest assumption already flags that edits confined to final tokens are only partially caught, and this stress-test sharpens that into a quantitative, falsifiable concern. The paper's own Section VI admits the tail-edit limitation, so this is not a hidden flaw; however, the abstract's blanket statement 'flags 89.5 to 100% of tampered texts' is position-dependent and would mislead without a qualifier. The robust signal's empirical resilience to insertions/deletions is a secondary worry, but the tamper-evidence tail failure is more directly tied to the paper's central contribution and is the most load-bearing. The proposed test would settle whether the unqualified claim is actually supported. Since the reader's CONDITIONAL verdict already accounts for this limitation, no change to the verdict is needed.","tokens_in":17536,"tokens_out":12521,"duration_ms":173907,"concrete_test":"Generate 500 watermarked texts with Cocktail (Llama-3.2-1B on C4, ratios 1:1 and 4:1) as in the paper. Apply the same sentiment-flip and token-substitution edit budgets, but restrict all edits to the final K tokens of each text, with K ∈ {5%, 10%, 20%} of the text length. Use the paper's threshold calibration (τ_r at 99th percentile of no-watermark z_r; τ_f at 1st percentile of intact z_f) and report tamper-evidence TPR@1%FPR. If TPR collapses for K=10% (e.g., below 50%) while uniformly distributed edits give the reported 89.5-100%, the headline claim must be qualified by edit position.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The fragile signal is seeded on the normalized prefix: s_f(t) = H(suffix_{n_f}(N(x_{1:t-1}))) (Eq. 2). An edit at position i changes the normalized prefix only for tokens t > i; tokens t <= i retain the original fragile seeds. The edited token at i itself contributes only one random hit, so a text with edits concentrated in the final fraction f of its length retains a fragile z-score of roughly 2(p_g - 0.5) * sqrt((1-f) T d_f), where p_g is the intact green-hit probability. For T=300, d_f=15, p_g≈0.55, and f=0.1, this gives z_f ≈ 5.5, likely above the 1% intact threshold. Thus an adversary can alter a conclusion or disclaimer in the last few tokens and the text is classified Intact. Section VI acknowledges 'edits confined to the final tokens corrupt only the tail of the fragile seeds,' but the abstract and Table I report unqualified '89.5-100%' tamper detection based on attacks that distribute edits across the text. This is load-bearing because the scheme's central novel capability—tamper evidence—fails for exactly the surgical, position-localized edits that piggyback spoofing enables, and no positional ablation or theoretical bound is provided.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes Cocktail, a generative watermark for LLM text that co-embeds a robust signal and a fragile signal into every generated token. The robust signal is seeded on a short token window and supports provenance attribution; the fragile signal is seeded on a long normalized-text suffix and is intended to expose any reader-visible edit. Embedding uses multiple rounds of unbiased tournament reweighting with a periodic robust/fragile round allocation, and detection maps the two z-scores through fixed thresholds into three states: Intact, Tampered, and No-Watermark. Experiments with Llama-3.2-1B and Gemma-3-4B on C4 and LFQA compare against KGW, Unigram, SynthID, and SIR, reporting TPR@1%FPR for attribution and tamper evidence plus perplexity. The authors also include ablations on normalization placement, co-embedding, and fragile-window length, and a measurement of the signature-based baseline Bileve.","tokens_in":17820,"tokens_out":6887,"duration_ms":77815,"significance":"If the results hold, Cocktail is a meaningful advance: it is the first scheme in this comparison to provide both provenance and tamper evidence at a 1% FPR operating point, with a mechanism that is elegant and does not rely on fitted constants. The experimental scope (two models, two datasets, five tasks), the ablations supporting the three claimed design necessities, and the honest reproduction of SynthID's unbiasedness theorems in Appendix A are strengths. However, the headline tamper-evidence recall is currently overclaimed because the fragile signal is position-dependent and the evaluation does not include localized edits; further, the reported operating points are calibrated in-sample. These issues are fixable and do not invalidate the design, but they must be addressed before the quantitative claims can be accepted.","major_comments":[{"comment":"The tamper-evidence claim is position-dependent in a way that the reported numbers do not reflect. Because s_f(t) is a hash of the normalized prefix ending at t-1 (Eq. (2)), an edit at position i changes the fragile seeds only for tokens t > i; tokens at or before i keep their original green-red assignments. Consequently, a text whose edits are concentrated in the final fraction f of its length retains a fragile z-score that grows with (1-f), so the detector can still label it Intact. The abstract and Table I report unqualified tamper-recalls of 89.5-100% based on attacks (random token substitution, sentiment flip) that distribute edits across the text, and Section VI's acknowledgment that \"edits confined to the final tokens corrupt only the tail of the fragile seeds\" does not quantify or bound this failure. Since piggyback spoofing is exactly a small, localized word substitution, this is load-bearing: the paper should provide a worst-case bound on z_f as a function of the edit position/fraction, or report a positional ablation (e.g., edits in the last 5%, 10%, 20% of tokens) and restrict the stated guarantee accordingly.","section":"Section IV.B, Eq. (2); Section V.A/Table I; Section VI"},{"comment":"The reported TPR@1%FPR values are obtained with thresholds calibrated on the same data used for the reported rates: tau_r is the 99th percentile of no-watermark z_r and tau_f the 1st percentile of intact z_f. This in-sample calibration can bias the point estimates upward, and Table I reports no confidence intervals or variability measures. The central quantitative claim (89.5-100% tamper detection while maintaining attribution) needs out-of-sample or cross-validated threshold selection, and the key TPRs should be reported with bootstrap or exact binomial confidence intervals.","section":"Section V.B and Appendix B"}],"minor_comments":[{"comment":"There is a definitional inconsistency in the seeding window: Eq. (2) defines the fragile seed as H(suffix_{n_f}(N(x_{1:t-1}))), i.e., a character window over the normalized prefix without the current token, while Table III states the fragile window is \"full normalized prefix + self token\" and the robust window is \"1 preceding token + self token\". Please clarify whether the current token is included in the hash input, since this affects the statistical independence of the hits and the exact meaning of the z-scores.","section":"Section IV.B and Table III"},{"comment":"Figure 4's caption says thresholds are \"calibrated at a 1% tail\", while Appendix B specifies tau_f at the 1st percentile of intact z_f; please state explicitly whether the thresholds are one-sided or two-sided, and report the exact FPR on held-out data.","section":"Appendix B and Figure 4"},{"comment":"The sentence \"The TPR@1%FPR of each task is the complement of the attacker's success rate\" is only true under a binary decision with a fixed false-alarm budget; consider rephrasing to avoid overstating the relationship.","section":"Section V.B"},{"comment":"The sentiment-flip evaluation is scored over 200 tokens while all other tasks are scored over the first 300 tokens; this difference should be stated in the Table I caption or in the evaluation protocol so that the columns are comparable.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest about relying on SynthID's theorems, and the Bileve comparison is thorough. The main concern is the unqualified tamper-evidence claim; if the authors can provide a positional analysis and validate thresholds out-of-sample, I would support acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real advance over single-signal watermarks. Co-embedding a robust and a fragile signal into every token through shared tournament rounds with different keys and windows is new, and the 2D three-state decision works for the attacks they actually test. The unqualified '89.5–100%' tamper-detection claim in the abstract and Table I is too strong, though: it holds for edits distributed across the text, but tail-localized edits can leave the fragile z-score well above threshold, so changing the last few tokens can still read as Intact. Section VI acknowledges this, but doesn't quantify it.\n\nWhat the paper does well: the mechanism is clearly specified, the ablations support the three claimed design necessities (normalization at seeding, co-embedding, and a long fragile window), and the Bileve comparison is unusually fair — it documents re-tokenization failure and the real cost of bit-carrying signatures. The appendix correctly imports SynthID's unbiasedness theorems, so the theory is not hand-waved. Perplexity stays in range, and the margin over baselines on distributed edits is enormous. Seeding on normalized text is a principled choice, and the homoglyph experiment makes the point concretely.\n\nSoft spots, in proportion: the position-dependence of tamper evidence is a real gap. The fragile signal is seeded on the full normalized prefix, so an edit at position i only corrupts tokens after i. The paper's own example attack (sentiment flip) spreads edits, and random substitution is uniform, so the reported 89.5–100% reflects that distribution. A positional ablation — e.g., recall as a function of edit location — would make the claim honest. The robust seed definition is also inconsistent: Eq. (2) and Algorithm 1 use the previous h_r tokens, while Table III says previous token plus self token. This should be one thing. Thresholds are calibrated in-sample and no error bars are given, which is minor but worth fixing. No code or data link is provided, so 'exactly reproducible' is an assertion, not a verifiable one.\n\nNone of this undermines the core mechanism as scoped. For the threat model of edits that appear throughout the text, Cocktail clearly works and beats the field. The limitation is that tamper evidence is a weaker guarantee for surgical tail edits than the abstract suggests.\n\nWho this is for: anyone working on LLM watermarking, provenance, or tamper detection. It deserves a serious referee, and I would send it out. The revision should add a positional analysis, error bars, and a code/data release.","headline":"Cocktail is a genuine dual-signal watermarking advance, but its headline tamper-detection number is position-dependent and needs scoping before the paper is fully honest.","tokens_in":18352,"tokens_out":3448,"would_cite":true,"duration_ms":40347,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dual-signal watermark can prove both where text came from and whether it was altered.","keywords":["LLM watermarking","provenance","tamper evidence","piggyback spoofing","robust and fragile signals","unbiased tournament reweighting","normalized text seeding","three-state detection"],"falsifier":"Generate a Cocktail-watermarked text, then flip the sentiment of only the final clause (the last 10-20 tokens), keeping the rest untouched. The fragile signal seeds on the full normalized prefix, so the corruption is confined to the tail; if the fragile z-score stays above the calibrated threshold and the detector outputs Intact, the claimed tamper-evidence guarantee is position-dependent and does not cover whole-text integrity.","tokens_in":17332,"feed_emoji":"🛡️","tokens_out":6476,"duration_ms":68458,"temperature":0.7,"pith_summary":"The paper claims that a single watermark can serve two opposing goals at once: tracing an LLM's output to its source even after rewriting, and flagging when the text has been altered. Existing watermarks prioritize edit-robustness for attribution, which lets an attacker change critical content while the text still scores as watermarked, a failure called piggyback spoofing. The proposed method, Cocktail, co-embeds a robust and a fragile signal into every generated token; the first survives edits, the second collapses when the reader-visible text changes. Detection then reads the two scores together and returns Intact, Tampered, or No-Watermark. Across two large language models and two prompt datasets, the method flags 89.5-100% of tampered texts at a 1% false-alarm rate while keeping attribution and generation quality near the strongest single-signal baselines.","feed_headline":"Two-signal watermark exposes tampered LLM text, not just origin","feed_subtitle":"Co-embedding a robust and a fragile signal in every token lets one detector say Intact, Tampered, or No-Watermark.","key_machinery":"The load-bearing mechanism is the co-embedding of two complementary signals through rounds of unbiased tournament reweighting. Each round reweights the token distribution by a fresh Bernoulli(0.5) green-red list, leaving the expected distribution unchanged; a periodic pattern assigns rounds alternately to the robust and fragile signals, so the round ratio (1:1, 2:1, 4:1) dials their relative strength. Because each signal is seeded on normalized text rather than token IDs, the fragile signal certifies the content the reader actually receives. The short-window robust signal provides attribution, the long-window fragile signal provides tamper evidence, and the joint two-threshold rule partitions the score plane into three states.","core_discovery":"The central claim is that provenance and tamper evidence are not in conflict if the same text carries two independent signals that are seeded differently. Both signals are green-red vocabulary partitions derived from a keyed pseudorandom function, but the robust signal keys on a short window of the preceding normalized text while the fragile signal keys on a long character window over the full normalized prefix. Any edit that changes the normalized text corrupts the fragile seeds of every subsequent token, driving the fragile z-score down, while the short-window robust seed recovers within a few tokens. In the resulting two-dimensional score space, tampered text occupies a distinct region: high robust score, low fragile score. The paper demonstrates that this separation holds across two large language models and two prompt datasets, with tamper detection far above single-signal baselines and without degrading perplexity.","pith_inferences":["One testable extension the paper leaves implicit: because the fragile window covers the full prefix, edits confined to the final tokens only corrupt the tail of the seeds; a detector that scores trailing segments separately could catch tail-only edits, and this could be verified directly.","The complementary-signal recipe may transfer to other generative modalities (images, audio) where a robust and a fragile signal are co-embedded in every output element; the paper notes this possibility for other modalities.","The fragile signal's long seeding window makes it harder to steal by querying the detector, so a forger who learns the robust green list will produce text that classifies as Tampered rather than Intact; this asymmetry between the two signals is a security property worth testing under an explicit stealing attack.","A practical deployment would need to define and normalize 'reader-visible content' carefully; any adversarial edit that survives the normalizer is by construction invisible to tamper evidence, so the guarantee is only as strong as the normalizer's coverage."],"forward_implications":["If the claim holds, one watermarked text can be simultaneously attributed to a model and checked for integrity, closing the piggyback-spoofing hole in single-signal schemes without adding a second, fragile signature.","The 2D score plane replaces the binary watermarked/not-watermarked decision with a three-state decision, so a detector can tell 'untouched model output' from 'edited model output' at a tunable false-alarm rate.","Because the fragile signal is seeded on normalized text, re-encoding attacks that preserve meaning but change bytes (homoglyphs, case changes, whitespace) are folded away rather than falsely flagged; the paper shows normalization at both seeding and detection keeps scores unchanged.","The round-allocation pattern gives practitioners a single knob to trade attribution robustness against tamper sensitivity while keeping generation quality, since each round preserves the expected distribution."],"supporting_citations":[{"why":"the green-red single-signal watermark whose z-score framework both signals use and whose spoofing vulnerability motivates the work","marker":"[4]"},{"why":"the tournament-sampling watermark that supplies the unbiased-reweighting theorems Cocktail's co-embedding relies on","marker":"[5]"},{"why":"the bi-level signature scheme that Cocktail is compared against and whose re-tokenization failures motivate the design","marker":"[9]"},{"why":"the attack definition of piggyback spoofing that tamper evidence must counter","marker":"[8]"},{"why":"a semantic-invariant single-signal baseline whose edit-insensitivity yields near-zero tamper evidence","marker":"[7]"},{"why":"a fixed-partition robust baseline showing the same trade-off","marker":"[6]"},{"why":"the paraphraser used as the attribution attack and as a prompt dataset source","marker":"[11]"},{"why":"the homoglyph perturbation study that motivates normalization at seeding time","marker":"[38]"},{"why":"the unbiased-watermark formulation that fixes the quality standard Cocktail preserves","marker":"[21]"},{"why":"the image-domain 'cocktail' pairing of robust and fragile watermarks that the paper adapts to text","marker":"[27]"}],"fun_headline_variants":["Dual-signal watermark flags tampered LLM text","LLM watermark now detects edits, not just provenance","Watermark with fragile signal reveals text tampering","Three-state watermark: intact, tampered, or no mark","Complementary signals make LLM watermarks tamper-evident"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The tamper-evidence guarantee rests on the fragile signal being seeded from the full normalized prefix, so 'intact' means 'unchanged after normalization'; any edit that leaves the normalized text the same, such as a homoglyph replacement, passes as intact, and edits confined to the final tokens are only partially caught.","fun_headline_variants_meta":{"raw":{"variants":["Dual-signal watermark flags tampered LLM text","LLM watermark now detects edits, not just provenance","Watermark with fragile signal reveals text tampering","Three-state watermark: intact, tampered, or no mark","Complementary signals make LLM watermarks tamper-evident"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000691,"raw_usage":{"total_tokens":3121,"prompt_tokens":930,"completion_tokens":2191,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":546,"completion_tokens_details":{"reasoning_tokens":2111}},"tokens_in":546,"tokens_out":2191,"duration_ms":16453,"temperature":1.0,"reasoning_tokens":2111,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T01:00:47.848742+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate a Cocktail-watermarked text, then flip the sentiment of only the final clause (the last 10-20 tokens), keeping the rest untouched. The fragile signal seeds on the full normalized prefix, so the corruption is confined to the tail; if the fragile z-score stays above the calibrated threshold and the detector outputs Intact, the claimed tamper-evidence guarantee is position-dependent and does not cover whole-text integrity.","supporting_citations":[{"cited_title":"A watermark for large language models,","cited_arxiv_id":null,"evidence_quote":"the green-red single-signal watermark whose z-score framework both signals use and whose spoofing vulnerability motivates the work"},{"cited_title":"Bileve: Securing text provenance in large language models against spoofing with bi-level signature,","cited_arxiv_id":null,"evidence_quote":"the bi-level signature scheme that Cocktail is compared against and whose re-tokenization failures motivate the design"},{"cited_title":"No free lunch in LLM watermarking: Trade-offs in watermarking design choices,","cited_arxiv_id":null,"evidence_quote":"the attack definition of piggyback spoofing that tamper evidence must counter"},{"cited_title":"A semantic invariant ro- bust watermark for large language models,","cited_arxiv_id":null,"evidence_quote":"a semantic-invariant single-signal baseline whose edit-insensitivity yields near-zero tamper evidence"},{"cited_title":"Provable robust watermarking for AI-generated text,","cited_arxiv_id":null,"evidence_quote":"a fixed-partition robust baseline showing the same trade-off"},{"cited_title":"Unbiased watermark for large language models,","cited_arxiv_id":null,"evidence_quote":"the unbiased-watermark formulation that fixes the quality standard Cocktail preserves"},{"cited_title":"Cocktail water- marking for digital image protection,","cited_arxiv_id":null,"evidence_quote":"the image-domain 'cocktail' pairing of robust and fragile watermarks that the paper adapts to text"}],"review_version":1}