{"id":"3640bf56-521b-469d-a11c-53fc31b8b0e0","arxiv_id":"2505.09924","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"An entropy-driven fusion of logits-based and sampling-based watermarks is claimed to outperform existing LLM watermarking methods on all four evaluation axes.","lead":"SymMark combines two established LLM watermarking techniques, logits-based and sampling-based, into serial, parallel, and hybrid configurations. The hybrid uses token and semantic entropy to choose where to embed watermarks, reporting gains in detectability, text quality, robustness, and security.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Algorithm 1's if/else-if chain cannot embed both watermarks on one token, contradicting the hybrid definition and Appendix H, so the reported Hybrid results may come from an unstated different procedure.","rationale":"The paper's core contribution is the hybrid selection rule, so the correctness of Algorithm 1 is load-bearing. The text and Appendix H describe simultaneous application of both watermarks; Algorithm 1's else-if makes that impossible. Algorithm 3 shows the authors know how to write independent conditions, which strengthens the case that Algorithm 1 is a typo rather than a principled design choice, but a typo at the exact point that defines the method is still a reproducibility failure for the preprint as written. Threshold tuning is a secondary worry, not the main one: even perfect thresholds cannot create tokens that receive both watermarks under the published code. I therefore agree with the reader's rejection but locate the decisive problem in the algorithm/description mismatch rather than in the 50-sample tuning. The concrete test distinguishes a harmless typo, if co-occurrence is rare, from a substantive discrepancy; until that test is run, the SOTA claim for Hybrid is unsubstantiated. The reader's stated weakest assumption was threshold generalization, which is related but not identical, hence partial agreement.","tokens_in":30832,"tokens_out":3819,"duration_ms":40206,"concrete_test":"Run the released code on a 200-sample C4 subset with OPT-6.7B, instrumenting each token's H_TE and H_SE. Count tokens with H_TE > alpha and H_SE < beta simultaneously (alpha=1.0, beta=0.5). If that fraction is material (e.g., >5%), compare Hybrid detection F1, PPL, and attack AUC under two implementations: Algorithm 1 exactly as written (else-if) versus the independent-if logic of Algorithm 3 and Appendix H. If the two differ by more than noise on the Table 1 and Figure 5 metrics, the paper's Hybrid results were produced by logic other than the published Algorithm 1. If the co-occurrence fraction is negligible, the else-if is immaterial and the concern would be downgraded.","verdict_should_be":"REJECT","load_bearing_attack":"Section 4.3 defines Hybrid as 'applying both watermarks, only one, or skipping altogether,' and Appendix H says Scheme 1 embeds two watermark signals when token entropy is high and semantic entropy is low. Algorithm 1, however, uses else-if: line 6 applies the logits watermark only when H_TE > alpha, and line 9 is unreachable for those tokens, so a token satisfying both conditions receives only the logits watermark. The same token cannot receive the sampling watermark. Algorithm 3 in Appendix I uses independent ifs and can append the same token to both Yl and Ys, showing that the intended logic is expressible and exists elsewhere in the paper. Thus the central mechanism that supposedly yields dual-signal robustness and the 'symbiotic' advantage is not what Algorithm 1 implements. If the experiments used the independent-if logic, the published algorithm must be corrected; if they used the published logic, the Hybrid results are for a conditional single-watermark selector, not the symbiotic hybrid described in the abstract and introduction. Either way, the SOTA claim is not supported by the stated method. This is an internal inconsistency, not a matter of threshold tuning or missing error bars.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SymMark, a framework that integrates logits-based and sampling-based LLM watermarking through three strategies: Serial, Parallel, and Hybrid. In the Hybrid strategy, token entropy and semantic entropy are used per token to decide whether to apply a logits watermark, a sampling watermark, both, or neither. Detection is performed by OR-ing the outputs of the two corresponding watermark detectors. The authors report experiments on OPT, LLaMA, and GPT-J models over C4, OpenGen, and WaterBench-style tasks, claiming state-of-the-art performance in detectability, robustness, text quality, and security, with code released at a public repository.","tokens_in":30989,"tokens_out":4592,"duration_ms":49648,"significance":"If the Hybrid mechanism works as described, the paper makes a useful contribution to LLM watermarking by directly attacking the known trade-offs among robustness, text quality, and security, and by systematically comparing three integration strategies. The experimental scope is broad: multiple models, datasets, downstream tasks, attacks, and a watermark-stealing security evaluation, and the authors also test several combinations of base watermarking schemes. The release of code is a concrete strength that supports reproducibility. However, the central algorithmic description of Hybrid is internally inconsistent, and the reported Hybrid results therefore cannot be unambiguously attributed to the method described in the abstract and introduction.","major_comments":[{"comment":"Algorithm 1's if/else-if structure prevents any token from receiving both watermarks. In Algorithm 1, line 5 checks H_TE > alpha and applies the logits watermark; because line 9 is an else-if branch, a token satisfying H_TE > alpha never reaches the sampling-watermark branch, even if it also has H_SE < beta. This directly contradicts the text of Section 4.3, which states that Hybrid may apply both watermarks, and Appendix H, which says the authors adopt Scheme 1 and 'embed two watermark signals when token entropy is high and semantic entropy is low.' Appendix I's Algorithm 3 uses independent if statements and is consistent with the intended behavior, which makes the discrepancy clear. The authors must state exactly which generation procedure was used in the experiments, correct Algorithm 1 if the independent-if logic was actually used, and re-derive any claims about dual-signal robustness from the procedure actually implemented.","section":"Section 4.3, Algorithm 1, Appendix H"},{"comment":"The robustness explanation attributed to 'Dual-signal Injection' is not supported under the literal Algorithm 1. The text claims Hybrid remains detectable because 'even if one watermarking signal is partially disrupted, the other remains intact,' but if no token receives both watermarks, then the Hybrid text does not contain two independent signals on the same tokens, and the stated mechanism cannot explain the reported robustness. The authors should either correct the algorithm to the independent-if version and confirm that the experiments use it, or provide a different, evidence-based explanation for the observed AUC values.","section":"Section 6.3"},{"comment":"The fixed thresholds alpha=1.0 and beta=0.5 are tuned on 50 samples from C4 using OPT-6.7B, yet they are then applied to all models, datasets, and attack settings without reporting sensitivity to these choices on the other evaluation settings. This is not circular, since the main evaluations appear to be separate experiments, but it is a generalization concern for the SOTA claim. The authors should report the entropy-threshold sensitivity for at least one additional model/dataset or otherwise justify that the OPT-6.7B/C4 tuning transfers.","section":"Appendix G and Tables 1-2"}],"minor_comments":[{"comment":"Equation (2) assigns the logits watermark to even positions and the sampling watermark to odd positions, whereas the text of Section 4.2 states the opposite assignment; the equation and text should be aligned.","section":"Equation (2)"},{"comment":"Algorithm 1 as printed is syntactically confusing: the softmax step appears between the if block and the else-if branch. The indentation and control flow should be reformatted so that the intended branch structure is unambiguous.","section":"Algorithm 1"},{"comment":"The sentence 'Series scheme exhibits a perfect TPR of 1.000, signifying no false positives' conflates TPR with TNR; a TPR of 1.000 means no false negatives, while TNR is the metric for false positives.","section":"Section 6.1"},{"comment":"The watermark-stealing experiment does not state the gamma and delta used for the attacked Unigram baseline; the appendix specifies gamma=0.25 and delta=0.4 for the Unigram component inside Hybrid, but not for the standalone Unigram comparator, which is needed for a fair comparison.","section":"Appendix J"},{"comment":"Tables 1 and 2 report values to three decimals without confidence intervals or multiple-seed statistics. Given that generation is stochastic, at least a note on fixed seeds or repeated trials would be helpful.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The Algorithm 1 issue is real and load-bearing, but it appears fixable: Appendix H and Algorithm 3 already describe the intended independent-if logic. If the code confirms that the experiments used independent ifs, the authors can correct the pseudocode and the central claims become defensible; if the code uses the else-if form, the Hybrid results need to be reinterpreted and the SOTA claim substantially weakened. I therefore recommend major revision rather than outright rejection, contingent on the authors clarifying and correcting this inconsistency."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the entropy-driven hybrid idea is genuinely new and the experimental breadth is impressive, but the published Algorithm 1 does not implement the hybrid as described. The paper claims a token can receive both watermarks, yet the algorithm's else-if chain guarantees no token ever does. The same intended logic appears correctly in Algorithm 3, so this is a fixable error, but it is load-bearing: the robustness and security arguments rely on dual-signal injection. If the experiments used the correct independent-if logic, the paper must be revised; if they used the printed logic, the Hybrid results describe a conditional single-watermark selector, not the 'symbiotic' method.\n\nWhat is new: token-entropy for deciding logits watermarking and semantic entropy for deciding sampling watermarking. That selection rule is plausible and well-motivated, and the paper works through serial, parallel, and hybrid integrations, with a broad evaluation across four models, two datasets, multiple attacks, and downstream tasks. The code release and the hyperparameter analysis in Appendix G are real assets.\n\nWhere it is soft: no error bars and many 1.000 scores in Tables 1-2 make the SOTA claim hard to assess, though saturation is common in this space. The thresholds α=1.0 and β=0.5 are tuned on 50 C4 samples with OPT-6.7B and then used everywhere; the sensitivity study mitigates this but does not remove it. Duwak (Zhu et al., 2024) does combine dual watermarks and is only cited in passing, so the novelty framing overstates the gap.\n\nBottom line: this deserves peer review, but as a major revision, not acceptance. A serious referee should ask for a corrected Algorithm 1, verification that the experiments match the intended logic, variance reporting, and a direct discussion of Duwak. I would not cite the current version in my own work because the central mechanism is ambiguous.","headline":"Entropy-driven hybrid watermarking is a real idea, but Algorithm 1 does not implement the claimed hybrid—a fixable but load-bearing inconsistency.","tokens_in":31586,"tokens_out":4295,"would_cite":false,"duration_ms":44049,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a per-token entropy gate lets logits-based and sampling-based watermarks reinforce each other, overcoming the usual trade-off between detectability and text quality.","keywords":["LLM watermarking","symbiotic watermark","logits-based watermark","sampling-based watermark","token entropy","semantic entropy","watermark security","watermark robustness"],"falsifier":"Take a model not in the paper's set (e.g., a multilingual or code-specialised model), generate with the hybrid method using the paper's fixed thresholds, and measure detection F1 and perplexity against the same baselines; if the hybrid's F1 or AUC falls below the best single-method baseline, or its perplexity rises above Unigram's, the generalisation claim is falsified.","tokens_in":30574,"feed_emoji":"🧬","tokens_out":6192,"duration_ms":56897,"temperature":0.7,"pith_summary":"The paper claims that the standard trade-off in LLM watermarking—stronger marks degrade text quality and invite attack—is not inevitable. It proposes SymMark, a framework that runs a logits-based watermark and a sampling-based watermark at the same time, and in its main 'Hybrid' configuration decides per token which of the two (or neither) to apply, using two entropy measures as the decision rule. The claim is that this adaptive combination reaches the best reported balance of detectability, resilience, text quality, and security across several models and datasets. If the claim holds, watermarking can be far less intrusive than the current best single-method baselines while staying more resistant to watermark-stealing and text-altering attacks.","feed_headline":"Hybrid watermarking beats the detectability-quality trade-off","feed_subtitle":"SymMark's entropy gate picks the watermark per token, cutting quality loss without losing detection.","key_machinery":"The central object is the entropy-gated hybrid watermarking rule. At each step $t$, the model's logits give a distribution $p_t$; token entropy $H^{TE} = -\\sum_i p_i^t \\log p_i^t$ gates the logits-based watermark, and semantic entropy $H^{SE}$ (computed as the entropy of the distribution merged across $n$ K-means clusters of the top-$k$ token embeddings) gates the sampling-based watermark. The rule 'apply logits watermark if $H^{TE} > \\alpha$; apply sampling watermark if $H^{SE} < \\beta$' is what carries the argument: it places the stronger, fluency-affecting mark only where the model is already uncertain, and the sampling mark only where swapping among semantically similar candidates will not change meaning. Detection is a simple OR of the two component detectors, applied to all tokens, so detection stays cheap and does not require knowing which tokens were watermarked.","core_discovery":"SymMark's core discovery is that two watermark families need not compete for the same token. In the serial strategy both watermarks are embedded in every token, maximizing signal but hurting fluency; in the parallel strategy they alternate by token position, preserving quality but losing resilience to edits. The hybrid strategy, the paper's main contribution, computes for each position the token entropy (Shannon entropy of the next-token distribution) and the semantic entropy (entropy over clusters of semantically similar top candidates), and applies the logits watermark only when token entropy is high, the sampling watermark only when semantic entropy is low, and both or neither otherwise. The authors report that this per-token selection outperforms each family alone and every tested single-method baseline, with the serial variant hitting perfect detection under their settings and the hybrid holding an average AUC of 0.984 under attacks against 0.951 for Unigram. The message is that the trade-off is a design choice, not a law.","pith_inferences":["Editorial extension: the fixed thresholds $\\alpha=1.0$ and $\\beta=0.5$ were chosen on 50 C4 samples with OPT-6.7B; a domain-adaptive threshold, estimated online from the generation context, could push the quality-detectability frontier further on short generations or code tasks.","Editorial extension: the semantic-entropy gate is doing duty as a proxy for 'will the reader notice a substitution'; a direct test of that proxy would be a human or NLI study comparing swapped-token sentences under the gating rule.","Editorial extension: because the semantic-entropy computation depends on a tokenizer-aligned embedding space, the method's portability to multilingual or code-tokenized models is an open question that the paper does not address."],"forward_implications":["On the paper's own results, the hybrid variant widens the operating window: it reaches near-perfect detection with the lowest perplexity among all tested watermarks, and on the four downstream tasks its quality drop is under 1% in two of them.","Because detection is an OR of two independent detectors, the paper's dual-signal design means that damaging the text enough to remove the logits signal will likely leave the sampling signal measurable, and vice versa.","The framework is modular: the paper's ablations swap Unigram for KGW and AAR for GumbelSoft, and the hybrid combination still outperforms its components, indicating that the entropy gating transfers across watermark families.","The serial and parallel variants bound the design space; the hybrid sits between them and inherits the serial's robustness and the parallel's fluency, which the paper demonstrates in the perplexity and attack experiments."],"supporting_citations":[{"why":"Defines the canonical logits-based red-green watermark that serves as the main baseline and the family targeted by watermark-stealing attacks.","marker":"(Kirchenbauer et al., 2023)"},{"why":"Provides the AAR exponential-sampling watermark, used as SymMark's sampling-based component and as a baseline.","marker":"(Aaronson, 2023)"},{"why":"Supplies Unigram, the fixed red-green logits watermark selected as SymMark's logits-based component and robustness baseline.","marker":"(Zhao et al., 2024)"},{"why":"Introduces EXP and ITS sampling watermarks, baselines for detectability and robustness comparisons.","marker":"(Kuditipudi et al., 2024)"},{"why":"Contributes SynthID, a tournament-sampling watermark that serves as a key sampling baseline.","marker":"(Dathathri et al., 2024)"},{"why":"Defines the watermark-stealing and spoofing attack setup used to evaluate SymMark's security.","marker":"(Jovanovi´c et al., 2024)"},{"why":"Supplies the concept of semantic entropy that motivates the sampling-watermark gating criterion.","marker":"(Farquhar et al., 2024)"},{"why":"Provides WaterBench, the downstream-task evaluation protocol used for the text-quality measurements.","marker":"(Tu et al., 2024)"}],"fun_headline_variants":["Watermark synergy: entropy picks the token-level method","SymMark: entropy-aware watermark mixing for zero trade-off","Hybrid watermarking: when entropy decides, no quality sacrifice","SymMark's hybrid watermarking: choose per token, not per model","Per-token watermark selection breaks the robustness-quality deadlock"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The fixed thresholds $\\alpha=1.0$ and $\\beta=0.5$, tuned on a 50-sample C4 subset with one model, are assumed to transfer across all models, datasets, and attacks, and the K-means clustering with $k=64,n=10$ is assumed to capture semantically similar candidates.","fun_headline_variants_meta":{"raw":{"variants":["Watermark synergy: entropy picks the token-level method","SymMark: entropy-aware watermark mixing for zero trade-off","Hybrid watermarking: when entropy decides, no quality sacrifice","SymMark's hybrid watermarking: choose per token, not per model","Per-token watermark selection breaks the robustness-quality deadlock"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001578,"raw_usage":{"total_tokens":6284,"prompt_tokens":919,"completion_tokens":5365,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":5281}},"tokens_in":535,"tokens_out":5365,"duration_ms":40630,"temperature":1.0,"reasoning_tokens":5281,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:21:17.813549+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a model not in the paper's set (e.g., a multilingual or code-specialised model), generate with the hybrid method using the paper's fixed thresholds, and measure detection F1 and perplexity against the same baselines; if the hybrid's F1 or AUC falls below the best single-method baseline, or its perplexity rises above Unigram's, the generalisation claim is falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the AAR exponential-sampling watermark, used as SymMark's sampling-based component and as a baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Contributes SynthID, a tournament-sampling watermark that serves as a key sampling baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the watermark-stealing and spoofing attack setup used to evaluate SymMark's security."}],"review_version":1}