{"id":"a53b872c-3675-4608-a8ee-9483d3e1e715","arxiv_id":"2505.14112","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A lightweight entropy classifier plus an adaptive threshold method can watermark and detect low-entropy LLM code outputs without querying the original model, matching much larger detectors at 99% fewer detection-phase parameters.","lead":"This paper replaces the large language model normally needed to detect watermarks in AI-generated code with a small neural network that predicts which tokens are predictable. If it works, AI-text detection becomes cheaper and safer for companies that do not want to share or run their proprietary models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The Threshold Navigator's stopping rule and its theoretical justification in §6.2 are internally inconsistent; the main detection numbers depend on a component whose claimed Type-II-error reduction is unsupported until the joint behavior of G and WR is tested.","rationale":"The reader's weakest assumption (out-of-distribution tagger accuracy) is legitimate, but the paper already evaluates one OOD setting, HumanEval, and Table 1 shows only a small AUROC drop there, so that risk is plausible but not yet demonstrable. The Threshold Navigator issue is different: its mathematical justification is internally inconsistent in a way that can be checked immediately, and it directly affects the headline detection numbers. The main Table 1 results use the Navigator, and Figure 5 claims it substantially improves AUROC and UES, so the central 'state-of-the-art detection' claim inherits whatever is wrong with the Navigator. Section 6.2's partial-derivative argument treats |S|G and WR as independent coordinates of z, although both are functions of the selected threshold; when WR increases because more tokens are watermark-eligible, G increases in expectation as well, so the sign of ∂z/∂WR at fixed G is not the correct experiment. Algorithm 4's stop condition (p>1, w<1) also conflicts with the prose: with w defined as WR_{τ_{i−1}}/WR_{τ_i}, w<1 means WR increased, not decreased, when the threshold is lowered. This makes the claimed 'optimistic threshold' and the Type-II-error analysis unreliable. The controlled threshold sweep I propose would settle whether the Navigator is finding a real detection peak or simply selecting thresholds on the evaluation set. The released code and tagger make this test feasible. This sharpens, but does not move, the reader's CONDITIONAL verdict.","tokens_in":18819,"tokens_out":14885,"duration_ms":185114,"concrete_test":"Run a controlled sweep on HumanEval with the released tagger: generate watermarked code at fixed thresholds τ ∈ {0.3, 0.6, 0.9, 1.2, 1.5} with γ=0.5, δ=3.0, and for each τ record WR, mean |S|G, mean z, and AUROC. If mean z and AUROC increase monotonically as τ decreases (WR increases), the ∂z/∂WR < 0 argument in §6.2 is contradicted under the actual coupling, and the Navigator's w < 1 branch is selecting lower detection power. If AUROC is instead non-monotonic, verify that the Navigator-selected τ coincides with the empirical AUROC peak. This distinguishes a real mechanism from test-set threshold selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central detection claim depends on the Threshold Navigator as much as on the tagger: Figure 5 attributes large AUROC/UES gains to it, and the main results in Table 1 are obtained with the Navigator. That component's statistical justification is internally inconsistent. In §6.2, z is written as z = (|S|G − γ·WR·|T|) / sqrt(WR·|T|·γ(1−γ)), and the paper takes partial derivatives treating |S|G and WR as independent, concluding ∂z/∂WR < 0 and hence that a smaller WR helps detection. But |S|G and WR are jointly determined: for watermarked text generated with fixed green-list bias δ, G ≈ p_δ·WR·|T|, so z ≈ (p_δ − γ)·sqrt(WR·|T|)/sqrt(γ(1−γ)), which increases with WR, not decreases. Moreover, Algorithm 4 in Appendix F lowers τ and stops when p = |S|G_{τ_{i−1}}/|S|G_{τ_i} > 1 and w = WR_{τ_{i−1}}/WR_{τ_i} < 1, i.e., at the point where the next lower threshold produced fewer green tokens but more watermarked tokens. The prose in §4.3 and §6.2 describes this condition as 'reducing the watermark ratio,' while w < 1 as defined means WR increased. Thus the sign conventions in the text, the algorithm, and the derivative analysis contradict one another. The reported Navigator gains may therefore be threshold search on the evaluation set rather than a principled Type-II-error reduction.","agreement_with_reader":"partial"},"referee_report":null,"author_rebuttal":null,"desk_editor":null,"rs_alignment":null,"lean_confirmation":null,"pith_extraction":null,"created_at":"2026-08-07T15:40:21.133011+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":null,"supporting_citations":[],"review_version":1}