{"id":"f52c140e-7b25-4f8f-adfa-eda000188df1","arxiv_id":"2505.15337","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"CoPA fools eight AI-text detectors by using a language model to paraphrase text while subtracting machine-like word probabilities during decoding, achieving high fooling rates without any training.","lead":"This paper introduces CoPA, a method that rewrites AI-generated text with an off-the-shelf language model so that common AI-text detectors can no longer tell it apart from human writing. The trick is to subtract the model's own 'machine-style' word choices during generation, which makes the output look more human. A smart generalist might read this to understand how easily current AI-text detectors can be fooled, and what that means for plagiarism and content moderation.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The empirical attack appears real, but the paper's theoretical guarantee does not apply to the implemented algorithm: Theorem 1 analyzes an arithmetic mixture (1+λ)p'_h − λ p_m, while Eq. (3) performs logit-space contrast, and the premise g′(0)<0 is never measured.","rationale":"The reader's conditional verdict is well aligned with my read. The empirical tables are striking and fairly comprehensive: CoPA reduces TPR across three datasets, eight detectors, and multiple source and paraphraser LLMs, with semantic similarity scores around 90%, and the code is promised publicly. There is no sign that the core empirical phenomenon is fabricated or accidental. The load-bearing weakness is the theory: Theorem 1 proves a statement about an arithmetic mixture of distributions, while Eq. (3) implements a logit-space subtraction that corresponds to a product-of-experts distribution. The gap is not cosmetic because the KL-convexity argument and the domain analysis in Appendix A rely on the affine structure q(λ)=(1+λ)p'_h−λp_m; the geometric contrast has different normalization and different curvature behavior. The premise g′(0)<0 is also not measured; Figure 4 is an illustrative simplex diagram with |V|=3, not evidence for real token distributions. These issues do not overturn the empirical contribution, but they make the theoretical section unsupported as written. A conditional verdict is therefore appropriate, and the requested concrete check would settle whether the theory can be repaired or should be relabeled as heuristic.","tokens_in":20403,"tokens_out":7258,"duration_ms":70409,"concrete_test":"Re-derive Theorem 1 for the implemented distribution. Define p_c(λ) ∝ (p'_h)^(1+λ) / (p_m)^λ, normalized, and set g(λ) = KL(p_h || p_c(λ)) using a human-text distribution estimated from held-out XSum samples and the Appendix B prompts with Qwen2.5-72B. Estimate g′(0) by finite differences and check whether λ* > 0 and g(λ) < g(0) for λ ∈ (0, λ*]. If this fails, the theorem is inapplicable to CoPA as written; if it holds, the paper should supply the corrected proof and the measured g′(0).","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central theory-backed claim rests on Theorem 1 (Sec. 3.3, Eq. 5-6), which proves KL(p_h || (1+λ)p'_h − λ p_m) < KL(p_h || p'_h) under g′(0)<0. The implemented CoPA (Eq. 3) instead samples from p_c ∝ exp((1+λ)f_h − λ f_m), which after normalization is p_c ∝ (p'_h)^(1+λ) / (p_m)^λ, a geometric/product-of-experts contrast, not the arithmetic mixture analyzed in the theorem. Thus the theorem as stated does not cover the algorithm as implemented. Moreover, the sufficient condition g′(0)<0 is justified only by a heuristic |V|=3 diagram (Fig. 4) and an assertion that it 'generally holds'; no direct estimate of g′(0) is reported anywhere. Because this theorem is the paper's only formal argument that the contrastive step moves the output distribution toward human text, the theoretical support for CoPA is currently unverified. The empirical results are large and consistent enough that they may stand independently, but the advertised theoretical guarantee should not be cited until it is either derived for the actual logit-space distribution or explicitly downgraded to a heuristic.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoPA, a training-free contrastive paraphrase attack against LLM-generated text detectors. At each decoding step, CoPA combines a human-style prompt distribution with a machine-style prompt distribution via the logit-space contrast p_c ∝ exp((1+λ)f_h − λ f_m), after adaptive truncation. The authors report that CoPA reduces TPR at 5% FPR to near-baseline levels across eight detectors, three datasets, several source LLMs, and several paraphraser LLMs, while preserving high semantic similarity. A theoretical section (Sec. 3.3) presents a theorem intended to show that the contrastive distribution is closer to human text than the human-like prompt distribution alone. The empirical results are extensive, and the code is released. The main weakness is that the theorem analyzes an arithmetic mixture of distributions, whereas the implemented algorithm uses a logit-space contrast, so the stated theoretical guarantee does not cover the method as implemented.","tokens_in":20668,"tokens_out":5629,"duration_ms":52354,"significance":"If the empirical findings hold, CoPA is a practical and inexpensive attack that substantially weakens current LLM-text detectors, with implications for detector robustness and for the deployment of detection systems. The paper's strengths are its broad evaluation (three datasets, eight detectors, multiple source and paraphraser models), its release of code, and its inclusion of human and automated text-quality evaluations. The adaptive-defense experiment and the ethical discussion are also valuable. The theoretical claim, however, is currently not established for the actual algorithm, and the key premise of the theorem is not directly validated. These issues affect the paper's central advertised contribution and require substantial revision before publication.","major_comments":[{"comment":"Theorem 1 analyzes the arithmetic mixture q_λ = (1+λ)p'_h − λ p_m, but the implemented CoPA samples from p_c ∝ exp((1+λ)f_h − λ f_m), which after normalization equals p_c ∝ (p'_h)^(1+λ) / p_m^λ. These are different distributions, so the theorem does not cover the algorithm evaluated in Section 4. I recommend deriving an analogous theorem for the logit-space geometric contrast, or explicitly repositioning Theorem 1 as a heuristic motivation and removing claims that the theory 'guarantees' or 'validates' the effectiveness of the implemented method.","section":"Sec. 3.2 Eq. (3) and Sec. 3.3 Eq. (5)"},{"comment":"The premise g′(0)<0 is never measured or verified. The only support is the |V|=3 contour diagram in Figure 4 and the assertion that p'_h is 'usually' between p_m and p_h. Since this condition is the basis of Theorem 1, the theoretical result is conditional on an unvalidated assumption. Please report direct estimates of g′(0) for the actual token distributions used in the experiments, or state clearly that this is an assumption rather than an established fact whose verification is left to the reader.","section":"Sec. 3.3, Fig. 4"},{"comment":"The machine prompt x_m was selected using detector likelihood as a proxy, and the contrast intensity λ was tuned on the first 50 XSum samples (Figure 6). This is detector-informed tuning, and it should be disclosed in the main text because the paper positions CoPA as training-free and surrogate-free. To support that positioning, the main results should either be confirmed with a fixed machine prompt and hyperparameters chosen without detector feedback, or the paper should acknowledge that the reported performance depends on detector-informed prompt selection.","section":"Sec. 4.2, Appendix B, Fig. 3"}],"minor_comments":[{"comment":"Several cells in the Dipper rows are merged or missing delimiters (for example, Table 1 XSum shows '15.672.6727.33' and Table 2 for Claude-3.5 shows '18.170.3320.67'). Please fix the table formatting so that each metric is a separate entry.","section":"Tables 1 and 2"},{"comment":"The proof of uniqueness of the minimum uses analyticity of g on the closed interval I, but g need not be analytic at points where the mixture assigns zero probability to a token with positive human probability, because the KL divergence becomes infinite there. Please clarify the domain of g and handle infinite KL values explicitly.","section":"Appendix A, Proposition 3"},{"comment":"The text claims that the trend of TPR over λ in Figure 6 'roughly aligns with our theoretical analysis,' but the theorem applies to the arithmetic mixture, not to the implemented logit-space contrast. The ablation result would be more appropriately presented as a heuristic observation unless the theory is re-derived for the actual sampling distribution.","section":"Sec. 3.3 and Sec. 4.4"}],"recommendation":"major_revision","confidential_remarks":"The empirical attack appears real and the evaluation is strong, so I do not see grounds for rejection. My main concern is that the theoretical section is advertised as a guarantee while applying to a different distribution than the implemented algorithm. This is fixable by reframing the theorem as a heuristic or by proving the appropriate analogue. The detector-informed prompt selection should also be made transparent. If the authors address these points, the paper would be a solid contribution to the robustness evaluation of LLM-text detectors."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, CoPA is a real and effective attack: using Qwen2.5-72B with a contrastive decoding trick, it drops detector TPR below 20% at 5% FPR across eight detectors on three datasets, while keeping semantic similarity around 90%. That is a strong, consistent empirical result. Second, the theory section is not actually about the algorithm. Theorem 1 proves a statement about an arithmetic mixture (1+λ)p'_h − λ p_m, but Eq. (3) implements a logit-space subtraction exp((1+λ)f_h − λ f_m), which normalizes to a geometric contrast, not the arithmetic mixture. Those are different objects, so the advertised theoretical guarantee does not cover the implemented method.\n\nWhat is genuinely new is the combination: a human-like prompt elicits one distribution, a machine-like \"repeat the paragraph\" prompt elicits a strongly machine-like distribution, and contrastive decoding subtracts the latter from the former, with adaptive truncation to keep coherence. The paper also shows the method works across multiple source LLMs and paraphraser LLMs, and includes ablations and quality metrics. The attack is training-free and code is released, so the empirical claim is reproducible. Credit where earned: this is a practical, well-executed attack paper.\n\nSoft spots, in proportion. The theory gap is real and should be fixed: either derive the result for the actual logit-space distribution or explicitly downgrade the theory to a heuristic. The premise g'(0)<0 is asserted to \"generally hold\" based on a |V|=3 diagram and is never measured directly. The machine prompt was selected using detector likelihood, which is a form of tuning; it is not circular because the headline numbers come from held-out evaluation with fixed hyperparameters, but it should be reported more transparently. There are no error bars or variance across seeds, and some near-zero TPRs may be noisy. Minor: the human-like prompt is a text-message tone, which may not transfer to academic writing, though the authors provide alternative prompts.\n\nWho is this for? Anyone working on AI-text detection or adversarial robustness. The empirical attack deserves serious attention even if the theory is overstated. I would bring it to reading group and would cite it for the attack results. Send it to peer review, but require the authors to reconcile the theory with the implemented algorithm before acceptance.","headline":"CoPA is a genuinely effective training-free paraphrase attack with a serious theory-algorithm gap: the theorem analyzes an arithmetic mixture while the code does logit-space contrast, but the empirical results stand on their own.","tokens_in":21236,"tokens_out":1695,"would_cite":true,"duration_ms":15597,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoPA, a training-free contrastive paraphrase attack, reduces the detection rate of eight AI-text detectors to near-baseline levels while preserving semantics.","keywords":["LLM-generated text detection","paraphrase attack","contrastive decoding","adversarial attack","AI text detection","training-free attack","Fast-DetectGPT","text authenticity"],"falsifier":"Compute, for a fixed sample of human texts and a fixed paraphraser, the empirical KL divergence $\\mathrm{KL}(p_h \\| (1+\\lambda) p'_h - \\lambda p_m)$ for a grid of $\\lambda \\in [0,1]$ by estimating the distributions from token logits. If for the paper's default $\\lambda=0.5$ the contrastive distribution is not closer to the human distribution than $p'_h$ on a majority of samples, or if $g'(0) \\ge 0$ for many samples, the theorem's premise fails and the claimed bound does not hold for those samples. A second test is to train a detector on CoPA-paraphrased texts (beyond the paper's RoBERTa-large adaption) and see whether the attack's TPR remains near baseline.","tokens_in":20200,"feed_emoji":"🤖","tokens_out":5154,"duration_ms":39624,"temperature":0.7,"pith_summary":"The paper claims that a training-free 'contrastive paraphrase attack' (CoPA) can rewrite AI-generated text so that eight current detectors, including the strong Fast-DetectGPT, flag it as human at rates close to the random baseline. The trick is to have an off-the-shelf LLM generate two versions of a sentence: one prompted to sound human and one prompted to sound machine-like, then subtract the machine-like token distribution from the human-like one during decoding. The authors prove a convexity theorem showing that, under a premise they argue generally holds, this subtraction moves the paraphrase closer to true human word distributions than a human-style prompt alone. If correct, the method undercuts the reliability of current detectors while requiring no training data or specialized paraphraser, using only the attacker's own LLM.","feed_headline":"No-training paraphrase attack beats eight AI-text detectors","feed_subtitle":"By subtracting a machine-like token distribution during decoding, CoPA makes GPT-3.5-turbo text look human—fast and free.","key_machinery":"The central object is the contrastive decoding distribution in Eq. (3), formed by subtracting the logits of a machine-like distribution from an amplified human-like distribution: $p_c \\propto \\exp((1+\\lambda) f_\\theta(\\cdot|x_h,y_m,y_{<t}) - \\lambda f_\\theta(\\cdot|x_m,y_m,y_{<t}))$. This is a training-free, self-corrective decoding mechanism: the machine-like prompt (in practice, asking the LLM to repeat the original text) provides a concentrated 'machine-style' token distribution that is subtracted to remove machine-inherent word preferences. An adaptive truncation constrains sampling to tokens with high probability under the human-like distribution, preserving plausibility. The theoretical support is a convexity/KL-divergence argument: $g(\\lambda) = \\mathrm{KL}(p_h \\| (1+\\lambda) p'_h - \\lambda p_m)$ is convex, and if $g'(0)<0$ then a positive $\\lambda$ strictly reduces the KL divergence to the human distribution.","core_discovery":"On the paper's own terms, the central discovery is that the machine-like statistical bias of an LLM can be turned against its own detectability. CoPA elicits two token distributions from the same off-the-shelf model, $p'_h$ from a human-style prompt and $p_m$ from a machine-style prompt, then samples from the contrastive distribution $p_c \\propto \\exp((1+\\lambda) f_h - \\lambda f_m)$. This penalizes tokens the LLM systematically prefers in its default mode while keeping tokens plausible under the human-style prompt, via an adaptive truncation. In experiments on XSum, SQuAD, and LongQA against eight detectors, CoPA drives the detection true positive rate at 5% FPR down to an average of about 12.63%, 11.06%, and 7.44% for GPT-3.5-turbo texts, versus 66.05%, 56.27%, and 57.09% without attack, with semantic similarity above 90%.","pith_inferences":["A broader principle suggested by this work is that detectors relying on LLM-inherent statistical biases can be fooled by making the generator sample away from its own mode, a strategy that may generalize to other modalities if a contrastive 'machine-style' distribution can be elicited.","Because CoPA is training-free and uses off-the-shelf models, it lowers the bar for adversarial users: anyone with API access to a capable LLM can launch the attack, so detector robustness should be studied against adaptive attackers rather than static baselines.","The theory's premise $g'(0)<0$ deserves direct measurement; the paper offers a heuristic diagram but no empirical check. A simple test is to compute KL divergences on real data for increasing $\\lambda$ and verify that Eq. (6) holds as predicted.","The attack's success depends on the choice of machine prompt; the paper uses 'repeat the paragraph' because it best concentrates machine-like features, but whether this holds for all domains and LLMs is an open question."],"forward_implications":["CoPA works across eight detector types, including zero-shot likelihood-based, curvature-based, and trained classifiers, with TPR near baseline at 5% FPR.","It transfers across source LLMs (GPT-3.5-turbo, GPT-4, Claude-3.5, GPT-4o, Gemini-1.5 Pro) and across paraphrasers (Qwen2.5-72B, R1-Distill-32B, QwQ-32B, GLM-4-9B-hf), with QwQ-32B performing best.","A single paraphrase suffices; multiple paraphrases give limited gains, and semantic similarity stays above 90%.","An adaptive defense that adversarially trains RoBERTa-large with CoPA-paraphrased texts raises detection from 22.67% to 78.00% TPR at 5% FPR, indicating the attack can be mitigated by training on such texts.","CoPA outperforms a surrogate-based RedTeaming attack even without using a surrogate detector, with 17.00% versus 30.00% TPR on Fast-DetectGPT on XSum."],"supporting_citations":[{"why":"Dipper baseline; the paraphrase attack whose detection rates CoPA must beat.","marker":"(Krishna et al., 2023)"},{"why":"Fast-DetectGPT, the advanced zero-shot detector that CoPA most clearly defeats.","marker":"(Bao et al., 2024)"},{"why":"Contrastive decoding, the token-constraint and logit-subtraction mechanism CoPA adapts.","marker":"(Li et al., 2023)"},{"why":"DetectGPT probability curvature, one of the detectors evaluated and a theoretical basis for likelihood-based detection.","marker":"(Mitchell et al., 2023)"},{"why":"Recursive paraphrasing baseline showing that LLM rewriting alone is insufficient, motivating the contrastive design.","marker":"(Sadasivan et al., 2023)"},{"why":"Raidar, the generate-then-rewrite detector, and Raidar-A baseline using the same paraphrasing LLM.","marker":"(Mao et al., 2024)"}],"fun_headline_variants":["Contrastive attack evades all eight detectors with no training","Training-free attack fools AI detectors by subtracting machine-like tokens","CoPA: contrastive decoding makes GPT-3.5 text undetectable","Detectors beaten by contrastive paraphrase attack from off-the-shelf LLM","Drop detection rates to ~7% by contrasting token distributions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole argument rests on the assumption that subtracting the machine-like token distribution in log space actually moves the rewritten text closer to how humans write, an assumption the paper justifies with a heuristic diagram rather than a direct measurement.","fun_headline_variants_meta":{"raw":{"variants":["Contrastive attack evades all eight detectors with no training","Training-free attack fools AI detectors by subtracting machine-like tokens","CoPA: contrastive decoding makes GPT-3.5 text undetectable","Detectors beaten by contrastive paraphrase attack from off-the-shelf LLM","Drop detection rates to ~7% by contrasting token distributions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000539,"raw_usage":{"total_tokens":2617,"prompt_tokens":1008,"completion_tokens":1609,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":624,"completion_tokens_details":{"reasoning_tokens":1518}},"tokens_in":624,"tokens_out":1609,"duration_ms":11132,"temperature":1.0,"reasoning_tokens":1518,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:18:50.815408+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute, for a fixed sample of human texts and a fixed paraphraser, the empirical KL divergence $\\mathrm{KL}(p_h \\| (1+\\lambda) p'_h - \\lambda p_m)$ for a grid of $\\lambda \\in [0,1]$ by estimating the distributions from token logits. If for the paper's default $\\lambda=0.5$ the contrastive distribution is not closer to the human distribution than $p'_h$ on a majority of samples, or if $g'(0) \\ge 0$ for many samples, the theorem's premise fails and the claimed bound does not hold for those samples. A second test is to train a detector on CoPA-paraphrased texts (beyond the paper's RoBERTa-large adaption) and see whether the attack's TPR remains near baseline.","supporting_citations":[],"review_version":1}