{"id":"30533dbf-d57c-4342-9126-dd313854f651","arxiv_id":"2505.04977","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"DNN watermarks can be made more robust and forge-resistant by generating trigger inputs as a one-way cryptographic chain and labels from the owner's signature, with a Monte Carlo-calibrated decision threshold.","lead":"ChainMarks embeds a DNN watermark as a chain of hash-based trigger images whose target labels come from the owner's digital signature, and verifies ownership with a statistically calibrated Hamming-distance threshold. A generalist should read it to see a concrete attempt to make AI model ownership claims harder to forge, using cryptographic chaining to block ambiguity attacks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The cryptographic chain does not prevent fine-tuning-based ambiguity attacks, leaving the central security claim unsupported.","rationale":"The reader's weakest assumption focused on the false-positive probability estimation for the decision threshold (Section 5.2, Eq. (3)). That is a legitimate concern about the statistical soundness of the p-values and the resulting match thresholds, but it does not attack the central security mechanism directly. The more load-bearing issue is that the proposed cryptographic chain does not actually prevent the canonical ambiguity attack described in the paper's own threat model. The chain ensures trigger inputs must be generated by forward hashing, but hashing is public and easy to compute; any adversary can generate an unlimited number of valid chains. Security therefore depends on the adversary being unable to embed a second watermark into the model. However, the threat model grants white-box access to a surrogate model, and the paper itself says watermark embedding can be done via fine-tuning. Therefore, an adversary can fine-tune a surrogate on their own valid chain triggers, creating a second valid watermark and causing ownership ambiguity. The paper's evaluation only tests an optimization-based ambiguity attack (Figure 7), not this fine-tuning attack, so the headline robustness claim is not established. This is not an 'outside consensus' disagreement; it is an internal gap between the stated threat model and the evaluated defense. If the concrete test confirms the attack, the paper's central claim collapses. Even if the test were to fail for some empirical reason (e.g., catastrophic forgetting), that would need to be demonstrated; as written, the argument does not rule it out. Hence the verdict should move from CONDITIONAL to REJECT: the paper's primary security contribution is unsupported by both argument and evaluation. The p-value/threshold concern remains relevant but secondary; it could be addressed by calibration experiments, whereas the ambiguity attack gap requires redesign or at least a substantial new evaluation.","tokens_in":24693,"tokens_out":6733,"duration_ms":71111,"concrete_test":"Implement a fine-tuning ambiguity attack on a ChainMarks-watermarked ResNet-18 trained on CIFAR-10. Generate a forged chain of 100 triggers using an adversary-chosen random seed and a forged signature (converted to base-10 labels). Fine-tune the watermarked model (or a surrogate) on these 100 trigger/label pairs for a small number of epochs (e.g., 5-20) with a modest learning rate, holding out the original test set. Then run ChainMarks' own verification procedure (regenerate the forged chain, feed to the fine-tuned model, compute Hamming distance to the forged signature) and check whether the forged watermark is accepted at the p=0.01 threshold used in Table 6, while test accuracy remains within 90% of the original model's accuracy. If the forged watermark is accepted, the ambiguity attack succeeds and ChainMarks does not defeat ambiguity attacks.","verdict_should_be":"REJECT","load_bearing_attack":"ChainMarks' central claim is that it 'can efficiently defeat both watermark removal attacks and watermark ambiguity attacks' (Section 1). The defense against ambiguity attacks rests on the assertion that the one-way chain prevents adversaries from finding trigger inputs that satisfy both the chaining constraint and the target labels (Section 8.1). But the threat model explicitly grants an adversary white-box access to a surrogate model (Section 3: 'A surrogate model ... effectively grants attackers access to a \"white-box\" version of the original model, including its parameters'). With such access, an adversary does not need to solve the hash-chain optimization problem at all: they can simply choose their own secret seed, generate a valid hash chain of trigger inputs, pick their own digital signature, and fine-tune the surrogate model on these trigger/label pairs to embed a second, forged watermark. The paper acknowledges that ChainMarks watermark embedding can be done via fine-tuning (Section 9.1: 'the watermarks can be embedded through fine-tuning'), so the same capability is available to an adversary. The evaluation of ambiguity attacks (Table 6, Section 8.1) only considers an optimization-based attack that adds trainable noise to base triggers; it does not test this straightforward fine-tuning attack. If a fine-tuning ambiguity attack succeeds, the model contains two valid watermarks and ownership is ambiguous, defeating the paper's primary security contribution. Thus, the one-way chain does not, by itself, prevent ambiguity attacks under the paper's own threat model.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ChainMarks, a dynamic DNN watermarking scheme in which trigger inputs are generated as a one-way hash chain from a secret seed and target labels are derived from the model owner's digital signature. Verification compares predicted labels on the regenerated chain with the claimed signature via Hamming distance, using a decision threshold obtained from a two-phase Monte Carlo method that estimates the model's classification distribution on random inputs. The paper evaluates ChainMarks on CIFAR-10 and CIFAR-100 against sixteen removal attacks and one ambiguity attack, compares with four baselines, and reports lower required watermark accuracy and higher marginal utility.","tokens_in":24891,"tokens_out":13021,"duration_ms":135534,"significance":"The two-phase Monte Carlo idea of grounding the decision threshold in the specific model's output distribution is a reasonable direction and is not circular, since the threshold is a false-positive probability for unwatermarked models. The experimental campaign is broad, covering two datasets, multiple model architectures, and 17 attack types. However, the central security claim that ChainMarks defeats watermark ambiguity attacks is not supported under the paper's own threat model, and the statistical derivation behind the p-values contains inconsistencies that affect the validity of the verification threshold. These are load-bearing issues, not presentation concerns.","major_comments":[{"comment":"The claim that ChainMarks defeats watermark ambiguity attacks is evaluated only against an optimization-based attack that adds trainable noise to base triggers. The threat model in Section 3 explicitly grants the attacker a white-box surrogate model, and Section 9.1 states that ChainMarks watermarks 'can be embedded through fine-tuning.' An adversary can therefore choose its own secret seed, generate its own cryptographic chain of triggers, choose its own digital signature, and fine-tune the surrogate on those trigger/label pairs to embed a second valid watermark. The hash-chain constraint does not prevent this, because the adversary never needs to invert the chain or optimize through the hash function. This straightforward fine-tuning ambiguity attack is within the paper's assumptions but is absent from Table 6, so the paper's primary security contribution is unsupported.","section":"§8.1 and Table 6"},{"comment":"The false-positive probability is computed for a fixed or random sequence of target labels, but a rational adversary can choose the claimed labels to maximize the match rate. For the ResNet-18 CIFAR-10 model in Table 1, one class has hit probability 0.9962 on random inputs. If an adversary claims a signature consisting mainly of that class, the expected number of matching triggers among L random inputs is close to L, so even a threshold of 14% matches is exceeded with probability essentially 1. The p-values in Table 2 therefore do not bound the success probability of an adaptive guessing attack; the analysis needs to maximize over adversarially chosen labels or justify why the signature format prevents such choices.","section":"§4.4 and §5.2"},{"comment":"The statement and proof of Claim 1 are internally inconsistent. Equation (4) defines the mean as μ = L/C, whereas the proof's Eq. (6) defines μ = ∑_{i=1}^L p_{c_i}; these are equal only under uniform class probabilities, which Table 1 shows is not the case. In addition, Eq. (12) minimizes ∑ p_{c_i}^2 over the class-index set U with the constraint ∑ p_{c_i} = p_U, but the quantity appearing in Eq. (11) is a sum over chain positions whose labels fall in U; the number of such positions need not equal |U|, so the conclusion that each p_{c_i} = p_U/k yields a valid upper bound on σ does not follow. Since the decision threshold and all reported p-values rely on Claim 1, this needs to be corrected before the verification statistics can be trusted.","section":"§5.2, Claim 1 (Eqs. (4)–(12))"}],"minor_comments":[{"comment":"The estimate of p_U is obtained from the average of first-hit waiting times over 50 simulations, but the geometric mean estimator is not linear; the MLE should be based on pooled trials, and using the average waiting time introduces bias.","section":"§5.1"},{"comment":"The statement that ownership can be verified with 14 of the first 20 triggers uses the threshold derived for L = 100 without recomputing it; the decision threshold depends on the chain length L and should be recalculated for L = 20.","section":"§9.1"},{"comment":"The p-values in Table 2 are not reproducible from the formulas stated in Section 5.2; the authors should provide the underlying parameters, the exact signature used, or the code used to generate the table.","section":"Table 2"},{"comment":"Reference [33] contains a garbled citation, 'SHA128' in Section 9.1 should be SHA-1 or SHA-256, and 'ResNet 28x10' should be identified precisely (e.g., as WideResNet) for reproducibility.","section":"References and notation"},{"comment":"The watermark marginal utility is a newly defined metric whose value depends on the arbitrary choice of consecutive p-values used in the difference quotient; the figure should state this dependence, and the missing CIFAR-100 bars for the baselines should be explained in the caption.","section":"§7.3 and Figure 6"}],"recommendation":"reject","confidential_remarks":"The fine-tuning ambiguity attack follows directly from the paper's own assumptions and is not addressed; the label-choice gap in the p-value analysis and the inconsistency in Claim 1 further undermine the verification statistics. These are not fixable with local corrections, because the security model needs to be rethought. The two-phase Monte Carlo threshold idea may be reusable, but the paper in its current form does not substantiate its central claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about ChainMarks. First, the two-phase Monte Carlo threshold estimator is a real contribution: it gives model owners a way to compute verification thresholds at very small p-values, where the empirical approach in the SoK paper breaks down. Second, the headline security claim—that the cryptographic chain defeats ambiguity attacks—does not survive contact with the paper's own threat model. An adversary with white-box access to a surrogate can simply choose a fresh seed, generate their own hash chain, pick their own signature, and fine-tune the surrogate to embed a second valid watermark. Section 8.1 only tests an optimization attack that tries to alter existing triggers; it never tests this fine-tuning route, even though Section 9.1 says watermarks can be embedded via fine-tuning. If that attack works, the model carries two valid chains and ownership is ambiguous, exactly what the scheme claims to prevent.\n\nThe generation procedure is a clean combination: triggers are a one-way chain from a secret key, labels come from the owner's signature in base C. That is new relative to Adi et al. and Zhang et al. The threshold analysis is the strongest part—modeling the match count as a Poisson binomial and using a two-phase Monte Carlo to estimate the zero-hit class probability is sensible, and the reported 14% match threshold at p≈1e-7 is plausible.\n\nSoft spots: Claim 1's bound has a minor flaw—Eq. (12) minimizes over all zero-hit classes U, but if the signature doesn't contain some of those classes, the sum of squares can be much smaller than p_U^2/k. Since p_U is roughly 1e-8, the error is negligible. More important, the evaluation compares ChainMarks with a two-phase threshold at p=0.01 against baselines using an empirical threshold at p=0.05, which is not apples-to-apples, and no code or data are released to re-check the numbers.\n\nWorth sending to a serious referee. The threshold estimator alone is useful. The authors need to release code, fix the Claim 1 statement, and either add the fine-tuning ambiguity attack to the evaluation or soften the security claim.","headline":"Useful threshold-estimation work, but the ambiguity-attack resistance claim is undercut by an unconsidered fine-tuning attack.","tokens_in":25483,"tokens_out":3766,"would_cite":true,"duration_ms":38817,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Watermarked DNN triggers linked by a one-way hash chain resist both removal and forgery attacks.","keywords":["deep neural network watermarking","cryptographic key chain","watermark ambiguity attack","watermark removal attack","two-phase Monte Carlo","backdoor triggers","model ownership verification","Poisson binomial distribution"],"falsifier":"Generate hash-chain trigger sets from many random seeds, feed them to an unwatermarked model trained on the same dataset, and compare the predicted labels with random base-$C$ signatures; if the observed proportion of trials achieving at least 14 of 100 matches exceeds the claimed ~$10^{-7}$ by a visible margin (say $>10^{-5}$), the two-phase Monte Carlo bound is not safe.","tokens_in":24433,"feed_emoji":"🔗","tokens_out":6025,"duration_ms":57232,"temperature":0.7,"pith_summary":"The paper proposes a dynamic DNN watermarking scheme in which trigger inputs are not independent samples but consecutive outputs of a one-way hash chain built from a secret seed, and the labels attached to those triggers are digits of the owner's digital signature. It argues that this makes optimization-based watermark forgery computationally infeasible, because an attacker cannot include a hash function in a gradient-based objective, and it makes removal attacks harder because hash-derived triggers behave like out-of-distribution noise. For verification, the paper introduces a two-phase Monte Carlo method that estimates the model's class probability distribution, including the probability mass of classes never hit by random inputs, and uses a Poisson-binomial model to bound the success probability of a random guessing attack. The authors report that on CIFAR-10, 14 matching triggers out of 100 suffice to claim ownership with an estimated random-guess probability near $10^{-7}$, and that ChainMarks resists the ambiguity attack that defeats the baseline schemes while withstanding most of the tested removal attacks.","feed_headline":"14% of chained triggers can prove DNN ownership","feed_subtitle":"Hash-linked trigger inputs resist removal and forgery; random-guess success is roughly one in ten million.","key_machinery":"The load-bearing object is the one-way key chain of trigger inputs, generated as $B_L = F(K)$ and $B_{i-1} = F(B_i)$ for a cryptographic hash $F$ and secret seed $K$, paired with a base-$C$ expansion of the owner's digital signature as target labels. The chain is what blocks ambiguity attacks: backpropagation cannot differentiate through $F$, so adversarial optimization cannot satisfy the inter-trigger constraint. The second piece of machinery is the two-phase Monte Carlo estimator: phase one gathers empirical classification probabilities $p_i = n_i/N$ from random inputs; phase two measures the waiting time until a hit in the zero-hit class set $U$, converting that geometric waiting time into an estimate of $p_U$, from which the variance bound $\\sigma'$ in the Poisson-binomial normal approximation is computed.","core_discovery":"ChainMarks' central claim is that a cryptographic chain over trigger inputs closes the ambiguity-attack hole that backdoor-style watermarking schemes leave open. Because each valid trigger is required to be the hash preimage of the next, an adversary who optimizes trainable perturbations to make fake triggers produce chosen labels will almost surely break the chain relation; the optimized inputs are no longer valid watermarks. The only residual avenue, random guessing from a seed, has success probability governed by the distribution of the number of label matches, which the paper approximates in closed form. The two-phase Monte Carlo estimator feeds millions of random inputs through the model to obtain empirical class probabilities, then treats the never-hit classes as a Bernoulli event and estimates their total probability mass by waiting for the first hit; the resulting upper bound on the standard deviation gives a normal-approximation $p$-value that remains usable when the target $p$-value is extremely small.","pith_inferences":["A natural stress test beyond the paper's experiments is to measure the true distribution of hash-chain triggers on a large set of unwatermarked models and compare the observed match rate with the claimed $p$-value; the exchangeability assumption is the point most worth probing.","The same construction could be lifted to non-image models: mapping hash output to token IDs or graph adjacency matrices would extend chain-based watermarking to language models and graph networks, an extension the paper mentions but does not implement.","The statistical ownership claim is only as strong as the $p$-value computation; in a legal or commercial dispute, the verifier may need to defend the Monte Carlo estimation and the independence assumption, not just the number of matching triggers."],"forward_implications":["Ownership can be established from a minority of surviving triggers: on CIFAR-10, 14 of 100 matched chain positions give a claimed random-guess success probability around $10^{-7}$.","Verification is black-box and partial: the owner can disclose only a subset of the chain positions, keeping the remaining triggers secret for later verification rounds.","Ambiguity attacks that forge a second watermark by optimizing fake triggers become infeasible, since the forged triggers would violate the hash-chain constraint.","The threshold estimation remains defined at very small $p$-values where empirical estimation fails, so the scheme can be configured for higher security levels.","Removal attacks that preserve model utility leave enough surviving triggers to clear the threshold; the paper reports resistance to all tested attack types except transfer learning, and notes knowledge distillation as a further limitation."],"supporting_citations":[{"why":"Supplies the backdoor-training embedding method and the abstract-image trigger baseline that ChainMarks is compared against.","marker":"[1]"},{"why":"Defines the Content, Noise, and Unrelated trigger-image baseline schemes whose robustness ChainMarks is compared with.","marker":"[92]"},{"why":"Provides the taxonomy of watermark removal attacks and the empirical threshold-estimation method used as the comparison baseline for $p$-values.","marker":"[53]"},{"why":"Describes the passport-based defense against ambiguity attacks, the prior approach that ChainMarks' chain design is intended to surpass.","marker":"[21]"},{"why":"Gives the Poisson-binomial distribution and normal-approximation formulas used to bound the guessing-attack success probability.","marker":"[27]"},{"why":"Introduces the one-way key chain construction that ChainMarks adapts to generate trigger inputs.","marker":"[35]"}],"fun_headline_variants":["ChainMarks: hash chain seals DNN watermarks","Hash-linked triggers thwart watermark ambiguity","Two-phase Monte Carlo proves DNN ownership","Cryptographic chain blocks watermark forgery","ChainMarks: provable DNN ownership via chain"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The threshold logic assumes the hash-generated trigger inputs behave, on the model, like the random inputs used to estimate class probabilities, and that the signature digits are independent across chain positions; if either fails, the claimed random-guess probability can understate the true forgery success rate.","fun_headline_variants_meta":{"raw":{"variants":["ChainMarks: hash chain seals DNN watermarks","Hash-linked triggers thwart watermark ambiguity","Two-phase Monte Carlo proves DNN ownership","Cryptographic chain blocks watermark forgery","ChainMarks: provable DNN ownership via chain"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000417,"raw_usage":{"total_tokens":2157,"prompt_tokens":961,"completion_tokens":1196,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":1137}},"tokens_in":577,"tokens_out":1196,"duration_ms":11464,"temperature":1.0,"reasoning_tokens":1137,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:16:46.982260+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Generate hash-chain trigger sets from many random seeds, feed them to an unwatermarked model trained on the same dataset, and compare the predicted labels with random base-$C$ signatures; if the observed proportion of trials achieving at least 14 of 100 matches exceeds the claimed ~$10^{-7}$ by a visible margin (say $>10^{-5}$), the two-phase Monte Carlo bound is not safe.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Content, Noise, and Unrelated trigger-image baseline schemes whose robustness ChainMarks is compared with."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the taxonomy of watermark removal attacks and the empirical threshold-estimation method used as the comparison baseline for $p$-values."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the Poisson-binomial distribution and normal-approximation formulas used to bound the guessing-attack success probability."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the one-way key chain construction that ChainMarks adapts to generate trigger inputs."}],"review_version":1}