{"id":"a383e5ae-54a4-47ed-ac51-a4c40eeec911","arxiv_id":"2608.10237","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A feed-forward generator trained offline can corrupt and invert the pairwise similarity structure of contrastive verification systems using small bounded image perturbations.","lead":"This paper trains a small generator network to attack contrastive and Siamese verification systems by warping their embedding geometry, pushing matching pairs apart and non-matching pairs together. In tests on ballot-mark and signature verification models, it drops accuracy sharply and inverts the similarity ranking between positive and negative pairs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Attack generator is trained on 80% of the victim's test set, contradicting the threat model's independent auxiliary data and likely inflating the reported 95.4→38.6 accuracy drop.","rationale":"The reader's weakest_assumption correctly identifies the train/test overlap in Section 4.2. I agree this is the most load-bearing issue: the paper's headline numbers depend on the generator seeing most of the victim's test set. However, the core method (a U-Net trained with a relational displacement objective) is not disproven by this concern; it just needs re-evaluation under the stated threat model. The internal table inconsistencies (logit/probability mismatch, Table 1 vs Table 7 disagreement, and the ZO/NES query table using 1,972 pairs instead of 394) make the current numerical claims harder to trust but do not change the fundamental need for a clean evaluation. The verdict should remain CONDITIONAL: the paper's central claim is plausible but not yet supported by the current experimental protocol.","tokens_in":13422,"tokens_out":14523,"duration_ms":133856,"concrete_test":"Retrain the geometry deformation generator on a truly disjoint auxiliary dataset (e.g., a separate ballot-mark collection or a held-out portion of Dtrain_victim that the victim never saw) and evaluate on the full 1,972-pair Dtest_victim. If the Accuracy Drop and Gap Reduction remain close to 0.569 and 15.170, the concern is resolved; if they shrink substantially, the reported attack strength is an artifact of test-set leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 states the attacker independently collects a small auxiliary dataset Dtrain_attack that is similar to, but disjoint from, the victim's training data. Section 4.2 instead sets Dtrain_attack to an 80% random split of Dtest_victim (1,578 of 1,972 pairs), using the victim's held-out test distribution for generator training. This directly violates the stated threat model: the attacker is given the target evaluation distribution, so the reported 95.4%→38.6% accuracy drop and the logit inversion (9.993→-2.478 vs 0.641→3.340) measure within-distribution memorization rather than generalization to independently collected queries. The remaining 20% test split is still from the same test set, so it does not test the realistic scenario in which the attacker must generalize from a different auxiliary set. This concern is load-bearing because the central claim is that a fast, generator-based attack works without online gradients under a realistic black-box threat model; if the generator has been trained on the test distribution, the evidence does not support that claim. Internal inconsistencies (e.g., Table 1 probabilities are not sigmoid of the reported logits; Table 7 'No Transform' gives 0.363 Adv Acc vs 0.386 in Table 1) further reduce confidence in the reported magnitudes.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a generator-based adversarial attack on contrastive/Siamese verification systems. Instead of crossing a decision boundary, an offline-trained U-Net generator produces bounded per-image perturbations that push positive pairs apart and pull negative pairs together in a frozen victim's temperature-scaled cosine-similarity embedding space. At deployment the generator emits perturbations in one forward pass, avoiding online gradient computation. Experiments on a MarkMatch ballot-mark verifier and on CEDAR signature verifiers report substantial accuracy degradation (e.g., 95.4% to 38.6% on MarkMatch), inversion of average positive/negative logits, and comparisons with Diff-PGD, ZO/NES, and surrogate-transfer baselines.","tokens_in":13800,"tokens_out":8191,"duration_ms":75029,"significance":"If the experimental evaluation is corrected, the paper would make a useful contribution: it identifies a distinct vulnerability class for similarity-based verification, and it shows that an amortized generator can, in principle, replace per-query iterative optimization. The problem formulation is clear, the attack objective is simple, and the cross-system experiments (SigNet, SigScatNet, CEDAR DenseNet) are a reasonable first step. The main strengths are the relational-geometry framing and the offline/online amortization idea. No code or machine-checked proofs are provided, and the current experimental protocol contains a serious threat-model violation and several internal inconsistencies, so the headline numbers cannot be taken at face value yet.","major_comments":[{"comment":"The attack generator is trained on the victim's test set, contradicting the stated threat model. Section 3.3 says the attacker 'independently collects a small auxiliary dataset Dtrain_attack that follows a similar distribution to, but has no overlap with, the victim model's training data Dtrain_victim.' Section 4.2 instead sets Dtrain_attack to an 80% random split of Dtest_victim (1,578 of 1,972 pairs), and the sentence 'ensuring no overlap with the victim model's training and test data' is incorrect because Dtrain_attack is a subset of Dtest_victim. The remaining 20% is still drawn from the same victim test distribution, so the evaluation does not test generalization from an independently collected auxiliary set to a different query distribution. Because the central claim is that a fast, generator-based attack works under a realistic black-box threat model, the reported 0.954-to-0.386 accuracy drop and the logit inversion (9.993 to -2.478 vs 0.641 to 3.340) may substantially overstate the attack's effectiveness. The evaluation must be rerun with a genuinely disjoint auxiliary dataset, or the threat model must be explicitly revised to grant the attacker access to the victim's test distribution.","section":"Section 4.2 vs Section 3.3"},{"comment":"The headline relational-geometry metrics are the training objective in closed form, so their large values are partly a restatement of the loss rather than independent evidence of manifold corruption. With w_y = 1, L_attack = (1/N) [sum over positives of Delta-l + sum over negatives of (-Delta-l)] = -(Positive Logit Drop + Negative Logit Rise) = -GapReduction, up to class-balance and averaging constants. Thus the reported PosDrop = 12.471, NegRise = 2.698, and GapReduction = 15.170 largely mirror the fact that the generator minimized the objective in Section 3.6. The accuracy drop is an independent result, but the paper's stronger claim of 'complete inversion' of relational structure should be supported by metrics that are not directly optimized by the loss, such as rank-correlation of pairwise similarities, neighborhood-overlap statistics, or the fraction of pairs whose similarity ordering flips.","section":"Section 3.6 vs Section 4.3"},{"comment":"There are internal inconsistencies in the reported probabilities and the decision rule. The text states that similarity logits are converted to probabilities with a sigmoid and compared with a threshold near 0.9962, but sigmoid(9.993) is about 0.99995, not 0.996; sigmoid(-2.478) is about 0.077, not 0.326; sigmoid(0.641) is about 0.655, not 0.559; and sigmoid(3.340) is about 0.966, not 0.589. With a threshold of 0.9962, a negative-pair logit of 3.340 would still be rejected, so the reported accuracy of 0.386 is not explainable from the Table 1 logits under the stated decision rule. In addition, the 'No Transform' row of Table 7 (Adv Acc 0.363, Pos Drop 13.529, Gap Red. 16.550) should match Table 1 (Adv Acc 0.386, Pos Drop 12.471, Gap Red. 15.170) but does not. Please clarify the exact decision rule, reconcile these tables, and report error bars over multiple train/test splits; currently every number in the paper is a point estimate from a single split.","section":"Table 1 and Table 7"}],"minor_comments":[{"comment":"The same attack-split protocol is not described for SigNet, SigScatNet, and CEDAR DenseNet; please state how Dtrain_attack and Dtest_attack were constructed for each victim model and whether the same test-set-leakage issue applies to those experiments.","section":"Section 4.2"},{"comment":"The regularization term L_dir uses ReLU(Delta-l_+) and ReLU(-Delta-l_-), but the main loss already encourages positive pairs to separate and negative pairs to collapse; please clarify whether L_dir changes the optimum or mainly affects optimization dynamics.","section":"Section 3.6"},{"comment":"The text says perturbation magnitude is constrained by clipping and projection 'introduced in Section 3.5,' but these operations are defined in Section 3.4; the cross-reference should be corrected.","section":"Section 3.6, final paragraph"},{"comment":"The manuscript uses placeholder ACM formatting data (conference acronym, DOI placeholder, and 'Received 20 February 2007' boilerplate), and Figure 2's panels are not clearly labeled; these should be fixed in a polished version.","section":"Front matter and Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The decisive issue is the Section 4.2 threat-model violation: the attack generator is trained on 80% of the victim's test set. If a corrected evaluation with a genuinely independent auxiliary set still shows large accuracy degradation, this could become a solid paper. If the authors cannot provide that experiment, the central claim will not be supported and I would recommend rejection. The manuscript also appears to be an early draft, given the placeholder ACM metadata and unresolved table inconsistencies."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a promising idea—training a U-Net generator to push positive embedding pairs apart and pull negative pairs together, so that similarity structure inverts—but the main experimental claim is undercut by a threat-model violation. The attack generator is trained on 80% of the victim's test set (Section 4.2), not on the independently collected auxiliary set described in Section 3.3. So the headline 95.4→38.6 accuracy drop and the logit inversion are within-distribution numbers; they say little about generalization to a genuinely separate auxiliary set. That's the load-bearing soft spot.\n\nCredit where due: the relational-geometry framing is a useful perspective, the generator amortization makes sense for real-time use, and the authors compare against several baselines including a query-heavy ZO/NES attack. The cross-system results suggest the effect is not a one-off artifact of one architecture. Also, the objective is simple and clearly stated.\n\nThe problems, in order: (1) The test-set leak. The threat model promises an independent auxiliary set; the experiment splits the victim's test set 8:2. This is not a minor detail—the paper's core claim is that the attack generalizes from offline training to online queries, and this setup gives it the evaluation distribution. (2) The probabilities in Tables 1 and 4 do not match sigmoid of the reported logits. Clean positive logit 9.993 would give ~0.99995, not 0.996; adversarial positive -2.478 gives ~0.077, not 0.326. Something else is going on (maybe a thresholding step), and it reduces confidence in the reported magnitudes. (3) Minor inconsistency: Table 7's No Transform Adv Acc 0.363 versus 0.386 in Table 1. (4) Novelty is modest—push/pull objectives and generator-based attack amortization already exist in the cited metric-learning and AdvGAN/GAP literature. The contribution is the application to contrastive verification, not a new attack paradigm.\n\nOverall, the central idea is plausible and worth a serious referee, but the paper as written does not support its headline claim. It should be sent to review with a request for major revision: rerun with a properly disjoint auxiliary set, add error bars, and fix the probability reporting.","headline":"A plausible generator-based attack on contrastive verification whose headline results are undermined by training the attack on the victim's test set and by inconsistent probability reporting.","tokens_in":14248,"tokens_out":4519,"would_cite":false,"duration_ms":39256,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a bounded, generator-produced perturbation can invert the pairwise similarity structure of a contrastive embedding manifold, dropping MarkMatch verification accuracy from 95.4% to 38.6%.","keywords":["contrastive learning","Siamese networks","adversarial attack","embedding manifold","pairwise similarity","verification systems","relational geometry attack","generative attack"],"falsifier":"Train the same geometry deformation generator on a dataset that is disjoint from and distributionally shifted relative to the victim's test set, for example signatures from a different population or writing instrument, then measure the accuracy drop and gap reduction on the original test set; if the drop collapses toward the surrogate-transfer baseline, the central claim of distribution-general relational corruption is refuted. A second check is to apply a certified input-space robustness method, such as randomized smoothing, and see whether the positive-negative inversion disappears at a small certified radius.","tokens_in":13255,"feed_emoji":"🔄","tokens_out":5996,"duration_ms":51862,"temperature":0.7,"pith_summary":"This paper sets out to show that contrastive and Siamese verification systems, which make decisions from pairwise similarity in an embedding space rather than from class boundaries, can be attacked by systematically corrupting that relational geometry instead of by flipping individual predictions. The authors train a lightweight U-Net generator offline to learn geometry deformation priors from a frozen victim model; at attack time the generator adds a bounded perturbation to each input in a single forward pass. Across the MarkMatch ballot-mark verification system the attack lowers accuracy from 95.4% to 38.6%, and on all tested architectures it collapses the separation between positive and negative pairs, in some cases reversing it so that negative pairs become more similar than positive pairs. If this holds, real-time black-box attacks on similarity-based verification are feasible without per-query gradient computation, and robustness for such systems may need to focus on preserving relational geometry rather than decision boundaries.","feed_headline":"A single forward pass drops verification accuracy from 95% to 39%","feed_subtitle":"Training a small generator to warp the embedding manifold reverses positive and negative pair similarity in Siamese verification systems.","key_machinery":"The load-bearing object is the geometry deformation generative network, a U-Net encoder-decoder with skip connections that maps each input image to a bounded perturbation field. Its training objective is built on the relational displacement $\\Delta\\ell = \\ell_{\\mathrm{adv}} - \\ell_{\\mathrm{clean}}$ between adversarial and clean temperature-scaled cosine logits, with a label-dependent sign $\\alpha(y)$ that pushes positive-pair logits down and negative-pair logits up, plus a directional regularization term that penalizes batches where positive pairs fail to separate or negative pairs fail to collapse. Perturbations are clipped per pixel and $L^2$-projected so the deformation stays visually subtle. This machinery makes the attack a single feed-forward pass at deployment: all gradient computation happens once, offline, against the frozen victim.","core_discovery":"The central discovery is that bounded image perturbations produced by a learned generator can invert the pairwise similarity structure that contrastive verification models rely on. The authors formulate the attack as manifold-level relational corruption: for positive pairs the objective is to separate embeddings, and for negative pairs to collapse them, encoded through a label-dependent direction term on the temperature-scaled cosine logit. On the MarkMatch victim model, the average positive-pair logit drops from 9.993 to -2.478 while the average negative-pair logit rises from 0.641 to 3.340, so the clean positive-negative gap of 9.352 becomes -5.818. The same pattern appears on SigNet, SigScatNet, and a CEDAR DenseNet model, with adversarial accuracy dropping from near-perfect to roughly 0.32-0.50. The paper argues this shows contrastive systems have a different adversarial vulnerability from classifiers: the vulnerable object is the relational organization of the embedding manifold, not any decision boundary.","pith_inferences":["Editorial inference: the offline generator may transfer to victim models it was not trained on, since it learns deformation patterns from embedding geometry rather than from one classifier's boundaries; this is testable by training on one victim and attacking a held-out Siamese model.","Editorial inference: if the vulnerability is intrinsic to contrastive training, then defenses should be evaluated in embedding space, for example by relational adversarial training that penalizes positive-negative gap collapse, rather than only with input transformations.","Editorial inference: the threat model's claim of an independent auxiliary dataset is not actually exercised in the experiments, because the attack training set is carved from the victim's test set; a cleaner test would train the generator on a disjoint acquisition to clarify how much of the reported 95.4% to 38.6% drop depends on test-set distribution overlap."],"forward_implications":["A trained generator becomes a real-time attack tool: after the offline stage, adversarial pairs are produced without iterative optimization, so low-latency verification endpoints are exposed.","Accuracy alone understates the effect; even where decisions partly survive, the positive-negative similarity order can be compressed or inverted, which would corrupt downstream ranking and retrieval built on the same embeddings.","Because the corruption is geometric rather than boundary-based, input-space defenses such as random transformations only soften it (adversarial accuracy rises from 0.363 to 0.480 but gap reduction remains 10.094) rather than eliminating it.","The consistency across DenseNet, SigNet, and SigScatNet victims suggests the vulnerability is anchored in the contrastive training paradigm rather than in one architecture.","A large $L^2$ budget is not required: a nearly unconstrained setting performs about the same as $\\epsilon_2 = 2.5$, indicating the effect comes from targeted geometry deformation rather than perturbation magnitude."],"supporting_citations":[{"why":"Supplies the MarkMatch victim verification framework and the ballot-mark dataset used for the headline accuracy drop.","marker":"[9]"},{"why":"Defines the classic adversarial optimization setting the threat model adopts and the baseline the attack extends.","marker":"[10]"},{"why":"Provides the U-Net encoder-decoder architecture used for the geometry deformation generator.","marker":"[28]"},{"why":"Supplies the temperature-scaled cosine similarity/logit formulation and the batch contrastive loss used to train the victim model.","marker":"[30]"},{"why":"Provides the DenseNet121 backbone for the primary MarkMatch victim encoder.","marker":"[29]"},{"why":"Provides the CEDAR signature dataset used in the cross-system evaluation.","marker":"[31]"},{"why":"One of the alternative Siamese verification architectures showing the attack generalizes across systems.","marker":"[12]"},{"why":"Another Siamese verification architecture in the cross-system evaluation, showing the largest gap reduction.","marker":"[11]"},{"why":"Diff-PGD baseline that the relational geometry attack is compared against on decision and geometry metrics.","marker":"[35]"},{"why":"ZO/NES query-limited black-box baseline used to contrast the single-pass generator's query cost and relational corruption.","marker":"[36]"}],"fun_headline_variants":["Contrastive embedding geometry attack reverses similarity relations","Learned generator inverts pair similarity, crashing accuracy to 39%","Relational attack on Siamese manifolds flips positive-negative similarity","Offline-trained attack warps embedding manifold, verification accuracy plummets"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The attack generator is trained on an 8:2 split of the victim's test set, but the threat model claims it uses only an independent auxiliary dataset; if the learned perturbations depend on that distributional overlap, the reported accuracy collapse will not transfer to a realistic attacker.","fun_headline_variants_meta":{"raw":{"variants":["Contrastive embedding geometry attack reverses similarity relations","Learned generator inverts pair similarity, crashing accuracy to 39%","Relational attack on Siamese manifolds flips positive-negative similarity","Offline-trained attack warps embedding manifold, verification accuracy plummets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000244,"raw_usage":{"total_tokens":1540,"prompt_tokens":963,"completion_tokens":577,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":504}},"tokens_in":579,"tokens_out":577,"duration_ms":6007,"temperature":1.0,"reasoning_tokens":504,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:11:16.477871+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same geometry deformation generator on a dataset that is disjoint from and distributionally shifted relative to the victim's test set, for example signatures from a different population or writing instrument, then measure the accuracy drop and gap reduction on the original test set; if the drop collapses toward the surrogate-transfer baseline, the central claim of distribution-general relational corruption is refuted. A second check is to apply a certified input-space robustness method, such as randomized smoothing, and see whether the positive-negative inversion disappears at a small certified radius.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MarkMatch victim verification framework and the ballot-mark dataset used for the headline accuracy drop."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the U-Net encoder-decoder architecture used for the geometry deformation generator."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the temperature-scaled cosine similarity/logit formulation and the batch contrastive loss used to train the victim model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the CEDAR signature dataset used in the cross-system evaluation."},{"cited_title":"SigScatNet: A Siamese + Scattering based Deep Learning Approach for Signature Forgery Detection and Similarity Assessment","cited_arxiv_id":"2311.05579","evidence_quote":"Another Siamese verification architecture in the cross-system evaluation, showing the largest gap reduction."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Diff-PGD baseline that the relational geometry attack is compared against on decision and geometry metrics."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ZO/NES query-limited black-box baseline used to contrast the single-pass generator's query cost and relational corruption."}],"review_version":1}