{"id":"0ffe913f-d4e1-4414-9c7b-ed0a8fe446e3","arxiv_id":"2508.11341","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Pretrained language models can select adversarial target labels more effectively than static lexical databases, particularly for semantically distant classes.","lead":"This paper proposes using pretrained language models like BERT, TinyLLAMA, and CLIP to pick the most semantically confusing or distinct target labels when attacking vision models. The authors report that these models beat static word databases like WordNet, especially for distant classes, and could make adversarial benchmarks more reproducible.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Uncontrolled target-difficulty confound may explain the reported advantage of LLM/CLIP embeddings over WordNet for adversarial target selection.","rationale":"The full text provided is corrupted, so the assessment rests on the abstract and the reader's summary. The reader's weakest assumption points to generalizability of semantic similarity; my concern is a more immediate internal-validity confound: target selection methods may differ in the difficulty of the targets they choose, independent of semantic quality. This does not contradict the reader's UNVERDICTED verdict—if anything, it reinforces that the evidence is insufficient. I do not move the verdict because the paper might already control for this confound in the unreadable full text. A concrete check would settle whether the concern lands.","tokens_in":6222,"tokens_out":4126,"duration_ms":49934,"concrete_test":"Run the same target-selection comparison on a held-out architecture/dataset (e.g., ResNet-50 and ViT-B/16 on ImageNet) and for each selected target set compute attack success alongside (a) a random-target baseline with identical target-label marginal frequencies, and (b) a 'most-confusable' baseline that selects the target by the attacked model's own softmax ranking. If the LLM/CLIP advantage over WordNet disappears or shrinks to non-significance when matched against either baseline, the semantic-similarity explanation is not supported. Also repeat on a novel label set not present in BERT/CLIP pretraining (e.g., CIFAR-100 or a custom label set) to check for pretraining leakage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—that embeddings from BERT/TinyLLAMA/CLIP select better adversarial targets than WordNet, especially for distant class relationships—depends on the comparison being fair. The abstract reports only attack success rates, not the properties of the selected target sets. If the embedding-based methods preferentially select targets that are intrinsically easy to attack (e.g., classes visually confusable with the source, low-confidence classes, or classes with high confusion under the attacked model), then the reported advantage is a proxy for target difficulty rather than for semantic quality. WordNet similarity is purely symbolic and is not aligned with the attacked model's feature geometry, so it may systematically select harder targets. The 'distant class relationships' claim is exactly where this confound bites: 'distant' in semantic space may coincide with visually similar but semantically unrelated classes, and targeted attacks to such labels may be easier for reasons unrelated to language-model semantics. Without controlling for per-target attackability—e.g., matching target sets by the attacked model's own softmax margin or comparing against a random-target baseline with identical marginal label frequencies—the headline comparison does not establish that language-model semantic similarity is the causal driver. The abstract also asserts generality 'across architectures and datasets' from only three vision models and five attacks, with no reported variance; that overgeneralization is secondary but reinforces the need for a confounding control.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a semantics-guided framework for selecting target labels in targeted adversarial attacks on vision models. It uses embeddings from pretrained text or vision-language models (BERT, TinyLLAMA, CLIP) to pick the most and least semantically similar labels with respect to the ground-truth class, thereby defining best- and worst-case attack scenarios. The authors report experiments on three vision models and five attacks, claiming that these similarity sources 'consistently render practical adversarial targets' and surpass static lexical databases such as WordNet, especially for distant class relationships. They also propose that static target evaluation provides an a priori assessment of similarity-source quality. The central claim is an empirical one: embedding-based target selection is superior and more generalizable than static lexical selection, enabling more interpretable and standardized adversarial benchmarks.","tokens_in":6541,"tokens_out":3381,"duration_ms":41199,"significance":"If substantiated, the proposed framework would be a useful step toward principled adversarial target selection, which is indeed an underexamined component in targeted attack evaluation. The use of pretrained multimodal models as similarity sources is timely and potentially valuable for constructing scalable, interpretable benchmarks. However, the significance is currently prospective: the submission as provided does not give enough experimental detail to verify the headline claims, and the main comparison may be confounded by target-difficulty effects. No machine-checked proofs, code, or data are provided; the contribution is an empirical methodology that must be validated by careful controlled experiments.","major_comments":[{"comment":"The abstract makes strong empirical claims—'consistently render practical adversarial targets' and 'surpass static lexical databases'—but gives no variance, confidence intervals, statistical tests, or experiment configuration. The three vision models, five attacks, and dataset(s) are not named, and no results table is visible in the provided text. As written, the central claim is not checkable. Please provide a full experimental section with model/attack/dataset details and uncertainty quantification.","section":"Abstract"},{"comment":"The headline comparison between embedding-based target selection and WordNet may be confounded by per-target attackability. If the embedding methods select targets that are intrinsically easier to attack (e.g., classes with low model confidence or high visual confusion with the source), the reported advantage merely reflects target difficulty, not semantic quality. This is especially relevant for the 'distant class relationships' claim: 'distant' in an embedding space may coincide with visually similar but semantically unrelated classes under the attacked model. Please control for this by including a random-target baseline matched on marginal label frequencies, or by matching selected target sets on the attacked model's own softmax margins, and report per-target success rates.","section":"Abstract; experiments (not numbered in the provided text)"},{"comment":"The notion of 'distant class relationships' is not operationally defined. If WordNet distance is measured by path length while embedding distance is measured by cosine similarity, the two methods select different target sets by construction. The claim that embedding-based selection is better for distant classes needs a concrete definition of 'distance' and a demonstration that the advantage is not an artifact of the distance measure or of the attacked model's feature geometry. A cross-condition comparison with matched target semantic-distance bins would address this.","section":"Abstract; 'particularly for distant class relationships'"},{"comment":"The generality claim is too strong for the evidence stated. Three vision models and five attacks is a reasonable start, but 'across architectures and datasets' requires a broader and more systematic evaluation—for example, CNNs versus ViTs, multiple dataset domains, and attacks with different budgets and threat models. The current abstract reports only examples, not a systematic ablation or transfer study. Please either provide such evidence or temper the claim to the tested settings.","section":"Abstract; 'across architectures and datasets'"},{"comment":"The claim that 'static testing of target labels offers a preliminary assessment of the effectiveness of similarity sources, a priori testing' is not operationalized. What is the static test? How does it relate to the downstream attack success? Without a definition and supporting experiments, this observation remains anecdotal. Please state the proposed static metric and show that it predicts the ranking of similarity sources.","section":"Abstract; 'a priori testing'"}],"minor_comments":[{"comment":"The body text supplied to me is heavily corrupted or unreadable in many places; only the abstract and fragments are intelligible. If this reflects the submitted PDF, the manuscript cannot be properly reviewed and a clean version must be provided.","section":"Full text"},{"comment":"Please define all acronyms and model/attack names on first use; also specify which CLIP variant, TinyLLAMA checkpoint, and BERT model are used.","section":"Abstract"},{"comment":"Please include a reproducibility statement with code/data availability and attack hyperparameters (e.g., number of iterations, epsilon bound, loss function) for each of the five attacks.","section":"Experiments (to be added)"}],"recommendation":"major_revision","confidential_remarks":"The provided full text is largely unreadable, so my assessment rests almost entirely on the abstract. The editor may want to verify that the submitted PDF is intact. The target-difficulty confound is serious and should be addressed experimentally before publication; it is fixable with additional control conditions, so I do not recommend rejection at this stage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things about this paper. First, the idea is a genuinely useful incremental step: use pretrained language and vision-language models (BERT, TinyLLAMA, CLIP) to pick adversarial target labels, instead of random, prediction-based, or fixed lexical choices like WordNet. That is a sensible, testable proposal, and the notion of \"static a priori testing\" of similarity sources before running attacks is a nice practical touch. Second, the empirical claim in the abstract—that these embedding sources \"consistently\" beat WordNet, especially for distant classes—is not backed by anything we can check. The full text I received is corrupted, so only the abstract is readable. No variance, no statistical tests, no per-target breakdown, no code or data artifacts. That alone would make me wary, but the stress-test note points at a deeper issue that the abstract cannot rule out: the comparison may be confounded by target difficulty. If the embedding-based methods pick targets that are intrinsically easy to attack under the specific vision model (visually confusable, low-confidence, or high-margin classes), then the reported advantage is about the targets, not about the semantic quality of the embeddings. WordNet is purely symbolic and may systematically select harder targets. The \"distant class relationships\" claim is exactly where this bites—semantic distance can correlate with visual similarity in ways that make attacks easier for unrelated reasons. Without controlling for per-target attackability—e.g., matching on the attacked model's softmax margins, or comparing against a random-target baseline with identical label frequencies—the headline conclusion is unsupported. That said, this is a fixable experimental design problem, not a fatal conceptual one. The authors seem to be thinking clearly about an overlooked component of targeted attacks, and the framework is reproducible in principle. If they ship code and data, the field gains a useful benchmarking tool. My recommendation: send it to peer review, but the reviewers should push hard for a confound-controlled comparison and for error bars. The idea deserves referee time; the claim as stated does not yet deserve acceptance.","headline":"Plausible incremental idea for adversarial target selection, but the evidence as presented is thin and the headline comparison may be confounded by target difficulty.","tokens_in":6926,"tokens_out":1067,"would_cite":false,"duration_ms":13657,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pretrained language and vision-language models can select adversarial target labels by semantic similarity, beating static lexical resources such as WordNet.","keywords":["targeted adversarial attacks","target label selection","semantic similarity","language models","vision-language models","CLIP","BERT","adversarial benchmarks"],"falsifier":"Take a vision model and an attack not among the three models and five attacks tested, choose targets by the paper's semantic rankings and by WordNet, and compare success rates on semantically distant pairs. If the lexical or random targets match or beat the embedding-based targets, the generality claim fails; likewise, if the rank correlation between embedding similarity and empirical per-target success is near zero across all classes, the proxy is not doing the work.","tokens_in":6189,"feed_emoji":"🎯","tokens_out":7088,"duration_ms":72025,"temperature":0.7,"pith_summary":"In targeted adversarial attacks on image classifiers, the attacker chooses a wrong label to force, and that choice largely determines whether the attack succeeds. This paper claims that pretrained language and vision-language models — BERT, TinyLLAMA, and CLIP — can select those target labels by semantic similarity to the true class, producing both easy (semantically close) and hard (semantically distant) attack scenarios. Evaluated on three vision models and five attack methods, the semantically chosen targets consistently work as adversarial targets and beat a static lexical database, WordNet, especially when the true and target classes are far apart. The authors argue this makes adversarial testing interpretable, reproducible, and scalable: the target-ranking itself can be tested before running any attack, as an a priori check of the similarity source.","feed_headline":"Language-model similarity beats WordNet at picking attack targets","feed_subtitle":"Semantic rankings from BERT, TinyLLAMA, and CLIP make reproducible best- and worst-case adversarial tests.","key_machinery":"The mechanism is cross-modal semantic similarity: take the ground-truth class name and every candidate target class name, embed them with a pretrained text encoder (BERT, TinyLLAMA, or CLIP), and rank candidate targets by embedding similarity to the ground-truth label. The ranked list defines the attack scenarios — most similar labels as one end, least similar as the other — and can be evaluated statically, without running attacks, as an a priori test of the similarity source. What carries the argument is the assumption that similarities computed in a text-pretrained space transfer to the difficulty of fooling a vision classifier.","core_discovery":"The central claim is that target-label selection, not just the perturbation method, is a major determinant of targeted attack success, and that cross-modal semantic similarity from pretrained models is the right tool for choosing targets. The paper builds best- and worst-case scenarios by ranking all candidate labels by their semantic similarity to the ground-truth label in embeddings from BERT, TinyLLAMA, and CLIP, then attacking vision classifiers (three architectures) with five attack methods. Across these setups, the similarity-based targets are practical adversarial targets and outperform static lexical databases such as WordNet, with the largest gap for distant class relationships — ca","pith_inferences":["Because the target-ranking step only needs class names, the method should extend to very large label sets (for example, the full 21,000-class ImageNet) where lexical resources such as WordNet become sparse; the paper does not test this scale.","Embedding similarity can encode corpus co-occurrence and dataset bias as much as genuine semantic distance, so some of the reported 'distant' targets may be easy for non-semantic reasons; separating those two effects would sharpen the benchmark.","The same semantic ranking could be used to predict natural model confusions, not just adversarial ones: classes that are close in the embedding space may be the ones a vision model already confuses, which would let the a priori test double as a diagnostic for learned representations."],"forward_implications":["If semantic similarity predicts target difficulty, attack success rates can be reported by semantic distance, making benchmark results interpretable rather than target-dependent.","The same target-ranking method can be applied to any new dataset or model without retraining, since it only needs class names.","Static a priori testing lets researchers compare similarity sources before expensive attack runs.","Best- and worst-case target selection brackets robustness, exposing where a model is most and least easily fooled.","Semantic target selection overtakes static lexical resources like WordNet for far-apart classes, where ontology links are missing."],"supporting_citations":[],"fun_headline_variants":["Semantic similarity from LMs beats WordNet for attack targets","Use BERT, TinyLLAMA, or CLIP to pick adversarial targets","Language models rank labels better than WordNet for attacks","Cross-modal similarity improves adversarial target selection","Pretrained LMs surpass WordNet in choosing attack targets"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that semantic similarity measured in a text-pretrained embedding space reliably predicts how easy it is to push a vision model to a given wrong label, for models, attacks, and datasets beyond the ones tested.","fun_headline_variants_meta":{"raw":{"variants":["Semantic similarity from LMs beats WordNet for attack targets","Use BERT, TinyLLAMA, or CLIP to pick adversarial targets","Language models rank labels better than WordNet for attacks","Cross-modal similarity improves adversarial target selection","Pretrained LMs surpass WordNet in choosing attack targets"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000325,"raw_usage":{"total_tokens":1645,"prompt_tokens":715,"completion_tokens":930,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":459,"completion_tokens_details":{"reasoning_tokens":849}},"tokens_in":459,"tokens_out":930,"duration_ms":8743,"temperature":1.0,"reasoning_tokens":849,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T19:57:52.704219+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a vision model and an attack not among the three models and five attacks tested, choose targets by the paper's semantic rankings and by WordNet, and compare success rates on semantically distant pairs. If the lexical or random targets match or beat the embedding-based targets, the generality claim fails; likewise, if the rank correlation between embedding similarity and empirical per-target success is near zero across all classes, the proxy is not doing the work.","supporting_citations":[],"review_version":1}