{"id":"3e77d63f-e631-4067-86dc-3ce8fac8dc75","arxiv_id":"2412.19523","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"Using transferable adversarial attacks like MIG and GRA inside AttEXplore raises insertion scores on ImageNet, but the best attack is chosen post hoc on the test set and deletion scores worsen.","lead":"The paper swaps nine adversarial attack methods into an existing explanation tool, AttEXplore, and reports better attribution scores on five image classifiers. The claimed improvements are hard to trust because the best attack is picked after seeing the test results, deletion scores get worse, and no error bars are shown.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"AttEXplore++ is a test-set oracle: Section V-D's per-model MIG/GRA choices match the best rows of Table III, so the claimed 7.57% average gain is selection bias, not evidence for a single improved method.","rationale":"I read the paper as claiming that plugging transferable adversarial attack gradients into AttEXplore's path-integral attribution yields a reproducible, better attribution method, quantified by the abstract's 7.57% and 32.62% improvements. For that claim to hold, the reported improvement must come from a fixed method or a validation-based selection, not from choosing the best attack per model on the same test set used for evaluation. Section V-D's per-model choices exactly reproduce the bold best-insertion entries of Table III, which is a classic test-set oracle. This concern is concrete, internally verifiable from the paper's own tables, and independent of contested definitions of attribution. The reader's weakest assumption about Eq. (5) is real but less decisive for the relative claim: AttEXplore and AttEXplore++ share Eq. (5), so a flawed path-integral definition would not by itself explain the reported margin; it would only change what both quantities mean. The oracle flaw directly undermines the headline numbers. The deletion-score contradiction in Table II reinforces the same conclusion. Because the reader already recommended REJECT, my stress-test does not change the verdict; it supplies a sharper, more directly testable basis for that rejection. The paper could be salvaged by fixing a single attack method, reporting held-out selection, and adding error bars, but as written the central empirical claim is not supported.","tokens_in":39609,"tokens_out":7125,"duration_ms":65863,"concrete_test":"Recompute the comparison with a fixed, prespecified strategy: run AttEXplore+ using MIG on all five models and GRA on all five models with the same hyperparameters, and average Table II-style insertion and deletion scores over a held-out set of 500 ImageNet images not used for any selection. Also perform nested selection: split the 1000 images into 500 selection and 500 test images, choose MIG versus GRA per model on the selection split by insertion score, then report test insertion and deletion scores. If the fixed-strategy average improvement over AttEXplore falls below 7.57%, or if the held-out selected strategy differs from the paper's MIG/GRA assignment, the claimed improvement is an artifact of test-set oracle selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim, that AttEXplore++ improves over AttEXplore by 7.57% and over other interpretability methods by 32.62%, depends on a comparison of a method selected after inspecting the same test results. In Section V-D the authors state they \"chose the most optimal strategy\" and use MIG for Inception-v3 and ViT-B/16 and GRA for ResNet-50, VGG16, and MaxViT-T. These are exactly the bolded best-insertion entries in Table III for each model. AttEXplore++ is therefore not a single algorithm; it is the argmax over nine attack variants evaluated on the same 1000 images reported in Table II. Selecting the best of nine rows guarantees an improvement over any fixed row even under pure noise, and no validation split or model-selection procedure is described. This is the most load-bearing weakness because the abstract's headline numbers are the paper's central contribution and they depend on this post hoc oracle. Recomputing from Table III, a fixed MIG strategy gives an average insertion improvement of about 6.2% over AttEXplore, and a fixed GRA strategy gives about 3.3%, both below the claimed 7.57%; fixed GRA on ViT-B/16 and fixed MIG on MaxViT-T are actually worse than AttEXplore. Additionally, Table II shows AttEXplore++ has worse deletion scores than AttEXplore on all five models, so the claim of overall improvement is not supported by the secondary metric either. The Eq. (5) path-integral definition is inherited from AttEXplore and deserves scrutiny, but the relative comparison to AttEXplore is not invalidated by that definition alone; the oracle selection is the load-bearing fault for the paper's headline empirical claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the authors' previous AttEXplore method by integrating transferable adversarial attack methods (MIG, GRA, and others) into a path-integral attribution formula, Eq. (5). The proposed AttEXplore+ framework is evaluated on five ImageNet classifiers (Inception-v3, ResNet-50, VGG16, MaxViT-T, ViT-B/16) using insertion and deletion scores. The paper further claims that an optimized variant, AttEXplore++, improves over AttEXplore by 7.57% on average and over other state-of-the-art interpretability methods by 32.62%, and it presents ablation studies of diversity probability, perturbation rate, and noise amplitude.","tokens_in":39950,"tokens_out":9343,"duration_ms":80253,"significance":"If the empirical claims were valid, the paper would make a useful empirical contribution by showing that transferable adversarial attacks can improve attribution maps, and the ablation studies on randomness and hyperparameters would be of practical interest. The manuscript is also accompanied by an anonymous code release, which is a positive practice. However, the central evaluation protocol is flawed: AttEXplore++ is defined as a per-model oracle chosen from the test results in Table III, and the reported headline numbers are not reproducible from the tables. The path-integral attribution definition in Eq. (5) is taken from prior work without validation, and the claimed dominance over baselines is contradicted by the paper's own Table II on some models. These issues undermine the main contribution as stated.","major_comments":[{"comment":"AttEXplore++ is not a single method but a post hoc oracle. The text states that the authors \"chose the most optimal strategy\" and selects MIG for Inception-v3 and ViT-B/16 and GRA for ResNet-50, VGG16, and MaxViT-T. These choices coincide exactly with the best insertion scores in Table III for each model. Because the selection is made on the same 1000-image test set that is then used for the final comparison in Table II, the reported average improvement is the result of maximizing over nine attack variants, not an evaluation of a single proposed algorithm. No validation split, nested model-selection procedure, or significance test is described. Recomputing from Table III, a fixed MIG strategy gives an average insertion gain of about 6.2% over AttEXplore, and a fixed GRA strategy gives about 3.3%; both are below the claimed 7.57%, and fixed MIG on MaxViT-T or fixed GRA on ViT-B/16 is worse than AttEXplore. The central empirical claim must be re-evaluated with a method selected on a validation set or reported separately for each attack variant.","section":"Section V-D, Table III"},{"comment":"The reported improvement numbers are not reproducible from the data in Table II. For insertion scores, the per-model relative gains of AttEXplore++ over AttEXplore are approximately 15.6% (Inception-v3), 6.3% (ResNet-50), 15.2% (VGG16), 8.3% (MaxViT-T), and 5.4% (ViT-B/16), which average to about 10.2%, not 7.57%. Moreover, on MaxViT-T the SmoothGrad baseline has an insertion score of 0.6432, which is higher than AttEXplore++'s 0.6368, contradicting the claim that AttEXplore++ is best across all models. The deletion scores of AttEXplore++ are worse than those of AttEXplore on all five models (e.g., 0.0496 vs. 0.0288 on Inception-v3), contradicting the conclusion that both metrics improve. The 32.62% gain over state-of-the-art methods also needs an explicit aggregation formula; using the best non-AttEXplore baseline in each column of Table II gives an average relative gain of roughly 25.6%, and on MaxViT-T the gain is negative.","section":"Section V-E, Table II"},{"comment":"The specification of AttEXplore++ is internally inconsistent. The text says 'For ResNet-50, VGG16, and MaxViT-T models, we applied the GRA method', but Table III shows that the best insertion score for VGG16 is achieved by MIG (0.3494), not GRA (0.3257), and the AttEXplore++ entry in Table II for VGG16 is 0.3494, matching MIG. The method description and the reported configuration must be reconciled so that the evaluated model is precisely defined.","section":"Section V-D, Table III"},{"comment":"The attribution definition A_i = ∫ Δx_t · g(x_t) dt is stated without derivation, axiomatic justification, or empirical validation, and it is inherited from AttEXplore [20]. Since all reported improvements are measured on attribution maps produced by this formula, a failure of Eq. (5) to capture true feature importance would invalidate the comparison. The authors should include sanity checks, such as synthetic data with known ground-truth features or model-randomization tests, and relate Eq. (5) to standard attribution axioms. This is a correctness-risk concern, not a claim that the definition is necessarily wrong.","section":"Section IV-A, Eq. (5)"},{"comment":"The paper claims that AttEXplore+ integrates 10 transferable attack methods, but Table III reports experimental results for only nine variants, and two methods listed in Table I (NAA and FSPS) are absent from the reported experiments. The claim of comprehensive coverage of transferable attack types is therefore not supported by the presented evidence.","section":"Section IV-B, Tables I and III"}],"minor_comments":[{"comment":"There is a typo in 'diversity probabilit' and the noise formula g'(x) = g(x) + η is not connected to the experiments; the authors should specify how differential-privacy noise is implemented in each attack method and what values of σ are used.","section":"Section IV-C"},{"comment":"The setting 'perturbation rate ϵ to 16' conflicts with the definition of ϵ as an Lp perturbation bound in Eq. (3); the authors should clarify whether ϵ denotes the total perturbation budget or a per-step bound.","section":"Section V-D"},{"comment":"The embedded figures are rendered as sequences of raw glyph codes and are unreadable; the captions and plots need to be regenerated so that the results of the seed, DP, ϵ, and β experiments can be inspected.","section":"Figures 3-6"},{"comment":"The claim that higher DP values are recommended on ResNet-50, VGG16, and ViT-B/16 is based on a figure that is currently unreadable; provide numerical tables or a readable figure to support the recommendation.","section":"Section V-H, Figure 4"},{"comment":"The bold/underline formatting does not follow the stated protocol: for MaxViT-T, the highest insertion score is SmoothGrad's 0.6432, which should be bold, not AttEXplore++'s 0.6368.","section":"Section V-E, Table II"}],"recommendation":"reject","confidential_remarks":"The paper is an incremental extension of the authors' own AttEXplore, and the central evaluation is compromised by test-set oracle selection: AttEXplore++ is the argmax over attack variants on the same benchmark that is used for the final comparison. The reported 7.57% and 32.62% improvements are not reproducible from the tables, and the paper's own Table II contradicts the claim of universal superiority (e.g., SmoothGrad beats AttEXplore++ on MaxViT-T insertion, and deletion scores are uniformly worse). These are load-bearing errors that would require redoing the experiments with a validation-based selection procedure and correcting the reported numbers. Even then, the conceptual novelty beyond AttEXplore is modest. I would not recommend further review unless the evaluation and claims are substantially revised."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the useful content is Table III, a clean comparison of nine transferable attacks inside the AttEXplore path-integral attribution on five ImageNet models. The advertised 7.57% improvement is not trustworthy: AttEXplore++ is a per-model argmax over those nine rows, chosen after inspecting the same test results, and the arithmetic doesn't reproduce from Table II.\n\nCredit where due: Table III is new and genuinely useful. It shows MIG and GRA consistently beat simpler attacks like BIM and PGD for attribution, and the per-model breakdown is exactly what someone working on adversarial-attack interpretability would want. The randomness and parameter sweeps are less informative, but they at least fix seeds and vary three values. Code is promised via an anonymous link.\n\nSoft spots, in proportion: (1) The oracle selection is load-bearing. Section V-D says \"we chose the most optimal strategy\" and picks MIG for Inception-v3 and ViT-B/16 and GRA for the other three; those are precisely the bolded best-insertion entries in Table III. AttEXplore++ is the best of nine, not a single method, so the 7.57% and 32.62% gains are selection bias. A fixed MIG strategy gives roughly 6.2% and fixed GRA roughly 3.3%, and both lose on some models. (2) The tables show deletion scores worse than AttEXplore on all five models, which contradicts the conclusion's \"substantial improvements in insertion and deletion scores.\" The paper's own argument that insertion is primary softens this, but the sentence should change. (3) The transferability claim is not isolated. There is no actual cross-model transfer experiment; the attacks run on the same model being explained. That is a framing gap, not a math error. (4) The path-integral definition is inherited from AttEXplore and not validated, so the whole evaluation measures changes in that quantity. For a relative comparison against AttEXplore this is acceptable.\n\nWho this is for: people who want a quick map of which transferable attacks improve this style of attribution. It deserves a serious referee because the empirical question is legitimate and Table III is reproducible in principle. But I would send it back for major revision: fix one attack per model or hold out a validation set, recompute the headline numbers, report variance, and either test cross-model transfer or stop claiming it. As submitted, I would not accept.\n\nRecommendation: send to peer review with a clear expectation of heavy revision, not desk reject.","headline":"Table III is the useful empirical core; the headline gains are a test-set oracle and the numbers don't reproduce from the tables.","tokens_in":40519,"tokens_out":3321,"would_cite":false,"duration_ms":31261,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Transferable adversarial attacks, by defining the integration path, improve image attribution scores by 32.62% over prior interpretability methods.","keywords":["interpretability","attribution methods","transferable adversarial attacks","integrated gradients","vision transformers","ImageNet","insertion and deletion scores","explainable AI"],"falsifier":"Train a small classifier on synthetic images where the only decision-relevant feature is a known patch, run AttEXplore++ on them, and check whether the highest-attributed pixels coincide with that patch. If the path integral points elsewhere while insertion and deletion scores remain high, the reported improvements would reflect artifacts of the score, not better explanations.","tokens_in":39403,"feed_emoji":"🧠","tokens_out":5276,"duration_ms":48546,"temperature":0.7,"pith_summary":"This paper claims that the quality of AI image explanations can be improved by choosing better adversarial attacks to define the path along which attributions are computed. It extends the AttEXplore framework, whose attribution integral runs along a non-linear adversarial attack path, by swapping in highly transferable attack methods—MIG for Inception-v3 and ViT-B/16, GRA for ResNet-50, VGG16 and MaxViT-T. On 1000 ImageNet samples, the resulting AttEXplore++ outperforms AttEXplore by 7.57% and other interpretability baselines by 32.62% in insertion and deletion scores. The paper also reports that randomness has little effect on the scores and that larger perturbation rates and noise amplitudes generally help. If correct, this shows that adversarial transferability, not just local gradient information, is a key ingredient in producing useful explanations.","feed_headline":"Transferable attacks lift AI attribution scores by 32.6%","feed_subtitle":"MIG and GRA attack paths inside the attribution integral beat 11 baselines across five ImageNet models.","key_machinery":"The load-bearing object is the adversarial path attribution integral, $A_i = \\int \\Delta x_t \\cdot g(x_t)\\,dt$, which attributes importance by accumulating the inner product of the input step $\\Delta x_t$ with the attack gradient $g(x_t)$ along the non-linear trajectory from the original image to an adversarial image. The paper's innovation is to choose which attack defines this path: MIG, which combines momentum with integrated gradients, and GRA, which samples gradient-relevant neighborhoods, both producing highly transferable adversarial examples. That transferability, rather than the path geometry alone, is what the paper identifies as the driver of better attribution scores.","core_discovery":"The central claim is that substituting transferable adversarial attacks into the path-integral attribution formula yields more accurate and robust explanations. Specifically, the attribution $A_i = \\int \\Delta x_t \\cdot g(x_t)\\,dt$ is computed along the non-linear path traced by an iterative attack, and the gradient $g$ is supplied by the attack method. When $g$ comes from MIG (momentum integrated gradients) or GRA (gradient relevance attack) rather than from plain BIM or PGD, insertion and deletion scores improve on every model tested. The paper reports an average gain of 7.57% over AttEXplore and 32.62% over eleven competing interpretability algorithms, establishing that the transferability of the attack is strongly correlated with attribution performance.","pith_inferences":["Because the paper never checks attributions against ground-truth features, the 32.62% headline is a gain on insertion and deletion scores, not necessarily on explanation correctness; a human-subject or ground-truth benchmark would be needed to confirm the practical value.","If transferability is the active ingredient, then future transfer attacks with even higher cross-model success should keep pushing these scores—an easily testable prediction the paper does not make.","The per-model choice of attack (MIG for some architectures, GRA for others) suggests a selection heuristic: match the attack's inductive bias to the model family; this could be formalized as an automated per-model attack selector.","The robustness to randomness is measured only across three seeds on 1000 samples; a tighter variance analysis over more seeds and more images would be the natural next check."],"forward_implications":["On all five tested models, replacing the attack in the attribution path with MIG or GRA raises insertion scores above every baseline, so attack selection is now a first-class design choice for attribution methods.","The reported 7.57% gain over AttEXplore and 32.62% gain over other interpretability algorithms imply that transferability is at least as important as the path-integral geometry itself.","Attribution performance is stable across random seeds, so results from the framework are reproducible despite the stochastic components in DIM, TIM, and GRA.","The parameter study indicates that users should tune perturbation rate $\\epsilon$ and noise amplitude $\\beta$ per model, since CNN and transformer responses differ."],"supporting_citations":[{"why":"Supplies the base AttEXplore framework and the path-integral attribution definition that this paper extends.","marker":"[20]"},{"why":"Provides the MIG gradient update used for Inception-v3 and ViT-B/16.","marker":"[39]"},{"why":"Provides the GRA gradient update used for ResNet-50, VGG16, and MaxViT-T.","marker":"[45]"},{"why":"Establishes the adversarial-path attribution approach and the insertion/deletion evaluation metrics.","marker":"[31]"},{"why":"Defines integrated gradients, the original path-integral attribution method that motivates replacing the straight-line baseline with an adversarial path.","marker":"[24]"},{"why":"Defines the basic iterative method that generates the non-linear attribution path.","marker":"[47]"}],"fun_headline_variants":["MIG and GRA attacks improve attribution across five ImageNet models","AttEXplore++ uses attack paths to explain nets more reliably","Transferable attacks yield 32.6% better attribution than baselines","Adversarial path integrals sharpen AI explanations on ImageNet"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole comparison assumes that the adversarial path integral in Eq. (5) is a faithful measure of which features truly drive the model's decision, an assumption inherited from AttEXplore and never validated against ground-truth importance in this paper.","fun_headline_variants_meta":{"raw":{"variants":["MIG and GRA attacks improve attribution across five ImageNet models","AttEXplore++ uses attack paths to explain nets more reliably","Transferable attacks yield 32.6% better attribution than baselines","Adversarial path integrals sharpen AI explanations on ImageNet"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000388,"raw_usage":{"total_tokens":2028,"prompt_tokens":905,"completion_tokens":1123,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":1049}},"tokens_in":521,"tokens_out":1123,"duration_ms":11192,"temperature":1.0,"reasoning_tokens":1049,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:15:03.959027+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a small classifier on synthetic images where the only decision-relevant feature is a known patch, run AttEXplore++ on them, and check whether the highest-attributed pixels coincide with that patch. If the path integral points elsewhere while insertion and deletion scores remain high, the reported improvements would reflect artifacts of the score, not better explanations.","supporting_citations":[{"cited_title":"Attexplore: Attribution for explanation with model parameters exploration,","cited_arxiv_id":null,"evidence_quote":"Supplies the base AttEXplore framework and the path-integral attribution definition that this paper extends."},{"cited_title":"Transferable adversarial attack for both vision transformers and convolutional networks via momentum integrated gradients,","cited_arxiv_id":null,"evidence_quote":"Provides the MIG gradient update used for Inception-v3 and ViT-B/16."},{"cited_title":"Boosting adversarial transferability via gradient relevance attack,","cited_arxiv_id":null,"evidence_quote":"Provides the GRA gradient update used for ResNet-50, VGG16, and MaxViT-T."},{"cited_title":"Explaining deep neural network models with adversarial gradient integration,","cited_arxiv_id":null,"evidence_quote":"Establishes the adversarial-path attribution approach and the insertion/deletion evaluation metrics."},{"cited_title":"Axiomatic attribution for deep networks,","cited_arxiv_id":null,"evidence_quote":"Defines integrated gradients, the original path-integral attribution method that motivates replacing the straight-line baseline with an adversarial path."}],"review_version":1}