{"id":"10e431ad-a083-4d7f-9127-a9772db85dfd","arxiv_id":"1908.02686","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FGVis produces fine-grained, faithful visual explanations by optimizing a pixel mask and clipping gradients during backpropagation to block adversarial evidence without added hyperparameters.","lead":"This paper introduces FGVis, a method that explains a neural network's image prediction by finding the specific pixels the network relies on, while a new gradient-filtering trick stops the explanation from inventing fake evidence. It produces fine-grained, image-like explanations that can be fed back into the model, and it performs well on medical eye-scan lesion localization and on detecting color biases.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 4 permits decreasing activations to zero, and signed classification weights can turn such deletions into adversarial target evidence; Table 1 tests only additive generation, so the defense's guarantee is unestablished.","rationale":"The paper's contribution is the adversarial defense; without it the method is a fairly direct masked-perturbation optimizer. The defense's guarantee rests on Eq. 4, but that constraint is one-sided in a critical way: all decreases are allowed, and decreases of negatively weighted features can raise an absent class's score. This is not merely abstract; trained classification layers have signed weights, so deletion can act as adversarial evidence. The paper's validation omits exactly this direction by testing only the generation game from the zero image. A second implementation gap compounds the issue: Eq. 6 clips gradients but does not clamp the forward pass, so even the stated bound is not guaranteed for the final explanation. I therefore cannot treat the defense as established. The reader's CONDITIONAL verdict already identified the zero-activation assumption as the weak point; this attack makes the failure mode concrete. If the proposed deletion-game test comes back clean, the practical claim survives on this benchmark and the conditional acceptance can stand. If it does not, the central novelty is unsupported and stronger action would be warranted. Given the current evidence, the reader's verdict remains CONDITIONAL, with the additional condition that the deletion-based attack must be tested before the defense claim is accepted.","tokens_in":19371,"tokens_out":12931,"duration_ms":152493,"concrete_test":"Rerun the Table 1 adversarial-class experiment with the deletion game: start from e=x (mask initialized to ones) instead of the zero-start generation game, keep the same 1000 ImageNet validation images, least-likely target class, λ=0, and the Eq. 6 gradient clipping, and maximize the target-class probability. If a non-negligible fraction of images yields a high-confidence explanation for the absent class, the defense does not block deletion-based adversarial evidence and the central claim of §3.2 fails; if the fraction stays near zero on this real-network benchmark, the loophole is at least not exploitable by that optimizer.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of §3.2 is that the activation-range constraint of Eq. 4 prevents the optimization from generating adversarial evidence. This is not a consequence of the stated assumption that zero activations have the lowest information. For a ReLU network, Eq. 4 forces each hidden activation h_i^l(e) to lie in [0, h_i^l(x)], but it explicitly permits h_i^l(e)=0 even when h_i^l(x)>0. The final logit for the adversarial target class c_A is linear in the top-layer activations, logit_cA(e) = Σ_i W_cA,i h_i^L(e) + b_cA, and the weights W_cA,i have mixed signs. If a feature that is active in the original image has a negative weight for c_A, then deleting that feature by setting its activation to zero increases logit_cA(e) above its original value. An explanation for a visually absent class can therefore be produced by deleting existing evidence, not by creating new evidence. The defense does not block this: a decrease to zero is inside the bounds of Eq. 4, and Eq. 6 only zeroes gradients that would violate those bounds. The adversarial validation in Table 1 and Table A1 uses the generation game starting from a zero/black image, so it measures whether the optimizer can add hallucinated features, not whether the feasible set of Eq. 4 contains an adversarial explanation obtained by deletion. Additionally, the forward pass is not clamped, so a numerical overshoot that violates Eq. 4 can persist in the final explanation. The near-zero success rates in Table 1 are thus compatible with the defense being a heuristic that blocks the particular attack trajectory tested, not a guarantee that adversarial evidence is impossible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FGVis, a post-hoc optimization-based visual explanation method that produces fine-grained, image-space explanations by optimizing a mask-based perturbation of the input. Its main technical contribution is an adversarial-defense technique that clips gradients during backpropagation whenever a hidden activation would exceed the range of activations produced by the original image (Eqs. 4–6). The defense is designed to prevent the optimization from hallucinating adversarial evidence while avoiding human-tuned hyperparameters. The authors evaluate the defense on an adversarial-generation task (Table 1), compare faithfulness against Grad-CAM, LIME, RISE, and sliding-window occlusion using the deletion metric (Table 2), and apply FGVis to weakly supervised lesion detection in diabetic retinopathy fundus images (Table 3). The paper also presents qualitative results on class discriminability, fine-grained detail, and training-data color bias.","tokens_in":19669,"tokens_out":6189,"duration_ms":67228,"significance":"If the central claims held as stated, the work would be valuable: it offers a parameter-free mechanism for constraining explanation optimization to use only evidence present in the original image, which would be a practically useful alternative to low-resolution masking or stochastic defenses. The derivation of the backward-pass clipping rule (Eq. 6) is clean and readily implementable, and the adversarial-generation experiment in Table 1 is a useful stress test showing that the defense blocks additive hallucination. The medical localization study goes beyond toy benchmarks. However, the paper's core guarantee that the defense prevents adversarial evidence is stronger than what Eq. 4 actually enforces, and several quantitative comparisons are not evaluated on equal terms. These issues bear directly on the paper's main contribution, so the manuscript needs substantive revision and additional experiments rather than cosmetic changes.","major_comments":[{"comment":"The claim that the activation-bounding constraint prevents adversarial evidence is too strong. For a ReLU network, Eq. 4 allows any activation h_i^l(e) to decrease to zero even when h_i^l(x) > 0. Because the logit for a target class c_A is linear in the top-layer activations, logit_{c_A}(e) = Σ_i W_{c_A,i} h_i^L(e) + b_{c_A}, deleting a feature that is active in x and has a negative weight for c_A increases logit_{c_A}(e). The feasible set of Eq. 4 therefore contains explanations for a visually absent class c_A that are obtained by deleting existing evidence, not by generating new evidence. Table 1 and Table A1 validate the defense only with the generation game starting from a zero image, so the defense's guarantee against deletion-based adversarial evidence is not established by the reported experiments. Please either extend the validation to the deletion/repression games or explicitly restrict the claim to additive hallucination.","section":"§3.2, Eqs. (4)–(6)"},{"comment":"The faithfulness evaluation is partly circular. The deletion metric measures how quickly the target-class probability drops when pixels are removed in order of an importance map, and that importance map is exactly the mask produced by the deletion game (Eq. 3), which optimizes a sparsity-weighted objective to drop the target-class probability. A method that directly optimizes this objective is expected to score well on the deletion metric relative to methods that are not trained against it. The paper should report an independent faithfulness measure (e.g., an insertion metric or a human-grounded evaluation) or at least discuss this optimization-to-metric alignment as a limitation before claiming that FGVis 'outperforms' the reference methods by a large margin.","section":"§5.1, Table 2"},{"comment":"The comparison with Gondal et al. is not threshold-matched: FGVis is evaluated with a 25% overlap criterion, whereas the reference weakly supervised method uses a 50% overlap criterion. At the 50% overlap that is also reported in Table 3, FGVis is below Gondal et al. for H and HE and comparable for SE, so the statement that FGVis 'performs comparable or outperforms fully supervised approaches' at image-level lesion detection is not supported when the same evaluation criterion is used. The headline comparison should be made at the same threshold, or the claim should be qualified accordingly.","section":"§5.2, Table 3"},{"comment":"The forward pass is not clamped, so Eq. 4 is not actually enforced at any iteration or at the final explanation. Equation 6 only zeros gradients whose update would violate the bounds; an SGD step can still move an activation beyond a bound, after which the indicator in Eq. 6 is zero for that neuron and the violation can persist. The paper states that the technique is equivalent to adding a clipping layer that 'acts as the identity in the forward pass', which means the constraint is never imposed on forward evaluations. The authors should either apply a forward clamp at inference time or empirically verify that final explanations satisfy Eq. 4, and they should soften the wording that claims the constraint 'ensures' the allowed activation range.","section":"§3.2, Eq. (6)"}],"minor_comments":[{"comment":"Table 2 reports no error bars, confidence intervals, or numbers of runs; the claim that FGVis outperforms the baselines by a large margin should be accompanied by variance estimates over image subsets or model seeds.","section":"§5.1, Table 2"},{"comment":"The reference-method results in Table 2 are taken from Petsiuk et al. [32] rather than rerun in the authors' pipeline. Please confirm that the experimental settings such as image preprocessing, deletion protocol, and model checkpoints are identical, or note any differences.","section":"§5.1, Table 2"},{"comment":"The color-bias experiment uses small sample sizes (21 minivan and 42 school bus images); the 8.3% versus 83.3% comparison would benefit from a confidence interval or a statistical significance test.","section":"§4.3"},{"comment":"The supplementary material states that FGVis highlights mutations in the optic disk and blood vessels that are not covered by the ground-truth markings, producing visual false positives; this limitation should also be acknowledged in the main text's medical application section.","section":"A4.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be the CVPR 2019 camera-ready version. If this is a journal submission, the editorial board should consider whether the incremental contribution beyond the conference version meets the journal's novelty bar. The central defense claim requires additional experiments and a more careful statement of what the constraint actually guarantees; the suggested changes in the major comments are substantial but feasible within revision scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this is the FGVis paper — a fine-grained, image-space explanation method that is genuinely useful, and the gradient-clipping defense is a real mechanism worth remembering. The headline claim, though, is stronger than the math supports: the defense does not prove that adversarial evidence is impossible, and the main faithfulness metric is partly circular with the optimization objective.\n\nWhat is actually new: Eq. 6, the backward-pass gradient clipping. Every post-nonlinearity activation of the explanation is bounded by the range it takes in the original image, and gradients that would push past the bound are zeroed. That is a clean, hyperparameter-free mechanism I have not seen in Fong-Vedaldi or Dabkowski-Gal, and it delivers what it promises: full-resolution masks, no smoothing, valid model inputs. The paper also does the empirical work well — adversarial generation tests on four architectures, the black-image experiment, the DiaretDB1 lesion localization against expert ground truth, and a color-bias analysis that checks out quantitatively. The RSD sensitivity (88–90% vs. 50% for the CAM baseline) is the strongest independent evidence that the fine-grained masks find real lesions.\n\nWhere it is soft. The stress-test concern lands. Eq. 4 bounds activations above but explicitly allows them to go to zero, and the final logit is linear in top-layer activations with signed weights. A feature that is active in the original image and negatively weighted for the target class can be deleted to raise that class's logit — so an adversarial explanation can be constructed by deletion, entirely inside the feasible set. Table 1 starts from a zero image and tests additive generation only; it does not test the deletion path. The forward pass is not clamped either, so a numerical overshoot that violates Eq. 4 persists in the final explanation. Treat the defense as a heuristic that empirically blocks the tested attack trajectory, not as a guarantee. Second, the deletion metric comparison is partly circular — the optimization maximizes softmax consistency and the metric measures exactly that — and the baseline numbers are copied from Petsiuk et al., not rerun. The medical comparison uses a 25% overlap against 50% for the CAM baseline, which flatters a fine-grained method, and there are no error bars or released code. None of this sinks the central idea; it does mean the large-margin claims need to be read with those caveats.\n\nWho it is for: anyone working on interpretability, model validation, or weakly-supervised medical localization. It deserves a serious referee — the mechanism is novel, the empirical work is substantial, and the weaknesses are addressable. I would send it to review and ask the authors to either tighten the defense claim or add a deletion-path experiment, rerun the baselines, and report variance.","headline":"FGVis delivers a genuinely new gradient-clipping defense and fine-grained image-space masks, but the defense's guarantee is overstated — Eq. 4 permits deletion-based adversarial evidence that Table 1 never tests.","tokens_in":20281,"tokens_out":5324,"would_cite":true,"duration_ms":51265,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FGVis produces fine-grained, class-discriminative explanations directly in image space by clipping gradients that would let the explanation activate neurons beyond the range seen in the original image.","keywords":["visual explanations","convolutional neural networks","adversarial evidence","gradient clipping","fine-grained saliency","perturbation-based explanations","weakly-supervised lesion detection","model faithfulness"],"falsifier":"Run the defended optimization (generation game with $\\lambda=0$) on a large held-out set of images, targeting a class that is visually absent, and count how often the resulting explanation is classified as that class with softmax score above a fixed threshold such as 10%; the paper reports near-zero success on 1000 ImageNet images, so any material fraction of successes would refute the claim that the defense blocks adversarial evidence. A second test would construct an input and target class where legitimate evidence requires a neuron to fire more strongly than it does in the original image, and check whether the per-location clipping suppresses that evidence; if it does, the constraint is too strong for fine-grained explanations.","tokens_in":19165,"feed_emoji":"🔍","tokens_out":7601,"duration_ms":70984,"temperature":0.7,"pith_summary":"This paper proposes FGVis, a post-hoc method that explains a CNN's prediction by optimizing a sparse mask that removes pixels from the image, producing an explanation that is itself a valid model input. The paper's central technical claim is that adversarial evidence -- spurious patterns introduced by the optimization that make the explanation look confident for a class that is visually absent -- can be blocked by clipping gradients in the backward pass so that no neuron's activation in the explanation may exceed the activation range it reaches on the original image. Because the defense is parameter-free and imposes no smoothness or resolution constraint on the mask, the resulting explanations preserve edges and colors and remain fine-grained down to individual pixels. The paper reports that FGVis outperforms Grad-CAM, LIME, RISE, and sliding-window occlusion on the deletion faithfulness metric on ImageNet, and that it matches or beats fully supervised lesion-detection methods on the DiaretDB1 diabetic retinopathy dataset even though it is only weakly supervised. If the method works as described, practitioners would gain testable pixel-level evidence of what a network sees, including small structures such as microaneurysms, and a way to expose biases such as reliance on color.","feed_headline":"Gradient clipping gives pixel-level CNN explanations","feed_subtitle":"A parameter-free defense blocks invented evidence, exposing true fine-grained features and color biases.","key_machinery":"The central object is the backward-pass gradient-clipping rule of Eq. 6, which is equivalent to inserting a clipping layer after each nonlinearity whose forward pass is the identity. For every neuron $i$ in layer $l$, the layer stores bounds $b_u = \\max(0, h_i^l(x))$ and $b_l = \\min(0, h_i^l(x))$ computed on the original image $x$, and in the backward pass the incoming gradient is multiplied by the indicator that the explanation's activation $h_i^l(e_{c_T})$ lies inside $[b_l, b_u]$. This enforces the constraint of Eq. 4 -- the explanation may activate a neuron no more strongly than the original image did -- while leaving the forward model untouched, adding no hyperparameters, and imposing no smoothness or resolution limit on the mask. The mask optimization of Eq. 2 and Eq. 3 with a zero reference image then yields sparse, pixel-level explanations that are valid model inputs.","core_discovery":"The paper's central claim is that optimization-based visual explanations can be made both fine-grained and free of adversarial evidence without any human-tuned defense parameters. The mechanism is a backward-pass gradient clip: after every nonlinear layer, the explanation's activation for each neuron is silently constrained to the range that the same neuron attains on the original input, which the paper justifies by the pruning convention that zero activations carry the least information. In effect the feature set the explanation may use is a subset of the features already present in the image, so the optimizer cannot sculpt brand-new evidence. The paper validates the defense by trying to generate explanations for the least-likely ImageNet class: without the defense an adversarial explanation is produced for 100% of images, and with the defense for roughly 0% across VGG16, AlexNet, ResNet50, and GoogleNet. Built on this defense, FGVis computes deletion and preservation explanations directly in image space and, to the authors' knowledge, is the first method able to produce fine-grained explanations directly in the image space.","pith_inferences":["Because the constraint is applied per neuron at a fixed spatial location, FGVis implicitly treats features as location-bound; a testable extension would allow location-shifted evidence using the network's own translational invariances and check whether that can be added without reintroducing adversarial artifacts.","The zero-activation premise is the most promising attack surface: an adversary could try to encode a class through a combination of already-active neurons, none of which individually exceeds its original bound, which would bypass the defense without violating Eq. 4.","The color-bias analysis suggests a systematic audit recipe: compute preservation explanations for a class, swap color channels, then measure both the accuracy drop and the shift in explanation color; applied broadly, that recipe could reveal dataset biases that accuracy alone hides.","On DiaretDB1 the paper compares FGVis at a 25% overlap threshold against a published CAM baseline reported at 50% overlap; a head-to-head with identical thresholds would separate a genuine fine-grained-localization advantage from a metric artifact."],"forward_implications":["Explanations from FGVis are valid model inputs, so faithfulness metrics such as deletion curves can be computed directly on the model without proxy measures or preprocessing that could distort the result.","The parameter-free defense removes the need to tune smoothness or resolution regularizers per model, making fine-grained explanations practical across different CNN architectures.","In medical imaging, weakly supervised localization of small lesions -- such as red small dots in diabetic retinopathy -- reaches sensitivities comparable to supervised detectors designed for a single lesion type.","The method can expose training-data biases: preservation explanations of school buses consistently feature yellow, and swapping color channels collapses the model's accuracy for color-dependent classes.","The defense gives a simple verification check: if the model is confident for a class that is visually absent, the explanation should be nearly empty, so users can test an explanation's credibility directly."],"supporting_citations":[{"why":"Introduces the perturbation-based explanation paradigm and the deletion/preservation games that FGVis extends.","marker":"[17]"},{"why":"Supplies the deletion metric and the comparison results for Grad-CAM, sliding-window occlusion, LIME, and RISE reported in Table 2.","marker":"[32]"},{"why":"Provides the pruning convention that zero activations carry the least information, which justifies the activation bounds of the defense.","marker":"[22]"},{"why":"Defines the iterative least-likely class attack used to validate that the defense blocks adversarial evidence.","marker":"[27]"},{"why":"Is the DiaretDB1 dataset with expert lesion annotations used for the weakly-supervised lesion-detection evaluation.","marker":"[25]"},{"why":"Provides the weakly-supervised CAM baseline and the overlap-based evaluation protocol for lesion detection in retinal fundus images.","marker":"[18]"},{"why":"Documents the vulnerability of optimization-based explanation methods to adversarial noise, motivating the proposed defense.","marker":"[11]"},{"why":"Is the Grad-CAM baseline and the source of the fine-grained and class-discriminative criteria applied to explanations.","marker":"[35]"}],"fun_headline_variants":["No-tune defense yields pixel-accurate CNN explanations","Gradient clip blocks fake evidence for fine-grained CNN maps","Fine-grained visual explanations without tuned defenses","Gradient clipping stops adversarial artifacts in CNN explanations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The defense assumes that a neuron whose activation is zero carries the least information, so that preventing any neuron from firing more strongly in the explanation than it does in the original image is enough to stop the optimizer from inventing new evidence; if that premise fails, the clipping would either admit some adversarial artifacts or filter out legitimate fine-grained evidence that needs a neuron to exceed its original activation.","fun_headline_variants_meta":{"raw":{"variants":["No-tune defense yields pixel-accurate CNN explanations","Gradient clip blocks fake evidence for fine-grained CNN maps","Fine-grained visual explanations without tuned defenses","Gradient clipping stops adversarial artifacts in CNN explanations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000142,"raw_usage":{"total_tokens":1131,"prompt_tokens":872,"completion_tokens":259,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":198}},"tokens_in":488,"tokens_out":259,"duration_ms":3082,"temperature":1.0,"reasoning_tokens":198,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:37:19.783940+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the defended optimization (generation game with $\\lambda=0$) on a large held-out set of images, targeting a class that is visually absent, and count how often the resulting explanation is classified as that class with softmax score above a fixed threshold such as 10%; the paper reports near-zero success on 1000 ImageNet images, so any material fraction of successes would refute the claim that the defense blocks adversarial evidence. A second test would construct an input and target class where legitimate evidence requires a neuron to fire more strongly than it does in the original image, and check whether the per-location clipping suppresses that evidence; if it does, the constraint is too strong for fine-grained explanations.","supporting_citations":[{"cited_title":"Fong and Andrea Vedaldi","cited_arxiv_id":null,"evidence_quote":"Introduces the perturbation-based explanation paradigm and the deletion/preservation games that FGVis extends."},{"cited_title":"Rise: Random- ized input sampling for explanation of black-box models","cited_arxiv_id":null,"evidence_quote":"Supplies the deletion metric and the comparison results for Grad-CAM, sliding-window occlusion, LIME, and RISE reported in Table 2."},{"cited_title":"Learning both weights and connections for efficient neural network","cited_arxiv_id":null,"evidence_quote":"Provides the pruning convention that zero activations carry the least information, which justifies the activation bounds of the defense."},{"cited_title":"The DIARETDB1 diabetic retinopathy database and evaluation protocol","cited_arxiv_id":null,"evidence_quote":"Is the DiaretDB1 dataset with expert lesion annotations used for the weakly-supervised lesion-detection evaluation."},{"cited_title":"Gondal, Jan M","cited_arxiv_id":null,"evidence_quote":"Provides the weakly-supervised CAM baseline and the overlap-based evaluation protocol for lesion detection in retinal fundus images."},{"cited_title":"Real time image saliency for black box classifiers","cited_arxiv_id":null,"evidence_quote":"Documents the vulnerability of optimization-based explanation methods to adversarial noise, motivating the proposed defense."},{"cited_title":"Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Ba- tra","cited_arxiv_id":null,"evidence_quote":"Is the Grad-CAM baseline and the source of the fine-grained and class-discriminative criteria applied to explanations."}],"review_version":1}