Pith. sign in

REVIEW 4 major objections 5 minor 17 references

Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read The paper argues that a CNN image classifier's decision can be explained by two sparse sets of top-layer filters: one that alone preserves the predicted class, and one whose boosted activations switch the prediction to a chosen…

desk verdict A genuinely novel filter-mask counterfactual explainer for DCNNs, but the 'minimum' claim is uncertified, the evaluation is thin, and the method still deserves a serious referee. read the letter →

arxiv 2501.06831 v1 pith:TTB7KVAO submitted 2025-01-12 cs.CV cs.AI

classification cs.CVcs.AI
keywords ExplainableAIcounterfactualexplanationcontrastivedeepconvolutionalneuralnetworksfilter-levelinterpretabilitypost-hocexplainabilitymisclassificationanalysisCUB-2011dataset
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a way to explain a deep convolutional image classifier by looking inside it rather than by changing its input. For any image, the method learns two sparse masks over the filters of the top convolutional layer: a minimum correct (MC) mask whose filters alone keep the model's prediction at the inferred class, and a minimum incorrect (MI) mask whose filter activations, when additively boosted, flip the prediction to a chosen alter class. The masks are trained by a small explainer head on top of the frozen convolutional features, using cross-entropy with L1 sparsity penalties, plus a negative-logits term for the MC mask. The authors argue that the selected filters correspond to interpretable, class-separating concepts and that examining them reveals why a model errs, for instance when a misclassification is driven by background branches or by a missing class-defining wing spot. If this holds, explanations of DCNN decisions can be generated at the level of the features the network actually uses.

What carries the argument

The machinery is a counterfactual explanation (CFE) model attached to a pre-trained DCNN at the global average pooling layer. The convolutional feature extractor is frozen and shared; a single dense layer predicts a filter mask. For MC filters the mask is binarized through sigmoid and thresholded ReLU and multiplied onto the pooled activations, $F_{MC} \circ g$, so only the selected filters contribute to the classifier head $h$. For MI filters the mask is a nonnegative additive vector produced by ReLU and added to the pooled activations, $g + F_{MI}$. The MC mask is trained with cross-entropy, an $\ell^1$ sparsity penalty, and a negative-logits term that favors filters with high weight toward the inferred class; the MI mask is trained with cross-entropy and an $\ell^1$ penalty on the additive values. The selected filters are then visualized by their receptive fields to show which visual features separate the inferred class from the alter class.

What would settle it

Take a set of test images for a class, introduce into each image the exact features that the MI filters highlight (such as the red eye or bluish wing tinge), and count how often the model's decision flips to the alter class; if flips occur no more often than for equally many randomly chosen feature edits, the MI filters are not the minimal cause of the decision change the paper claims.

Watch

Extended reading notes

Core claim

The central discovery is that a DCNN's decision on an image can be accounted for by a small, learnable set of filters in its top convolutional layer: those that are sufficient to reproduce the decision and those whose augmented activations are sufficient to overturn it. The paper calls the first set minimum correct (MC) and the second minimum incorrect (MI). It claims that these sets are sparse, that the filters are visually interpretable as high-level concepts, and that the contrast between them provides both contrastive ("why this class and not that one") and counterfactual ("what would have to be different") explanations that make the model transparent. It further claims that the same machinery exposes faulty reasoning in misclassifications, and reports that disabling the globally important MC filters for a class collapses class recall while leaving overall accuracy nearly unchanged.

Load-bearing premise

The load-bearing premise is that modifying a filter's post-pooling activation acts as a faithful intervention on the visual feature that filter detects, so the sparse masks reveal the concepts the model actually uses.

Editorial extensions

If this is right

  • If a small MC mask suffices to reproduce the original prediction, the remaining top-layer filters are not doing load-bearing work for that image, so explanations can identify where the model's decision actually lives.
  • The MI mask gives an actionable counterfactual: boosting the pooled activations of a handful of filters flips the decision to the alter class, meaning those filters encode features that separate the two classes.
  • Misclassification analysis becomes possible by contrasting MC filters for the wrong class with MI filters for the true class; in the paper's example, a background-branch filter drives a wrong prediction and a missing red wing spot explains why the true class was not chosen.
  • Disabling the globally important MC filters for a class drops class recall sharply (e.g., 93.3% to 30% for Red-winged blackbird) while overall accuracy falls by only about 2%, indicating these filters encode class-exclusive concepts.
  • User evaluations with Explanation Satisfaction scores suggest that both expert and non-expert users find the filter-based explanations understandable and useful compared with heatmap and attribute-based baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's "minimum" is a claim about what L1-regularized training returns, not a certificate of true minimality; a stronger test would compare the MC mask against exhaustive or random subsets of the same size.
  • The causal reading is strongest when the features highlighted by MI filters are introduced into real images; the paper demonstrates this on hand-edited examples, and a systematic study across many images and classes would show whether the identified filters are genuinely sufficient to flip decisions.
  • Because the method operates on post-pooling activations, its transparency likely degrades for lower layers, where filters are less semantic; the same architecture could be tested there to map the limits of concept-level explanations.
  • The MI filters point to features whose presence pushes the model across a decision boundary, which suggests a natural connection to adversarial perturbations: features that flip decisions could be checked for overlap with adversarial patterns.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a post-hoc explainability method for DCNN image classifiers, applied to VGG-16 on CUB-2011. Given an input image, a per-class 'counterfactual explanation' (CFE) network predicts two sets of masks over the top convolutional layer's post-GAP filter activations: a binary 'minimum correct' (MC) mask that, when multiplied with the activations, is claimed to be the minimum set of filters sufficient to preserve the model's original decision, and a nonnegative 'minimum incorrect' (MI) additive perturbation that, when added to the activations, is claimed to flip the decision to a specified alter class. The masks are trained by minimizing cross-entropy, an L1 sparsity penalty, and (for MC) a negative logits loss. The paper evaluates the masks through qualitative visualizations, a misclassification case study, a user study, a class-recall ablation, and a comparison with GradCAM and SCOUT.

Significance. If the claims were fully established, the method would be a useful addition to the explainability toolbox: it is model-intrusive, produces compact filter-level explanations, and the misclassification analysis in Section 4.2.2 is a compelling demonstration that filter-level reasoning can diagnose an erroneous decision. The paper also ships a user study and an attempt at quantitative comparison, which is more than many XAI papers provide. However, the central 'minimum set' claim is currently a fitted property of a learned surrogate, not a verified property of the explained model, and the quantitative comparison is underspecified. The significance of the contribution is therefore conditional on reframing and additional validation.

major comments (4)
  1. [Section 1 and Section 3.1-3.2, Eqs. (3) and (9)] The central claim that the method identifies the 'minimum set' of filters is not established. The masks are outputs of a per-class dense network trained on the class-averaged loss, not solutions of the per-image optimization written in Eqs. (3) and (9). L1 regularization selects one sparse solution and provides no certificate that no smaller or alternative mask exists. Table 3 itself shows that at the reported lambda=2 the testing accuracy is 99.1%, meaning that for about 1% of test images the predicted MC mask does not even preserve the decision, which contradicts the 'necessary' interpretation. Please either reframe the claims as 'sparse sufficient masks' or provide per-image minimality evidence, for example by comparing predicted masks against greedy or exhaustive search on a sample of images, reporting the distribution of mask sizes, and testing whether removing any individual predicted filter breaks the prediction.
  2. [Section 4.5, Table 5] The quantitative comparison with GradCAM and SCOUT is under-specified for the proposed filter-based method. The recall and precision metrics are defined over an 'explanation region', but the paper never states how a binary MC filter mask or an MI perturbation is converted into a spatial explanation region on the input image. Without this mapping, the numbers in Table 5 are not reproducible and the comparison is not meaningful. Please specify the spatial aggregation rule, describe how part annotations are matched to the region, and ideally release code for the evaluation.
  3. [Section 3.1, Eq. (2), and Section 3.2, Eq. (8)] The paper treats scalar modification of post-GAP filter activations as equivalent to enabling or disabling visual concepts in the input. This assumption is not validated: a GAP value is an average over spatial locations, and zeroing or adding to that scalar does not remove or insert the corresponding visual feature in the image or in the feature maps. The manual image edits in Section 4.2.1 (Fig. 4) and Section 4.2.2 (Fig. 7) are a different intervention and do not test the scalar-level operation. As a result, the claims that the masks correspond to 'features and concepts' and that the model is made 'transparent' rest on an unverified causal linkage. Please add experiments that compare scalar-level interventions with spatial feature-map interventions or input-level perturbations on the same images.
  4. [Section 4.4.2, Table 3] The sparsity-loss analysis does not support the 'minimum' conclusion. The table reports average filter counts and accuracy values as a function of lambda, but it never checks whether a strictly smaller subset of the predicted filters would still suffice for a given image, nor whether a different sparse subset would be equally effective. Thus the trade-off shown in Table 3 is between the surrogate's training objective and its output size, not a demonstration that the CFE model finds the minimal sufficient set. Please include a minimality check or weaken the conclusion accordingly.
minor comments (5)
  1. [Section 3.1, Eq. (4)] The cross-entropy loss is written as if ci were a probability vector, but ci is defined earlier as a class index. Please use one-hot notation or a sparse softmax cross-entropy formulation to avoid ambiguity.
  2. [Table 2] The 'Rand. disabled' column appears to report a single random draw of filters; please state how many random subsets were used and report the variance, since the conclusion that random disabling has little effect depends on that.
  3. [Section 4.4.2, Tables 3 and 4] The 'Filters' columns report averages without standard deviations or percentiles; given that the masks vary per image, a distribution would be more informative than a point average.
  4. [Section 4.1] The reported VGG-16 test accuracy of 69.5% on CUB-2011 is not contextualized against standard baselines; adding a reference to prior reported accuracies on this benchmark would help the reader judge the explained model.
  5. [Section 4.5] The paper claims 'consistently better' recall/precision than SCOUT, but Table 5 reports only a few classes and no significance testing; please state how many images and class pairs were used and whether the differences are statistically reliable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper's mask predictors are trained and evaluated out-of-sample, and the 'minimum' overclaim is an evidence gap rather than a circular reduction.

full rationale

The paper's derivation chain is not circular in the sense required by the review criteria. The MC and MI filter masks are outputs of a learned single-layer network trained with cross-entropy, L1 sparsity, and (for MC) logits losses (Eqs. 3 and 9); the later demonstrations that the masks preserve or flip the classifier's decision are out-of-sample evaluations of that trained predictor (e.g., Table 3), which is standard supervised learning rather than a construction that forces the reported behavior. The 'minimum' wording is an overclaim, because L1 regularization selects a sparse solution and does not certify global minimality, and the average mask size varies with the chosen sparsity weight λ in Table 3; however, that is an evidentiary or correctness gap, not a circular reduction: the paper does not define minimality as the trained output and then use that same definition as proof. There are no load-bearing self-citations, no imported uniqueness theorems, and no known result merely renamed. Therefore no specific circular step can be exhibited, and the appropriate score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claim rests on four assumptions about filter semantics and the faithfulness of scalar GAP interventions, plus the heuristic that L1 sparsity produces minimal masks. The MC and MI masks are learned constructs with no independent external handle. No new physical entities are introduced.

free parameters (4)
  • lambda_MC = 2
    Hand-chosen sparsity weight for the MC mask loss in Eq. (3); Table 3 explores it only for the Red-winged blackbird class, so sensitivity across classes is unknown.
  • lambda_MI = 1
    Hand-chosen sparsity weight for the MI mask loss in Eq. (9); no sensitivity analysis is given.
  • binarization_threshold_t = 0.5
    Threshold in Eq. (1) used to binarize MC masks at inference time; no ablation is reported for other thresholds.
  • CFE dense layer weights = learned on CUB-200-2011 training split
    The 512-output dense layers for MC and MI are fit to data; every predicted mask and explanation depends on these weights, and their initialization or regularization beyond SGD is not detailed.
assumptions (4)
  • domain assumption Top convolutional layer filters encode high-level, semantically meaningful concepts.
    Invoked in Section 1 with citations to Bau et al. and Zhou et al.; the meaningfulness of explanations depends on this holding for the VGG-16 model trained on CUB.
  • ad hoc to paper Multiplying or adding scalar values to post-GAP filter activations is a faithful causal intervention on the model's visual decision process.
    Used in Eqs. (2) and (8); exact for the linear softmax head, but interpreting scalar changes as visual concept additions or removals is not independently validated.
  • domain assumption The class weights W_{k,c} in the negative logits loss correctly rank how much each filter contributes to class c.
    Used in Eq. (6); the logits are linear in the post-GAP activations, but using W as the measure of semantic importance is an interpretability choice, not a discovered fact.
  • ad hoc to paper L1-regularized minimization finds the true minimum sufficient filter masks.
    This is the load-bearing support for the 'minimum' wording in Algorithms 1 and 2; L1 returns a sparse solution but offers no certificate of minimality or uniqueness.
invented entities (2)
  • MC filter mask FM_C
    purpose: A binary selection of top-layer filters claimed to be the minimum set that maintains the source-class prediction.
    Defined in Eq. (1) and trained with Eq. (3); minimality and semantic content are asserted, not measured independently.
  • MI filter perturbation FM_I
    purpose: A nonnegative additive vector over filter activations claimed to be the minimum change that flips the prediction to an alter class.
    Defined in Eq. (7) and trained with Eq. (9); the paper provides no lower-bound or exhaustive-search check for minimality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers." pith.science (2026). https://pith.science/paper/TTB7KVAO

@misc{pith2026250106831,
  author       = {Pith},
  title        = {Pith review of: Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TTB7KVAO}},
  note         = {Machine review of arXiv:2501.06831}
}
read the original abstract

Explainability of deep convolutional neural networks (DCNNs) is an important research topic that tries to uncover the reasons behind a DCNN model's decisions and improve their understanding and reliability in high-risk environments. In this regard, we propose a novel method for generating interpretable counterfactual and contrastive explanations for DCNN models. The proposed method is model intrusive that probes the internal workings of a DCNN instead of altering the input image to generate explanations. Given an input image, we provide contrastive explanations by identifying the most important filters in the DCNN representing features and concepts that separate the model's decision between classifying the image to the original inferred class or some other specified alter class. On the other hand, we provide counterfactual explanations by specifying the minimal changes necessary in such filters so that a contrastive output is obtained. Using these identified filters and concepts, our method can provide contrastive and counterfactual reasons behind a model's decisions and makes the model more transparent. One of the interesting applications of this method is misclassification analysis, where we compare the identified concepts from a particular input image and compare them with class-specific concepts to establish the validity of the model's decisions. The proposed method is compared with state-of-the-art and evaluated on the Caltech-UCSD Birds (CUB) 2011 dataset to show the usefulness of the explanations provided.

Figures

Figures reproduced from arXiv: 2501.06831 by the authors.

Figure 1
Figure 1. Explanation provided by the proposed approach. Our method identifies the most important minimum correct (MC) and minimum incorrect (MI) filters using which the pre-trained model either predicts the input image to its original inferred class or some chosen alter class, respectively. The top-3 MC filters for the example image classified as “Red-winged blackbird" show that the red spot on this bird’s wing is the most d… view at source ↗
Figure 2
Figure 2. Overall block diagram of the proposed counterfactual and contrastive explanation model. Given an input [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Contrastive and counterfactual explanations for a sample image from CUB dataset. Contrastive explanation highlights the top-3 MC filters representing features important for the inferred class. Counterfactual explanation highlights top-3 MI filters that represent features required for the alter class. that the output dense layer after the top convolution layer is followed by ReLU activation function instead of sigmoi… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Artificially introducing the most important features relevant to the alter class identified in Fig. 3 (Bronzed [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Fig. 5 shows explanations for a test image of the “Bronzed cowbird" class that the DCNN correctly classified. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 5
Figure 5. Figure 5: Explanation comparison of GradCAM Selvaraju et al. [2017], SCOUT Wang and Vasconcelos [2020], and proposed CFE method for a query image that was classified as “Bronzed cowbird", while the alter class is set as “Red-winged blackbird". (a) GradCAM explanation for the inf…
Figure 6
Figure 6. Figure 6: Identifying the erroneous filters that result in misclassification. (a) Input image of class “Red-winged blackbird" with RFs of top-3 MC filters involved in incorrect classification of the image as “Myrtle warbler". (b) Top-3 images from the inferred class that activat…
Figure 7
Figure 7. Figure 7: Modifying the misclassified input image according to features identified by MI filters. (a) Image of class [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Misclassification case explanations of GradCAM Selvaraju et al. [2017] and SCOUT Wang and Vasconcelos [2020]. (a) GradCAM explanation for the incorrect inferred class (Myrtle warbler). (b) GradCAM explanation for the true class (Red-winged blackbird). (c) SCOUT explana…
Figure 9
Figure 9. Figure 9: Statistical analysis of MC filters predicted by CFE model for explaining VGG-16 model with respect to [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 5 canonical work pages

  1. [3]

    What do we need to build explainable ai systems for the medical domain? arXiv preprint arXiv:1712.09923,

    Andreas Holzinger, Chris Biemann, Constantinos S Pattichis, and Douglas B Kell. What do we need to build explainable ai systems for the medical domain? arXiv preprint arXiv:1712.09923,

  2. [5]

    Interpretable Neural Network Decoupling

    PMLR. Yuchao Li, Rongrong Ji, Shaohui Lin, Baochang Zhang, Chenqian Yan, Yongjian Wu, Feiyue Huang, and Ling Shao. Dynamic neural network decoupling. arXiv preprint arXiv:1906.01166,

  3. [6]

    Training Interpretable Convolutional Neural Networks by Differentiating Class-specific Filters

    Haoyu Liang, Zhihao Ouyang, Yuyuan Zeng, Hang Su, Zihao He, Shu-Tao Xia, Jun Zhu, and Bo Zhang. Training interpretable convolutional neural networks by differentiating class-specific filters. arXiv preprint arXiv:2007.08194,

  4. [8]

    Efficientps: Efficient panoptic segmentation

    Rohit Mohan and Abhinav Valada. Efficientps: Efficient panoptic segmentation. arXiv preprint arXiv:2004.02307,

  5. [9]

    Rise: Randomized input sampling for explanation of black-box models

    Vitali Petsiuk, Abir Das, and Kate Saenko. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421,

  6. [11]

    Very deep convolutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556,

  7. [13]

    Efficientnet: Rethinking model scaling for convolutional neural networks.arXiv preprint arXiv:1905.11946,

    Mingxing Tan and Quoc V Le. Efficientnet: Rethinking model scaling for convolutional neural networks.arXiv preprint arXiv:1905.11946,

  8. [14]

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. The Caltech-UCSD Birds-200-2011 Dataset. Technical Report CNS-TR-2011-001, California Institute of Technology,

Show all 17 references
  1. [15]

    Explainability of vision-based autonomous driving systems: Review and challenges

    Éloi Zablocki, Hédi Ben-Younes, Patrick Pérez, and Matthieu Cord. Explainability of vision-based autonomous driving systems: Review and challenges. arXiv preprint arXiv:2101.05307,

  2. [17]

    Object detectors emerge in deep scene cnns

    Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Object detectors emerge in deep scene cnns. arXiv preprint arXiv:1412.6856,

  3. [2014]

    Deep inside convolutional networks: Visualising image classification models and saliency maps

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034,

  4. [2015]

    Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller

    doi:10.1007/s11263-015-0816- y. Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296,

  5. [2017]

    Architecture disentanglement for deep neural networks

    Jie Hu, Rongrong Ji, Qixiang Ye, Tong Tong, ShengChuan Zhang, Ke Li, Feiyue Huang, and Ling Shao. Architecture disentanglement for deep neural networks. arXiv preprint arXiv:2003.13268,

  6. [2018]

    Metrics for explainable ai: Challenges and prospects

    Robert R Hoffman, Shane T Mueller, Gary Klein, and Jordan Litman. Metrics for explainable ai: Challenges and prospects. arXiv preprint arXiv:1812.04608,

  7. [2019]

    Neuron shapley: Discovering the responsible neurons

    Amirata Ghorbani and James Zou. Neuron shapley: Discovering the responsible neurons. arXiv preprint arXiv:2002.09815,

  8. [2020]

    Adaptiveface: Adaptive margin and sampling for face recognition

    Hao Liu, Xiangyu Zhu, Zhen Lei, and Stan Z Li. Adaptiveface: Adaptive margin and sampling for face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 11947–11956, 2019a. Haomiao Liu, Ruiping Wang, Shiguang Shan, and Xilin Chen....

  9. [2021]

    Resnest: Split-attention networks

    Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Zhi Zhang, Haibin Lin, Yue Sun, Tong He, Jonas Mueller, R Manmatha, et al. Resnest: Split-attention networks. arXiv preprint arXiv:2004.08955,

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.