Pith. sign in

REVIEW 4 major objections 5 minor 17 references

Faithful Counterfactual Visual Explanations (FCVE)

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that counterfactual visual explanations can be generated by changing a CNN's internal filter activations and decoding them, so the resulting image shows what the model's own learned features would need to change to flip…

desk verdict New decoder-based filter visualization with nice qualitative results, but faithfulness is untested and the reported metrics contradict the paper's own claims. read the letter →

arxiv 2501.06841 v1 pith:D44BYOZO submitted 2025-01-12 cs.CV

Bismillah Khan , Syed Ali Tariq , Tehseen Zia , Muhammad Ahsan , David Windridge This is my paper · ORCID
classification cs.CV
keywords counterfactualexplanationsvisualexplainableAIfaithfulnessplausibilityCNNfiltersfeaturevisualizationimageclassifiers
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to make counterfactual explanations of image classifiers both plausible and faithful by moving the explanation out of pixel space. Instead of searching for pixels to perturb, the proposed FCVE model locates the classifier's own high-level filters: which last-layer activations must stay on to keep a prediction, and which must be boosted to flip it toward a target class. A decoder is first trained to reconstruct the input from the unmodified activation vector, and is then applied to the vector after those filter changes, producing an image that shows the model's internal features changing with its decision. If this works as claimed, a user could ask what would make this image classified differently and receive a visual answer generated from inside the network, not from a pixel-level perturbation search.

What carries the argument

The load-bearing objects are the two counterfactual-filter masks and the decoder that visualizes them. $F_{MC}$, called the minimum-correct mask, is an approximately binary vector marking which of the $n$ filters in the top convolutional layer must remain active for the classifier to keep its original prediction. $F_{MI}$, the minimum-incorrect mask, is a vector of positive magnitudes that specifies how much each filter activation should be boosted or altered to move the prediction toward a target class. The decoder, trained once with the classifier frozen, learns the inverse map from the last convolutional feature vector back to the input image; applying the masks to that feature vector and decoding produces the counterfactual image. The masks supply what to change, and the decoder supplies how that change looks.

What would settle it

Take the counterfactual image $x'$ produced by FCVE for a source image $x$ and a target class, run the original classifier on $x'$, and record its predicted label. If this label is not the target class, or if the prediction fails to stay in the source class when only $F_{MC}$ is applied, the explanation is not faithful to the model; this classifier-on-counterfactual accuracy is not reported in the paper.

Watch

Extended reading notes

Core claim

On its own terms, the central claim is that a CNN decision can be faithfully visualized without touching pixel values. The method takes a previously trained counterfactual-filter model that outputs two masks over the $n$ filters of the last convolutional layer: a minimum-correct mask $F_{MC} \in [0,1]^n$, which keeps the original class active, and a minimum-incorrect mask $F_{MI} \in \mathbb{R}_+^n$, which gives the magnitudes by which filter activations must be altered to move the prediction to a chosen target class. A decoder $D$ is trained with mean-absolute-error loss to invert the intact feature vector $C_{\mathrm{conv}}(x)$, and the counterfactual is defined as $x' = D(C(x, F_{MC}, F_{MI}))$, where $C(x, F_{MC}, F_{MI})$ denotes the classifier's feature vector after the masks are applied. The paper argues that because $x'$ is produced by changing internal activations rather than by editing pixels, the visual explanation reflects the model's own decision process, and because the filter changes are minimal, the counterfactual stays close to the original image.

Load-bearing premise

The entire visual explanation rests on the decoder being able to render feature vectors it never saw during training: it is trained only on intact activations, but is then used on activations in which filters have been zeroed or boosted, and the paper does not establish that these modified vectors stay inside the decoder's reliable reconstruction region.

Editorial extensions

If this is right

  • Once the decoder is trained, counterfactual images are produced in a single forward pass, with no per-example optimization loop over pixels.
  • The explanation is anchored in the network's own filter vocabulary, so it can show which learned concepts, not which pixels, the model treats as decisive.
  • Because only the minimal correct and incorrect filter masks are edited, the method aims to satisfy the counterfactual proximity property by construction.
  • The same filter-plus-decoder recipe is tied to any convolutional classifier with an accessible last-layer feature vector, independent of the image dataset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implicit test the paper leaves open is decision-level validity: feeding each generated $x'$ back into the classifier and measuring how often the label actually becomes the target class would directly separate faithful counterfactuals from visually plausible artifacts.
  • Interpolating the $F_{MI}$ boost from zero to its full value would trace a visual trajectory from the source class to the target class, offering a concept-level analogue of latent-space interpolation that the paper does not explore.
  • The decoder's reliability could be audited by constructing modified activations for images whose class is known, for example zeroing the predicted filters of a digit and decoding, and checking that the reconstructions degrade only in the class-relevant regions.
  • The very low FID scores reported relative to pixel-based baselines may partly reflect that the decoder outputs are reconstructions from the training distribution; a comparison against classification accuracy on the generated images would disentangle reconstruction quality from counterfactual validity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes FCVE, a post-hoc visual explanation method that generates counterfactual images by modifying filter activations in a pretrained CNN's last convolutional layer rather than altering input pixels. The method combines a previously developed counterfactual/contrastive filter identification model (Tariq et al. 2022) with a decoder trained to reconstruct input images from intact feature vectors; during explanation, the decoder renders images from filter-modified activations as visual counterfactual explanations. The authors claim these explanations are both plausible and faithful to the classifier's internal decision process, and report qualitative comparisons on MNIST and Fashion-MNIST along with quantitative proximity and FID scores against several baselines.

Significance. If the central claims were established, the paper would make a useful contribution by offering a visualization of internal filter-level changes rather than pixel perturbations, which could make counterfactual explanations more interpretable. The qualitative examples on MNIST and FMNIST are suggestive, and the idea of decoding modified feature activations is a reasonable direction. However, the paper does not actually evaluate faithfulness, its quantitative proximity results contradict the 'minimum changes' claim, and the FID computation appears incorrect. As it stands, the evidence does not support the paper's main assertions, so the contribution remains a promising but unvalidated idea rather than a demonstrated method.

major comments (4)
  1. [Section 4.3.1, Table 1] The proximity scores reported for FCVE (0.098 on MNIST, 0.198 on FMNIST) are the highest among all methods in Table 1; for comparison, CEM achieves 0.016 and 0.013 on the same datasets. The paper defines proximity as an L1 distance where lower values indicate counterfactuals closer to the original input. These numbers directly contradict the central claim, stated in the Abstract and Section 5, that the method makes 'minimum changes necessary' to the input. The authors need to reconcile this discrepancy or substantively revise the minimality claim, since the 'minimum change' property is a core selling point of the paper.
  2. [Section 4.3.2, Eq. (10)] Equation (10) is not the Fréchet Inception Distance. The standard FID is ||mu - mu'||^2 + Tr(Σ + Σ' - 2(ΣΣ')^{1/2}) where mu and Σ are the mean and covariance of feature distributions; the paper instead writes Tr(x + x' - 2*sqrt(x.x')) without defining x and x' as covariance matrices, and the surrounding text describes the metric as 'feature-wise subtraction of the query images and their respective counterfactuals,' which is a per-pair distance, not a distributional metric. Furthermore, the reported FID values (0.50 for MNIST, 2.02 for FMNIST) are strikingly low and given without error bars or any description of how many samples were used. The quantitative claim of outperforming baselines is therefore not supported.
  3. [Section 4, first paragraph] Faithfulness, the property that appears in the title and the abstract, is not directly evaluated for the proposed visual explanations. The text states that the authors 'refer the reader to Tariq et al. [2022]' for evidence that the identified filters are faithful. That prior work validates the filters themselves, not the decoded images produced by the decoder used here. The present paper's claim is that the generated counterfactual images 'reflect the internal decision-making process of the model,' but no experiment checks whether a generated image x' actually flips the classifier's prediction or whether the decoded visual changes correspond to the internal filter modifications. Without such a direct evaluation, the faithfulness claim is inherited from a prior work rather than demonstrated for the proposed method.
  4. [Section 3.2, Eq. (3) and Eq. (8)] The decoder D is trained on intact feature vectors C_conv(x_i) but is applied at test time to modified feature vectors C(x, F_MC_i, F_MI_i) produced by the CFE filter changes. The paper provides no evidence that D generalizes to these out-of-distribution activations; if it does not, the generated counterfactuals could be artifacts of the decoder rather than visualizations of the classifier's reasoning. The paper should include an inversion-fidelity check on modified activations (for example, measuring reconstruction error on feature vectors perturbed to the same degree as those used in Eq. (8)) or verify that the decoded image is classified into the intended target class. Since the entire visual explanation rests on this step, this issue is load-bearing.
minor comments (5)
  1. [Section 3.2] The text says 'The decoder is asynchronous as the encoder and decoder have variable depths'; the intended word is likely 'asymmetric.'
  2. [Section 4.3.1] The sentence 'Proximity property explanations the counterfactuals, meaning Faithful to the original instance' is grammatically garbled and should be rewritten.
  3. [Section 4.3.1, Eq. (9)] The notation dsict in Eq. (9) is undefined; presumably this is an L1 distance and the notation should be clarified.
  4. [General] The paper discusses several diffusion-based methods (e.g., Augustin et al. 2022, Jeanneret et al. 2022) in the related work but does not include them in the experimental comparison; adding such recent baselines would strengthen the evaluation.
  5. [References] The reference list has inconsistent formatting; for example, some entries include DOIs or volume/page numbers while others are incomplete, and the Tariq et al. 2022 entry appears to be self-cited with a DOI but no page range.

Circularity Check

1 steps flagged · score 4.0 of 10

Faithfulness claim rests on a load-bearing self-citation; core equations are not otherwise self-referential.

  1. self citation load bearing [Section 4, first paragraph (faithfulness evaluation)]
    "To evaluate faithfulness of the explanations provided by the proposed method, we refer the reader to Tariq et al. [2022] that used the class recall metric to demonstrate that the identified counterfactual and contrastive filters are faithful to their respective classes."

    The paper's central advertised property is faithfulness to the model's internal decision-making, asserted in the abstract and Section 3. The only support offered for this property is a reference to the authors' own prior paper, which evaluates filter-level class recall, not whether the decoder-rendered images faithfully invert the modified feature vectors. The present paper therefore inherits its central claim from an overlapping-author self-citation rather than deriving or measuring it for its own visual explanations. This is load-bearing because no other faithfulness evaluation is provided, and the cited result does not cover the actual explanation object (the generated images).

full rationale

The construction itself is not circular: Eq. (3) trains the decoder on intact features, and Eq. (8) applies it to CFE-modified features; the output is a forward pass, not the minimizer of Eq. (3), so no equation equals its own input by construction. However, the central advertised property—faithfulness to the model's internal reasoning—is not evaluated on the generated images. Section 4 explicitly outsources this evaluation to Tariq et al. [2022], an overlapping-author prior paper, and even that evaluation concerns filter-level class recall rather than decoder-rendered visuals. Because the abstract and Section 3 assert faithfulness as the main result, this self-citation is load-bearing. The decoder out-of-distribution concern (training on intact activations, application to modified activations) is a real validity gap but is not a circularity; it raises correctness risk, not self-referentiality.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new theoretical entities. Its central method rests on inherited CFE filter identification and a decoder that must generalize to modified activations; the threshold t=0.5 is a hand-set hyperparameter with no sensitivity analysis.

free parameters (1)
  • Binarization threshold t = 0.5
    Used in Eq. 5 to convert MC filter outputs to a binary mask; no sensitivity analysis is provided.
assumptions (4)
  • domain assumption Last convolution layer filters represent high-level, class-distinctive concepts.
    Invoked in Section 3 to justify modifying these filters; based on cited work (Bau et al., Zhou et al.).
  • domain assumption The CFE model from Tariq et al. 2022 correctly identifies MC and MI filters for the classifier.
    The paper relies on this for all counterfactual filter selection and does not independently verify it.
  • domain assumption The decoder trained on unmodified activations can map modified activations to plausible images.
    Section 3.2 and Eq. 8; this is not tested and is a key generalization assumption.
  • domain assumption FID computed on MNIST and FMNIST with Inception features is a valid plausibility measure.
    Used in Section 4.3.2; standard FID may behave differently on low-resolution images.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Faithful Counterfactual Visual Explanations (FCVE)." pith.science (2026). https://pith.science/paper/D44BYOZO

@misc{pith2026250106841,
  author       = {Pith},
  title        = {Pith review of: Faithful Counterfactual Visual Explanations (FCVE)},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/D44BYOZO}},
  note         = {Machine review of arXiv:2501.06841}
}
read the original abstract

Deep learning models in computer vision have made remarkable progress, but their lack of transparency and interpretability remains a challenge. The development of explainable AI can enhance the understanding and performance of these models. However, existing techniques often struggle to provide convincing explanations that non-experts easily understand, and they cannot accurately identify models' intrinsic decision-making processes. To address these challenges, we propose to develop a counterfactual explanation (CE) model that balances plausibility and faithfulness. This model generates easy-to-understand visual explanations by making minimum changes necessary in images without altering the pixel data. Instead, the proposed method identifies internal concepts and filters learned by models and leverages them to produce plausible counterfactual explanations. The provided explanations reflect the internal decision-making process of the model, thus ensuring faithfulness to the model.

Figures

Figures reproduced from arXiv: 2501.06841 by the authors.

Figure 1
Figure 1. Block diagram of the proposed visual counterfactual explanation model. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 1
Figure 1. The reconstructed image by the decoder provides visual explanations of the features-modification made by [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Visual comparison of counterfactual explanation methods. [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Plausible counterfactuals generated for digit seven as a source class and digit nine as target class. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png]
Figure 4
Figure 4. Figure 4: Counterfactuals generated for random source and target classes of MNIST dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Plausible counterfactuals generated for the FMNIST dataset. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Counterfactual generation from visually identical classes of “T-shirt" (source class, row 1) and “Pullover" [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Counterfactual generation from visually non-identical classes of “Trouser" (source class, row 1) and “Shirt" [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 8 canonical work pages

  1. [2022]

    Latent-CF: A Simple Baseline for Reverse Counterfactual Explanations

    Rachana Balasubramanian, Samuel Sharpe, Brian Barr, Jason Wittenbach, and C Bayan Bruss. Latent-cf: a simple baseline for reverse counterfactual explanations. arXiv preprint arXiv:2012.09301,

    work page Pith review arXiv 2012

  2. [1]

    Towards an efficient real-time approach to loan credit approval using deep learning

    Youness Abakarim, Mohamed Lahby, and Abdelbaki Attioui. Towards an efficient real-time approach to loan credit approval using deep learning. In 2018 9th International Symposium on Signal, Image, Video and Communications (ISIVC), pages 306–313. IEEE,

    work page 2018

  3. [3]

    Diffusion Visual Counterfactual Explanations

    Maximilian Augustin, Valentyn Boreiko, Francesco Croce, and Matthias Hein. Diffusion visual counterfactual explanations. arXiv preprint arXiv:2210.11841,

    work page Pith review arXiv

  4. [5]

    Explaining image classifiers by counterfac- tual generation

    Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Explaining image classifiers by counterfac- tual generation. arXiv preprint arXiv:1807.08024,

    arXiv

  5. [10]

    Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models

    Wojciech Samek, Thomas Wiegand, and Klaus-Robert Müller. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models. arXiv preprint arXiv:1708.08296,

    arXiv

  6. [11]

    A novel cognitive computing technique using convolutional networks for automating the criminal investigation process in policing

    Francesco Schiliro, Amin Beheshti, and Nour Moustafa. A novel cognitive computing technique using convolutional networks for automating the criminal investigation process in policing. In Intelligent Systems and Applications: Proceedings of the 2020 Intelligent Systems Conference (IntelliSys) Volume 1, pages 528–539. Springer,

    work page 2020

  7. [12]

    doi:https://doi.org/10.1016/j.knosys.2022.109901

    ISSN 0950-7051. doi:https://doi.org/10.1016/j.knosys.2022.109901. Simen Thys, Wiebe Van Ranst, and Toon Goedemé. Fooling automated surveillance cameras: adversarial patches to attack person detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 0–0,

    arXiv 2022

  8. [13]

    Interpretable counterfactual explanations guided by prototypes

    Arnaud Van Looveren and Janis Klaise. Interpretable counterfactual explanations guided by prototypes. In Machine Learning and Knowledge Discovery in Databases. Research Track: European Conference, ECML PKDD 2021, Bilbao, Spain, September 13–17, 2021, Proceedings, Part II 21, pages 650–665. Springer,

    work page 2021

Show all 17 references
  1. [14]

    Making heads or tails: Towards semantically consistent visual counterfactuals

    Simon Vandenhende, Dhruv Mahajan, Filip Radenovic, and Deepti Ghadiyaram. Making heads or tails: Towards semantically consistent visual counterfactuals. In Computer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XII, pages ...

    2022

  2. [15]

    Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms

    Han Xiao, Kashif Rasul, and Roland V ollgraf. Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747,

    arXiv

  3. [16]

    Explainability of vision-based autonomous driving systems: Review and challenges

    14 Faithful Counterfactual Visual Explanations (FCVE) A PREPRINT Éloi Zablocki, Hédi Ben-Younes, Patrick Pérez, and Matthieu Cord. Explainability of vision-based autonomous driving systems: Review and challenges. arXiv preprint arXiv:2101.05307,

    arXiv

  4. [17]

    Examining cnn representations with respect to dataset bias

    Quanshi Zhang, Wenguan Wang, and Song-Chun Zhu. Examining cnn representations with respect to dataset bias. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018a. Xihai Zhang, Yue Qiao, Fanfeng Meng, Chengguo Fan, and Mingming Zhang. Identification...

    arXiv

  5. [2017]

    Neuron shapley: Discovering the responsible neurons

    Amirata Ghorbani and James Zou. Neuron shapley: Discovering the responsible neurons. arXiv preprint arXiv:2002.09815,

    2002 arXiv

  6. [2018]

    What do we need to build explainable ai systems for the medical domain? arXiv preprint arXiv:1712.09923,

    Andreas Holzinger, Chris Biemann, Constantinos S Pattichis, and Douglas B Kell. What do we need to build explainable ai systems for the medical domain? arXiv preprint arXiv:1712.09923,

    arXiv

  7. [2019]

    Deep cnn and deep gan in computational visual perception-driven image analysis

    R Nandhini Abirami, PM Durai Raj Vincent, Kathiravan Srinivasan, Usman Tariq, and Chuan-Yu Chang. Deep cnn and deep gan in computational visual perception-driven image analysis. Complexity, 2021:1–30,

    2021

  8. [2020]

    Explaining image classifiers using contrastive counterfactuals in generative latent spaces

    Kamran Alipour, Aditya Lahiri, Ehsan Adeli, Babak Salimi, and Michael Pazzani. Explaining image classifiers using contrastive counterfactuals in generative latent spaces. arXiv preprint arXiv:2206.05257,

    arXiv

  9. [2021]

    Generating contrastive explanations with monotonic attribute functions

    Ronny Luss, Pin-Yu Chen, Amit Dhurandhar, Prasanna Sattigeri, Yunfeng Zhang, Karthikeyan Shanmugam, and Chun- Chen Tu. Generating contrastive explanations with monotonic attribute functions. arXiv preprint arXiv:1905.12698,

    1905 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.