Pith. sign in

REVIEW 4 major objections 5 minor 13 references

Towards generating more interpretable counterfactuals via concept vectors: a preliminary study on chest X-rays

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that user-defined clinical concepts can be recovered as stable directions in a chest X-ray autoencoder's latent space, and that traversing along those directions produces visual counterfactual explanations that highlight…

desk verdict Honest preliminary CAV+latent shift study for chest X-rays; the stability claim holds up, but the explanation-quality claim needs a random-direction control before it convinces. read the letter →

arxiv 2506.04058 v1 pith:ZPDZZBRL submitted 2025-06-04 eess.IV cs.AIcs.CV

Bulat Maksudov , Kathleen Curran , Alessandra Mileo This is my paper
classification eess.IVcs.AIcs.CV
keywords InterpretabilityConceptActivationVectorsAutoencodersCounterfactualExplanationsVisualExplainableAIFeatureAttributionChestX-rays
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that clinical concepts, such as an enlarged heart, have recognizable directions in the mathematical space learned by a simple image-reconstruction model, and that moving an X-ray's representation along those directions makes the feature appear or disappear. If true, this matters because explanations would be stated in clinician-defined terms rather than in model gradients, and because the directions transfer across datasets, the same concept vectors could be used on images from other hospitals. The authors' evidence is a set of stability measurements, with an averaged cross-dataset cosine similarity of 0.406 versus near zero for random vectors, and localization measurements, with a cardiomegaly IoU of 0.363 for the averaged NIH vector versus 0.286 for the Latent Shift baseline. They are explicit that the method does not beat the baseline overall and that small pathologies remain challenging due to reconstruction limits. The contribution is therefore a demonstration of a feasible, label-free route to concept-based counterfactual explanations, with limitations clearly stated.

What carries the argument

The central object is the Concept Activation Vector (CAV): the vector orthogonal to the decision boundary of a linear classifier trained in the autoencoder's flattened latent space to separate positive concept samples from negative samples. The paper uses this vector as a movable direction in latent space, adding it to an image's embedding to exaggerate the concept and subtracting it to curtail the concept; decoding both modified embeddings and taking the absolute pixelwise difference produces the 2D attribution map used as the visual explanation. The same traversal protocol and IoU evaluation are inherited from the Latent Shift baseline, with the step size fixed at 10 and concept vectors averaged over multiple batches.

What would settle it

Recompute the cross-dataset cosine similarities after permuting the positive and negative labels within each dataset, or after training on NIH positives versus CheXpert negatives; if the similarity does not drop toward the random-vector baseline near zero, the stability in Table 1 is not evidence of concept capture. A complementary check is to measure reconstruction error inside the annotated bounding boxes for small pathologies, since the paper itself attributes the atelectasis failures to reconstruction fidelity.

Watch

Extended reading notes

Core claim

The paper's central claim is that Concept Activation Vectors computed in the latent space of a reconstruction autoencoder recover user-defined clinical concepts, even though the autoencoder is trained only to reconstruct images and never sees pathology labels. The vectors are obtained by training a linear classifier to separate latent codes of images labeled with the target concept from latent codes of images without it, and the vector orthogonal to the decision boundary is taken as the concept direction. Averaging several such vectors yields a stable direction: the mean NIH and mean CheXpert vectors for the overlapping pathologies have a cosine similarity of 0.406, versus near zero for random vectors. Traversing an image's embedding along this direction and decoding produces exaggerated and curtailed reconstructions, and the pixelwise difference between them is used as a visual explanation. The paper's best result is for cardiomegaly, where the NIH averaged vector reaches an Intersection over Union (IoU) of 0.363 against 0.286 for the Latent Shift baseline; for smaller and more variably localized pathologies such as atelectasis the IoU is much lower, and the method does not outperform the baseline across the board.

Load-bearing premise

The argument rests on the premise that a straight boundary learned to separate images that carry a disease label from images that do not points at the disease itself rather than at incidental differences between the two image groups; if that premise fails, the reported stability and overlap numbers would not show that the explanations are about clinical concepts.

Editorial extensions

If this is right

  • Averaging several concept vectors noticeably improves their stability and cross-dataset alignment, so a mean vector rather than a single-batch vector is the more reliable unit of explanation.
  • Because concept vectors transfer between datasets, the same directions could in principle be used to explain images from another scanner or hospital without retraining the generative model.
  • The generative model itself needs no label training: clinical knowledge enters only through the concept vectors, so one reconstruction model can be steered toward many different concepts.
  • For large, consistently located pathologies such as cardiomegaly the traversal localizes the feature better than the baseline, while small, variably located pathologies need better reconstruction fidelity before the same mechanism can work.
  • Modeling a concept as a subspace or a distribution, rather than a single vector, is the paper's own next step for handling concept variability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors do not run this experiment, but the same concept vectors could be used as probes on other chest X-ray models: measuring whether a classifier's decision boundary aligns with the NIH-mean cardiomegaly vector would test whether the model and the concept share a common geometry.
  • IoU rewards tight pixel overlap with bounding boxes, so it may undervalue explanations that point a clinician to the right region without exact alignment; a small reader study asking clinicians to localize the concept from the attribution map would test the interpretability claim more directly.
  • If stability across datasets is genuine, a practical byproduct is a dataset-comparability check: a new, unlabeled X-ray collection whose concept directions lie close to the NIH-mean vectors is probably clinically similar to the training distribution.
  • The linear-separator premise could be stress-tested by fitting a low-dimensional concept subspace, for example the principal components of the positive-set latent codes, and traversing along several components; this would reveal whether the concept is better described by a cone or a manifold than by a single direction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes using Concept Activation Vectors (CAVs) in the latent space of a pre-trained reconstruction autoencoder to produce counterfactual visual explanations for chest X-rays. Concepts are defined by class labels as proxies (Section 3.1), linear classifiers are trained in the flattened latent space to obtain concept directions, and explanations are generated by adding/subtracting the CAV to image embeddings and taking absolute differences between reconstructions. The authors evaluate concept-vector stability via cosine similarity across NIH and CheXpert datasets, and explanation quality via IoU against bounding boxes, comparing with the Latent Shift baseline. They report that the approach is preliminary, does not outperform the baseline overall, works better for large pathologies such as cardiomegaly, and struggles on smaller pathologies such as atelectasis. The paper includes a link to code and appendices with hyperparameters and example attributions.

Significance. If the central claim were fully supported—that class-label-derived concept directions in a reconstruction autoencoder are stable across datasets and produce concept-specific, clinically meaningful visual explanations—the work would be a useful contribution to interpretable medical imaging and would align with the growing interest in concept-based XAI. The authors are honest about the preliminary nature of the results, and the public code link is a strength, as is the direct comparison with the Latent Shift baseline. However, the current evidence does not yet establish concept-specificity: the explanation-evaluation pipeline lacks any null-model or random-direction control, and the label-free framing in the abstract conflicts with the actual use of class labels to train the CAV classifiers. Because the interpretation of Tables 2 and 3 as evidence of clinically meaningful explanations depends on this control, the central claim is not yet fully supported.

major comments (4)
  1. [Section 3.2 / Section 5.2 / Tables 2-3] The explanation-quality evaluation lacks a random-direction control. The 2D attribution map is defined as the absolute difference between a traversed reconstruction and the original image (Section 3.2), so any latent direction that produces global intensity or contrast changes will yield high overlap with a large, centrally located bounding box such as the cardiomegaly ground truth, irrespective of whether the direction encodes the clinical concept. The reported IoU of 0.363 for the averaged NIH CAV versus 0.286 for Latent Shift therefore does not establish concept-specific explanation quality unless the same traversal protocol with random latent directions is shown to produce substantially lower IoU. Please add this control, with the same step size, averaging procedure, and attribution-mapping pipeline, and report the distribution over multiple random directions.
  2. [Abstract / Section 3.1] The abstract and conclusion state that concepts are linked to image-level features 'without requiring explicit training on class labels' and 'without explicit label training,' yet Section 3.1 explicitly says 'we use class labels as proxies for the concepts' and the CAV is obtained by training a linear classifier on positive samples labeled with the target class and negative samples without it. This is a direct internal inconsistency and is load-bearing for the framing of the contribution. Please either change the claim to acknowledge the label-based supervision used to define concepts, or provide a genuinely label-free construction of the concept sets.
  3. [Section 4.2 / Appendix C, Table 5] The number of concept vectors used for averaging is inconsistent. Section 4.2 states 'we generate explanations using 4 sets of concept vectors for each dataset and then average those vectors,' while Appendix C, Table 5 reports 'Number of concept vectors for averaging: 10.' The reported mean values and standard deviations in Tables 1-3 depend on this number, and the discrepancy prevents exact reproducibility. Please unify the text and tables and state the exact number of concept-vector replicates used for each reported statistic.
  4. [Section 5.1 / Table 1] The interpretation of the cross-dataset cosine similarity is not adequately supported. The paper states that 'in high-dimensional latent space even the smallest observed value (0.1011 for NIH to CheXpert similarity) is very high,' but the absolute value is low, and no statistical test or comparison against a baseline direction (e.g., random vectors with the same norm or a label-agnostic PCA direction) is provided beyond the raw random-vector similarity shown in Table 1. A cosine similarity of 0.1011 that is several standard deviations from the random baseline still represents weak absolute alignment, and it does not by itself establish that the direction is clinically specific rather than merely dataset-correlated. Please add a significance test or a meaningful reference point for 'high' similarity, and avoid over-interpreting low absolute values.
minor comments (5)
  1. [Introduction] There is a typo, 'the the model predictions,' and the phrase 'CA Vs' uses inconsistent spacing throughout; please standardize to 'CAVs.'
  2. [Section 4.1] The paper says 'we compute the similarity of concept vectors with random vectors' and later refers to 'inter and intra dataset' experiments, but the terminology is applied interchangeably; please clarify which of the three experiments corresponds to Table 1 and which to the appendix.
  3. [Section 4.2] The sentence 'In out case, Intersection over Union provides a measure' contains a typo ('out' should be 'our'), and the metric definition would benefit from a precise formula or reference to the exact equation used.
  4. [Abstract / Conclusion] The sentence 'Despite the approach does not beat the baseline in its current form' is grammatically incorrect; consider 'Although the approach does not beat the baseline in its current form.'
  5. [Tables 2-3] The tables report standard deviations for individual CAV results but not for the averaged-vector results; adding confidence intervals or a small-number-of-vectors caveat would help readers judge the stability of the mean IoU values.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: concept directions are fitted to label proxies but validated against external bounding boxes and random-vector baselines.

full rationale

The paper's derivation chain is empirical rather than formal. CAVs are obtained by training a linear classifier in the autoencoder's latent space on label-defined positive and negative sets (Section 3.1: 'we use class labels as proxies for the concepts'). This is a supervised fit of the concept direction, but the paper's central claims are not obtained from that fit by construction. Stability is measured by cosine similarity between CAVs estimated on different datasets (NIH vs CheXpert) and against random vectors (Table 1); inter-dataset alignment is not logically forced by the label proxy because the datasets have different label protocols and image distributions. Explanation quality is evaluated externally against anatomical bounding boxes (Section 4.1, Tables 2-3), which were not used to fit the CAVs, so the IoU numbers are independent evidence rather than a renamed fit. The paper also concedes the baseline is not beaten and that IoU is only a proxy for human evaluation, and it identifies reconstruction fidelity as a limitation. The only concern, namely that IoU lacks a random-direction traversal control, is a validity or interpretational limitation, not a circularity, because the same protocol could in principle yield low IoU for random directions. The abstract's phrase 'without explicit label training' is imprecise since CAV generation is supervised by labels, but Section 3.1 discloses the label proxy and the autoencoder itself is label-free; this is a wording issue, not a circular derivation. No load-bearing self-citations, uniqueness theorems, or ansatz-smuggling citations appear; all cited external methods (TCAV, Latent Shift, torchxrayvision) are used as benchmarks or tooling.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper adds no new entities and fits no curve to derive constants. Its contributions are empirical, resting on the TCAV assumption of linear concept directions, on label validity in NIH and CheXpert, on IoU as an explanation-quality proxy, and on the pre-trained autoencoder's reconstruction fidelity. The traversal step size and number of averaged vectors are hand-chosen hyperparameters that affect the reported numbers; the paper does not ablate them.

free parameters (3)
  • Latent traversal step size = 10
    Manual choice in Section 4.2; no ablation is reported, so results depend on this scale.
  • Number of concept vectors averaged = 10 (but Section 4.2 says 4)
    Appendix C Table 5 lists 10, while Section 4.2 says 4 sets per dataset; the reported means depend on this count.
  • CAV batch composition = 250 positive, 250 negative
    Chosen by hand (Appendix C Table 5); affects the estimated concept direction and all downstream similarities and IoUs.
assumptions (4)
  • standard math A linear hyperplane in the flattened latent space separates concept-positive from concept-negative embeddings, and its orthogonal vector captures the concept direction.
    Core CAV premise from Kim et al. 2018, adopted in Section 3.1.
  • domain assumption Class labels from NIH and CheXpert are reliable proxies for the clinical concepts being explained.
    Section 3.1 explicitly uses class labels as concept proxies; label noise is not analyzed.
  • domain assumption IoU between attribution maps and bounding boxes is a meaningful proxy for explanation quality.
    Section 5.2 acknowledges IoU is a proxy and lacks human evaluation; the numbers are interpreted as clinical relevance.
  • domain assumption The pre-trained torchxrayvision autoencoder reconstructs small pathologies with enough fidelity for traversal to change the relevant pixels.
    Section 5.2 attributes failures for Atelectasis and Nodule to reconstruction fidelity and localization.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Towards generating more interpretable counterfactuals via concept vectors: a preliminary study on chest X-rays." pith.science (2026). https://pith.science/paper/ZPDZZBRL

@misc{pith2026250604058,
  author       = {Pith},
  title        = {Pith review of: Towards generating more interpretable counterfactuals via concept vectors: a preliminary study on chest X-rays},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZPDZZBRL}},
  note         = {Machine review of arXiv:2506.04058}
}
read the original abstract

An essential step in deploying medical imaging models is ensuring alignment with clinical knowledge and interpretability. We focus on mapping clinical concepts into the latent space of generative models to identify Concept Activation Vectors (CAVs). Using a simple reconstruction autoencoder, we link user-defined concepts to image-level features without explicit label training. The extracted concepts are stable across datasets, enabling visual explanations that highlight clinically relevant features. By traversing latent space along concept directions, we produce counterfactuals that exaggerate or reduce specific clinical features. Preliminary results on chest X-rays show promise for large pathologies like cardiomegaly, while smaller pathologies remain challenging due to reconstruction limits. Although not outperforming baselines, this approach offers a path toward interpretable, concept-based explanations aligned with clinical knowledge.

Figures

Figures reproduced from arXiv: 2506.04058 by the authors.

Figure 1
Figure 1. Overview of our method: (a) Construction of Concept Activation Vector in the latent space of autoencoder using linear classifier (b) Generation of explanations by traversing the latent space using CAV. Visual explanation is constructed by pixelwise subtracting exaggerated and curtailed samples in the latent space that best captures the target concept. This orthogonal vector is referred to as the Concept Activation V… view at source ↗
Figure 2
Figure 2. Visual representation of feature manipulation for the ”Cardiomegaly” concept. By traversing the latent space of the autoencoder, we add or subtract the concept vector from the image embedding to generate reconstructions that exaggerate or curtail cardiomegaly-related features. From left to right: (a) Feature curtail, reducing the heart area; (b) Original chest X-ray; (c) Feature exaggeration, enhancing cardiomegaly-… view at source ↗
Figure 3
Figure 3. Successful visual explanation of the ”Cardiomegaly” pathology using concept vectors. From left to right: (a) Concept mask highlighting the region of interest, (b) Original chest X-ray image, (c) Reconstructed image from the autoencoder, (d) Latent shift 2D attribution map generated by the baseline method, and (e) Attribution map generated using the NIH concept vector [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Example of a failed explanation for ”Atelectasis”. From left to right: (a) Concept mask highlighting the region of interest; (b) Original chest X-ray; (c) Reconstructed image; (d) Baseline latent shift attribution map; (e) Attribution map. The failure is likely due to …

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

13 extracted references · 7 canonical work pages

  1. [1]

    Sanity checks for saliency maps

    Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in neural information processing systems, 31, 2018

    2018

  2. [2]

    Chexplaining in style: Counterfactual explanations for chest x-rays using stylegan

    Matan Atad, Vitalii Dmytrenko, Yitong Li, Xinyue Zhang, Matthias Keicher, Jan Kirschke, Bene Wiestler, Ashkan Khakzar, and Nassir Navab. Chexplaining in style: Counterfactual explanations for chest x-rays using stylegan. arXiv preprint arXiv:2207.07553, 2022

    arXiv 2022

  3. [3]

    Diffusion visual counterfactual explanations

    Maximilian Augustin, Valentyn Boreiko, Francesco Croce, and Matthias Hein. Diffusion visual counterfactual explanations. In NeurIPS, 2022

    work page 2022

  4. [4]

    Gifsplanation via latent shift: a simple autoencoder approach to counterfactual generation for chest x-rays

    Joseph Paul Cohen, Rupert Brooks, Sovann En, Evan Zucker, Anuj Pareek, Matthew P Lungren, and Akshay Chaudhari. Gifsplanation via latent shift: a simple autoencoder approach to counterfactual generation for chest x-rays. In Medical Imaging with Deep Learning, pages 74--104. PMLR, 2021

    work page 2021

  5. [5]

    Viviano, Paul Bertin, Paul Morrison, Parsa Torabian, Matteo Guarrera, Matthew P Lungren, Akshay Chaudhari, Rupert Brooks, Mohammad Hashir, and Hadrien Bertrand

    Joseph Paul Cohen, Joseph D. Viviano, Paul Bertin, Paul Morrison, Parsa Torabian, Matteo Guarrera, Matthew P Lungren, Akshay Chaudhari, Rupert Brooks, Mohammad Hashir, and Hadrien Bertrand. TorchXRayVision: A library of chest X-ray datasets and models . In Medical Imaging with Deep Learning, 2022. URL https://github.com/mlmed/torchxrayvision

    work page 2022

  6. [6]

    Towards Benchmarking Explainable Artificial Intelligence Methods

    Lars Holmberg. Towards benchmarking explainable artificial intelligence methods. ArXiv, abs/2208.12120, 2022. URL https://api.semanticscholar.org/CorpusID:251800062

    work page Pith review arXiv 2022

  7. [7]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea-Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 590--597, 2019

    2019

  8. [8]

    Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)

    Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pages 2668--2677. PMLR, 2018

    2018

Show all 13 references
  1. [9]

    In-context vectors: making in context learning more effective and controllable through latent space steering

    Sheng Liu, Haotian Ye, Lei Xing, and James Zou. In-context vectors: making in context learning more effective and controllable through latent space steering. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024

    2024

  2. [10]

    An overview of the empirical evaluation of explainable ai (xai): A comprehensive guideline for user-centered evaluation in xai

    Sidra Naveed, Gunnar Stevens, and Dean Robin-Kern. An overview of the empirical evaluation of explainable ai (xai): A comprehensive guideline for user-centered evaluation in xai. Applied Sciences, 14 0 (23), 2024. ISSN 2076-3417. doi:10.3390/app142311288. URL https://www.mdpi....

    2024 doi

  3. [11]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017

    2017

  4. [12]

    Saliency is a possible red herring when diagnosing poor generalization

    Joseph D Viviano, Becks Simpson, Francis Dutil, Yoshua Bengio, and Joseph Paul Cohen. Saliency is a possible red herring when diagnosing poor generalization. arXiv preprint arXiv:1910.00199, 2019

    1910 arXiv

  5. [13]

    Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases

    Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M Summers. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE conference on co...

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.