Pith. sign in

REVIEW 4 major objections 3 minor 3 cited by

LCBM claims that per-concept prototype learning, supervised by CLIP region similarity, makes label-free concept bottleneck models localize concepts correctly without losing classification accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A label-free concept bottleneck model using per-concept prototypes aligned by CLIP to localize concept predictions to the correct image regions.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The paper plausibly improves concept localization in label-free CBMs, but the visible text omits the method and all quantitative results, so the central claim is unverifiable and the CLIP-based training signal raises a real circularity concern. the 4 major comments →

arxiv 2508.14562 v1 pith:PKFEJU7Y submitted 2025-08-20 cs.CV

Locality-aware Concept Bottleneck Model

Sujin Jeon , Hyundo Lee , Eungseo Kim , Sanghack Lee , Byoung-Tak Zhang , Inwoo Hwang This is my paper
classification cs.CV
keywords concept bottleneck modelslabel-free concept bottleneckprototype learningconcept localizationinterpretable machine learningCLIPGradCAMfine-grained classification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Label-free concept bottleneck models predict human-readable concepts from images but often explain a concept like "barred black feather" by attending to the wrong region. LCBM is an attempt to fix that by giving each concept a learnable prototype that encodes the concept's prototypical local appearance, with CLIP similarity scores keeping the prototype semantically tied to the concept's text. Concept presence is then read from similarity between image regions and prototypes, so each prediction carries a spatial answer to "where is this concept?". The paper reports improved agreement between concept-level GradCAM maps and human-annotated regions on CUB, ImageNet-animal, and Stanford Cars, and shows that erasing those regions lowers concept presence scores while classification accuracy stays comparable.

Core claim

The central claim is that label-free concept bottleneck models misbehave spatially—they predict a concept like "barred black feather" while their activation lands on the wrong part of the image—and that this is fixable without dense concept annotations. The proposed mechanism is to learn one prototype per concept so that concept prediction is mediated by local-region-to-prototype similarity. Prototypes are trained with an auxiliary classification objective and with CLIP similarity between prototype features and concept text, which teaches each prototype which local appearance it stands for. On CUB-200-2011, ImageNet-animal, and Stanford Cars, LCBM's concept-level GradCAM maps fall inside hum

What carries the argument

Each concept is assigned one prototype vector, a learnable embedding meant to represent the prototypical local appearance of that concept. Two additional losses drive the learning: one auxiliary classification loss makes the prototype encode features shared by visually similar local regions, and one alignment loss uses CLIP similarity scores to keep the prototype tied to its concept's text. At inference, similarity between an image's local features and a prototype produces a spatial relevance map, and the concept presence score is read from the regions that match the prototype. The prototype is the load-bearing object: it turns a global concept score into a "this looks like that" region, and

Load-bearing premise

The entire localization improvement rests on trusting CLIP's similarity scores on small image regions as ground truth for whether a concept is present there; if those scores are unreliable or biased for fine-grained concepts, the prototypes learn to attend to the wrong places and the reported localization is an artifact.

What would settle it

Take a synthetic dataset with known part locations and a concept whose CLIP text embedding is confounded (e.g., "white belly" on an animal with a white background). Train LCBM on images where the confound is decorrelated, then evaluate whether GradCAM peaks at the belly or the background; if it tracks CLIP's highest-scoring region rather than the annotated belly, the prototype mechanism is just relaying CLIP's patch-level errors. A stronger version uses human part masks during training as an oracle and asks whether LCBM approaches oracle localization; if it does not, the gap is the cost of tru

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Concept predictions in label-free CBMs acquire a usable spatial reference: the region of highest prototype similarity is where the concept is claimed to be, so a user can verify each concept against the image instead of trusting a score.
  • Because classification accuracy remains comparable, LCBM can replace existing label-free CBM backbones as a drop-in module for interpretable fine-grained classification.
  • The removal experiment gives evidence that the localized region is actually used: deleting the region decreases the concept presence score, tying explanations to the model's computation rather than only to post-hoc saliency.
  • No dense concept annotations are needed to obtain this localization, which matters for scaling interpretable models to new datasets and domains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next step, not explored in the paper, is treating the learned prototype-region similarity maps as soft part segmentations; if they are as well-localized as the GradCAM results suggest, they could seed part discovery on datasets that lack part annotations.
  • Because CLIP is used only as a teacher for the alignment loss, the same architecture could be retrained with a different vision-language teacher; comparing localization quality across teachers would quantify how much of the gain comes from the prototype mechanism versus from CLIP's own spatial knowledge.
  • The paper's localization metric (GradCAM overlap with annotated regions) can be gamed by models that produce peaked heatmaps for the wrong reason; the removal experiment is the stronger evidence, and a direct comparison of both metrics across baselines would separate true causal use from saliency artifacts.
  • For concepts where CLIP is known to be systematically biased (e.g., color adjectives that actually describe the background), LCBM would inherit that bias; testing on a bias-controlled set would clarify how far the locality guarantee extends.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper proposes LCBM (Locality-aware Concept Bottleneck Model), a label-free CBM in which each concept is assigned a prototype that is trained to encode local image features associated with that concept. Prototype learning is guided by CLIP similarity scores between local patches and concept text, and the prototypes are then used to predict concept presence from local regions. The central claim is that LCBM improves the spatial localization of concept predictions—measured via GradCAM alignment with human-annotated points/boxes and via a concept-removal protocol—while maintaining classification performance comparable to prior label-free CBMs. The supplied manuscript text includes the abstract, introduction, related work, a qualitative results section, the limitation section, and the conclusion, but Sections 3 (method) and the quantitative portions of Section 4 (experiments) are not visible in the provided version. The limitation section explicitly concedes that LCBM inherits biases from CLIP and that CLIP is not optimized for identifying detailed concepts within an image.

Significance. If the central claim is substantiated, the work addresses an important and practical gap: label-free CBMs often produce concept explanations that attend to visually irrelevant regions. The prototype-based locality mechanism is a sensible and potentially effective approach, and the two proposed evaluation protocols (GradCAM vs. human annotations, and concept removal) are appropriate ways to assess localization fidelity. The paper also provides qualitative evidence and an appendix with additional examples. However, the significance is conditional: the current supplied text lacks the quantitative experimental tables needed to verify the 'significantly improves' claim, and the method's reliance on CLIP patch-text similarity as the training target creates a risk of circularity when evaluation also uses CLIP-derived concepts. These issues must be resolved before the contribution can be assessed reliably.

major comments (4)
  1. [Section 5 (Limitation) and Section 3 (method, as described in the Abstract)] The core training signal for prototype localization is CLIP similarity between local image patches and concept text. The paper's own Limitation section (Section 5) admits that LCBM 'inherits inherent biases from CLIP' and that 'CLIP is not optimized for identifying detailed concepts within an image.' This is load-bearing because the central claim—localization improvement—is evaluated against human-annotated points/boxes in Section 4.2. If CLIP's patch-level similarity is biased or weakly calibrated for fine-grained concepts such as 'barred black feather' or 'black beak,' the prototype will be attracted to the regions CLIP associates with the phrase rather than the regions a human annotator labels. The manuscript provides no quantitative check that CLIP patch-level similarity agrees with human spatial annotations for the tested concepts. I would ask for an analysis of this agreement, or a
  2. [Section 4 and Conclusion (quantitative results missing in supplied text)] The Conclusion states that LCBM 'significantly improves the localization ability of concept predictions,' but the supplied manuscript does not contain the quantitative experimental tables that would support this. The only visible evidence is the qualitative Figure 6 and selected appendix examples. Without exact metrics—for example, localization point-hit rates or IoU, GradCAM agreement with human points/boxes, concept-removal score decreases, classification accuracy, and concept-presence F1—it is impossible to verify the central claim or compare with baselines such as LfCBM and VLG-CBM. The manuscript needs to report these numbers, including error bars and significance tests, and to specify how many concepts and images are used.
  3. [Evaluation protocol (Section 4.2, as referenced in the text)] There is a potential circularity in the evaluation. The concept vocabulary and concept-presence pseudo-labels are derived from CLIP (or from a language model in prior label-free CBMs), and LCBM is trained to align prototypes with CLIP similarity. GradCAM maps are then computed from the concept scores. Consequently, evaluating concept-presence accuracy and localization against CLIP-derived pseudo-labels partly measures how well the model reproduces CLIP's own outputs, rather than how well it captures human-aligned concepts. The manuscript mentions comparison to human-annotated points/boxes for localization, which is the right direction, but it should also report the correlation between CLIP pseudo-labels and human labels, and separate the CLIP-supervised components from the claims validated against human ground truth.
  4. [Section 2 (Related Works, SALF-CBM discussion)] The paper claims 'to the best of our knowledge, this is the first work to perform a quantitative analysis of concept-level localization,' yet the preceding paragraph discusses SALF-CBM, which 'aims to resolve the issue of concept localization.' The distinction drawn—that SALF-CBM 'cannot determine which concept is most likely to be present in a specific region'—needs to be made concrete with a quantitative comparison against SALF-CBM. If SALF-CBM already performs localization and is evaluated quantitatively, the claimed novelty is weakened. Please clarify what exactly is new and include SALF-CBM in the experimental comparison.
minor comments (3)
  1. [Figure 6 (Qualitative Analysis, Section 4.6)] The description of the GradCAM maps says 'thresholded GradCAM maps' but does not specify the threshold value or how the color scale is defined. Please provide this information so the qualitative results are reproducible.
  2. [Appendix A.5.2 (Local Explanation Examples)] Figures 22–24 in the appendix display quantities such as $\chi$, $\hat{X}$, and $\hat{P}$ without explanation in the visible text. If these are part of the local-explanation visualization, the notation should be defined and connected to the concept contribution bar graphs.
  3. [General] Several references (e.g., [10], [19], [27], [29], [33]) appear only in the appendix or future-work discussion; the main text should cite them where they are first relevant. Also, the supplied text omits Sections 3 and the experimental tables, making the manuscript incomplete as submitted; please ensure the full version is used in review.

Circularity Check

0 steps flagged

No load-bearing circularity: the localization claim is evaluated against human annotations, not against CLIP, and the CLIP reliance is disclosed as a limitation.

full rationale

The claimed derivation chain is: (1) assign a prototype to each concept; (2) train prototypes with an auxiliary classification loss plus a CLIP-similarity alignment loss; (3) compute concept predictions from patch-prototype similarity; (4) visualize with GradCAM and compare to human-annotated points/boxes. Steps (1)-(3) make CLIP's spatial similarity a training signal, but the paper's central localization claim is tested externally: 'we adopted GradCAM to identify the activated regions ... compared these activated regions against ground-truth points or bounding boxes of present concepts.' This is an independent benchmark, so the improved-localization result is not equivalent to the CLIP input by construction. The limitation section explicitly concedes the load-bearing external assumption: 'it still inherits inherent biases from CLIP. Moreover, because CLIP is not optimized for identifying detailed concepts within an image' — an honest falsifiability caveat, not a circular step. No self-citation is load-bearing: the cited works by the authors appear only in future-work suggestions (e.g., augmentations [10,19,33], sketches [18], causality [11]) and do not supply the method's premises. No uniqueness theorem, ansatz smuggled via citation, or renaming of a known result is present. The only potential circularity would be if the concept-presence accuracy evaluation used CLIP-generated pseudo-labels as ground truth, but the visible text does not say that and the localization protocol is explicitly human-annotated; without a quoted reduction, this remains speculation and is not scored.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The paper does not introduce new physical entities, mediators, or conserved quantities. Prototypes are a standard construct from prototype learning literature, and CLIP is an existing foundation model used as an external knowledge source. The main external inputs are CLIP and an LLM-generated concept vocabulary.

free parameters (2)
  • Prototype alignment loss weight
    The paper introduces two additional losses (one for prototype learning, one for concept-region alignment) but the visible text does not report their balancing hyperparameters; these are likely tuned on validation data.
  • Similarity temperature or scale for CLIP-prototype alignment
    CLIP similarity scores are used to align prototypes to concepts; any temperature or scaling applied to these scores affects the gradient strength and is not specified in the visible text.
axioms (3)
  • domain assumption CLIP similarity between a local image patch and a concept text is a reliable indicator of that concept's presence in the patch.
    This underpins the prototype alignment loss; if CLIP's patch-level semantics are weak for fine-grained concepts, the prototypes will be aligned to the wrong regions.
  • domain assumption GradCAM activations and concept-removal performance faithfully reflect the regions the model used for a concept prediction.
    Both localization evaluation protocols in Section 4.2 assume that GradCAM highlights and removal sensitivity are valid proxies for the model's spatial reliance.
  • domain assumption The LLM-generated concept vocabulary covers the relevant visual cues for each class.
    The concept bottleneck can only explain decisions through its vocabulary; if important concepts are missing, the explanation is incomplete regardless of localization quality.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Locality-aware Concept Bottleneck Model." pith.science (2026). https://pith.science/paper/PKFEJU7Y

@misc{pith2026250814562,
  author       = {Pith},
  title        = {Pith review of: Locality-aware Concept Bottleneck Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PKFEJU7Y}},
  note         = {Machine review of arXiv:2508.14562}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Concept bottleneck models (CBMs) are inherently interpretable models that make predictions based on human-understandable visual cues, referred to as concepts. As obtaining dense concept annotations with human labeling is demanding and costly, recent approaches utilize foundation models to determine the concepts existing in the images. However, such label-free CBMs often fail to localize concepts in relevant regions, attending to visually unrelated regions when predicting concept presence. To this end, we propose a framework, coined Locality-aware Concept Bottleneck Model (LCBM), which utilizes rich information from foundation models and adopts prototype learning to ensure accurate spatial localization of the concepts. Specifically, we assign one prototype to each concept, promoted to represent a prototypical image feature of that concept. These prototypes are learned by encouraging them to encode similar local regions, leveraging foundation models to assure the relevance of each prototype to its associated concept. Then we use the prototypes to facilitate the learning process of identifying the proper local region from which each concept should be predicted. Experimental results demonstrate that LCBM effectively identifies present concepts in the images and exhibits improved localization while maintaining comparable classification performance.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Prototype-Grounded Concept Models for Verifiable Concept Alignment

    cs.LG 2026-04 unverdicted novelty 7.0

    Prototype-Grounded Concept Models ground concepts in visual prototypes to enable verifiable alignment and targeted human intervention while matching CBM predictive performance.

  2. Measuring What Matters: Synthetic Benchmarks for Concept Bottleneck Models

    cs.LG 2026-06 unverdicted novelty 6.0

    Introduces synthetic benchmarks for concept bottleneck models that control data modality, concept choice, annotation quality, and completeness to evaluate performance in decision support and automation.

  3. Prototype-Grounded Concept Models for Verifiable Concept Alignment

    cs.LG 2026-04 unverdicted novelty 6.0

    Prototype-Grounded Concept Models ground concepts in learned visual prototypes to enable verifiable alignment and targeted interventions, matching Concept Bottleneck Model performance with improved transparency and in...

Reference graph

Works this paper leans on

40 extracted references · 35 canonical work pages · cited by 2 Pith papers

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    Pith/arXiv arXiv

  2. [2]

    Show and tell: Visually explainable deep neural nets via spatially-aware concept bot- tleneck models

    Itay Benou and Tammy Riklin Raviv. Show and tell: Visually explainable deep neural nets via spatially-aware concept bot- tleneck models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 30063–30072, 2025. 2

    work page 2025

  3. [3]

    This looks like that: deep learn- ing for interpretable image recognition

    Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. This looks like that: deep learn- ing for interpretable image recognition. Advances in neural information processing systems, 32, 2019. 2

    work page 2019

  4. [4]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. 4, 11

    work page 2009

  5. [5]

    Counterfactual concept bottleneck models

    Gabriele Dominici, Pietro Barbiero, Francesco Giannini, Martin Gjoreski, Giuseppe Marra, and Marc Langheinrich. Counterfactual concept bottleneck models. arXiv preprint arXiv:2402.01408, 2024. 18

    Pith/arXiv arXiv 2024

  6. [6]

    Concept embedding mod- els: Beyond the accuracy-explainability trade-off

    Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelan- gelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, et al. Concept embedding mod- els: Beyond the accuracy-explainability trade-off. Advances in Neural Information Processing Systems, 35:21400–21413,

    work page

  7. [7]

    Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset

    Matthew Groh, Caleb Harris, Luis Soenksen, Felix Lau, Rachel Han, Aerin Kim, Arash Koochek, and Omar Badri. Evaluating deep neural networks trained on clinical images in dermatology with the fitzpatrick 17k dataset. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1820–1828, 2021. 5, 13

    work page 2021

  8. [8]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 15

    work page 2016

  9. [9]

    On the concept trustworthiness in concept bottleneck models

    Qihan Huang, Jie Song, Jingwen Hu, Haofei Zhang, Yong Wang, and Mingli Song. On the concept trustworthiness in concept bottleneck models. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 21161–21168, 2024. 2

    work page 2024

  10. [10]

    Selecmix: Debiased learning by contradicting-pair sampling

    Inwoo Hwang, Sangjun Lee, Yunhyeok Kwak, Seong Joon Oh, Damien Teney, Jin-Hwa Kim, and Byoung-Tak Zhang. Selecmix: Debiased learning by contradicting-pair sampling. Advances in Neural Information Processing Systems , 35: 14345–14357, 2022. 18

    work page 2022

  11. [11]

    From black-box to causal-box: Towards building more inter- pretable models

    Inwoo Hwang, Yushu Pan, and Elias Bareinboim. From black-box to causal-box: Towards building more inter- pretable models. Technical Report R-127, Columbia CausalAI Laboratory, 2025. Columbia CausalAI Laboratory, Technical Report (R-127). 18

    work page 2025

  12. [12]

    Probabilistic concept bottleneck models

    Eunji Kim, Dahuin Jung, Sangha Park, Siwon Kim, and Sun- groh Yoon. Probabilistic concept bottleneck models. In In- ternational Conference on Machine Learning, pages 16521– 16540. PMLR, 2023. 1, 2

    work page 2023

  13. [13]

    Towards causal representation learning with observable sources as auxiliaries

    Kwonho Kim, Heejeong Nam, Inwoo Hwang, and Sanghack Lee. Towards causal representation learning with observable sources as auxiliaries. In UAI 2025 Workshop on Causal Abstractions and Representations, 2025. 18

    work page 2025

  14. [14]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 18

    work page 2023

  15. [15]

    Concept bottleneck models

    Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning, pages 5338–5348. PMLR, 2020. 1, 2

    work page 2020

  16. [16]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on com- puter vision workshops, pages 554–561, 2013. 4, 12

    work page 2013

  17. [17]

    Faithful vision-language interpretation via concept bottleneck models

    Songning Lai, Lijie Hu, Junxiao Wang, Laure Berti-Equille, and Di Wang. Faithful vision-language interpretation via concept bottleneck models. In The Twelfth International Conference on Learning Representations, 2024. 1, 2

    work page 2024

  18. [18]

    Learning geometry- aware representations by sketching

    Hyundo Lee, Inwoo Hwang, Hyunsung Go, Won-Seok Choi, Kibeom Kim, and Byoung-Tak Zhang. Learning geometry- aware representations by sketching. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23315–23326, 2023. 18

    work page 2023

  19. [19]

    Improving robustness to texture bias via shape- focused augmentation

    Sangjun Lee, Inwoo Hwang, Gi-Cheon Kang, and Byoung- Tak Zhang. Improving robustness to texture bias via shape- focused augmentation. In Proceedings of the ieee/cvf con- ference on computer vision and pattern recognition , pages 4323–4331, 2022. 18

    work page 2022

  20. [20]

    Dynamic computational time for visual attention

    Zhichao Li, Yi Yang, Xiao Liu, Feng Zhou, Shilei Wen, and Wei Xu. Dynamic computational time for visual attention. In Proceedings of the IEEE international conference on com- puter vision workshops, pages 1199–1209, 2017. 16

    work page 2017

  21. [21]

    Decoupled weight de- cay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2019. 15

    work page 2019

  22. [22]

    This looks like those: Illuminating prototypical con- cepts using multiple visualizations

    Chiyu Ma, Brandon Zhao, Chaofan Chen, and Cynthia Rudin. This looks like those: Illuminating prototypical con- cepts using multiple visualizations. Advances in Neural In- formation Processing Systems, 36:39212–39235, 2023. 2

    work page 2023

  23. [23]

    Do concept bottleneck models learn as intended? arXiv preprint arXiv:2105.04289, 2021

    Andrei Margeloiu, Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller. Do concept bottleneck models learn as intended? arXiv preprint arXiv:2105.04289, 2021. 2

    Pith/arXiv arXiv 2021

  24. [24]

    Pip-net: Patch-based intuitive prototypes for interpretable image classification

    Meike Nauta, J ¨org Schl ¨otterer, Maurice van Keulen, and Christin Seifert. Pip-net: Patch-based intuitive prototypes for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2744–2753, 2023. 2

    work page 2023

  25. [25]

    Nguyen, and Tsui- Wei Weng

    Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, and Tsui- Wei Weng. Label-free concept bottleneck models. In The Eleventh International Conference on Learning Representa- tions, 2023. 1, 2, 3, 5, 6, 15 9

    work page 2023

  26. [26]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, pages 8748–8763, 2021. 2

    work page 2021

  27. [27]

    From causal to concept-based representation learning

    Goutham Rajendran, Simon Buchholz, Bryon Aragam, Bernhard Sch ¨olkopf, and Pradeep Ravikumar. From causal to concept-based representation learning. Advances in Neu- ral Information Processing Systems , 37:101250–101296,

    work page

  28. [28]

    Do concept bottleneck models obey local- ity?, 2024

    Naveen Raman, Mateo Espinosa Zarlenga, Juyeon Heo, and Mateja Jamnik. Do concept bottleneck models obey local- ity?, 2024. 2

    work page 2024

  29. [29]

    Toward causal representation learning.Pro- ceedings of the IEEE, 109(5):612–634, 2021

    Bernhard Sch ¨olkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning.Pro- ceedings of the IEEE, 109(5):612–634, 2021. 18

    work page 2021

  30. [30]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE in- ternational conference on computer vision , pages 618–626,

  31. [31]

    Incremental residual con- cept bottleneck models

    Chenming Shang, Shiji Zhou, Hengyuan Zhang, Xinzhe Ni, Yujiu Yang, and Yuwang Wang. Incremental residual con- cept bottleneck models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11030–11040, 2024. 1, 2

    work page 2024

  32. [32]

    Auxiliary losses for learning generalizable concept-based models

    Ivaxi Sheth and Samira Ebrahimi Kahou. Auxiliary losses for learning generalizable concept-based models. In Thirty- seventh Conference on Neural Information Processing Sys- tems, 2023. 1, 2

    work page 2023

  33. [33]

    A survey on image data augmentation for deep learning

    Connor Shorten and Taghi M Khoshgoftaar. A survey on image data augmentation for deep learning. Journal of big data, 6(1):1–48, 2019. 18

    work page 2019

  34. [34]

    Vlg-cbm: Training concept bottleneck models with vision-language guidance

    Divyansh Srivastava, Ge Yan, and Tsui-Wei Weng. Vlg-cbm: Training concept bottleneck models with vision-language guidance. arXiv preprint arXiv:2408.01432, 2024. 1, 2, 3, 5, 6, 15

    Pith/arXiv arXiv 2024

  35. [35]

    C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. Caltech-ucsd birds-200-2011. Technical Report CNS-TR- 2011-001, California Institute of Technology, 2011. 4, 11

    work page 2011

  36. [36]

    Energy-based concept bottleneck models: Unifying predic- tion, concept intervention, and probabilistic interpretations

    Xinyue Xu, Yi Qin, Lu Mi, Hao Wang, and Xiaomeng Li. Energy-based concept bottleneck models: Unifying predic- tion, concept intervention, and probabilistic interpretations. In The Twelfth International Conference on Learning Repre- sentations, 2024. 1, 2

    work page 2024

  37. [37]

    Learning concise and descriptive attributes for visual recognition

    An Yan, Yu Wang, Yiwu Zhong, Chengyu Dong, Zexue He, Yujie Lu, William Yang Wang, Jingbo Shang, and Julian McAuley. Learning concise and descriptive attributes for visual recognition. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 3090–3100,

    work page

  38. [38]

    Language in a bottle: Language model guided concept bottlenecks for interpretable image classification

    Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19187–19197, 2023. 1, 3, 5, 6, 15

    work page 2023

  39. [39]

    Post-hoc concept bottleneck models

    Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. In The Eleventh International Conference on Learning Representations, 2023. 2 10 Figure 7. Histogram for MNIST experiment. A.5. Additional Qualitative Examples Fig. 8 to Fig. 10 present the results on three datasets, using the same examples as section 4.6. Concepts highlight...

    work page 2023

  40. [40]

    18 Figure 8

    or sketches [18] during training. 18 Figure 8. Qualitative Results for VLG-CBM. 19 Figure 9. Qualitative Results for LfCBM. 20 Figure 10. Qualitative Results for LCBM. Figure 11. Qualitative Results of LCBM for CUB-200-2011. 21 Figure 12. Qualitative Results of LCBM for ImageNet-animal. Figure 13. Qualitative Results of LCBM for Stanford Cars. 22 Figure 1...

    work page 2011

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.