Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Discovering Fine-Grained Visual-Concept Relations by Disentangled Optimal Transport Concept Bottleneck Models

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Modeling concept prediction as optimal transport between image patches and concept embeddings yields explicit visual-concept alignment, more reliable concept predictions, and state-of-the-art classification, part detection, and OOD…

desk verdict A genuinely interesting CBM extension that marries OT-based patch-concept alignment to concept bottlenecks, but Eq. 18's concept prior is formally broken as written and the method section needs a fix before the SOTA claims can be taken at face value. read the letter →

arxiv 2505.07209 v1 pith:6PQWHRHL submitted 2025-05-12 cs.CV

Yan Xie , Zequn Zeng , Hao Zhang , Yucheng Ding , Yi Wang , Zhengjue Wang , Bo Chen , Hongwei Liu This is my paper · ORCID
classification cs.CV
keywords conceptbottleneckmodelsoptimaltransportfine-grainedvisual-conceptalignmentinterpretabilityspuriouscorrelationpartdetectionout-of-distributiongeneralizationdisentanglement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a Concept Bottleneck Model (an interpretable classifier that predicts human-understandable concepts before the final class) can align those concepts to specific image regions by treating concept prediction as an optimal transport problem—a matching that moves mass from local image patches to concept embeddings. The payoff is a single transport plan that serves both as the concept predictor and as an explicit inversion heatmap, so users can see which pixels caused each concept prediction. The paper reports that this design outperforms prior concept models on ImageNet (83.84%), CUB (85.39%), and part detection (50.12 mAP on PartImageNet), and improves out-of-distribution accuracy on background-shifted datasets. The takeaway is that fine-grained, explicit alignment plus two data-bias priors can make concept models both more accurate and more interpretable.

What carries the argument

The carrying object is the optimal transport plan $Q^*$, an $N \times M$ matrix coupling image patches to concept embeddings, computed by Sinkhorn fixed-point iterations on a cosine-distance cost with prescribed row and column sums $\theta$ and $\gamma$. The same matrix is read in two directions: each concept activation $\hat{a}_j$ comes from $1 - \min_i (q^*_{ij} d_{ij})$ after a sigmoid, and each concept's inversion mask is the column vector $(q^*_{1j}, \dots, q^*_{Nj})$. Two orthogonal projection losses, one on patch features and one on concept features, push each modality onto a unit hypersphere with low cosine similarity, while the saliency and co-occurrence priors shape the marginals. The Sinkhorn plan is what makes the image-to-concept map no longer a black box: it is an explicit, invertible assignment rather than a learned global feature.

What would settle it

On any dataset used in the paper, compute $\sum_{j=1}^M \gamma_j$ from Eq. (18) with $M$ the number of concepts; it equals $M-1$, not $1$. If the released training code is run with $\gamma$ exactly as stated, the Sinkhorn iteration cannot satisfy the column marginal $Q^\top \mathbf{1}_N = \gamma$, so the concept activations and inversion masks would either fail or depend on an invisible normalization; changing the normalization rule should measurably change every reported mask and accuracy.

Watch

Extended reading notes

Core claim

The central claim is that replacing the black-box mapping from an entire image to concept labels with an optimal transport problem between local image patches and concept features eliminates the coarse-grained, spurious visual-concept associations typical of vanilla Concept Bottleneck Models. After orthogonal-projection losses disentangle patch features and concept features, the Sinkhorn solution $Q^*$—an $N \times M$ assignment matrix—defines every downstream quantity: concept activations are computed from the minimum transport distance between a concept and all patches, and the concept inversion mask is the corresponding column of $Q^*$. Two prior distributions (a saliency-based patch prior and a conditional co-occurrence concept prior) penalize shortcut alignments to background and to frequently co-occurring concepts. The paper's evidence is that this single mechanism simultaneously raises classification accuracy, part-detection mAP by roughly eight points over the best reimplemented baseline, and out-of-distribution generalization under background shift.

Load-bearing premise

The load-bearing premise is that the concept-side prior is a true probability distribution adding up to one; as written it adds up to one fewer than the number of concepts, so without an unstated renormalization the matching problem has no solution and every reported result is undefined.

Editorial extensions

If this is right

  • Concept predictions no longer need post-hoc localization: the inversion mask is a free byproduct of the transport plan, and the paper measures it directly with part-detection mAP.
  • Fine-grained patch-concept alignment reduces part-background, part-object, and part-part spurious correlations, which the paper links to classification gains of 0.91 points on ImageNet and 3.29 points on CUB over the best reimplemented baselines.
  • Saliency-based and co-occurrence-based priors improve out-of-distribution accuracy sharply under background shift: +10.3 on CUB and +10.7 on Dogs over the strongest baseline.
  • Because the concept-to-class layer spontaneously becomes sparse under the orthogonal losses, the model retains a human-readable concept-to-class explanation while approaching black-box accuracy.
  • The framework is compatible with frozen pretrained encoders, so the same recipe can be applied at ImageNet scale without heavy training compute.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not pursue is interactive intervention: because $Q^*$ is explicit, a user could manually edit the alignment or point to a patch and ask which concepts it feeds, turning the inversion mask into a debugging tool rather than just a visualization.
  • The reported OOD gains likely depend on the quality of the saliency map from the pretrained vision transformer; an ablation that replaces the fixed saliency prior with a self-refined mask would show how much robustness comes from the prior versus from the OT alignment itself.
  • The concept-set prior in Eq. (18) is written as $1 - \mathrm{Softmax}(r_i)$, which sums to $M-1$, not $1$; if an unstated renormalization is applied in code, its exact form is an underdocumented degree of freedom that changes the Sinkhorn marginals and therefore every inversion mask.
  • The same patch-to-concept transport could transfer to other bottleneck tasks where part-level attribution matters, such as medical image diagnosis, by swapping the concept bank without changing the alignment machinery.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DOT-CBM, a Concept Bottleneck Model that models the image-to-concept mapping as an optimal transport problem between local image patches (from DINOv2) and concept embeddings (from a CLIP text encoder). Two orthogonality losses are used to disentangle patch features and concept features, and two priors (a saliency-based patch prior and a co-occurrence-based concept prior) are introduced to reduce spurious visual-concept correlations. The learned transport plan Q* yields concept activations through a minimum-product rule and inversion masks through its columns. The model is trained in two stages and evaluated on image classification (ImageNet, CUB, CIFAR-100, AwA2), part detection (PartImageNet, CUB, RIVAL-10, PASCAL-Parts), and out-of-distribution generalization, reporting state-of-the-art results against re-implemented baselines. Ablations and qualitative visualizations are also provided.

Significance. If the technical issues are resolved, the paper addresses a real limitation of existing CBMs—the lack of explicit fine-grained alignment between image regions and concepts—and offers a natural mechanism for visualizing concept predictions. The reported gains on part detection and OOD generalization are substantial, and the inclusion of dual priors to counter dataset bias is a thoughtful design. The manuscript also credibly reimplements baselines with a controlled backbone and concept set. However, the central derivation contains a formal gap (Eq. 18) and the concept activation rule (Eq. 12) is not justified, which currently prevents the acceptance of the method as described. The absence of error bars and code further weakens the empirical claims.

major comments (4)
  1. [Section 3.4, Eq. (18)] The definition γ = 1 − Softmax(r_i) produces a vector whose entries sum to M − 1, not 1, when r_i is a scalar per concept and M > 2. This violates the simplex constraint on γ and makes the column marginal Q^T 1_N = γ in Eq. (9) infeasible. As a result, the Sinkhorn problem in Eqs. (9)–(10) has no feasible solution, and the optimal Q* used in Eqs. (11)–(14) is undefined as written. The paper must explicitly renormalize the vector, define a different mapping (e.g., 1 − Softmax(r_i) normalized by M−1, or a softmax over the negative co-occurrence counts), or otherwise state the missing normalization step. This is a load-bearing issue because all downstream quantities—concept activations, inversion masks, and the concept loss—depend on Q*.
  2. [Section 3.3, Eq. (12)] The concept activation rule â_j = σ(1 − min_i(q_ij d_ij)) is introduced without derivation or empirical justification. The standard OT cost per concept would be Σ_i q_ij d_ij or an aggregate transport quantity; the minimum of products is not a natural consequence of the OT formulation. The paper should either derive this rule from the OT objective, motivate it as a design choice with reference to competing alternatives, or include an ablation study comparing min, mean, and sum aggregations. Without this, the connection between the OT plan and the reported concept prediction quality is not established.
  3. [Section 4.3, part detection evaluation] The inversion mask for concept c_j is defined directly as the column vector m_j = [q_1j, ..., q_Nj] (Eq. 13), and the same Q* is used to compute concept activations (Eq. 12) and to produce part detection masks. Consequently, part detection mAP partially measures how well the training objective (concept-label BCE loss) has been optimized, rather than providing independent evidence of fine-grained alignment. The classification and OOD experiments are external, but the part detection claim is a central contribution. I recommend validating the masks on held-out concepts or images, or comparing with an OT plan that is not supervised by concept labels, to demonstrate that the alignment is genuinely meaningful.
  4. [Section 4, experimental reporting] All tables (Tables 1, 2, 3) report a single run without standard deviations, error bars, or significance tests. The reported margins are often small (e.g., +0.91 on ImageNet, −0.25 on AwA2), so the paper should include multiple seeds and confidence intervals to support the claimed improvements. Additionally, the manuscript does not provide a code link or any other reproduction artifact; given the formal gap in Eq. (18), releasing code would be particularly important to confirm the method as described.
minor comments (6)
  1. [Section 3.3, Eq. (9)] The text refers to Q as a 'doubly stochastic matrix,' but with general marginals θ and γ the feasible set is a transportation polytope, not a doubly stochastic matrix. Please correct the terminology to avoid confusion with the standard definition.
  2. [Figure 2 caption] The phrase 'sum= Softmax(1− )' in the figure caption appears garbled and should be rewritten to clearly describe the prior computation for the concept set.
  3. [Section 4.2, hyperparameters] The hyperparameters λ1, λ2, λ3 are specified only for CUB; the appendix should state the values used for all datasets, or explicitly confirm that the same values are used everywhere.
  4. [Table 3 caption] The abbreviation 'LrD' is used in the table but is not defined in the caption; it is only explained in the body text of Section 4.5. Please define it in the caption or at first use in the table.
  5. [Section 4.4, OOD experiment] The OOD experiment relies on SAM-generated segmentation masks for the foreground/background split; please report a validation of the mask quality (e.g., IoU against ground truth, if any) so that the experimental setup is fully transparent.
  6. [General presentation] Some references are incomplete (e.g., entries for arXiv preprints lack version numbers or access dates). Please ensure all bibliographic entries are complete and consistent.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the fine-grained OT alignment is an independently testable mechanism, and the part-detection evaluation uses external ground-truth boxes not present in the training objective.

full rationale

The paper's derivation chain is self-contained in the relevant sense: the optimal transport assignment Q* is computed from patch and concept features, priors, and the Sinkhorn objective; concept activations (Eq. 12) and inversion masks (Eq. 13) are both read from Q*, but this is a shared internal representation, not an input fitted to the claimed output. The part-detection benchmark compares the inversion masks against ground-truth part bounding boxes from CUB, PartImageNet, RIVAL-10, and PASCAL-Parts; those masks are never trained against the box annotations, so the localization result is external evidence rather than a relabeled training target. Classification and OOD experiments use standard held-out benchmarks, and no fitted constant is renamed as a prediction. The paper cites related work, including PatchCT by overlapping authors, but only as background and not as a load-bearing justification for the method's correctness. The formal gap flagged by the skeptic—Eq. 18 defining γ = 1 − Softmax(r_i) does not sum to 1 as required by Eq. 8–9—is a correctness/reproducibility concern, not a circularity concern, because it does not make any output equivalent to an input by construction. No self-definitional step, fitted-input-called-prediction step, or self-citation chain is present in the manuscript.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The method rests on the assumption that CLIP text embeddings and DINOv2 patch embeddings are comparable in one metric space, that the saliency prior and concept co-occurrence prior are valid, and on two ad hoc modeling choices: the min rule in Eq. 12 and the unnormalized gamma prior in Eq. 18. No new physical entities are introduced.

free parameters (3)
  • Loss weights lambda1, lambda2, lambda3 = 0.15, 0.2, 0.8 on CUB; other datasets in missing appendix
    Tuned by hand; no sensitivity curves for all datasets in the main text.
  • OT entropy regularization coefficient epsilon = not reported
    Sinkhorn iterations need epsilon in Eq. 9, but its value is not given in the main text.
  • Part-detection binarization threshold = 0.6 of maximum mask peak
    Chosen for converting inversion masks to bounding boxes; no sensitivity analysis is reported.
assumptions (6)
  • standard math Sinkhorn fixed-point iterations solve the entropic optimal transport problem in Eq. 9.
    Standard result from Cuturi (reference 7), used in Eq. 10.
  • domain assumption Cosine distance in the shared CLIP and DINOv2 feature space is a semantically meaningful transport cost between image patches and concept descriptions.
    Used in Eq. 9 and Eq. 12; no calibration of the raw distances beyond the learned adapters.
  • domain assumption The saliency map S_I = g_I dot P from the frozen DINOv2 model separates foreground from background well enough to form the patch prior theta.
    Section 3.4, Eq. 16; if saliency is wrong, the prior will suppress relevant patches.
  • domain assumption Concept co-occurrence statistics from the class-concept label matrix L are a valid proxy for shortcut bias that should be penalized.
    Section 3.4, Eqs. 17 and 18; dataset-level statistics may not reflect image-level concept co-occurrence.
  • ad hoc to paper Concept activation uses the single minimum product min_i(q_ij d_ij) rather than an aggregate transport cost.
    Eq. 12 is not derived from the optimal transport objective; it is a heuristic that ignores all but one patch.
  • ad hoc to paper The transformation 1 - Softmax(r_i) is a probability distribution satisfying the optimal transport marginal constraints.
    As written it sums to M - 1, violating Eq. 8 and Eq. 9; this is the main formal flaw.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Discovering Fine-Grained Visual-Concept Relations by Disentangled Optimal Transport Concept Bottleneck Models." pith.science (2026). https://pith.science/paper/6PQWHRHL

@misc{pith2026250507209,
  author       = {Pith},
  title        = {Pith review of: Discovering Fine-Grained Visual-Concept Relations by Disentangled Optimal Transport Concept Bottleneck Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6PQWHRHL}},
  note         = {Machine review of arXiv:2505.07209}
}
read the original abstract

Concept Bottleneck Models (CBMs) try to make the decision-making process transparent by exploring an intermediate concept space between the input image and the output prediction. Existing CBMs just learn coarse-grained relations between the whole image and the concepts, less considering local image information, leading to two main drawbacks: i) they often produce spurious visual-concept relations, hence decreasing model reliability; and ii) though CBMs could explain the importance of every concept to the final prediction, it is still challenging to tell which visual region produces the prediction. To solve these problems, this paper proposes a Disentangled Optimal Transport CBM (DOT-CBM) framework to explore fine-grained visual-concept relations between local image patches and concepts. Specifically, we model the concept prediction process as a transportation problem between the patches and concepts, thereby achieving explicit fine-grained feature alignment. We also incorporate orthogonal projection losses within the modality to enhance local feature disentanglement. To further address the shortcut issues caused by statistical biases in the data, we utilize the visual saliency map and concept label statistics as transportation priors. Thus, DOT-CBM can visualize inversion heatmaps, provide more reliable concept predictions, and produce more accurate class predictions. Comprehensive experiments demonstrate that our proposed DOT-CBM achieves SOTA performance on several tasks, including image classification, local part detection and out-of-distribution generalization.

Figures

Figures reproduced from arXiv: 2505.07209 by the authors.

Figure 1
Figure 1. Comparison between vanilla CBM and our proposed [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our proposed DOT-CBM. The overall CBM framework proceeds from left to right. In the first stage, the model [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Conditioned co-occurrence matrix of concept labels. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Qualitative results for the concept inversion. VanillaCBM and CoopCBM adopt Grad-CAM to provide a concept inversion [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The SuperActivator Mechanism: Transformers Concentrate Reliable Concept Signals in the Tail

    cs.LG 2025-12 conditional novelty 6.0 of 10

    Reliable concept presence in transformers is concentrated in the extreme high-activation tail of in-concept tokens; thresholding that tail improves concept detection and localization.

Reference graph

Works this paper leans on

52 extracted references · 29 canonical work pages · cited by 1 Pith paper

  1. [1]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Dosovitskiy Alexey. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv: 2010.11929, 2020. 4

    arXiv 2010

  2. [2]

    Interactive concept bottleneck models

    Kushal Chauhan, Rishabh Tiwari, Jan Freyberg, Pradeep Shenoy, and Krishnamurthy Dvijotham. Interactive concept bottleneck models. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 5948–5955, 2023. 2

    work page 2023

  3. [3]

    Plot: Prompt learning with optimal transport for vision-language models

    Guangyi Chen, Weiran Yao, Xiangchen Song, Xinyue Li, Yongming Rao, and Kun Zhang. Plot: Prompt learning with optimal transport for vision-language models. arXiv preprint arXiv:2210.01253, 2022. 3

    arXiv 2022

  4. [4]

    Graph optimal transport for cross-domain alignment

    Liqun Chen, Zhe Gan, Yu Cheng, Linjie Li, Lawrence Carin, and Jingjing Liu. Graph optimal transport for cross-domain alignment. In International Conference on Machine Learn- ing, pages 1542–1553. PMLR, 2020. 3

    work page 2020

  5. [5]

    Detect what you can: Detecting and representing objects using holistic mod- els and body parts

    Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu, Sanja Fi- dler, Raquel Urtasun, and Alan Yuille. Detect what you can: Detecting and representing objects using holistic mod- els and body parts. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1971–1978,

    1971

  6. [6]

    Interpretable machine learning: A guide for making black box models explainable

    Molnar Christoph. Interpretable machine learning: A guide for making black box models explainable. Leanpub, 2020. 1

    work page 2020

  7. [7]

    Sinkhorn distances: Lightspeed computation of optimal transport

    Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. Advances in neural information pro- cessing systems, 26, 2013. 4

    work page 2013

  8. [8]

    Decision trees

    Barry De Ville. Decision trees. Wiley Interdisciplinary Re- views: Computational Statistics, 5(6):448–455, 2013. 1

    work page 2013

Show all 52 references
  1. [9]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 6

    2009

  2. [10]

    Concept embedding mod- els: Beyond the accuracy-explainability trade-off

    Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelan- gelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, et al. Concept embedding mod- els: Beyond the accuracy-explainability trade-off. Advance...

  3. [11]

    Learning to receive help: Intervention-aware concept embedding models

    Mateo Espinosa Zarlenga, Katie Collins, Krishnamurthy Dvijotham, Adrian Weller, Zohreh Shams, and Mateja Jam- nik. Learning to receive help: Intervention-aware concept embedding models. Advances in Neural Information Pro- cessing Systems, 36, 2024. 2

    2024

  4. [12]

    Learning with a wasserstein loss

    Charlie Frogner, Chiyuan Zhang, Hossein Mobahi, Mauricio Araya, and Tomaso A Poggio. Learning with a wasserstein loss. Advances in neural information processing systems, 28,

  5. [13]

    Partimagenet: A large, high- quality dataset of parts

    Ju He, Shuo Yang, Shaokang Yang, Adam Kortylewski, Xi- aoding Yuan, Jie-Neng Chen, Shuai Liu, Cheng Yang, Qi- hang Yu, and Alan Yuille. Partimagenet: A large, high- quality dataset of parts. In European Conference on Com- puter Vision, pages 128–145. Springer, 2022. 6

    2022

  6. [14]

    An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models

    Johan Huysmans, Karel Dejaeger, Christophe Mues, Jan Vanthienen, and Bart Baesens. An empirical evaluation of the comprehensibility of decision table, tree and rule based predictive models. Decision Support Systems , 51(1):141– 154, 2011. 1

    2011

  7. [15]

    Novel dataset for fine-grained image categorization

    Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. In First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, 2011. 7

    2011

  8. [16]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF International Con- ference on Computer Vision, pages 4015–4026, 2023. 7

    2023

  9. [17]

    Concept bottleneck models

    Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning, pages 5338–5348. PMLR, 2020. 1, 2, 3, 6, 7

    2020

  10. [18]

    Incorporating ex- pert rules into neural networks in the framework of concept- based learning

    Andrei V Konstantinov and Lev V Utkin. Incorporating ex- pert rules into neural networks in the framework of concept- based learning. arXiv preprint arXiv:2402.14726, 2024. 2

    2024 arXiv

  11. [19]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 6

    2009

  12. [20]

    Hierarchical optimal transport for multimodal distribution alignment

    John Lee, Max Dabagia, Eva Dyer, and Christopher Rozell. Hierarchical optimal transport for multimodal distribution alignment. Advances in neural information processing sys- tems, 32, 2019. 3

    2019

  13. [21]

    Patchct: Align- ing patch set and label set with conditional transport for multi-label image classification

    Miaoge Li, Dongsheng Wang, Xinyang Liu, Zequn Zeng, Ruiying Lu, Bo Chen, and Mingyuan Zhou. Patchct: Align- ing patch set and label set with conditional transport for multi-label image classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision ,...

    2023

  14. [22]

    Deal: Disentan- gle and localize concept-level explanations for vlms

    Tang Li, Mengmeng Ma, and Xi Peng. Deal: Disentan- gle and localize concept-level explanations for vlms. In European Conference on Computer Vision, pages 383–401. Springer, 2025. 2, 4

    2025

  15. [23]

    Multi-granularity correspondence learning from long-term noisy videos

    Yijie Lin, Jie Zhang, Zhenyu Huang, Jia Liu, Zujie Wen, and Xi Peng. Multi-granularity correspondence learning from long-term noisy videos. arXiv preprint arXiv:2401.16702 ,

    arXiv

  16. [24]

    Mode: Clip data experts via clustering

    Jiawei Ma, Po-Yao Huang, Saining Xie, Shang-Wen Li, Luke Zettlemoyer, Shih-Fu Chang, Wen-Tau Yih, and Hu Xu. Mode: Clip data experts via clustering. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26354–26363, 2024. 2

    2024

  17. [25]

    Joint wasserstein autoencoders for aligning multi- modal embeddings

    Shweta Mahajan, Teresa Botschen, Iryna Gurevych, and Ste- fan Roth. Joint wasserstein autoencoders for aligning multi- modal embeddings. In Proceedings of the IEEE/CVF Inter- national Conference on Computer Vision Workshops, pages 0–0, 2019. 3

    2019

  18. [26]

    Do concept bottleneck models learn as intended? arXiv preprint arXiv:2105.04289, 2021

    Andrei Margeloiu, Matthew Ashman, Umang Bhatt, Yanzhi Chen, Mateja Jamnik, and Adrian Weller. Do concept bottleneck models learn as intended? arXiv preprint arXiv:2105.04289, 2021. 2, 3, 6

    2021 arXiv

  19. [27]

    [re] on the reproducibil- ity of post-hoc concept bottleneck models

    Nesta Midavaine, Gregory Hok Tjoan Go, Diego Canez, Ioana Simion, and Satchit Chatterji. [re] on the reproducibil- ity of post-hoc concept bottleneck models. Transactions on Machine Learning Research. 2

  20. [28]

    A comprehensive study of image classification model sensitivity to foregrounds, backgrounds, and visual attributes

    Mazda Moayeri, Phillip Pope, Yogesh Balaji, and So- heil Feizi. A comprehensive study of image classification model sensitivity to foregrounds, backgrounds, and visual attributes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 6

    2022

  21. [29]

    Text2concept: Concept activation vectors di- rectly from text

    Mazda Moayeri, Keivan Rezaei, Maziar Sanjabi, and So- heil Feizi. Text2concept: Concept activation vectors di- rectly from text. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 3744–3749, 2023. 2, 4

    2023

  22. [30]

    Attention- based joint detection of object and semantic part

    Keval Morabia, Jatin Arora, and Tara Vijaykumar. Attention- based joint detection of object and semantic part. arXiv preprint arXiv:2007.02419, 2020. 6

    2007 arXiv

  23. [31]

    Label-free concept bottleneck models

    Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui- Wei Weng. Label-free concept bottleneck models. arXiv preprint arXiv:2304.06129, 2023. 2

    2023 arXiv

  24. [32]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 6

    2023 arXiv

  25. [33]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  26. [34]

    Do vision trans- formers see like convolutional neural networks? Advances in neural information processing systems, 34:12116–12128,

    Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, and Alexey Dosovitskiy. Do vision trans- formers see like convolutional neural networks? Advances in neural information processing systems, 34:12116–12128,

  27. [35]

    Do concept bottleneck models obey locality? In XAI in Action: Past, Present, and Future Applications ,

    Naveen Raman, Mateo Espinosa Zarlenga, Juyeon Heo, and Mateja Jamnik. Do concept bottleneck models obey locality? In XAI in Action: Past, Present, and Future Applications ,

  28. [36]

    Optimal transport for multi-source domain adaptation under target shift

    Ievgen Redko, Nicolas Courty, R ´emi Flamary, and Devis Tuia. Optimal transport for multi-source domain adaptation under target shift. In The 22nd International Conference on artificial intelligence and statistics, pages 849–858. PMLR,

  29. [37]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE in- ternational conference on computer vision , pages 618–626,

  30. [38]

    Sparse concept bottleneck mod- els: Gumbel tricks in contrastive learning

    Andrei Semenov, Vladimir Ivanov, Aleksandr Beznosikov, and Alexander Gasnikov. Sparse concept bottleneck mod- els: Gumbel tricks in contrastive learning. arXiv preprint arXiv:2404.03323, 2024. 2, 6, 7

    2024 arXiv

  31. [39]

    Auxiliary losses for learning generalizable concept-based models

    Ivaxi Sheth and Samira Ebrahimi Kahou. Auxiliary losses for learning generalizable concept-based models. Advances in Neural Information Processing Systems, 36, 2024. 2, 6, 7

    2024

  32. [40]

    A closer look at the intervention procedure of concept bot- tleneck models

    Sungbin Shin, Yohan Jo, Sungsoo Ahn, and Namhoon Lee. A closer look at the intervention procedure of concept bot- tleneck models. In International Conference on Machine Learning, pages 31504–31520. PMLR, 2023. 2

    2023

  33. [41]

    The caltech-ucsd birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Per- ona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. 2011. 6, 7

    2011

  34. [42]

    Beyond matryoshka: Revisit- ing sparse coding for adaptive representation

    Tiansheng Wen, Yifei Wang, Zequn Zeng, Zhong Peng, Yudi Su, Xinyang Liu, Bo Chen, Hongwei Liu, Stefanie Jegelka, and Chenyu You. Beyond matryoshka: Revisit- ing sparse coding for adaptive representation. arXiv preprint arXiv:2503.01776, 2025. 2

    2025 arXiv

  35. [43]

    Zero-shot learning—a comprehensive eval- uation of the good, the bad and the ugly

    Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata. Zero-shot learning—a comprehensive eval- uation of the good, the bad and the ugly. IEEE transactions on pattern analysis and machine intelligence , 41(9):2251– 2265, 2018. 6

    2018

  36. [44]

    Attribute prototype network for zero-shot learning

    Wenjia Xu, Yongqin Xian, Jiuniu Wang, Bernt Schiele, and Zeynep Akata. Attribute prototype network for zero-shot learning. Advances in Neural Information Processing Sys- tems, 33:21969–21980, 2020. 6

    2020

  37. [45]

    Language in a bottle: Language model guided concept bottlenecks for interpretable image classification

    Yue Yang, Artemis Panagopoulou, Shenghao Zhou, Daniel Jin, Chris Callison-Burch, and Mark Yatskar. Language in a bottle: Language model guided concept bottlenecks for interpretable image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2023

  38. [46]

    Post-hoc concept bottleneck models

    Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. arXiv preprint arXiv:2205.15480, 2022. 2

    2022 arXiv

  39. [47]

    Pre-trained vision-language models learn discoverable visual concepts

    Yuan Zang, Tian Yun, Hao Tan, Trung Bui, and Chen Sun. Pre-trained vision-language models learn discoverable visual concepts. arXiv preprint arXiv:2404.12652, 2024. 2, 4, 6

    2024 arXiv

  40. [48]

    Concept embedding models: Beyond the accuracy-explainability trade-off

    Mateo Espinosa Zarlenga, Pietro Barbiero, Gabriele Ciravegna, Giuseppe Marra, Francesco Giannini, Michelan- gelo Diligenti, Zohreh Shams, Frederic Precioso, Stefano Melacci, Adrian Weller, et al. Concept embedding models: Beyond the accuracy-explainability trade-off. arXiv pre...

    2022 arXiv

  41. [49]

    Conzic: Controllable zero-shot image captioning by sampling-based polishing

    Zequn Zeng, Hao Zhang, Ruiying Lu, Dongsheng Wang, Bo Chen, and Zhengjue Wang. Conzic: Controllable zero-shot image captioning by sampling-based polishing. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 23465–23476, 2023. 2

    2023

  42. [50]

    Meacap: Memory-augmented zero- shot image captioning

    Zequn Zeng, Yan Xie, Hao Zhang, Chiyu Chen, Bo Chen, and Zhengjue Wang. Meacap: Memory-augmented zero- shot image captioning. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 14100–14110, 2024. 2

    2024

  43. [51]

    Spda-cnn: Unifying semantic part detection and abstraction for fine-grained recognition

    Han Zhang, Tao Xu, Mohamed Elhoseiny, Xiaolei Huang, Shaoting Zhang, Ahmed Elgammal, and Dimitris Metaxas. Spda-cnn: Unifying semantic part detection and abstraction for fine-grained recognition. InProceedings of the IEEE con- ference on computer vision and pattern recognition...

    2016

  44. [52]

    Label distribution learning by optimal transport

    Peng Zhao and Zhi-Hua Zhou. Label distribution learning by optimal transport. In Proceedings of the AAAI Conference on Artificial Intelligence, 2018. 3

    2018

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.