Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts

Jayaram Raghuram; Jihye Choi; Somesh Jha; Yixuan Li

REVIEW 3 major objections 6 minor 53 references

Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts

T0 review · 3 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper proposes CONDA, the first test-time adaptation method for concept bottleneck models with foundation-model backbones, and reports that it restores post-deployment accuracy by up to 28% using only unlabeled target data.

desk verdict A genuine first stab at test-time adaptation for concept bottlenecks, with real gains in some settings but a headline claim that softens once you look at pseudo-label dependence and the severity-2 evaluation. read the letter →

arxiv 2412.14097 v1 pith:37KAMHYT submitted 2024-12-18 cs.LG cs.AIcs.CV

Jihye Choi , Jayaram Raghuram , Yixuan Li , Somesh Jha This is my paper · ORCID

classification cs.LGcs.AIcs.CV

keywords conceptbottleneckmodelstest-timeadaptationdistributionshiftfoundationinterpretabilitypseudo-labelingalignmentresidual

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Concept bottleneck models make foundation-model predictions interpretable by routing features through a small set of high-level concept scores before classification, but the paper shows that these concept-level predictions are not automatically robust to distribution shifts at deployment. It proposes CONDA, a test-time adaptation method that updates the concept bank and the prediction layer using only unlabeled batches from the target domain, with no access to the source training data. Across CIFAR-C, Waterbirds, Metashift, and Camelyon17 benchmarks, the method reports post-deployment accuracy gains of up to 28%, bringing concept-based predictions to parity with non-interpretable zero-shot and linear-probing baselines. The overall claim is that interpretability and deployment robustness are not in tension if the concept bottleneck itself is adapted rather than frozen.

What carries the argument

The load-bearing object is the adaptable concept bottleneck itself, treated as a matrix of unit-norm concept vectors followed by a linear head. CSA re-solves the concept matrix so that, under pseudo-labels, Mahalanobis distances from target concept scores to source class-conditional Gaussians shrink within each class and grow across classes. LPA then minimizes the cross-entropy of the main branch against pseudo-labels with an elastic-net penalty that keeps the head interpretable. RCB appends a second branch of fresh concept vectors and a linear head, trained jointly with a cosine-similarity penalty for diversity and a coherency term that ties each new concept to the target patches that activate it most; because the branches share the backbone, the combined predictor is exactly the main branch plus the residual branch, equivalent to augmenting the concept matrix with new rows and the weight matrix with new columns. These three steps isolate the three failure modes so that each adaptation stage can be ablated and attributed.

What would settle it

Evaluate CONDA on a target batch whose pseudo-labels are deliberately replaced with uniform random labels; if the adapted batch accuracy does not degrade relative to the unadapted CBM, the paper's stated dependence on pseudo-label quality would be contradicted, whereas the paper's own severity-5 Metashift result already exhibits the predicted degradation.

Watch

Extended reading notes

Core claim

The central claim is that a deployed concept bottleneck need not be static: each failure mode caused by distribution shift corresponds to a component that can be repaired online. The paper formalizes two shift types, low-level shifts that change inputs but not concept semantics and concept-level shifts that change high-level semantics, and names three failure modes: a non-robust concept mapping, a classifier that no longer maps concepts to labels consistently, and a concept set that is incomplete for the target domain. CONDA addresses them in order: concept-score alignment (CSA) adapts the concept vectors so target concept scores match the source class-conditional distributions; linear-probing adaptation (LPA) re-tunes the label predictor using pseudo-labels; and a residual concept bottleneck (RCB) adds new concepts to cover what the original bank missed. With a frozen foundation-model backbone and only unlabeled test batches, the paper reports target-domain accuracy gains up to 28% and worst-group gains that often exceed the non-interpretable baselines, while the adapted concept weights visibly shift toward target-relevant semantics, such as land concepts contributing to waterbird predictions and shelf concepts to both Metashift classes.

Load-bearing premise

The load-bearing premise is that the pseudo-labels produced by the zero-shot and linear-probing ensemble are accurate enough to supervise all three adaptation objectives; Appendix F's Table 8 shows that on severity-5 Gaussian noise over Metashift, where both baselines collapse, CONDA's adapted accuracy falls below the unadapted CBM, so the claim is conditional on a backbone that is at least partly reliable on the target domain.

Editorial extensions

If this is right

If the paper's claim is correct, a deployed concept bottleneck can recover from distribution shifts without any labeled target data or access to the source dataset, matching or exceeding non-interpretable baselines in worst-group accuracy.
The failure-mode decomposition implies that practitioners can choose the component they need: CSA for low-level input corruptions, LPA and RCB for concept-level or semantic shifts, with little cost from the unused components.
The residual branch demonstrates that the original concept bank may be incomplete, and that automatically discovered residual concepts can restore both accuracy and interpretability, as seen with bird-related concepts on Waterbirds.
Because adaptation is online and batchwise, the method can be deployed in streaming settings where target data arrives continuously and the model must not be reset between batches.
The paper's component analyses support a broader design rule: adapting the bottleneck layer matters most for input-level shifts, while adapting the prediction layer matters most for output-level shifts.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper, the three-stage recipe of align, re-fit the head, and extend the concept set is a template for other interpretable architectures: any bottleneck that exposes a linear concept projection could be repaired by the same sequence, even if the concepts are built differently.
The residual branch's ability to discover target-specific concepts suggests a testable extension: feed the learned residual concept vectors back into the original concept-annotation pipeline, then measure whether adding them to a static concept bank recovers most of CONDA's gain without online adaptation.
Because CSA only needs class-conditional Gaussian statistics of source concept scores, a deployment team could publish those statistics alongside the model and let each test site run CONDA without transferring source data; the paper does not explore the privacy or bandwidth aspects of this protocol.
The dependence on pseudo-label quality implies an upper bound on the method: with perfect pseudo-labels, the paper's own ablation shows near-perfect accuracy on Metashift and large gains on CIFAR100-C, so future improvements in test-time pseudo-labeling should transfer almost directly into CONDA's accuracy.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First take: this is a genuine first: TTA for CBMs with a frozen FM backbone, and the three-component recipe (CSA, LPA, RCB) maps cleanly onto the failure modes they define. The reported gains on Waterbirds and Metashift for the Yuksekgonul and Oikarinen bottlenecks are substantial — e.g., Waterbirds AVG from 32 to 60 for PCBM — and the paper does a lot of things right: honest ablations, hyperparameter sensitivity, pseudo-labeling variants, a perfect-label upper bound, and a stated limitation section that actually names where the method fails. I appreciate that they report negative results in Appendix F/Table 8 instead of hiding them.

The soft spot is exactly where the stress-test note lands. All three adaptation stages are supervised by pseudo-labels from a ZS/LP ensemble, and those are the same feature-based predictors used as baselines. So the "on par with non-interpretable classification" claim is partly inherited: if ZS/LP are good, CONDA can ride them; if they are bad, CONDA can be worse than doing nothing. The severity-5 Metashift results in Table 8 show CONDA underperforming the unadapted CBM, and the main text sidesteps this by evaluating CIFAR-C at severity 2 instead of the standard severity 5. That's a meaningful gap between the abstract's "up to 28%" claim and what is actually established. I also note that Table 1 shows CONDA hurting accuracy for the Yeh et al. bottleneck on CIFAR10, CIFAR100, and Camelyon17; the paper mentions this in Appendix F, but it should be in the main text.

Minor but worth saying: no code is provided, which makes the many hyperparameters (five per dataset plus r, ngrad, k) hard to trust as anything other than tuned to the reported benchmarks. The source-domain class statistics assumption is stated clearly, so that's fine as a limitation, not a flaw.

Overall: the central idea is sound, the engineering is careful, and the negative evidence is honestly reported. What's missing is a narrowed claim: CONDA helps when the backbone's ZS/LP pseudo-labels are reliable, and the evaluation should include severity-5 and the failure cases in the main text. I would send this to review — a serious referee could push the authors to add code, TTA baselines (e.g., TENT on the feature classifier), and a clearer statement of the pseudo-label condition. I'd cite it if working on interpretable robustness; it's a useful reference point even with the caveats.

Referee Report

3 major / 6 minor

Summary. The paper studies test-time adaptation of concept bottleneck models (CBMs) built on frozen foundation models under distribution shifts. It formalizes three failure modes—non-robust concept bottleneck under low-level shift, non-robust classifier under concept-level shift, and incomplete concept set—and proposes CONDA, a three-stage adaptation method: concept-score alignment (CSA, Eq. 9), linear-probing adaptation (LPA, Eq. 11), and a residual concept bottleneck (RCB, Eq. 14). All stages are supervised by pseudo-labels from an ensemble of zero-shot and linear-probing predictors. Experiments on CIFAR10-C/CIFAR100-C, Waterbirds, Metashift, and Camelyon17, across three CBM construction methods, report accuracy gains in several settings, with the largest gains on concept-level shifts. The paper includes ablations, interpretability analysis, complexity analysis, and a candid limitations appendix.

Significance. CONDA is, to my knowledge, the first test-time adaptation framework specifically for concept bottleneck models with a foundation-model backbone. If the results held broadly, the paper would make a real contribution by showing that interpretable concept-based pipelines can be adapted online without labels. Strengths include the clean decomposition of failure modes, the residual concept bank idea, the use of multiple CBM construction methods, and the honest reporting of negative results (Appendix F, Table 8) and pseudo-label sensitivity (Table 5). However, the main claims are partly undercut by the method's dependence on the same non-interpretable predictors it aims to match, and by the non-standard severity-2 CIFAR-C evaluation; these issues are fixable but require re-framing or additional experiments.

major comments (3)

[Section 3 (Pseudo-labeling), Eqs. (9), (11), (14); Tables 5 and 8] The adaptation objectives in all three stages are supervised by pseudo-labels taken from the zero-shot (ZS) and linear-probing (LP) predictors, which are exactly the non-interpretable baselines used as comparison targets in Table 1. As a result, the statement that CONDA 'aligns the CBM performance with that of non-interpretable classification' (abstract) is to a nontrivial degree a propagation of ZS/LP accuracy rather than an independent property of the concept-bottleneck adaptation. Table 5 shows the strong dependence on pseudo-label quality (e.g., CIFAR100-C AVG increases from 53.88 with ZS/LP pseudo-labels to 97.31 with perfect pseudo-labels), and Table 8 documents a regime where poor pseudo-labels cause CONDA to underperform the unadapted CBM. The manuscript discloses this in Appendix F, but the main text and abstract should either be reworded to state the conditional nature of the claim or supplemented with an ablation that uses a pseudo-labeling scheme not derived from the comparison baselines (e.g., entropy minimization or rotation-based self-supervision).
[Appendix C.1; CIFAR-C evaluation; Table 8] The low-level shift benchmark is evaluated at corruption severity 2 rather than the standard severity 5, with the rationale that severity 5 degrades the backbone's pseudo-label oracle. This choice is consequential because low-level shift is one of the three failure modes motivating CONDA, and CIFAR-C is the only low-level-shift dataset. The negative result in Table 8 (Metashift with severity-5 Gaussian noise) shows exactly the boundary regime where the method fails. To establish the headline 'boosts post-deployment accuracy by up to 28%' for low-level shifts, please report CIFAR-C at multiple severities (at least severity 5) or, if that is infeasible due to pseudo-label failure, state in the abstract and Section 4.2 that the low-level-shift results are limited to mild corruptions.
[Table 1; Appendix F] CONDA reduces accuracy for the Yeh et al. (2020) bottleneck on three of five datasets (CIFAR10-C AVG 89.76 to 85.14, CIFAR100-C AVG 72.33 to 70.82, Camelyon17 AVG 95.01 to 92.54). The caption's claim that CONDA 'significantly improves' target accuracy is therefore not true for a substantial subset of the reported configurations. Appendix F acknowledges this, but the main text should explicitly delimit the method's scope (e.g., to concept banks with interpretable/annotatable concepts) and, ideally, analyze why the optimization-based concepts of Yeh et al. behave differently. Without this qualification, the central claim of broad effectiveness is overgeneralized.

minor comments (6)

[Section 3.1] The definition of the Mahalanobis distance has unbalanced parentheses: Dmah(xt ; µy, Σy) should be (vC(xt) − µy)ᵀ Σ_y⁻¹ (vC(xt) − µy).
[Section 4.3 and Figure 3] The main text says Figure 3 corresponds to adapting the CBM of Yeh et al. (2020), while the figure caption says it is for the CBM method of Yuksekgonul et al. (2023); please reconcile this discrepancy.
[Section 3 and Algorithm 1] The pseudo-labeling ensemble is described informally as taking 'the class predicted with higher confidence across both predictors'; a precise definition (e.g., maximum softmax probability or margin) would improve reproducibility.
[Section 3.1] The method assumes access to source-domain class-conditional Gaussian statistics {(µy, Σy)}; since these are derived from labeled source data, the claim of operating 'without access to the source dataset' should be qualified to say that precomputed source statistics are required.
[Table 4] The hyper-parameters are dataset-specific; please state how they were selected (e.g., source-domain validation, grid search) rather than reporting only the final values.
[Abstract and Section 4.2] The phrase 'boosts post-deployment accuracy by up to 28%' should be tied to a specific table entry (e.g., Waterbirds AVG for Yuksekgonul et al., 32.03 to 60.69) rather than left as a global claim, since the method does not improve all reported configurations.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CONDA's pseudo-label supervision is a limitation, not a circularity, and its self-citations are not load-bearing.

full rationale

The paper's claimed derivation—adapting the concept bank, label predictor, and residual concept bottleneck using unlabeled target data—does not reduce to its inputs. The adaptation objectives (Eqs. 6–14) use pseudo-labels from an ensemble of zero-shot and linear-probing predictors, but the reported post-deployment accuracy is measured against ground-truth labels (Table 1 and Appendix C), so the headline 'up to 28%' gain is an external empirical outcome rather than a definitional identity. The dependence on pseudo-label quality is real and is explicitly acknowledged in Appendix F (Table 8), where severe corruption degrades performance below the unadapted CBM; this is an honest robustness limitation, not a circular argument. The failure-mode taxonomy in Section 2.3 is descriptive and not used to derive Eqs. 9, 11, or 14 by formal entailment. The only self-citation (Choi et al. 2023, Appendix C.2) is used as an example of unsupervised concept learning and is not load-bearing; no uniqueness theorem or ansatz is imported from the authors' prior work. The main results are benchmark comparisons against external CBM baselines and ZS/LP feature-based baselines, so the central claims stand independently.

Assumptions & free parameters 8 free parameters · 5 assumptions · 1 invented entities

The method is an engineered adaptation routine rather than a derivation. Its central claim rests on source-domain Gaussian statistics for concept scores, reliable pseudo-labels, the expressiveness of a small residual concept bank, and the absence of catastrophic drift during online batch-by-batch updates. All of these are stated or implicitly assumed in Sections 3 and 4, and none are proven.

free parameters (8)

lambda_frob per dataset = CIFAR10/CIFAR100: 0.1; Waterbirds: 2.5; Metashift: 5.0; Camelyon17: 0.5
Controls how far adapted concept vectors may move from the source concept bank (Eq. 9). Ablation in Figs. 5a and 5b shows strong sensitivity: low values destabilize and high values block adaptation.
lambda_sparse per dataset = 1.0 or 2.0
Elastic net penalty on the linear probing layer in Eq. 11; tuned per dataset in Table 4.
lambda_sim per dataset = 0.1 to 1.0
Cosine similarity penalty on residual concepts in Eq. 12; chosen per dataset.
lambda_coh per dataset = 0.1 to 2.0
Coherency regularizer in Eq. 13; chosen per dataset.
number of residual concepts r = 5
Selected via ablation in Fig. 5d; performance saturates after 5 and begins to drop in interpretability.
adaptation gradient steps ngrad = 20 except Metashift 50
Number of gradient steps for each of CSA, LPA, and RCB; selected per dataset.
k for coherency top-k = batch_size / (2 * num_classes)
Heuristic for Eq. 13 nearest neighbors; no justification beyond 'works well in practice'.
corruption severity for CIFAR-C = 2
Chosen instead of standard severity 5 because severity 5 degrades the foundation model and pseudo-label quality (Appendix C.1, Table 8). This evaluation-regime choice affects the headline low-level-shift results.

assumptions (5)

domain assumption Source class-conditional concept scores follow multivariate Gaussians with known means and covariances.
Section 3.1 models P(v_Cs(x_s)|y_s) as N(mu_y, Sigma_y) and assumes these statistics are available even though the source dataset is not. Used to define Mahalanobis distances in Eqs. 6 and 7.
domain assumption Pseudo-labels from the zero-shot and linear-probing ensemble are accurate enough to supervise adaptation.
Algorithm 1 uses pseudo-labels in all three objectives; Appendix F reports negative results when the ensemble degrades, so this assumption is load-bearing.
ad hoc to paper Online coordinate-wise gradient updates on unlabeled batches improve true target accuracy.
No convergence guarantee or error bound is provided; Algorithm 1 lines 5-28 rely on this empirical assumption.
ad hoc to paper The source concept set plus r random-initialized residual concepts is expressive enough for the target task.
Eq. 14 jointly adapts residual concepts and the residual linear layer; the paper offers no completeness condition for choosing r, only an ablation in Fig. 5d.
domain assumption The transformation model in Section 2.2 is a faithful description of the tested shifts.
Section 2.2 formalizes shifts via invertible transformations t_i, but Waterbirds and Metashift change background correlations, which are not observed as pixel-space transform mixtures in the data.

invented entities (1)

Residual concept vectors eC
purpose: Add r new concept directions to represent target-specific information missing from the source concept bank.
Randomly initialized and optimized on unlabeled target batches (Eq. 14); post hoc CLIP-DISSECT annotations are descriptive, not falsifiable external predictions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts." pith.science (2026). https://pith.science/paper/37KAMHYT

@misc{pith2026241214097,
  author       = {Pith},
  title        = {Pith review of: Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/37KAMHYT}},
  note         = {Machine review of arXiv:2412.14097}
}

read the original abstract

Advancements in foundation models (FMs) have led to a paradigm shift in machine learning. The rich, expressive feature representations from these pre-trained, large-scale FMs are leveraged for multiple downstream tasks, usually via lightweight fine-tuning of a shallow fully-connected network following the representation. However, the non-interpretable, black-box nature of this prediction pipeline can be a challenge, especially in critical domains such as healthcare, finance, and security. In this paper, we explore the potential of Concept Bottleneck Models (CBMs) for transforming complex, non-interpretable foundation models into interpretable decision-making pipelines using high-level concept vectors. Specifically, we focus on the test-time deployment of such an interpretable CBM pipeline "in the wild", where the input distribution often shifts from the original training distribution. We first identify the potential failure modes of such a pipeline under different types of distribution shifts. Then we propose an adaptive concept bottleneck framework to address these failure modes, that dynamically adapts the concept-vector bank and the prediction layer based solely on unlabeled data from the target domain, without access to the source (training) dataset. Empirical evaluations with various real-world distribution shifts show that our adaptation method produces concept-based interpretations better aligned with the test data and boosts post-deployment accuracy by up to 28%, aligning the CBM performance with that of non-interpretable classification.

Figures

Figures reproduced from arXiv: 2412.14097 by the authors.

**Figure 1.** Concept-based predictions are not inherently more robust to distribution shifts than featurebased predictions, necessitating dynamic adaptation after deployment. We observe significant drops in the averaged group accuracy (AVG) and worst-group accuracy (WG) from the source to the target (test) domain under two types of distribution shifts: (1) low-level shift (left), where inputs are perturbed without modifying cla… view at source ↗

**Figure 2.** Overview of CONDA, our proposed adaptation framework. The foundation model and CBM pipeline trained on the source domain is shown at the top, while the adapted CBM, consisting of a main branch and residual branch, is shown at the bottom. The components of CBM that are adapted during each stage of the proposed method (i.e., CSA, LPA, and RCB) are shown in different colors. adaptation, where the foundation model ϕ(x) … view at source ↗

**Figure 3.** Effectiveness of individual components of CONDA for the CBM method of Yuksekgonul et al. (2023). We report the relative AVG and WG, which is the (acc. after adaptation) − (acc. before adaptation). classifier. The concept bottleneck layer corresponds to the first layer, which is particularly important for addressing input-level shifts (following their definition), while the linear probing layer corresponds to the sec… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: CONDA adapts the concept weights to be tailored to the target data. We visualize the linear probing layer weights (width of each mapping) before vs. after applying CONDA to the PCBM baseline (Yuksekgonul et al., 2023) on the Waterbirds dataset. We only show the mapping…

**Figure 5.** Figure 5: Ablations on the hyper-parameters in CONDA. We ablate on the individual hyper-parameters in CONDA for each type of distribution shift: (1) CIFAR10-C (impulse noise) simulating low-level shift, and (2) Waterbirds simulating concept-level shift. D Ablation Experiments Ab…

**Figure 6.** Figure 6: RCB intervenes and compensates for mis-classifications of PCBM + CSA + LPA, particularly in [PITH_FULL_IMAGE:figures/full_fig_p029_6.png]

**Figure 7.** Figure 7: Accuracy vs. Interpretability of the Residual Concept Bottleneck. with PCBM (CLIP) deployed on the Waterbirds dataset (target domain) [PITH_FULL_IMAGE:figures/full_fig_p030_7.png]

**Figure 8.** Figure 8: CONDA adapts the concept weights to be tailored to the target data. We visualize the linear probing layer weights (width of each mapping) before vs after applying CONDA to PCBM (Yuksekgonul et al., 2023) on the Metashift dataset. We only show the mappings with positive…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

53 extracted references · 33 canonical work pages

[1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
[2]

Meaningfully debugging model mistakes using conceptual counterfactual explanations

Abubakar Abid, Mert Yuksekgonul, and James Zou. Meaningfully debugging model mistakes using conceptual counterfactual explanations. In International Conference on Machine Learning, pp.\ 66--88. PMLR , 2022

work page 2022
[3]

Debugging tests for model explanations

Julius Adebayo, Michael Muelly, Ilaria Liccardi, and Been Kim. Debugging tests for model explanations. In Proceedings of the 34th International Conference on Neural Information Processing Systems, pp.\ 700--712, 2020

work page 2020
[4]

Zero-shot robustification of zero-shot models

Dyah Adila, Changho Shin, Linrong Cai, and Frederic Sala. Zero-shot robustification of zero-shot models. In The Twelfth International Conference on Learning Representations, 2024

work page 2024
[5]

Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing

Ehab A AlBadawy, Ashirbani Saha, and Maciej A Mazurowski. Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing. Medical physics, 45 0 (3): 0 1150--1158, 2018

work page 2018
[6]

Network dissection: Quantifying interpretability of deep visual representations

David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 6541--6549, 2017

work page 2017
[7]

On the opportunities and risks of foundation models

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. CoRR, abs/2108.07258, 2021. URL https://arxiv.org/abs/2108.07258

arXiv 2021
[8]

Contrastive test-time adaptation

Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition ( CVPR ) , pp.\ 295--305. IEEE , 2022. doi:10.1109/CVPR52688.2022.00039. URL https://doi.org/10.1109/CVPR52688.2022.00039

arXiv 2022

Show all 53 references

[9]

Concept-based explanations for out-of-distribution detectors

Jihye Choi, Jayaram Raghuram, Ryan Feng, Jiefeng Chen, Somesh Jha, and Atul Prakash. Concept-based explanations for out-of-distribution detectors. In International Conference on Machine Learning, pp.\ 5817--5837. PMLR , 2023

2023
[10]

Debiasing vision-language models via biased prompts

Ching-Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, and Stefanie Jegelka. Debiasing vision-language models via biased prompts. arXiv preprint arXiv:2302.00070, 2023

2023 arXiv
[11]

Sedigheh Eslami, Christoph Meinel, and Gerard De Melo. PubMedCLIP : How much does CLIP benefit visual question answering in the medical domain? In Findings of the Association for Computational Linguistics: EACL 2023 , pp.\ 1151--1163, 2023

2023
[12]

Imagebind: One embedding space to bind them all

Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In CVPR, 2023

2023
[13]

Addressing leakage in concept bottleneck models

Marton Havasi, Sonali Parbhoo, and Finale Doshi-Velez. Addressing leakage in concept bottleneck models. Advances in Neural Information Processing Systems, 35: 0 23386--23397, 2022

2022
[14]

Benchmarking neural network robustness to common corruptions and perturbations

Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. Proceedings of the International Conference on Learning Representations, 2019

2019
[15]

Scaling up visual and vision-language representation learning with noisy text supervision

Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pp.\ 4904--4916....

2021
[16]

CAFA : Class-aware feature alignment for test-time adaptation

Sanghun Jung, Jungsoo Lee, Nanhee Kim, Amirreza Shaban, Byron Boots, and Jaegul Choo. CAFA : Class-aware feature alignment for test-time adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pp.\ 19060--19071, 2023

2023
[17]

Interpretability beyond feature attribution: Quantitative testing with concept activation vectors ( TCAV )

Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors ( TCAV ). In International conference on machine learning, pp.\ 2668--2677. PMLR , 2018

2018
[18]

Concept bottleneck models

Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine ...

2020
[19]

Fine-tuning can distort pretrained features and underperform out-of-distribution

Ananya Kumar, Aditi Raghunathan, Robbie Jones, Tengyu Ma, and Percy Liang. Fine-tuning can distort pretrained features and underperform out-of-distribution. In International Conference on Learning Representations, 2022

2022
[20]

Measure theory, Probability, and Stochastic Processes

Jean-Fran c ois Le Gall. Measure theory, Probability, and Stochastic Processes. Springer, 2022

2022
[21]

Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks

Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML , volume 3, pp.\ 896. Atlanta, 2013

2013
[22]

Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn

Yoonho Lee, Annie S. Chen, Fahim Tajwar, Ananya Kumar, Huaxiu Yao, Percy Liang, and Chelsea Finn. Surgical fine-tuning improves adaptation to distribution shifts. In The Eleventh International Conference on Learning Representations, ICLR . OpenReview.net, 2023. URL https://ope...

2023
[23]

A comprehensive survey on test-time adaptation under distribution shifts

Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribution shifts. CoRR, abs/2303.15361, 2023. doi:10.48550/ARXIV.2303.15361. URL https://doi.org/10.48550/arXiv.2303.15361

2023 doi arXiv
[24]

Metashift: A dataset of datasets for evaluating contextual distribution shifts and training conflicts

Weixin Liang and James Zou. Metashift: A dataset of datasets for evaluating contextual distribution shifts and training conflicts. In International Conference on Learning Representations, 2021

2021
[25]

How to exploit hyperspherical embeddings for out-of-distribution detection? In The Eleventh International Conference on Learning Representations ( ICLR )

Yifei Ming, Yiyou Sun, Ousmane Dia, and Yixuan Li. How to exploit hyperspherical embeddings for out-of-distribution detection? In The Eleventh International Conference on Learning Representations ( ICLR ) . OpenReview.net, 2023. URL https://openreview.net/pdf?id=aEFaE0W5pAd

2023
[26]

Text-to-concept (and back) via cross-model alignment

Mazda Moayeri, Keivan Rezaei, Maziar Sanjabi, and Soheil Feizi. Text-to-concept (and back) via cross-model alignment. In International Conference on Machine Learning, pp.\ 25037--25060. PMLR , 2023

2023
[27]

Evaluating prediction-time batch normalization for robustness under covariate shift

Zachary Nado, Shreyas Padhy, D Sculley, Alexander D'Amour, Balaji Lakshminarayanan, and Jasper Snoek. Evaluating prediction-time batch normalization for robustness under covariate shift. arXiv preprint arXiv:2006.10963, 2020

2006 arXiv
[28]

CLIP-Dissect : Automatic description of neuron representations in deep vision networks

Tuomas Oikarinen and Tsui-Wei Weng. CLIP-Dissect : Automatic description of neuron representations in deep vision networks. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=iPWiwWHc1V

2023
[29]

Nguyen, and Tsui-Wei Weng

Tuomas Oikarinen, Subhro Das, Lam M. Nguyen, and Tsui-Wei Weng. Label-free concept bottleneck models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=FlCg47MNvBA

2023
[30]

Dataset shift in machine learning

Joaquin Qui \ n onero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. Dataset shift in machine learning. Mit Press, 2022

2022
[31]

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp...

2021
[32]

High-resolution image synthesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp.\ 10684--10695, 2022

2022
[33]

Distributionally robust neural networks

Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks. In International Conference on Learning Representations, 2019

2019
[34]

Robust CLIP : Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models

Christian Schlarmann, Naman Deep Singh, Francesco Croce, and Matthias Hein. Robust CLIP : Unsupervised adversarial fine-tuning of vision embeddings for robust large vision-language models. In Forty-first International Conference on Machine Learning ( ICML ) . OpenReview.net, 2...

2024
[35]

Incremental residual concept bottleneck models

Chenming Shang, Shiji Zhou, Hengyuan Zhang, Xinzhe Ni, Yujiu Yang, and Yuwang Wang. Incremental residual concept bottleneck models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 11030--11040, 2024

2024
[36]

FixMatch : Simplifying semi-supervised learning with consistency and confidence

Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun - Liang Li. FixMatch : Simplifying semi-supervised learning with consistency and confidence. In Advances in Neural Information Processing Systems 3...

2020
[37]

Conceptnet 5.5: An open multilingual graph of general knowledge

Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the AAAI conference on artificial intelligence, volume 31, 2017

2017
[38]

Test-time training with self-supervision for generalization under distribution shifts

Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei Efros, and Moritz Hardt. Test-time training with self-supervision for generalization under distribution shifts. In International conference on machine learning, pp.\ 9229--9248. PMLR, 2020

2020
[39]

Learning bottleneck concepts in image classification

Bowen Wang, Liangzhi Li, Yuta Nakashima, and Hajime Nagahara. Learning bottleneck concepts in image classification. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pp.\ 10962--10971, 2023

2023
[40]

Tent: Fully test-time adaptation by entropy minimization

Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=uXl3bZLkr3c

2021
[41]

Medclip: Contrastive learning from unpaired medical images and text

Zifeng Wang, Zhenbang Wu, Dinesh Agarwal, and Jimeng Sun. Medclip: Contrastive learning from unpaired medical images and text. arXiv preprint arXiv:2210.10163, 2022

2022 arXiv
[42]

Bloomberggpt: A large language model for finance

Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. Bloomberggpt: A large language model for finance. arXiv preprint arXiv:2303.17564, 2023 a

2023 arXiv
[43]

Discover and cure: Concept-aware mitigation of spurious correlation

Shirley Wu, Mert Yuksekgonul, Linjun Zhang, and James Zou. Discover and cure: Concept-aware mitigation of spurious correlation. arXiv preprint arXiv:2305.00650, 2023 b

2023 arXiv
[44]

Towards a theoretical framework of out-of-distribution generalization

Haotian Ye, Chuanlong Xie, Tianle Cai, Ruichen Li, Zhenguo Li, and Liwei Wang. Towards a theoretical framework of out-of-distribution generalization. Advances in Neural Information Processing Systems, 34: 0 23519--23531, 2021

2021
[45]

On completeness-aware concept-based explanations in deep neural networks

Chih-Kuan Yeh, Been Kim, Sercan Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. On completeness-aware concept-based explanations in deep neural networks. Advances in neural information processing systems, 33: 0 20554--20565, 2020

2020
[46]

Bdd100k: A diverse driving dataset for heterogeneous multitask learning

Fisher Yu, Haofeng Chen, Xin Wang, Wenqi Xian, Yingying Chen, Fangchen Liu, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving dataset for heterogeneous multitask learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp....

2020
[47]

Post-hoc concept bottleneck models

Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=nA5AZ8CEyow

2023
[48]

MEMO : Test time robustness via adaptation and augmentation

Marvin Zhang, Sergey Levine, and Chelsea Finn. MEMO : Test time robustness via adaptation and augmentation. Advances in neural information processing systems, 35: 0 38629--38642, 2022

2022
[49]

Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs

Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al. Biomedclip: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv:2303.00915, 2023

2023 arXiv
[50]

Regularization and variable selection via the elastic net

Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67 0 (2): 0 301--320, 2005

2005
[51]

@esa (Ref

\@ifxundefined[1] #1\@undefined \@firstoftwo \@secondoftwo \@ifnum[1] #1 \@firstoftwo \@secondoftwo \@ifx[1] #1 \@firstoftwo \@secondoftwo [2] @ #1 \@temptokena #2 #1 @ \@temptokena \@ifclassloaded agu2001 natbib The agu2001 class already includes natbib coding, so you should ...
[52]

\@lbibitem[] @bibitem@first@sw\@secondoftwo \@lbibitem[#1]#2 \@extra@b@citeb \@ifundefined br@#2\@extra@b@citeb \@namedef br@#2 \@nameuse br@#2\@extra@b@citeb \@ifundefined b@#2\@extra@b@citeb @num @parse #2 @tmp #1 NAT@b@open@#2 NAT@b@shut@#2 \@ifnum @merge>\@ne @bibitem@firs...
[53]

@open @close @open @close and [1] URL: #1 \@ifundefined chapter * \@mkboth \@ifxundefined @sectionbib * \@mkboth * \@mkboth\@gobbletwo \@ifclassloaded amsart * \@ifclassloaded amsbook * \@ifxundefined @heading @heading NAT@ctr thebibliography [1] @ \@biblabel @NAT@ctr \@bibset...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

[2] [2]

Meaningfully debugging model mistakes using conceptual counterfactual explanations

Abubakar Abid, Mert Yuksekgonul, and James Zou. Meaningfully debugging model mistakes using conceptual counterfactual explanations. In International Conference on Machine Learning, pp.\ 66--88. PMLR , 2022

work page 2022

[3] [3]

Debugging tests for model explanations

Julius Adebayo, Michael Muelly, Ilaria Liccardi, and Been Kim. Debugging tests for model explanations. In Proceedings of the 34th International Conference on Neural Information Processing Systems, pp.\ 700--712, 2020

work page 2020

[4] [4]

Zero-shot robustification of zero-shot models

Dyah Adila, Changho Shin, Linrong Cai, and Frederic Sala. Zero-shot robustification of zero-shot models. In The Twelfth International Conference on Learning Representations, 2024

work page 2024

[5] [5]

Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing

Ehab A AlBadawy, Ashirbani Saha, and Maciej A Mazurowski. Deep learning for segmentation of brain tumors: Impact of cross-institutional training and testing. Medical physics, 45 0 (3): 0 1150--1158, 2018

work page 2018

[6] [6]

Network dissection: Quantifying interpretability of deep visual representations

David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp.\ 6541--6549, 2017

work page 2017

[7] [7]

On the opportunities and risks of foundation models

Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. CoRR, abs/2108.07258, 2021. URL https://arxiv.org/abs/2108.07258

arXiv 2021

[8] [8]

Contrastive test-time adaptation

Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition ( CVPR ) , pp.\ 295--305. IEEE , 2022. doi:10.1109/CVPR52688.2022.00039. URL https://doi.org/10.1109/CVPR52688.2022.00039

arXiv 2022