Pith. sign in

REVIEW 3 major objections 6 minor 33 references

PRISM: Reducing Spurious Implicit Biases in Vision-Language Models with LLM-Guided Embedding Projection

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PRISM claims a text-trained projection removes CLIP's spurious biases, lifting worst-group accuracy from 36.4% to 84.2% (Waterbirds) and 72.8% to 84.0% (CelebA).

desk verdict A promising data-free debiasing idea undercut by a central equation that, as written, does not depend on the learned projection; likely a typo, but must be fixed before the claims can be assessed. read the letter →

arxiv 2507.08979 v1 pith:VJMEMLTQ submitted 2025-07-11 cs.CV cs.LG

Mahdiyar Molahasani , Azadeh Motamedi , Michael Greenspan , Il-Min Kim , Ali Etemad This is my paper
classification cs.CVcs.LG
keywords spuriouscorrelationsvision-languagemodelsCLIPLLM-guideddebiasingembeddingprojectioncontrastivelossworst-groupaccuracyzero-shotclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PRISM claims that CLIP's tendency to lean on spurious correlations—a bird's background, a face's incidental context—can be largely neutralized without retraining the model, collecting new images, or pre-specifying which biases to hunt for. The recipe is to prompt a large language model with plain class names, get back scene descriptions that pair each class with its stereotyped surroundings, encode those descriptions with CLIP's text encoder, and learn a single linear projection of the shared embedding space using a contrastive loss that pulls same-class descriptions together and pushes different-class descriptions apart. The same projection, applied to image and text embeddings at test time, is reported to raise worst-group accuracy (the accuracy of the model's most poorly predicted subgroup) from 36.4% to 84.2% on Waterbirds and from 72.8% to 84.0% on CelebA, while overall accuracy stays roughly level. If the claim holds, it means debiasing a vision-language model can be done in a data-free, task-agnostic way: any new task needs only an LLM call and one short projection-training pass.

What carries the argument

The central object is the learned linear projection $P$ of CLIP's shared embedding space, trained on text only. It is driven by two components: an LLM that, given simple class prompts such as 'A photo of a duck', returns likely spurious attributes (e.g., a lake or a jungle background) and then generates controlled scene descriptions per group; and the Latent space Debiasing loss (LD, Eq. 7), a contrastive-style objective whose intra-class, inter-attribute term minimizes $1 - \langle \phi_T(T_{a,y}), \phi_T(T_{a',y}) \rangle$ for same-class pairs with different attributes, and whose inter-class, intra-attribute term penalizes similarity beyond $m$ for different classes sharing an attribute. Optimizing $P$ on these text embeddings alone is claimed to strip the spurious subspace out of the shared representation, so applying $P$ to both image and text embeddings at test time debiases zero-shot classification without fine-tuning CLIP or using external data. A cheaper variant, PRISM-mini, skips optimization and sets $P$ by orthogonally projecting against the embedding matrix of the LLM-suggested attributes (Eq. 10).

What would settle it

Take a dataset whose known spurious correlation has no natural textual counterpart—for example, classify two bird species where the confound is image resolution or lighting angle, attributes an LLM prompted for scene descriptions will not name—and run PRISM exactly as described. If worst-group accuracy does not improve over the zero-shot baseline, the text-to-image transfer hypothesis fails. A cheaper check on Waterbirds: strip the words for the spurious attributes (land, water, and their synonyms) out of the LLM-generated scene descriptions and retrain the projection; if the worst-group gain persists, the LD loss is removing something other than the stated spurious signal.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that spurious biases in CLIP occupy a subspace of the joint embedding space that can be found and excised using text alone. Because CLIP aligns images and text in one space, the paper argues that the same spurious correlations visible in images leave detectable traces in text embeddings: a picture of a duck over water and a picture of a duck in a jungle are encoded closer to 'water' or 'jungle' than to each other. PRISM harvests those traces by prompting an LLM with class prompts to generate scene descriptions for every (class, spurious-attribute) group, then learns a projection $P$ that minimizes the Latent space Debiasing loss (Eq. 7), which pushes embeddings of the same class across different attributes together and pushes embeddings of different classes sharing an attribute apart by a margin $m$. At inference the same $P$ is applied to both image and text embeddings (Eq. 8), so the alignment between modalities is preserved while the spurious direction is suppressed. The reported outcome is a large gain in worst-group accuracy on both benchmarks with essentially unchanged overall accuracy, and a representation map in which the four (class, background) groups form visibly separated clusters.

Load-bearing premise

The load-bearing premise, which the paper states as a hypothesis in Section 3.1 and verifies only in the 'Bias text' experiment (Table 2), is that spurious correlations in images leave matching traces in the text domain, so a debiasing projection learned from text descriptions alone will also remove the bias from image embeddings.

Editorial extensions

If this is right

  • Debiasing becomes a data-free, task-agnostic step: for any new set of class names, the user needs only an LLM call to generate scene descriptions and a single quick projection-training pass, with no images, no task labels, and no fixed list of bias categories.
  • Because only the projector $P$ is learned and CLIP's weights stay frozen, the method preserves CLIP's general zero-shot ability; the paper reports overall accuracy rising 4.3% on Waterbirds and dipping 0.7% on CelebA while worst-group accuracy jumps 47.8 and 11.2 points respectively.
  • PRISM beats all compared data-free debiasing methods on both benchmarks and outperforms the image-using baselines on Waterbirds worst-group accuracy, suggesting text-only guidance can substitute for image-side supervision when removing spurious correlations.
  • The main controls are the margin $m$ in the LD loss and the number of LLM-generated scene descriptions; the paper finds an optimal range for both, with $m = 0.6$ maximizing worst-group accuracy on Waterbirds and CelebA.
  • The choice of LLM sets the method's ceiling: in the paper's comparison, GPT-4o (the largest model tested) identifies spurious correlations best and yields the highest worst-group accuracy, while a smaller LLM like Llama 3.2 70B drops worst-group accuracy to 58.9% on CelebA.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • [Inference] If the text-to-image transfer hypothesis holds generally, the same projection recipe could debias other aligned multimodal models (for example SigLIP or ALIGN) by swapping the encoders, making LLM-guided projection a general debiasing primitive rather than a CLIP-specific fix.
  • [Inference] Because the LLM's world knowledge is the sole source of bias attributes, purely visual confounds that have no natural language counterpart—such as camera angle, resolution, or colour balance—would be invisible to PRISM; testing on a confound of that kind would probe the method's true boundary.
  • [Inference] The margin $m$ between different classes sharing an attribute is a measurable quantity before and after projection, so the LD loss could double as a lightweight bias meter that reports how much spurious signal remains in the embedding space after debiasing.
  • [Inference] The paper's single-epoch, batch-of-64 training recipe hints that the debiasing direction is a low-dimensional subspace; if a closed-form whitening or orthogonalization matches the gradient-trained $P$, optimization could be removed entirely without losing the worst-group gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes PRISM, a data-free method to reduce spurious correlations in CLIP's embedding space. In Stage 1 an LLM generates scene descriptions conditioned on class labels and discovered spurious attributes; in Stage 2 a linear projection P is learned by minimizing a Latent space Debiasing loss (Eq. 7) over text embeddings of these descriptions, and zero-shot classification is performed by applying P to both image and text embeddings (Eq. 8). A lighter variant, PRISM-mini, removes spurious directions via an orthogonal projection. Experiments on Waterbirds and CelebA with CLIP-ViT-L/14 report worst-group accuracy 84.2% and 84.0%, respectively, outperforming data-free baselines, with additional RN50 results in Appendix C.

Significance. If the mechanism is as claimed, PRISM would be a valuable contribution: it avoids predefined bias categories and external image data, uses only LLM-generated text during training, preserves overall accuracy, and ships public code. The reported gains are large and the design is simple to reproduce. However, the formal definition of the loss and the evidence for text-to-image transfer need strengthening before the empirical claims can be assessed.

major comments (3)
  1. [Section 3.2, Eq. (7), Algorithm 1] As written, every term in LLD is an inner product of the form <phi_T(...), phi_T(...)>, with no occurrence of the learned projection P. Therefore LLD is constant in P and the gradient update in Algorithm 1 step 8 is identically zero; the reported WG improvements cannot arise from the described objective. Please define the loss on the projected embeddings, e.g. replace phi_T(...) with P(phi_T(...)) throughout Eq. (7), and align Algorithm 1 and Figure 2 with this definition.
  2. [Section 5, "Bias text", Table 2] The paper's transfer hypothesis (Section 3.1) is that spurious correlations in text mirror those in images, so a projection trained on text embeddings will debias image embeddings. Table 2 only demonstrates that raw CLIP text embeddings of scene descriptions exhibit a similar worst-group gap to image embeddings; it does not test whether the learned P transfers to images. Please add a direct experiment: train P on scene descriptions only, then apply it to image embeddings and report WG/Acc on the actual benchmarks, including CelebA.
  3. [Section 4, Figures 5 and 6] The margin m and the number of scene descriptions are tuned on worst-group accuracy, but no validation protocol is described; if these values are selected using the test WG, the headline numbers are partially fitted. Please specify how m and the number of descriptions were chosen, whether a validation split was used, and report per-seed WG/Acc variability for the main results.
minor comments (6)
  1. [Appendix C] Appendix C contains an unresolved "Table X" reference in the comparison with BendVLM; please replace it with the actual table number (Table A2).
  2. [Section 3.2, Eqs. (9) and (10)] The symbol A is used both for the set of spurious attributes and for the matrix of their embeddings; please use distinct notation to avoid ambiguity.
  3. [Table 3] With Llama 3.2, the CelebA WG of 58.9% is below the zero-shot baseline of 72.8%, so PRISM is not uniformly beneficial; a sentence acknowledging this failure mode would improve the presentation.
  4. [Abstract and Appendix C, Table A1] The abstract's claim that PRISM outperforms current debiasing methods on Waterbirds and CelebA is not true for the RN50 backbone on Waterbirds, where FairerCLIP and Orth-Cali achieve higher WG; please qualify the claim by backbone.
  5. [Section 5, "Bias text", reference [24]] Reference [24] concerns object hallucination and does not directly support the claim that spurious correlations in CLIP's text domain mirror those in the image domain; please cite a more appropriate source.
  6. [Throughout] There are several typographical errors, including "orthogonlizing" (Section 3.2), "attibutes" (Appendix B), and "effectivenss" (Appendix C); a careful proofread is needed.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: PRISM's projection is optimized on LLM-generated text, not on the image labels or groups used for evaluation; the reported worst-group gains are not restatements of the loss. Two non-circular concerns remain: a minor self-citation for the text-bias transfer hypothesis and an internal inconsistency in Eq. 7, where LLD does not depend on P.

full rationale

The claimed derivation chain is not circular. In Stage 2 (Section 3.2, Eq. 7, Algorithm 1), the projection P is supposed to be learned by minimizing LLD on LLM-generated scene descriptions, and the evaluation (Eq. 8, Table 1) is on held-out image embeddings. The training objective contains no image labels, no group annotations, and no measured worst-group accuracy, so the headline Waterbirds WG gain (36.4% to 84.2%) is not a fitted value from the evaluation metric. The text-to-image transfer premise (Section 3.1, "we hypothesize that ... similar biases should also be present in the text domain") is explicitly stated as a hypothesis and checked in the "Bias text" experiment (Table 2), so it does not reduce to citation [24] even though that citation shares a co-author (Etemad) with this paper. PRISM-mini's orthogonal projection (Eq. 10) is a standard construction inspired by prior non-self work [5]. The most serious issue in the text is non-circular: Eq. (7) as printed contains no occurrence of P, so LLD(P) = LLD(I) and Algorithm 1's gradient update is formally ill-posed; the authors most likely intend inner products of the form <P(phi_T(T_{a,y})), P(phi_T(T_{a',y'}))>, but as written the central training step is undefined. This is a correctness or notation defect, not an equivalence between input and output. Hyperparameter selection for m and scene-description count (Figs. 5-6) is performed against WG, which is a test-set selection concern, but it does not make the projection's construction circular. Overall, no load-bearing circular step was identified.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method rests on three domain assumptions: text-image bias transfer, LLM co-occurrence encoding bias, and linear separability of bias. None are proved, though Table 2 provides partial evidence for the first. No new physical or categorical entities are introduced.

free parameters (4)
  • margin m in LD loss = 0.6
    Selected based on worst-group accuracy, as shown in Figure 6; affects the inter-class separation strength in Eq. 7.
  • number of scene descriptions N = not specified
    The paper shows performance varies with the number of LLM-generated scene descriptions (Figure 5) but does not state the value used in the main experiments.
  • projection matrix P = learned d x d linear map
    Optimized on text embeddings via LD loss; no regularization or norm constraint described, and its architecture (rank, initialization) is not specified.
  • learning rate = 0.1 (Waterbirds), 0.01 (CelebA)
    Chosen per dataset; no schedule or sensitivity analysis reported.
assumptions (3)
  • domain assumption Spurious correlations present in the image modality are also encoded in the text modality.
    Introduced as a hypothesis in Section 3.1 and partially validated by the 'Bias text' experiment (Table 2) on one task.
  • domain assumption An LLM's conditional probabilities encode co-occurrence-based spurious correlations from its training corpus.
    Invoked in Section 3.2 via Eq. 5-6 to justify using the LLM to produce bias attributes from class prompts.
  • domain assumption A linear projection of the shared embedding space is sufficient to remove the bias while preserving class information.
    The method only learns linear P; the Limitations section admits that non-linear biases may not be captured.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PRISM: Reducing Spurious Implicit Biases in Vision-Language Models with LLM-Guided Embedding Projection." pith.science (2026). https://pith.science/paper/VJMEMLTQ

@misc{pith2026250708979,
  author       = {Pith},
  title        = {Pith review of: PRISM: Reducing Spurious Implicit Biases in Vision-Language Models with LLM-Guided Embedding Projection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VJMEMLTQ}},
  note         = {Machine review of arXiv:2507.08979}
}
read the original abstract

We introduce Projection-based Reduction of Implicit Spurious bias in vision-language Models (PRISM), a new data-free and task-agnostic solution for bias mitigation in VLMs like CLIP. VLMs often inherit and amplify biases in their training data, leading to skewed predictions. PRISM is designed to debias VLMs without relying on predefined bias categories or additional external data. It operates in two stages: first, an LLM is prompted with simple class prompts to generate scene descriptions that contain spurious correlations. Next, PRISM uses our novel contrastive-style debiasing loss to learn a projection that maps the embeddings onto a latent space that minimizes spurious correlations while preserving the alignment between image and text embeddings.Extensive experiments demonstrate that PRISM outperforms current debiasing methods on the commonly used Waterbirds and CelebA datasets We make our code public at: https://github.com/MahdiyarMM/PRISM.

Figures

Figures reproduced from arXiv: 2507.08979 by the authors.

Figure 1
Figure 1. Performance comparison among data-free CLIP debias [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of the proposed PRISM. In Stage 1, by passing text prompts as ‘A photo of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Per-group accuracy of our method on (a) Waterbirds and (b) CelebA datasets. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The representation map at the output of CLIP (a) without [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Worst-group performance of PRISM for different num [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Worst-group performance of PRISM for different values [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 24 canonical work pages

  1. [1]

    Zero-shot robustification of zero-shot models

    Dyah Adila, Changho Shin, Linrong Cai, and Frederic Sala. Zero-shot robustification of zero-shot models. In International Conference on Learning Representa- tions, 2024. 1, 2, 3, 6

    work page 2024

  2. [2]

    Evaluating clip: towards characterization of broader capabilities and downstream implications

    Sandhini Agarwal, Gretchen Krueger, Jack Clark, Alec Radford, Jong Wook Kim, and Miles Brundage. Evaluating clip: towards characterization of broader capabilities and downstream implications. arXiv preprint arXiv:2108.02818, 2021. 2

    arXiv 2021

  3. [3]

    CLIP the bias: How useful is balancing data in multimodal learning? In International Conference on Learning Representations, 2024

    Ibrahim Alabdulmohsin, Xiao Wang, Andreas Peter Steiner, Priya Goyal, Alexander D’Amour, and Xiao- hua Zhai. CLIP the bias: How useful is balancing data in multimodal learning? In International Conference on Learning Representations, 2024. 2

    work page 2024

  4. [4]

    Learning de-biased rep- resentations with biased representations

    Hyojin Bahng, Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased rep- resentations with biased representations. In Interna- tional conference on machine learning , pages 528–

    work page

  5. [5]

    Debiasing vision-language models via biased prompts

    Ching-Yao Chuang, Varun Jampani, Yuanzhen Li, Antonio Torralba, and Stefanie Jegelka. Debiasing vision-language models via biased prompts. arXiv preprint arXiv:2302.00070, 2023. 1, 2, 5, 6

    arXiv 2023

  6. [6]

    Refining Skewed Perceptions in Vision-Language Contrastive Models through Visual Representations

    Haocheng Dai and Sarang Joshi. Refining skewed perceptions in vision-language models through visual representations. arXiv preprint arXiv:2405.14030 ,

    work page Pith review arXiv

  7. [7]

    FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs

    Sepehr Dehdashtian, Lan Wang, and Vishnu Naresh Boddeti. Fairerclip: Debiasing clip’s zero-shot pre- dictions using functions in rkhss. arXiv preprint arXiv:2403.15593, 2024. 1, 2, 5, 6

    work page Pith review arXiv 2024

  8. [8]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. 8

    arXiv 2024

Show all 33 references
  1. [9]

    Bias and fairness in large language models: A survey

    Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernon- court, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language models: A survey. Computational Linguistics, pages 1–79, 2024. 2

    2024

  2. [10]

    Bendvlm: Test-time debiasing of vision-language embeddings

    Walter Gerych, Haoran Zhang, Kimia Hamidieh, Eileen Pan, Maanas K Sharma, Tom Hartvigsen, and Marzyeh Ghassemi. Bendvlm: Test-time debiasing of vision-language embeddings. NeurIPS, 37:62480– 62502, 2024. 3

    2024

  3. [11]

    Cobias: Contex- tual reliability in bias assessment

    Priyanshul Govil, Hemang Jain, Vamshi Krishna Bonagiri, Aman Chadha, Ponnurangam Kumaraguru, Manas Gaur, and Sanorita Dey. Cobias: Contex- tual reliability in bias assessment. arXiv preprint arXiv:2402.14889, 2024. 2

    2024 arXiv

  4. [12]

    Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incen- tivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. 8

    2025 arXiv

  5. [13]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Rad- ford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. 8

    2024 arXiv

  6. [14]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Hel- yar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720, 2024. 8

    2024 arXiv

  7. [15]

    A unified debiasing approach for vision-language models across modalities and tasks

    Hoin Jung, Taeuk Jang, and Xiaoqian Wang. A unified debiasing approach for vision-language models across modalities and tasks. Advances in Neural Information Processing Systems, 37:21034–21058, 2025. 2

    2025

  8. [16]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 6

    2014 arXiv

  9. [17]

    Last layer re-training is sufficient for robustness to spurious correlations

    Polina Kirichenko, Pavel Izmailov, and Andrew Gor- don Wilson. Last layer re-training is sufficient for robustness to spurious correlations. In International Conference on Learning Representations, 2023. 1, 2, 3, 6

    2023

  10. [18]

    Deep learning face attributes in the wild

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pages 3730–3738, 2015. 5

    2015

  11. [19]

    Gender artifacts in visual datasets

    Nicole Meister, Dora Zhao, Angelina Wang, Vikram V Ramaswamy, Ruth Fong, and Olga Russakovsky. Gender artifacts in visual datasets. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4837–4848, 2023. 1, 2

    2023

  12. [20]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sas- try, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, ...

    2021

  13. [21]

    Beads: Bias evaluation across domains.arXiv preprint arXiv:2406.04220, 2024

    Shaina Raza, Mizanur Rahman, and Michael R Zhang. Beads: Bias evaluation across domains.arXiv preprint arXiv:2406.04220, 2024. 2

    2024

  14. [22]

    Towards learning an unbiased classifier from biased data via conditional adversarial debiasing

    Christian Reimers, Paul Bodesheim, Jakob Runge, and Joachim Denzler. Towards learning an unbiased classifier from biased data via conditional adversarial debiasing. arXiv preprint arXiv:2103.06179, 2021. 2

    2021 arXiv

  15. [23]

    Distributionally robust neural networks for group shifts: On the 9 importance of regularization for worst-case gener- alization

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. Distributionally robust neural networks for group shifts: On the 9 importance of regularization for worst-case gener- alization. International Conference on Learning Representations, 2020. 5

    2020

  16. [24]

    Data- augmented phrase-level alignment for mitigating ob- ject hallucination

    Pritam Sarkar, Sayna Ebrahimi, Ali Etemad, Ahmad Beirami, Sercan ¨O Arık, and Tomas Pfister. Data- augmented phrase-level alignment for mitigating ob- ject hallucination. International Conference on Learn- ing Representations, 2025. 4, 7

    2025

  17. [25]

    Investigating the limita- tion of clip models: The worst-performing categories

    Jie-Jing Shao, Jiang-Xin Shi, Xiao-Wen Yang, Lan- Zhe Guo, and Yu-Feng Li. Investigating the limita- tion of clip models: The worst-performing categories. CoRR, 2023. 2

    2023

  18. [26]

    Revise: A tool for measuring and mitigating bias in visual datasets

    Angelina Wang, Alexander Liu, Ryan Zhang, Anat Kleiman, Leslie Kim, Dora Zhao, Iroha Shirai, Arvind Narayanan, and Olga Russakovsky. Revise: A tool for measuring and mitigating bias in visual datasets. In- ternational Journal of Computer Vision, 130(7):1790– 1810, 2022. 1

    2022

  19. [27]

    Are gender- neutral queries really gender-neutral? mitigating gen- der bias in image search

    Jialu Wang, Yang Liu, and Xin Wang. Are gender- neutral queries really gender-neutral? mitigating gen- der bias in image search. In Proceedings of the 2021 Conference on Empirical Methods in Natural Lan- guage Processing, 2021. 1

    2021

  20. [28]

    Fair- clip: Social bias elimination based on attribute proto- type learning and representation neutralization

    Junyang Wang, Yi Zhang, and Jitao Sang. Fair- clip: Social bias elimination based on attribute proto- type learning and representation neutralization. arXiv preprint arXiv:2210.14562, 2022. 2

    2022 arXiv

  21. [29]

    Robust fine- tuning of zero-shot models

    Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gontijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, et al. Robust fine- tuning of zero-shot models. In Proceedings of the IEEE/CVF conference on computer visio...

    2022

  22. [30]

    Towards fairer datasets: Filter- ing and balancing the distribution of the people sub- tree in the imagenet hierarchy

    Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky. Towards fairer datasets: Filter- ing and balancing the distribution of the people sub- tree in the imagenet hierarchy. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pag...

    2020

  23. [31]

    Understand- ing bias in large-scale visual datasets

    Boya Zeng, Yida Yin, and Zhuang Liu. Understand- ing bias in large-scale visual datasets. arXiv preprint arXiv:2412.01876, 2024. 1, 2

    2024 arXiv

  24. [32]

    Contrastive adapters for foundation model group robustness

    Michael Zhang and Christopher R ´e. Contrastive adapters for foundation model group robustness. Ad- vances in Neural Information Processing Systems, 35: 21682–21697, 2022. 3

    2022

  25. [33]

    Provide a list of potential bias attibutes associated with the following zero-shot classification using CLIP: <{tk}>

    Kankan Zhou, Yibin LAI, and Jing Jiang. Vlstereoset: A study of stereotypical bias in pre-trained vision- language models. In Proceedings of the Associa- tion for Computational Linguistics. International Joint Conference on Natural Language Processing, 2022. 1 10 Appendix A. N...

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.