Pith. sign in

REVIEW 3 major objections 6 minor 57 references

Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoders

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pretraining data, not architecture, predicts bias in CLIP models, and high-performing models tend to carry more bias.

desk verdict The dataset-dominance finding is likely right in broad strokes, but the regression's p-values are overconfident because the 3,406 measurements are clustered in 131 models and that clustering is not modeled. read the letter →

arxiv 2502.07957 v2 pith:QAMRGIUJ submitted 2025-02-11 cs.AI

Kshitish Ghate , Isaac Slaughter , Kyra Wilson , Mona Diab , Aylin Caliskan This is my paper · ORCID
classification cs.AI
keywords CLIPintrinsicbiasembeddingassociationtestsvision-languagemodelspretrainingdatadatasetcurationzero-shotperformancesocial
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

What this paper is trying to establish: the biases that show up inside vision-language encoders are primarily a property of the data they were pretrained on, not of the architecture or size of the model, and that those biases move together with downstream task performance. It reaches this by measuring 131 CLIP-style models with 26 association tests spanning images, text, and cross-modal combinations, producing 3,406 bias effect sizes. The strongest upstream predictor is the dataset family, and datasets that were aggressively filtered for accuracy like dfn, commonpool, and datacomp carry the highest bias. The paper also reports that bias and performance are often positively correlated ($0.3 \le r \le 0.8$), meaning the same training choices that make models better at benchmarks can make them more stereotyped. If true, this shifts the burden of bias mitigation onto data curation rather than model design alone.

What carries the argument

The load-bearing measurement is the EAT effect size $d$, the standardized difference in mean cosine similarity between target-concept embeddings (e.g., flowers, instruments, women, European Americans, young people) and pleasant versus unpleasant attribute embeddings. The statistical engine is a mixed-effects regression with random intercepts and slopes that attributes variance in 3,406 $d$ observations to dataset family, architecture family, log-parameter count, and log-dataset size; Pearson correlations against the VTAB+ benchmark then link bias to downstream zero-shot performance. The paper also introduces human-grounded attribute stimuli (OASIS images and NRC-VAD words) to reduce noise in the EAT estimates.

What would settle it

Train two sets of CLIP encoders with identical architecture, parameter count, and training recipe on the same raw image-text pool, one filtered with a performance-oriented pipeline and one with a low-bias heuristic pipeline, then rerun the 26 EATs; if the bias gap disappears, the 'dataset choice drives bias' claim fails. Alternatively, add training-recipe covariates such as batch size, epochs, or filtering network weights to the mixed-effects model and check whether the dataset-family coefficients survive.

Watch

Extended reading notes

Core claim

Studying 131 CLIP encoders with 26 Embedding Association Tests (EATs) across four modality combinations, the paper's central claim is that the pretraining dataset family is the dominant upstream predictor of intrinsic bias, with statistically significant positive coefficients for performance-focused families like 'dfn' ($\beta = 0.608$), 'commonpool' ($\beta = 0.399$), and 'merged2b' ($\beta = 0.396$) relative to the CC12m baseline, while no architecture family shows a significant effect. The same analysis finds that stronger intrinsic bias often accompanies better downstream zero-shot performance, with correlations between $0.3$ and $0.8$ for non-human associations and negative correlations for gender/valence in some modality settings. The authors interpret this as evidence that optimizing models for performance, especially through automated data filtering, can inadvertently amplify representational stereotypes, and that bias is modality-dependent rather than uniform across text and image.

Load-bearing premise

The claim that dataset choice, not architecture or scale, drives bias rests on the assumption that grouping 131 models into dataset families and architecture families leaves each family homogeneous enough in training recipe and compute that the regression's dataset-family coefficients isolate the effect of data composition rather than capturing differences in optimization or checkpoint availability.

Editorial extensions

If this is right

  • Bias audits of CLIP-style models should report the pretraining dataset family; models from the same family will likely cluster in bias regardless of architecture.
  • Performance-oriented data filtering pipelines need fairness constraints built in; the current approach of filtering for benchmark accuracy alone appears to raise intrinsic bias.
  • Correlations between bias and zero-shot performance mean benchmark rankings can systematically favor more stereotyped models, so accuracy and fairness should be evaluated together.
  • Modality-specific results imply unimodal audits, whether text-only or image-only, understate cross-modal bias; evaluation should cover all four modality combinations.
  • Data curation choices such as hypernymizing names to a generic '[PERSON]' token may lower bias, consistent with the low-bias CC12m reference family.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If dataset curation is the dominant lever, then interventions like balanced resampling or demographic parity filtering are likely to be more cost-effective than architectural changes or scaling alone, a testable prediction the paper does not make.
  • The positive bias-performance correlation suggests intrinsic bias measures could be used as cheap, representation-level signals during training, but only if the correlation is causal rather than a shared confound with data quality, something the paper's correlational design cannot separate.
  • The modality-dependence of bias raises a route to mitigation: aligning text and image representations for a social category may dampen cross-modal stereotypes, since opposite-sign age associations appear in text versus image.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper presents a large-scale empirical study of intrinsic social bias in 131 CLIP models. The authors measure bias using 26 EAT/SEAT/iEAT-style tests across four modality combinations, and relate the resulting 3,406 effect sizes to upstream pretraining factors (dataset, architecture, parameter count, dataset size) via a mixed-effects regression, and to downstream VTAB+ performance via Pearson correlations. They report that pretraining dataset family is the most significant upstream predictor of intrinsic bias, that datasets curated with filtering techniques aimed at downstream performance tend to be associated with higher bias, and that intrinsic bias often correlates with downstream performance. They also introduce controlled, human-grounded attribute stimuli for the EATs and release their code and data.

Significance. If the regression result were statistically sound, this would be a valuable and much-needed large-scale comparison, indeed the largest such analysis of CLIP bias to date. The paper's strengths include the breadth of models (131), the use of established bias tests with improved human-grounded stimuli, the public release of code and data, and the connection between upstream curation choices and downstream performance. However, the central dataset-family claim is currently supported by a model that does not account for model-level clustering, so the key quantitative conclusion is not yet established. The contribution is significant and the empirical corpus is a useful resource, but the headline 'predicted by pretraining data' claim requires a statistically defensible regression.

major comments (3)
  1. [Section 4, Eq. (1); Section 5, Figure 3] The mixed-effects model includes random intercepts and slopes only for modality-by-test-order groups, not for models or dataset families. Each of the 131 models contributes 26 EAT observations that share a dataset family, so the residuals are correlated within model and within dataset family; the effective sample size for the dataset-family fixed effects is at most 131, not 3,406. The Wald p-values reported in Figure 3 are therefore likely anti-conservative, and it is uncertain whether coefficients such as dfn (beta = 0.608) or commonpool (beta = 0.399) survive when standard errors are clustered at the model or dataset-family level, or under a permutation test that shuffles models between families. Please re-fit with model-level random intercepts, cluster-robust standard errors, or a model-level permutation test, and report the corresponding confidence intervals.
  2. [Appendix B] The convergence-driven grouping of datasets into families, combined with the absence of model-level random effects, leaves the dataset-family effect not separately identified from training recipe, optimization budget, or checkpoint-selection effects. A family such as 'dfn' or 'merged2b' may be represented by checkpoints from a single training run, and the regression conditions only on architecture and parameter count. The claim in the abstract and Section 6 that dataset choice is significant 'independent of other upstream factors such as model architecture or parameter count' is too strong; at best the analysis controls for the listed covariates. Please report the number of distinct models per dataset family and per architecture family, and discuss the remaining confounding with training recipe and scale.
  3. [Title and Abstract] The title and abstract state that intrinsic bias is 'predicted' by pretraining data, but the analysis is an in-sample regression with no out-of-sample validation, no cross-validation, and no predictive metric. The results are associational, not predictive. Please either rephrase to 'associated with' and 'explained by' throughout, or add a proper out-of-sample prediction evaluation, such as holding out entire dataset families before quantifying predictive accuracy.
minor comments (6)
  1. [Section 5 heading] The heading 'EA Ts as an Aggregate Measure of Bias' contains an unnecessary space between 'EA' and 'Ts'; it should read 'EATs'.
  2. [Appendix A.1.3] The sentence about the YFCC15M subset contains a duplicated 'whose': 'whose whose title contains natural language' should be 'whose title contains natural language.'
  3. [Ethical Considerations] The word 'instrinsic' is misspelled; it should be 'intrinsic.'
  4. [Section 8 (Limitations)] The sentence 'while with training datasets are curated using multilingual and multicultural sources such as webli' is grammatically incomplete; it should read 'while some training datasets are curated using multilingual and multicultural sources such as webli.'
  5. [References] In the reference to Goh et al. (2021), 'V oss' should be 'Voss'.
  6. [Appendix A] The text says 'All code and data used in this study will be made available publicly' while the Introduction states 'We release our code and data at https://github.com/kshitishghate/CLIP_bias/.' Please make the release status consistent and specify the license and version for the released artifacts.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: bias measurements, upstream predictors, and downstream benchmarks are empirically independent.

full rationale

The derivation chain is empirical rather than deductive, so no load-bearing step reduces to its own inputs. EAT effect sizes are computed from externally established EAT/SEAT/iEAT instruments, and the new attribute stimuli are grounded in OASIS and NRC-VAD human valence ratings rather than in the models' pretraining labels or downstream scores. The mixed-effects regression treats dataset family, architecture, parameter count, and dataset size as predictors and EAT effect size as the outcome; dataset family is not defined from the effect sizes, and the finding that dataset family is significant is a fitted coefficient, not a prediction derived from that coefficient's definition. The downstream analysis uses independently reported VTAB+ performance scores. Several citations are to prior work by co-authors, but they serve as method validation, human-benchmark references, or prior empirical findings, and none is used to concede the paper's central conclusion. The skeptic's model-clustering and p-value concern is a statistical validity issue, not a circularity issue. No quoted equation reduces to another by construction, no fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors, and no ansatz is smuggled in via citation.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper's central claims rest on the validity of EATs as bias measures, the human grounding of the new stimuli, and the statistical identification of dataset-family effects after grouping. It introduces no new theoretical entities or fitted parameters beyond the regression coefficients; the regression coefficients are estimates of the empirical relationship, not assumptions.

assumptions (5)
  • domain assumption EAT effect sizes computed via cosine similarity in CLIP embedding spaces are valid measures of intrinsic social bias across models.
    Adopted from prior work (Caliskan et al., 2017; Steed and Caliskan, 2021) without re-validation across the 131 models; the paper assumes comparability of embedding spaces across architectures.
  • domain assumption Valence ratings from OASIS and NRC-VAD are human-grounded and the selected top-25 pleasant/unpleasant stimuli are balanced.
    The paper selects 25 images and 25 words per attribute set and reports aggregate valence balance, but does not provide per-stimulus controls or show that balance holds within each CLIP embedding space.
  • ad hoc to paper Grouping models into dataset families and architecture families yields categories homogeneous enough for mixed-effects regression.
    Appendix B notes convergence issues led to grouping; if families conflate dataset with training recipe or compute, the fixed-effect estimates are biased. This is the weakest structural assumption.
  • standard math The mixed-effects model with random intercepts and slopes for log(param) and log(dataset_size) is correctly specified.
    Wald z-tests assume asymptotic normality; no residual diagnostics or multicollinearity checks are reported.
  • domain assumption VTAB+ performance scores are comparable across the subset of models with available scores.
    Correlations use only models with VTAB+ scores; missingness may correlate with dataset family, which is not tested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoders." pith.science (2026). https://pith.science/paper/QAMRGIUJ

@misc{pith2026250207957,
  author       = {Pith},
  title        = {Pith review of: Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QAMRGIUJ}},
  note         = {Machine review of arXiv:2502.07957}
}
abstract

While recent work has found that vision-language models trained under the Contrastive Language Image Pre-training (CLIP) framework contain intrinsic social biases, the extent to which different upstream pre-training features of the framework relate to these biases, and hence how intrinsic bias and downstream performance are connected has been unclear. In this work, we present the largest comprehensive analysis to-date of how the upstream pre-training factors and downstream performance of CLIP models relate to their intrinsic biases. Studying 131 unique CLIP models, trained on 26 datasets, using 55 architectures, and in a variety of sizes, we evaluate bias in each model using 26 well-established unimodal and cross-modal principled Embedding Association Tests. We find that the choice of pre-training dataset is the most significant upstream predictor of bias, whereas architectural variations have minimal impact. Additionally, datasets curated using sophisticated filtering techniques aimed at enhancing downstream model performance tend to be associated with higher levels of intrinsic bias. Finally, we observe that intrinsic bias is often significantly correlated with downstream performance ($0.3 \leq r \leq 0.8$), suggesting that models optimized for performance inadvertently learn to amplify representational biases. Comparisons between unimodal and cross-modal association tests reveal that social group bias depends heavily on the modality. Our findings imply that more sophisticated strategies are needed to address intrinsic model bias for vision-language models across the entire model development pipeline.

Figures

Figures reproduced from arXiv: 2502.07957 by the authors.

Figure 1
Figure 1. We use Embedding Association Tests, bias [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Aggregate Effect Size (d) by Test Category Across Modality Orders (NRC-OASIS) along with error bars in black representing standard deviation. EATs as an Aggregate Measure of Bias As shown in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Fixed effects coefficients with 95% confidence [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Measure of Pearson’s correlation r between effect size magnitude and downstream VTAB+ performance across test categories and modality combinations. Significant values are marked with an asterisk. more positive terms than men (Caliskan et al., 2022; Charlesworth et al.,…
Figure 5
Figure 5. Figure 5: List of Test Categories and Stimuli selected from [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Full mixed effects regression results to measure impact of upstream factors on intrinsic bias. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Aggregate Effect Size (d) with error bars representing standard deviation by Test Category Across Modality Orders from SEAT and iEAT stimuli. The direction of effect sizes are largely consistent with the effect sizes obtained from new stimuli in [PITH_FULL_IMAGE:figur…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 32 canonical work pages

  1. [1]

    Andrew Scott Baron, Toni Schmader, Dario Cvencek, and Andrew N Meltzoff. 2013. The gendered self-concept: How implicit gender stereotypes and attitudes shape self-definition, pages 109--132. Psychology Press

    work page 2013

  2. [2]

    Hugo Berg, Siobhan Hall, Yash Bhalgat, Hannah Kirk, Aleksandar Shtedritski, and Max Bain. 2022. https://aclanthology.org/2022.aacl-main.61 A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learning . In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the...

    work page 2022

  3. [3]

    Su Lin Blodgett, Solon Barocas, Hal Daum \' e III, and Hanna Wallach. 2020. https://doi.org/10.18653/v1/2020.acl-main.485 Language (Technology) is Power: A Critical Survey of "Bias" in NLP . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454--5476. Association for Computational Linguistics (ACL)

    doi:10.18653/v1/2020.acl-main.485 2020

  4. [4]

    Laura Cabello, Emanuele Bugliarello, Stephanie Brandl, and Desmond Elliott. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.525 Evaluating bias and fairness in gender-neutral pretrained vision-and-language models . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 8465--8483, Singapore. Association for Comp...

    doi:10.18653/v1/2023.emnlp-main.525 2023

  5. [5]

    Aylin Caliskan, Pimparkar Parth Ajay, Tessa Charlesworth, Robert Wolfe, and Mahzarin R Banaji. 2022. Gender bias in word embeddings: A comprehensive analysis of frequency, syntax, and semantics. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pages 156--170

    work page 2022

  6. [6]

    Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. https://doi.org/10.1126/science.aal4230 Semantics derived automatically from language corpora contain human-like biases . Science, 356(6334):183--186

    doi:10.1126/science.aal4230 2017

  7. [7]

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. 2021. Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3558--3568

    work page 2021

  8. [8]

    Tessa ES Charlesworth, Kshitish Ghate, Aylin Caliskan, and Mahzarin R Banaji. 2024. Extracting intersectional stereotypes from embeddings: Developing and validating the flexible intersectional stereotype extraction procedure. PNAS nexus, 3(3):pgae089

    2024

Show all 57 references
  1. [9]

    Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. 2022. Pali: A jointly-scaled multilingual language-image model. arXiv preprint arXiv:2209.06794

    2022 arXiv

  2. [10]

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. 2023. Reproducible Scaling Laws for Contrastive Language-Image Learning . In Proceedings of the IEEE/CVF Conference on Computer...

    2023

  3. [11]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database . In 2009 IEEE conference on computer vision and pattern recognition, pages 248--255

    2009

  4. [12]

    Jesse Dodge, Ana Marasovic, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, Matt Gardner, and William Agnew. 2021. https://api.semanticscholar.org/CorpusID:237568724 Documenting large webtext corpora: A case study on the colossal clean crawled corpus . In Conference on Em...

    2021

  5. [13]

    Simone Fabbrizzi, Symeon Papadopoulos, Eirini Ntoutsi, and Ioannis Kompatsiaris. 2022. A survey on bias in visual datasets. Computer Vision and Image Understanding, 223:103552

    2022

  6. [14]

    Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. 2023 a . Data filtering networks. arXiv preprint arXiv:2309.17425

    2023 arXiv

  7. [15]

    Toshev, and Vaishaal Shankar

    Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander T. Toshev, and Vaishaal Shankar. 2023 b . https://openreview.net/forum?id=KAk6ngZ09F Data Filtering Networks

    2023

  8. [16]

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. 2024. Datacomp: In search of the next generation of multimodal datasets. Advances in Neural Information Processing...

    2024

  9. [17]

    Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah. 2021. https://doi.org/10.23915/distill.00030 Multimodal Neurons in Artificial Neural Networks . Distill

    2021 doi

  10. [18]

    Anthony G Greenwald, Debbie E Mcghee, and Jordan L K Schwartz. 1998 a . https://doi.org/10.1037/0022-3514.74.6.1464 Measuring Individual Differences in Implicit Cognition: The Implicit Association Test . Journal of Personality and Social Psychology, 74(6):1464--1480

    1998 doi

  11. [19]

    Anthony G Greenwald, Debbie E McGhee, and Jordan LK Schwartz. 1998 b . Measuring individual differences in implicit cognition: the implicit association test. Journal of personality and social psychology, 74(6):1464

    1998

  12. [20]

    Varun Gulshan, Lily Peng, Marc Coram, Martin C Stumpe, Derek Wu, Arunachalam Narayanaswamy, Subhashini Venugopalan, Kasumi Widner, Tom Madams, Jorge Cuadros, and others . 2016. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in ret...

    2016

  13. [21]

    Wei Guo and Aylin Caliskan. 2021. https://doi.org/10.1145/3461702.3462536 Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases . In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 122...

    2021

  14. [22]

    Suchin Gururangan, Dallas Card, Sarah Dreier, Emily Gade, Leroy Wang, Zeyu Wang, Luke Zettlemoyer, and Noah A. Smith. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.165 Whose language counts as high quality? measuring language ideologies in text data selection . In Proceedi...

    2022 doi

  15. [23]

    Melissa Hall, Laurens van der Maaten, Laura Gustafson, Maxwell Jones, and Aaron Adcock. 2022. A systematic study of bias amplification. arXiv preprint arXiv:2201.11706

    2022 arXiv

  16. [24]

    Siobhan Mackenzie Hall, Fernanda Gonçalves Abrantes, Hanwen Zhu, Grace Sodunke, Aleksandar Shtedritski, and Hannah Rose Kirk. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/hash/c93f26b1381b17693055a611a513f1e9-Abstract-Datasets_and_Benchmarks.html VisoGender : A ...

    2023

  17. [25]

    Eddie Harmon-Jones, Philip A Gable, and Tom F Price. 2013. Does negative affect always narrow and positive affect always broaden the mind? considering the influence of motivational intensity on cognitive scope. Current Directions in Psychological Science, 22(4):301--307

    2013

  18. [26]

    Rachel Hong, William Agnew, Tadayoshi Kohno, and Jamie Morgenstern. 2024. Who's in and who's out? a case study of multimodal clip-filtering in datacomp. In Proceedings of the 4th ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pages 1--17

    2024

  19. [27]

    Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. 2021. https://doi.org/10.5281/zenodo.5143773 OpenCLIP

    2021 doi

  20. [28]

    Sepehr Janghorbani and Gerard De Melo. 2023. https://aclanthology.org/2023.eacl-main.126 Multi-Modal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision–Language Models . In Proceedings of the 17th Conference of the European Chapter...

    2023

  21. [29]

    Benedek Kurdi, Shayn Lozano, and Mahzarin R Banaji. 2017. Introducing the open affective standardized image set (oasis). Behavior research methods, 49:457--470

    2017

  22. [30]

    Faisal Ladhak, Esin Durmus, Mirac Suzgun, Tianyi Zhang, Dan Jurafsky, Kathleen McKeown, and Tatsunori Hashimoto. 2023. https://aclanthology.org/2023.eacl-main.234 When Do Pre-Training Biases Propagate to Downstream Tasks? A Case Study in Text Summarization . In Proceedings of ...

    2023

  23. [31]

    Yann LeCun, Fu Jie Huang, and Leon Bottou. 2004. Learning methods for generic object recognition with invariance to pose and lighting . In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004., volume 2, page II–104

    2004

  24. [32]

    Fei-Fei Li, Marco Andreeto, Marc'Aurelio Ranzato, and Pietro Perona. 2022. https://doi.org/10.22002/D1.20086 Caltech 101

    2022 doi

  25. [33]

    Steven G Luke. 2017. Evaluating significance in linear mixed-effects models in r. Behavior research methods, 49:1494--1502

    2017

  26. [34]

    Abhishek Mandal, Suzanne Little, and Susan Leavy. 2023. https://doi.org/10.1145/3577190.3614156 Multimodal Bias : Assessing Gender Bias in Computer Vision Models with NLP Techniques . In Proceedings of the 25th International Conference on Multimodal Interaction , ICMI '23, pag...

    2023

  27. [35]

    Bowman, and Rachel Rudinger

    Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. https://doi.org/10.18653/v1/N19-1063 On Measuring Social Biases in Sentence Encoders . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational ...

    2019 doi

  28. [36]

    Saif Mohammad. 2018. Obtaining reliable human ratings of valence, arousal, and dominance for 20,000 english words. In Proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: Long papers), pages 174--184

    2018

  29. [37]

    Shiva Omrani Sabbaghi, Robert Wolfe, and Aylin Caliskan. 2023. Evaluating biased attitude associations of language models in an intersectional context. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pages 542--553

    2023

  30. [38]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. https://proceedings.mlr.press/v139/radford21a.html Learning Transferable Visual Model...

    2021

  31. [39]

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving Language Understanding by Generative Pre-Training

    2018

  32. [40]

    Gabriele Ruggeri, Debora Nozza, et al. 2023. A multi-dimensional study on bias in vision-language models. In Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics

    2023

  33. [41]

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. 2...

    2022

  34. [42]

    Ryan Steed and Aylin Caliskan. 2021. https://doi.org/10.1145/3442188.3445932 Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biases . In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 701--713, New Yo...

    2021

  35. [43]

    Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. 2023. Eva-clip: Improved training techniques for clip at scale. arXiv preprint arXiv:2303.15389

    2023 arXiv

  36. [44]

    Hao Tan and Mohit Bansal. 2019. https://doi.org/10.18653/v1/D19-1514 LXMERT : Learning cross-modality encoder representations from transformers . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferenc...

    2019 doi

  37. [45]

    Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 2016. https://doi.org/10.1145/2812802 YFCC100M: The New Data in Multimedia Research . Commun. ACM, 59(2):64--73

    2016 doi

  38. [46]

    Autumn Toney-Wails and Aylin Caliskan. 2020. Valnorm quantifies semantics to reveal consistent valence biases across languages and over centuries. arXiv preprint arXiv:2006.03950

    2020 arXiv

  39. [47]

    Kyra Wilson and Aylin Caliskan. 2024. Gender, race, and intersectional bias in resume screening via language model retrieval. arXiv preprint arXiv:2407.20371

    2024 arXiv

  40. [49]

    Robert Wolfe, Mahzarin R Banaji, and Aylin Caliskan. 2022 b . Evidence for hypodescent in visual semantic ai. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1293--1304

    2022

  41. [50]

    Robert Wolfe and Aylin Caliskan. 2021. Low frequency names exhibit bias and overfitting in contextualizing language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 518--532

    2021

  42. [51]

    Robert Wolfe and Aylin Caliskan. 2022 a . https://doi.org/10.1145/3514094.3534136 American == White in Multimodal Language-and-Image AI . In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, AIES '22, pages 800--812, New York, NY, USA. Association for Com...

    2022

  43. [52]

    Robert Wolfe and Aylin Caliskan. 2022 b . https://doi.org/10.1145/3531146.3533183 Markedness in Visual Semantic AI . In ACM International Conference Proceeding Series, pages 1269--1279. Association for Computing Machinery

    2022

  44. [53]

    Robert Wolfe and Aylin Caliskan. 2022 c . https://doi.org/10.1609/aaai.v36i10.21400 VAST: The Valence-Assessing Semantics Test for Contextualizing Language Models . Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):11477--11485

    2022 doi

  45. [54]

    Robert Wolfe, Yiwei Yang, Bill Howe, and Aylin Caliskan. 2023. https://doi.org/10.1145/3593013.3594072 Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias . In Proceedings of the 2023 ACM Conference on Fairness, A...

    2023

  46. [55]

    Hu Xu, Saining Xie, Xiaoqing Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. 2023 a . https://openreview.net/forum?id=5BCFlnfE1g Demystifying CLIP Data

    2023

  47. [56]

    Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. 2023 b . Demystifying clip data. arXiv preprint arXiv:2309.16671

    2023 arXiv

  48. [57]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  49. [58]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.