Pith. sign in

REVIEW 2 major objections 5 minor 56 references

Scaling vision-language models improves average accuracy but fails against multi-attribute bias; training data size and curation hold up instead.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 14:30 UTC pith:BTBM227O

load-bearing objection Large OpenCLIP audit shows scale stops predicting multi-attribute WGA while data size/curation hold up; scoped but solid and worth engaging. the 2 major comments →

arxiv 2607.28211 v1 pith:BTBM227O submitted 2026-07-30 cs.CV

Scaling Vision-Language Models Is Not Enough to Mitigate Bias

Ioannis Sarridis , Ioannis Kompatsiaris , Symeon Papadopoulos This is my paper
classification cs.CV
keywords spurious correlationsbiasvision-language modelsscaling lawsworst-group accuracydata curationzero-shot robustnessshortcut learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-language models are usually chosen by how well they score on broad tests like ImageNet, under the quiet assumption that bigger, stronger models are also less fooled by shortcuts. This paper tests that assumption across 194 public models by comparing overall accuracy with worst-group accuracy on a single spurious cue (CelebA hair color vs gender) and on two simultaneous cues (UrbanCars car type vs background and co-occurring object). The link between parameter count and performance collapses as the bias gets more complex, and controlled size-only comparisons even show multi-attribute worst-group accuracy falling when models get larger. Dataset size and especially curation quality keep a clearer relationship with robustness, with curated data beating size-matched unfiltered data by large margins. The practical message is that practitioners who care about shortcut resistance should treat data quality as a first-class lever and stop treating scale as a reliable fix.

Core claim

As evaluation moves from overall ImageNet accuracy to single-attribute then multi-attribute worst-group accuracy, the Spearman correlation of model scale with performance falls from 0.68 to 0.48 to a non-significant 0.05, while training-data size stays meaningfully correlated and curated datasets improve worst-group accuracy by up to about 25 percent over uncurated alternatives at matched scale. In matched comparisons that vary only backbone size, scaling averages +2.6 percent ImageNet accuracy but −4.2 percent UrbanCars worst-group accuracy.

What carries the argument

Bias complexity sensitivity: the progressive decay, across a ladder of benchmarks with more simultaneous spurious attributes, of the usual scaling relationship between model size and performance, measured by worst-group accuracy on the group structure induced by label times spurious attributes.

Load-bearing premise

That two zero-shot group-structured benchmarks—one face attribute with a single demographic cue, one synthetic car scene with two controlled shortcuts—plus the public OpenCLIP checkpoint set are enough to stand in for how scale behaves under real multi-attribute bias.

What would settle it

Find a large matched family of VLMs where increasing parameter count alone, with data and architecture fixed, reliably raises worst-group accuracy on a multi-attribute bias benchmark comparable to UrbanCars; or show that on several additional multi-attribute benchmarks the scale–WGA correlation stays strong rather than collapsing near zero.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Model cards and leaderboards that rank VLMs only by ImageNet-style averages will systematically mis-rank models for shortcut robustness.
  • At fixed compute, investing in data filtering and curation is a more reliable path to worst-group gains than simply enlarging the backbone.
  • Multi-attribute bias suites should become standard release criteria alongside average zero-shot accuracy.
  • Token granularity (patch size and resolution) should be tuned to the spatial nature of the expected shortcuts, not only to average accuracy.
  • Pareto-strong models in this audit share curated or large data, finer tokens, and moderate—not maximal—scale.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If scale mainly improves context exploitation, larger models may look better on bias-aligned slices while quietly worsening the fully conflicting minority groups that WGA cares about.
  • The same pattern may appear in other foundation-model settings (audio-language, video-language) whenever multiple non-causal cues can be exploited at once.
  • Procurement and safety review for deployed VLMs may need mandatory multi-shortcut audits rather than relying on demographic single-attribute probes alone.
  • Data-curation methods that optimize average CLIP score may still leave multi-attribute holes; filters may need explicit worst-group or conflict-aware objectives.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper audits 194 public OpenCLIP VLMs (16 families, 63M–3.6B params, 24 training datasets) on ImageNet accuracy and worst-group accuracy (WGA) on CelebA (single spurious attribute) and UrbanCars (two simultaneous spurious attributes). It reports that Spearman correlation of model scale with performance decays from ρ=0.68 (ImageNet) to 0.48 (CelebA WGA) to 0.05 n.s. (UrbanCars WGA), while data size remains more stable (ρ=0.59→0.57→0.41) and curated data can improve UrbanCars WGA by up to ~25% at matched scale. Matched comparisons isolating scale, data, patch size, and resolution, plus a subgroup analysis on bias-aligned vs. bias-conflicting UrbanCars groups, support a “bias complexity sensitivity” framing and practical guidance favoring data quality and fine tokenization over pure scale.

Significance. If the scoped empirical pattern holds, the work is a useful corrective to scale-centric VLM selection: overall accuracy and parameter count are weak proxies for multi-attribute shortcut robustness, whereas training-data size and curation remain more informative. Strengths include the breadth of the public-checkpoint audit (194 models), explicit matched-group controls (Tables 2, 4, 5; dataset isolation in Fig. 4), subgroup decomposition (Table 7), a prompt-sensitivity check in §5, Pareto/safe-model tables, and released code. The contribution is primarily empirical and ecosystem-level rather than methodological, but it is timely for practitioners choosing open VLMs and for evaluation practice that still overweights ImageNet-style averages.

major comments (2)
  1. [Table 2; Abstract; Figure 1; §4.2 Model Scale] Table 2 and the abstract/Fig. 1 framing: the mean UrbanCars effect of scaling is −4.22%, but the sign split is 12 positive / 13 negative with large configuration-dependent swings (e.g., large drops in some SigLIP2 and MetaCLIP2 rows, large gains in some CLIPA rows). “Scaling reduces WGA by −4.2%” overstates a systematic negative effect; the load-bearing, better-supported claim is that scale is unreliable / loses predictive power and is not a robust lever once bias is multi-attribute. Please rephrase abstract, Fig. 1, §4.2, and Conclusion to lead with unreliability/no consistent benefit, and report the mean with dispersion (e.g., median and IQR or a simple sign test) so the average is not read as a near-universal reduction.
  2. [§3.3 Benchmarks; §4.1; §5 Discussion] §3.3 and the “bias complexity sensitivity” construct treat the ImageNet→CelebA→UrbanCars progression as increasing spurious-correlation complexity (M=0/1/2). That axis is confounded with domain (faces vs. synthetic car scenes), realism (natural CelebA vs. pasted UrbanCars), and spatial structure of cues. §5 discloses limited benchmarks, but the central named claim still leans on “complexity.” Either (i) add at least one additional group-structured multi-attribute or non-synthetic bias benchmark, or (ii) narrow the claim language throughout to “across these two established group-structured bias benchmarks of increasing attribute count,” and discuss the confound explicitly in §4.1/§5 so the title-level generalization is not stronger than the design.
minor comments (5)
  1. [Table 1; §4.1] Table 1 reports Tokens ρ=+0.27*** on UrbanCars while Parameters are n.s.; a brief partial-correlation or note on collinearity among parameters, tokens, patch size, and resolution would clarify how much “representation” is separable from scale.
  2. [§3.6; §5] §3.6 prompt construction for CelebA is asymmetric (more non-blonde descriptors plus brunette/bald). The 80-template check in §5 is helpful; consider reporting the primary WGA numbers under that protocol in an appendix table for reproducibility.
  3. [Figure 4; Table 3] Figure 4 sorts datasets by UrbanCars mean but vertical spread is large for some corpora; stating n per dataset (models entering each mean) in Table 3/Fig. 4 would prevent over-reading small-n leaders.
  4. [Figure 1; Table 2] Minor clarity: define WGA at first use in the main text before Fig. 1’s caption reliance; fix “gopt” / naming consistency for SigLIP2 variants if space allows in tables.
  5. [§2] Related work could briefly situate UrbanCars’ multi-shortcut “whac-a-mole” finding [25] against the paper’s pretraining-design focus so the novelty boundary is sharper.

Circularity Check

0 steps flagged

No significant circularity: empirical audit with external checkpoints and established benchmarks; correlations and controlled deltas are measured, not derived from fitted inputs.

full rationale

This paper is a large-scale observational and controlled empirical study, not a first-principles derivation. Its load-bearing claims are Spearman correlations (parameters/data size vs ImageNet acc., CelebA WGA, UrbanCars WGA) and mean deltas from matched groups where only one design factor varies (Tables 2, 4, 5). WGA is the standard min-group accuracy (Eq. 2); zero-shot inference is the standard CLIP protocol (Eq. 1). Neither quantity is fitted to produce the reported decay of scale’s association, nor is any parameter estimated on a subset and then ‘predicted’ on a closely related target. Benchmarks (ImageNet, CelebA, UrbanCars) and the OpenCLIP checkpoint population are external to the authors’ construction of the result. Author self-citations appear only in Related Work (prior bias-mitigation papers) and do not justify the central measurements. Coining ‘bias complexity sensitivity’ merely names the observed ρ decay (0.68→0.48→0.05); renaming an empirical pattern is not circular when the pattern is independently computed. Scope limits (two bias benchmarks, ≤3.6B public models, fixed prompts) are validity concerns, not circularity. Conclusion: derivation chain is self-contained measurement; score 0.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 1 invented entities

Load-bearing content is empirical measurement, not a derivation from deep axioms. The claim rests on standard zero-shot VLM evaluation assumptions, the validity of WGA on two chosen group-structured benchmarks as a ladder of “bias complexity,” and the adequacy of matched public checkpoints as quasi-controlled experiments. No physical constants or fitted universal scales; prompt templates and benchmark choice are the main discretionary knobs.

free parameters (2)
  • Zero-shot prompt template sets (CelebA hair descriptors; UrbanCars subtype lists; ImageNet 80 templates) = Hand-specified descriptor lists (e.g., blonde/light blonde/golden/platinum vs dark/black/brown/red/grey/auburn + brunett
    Prompt wording is an author choice that defines the classifier in Eq. (1); the paper partially checks sensitivity with ImageNet-style templates in Discussion but primary numbers use the custom sets in §3.6.
  • Matched-group inclusion criteria for “isolated” factor comparisons = 11 scale groups (26 comparisons); 7 patch groups; 25 resolution comparisons; 7 data groups
    Which checkpoint pairs count as holding family/data/resolution/patch fixed is a design choice that determines the +2.63%/−4.22% mean effects in Tables 2, 4, 5.
axioms (5)
  • domain assumption Zero-shot cosine-similarity classification with class text prompts (Eq. 1) is the right exposure of pretraining bias for deployment-relevant evaluation.
    Stated in §3.1 and §3.6; Discussion notes this is standard deployment mode with no task-specific correction.
  • domain assumption Worst-group accuracy on known group structures is the appropriate robustness metric for spurious correlations.
    §3.2 cites the group-robustness literature (e.g., Sagawa et al.); WGA is used as the bias-aware metric throughout.
  • domain assumption CelebA (M=1) and UrbanCars (M=2) form a meaningful progression of spurious-correlation complexity, including combinatorial group growth.
    §3.3 and the “bias complexity sensitivity” framing; UrbanCars is synthetic composition, which is accepted as harder multi-shortcut stress test.
  • domain assumption Public OpenCLIP checkpoints with reported metadata allow approximate causal isolation when only one declared factor varies.
    §4.2 controlled comparisons; training recipes may still differ in unreported hyperparameters, optimization, or filtering details.
  • standard math Spearman rank correlation is an appropriate summary of monotone association between design factors and metrics across a heterogeneous model population.
    Table 1 and Figs. 2–3; standard nonparametric association measure with reported p-value stars.
invented entities (1)
  • bias complexity sensitivity no independent evidence
    purpose: Name the observed progressive breakdown of VLM scaling-law-style associations as evaluation moves from average accuracy to single- then multi-attribute WGA.
    Introduced in Introduction/Results as a term for the empirical pattern (ρ decay and controlled scale harm on UrbanCars). It is a label for measurements, not a new physical mechanism; independent evidence is exactly the reported correlations and matched comparisons on these benchmarks.

pith-pipeline@v1.2.0-daily-grok45 · 21029 in / 3808 out tokens · 80209 ms · 2026-07-31T14:30:50.201818+00:00 · methodology

0 comments
read the original abstract

Vision-Language Models (VLMs) such as CLIP are now foundational to multimodal systems, yet their robustness to spurious correlations remains poorly understood at scale. We present the first large-scale empirical study of 194 publicly available VLMs, including 16 model families, covering a wide range of model sizes, 24 training datasets, and three evaluation benchmarks, namely ImageNet (overall performance), CelebA (typical single-attribute bias), and UrbanCars (complex multi-attribute biases). Across these settings, the Spearman correlation between model scale and performance weakens as evaluation shifts from ImageNet ($\rho{=}0.68$) to single-attribute ($\rho{=}0.48$) and further to multi-attribute ($\rho{=}0.05$) bias benchmarks. In contrast, properties of the training data (size and quality) show more consistent relationships with worst-group accuracy across both bias benchmarks. Notably, curated datasets yield improvements of up to 25% over uncurated alternatives at a comparable scale. Finally, the effect of architectural choices (e.g., patch size, image resolution) is highly context-dependent, varying with the nature of the benchmark, including the type of bias and its spatial distribution within images.

Figures

Figures reproduced from arXiv: 2607.28211 by Ioannis Kompatsiaris, Ioannis Sarridis, Symeon Papadopoulos.

Figure 1
Figure 1. Figure 1: Model scale — the dominant predictor of over [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The bias complexity sensitivity through model scale. The trend progressively flattens from left to right: parameters [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Training data size across benchmarks. Unlike model parameters (Figure 2), the positive trend persists on UrbanCars [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Isolated training data effect: per-dataset accuracy across matched architecture groups. Small semi-transparent dots [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Effect of patch and image size on model performance. Higher resolution is associated with improved performance [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 5 linked inside Pith

  1. [1]

    Sandhini Agarwal, Gretchen Krueger, Jack Clark, Alec Radford, Jong Wook Kim, and Miles Brundage. 2021. Evaluating CLIP: Towards characterization of broader capabilities and downstream implications.arXiv preprint arXiv:2108.02818(2021)

    Pith/arXiv arXiv 2021

  2. [2]

    Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. 2018. Don’t just assume; look and answer: Overcoming priors for visual question answering. InIEEE/CVF Conference on Computer Vision and Pattern Recognition

    2018

  3. [3]

    Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. 2019. Invariant risk minimization.arXiv preprint arXiv:1907.02893(2019)

    Pith/arXiv arXiv 2019

  4. [4]

    Sara Beery, Grant Van Horn, and Pietro Perona. 2018. Recognition in terra incognita. InEuropean Conference on Computer Vision

    2018

  5. [5]

    Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, et al. 2025. Perception encoder: The best visual embeddings are not at the output of the network.arXiv preprint arXiv:2504.13181(2025). Scaling Vision-Language Models Is Not Enough to Mitigate Bias

    Pith/arXiv arXiv 2025

  6. [6]

    Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim. 2022. COYO-700M: Image-Text Pair Dataset. https://github. com/kakaobrain/coyo-dataset

    2022

  7. [7]

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. 2021. Concep- tual 12M: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. InIEEE/CVF Conference on Computer Vision and Pattern Recognition

    2021

  8. [8]

    Jieneng Chen, Qihang Yu, Xiaohui Shen, Alan Yuille, and Liang-Chieh Chen

  9. [9]

    Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al

  10. [10]

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev

  11. [11]

    Yung-Sung Chuang, Yang Li, Dong Wang, Ching-Feng Yeh, Kehan Lyu, Ramya Raghavendra, Jim Glass, Lifei Huang, Jason Weston, Luke Zettlemoyer, et al

  12. [12]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guil- laume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. InProceedings of the 58th annual meeting of the association for computa- tional linguistics. 8440–8451

    2020

  13. [13]

    In IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Reproducible scaling laws for contrastive language-image learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition

  14. [14]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Im- ageNet: A large-scale hierarchical image database. InIEEE/CVF Conference on Computer Vision and Pattern Recognition

    2009

  15. [15]

    Fartash Faghri, Pavan Kumar Anasosalu Vasu, Cem Koc, Vaishaal Shankar, Alexander T Toshev, Oncel Tuzel, and Hadi Pouransari. 2025. MobileCLIP2: Improving Multi-Modal Reinforced Training.Transactions on Machine Learning Research(2025). https://openreview.net/forum?id=WeF9zolng8

    2025

  16. [16]

    Alex Fang, Gabriel Ilharco, Mitchell Wortsman, Yuhao Wan, Vaishaal Shankar, Achal Dave, and Ludwig Schmidt. 2022. Data determines distributional robustness in contrastive language image pre-training (CLIP). InInternational Conference on Machine Learning

    2022

  17. [17]

    Elliot Creager, Jörn-Henrik Jacobsen, and Richard Zemel. 2021. Environment inference for invariant learning. InInternational Conference on Machine Learning

    2021

  18. [18]

    Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. 2023. EVA: Exploring the limits of masked visual representation learning at scale. InIEEE/CVF Conference on Computer Vision and Pattern Recognition

    2023

  19. [19]

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. 2024. DataComp: In search of the next generation of multimodal datasets. InAdvances in Neural Information Processing Systems

    2024

  20. [20]

    Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A Wichmann. 2020. Shortcut learning in deep neural networks.Nature Machine Intelligence2, 11 (2020)

    2020

  21. [21]

    Alex Fang, Albin Madappally Jose, Amit Jain, Ludwig Schmidt, Alexander Toshev, and Vaishaal Shankar. 2024. Data filtering networks. InInternational Conference on Learning Representations

    2024

  22. [22]

    Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. 2023. Last layer re-training is sufficient for robustness to spurious correlations. InInternational Conference on Learning Representations

    2023

  23. [23]

    Chunyuan Li, Haotian Liu, Liunian Harold Li, Pengchuan Zhang, Jyoti Aneja, Jianwei Yang, Ping Jin, Yong Jae Lee, Houdong Hu, Zicheng Liu, and Jianfeng Gao

  24. [24]

    Xianhang Li, Zeyu Wang, and Cihang Xie. 2023. An Inverse Scaling Law for CLIP Training.Advances in Neural Information Processing Systems36 (2023)

    2023

  25. [25]

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision- language representation learning with noisy text supervision. InInternational Conference on Machine Learning

    2021

  26. [26]

    Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. 2021. Just train twice: Improving group robustness without training group labels. InInternational Conference on Machine Learning

    2021

  27. [27]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2020. RoBERTa: A Robustly Optimized BERT Pretraining Approach. InInternational Conference on Learning Representations

    2020

  28. [28]

    Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. 2015. Deep learning face attributes in the wild. InInternational Conference on Computer Vision

    2015

  29. [29]

    Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A ConvNet for the 2020s. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition

    2022

  30. [30]

    Zhiheng Li, Ivan Evtimov, Albert Gordo, Caner Hazirbas, Tal Hassner, Cris- tian Canton Ferrer, Chenliang Xu, and Mark Ibrahim. 2023. A whac-a-mole dilemma: Shortcuts come in multiples where mitigating one amplifies others. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion

    2023

  31. [31]

    John P Miller, Rohan Taori, Aditi Raghunathan, Shiori Sagawa, Pang Wei Koh, Vaishaal Shankar, Percy Liang, Yair Carmon, and Ludwig Schmidt. 2021. Ac- curacy on the line: On the strong correlation between out-of-distribution and in-distribution generalization. InInternational Conference on Machine Learning

    2021

  32. [32]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. InInternational Conference on Machine Learning

    2021

  33. [33]

    Shiori Sagawa, Pang Wei Koh, Tatsunori B Hashimoto, and Percy Liang. 2020. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. InInternational Conference on Learning Representations

    2020

  34. [35]

    Tom McCoy, Ellie Pavlick, and Tal Linzen. 2019. Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference. InAnnual Meeting of the Association for Computational Linguistics

    2019

  35. [36]

    Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, and Christos Diou. 2025. FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class Associations.IEEE Transactions on Pattern Analysis and Machine Intelligence47, 2 (2025)

    2025

  36. [37]

    Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, and Christos Diou. 2025. MAVias: Mitigate any Visual Bias. InProceedings of the IEEE/CVF International Conference on Computer Vision

    2025

  37. [38]

    Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, and Christos Diou

  38. [39]

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. 2022. LAION-5B: An open large-scale dataset for training next generation image-text models.Advances in Neural Information Processing Systems (2022)

    2022

  39. [40]

    InProceedings of the 2024 International Conference on Multimedia Retrieval

    Facex: Understanding face attribute classifiers through summary model explanations. InProceedings of the 2024 International Conference on Multimedia Retrieval

    2024

  40. [42]

    Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. 2025. SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features.arXiv preprint arXiv:2502.14786(2025)

    Pith/arXiv arXiv 2025

  41. [43]

    Pavan Kumar Anasosalu Vasu, Hadi Pouransari, Fartash Faghri, Raviteja Vem- ulapalli, and Oncel Tuzel. 2024. MobileCLIP: Fast image-text models through multi-modal reinforced training. InIEEE/CVF Conference on Computer Vision and Pattern Recognition

    2024

  42. [44]

    Alexander Visheratin. 2023. NLLB-CLIP—train performant multilingual image retrieval model on a budget.arXiv preprint arXiv:2309.01859(2023)

    Pith/arXiv arXiv 2023

  43. [45]

    Jialu Wang, Yang Liu, and Xin Wang. 2021. Are gender-neutral queries really gender-neutral? Mitigating gender bias in image search. InProceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

    2021

  44. [46]

    VB-Mitigator: An Open-source Framework for Evaluating and Advancing Visual Bias Mitigation.In 3nd AEQUITAS Workshop on Fairness and Bias in AI at ECAI 2025(2025)

    2025

  45. [47]

    Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feicht- enhofer. 2024. Demystifying CLIP data. InInternational Conference on Learning Representations

    2024

  46. [48]

    Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. 2020. Measuring robustness to natural distribution shifts in image classification. InAdvances in Neural Information Processing Systems

    2020

  47. [49]

    Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 2016. YFCC100M: The new data in multimedia research.Commun. ACM59, 2, 64–73

    2016

  48. [50]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sig- moid loss for language image pre-training. InInternational Conference on Com- puter Vision

    2023

  49. [54]

    Kai Xiao, Logan Engstrom, Andrew Ilyas, and Aleksander Madry. 2021. Noise or signal: The role of image backgrounds in object recognition. InInternational Conference on Learning Representations

    2021

  50. [56]

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. 2022. CoCa: Contrastive Captioners are Image-Text Foundation Models.Transactions on Machine Learning Research(2022). https://openreview. net/forum?id=Ee277P3AYC

    2022

  51. [57]

    Daochen Zha, Zaid Pervaiz Bhat, Kwei-Herng Lai, Fan Yang, Zhimeng Jiang, Shaochen Zhong, and Xia Hu. 2023. Data-centric artificial intelligence: A survey. Comput. Surveys(2023). Sarridis et al

    2023

  52. [2022]

    InAdvances in Neural Information Processing Systems

    ELEVATER: A benchmark and toolkit for evaluating language-augmented visual models. InAdvances in Neural Information Processing Systems

  53. [2023]

    InInternational Conference on Learning Representations

    PaLI: A jointly-scaled multilingual language-image model. InInternational Conference on Learning Representations

  54. [2024]

    In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    ViTamin: Designing scalable vision models in the vision-language era. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

  55. [2025]

    InProceedings of the IEEE/CVF International Conference on Computer Vision

    Badd: Bias mitigation through bias addition. InProceedings of the IEEE/CVF International Conference on Computer Vision

  56. [2026]

    Meta CLIP 2: A Worldwide Scaling Recipe.Advances in Neural Information Processing Systems(2026)

    2026