Pith. sign in

REVIEW 5 major objections 7 minor 46 references

BioPro shows that vision-language models can be selectively debiased: the projection removes gender bias from ambiguous inputs while preserving explicit gender cues, all without retraining.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

BioPro uses orthogonal projection on a gender-variation subspace to selectively debias vision-language models, reducing gender bias in neutral contexts while preserving explicit gender cues.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection The difference-aware formulation and closed-form calibration are worth a look, but Table 1 contradicts the faithfulness claim and the generation results are tuned per test category, so the headline overstates what is shown. the 5 major comments →

arxiv 2512.00807 v2 pith:GT6NU7VV submitted 2025-11-30 cs.AI

BioPro: Towards Difference-Aware Gender Fairness for Vision-Language Models

Yujie Lin , Jiayao Ma , Qingguo Hu , Wenbo Li , Genji Li , Derek Wong , Jinsong Su This is my paper
classification cs.AI
keywords difference-aware fairnessgender biasvision-language modelsorthogonal projectioncounterfactual embeddingstext-to-image generationimage captioningtraining-free debiasing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that fairness interventions in vision-language models should be difference-aware: they should suppress gender stereotypes when the input is ambiguous (such as a person seen from behind), but preserve genuine gender information when it is clearly visible. To achieve this, the authors propose BioPro, a training-free method that estimates a low-dimensional gender-variation subspace from paired counterfactual images or prompts that differ only in gender, then projects embeddings onto the orthogonal complement of that subspace. A selection threshold ensures the projection is applied mainly to neutral samples, while a calibration term balances gender proportions in text-to-image generation. Experiments on captioning and generation backbones report reduced gender bias in neutral cases with negligible loss of explicit gender faithfulness and minimal impact on semantic quality. The same technique is extended to a continuous bias variable, scene brightness, suggesting broader applicability to other bias dimensions.

Core claim

On its own terms, the paper establishes that gender information in VLM embeddings occupies a low-dimensional subspace that can be identified from paired counterfactual examples, and that removing that subspace by orthogonal projection yields selective debiasing: neutral inputs no longer receive stereotyped gender words or skewed gender distributions, while explicit inputs retain their specified gender. The projection-based selection mechanism, which thresholds on the magnitude of the embedding's projection onto the gender subspace, prevents over-correction. For text-to-image generation, a calibration term with a closed-form solution further balances the generated gender ratio. The authors de

What carries the argument

The gender-variation subspace S_c is computed by singular value decomposition of the difference matrix D_c = H_m − H_f built from paired male/female multimodal embeddings taken from the SCFs counterfactual image dataset; its top-k left singular vectors span S_c. The debiasing transform is the orthogonal projector P⊥ = I − U_k U_k^T onto the orthogonal complement of S_c, which removes the gender component while leaving the orthogonal semantic component intact. A skew-normal fit to projection magnitudes of neutral versus explicit samples yields a threshold δ_c that decides when to apply the projector. For generation, the projection matrix is refined by a calibration term (with closed-form solu

Load-bearing premise

The method assumes the difference between paired male and female counterfactual embeddings isolates a pure gender-variation subspace, so that projecting it away removes gender while leaving all other semantics untouched.

What would settle it

Take a diverse set of neutral images whose subjects differ systematically from the counterfactual training pairs in pose, background, or clothing, and measure whether the projection still removes gender bias without degrading caption accuracy; a significant drop in semantic-preservation metrics (e.g., METEOR or CLIP score) or residual bias beyond chance would indicate that the gender subspace is entangled with other attributes.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If BioPro works as claimed, VLMs can be made fairer at inference time without fine-tuning, reducing the cost and risk of debiasing.
  • Selective debiasing means captioning models could stop mis-gendering ambiguous subjects (e.g., people seen from behind) without losing correct gender terms for clear photos.
  • Text-to-image generation from neutral prompts could be balanced across gender, countering stereotyped outputs like chef→male and nurse→female, while explicit prompts remain obeyed.
  • The same projection machinery generalizes to non-social continuous attributes such as scene brightness, potentially increasing the diversity of generated images.
  • The closed-form calibration solution keeps the debiasing transform cheap to compute and re-tunable per concept, which is useful for practical deployment at scale.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A key untested implication is whether the gender subspace estimated from SCFs counterfactual images is portable across different base models: the paper constructs the subspace separately for each backbone, so cross-model transfer of a single subspace remains an open question.
  • The selection threshold assumes that projection magnitude alone separates neutral from explicit samples; in harder cases such as partial occlusion or diverse cultural presentations of gender, the threshold could misfire and either leave bias behind or over-correct explicit samples.
  • The scene-brightness result hints that the same orthogonal-projection recipe could apply to other continuous bias axes (age, skin tone, scene type), but the paper only demonstrates brightness; a test of whether those axes are as cleanly separable in embedding space is still open.
  • Because BioPro operates on fused embeddings for captioning, its effectiveness depends on the model's internal representation of gender being largely orthogonal to other semantics; models with heavily entangled representations might need a different intervention.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper formalizes difference-aware gender fairness for vision-language models, distinguishing neutral (gender-ambiguous) inputs where gender bias should be suppressed from explicit (gender-evident) inputs where gender should be preserved. It proposes BioPro, a training-free framework that constructs a low-dimensional gender-variation subspace from counterfactual embedding differences via SVD, removes that subspace by orthogonal projection, and adds a projection-based selection threshold for captioning and a calibration term for text-to-image generation. Experiments on LLaVA-1.5/LLaVA-NeXT captioning and FLUX.1-dev/FLUX.1-schnell generation report reduced gender bias on neutral samples while claiming preserved explicit gender semantics, and the method is further extended to continuous scene-brightness bias. The central claim is that BioPro achieves selective debiasing at inference time without training.

Significance. If the claims hold, BioPro would be a useful training-free intervention for selective fairness in multimodal systems, with a clean formulation of neutral fairness, explicit faithfulness, and semantic preservation, and a closed-form calibration solution (Lemma 1, Appendix C). The paper also raises an important distinction between difference-unaware and difference-aware fairness in the multimodal setting, extending recent text-only work. The generalization to continuous bias variables such as brightness is a novel and potentially valuable direction. However, the current evidence does not fully support the headline claims: the captioning results contradict the explicit-faithfulness constraint as stated, the generation results rely on per-category calibration coefficients, and the scene-bias evaluation lacks a quantitative brightness metric. These issues are addressable in revision, but they are load-bearing for the paper's main conclusions.

major comments (5)
  1. [Section 5.2, Table 1 / Eq. (2)] The explicit gender-faithfulness constraint in Eq. (2) requires the ratio P(w_g | M_c(tilde h(x_e))) / P(w_g | M_c(h(x_e))) to be approximately 1. The reported BR_e values contradict this: for LLaVA-1.5, BR_e drops from 80.27 to 68.74 (ratio 0.86), and for LLaVA-NeXT from 72.55 to 64.06 (ratio 0.88). An 11–15 percentage point reduction means a substantial fraction of explicit images lose their gender word, so the headline claim of 'maintaining gender faithfulness in explicit ones' is not supported by the numbers. The CBR metric in Eq. (17) collapses neutral fairness and explicit faithfulness into one scalar and hides this loss. Please report per-sample statistics on explicit images that are altered, the distribution of projection magnitudes for explicit samples around the threshold, and a direct evaluation of Eq. (2).
  2. [Section 6.1 / Appendix D.1, Table 6] The calibration coefficient lambda_g is chosen separately for each of the ten test professions (e.g., 0.1 for farmer, 50 for driver, 25 for banker, 100 for chef) and for each scene. Because the reported Skew values are computed on exactly these categories for which lambda_g was fitted, the main generation result partly reflects per-category coefficient selection rather than a generic debiasing rule. The paper gives no selection criterion for lambda_g, no sensitivity analysis for this parameter, and no evaluation on held-out professions with a fixed lambda_g. Please provide a validation protocol or demonstrate robustness across a range of lambda_g values and unseen categories.
  3. [Section 7 / Fig. 5] The scene-bias generalization claim is not quantitatively supported. The only reported number is CLIP Score (Table 3), which does not measure brightness. There is no metric for illumination level, no distribution comparison, and no comparison against the base model's brightness distribution. Without a quantitative brightness metric (e.g., mean luminance or perceptual darkness score) and a controlled comparison across lambda_g values, the statement that BioPro 'can effectively control the model to generate more scenes with lower brightness' is not verifiable from the presented evidence.
  4. [Section 4.1 / Eqs. (9)–(13)] The method assumes that the difference between paired male/female embeddings from SCFs isolates a pure, low-dimensional gender-variation subspace, so that projection onto its orthogonal complement removes gender while preserving all other semantics. This is not verified. The paper relies on SCFs being 'similar in all features except gender,' but no analysis is provided to show that the top-k singular vectors of D_c do not also capture pose, background, or other attributes. The semantic-preservation evidence is only aggregate METEOR/CLIP scores; please include per-sample or distributional checks, such as comparing captions of the same explicit image with and without projection and measuring changes in non-gender content.
  5. [Section 4.1 / Eq. (14)] The projection threshold delta_c is obtained by fitting skew-normal distributions to the validation set and optimizing with a manually chosen lambda_c. This is a fitted hyperparameter, not a parameter-free procedure. The paper should report how delta_c transfers across random splits or datasets, and provide confidence intervals for BR_n and BR_e. The sensitivity analysis in Table 7 varies (k, lambda_c) on the same validation-based protocol, but does not assess the robustness of the fitted threshold itself.
minor comments (7)
  1. [Throughout] The model name is inconsistently written as 'LLaV A-1.5' and 'LLaV A-1.6'; the standard spelling is 'LLaVA-1.5' and 'LLaVA-NeXT' (or LLaVA-1.6).
  2. [References] Reference [3] has a garbled author string: 'Aleksandar Kirk, Hannah andShtedritski' should be corrected to the proper author list.
  3. [Appendix D.1, Table 5] The table heading uses '∆ c' while the text uses 'δ_c'; the notation should be unified.
  4. [Figures 4 and 7] The heatmaps lack colorbars and axis labels, and the 'white boundary' mentioned in the text is not clearly visible. Please add color scales and annotations.
  5. [Eq. (14)] The integrals are taken from 0 to δ for the absolute projection values. Justify the lower limit of 0 and the use of a skew-normal distribution for absolute values (or explain that a folded distribution is intended).
  6. [Section 5.1 / Eq. (17)] BR_base_e is defined, but BR_base_n is never introduced. Clarify that the CBR uses the base model's explicit rate only, and state why the base neutral rate is not part of the composite.
  7. [Section 6 / Table 2] The MR column reports 0.1% and 0.2% for some methods. State the number of explicit prompts/images used to compute MR and provide confidence intervals, given that only 100 images are generated per prompt.

Circularity Check

1 steps flagged

Text-to-image balancing is partly a fitted calibration effect; captioning and subspace derivation are independent. Reported BR_e drops contradict Eq. 2, but that is a consistency issue, not circularity.

specific steps
  1. fitted input called prediction [Section 4.2, Eq. 15; Appendix D.1 Table 6; Section 6.2 Table 2]
    "Since the magnitude of inherent bias varies across different concepts, we adopt category-specific λ_g values. ... This adjustment helps balance the gender distribution of generated images and mitigates residual gender bias."

    The generation-balance result is an optimization target, not an independent prediction. Table 6 assigns a different λ_g to each test profession/scene, and the calibration term in Eq. 15 directly minimizes ||PZ_f − Z_m||, i.e., it drives female-prompt embeddings toward male-prompt embeddings. Reporting reduced Skew for those same professions in Table 2 therefore restates the fitted calibration objective: the debiasing strength is tuned per concept before the metric is computed. The captioning side does not share this issue, so the circularity is partial.

full rationale

BioPro's captioning derivation is largely self-contained: S_c is obtained by SVD on external SCFs counterfactual pairs (Eq. 9-10), P⊥ (Eq. 11) is a standard orthogonal projector, and δ_c (Eq. 14) is fit on the MS-COCO validation split and evaluated on a disjoint test split, which is ordinary hyperparameter selection rather than circularity. The explicit/neutral decomposition in Eq. 12-13 is definitional linear algebra, not a proof of semantic preservation; that remains an empirical assumption (a correctness risk, not circularity). The only partial circularity is in text-to-image generation: the calibration term in Eq. 15 is explicitly constructed to balance gender (λ_g controls its strength), and Table 6 sets λ_g per test profession/scene, so the Skew improvements in Table 2 are partly a fitted effect rather than an emergent prediction. Separately, I flag an internal-consistency problem: Table 1 shows BR_e dropping from 80.27 to 68.74 for LLaVA-1.5 and 72.55 to 64.06 for LLaVA-NeXT, ratios of 0.86 and 0.88, far from Eq. 2's ≈1 requirement; CBR (Eq. 17) can mask this by combining squared deviations. This contradicts the headline faithfulness claim but is a correctness/evidence issue, not a circular-derivation issue. Reference [40] is a self-citation by one author but is only a general bias-survey citation in the Introduction and is not load-bearing; it does not raise the score.

Axiom & Free-Parameter Ledger

3 free parameters · 6 axioms · 0 invented entities

The method rests on empirically fitted hyperparameters (subspace rank k, selection threshold delta_c via lambda_c, and per-category calibration lambda_g) and on assumptions that counterfactual differences isolate gender and that orthogonal projection preserves semantics. No new physical or ontological entities are introduced.

free parameters (3)
  • k (gender subspace dimension) = 2 (main; 1 and 4 tested in sensitivity analysis)
    Top-k SVD dimensions chosen to optimize CBR; sensitivity analysis Table 7 shows it affects results.
  • lambda_c (selection threshold trade-off) = 2, 3, 4; main lambda_c=3 giving delta_c=8.70 (LLaVA-1.5) and 19.78 (LLaVA-1.6)
    Chosen on MS-COCO validation set to balance neutral coverage vs explicit preservation; directly sets which samples are projected.
  • lambda_g (calibration coefficient for T2I) = Per category: farmer 0.1, driver 1/50, banker 25/50, chef 100, police 100, cashier 1, teacher 1, secretary 2, assistant
    Hand-tuned per test profession/scene to produce balanced outputs; a different value for each concept means the method is not parameter-free.
axioms (6)
  • domain assumption SCFs counterfactual image pairs differ only in gender, so the top SVD directions of H_m - H_f form a pure gender-variation subspace.
    Section 4.1: SVD on D_c = H_m - H_f; if counterfactual pairs also differ in pose/background, the subspace captures non-gender variation.
  • domain assumption Orthogonal projection onto S_c^orthogonal removes gender information without removing task-relevant semantics.
    Eq. 12-13 assume H = H_bias + H_sem with H_bias in S_c and H_sem in S_c^orthogonal; real embeddings may have entangled gender and semantic features.
  • domain assumption Explicit samples have larger projection magnitudes onto U_c than neutral samples, enabling a threshold.
    Section 4.1 'Projection-Based Selection'; this is an empirical observation fitted to validation, not a proven separation.
  • domain assumption For T2I, mapping female prompt embeddings toward male embeddings with P_f->m balances neutral prompts while preserving explicit prompts.
    Section 4.2; relies on linearity of text embedding geometry and on lambda_g being small enough for explicit cases.
  • domain assumption Gender can be treated as binary in evaluation (male/female classifier).
    Metrics Skew and MR use BLIP-2 binary gender classification; non-binary gender is outside scope.
  • standard math Standard linear algebra (SVD, convex quadratic optimum) is valid.
    Eq. 9 and Lemma 1 use standard SVD and matrix calculus; no issue.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of BioPro: Towards Difference-Aware Gender Fairness for Vision-Language Models." pith.science (2026). https://pith.science/paper/GT6NU7VV

@misc{pith2026251200807,
  author       = {Pith},
  title        = {Pith review of: BioPro: Towards Difference-Aware Gender Fairness for Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GT6NU7VV}},
  note         = {Machine review of arXiv:2512.00807}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Vision-Language Models (VLMs) inherit significant social biases from their training data, notably in gender representation. Current fairness interventions often adopt a difference-unaware perspective that enforces uniform treatment across demographic groups. These approaches, however, fail to distinguish between contexts where neutrality is required and those where group-specific attributes are legitimate and must be preserved. Building upon recent advances in difference-aware fairness for text-only models, we extend this concept to the multimodal domain and formalize the problem of difference-aware gender fairness for image captioning and text-to-image generation. We advocate for selective debiasing, which aims to mitigate unwanted bias in neutral contexts while preserving valid distinctions in explicit ones. To achieve this, we propose BioPro (Bias Orthogonal Projection), an entirely training-free framework. BioPro identifies a low-dimensional gender-variation subspace through counterfactual embeddings and applies projection to selectively neutralize gender-related information. Experiments show that BioPro effectively reduces gender bias in neutral cases while maintaining gender faithfulness in explicit ones, thus providing a promising direction toward achieving selective fairness in VLMs. Beyond gender bias, we further demonstrate that BioPro can effectively generalize to continuous bias variables, such as scene brightness, highlighting its broader applicability.

Figures

Figures reproduced from arXiv: 2512.00807 by Derek Wong, Genji Li, Jiayao Ma, Jinsong Su, Qingguo Hu, Wenbo Li, Yujie Lin.

Figure 1
Figure 1. Figure 1: BioPro can selectively perform debiasing primarily on [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An example of text-to-image generation. The figure shows eight images generated from the same prompt with fixed seeds ranging [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of BioPro on two multimodal tasks. For the image captioning task, the feature space of the samples is divided into [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Absolute values after projection onto Uc when using the FLUX.1-dev base model. complement S ⊥ c . Applying the projection matrix (I − Uk c (Uk c ) ⊤) yields H′ = P⊥H = Hsem. (13) Hence, the operation removes Hbias while preserving Hsem, ensuring that semantic information (e.g., objects and ac￾tions) remains intact. Since the dimension k of Sc is small (k ≪ d), the impact on the overall representational cap… view at source ↗
Figure 5
Figure 5. Figure 5: Debiasing scene bias. The figure shows eight images generated from the same prompt with fixed seeds ranging from 0 to 7. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Skew-normal distribution modeling of the absolute projection values on the first dimension of [PITH_FULL_IMAGE:figures/full_fig_p014_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Absolute values after projection onto Uc when using the FLUX.1-schnell base model. λg = 1, the model generates scenes with moderate lighting or overcast conditions. Increasing the strength to λg = 100 effectively shifts the distribution towards darker, night-time scenes, confirming that our projection framework can ma￾nipulate continuous attributes along the identified bias sub￾space. F. All the Prompts Us… view at source ↗
Figure 8
Figure 8. Figure 8: Qualitative comparison of text-to-image generation for various professions. For each profession, the top row shows samples [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative results for mitigating scene bias (brightness). The top row (Initial) shows the default high-brightness bias. The [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 13 linked inside Pith

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [2]

    Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments

    Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with hu- man judgments. InProceedings of the acl workshop on in- trinsic and extrinsic evaluation measures for machine trans- lation and/or summarization, pages 65–72, 2005. 6

    2005

  3. [3]

    A prompt array keeps the bias away: Debiasing vision-language models with ad- versarial learning

    Hugo Berg, Siobhan Hall, Yash Bhalgat, Aleksandar Kirk, Hannah andShtedritski, and Max Bain. A prompt array keeps the bias away: Debiasing vision-language models with ad- versarial learning. InProceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Compu- tational Linguistics and the 12th International Joint Confer- ence on Na...

    2022

  4. [4]

    In- structpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. In- structpix2pix: Learning to follow image editing instructions. InProceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 18392–18402, 2023. 5

    2023

  5. [5]

    Dall-eval: Probing the reasoning skills and social biases of text-to- image generation models

    Jaemin Cho, Abhay Zala, and Mohit Bansal. Dall-eval: Probing the reasoning skills and social biases of text-to- image generation models. InProceedings of the IEEE/CVF international conference on computer vision, pages 3043– 3054, 2023. 7

    2023

  6. [6]

    Debiasing vision- language models via biased prompts.arXiv preprint arXiv:2302.00070, 2023

    Ching-Yao Chuang, Varun Jampani, Yuanzhen Li, Anto- nio Torralba, and Stefanie Jegelka. Debiasing vision- language models via biased prompts.arXiv preprint arXiv:2302.00070, 2023. 3, 7, 11

    Pith/arXiv arXiv 2023

  7. [7]

    The llama 3 herd of models.arXiv e-prints, pages arXiv–2407,

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv e-prints, pages arXiv–2407,

  8. [8]

    Fair diffusion: Instructing text-to- image generation models on fairness.arXiv preprint arXiv:2302.10893, 2023

    Felix Friedrich, Manuel Brack, Lukas Struppek, Dominik Hintersdorf, Patrick Schramowski, Sasha Luccioni, and Kristian Kersting. Fair diffusion: Instructing text-to- image generation models on fairness.arXiv preprint arXiv:2302.10893, 2023. 3

    Pith/arXiv arXiv 2023

  9. [9]

    Fairimagen: Post-processing for bias mitigation in text-to-image models.arXiv preprint arXiv:2510.21363, 2025

    Zihao Fu, Ryan Brown, Shun Shao, Kai Rawal, Eoin De- laney, and Chris Russell. Fairimagen: Post-processing for bias mitigation in text-to-image models.arXiv preprint arXiv:2510.21363, 2025. 3, 7, 11

    arXiv 2025

  10. [10]

    Bias and fairness in large language models: A survey.Computational Linguistics, 50 (3):1097–1179, 2024

    Isabel O Gallegos, Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and fairness in large language models: A survey.Computational Linguistics, 50 (3):1097–1179, 2024. 1

    2024

  11. [11]

    Unified concept editing in diffusion models

    Rohit Gandikota, Hadas Orgad, Yonatan Belinkov, Joanna Materzy´nska, and David Bau. Unified concept editing in diffusion models. InProceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision, pages 5111–5120, 2024. 3

    2024

  12. [12]

    The capacity for moral self-correction in large language models

    Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I Liao, Kamil ˙e Lukoˇsi¯ut˙e, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, et al. The capacity for moral self-correction in large language models. arXiv preprint arXiv:2302.07459, 2023. 3

    Pith/arXiv arXiv 2023

  13. [13]

    Bendvlm: Test-time debiasing of vision-language em- beddings.Advances in Neural Information Processing Sys- tems, 37:62480–62502, 2024

    Walter Gerych, Haoran Zhang, Kimia Hamidieh, Eileen Pan, Maanas K Sharma, Tom Hartvigsen, and Marzyeh Ghas- semi. Bendvlm: Test-time debiasing of vision-language em- beddings.Advances in Neural Information Processing Sys- tems, 37:62480–62502, 2024. 3, 7, 11

    2024

  14. [14]

    Women also snowboard: Over- coming bias in captioning models

    Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. Women also snowboard: Over- coming bias in captioning models. InProceedings of the Eu- ropean Conference on Computer Vision, 2018. 3

    2018

  15. [15]

    Clipscore: A reference-free evaluation met- ric for image captioning.arXiv preprint arXiv:2104.08718,

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning.arXiv preprint arXiv:2104.08718,

    Pith/arXiv arXiv

  16. [16]

    Quantify- ing societal bias amplification in image captioning

    Yusuke Hirota, Yuta Nakashima, and Noa Garcia. Quantify- ing societal bias amplification in image captioning. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13440–13449, 2022. 3

    2022

  17. [17]

    Model- agnostic gender debiased image captioning

    Yusuke Hirota, Yuta Nakashima, and Noa Garcia. Model- agnostic gender debiased image captioning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 15191–15200, 2023. 3, 6, 7, 11

    2023

  18. [18]

    Social- counterfactuals: Probing and mitigating intersectional social biases in vision-language models with counterfactual exam- ples

    Phillip Howard, Avinash Madasu, Tiep Le, Gustavo Lujan Moreno, Anahita Bhiwandiwalla, and Vasudev Lal. Social- counterfactuals: Probing and mitigating intersectional social biases in vision-language models with counterfactual exam- ples. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pages 11975–11985,

  19. [19]

    A unified debi- asing approach for vision-language models across modalities and tasks.Advances in Neural Information Processing Sys- tems, 37:21034–21058, 2024

    Hoin Jung, Taeuk Jang, and Xiaoqian Wang. A unified debi- asing approach for vision-language models across modalities and tasks.Advances in Neural Information Processing Sys- tems, 37:21034–21058, 2024. 3, 6, 7, 8, 11

    2024

  20. [20]

    De-stereotyping text-to-image models through prompt tun- ing

    Eunji Kim, Siwon Kim, Chaehun Shin, and Sungroh Yoon. De-stereotyping text-to-image models through prompt tun- ing. InICML 2023 Workshop on Challenges in Deployable Generative AI, 2023. 3

    2023

  21. [21]

    Self-discovering interpretable diffusion latent di- rections for responsible text-to-image generation

    Hang Li, Chengzhi Shen, Philip Torr, V olker Tresp, and Jin- dong Gu. Self-discovering interpretable diffusion latent di- rections for responsible text-to-image generation. InPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12006–12016, 2024. 3

    2024

  22. [22]

    Blip-2: Bootstrapping language-image pre-training with 9 frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with 9 frozen image encoders and large language models. InIn- ternational conference on machine learning, pages 19730– 19742. PMLR, 2023. 8

    2023

  23. [23]

    Fair text-to-image diffusion via fair mapping.Proceedings of the AAAI Conference on Artificial Intelligence, 39(25):26256–26264, 2025

    Jia Li, Lijie Hu, Jingfeng Zhang, Tianhang Zheng, Hua Zhang, and Di Wang. Fair text-to-image diffusion via fair mapping.Proceedings of the AAAI Conference on Artificial Intelligence, 39(25):26256–26264, 2025. 3

    2025

  24. [24]

    Towards understanding and mitigating social biases in language models

    Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Rus- lan Salakhutdinov. Towards understanding and mitigating social biases in language models. InInternational confer- ence on machine learning, pages 6565–6576. PMLR, 2021. 1

    2021

  25. [25]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 5

    2014

  26. [26]

    Intrinsic self-correction for enhanced morality: An anal- ysis of internal mechanisms and the superficial hypothesis

    Guangliang Liu, Haitao Mao, Jiliang Tang, and Kristen John- son. Intrinsic self-correction for enhanced morality: An anal- ysis of internal mechanisms and the superficial hypothesis. InProceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing, pages 16439–16455,

    2024

  27. [27]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023. 2

    2023

  28. [28]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 26296–26306, 2024. 6

    2024

  29. [29]

    Llavanext: Improved reasoning, ocr, and world knowledge, 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llavanext: Improved reasoning, ocr, and world knowledge, 2024. 6

    2024

  30. [30]

    Rectified flow: A simple and deterministic flow modeling framework.arXiv preprint arXiv:2309.03183, 2023

    Xiaohan Liu, Xiangning Chen, Jonathan Ho, Jiaming Song, Tim Salimans, and Stefano Ermon. Rectified flow: A simple and deterministic flow modeling framework.arXiv preprint arXiv:2309.03183, 2023. 7

    Pith/arXiv arXiv 2023

  31. [31]

    Stereoset: Measuring stereotypical bias in pretrained language models

    Moin Nadeem, Anna Bethke, and Siva Reddy. Stereoset: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456, 2020. 1

    Pith/arXiv arXiv 2004

  32. [32]

    Bias in the picture: Benchmarking vlms with social-cue news images and llm-as-judge assessment.arXiv preprint arXiv:2509.19659, 2025

    Aravind Narayanan, Vahid Reza Khazaie, and Shaina Raza. Bias in the picture: Benchmarking vlms with social-cue news images and llm-as-judge assessment.arXiv preprint arXiv:2509.19659, 2025. 3

    arXiv 2025

  33. [33]

    Springer, 2006

    Jorge Nocedal and Stephen J Wright.Numerical optimiza- tion. Springer, 2006. 6

    2006

  34. [34]

    Gpt-5 system card

    OpenAI. Gpt-5 system card. Technical report, OpenAI,

  35. [35]

    Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. Automat- ically correcting large language models: Surveying the land- scape of diverse automated correction strategies.Transac- tions of the Association for Computational Linguistics, 12: 484–506, 2024. 3

    2024

  36. [36]

    Bbq: A hand-built bias benchmark for question answering.arXiv preprint arXiv:2110.08193, 2021

    Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Pad- makumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R Bowman. Bbq: A hand-built bias benchmark for question answering.arXiv preprint arXiv:2110.08193, 2021. 1

    Pith/arXiv arXiv 2021

  37. [37]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  38. [38]

    Fair- cot: Enhancing fairness in text-to-image generation via chain of thought reasoning with multimodal large language mod- els.arXiv preprint arXiv:2406.09070, 2025

    Zahraa Al Sahili, Ioannis Patras, and Matthew Purver. Fair- cot: Enhancing fairness in text-to-image generation via chain of thought reasoning with multimodal large language mod- els.arXiv preprint arXiv:2406.09070, 2025. 3

    Pith/arXiv arXiv 2025

  39. [39]

    Dear: Debiasing vision-language models with additive residuals

    Ashish Seth, Mayur Hemani, and Chirag Agarwal. Dear: Debiasing vision-language models with additive residuals. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6820–6829, 2023. 3

    2023

  40. [40]

    Supervised algorithmic fairness in distribution shifts: A survey.arXiv preprint arXiv:2402.01327, 2024

    Minglai Shao, Dong Li, Chen Zhao, Xintao Wu, Yujie Lin, and Qin Tian. Supervised algorithmic fairness in distribution shifts: A survey.arXiv preprint arXiv:2402.01327, 2024. 1

    Pith/arXiv arXiv 2024

  41. [41]

    Finetuning text-to- image diffusion models for fairness.arXiv preprint arXiv:2311.07604, 2024

    Xudong Shen, Chao Du, Tianyu Pang, Min Lin, Yongkang Wong, and Mohan Kankanhalli. Finetuning text-to- image diffusion models for fairness.arXiv preprint arXiv:2311.07604, 2024. 3

    Pith/arXiv arXiv 2024

  42. [42]

    Fairness through difference awareness: Measur- ing desired group discrimination in llms.arXiv preprint arXiv:2502.01926, 2025

    Angelina Wang, Michelle Phan, Daniel E Ho, and Sanmi Koyejo. Fairness through difference awareness: Measur- ing desired group discrimination in llms.arXiv preprint arXiv:2502.01926, 2025. 1, 3

    Pith/arXiv arXiv 2025

  43. [43]

    Iti- gen: Inclusive text-to-image generation

    Cheng Zhang, Xuanbai Chen, Siqi Chai, Chen Henry Wu, Dmitry Lagun, Thabo Beeler, and Fernando De la Torre. Iti- gen: Inclusive text-to-image generation. InProceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 3969–3980, 2023. 3

    2023

  44. [44]

    Under- standing and evaluating racial biases in image captioning

    Dora Zhao, Angelina Wang, and Olga Russakovsky. Under- standing and evaluating racial biases in image captioning. In ICCV, pages 14830–14840, 2021. 6

    2021

  45. [45]

    Men also like shopping: Reducing gen- der bias amplification using corpus-level constraints

    Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shopping: Reducing gen- der bias amplification using corpus-level constraints. InPro- ceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2979–2989, 2017. 3

    2017

  46. [46]

    Describe this photo in detail. When the person’s gender is ambiguous or not clearly discernible, avoid inferring it from stereotypical vi- sual cues

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models.arXiv preprint arXiv:2304.10592, 2023. 2 10 On Difference-Aware Gender Fairness for Vision-Language Models Supplementary Material A. Notations We summarize all key notations in Table 4. B. Baseline I...

    Pith/arXiv arXiv 2023

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.