Interpreting Social Bias in LVLMs via Information Flow Analysis and Multi-Round Dialogue Evaluation

Shang Gao; Yifan Jia; Yutao Yue; Zhengyang Ji

REVIEW 4 major objections 5 minor 12 references

Interpreting Social Bias in LVLMs via Information Flow Analysis and Multi-Round Dialogue Evaluation

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that social bias in large vision-language models is rooted in the internal reasoning dynamics of the model, showing that demographic groups with lower accuracy on neutral questions are precisely those for which the model…

desk verdict A genuinely new bias-diagnostic framework, but the headline sign-alignment is confounded by an unspecified attribution target and an over-selected occupation set. read the letter →

arxiv 2505.21106 v1 pith:TEAEWCAX submitted 2025-05-27 cs.AI

Zhengyang Ji , Yifan Jia , Shang Gao , Yutao Yue This is my paper

classification cs.AI

keywords largevisionlanguagemodelssocialbiasinformationflowanalysisfairnessscoremulti-rounddialoguetokenattributiondemographicdisparityLLaVA

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to explain, not just detect, social bias in large vision-language models. It argues that when a model answers a neutral question about a person in an image, it uses visual information unevenly across demographic groups: for the group that answers less accurately, the model leans more heavily on image tokens that encode sensitive attributes such as gender, age, or skin tone. To make this quantitative, the paper introduces a fairness score based on a multi-round dialogue in which the model is shown either the full image or only its highest-contribution tokens. The main reported result is that accuracy differences between demographic groups line up in sign with fairness-score differences, and a complementary textual analysis shows that neutral concepts already sit closer to some sensitive directions in embedding space.

What carries the argument

The machinery is an attribution-plus-dialogue pipeline. First, gradient-based token attribution over intermediate layers identifies the image tokens $I_{\mathrm{key}}$ that contribute most to the model's answer to a neutral question, using scores $A^l = \mathrm{ReLU}(\sum_k \alpha_{l,k} F^{l,k})$ aggregated across layers. Second, a two-round dialogue with an added counterfactual round presents the model with either the full image token sequence or only $I_{\mathrm{key}}$, and a fairness score in $[-1, +1]$ is computed from the consistency and confidence of responses; the counterfactual round is meant to separate genuine leakage of sensitive information from hallucination caused by pruning. Finally, a text-side measure $TSB_i = 1 - \langle h_n, \hat{h}_i^n \rangle$, with $\hat{h}_i^n = h_n - h_i^s$, quantifies how much a neutral concept embedding leans toward a sensitive direction.

What would settle it

An independent probe would settle the claim: if a classifier trained on the key-token representations of one group finds no more sensitive information than for the other group while accuracy gaps persist, the reliance-on-sensitive-tokens mechanism would be refuted. So would observing the same fairness-score sign alignment when the key tokens are replaced by a same-size random token subset.

Watch

Extended reading notes

Core claim

The paper claims that social bias in LVLMs is rooted in the model's internal reasoning dynamics rather than being only a surface artifact. Its central evidence is a sign alignment between the demographic accuracy difference (Acc Diff) and the fairness-score difference (FSD): for occupations where one group answers neutral questions markedly better, that dominant group also receives the higher fairness score, meaning the model relies less on sensitive visual information when reasoning about it. The paper further claims that the same biased pattern appears in the text modality, where neutral occupation names have embeddings that are systematically closer to one gender direction than another. Together these results are offered as a cross-modal explanation of where social bias comes from.

Load-bearing premise

The fairness-score measurement assumes that feeding only the pruned, high-contribution image tokens back to the model is a faithful way to read what those tokens encode, and that the counterfactual round fully removes pruning-induced hallucination; the threshold and layer choices that define the key-token set are not validated, so a group-dependent artifact of token deletion could look like bias.

Editorial extensions

If this is right

Fairness scores computed from token pruning can serve as a diagnostic that reveals group disparities even when accuracy does not yet show a gap.
A model that is made to rely less on sensitive image tokens during neutral reasoning should show smaller accuracy differences between demographic groups.
Because the biased proximity also appears in text embeddings, debiasing a vision-language model may require intervening on the language backbone, not just on the vision encoder.
Per-occupation fairness-score differences give a concrete target list: the occupations with the most negative FSD are the ones where the model leans most on sensitive cues for the disadvantaged group.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A sharper test of the mechanism would train a linear probe directly on the $I_{\mathrm{key}}$ representations to confirm that they encode more sensitive information for disadvantaged groups, rather than inferring this from dialogue behavior.
If the causal reading is right, suppressing the identified sensitive tokens at decoding time could be a practical debiasing method; the paper does not test this intervention.
Replacing the key-token set with a same-size random subset would check whether the fairness-score pattern is genuinely driven by attribution or by token-count and pruning artifacts.
The same framework could be applied to neutral concepts beyond people, such as objects and scenes, to see whether token-reliance asymmetries generalize to biases not tied to demographic attributes.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the short version: this paper has a genuinely new idea—using token-level attribution plus a pruned-token multi-round dialogue to quantify how much a model leans on sensitive visual information when answering neutral questions. If it works, it gives bias researchers a per-token, per-group diagnostic. But the headline result, sign alignment between accuracy disparity and the fairness score, is not yet supported because a load-bearing detail is left unspecified and the selection protocol inflates the apparent correlation.

What's actually new: prior work (LLaVA-CAM, ModSCAN) either visualized information flow or measured stereotypes, but didn't tie either to sensitive-signal leakage in neutral reasoning via token pruning. The counterfactual round in the dialogue is a sensible attempt to separate genuine leakage from hallucination caused by pruning. The experiments use real FACET images across four LLaVA models, and the Limitations section is unusually frank.

Where it falls short: First, Eq. 2 never says whether z_answer is the ground-truth token or the token the model actually generates. The surrounding text says 'model's output logits,' which reads like the predicted token. If that's right, then for low-accuracy groups you're computing attribution for the model's wrong answer, and the key tokens are the ones driving that wrong answer. The fairness score then naturally aligns with accuracy disparity because the attribution target differs by correctness. That's a direct confound, and the stress-test note lands. Second, the threshold tau and the layer range are never given and never ablated; if tau differs across groups, the pruned token set size differs and the scores aren't comparable. Third, the headline correlation is only a sign match on 12 hand-picked most-biased occupations per model—no confidence intervals, no aggregate correlation across all 51, no prompt-seeded repetition. Fourth, the textual TSB analysis reduces to cosine similarity with a direction vector—fine as a sanity check, not a new result.

If I'm right about z_answer, the central claim is not established, though it's fixable: specify and justify the attribution target, run the analysis with ground-truth tokens, release tau and code, report all-occupation correlations with error bars. This deserves a serious referee who can ask for those revisions; it's not a desk reject. I'd bring it to a reading group in fairness, but I wouldn't cite it in its current form. The thinking is clear and the limitations are honestly stated—this is serious work with a fixable flaw.

Referee Report

4 major / 5 minor

Summary. This paper proposes an explanatory framework for social bias in LVLMs. It first uses gradient-based attribution (Eqs. 2-5) to identify image tokens Ikey that most influence the model's answer to a neutral yes/no question, then feeds only Ikey back to the model in a second dialogue round with a sensitive prompt, and defines a fairness score from the consistency and confidence of responses across full and pruned inputs (Sec. 3.3, Table 1). The main empirical claim is that the difference in fairness scores between demographic groups aligns in sign with the difference in neutral-question accuracy, based on 12 hand-selected occupations per model (Sec. 5.2, Table 2). A supplementary textual analysis computes embedding similarity between neutral and sensitive concepts (Sec. 3.4).

Significance. If the sign-alignment result survives scrutiny, the paper would provide a concrete, token-level mechanism for social bias in LVLMs, going beyond detection. The paper has several strengths: it uses real FACET images rather than synthetic ones; it evaluates four open-source LLaVA variants; the multi-round protocol includes a counterfactual round intended to separate pruning-induced hallucination from genuine sensitive-information leakage; and the textual-modality analysis is a useful complement. However, the current evidence is a set of point estimates on a biased sample of occupations, without uncertainty quantification, and two load-bearing methodological choices (attribution target and threshold tau) are unspecified. These issues are addressable and do not by themselves invalidate the framework.

major comments (4)

[§3.2, Eq. (2)] Eq. (2) computes the derivative of z_answer with respect to the feature map, but the paper never states whether z_answer is the ground-truth answer token or the token the model actually generates. The phrase "the model's output logits" suggests the latter. If the attribution target is the model-generated token, then Ikey is built from the realized prediction: for high-accuracy groups this is the correct yes/no token, while for low-accuracy groups it is an incorrect token. In that case the fairness score measures sensitive information in the tokens that support a wrong neutral answer, rather than group-dependent information usage for the same target, creating a direct confound with accuracy. Please specify the target explicitly and, if it is the generated token, rerun the analysis with the ground-truth token (or both) to show that the results are not an artifact of attribution-target choice.
[§3.2, Eq. (5); §5.2] The threshold tau and the resulting size of Ikey are never reported, and no ablation is given. Since the Round-2 input is defined as "only Ikey", the fairness score depends directly on this threshold; if tau produces systematically larger pruned sets for disadvantaged groups, the comparison across groups is not meaningful. Report tau, the distribution of |Ikey| per group and per model, and a sensitivity analysis over tau. A related gap is that the chosen intermediate layers in Appendix A.3 are asserted from prior work without a quantitative justification, so the key-token set itself is an unvalidated modeling choice.
[§5.2, Table 2] The core evidence is sign alignment on 12 hand-selected occupations per model. For the gender attribute, where the authors use a signed male-minus-female convention, the selected occupations do show consistent alignment, and this is the most informative part of the table. However, for age and skin-tone attributes the table reports only positive Acc Diff values while Eq. (1) defines Acc Diff as an absolute max-min difference; with more than two groups, "sign alignment" is not well-defined unless a signed dominant-minus-disadvantaged difference is used. Moreover, no correlation coefficient, confidence interval, or significance test is provided, and selecting the 12 most biased occupations makes the alignment test weak. Report a signed version of Eq. (1), compute per-model Spearman correlations over all 51 occupations, and include uncertainty quantification.
[§3.3, Table 1] The fairness scoring rules are ad hoc and are not derived from a stated model of how pruning and hallucination interact. For instance, Cases 1 and 3 both receive a score of +1 despite opposite Round-2 outcomes, the counterfactual round is applied only when Round 2 is positive, and Case 6 computes Conf2 - Conf1 without any hallucination check even though token pruning can reduce confidence for reasons unrelated to sensitive information. Please justify each case, ideally with an explicit generative model of the response process, and validate the scoring on data where the sensitive ground truth is known.

minor comments (5)

[§3.4, Eq. (8)] After simplification, TSB_i equals the cosine similarity between h_n and h_i^s rather than a quantity that needs the "1 -" form; stating this directly would make the direction of the claimed bias easier to check. Also fix the typo "for for each" in the same section.
[Figure 4] The caption for panel (d) repeats "LLaVA-v1.6-13B"; from the surrounding models it should presumably read "LLaVA-v1.6-7B".
[Table 2] The skin-tone row for LLaVA-v1.6-7B contains the typo "liferguard" instead of "lifeguard".
[Appendix A.3] The layer ranges in Table 4 are the same across model sizes of each version, but no evidence is given that the attribution scores are stable across these ranges; a small ablation over layer choices would address this.
[Abstract and Conclusion] The statement that social bias is "deeply rooted in the model's internal reasoning dynamics" is stronger than what the correlational evidence supports; phrasing such as "consistent with a mechanism in the model's internal reasoning" would be more proportionate.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the fairness-score/accuracy sign alignment is an empirical correlation, not a fitted or definitional identity; one minor non-load-bearing self-citation lowers the score slightly.

full rationale

The paper's central evidence is the sign alignment between Acc Diff (Eq. 1) and FSD (Eq. 9) in Sec. 5.2. FSD is computed from the model's own responses in a two-round dialogue under full vs. pruned token inputs (Sec. 3.3), and it is not derived from, fitted to, or algebraically equal to the neutral-question accuracy that defines Acc Diff. The key-token set Ikey (Eq. 5) is selected by gradient attribution for 'a given answer output zanswer' (Eq. 2); while the paper's failure to specify whether zanswer is the ground-truth or the model-predicted token is a genuine methodological confound for the causal interpretation, it is not a demonstrated reduction of FSD to Acc Diff by construction. The threshold tau and layer choices are unablated modeling choices, not fitted predictions of the reported correlation. The only overlap with the authors' prior work is the citation of Huo et al. (2024) in Appendix A.3 to justify selected intermediate layers; this is a minor implementation detail and is not load-bearing for the main claim. The Limitations paragraph itself concedes that the fairness score aligns with accuracy disparities 'only in sign, but not in magnitude,' which is consistent with an empirical correlation rather than a tautology. No self-definitional, fitted-input, self-citation-chain, or ansatz-smuggling pattern is exhibited by the paper's equations.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The framework rests on interpretability assumptions about gradient attribution, a pruning protocol whose validity is not independently checked, a hand-designed scoring table, and an unstated Ikey threshold. These are the main costs that the paper does not pay for upstream; there are no new physical entities.

free parameters (4)

Ikey threshold tau = not specified
Eq. 5 defines high-contribution tokens as those with average attribution above tau; the paper never reports tau or an ablation, and the fairness score is sensitive to this cutoff.
Intermediate layer range L = layers 10-12 (7B) or 10-14 (13B)
Table 4 fixes per-model layer ranges for attribution, taken from prior empirical findings (Appendix A.3); different ranges could change which tokens count as key.
Fairness score rule coefficients = +1, 0, -1, Conf1-Conf2, Conf2-Conf1
Table 1 assigns hand-chosen scores to the six response-pattern cases; these coefficients encode the construct of 'reliance on sensitive information' and are not derived or validated.
Number of selected occupations = 12 per model
Section 5.2 takes only the 12 occupations with the largest bias to show sign alignment; the aggregate pattern over all 51 occupations is not reported.

assumptions (5)

domain assumption Gradient-based token attribution (Eqs. 2-5) identifies the image tokens causally relevant to the model's answer.
The method assumes CAM-style channel-weighted ReLU attributions faithfully isolate reasoning-critical tokens; this is a common but unproven interpretability assumption (Section 3.2).
domain assumption Feeding only pruned Ikey tokens back as the image input yields a valid conditional distribution for comparison.
Section 3.3 compares full vs pruned token sequences; transformer models are sensitive to token sequence structure, and the paper does not validate that the pruned input is a meaningful image surrogate.
ad hoc to paper The counterfactual round labels a positive response as hallucination and tells genuine sensitive-information leakage from pruning artifacts.
Table 1's Case 1/4 rules assert that a positive answer to the opposite sensitive concept implies hallucination; this is a modeling assumption specific to the proposed scoring.
domain assumption FACET annotations (occupation, gender, age, skin tone) are correct for all images used.
The entire accuracy and fairness computation inherits annotation quality (Appendix A.2).
domain assumption One randomly selected prompt per image is sufficient to estimate response accuracy and confidence.
Appendix A.1 says one prompt is randomly selected per input; no repeated sampling or variance estimates are reported, so prompt variability is ignored.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Interpreting Social Bias in LVLMs via Information Flow Analysis and Multi-Round Dialogue Evaluation." pith.science (2026). https://pith.science/paper/TEAEWCAX

@misc{pith2026250521106,
  author       = {Pith},
  title        = {Pith review of: Interpreting Social Bias in LVLMs via Information Flow Analysis and Multi-Round Dialogue Evaluation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TEAEWCAX}},
  note         = {Machine review of arXiv:2505.21106}
}

read the original abstract

Large Vision Language Models (LVLMs) have achieved remarkable progress in multimodal tasks, yet they also exhibit notable social biases. These biases often manifest as unintended associations between neutral concepts and sensitive human attributes, leading to disparate model behaviors across demographic groups. While existing studies primarily focus on detecting and quantifying such biases, they offer limited insight into the underlying mechanisms within the models. To address this gap, we propose an explanatory framework that combines information flow analysis with multi-round dialogue evaluation, aiming to understand the origin of social bias from the perspective of imbalanced internal information utilization. Specifically, we first identify high-contribution image tokens involved in the model's reasoning process for neutral questions via information flow analysis. Then, we design a multi-turn dialogue mechanism to evaluate the extent to which these key tokens encode sensitive information. Extensive experiments reveal that LVLMs exhibit systematic disparities in information usage when processing images of different demographic groups, suggesting that social bias is deeply rooted in the model's internal reasoning dynamics. Furthermore, we complement our findings from a textual modality perspective, showing that the model's semantic representations already display biased proximity patterns, thereby offering a cross-modal explanation of bias formation.

Figures

Figures reproduced from arXiv: 2505.21106 by the authors.

**Figure 2.** Social bias explanation framework for LVLMs. (a) Information flow analysis identifies key image tokens in neutral reasoning. (b) Multi-round dialogue assesses the sensitive content of key image tokens. a benchmark dataset covering a range of social biases, enabling comprehensive evaluation of model behavior. Other studies employ counterfactual approaches by modifying sensitive attributes (e.g., gender, skin tone) … view at source ↗

**Figure 3.** Visualization of image token contributions in neutral question reasoning using LLaVA-v1.5. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: TSB Results on Gender Attribute for LVLMs [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 5 canonical work pages

[2]

arXiv preprint arXiv:2308.12966

Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966. Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoor- thi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023a. Minigpt-v2: large language model as a unified interface for vision-language multi-task...

arXiv
[5]

In Pro- ceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing, pages 12814– 12845

Modscan: Measuring stereotypical bias in large vision-language models from vision and language modalities. In Pro- ceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing, pages 12814– 12845. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao

work page 2024
[6]

arXiv preprint arXiv:2310.03744

Improved baselines with visual instruc- tion tuning. arXiv preprint arXiv:2310.03744. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee

arXiv
[8]

In The 2023 Conference on Empirical Methods in Natural Lan- guage Processing

Label words are anchors: An information flow perspective for understanding in-context learning. In The 2023 Conference on Empirical Methods in Natural Lan- guage Processing. Jinfeng Wei and Xiaofeng Zhang

work page 2023
[9]

Peiran Wu, Che Liu, Canyu Chen, Jun Li, Cosmin I Bercea, and Rossella Arcucci. 2024a. Fmbench: Benchmarking fairness in multimodal large lan- guage models on medical tasks. arXiv preprint arXiv:2410.01089. Xuyang Wu, Yuan Wang, Hsin-Tai Wu, Zhiqiang Tao, and Yi Fang. 2024b. Evaluating fairness in large vision-language models across diverse demo- graphic a...

arXiv
[10]

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi

Lan- guage representation projection: Can we transfer factual knowledge across languages in multilingual language models? In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 3692–3702. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi

work page 2023
[12]

In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 2289–2299

From redundancy to relevance: Information flow in LVLMs across reasoning tasks. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 2289–2299. Zhi Zhang, Srishti Yadav, Fengze Han, and Ekaterina Shutova. 2024b. Cross-modal information flow in mu...

arXiv 2025
[2017]

In Artificial intelligence and statistics, pages 962–970

Fairness constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pages 962–970. Jie Zhang, Sibo Wang, Xiangkui Cao, Zheng Yuan, Shiguang Shan, Xilin Chen, and Wen Gao. 2024a. Vlbiasbench: A comprehensive benchmark for eval- uating bias in large vision-language model. arXiv preprint arXiv:2406.14194. Xiaofeng Zhang, Yiha...

arXiv

Show all 12 references

[2019]

arXiv preprint arXiv:1908.01224

Smooth grad- cam++: An enhanced inference level visualization technique for deep convolutional neural network models. arXiv preprint arXiv:1908.01224. Tianyi Tang, Wenyang Luo, Haoyang Huang, Dong- dong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen

1908 arXiv
[2023]

arXiv preprint arXiv:2303.08774

Gpt-4 techni- cal report. arXiv preprint arXiv:2303.08774. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou

arXiv
[2024]

In Proceedings of the 2024 Con- ference on Empirical Methods in Natural Language Processing, pages 6801–6816

MMNeuron: Discovering neuron-level domain-specific interpretation in multimodal large language model. In Proceedings of the 2024 Con- ference on Empirical Methods in Natural Language Processing, pages 6801–6816. Yukun Jiang, Zheng Li, Xinyue Shen, Yugeng Liu, Michael Backes, a...

2024
[2025]

In Proceedings of the 2025 Confer- ence of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5946–5991

Uncovering bias in large vision-language models at scale with counterfactuals. In Proceedings of the 2025 Confer- ence of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5946–5991. Qidong Huang, Xiaoyi Do...

2025

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [2]

arXiv preprint arXiv:2308.12966

Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966. Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoor- thi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023a. Minigpt-v2: large language model as a unified interface for vision-language multi-task...

arXiv

[2] [5]

In Pro- ceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing, pages 12814– 12845

Modscan: Measuring stereotypical bias in large vision-language models from vision and language modalities. In Pro- ceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing, pages 12814– 12845. Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Nau- mann, Hoifung Poon, and Jianfeng Gao

work page 2024

[3] [6]

arXiv preprint arXiv:2310.03744

Improved baselines with visual instruc- tion tuning. arXiv preprint arXiv:2310.03744. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee

arXiv

[4] [8]

In The 2023 Conference on Empirical Methods in Natural Lan- guage Processing

Label words are anchors: An information flow perspective for understanding in-context learning. In The 2023 Conference on Empirical Methods in Natural Lan- guage Processing. Jinfeng Wei and Xiaofeng Zhang

work page 2023

[5] [9]

Peiran Wu, Che Liu, Canyu Chen, Jun Li, Cosmin I Bercea, and Rossella Arcucci. 2024a. Fmbench: Benchmarking fairness in multimodal large lan- guage models on medical tasks. arXiv preprint arXiv:2410.01089. Xuyang Wu, Yuan Wang, Hsin-Tai Wu, Zhiqiang Tao, and Yi Fang. 2024b. Evaluating fairness in large vision-language models across diverse demo- graphic a...

arXiv

[6] [10]

Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi

Lan- guage representation projection: Can we transfer factual knowledge across languages in multilingual language models? In Proceedings of the 2023 Con- ference on Empirical Methods in Natural Language Processing, pages 3692–3702. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi

work page 2023

[7] [12]

In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 2289–2299

From redundancy to relevance: Information flow in LVLMs across reasoning tasks. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Hu- man Language Technologies, pages 2289–2299. Zhi Zhang, Srishti Yadav, Fengze Han, and Ekaterina Shutova. 2024b. Cross-modal information flow in mu...

arXiv 2025

[8] [2017]

In Artificial intelligence and statistics, pages 962–970

Fairness constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pages 962–970. Jie Zhang, Sibo Wang, Xiangkui Cao, Zheng Yuan, Shiguang Shan, Xilin Chen, and Wen Gao. 2024a. Vlbiasbench: A comprehensive benchmark for eval- uating bias in large vision-language model. arXiv preprint arXiv:2406.14194. Xiaofeng Zhang, Yiha...

arXiv