{"id":"1b57abe8-f431-4573-9e6f-0cbc50e68341","arxiv_id":"2505.21106","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A fairness score built from token-level attribution and pruned multi-round dialogues links LLaVA models' reliance on sensitive image regions to demographic accuracy gaps.","lead":"The paper proposes a way to see inside vision-language models and measure whether they lean on sensitive visual cues, such as gender or skin color, when answering neutral questions. It reports that models use such cues unevenly across demographic groups and that this mismatch tracks accuracy gaps.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Sign alignment in Table 2 may be an artifact of attributing Ikey to the model's predicted (often wrong) answer rather than the ground-truth answer; Eq. 2–5 leave the attribution target unspecified.","rationale":"The paper's core claim is that bias is internal: lower-accuracy demographic groups are those for which neutral reasoning relies more on sensitive image tokens. The proposed FSD is a creative way to test this, and the framework is not circular because the fairness prompt is distinct from the neutral prompt. However, the load-bearing link between Ikey and the neutral answer is under-specified. If the gradient target is the model's own answer, then the entire comparison is between attributions for correct versus incorrect predictions. That would make the sign alignment in Table 2 nearly a restatement of the accuracy gap rather than independent evidence about internal reasoning. The reader's pruning/hallucination concern is real and related, but the predicted-vs-ground-truth logit issue is more fundamental because it affects the construction of Ikey itself. The proposed check directly settles it. Since the authors can fix this by clarifying the target and rerunning the analysis, the conditional verdict stands; acceptance should require reporting this choice and the corrected table.","tokens_in":13439,"tokens_out":7169,"duration_ms":89153,"concrete_test":"Recompute Ikey for LLaVA-v1.5-7B on the gender occupations in Table 2a using zanswer equal to the logit of the ground-truth answer token (yes/no based on FACET annotation), holding the remaining pipeline fixed, and recompute the FSD column. If the sign alignment with Acc Diff weakens or disappears, the reported correlation is an artifact of targeting predicted versus correct logits. Also rerun with tau swept over a range and with a fixed Ikey token budget per image to ensure FSD is not driven by token-count differences.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central evidence for \"systematic disparities in information usage\" is the sign alignment between Acc Diff and FSD in Sec. 5.2. The most load-bearing methodological choice is in Sec. 3.2: Eq. 2 computes gradients of \"a given answer output zanswer\", but the paper never states whether zanswer is the ground-truth answer token or the token the model actually generates. The phrase \"the model's output logits\" suggests the latter. If zanswer is the model's predicted token, then Ikey is constructed from the realized prediction: for high-accuracy groups this is the correct yes/no token, while for low-accuracy groups it is the incorrect token. FSD then measures sensitive information in the tokens that support a wrong neutral answer, not group-dependent information usage for the same target. This creates a direct confound: the fairness score can align with accuracy disparity simply because attribution targets differ by correctness. The counterfactual round in Sec. 3.3 mitigates hallucination from pruning but does not remove this confound, since the input to Round 2 was already selected under a different objective. A secondary, related gap is that threshold tau and the resulting Ikey size are not ablated; if disadvantaged groups have systematically larger Ikey, the pruned prompt retains more information and the fairness score is not comparable across groups.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes an explanatory framework for social bias in LVLMs. It first uses gradient-based attribution (Eqs. 2-5) to identify image tokens Ikey that most influence the model's answer to a neutral yes/no question, then feeds only Ikey back to the model in a second dialogue round with a sensitive prompt, and defines a fairness score from the consistency and confidence of responses across full and pruned inputs (Sec. 3.3, Table 1). The main empirical claim is that the difference in fairness scores between demographic groups aligns in sign with the difference in neutral-question accuracy, based on 12 hand-selected occupations per model (Sec. 5.2, Table 2). A supplementary textual analysis computes embedding similarity between neutral and sensitive concepts (Sec. 3.4).","tokens_in":13638,"tokens_out":7215,"duration_ms":74703,"significance":"If the sign-alignment result survives scrutiny, the paper would provide a concrete, token-level mechanism for social bias in LVLMs, going beyond detection. The paper has several strengths: it uses real FACET images rather than synthetic ones; it evaluates four open-source LLaVA variants; the multi-round protocol includes a counterfactual round intended to separate pruning-induced hallucination from genuine sensitive-information leakage; and the textual-modality analysis is a useful complement. However, the current evidence is a set of point estimates on a biased sample of occupations, without uncertainty quantification, and two load-bearing methodological choices (attribution target and threshold tau) are unspecified. These issues are addressable and do not by themselves invalidate the framework.","major_comments":[{"comment":"Eq. (2) computes the derivative of z_answer with respect to the feature map, but the paper never states whether z_answer is the ground-truth answer token or the token the model actually generates. The phrase \"the model's output logits\" suggests the latter. If the attribution target is the model-generated token, then Ikey is built from the realized prediction: for high-accuracy groups this is the correct yes/no token, while for low-accuracy groups it is an incorrect token. In that case the fairness score measures sensitive information in the tokens that support a wrong neutral answer, rather than group-dependent information usage for the same target, creating a direct confound with accuracy. Please specify the target explicitly and, if it is the generated token, rerun the analysis with the ground-truth token (or both) to show that the results are not an artifact of attribution-target choice.","section":"§3.2, Eq. (2)"},{"comment":"The threshold tau and the resulting size of Ikey are never reported, and no ablation is given. Since the Round-2 input is defined as \"only Ikey\", the fairness score depends directly on this threshold; if tau produces systematically larger pruned sets for disadvantaged groups, the comparison across groups is not meaningful. Report tau, the distribution of |Ikey| per group and per model, and a sensitivity analysis over tau. A related gap is that the chosen intermediate layers in Appendix A.3 are asserted from prior work without a quantitative justification, so the key-token set itself is an unvalidated modeling choice.","section":"§3.2, Eq. (5); §5.2"},{"comment":"The core evidence is sign alignment on 12 hand-selected occupations per model. For the gender attribute, where the authors use a signed male-minus-female convention, the selected occupations do show consistent alignment, and this is the most informative part of the table. However, for age and skin-tone attributes the table reports only positive Acc Diff values while Eq. (1) defines Acc Diff as an absolute max-min difference; with more than two groups, \"sign alignment\" is not well-defined unless a signed dominant-minus-disadvantaged difference is used. Moreover, no correlation coefficient, confidence interval, or significance test is provided, and selecting the 12 most biased occupations makes the alignment test weak. Report a signed version of Eq. (1), compute per-model Spearman correlations over all 51 occupations, and include uncertainty quantification.","section":"§5.2, Table 2"},{"comment":"The fairness scoring rules are ad hoc and are not derived from a stated model of how pruning and hallucination interact. For instance, Cases 1 and 3 both receive a score of +1 despite opposite Round-2 outcomes, the counterfactual round is applied only when Round 2 is positive, and Case 6 computes Conf2 - Conf1 without any hallucination check even though token pruning can reduce confidence for reasons unrelated to sensitive information. Please justify each case, ideally with an explicit generative model of the response process, and validate the scoring on data where the sensitive ground truth is known.","section":"§3.3, Table 1"}],"minor_comments":[{"comment":"After simplification, TSB_i equals the cosine similarity between h_n and h_i^s rather than a quantity that needs the \"1 -\" form; stating this directly would make the direction of the claimed bias easier to check. Also fix the typo \"for for each\" in the same section.","section":"§3.4, Eq. (8)"},{"comment":"The caption for panel (d) repeats \"LLaVA-v1.6-13B\"; from the surrounding models it should presumably read \"LLaVA-v1.6-7B\".","section":"Figure 4"},{"comment":"The skin-tone row for LLaVA-v1.6-7B contains the typo \"liferguard\" instead of \"lifeguard\".","section":"Table 2"},{"comment":"The layer ranges in Table 4 are the same across model sizes of each version, but no evidence is given that the attribution scores are stable across these ranges; a small ablation over layer choices would address this.","section":"Appendix A.3"},{"comment":"The statement that social bias is \"deeply rooted in the model's internal reasoning dynamics\" is stronger than what the correlational evidence supports; phrasing such as \"consistent with a mechanism in the model's internal reasoning\" would be more proportionate.","section":"Abstract and Conclusion"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim depends on a methodology that needs more validation than the manuscript currently provides. I recommend major revision rather than rejection because the required experiments (specifying the attribution target, ablating the threshold tau, using a signed metric over all occupations with confidence intervals, and justifying the fairness scoring rules) are within the scope of a revision. The framework is reusable and the core idea is plausible, so the issues are correctable without changing the paper's focus."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this paper has a genuinely new idea—using token-level attribution plus a pruned-token multi-round dialogue to quantify how much a model leans on sensitive visual information when answering neutral questions. If it works, it gives bias researchers a per-token, per-group diagnostic. But the headline result, sign alignment between accuracy disparity and the fairness score, is not yet supported because a load-bearing detail is left unspecified and the selection protocol inflates the apparent correlation.\n\nWhat's actually new: prior work (LLaVA-CAM, ModSCAN) either visualized information flow or measured stereotypes, but didn't tie either to sensitive-signal leakage in neutral reasoning via token pruning. The counterfactual round in the dialogue is a sensible attempt to separate genuine leakage from hallucination caused by pruning. The experiments use real FACET images across four LLaVA models, and the Limitations section is unusually frank.\n\nWhere it falls short: First, Eq. 2 never says whether z_answer is the ground-truth token or the token the model actually generates. The surrounding text says 'model's output logits,' which reads like the predicted token. If that's right, then for low-accuracy groups you're computing attribution for the model's wrong answer, and the key tokens are the ones driving that wrong answer. The fairness score then naturally aligns with accuracy disparity because the attribution target differs by correctness. That's a direct confound, and the stress-test note lands. Second, the threshold tau and the layer range are never given and never ablated; if tau differs across groups, the pruned token set size differs and the scores aren't comparable. Third, the headline correlation is only a sign match on 12 hand-picked most-biased occupations per model—no confidence intervals, no aggregate correlation across all 51, no prompt-seeded repetition. Fourth, the textual TSB analysis reduces to cosine similarity with a direction vector—fine as a sanity check, not a new result.\n\nIf I'm right about z_answer, the central claim is not established, though it's fixable: specify and justify the attribution target, run the analysis with ground-truth tokens, release tau and code, report all-occupation correlations with error bars. This deserves a serious referee who can ask for those revisions; it's not a desk reject. I'd bring it to a reading group in fairness, but I wouldn't cite it in its current form. The thinking is clear and the limitations are honestly stated—this is serious work with a fixable flaw.","headline":"A genuinely new bias-diagnostic framework, but the headline sign-alignment is confounded by an unspecified attribution target and an over-selected occupation set.","tokens_in":14228,"tokens_out":2917,"would_cite":false,"duration_ms":32324,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that social bias in large vision-language models is rooted in the internal reasoning dynamics of the model, showing that demographic groups with lower accuracy on neutral questions are precisely those for which the model…","keywords":["large vision language models","social bias","information flow analysis","fairness score","multi-round dialogue","token attribution","demographic disparity","LLaVA"],"falsifier":"An independent probe would settle the claim: if a classifier trained on the key-token representations of one group finds no more sensitive information than for the other group while accuracy gaps persist, the reliance-on-sensitive-tokens mechanism would be refuted. So would observing the same fairness-score sign alignment when the key tokens are replaced by a same-size random token subset.","tokens_in":13153,"feed_emoji":"⚖️","tokens_out":7329,"duration_ms":65204,"temperature":0.7,"pith_summary":"The paper sets out to explain, not just detect, social bias in large vision-language models. It argues that when a model answers a neutral question about a person in an image, it uses visual information unevenly across demographic groups: for the group that answers less accurately, the model leans more heavily on image tokens that encode sensitive attributes such as gender, age, or skin tone. To make this quantitative, the paper introduces a fairness score based on a multi-round dialogue in which the model is shown either the full image or only its highest-contribution tokens. The main reported result is that accuracy differences between demographic groups line up in sign with fairness-score differences, and a complementary textual analysis shows that neutral concepts already sit closer to some sensitive directions in embedding space.","feed_headline":"Over-reliance on sensitive tokens drives vision-language AI bias","feed_subtitle":"Lower-accuracy groups are those whose key image tokens carry the most sensitive information.","key_machinery":"The machinery is an attribution-plus-dialogue pipeline. First, gradient-based token attribution over intermediate layers identifies the image tokens $I_{\\mathrm{key}}$ that contribute most to the model's answer to a neutral question, using scores $A^l = \\mathrm{ReLU}(\\sum_k \\alpha_{l,k} F^{l,k})$ aggregated across layers. Second, a two-round dialogue with an added counterfactual round presents the model with either the full image token sequence or only $I_{\\mathrm{key}}$, and a fairness score in $[-1, +1]$ is computed from the consistency and confidence of responses; the counterfactual round is meant to separate genuine leakage of sensitive information from hallucination caused by pruning. Finally, a text-side measure $TSB_i = 1 - \\langle h_n, \\hat{h}_i^n \\rangle$, with $\\hat{h}_i^n = h_n - h_i^s$, quantifies how much a neutral concept embedding leans toward a sensitive direction.","core_discovery":"The paper claims that social bias in LVLMs is rooted in the model's internal reasoning dynamics rather than being only a surface artifact. Its central evidence is a sign alignment between the demographic accuracy difference (Acc Diff) and the fairness-score difference (FSD): for occupations where one group answers neutral questions markedly better, that dominant group also receives the higher fairness score, meaning the model relies less on sensitive visual information when reasoning about it. The paper further claims that the same biased pattern appears in the text modality, where neutral occupation names have embeddings that are systematically closer to one gender direction than another. Together these results are offered as a cross-modal explanation of where social bias comes from.","pith_inferences":["A sharper test of the mechanism would train a linear probe directly on the $I_{\\mathrm{key}}$ representations to confirm that they encode more sensitive information for disadvantaged groups, rather than inferring this from dialogue behavior.","If the causal reading is right, suppressing the identified sensitive tokens at decoding time could be a practical debiasing method; the paper does not test this intervention.","Replacing the key-token set with a same-size random subset would check whether the fairness-score pattern is genuinely driven by attribution or by token-count and pruning artifacts.","The same framework could be applied to neutral concepts beyond people, such as objects and scenes, to see whether token-reliance asymmetries generalize to biases not tied to demographic attributes."],"forward_implications":["Fairness scores computed from token pruning can serve as a diagnostic that reveals group disparities even when accuracy does not yet show a gap.","A model that is made to rely less on sensitive image tokens during neutral reasoning should show smaller accuracy differences between demographic groups.","Because the biased proximity also appears in text embeddings, debiasing a vision-language model may require intervening on the language backbone, not just on the vision encoder.","Per-occupation fairness-score differences give a concrete target list: the occupations with the most negative FSD are the ones where the model leans most on sensitive cues for the disadvantaged group."],"supporting_citations":[{"why":"Provides the LLaVA-CAM attribution method that the paper extends to token-level scoring.","marker":"Zhang et al., 2025"},{"why":"Supplies the FACET dataset of real images with demographic labels used for all experiments.","marker":"Gustafson et al., 2023"},{"why":"Defines the demographic accuracy difference metric used as the bias measure.","marker":"Zafar et al., 2017"},{"why":"Provides the LLaVA-v1.5 model variants used as testbeds.","marker":"Liu et al., 2023"},{"why":"Provides the LLaVA-v1.6 model variants used as testbeds.","marker":"Liu et al., 2024"},{"why":"Supplies the directional mapping construction used for the textual sensitivity bias.","marker":"Xu et al., 2023"}],"fun_headline_variants":["Bias in vision-language AI is rooted in reasoning dynamics","Sensitive image tokens skew vision-language model fairness","Cross-modal bias in LVLMs traced to internal information flow","LVLM bias linked to uneven use of key sensitive tokens","Sensitive tokens in vision-language models drive demographic bias"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The fairness-score measurement assumes that feeding only the pruned, high-contribution image tokens back to the model is a faithful way to read what those tokens encode, and that the counterfactual round fully removes pruning-induced hallucination; the threshold and layer choices that define the key-token set are not validated, so a group-dependent artifact of token deletion could look like bias.","fun_headline_variants_meta":{"raw":{"variants":["Bias in vision-language AI is rooted in reasoning dynamics","Sensitive image tokens skew vision-language model fairness","Cross-modal bias in LVLMs traced to internal information flow","LVLM bias linked to uneven use of key sensitive tokens","Sensitive tokens in vision-language models drive demographic bias"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000222,"raw_usage":{"total_tokens":1418,"prompt_tokens":875,"completion_tokens":543,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":464}},"tokens_in":491,"tokens_out":543,"duration_ms":5759,"temperature":1.0,"reasoning_tokens":464,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:35:14.355604+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An independent probe would settle the claim: if a classifier trained on the key-token representations of one group finds no more sensitive information than for the other group while accuracy gaps persist, the reliance-on-sensitive-tokens mechanism would be refuted. So would observing the same fairness-score sign alignment when the key tokens are replaced by a same-size random token subset.","supporting_citations":[],"review_version":1}