{"id":"43c98d4d-74bc-4133-9713-ad7c3f51994f","arxiv_id":"2608.08284","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"FairGap jointly measures output and hidden-state shifts under counterfactual identity probes and reports that these two fairness signals decouple in LLM recommenders.","lead":"This paper introduces FairGap, a benchmark that measures whether LLM recommenders change their visible recommendations and their hidden internal states when a user's stated gender, age, or race is swapped. It finds that output-only fairness audits can miss large internal shifts, and that removing an internal demographic signal can make recommendations more biased rather than less.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ROA sign flips under alternative internal anchors in the paper's own ablation (§A.3), so the 'pervasive hidden-output decoupling' claim may be an artifact of last-prompt-token extraction.","rationale":"The reader's weakest-assumption analysis correctly identifies the representation-extraction anchor as the load-bearing measurement choice. My independent reading of the paper confirms that this is not a peripheral robustness detail: the entire empirical contribution is the joint IBS/OBS comparison, and the paper's own ablation in §A.3 demonstrates that the alignment statistic changes sign across plausible extraction rules. That is exactly the kind of internal inconsistency that should gate the central claim. I considered whether a different concern—such as the abstract overstating 'ROA rarely exceeds 0.22' or 'every tested condition' in the steering result—might be more load-bearing. Those are real overclaims, but they are presentation-level issues; even if corrected, the benchmark's core finding would still depend on the stability of IBS. The anchor problem is more fundamental because it determines whether the benchmark measures anything at all. I also considered whether the steering intervention result could independently support the decoupling claim even if IBS is anchor-dependent. It cannot: steering explicitly targets the internal direction computed from hidden states, so the observed IBS reduction is partly by construction, and its OBS worsening is measured against the same anchor-dependent baseline. A concrete test that recomputes the full pipeline under alternative anchors would settle whether the decoupling phenomenon survives. Since the reader already made this the condition for their verdict, my stress-test does not change the recommended outcome; it strengthens the condition and makes the required check more explicit.","tokens_in":24123,"tokens_out":3505,"duration_ms":35835,"concrete_test":"Recompute Table 3 for the six MovieLens gender and race conditions using the two alternative extraction anchors from Appendix A.3—cue-span mean and prompt-mean pooling—while holding fixed the OBS definition, layer selection, dev-set probe weighting, and Joint Otsu thresholds. If the sign of ROA flips or the Q4 fraction drops below 5% under either alternative for a condition whose headline finding is decoupling, then the pervasive hidden-output decoupling claim is a measurement artifact rather than a robust model property. Specifically, verify whether the sign flip already visible for Llama-8B in Table 6 replicates for Mistral-7B/MovieLens/Race, where the headline ROA is 0.235.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that output-only fairness evaluation is structurally incomplete rests on the assertion that IBS, as measured, is a faithful index of fairness-relevant internal processing. Every downstream finding—low ROA, non-negligible Q4, and the steering paradox—is a comparison between OBS and this IBS measurement. Appendix A.3 shows the opposite: for Llama-8B/MovieLens/Gender, ROA is -0.244 under cue-span mean, +0.064 under last-prompt-token, and +0.181 under prompt-mean pooling; for race, ROA ranges from -0.104 to +0.289. IBS itself varies by over three orders of magnitude (4.59e-5 vs 0.0398 for gender). The paper's justification of last-prompt-token as a 'principled middle ground' is a heuristic, not a measurement-theoretic argument. Consequently, the existence of a non-negligible Q4 'silent bias' population, and the specific claim that outputs remain stable despite substantial internal shifts, may be properties of the chosen anchor rather than properties of the models. The paper itself acknowledges in A.3 that the results support a 'measurement-sensitive interpretation,' yet the abstract and Section 5 present decoupling as a discovered property of LLM recommenders. This is an internal tension: the paper's own evidence undercuts the operational meaning of its headline metric. If IBS is not a stable measure of internal processing, then the 'structural incompleteness' conclusion is not established by this benchmark, even if the conceptual point has independent plausibility.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FairGap introduces a benchmark for jointly evaluating LLM recommender fairness at the output level (OBS, based on rank-biased overlap of counterfactual recommendation lists) and at the internal-representation level (IBS, based on cosine distances between hidden states aggregated over four relative layer depths). The paper reports Representation-Output Alignment (ROA) as a Spearman correlation between these two axes, a four-quadrant taxonomy over the joint IBS-OBS space, and a steering intervention that is claimed to reduce IBS while worsening OBS. The central claims are that output-only fairness evaluation is structurally incomplete, that hidden-output decoupling is pervasive (with ROA rarely exceeding 0.22), and that a non-negligible population of users shows stable outputs despite substantial internal shifts. The paper also argues that activation steering demonstrates a fundamental tension between internal and output-level fairness.","tokens_in":24406,"tokens_out":5183,"duration_ms":44546,"significance":"If the central claims held, FairGap would be a useful contribution: the counterfactual minimal-pair construction is programmatically verified, the benchmark covers six model families across three domains and three attributes, and the paper includes several robustness checks (RBO persistence, percentile thresholding, prompt-family variants, age-cue variants). The release of benchmark data, probes, and evaluation code is a concrete strength. However, the load-bearing internal-representation measurement is not stable under alternative extraction anchors in the paper's own ablation, and the steering evidence is partly circular because the same representations define both the steering direction and the outcome metric. Because every downstream claim (low ROA, Q4 silent-bias population, steering paradox) is a comparison against this IBS measurement, the headline claims are not yet established as properties of the models. The framework is promising and the manuscript is repairable, but the abstract and Section 5 substantially oversell what the current evidence supports.","major_comments":[{"comment":"The claim that \"ROA rarely exceeds 0.22\" is contradicted by the paper's own Table 3. Ten of the 54 reported condition cells exceed 0.22, including Qwen-7B/MovieLens/Gender (0.315), Mistral-7B/SteamReviews/Age (0.379), Llama-8B/SteamReviews/Age (0.267), Dolphin-34B/SteamReviews/Gender (0.267), and four MovieLens/Race cells (0.235, 0.286, 0.289, 0.370). Even the cross-model average for MovieLens/Race is 0.224, itself above the stated threshold. This is not a minor wording issue: the abstract's central quantitative claim and the \"pervasive decoupling\" narrative are not supported by the reported distribution. The manuscript should either report the actual distribution of ROA values or revise the claim to match the data.","section":"Abstract and §5.1 (Table 3)"},{"comment":"The extraction ablation in Appendix A.3 shows that IBS and ROA are highly sensitive to the choice of hidden-state anchor. For Llama-8B/MovieLens/Gender, ROA ranges from -0.244 (cue-span mean) to 0.181 (prompt-mean pooling), and IBS ranges from 4.59e-5 to 0.0398, a variation of nearly three orders of magnitude; for Race, ROA ranges from -0.104 to 0.289. Since IBS is the internal axis used to define Q4, to compute every ROA value, and to motivate the steering experiment, the headline decoupling phenomenon and the \"silent bias\" population are properties of the last-prompt-token extraction rule, not established model properties. The paper acknowledges this in A.3, where it concludes that the results support \"a measurement-sensitive interpretation,\" but the Abstract and Section 5 present decoupling as a discovered property of LLM recommenders. This internal tension is load-bearing and must be resolved, either by demonstrating robustness across anchors or by explicitly reframing all conclusions as protocol-dependent.","section":"§A.3 (Table 6)"},{"comment":"The steering experiment is partly circular. The steering direction is the normalized mean difference between female and male hidden states at each layer, the layer selection uses separability defined as the cosine distance between these class means, and the outcome IBS is a cosine-distance-based measure on the same hidden states. Projecting out the mean-difference vector from those representations will reduce the component of variance along that direction, so a drop in IBS is expected by construction and does not independently confirm that a demographic direction is \"mechanistically present.\" The manuscript needs a control condition, for example steering with a random or permuted direction of the same norm, or evaluating IBS on held-out layers or held-out users not used to construct the vector, before the 3-8x reduction can be claimed as evidence of a genuine mechanistic effect.","section":"§5.4 and §C.6"},{"comment":"Table 5 presents six representative intervention conditions, but three rows are marked as estimated (‡) with methodology deferred to Appendix C.6, while the measured steering results in Table 12 cover only three conditions. The main text nevertheless states that \"in every tested condition\" OBS increases and describes this as \"the strongest evidence\" for FairGap's diagnostic utility. The abstract's \"up to 8x\" reduction is based on one measured row, but the generalization to all tested conditions relies on estimated rows. The manuscript should separate measured from estimated results in the main text and should restrict the \"every tested condition\" claim to the measured conditions, or move the estimated rows entirely to the appendix.","section":"Table 5 and §5.4"}],"minor_comments":[{"comment":"The sentence \"Although IBS is in absolute magnitude and diagnostically informative\" is grammatically incomplete; it appears to be missing a predicate such as \"is small in absolute magnitude.\"","section":"Section 6"},{"comment":"The citation \"[18,22]\" for activation steering appears to point to CrowS-Pairs and a fairness-survey reference rather than the intended representation-steering works; it should likely be [34] and [44].","section":"Appendix C.6"},{"comment":"The caption states \"Darker shading indicates more favourable values,\" but the table as rendered contains no shading; either implement the shading or remove the caption sentence.","section":"Table 3 caption"},{"comment":"The extraction ablation text uses the example \"a Black user/a White user,\" while Section B.1 specifies the race cue as \"a Black individual/a white individual\"; the wording should be aligned to avoid confusion about which cue was actually tested.","section":"Appendix A.3 vs B.1"},{"comment":"The mapping from relative depths L={1/4,2/4,3/4,4/4} to concrete layer indices is not specified for models where the layer count is not divisible by four; an off-by-one ambiguity could affect the layerwise results in Table 13.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"I agree with the conditional verdict: the benchmark infrastructure and the counterfactual-pair design are solid and worth publishing after revision, but the central empirical claims need to be recalibrated to the measurement-sensitive findings in Appendix A.3. The paper's own ablation undermines the operational meaning of the headline metric, so the broad \"structurally incomplete\" conclusion is not established by this benchmark as written. The steering experiment also needs a non-circular control before it can be presented as the sharpest diagnostic test. This is repairable within the manuscript's scope, hence major revision rather than reject."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the real contribution here is the benchmark scaffolding, not the discovery that internal and output fairness can diverge. That conceptual point is already in Bai et al. and Cassese et al., which the paper cites. What is new is the concrete operationalization for recommenders—OBS/IBS/ROA, counterfactual identity probes, quadrant taxonomy—and that is a genuinely useful, well-scoped evaluation package. The sweep across six model families, three domains, and three attributes, with RBO persistence, threshold, and prompt-family robustness checks, is serious work, and the released code/data make it reproducible.\n\nThe soft spot is load-bearing. The central metric IBS depends on which hidden-state anchor you pick, and the paper's own Appendix A.3 shows it. For Llama-8B/MovieLens, ROA flips sign across cue-span, last-prompt-token, and prompt-mean pooling; IBS itself varies by three orders of magnitude. That means the headline 'pervasive decoupling' and the Q4 'silent bias' population may be properties of the extraction rule, not the models. The authors acknowledge a 'measurement-sensitive interpretation' in A.3, but the abstract and Section 5 drop that caveat and present decoupling as a discovered fact. Also, 'ROA rarely exceeds 0.22' is contradicted by several cells in Table 3 (0.315, 0.379, 0.370). And the steering 'every tested condition' claim includes rows marked as estimates; those should be labeled cautiously in the main text.\n\nThe circularity is minor but real: IBS weights are fitted on separability, and steering layers are selected by the same separability criterion, so the IBS drop under steering is partly a mathematical consequence of projecting out the mean-difference vector used to define the direction.\n\nBottom line: this is a solid benchmark paper with an overstated central conclusion. The measurement sensitivity is acknowledged internally but not allowed to constrain the advertised claims. A good referee could get the authors to either show robustness of the decoupling across anchors or re-scope the claims as 'decoupling under a specified extraction protocol.' The conceptual insight isn't new, but the benchmark itself is worth having. I'd send it to review, but the revision needs to front-load A.3 and recalibrate the abstract.","headline":"Solid and useful benchmark scaffolding for hidden-vs-output fairness in LLM recommenders, but the central claim of pervasive decoupling is not established by the paper's own metric-robustness ablation.","tokens_in":25019,"tokens_out":1553,"would_cite":true,"duration_ms":13790,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Output-level fairness audits of LLM recommenders are structurally incomplete because hidden representation shifts and recommendation shifts decouple.","keywords":["fairness auditing","LLM recommenders","hidden representations","counterfactual probes","representation-output alignment","activation steering","internal bias","output bias"],"falsifier":"Recompute ROA and quadrant assignments across the full benchmark using cue-span mean and prompt-mean pooling instead of the last-prompt-token hidden state, holding everything else fixed, as the paper does for Llama-8B on MovieLens. If the low-ROA pattern and the non-negligible Q4 population disappear, change sign, or shrink to noise under these alternative anchors, then the claimed hidden-output decoupling is a property of the extraction rule, not of the models.","tokens_in":23859,"feed_emoji":"⚖️","tokens_out":6665,"duration_ms":52620,"temperature":0.7,"pith_summary":"The paper argues that checking only the recommendations an LLM produces cannot establish whether the model is fair, because the hidden representations behind those recommendations can shift in ways the outputs do not reveal. It introduces FairGap, a benchmark that measures observable output shift and internal representation shift from counterfactual identity probes across gender, age, and race, applied to six open-weight LLM families and three recommendation domains. Across conditions the two measures align only weakly, with the Spearman correlation rarely above 0.22, and a non-negligible share of users showing stable outputs alongside substantial internal shifts. The paper further claims this decoupling is not a measurement oddity: an intervention that reduces internal shift by up to eight times in every tested condition simultaneously worsens output shift, revealing a genuine tension between internal and output-level fairness.","feed_headline":"Output-level fairness audits miss hidden bias in LLM recommenders","feed_subtitle":"Across six model families, hidden shifts and output shifts decouple; steering for one axis worsens the other.","key_machinery":"The central object is the counterfactual identity probe: a pair of prompts identical in preference history and task instruction except for one protected-attribute sentence, such as \"The user is a woman\" versus \"The user is a man.\" Each probe produces two ranked recommendation lists and two hidden-state sequences, and from these FairGap computes the output shift as one minus rank-biased overlap and the internal shift as a probe-weight-averaged cosine distance across four relative layer depths. The connection between the two axes is summarized by ROA, the Spearman correlation across users, and turned into a per-user diagnosis by adaptive two-dimensional Otsu thresholds that partition the joint shift plane into four quadrants. Activation steering on the top-5 gender-separable layers acts as the mechanical check: it confirms the internal axis is linearly decodable and shows that compressing it can destabilize outputs.","core_discovery":"The central claim is that output-level fairness evaluation of LLM recommenders is structurally incomplete: the ranked lists a model emits and the hidden states that produce them frequently disalign under a single protected-attribute cue. FairGap operationalizes this with Output Bias Score, defined as one minus rank-biased overlap between counterfactual top-10 lists; Internal Bias Score, defined as a separability-weighted average of cosine distances between hidden states at four relative layer depths; and Representation-Output Alignment, the Spearman correlation between the two across users. It then assigns each user pair to one of four quadrants of the joint shift space, isolating output-visible mismatch, joint sensitivity, joint stability, and hidden-internal mismatch, the last being invisible to output-only audits. The paper reports that ROA rarely exceeds 0.22 across the benchmark and that Q4 holds a non-negligible user population, and its steering experiments show that projection-mode activation steering reduces IBS by 3-8x while increasing OBS in every condition, which it reads as evidence of a fundamental tension rather than a measurement artifact.","pith_inferences":["If this paper is right, any output-only fairness certification for LLM recommenders is incomplete, and future audit standards may need to specify which internal representation anchor is being used, since the paper's own ablation shows ROA can flip sign across extraction rules.","The steering paradox suggests that removing a linearly decodable demographic direction is not sufficient for output fairness; a natural testable extension is steering on nonlinear or layer-targeted directions to see whether joint IBS/OBS improvement is possible.","Because IBS magnitudes are much smaller than OBS magnitudes, raw internal-shift numbers may be less interpretable across models than quadrant profiles; a practical extension is to report anchor-specific confidence intervals rather than point thresholds.","The same benchmark design could transfer to closed-weight models, but only where hidden-state access is available, so API-based audits would need a proxy internal signal or a stated limitation."],"forward_implications":["An auditor who looks only at recommendation outputs can certify a model as fair while the model silently shifts its internal processing; benchmark results show Q4 populations persist across multiple model-domain pairs.","Utility metrics such as Match@10 do not certify fairness stability, since models with strong preference alignment can still show large internal displacement under attribute probes.","Debiasing interventions should not be assumed to improve both axes at once: steering that suppresses linear demographic directions can worsen observable divergence, so joint evaluation is needed to interpret intervention results.","New models can be compared against FairGap's standardized OBS, IBS, ROA, and quadrant profiles across the six model families and three domains."],"supporting_citations":[{"why":"Defines rank-biased overlap, the ranked-list similarity measure that underlies the output shift d_out.","marker":"[38]"},{"why":"Supplies the two-dimensional Otsu thresholding procedure that partitions users into the four IBS-OBS quadrants.","marker":"[39]"},{"why":"Provides the demographic counterfactual output-similarity framing that FairGap extends with an internal representation axis.","marker":"[7]"},{"why":"Motivates the representation-engineering view that protected-attribute information can be read as a direction in hidden states.","marker":"[34]"},{"why":"Supplies the mean-difference activation vector mechanism used in the steering intervention experiments.","marker":"[44]"},{"why":"Provides the book-domain user-item interaction histories used to construct counterfactual probes.","marker":"[40]"},{"why":"Provides the movie-domain user-item interaction histories used to construct counterfactual probes.","marker":"[41]"},{"why":"Provides the game-domain user-item interaction histories used to construct counterfactual probes.","marker":"[42]"}],"fun_headline_variants":["Output fairness audits blind to hidden bias in LLM recommenders","Steering LLM recommenders for less hidden bias worsens outputs","Hidden shifts in LLM recommenders escape output-only audits","LLM recommender fairness: hidden bias survives output checks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing measurement assumption is that the hidden state at the last prompt token, sampled at four relative layer depths and compared by cosine distance, faithfully captures fairness-relevant internal processing; the paper's own ablation shows ROA and IBS change substantially under alternative extraction rules, so a wrong anchor would make the decoupling and Q4 populations artifacts of the extraction rule.","fun_headline_variants_meta":{"raw":{"variants":["Output fairness audits blind to hidden bias in LLM recommenders","Steering LLM recommenders for less hidden bias worsens outputs","Hidden shifts in LLM recommenders escape output-only audits","LLM recommender fairness: hidden bias survives output checks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001177,"raw_usage":{"total_tokens":4863,"prompt_tokens":939,"completion_tokens":3924,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":3854}},"tokens_in":555,"tokens_out":3924,"duration_ms":25139,"temperature":1.0,"reasoning_tokens":3854,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:10:56.177467+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute ROA and quadrant assignments across the full benchmark using cue-span mean and prompt-mean pooling instead of the last-prompt-token hidden state, holding everything else fixed, as the paper does for Llama-8B on MovieLens. If the low-ROA pattern and the non-negligible Q4 population disappear, change sign, or shrink to noise under these alternative anchors, then the claimed hidden-output decoupling is a property of the extraction rule, not of the models.","supporting_citations":[{"cited_title":"Webber, A","cited_arxiv_id":null,"evidence_quote":"Defines rank-biased overlap, the ranked-list similarity measure that underlies the output shift d_out."},{"cited_title":"Zhang, K","cited_arxiv_id":null,"evidence_quote":"Provides the demographic counterfactual output-similarity framing that FairGap extends with an internal representation axis."},{"cited_title":"Game recommendations on steam","cited_arxiv_id":null,"evidence_quote":"Provides the game-domain user-item interaction histories used to construct counterfactual probes."}],"review_version":1}