Pith. sign in

REVIEW 4 major objections 7 minor 12 references

(Fact) Check Your Bias

T0 review · 4 major / 7 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Injecting directional bias into a fact-checking LLM's hypothetical documents changes about half of the retrieved evidence, yet final veracity verdicts shift by at most 4 percentage points, while asymmetric safety refusals create an…

desk verdict A useful empirical look at prompt bias in an LLM fact-checking pipeline, but the headline 'verdict stability' claim rests on aggregate shifts and needs per-claim agreement data before it can be trusted. read the letter →

arxiv 2506.21745 v1 pith:FNSTOKHP submitted 2025-06-26 cs.CL

Eivind Morris Bakke , Nora Winger Heggelund This is my paper
classification cs.CL
keywords factverificationLLMbiasparametricknowledgeevidenceretrievalpromptsensitivitysafetyrefusalHerOAveritec
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper investigates whether the parametric knowledge of a large language model biases an automatic fact-checking pipeline, using the HerO system and its benchmark dataset. It first asks the model to verify claims with no external evidence, finding that Llama 3.1 8B labels nearly half of claims 'Not Enough Evidence' and reaches a verdict on the rest. It then injects directional bias into the generated hypothetical fact-checking documents, running separate retrieval and verification pipelines for supportive, refuting, and neutral prompts. The injected bias changes roughly half of the retrieved evidence documents, but final veracity labels shift by at most 4 percentage points, and the model refuses supportive document generation much more often than refuting generation. If the findings hold, evidence collection in LLM fact-checkers is prompt-sensitive while verdicts are robust, and safety guardrails asymmetrically suppress supportive evidence for controversial claims.

What carries the argument

The load-bearing object is the modified first stage of the HerO pipeline: the hypothetical fact-checking document generator, which the authors re-prompt with three distinct instructions (support, refute, balanced/objective). These generated documents act as queries for BM25 retrieval and embedding-based reranking, so the prompt's perspective steers which external evidence is selected; the same claim is then run through parallel pipelines that share the retrieval, question-generation, and veracity-prediction components, isolating the effect of the injected bias. A second, simpler variant bypasses retrieval entirely and asks the model to answer three self-generated questions from parametric knowledge alone, exposing the model's native cautious distribution and refusal behavior.

What would settle it

Run the same two experiments with Llama 3.1 70B (or a frontier model) at the original top-10,000 retrieval setting, and check whether final veracity label shifts exceed the reported 4-percentage-point ceiling and whether the supportive-document refusal rate remains far above the refuting rate; either outcome would overturn the paper's central claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that directional bias in the HerO pipeline's hypothetical document generation significantly changes which evidence documents are retrieved, yet has limited impact on final veracity predictions, and that asymmetric safety refusals introduce a systematic negative bias. In the direct prediction experiment, Llama 3.1 8B, prompted to answer verification questions from parametric knowledge alone, assigns 'Not Enough Evidence' to roughly 47% of claims, a much more cautious distribution than the knowledge-based baseline. In the biased-retrieval experiment, Jaccard similarity scores of 0.42–0.56 across conditions imply that about half of the retrieved documents are unique to each perspective, while the largest final-label shift is 4 percentage points in the supportive condition and under 0.8 points in the others. The model also refuses to produce supportive passages for 5.3% of documents, affecting 18.8% of claims, while refusal rates for refuting, neutral, and baseline prompts stay near 0.1%; the paper argues this creates an inherent negative bias because the model will write critical content about controversial claims it will not support.

Load-bearing premise

The findings rest on the assumption that Llama 3.1 8B with top-5,000 retrieval reproduces the bias behavior of the larger 70B models with top-10,000 retrieval that the original HerO system uses; if the smaller model's cautiousness and refusal asymmetries do not carry over, the central conclusions about bias in the system do not transfer.

Editorial extensions

If this is right

  • Fact-checking pipelines that generate hypothetical evidence with LLMs will retrieve substantially different document sets depending on prompt phrasing, so single-perspective evidence collection is not neutral.
  • Final veracity verdicts appear robust to differences in retrieved evidence, suggesting verification components focus on key or semantically redundant evidence rather than the full retrieved set.
  • Safety guardrails create a measurable asymmetry: the model more readily produces refuting than supporting evidence for controversial claims, which systematically narrows the evidence available for the supportive side.
  • Multi-perspective evidence collection, including explicitly balanced or parallel biased prompts, is a candidate design response to reduce this evidence skew.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The verdict stability the paper observes may reflect that the veracity prediction step condenses evidence into question-answer pairs, which could mask retrieval bias in downstream tasks that use the retrieved documents directly; this is an extension the paper does not test.
  • The refusal asymmetry could be tested across other open and closed models to see whether it is a Llama-specific safety calibration or a general property of instruction-tuned LLMs.
  • Because the paper only measures structural overlap (Jaccard and Kendall), semantically different documents that receive identical verdicts might still affect users who read the evidence; a human or downstream-task evaluation of the evidence sets would extend the finding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper investigates whether parametric-knowledge biases in Llama 3.1 affect the HerO fact-checking pipeline. In Experiment 1, a direct-prediction variant that uses only the model's parametric knowledge labels 47% of claims as "Not Enough Evidence" and agrees with the retrieval-based baseline on only 31.2% of claims. In Experiment 2, the authors prompt the hypothetical-document generator to produce supportive, refuting, or neutral passages, then run parallel retrieval and verification pipelines. They report Jaccard similarities of 0.42–0.56 and Kendall rank correlations of 0.48–0.61 between evidence sets, but only 0–4 percentage-point shifts in label distributions and near-identical legacy AVeriTeC scores. They also observe that the model refuses to generate supportive documents more often than critical documents (5.3% vs. 0.1% document refusal rates, affecting 18.8% of claims in the positive condition). The paper concludes that evidence retrieval is prompt-sensitive while final verdicts are robust, and that asymmetric safety refusals create an inherent negative bias.

Significance. If the retrieval-sensitivity and verdict-stability findings survive per-claim analysis, they would be a useful empirical contribution: they separate evidence-collection bias from verdict-level bias in an open pipeline. The refusal asymmetry for supportive documents on controversial claims is a novel, concrete observation with implications for fairness of automated fact-checking. The manuscript is honest about limitations (8B model, development set only) and ships code. However, the key stability claim currently rests on aggregate statistics, and the refusal-rate numbers appear internally inconsistent; both issues are fixable with additional reporting.

major comments (4)
  1. [§4.3 and §6] The central claim that final verdict predictions are stable across prompting strategies is not supported by per-claim agreement data. Table 2 report only aggregate label-distribution shifts, and Table 3 reports only aggregate legacy AVeriTeC scores; these are compatible with large numbers of opposing per-claim flips that cancel out. With 500 claims, a net shift of 4 percentage points corresponds to only 20 claims net, while hundreds of claims could change in opposite directions. The paper never reports the fraction of claims whose verdict actually changes between conditions, nor pairwise confusion matrices. Appendix B itself shows a single claim receiving four different verdicts across the positive, negative, objective, and baseline conditions (Conflicting Evidence/Cherrypicking, Refuted, Refuted, Not Enough Evidence). The authors should report per-claim pairwise agreement rates or confusion matrices; without this, the "robustness" conclusion is unsubstantiated.
  2. [§4.3, Table 4] The refusal-rate statistics are internally inconsistent. The text says "we analyzed all 4,000 generated documents," yet the positive condition is reported as having 214 refusals at a 5.3% document refusal rate; 214/0.053 ≈ 4,038, which would imply the positive condition alone generated about 4,000 documents. If instead each of the four conditions generated roughly 1,000 documents (500 claims × 2 documents per claim), the positive condition's refusal rate would be about 21.4%, not 5.3%. The baseline row (5/500 claims with refusals but a 0.1% document refusal rate) is also incompatible with a 1,000-document condition denominator, because even one refusal per claim would require at least five refusals. The paper never states how many hypothetical documents are generated per claim, so the denominator for every refusal rate is ambiguous. This undermines the quantitative "47-fold" or "53-fold" asymmetry claim that is central to the negative-bias finding.
  3. [§3.1 and Limitations] The experiments use Llama 3.1 8B with top-5,000 retrieval, while the original HerO baseline used 70B models and top-10,000 retrieval. The paper acknowledges this gap in Section 3.1 and in the Limitations, but the abstract and conclusions generalize to "the HerO system" and claim an "inherent negative bias" in the system. Because the model size and retrieval cutoff materially affect both parametric-knowledge behavior and retrieval quality, the conclusions should be explicitly scoped to the adapted 8B pipeline, or the authors should provide a small verification with a larger model or at least with top-10,000 retrieval on a subset of claims. Without this, the external validity of the central claims is unestablished.
  4. [§4.3] The retrieval-difference statistics (Jaccard 0.42–0.56, Kendall 0.48–0.61) are reported as point estimates without confidence intervals or significance tests. Given the 500-claim sample, bootstrap confidence intervals or a paired test would help establish that the observed differences between conditions are not sampling noise. This is particularly important for the qualitative claim that the objective condition is closer to the negative condition than to the positive condition (Jaccard 0.56 vs. 0.42), since no uncertainty is reported for either value.
minor comments (7)
  1. [§3.3] Section 3.3 contains a typo: "bypasses all knowledge retrieval components in the in the HerO-system" should read "components in the HerO-system."
  2. [§5] The reference "Hoe et. al." should be "Hoes et al." and the citation should be formatted consistently with the reference list.
  3. [§4.3] The text reports a "47-fold difference" in refusal rates, but Table 4 implies about a 53-fold difference (5.3% / 0.1%). Please reconcile the two numbers.
  4. [Table 4] The column "Document Refusal Rate" is not defined. Specify the denominator (per condition, per total documents, or per claim) and describe how the 4,000 generated documents are distributed across the four conditions.
  5. [§6] The expression "0-4%" would be clearer as "0–4 percentage points" to avoid ambiguity with relative changes.
  6. [Limitations] The Limitations section is thorough, but it does not mention the absence of per-claim agreement analysis, which is directly relevant to the stability claim. Adding this would help readers calibrate the strength of the conclusions.
  7. [Appendix A] The sample output in Table 5 shows the model reaching "Not Enough Evidence" even though its own justification states that "the claim appears to be unsubstantiated and lacks concrete evidence." The paper does not discuss this tension; a brief comment would be useful for interpreting the cautious-bias finding.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all central claims are empirical measurements against the external Averitec benchmark and the external HerO baseline, with no fitted parameters or self-citation chain used to derive the results.

full rationale

The paper's two experiments are direct empirical interventions on a fixed external pipeline (HerO) and a fixed external benchmark (Averitec). Experiment 1 measures Llama 3.1's parametric-knowledge verdict distribution against a knowledge-based baseline; Experiment 2 injects directional prompts into HyDE-FC document generation and measures downstream retrieval overlap, label-shift, and legacy AVeriTeC scores, all compared to the external HerO baseline. No parameter is fitted to the reported outcomes, no result is defined in terms of another reported result, and no load-bearing claim is justified by the authors' own prior work (the only self-references are to the public code repository). The 'inherent negative bias' conclusion is supported by an independently countable refusal-rate asymmetry (214 vs 0.1% document refusal rates), which is a measured behavioral fact rather than an imported assumption. The skeptical concern about aggregate label shifts not establishing per-claim stability is a correctness/statistical-inference issue, not a circularity issue, and it does not reduce any prediction to its input. The paper also explicitly acknowledges external-validity limits (smaller 8B model, top-5000 retrieval, dev-set evaluation), which further indicates the findings are empirical claims contingent on experimental setup rather than analytic equivalences. Hence the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or invented entities are needed because this is an empirical study. Four domain assumptions carry the interpretation: the 8B model represents the larger HerO models, the dev split represents the withheld test set, phrase-based refusal detection measures refusal behavior, and legacy A VeriTeC scores are adequate for comparison. These are partly flagged in the Limitations section.

assumptions (4)
  • domain assumption Llama 3.1 8B behavior is representative of HerO's 70B behavior for the bias phenomena studied.
    Section 3.1 uses 8B instead of 70B and changes retrieval depth from 10,000 to 5,000; the Limitations section admits this may affect results, yet the main conclusions are framed about the HerO baseline.
  • domain assumption The 500-claim development set and the non-random first-1,000 training split represent the withheld FEVER-25 test distribution.
    Section 3.2 states the split and that no test set was available; non-random ordering could introduce bias, and the Limitations section notes results may vary on the official test set.
  • domain assumption Automated phrase matching for refusal language captures systematic refusal behavior.
    Section 4.3 counts only explicit phrases such as 'I am unable to', 'I cannot', and 'I can't provide'; indirect refusals or rephrased refusals are not counted, which could undercount refusals.
  • domain assumption Legacy A VeriTeC scores are an adequate performance measure for comparing prompting strategies.
    Section 4.1 and the Limitations section state that new A VeriTeC scores were not computed because of Llama API cost, so the evaluation relies on the legacy metric.

how reviews work

0 comments
Cite this review

Pith. "Pith review of (Fact) Check Your Bias." pith.science (2026). https://pith.science/paper/FNSTOKHP

@misc{pith2026250621745,
  author       = {Pith},
  title        = {Pith review of: (Fact) Check Your Bias},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FNSTOKHP}},
  note         = {Machine review of arXiv:2506.21745}
}
read the original abstract

Automatic fact verification systems increasingly rely on large language models (LLMs). We investigate how parametric knowledge biases in these models affect fact-checking outcomes of the HerO system (baseline for FEVER-25). We examine how the system is affected by: (1) potential bias in Llama 3.1's parametric knowledge and (2) intentionally injected bias. When prompted directly to perform fact-verification, Llama 3.1 labels nearly half the claims as "Not Enough Evidence". Using only its parametric knowledge it is able to reach a verdict on the remaining half of the claims. In the second experiment, we prompt the model to generate supporting, refuting, or neutral fact-checking documents. These prompts significantly influence retrieval outcomes, with approximately 50\% of retrieved evidence being unique to each perspective. Notably, the model sometimes refuses to generate supporting documents for claims it believes to be false, creating an inherent negative bias. Despite differences in retrieved evidence, final verdict predictions show stability across prompting strategies. The code is available at: https://github.com/eibakke/FEVER-8-Shared-Task

Figures

Figures reproduced from arXiv: 2506.21745 by the authors.

Figure 1
Figure 1. Our direct to prediction system, made to assess [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Our multi-prediction system, made to assess the impact of intentionally introduced bias into the retrieval [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. An example of the instruction prompt used for [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: An example of the instruction prompt used [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: When asked to write a fact-checking article [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Claims about certain topics, such as Covid-19, [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 6 canonical work pages

  1. [3]

    Preprint, arXiv:2410.16927

    Revealing hidden bias in ai: Lessons from large language models. Preprint, arXiv:2410.16927. Alex Campolo, Madelyn Rose Sanfilippo, Meredith Whittaker, and Kate Crawford

    arXiv

  2. [5]

    In Companion Proceedings of the ACM Web Conference 2024, WWW ’24, page 883–886, New York, NY , USA

    Fact-gpt: Fact-checking augmentation via claim matching with llms. In Companion Proceedings of the ACM Web Conference 2024, WWW ’24, page 883–886, New York, NY , USA. Association for Computing Machin- ery. Emma Hoes, Sacha Altay, and Juan Bermeo

    work page 2024

  3. [6]

    Preprint, arXiv:2405.20362

    Hallucination-free? assessing the reliabil- ity of leading ai legal research tools. Preprint, arXiv:2405.20362. Alexandra Olteanu, Carlos Castillo, Fernando Diaz, and Emre Kıcıman

    arXiv

  4. [7]

    In Proceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP 2024), pages 410–424, Trento

    Investigating gender bias in large language models through text generation. In Proceedings of the 7th International Conference on Natural Language and Speech Processing (ICNLSP 2024), pages 410–424, Trento. Association for Computational Linguistics. Zhongxiang Sun, Xiaoxue Zang, Kai Zheng, Yang Song, Jun Xu, Xiao Zhang, Weijie Yu, Yang Song, and Han Li

    work page 2024

  5. [11]

    In Pro- ceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing , pages 8541– 8565, Miami, Florida, USA

    Knowledge conflicts for LLMs: A survey. In Pro- ceedings of the 2024 Conference on Empirical Meth- ods in Natural Language Processing , pages 8541– 8565, Miami, Florida, USA. Association for Compu- tational Linguistics. Yejun Yoon, Jaeyoon Jung, Seunghyun Yoon, and Kun- woo Park

    work page 2024

  6. [12]

    In a letter to Steve Jobs, Sean Connery refused to appear in an apple commercial

    Gender bias in large lan- guage models across multiple languages. Preprint, arXiv:2403.00277. 10 Appendix A. Experiment 1: Direct Prediction Prompt and Sample Output The following prompt was used for the direct pre- diction system to evaluate Llama 3.1’s parametric knowledge without external retrieval: System Prompt: You are a fact-checking assistant. You...

    arXiv 2004

  7. [2014]

    In Proceedings of the ACL 2014 Workshop on Lan- guage Technologies and Computational Social Sci- ence, pages 18–22, Baltimore, MD, USA

    Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Lan- guage Technologies and Computational Social Sci- ence, pages 18–22, Baltimore, MD, USA. Associa- tion for Computational Linguistics. Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou

    work page 2014

  8. [2017]

    AI Now Institute at New York University

    AI Now 2017 Report. AI Now Institute at New York University. Eun Cheol Choi and Emilio Ferrara

    work page 2017

Show all 12 references
  1. [2020]

    Preprint, arXiv:2002.10957

    Minilm: Deep self-attention distillation for task-agnostic com- pression of pre-trained transformers. Preprint, arXiv:2002.10957. Rongwu Xu, Zehan Qi, Zhijiang Guo, Cunxiang Wang, Hongru Wang, Yue Zhang, and Wei Xu

    2002 arXiv

  2. [2021]

    Preprint, arXiv:2101.05783

    Persistent anti-muslim bias in large language models. Preprint, arXiv:2101.05783. Orlando Ayala and Patrice Bechard

    arXiv

  3. [2024]

    Reduc- ing hallucination in structured outputs via retrieval- augmented generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track), page 228–238. Assoc...

    2024

  4. [2025]

    Preprint, arXiv:2410.11414

    Redeep: Detecting hallucination in retrieval-augmented generation via mechanistic inter- pretability. Preprint, arXiv:2410.11414. Andreas Vlachos and Sebastian Riedel

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.