Pith. sign in

REVIEW 4 major objections 6 minor 42 references

The paper claims that a turn-level catch-and-repair loop — a small classifier that flags ungrounded conversational turns plus a re-prompting step that rewrites them — improves faithfulness scores by roughly 0.1 to 0.5 points on a 5-point sc

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:12 UTC pith:YJJCRFBL

load-bearing objection A solid first benchmark for faithfulness in document-to-podcast generation, but the headline gains rest on an LLM judge that is only validated on vanilla turns, not on catch-n-repair outputs. the 4 major comments →

arxiv 2607.21961 v1 pith:YJJCRFBL submitted 2026-07-24 cs.CL

On Improving Faithfulness of Podcasts from Documents

Soumya Dutta , Tejas Indulal Dhamecha , Pannaga Shivaswamy This is my paper
classification cs.CL
keywords faithfulnesspodcast generationdocument groundinghallucination mitigationLLM-as-a-judgecatch-n-repairlong-form dialogueturn-level evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AI-generated podcasts that are supposed to summarize a document often add plausible but unsupported claims. This paper argues that faithfulness can be measured turn by turn, and that unfaithful turns can be caught and rewritten before they contaminate the rest of the conversation. The authors build a dataset of more than 1,500 documents across five domains, show that even strong models produce ungrounded turns, and propose catch-n-repair: a lightweight detector that flags unfaithful turns plus a re-prompting step that has the generator rewrite them. Across five generator models and both in-domain and out-of-domain documents, the method raises faithfulness scores by roughly 0.1 to 0.5 points on a 5-point scale while leaving coverage essentially unchanged. If the judge scores are trusted, the result shows that long-form conversational grounding can be improved without retraining the generator.

Core claim

The central claim is that turn-level grounding, not transcript-level quality, is the right unit for faithfulness in document-grounded podcast generation, and that a model-agnostic detect-and-repair loop reliably improves judged faithfulness. The loop uses a fine-tuned classifier to flag each conversational turn as faithful or unfaithful; the generator is then re-prompted to rewrite flagged turns using only facts from the source document. The paper validates a turn-level LLM judge against human ratings on 328 turns (Pearson r=0.63, Krippendorff α=0.69), then applies catch-n-repair to five generators. It reports consistent improvements in both in-domain and out-of-domain settings, with larger

What carries the argument

The load-bearing object is the turn-level 'catch' classifier: a LoRA fine-tuned 8-billion-parameter instruction model that, given a source document and a single conversational turn, outputs faithful or unfaithful. It is trained on synthetic turns generated and labeled by a 70-billion-parameter model using a 1–5 Likert grounding prompt, with scores of 4 or above treated as faithful. During sequential generation, flagged turns are re-prompted for repair; because repair happens before subsequent turns are generated, errors do not propagate. The judge that measures outcomes is a separate LLM prompted to rate each turn's grounding on the same 1–5 scale.

Load-bearing premise

The whole result rests on the assumption that the LLM judge's 1–5 faithfulness scores, which are used to train the detector, select repairs, and measure success, agree with human judgment on the repaired transcripts; the paper only validates the judge against humans on 328 original turns, with a Pearson correlation of 0.63.

What would settle it

Have human annotators rate catch-n-repair transcripts from the same 40 test documents using the paper's own grounding instructions. If human-judge agreement on repaired turns is no better than chance, or if any domain shows human-rated faithfulness dropping while judge-rated faithfulness rises, the reported 0.1–0.5 gains would not establish real grounding.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Faithfulness gains appear across all five generators tested, including a state-of-the-art closed model, so the method is not tied to a particular generator.
  • Out-of-domain gains are at least as large as in-domain gains, suggesting the detector generalizes beyond the five training domains.
  • Coverage is largely preserved; the largest drops are about 0.2 points on a 5-point scale, and in some cases coverage rises.
  • The improvement pattern holds when a different evaluator model replaces the primary judge, reducing the chance that gains are an artifact of one judge.
  • Turn-level repair is cheap: it requires only a small classifier and a re-prompt, not access to model internals or external retrieval.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same catch-and-repair pattern could transfer to other long-form grounded generation tasks — lecture scripts, meeting summaries, narrated articles — wherever turns can be judged against a source; the paper only demonstrates podcasts.
  • Because the judge is both the training-signal source and the outcome measure, part of the reported gain may reflect the judge's preference for repaired turns rather than human-perceived grounding; a human evaluation of catch-n-repair transcripts would settle this.
  • Detector accuracy on unfaithful turns (67.3% F1) sets a ceiling; improving recall on subtle hallucinations, or training on human-labeled rather than model-labeled data, should amplify the repair gains.
  • The judge could be replaced by a human-in-the-loop or a retrieval-verified scorer to make the loop work in high-stakes domains where a mislabeled turn is costly.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents the first systematic study of faithfulness in document-grounded podcast generation. The authors construct Doc-to-Podcast, a dataset of 1520 documents across five domains, and generate transcripts with five LLMs (GPT-4o, LLaMA3.3-70B, and Qwen2.5 14B/32B/72B). They introduce a turn-level LLM-as-a-judge protocol assigning Likert 1-5 grounding scores to each conversational turn, and validate this protocol against human annotations on 328 turns (Pearson r=0.63, Krippendorff α=0.69). They then propose catch-n-repair (CnR), a model-agnostic mitigation framework: a LoRA-fine-tuned LLaMA-3.1-8B binary classifier detects unfaithful turns, and the generator is re-prompted to rewrite flagged turns using only facts explicitly stated in the source. Experiments report consistent faithfulness gains with CnR across in-domain (ID) and out-of-domain (OOD) sets for all generators, as measured by GPT-4o and Qwen judges (Tables 3-4), with minimal impact on coverage (Table 5). The paper claims to be the first to study faithfulness in document-grounded podcast generation and to provide an effective turn-level grounding intervention.

Significance. The work addresses an important and underexplored problem: grounding in long-form, multi-speaker conversational generation. The Doc-to-Podcast dataset and the human annotation study for the judge are useful community resources. The proposed catch-n-repair is simple, model-agnostic, and black-box compatible, which makes it practically appealing. If the reported gains were shown to correspond to human-perceived faithfulness, the paper would make a solid contribution to trustworthy generative AI. However, as presented, the central claim that CnR 'improves faithfulness' is not fully established because the evaluation metric is validated only on original (vanilla) turns, while the repaired turns are scored exclusively by LLM judges. The whole pipeline is optimized toward LLM preferences: training labels come from LLaMA-3.1-70B as both generator and judge, the catch model is trained to imitate that judge, and the final scores are produced by GPT-4o or Qwen judges. The single external anchor—a moderate 0.63 correlation on vanilla turns—does not cover the distribution shift introduced by the repair step. This gap, combined with the lack of significance testing on the reported deltas, preve

major comments (4)
  1. [Sec. 5.3 vs. Sec. 5.5 (Table 3)] The central claim that CnR improves faithfulness rests on LLM-as-a-judge scores for repaired turns. However, the human validation of the judge is limited to 328 original turns (Sec. 5.3). Repaired turns differ systematically: the repair prompt (Sec. 4.3) instructs the generator to rewrite using only facts explicitly stated in the source, which likely makes the turns more conservative and more lexically/semantically similar to the document. LLM judges are known to be sensitive to style and surface overlap; without human scores on CnR transcripts, the reported Δ of +0.1 to +0.5 in Table 3 may reflect judge preference rather than actual grounding gains. Please provide a human evaluation of a sample of CnR outputs, or at least a judge–human correlation computed separately on repaired turns. Until then, the paper should temper its claim of 'improving faithfulness.' This is the primary gap and
  2. [Sec. 4.1 and Sec. 5.5 (Tables 3-5)] There is a potential circularity in the evaluation pipeline: the catch model is trained on labels produced by LLaMA-3.1-70B acting as both generator and evaluator (Sec. 4.1), and the final improvements are measured using GPT-4o and Qwen LLM judges (Tables 3-5). While the use of a different judge model for the final evaluation mitigates the concern of judge self-preference, all judges are LLMs and may share systematic biases toward conservative, source-aligned text. The only external anchor is a 0.63 correlation with humans on original turns. The paper does not demonstrate that this correlation holds for repaired turns. Concretely, if the LLM judge rewards the verbatim repetition of source phrases, the repair strategy would inflate scores without improving human-perceived grounding. A simple and decisive test would be to compute the judge–human correlation on repaired turns; if it is sign
  3. [Sec. 5.5, Table 3] The reported improvements are not accompanied by statistical significance tests. Several deltas are small relative to the reported standard deviations—e.g., Qwen2.5-14B ID Δ=+0.1 with SDs of 0.5/0.6, and LLaMA3.3-70B ID Δ=+0.1 with SDs of 0.5. The claim that 'Across all models and both ID and OOD settings, CnR improves faithfulness scores' is therefore not supported by the data as presented. Please report per-document paired analyses (e.g., two-sided bootstrap or signed-rank tests with effect-size confidence intervals) and qualify any improvements that are not statistically reliable. This does not require a huge new experiment, but it is essential for a quantitative claim of consistent gains.
  4. [Sec. 4.2 and Sec. 5.4] The catch model achieves an F1-score of only 67.3% on the unfaithful class (Sec. 5.4). Since the repair mechanism only corrects turns that are caught, the headline improvement may be driven by a small and potentially biased subset of turns that the detector happens to identify. The paper does not report the percentage of turns that were flagged and repaired, nor the per-turn outcomes for those cases. Reporting the repair rate and the judge-assigned improvements among repaired turns would clarify the mechanism. Moreover, the synthetic training data are generated by prompting LLaMA-3.1-70B to intentionally hallucinate, which may produce a narrow distribution of unfaithfulness; the detector's behavior on naturally occurring ungrounded content—which may be more subtle—remains uncharacterized.
minor comments (6)
  1. [Abstract / Intro] The Figure 1 example uses 'Attention Is All You Need' and notes that ChatGPT is mentioned anachronistically. This is a good illustrative example, but the temporal framing (the paper is from 2017) should be stated explicitly to avoid confusion about the difference between factual correctness and grounding.
  2. [Sec. 5.5] Typo: 'Appendex' should be 'Appendix' in the pointer to A.2.
  3. [Table 3 and Table 4] The 'Avg.Δ' column is computed as the simple mean of ID and OOD deltas. Please clarify whether the per-setting sample sizes are equal, and consider reporting the weighted mean or the per-setting confidence intervals.
  4. [Appendix A.1] The financial-domain link (annualreports.com) is a commercial site and may be volatile; please provide an access date or a more stable snapshot.
  5. [Sec. 7 Limitations] The limitations section acknowledges dependence on the catch classifier but does not mention that the LLM-judge evaluation is validated only on original turns. Adding a sentence about this would be appropriate and consistent with the paper's apparent honesty about scope.
  6. [Sec. 3.3] The definition of coverage is vague: 'important information' is not operationalized. Consider specifying a rubric or using a small set of key points extracted from the source, so that the coverage metric is reproducible.

Circularity Check

0 steps flagged

No significant circularity: CnR's gains are measured by held-out LLM judges (GPT-4o/Qwen), not by the LLaMA-3.1-70B judge that produced the training labels; the main gap is external validity, not a by-construction reduction.

full rationale

The formal target is defined in Sec. 3.3 as an LLM-as-a-judge score (Feval(t_ij,d_i)=r_ij, Eq. 1), and the catch detector is trained on LLaMA-3.1-70B judge labels with a >=4 threshold (Sec. 4.1). However, the paper deliberately evaluates catch-n-repair with a different judge: 'All podcast turns are evaluated by GPT-4o acting as the judge LLM' (Sec. 5.2), and App. A.2 uses Qwen2.5-72B. There is no equation or training objective that forces f'_i > f_i under these independent judges; the gain in Tables 3-5 is an empirical outcome that could have failed. The human validation (Sec. 5.3, Pearson r=0.63, Krippendorff alpha=0.69) anchors the GPT-4o judge on 328 vanilla turns, and Sec. A.8 confirms those annotations came from the Sec. 3.2 vanilla system rather than from catch-n-repair transcripts. The absence of human scores on repaired outputs is a genuine external-validity limitation (and Sec. 7 concedes prompt sensitivity), but it does not reduce the claimed improvement to its own input: the catch model is not the final judge, the final judge is not the labeler, and no result is imported by self-citation. No load-bearing self-citations, imported uniqueness theorems, or ansatz-by-citation steps are present. The concern raised about LLM-judge self-preference is a correctness/validity risk, not a circularity step.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper's core contribution is empirical; it introduces no new formal objects. The load-bearing assumptions are about evaluator validity, the grounding definition, and the transfer of synthetic training data and judge scores to new models/domains. The main worry is that the evaluation loop is closed inside LLM judgments: labels for training come from LLM-as-judge and final scores come from LLM-as-judge, with human validation only for the judge and only on a small sample.

free parameters (3)
  • faithfulness label threshold (Likert >=4 = faithful) = >=4 faithful; <=3 unfaithful (1-5 scale)
    Hand-set binarization converts judge scores into training labels for the catch model (Sec 4.1); determines the 17,561/9,436 class split; no sensitivity analysis.
  • context truncation limit = 32,768 tokens
    Documents are truncated to fit LLaMA-3.1-8B context for the catch model (Sec 4.2, 5.4); grounding evidence beyond the limit is invisible to the detector.
  • LoRA hyperparameters = rank=8, alpha=16, dropout=0.05
    Chosen implementation details for catch model fine-tuning (Sec 5.4); standard but hand-set, and the paper does not study their effect.
axioms (4)
  • domain assumption LLM-as-a-judge scores are a valid proxy for human faithfulness judgments in long-form conversational turns.
    Sec 3.3 defines Feval through Leval; Sec 5.3 reports only 0.63 Pearson correlation over 328 turns, and no human check on repaired turns.
  • domain assumption Faithfulness means grounding in the source document, not real-world correctness; plausible additions count as unfaithful.
    Definition repeated in Sec 1-3 and in annotator instructions (A.8); the ChatGPT example in Fig 1 is factually true but ungrounded.
  • domain assumption Synthetic hallucinations generated by LLaMA-3.1-70B with an instruction to embellish are representative of hallucinations from GPT-4o, Qwen2.5, and LLaMA3.3.
    Sec 4.1 trains the catch model only on this synthetic distribution; the paper does not measure distribution shift.
  • domain assumption Re-prompting the generator to rewrite a flagged turn using only source facts preserves conversational flow and does not create new hallucinated content.
    Sec 4.3 repair strategy; final transcripts are scored only for faithfulness/coverage by LLM judges, with no human flow or factuality check.

pith-pipeline@v1.3.0-alltime-deepseek · 13684 in / 14059 out tokens · 146411 ms · 2026-08-01T06:12:36.438651+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) are increasingly used to generate long-form conversational content such as podcasts from textual sources. While these systems produce fluent and engaging narratives, they often introduce ungrounded information. In this work, we present the first systematic study of faithfulness in document-grounded podcast generation, where grounding must be maintained across conversational turns in long-form, multi-speaker transcripts. We construct a dataset of over 1500 documents spanning five domains and generate podcast transcripts using multiple LLMs. We introduce a turn-level LLM-as-a-judge framework for evaluating whether conversational turns are supported by the source document, and validate its reliability through human studies. Our analysis shows that even state-of-the-art models, including GPT-4o, frequently generate ungrounded content. To mitigate this issue, we propose catch-n-repair, a model-agnostic framework that detects and rewrites unfaithful conversational turns while preserving conversational flow. Experiments demonstrate consistent improvements in faithfulness across both in-domain and out-of-domain settings.

Figures

Figures reproduced from arXiv: 2607.21961 by Pannaga Shivaswamy, Soumya Dutta, Tejas Indulal Dhamecha.

Figure 1
Figure 1. Figure 1: Example of unfaithfulness in generated pod [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Faithfulness evaluation: Once these transcripts are generated, LLM-as-a-judge (GPT4o) evaluation is used to classify each turn of the podcast on a scale of 1 to 5 based on its faithfulness to the source document. Mitigation: The proposed mitigation strategy - catch-n-repair - is used in conjunction with the LLM to generate transcripts that are faithful to the source document. 2025; Xu et al., 2026). Howeve… view at source ↗
Figure 3
Figure 3. Figure 3: Step 1: Using an LLM to create training data for the catch model. Step 2: Training another LLM for predicting faithful/unfaithful given any podcast turn. Step 3: The catch-n-repair mechanism for mitigating the unfaithful generations. The catch model flags each generation as faithful or unfaithful. The unfaithful generations are replaced by the same LLM with the old unfaithful generation as a negative examp… view at source ↗
Figure 4
Figure 4. Figure 4: The average faithfulness score as evaluated [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The average faithfulness score as evaluated [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The number of documents in each page range [PITH_FULL_IMAGE:figures/full_fig_p011_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The average coverage score as evaluated with [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

42 extracted references · 4 canonical work pages

  1. [1]

    Audio and Podcasting Fact Sheet , year =

  2. [2]

    Liu, Nayu and Zhu, Junnan and Ma, Yiming and Lu, Zhicong and Xu, Wenlei and Yang, Yong and Zhong, Jiang and Wei, Kaiwen , booktitle=

  3. [3]

    The Fourteenth International Conference on Learning Representations , year=

    VibeVoice: Expressive Podcast Generation with Next-Token Diffusion , author=. The Fourteenth International Conference on Learning Representations , year=

  4. [4]

    arXiv preprint arXiv:2601.14903 , year=

    PodBench: A Comprehensive Benchmark for Instruction-Aware Audio-Oriented Podcast Script Generation , author=. arXiv preprint arXiv:2601.14903 , year=

    arXiv

  5. [5]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Rarr: Researching and revising what language models say, using language models , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  6. [6]

    arXiv preprint arXiv:1904.09675 , year=

    Bertscore: Evaluating text generation with bert , author=. arXiv preprint arXiv:1904.09675 , year=

    Pith/arXiv arXiv 1904

  7. [7]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Towards mitigating LLM hallucination via self reflection , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  8. [8]

    Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    2023

  9. [9]

    N exus S um: Hierarchical LLM Agents for Long-Form Narrative Summarization

    Kim, Hyuntak and Kim, Byung-Hak. N exus S um: Hierarchical LLM Agents for Long-Form Narrative Summarization. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.500

    doi:10.18653/v1/2025.acl-long.500 2025

  10. [10]

    Transactions of the Association for Computational Linguistics , volume=

    Benchmarking large language models for news summarization , author=. Transactions of the Association for Computational Linguistics , volume=. 2024 , publisher=

    2024

  11. [11]

    Knowledge and Information Systems , volume=

    Conversational question answering: A survey , author=. Knowledge and Information Systems , volume=. 2022 , publisher=

    2022

  12. [12]

    BELLE : A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering

    Zhang, Taolin and Li, Dongyang and Chen, Qizhou and Wang, Chengyu and He, Xiaofeng. BELLE : A Bi-Level Multi-Agent Reasoning Framework for Multi-Hop Question Answering. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.211

    doi:10.18653/v1/2025.acl-long.211 2025

  13. [13]

    Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering

    Hu, Zhanghao and Yan, Hanqi and Zhu, Qinglin and Shen, Zhenyi and He, Yulan and Gui, Lin. Beyond Prompting: An Efficient Embedding Framework for Open-Domain Question Answering. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.981

    work page doi:10.18653/v1/2025.acl-long.981 2025

  14. [14]

    The Moon is Made of Marshmallows

    FaithEval: Can Your Language Model Stay Faithful to Context, Even If" The Moon is Made of Marshmallows" , author=. arXiv preprint arXiv:2410.03727 , year=

    Pith/arXiv arXiv

  15. [15]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Zero-shot Faithfulness Evaluation for Text Summarization with Foundation Language Model , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    2023

  16. [16]

    Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    The Dangers of trusting Stochastic Parrots: Faithfulness and Trust in Open-domain Conversational Question Answering , author=. Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    2023

  17. [17]

    P od A gent: A Comprehensive Framework for Podcast Generation

    Xiao, Yujia and He, Lei and Guo, Haohan and Xie, Feng-Long and Lee, Tan. P od A gent: A Comprehensive Framework for Podcast Generation. Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.18653/v1/2025.findings-acl.1226

    work page doi:10.18653/v1/2025.findings-acl.1226 2025

  18. [18]

    arXiv preprint arXiv:2503.14345 , year=

    MoonCast: High-quality zero-shot podcast generation , author=. arXiv preprint arXiv:2503.14345 , year=

    Pith/arXiv arXiv

  19. [19]

    arXiv preprint arXiv:2412.10117 , year=

    Cosyvoice 2: Scalable streaming speech synthesis with large language models , author=. arXiv preprint arXiv:2412.10117 , year=

    Pith/arXiv arXiv

  20. [20]

    arXiv preprint arXiv:2303.08774 , year=

    Gpt-4 technical report , author=. arXiv preprint arXiv:2303.08774 , year=

    Pith/arXiv arXiv

  21. [21]

    5 Technical Report , author=

    Qwen2. 5 Technical Report , author=. arXiv e-prints , pages=

  22. [22]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    Pith/arXiv arXiv

  23. [23]

    Mapping the Podcast Ecosystem with the Structured Podcast Research Corpus

    Litterer, Benjamin Roger and Jurgens, David and Card, Dallas. Mapping the Podcast Ecosystem with the Structured Podcast Research Corpus. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.1222

    work page doi:10.18653/v1/2025.acl-long.1222 2025

  24. [24]

    Groundedness in Retrieval-augmented Long-form Generation: An Empirical Study

    Stolfo, Alessandro. Groundedness in Retrieval-augmented Long-form Generation: An Empirical Study. Findings of the Association for Computational Linguistics: NAACL 2024. 2024. doi:10.18653/v1/2024.findings-naacl.100

    doi:10.18653/v1/2024.findings-naacl.100 2024

  25. [25]

    ACM computing surveys , volume=

    Survey of hallucination in natural language generation , author=. ACM computing surveys , volume=. 2023 , publisher=

    2023

  26. [26]

    Findings of the Association for Computational Linguistics: EACL 2024 , pages=

    Do Language Models Know When They’re Hallucinating References? , author=. Findings of the Association for Computational Linguistics: EACL 2024 , pages=

    2024

  27. [27]

    Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    Neural Path Hunter: Reducing Hallucination in Dialogue Systems via Path Grounding , author=. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages=

    2021

  28. [28]

    Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding , author=. Findings of the Association for Computational Linguistics: ACL 2023 , pages=

    2023

  29. [29]

    Trusting Your Evidence: Hallucinate Less with Context-aware Decoding

    Shi, Weijia and Han, Xiaochuang and Lewis, Mike and Tsvetkov, Yulia and Zettlemoyer, Luke and Yih, Wen-tau. Trusting Your Evidence: Hallucinate Less with Context-aware Decoding. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers). 2024. doi:...

    doi:10.18653/v1/2024.naacl-short.69 2024

  30. [30]

    Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models

    Huang, Yanwen and Zhang, Yong and Cheng, Ning and Li, Zhitao and Wang, Shaojun and Xiao, Jing. Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models. Findings of the Association for Computational Linguistics: ACL 2025. 2025. doi:10.18653/v1/2025.findings-acl.269

    work page doi:10.18653/v1/2025.findings-acl.269 2025

  31. [31]

    Findings of the Association for Computational Linguistics: EMNLP 2021 , pages=

    Retrieval Augmentation Reduces Hallucination in Conversation , author=. Findings of the Association for Computational Linguistics: EMNLP 2021 , pages=

    2021

  32. [32]

    Honovich, Or and Choshen, Leshem and Aharoni, Roee and Neeman, Ella and Szpektor, Idan and Abend, Omri , booktitle=

  33. [33]

    Transactions of the Association for Computational Linguistics , volume=

    Faithdial: A faithful benchmark for information-seeking dialogue , author=. Transactions of the Association for Computational Linguistics , volume=. 2022 , publisher=

    2022

  34. [34]

    Advances in neural information processing systems , volume=

    Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in neural information processing systems , volume=

  35. [35]

    Transactions of the Association for Computational Linguistics , volume=

    Evaluating correctness and faithfulness of instruction-following models for question answering , author=. Transactions of the Association for Computational Linguistics , volume=. 2024 , publisher=

    2024

  36. [36]

    Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference

    Warner, Benjamin and Chaffin, Antoine and Clavi. Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2025. doi:10.18653/v1/2025.acl-long.127

    doi:10.18653/v1/2025.acl-long.127 2025

  37. [37]

    , author=

    Lora: Low-rank adaptation of large language models. , author=. ICLR , volume=

  38. [38]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Steering Llama 2 via Contrastive Activation Addition , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  39. [39]

    arXiv preprint arXiv:2308.10248 , year=

    Steering language models with activation engineering , author=. arXiv preprint arXiv:2308.10248 , year=

    Pith/arXiv arXiv

  40. [40]

    context: Fine-grained control of knowledge reliance in language models , author=

    Parameters vs. context: Fine-grained control of knowledge reliance in language models , author=. arXiv preprint arXiv:2503.15888 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2502.09604 , year=

    SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models , author=. arXiv preprint arXiv:2502.09604 , year=

    Pith/arXiv arXiv

  42. [42]

    arXiv preprint arXiv:2502.08767 , year=

    SelfElicit: Your language model secretly knows where is the relevant evidence , author=. arXiv preprint arXiv:2502.08767 , year=

    Pith/arXiv arXiv