{"paper":{"title":"Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback","license":"http://creativecommons.org/licenses/by/4.0/","headline":"LLM-Augmenter augments black-box models like ChatGPT with external knowledge modules and automated feedback to reduce hallucinations while preserving fluency.","cross_cats":["cs.AI"],"primary_cat":"cs.CL","authors_text":"Baolin Peng, Hao Cheng, Jianfeng Gao, Lars Liden, Michel Galley, Pengcheng He, Qiuyuan Huang, Weizhu Chen, Yu Hu, Yujia Xie, Zhou Yu","submitted_at":"2023-02-24T18:48:43Z","abstract_excerpt":"Large language models (LLMs), such as ChatGPT, are able to generate human-like, fluent responses for many downstream tasks, e.g., task-oriented dialog and question answering. However, applying LLMs to real-world, mission-critical applications remains challenging mainly due to their tendency to generate hallucinations and their inability to use external knowledge. This paper proposes a LLM-Augmenter system, which augments a black-box LLM with a set of plug-and-play modules. Our system makes the LLM generate responses grounded in external knowledge, e.g., stored in task-specific databases. It al"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"LLM-Augmenter significantly reduces ChatGPT's hallucinations without sacrificing the fluency and informativeness of its responses.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That utility functions such as factuality scores can reliably detect and guide correction of hallucinations without introducing new errors or degrading response quality.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"LLM-Augmenter reduces hallucinations in LLMs like ChatGPT by grounding responses in external knowledge and using automated feedback loops to iteratively revise outputs.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"LLM-Augmenter augments black-box models like ChatGPT with external knowledge modules and automated feedback to reduce hallucinations while preserving fluency.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"6008599ef363fb51d705b05be76bbedb830d302b6294e29ef95c0adbb64ecb56"},"source":{"id":"2302.12813","kind":"arxiv","version":3},"verdict":{"id":"57f018a2-bad6-4d6b-9075-36d0b856329c","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-17T09:56:55.544089Z","strongest_claim":"LLM-Augmenter significantly reduces ChatGPT's hallucinations without sacrificing the fluency and informativeness of its responses.","one_line_summary":"LLM-Augmenter reduces hallucinations in LLMs like ChatGPT by grounding responses in external knowledge and using automated feedback loops to iteratively revise outputs.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That utility functions such as factuality scores can reliably detect and guide correction of hallucinations without introducing new errors or degrading response quality.","pith_extraction_headline":"LLM-Augmenter augments black-box models like ChatGPT with external knowledge modules and automated feedback to reduce hallucinations while preserving fluency."},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"d242727f8af11c7ba870a1ddea1a8d1ede51b6906cbd44015231a29e4d082674"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}