{"record_type":"pith_number_record","schema_url":"https://pith.science/schemas/pith-number/v1.json","pith_number":"pith:2024:P7K7CM6TL6FGXDI4LGZV6ZE7AH","short_pith_number":"pith:P7K7CM6T","schema_version":"1.0","canonical_sha256":"7fd5f133d35f8a6b8d1c59b35f649f01d8f1f0caf3e9cd59f9e22a35a255e93b","source":{"kind":"arxiv","id":"2406.04744","version":2},"attestation_state":"computed","paper":{"title":"CRAG -- Comprehensive RAG Benchmark","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Anuj Kumar, An Yan, Brian Moran, Chenyu Yang, Eting Yuan, Hanwen Zha, Hao Xin, Jiaqi Wang, Kai Sun, Lei Chen, Lingkun Kong, Nan Tang, Nicolas Scheffer, Nikita Bhalla, Nirav Shah, Rakesh Wanga, Rongze Daniel Gui, Sajal Choudhary, Wen-tau Yih, Xiangsen Chen, Xiao Yang, Xin Luna Dong, Yifan Ethan Xu, Yue Liu, Yushi Sun, Ziran Will Jiang, Ziyu Jiang","submitted_at":"2024-06-07T08:43:07Z","abstract_excerpt":"Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution to alleviate Large Language Model (LLM)'s deficiency in lack of knowledge. Existing RAG datasets, however, do not adequately represent the diverse and dynamic nature of real-world Question Answering (QA) tasks. To bridge this gap, we introduce the Comprehensive RAG Benchmark (CRAG), a factual question answering benchmark of 4,409 question-answer pairs and mock APIs to simulate web and Knowledge Graph (KG) search. CRAG is designed to encapsulate a diverse array of questions across five domains and eight question c"},"verification_status":{"content_addressed":true,"pith_receipt":true,"author_attested":false,"weak_author_claims":0,"strong_author_claims":0,"externally_anchored":false,"storage_verified":false,"citation_signatures":0,"replication_records":0,"graph_snapshot":true,"references_resolved":false,"formal_links_present":false},"canonical_record":{"source":{"id":"2406.04744","kind":"arxiv","version":2},"metadata":{"license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","primary_cat":"cs.CL","submitted_at":"2024-06-07T08:43:07Z","cross_cats_sorted":[],"title_canon_sha256":"f0c5b7515da838a6e14bce3e321b9d819d02f2e7bdc1441184c34e7d88aa33ef","abstract_canon_sha256":"2d4e588751b9fbf80a532e08f4edb4b202c15ca7a89c21a41957a68ec1afd0d2"},"schema_version":"1.0"},"receipt":{"kind":"pith_receipt","key_id":"pith-v1-2026-05","algorithm":"ed25519","signed_at":"2026-07-05T09:29:30.154750Z","signature_b64":"vzmwS9S9RZ/YF+cKOChpecCx3xWHqm977wXLFWPd1nTNQuzMgsItWyJRNrRbZh54MHgl+/R3yEsXBWhaljVQAA==","signed_message":"canonical_sha256_bytes","builder_version":"pith-number-builder-2026-05-17-v1","receipt_version":"0.3","canonical_sha256":"7fd5f133d35f8a6b8d1c59b35f649f01d8f1f0caf3e9cd59f9e22a35a255e93b","last_reissued_at":"2026-07-05T09:29:30.154221Z","signature_status":"signed_v1","first_computed_at":"2026-07-05T09:29:30.154221Z","public_key_fingerprint":"8d4b5ee74e4693bcd1df2446408b0d54"},"graph_snapshot":{"paper":{"title":"CRAG -- Comprehensive RAG Benchmark","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"","cross_cats":[],"primary_cat":"cs.CL","authors_text":"Anuj Kumar, An Yan, Brian Moran, Chenyu Yang, Eting Yuan, Hanwen Zha, Hao Xin, Jiaqi Wang, Kai Sun, Lei Chen, Lingkun Kong, Nan Tang, Nicolas Scheffer, Nikita Bhalla, Nirav Shah, Rakesh Wanga, Rongze Daniel Gui, Sajal Choudhary, Wen-tau Yih, Xiangsen Chen, Xiao Yang, Xin Luna Dong, Yifan Ethan Xu, Yue Liu, Yushi Sun, Ziran Will Jiang, Ziyu Jiang","submitted_at":"2024-06-07T08:43:07Z","abstract_excerpt":"Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution to alleviate Large Language Model (LLM)'s deficiency in lack of knowledge. Existing RAG datasets, however, do not adequately represent the diverse and dynamic nature of real-world Question Answering (QA) tasks. To bridge this gap, we introduce the Comprehensive RAG Benchmark (CRAG), a factual question answering benchmark of 4,409 question-answer pairs and mock APIs to simulate web and Knowledge Graph (KG) search. CRAG is designed to encapsulate a diverse array of questions across five domains and eight question c"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2406.04744","kind":"arxiv","version":2},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2406.04744/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"},"aliases":[{"alias_kind":"arxiv","alias_value":"2406.04744","created_at":"2026-07-05T09:29:30.154279+00:00"},{"alias_kind":"arxiv_version","alias_value":"2406.04744v2","created_at":"2026-07-05T09:29:30.154279+00:00"},{"alias_kind":"doi","alias_value":"10.48550/arxiv.2406.04744","created_at":"2026-07-05T09:29:30.154279+00:00"},{"alias_kind":"pith_short_12","alias_value":"P7K7CM6TL6FG","created_at":"2026-07-05T09:29:30.154279+00:00"},{"alias_kind":"pith_short_16","alias_value":"P7K7CM6TL6FGXDI4","created_at":"2026-07-05T09:29:30.154279+00:00"},{"alias_kind":"pith_short_8","alias_value":"P7K7CM6T","created_at":"2026-07-05T09:29:30.154279+00:00"}],"events":[],"event_summary":{},"paper_claims":[],"inbound_citations":{"count":9,"internal_anchor_count":0,"sample":[{"citing_arxiv_id":"2606.05415","citing_title":"Executable Schema Contracts: From Automatic Ingestion to Multi-Source Retrieval","ref_index":2,"is_internal_anchor":false},{"citing_arxiv_id":"2606.30306","citing_title":"Always-OnAgents:A Survey of Persistent Memory, State, and Governance in LLMAgents","ref_index":47,"is_internal_anchor":false},{"citing_arxiv_id":"2604.26176","citing_title":"CacheRAG: A Semantic Caching System for Retrieval-Augmented Generation in Knowledge Graph Question Answering","ref_index":31,"is_internal_anchor":false},{"citing_arxiv_id":"2606.30062","citing_title":"Little Brains, Big Feats: Exploring Compact Language Models","ref_index":39,"is_internal_anchor":false},{"citing_arxiv_id":"2604.16310","citing_title":"RAG-DIVE: A Dynamic Approach for Multi-Turn Dialogue Evaluation in Retrieval-Augmented Generation","ref_index":38,"is_internal_anchor":false},{"citing_arxiv_id":"2604.20850","citing_title":"Association Is Not Similarity: Learning Corpus-Specific Associations for Multi-Hop Retrieval","ref_index":4,"is_internal_anchor":false},{"citing_arxiv_id":"2604.06211","citing_title":"Illocutionary Explanation Planning for Source-Faithful Explanations in Retrieval-Augmented Language Models","ref_index":53,"is_internal_anchor":false},{"citing_arxiv_id":"2504.19678","citing_title":"From LLM Reasoning to Autonomous AI Agents: A Comprehensive Review","ref_index":80,"is_internal_anchor":false},{"citing_arxiv_id":"2604.26176","citing_title":"CacheRAG: A Semantic Caching System for Retrieval-Augmented Generation in Knowledge Graph Question Answering","ref_index":33,"is_internal_anchor":false}]},"formal_canon":{"evidence_count":0,"sample":[],"anchors":[]},"links":{"html":"https://pith.science/pith/P7K7CM6TL6FGXDI4LGZV6ZE7AH","json":"https://pith.science/pith/P7K7CM6TL6FGXDI4LGZV6ZE7AH.json","graph_json":"https://pith.science/api/pith-number/P7K7CM6TL6FGXDI4LGZV6ZE7AH/graph.json","events_json":"https://pith.science/api/pith-number/P7K7CM6TL6FGXDI4LGZV6ZE7AH/events.json","paper":"https://pith.science/paper/P7K7CM6T"},"agent_actions":{"view_html":"https://pith.science/pith/P7K7CM6TL6FGXDI4LGZV6ZE7AH","download_json":"https://pith.science/pith/P7K7CM6TL6FGXDI4LGZV6ZE7AH.json","view_paper":"https://pith.science/paper/P7K7CM6T","resolve_alias":"https://pith.science/api/pith-number/resolve?arxiv=2406.04744&json=true","fetch_graph":"https://pith.science/api/pith-number/P7K7CM6TL6FGXDI4LGZV6ZE7AH/graph.json","fetch_events":"https://pith.science/api/pith-number/P7K7CM6TL6FGXDI4LGZV6ZE7AH/events.json","actions":{"anchor_timestamp":"https://pith.science/pith/P7K7CM6TL6FGXDI4LGZV6ZE7AH/action/timestamp_anchor","attest_storage":"https://pith.science/pith/P7K7CM6TL6FGXDI4LGZV6ZE7AH/action/storage_attestation","attest_author":"https://pith.science/pith/P7K7CM6TL6FGXDI4LGZV6ZE7AH/action/author_attestation","sign_citation":"https://pith.science/pith/P7K7CM6TL6FGXDI4LGZV6ZE7AH/action/citation_signature","submit_replication":"https://pith.science/pith/P7K7CM6TL6FGXDI4LGZV6ZE7AH/action/replication_record"}},"created_at":"2026-07-05T09:29:30.154279+00:00","updated_at":"2026-07-05T09:29:30.154279+00:00"}