REVIEW 8 cited by
KILT: a Benchmark for Knowledge Intensive Language Tasks
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Challenging problems such as open-domain question answering, fact checking, slot filling and entity linking require access to large, external knowledge sources. While some models do well on individual tasks, developing general models is difficult as each task might require computationally expensive indexing of custom knowledge sources, in addition to dedicated infrastructure. To catalyze research on models that condition on specific information in large textual resources, we present a benchmark for knowledge-intensive language tasks (KILT). All tasks in KILT are grounded in the same snapshot of Wikipedia, reducing engineering turnaround through the re-use of components, as well as accelerating research into task-agnostic memory architectures. We test both task-specific and general baselines, evaluating downstream performance in addition to the ability of the models to provide provenance. We find that a shared dense vector index coupled with a seq2seq model is a strong baseline, outperforming more tailor-made approaches for fact checking, open-domain question answering and dialogue, and yielding competitive results on entity linking and slot filling, by generating disambiguated text. KILT data and code are available at https://github.com/facebookresearch/KILT.
Forward citations
Cited by 8 Pith papers
-
ARC-Encoder: learning compressed text representations for large language models
ARC-Encoder pools queries in an encoder's last attention layer to produce compressed continuous representations that a frozen decoder consumes as token embeddings.
-
CROP: Circuit Retrieval and Optimization with Parameter Guidance using LLMs
LLM-based retrieval-augmented tuning of EDA flow parameters found a 9.9% lower-power configuration on one industrial core than classical optimizers.
-
FlexRAG: A Flexible and Comprehensive Framework for Retrieval-Augmented Generation
FlexRAG is a modular, open-source RAG framework with text, multimodal, and web retrieval, plus evaluation tools and efficient memory-mapped indexing.
-
ComposeRAG: A Modular and Composable RAG for Corpus-Grounded Multi-Hop Question Answering
A modular, verifier-driven RAG pipeline with iterative re-decomposition outperforms fine-tuned and agentic baselines on four multi-hop QA benchmarks.
-
HF-RAG: Hierarchical Fusion-based RAG with Multiple Sources and Rankers
By first fusing multiple retrievers within labeled and unlabeled sources with RRF, then merging z-score normalized lists, HF-RAG improves fact-verification F1 in-domain and out-of-domain.
-
DiffLoRA: Differential Low-Rank Adapters for Large Language Models
DiffLoRA, a low-rank adapter variant of differential attention, generally underperforms LoRA but shows gains on HumanEval and multi-value needle retrieval.
-
MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAG
A fully on-device RAG pipeline using a partitioned, partially disk-loaded graph index and selective sentence-window reduction claims 1.72-8.89x faster vector search and up to 40.2% lower power than baselines, with com...
-
A comprehensive taxonomy of hallucinations in Large Language Models
A survey that organizes LLM hallucination types, causes, benchmarks, and mitigations, and restates the theorem that hallucination is inevitable for computable LLMs.
Discussion (0). Sign in to comment.