Activation patching reveals that citation decisions in Llama-3.1-8B RAG are implemented by a distributed attributional ensemble of heads and layers; targeted interventions fix most missed and spurious citations on PopQA.
Hallucination-free? assessing the reliability of leading ai legal research tools
11 Pith papers cite this work, alongside 34 external citations. Polarity classification is still indexing.
11
Pith papers citing it
34
external citations · external index
citation-role summary
background 2
citation-polarity summary
roles
background 2representative citing papers
The work creates a new ground truth dataset for mapping queries to verbatim text spans in research papers and shows a 150M-parameter ModernBERT token classifier achieving 53.6 word-level F1, outperforming LLM extractors at 48.7.
TRIVIA+ is a new long-context RAG hallucination benchmark with four noisy label variants that shows current detectors have substantial room for improvement and are hindered by label noise.
Expert evaluation of LLMs on Japanese bar exam writing tasks shows clear limitations in open-ended legal reasoning and frequent hallucinations unsupported by law or precedent.
A new structured prompting method (SPEC) helps AI detect insufficient evidence in adjudication tasks and defer decisions appropriately, reaching 89% accuracy on a benchmark varying information completeness from Colorado unemployment insurance cases.
Tool use in LLMs improves final-answer accuracy but degrades reasoning quality through Tool-Induced Myopia, with the effect worsening as tool calls increase and shifting errors toward logic and assumption failures.
GrACE is a fine-tuned generative method that uses similarity to a special token embedding for real-time calibrated confidence in LLMs and enables efficient confidence-based test-time scaling.
Authority-style prefixes increase refusal rates 2-20x in small on-prem LLMs on criminal legal prompts while a jailbreak prefix shows mixed effects.
Uncertainty-aware fine-tuning with a decision-theory-based loss produces better-calibrated uncertainty estimates than standard training on free-form QA tasks.
LegalCheck automates drafting of municipal legal advice letters via RAG and CAG, producing near-final drafts in minutes with 80-100% coverage of essential legal reasoning in an Amsterdam deployment.
citing papers explorer
-
How Do LLMs Cite? A Mechanistic Interpretation of Attribution in Retrieval-Augmented Generation
Activation patching reveals that citation decisions in Llama-3.1-8B RAG are implemented by a distributed attributional ensemble of heads and layers; targeted interventions fix most missed and spurious citations on PopQA.
-
ACL-Verbatim: hallucination-free question answering for research
The work creates a new ground truth dataset for mapping queries to verbatim text spans in research papers and shows a 150M-parameter ModernBERT token classifier achieving 53.6 word-level F1, outperforming LLM extractors at 48.7.
-
Rethinking Evaluation for LLM Hallucination Detection: A Desiderata, A New RAG-based Benchmark, New Insights
TRIVIA+ is a new long-context RAG hallucination benchmark with four noisy label variants that shows current detectors have substantial room for improvement and are hindered by label noise.
-
Expert Evaluation of LLM's Open-Ended Legal Reasoning on the Japanese Bar Exam Writing Task
Expert evaluation of LLMs on Japanese bar exam writing tasks shows clear limitations in open-ended legal reasoning and frequent hallucinations unsupported by law or precedent.
-
Learning When Not to Decide: A Framework for Overcoming Factual Presumptuousness in AI Adjudication
A new structured prompting method (SPEC) helps AI detect insufficient evidence in adjudication tasks and defer decisions appropriately, reaching 89% accuracy on a benchmark varying information completeness from Colorado unemployment insurance cases.
-
From Proof to Program: Characterizing Tool-Induced Reasoning Hallucinations in Large Language Models
Tool use in LLMs improves final-answer accuracy but degrades reasoning quality through Tool-Induced Myopia, with the effect worsening as tool calls increase and shifting errors toward logic and assumption failures.
-
GrACE: A Generative Approach to Better Confidence Elicitation and Efficient Test-Time Scaling in Large Language Models
GrACE is a fine-tuned generative method that uses similarity to a special token embedding for real-time calibrated confidence in LLMs and enables efficient confidence-based test-time scaling.
-
LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context
Authority-style prefixes increase refusal rates 2-20x in small on-prem LLMs on criminal legal prompts while a jailbreak prefix shows mixed effects.
-
Enhancing Trust in Large Language Models via Uncertainty-Calibrated Fine-Tuning
Uncertainty-aware fine-tuning with a decision-theory-based loss produces better-calibrated uncertainty estimates than standard training on free-form QA tasks.
-
LegalCheck: Retrieval- and Context-Augmented Generation for Drafting Municipal Legal Advice Letters
LegalCheck automates drafting of municipal legal advice letters via RAG and CAG, producing near-final drafts in minutes with 80-100% coverage of essential legal reasoning in an Amsterdam deployment.
- Citation Grounding Measures the Oracle: Graph Coverage Determines Reported LLM Hallucination Rates in Law