REVIEW 21 cited by
Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI). Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. But the large language models used in these tools are prone to "hallucinate," or make up false information, making their use risky in high-stakes domains. Recently, certain legal research providers have touted methods such as retrieval-augmented generation (RAG) as "eliminating" (Casetext, 2023) or "avoid[ing]" hallucinations (Thomson Reuters, 2023), or guaranteeing "hallucination-free" legal citations (LexisNexis, 2023). Because of the closed nature of these systems, systematically assessing these claims is challenging. In this article, we design and report on the first preregistered empirical evaluation of AI-driven legal research tools. We demonstrate that the providers' claims are overstated. While hallucinations are reduced relative to general-purpose chatbots (GPT-4), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also document substantial differences between systems in responsiveness and accuracy. Our article makes four key contributions. It is the first to assess and report the performance of RAG-based proprietary legal AI tools. Second, it introduces a comprehensive, preregistered dataset for identifying and understanding vulnerabilities in these systems. Third, it proposes a clear typology for differentiating between hallucinations and accurate legal responses. Last, it provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains a central open question for the responsible integration of AI into law.
Forward citations
Cited by 21 Pith papers
-
AI for Statutory Simplification: A Comprehensive State Legal Corpus and Labor Benchmark
State-of-the-art LLMs with retrieval answer simplified boolean questions about state unemployment insurance law with at best 0.69 F1, well short of reliable end-to-end code simplification.
-
Linguistic Nepotism: Trading-off Quality for Language Preference in Multilingual RAG
In multilingual retrieval-augmented generation, models cite English evidence more accurately than translated evidence, and this language preference can outweigh document relevance.
-
AI Self-preferencing in Algorithmic Hiring: Empirical Evidence and Insights
LLMs that screen resumes systematically prefer their own generated summaries over human-written ones, with simulated shortlisting advantages of 23 to 60 percent for same-model users.
-
(Fact) Check Your Bias
Biased prompts change the evidence an LLM fact-checker retrieves but barely change its verdicts, while safety refusals create an asymmetric negative bias in evidence collection.
-
A Reasoning-Focused Legal Retrieval Benchmark
Two hand-annotated legal RAG benchmarks with low query-passage lexical overlap show that existing retrievers struggle, and that structured legal reasoning query expansion improves retrieval.
-
AI for Scaling Legal Reform: Mapping and Redacting Racial Covenants in Santa Clara County
A fine-tuned open language model detected thousands of racial covenants in Santa Clara County deed records at low cost, leading to an estimate that one in four county properties was racially restricted by 1950.
-
CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering
Introduces a 2,000-question benchmark where ambiguous QA queries are annotated with explicit conditions, and shows that condition-aware answering improves accuracy across seven LLMs.
-
LLMs & Legal Aid: Understanding Legal Needs Exhibited Through User Queries
In a real-world Czech legal aid LLM deployment, 70% of user queries contained no factual details, 65% requested legal information, and 71% let the model shape the answer.
-
Evaluating LLM-based Approaches to Legal Citation Prediction: Domain-specific Pre-training, Fine-tuning, or RAG? A Benchmark and an Australian Law Case Study
The paper releases the AusLaw Citation Benchmark and shows that instruction-tuned 7B-8B LLMs plus retrieval re-ranking outperform general and law-specific pretrained LLMs for legal citation prediction, reaching about ...
-
Automating Feedback Analysis in Surgical Training: Detection, Categorization, and Assessment
An automated pipeline reconstructs surgical teaching dialogues from operating-room audio and detects feedback, with a hallucination-filtering step that improves detection by about 14%.
-
Zero Hallucination, by Construction: Hallucination-Aware Layered Oversight for Trustworthy Enterprise AI
HALO is a layered oversight architecture that grounds, constrains, verifies, abstains, traces, and monitors LLM outputs to make hallucinations containable rather than eliminated.
-
The Consistency-Acceptability Divergence of LLMs in Judicial Decision-Making: Task and Stakeholder Dimensions
The paper introduces the consistency-acceptability divergence concept and proposes the DTDMR-LJGF framework for governing LLMs in judicial decision-making.
-
BR-TaxQA-R: A Dataset for Question Answering with References for Brazilian Personal Income Tax Law, including case law
BR-TaxQA-R: a 715-question Brazilian personal income tax QA dataset with statutory and case-law references, plus a RAG baseline that beats commercial chatbots on response relevancy but not on factual correctness.
-
Hermit Kingdom Through the Lens of Multiple Perspectives: A Case Study of LLM Hallucination on North Korea
Across five LLMs and three languages, answers to the same 13 fact-checkable North Korea prompts differ substantially by model and by language, with refusal rates and accuracy varying.
-
Inteligencia Artificial jur\'idica y el desaf\'io de la veracidad: an\'alisis de alucinaciones, optimizaci\'on de RAG y principios para una integraci\'on responsable
Legal AI hallucination persists in commercial RAG tools (17-34%+ of queries), so the report argues the fix is consultative, source-citing system design plus mandatory human oversight, not better generative models.
-
Data and AI governance: Promoting equity, ethics, and fairness in large language models
The paper proposes a lifecycle governance framework, built on the authors' BEATS benchmark, to quantify and mitigate bias, ethics, fairness, and factuality failures in large language models.
-
Incorporating Legal Structure in Retrieval-Augmented Generation: A Case Study on Copyright Fair Use
A fair-use legal assistant that reranks RAG results using PageRank and court hierarchy retrieves more highly cited opinions than semantic search alone, but the evaluation is partly circular.
-
The Impact of AI on the Cyber Offense-Defense Balance and the Character of Cyber Conflict
After reviewing 66 arguments about cyber conflict, the paper concludes AI's effect on the offense-defense balance is mixed and lists 44 pathways through which AI could change cyber conflict.
-
Artificial Intelligence and Legal Analysis: Implications for Legal Education and the Profession
In a seven-exercise IRAC study, general-purpose LLMs outperformed Lexis+ AI, and all tested LLMs showed limitations including brevity, hedging, false confidence, and hallucinations.
-
A Comprehensive Framework for Reliable Legal AI: Combining Specialized Expert Systems and Adaptive Refinement
The paper proposes a hybrid legal AI architecture and claims large accuracy gains, but it presents no numerical evidence, code, or data to support the claim.
-
Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models
A survey of LLM hallucination research that formalizes hallucination types and argues, via incompleteness and undecidability arguments, that hallucinations cannot be fully eliminated.
Discussion (0). Continue with ORCID to comment.