REVIEW 4 cited by
CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Legal professionals need to write analyses that rely on citations to relevant precedents, i.e., previous case decisions. Intelligent systems assisting legal professionals in writing such documents provide great benefits but are challenging to design. Such systems need to help locate, summarize, and reason over salient precedents in order to be useful. To enable systems for such tasks, we work with legal professionals to transform a large open-source legal corpus into a dataset supporting two important backbone tasks: information retrieval (IR) and retrieval-augmented generation (RAG). This dataset CLERC (Case Law Evaluation Retrieval Corpus), is constructed for training and evaluating models on their ability to (1) find corresponding citations for a given piece of legal analysis and to (2) compile the text of these citations (as well as previous context) into a cogent analysis that supports a reasoning goal. We benchmark state-of-the-art models on CLERC, showing that current approaches still struggle: GPT-4o generates analyses with the highest ROUGE F-scores but hallucinates the most, while zero-shot IR models only achieve 48.3% recall@1000.
Forward citations
Cited by 4 Pith papers
-
How Can AI Augment Access to Justice? Public Defenders' Perspectives on AI Adoption
Public defenders view AI as most useful for evidence investigation but limited in courtroom work and strategy, with adoption blocked by costs, confidentiality risks, and norms, requiring human oversight and open development.
-
CPA-RAG:Covert Poisoning Attacks on Retrieval-Augmented Generation in Large Language Models
A black-box poisoning framework, CPA-RAG, generates fluent fake documents that steer retrieval-augmented language models toward attacker-chosen wrong answers, achieving over 90% success in the reported experiments.
-
Assessing the Performance Gap Between Lexical and Semantic Models for Information Retrieval With Formulaic Legal Language
On CJEU paragraph retrieval, BM25 beats off-the-shelf dense models on most metrics, fine-tuned dense models beat BM25, and BM25 wins mainly when queries have less verbatim overlap with the target.
-
ASP2LJ : An Adversarial Self-Play Laywer Augmented Legal Judgment Framework
ASP2LJ combines synthetic case generation with adversarial self-play for lawyer agents, improving legal judgment prediction on a Chinese benchmark and on a new rare-case dataset.
Discussion (0). Continue with ORCID to comment.