The paper defines Prosecution Decision Prediction (PDP) and PDP-Bench with 4,630 cases, showing LLMs perform worse on PDP than LJP and that outcome-based RLVR fails to improve generalization.
CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
abstract
In this paper, we introduce the \textbf{C}hinese \textbf{AI} and \textbf{L}aw challenge dataset (CAIL2018), the first large-scale Chinese legal dataset for judgment prediction. \dataset contains more than $2.6$ million criminal cases published by the Supreme People's Court of China, which are several times larger than other datasets in existing works on judgment prediction. Moreover, the annotations of judgment results are more detailed and rich. It consists of applicable law articles, charges, and prison terms, which are expected to be inferred according to the fact descriptions of cases. For comparison, we implement several conventional text classification baselines for judgment prediction and experimental results show that it is still a challenge for current models to predict the judgment results of legal cases, especially on prison terms. To help the researchers make improvements on legal judgment prediction, both \dataset and baselines will be released after the CAIL competition\footnote{http://cail.cipsc.org.cn/}.
fields
cs.CL 6verdicts
UNVERDICTED 6representative citing papers
TypedCSIP applies typed counterfactual selective intervention pretraining on expert revisions to lift macro-F1 by 0.9-1.3 pp on the LCR-CN Chinese legislative conflict classification benchmark under a pre-registered multi-seed test.
Retrieval with frozen embeddings and k-NN delivers competitive accuracy, high data efficiency, and zero hallucinations on legal multi-label annotation across ECtHR and Eurlex datasets.
Internal LLM artifacts can be used to build classifiers that identify incorrect predictions on legal classification tasks.
LegalGraphRAG adds hierarchical organization to legal knowledge graphs and a multi-agent verification loop to reach claimed state-of-the-art accuracy and trustworthiness on legal reasoning benchmarks.
A two-phase external-knowledge plus number-learning-network method for multi-label charge prediction yields 3-5% macro-F1 and 5-15% micro-F1 gains when added to existing deep models on a Chinese legal dataset.
citing papers explorer
-
The Cases LJP Never Sees: Prosecution Decision Prediction for More Complete Criminal Liability Assessment
The paper defines Prosecution Decision Prediction (PDP) and PDP-Bench with 4,630 cases, showing LLMs perform worse on PDP than LJP and that outcome-based RLVR fails to improve generalization.
-
TypedCSIP: Typed Counterfactual Pretraining for Chinese Legislative Conflict Classification
TypedCSIP applies typed counterfactual selective intervention pretraining on expert revisions to lift macro-F1 by 0.9-1.3 pp on the LCR-CN Chinese legislative conflict classification benchmark under a pre-registered multi-seed test.
-
Retrieval-Based Multi-Label Legal Annotation: Extensible, Data-Efficient and Hallucination-Free
Retrieval with frozen embeddings and k-NN delivers competitive accuracy, high data efficiency, and zero hallucinations on legal multi-label annotation across ECtHR and Eurlex datasets.
-
Peeking Inside LLMs: Leveraging Internal Artifacts of LLMs for Enhancing Reliability in Legal Classification
Internal LLM artifacts can be used to build classifiers that identify incorrect predictions on legal classification tasks.
-
LegalGraphRAG: Multi-Agent Graph Retrieval-Augmented Generation for Reliable Legal Reasoning
LegalGraphRAG adds hierarchical organization to legal knowledge graphs and a multi-agent verification loop to reach claimed state-of-the-art accuracy and trustworthiness on legal reasoning benchmarks.