CaseSumm, a 25.6K-pair dataset of Supreme Court opinions and official syllabuses, shows automated metrics favor fine-tuned Mistral while human experts prefer GPT-4, and LLM judges do not align with humans better than ROUGE.
Legal Extractive Summarization of U.S. Court Opinions
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
This paper tackles the task of legal extractive summarization using a dataset of 430K U.S. court opinions with key passages annotated. According to automated summary quality metrics, the reinforcement-learning-based MemSum model is best and even out-performs transformer-based models. In turn, expert human evaluation shows that MemSum summaries effectively capture the key points of lengthy court opinions. Motivated by these results, we open-source our models to the general public. This represents progress towards democratizing law and making U.S. court opinions more accessible to the general public.
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
CaseSumm: A Large-Scale Dataset for Long-Context Summarization from U.S. Supreme Court Opinions
CaseSumm, a 25.6K-pair dataset of Supreme Court opinions and official syllabuses, shows automated metrics favor fine-tuned Mistral while human experts prefer GPT-4, and LLM judges do not align with humans better than ROUGE.