REVIEW 16 cited by
LEGAL-BERT: The Muppets straight out of Law School
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
BERT has achieved impressive performance in several NLP tasks. However, there has been limited investigation on its adaptation guidelines in specialised domains. Here we focus on the legal domain, where we explore several approaches for applying BERT models to downstream legal tasks, evaluating on multiple datasets. Our findings indicate that the previous guidelines for pre-training and fine-tuning, often blindly followed, do not always generalize well in the legal domain. Thus we propose a systematic investigation of the available strategies when applying BERT in specialised domains. These are: (a) use the original BERT out of the box, (b) adapt BERT by additional pre-training on domain-specific corpora, and (c) pre-train BERT from scratch on domain-specific corpora. We also propose a broader hyper-parameter search space when fine-tuning for downstream tasks and we release LEGAL-BERT, a family of BERT models intended to assist legal NLP research, computational law, and legal technology applications.
Forward citations
Cited by 16 Pith papers
-
SteuerLLM: Local specialized large language model for German tax law analysis
A tax-specialized 28B model beats larger general-purpose LLMs on a new authentic German tax-law exam benchmark, but its edge may be inflated by overlap between training and evaluation exams.
-
Health Insurance Coverage Rule Interpretation Corpus: Law, Policy, and Medical Guidance for Health Insurance Coverage Understanding
A new corpus and a pseudo-annotated benchmark for predicting health insurance external appeal outcomes, with baseline transformer models.
-
The Missing Link: Joint Legal Citation Prediction using Heterogeneous Graph Enrichment
A graph neural network that enriches legal citation graphs with categorical metadata nodes predicts case and law citations more accurately than prior GNN baselines, and joint training boosts case citation prediction.
-
BriefMe: A Legal NLP Benchmark for Assisting with Legal Briefs
BriefMe introduces a legal brief benchmark with argument summarization, argument completion, and case retrieval, and shows LLMs beat human headings on the first two but struggle on the latter two.
-
CHANCERY: Evaluating Corporate Governance Reasoning Capabilities in Language Models
A 502-question benchmark for corporate governance reasoning shows current language models reach at most 78.1 percent accuracy.
-
LAARA: Layer-Aware Adaptive Rank Allocation for Parameter-Efficient Fine-Tuning
LAARA allocates LoRA ranks per layer from diagonal Fisher (gradient-based) estimates, reporting improved accuracy with fewer trainable parameters on GLUE and MathInstruct.
-
LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documents
On CUAD legal contracts, a prompt-engineered QWEN-2 pipeline with chunking and two answer-selection heuristics reportedly outperforms the fine-tuned DeBERTa-large baseline by about 9%, reaching claimed state-of-the-ar...
-
EMA Without the Lag: Bias-Corrected Iterate Averaging Schemes
A bias-corrected exponential moving average (BEMA) is claimed to remove the lag of standard EMA weight averaging during LLM fine-tuning, improving convergence and final performance over EMA and vanilla training.
-
TRIDENT: Benchmarking LLM Safety in Finance, Medicine, and Law
Trident-Bench provides 2,652 professionally validated harmful prompts across finance, law, and medicine, and shows that domain-specialized LLMs often comply with unethical requests more than generalist models.
-
PromptAL: Sample-Aware Dynamic Soft Prompts for Few-Shot Active Learning
PromptAL combines sample-aware dynamic soft prompts with uncertainty and diversity scores to select better annotation candidates in few-shot active learning.
-
ASP2LJ : An Adversarial Self-Play Laywer Augmented Legal Judgment Framework
ASP2LJ combines synthetic case generation with adversarial self-play for lawyer agents, improving legal judgment prediction on a Chinese benchmark and on a new rare-case dataset.
-
Hybrid Topic-Semantic Labeling and Graph Embeddings for Unsupervised Legal Document Clustering
Concatenating Top2Vec and Node2Vec embeddings, where the Node2Vec graph encodes Top2Vec's own topic labels, yields compact clusters, but the gain is largely circular.
-
L-MARS: Legal Multi-Agent System with Agentic Search and Citation-Faithfulness Audit
The abstract reports that a judge-driven multi-agent loop improves legal-citation faithfulness from 0.13 to 0.25 strict F1 and cuts the no-citation rate from 34% to 13%, but the provided manuscript text does not conta...
-
A Data Science Approach to Calcutta High Court Judgments: An Efficient LLM and RAG-powered Framework for Summarization and Similar Cases Retrieval
Fine-tuning Pegasus on LLM-annotated headnotes improves part of the legal summarization pipeline, and a RAG framework retrieves similar Calcutta High Court cases, though retrieval quality is never measured.
-
CoLA: Collaborative Low-Rank Adaptation
CoLA generalizes LoRA to multiple A and B matrices with a principal-component initialization and reports gains of roughly 2-4 accuracy points over PiSSA on low-sample fine-tuning benchmarks.
-
LLMPR: A Novel LLM-Driven Transfer Learning based Petition Ranking Model
A petition-ranking model that reports near-perfect accuracy, but its target ranking is derived from the same gap-days features it feeds the model, making the result circular.
Discussion (0). Sign in to comment.