REVIEW 3 cited by
Legal Prompting: Teaching a Language Model to Think Like a Lawyer
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models that are capable of zero or few-shot prompting approaches have given rise to the new research area of prompt engineering. Recent advances showed that for example Chain-of-Thought (CoT) prompts can improve arithmetic or common sense tasks significantly. We explore how such approaches fare with legal reasoning tasks and take the COLIEE entailment task based on the Japanese Bar exam for testing zero-shot/few-shot and fine-tuning approaches. Our findings show that while CoT prompting and fine-tuning with explanations approaches show improvements, the best results are produced by prompts that are derived from specific legal reasoning techniques such as IRAC (Issue, Rule, Application, Conclusion). Based on our experiments we improve the 2021 best result from 0.7037 accuracy to 0.8148 accuracy and beat the 2022 best system of 0.6789 accuracy with an accuracy of 0.7431.
Forward citations
Cited by 3 Pith papers
-
From Judgments to Issues: Structured Extraction of Legal Reasoning with Citation-Hallucination Control
A cost-efficient DeepSeek-V3 pipeline extracts IRAC-grounded issue-level XML from ~330k Italian tax judgments and cuts citation hallucinations from 11.7% to 0.9% via Linkoln matching, validated by two tax-law PhDs on ...
-
LLMs for LLMs: A Structured Prompting Methodology for Long Legal Documents
On CUAD legal contracts, a prompt-engineered QWEN-2 pipeline with chunking and two answer-selection heuristics reportedly outperforms the fine-tuned DeBERTa-large baseline by about 9%, reaching claimed state-of-the-ar...
-
Prompt Engineering Guidelines for Using Large Language Models in Requirements Engineering
A literature review and three expert interviews yield a proposed mapping of prompt engineering guideline themes onto five requirements engineering activities, with no empirical validation of the mapping.
Discussion (0). Sign in to comment.