REVIEW 2 cited by
Promises and pitfalls of artificial intelligence for legal applications
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Is AI set to redefine the legal profession? We argue that this claim is not supported by the current evidence. We dive into AI's increasingly prevalent roles in three types of legal tasks: information processing; tasks involving creativity, reasoning, or judgment; and predictions about the future. We find that the ease of evaluating legal applications varies greatly across legal tasks, based on the ease of identifying correct answers and the observability of information relevant to the task at hand. Tasks that would lead to the most significant changes to the legal profession are also the ones most prone to overoptimism about AI capabilities, as they are harder to evaluate. We make recommendations for better evaluation and deployment of AI in legal contexts.
Forward citations
Cited by 2 Pith papers
-
BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices
A new 46-criteria assessment framework scores 24 AI benchmarks and finds that commonly used benchmarks are weak in implementation and statistical rigor.
-
Data and AI governance: Promoting equity, ethics, and fairness in large language models
The paper proposes a lifecycle governance framework, built on the authors' BEATS benchmark, to quantify and mitigate bias, ethics, fairness, and factuality failures in large language models.
Discussion (0). Continue with ORCID to comment.