Pith. sign in

REVIEW 3 cited by

Better Call GPT, Comparing Large Language Models Against Lawyers

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2401.16212 v1 pith:27FFU2KA submitted 2024-01-24 cs.CY cs.CL

Better Call GPT, Comparing Large Language Models Against Lawyers

Lauren Martin , Nick Whitehouse , Stephanie Yiu , Lizzie Catterson , Rivindu Perera (Onit AI Centre of Excellence) This is my paper
classification cs.CY cs.CL
keywords legalllmscontractcostlawyersmodelsaccuracyefficiency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This paper presents a groundbreaking comparison between Large Language Models and traditional legal contract reviewers, Junior Lawyers and Legal Process Outsourcers. We dissect whether LLMs can outperform humans in accuracy, speed, and cost efficiency during contract review. Our empirical analysis benchmarks LLMs against a ground truth set by Senior Lawyers, uncovering that advanced models match or exceed human accuracy in determining legal issues. In speed, LLMs complete reviews in mere seconds, eclipsing the hours required by their human counterparts. Cost wise, LLMs operate at a fraction of the price, offering a staggering 99.97 percent reduction in cost over traditional methods. These results are not just statistics, they signal a seismic shift in legal practice. LLMs stand poised to disrupt the legal industry, enhancing accessibility and efficiency of legal services. Our research asserts that the era of LLM dominance in legal contract review is upon us, challenging the status quo and calling for a reimagined future of legal workflows.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. DECODEM: Data Extraction from Corporate Organizational Documents via Enhanced Methods

    cs.CL 2026-07 conditional novelty 6.0

    Frontier LLMs accurately extract many corporate governance variables from charters and bylaws on the new DECODEM benchmarks, but complex provisions and label noise remain the main error sources.

  2. A Few Good Clauses: Comparing LLMs vs Domain-Trained Small Language Models on Structured Contract Extraction

    cs.CL 2026-05 unverdicted novelty 5.0

    Domain-trained small language model Olava Extract outperforms frontier LLMs on structured contract extraction with macro F1 0.812, micro F1 0.842, highest precision, and 78-97% lower inference cost.

  3. Lightweight Domain Adaptation of a Large Language Model for Legal Assistance in the Indian Context

    cs.CL 2025-05 unverdicted novelty 5.0

    An 8B Llama model with RAG and prompt engineering scores 60.08% on the All-India Bar Examination, slightly above GPT-3.5 Turbo while claiming 22 times greater parameter efficiency via a new PEI metric.