Pith. sign in

REVIEW 5 major objections 6 minor 97 references

Legal AI tools still hallucinate on 17% to over 33% of real queries; the report argues the fix is not a bigger model but a consultative design that retrieves, cites, and abstains.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 19:02 UTC pith:7RRHTPDO

load-bearing objection A solid Spanish-language survey of legal LLM hallucination and RAG, but the paper's own quantitative additions are thin or self-promotional, and the 'consultative paradigm' conclusion is argued more than demonstrated. the 5 major comments →

arxiv 2509.09467 v1 pith:7RRHTPDO submitted 2025-09-11 cs.AI

Inteligencia Artificial jur\'idica y el desaf\'io de la veracidad: an\'alisis de alucinaciones, optimizaci\'on de RAG y principios para una integraci\'on responsable

Alex Dantart This is my paper
classification cs.AI
keywords legal AIhallucinationretrieval-augmented generationRAGlegal researchhuman oversightconsultative AIAI regulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This technical report tries to establish that generative legal AI is, as a class, too unreliable for professional practice unless it is rebuilt around retrieval, citation, and abstention. Its evidence: commercial legal-research tools built on retrieval-augmented generation still return wrong or unfounded answers on 17% to more than 33% of real lawyer queries, and general-purpose models do far worse on legal tasks. The author's central contention is that the fix is not incremental model scaling but a different paradigm—'consultative AI'—that retrieves and cites sources before generating, treats veracity and traceability as design requirements rather than add-ons, and keeps a human expert in the loop. A sympathetic reader would care because the paper names a concrete, measurable reliability gap in a high-stakes domain and proposes a design principle—show sources or abstain—that is testable.

Core claim

The paper argues that hallucinations in legal LLMs are a design feature of generative AI, not a bug to be tuned away, and that the standard mitigation, retrieval-augmented generation (RAG), reduces but does not remove the failure: measured hallucination rates in leading commercial legal tools range from roughly 17% (responses that misground a real source) to over one-third of queries. On this evidence the author concludes the effective path to reliable legal AI is a consultative paradigm: a system whose job is to retrieve, structure, and present verified knowledge with explicit citations, that is optimized to abstain when sources are insufficient, and whose design makes traceability and huma

What carries the argument

The central object is the proposed binary between generative AI ('a creative oracle' that predicts the next word and is incentivized never to say 'I don't know') and consultative AI ('an expert archivist' built on retrieval-augmented generation, whose output is anchored to an external curated corpus and must cite it). The load-bearing mechanism is the RAG pipeline at the core of consultative AI: retrieve relevant legal fragments, force the generator to ground its answer in those fragments, cite them precisely, and withhold an answer when grounding fails. The paper's refined taxonomy of legal hallucinations (factual/extrinsic, source-based misgrounding/ungrounding, inference, and intrinsic) i

Load-bearing premise

The load-bearing premise is that the proposed consultative optimizations would actually cut the residual error documented in existing RAG systems down to professionally acceptable levels, but the report itself provides no evaluation of such an optimized consultative system—the optimization gains in its table are explicitly hypothetical.

What would settle it

Run a pre-registered evaluation of a production legal-research assistant that implements the full consultative design (curated and hierarchy-tagged corpus, structure-aware chunking, legal embeddings with hybrid retrieval, grounding-forcing prompts with required citation, post-hoc verification, and abstention thresholds) on a set of 200+ real lawyer queries with double-blind expert annotation; if the residual hallucination rate is not clearly below the reported 17–34% range for current commercial RAG tools, the paper's central claim is refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the consultative paradigm is right, no amount of generative-model scaling makes a legal AI safe: safety comes from external grounding, citation-level traceability, and an explicit abstention mechanism.
  • Professional-use and procurement policies for legal AI should require auditability and error reporting, and lawyers remain legally responsible for content regardless of the tool that produced it—the paper documents courts enforcing exactly this principle.
  • RAG systems in production should be treated as engineered systems with measurable failure modes, not as prototypes; optimizing retrieval, generation, and post-hoc verification separately is necessary but only holistic integration reduces residual hallucination.
  • Regulators, rather than trying to ban generative legal tools, may find the generative/consultative distinction a useful line for risk classification under high-risk AI rules.
  • The 'user hallucination'—uncritical reliance on fluent output—is identified as the most dangerous failure point, so training lawyers to verify, not advanced prompting skills, is the key professional skill for the AI era.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper stops short of quantifying an acceptable residual hallucination rate; a natural extension is to set an explicit threshold (for instance, below 5% misgrounding with mandatory abstention above that) and to test whether each proposed optimization moves the rate monotonically.
  • The cite-or-abstain principle could transfer to other high-stakes professions—medicine, finance, public administration—where an AI that admits ignorance is safer than one that improvises; a testable hypothesis is that abstention-aware systems are trusted more by professionals even if their raw hit rate is lower.
  • The paper's norm-hierarchy-aware agent idea implies that corpus metadata should encode legal hierarchy, which suggests a concrete benchmark: measure whether hierarchy-aware retrieval reduces the specific class of 'validity hallucinations' compared with relevance-only retrieval.
  • Since the report's only original measurement shows a human expert outperforming every tested model on legal research, an implicit extension is that 'human-in-the-loop accuracy'—the joint human-plus-AI system's performance—should be the evaluation target, not the model's standalone score.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This technical report (in Spanish) analyzes hallucination in LLMs applied to law. It proposes a taxonomy of legal hallucinations, reviews root causes and evaluation methodologies, and documents empirical findings from commercial legal AI tools (e.g., Magesh et al.) showing residual hallucination rates of 17% to over 33% even with RAG. The report argues that RAG is a mitigation, not a cure, and then advances a set of optimization strategies: data curation, advanced retrieval, fine-tuning, post-hoc verification, and a "Kelsenian" legal agent. Its central prescriptive claim is that effective mitigation requires a shift toward a "consultative AI" paradigm that prioritizes veracity and traceability over incremental generative-model improvement, with mandatory human oversight. The report also discusses case studies (Mata v. Avianca, a Spanish Constitutional Court sanction, Thackston v. Driscoll) and relevant EU/Spain regulatory frameworks.

Significance. If the central claim were fully supported, the report would be a useful synthesis for practitioners and regulators: it consolidates recent empirical evidence on RAG's limitations, gives a structured taxonomy, and grounds its recommendations in concrete incidents and in the EU AI Act and the CTEAJE policy. The emphasis on human oversight and on the difference between "generative" and "consultative" AI is timely and pedagogically valuable. However, the paper is primarily a review/position piece, and its most novel contribution—the assertion that an optimized consultative RAG system is the way forward—is not backed by original evaluation. The quantitative support it does provide (Table 2, Table 3, Fig. 1, Table 4) is either from third parties or explicitly hypothetical/unsourced, so the significance of the report is contingent on seeing those claims properly substantiated.

major comments (5)
  1. [Abstract; §1; §3.3; §5, Table 4] The central prescriptive thesis—that effective hallucination mitigation does not lie in incremental generative-model improvements but in adopting a consultative paradigm—is asserted but not demonstrated. The only quantitative evidence on RAG systems (Sec. 3.3, Table 2) shows that current commercial RAG tools still hallucinate in 17% to >33% of queries. The optimization gains in Table 4 are explicitly labeled 'hipotéticos' and illustrative. Fig. 1 evaluates base LLMs and a hypothetical GPT-5, not a consultative system. No comparison is made between an optimized consultative system and generative-model scaling. The conclusion may be plausible, but it is underdetermined by the manuscript's own evidence.
  2. [Table 3; §4.4.1] Table 3 is presented as a benchmark of embedding models on 'FinMTEB,' but no dataset, evaluation code, retrieval corpus, or methodology is cited. Some model/parameter sizes are implausible (e.g., 'LegalBERT-v1 7B', 'LegalBERT-v2 335M'; LegalBERT is a 110M-parameter BERT variant). The table also includes models associated with the author's firm (bge-m3-spa-law variants), which are not publicly documented, and these models are recommended in §4.4.1. Because this table is the only quantitative support for the legal-embedding recommendation, it is load-bearing for the §4–5 optimization claims and must either be fully documented and reproducible or removed.
  3. [Fig. 1] Figure 1 is described as a manual expert evaluation of LLMs and a human expert, but it includes 'GPT5,' which the caption itself calls 'una mejora hipotética sobre GPT4.' A hypothetical model cannot appear in an empirical evaluation. The caption also reports a 50-query set and Cohen's κ=0.85, but no prompt template, model versions, annotation protocol, or raw scores are provided. Since the figure is used to argue that human expert performance remains superior, its evidential status must be clarified: either remove the GPT5 series and provide full methodological details, or explicitly label the figure as a synthetic/illustrative projection.
  4. [§5.3; Table 4] The 'Kelsenian agent' (§5.3) is proposed as a method to reduce 'alucinación de invalidez' and is assigned an 86.7% relative error reduction in Table 4, but the table's note states this value is 'ilustrativa' and hypothetical. No implementation, dataset, or test is described. This is acceptable in a position paper only if it is presented as a research agenda, not as evidence supporting the claimed superiority of the consultative paradigm. The current text blends speculative gains with empirical claims, which undermines the report's credibility.
  5. [Title page; §4.1; Table 3] The report's author is identified as CIO of LittleJohn, and §4.1 recommends a LittleJohn model (bge-m3-spa-law-qa-large). Table 3 includes several bge-m3-spa-law models that outperform alternatives, yet no conflict-of-interest statement accompanies these recommendations or the benchmark. This is a transparency issue that should be addressed prominently; it also reinforces the need for independent verification of Table 3.
minor comments (6)
  1. [§3.3] The numbered list of findings (items 1–4) is duplicated almost verbatim later in the same section (e.g., 'RAG es una (gran) mitigación' appears twice). This appears to be an editing artifact.
  2. [§5.6] The paragraph beginning 'Este principio de la indispensabilidad de la supervisión humana...' is repeated almost verbatim in two places in the same section.
  3. [Throughout] There are numerous typos and spacing errors, e.g., 'V entajas' (§2.1.1), 'V erificación' (§5.5), 'Y a' (§5), 'rstructuradas' (§5.4), 'desconfiguración' (§6.2.2), and 'de la popular' (§2.4, item 2).
  4. [References] Citation years are inconsistent: Magesh et al. is cited as both 2024 and 2025, and Dahl et al. appears as both 2024 and 2025. Please standardize.
  5. [Table 3] The benchmark is called 'FinMTEB,' which is not a legal benchmark; if this is a custom/legal MTEB variant, the name should be clarified and a source provided.
  6. [Fig. 1] The y-axis mixes English and Spanish ('Accurate', 'Precisión', 'Exhaustividad', 'Utilidad'). Please use consistent labels.

Circularity Check

0 steps flagged

No circular derivation; the paper's central claim is a normative recommendation supported by external evidence, not a result that reduces to its own inputs.

full rationale

This technical report contains no equations or fitted models from which a 'prediction' is derived, so the equation-level circularity patterns do not apply. The central thesis—adopt a consultative RAG paradigm rather than merely scaling generative models—is an argued recommendation, not a computed output that collapses into its inputs. Its main empirical support is external (Magesh et al., Dahl et al., Hemrajani, etc.), and the paper explicitly labels Table 4's optimization gains as 'hipotéticos' with illustrative intent; those figures are therefore not fitted parameters being relabeled as predictions. The only self-referential element is the appearance of the author's firm's embedding model, bge-m3-spa-law-qa-large, in Sec. 4.1 and in the unsourced Table 3; this is promotional and not load-bearing, since the central claim does not depend on that model's benchmark ranking. The paper's real weakness is an evidence gap—it asserts that a sufficiently optimized consultative system would reduce residual RAG errors to acceptable levels without evaluating such a system—but that is a correctness/support concern, not circularity. No load-bearing step reduces to a self-citation, a definitional identity, or a renamed known result presented as a derivation.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 2 invented entities

The report introduces no fitted parameters in a derivation sense, but its quantitative illustrations (Table 4 gains, hypothetical GPT-5 scores) are hand-picked. Its main new constructs (consultative paradigm, Kelsenian agent) are proposed frameworks without empirical validation, and the central factual premises are borrowed from external studies the report does not independently check.

free parameters (2)
  • Illustrative optimization gains (Table 4) = e.g., misgrounding 20%->5%, undetected hallucinations 15%->3%, Kelsenian-agent normative-conflict error 15%->2%
    Chosen by hand for illustration; the table's own note states only the retrieval-optimization row is inspired by Addleshaw Goddard (2024) and the rest are 'valores hipotéticos'. Presenting them in a table titled 'Impacto cuantificable' risks reading as measured results.
  • Hypothetical GPT-5 scores in Figure 1 = Approximately 3.0-3.3 on a 1-4 scale (visual estimate)
    The figure caption states GPT5 represents a hypothetical improvement over GPT4; no such scores are measured, so these data points are constructed rather than observed.
axioms (4)
  • domain assumption The correctness/groundedness definition of legal hallucination (Magesh et al. 2025) is the operative standard
    Adopted in Sec. 2.2 to define hallucination as any response that is incorrect or misgrounded; the report's headline rates (17-34%) inherit this definition.
  • domain assumption The cited benchmark rates are accurate and representative of legal AI generally, including the Spanish/EU context the report advises on
    Sec. 3.3 builds the severity claim on Magesh et al. (~200 queries to two US tools) and Dahl et al.; no primary data for Spanish/EU tools is provided.
  • domain assumption LLM hallucination is intrinsic by design because models are trained to avoid abstention (Kalai et al. 2025)
    Invoked in Secs. 2.1.1 and 4.3.3 to justify why optimizing generative models cannot solve the problem; this causal framing motivates the consultative recommendation.
  • domain assumption Kelsen's normative pyramid is a faithful model of legal validity and can be encoded into retrieval/agent metadata
    Sec. 5.3 proposes hierarchy-aware retrieval on this legal-theory premise; it is standard legal theory, but its engineering translation is not validated.
invented entities (2)
  • 'Consultative AI' paradigm no independent evidence
    purpose: Recommended architecture class: RAG-based systems that retrieve, cite, and abstain rather than generate free-form answers
    Central proposal of the report; no implementation or evaluation of the paradigm as specified is provided, and the report's own cited evidence shows current RAG products still hallucinate at 17-34%.
  • Kelsen-conscious legal agent no independent evidence
    purpose: Agent that validates answers against normative hierarchy and flags validity conflicts
    Proposed in Sec. 5.3 with an illustrative benefit row (86.7% error reduction) that the table itself labels hypothetical.

pith-pipeline@v1.3.0-alltime-deepseek · 53337 in / 16528 out tokens · 178498 ms · 2026-08-04T19:02:02.235907+00:00 · methodology

0 comments
read the original abstract

This technical report analyzes the challenge of "hallucinations" (false information) in LLMs applied to law. It examines their causes, manifestations, and the effectiveness of the RAG mitigation strategy, highlighting its limitations and proposing holistic optimizations. The paper explores the ethical and regulatory implications, emphasizing human oversight as an irreplaceable role. It concludes that the solution lies not in incrementally improving generative models, but in adopting a "consultative" AI paradigm that prioritizes veracity and traceability, acting as a tool to amplify, not replace, professional judgment. -- Este informe t\'ecnico analiza el desaf\'io de las "alucinaciones" (informaci\'on falsa) en los LLMs aplicados al derecho. Se examinan sus causas, manifestaciones y la efectividad de la estrategia de mitigaci\'on RAG, exponiendo sus limitaciones y proponiendo optimizaciones hol\'isticas. Se exploran las implicaciones \'eticas y regulatorias, enfatizando la supervisi\'on humana como un rol insustituible. El documento concluye que la soluci\'on no reside en mejorar incrementalmente los modelos generativos, sino en adoptar un paradigma de IA "consultiva" que priorice la veracidad y la trazabilidad, actuando como una herramienta para amplificar, y no sustituir, el juicio profesional.

Figures

Figures reproduced from arXiv: 2509.09467 by Alex Dantart.

Figure 1
Figure 1. Figure 1: Evaluación comparativa del rendimiento de Modelos de Lenguaje (LLMs) y un experto humano en la tarea de [PITH_FULL_IMAGE:figures/full_fig_p015_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Descomposición de la estructura argumentativa del informe según el modelo de Toulmin. La figura ilustra el [PITH_FULL_IMAGE:figures/full_fig_p016_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Diagrama esquemático de un sistema de generación aumentada por recuperación (RAG). El flujo ilustra cómo [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Esta gráfica ilustra cómo estrategias de ataque más sofisticadas (análogas a la falta de estrategias de mitigación [PITH_FULL_IMAGE:figures/full_fig_p026_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Modelo cíclico de un sistema RAG Legal y puntos de intervención estratégica para la optimización y [PITH_FULL_IMAGE:figures/full_fig_p027_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Distribución porcentual de los diferentes tipos de alucinaciones efectivamente realizadas en las respuestas de [PITH_FULL_IMAGE:figures/full_fig_p028_6.png] view at source ↗
Figure 1
Figure 1. Figure 1: Comparative evaluation of the performance of Language Models (LLMs) and a human expert in the legal [PITH_FULL_IMAGE:figures/full_fig_p075_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Decomposition of the argumentative structure of the report according to Toulmin’s model. The figure [PITH_FULL_IMAGE:figures/full_fig_p076_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Schematic diagram of a retrieval-augmented generation (RAG) system. The flow illustrates how a user query [PITH_FULL_IMAGE:figures/full_fig_p078_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: This graph illustrates how more sophisticated attack strategies (analogous to the lack of robust mitigation [PITH_FULL_IMAGE:figures/full_fig_p086_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Cyclical model of a Legal RAG system and strategic intervention points for optimization and mitigation of [PITH_FULL_IMAGE:figures/full_fig_p087_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Percentage distribution of the different types of hallucinations actually produced in the responses of language [PITH_FULL_IMAGE:figures/full_fig_p087_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

97 extracted references · 30 linked inside Pith

  1. [2]

    and Schwarcz, Daniel

    Choi, Jonathan H. and Schwarcz, Daniel. AI Assistance in Legal Analysis: An Empirical Study. In Journal of Legal Education, Forthcoming, 2024

    2024

  2. [3]

    and Herron, Felix and Rockmore, Daniel

    Livermore, Michael A. and Herron, Felix and Rockmore, Daniel. Language Model Interpretability and Empirical Legal Studies. In Journal of Institutional and Theoretical Economics , Forthcoming, 2024

    2024

  3. [4]

    How Much Content Do LLMs Generate That Induces Cognitive Bias in Users? In arXiv preprint arXiv:2507.03194, 2025

    Alessa, Abeer and Lakshminarasimhan, Akshaya and Somane, Param and Skirzynski, Julian and McAuley, Julian and Echterhoff, Jessica. How Much Content Do LLMs Generate That Induces Cognitive Bias in Users? In arXiv preprint arXiv:2507.03194, 2025

    arXiv 2025

  4. [5]

    How Technology Is (or Is Not) Transforming Law Firms

    Rodgers, Ian and Armour, John and Sako, Mari. How Technology Is (or Is Not) Transforming Law Firms. In Annual Review of Law and Social Science , 19:299–317, 2023

    2023

  5. [6]

    and Hickman, Kristin E

    Choi, Jonathan H. and Hickman, Kristin E. and Monahan, Amy and Schwarcz, Daniel. ChatGPT Goes to Law School. In Journal of Legal Education, 71(3):387–400, 2022. 52 Legal Artificial Intelligence and the challenge of veracity TECHNICAL REPORT

    2022

  6. [7]

    Evaluating the Role of Large Language Models in Legal Practice in India

    Rahul Hemrajani. Evaluating the Role of Large Language Models in Legal Practice in India. In arXiv preprint arXiv:2508.09713, 2025

    Pith/arXiv arXiv 2025

  7. [8]

    Legal Assist AI: Leveraging Transformer-based Model for Effective Legal Assistance

    Gupta, Jatin and Sharma, Akhil and Singhania, Saransh and Abidi, Ali Imam. Legal Assist AI: Leveraging Transformer-based Model for Effective Legal Assistance. In arXiv preprint arXiv:2505.22003, 2025

    Pith/arXiv arXiv 2025

  8. [9]

    GPT-4 Passes the Bar Exam

    Katz, Daniel Martin and Bommarito, Michael James and Gao, Shang and Arredondo, Pablo. GPT-4 Passes the Bar Exam. SSRN Working Paper, 2023

    2023

  9. [10]

    V empala, and Edwin Zhang

    Kalai, Adam Tauman, Ofir Nachum, Santosh S. V empala, and Edwin Zhang. Why Language Models Hallucinate. OpenAI, Technical Report, September 2025

    2025

  10. [11]

    Can GPT-3 Perform Statutory Reason- ing? In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law (ICAIL 2023) , pages Braga, Portugal

    Blair-Stanek, Andrew and Holzenberger, Nils and V an Durme, Benjamin. Can GPT-3 Perform Statutory Reason- ing? In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law (ICAIL 2023) , pages Braga, Portugal. Association for Computing Machinery, 2023

    2023

  11. [12]

    and Ré, Christopher and Chilton, Adam and Narayana, Aditya and Chohlas-Wood, Alex and Peters, Austin and Waldon, Brandon and Rockmore, Daniel N

    Guha, Neel and Nyarko, Julian and Ho, Daniel E. and Ré, Christopher and Chilton, Adam and Narayana, Aditya and Chohlas-Wood, Alex and Peters, Austin and Waldon, Brandon and Rockmore, Daniel N. and others. LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models. Thirty-seventh Conference on Neural Information Pr...

    2023

  12. [13]

    Here’s What Happens When Y our Lawyer Uses ChatGPT

    Weiser, Benjamin. Here’s What Happens When Y our Lawyer Uses ChatGPT. The New Y ork Times, May 2023. [14] Romoser, James. No, Ruth Bader Ginsburg Did Not Dissent in Obergefell — and Other Things ChatGPT Gets Wrong about the Supreme Court. SCOTUSblog, Jan 2023

    2023

  13. [15]

    Conditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech

    Ludwig, Florian and Zesch, Torsten and Zufall, Frederike. Conditioning Large Language Models on Legal Systems? Detecting Punishable Hate Speech. In arXiv preprint arXiv:2508.06456, 2025

    arXiv 2025

  14. [16]

    and Ho, Daniel E

    Magesh, V arun and Surani, Faiz and Dahl, Matthew and Suzgun, Mirac and Manning, Christopher D. and Ho, Daniel E. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. In Journal of Empirical Legal Studies, 2025

    2025

  15. [17]

    When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governance

    Shao, Peizhang and Xu, Linrui and Wang, Jinxi and Zhou, Wei and Wu, Xingyu. When Large Language Models Meet Law: Dual-Lens Taxonomy, Technical Advances, and Ethical Governance. In arXiv preprint arXiv:2507.07748, 2025

    Pith/arXiv arXiv 2025

  16. [18]

    Roberts, John G. Jr. 2023 Y ear-End Report on the Federal Judiciary. Supreme Court of the United States, 2023. [19] Engstrom, David Freeman and Ho, Daniel E. Algorithmic Accountability in the Administrative State. In Yale Journal on Regulation, 37:800–854, 2020

    2023

  17. [20]

    and Sharkey, Catherine M

    Engstrom, David Freeman and Ho, Daniel E. and Sharkey, Catherine M. and Cuéllar, Mariano-Florentino. Government by Algorithm: Artificial Intelligence in Federal Administrative Agencies. Administrative Conference of the United States, 2020

    2020

  18. [21]

    Administering Artificial Intelligence

    Solow-Niederman, Alicia. Administering Artificial Intelligence. In Southern California Law Review , 93(4):633– 696, 2020

    2020

  19. [22]

    Machine Advice with a Warning about Machine Limitations: Experimen- tally Testing the Solution Mandated by the Wisconsin Supreme Court

    Engel, Christoph and Grgi ´c-Hlaˇca, Nina. Machine Advice with a Warning about Machine Limitations: Experimen- tally Testing the Solution Mandated by the Wisconsin Supreme Court. In Journal of Legal Analysis, 13(1):284–340, 2021

    2021

  20. [23]

    Big Data’s Disparate Impact

    Barocas, Solon and Selbst, Andrew D. Big Data’s Disparate Impact. In California Law Review, 104(3):671–732, 2016

    2016

  21. [24]

    Privacy Protection, At What Cost? Exploring the Regulatory Resistance to Data Technology in Auto Insurance

    Ben-Shahar, Omri. Privacy Protection, At What Cost? Exploring the Regulatory Resistance to Data Technology in Auto Insurance. In Journal of Legal Analysis, 15(1):129–157, 2023

    2023

  22. [25]

    The Privacy-Bias Tradeoff: Data Minimization and Racial Disparity Assessments in U.S

    King, Jennifer and Ho, Daniel and Gupta, Arushi and Wu, Victor and Webley-Brown, Helen. The Privacy-Bias Tradeoff: Data Minimization and Racial Disparity Assessments in U.S. Government. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 492–505. ACM, 2023

    2023

  23. [26]

    Where’s the Liability in Harmful AI Speech? In Journal of Free Speech Law, 3(2):589–650, 2023

    Henderson, Peter and Hashimoto, Tatsunori and Lemley, Mark. Where’s the Liability in Harmful AI Speech? In Journal of Free Speech Law, 3(2):589–650, 2023

    2023

  24. [27]

    and Casey, Bryan

    Lemley, Mark A. and Casey, Bryan. Remedies for Robots. In The University of Chicago Law Review , 86(5):1311– 1396, 2019

    2019

  25. [28]

    Large Libel Models? Liability for AI Output

    V olokh, Eugene. Large Libel Models? Liability for AI Output. In Journal of Free Speech Law, 3(2):489–558, 2023

    2023

  26. [29]

    and Kim, Miriam and Akhil, Raj and Rathish, Rohit

    Chien, Colleen V . and Kim, Miriam and Akhil, Raj and Rathish, Rohit. How Generative AI Can Help Address the Access to Justice Gap Through the Courts. In Loyola of Los Angeles Law Review , Forthcoming, 2024. 53 Legal Artificial Intelligence and the challenge of veracity TECHNICAL REPORT

    2024

  27. [30]

    Nota Informativa N° 90/2024: La Sala Primera del TC por unanimidad sanciona a un abogado por la falta del debido respeto al tribunal

    Tribunal Constitucional de España. Nota Informativa N° 90/2024: La Sala Primera del TC por unanimidad sanciona a un abogado por la falta del debido respeto al tribunal. Oficina de Prensa del Tribunal Constitucional , 19 de septiembre de 2024

    2024

  28. [31]

    The Implications of ChatGPT for Legal Services and Society

    Perlman, Andrew. The Implications of ChatGPT for Legal Services and Society. In The Practice, March/April 2023

    2023

  29. [32]

    ChatGPT as an Artificial Lawyer? In Proceedings of the ICAIL 2023 Workshop on Artificial Intelligence for Access to Justice

    Tan, Jinzhe and Westermann, Hannes and Benyekhlef, Karim. ChatGPT as an Artificial Lawyer? In Proceedings of the ICAIL 2023 Workshop on Artificial Intelligence for Access to Justice . CEUR Workshop Proceedings, 2023

    2023

  30. [33]

    The Potential for Jurisdictional Challenges to AI or LLM Training Datasets

    Draper, Chris and Gillibrand, Nicky. The Potential for Jurisdictional Challenges to AI or LLM Training Datasets. In Proceedings of the ICAIL 2023 Workshop on Artificial Intelligence for Access to Justice . CEUR Workshop Proceedings, 2023

    2023

  31. [34]

    Access to A.I

    Simshaw, Drew. Access to A.I. Justice: Avoiding an Inequitable Two-Tiered System of Legal Services. In Yale Journal of Law & Technology, 24:150–226, 2022

    2022

  32. [35]

    Algorithmic Harm in Consumer Markets

    Bar-Gill, Oren and Sunstein, Cass R and Talgam-Cohen, Inbal. Algorithmic Harm in Consumer Markets. In Journal of Legal Analysis, 15(1):1–47, 2023

    2023

  33. [36]

    and Spiess, Jann L

    Gillis, Talia B. and Spiess, Jann L. Big Data and Discrimination. In The University of Chicago Law Review , 86(2):459–488, 2019

    2019

  34. [37]

    Discrimination in the Age of Algorithms

    Kleinberg, Jon and Ludwig, Jens and Mullainathan, Sendhil and Sunstein, Cass R. Discrimination in the Age of Algorithms. In Journal of Legal Analysis, 10(1):113–174, 2018

    2018

  35. [38]

    Bias In, Bias Out

    Mayson, Sandra G. Bias In, Bias Out. In The Yale Law Journal, 128(8):2122–2473, 2019

    2019

  36. [39]

    and Adeli, Ehsan and Altman, Russ and Arora, Simran and von Arx, Sydney and Bernstein, Michael S

    Bommasani, Rishi and Hudson, Drew A. and Adeli, Ehsan and Altman, Russ and Arora, Simran and von Arx, Sydney and Bernstein, Michael S. and Bohg, Jeannette and Bosselut, Antoine and Brunskill, Emma and others. On the Opportunities and Risks of Foundation Models. arXiv preprint arXiv:2108.07258, 2022

    Pith/arXiv arXiv 2022

  37. [40]

    The Algorithmic Leviathan: Arbitrariness, Fairness, and Opportunity in Algorithmic Decision-Making Systems

    Creel, Kathleen and Hellman, Deborah. The Algorithmic Leviathan: Arbitrariness, Fairness, and Opportunity in Algorithmic Decision-Making Systems. In Canadian Journal of Philosophy , 52(1):26–43, 2022

    2022

  38. [41]

    Algorithmic Monoculture and Social Welfare

    Kleinberg, Jon and Raghavan, Manish. Algorithmic Monoculture and Social Welfare. In Proceedings of the National Academy of Sciences , 118(22), 2021

    2021

  39. [42]

    Survey of Hallucination in Natural Language Generation

    Ji, Ziwei and Lee, Nayeon and Frieske, Rita and Y u, Tiezheng and Su, Dan and Xu, Y an and Ishii, Etsuko and Bang, Y ejin and Madotto, Andrea and Fung, Pascale. Survey of Hallucination in Natural Language Generation. In ACM Computing Surveys, 55(12):1–38, 2023

    2023

  40. [43]

    Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models

    Zhang, Y ue and Li, Y afu and Cui, Leyang and Cai, Deng and Liu, Lemao and Fu, Ting and Huang, Xinting and Shi, Enbo and Wang, Y ulong and Tan, Y ulong and Gao, Liqun and He, Bang and Sun, Wei and Bi, Y ongjing and Fu, Y ou and Y uan, Furu and Zhang, Wei. Siren’s Song in the AI Ocean: A Survey on Hallucination in Large Language Models. arXiv preprint arXi...

    Pith/arXiv arXiv 2023

  41. [44]

    The Pitfalls of Defining Hallucination

    van Deemter, Kees. The Pitfalls of Defining Hallucination. In Computational Linguistics, Forthcoming, 2024

    2024

  42. [45]

    Yiming Xu, Junfeng Jiao Evaluating Retrieval-Augmented Generation Strategies for Large Language Models in Travel Mode Choice Prediction In arXiv preprint arXiv:2508.17527, 2025

    Pith/arXiv arXiv 2025

  43. [46]

    Calibrated Language Models Must Hallucinate

    Kalai, Adam Tauman and V empala, Santosh S. Calibrated Language Models Must Hallucinate. arXiv preprint arXiv:2311.14648, 2023

    Pith/arXiv arXiv 2023

  44. [47]

    Hallucination Is Inevitable: An Innate Limitation of Large Language Models

    Xu, Ziwei and Jain, Sanjay and Kankanhalli, Mohan. Hallucination Is Inevitable: An Innate Limitation of Large Language Models. arXiv preprint arXiv:2401.11817, 2024

    Pith/arXiv arXiv 2024

  45. [48]

    and Zheng, Lucia and Guha, Neel and Manning, Christopher D

    Henderson, Peter and Krass, Mark S. and Zheng, Lucia and Guha, Neel and Manning, Christopher D. and Jurafsky, Dan and Ho, Daniel E. Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset. arXiv preprint arXiv:2207.00220, 2022

    Pith/arXiv arXiv 2022

  46. [49]

    How AI Can Improve Access to Justice

    Tito, Joel. How AI Can Improve Access to Justice. Centre for Public Impact, 2017. [50] Sharma, Mrinank and Tong, Meg and Korbak, Tomasz and Duvenaud, David and Askell, Amanda and Bowman, Samuel R. and Cheng, Newton and Durmus, Esin and Dodds, Zac Hatfield and Johnston, Scott R. and others. Towards Understanding Sycophancy in Language Models. arXiv preprint...

    Pith/arXiv arXiv 2017

  47. [51]

    Simple Synthetic Data Reduces Sycophancy in Large Language Models

    Wei, Jerry and Huang, Da and Lu, Yifeng and Zhou, Denny and Le, Quoc V . Simple Synthetic Data Reduces Sycophancy in Large Language Models. arXiv preprint arXiv:2308.03958, 2023

    Pith/arXiv arXiv 2023

  48. [52]

    Capturing Failures of Large Language Models via Human Cognitive Biases

    Jones, Erik and Steinhardt, Jacob. Capturing Failures of Large Language Models via Human Cognitive Biases. In Advances in Neural Information Processing Systems , 35:11411–11426, 2022. 54 Legal Artificial Intelligence and the challenge of veracity TECHNICAL REPORT

    2022

  49. [53]

    and Ziaee, Ali and Nguyen, Morgan

    Suri, Gaurav and Slater, Lily R. and Ziaee, Ali and Nguyen, Morgan. Do Large Language Models Show Decision Heuristics Similar to Humans? A Case Study Using GPT-3.5. arXiv preprint arXiv:2305.04400, 2023

    Pith/arXiv arXiv 2023

  50. [54]

    The Internal State of an LLM Knows When It’s Lying

    Azaria, Amos and Mitchell, Tom. The Internal State of an LLM Knows When It’s Lying. arXiv preprint arXiv:2304.13734, 2023

    Pith/arXiv arXiv 2023

  51. [55]

    Language Models (Mostly) Know What They Know

    Kadavath, Saurav and Conerly, Tom and Askell, Amanda and Henighan, Tom and Drain, Dawn and Perez, Ethan and Schiefer, Nicholas and Hatfield-Dodds, Zac and Maxwell, Jackson Kernion and others. Language Models (Mostly) Know What They Know. arXiv preprint arXiv:2207.05221, 2022

    Pith/arXiv arXiv 2022

  52. [56]

    Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback

    Tian, Katherine and Mitchell, Eric and Zhou, Allan and Sharma, Archit and Rafailov, Rafael and Y ao, Huaxiu and Finn, Chelsea and Manning, Christopher D. Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback. arXiv preprint arXiv:2305.14975, 2023

    Pith/arXiv arXiv 2023

  53. [57]

    Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs

    Xiong, Miao and Hu, Zhiyuan and Lu, Xinyang and Li, Yifei and Fu, Jie and He, Junxian and Hooi, Bryan. Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs. arXiv preprint arXiv:2306.13063, 2023

    Pith/arXiv arXiv 2023

  54. [58]

    Do Large Language Models Know What They Don’t Know? arXiv preprint arXiv:2305.18153, 2023

    Yin, Zhangyue and Sun, Qiushi and Guo, Qipeng and Wu, Jiawen and Qiu, Xipeng and Huang, Xuanjing. Do Large Language Models Know What They Don’t Know? arXiv preprint arXiv:2305.18153, 2023

    Pith/arXiv arXiv 2023

  55. [59]

    V era and Bellamy, Rachel K

    Zhang, Y unfeng and Liao, Q. V era and Bellamy, Rachel K. E. Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-assisted Decision Making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 295–305, 2020

    2020

  56. [60]

    Retrieval Augmentation Reduces Hallucination in Conversation

    Shuster, Kurt and Poff, Spencer and Chen, Moya and Kiela, Douwe and Weston, Jason. Retrieval Augmentation Reduces Hallucination in Conversation. arXiv preprint arXiv:2104.07567, 2021

    Pith/arXiv arXiv 2021

  57. [61]

    Check Y our Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback

    Peng, Baolin and Galley, Michel and He, Pengcheng and Cheng, Hao and Xie, Y ujia and Hu, Y u and Huang, Qiuyuan and Liden, Lars and Y u, Zhou and Chen, Weizhu and Gao, Jianfeng. Check Y our Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback. arXiv preprint arXiv:2302.12813, 2023

    Pith/arXiv arXiv 2023

  58. [62]

    Prompting GPT-3 To Be Reliable

    Si, Chenglei and Gan, Zhe and Y ang, Zhengyuan and Wang, Shuohang and Wang, Jianfeng and Boyd-Graber, Jordan and Wang, Lijuan. Prompting GPT-3 To Be Reliable. Eleventh International Conference on Learning Representations, 2023

    2023

  59. [63]

    Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinations

    Lei, Deren and Li, Y axi and Wang, Mingyu and Y un, Vincent and Ching, Emily and Kamal, Eslam and Liu, Y aqing and Liu, Wen-Ding and Y ang, Ellen and Liu, Daniel. Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinations. arXiv preprint arXiv:2310.03951, 2023

    Pith/arXiv arXiv 2023

  60. [64]

    Meta-prompting: Enhancing Language Models with Task-agnostic Scaffolding

    Suzgun, Mirac and Kalai, Adam Tauman. Meta-prompting: Enhancing Language Models with Task-agnostic Scaffolding. arXiv preprint arXiv:2401.12954, 2024

    Pith/arXiv arXiv 2024

  61. [65]

    and Finn, Chelsea

    Tian, Katherine and Mitchell, Eric and Y ao, Huaxiu and Manning, Christopher D. and Finn, Chelsea. Fine-Tuning Language Models for Factuality. arXiv preprint arXiv:2311.08401, 2023

    Pith/arXiv arXiv 2023

  62. [66]

    Dial BeInfo for Faithfulness: Improving Factuality of Information-Seeking Dialogue via Behavioural Fine-Tuning

    Razumovskaia, Evgeniia and Vuli ´c, Ivan and Markovi ´c, Pavle and Cichy, Tomasz and Zheng, Qian and Wen, Tsung-Hsien and Budzianowski, Paweł. Dial BeInfo for Faithfulness: Improving Factuality of Information-Seeking Dialogue via Behavioural Fine-Tuning. arXiv preprint arXiv:2311.09800, 2023

    Pith/arXiv arXiv 2023

  63. [67]

    R-Tuning: Teaching Large Language Models to Refuse Unknown Questions

    Zhang, Hanning and Diao, Shizhe and Lin, Y ong and Fung, Yi R and Lian, Qing and Wang, Xingyao and Chen, Y angyi and Ji, Heng and Zhang, Tong. R-Tuning: Teaching Large Language Models to Refuse Unknown Questions. arXiv preprint arXiv:2311.09677, 2023

    Pith/arXiv arXiv 2023

  64. [68]

    Trusting Y our Evidence: Hallucinate Less with Context-aware Decoding

    Shi, Weijia and Han, Xiaochuang and Lewis, Mike and Tsvetkov, Y ulia and Zettlemoyer, Luke and Yih, Scott Wen- tau. Trusting Y our Evidence: Hallucinate Less with Context-aware Decoding. arXiv preprint arXiv:2305.14739, 2023

    Pith/arXiv arXiv 2023

  65. [69]

    When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories

    Mallen, Alex and Asai, Akari and Zhong, Victor and Das, Rajarshi and Khashabi, Daniel and Hajishirzi, Hannaneh. When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers) , pages 9802–9822, 2023

    2023

  66. [70]

    Inference-time Intervention: Eliciting Truthful Answers from a Language model

    Li, Kenneth and Patel, Oam and Viégas, Fernanda and Pfister, Hanspeter and Wattenberg, Martin. Inference-time Intervention: Eliciting Truthful Answers from a Language model. 2024

    2024

  67. [71]

    and He, Pengcheng

    Chuang, Y ung-Sung and Xie, Y ujia and Luo, Hongyin and Kim, Y oon and Glass, James R. and He, Pengcheng. DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models. Twelfth International Conference on Learning Representations, 2024. 55 Legal Artificial Intelligence and the challenge of veracity TECHNICAL REPORT

    2024

  68. [72]

    FacTool: Factuality Detection in Generative AI–A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios

    Chern, I and Chern, Steffi and Chen, Shiqi and Y uan, Weizhe and Feng, Kehua and Zhou, Chunting and He, Junxian and Neubig, Graham and Liu, Pengfei and others. FacTool: Factuality Detection in Generative AI–A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios. arXiv preprint arXiv:2307.13528, 2023

    Pith/arXiv arXiv 2023

  69. [73]

    Tool Learning with Foundation Models

    Qin, Y ujia and Hu, Shengding and Lin, Y ankai and Chen, Weize and Ding, Ning and Cui, Ganqu and Zeng, Zheni and Huang, Y ufei and Xiao, Chaojun and Han, Chi and others. Tool Learning with Foundation Models. arXiv preprint arXiv:2304.08354, 2023

    Pith/arXiv arXiv 2023

  70. [74]

    CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing

    Gou, Zhibin and Shao, Zhihong and Gong, Y eyun and shen, yelong and Y ang, Y ujiu and Duan, Nan and Chen, Weizhu. CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing. Twelfth International Conference on Learning Representations, 2024

    2024

  71. [75]

    arXiv:2308.03688

    Xiao Liu, Hao Y u, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Y u Gu, Hangliang Ding, Kaiwen Men, Kejuan Y ang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Y u Su, Huan Sun, Minlie Huang, Y uxiao Dong, Jie Tang AgentBench: Evaluating LLMs as Agents. arXiv:2308.03688

    Pith/arXiv arXiv

  72. [76]

    A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models

    Tonmoy, SM and Zaman, SM and Jain, Vinija and Rani, Anku and Rawte, Vipula and Chadha, Aman and Das, Amitava. A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models. arXiv preprint arXiv:2401.01313, 2024

    Pith/arXiv arXiv 2024

  73. [77]

    and Ho, Daniel E

    Magesh, V arun and Surani, Faiz and Dahl, Matthew and Suzgun, Mirac and Manning, Christopher D. and Ho, Daniel E. Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools. arXiv preprint arXiv:2405.20362, 2024

    Pith/arXiv arXiv 2024

  74. [78]

    GPT-4 alone is not a reliable legal solution—but it does enable one: CoCounsel harnesses GPT-4’s power to deliver results that legal professionals can rely on

    Casetext. GPT-4 alone is not a reliable legal solution—but it does enable one: CoCounsel harnesses GPT-4’s power to deliver results that legal professionals can rely on. Casetext Blog, 2023

    2023

  75. [79]

    Introducing AI-Assisted Research: Legal research meets generative AI

    Thomson Reuters. Introducing AI-Assisted Research: Legal research meets generative AI. Press Release, 2023. [80] LexisNexis. LexisNexis Launches Lexis+ AI, a Generative AI Solution with Linked Hallucination-Free Legal Citations. Press Release, 2023

    2023

  76. [81]

    Lawyer uses ChatGPT in brief, gets called out for ’bogus’ case citations

    Weiser, Benjamin and Bromwich, Jonah E. Lawyer uses ChatGPT in brief, gets called out for ’bogus’ case citations. In The New York Times, May 2023

    2023

  77. [82]

    Recent Integration of Large Language Models (LLMs) into Research and Writing Tools Presents Both Unprecedented Opportunities and Significant Challenges

    Kite-Jackson, Darla Wynon. Recent Integration of Large Language Models (LLMs) into Research and Writing Tools Presents Both Unprecedented Opportunities and Significant Challenges. In 2023 Artificial Intelligence (AI) TechReport. American Bar Association, 2023

    2023

  78. [83]

    How Lexis+ AI Delivers Hallucination-Free Linked Legal Citations

    Wellen, Serena. How Lexis+ AI Delivers Hallucination-Free Linked Legal Citations. LexisNexis Blog, Feb 2024

    2024

  79. [84]

    Tech Innovation with LLMs Producing More Secure and Reliable Gen AI Results

    Wellen, Serena. Tech Innovation with LLMs Producing More Secure and Reliable Gen AI Results. LexisNexis Blog, May 2024

    2024

  80. [85]

    Introducing Ask Practical Law AI on Practical Law: Generative AI meets legal how-to

    Thomson Reuters. Introducing Ask Practical Law AI on Practical Law: Generative AI meets legal how-to. Product Information, 2024

    2024

Showing first 80 references.