pith. sign in

arxiv: 2606.30828 · v1 · pith:F5XHCL6Jnew · submitted 2026-06-29 · 💻 cs.HC

Drawing Out Legal Risks: Co-Designing with Lawyers to Predict and Manage Legal Uncertainties of Medical AI Tools

Gennie Mansi , Julia Kim , Michael Rosenbloom , Mark Riedl This is my paper

Pith reviewed 2026-07-01 01:43 UTC · model grok-4.3

classification 💻 cs.HC
keywords medical AIlegal risksco-designvisualizationsrisk managementadaptive systemshuman-computer interaction
0
0 comments X

The pith

Co-design with U.S. lawyers produces visualizations to predict and manage legal risks in adaptive medical AI tools

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that a two-year co-design process with lawyers can translate their expertise into visualizations and strategies for handling legal uncertainties created by medical AI that adapts across users and environments. Existing laws assume more static tools, so adaptive behavior makes risk prediction difficult for developers and users. The visualizations map how legal risks arise and provide concrete ways for organizations to anticipate and address them. This matters because it turns specialized legal knowledge into something actionable for teams without legal training.

Core claim

Through iterative co-design with U.S. lawyers, the authors developed visualizations that translate legal expertise into methods for identifying how legal risks arise from the adaptive nature of medical AI tools and strategies for people and organizations to predict and manage those risks under current laws and regulations.

What carries the argument

The co-design process and resulting visualizations that map legal risk factors onto the adaptive behaviors of medical AI tools.

If this is right

  • Organizations gain practical strategies to forecast legal outcomes for specific adaptive AI deployments.
  • The visualizations allow non-lawyers to apply legal reasoning to AI design choices.
  • Cross-disciplinary efforts can close the gap between regulatory expertise and technical development of adaptive systems.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same co-design method could be applied to other regulated adaptive technologies such as autonomous systems.
  • The visualizations might be evaluated in live development settings to measure their impact on decision-making.
  • Ongoing adaptation of AI would require periodic updates to the risk visualizations as new legal precedents emerge.

Load-bearing premise

That lawyers without technical AI knowledge can still produce visualizations and strategies that accurately capture and help manage the legal uncertainties of adaptive medical AI.

What would settle it

A direct comparison showing whether development teams using the visualizations encounter fewer or better-prepared legal challenges in actual deployed medical AI products than teams that do not use them.

read the original abstract

While there's optimism around medical AI tools due to their abilities to adapt from user-to-user and across environments, these new abilities complicate how people and organizations are able to predict and manage risk based on existing laws and regulations. Lawyers are trained to identify potential legal outcomes, but they lack technical AI knowledge, making it difficult to translate their expertise to creators and users of AI tools. We contribute insights from our co-design process with U.S. lawyers to identify and translate ways to predict and manage risks of medical AI tools. We present the visualizations we developed through two years of cross-disciplinary efforts and thereby illustrate our findings about how legal risks are determined and our strategies for people and organizations to predict and manage these risks. We offer insights about leveraging lawyers' expertise to understand, predict, and manage legal risks.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The paper reports on a two-year co-design process with U.S. lawyers to develop visualizations and strategies for predicting and managing legal risks of adaptive medical AI tools. It claims to contribute insights on how legal risks are determined, present the resulting visualizations, and offer strategies for people and organizations to leverage lawyers' expertise despite their lack of technical AI knowledge.

Significance. If the visualizations and insights hold as described, the work could offer a useful case study bridging HCI and legal informatics for medical AI, highlighting challenges unique to adaptive systems. The descriptive nature of the contribution limits broader impact without concrete examples or validation of the outputs.

major comments (2)
  1. Abstract: The central claims rest on a high-level description of the two-year co-design effort and the visualizations developed, but no specific evidence, data, outcomes, or validation of those visualizations is provided; this is load-bearing for the contribution of 'insights' and 'strategies' that can be used to predict and manage risks.
  2. Abstract: The paper does not address how the co-design process with lawyers lacking technical AI knowledge produces accurate translations of legal expertise into actionable risk-management tools for adaptive AI, leaving the effectiveness of the reported visualizations unexamined.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments on our manuscript. We agree that the abstract could more explicitly convey specific evidence from the co-design process and address the translation challenges. We have revised the abstract and added discussion sections to strengthen these aspects while preserving the descriptive nature of the contribution.

read point-by-point responses

  1. Referee: Abstract: The central claims rest on a high-level description of the two-year co-design effort and the visualizations developed, but no specific evidence, data, outcomes, or validation of those visualizations is provided; this is load-bearing for the contribution of 'insights' and 'strategies' that can be used to predict and manage risks.

    Authors: The body of the manuscript details the two-year co-design process with specific examples of visualizations (e.g., risk-mapping diagrams developed through lawyer sessions) and resulting strategies for adaptive medical AI. We acknowledge the abstract's high-level presentation and will revise it to reference key outcomes and evidence from the process, such as particular legal risk factors identified. As this is an exploratory co-design study, we do not include formal validation data, but the revisions will better highlight the concrete outputs. revision: yes

  2. Referee: Abstract: The paper does not address how the co-design process with lawyers lacking technical AI knowledge produces accurate translations of legal expertise into actionable risk-management tools for adaptive AI, leaving the effectiveness of the reported visualizations unexamined.

    Authors: The manuscript describes the co-design methods, including iterative sessions that used analogies and prototyping to bridge the knowledge gap between lawyers and AI developers. We agree that effectiveness is not empirically examined. We will revise to add explicit discussion of how translations were achieved, the limitations of this approach, and plans for future evaluation studies to assess the visualizations' accuracy and utility. revision: yes

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper is a descriptive HCI contribution reporting insights and visualizations produced via a two-year co-design process with external U.S. lawyers. It contains no equations, derivations, fitted parameters, or predictions that could reduce to inputs by construction. No load-bearing self-citations, uniqueness theorems, or ansatzes are invoked; the central claim is simply the output of the reported collaboration itself, which is externally grounded and self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is a qualitative HCI paper; the abstract identifies no free parameters, mathematical axioms, or invented entities. The central claim rests entirely on the described co-design process and its outputs.

pith-pipeline@v0.9.1-grok · 5675 in / 1124 out tokens · 50894 ms · 2026-07-01T01:43:12.693960+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 18 canonical work pages

  1. [1]

    AMA Future of Health: The Emerging Landscape of Augmented Intelligence in Health Care

    2024. AMA Future of Health: The Emerging Landscape of Augmented Intelligence in Health Care. https://www.ama-assn.org/practice-manage- ment/digital-health/ama-future-health-emerging- landscape-augmented-intelligence

    2024

  2. [2]

    How to develop AI policies that work for your organization’s needs

    2025. How to develop AI policies that work for your organization’s needs. https://www.ama-assn. org/practice-management/digital-health/how-de- velop-ai-policies-work-your-organization-s-needs

    2025

  3. [3]

    Glenn Cohen

    Boris Babic and I. Glenn Cohen. 2023. The Algorithmic Explainability ’Bait and Switch’. 4541487 (2023). https://papers.ssrn.com/ab- stract=4541487

    2023

  4. [4]

    Kimberly Chew, Colleen Pert, and Kathleen Sny- der. 2025. Navigating Liability in Virtual Care: Risk Management Strategies for Physicians in a Rapidly Evolving Landscape. Missouri Medicine 122, 6 (2025), 466–472. https://pmc.ncbi.nlm.nih. gov/articles/PMC12721841/

    2025

  5. [5]

    Avishek Choudhury and Onur Asan. 2022. Impact of accountability, training, and human factors on the use of artificial intelligence in healthcare: Ex- ploring the perceptions of healthcare practitioners in the US. Human Factors in Healthcare 2 (2022), 100021. doi:10.1016/j.hfh.2022.100021

    work page doi:10.1016/j.hfh.2022.100021 2022

  6. [6]

    Kirsten K. Davis. 2013. ’The Reports of My Death are Greatly Exaggerated’: Reading and Writing Objective Legal Memoranda in a Mobile Computing Age. doi:10.2139/ssrn.2281252

    work page doi:10.2139/ssrn.2281252 2013

  7. [7]

    Robert Field. 2017. Regulation of Health Care in the United States: Complexity, Confrontation and Compromise. https://papers.ssrn.com/ab- stract=3699036

    2017

  8. [8]

    Lukasz Gorski and Shashishekar Ramakrishna

  9. [9]

    In Proceedings of the Eighteenth In- ternational Conference on Artificial Intelligence and Law (New York, NY , USA) (ICAIL ’21)

    Explainable artificial intelligence, lawyer’s perspective. In Proceedings of the Eighteenth In- ternational Conference on Artificial Intelligence and Law (New York, NY , USA) (ICAIL ’21). Association for Computing Machinery, 60–68. doi:10.1145/3462757.3466145

    work page doi:10.1145/3462757.3466145

  10. [10]

    Helena Haapio. 2014. Lawyers as Designers, En- gineers and Innovators: Better Legal Documents Through Information Design and Visualization. In Proceedings of the 17th International Legal Informatics Symposium IRIS 2014 (Rochester, NY). https://papers.ssrn.com/abstract=2651066

    2014

  11. [11]

    Margaret Hagan. 2020. Legal Design as a Thing: A Theory of Change and a Set of Methods to Craft a Human-Centered Legal System. 36, 3 (2020), 3–15. doi:10.1162/desi_a_00600

    work page doi:10.1162/desi_a_00600 2020

  12. [12]

    Aishali Hegde. 2011. Case study — Risk man- agement for medical devices (based on ISO 14971). In 2011 Proceedings - Annual Reli- ability and Maintainability Symposium. 1–6. doi:10.1109/RAMS.2011.5754492 ISSN: 0149- 144X

    work page doi:10.1109/rams.2011.5754492 2011

  13. [13]

    Angela Jin and Niloufar Salehi. 2024. (Beyond) Reasonable Doubt: Challenges that Public Defenders Face in Scrutinizing AI in Court. In Proceedings of the CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for Computing Machinery, New York, NY , USA, Article 467, 19 pages. doi:10.1145/3613904.3641902

    work page doi:10.1145/3613904.3641902 2024

  14. [14]

    Matthew P. Keris. 2020. Artificial intelligence in medicine creates real risk management and litiga- tion issues. Journal of Healthcare Risk Manage- ment: The Journal of the American Society for Healthcare Risk Management 40, 2 (Oct. 2020), 21–26. doi:10.1002/jhrm.21445

    work page doi:10.1002/jhrm.21445 2020

  15. [15]

    Thomas Maddox, Dianne Babski, Peter Embi, Jackie Gerhart, Jennifer Goldsack, Ravi Parikh, Troy Sarich, Sunita Krishnan, and Audrey Elliott

  16. [16]

    National Acad- emies Press

    Generative Artificial Intelligence in Health and Medicine: Opportunities and Responsibilities for Transformative Innovation. National Acad- emies Press. doi:10.17226/28907

    work page doi:10.17226/28907

  17. [17]

    Anonymized for submission

  18. [18]

    Gennie Mansi and Mark O. Riedl. 2024. Recog- nizing Lawyers as AI Creators and Intermediar - ies in Contestability. In Proceedings of the 2024 Workshop From Stem to Stern: Contestability Along AI Value Chains” at the Conference for Computer Supported Collaborative Work. https:// arxiv.org/abs/2409.17626

    work page arXiv 2024

  19. [19]

    Mello, Amitabh Chandra, Atul A

    Michelle M. Mello, Amitabh Chandra, Atul A. Gawande, and David M. Studdert. 2010. National Costs Of The Medical Liability System. Health affairs (Project Hope) 29, 9 (Sept. 2010), 1569–

    2010

  20. [20]

    doi:10.1377/hlthaff.2009.0807

    work page doi:10.1377/hlthaff.2009.0807 2009

  21. [21]

    Mello and Neel Guha

    Michelle M. Mello and Neel Guha. 2024. Under - standing Liability Risk from Using Health Care Artificial Intelligence Tools. The New England Journal of Medicine 390, 3 (Jan. 2024), 271–278. doi:10.1056/NEJMhle2308901

    work page doi:10.1056/nejmhle2308901 2024

  22. [22]

    Stefania Passera and Helena Haapio. 2013. Transforming contracts from legal rules to user- centered communication tools: a human-infor - mation interaction challenge. 1, 3 (2013), 38–45. doi:10.1145/2466489.2466498 Place: New York, NY , USA

    work page doi:10.1145/2466489.2466498 2013

  23. [23]

    Tuan Pham. 2025. Ethical and legal consider - ations in healthcare AI: innovation and policy for safe and fair use. Royal Society Open Science 12, 5 (2025), 241873. doi:10.1098/rsos.241873

    work page doi:10.1098/rsos.241873 2025

  24. [24]

    Nicholson Price, Sara Gerke, and I

    W. Nicholson Price, Sara Gerke, and I. Glenn Cohen. 2019. Potential Liability for Physicians Using Artificial Intelligence. 322, 18 (2019), 1765–1766. doi:10.1001/jama.2019.15064

    work page doi:10.1001/jama.2019.15064 2019

  25. [25]

    Suzanne E. Rowe. 1999. Legal Research, Legal Writing, and Legal Analysis: Putting Law School into Practice Maximizing the Law School Experi- ence II. Stetson Law Review 29, 4 (1999), 1193–

    1999

  26. [26]

    journals/stet29&i=1210

    https://heinonline.org/HOL/P?h=hein. journals/stet29&i=1210

  27. [27]

    Rowland, J

    Simon P. Rowland, J. Edward Fitzgerald, Mat- thew Lungren, Elizabeth (Hsieh) Lee, Zach Harned, and Alison H. McGregor. 2022. Digital health technology-specific risks for medical mal- practice liability. npj Digital Medicine 5, 1 (Oct. 2022), 157. doi:10.1038/s41746-022-00698-3

    work page doi:10.1038/s41746-022-00698-3 2022

  28. [28]

    Studdert, Michelle M

    David M. Studdert, Michelle M. Mello, Atul A. Gawande, Tejal K. Gandhi, Allen Kachalia, Cath- erine Yoon, Ann Louise Puopolo, and Troyen A. Brennan. 2006. Claims, Errors, and Compensa- tion Payments in Medical Malpractice Litigation. New England Journal of Medicine 354, 19 (May 2006), 2024–2033. doi:10.1056/NEJMsa054479 Publisher: Massachusetts Medical Society

    work page doi:10.1056/nejmsa054479 2006

  29. [29]

    Sullivan and Scott J

    Hannah R. Sullivan and Scott J. Schweikart

  30. [30]

    2019), 160–166

    Are Current Tort Liability Doctrines Ade- quate for Addressing Injury Caused by AI? AMA Journal of Ethics 21, 2 (Feb. 2019), 160–166. doi:10.1001/amajethics.2019.160

    work page doi:10.1001/amajethics.2019.160 2019

  31. [31]

    Benjamin Sundholm. 2024. Navigating the Fron- tiers of MedTech. Arizona State Law Journal 56, 3 (2024), 1403–1470

    2024

  32. [32]

    Marisol Wong-Villacres, Aakash Gautam, Debo- rah Tatar, and Betsy DiSalvo. 2021. Reflections on Assets-Based Design: A Journey Towards A Collective of Assets-Based Thinkers. Proc. ACM Hum.-Comput. Interact. 5, CSCW2, Article 401 (Oct. 2021), 32 pages. doi:10.1145/3479545

    work page doi:10.1145/3479545 2021