Pith. sign in

REVIEW 4 major objections 5 minor 44 references

An autonomous agent for auditing and improving the reliability of clinical AI models

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An autonomous LLM agent audits clinical AI models, identifies hidden failure modes, and prescribes targeted retraining that recovers 15-25% of performance lost under real-world distribution shift.

desk verdict ModelAuditor is a useful new workflow for clinical AI reliability auditing, but the 15–25% recovery headline is not yet established because the evaluation lacks a control for random or rule-based shift selection. read the letter →

arxiv 2507.05755 v1 pith:4NAAQKLO submitted 2025-07-08 cs.AI

Lukas Kuhn , Florian Buettner This is my paper · ORCID
classification cs.AI
keywords LLMagentdistributionshiftclinicalAIauditingrobustnessmedicalimagingdataaugmentationcalibrationmulti-agentdebate
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ModelAuditor is a conversational, LLM-driven agent that audits a trained medical-imaging AI before deployment. It asks the clinician about the intended use, picks task-specific metrics and distribution shifts, runs perturbation experiments, and explains which failure modes matter and how to fix them. The paper claims that its targeted retraining recommendations recover 15-25% of the performance lost under real-world distribution shift across histopathology, chest radiography, and dermatology, while generic augmentation like AutoAugment often helps less or even hurts. If this holds, reliability auditing becomes a fast, cheap, interpretable step that a practitioner can run on a laptop.

What carries the argument

The load-bearing mechanism is a proposer-critic debate between two LLM sub-agents that converts a natural-language deployment description into a parameterized audit specification: a set of metrics pinned to the MetricsReloaded framework and a subset of 22 perturbation types (brightness, stain mixing, JPEG compression, zoom, blur, and others). That specification is executed by the ModelAuditor core on a stratified 10% sample of the data, and the results stream back to the same dialogue thread, which writes the report and recommends torchvision transforms for retraining. The audit-retrain-verify loop is what carries the performance-recovery claims.

What would settle it

Take a model audited by ModelAuditor and deploy it against a hospital-specific artifact absent from the 22-shift catalog, such as a proprietary scanner reconstruction artifact or a novel stain; if the agent neither predicts the degradation nor proposes an effective mitigation, the claim that it anticipates real-world failure modes is bounded by catalog coverage.

Watch

Extended reading notes

Core claim

The central claim is that a single LLM agent, equipped with a catalog of 22 clinically motivated perturbations and the MetricsReloaded metric rubric, can autonomously diagnose context-specific failure modes of clinical vision models and prescribe mitigations that improve out-of-distribution performance. In the SIIM-ISIC melanoma classifier case, the agent predicted lighting sensitivity, zoom and centering dependence, and overconfidence, and those predictions were confirmed on Fitzpatrick17, where AUROC dropped more than 15%. In three controlled scenarios, re-training with the agent's targeted augmentations recovered roughly 15-25% of lost accuracy and improved calibration, beating AutoAugment and the unaugmented baseline in nearly every comparison.

Load-bearing premise

The whole audit depends on the 22 perturbation types in the catalog covering the failure modes that actually occur in the deployment environment; a real-world shift that is not in the catalog will be missed and will not be fixed.

Editorial extensions

If this is right

  • Practitioners can run a reliability audit in under 10 minutes on consumer hardware for under $0.50, making pre-deployment checks feasible for resource-limited settings.
  • Targeted, context-aware augmentation can recover a substantial share of distribution-shift loss without stronger regularization or longer training.
  • Generic one-size-fits-all augmentation can hurt out-of-distribution performance where targeted shifts help, as observed in the HAM10000 demographic-shift scenario.
  • The same agent can re-audit a retrained model and recognize when improvements have been successfully implemented, closing the loop from diagnosis to remediation.
  • The audit produces reproducible shift specifications and metric choices that could double as documentation for regulatory compliance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors' implicit claim is that the gap between simulated shift robustness and real-world robustness is small; a stronger test would use a prospective deployment drift discovered after the audit, not held-out datasets chosen after the catalog was built.
  • The method could be extended beyond vision: the same debate-over-metrics structure would need new shift generators for text, waveform, or multimodal clinical data, likely exposing the catalog bottleneck.
  • The $0.50-per-audit figure depends on current API pricing of one commercial LLM; institutions that switch to local models for privacy may trade cost or latency for data control.
  • If the approach generalizes, a natural next step is a continuous monitoring loop that generates new shifts from deployment telemetry and feeds them back into the catalog.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces ModelAuditor, an LLM-based multi-agent system that audits trained clinical image classifiers. The agent selects task-specific evaluation metrics using a rubric distilled from MetricsReloaded, chooses clinically motivated distribution shifts from a catalog of 22 perturbations, runs a perturbation-based audit, produces interpretable reports, and recommends targeted augmentations for retraining. The authors evaluate the approach on three clinical distribution-shift scenarios: Camelyon17 histopathology (hold-out hospital), CheXpert-to-ChestXray14 chest radiography (equipment/sampling shift), and HAM10000 dermatology (Vienna-to-Australia demographic shift), comparing Base, AutoAugment, and ModelAuditor-augmented training across four backbones. They also present a case study on the SIIM-ISIC melanoma classifier, claiming that the agent's identified failure modes are confirmed on Fitzpatrick17. The headline claim is that ModelAuditor recovers 15-25% of performance lost under real-world distribution shift while running in under 10 minutes and costing less than US$0.50 per audit.

Significance. If the central causal claim is supported, ModelAuditor would be a practical and inexpensive tool for pre-deployment reliability auditing of clinical AI models, addressing a real need given regulatory pressure and documented OOD failures. The paper has notable strengths: it uses realistic OOD benchmarks (Camelyon17 held-out hospital, ChestXray14, Australian HAM10000 subset), evaluates multiple architectures with three seeds, releases code, and provides a concrete validation of failure-mode identification against Fitzpatrick17. The metric-selection mechanism built on MetricsReloaded is also potentially valuable. However, the main attribution claim—that targeted agent reasoning, rather than generic augmentation breadth, drives the gains—is not yet established, and the headline recovery figure is not reconcilable with the appendix tables. These issues are load-bearing for the paper's primary contribution and require additional experiments or a substantially revised claims.

major comments (4)
  1. [Abstract; §2.2; Tables A1–A3] The headline claim that targeted recommendations 'recover 15–25% of performance lost under real-world distribution shift' is not supported by the appendix tables. For Camelyon17 balanced accuracy (Table A1), the recovered fraction of the ID→OOD gap is about 69% for ResNet50 ((95.28−87.05)/(98.88−87.05)), 61% for ResNet152, 29% for VGG19, and 60% for ViT, i.e., roughly 29–70%, not 15–25%. For HAM10000 (Table A3) no in-distribution baseline is reported, so 'lost performance' is undefined. Please specify the exact definition of recovery, report the computation for each dataset/architecture, and correct the abstract and Section 2.2 accordingly.
  2. [§2.2; §3; Methods 'ModelAuditor'] The central attribution claim—that gains arise from the agent's context-aware shift selection rather than from any augmentation drawn from the catalog—is not tested. The evaluation compares only Base, AutoAugment, and ModelAuditor; there is no control that trains with a random subset of the 22 perturbations from the shift catalog, or with a simple rule-based pipeline, using identical budgets, seeds, and architectures. Without such a control, the Discussion statement that improvements arose from 'auditor-specific, context-aware interventions rather than stronger regularization or longer training' is presupposed, not demonstrated. Please add the missing control or weaken the causal claim accordingly.
  3. [Tables A2–A3; §2.2] Several reported improvements are not statistically distinguishable, which undercuts the 'substantially outperforming' language. In Table A3, ResNet152 ModelAuditor AUROC (80.99 ± 1.37) versus Base (76.06 ± 2.38) gives a difference of about 4.9 with a standard error of about 2.75 (p > 0.05), and ViT ModelAuditor (80.24 ± 0.83) versus Base (80.08 ± 0.78) is negligible; the AutoAugment ViT result (68.53 ± 9.04) has enormous variance. In Table A1, VGG19 balanced accuracy is 96.25 ± 0.10 for ModelAuditor versus 96.21 ± 0.14 for AutoAugment. Please report per-comparison confidence intervals or significance tests and revise overclaims such as 'consistently larger' and 'significantly outperforms.'
  4. [§2.2; §3; Methods Phase 1] The validation of metric selection is circular. The agent's instruction prompt is distilled from the MetricsReloaded guideline grid ([12], Methods Phase 1), and the claimed 100% overlap with expert selection is then verified 'against the MetricsReloaded checklist' (§2.2 and §3). Since one of the present authors is also a co-author of MetricsReloaded, an independent expert panel or a held-out set of task descriptions is needed to support the claim that the agent translates free text into metrics as well as a domain specialist. Please provide such an evaluation or substantially soften the claim.
minor comments (5)
  1. [Throughout] The text contains multiple typographical errors, including 'able correctly' (Abstract), 'aufit' (Discussion), 'wether' (§2.2), and 'clincal' (Fig. 2); a thorough proofread is needed.
  2. [Methods, Datasets, Camelyon17] The statement 'We only used the data of two of the hospitals in the training split and used the third for our in-distribution results' is ambiguous and appears to conflict with the WILDS split description; please clarify exactly which hospitals are used for training, validation, ID testing, and OOD testing.
  3. [Methods, ModelAuditor] The exact augmentation pipelines recommended by the agent are not reported; the text only says operations were mapped to torchvision transforms or wrapped as Python snippets. Without listing the final pipelines per dataset and architecture, the retraining experiments cannot be reproduced from the manuscript alone.
  4. [§2.3; §4] The cost and runtime claims (median US$0.50, 5–10 minutes per model) lack measurement details; please report the number of audits, total token counts, and variance across runs.
  5. [Appendix B.2] In the shifts prompt, the sentence 'Only end the conversation by outputting the metrics' should read 'shifts', since the agent is selecting distribution shifts.

Circularity Check

2 steps flagged · score 4.0 of 10

Metric-selection validation is self-referential: the agent is prompted from MetricsReloaded and then verified against MetricsReloaded; the main OOD retraining claim is independently benchmarked and not circular.

  1. self definitional [Methods (Agentic framework); Section 2.2 (Targeted augmentations recover lost performance)]
    "Phase 1 supplies the model with a concise rubric distilled from the MetricsReloaded guideline grid [12]; ... In all three scenarios the selected metric sets exactly matched those a domain specialist would recommend, verified by a manual review against the MetricsReloaded checklist."

    The agent's metric choice is generated from a prompt distilled from MetricsReloaded; the paper then validates it against the MetricsReloaded checklist. The 'domain specialist' is thus the same document that defined the decision logic, so the 100% overlap is a self-consistency check rather than an independent prediction. The manual review is also not independent of the framework's authors, since the present paper's corresponding author (F. Buettner) is a MetricsReloaded author. The agreement therefore holds by construction rather than as evidence of external validity.

  2. self citation load bearing [Discussion]
    "The resulting panels overlapped 100% with the task-specific MetricsReloaded [12] checklist, showing that plain-language task descriptions can indeed be translated into rigorous evaluation metrics."

    The cited framework [12] is the same source from which the Phase 1 instruction rubric was distilled, and its author list includes the present paper's F. Buettner. The claimed overlap therefore rests on a self-citation chain: the input specification and the validation standard are both from [12]. This is load-bearing for the agent's metric-selection claim, although it does not affect the independent OOD retraining results.

full rationale

ModelAuditor's principal performance claims are tested against external OOD datasets (Camelyon17 held-out hospital, ChestX-ray14, Australian HAM10000 subset), and the agent does not see OOD labels when proposing augmentations, so the 15-25% recovery figure is not circular in the strict sense. The one genuinely circular element is the metric-selection validation: the agent is prompted with a rubric distilled from MetricsReloaded, then verified against the MetricsReloaded checklist 100%. This is an instruction-following tautology rather than an independent check, and the independence is further weakened because F. Buettner is also a MetricsReloaded author. A missing random/rule-based shift-selection control weakens causal attribution of the augmentation gains, but that is an experimental-design confound rather than a constructional circularity, so it is not counted here. Overall: one self-definitional validation step in a supporting claim; the central recovery claim retains independent empirical content.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The ledger shows the main input dependencies: the perturbation catalog and the OOD proxy sets carry the load, and the agent-selected augmentation pipelines are the least specified free parameters. No invented physical entities are introduced.

free parameters (3)
  • Agent-selected augmentation pipeline = Not quantified in the paper (reported only as 'specific color-jitter and gaussian blur', 'randomized geometric…
    These per-dataset, per-architecture pipelines are the main intervention. Without fixed values or ablations over magnitudes, the reported recovery cannot be independently reconstructed.
  • Perturbation parameter ranges in the shift catalog = e.g., BrightnessShift([0.8,1.2,1.5]), GaussianNoise([0,0.05,0.1])
    Hand-set ranges in the prompt (Appendix B.2) determine which stress levels are audited; different ranges would change which failure modes appear.
  • Stratified audit sample fraction = 10% (default)
    All perturbation evaluations run on a stratified 10% sample of the dataset (Methods); the sampling fraction affects the reliability of the shift-based metric estimates and therefore the agent's conclusions.
assumptions (4)
  • domain assumption The 22 perturbation types in the catalog capture clinically relevant real-world distribution shifts.
    Methods Phase 2: 'a catalogue of twenty-two clinically motivated perturbations'; if a real deployment shift is not represented, the audit cannot detect it.
  • domain assumption The held-out/OOD sets are valid proxies for real deployment: Fitzpatrick17, Camelyon17 held-out hospital, ChestX-ray14, HAM10000 Australia.
    Sections 2 and 2.2 evaluate recovery on these sets; the paper does not verify that the dominant real-world shift in each set matches the synthetic shift the agent targeted.
  • domain assumption MetricsReloaded is the correct reference standard for task-appropriate metrics.
    The agent is prompted with a rubric distilled from MetricsReloaded, and success is later verified 'against the MetricsReloaded checklist'; one of the present authors is also a MetricsReloaded author, so the standard is not fully external.
  • domain assumption Ten epochs of fine-tuning from ImageNet weights is a sufficient training regime to compare augmentation strategies.
    Methods 'Architectures and Training'; shorter-than-typical training may compress the differences between augmentation policies.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An autonomous agent for auditing and improving the reliability of clinical AI models." pith.science (2026). https://pith.science/paper/4NAAQKLO

@misc{pith2026250705755,
  author       = {Pith},
  title        = {Pith review of: An autonomous agent for auditing and improving the reliability of clinical AI models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4NAAQKLO}},
  note         = {Machine review of arXiv:2507.05755}
}
abstract

The deployment of AI models in clinical practice faces a critical challenge: models achieving expert-level performance on benchmarks can fail catastrophically when confronted with real-world variations in medical imaging. Minor shifts in scanner hardware, lighting or demographics can erode accuracy, but currently reliability auditing to identify such catastrophic failure cases before deployment is a bespoke and time-consuming process. Practitioners lack accessible and interpretable tools to expose and repair hidden failure modes. Here we introduce ModelAuditor, a self-reflective agent that converses with users, selects task-specific metrics, and simulates context-dependent, clinically relevant distribution shifts. ModelAuditor then generates interpretable reports explaining how much performance likely degrades during deployment, discussing specific likely failure modes and identifying root causes and mitigation strategies. Our comprehensive evaluation across three real-world clinical scenarios - inter-institutional variation in histopathology, demographic shifts in dermatology, and equipment heterogeneity in chest radiography - demonstrates that ModelAuditor is able correctly identify context-specific failure modes of state-of-the-art models such as the established SIIM-ISIC melanoma classifier. Its targeted recommendations recover 15-25% of performance lost under real-world distribution shift, substantially outperforming both baseline models and state-of-the-art augmentation methods. These improvements are achieved through a multi-agent architecture and execute on consumer hardware in under 10 minutes, costing less than US$0.50 per audit.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 27 canonical work pages

  1. [12]

    Nature Methods 21(2), 195–212 (2024) https://doi.org/10.1038/s41592-023-02151-z

    Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M.D., Buettner, F., Christodoulou, E., Glocker, B., Isensee, F., Kleesiek, J., Kozubek, M., Reyes, M., Riegler, M.A., Wiesenfarth, M., Kavur, A.E., Sudre, C.H., Baumgartner, M., Eisenmann, M., Heckmann-N¨ otzel, D., R¨ adsch, T., Acion, L., Antonelli, M., Arbel, T., Bakas, S., Benis, A., Blaschko, M.B., Cardo...

    2024

  2. [1]

    Nature Medicine 25, 30–36 (2019) https://doi.org/10.1038/ s41591-018-0307-0

    Kelly, C.J., Karthikesalingam, A., Suleyman, M., Corrado, G., King, D.: Key challenges for delivering clinical impact with artificial intelligence. Nature Medicine 25, 30–36 (2019) https://doi.org/10.1038/ s41591-018-0307-0

    work page 2019

  3. [2]

    Lancet Digital Health 1(4), 192–200 (2019) https://doi.org/10.1016/S2589-7500(19)30100-0

    Zech, J.R., Badgeley, M.A., Liu, M., Costa, A.B., Titano, J.J., Oermann, E.K.: Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: A cross-sectional study. Lancet Digital Health 1(4), 192–200 (2019) https://doi.org/10.1016/S2589-7500(19)30100-0

    work page doi:10.1016/s2589-7500(19)30100-0 2019

  4. [3]

    In: Advances in Neural Information Processing Systems, vol

    Koh, P.W.W., Sagawa, S., Hashimoto, T., Khisti, L., Liang, P.,et al.: Wilds: A benchmark of in-the-wild distribution shifts. In: Advances in Neural Information Processing Systems, vol. 34, pp. 23268–23280 (2021)

    work page 2021

  5. [4]

    https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206

    European Commission: Proposal for a Regulation of the European Parliament and of the Council laying down harmonised rules on Artificial Intelligence (Artificial Intelligence Act) and amending certain Union legislative acts. https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206. Accessed 7 July 2025 (2021) 14

    work page 2021

  6. [5]

    Food and Drug Administration: Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan

    U.S. Food and Drug Administration: Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. https://www.fda.gov/media/145022/download. Accessed 7 July 2025 (2021)

    work page 2021

  7. [6]

    Nature 542(7639), 115–118 (2017) https: //doi.org/10.1038/nature21056

    Esteva, A., Kuprel, B., Novoa, R.A., Ko, J., Swetter, S.M., Blau, H.M., Thrun, S.: Dermatologist- level classification of skin cancer with deep neural networks. Nature 542(7639), 115–118 (2017) https: //doi.org/10.1038/nature21056

    doi:10.1038/nature21056 2017

  8. [7]

    Science Advances 8(32), 6147 (2022) https://doi.org/10.1126/ sciadv.abq6147

    Daneshjou, R., Vodrahalli, K., Novoa, R.A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S.M., Bailey, E.E., Gevaert, O., Mukherjee, P., Phung, M., Yekrang, K., Fong, B., Sahasrabudhe, R., Allerup, J.A.C., Okada-Karigane, U., Zou, J., Chiou, A.S.: Disparities in dermatology ai performance on a diverse, curated clinical image set. Science Advanc...

    work page 2022

Show all 44 references
  1. [8]

    PLOS Medicine 15(11), 1002683 (2018) https://doi.org/10.1371/journal.pmed.1002683

    Zech, J.R., Badgeley, M.A., Liu, M., Costa, A.B., Titano, J.J., Oermann, E.K.: Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLOS Medicine 15(11), 1002683 (2018) https://doi.org/10.1371/journal.p...

    2018 doi

  2. [9]

    Scientific Reports 13, 16791 (2023) https://doi.org/ 10.1038/s41598-023-43715-5

    Kushol, R., Parnianpour, P., Wilman, A.H., Kalra, S., Yang, Y.-H.: Effects of mri scanner manufacturers in classification tasks with deep learning models. Scientific Reports 13, 16791 (2023) https://doi.org/ 10.1038/s41598-023-43715-5

    2023 doi

  3. [10]

    JAMA318(22), 2199–2210 (2017) https://doi.org/10.1001/jama.2017.14585

    Ehteshami Bejnordi, B., Veta, M., Diest, P., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA318(22), 2199–2210 (2017) https://doi.org/10.1001/jama.2017.14585

    2017

  4. [11]

    Computerized Medical Imaging and Graphics 73, 60–71 (2019) https://doi.org/10

    Bandi, P., Chaudhary, V., Albarqouni, S., et al.: Benchmarking stroma segmentation in breast cancer histology images. Computerized Medical Imaging and Graphics 73, 60–71 (2019) https://doi.org/10. 1016/j.compmedimag.2019.01.004

    2019

  5. [13]

    Food and Drug Administration: Proposed Regulatory Framework for Modifications to AI/ML- Based Software as a Medical Device (SaMD) – Discussion Paper and Request for Feedback (2019)

    U.S. Food and Drug Administration: Proposed Regulatory Framework for Modifications to AI/ML- Based Software as a Medical Device (SaMD) – Discussion Paper and Request for Feedback (2019). https://www.fda.gov/media/122535/download

    2019

  6. [14]

    https://arxiv.org/abs/2406.17536

    Salvo, F.D., Doerrich, S., Ledig, C.: MedMNIST-C: Comprehensive benchmark and improved classifier robustness by simulating realistic image corruptions (2024). https://arxiv.org/abs/2406.17536

    2024 arXiv

  7. [15]

    arXiv preprint arXiv:2105.06986 (2021)

    Navarro, F., Watanabe, C., Shit, S., Sekuboyina, A., Peeken, J.C., Combs, S.E., Menze, B.H.: Evaluating the robustness of self-supervised learning in medical imaging. arXiv preprint arXiv:2105.06986 (2021)

    2021 arXiv

  8. [16]

    https://arxiv.org/abs/1805.09501

    Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: AutoAugment: Learning Augmentation Policies from Data (2019). https://arxiv.org/abs/1805.09501

    2019 arXiv

  9. [17]

    Artificial Intelligence Review 56(11), 12561–12605 (2023)

    Goceri, E.: Medical image data augmentation: techniques, comparisons and interpretations. Artificial Intelligence Review 56(11), 12561–12605 (2023)

    2023

  10. [18]

    In: Proceedings of the 2024 ACM Conference on Health, Inference, and Learning, pp

    Liu, C., Garc ´ ıa, M., Thompson, D.: Medchain-agent: A multi-agent llm framework for sequential clinical decision support. In: Proceedings of the 2024 ACM Conference on Health, Inference, and Learning, pp. 112–122 (2024). https://doi.org/10.1145/3539254.3540001

    2024

  11. [19]

    Journal of Medical Internet Research 26(4), 39672 (2024) https://doi.org/10.2196/ 39672

    Lu, W., Fernandez, A., Patel, V.: Clinicalrag: A retrieval-augmented generation agent for clinical decision support. Journal of Medical Internet Research 26(4), 39672 (2024) https://doi.org/10.2196/ 39672

    2024

  12. [20]

    https://arxiv.org/abs/2412.06828

    Zeng, F., Lyu, Z., Li, Q., Li, X.: Enhancing LLMs for Impression Generation in Radiology Reports through a Multi-Agent System (2024). https://arxiv.org/abs/2412.06828

    2024 arXiv

  13. [21]

    Nature Biomedical Engineering 9(3), 294–306 (2025) https://doi.org/10.1038/s41551-023-01160-9

    DeGrave, A.J., Cai, Z.R., Janizek, J.D., Daneshjou, R., Lee, S.-I.: Auditing the inference processes of medical-image classifiers by leveraging generative AI and the expertise of physicians. Nature Biomedical Engineering 9(3), 294–306 (2025) https://doi.org/10.1038/s41551-023-01160-9

    2025 doi

  14. [22]

    https://arxiv.org/abs/2305.14325

    Du, Y., Li, S., Torralba, A., Tenenbaum, J.B., Mordatch, I.: Improving Factuality and Reasoning in Language Models through Multiagent Debate (2023). https://arxiv.org/abs/2305.14325

    2023 arXiv

  15. [23]

    https: //arxiv.org/abs/1512.03385

    He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition (2015). https: //arxiv.org/abs/1512.03385

    2015 arXiv

  16. [24]

    In: International Conference on 16 Learning Representations (2021)

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., et al.: An image is worth 16×16 words: Transformers for image recognition at scale. In: International Conference on 16 Learning Representations (2021)

    2021

  17. [25]

    arXiv:2012.07421 (2021)

    Koh, P.W., Sagawa, S., Marklund, H., Xie, S.M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R.L., Gao, I., Lee, T., David, E., Stavness, I., Guo, W., Earnshaw, B.A., Haque, I.S., Beery, S., Leskovec, J., Kundaje, A., Pierson, E., Levine, S., Finn, C., Liang, P...

    2021 arXiv

  18. [26]

    Proceedings of the National Academy of Sciences 117(23), 12592–12594 (2020) https://doi.org/10.1073/pnas.1919012117 https://www.pnas.org/doi/pdf/10.1073/pnas.1919012117

    Larrazabal, A.J., Nieto, N., Peterson, V., Milone, D.H., Ferrante, E.: Gender imbalance in med- ical imaging datasets produces biased classifiers for computer-aided diagnosis. Proceedings of the National Academy of Sciences 117(23), 12592–12594 (2020) https://doi.org/10.1073/p...

    2020 doi

  19. [27]

    Nature medicine 27(12), 2176–2182 (2021)

    Seyyed-Kalantari, L., Zhang, H., McDermott, M.B., Chen, I.Y., Ghassemi, M.: Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature medicine 27(12), 2176–2182 (2021)

    2021

  20. [28]

    In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

    Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., Marklund, H., Haghgoo, B., Ball, R., Shpanskaya, K., Seekins, J., Mong, D.A., Halabi, S.S., Sandberg, J.K., Jones, R., Larson, D.B., Langlotz, C.P., Patel, B.N., Lungren, M.P., Ng, A.Y.: CheXpert: A large c...

    2019

  21. [29]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., Summers, R.M.: ChestX-Ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common tho- rax diseases. In: Proceedings of the IEEE Conference on Computer Vision and Patte...

    2017

  22. [30]

    Scientific Data 5, 180161 (2018) https://doi

    Tschandl, P., Rosendahl, C., Kittler, H.: The ham10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Scientific Data 5, 180161 (2018) https://doi. org/10.1038/sdata.2018.161

    2018 doi

  23. [31]

    https: //arxiv.org/abs/2407.01502

    Kapoor, S., Stroebl, B., Siegel, Z.S., Nadgir, N., Narayanan, A.: AI Agents That Matter (2024). https: //arxiv.org/abs/2407.01502

    2024 arXiv

  24. [32]

    https://arxiv.org/abs/2411.05824

    Su, Z., Guo, J., Yang, X., Wang, Q., Coenen, F., Huang, K.: Navigating Distribution Shifts in Medical Image Analysis: A Survey (2024). https://arxiv.org/abs/2411.05824

    2024 arXiv

  25. [33]

    Radiographics 43(5), 220105 (2023) 17

    Choi, Y., Yu, W., Nagarajan, M.B., Teng, P., Goldin, J.G., Raman, S.S., Enzmann, D.R., Kim, G.H.J., Brown, M.S.: Translating ai to clinical practice: overcoming data shift with explainability. Radiographics 43(5), 220105 (2023) 17

    2023

  26. [34]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    2016

  27. [35]

    In: International Conference on Learning Representations (2015)

    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

    2015

  28. [36]

    Kingma, D.P., Ba, J.: Adam: A Method for Stochastic Optimization (2017). https://arxiv.org/abs/ 1412.6980 18 Appendix A Extended Results Balanced Accuracy↑ Matthews Corr.↑ Brier Score↓ Model Variant ID OOD ID OOD ID OOD ResNet50 Base 98.88 ± 0.06 87.05 ± 2.91 98.05± 0.50 74.57...

    2017 arXiv

  29. [37]

    Multi-class Counting Metrics: • When classes are balanced: accuracy • When classes are imbalanced with compensation requested: balanced accuracy or matthews correlation coefficient • When there’s unequal severity of class confusions: expected cost 20

  30. [38]

    Per-class Counting Metrics: • For binary problems: sensitivity (recall) specificity • When preference for minimizing false positives: positive predictive value or f beta score (β <1) • When preference for minimizing false negatives: negative predictive value or f beta score (β...

  31. [39]

    Multi-threshold Metrics: • Default recommendation: auroc • When classes are highly imbalanced: average precision

  32. [40]

    Calibration Metrics: • When calibration assessment is requested: expected calibration error and root brier score • For comparison of calibration methods: kernel calibration error • For overall probabilistic performance: negative log likelihood or brier score Y ou’ll look for: ...

  33. [41]

    You don’t need to list a lot of different numbers, just the ones that are important

    List and explain failure cases as well as good results of the model. You don’t need to list a lot of different numbers, just the ones that are important

  34. [42]

    Explain how this could impact real-world deployment

  35. [43]

    Give a simple compose list of transformations that could help mitigate this in training

  36. [44]

    After the initial advice, answer questions in a concise but helpful manner

    Any other information you want to convey to the user. After the initial advice, answer questions in a concise but helpful manner. 25

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.