{"paper":{"title":"MedPerf: Open Benchmarking Platform for Medical Artificial Intelligence using Federated Evaluation","license":"http://creativecommons.org/licenses/by-nc-nd/4.0/","headline":"","cross_cats":["cs.DC","cs.PF","cs.SE"],"primary_cat":"cs.LG","authors_text":"Abhishek Singh, Akshay Chaudhari, Alejandro Aristizabal, Alexander Chowdhury, Alexandros Karargyris, Anna Wuest, Bala Desinghu, Cody Coleman, Daguang Xu, Daniel J. Beutel, David Kanter, Debo Dutta, Diane Feddema, G Anthony Reina, Gennady Pekhimenko, Gregory Diamos, Grigori Fursin, Indranil Mallick, Jacob Rosenthal, Jason M. Johnson, Jayaraman J. Thiagarajan, Johnu George, Junyi Guo, Maria Xenochristou, Massimo Loda, Micah J. Sheller, Michael Rosenthal, Nicholas Lane, Nicolas Padoy, Nikola Nikolov, Pablo Ribalta, Peter Mattson, Pietro Mascagni, Poonam Yadav, Renato Umeton, Satyananda Kashyap, Srini Bala, Victor Bittorf, Vijay Janapa Reddi, Virendra Mehta, Vivek Natarajan, Xinyuan Huang","submitted_at":"2021-09-29T18:09:41Z","abstract_excerpt":"Medical AI has tremendous potential to advance healthcare by supporting the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving provider and patient experience. We argue that unlocking this potential requires a systematic way to measure the performance of medical AI models on large-scale heterogeneous data. To meet this need, we are building MedPerf, an open framework for benchmarking machine learning in the medical domain. MedPerf will enable federated evaluation in which models are securely distributed to different facilities for evaluation, th"},"claims":{"count":0,"items":[],"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"source":{"id":"2110.01406","kind":"arxiv","version":3},"verdict":{"id":null,"model_set":{},"created_at":null,"strongest_claim":"","one_line_summary":"","pipeline_version":null,"weakest_assumption":"","pith_extraction_headline":""},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2110.01406/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}