Pith Number

pith:MSLRSR3M

pith:2025:MSLRSR3MTSAHDW7QNMSIX2WXNG

not attested not anchored not stored refs pending

MedHELM: Holistic Evaluation of Large Language Models for Medical Tasks

Aditya Sharma, Akshay Chaudhari, Akshay Swaminathan, Albert S. Chiou, Alex Alekseyev, Alyssa Unell, Anurang Revri, Asad Aali, Ashwin Nayak, Aydin Zahedivash, Bethel Mieso, Bilal Mawji, Birju Patel, Brian Soetikno, Carlene Lugtu, Chia-Chun Chiang, Christy Hong, Danton Char, Dev Dash, Dong-han Yao, Eduardo Reis, Emily Alsentzer, Eric Horvitz, Ethan Goh, Faraz Ghoddusi, Fateme Nateghi, Francois Grolleau, Hannah Kirsch, Hao Qiu, Harshita Sharma, Hejie Cui, Hinesh Patel, Jarrod Helzer, Jason Alan Fries, Jenelle Jindal, Jennifer Lee, Jonathan Chen, Juan M. Banda, Kameron Black, Keith Morse, Kevin Schulman, Lance Downing, Leonardo Schettini, Matthew P. Lungren, Mehr Kashyap, Mert Oez, Michael F. Gensheimer, Michael Wornow, Miguel Fuentes, Mike Pfeffer, Mohana Roy, Nerissa Ambers, Nigam H. Shah, Nikesh Kotecha, Nima Aghaeepour, Nirmal Ravi, Oluseyi Fayanju, Percy Liang, Philip Chung, Rachna Saralkar, Robson Capasso, Roxana Daneshjou, Sanmi Koyejo, Sergios Gatidis, Shivam Vedak, Shreya Shah, Shrey Jain, Sneha S. Jain, Steven Lin, Suhana Bedi, Thomas Wang, Tho Pham, Timothy Keyes, Tony Lee, Vanessa Kennedy, Vasu Divi, Vicky Zhou, Vikas Kakkar, Wen-wai Yim, Yair Bannett, Yifan Mai

arxiv:2505.23802 v2 · 2025-05-26 · cs.CL · cs.AI

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{MSLRSR3MTSAHDW7QNMSIX2WXNG}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Cited by

20 papers in Pith

A rubric-based controlled comparison of frontier language models on expert-authored clinical reasoning tasks

Beyond English benchmarks: clinical llm evaluation in Brazilian Portuguese

AutoMedBench: Towards Medical AutoResearch with Agentic AI Models

CLExEval: A Human-in-the-Loop Framework for Qualitative Evaluation of LLM Clinical Reasoning

CHI-Bench: Can AI Agents Automate End-to-End, Long-Horizon, Policy-Rich Healthcare Workflows?

Receipt and verification

First computed	2026-07-05T11:14:08.832278Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

649719476c9c8071dbf06b248bead769bebcbca10bf36aa7b7f30a82a1865720

Aliases

arxiv: 2505.23802 · arxiv_version: 2505.23802v2 · doi: 10.48550/arxiv.2505.23802 · pith_short_12: MSLRSR3MTSAH · pith_short_16: MSLRSR3MTSAHDW7Q · pith_short_8: MSLRSR3M

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/MSLRSR3MTSAHDW7QNMSIX2WXNG \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 649719476c9c8071dbf06b248bead769bebcbca10bf36aa7b7f30a82a1865720

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "56d5f3e9e7266a1d5fb605e308e95ae04d1c5f810305abf54c0d0f37e9d5c928",
    "cross_cats_sorted": [
      "cs.AI"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2025-05-26T22:55:49Z",
    "title_canon_sha256": "a19084995afa31ab5ccde2a1e0a840b0f3f15eb6fe01051fa5cc3caaaad2d85e"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2505.23802",
    "kind": "arxiv",
    "version": 2
  }
}