Pith. sign in

REVIEW 4 major objections 5 minor 18 cited by

This paper claims a 32B medical model beats far larger rivals by training inside a dynamic patient-simulator verifier.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Baichuan-M2, a 32B medical LLM trained with a patient simulator and a clinical rubric generator as RL verifiers, reports state-of-the-art HealthBench scores (60.1 overall, 34.7 hard), ahead of all open-source models.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Strong engineering, weak evidential hygiene: the paper's own Figure 5 shows HealthBench was used to guide training, so the SOTA claim is not out-of-sample until the authors rule out contamination. the 4 major comments →

arxiv 2509.02208 v1 pith:PK2ZJANY submitted 2025-09-02 cs.LG cs.AI

Baichuan-M2: Scaling Medical Capability with Large Verifier System

Baichuan-M2 Team: Chengfeng Dou , Chong Liu , Fan Yang , Fei Li , Jiyuan Jia , Mingyang Chen , Qiang Ju , Shuai Wang
show 25 more authors
Shunya Dang Tianpeng Li Xiangrong Zeng Yijie Zhou Chenzheng Zhu Da Pan Fei Deng Guangwei Ai Guosheng Dong Hongda Zhang Jinyang Tai Jixiang Hong Kai Lu Linzhuang Sun Peidong Guo Qian Ma Rihui Xin Shihui Yang Shusen Zhang Yichuan Mo Zheng Liang Zhishou Zhang Hengfu Cui Zuyi Zhu Xiaochuan Wang
This is my paper
classification cs.LG cs.AI
keywords medical LLMreinforcement learningHealthBenchpatient simulatorclinical rubricsGRPORLVRmedical reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to establish that a 32-billion-parameter model, Baichuan-M2, can outperform much larger open and closed models on the HealthBench clinical benchmark by replacing static answer verifiers with a large dynamic verifier system. The system combines a Patient Simulator, built from de-identified medical records and personality profiles, with a Clinical Rubrics Generator that produces per-context, multi-dimensional evaluation criteria. Trained through staged reinforcement learning on an improved GRPO objective, Baichuan-M2 scores 60.1 overall and 34.7 on HealthBench Hard, the highest among open models in the comparison and above most closed models; the paper notes only GPT-5's 46.2 exceeds 32 on Hard. If true, the result matters because it locates the source of clinical capability in verifier fidelity rather than raw scale, pointing toward affordable private medical AI.

Core claim

On its own terms, the paper's discovery is that a 32B model can reach a HealthBench overall score of 60.1 and Hard score of 34.7 while retaining general abilities, surpassing all compared open-source models (gpt-oss-120B, Qwen3-235B-A22B, DeepSeek-R1, GLM-4.5, Kimi-K2) and most compared closed-source models (o3, Grok 3, Gemini 2.5 Pro, GPT-4.1). The claimed cause is the training environment: a dynamic interactive verifier rather than a static answer key. In this environment, a patient simulator produces diverse multi-turn consultations, a clinical rubrics generator emits weighted expert-style rubrics for each context, and an improved GRPO loop converts those rubrics into dense reward. The pa

What carries the argument

The load-bearing object is the closed-loop verifier system. It has two modules: a Patient Simulator, whose three components (termination gate, affective unit, factual unit) keep simulated patients diverse yet consistent while preventing information leakage; and a Clinical Rubrics Generator, which turns each dialogue slice into weighted rubrics across dimensions such as diagnostic accuracy, consultation logic, treatment rationality, communication and empathy, and medical ethics. Around this loop the paper runs multi-stage RL with a modified GRPO objective—no KL term, asymmetric clipping with a raised upper bound, length-normalized loss, and simplified advantage normalization. The verifier sup

Load-bearing premise

The learned reward from the Clinical Rubrics Generator is trusted as a proxy for expert clinical judgment; the paper's only validation is a 92.7% consistency rate against expert rubrics, measured with GPT-4.1 as referee on 100 cases—not against patient outcomes.

What would settle it

Re-run Baichuan-M2 on HealthBench Hard under an independent protocol with the same scoring rules; if its point estimate drops below 32 or into the noise band of gpt-oss-120B, the headline claim fails. Separately, have blinded clinicians rate the model's consultations on held-out real cases; if clinician-rated quality does not track the HealthBench gains, the dynamic verifier is rewarding rubric-shaped responses rather than clinical competence.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A 32B open model becomes a practical alternative to much larger closed systems for private and local medical deployment, including on consumer-grade hardware.
  • Training reward can be generated dynamically for open-ended clinical tasks such as history taking, triage, communication, and safety, which static USMLE-style QA cannot reward.
  • The interaction between patient simulator and rubric generator allows reinforcement learning to keep improving skills that are normally evaluated only by human experts.
  • The same staged RL recipe should transfer to other medical benchmarks, matching the paper's reported gains on SuperGPQA and MedXQA after rule-based RL.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The 92.7% rubrics-agreement figure is measured by GPT-4.1 as referee, so the reward signal's validity ultimately rests on an LLM judge; blinded clinician ratings on held-out real cases would test whether the trained model's gains reflect clinical competence rather than rubric-shaped output.
  • The Pareto-front framing implies a scaling hypothesis the authors do not test: verifier quality matters more than parameter count for clinical benchmarks; running the same verifier pipeline on a larger base model would test that.
  • The dynamic-verifier recipe is domain-general in principle: any high-stakes conversational setting with incomplete information and multi-turn judgment—legal counsel, psychiatric triage—could use a simulator plus rubric generator to create dense reward where static exams fail.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Baichuan-M2, a 32B medical LLM trained with mid-training, supervised fine-tuning, and a multi-stage RL pipeline. The RL loop uses a learned Clinical Rubrics Generator as a dense reward and a Patient Simulator for multi-turn interactions. Headline results are on OpenAI's HealthBench: 60.1 overall, 34.7 Hard, and 91.5 Consensus, which the authors claim outperforms all compared open-source models and most closed-source models, with GPT-5 being the only model above 32 on HealthBench Hard. The paper also reports general-capability benchmarks, quantization results, and speculative decoding speedups. The central claim is that a 32B open-source model sets a new Pareto front for medical AI at deployable scale.

Significance. If the HealthBench numbers are out-of-sample and produced under a fair, comparable protocol, the result is significant: it would show that a 32B open-source model can surpass far larger open-source and many closed-source models on a challenging medical benchmark, with practical deployment implications. The paper provides a detailed description of a novel interactive verifier system and a multi-stage RL recipe, and it openly discusses limitations. However, the significance is currently conditional: the development-time use of HealthBench is not shown to be disjoint from the final evaluation, no error bars or evaluation protocols are given for the headline comparisons, and the reward-validity chain from the learned rubrics to clinical competence is not empirically calibrated. These issues must be resolved before the SOTA claim can be accepted.

major comments (4)
  1. [§3.3.2, Fig. 5] The caption states 'All results are evaluated on a random subset of HealthBench.' This is the same benchmark on which the headline 60.1/34.7 scores are reported in §4.1 and Figs. 6–7. The paper nowhere states that this random subset was excluded from the final evaluation, nor that no other HealthBench items informed checkpoint selection, length-penalty threshold tuning, or hyperparameter decisions. As written, the SOTA claim is compatible with the model being selected on the evaluation set. Please provide an explicit disjointness statement with item-level logs, or re-evaluate on a fully held-out split that is demonstrably separate from all development-time uses of HealthBench.
  2. [§4.1, Figs. 6–7] The headline comparisons are point estimates with no confidence intervals, no number of repeated runs, and no stated evaluation protocol. The reported margins over gpt-oss-120B (60.1 vs 57.6 overall; 34.7 vs 30 on Hard) and over o3 (60.1 vs 59.8 overall) are small relative to sampling and scorer variance. Please report the full evaluation protocol: scoring prompts, temperature, max tokens, rubric aggregation method, and whether all models were run through the identical HealthBench harness. Without this, the claim of outperforming 'most advanced closed-source counterparts' is under-specified.
  3. [§2.2.4 and §3.3.2] The Clinical Rubrics Generator is validated only by a 92.7% consistency rate against expert rubrics, with GPT-4.1 as referee, and is then used as the RL reward; the final evaluation, HealthBench, also uses doctor-written rubric criteria. This is not circularity by construction, but it is a validity gap for the claim that the model is 'deeply aligned with clinical reasoning.' Optimizing a rubric-shaped reward could improve HealthBench-style rubric scores without improving clinical decision-making. Please provide external calibration: e.g., on a held-out sample, compare reward-model scores with independent clinician ratings, or report performance on a non-rubric clinical outcome task.
  4. [Abstract and §3.3] The abstract and §1 claim that the dynamic verifier system is 'essential' for aligning LLM capabilities with clinical practice, and §3.3 attributes the gains to multi-stage RL with the patient simulator and rubrics generator. No ablation isolates these components. For example, the paper does not compare training with a static expert-written rubric reward, nor training without the patient simulator, nor rule-based RL alone. Without at least one ablation, the causal claim about the verifier system is unsupported. Please add an ablation or soften the causal language to what the experiments actually establish.
minor comments (5)
  1. [§2.2] Typo: 'which are are modeled through patient simulators' should read 'which are modeled through patient simulators.'
  2. [§2.1.2 and Fig. 2] The module is called 'Fact Unit' in Fig. 2 but 'Factual Unit' in the text; please unify the terminology.
  3. [§2] Grammar: 'transcends conventional answer verifier' should be 'transcends conventional answer verifiers'; similar phrasing appears in the abstract and introduction.
  4. [§4.1, Figs. 6–7] GPT-5 is mentioned in the text and abstract as the only model above 32 on HealthBench Hard, with a score of 46.2, but it is not shown in Fig. 7. Including GPT-5's score and its evaluation setting would make the comparison complete and reproducible.
  5. [§4.1, Fig. 9/10] The axis figures show '95% CI' in the legend but the displayed values are point estimates only; please clarify whether the intervals are omitted for readability or were not computed.

Circularity Check

1 steps flagged

Development-time use of HealthBench for length-penalty analysis is not disclosed as excluded from the final evaluation, so the headline 60.1/34.7 scores may be partially in-sample. The rubric-based RL pipeline itself is not circular.

specific steps
  1. fitted input called prediction [Section 3.3.2, Figure 5 caption; Section 4.1]
    "Figure 5: Impact of length penalty. ... All results are evaluated on a random subset of HealthBench. ... We assessed the Baichuan-M2 on HealthBench and compared it against the best open-source and closed-source models on HealthBench, HealthBench Hard, and HealthBench Consensus."

    The length-penalty mechanism (Eq. 3) is the final reward component R = R_rubric + R_length, and Figure 5 evaluates its effect on 'a random subset of HealthBench' during training. The same benchmark is then used in Section 4.1 to report the paper's central SOTA claim (60.1 overall, 34.7 Hard). The paper never states that the random subset used for length-penalty analysis and checkpoint/ hyperparameter decisions was excluded from the final HealthBench evaluation. If any of those items remained in the final evaluation set, the reported scores are in-sample point estimates: a parameter (length penalty, training checkpoints) fitted to HealthBench is renamed as an independent HealthBench prediction. The lack of item-level logs or an explicit train/eval split prevents the reader from ruling out t

full rationale

Baichuan-M2's derivation chain is mostly self-contained: the Clinical Rubrics Generator is trained on internally curated expert rubrics, not on HealthBench items, and HealthBench is an external benchmark with human-written rubrics, so the rubric-shaped reward/evaluation overlap is not equivalence by construction. The patient-simulator component cites prior work [16] with overlapping authors, but the paper also provides its own fidelity evaluations (Figure 3) and that component is not the load-bearing step for the headline score. The one serious circularity concern is the evaluation protocol: Section 3.3.2 explicitly says results are evaluated on a random subset of HealthBench during the length-penalty analysis, while Section 4.1 reports final HealthBench scores without stating that this subset was held out. Since the length-penalty reward is part of the final model and training progress / threshold choices appear to be informed by that HealthBench subset, the reported 60.1/34.7 point estimates are not demonstrated to be out-of-sample. This is a partial circularity: the final evaluation may be measuring performance on data already used to select the model, rather than an independent prediction. The absence of released harness, weights, and item-level logs compounds the issue. I therefore assign a score of 6 rather than 0-2, while noting that the core RL training design is not circular by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 2 invented entities

The report counts three hand-chosen design quantities, two domain assumptions, and two software entities as the unverified load-bearing pillars of the claimed performance. The primary measured quantity, HealthBench score, is external, which keeps circularity moderate, but the training reward and the evaluation are both rubric-shaped, and neither the simulator nor the rubric generator is validated against external clinical outcomes.

free parameters (3)
  • Quality threshold (thresh) for length penalty = not reported
    Eq. (3) gates the length reward on P80 > thresh; thresh is a hand-picked value that controls response compression, and no value or sensitivity analysis is given.
  • Mid-training corpus mixing ratio = 2:2:1 (medical:general:math)
    Section 3.1: the ratio is chosen by hand to balance domain adaptation and general retention; no sweep is reported.
  • Rubric weight annotations = integer weights in [-10, 10]
    Section 2.2.2: experts assign integer weights to each rubric; these weights are human-set inputs to the reward, and no inter-annotator agreement is reported.
axioms (3)
  • domain assumption HealthBench rubric scores are a valid proxy for real clinical decision-making capability
    The paper's central evaluation and motivation assume HealthBench, a rubric-based LLM-judged benchmark, reflects practical clinical utility (Sections 1 and 4.1).
  • ad hoc to paper Agreement of the Clinical Rubrics Generator with experts (92.7%, judged by GPT-4.1) transfers to reward validity during RL
    Section 2.2.4: consistency is measured by GPT-4.1 as referee, not by independent clinician agreement, and no evidence connects this consistency to better clinical outcomes.
  • domain assumption Patient simulator fidelity metrics (privacy, fact, personification) guarantee that RL on simulated dialogues improves real consultation skill
    Section 2.1.3: the simulator is validated only on internal turn-level metrics; there is no external clinical validation or human evaluation of simulated dialogues.
invented entities (2)
  • Patient Simulator no independent evidence
    purpose: Generates multi-turn clinical dialogues for RL training
    Validated only by internal privacy/fact/personification scores against DeepSeek-V3 baselines (Section 2.1.3); no external evidence that simulated encounters behave like real patients.
  • Clinical Rubrics Generator no independent evidence
    purpose: Dynamically generates multi-dimensional scoring rubrics used as RL rewards
    Assessed only via GPT-4.1-judged consistency with expert rubrics (92.7%); there is no direct validation that optimizing these rubrics improves clinical performance.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Baichuan-M2: Scaling Medical Capability with Large Verifier System." pith.science (2026). https://pith.science/paper/PK2ZJANY

@misc{pith2026250902208,
  author       = {Pith},
  title        = {Pith review of: Baichuan-M2: Scaling Medical Capability with Large Verifier System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PK2ZJANY}},
  note         = {Machine review of arXiv:2509.02208}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

As large language models (LLMs) advance in conversational and reasoning capabilities, their practical application in healthcare has become a critical research focus. However, there is a notable gap between the performance of medical LLMs on static benchmarks such as USMLE and their utility in real-world clinical decision-making. This discrepancy arises because traditional exams fail to capture the dynamic, interactive nature of medical consultations. To address this challenge, we introduce a novel dynamic verification framework that moves beyond static answer verifier, establishing a large-scale, high-fidelity interactive reinforcement learning system. Our framework comprises two key components: a Patient Simulator that creates realistic clinical environments using de-identified medical records, and a Clinical Rubrics Generator that dynamically produces multi-dimensional evaluation metrics. Building on this foundation, we develop Baichuan-M2, a 32B-parameter medical augmented reasoning model trained through a multi-stage reinforcement learning strategy with an improved Group Relative Policy Optimization (GRPO) algorithm. Evaluated on HealthBench, Baichuan-M2 outperforms all other open-source models and most advanced closed-source counterparts, achieving a score above 32 on the challenging HealthBench Hard benchmark-previously exceeded only by GPT-5. Our work demonstrates that robust dynamic verifier system is essential for aligning LLM capabilities with practical clinical applications, establishing a new Pareto front in the performance-parameter trade-off for medical AI deployment.

Figures

Figures reproduced from arXiv: 2509.02208 by Baichuan-M2 Team: Chengfeng Dou, Chenzheng Zhu, Chong Liu, Da Pan, Fan Yang, Fei Deng, Fei Li, Guangwei Ai, Guosheng Dong, Hengfu Cui, Hongda Zhang, Jinyang Tai, Jixiang Hong, Jiyuan Jia, Kai Lu, Linzhuang Sun, Mingyang Chen, Peidong Guo, Qiang Ju, Qian Ma, Rihui Xin, Shihui Yang, Shuai Wang, Shunya Dang, Shusen Zhang, Tianpeng Li, Xiangrong Zeng, Xiaochuan Wang, Yichuan Mo, Yijie Zhou, Zheng Liang, Zhishou Zhang, Zuyi Zhu.

Figure 1
Figure 1. Figure 1: Verifier System Framework [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An illustration of Patient Simulator. The system is composed of three primary modules: the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Patient Simulator Comparison. We observe that the Privacy Score and Fact Score of [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of Training Pipeline. 3.1 Mid-Training Given that general pretrained models in medical scenarios often suffer from insufficient medical knowledge reserves, lack of authority, and temporal lag, direct medical post-training tends to fall into a dilemma of either inadequate alignment or aggravated hallucinations [21]. Therefore, we adopt lightweight mid-training, aiming to effectively enhance the mod… view at source ↗
Figure 5
Figure 5. Figure 5: Impact of length penalty. The results demonstrate that the model can effectively compress [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The comparison of Baichuan-M2 with prevailing open-source models on the HealthBench [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The comparison of Baichuan-M2 with prevailing closed-source models on the HealthBench [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The comparison of Baichuan-M2 with leading open-source models on Model Parameters [PITH_FULL_IMAGE:figures/full_fig_p013_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: HealthBench scores by axis. All HealthBench rubric criterias are partitioned into five axes [PITH_FULL_IMAGE:figures/full_fig_p013_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: HealthBench scores by theme. HealthBench examples are partitioned into seven themes to [PITH_FULL_IMAGE:figures/full_fig_p014_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: The comparison of Baichuan-M2 and gpt-oss-120B in China’s medical settings. [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Case of Gestational Diabetes answered by Baichuan-M2, which shows superior perfor [PITH_FULL_IMAGE:figures/full_fig_p025_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Gestational Diabetes case responded by gpt-oss-120b. [PITH_FULL_IMAGE:figures/full_fig_p026_13.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. MedDDC-Eval: Diagnosis-Decoupled Evaluation of Multi-Turn Medical Consultation Agents

    cs.CL 2026-07 conditional novelty 7.0

    MedDDC-Eval decouples evaluation of multi-turn consultation agents from their terminal diagnosis generators by scoring policy-elicited histories under one frozen shared diagnostic reader.

  2. MentalHospital: A Virtual Environment for Evaluating Psychiatric Clinical Encounters

    cs.AI 2026-07 conditional novelty 7.0

    EHR-derived standardized patients and dual-track evaluation reveal LLMs trail clinicians by 37.28 points on full psychiatric encounters, with mental-status assessment the main bottleneck.

  3. Experience Makes Skillful: Enabling Generalizable Medical Agent Reasoning via Self-Evolving Skill Memory

    cs.AI 2026-06 unverdicted novelty 7.0

    SkeMex distills agent trajectories into value-aware skills organized in general/task/action branches and evolves them via a closed-loop Read-Write-Assess-Govern process, outperforming prior memory agents on clinical tasks.

  4. ClinicalMC: A Benchmark for Multi-Course Clinical Decision-Making with Large Language Models

    cs.AI 2026-06 unverdicted novelty 7.0

    ClinicalMC is a benchmark of 1,275 Chinese and 5,804 English multi-course clinical samples across four stages, evaluated via a multi-agent framework on closed-source, open-source, and medical LLMs in static and dynami...

  5. LingxiDiagBench: A Multi-Agent Framework for Benchmarking LLMs in Chinese Psychiatric Consultation and Diagnosis

    cs.MA 2026-02 conditional novelty 7.0

    A 16,000-case Chinese psychiatric consultation benchmark shows LLMs reach ~92% accuracy on depression-vs-anxiety but only ~29–43% on comorbidity and 12-way differential diagnosis, and dynamic interviewing does not rel...

  6. MedRealMM: A Real-World Multimodal Benchmark for Chinese Online Medical Consultation

    cs.AI 2026-07 conditional novelty 6.5

    A real multimodal Chinese online-consultation benchmark of 5,620 cases finds frontier LLMs below physicians, with safety-sensitive error avoidance as the main gap.

  7. MedRealMM: A Real-World Multimodal Benchmark for Chinese Online Medical Consultation

    cs.AI 2026-07 conditional novelty 6.0

    On a new benchmark of 5,620 real multimodal online consultations, top LLMs trail the original physicians mainly because they trigger more unsafe or unsupported negative criteria.

  8. Active Learning for Neurosymbolic Program Synthesis

    cs.PL 2025-08 unverdicted novelty 6.0

    The abstract claims a new active learning technique, constrained conformal evaluation (tool SmartLabel), that finds the ground-truth program in 98% of benchmarks, but the delivered full text is a different paper, leav...

  9. Lung-R1: A Knowledge Graph-Guided LLM for Pulmonary Diagnostic Reasoning

    cs.AI 2026-06 unverdicted novelty 5.0

    Introduces the first structured pulmonary knowledge graph LungKG and uses it to train Lung-R1, which reaches SOTA on EMR-based pulmonary diagnosis tasks.

  10. C-MIG: Multi-view Information Gain-based Retrieval-Augmented Generation for Clinical Diagnosis Reasoning

    cs.AI 2026-05 unverdicted novelty 5.0

    C-MIG uses multi-view information gain from retrieved documents and refinements to supervise RAG-RL for clinical diagnosis, claiming top performance on four medical benchmarks.

  11. C-MIG: Multi-view Information Gain-based Retrieval-Augmented Generation for Clinical Diagnosis Reasoning

    cs.AI 2026-05 conditional novelty 5.0

    C-MIG converts retrieval and refinement steps into continuous information-gain rewards computed by a frozen reference model, improving RAG-based clinical diagnosis over hard exact-match reward baselines.

  12. Medical Reasoning with Large Language Models: A Survey and MR-Bench

    cs.CL 2026-03 accept novelty 5.0

    LLMs show strong exam performance on medical tasks but exhibit a clear gap in accuracy on authentic clinical decision-making as measured by the new MR-Bench benchmark and unified evaluations.

  13. OpenHospital: A Thing-in-itself Arena for Evolving and Benchmarking LLM-based Collective Intelligence

    cs.AI 2026-03 conditional novelty 5.0

    OpenHospital is an interactive physician-patient multi-agent arena that improves clinical metrics via ground-truth reflection and reports cooperative behaviors as evidence of evolving LLM collective intelligence.

  14. InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Training

    cs.CL 2025-10 conditional novelty 5.0

    Rubric-based incremental RL with LLM-generated case-specific checklists lifts Qwen3-4B's HealthBench-Hard score from 7.0 to 27.5 with 2k samples, and improves InfoBench instruction-following from 42.0 to 82.9.

  15. Latent-CURE for Breast Cancer Diagnosis

    cs.CV 2026-06 unverdicted novelty 4.0

    Latent-CURE introduces latent-space chain-of-thought reasoning and dual-asymmetric optimization to produce transparent, robust breast cancer diagnoses in imbalanced cohorts.

  16. UniReason-Med: A Shared Grounded Reasoning Interface for 2D-to-3D Transfer in Medical VQA

    cs.CV 2026-06 unverdicted novelty 4.0

    UniReason-Med introduces a unified framework for 2D and 3D medical VQA with shared grounded reasoning, trained on a 220K dataset, claiming that joint 2D+3D supervision improves 3D performance over 3D-only training.

  17. Baichuan-M4: A Clinical-Grade Medical Agent System for Continuous Care

    cs.AI 2026-06 unverdicted novelty 3.0

    The paper describes Baichuan-M4, a coordinated medical agent system that reports leading scores across static knowledge, dynamic consultation, long-context memory, retrieval, OCR, and multimodal tasks with a 3.3% hall...

  18. A Survey of Reinforcement Learning for Large Reasoning Models

    cs.CL 2025-09 accept novelty 3.0

    A survey compiling RL methods, challenges, data resources, and applications for enhancing reasoning in large language models and large reasoning models since DeepSeek-R1.

Reference graph

Works this paper leans on

89 extracted references · 41 canonical work pages · cited by 16 Pith papers · 1 internal anchor

  1. [1]

    Healai: A healthcare LLM for effective medical documentation

    Sagar Goyal, Eti Rastogi, Sree Prasanna Rajagopal, Dong Yuan, Fen Zhao, Jai Chintagunta, Gautam Naik, and Jeff Ward. Healai: A healthcare LLM for effective medical documentation. In Luz Angelica Caudillo-Mata, Silvio Lattanzi, Andrés Muñoz Medina, Leman Akoglu, Aristides Gionis, and Sergei Vassilvitskii, editors,Proceedings of the 17th ACM International C...

    arXiv 2024

  2. [2]

    Evalu- ating the feasibility of chatgpt in healthcare: An analysis of multiple clinical and research scenarios

    Marco Cascella, Jonathan Montomoli, Valentina Bellini, and Elena Giovanna Bignami. Evalu- ating the feasibility of chatgpt in healthcare: An analysis of multiple clinical and research scenarios. J. Medical Syst. , 47(1):33, 2023. doi: 10.1007/S10916-023-01925-4. URL https://doi.org/10.1007/s10916-023-01925-4

    work page doi:10.1007/s10916-023-01925-4 2023

  3. [3]

    Intille, Nawar Shara, Guodong Gordon Gao, and Dakuo Wang

    Ziqi Yang, Xuhai Xu, Bingsheng Yao, Ethan Rogers, Shao Zhang, Stephen S. Intille, Nawar Shara, Guodong Gordon Gao, and Dakuo Wang. Talk2care: An llm-based voice assistant for communication between healthcare providers and older adults. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 8(2):73:1–73:35, 2024. doi: 10.1145/3659625. URL https: //doi.org/...

    doi:10.1145/3659625 2024

  4. [4]

    Beyond distillation: Pushing the limits of medical LLM reasoning with minimalist rule-based RL

    Che Liu, Haozhe Wang, Jiazhen Pan, Zhongwei Wan, Yong Dai, Fangzhen Lin, Wenjia Bai, Daniel Rueckert, and Rossella Arcucci. Beyond distillation: Pushing the limits of medical LLM reasoning with minimalist rule-based RL. CoRR, abs/2505.17952, 2025. doi: 10.48550/ARXIV . 2505.17952. URL https://doi.org/10.48550/arXiv.2505.17952

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.17952 2025

  5. [5]

    Huatuogpt-o1, towards medical complex reasoning with llms

    Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wang, Jianye Hou, and Benyou Wang. Huatuogpt-o1, towards medical complex reasoning with llms. CoRR, abs/2412.18925, 2024. doi: 10.48550/ARXIV .2412.18925. URL https://doi.org/10. 48550/arXiv.2412.18925

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.18925 2024

  6. [6]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. CoRR, abs/2501.12948, 2025. doi: 10.48550/ARXIV .2501.12948. URL https://doi.org/10.48550/arXiv.2501.12948

    Pith/arXiv arXiv doi:10.48550/arxiv.2501.12948 2025

  7. [7]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. CoRR, abs/2412.16720, 2024. doi: 10.48550/ARXIV .2412.16720. URL https://doi.org/ 10.48550/arXiv.2412.16720

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.16720 2024

  8. [8]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024. doi: 10.48550/ARXIV .2402.03300. URL https://doi.org/10.48550/arXiv.2402.03300

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.03300 2024

  9. [9]

    Introducing Claude 4

    Anthropic. Introducing Claude 4. https://www.anthropic.com/news/claude-4, May

  10. [10]

    Kimi K2: open agentic intelligence

    Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi K2: open agentic intelligence. CoRR, abs/2507.20534, 2025. doi: 10.48550/ARXIV .2507.20534. URL https://doi.org/10. 48550/arXiv.2507.20534

    Pith/arXiv arXiv doi:10.48550/arxiv.2507.20534 2025

  11. [11]

    Glm-4.5: Agentic, reasoning, and coding (arc) foundation models

    Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, et al. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models. arXiv preprint arXiv:2508.06471, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Glm-4.1v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning

    Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, et al. Glm-4.1v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning. CoRR, abs/2507.01006, 2025. doi: 10.48550/ARXIV . 2507.01006. URL https://doi.org/10.48550/arXiv.2507.01006. 19

    Pith/arXiv arXiv doi:10.48550/arxiv.2507.01006 2025

  13. [13]

    Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning

    LASA Team, Weiwen Xu, Hou Pong Chan, Long Li, Mahani Aljunied, Ruifeng Yuan, Jianyu Wang, Chenghao Xiao, Guizhen Chen, Chaoqun Liu, Zhaodonghui Li, Yu Sun, Junao Shen, Chaojun Wang, Jie Tan, Deli Zhao, Tingyang Xu, Hao Zhang, and Yu Rong. Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning. CoRR, abs/2506.0704...

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.07044 2025

  14. [14]

    Usmle scoring policies and score reporting guidelines 2024

    National Board of Medical Examiners (NBME). Usmle scoring policies and score reporting guidelines 2024. Technical Report USMLE-POL-2024-01, Federation of State Medical Boards (FSMB) and National Board of Medical Examiners (NBME), 2024. URL https://www. usmle.org/scoring/policies

    2024

  15. [15]

    Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Quiñonero Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. Healthbench: Evaluating large language models towards improved human health. CoRR, abs/2505.08775, 2025. doi: 10.48550/ARXIV .2505.08775. URL https:...

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.08775 2025

  16. [16]

    Exploring the inquiry-diagnosis relationship with advanced patient simulators

    Zhaocheng Liu, Quan Tu, Wen Ye, Yu Xiao, Zhishou Zhang, Hengfu Cui, Yalun Zhu, Qiang Ju, Shizheng Li, and Jian Xie. Exploring the inquiry-diagnosis relationship with advanced patient simulators. CoRR, abs/2501.09484, 2025. doi: 10.48550/ARXIV .2501.09484. URL https://doi.org/10.48550/arXiv.2501.09484

    Pith/arXiv arXiv doi:10.48550/arxiv.2501.09484 2025

  17. [17]

    Patient safety, what does clinical simulation and teaching innovation contribute? Medicina Intensiva (English Edition) , 49(3): 165–173, 2025

    María Jesús Broch Porcar and Álvaro Castellanos-Ortega. Patient safety, what does clinical simulation and teaching innovation contribute? Medicina Intensiva (English Edition) , 49(3): 165–173, 2025

    2025

  18. [18]

    Agent hospital: A simulacrum of hospital with evolvable medical agents

    Junkai Li, Siyu Wang, Meng Zhang, Weitao Li, Yunghwei Lai, Xinhui Kang, Weizhi Ma, and Yang Liu. Agent hospital: A simulacrum of hospital with evolvable medical agents. CoRR, abs/2405.02957, 2024. doi: 10.48550/ARXIV .2405.02957. URL https://doi.org/10. 48550/arXiv.2405.02957

    Pith/arXiv arXiv doi:10.48550/arxiv.2405.02957 2024

  19. [19]

    Wenxuan Wang, Zizhan Ma, Zheng Wang, Chenghan Wu, Jiaming Ji, Wenting Chen, Xiang Li, and Yixuan Yuan. A survey of llm-based agents in medicine: How far are we from baymax? In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 -...

    2025

  20. [20]

    Consulting Psychologists Press Palo Alto, CA, 1962

    Isabel Briggs Myers et al.The myers-briggs type indicator, volume 34. Consulting Psychologists Press Palo Alto, CA, 1962

    1962

  21. [21]

    Challenges and barriers of using large language models (llm) such as chatgpt for diagnostic medicine with a focus on digital pathology –a recent scoping review

    Ehsan Ullah, Anil Parwani, Mirza Mansoor Baig, and Rajendra Singh. Challenges and barriers of using large language models (llm) such as chatgpt for diagnostic medicine with a focus on digital pathology –a recent scoping review. Diagnostic Pathology, 19(1):43, 2024. doi: 10.1186/s13000-024-01464-7. URL https://doi.org/10.1186/s13000-024-01464-7

    doi:10.1186/s13000-024-01464-7 2024

  22. [22]

    Baichuan4-Finance Technical Report

    Hanyu Zhang, Boyu Qiu, Yuhao Feng, Shuqi Li, Qian Ma, Xiyuan Zhang, Qiang Ju, Dong Yan, and Jian Xie. Baichuan4-finance technical report. CoRR, abs/2412.15270, 2024. doi: 10.48550/ARXIV .2412.15270. URLhttps://doi.org/10.48550/arXiv.2412.15270

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2412.15270 2024

  23. [23]

    The learnability of in-context learning

    Noam Wies, Yoav Levine, and Amnon Shashua. The learnability of in-context learning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16,...

    2023

  24. [24]

    Jailbreak and guard aligned language models with only few in-context demonstrations

    Zeming Wei, Yifei Wang, and Yisen Wang. Jailbreak and guard aligned language models with only few in-context demonstrations. CoRR, abs/2310.06387, 2023. doi: 10.48550/ARXIV .2310. 06387. URL https://doi.org/10.48550/arXiv.2310.06387. 20

    Pith/arXiv arXiv doi:10.48550/arxiv.2310.06387 2023

  25. [25]

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, Uni...

    work page doi:10.18653/v1/2022.emnlp-main.7 2022

  26. [26]

    Baichuan-m1: Pushing the medical capability of large language models, 2025

    Bingning Wang, Haizhou Zhao, Huozhi Zhou, Liang Song, Mingyu Xu, Wei Cheng, Xiangrong Zeng, Yupeng Zhang, Yuqi Huo, Zecheng Wang, Zhengyun Zhao, Da Pan, Fei Kou, Fei Li, Fuzhong Chen, Guosheng Dong, Han Liu, Hongda Zhang, Jin He, Jinjie Yang, Kangxi Wu, Kegeng Wu, Lei Su, Linlin Niu, Linzhuang Sun, Mang Wang, Pengcheng Fan, Qianli Shen, Rihui Xin, Shunya ...

    Pith/arXiv arXiv 2025

  27. [27]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neur...

    2022

  28. [28]

    Large language models are zero-shot reasoners

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwa- sawa. Large language models are zero-shot reasoners. In Sanmi Koyejo, S. Mo- hamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neu- ral Information Processing Systems 35: Annual Conference on Neural Information Pro- cessing Systems 2022, NeurIPS 2022...

    2022

  29. [29]

    When more is less: Understanding chain-of-thought length in llms

    Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. When more is less: Understanding chain-of-thought length in llms. CoRR, abs/2502.07266, 2025. doi: 10.48550/ARXIV .2502.07266. URLhttps://doi.org/10.48550/arXiv.2502.07266

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.07266 2025

  30. [30]

    Baichuan alignment technical report

    Mingan Lin, Fan Yang, Yan-Bin Shen, Haoze Sun, Tianpeng Li, Tao Zhang, Chenzheng Zhu, Miao Zheng, Xu Li, Yijie Zhou, Mingyang Chen, Yanzhao Qin, Youquan Li, Hao Liang, Fei Li, Yadong Li, Mang Wang, Guosheng Dong, Kuncheng Fang, Jianhua Xu, Bin Cui, Wentao Zhang, Zenan Zhou, and Weipeng Chen. Baichuan alignment technical report. ArXiv, abs/2410.14940,

    arXiv

  31. [31]

    DAPO: an open-source LLM reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, W...

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.14476 2025

  32. [32]

    Understanding r1-zero-like training: A critical perspective

    Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding r1-zero-like training: A critical perspective. CoRR, abs/2503.20783,

    Pith/arXiv arXiv

  33. [33]

    AIME problems and solutions, 2025

    AIME. AIME problems and solutions, 2025. URL https://artofproblemsolving.com/ wiki/index.php/AIME_Problems_and_Solutions

    work page 2025

  34. [34]

    Supergpqa: Scaling LLM evaluation across 285 graduate disciplines

    Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, King Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, et al. Supergpqa: Scaling LLM evaluation across 285 graduate disciplines. CoRR, abs/2502.14739, 2025. doi: 10.48550/ARXIV .2502.14739. URL https://doi.org/10.48550/arXiv.2502.14739. 21

    Pith/arXiv arXiv doi:10.48550/arxiv.2502.14739 2025

  35. [35]

    URL https://doi.org/10.48550/arXiv.2503

    doi: 10.48550/ARXIV .2503.20783. URL https://doi.org/10.48550/arXiv.2503. 20783

    Pith/arXiv arXiv doi:10.48550/arxiv.2503.20783

  36. [36]

    Reinforcement learning with rubric anchors

    Zenan Huang, Yihong Zhuang, Guoshan Lu, Zeyu Qin, Haokai Xu, Tianyu Zhao, Ru Peng, Jiaqi Hu, Zhanming Shen, Xiaomeng Hu, et al. Reinforcement learning with rubric anchors. arXiv preprint arXiv:2508.12790, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Fast on the easy, deep on the hard: Efficient reasoning via powered length penalty

    Zehui Ling, Deshu Chen, Hongwei Zhang, Yifeng Jiao, Xin Guo, and Yuan Cheng. Fast on the easy, deep on the hard: Efficient reasoning via powered length penalty. CoRR, abs/2506.10446,

    Pith/arXiv arXiv

  38. [38]

    MedxpertQA: Benchmarking expert-level medical reasoning and understanding

    Yuxin Zuo, Shang Qu, Yifei Li, Zhang-Ren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. MedxpertQA: Benchmarking expert-level medical reasoning and understanding. In F orty-second International Conference on Machine Learning, 2025. URL https://openreview.net/forum?id=IyVcxU0RKI

    work page 2025

  39. [39]

    Qwen3 technical report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. CoRR, abs/2505.09388,

    Pith/arXiv arXiv

  40. [40]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. CoRR, abs/2507.06261, 2025. doi: 10.48550/ARXIV .2507.06261. URL https: /...

    Pith/arXiv arXiv doi:10.48550/arxiv.2507.06261 2025

  41. [41]

    URL https://doi.org/10.48550/arXiv.2506

    doi: 10.48550/ARXIV .2506.10446. URL https://doi.org/10.48550/arXiv.2506. 10446

    Pith/arXiv arXiv doi:10.48550/arxiv.2506.10446

  42. [42]

    gpt-oss-120b & gpt-oss-20b model card

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025

    Pith/arXiv arXiv 2025

  43. [43]

    Gonzalez, and Ion Stoica

    Tianle Li, Wei-Lin Chiang, Evan Frick, Lisa Dunlap, Tianhao Wu, Banghua Zhu, Joseph E. Gonzalez, and Ion Stoica. From crowdsourced data to high-quality benchmarks: Arena-hard and benchbuilder pipeline. CoRR, abs/2406.11939, 2024. doi: 10.48550/ARXIV .2406.11939. URL https://doi.org/10.48550/arXiv.2406.11939

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.11939 2024

  44. [44]

    URL https://doi.org/10.48550/arXiv.2505

    doi: 10.48550/ARXIV .2505.09388. URL https://doi.org/10.48550/arXiv.2505. 09388

    Pith/arXiv arXiv doi:10.48550/arxiv.2505.09388

  45. [45]

    Writingbench: A comprehensive benchmark for generative writing

    Yuning Wu, Jiahao Mei, Ming Yan, Chenliang Li, Shaopeng Lai, Yuran Ren, Zijia Wang, Ji Zhang, Mengyue Wu, Qin Jin, and Fei Huang. Writingbench: A comprehensive benchmark for generative writing. CoRR, abs/2503.05244, 2025. doi: 10.48550/ARXIV .2503.05244. URL https://doi.org/10.48550/arXiv.2503.05244. 22

    doi:10.48550/arxiv.2503.05244 2025

  46. [46]

    Instruction-following evaluation for large language models

    Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models. CoRR, abs/2311.07911, 2023. doi: 10.48550/ARXIV .2311.07911. URL https://doi.org/10. 48550/arXiv.2311.07911

    Pith/arXiv arXiv doi:10.48550/arxiv.2311.07911 2023

  47. [47]

    Cfbench: A comprehensive constraints-following benchmark for llms

    Tao Zhang, Chenglin Zhu, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang, and Zenan Zhou. Cfbench: A comprehensive constraints-following benchmark for llms. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting o...

    work page 2025

  48. [48]

    GPTQ: accurate post- training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: accurate post- training quantization for generative pre-trained transformers. CoRR, abs/2210.17323, 2022. doi: 10.48550/ARXIV .2210.17323. URLhttps://doi.org/10.48550/arXiv.2210.17323

    Pith/arXiv arXiv doi:10.48550/arxiv.2210.17323 2022

  49. [49]

    Alignbench: Benchmarking chinese alignment of large language models

    Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Andrew Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, Xiaohan Zhang, Lichao Sun, Xiaotao Gu, Hongning Wang, Jing Zhang, Minlie Huang, Yuxiao Dong, and Jie Tang. Alignbench: Benchmarking chinese alignment of large language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proce...

    2024

  50. [50]

    Gonzalez, Clark W

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark W. Barrett, and Ying Sheng. Sglang: Efficient execution of structured language model programs. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zh...

    work page 2024

  51. [51]

    Optimize weight rounding via signed gradient descent for the quantization of llms

    Wenhua Cheng, Weiwei Zhang, Haihao Shen, Yiyang Cai, Xin He, Kaokao Lv, and Yi Liu. Optimize weight rounding via signed gradient descent for the quantization of llms. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024 , pages 11332–...

    doi:10.18653/v1/2024 2024

  52. [52]

    Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L. Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. Quarot: Outlier- free 4-bit inference in rotated llms. In Amir Globersons, Lester Mackey, Danielle Bel- grave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Ad- vances in Neural Infor...

    work page 2024

  53. [54]

    QQQ: quality quattuor-bit quantization for large language models

    Ying Zhang, Peng Zhang, Mincong Huang, Jingyang Xiang, Yujie Wang, Chao Wang, Yineng Zhang, Lei Yu, Chuan Liu, and Wei Lin. QQQ: quality quattuor-bit quantization for large language models. CoRR, abs/2406.09904, 2024. doi: 10.48550/ARXIV .2406.09904. URL https://doi.org/10.48550/arXiv.2406.09904

    Pith/arXiv arXiv doi:10.48550/arxiv.2406.09904 2024

  54. [56]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Jason Flinn, Margo I. Seltzer, Peter Druschel, Antoine Kaufmann, and Jonathan Mace, editors, Proceedings of the 29th Symposium on Operating Systems ...

    arXiv 2023

  55. [57]

    explanation

    Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle-3: Scaling up inference acceleration of large language models via training-time test. arXiv preprint arXiv:2503.01840, 2025. A Example of Evaluation Prompt for Rubrics Here are examples of evaluation prompts for positive and negative rubrics, where<conversation> denotes the conversation history ...

    Pith/arXiv arXiv 2025

  56. [58]

    Are these capillary (fingerstick) or venous measurements? Fasting = ≥8 hours without caloric intake

    work page

  57. [59]

    Check the trend: Is this consistently >105, or an isolated reading? ACOG targets are fasting <95 mg/dL (1-hr postprandial <140, 2-hr <120)

    work page

  58. [60]

    If postprandial values are also elevated, basal alone may be insufficient

    Review postprandial (1- and 2-hour) glucose logs. If postprandial values are also elevated, basal alone may be insufficient. 2.Assess Patient Context:

    work page

  59. [61]

    Weight/BMI: Higher BMI often requires more insulin

    work page

  60. [62]

    Insulin Type: Is it glargine/detemir? Glargine U300 may provide more stable coverage than U100

    work page

  61. [63]

    Non-compliance or inadequate carb control can drive hyperglycemia

    Diet/Activity: Review carb intake, meal timing, and exercise. Non-compliance or inadequate carb control can drive hyperglycemia

    work page

  62. [64]

    Hypoglycemia Risk: Any prior lows? Basal increases can cause nocturnal hypoglycemia

    work page

  63. [65]

    Other Factors: Ketones? Renal function? Other meds? Fetal growth (ultrasound)? History of macrosomia?

    work page

  64. [66]

    3.ACOG Recommendations (Practice Bulletin #190):

    Gestational Age: At 32 weeks, insulin resistance increases, but dosing must balance efficacy and safety. 3.ACOG Recommendations (Practice Bulletin #190):

    work page

  65. [67]

    Fasting >95 mg/dL warrants intensification

    work page

  66. [68]

    Start conservatively: Increase basal by 10-20% (e.g., 18 units) instead of 25% (20 units)

    work page

  67. [69]

    Recheck in 2-3 days: Monitor fasting + postprandial levels after adjustment

    work page

  68. [70]

    2.Adding prandial insulin (rapid-acting) for meals if postprandial values are elevated

    If fasting remains >95 after 18 units, consider: 1.Splitting basal (e.g., 10 units at bedtime + 8 units at 3 AM). 2.Adding prandial insulin (rapid-acting) for meals if postprandial values are elevated. 3.Switching to an insulin pump or U300 glargine for more stable coverage. 4.Safety First: Hypoglycemia Education: Counsel patient on symptoms, glucagon use...

    work page

  69. [71]

    Ketone Testing: If fasting >130 mg/dL or symptoms of DKA (nausea, vomiting), check urine/blood ketones

    work page

  70. [72]

    Avoid Aggressive Increases: A 4-unit jump risks hypoglycemia, especially if the patient is sensitive or has variable absorption

    work page

  71. [73]

    1.When to Escalate:

    Re-evaluate Diet: Refer to a diabetes educator for carb counting and meal planning. 1.When to Escalate:

    work page

  72. [74]

    If fasting remains >100 after 18 units + dietary review → Consider adding prandial insulin or endocrinology consult

    work page

  73. [75]

    If fasting >130 or ketonuria → urgent evaluation (possible pre-existing diabetes or insulin resistance)

    work page

  74. [76]

    2.Documentation & Follow-up:

    Document A1c (if recent) to assess overall control. 2.Documentation & Follow-up:

    work page

  75. [77]

    Per ACOG, intensification indicated

    Note: "Fasting glucose 105 mg/dL on 16U basal insulin. Per ACOG, intensification indicated. Adjusting basal insulin to 18U nightly. Will recheck fasting + 1-hr postprandial in 72 hours. Educated on hypoglycemia signs. Ketone testing if BG >130 or symptoms."

    work page

  76. [78]

    Schedule close follow-up (within 3-5 days)

    work page

  77. [79]

    Recommended Action Plan: 1.Immediate:

    Coordinate with endocrinology if needed (complex cases, ketonuria, or if control remains suboptimal). Recommended Action Plan: 1.Immediate:

    work page

  78. [80]

    Increase basal insulin to 18 units nightly (not 20)

    work page

  79. [81]

    Reinforce glucose monitoring (fasting + 1-hr post each meal)

    work page

  80. [82]

    2.Within 72 hours:

    Check for nocturnal hypoglycemia (2-3 AM glucose). 2.Within 72 hours:

    work page

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.