Pith. sign in

REVIEW 1 cited by

An entropy-guided reward that shortens answers to easy, confident questions and extends reasoning on hard ones reduces medical QA tokens by 4.7x to 6.4x with minimal accuracy loss.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 13:51 UTC pith:TCVXVCSG

arxiv 2509.24560 v2 pith:TCVXVCSG submitted 2025-09-29 cs.CL cs.AI

AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantification

Shaohao Rui , Kaitao Chen , Weijie Ma , Xiaosong Wang This is my paper
classification cs.CL cs.AI
keywords adathink-medmedicalreasoningcomputationaldiagnosticextendedinferencemodels
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Medical question-answering models that 'think' for a long time are accurate but expensive. AdaThink-Med is a way to train one model to think short when a question is easy and long when it is hard. The model generates several answers to each training question, checks whether each answer is right, and measures how unsure the model was by looking at the spread of token probabilities (entropy).

These two signals, correctness and uncertainty, are combined into a difficulty score. For questions judged easy, the model is rewarded for giving short correct answers. For questions judged hard, if the model gets the answer wrong, it is rewarded for making its reasoning longer, so it keeps exploring. A threshold decides which questions are easy and which are hard. The whole thing is done inside reinforcement learning, so no separate router or classifier is needed at deployment.

On six medical benchmarks, the trained models used between 4.7 and 6.4 times fewer tokens on average, with accuracy roughly the same as a strong baseline. The model did learn two habits: quick answers on simple items, longer chains on harder ones. But the paper's headline numbers should be read with care: the threshold and uncertainty weight were picked by testing values on these same six benchmarks, and baseline checkpoints were chosen to avoid a known collapse. The method is plausible, but the exact size of the gain may be optimistic.

Core claim

The central claim, from the abstract: 'Across six medical benchmarks, AdaThink-Med reduces inference token consumption by 4.7x to 6.4x on Qwen and Llama architectures, respectively, with minimal performance trade-offs.' If true, a single model produces short outputs for easy questions and long chains for hard ones, with an average accuracy difference of about 1 point on the studied benchmarks.

Load-bearing premise

The method rests on the assumption that the mean entropy of the top 20% most uncertain tokens (Eq. 4, adopted from Wang et al. 2025) is a faithful estimate of output-level uncertainty, and that substituting it into Eq. 5 yields a true measure of question difficulty. This is load-bearing because the length reward in Eq. 6 is entirely shaped by that difficulty score. If the entropy proxy is miscalibrated for medical QA, the reward will not allocate compute to the right questions and the claimed adaptive behavior collapses. The paper does not independently validate this proxy in the medical setting.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The central contribution relies on a chain of hand-set or test-fitted choices: an entropy proxy from prior work, a hand-designed difficulty formula, a threshold and uncertainty weight tuned on the evaluation benchmarks, and manually set reward weights. No new conceptual entities are introduced. The 'thinking' and 'non-thinking' modes are behavioral descriptions of output length distributions produced by the reward design, not separate models or new mechanisms.

free parameters (6)
  • tau (difficulty threshold quantile) = 0.7 (grid-searched over 0.3, 0.5, 0.7, 0.9)
    Controls the easy-hard split in Eq. 6; selected by best AES on the six evaluation benchmarks (Table 3).
  • alpha (uncertainty weight) = 0.5 (grid-searched over 0.1, 0.5, 0.9)
    Weights uncertainty vs correctness in Eq. 5; selected by AES on the six evaluation benchmarks.
  • alpha_m (EMA momentum) = 0.1
    EMA momentum for threshold and length statistics, set by hand in Sec. 3.1.
  • length reward weight = 0.5
    Weight of R_len in Eq. 7; set by hand, no ablation.
  • K (top entropy tokens) = top 20%
    Number of highest-entropy tokens averaged in Eq. 4, adopted from Wang et al. 2025; not re-tuned but load-bearing.
  • rho scaling formula = rho_i = 0.5 + 0.5*(1-eH_i)
    Hand-designed scaling factor in Eq. 6; no ablation.
axioms (4)
  • domain assumption The mean entropy of the top 20% most uncertain tokens faithfully represents output-level uncertainty.
    Adopted from Wang et al. 2025 (Eq. 4); the difficulty estimation and reward shaping rely on this proxy, but its validity for medical QA outputs is not re-established.
  • ad hoc to paper Problem difficulty can be estimated by uncertainty-discounted sampling correctness (Eq. 5).
    This is a new formulation introduced by the paper; it is plausible but not derived from first principles.
  • domain assumption For difficult questions, extending output length improves the chance of correctness; for easy questions, short correct answers are preferable.
    Underpins the sign and shape of the length reward in Eq. 6; supported by prior literature but not demonstrated within this paper's data.
  • domain assumption RL training on AlphaMed19k (19k samples) transfers to the six evaluation benchmarks.
    In-domain training on MedQA/MedMCQA training splits is evaluated on out-of-domain benchmarks; no analysis of distribution shift is provided.

pith-pipeline@v1.3.0-alltime-deepseek · 16844 in / 13629 out tokens · 98802 ms · 2026-08-04T13:51:44.609457+00:00 · methodology

0 comments
read the original abstract

Extended Chain-of-Thought (CoT) reasoning has significantly bolstered the capabilities of medical large language models (LLMs). However, current models exhibit static computational expenditure, applying lengthy reasoning processes indiscriminately to both simple queries and complex diagnostic cases. This inefficiency is particularly prohibitive in real-world healthcare, where clinical scenarios range from time-sensitive emergencies requiring rapid response to intricate pathologies demanding deep analysis. To address this, we propose \textbf{AdaThink-Med}\footnote{Code available at https://github.com/shaohao011/AdaThinkMed}, an end-to-end framework for adaptive reasoning via uncertainty-guided length calibration. Although the underlying mechanism is generalizable, we demonstrate its critical value in the medical domain, where balancing inference latency with diagnostic precision is paramount. AdaThink-Med leverages entropy-based uncertainty estimation within reinforcement fine-tuning to dynamically shape reward signals: it penalizes verbosity for high-confidence correct answers (e.g., straightforward knowledge retrieval) while incentivizing extended exploration for uncertain or ambiguous scenarios. Across six medical benchmarks, AdaThink-Med reduces inference token consumption by \textbf{4.7$\times$ to 6.4$\times$} on Qwen and Llama architectures, respectively, with minimal performance trade-offs. Notably, our reward design naturally produces distinct ``non-thinking'' and ``thinking'' modes within a single model, enabling efficient allocation of computational resources without any external router or classifier.

Figures

Figures reproduced from arXiv: 2509.24560 by Kaitao Chen, Shaohao Rui, Weijie Ma, Xiaosong Wang.

Figure 1
Figure 1. Figure 1: Overview of AdaThink-Med. (a) A medical reasoning model is first trained via RFT on [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Length reward hack during RL with greedy length calibration. (a) RL with a close-ended [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a) Output length distribution w and w/o length calibration. (b) Dataset selection results. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Performance comparison w and w/o staged adaptive training. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Case 1. AdaThink-Med delivers correct answer while consumes far fewer tokens than [PITH_FULL_IMAGE:figures/full_fig_p016_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Case 2. AdaThink-Med delivers correct answer while consumes far fewer tokens than [PITH_FULL_IMAGE:figures/full_fig_p017_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Case 3. An overthinking case of UltraMedical-8B-3.1 and HuatuoGPT-o1-8B. [PITH_FULL_IMAGE:figures/full_fig_p018_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Case 4. An overthinking case of HuatuoGPT-o1-8B. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Medical Reasoning with Large Language Models: A Survey and MR-Bench

    cs.CL 2026-03 accept novelty 5.0

    LLMs show strong exam performance on medical tasks but exhibit a clear gap in accuracy on authentic clinical decision-making as measured by the new MR-Bench benchmark and unified evaluations.

Reference graph

Works this paper leans on

33 extracted references · 26 linked inside Pith · cited by 1 Pith paper

  1. [1]

    A survey on data selection for language models

    Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Muennighoff, Bairu Hou, Liangming Pan, Haewon Jeong, et al. A survey on data selection for language models. arXiv preprint arXiv:2402.16827,

    Pith/arXiv arXiv

  2. [4]

    Med42–evaluating fine-tuning strategies for medical llms: Full-parameter vs

    Cl´ement Christophe, Praveen K Kanithi, Prateek Munjal, Tathagata Raha, Nasir Hayat, Ronnie Rajan, Ahmed Al-Mahrooqi, Avani Gupta, Muhammad Umar Salman, Gurpreet Gosal, et al. Med42–evaluating fine-tuning strategies for medical llms: Full-parameter vs. parameter-efficient approaches. arXiv preprint arXiv:2404.14779,

    Pith/arXiv arXiv

  3. [5]

    The entropy mechanism of reinforcement learning for reasoning language models

    Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al. The entropy mechanism of reinforcement learning for reasoning language models. arXiv preprint arXiv:2505.22617,

    Pith/arXiv arXiv

  4. [6]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv

  5. [7]

    Thinkless: Llm learns when to think.arXiv preprint arXiv:2505.13379,

    Gongfan Fang, Xinyin Ma, and Xinchao Wang. Thinkless: Llm learns when to think.arXiv preprint arXiv:2505.13379,

    Pith/arXiv arXiv

  6. [8]

    Deepseek-coder: When the large language model meets programming–the rise of code intelligence

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196,

    Pith/arXiv arXiv

  7. [9]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948,

    Pith/arXiv arXiv

  8. [10]

    m1: Unleash the po- tential of test-time scaling for medical reasoning with large language models

    Xiaoke Huang, Juncheng Wu, Hui Liu, Xianfeng Tang, and Yuyin Zhou. m1: Unleash the po- tential of test-time scaling for medical reasoning with large language models. arXiv preprint arXiv:2504.00869,

    arXiv

  9. [11]

    Think only when you need with large hybrid-reasoning models

    Lingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong, Zewen Chi, Li Dong, Xingxing Zhang, Tengchao Lv, Lei Cui, and Furu Wei. Think only when you need with large hybrid-reasoning models. arXiv preprint arXiv:2505.14631,

    Pith/arXiv arXiv

  10. [14]

    Reasoning models can be effective without thinking

    Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can be effective without thinking. arXiv preprint arXiv:2504.09858,

    Pith/arXiv arXiv

  11. [15]

    Improving medical reasoning with curriculum-aware reinforcement learning

    Shaohao Rui, Kaitao Chen, Weijie Ma, and Xiaosong Wang. Improving medical reasoning with curriculum-aware reinforcement learning. arXiv preprint arXiv:2505.19213, 2025a. Shaohao Rui, Haoyang Su, Jinyi Xiang, Lian-Ming Wu, and Xiaosong Wang. Cardiocot: Hierarchi- cal reasoning for multimodal survival analysis. arXiv preprint arXiv:2505.19195, 2025b. Yi She...

    Pith/arXiv arXiv

  12. [16]

    Hybridflow: A flexible and efficient rlhf framework

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv:2409.19256,

    Pith/arXiv arXiv

  13. [17]

    Between underthinking and over- thinking: An empirical study of reasoning length and correctness in llms

    Jinyan Su, Jennifer Healey, Preslav Nakov, and Claire Cardie. Between underthinking and over- thinking: An empirical study of reasoning length and correctness in llms. arXiv preprint arXiv:2505.00127,

    Pith/arXiv arXiv

  14. [18]

    Stop overthinking: A survey on efficient reasoning for large language models

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419,

    Pith/arXiv arXiv

  15. [19]

    Challenging big-bench tasks and whether chain-of-thought can solve them

    Mirac Suzgun, Nathan Scales, Nathanael Sch¨arli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261,

    Pith/arXiv arXiv

  16. [21]

    The Register

    URLhttps://arxiv.org/abs/2505.09388. The Register. Alibaba launches qwen3 with hybrid thinking capabilities, July

    Pith/arXiv arXiv

  17. [22]

    Accessed: 2025-08-27

    URLhttps://www.theregister.com/2025/07/31/alibaba_qwen3_hybrid_ thinking/. Accessed: 2025-08-27. Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving. Advances in Neural Information Processing Systems, 37:7821–7846,

    2025

  18. [23]

    Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning

    Shenzhi Wang, Le Yu, Chang Gao, Chujie Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xionghui Chen, Jianxin Yang, Zhenru Zhang, et al. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning. arXiv preprint arXiv:2506.01939,

    Pith/arXiv arXiv

  19. [24]

    Fast-slow thinking for large vision-language model reasoning

    Wenyi Xiao, Leilei Gan, Weilong Dai, Wanggui He, Ziwei Huang, Haoyuan Li, Fangxun Shu, Zhelun Yu, Peng Zhang, Hao Jiang, et al. Fast-slow thinking for large vision-language model reasoning. arXiv preprint arXiv:2504.18458,

    arXiv

  20. [25]

    Demystifying long chain- of-thought reasoning in llms

    Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long chain- of-thought reasoning in llms. arXiv preprint arXiv:2502.03373,

    Pith/arXiv arXiv

  21. [26]

    Shorterbetter: Guiding reasoning models to find optimal inference length for efficient reasoning

    Jingyang Yi, Jiazheng Wang, and Sida Li. Shorterbetter: Guiding reasoning models to find optimal inference length for efficient reasoning. arXiv preprint arXiv:2504.21370,

    arXiv

  22. [27]

    Dapo: An open-source llm reinforcement learning system at scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476,

    Pith/arXiv arXiv

  23. [28]

    Synapseroute: An auto-route switching framework on dual-state large language model

    Wencheng Zhang, Shiqin Qiao, Lingjie Luo, Yinfeng Li, Chuanyang Zheng, Qian Xu, Meng Li, Yong Gui, Yijun He, Jianing Qiu, et al. Synapseroute: An auto-route switching framework on dual-state large language model. arXiv preprint arXiv:2507.02822,

    Pith/arXiv arXiv

  24. [29]

    Least-to-most prompting enables complex reasoning in large language models

    Denny Zhou, Nathanael Sch ¨arli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuur- mans, Claire Cui, Olivier Bousquet, Quoc Le, et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625,

    Pith/arXiv arXiv

  25. [30]

    Medxpertqa: Benchmarking expert-level medical reasoning and understanding

    Yuxin Zuo, Shang Qu, Yifei Li, Zhangren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. Medxpertqa: Benchmarking expert-level medical reasoning and understanding. arXiv preprint arXiv:2501.18362,

    Pith/arXiv arXiv

  26. [31]

    equation 1, both critic-based reinforcement learning methods (e.g., PPO) and critic-free methods (e.g., GRPO (Guo et al., 2024)) can be applied

    12 Arxiv Preprint A APPENDIX B EXPERIMENTALSETUP B.1 TRAININGRL ALGORITHM Group Relative Policy Optimization (GRPO).To optimize the efficient reasoning objective in Eq. equation 1, both critic-based reinforcement learning methods (e.g., PPO) and critic-free methods (e.g., GRPO (Guo et al., 2024)) can be applied. Considering its simplicity and effectivenes...

    2024

  27. [32]

    We adopt the Qwen2.5 (Team, 2024)-7B-Instruct model and LLama3.1-Instruct-8B as the backbone models

    framework to accelerate the training. We adopt the Qwen2.5 (Team, 2024)-7B-Instruct model and LLama3.1-Instruct-8B as the backbone models. The training uses a total batch size of 256, with a constant learning rate of1e −6. The KL penalty coef- ficient is set toβ= 0.01. For inference during GRPO, we deploy the model usingvllm(Kwon et al.,

    2024

  28. [33]

    yes,” “no,

    on 2 GPUs, generating 8 completions per sample, corresponding to the group sizeG in GRPO. The temperature forvllmsampling is set to 1.0. As for our staged adaptive training, we first train the model using GRPO for 300 steps until full convergence is achieved. Subsequently, the proposed length calibration method is applied and train for 200 steps. Repetiti...

    2025

  29. [2021]

    Pubmedqa: A dataset for biomedical research question answering

    Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William W Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering. arXiv preprint arXiv:1909.06146,

    Pith/arXiv arXiv 1909

  30. [2022]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599,

    Pith/arXiv arXiv

  31. [2023]

    Beyond distillation: Pushing the limits of medical llm reasoning with minimalist rule-based rl

    Che Liu, Haozhe Wang, Jiazhen Pan, Zhongwei Wan, Yong Dai, Fangzhen Lin, Wenjia Bai, Daniel Rueckert, and Rossella Arcucci. Beyond distillation: Pushing the limits of medical llm reasoning with minimalist rule-based rl. arXiv preprint arXiv:2505.17952, 2025a. Wei Liu, Ruochen Zhou, Yiyun Deng, Yuzhen Huang, Junteng Liu, Yuntian Deng, Yizhe Zhang, and Junx...

    Pith/arXiv arXiv

  32. [2024]

    Language models are hidden reasoners: Unlock- ing latent reasoning capabilities via self-rewarding

    Haolin Chen, Yihao Feng, Zuxin Liu, Weiran Yao, Akshara Prabhakar, Shelby Heinecke, Ricky Ho, Phil Mui, Silvio Savarese, Caiming Xiong, et al. Language models are hidden reasoners: Unlock- ing latent reasoning capabilities via self-rewarding. arXiv preprint arXiv:2411.04282, 2024a. Junying Chen, Zhenyang Cai, Ke Ji, Xidong Wang, Wanlong Liu, Rongsheng Wan...

    Pith/arXiv arXiv

  33. [2025]

    Do not think that much for 2+ 3=? on the overthinking of o1-like llms

    Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of o1-like llms. arXiv preprint arXiv:2412.21187, 2024c. Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. Reasoning with ...

    Pith/arXiv arXiv