Pith. sign in

REVIEW 2 major objections 6 minor 169 references

Small language models can reason accurately on mobile devices when LoRA adapters, length-forcing reinforcement learning, and parallel verification are combined under tight memory limits.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 23:26 UTC pith:VR2R5PCQ

load-bearing objection Solid edge-systems integration with a few real engineering tricks; the parallel-TTS claim is the softest link, not the whole stack. the 2 major comments →

arxiv 2603.16867 v2 pith:VR2R5PCQ submitted 2026-03-17 cs.LG cs.CL

Efficient Reasoning on the Edge

Yelysei Bondarenko , Thomas Hehn , Rob Hesselink , Romain Lepert , Fabio Valerio Massoli , Evgeny Mironov , Leyla Mirvakhabova , Tribhuvanesh Orekondy
show 10 more authors
Spyridon Stasis Andrey Kuzmin Anna Kuzina Markus Nagel Ankita Nayak Corrado Rainone Ork de Rooij Paul N Whatmough Arash Behboodi Babak Ehteshami Bejnordi
This is my paper
classification cs.LG cs.CL
keywords edge LLM reasoningLoRA adaptersbudget forcingparallel test-time scalingKV-cache sharingon-device quantizationhybrid reasoning modelsoft-barrier reward
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Chain-of-thought reasoning is powerful but usually too long and memory-heavy for phones. This paper shows a practical path: freeze a compact instruct model, attach low-rank adapters, and train those adapters first on high-quality reasoning traces and then with reinforcement learning that multiplies accuracy by a soft length barrier. A tiny switcher turns reasoning on only when a query needs it, and masked adapter training lets the base model and the reasoning mode share one prompt cache so switching does not force a full re-encode. Parallel answer streams plus a lightweight verifier head reclaim accuracy that short traces would otherwise lose, while 4-bit weights keep the stack inside mobile memory. The result is a hybrid system that stays fast for ordinary chat yet still solves hard math and coding problems on-device.

Core claim

On Qwen2.5-7B, rank-128 LoRA adapters trained with supervised fine-tuning on OpenThoughts3 traces and then GRPO under a multiplicative soft-barrier budget reward recover most dense-distillation accuracy while cutting average completion length by roughly 2.4 imes. Dynamic routing, KV-cache sharing via masked prefill, parallel test-time scaling with a linear verifier head, and W4A16KV8 quantization together make accurate reasoning practical under strict edge budgets, staying within about 2% of full-precision reasoning performance.

What carries the argument

Budget forcing: a multiplicative soft-barrier reward R = R_accuracy × R_budget(L) applied to LoRA adapters via GRPO, combined with masked LoRA prefill so adapters reuse the base-model KV cache, and a lightweight switcher that activates reasoning only when needed.

Load-bearing premise

On-device decoding stays memory-bound enough that generating several parallel answer streams adds only minor latency; if the chip saturates earlier, that accuracy boost is no longer nearly free.

What would settle it

On the target mobile NPU, measure wall-clock time-to-final-answer and MATH500 accuracy for 1, 2, 4, and 8 parallel streams of the same quantized model; if latency rises roughly linearly with stream count while accuracy gains shrink, the claimed minor-latency parallel scaling fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Edge devices can host hybrid chat/reasoning models without full-parameter distillation or cloud offload for complex queries.
  • Average reasoning-token cost can fall by about 2.4× (up to 8× on some queries) with only small accuracy loss on MATH500.
  • A few parallel streams plus a linear verifier head can raise accuracy by roughly 10% over greedy decoding under memory-bound on-device generation.
  • One frozen backbone can switch at runtime between ordinary instruct mode and specialized reasoning by enabling or disabling adapters.
  • 4-bit weight quantization with function-preserving transforms plus quantization-aware modular reasoning keeps performance within about 2% of full-precision on reasoning benchmarks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the memory-bound premise generalizes, the same parallel-stream plus lightweight-head pattern could help other autoregressive edge models beyond language.
  • Learning the router with reinforcement learning rather than supervised labels could jointly optimize accuracy and length by sending easy queries straight to the base model.
  • Extending the switcher to a bank of task-specific or latent-reasoning adapters would let one backbone support multiple specialized modes without loading separate models.
  • Penalizing tokens by information density rather than raw count could compress traces further while protecting high-utility reasoning steps.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper presents an end-to-end pipeline for deploying chain-of-thought reasoning on edge devices, starting from a non-reasoning Qwen2.5 instruct backbone. Reasoning is elicited via LoRA adapters trained with SFT on OpenThoughts3 (and MoT) traces, then refined with GRPO reinforcement learning under a multiplicative soft-barrier budget reward that shortens completions. A lightweight switcher routes queries to base or reasoning mode; masked LoRA training enables KV-cache reuse across modes. Parallel test-time scaling with a linear verifier head and weighted majority voting is proposed to raise accuracy under memory-bound decoding. The stack is quantized to W4A16KV8 via FPTQuant and Quantization-Aware Modular Reasoning (QAMR). Experiments on Qwen2.5-3B/7B report competitive math/science/coding scores, ~2.4× average length reduction with small accuracy loss, switcher accuracy–cost curves, and near full-precision quantized reasoning accuracy.

Significance. If the claims hold, the work is a practical blueprint for on-device reasoning: modular LoRA adapters avoid full distillation, budget-forced RL addresses verbosity without hard truncation hacks, masked LoRA solves a real KV-cache switching cost, and QAMR shows that reasoning adapters can be trained on a 4-bit base with limited loss. The soft-barrier reward and qualitative traces of reduced epistemic hesitation are useful contributions. Strengths include extensive LoRA hyperparameter sweeps (Tables 2–7, 12–13), multi-benchmark evaluation (Table 1), budget-forcing length CDFs (Figures 4–5, Table 8), and a careful quantization stack (Tables 10–11). The open tooling path (FastForward, GENIE) and mobile demos further support reproducibility. The main significance is systems integration under edge constraints rather than a single algorithmic breakthrough.

major comments (2)
  1. [Section 6.3, Table 9] §6.3 and Table 9 evaluate greedy, majority, and weighted majority voting only on 4-bit Qwen2.5-7B-Instruct (MATH500 greedy ~71%), not on the OT3 LoRA-r128 + budget-forced reasoner that reaches ~90–95% single-sample accuracy (Tables 1, 8, 11). At that higher base accuracy, sample diversity and Best-of-N gains typically shrink. The abstract and Fig. 1b package parallel TTS as part of the edge reasoning system; without results on the actual adapters, it is unclear whether parallel scaling still moves the needle for the deployed artifact. Please report the same protocol on the budget-forced LoRA model (and ideally the quantized QAMR model).
  2. [Section 6, Figure 1b] The claim of accuracy gains 'at minor latency increase' (abstract; §6; Fig. 1b) rests on the premise that on-device decoding is memory-bound so N parallel streams cost little extra wall-clock time. The manuscript reports only accuracy for N∈{1…8}; there are no TPS, TTFT, end-to-end latency, or peak-memory figures for N>1 on the target NPU/Genie stack. If the memory hierarchy or NPU saturates earlier, the accuracy–latency trade-off collapses. For an edge-deployment paper this measurement is load-bearing; please add on-device latency/memory profiles for sequential vs. parallel decoding (and for switcher on/off).
minor comments (6)
  1. [Section 4.2, Figure 3] Switcher evaluation (Fig. 3) is confined to MATH500. A brief multi-domain check (e.g., mixed chat + GPQA + simple MMLU) would better support the claim that day-to-day queries are correctly routed away from reasoning mode.
  2. [Section 5.1, Equation (2)] Soft-barrier hyperparameters m and p (Eq. 2) and the discrete budget buckets {1K,3K,4K,6K} are free parameters; a short sensitivity note (or fixed defaults with justification) would help reproducibility.
  3. [Section 3.4, Table 1] Table 1 shows clear degradation on HumanEval/MBPP after reasoning SFT; the text acknowledges the trade-off but could more explicitly state whether budget-forced RL or the switcher recovers any of that direct-answer coding performance.
  4. [Section 4, Masked LoRA paragraph] Masked LoRA is stated to incur 'no drop in reasoning accuracy' but no side-by-side numbers (masked vs. unmasked) appear in the main tables; a one-row ablation would strengthen that axiom.
  5. [Throughout] Minor presentation: 'T o' / 'T able' spacing artifacts appear throughout (e.g., 'T o address', 'T able 1'); clean for camera-ready. Also unify 'Co T' vs. 'CoT' and 'MA TH500' vs. 'MATH500'.
  6. [Section 7.4 / Conclusion] Project-page videos are cited as evidence of mobile deployment; a short quantitative table (model size, peak DRAM, tokens/s, TTFT on a named device) in the main text would make the edge claim self-contained without requiring external media.

Circularity Check

1 steps flagged

Empirical systems paper: accuracy/length metrics are measured on held-out benchmarks; no claimed prediction reduces to a fitted input by construction. Only minor non-load-bearing self-citation of the authors' FPTQuant transforms.

specific steps
  1. other [Section 7.1–7.2, Table 10; citation [143] FPTQuant]
    "To maximize the accuracy of the quantized model, we apply the subset of fully-mergeable transformations from FPTQuant [143] (Figure 8)... We simulate quantization using FastForward [18]. For brevity, we denote to the aforementioned quantization pipeline as FPTQuant◦."

    FPTQuant has overlapping authors (Bondarenko, Whatmough, Nagel). This is ordinary method reuse, not load-bearing circularity: transforms are re-trained end-to-end on this backbone and accuracy is re-measured on WikiText/CSR/MMLU and later reasoning suites. No uniqueness claim is imported, and the main reasoning/length claims do not reduce to this citation.

full rationale

The paper's load-bearing claims are engineering results measured after training: LoRA-SFT on OT3/MoT evaluated on AIME/MATH500/GPQA/LCB (Tables 1–3), GRPO soft-barrier budget forcing evaluated by completion-length CDFs and MATH500 accuracy under hard caps (Figs. 4–5, Table 8), switcher routing curves on MATH500 (Fig. 3), parallel TTS with a linear verifier head trained on MATH train and scored on MATH500 (Table 9), and W4A16KV8 QAMR accuracy (Tables 10–11). None of these quantities is defined as, or algebraically forced by, the free parameters of the soft barrier (m, p, buckets), the switcher labels, or the verifier BCE objective. The soft-barrier reward R = R_accuracy × R_budget(L) is an optimization objective, not a prediction of length; length reduction (~2.4×) is an empirical outcome. The only self-citation of note is FPTQuant (overlapping authors) for mergeable transforms; those transforms are re-initialized and re-trained on Qwen2.5-7B and re-measured, so they do not smuggle uniqueness or force the reasoning results. No uniqueness theorem, ansatz-via-citation, or fitted-constant-as-prediction pattern appears in the derivation chain. Score 1 only for the minor FPTQuant self-reference; central claims remain independently measured.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 5 invented entities

The central empirical claims rest on a handful of hand-chosen hyper-parameters (LoRA rank, β_KL, budget buckets, soft-barrier window) and on standard domain assumptions about edge hardware and teacher-trace quality; no new physical entities are postulated.

free parameters (5)
  • LoRA rank = 128
    Chosen as 128 after ablation; directly controls adapter capacity and the fraction of trainable parameters (4.24 % for 7B).
  • β_KL (GRPO KL coefficient) = 1e-3
    Tuned to 1e-3 (or 1e-4) to trade length compression against accuracy; acts as the effective strength of budget forcing.
  • soft-barrier half-window m and floor p = p=0, m tuned
    m ∈ [0,1] and p=0 define the linear decay region around each prompted budget B; chosen by hand to avoid reward hacking.
  • budget buckets = {1k,3k,4k,6k}
    Discrete lengths {1000,3000,4000,6000} used both for prompting and for the soft barrier; arbitrary discretization of the continuous length axis.
  • switcher threshold / EMA α = α=0.5
    Confidence threshold and α=0.5 for chunked prefill control the accuracy–latency operating point; selected on MATH500.
axioms (4)
  • domain assumption On-device decoding is memory-bound, so N parallel streams incur only minor extra latency
    Stated in Section 6 and used to justify parallel test-time scaling; if false on the target NPU the latency claim fails.
  • domain assumption High-quality CoT traces from larger teachers (DeepSeek-R1, QwQ-32B) are sufficient to elicit reasoning via LoRA SFT
    Underpins the entire SFT stage (Section 3); no proof that denser or different data would not be required.
  • ad hoc to paper Masked LoRA (adapters off during prompt prefill) does not degrade final reasoning accuracy
    Empirically claimed in Section 4 to enable KV-cache sharing; treated as free after a single check.
  • domain assumption Standard GRPO + multiplicative soft-barrier reward is free of residual reward hacking
    Authors report that penalizing total length (not just CoT) closes the </think> exploit; assumed to generalize.
invented entities (5)
  • soft-barrier multiplicative budget reward no independent evidence
    purpose: Replace additive length penalty with a piecewise-linear multiplier that decays from 1 to 0 around a prompted budget, blocking common reward hacks.
    Defined in Eqs. (2)–(3); no independent theoretical derivation, only empirical success on DeepScaleR.
  • masked LoRA training for KV-cache reuse no independent evidence
    purpose: Force adapters to condition on base-model prompt KV so that dynamic switching never re-encodes the prompt.
    Introduced in Section 4; claimed zero accuracy drop but not independently validated outside this pipeline.
  • Quantization-Aware Modular Reasoning (QAMR) no independent evidence
    purpose: Train LoRA adapters on an already 4-bit quantized base so that distribution shift does not destroy reasoning.
    Section 7.3; necessary for the quantized results in Table 11.
  • lightweight linear verifier head + verification prompt no independent evidence
    purpose: Score parallel candidates while re-using the generator’s KV cache, enabling weighted majority vote at near-zero extra cost.
    Section 6.1; trained on MATH training set generations.
  • Switcher classifier head (MLP dim-8) no independent evidence
    purpose: Binary decision to activate or bypass reasoning LoRA based on prompt hidden states.
    Section 4; trained on a 2 k mix of easy/hard prompts.

pith-pipeline@v1.1.0-grok45 · 46816 in / 3474 out tokens · 37839 ms · 2026-07-13T23:26:23.538250+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) with chain-of-thought reasoning achieve state-of-the-art performance across complex problem-solving tasks, but their verbose reasoning traces and large context requirements make them impractical for edge deployment. These challenges include high token generation costs, large KV-cache footprints, and inefficiencies when distilling reasoning capabilities into smaller models for mobile devices. Existing approaches often rely on distilling reasoning traces from larger models into smaller models, which are verbose and stylistically redundant, undesirable for on-device inference. In this work, we propose a lightweight approach to enable reasoning in small LLMs using LoRA adapters combined with supervised fine-tuning. We further introduce budget forcing via reinforcement learning on these adapters, significantly reducing response length with minimal accuracy loss. To address memory-bound decoding, we exploit parallel test-time scaling, improving accuracy at minor latency increase. Finally, we present a dynamic adapter-switching mechanism that activates reasoning only when needed and a KV-cache sharing strategy during prompt encoding, reducing time-to-first-token for on-device inference. Experiments on Qwen2.5-7B demonstrate that our method achieves efficient, accurate reasoning under strict resource constraints, making LLM reasoning practical for mobile scenarios. Videos demonstrating our solution running on mobile devices are available on our project page.

Figures

Figures reproduced from arXiv: 2603.16867 by Andrey Kuzmin, Ankita Nayak, Anna Kuzina, Arash Behboodi, Babak Ehteshami Bejnordi, Corrado Rainone, Evgeny Mironov, Fabio Valerio Massoli, Leyla Mirvakhabova, Markus Nagel, Ork de Rooij, Paul N Whatmough, Rob Hesselink, Romain Lepert, Spyridon Stasis, Thomas Hehn, Tribhuvanesh Orekondy, Yelysei Bondarenko.

Figure 1
Figure 1. Figure 1: Overview of the proposed efficient reasoning framework for edge devices. (a) The model architecture utilizes parameter-efficient LoRA adapters and a lightweight switcher to dynamically route queries. This design allows the base model and the reasoning-activated mode to seamlessly share a reusable KV cache during prefill. (b) Parallel test-time scaling strategy, generating multiple reasoning streams concurr… view at source ↗
Figure 2
Figure 2. Figure 2: Architecture of the Hybrid Reasoning Model. The pipeline begins with a compact base LLM, which is specialized for reasoning via LoRA-based supervised fine-tuning (SFT). To enforce concise generation and prevent excessive verbosity, these adapters undergo reinforcement learning (RL) with Budget Forcing. Finally, a lightweight Switcher module is introduced to act as a reasoning-needed classifier, creating a … view at source ↗
Figure 3
Figure 3. Figure 3: Impact of the Switcher module on MATH500. Left: Combined model accuracy as the fraction of queries routed to the reasoning adapters. Right: Average completion length versus overall accuracy across different switcher thresholds. ing a higher-accuracy operating model requires a proportional increase in computational costs, while lower-cost regimes are possible when accuracy demands are modest. The switcher t… view at source ↗
Figure 4
Figure 4. Figure 4: Average Completion Length Distributions. Left: Evaluation with a forced maximum completion length of 4K tokens. Right: Evaluation with a maximum of 6K tokens. Note that distribution tails extending below zero or above the maximum budget are standard artifacts of Kernel Density Estimation (KDE) curve smoothing. The progression from the baseline (purple) through the intermediate (blue) to the final RL fine-t… view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p014_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative comparison on algebraic simplification. Middle: The Baseline trace correctly identifies the difference of squares strategy immediately but engages in excessive self-verification, re-calculating the result via expansion, direct computation, and alternative factorizations. Bottom: The Budget Forced trace recognizes the nested difference of squares structure and executes the solution linearly with… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative comparison on modular arithmetic. Middle: The Baseline trace correctly computes the sum and remainder immediately but engages in extensive, redundant verification using four different methods (step￾by-step addition, digit sum rule, pairing, and re-calculation). Bottom: The Budget Forced trace performs the direct calculation and returns the result without hesitation. Parallel generation is not o… view at source ↗
Figure 8
Figure 8. Figure 8: Function-Preserving Transformations. We use 4 transform types from FPTQuant: scale-and-rotate transform Tk merged into query and key, a per-channel scaler Tu merged into up and down projection, and Tv that consists of invertible matrices per head merged into value and output weights, and a rotation matrix Tr for rotating residuals (shared across layers). After training of the transforms is complete, the tr… view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative comparison on number theory reasoning. Text highlighted in red denotes redundant verifi￾cation and verbal parsing, while bold text identifies essential reasoning steps. We use “[... ]” as a placeholder for brevity. Top: Prompt. Middle: The Baseline trace correctly identifies the property (p 2 ) early on but falls into exten￾sive, redundant verification loops checking composite numbers and re-li… view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative comparison on pattern recognition. Middle: The Baseline trace correctly identifies the formula 2n − 1 initially but spends nearly 1000 tokens validating it against alternative arithmetic formulas (a + (n − 1)d) and hypothetical user errors (confusing term number with value, testing n 2 , etc.). Bottom: The Budget Forced trace directly retrieves the formula and computes the specific term reques… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

169 extracted references · 1 canonical work pages

  1. [1]

    The claude 3 model family: Opus, sonnet, haiku

    Anthropic. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card , 1(1):4, 2024. (Cited on page 1)

    2024

  2. [2]

    Openai o1 system card

    Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Alek- sander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720 ,

    Pith/arXiv arXiv

  3. [3]

    (Cited on page 1, 3)

  4. [4]

    First proof.arXiv preprint arXiv:2602.05192, 5 February 2026

    Mohammed Abouzaid, Andrew J Blumberg, Martin Hairer, Joe Kileel, T amara G Kolda, Paul D Nelson, Daniel Spielman, Nikhil Srivastava, Rachel Ward, Shmuel Weinberger, and Lauren Williams. First proof.arXiv preprint arXiv:2602.05192, 5 February 2026. (Cited on page 1)

    arXiv 2026

  5. [5]

    Evaluating frontier LLMs on PhD-level mathematical reasoning: A benchmark on a textbook in theoretical computer science about randomized algorithms

    Yang Cao, Yubin Chen, Xuyang Guo, Zhao Song, Song Yue, Jiahao Zhang, and Jiale Zhao. Evaluating frontier LLMs on PhD-level mathematical reasoning: A benchmark on a textbook in theoretical computer science about randomized algorithms. arXiv [cs.AI], 16 December 2025. (Cited on page 1)

    2025

  6. [6]

    T owards autonomous mathematics research

    T ony Feng, Trieu H Trinh, Garrett Bingham, Dawsen Hwang, Yuri Chervonyi, Junehyuk Jung, Joonkyung Lee, Carlo Pagano, Sang-Hyun Kim, Federico Pasqualotto, Sergei Gukov, Jonathan N Lee, Junsu Kim, Kaiying Hou, Golnaz Ghiasi, Yi T ay, Yaguang Li, Chenkai Kuang, Yuan Liu, Hanzhao Lin, Evan Zheran Liu, Nigamaa Nayakanti, Xiaomeng Yang, Heng- Tze Cheng, Demis ...

    arXiv 2026

  7. [7]

    David P Woodruff, Vincent Cohen-Addad, Lalit Jain, Jieming Mao, Song Zuo, Mohammadhossein Bateni, Simina Branzei, Michael P Brenner, Lin Chen, Ying Feng, Lance Fortnow, Gang Fu, Ziyi Guan, Zahra Hadizadeh, Mohammad T Hajiaghayi, Mahdi JafariRaviz, Adel Javanmard, Karthik C. S., Ken-Ichi Kawarabayashi, Ravi Kumar, Silvio Lattanzi, Euiwoong Lee, Yi Li, Ioan...

    arXiv

  8. [8]

    ReAct: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. InThe Eleventh International Conference on Learning Representations, 29 September 2022. (Cited on page 1)

    2022

  9. [9]

    MAI-UI technical report: Real-world centric foundation GUI agents

    Hanzhang Zhou, Xu Zhang, Panrong T ong, Jianan Zhang, Liangyu Chen, Quyu Kong, Chenglin Cai, Chen Liu, Yue Wang, Jingren Zhou, and Steven Hoi. MAI-UI technical report: Real-world centric foundation GUI agents. arXiv [cs.CV], 26 December 2025. (Cited on page 1)

    2025

  10. [10]

    26 Chaolin Jin, Chen Li, Hao Chen, Haoli Chen, Jian Chen, Qinghao Zhao, and Guang Shi

    Haoming Wang, Haoyang Zou, Huatong Song, Jiazhan Feng, Junjie Fang, Junting Lu, Longxiang Liu, Qinyu Luo, Shihao Liang, Shijue Huang, Wanjun Zhong, Yining Ye, Yujia Qin, Yuwen Xiong, Yuxin Song, Zhiyong Wu, Aoyan Li, Bo Li, Chen Dun, Chong Liu, Daoguang Zan, Fuxing Leng, Hanbin Wang, Hao Yu, Haobin Chen, Hongyi Guo, Jing Su, Jingjia Huang, Kai Shen, Kaiyu...

    2025

  11. [11]

    UI-venus-1.5 technical report

    Venus T eam, Changlong Gao, Zhangxuan Gu, Yulin Liu, Xinyu Qiu, Shuheng Shen, Yue Wen, Tianyu Xia, Zhenyu Xu, Zhengwen Zeng, et al. UI-venus-1.5 technical report. arXiv [cs.CV], 9 February 2026. (Cited on page 1)

    2026

  12. [12]

    https://nousresearch

    Measuring thinking efficiency in reasoning models: The missing benchmark. https://nousresearch. com/measuring-thinking-efficiency-in-reasoning-models-the-missing-benchmark/ , 14 August 2025. Accessed: 2026-2-19. (Cited on page 1)

    2025

  13. [13]

    LLM in a flash: Efficient large language model inference with limited memory

    Keivan Alizadeh, Iman Mirzadeh, Dmitry Belenko, S Karen Khatamifard, Minsik Cho, Carlo C Del Mundo, Mo- hammad Rastegari, and Mehrdad Farajtabar. LLM in a flash: Efficient large language model inference with limited memory. In ACL, 2024. (Cited on page 2)

    2024

  14. [14]

    Understanding large language models in your pockets: Performance study on COTS mobile devices

    Jie Xiao, Qianyi Huang, Xu Chen, and Chen Tian. Understanding large language models in your pockets: Performance study on COTS mobile devices. arXiv [cs.LG], 9 February 2026. (Cited on page 2)

    2026

  15. [15]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In International Conference on Learning Repre- sentations, 2022. (Cited on page 2, 4)

    2022

  16. [16]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, and et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948. (Cited on page 2, 4)

    Pith/arXiv arXiv 2025

  17. [17]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettle- moyer, Percy Liang, Emmanuel Candès, and T atsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393 , 2025. doi: 10.48550/arXiv.2501.19393. URL https://arxiv.org/abs/ 2501.19393. (Cited on page 2, 10, 11)

    Pith/arXiv arXiv doi:10.48550/arxiv.2501.19393 2025

  18. [18]

    Steering llm thinking with budget guidance

    Junyan Li, Wenshuo Zhao, Yang Zhang, and Chuang Gan. Steering llm thinking with budget guidance. arXiv preprint arXiv:2506.13752, 2025. (Cited on page 2)

    Pith/arXiv arXiv 2025

  19. [19]

    https://github.com/ Qualcomm-AI-research/fastforward

    Fastforward: Neural network quantization for research and prototyping. https://github.com/ Qualcomm-AI-research/fastforward . (Cited on page 3, 20, 23)

  20. [20]

    https://www.qualcomm.com/developer/ software/gen-ai-inference-extensions

    Qualcomm gen ai inference extensions (genie). https://www.qualcomm.com/developer/ software/gen-ai-inference-extensions . (Cited on page 3, 23)

  21. [21]

    Large language models are zero-shot reasoners

    T akeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems , 35:22199–22213,

  22. [22]

    Show your work: Scratchpads for intermediate computation with language models, 2022

    Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. Show your work: Scratchpads for intermediate computation with language models, 2022. URL https:// openreview.net/forum?id=iedYJm92o0a. (Cited on page 3)

    2022

  23. [23]

    Chain-of-thought prompting elicits reasoning in large language mod- els

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language mod- els. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 24824–24837. Curran As...

    2022

  24. [24]

    Tina: Tiny reasoning models via lora

    Shangshang Wang, Julian Asilis, Ömer Faruk Akgül, Enes Burak Bilgin, Ollie Liu, and Willie Neiswanger. Tina: Tiny reasoning models via lora. arXiv preprint arXiv:2504.15777, 2025. (Cited on page 3)

    Pith/arXiv arXiv 2025

  25. [25]

    Phi-4-mini-reasoning: Exploring the limits of small reasoning lan- guage models in math

    Haoran Xu, Baolin Peng, Hany Awadalla, Dongdong Chen, Yen-Chun Chen, Mei Gao, Young Jin Kim, Yun- sheng Li, Liliang Ren, Yelong Shen, et al. Phi-4-mini-reasoning: Exploring the limits of small reasoning lan- guage models in math. arXiv preprint arXiv:2504.21233, 2025. (Cited on page 3)

    Pith/arXiv arXiv 2025

  26. [26]

    Think only when you need with large hybrid-reasoning models

    Lingjie Jiang, Xun Wu, Shaohan Huang, Qingxiu Dong, Zewen Chi, Li Dong, Xingxing Zhang, T engchao Lv, Lei Cui, and Furu Wei. Think only when you need with large hybrid-reasoning models. arXiv preprint arXiv:2505.14631, 2025. (Cited on page 3)

    Pith/arXiv arXiv 2025

  27. [27]

    Lora without regret, 2025

    John Schulman and Thinking Machines Lab. Lora without regret, 2025. https://thinkingmachines.ai/blog/lora/. (Cited on page 4)

    2025

  28. [28]

    Acereason-nemotron: Advancing math and code reasoning through reinforcement learning

    Yang Chen, Zhuolin Yang, Zihan Liu, Chankyu Lee, Peng Xu, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Acereason-nemotron: Advancing math and code reasoning through reinforcement learning. arXiv preprint arXiv:2505.16400, 2025. (Cited on page 4)

    Pith/arXiv arXiv 2025

  29. [29]

    Reinforcement learning for reasoning in small llms: What works and what doesn’t

    Quy-Anh Dang and Chris Ngo. Reinforcement learning for reasoning in small llms: What works and what doesn’t. arXiv preprint arXiv:2503.16219, 2025. (Cited on page 4)

    arXiv 2025

  30. [30]

    Reasoning on a budget: A survey of adaptive and controllable test-time compute in llms

    Mohammad Ali Alomrani, Yingxue Zhang, Derek Li, Qianyi Sun, Soumyasundar Pal, Zhanguang Zhang, Yaochen Hu, Rohan Deepak Ajwani, Antonios Valkanas, Raika Karimi, et al. Reasoning on a budget: A survey of adaptive and controllable test-time compute in llms. arXiv preprint arXiv:2507.02076, 2025. (Cited on page 4)

    Pith/arXiv arXiv 2025

  31. [31]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language mod- els

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language mod- els. arXiv preprint arXiv:2402.03300, 2024. (Cited on page 4, 12)

    Pith/arXiv arXiv 2024

  32. [32]

    Qwq-32b: Embracing the power of reinforcement learning

    Qwen T eam. Qwq-32b: Embracing the power of reinforcement learning. https://qwen.ai/blog?id= qwq-32b, 2025. (Cited on page 4)

    2025

  33. [33]

    Openthoughts: Data recipes for reasoning models

    Etash Guha, Ryan Marten, Sedrick Keh, Negin Raoof, Georgios Smyrnis, Hritik Bansal, Marianna Nezhurina, Jean Mercat, Trung Vu, Zayne Sprague, et al. Openthoughts: Data recipes for reasoning models. arXiv preprint arXiv:2506.04178, 2025. (Cited on page 4, 5, 7, 22)

    Pith/arXiv arXiv 2025

  34. [34]

    Qwen2.5 technical report, 2025

    Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jin- gren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, ...

    Pith/arXiv arXiv 2025

  35. [35]

    Open r1: A fully open reproduction of deepseek-r1, January 2025

    Hugging Face. Open r1: A fully open reproduction of deepseek-r1, January 2025. URL https://github. com/huggingface/open-r1. (Cited on page 5)

    2025

  36. [36]

    aime problems and solutions

    Art of problem solving. aime problems and solutions. 2024. URL https://artofproblemsolving. com/wiki/index.php/AIME_Problems_and_Solutions. (Cited on page 5, 22, 39)

    2024

  37. [37]

    amc problems and solutions

    Art of problem solving. amc problems and solutions. 2023. URLhttps://artofproblemsolving.com/ wiki/index.php/AMC_12_Problems_and_Solutions. (Cited on page 5, 22, 39)

    2023

  38. [38]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric T ang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. (Cited on page 5, 10, 17, 22, 39) Qualcomm AI Research is an initiative of Qualcomm T echnologies, Inc. 28

    Pith/arXiv arXiv 2021

  39. [39]

    Gpqa: A graduate-level google-proof q&a benchmark

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024. (Cited on page 5, 22, 39)

    2024

  40. [40]

    LiveCodeBench: Holistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. LiveCodeBench: Holistic and contamination free evaluation of large language models for code. arXiv [cs.SE], March 2024. (Cited on page 5, 39)

    2024

  41. [41]

    Evaluating large language models trained on code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    2021

  42. [42]

    Program synthesis with large language mod- els

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael T erry, Quoc Le, and Charles Sutton. Program synthesis with large language mod- els. arXiv [cs.PL], August 2021. (Cited on page 5, 39)

    2021

  43. [43]

    Is your code generated by ChatGPT really correct? rigorous evaluation of large language models for code generation

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. Is your code generated by ChatGPT really correct? rigorous evaluation of large language models for code generation. arXiv [cs.SE], May 2023. (Cited on page 5, 39)

    2023

  44. [44]

    Lighteval: A lightweight framework for llm evaluation, 2023

    Nathan Habib, Clémentine Fourrier, Hynek Kydlíček, Thomas Wolf, and Lewis Tunstall. Lighteval: A lightweight framework for llm evaluation, 2023. URL https://github.com/huggingface/ lighteval. (Cited on page 6)

    2023

  45. [45]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedat- tention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. (Cited on page 6)

    2023

  46. [46]

    Does math reasoning improve general llm capabilities? understanding transferability of llm reasoning

    Maggie Huan, Yuetai Li, Tuney Zheng, Xiaoyu Xu, Seungone Kim, Minxin Du, Radha Poovendran, Graham Neubig, and Xiang Yue. Does math reasoning improve general llm capabilities? understanding transferability of llm reasoning. arXiv preprint arXiv:2507.00432, 2025. (Cited on page 7)

    Pith/arXiv arXiv 2025

  47. [47]

    Reinforcement fine-tuning naturally mitigates forgetting in continual post-training

    Song Lai, Haohan Zhao, Rong Feng, Changyi Ma, Wenzhuo Liu, Hongbo Zhao, Xi Lin, Dong Yi, Qingfu Zhang, Hongbin Liu, et al. Reinforcement fine-tuning naturally mitigates forgetting in continual post-training. arXiv preprint arXiv:2507.05386, 2025. (Cited on page 7)

    Pith/arXiv arXiv 2025

  48. [48]

    Know what you don’t know: Unanswerable questions for squad,

    Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswerable questions for squad,

  49. [49]

    (Cited on page 10)

    URL https://arxiv.org/abs/1806.03822. (Cited on page 10)

    Pith/arXiv arXiv

  50. [50]

    Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies, 2021

    Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies, 2021. URL https://arxiv. org/abs/2101.02235. (Cited on page 10)

    Pith/arXiv arXiv 2021

  51. [51]

    When reasoning meets its laws

    Junyu Zhang, Yifan Sun, Tianang Leng, Jingyan Shen, Liu Ziyin, Paul Pu Liang, and Huan Zhang. When reasoning meets its laws. In NeurIPS 2025 Workshop on Efficient Reasoning , 2025. URL https:// openreview.net/forum?id=lWjcbodr4M. (Cited on page 11) Qualcomm AI Research is an initiative of Qualcomm T echnologies, Inc. 29

    2025

  52. [52]

    Chain of draft: Thinking faster by writing less

    Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600, 2025. URL https://arxiv.org/pdf/2502.18600. (Cited on page 11)

    Pith/arXiv arXiv 2025

  53. [53]

    The benefits of a concise chain of thought on problem-solving in large lan- guage models

    Matthew Renze and Erhan Guven. The benefits of a concise chain of thought on problem-solving in large lan- guage models. arXiv preprint arXiv:2401.05618, 2024. URL https://arxiv.org/pdf/2401.05618v1. pdf. (Cited on page 11)

    Pith/arXiv arXiv 2024

  54. [54]

    Think or not? selective reasoning via reinforcement learning for vision-language models

    Jiaqi Wang, Kevin Qinghong Lin, James Cheng, and Mike Zheng Shou. Think or not? selective reasoning via reinforcement learning for vision-language models. arXiv preprint arXiv:2505.16854, 2025. URL https: //arxiv.org/pdf/2505.16854. (Cited on page 11)

    arXiv 2025

  55. [55]

    Hapo: Training language models to reason concisely via history-aware policy optimization

    Chengyu Huang, Zhengxin Zhang, and Claire Cardie. Hapo: Training language models to reason concisely via history-aware policy optimization. arXiv preprint arXiv:2505.11225, 2025. (Cited on page 11)

    arXiv 2025

  56. [56]

    L1: Controlling how long a reasoning model thinks with reinforcement learning

    Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697, 2025. URL https://arxiv.org/pdf/2503.04697. (Cited on page 11)

    Pith/arXiv arXiv 2025

  57. [57]

    Dler: Doing length penalty right-incentivizing more intelligence per token via reinforcement learning

    Shih-Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang, Kwang- Ting Cheng, Yejin Choi, et al. Dler: Doing length penalty right-incentivizing more intelligence per token via reinforcement learning. arXiv preprint arXiv:2510.15110, 2025. (Cited on page 11)

    arXiv 2025

  58. [58]

    Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl

    Michael Luo, Sijun T an, Justin Wong, Xiaoxiang Shi, William Y T ang, Manan Roongta, Colin Cai, Jeffrey Luo, Tianjun Zhang, Li Erran Li, et al. Deepscaler: Surpassing o1-preview with a 1.5 b model by scaling rl. Notion Blog, 2025. (Cited on page 12)

    2025

  59. [59]

    TRL: Transformers Reinforcement Learning, 2020

    Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. TRL: Transformers Reinforcement Learning, 2020. URL https://github.com/huggingface/trl. (Cited on page 13)

    2020

  60. [60]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, T eddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In International Conference on Learning Representations, volume 2024, pages 39578–39601, 2024. (Cited on page 13, 16)

    2024

  61. [61]

    Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plap- pert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168, 2021. (Cited on page 16)

    Pith/arXiv arXiv 2021

  62. [62]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022. (Cited on page 16)

    Pith/arXiv arXiv 2022

  63. [63]

    Large language monkeys: Scaling inference compute with repeated sampling

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirho- seini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787, 2024. (Cited on page 16)

    Pith/arXiv arXiv 2024

  64. [64]

    Inference scaling laws: An empirical analysis of compute-optimal inference for llm problem-solving

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for llm problem-solving. In The Thirteenth International Conference on Learning Representations, 2025. (Cited on page 16, 17)

    2025

  65. [65]

    Scaling llm test-time compute optimally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling llm test-time compute optimally can be more effective than scaling model parameters. In The Thirteenth International Conference on Learning Rep- resentations, 2025. (Cited on page 16) Qualcomm AI Research is an initiative of Qualcomm T echnologies, Inc. 30

    2025

  66. [66]

    The effect of sampling temperature on problem solving in large language models

    Matthew Renze. The effect of sampling temperature on problem solving in large language models. In Find- ings of the association for computational linguistics: EMNLP 2024, pages 7346–7356, 2024. (Cited on page 16)

    2024

  67. [67]

    Group think: Multiple concurrent reasoning agents collaborating at token level granularity

    Chan-Jan Hsu, Davide Buffelli, Jamie McGowan, Feng- Ting Liao, Yi-Chang Chen, Sattar Vakili, and Da-shan Shiu. Group think: Multiple concurrent reasoning agents collaborating at token level granularity. arXiv preprint arXiv:2505.11107, 2025. (Cited on page 16, 24)

    Pith/arXiv arXiv 2025

  68. [68]

    Learning adaptive parallel reasoning with language models

    Jiayi Pan, Xiuyu Li, Long Lian, Charlie Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr. Learning adaptive parallel reasoning with language models. arXiv preprint arXiv:2504.15466, 2025. (Cited on page 16)

    Pith/arXiv arXiv 2025

  69. [69]

    Hogwild! inference: Parallel llm generation via concurrent attention

    Gleb Rodionov, Roman Garipov, Alina Shutova, George Yakushev, Erik Schultheis, Vage Egiazarian, Anton Sinitsin, Denis Kuznedelev, and Dan Alistarh. Hogwild! inference: Parallel llm generation via concurrent attention. arXiv preprint arXiv:2504.06261, 2025. (Cited on page 16, 24)

    arXiv 2025

  70. [70]

    Parallel-r1: T owards parallel thinking via reinforcement learning.arXiv preprint arXiv:2509.07980, 2025

    T ong Zheng, Hongming Zhang, Wenhao Yu, Xiaoyang Wang, Xinyu Yang, Runpeng Dai, Rui Liu, Huiwen Bao, Chengsong Huang, Heng Huang, et al. Parallel-r1: T owards parallel thinking via reinforcement learning.arXiv preprint arXiv:2509.07980, 2025. (Cited on page 16)

    Pith/arXiv arXiv 2025

  71. [71]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, T om Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information process- ing systems, 36:11809–11822, 2023. (Cited on page 16)

    2023

  72. [72]

    Skeleton-of-thought: Prompt- ing llms for efficient parallel generation

    Xuefei Ning, Zinan Lin, Zixuan Zhou, Zifu Wang, Huazhong Yang, and Yu Wang. Skeleton-of-thought: Prompt- ing llms for efficient parallel generation. arXiv preprint arXiv:2307.15337, 2023. (Cited on page 16)

    Pith/arXiv arXiv 2023

  73. [73]

    Enhancing reasoning through process supervision with monte carlo tree search

    Shuangtao Li, Shuaihao Dong, Kexin Luan, Xinhan Di, and Chaofan Ding. Enhancing reasoning through process supervision with monte carlo tree search. arXiv preprint arXiv:2501.01478, 2025. (Cited on page 16)

    Pith/arXiv arXiv 2025

  74. [74]

    Skywork-reward: Bag of tricks for reward modeling in llms

    Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. Skywork-reward: Bag of tricks for reward modeling in llms. arXiv preprint arXiv:2410.18451 , 2024. (Cited on page 16)

    Pith/arXiv arXiv 2024

  75. [75]

    Helpsteer 2: Open-source dataset for training top-performing reward mod- els, 2024

    Zhilin Wang, Yi Dong, Olivier Delalleau, Jiaqi Zeng, Gerald Shen, Daniel Egert, Jimmy J Zhang, Makesh N Sreedhar, and Oleksii Kuchaiev. Helpsteer 2: Open-source dataset for training top-performing reward mod- els, 2024. (Cited on page 16)

    2024

  76. [76]

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. arXiv preprint arXiv:2312.08935, 2023. (Cited on page 16)

    Pith/arXiv arXiv 2023

  77. [77]

    The lessons of developing process reward models in mathematical reasoning

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301, 2025. (Cited on page 16)

    Pith/arXiv arXiv 2025

  78. [78]

    Inference-time scaling for generalist reward modeling

    Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling. arXiv preprint arXiv:2504.02495, 2025. (Cited on page 16)

    arXiv 2025

  79. [79]

    Web-shepherd: Advancing prms for reinforcing web agents

    Hyungjoo Chae, Sunghwan Kim, Junhee Cho, Seungone Kim, Seungjun Moon, Gyeom Hwangbo, Dongha Lim, Minjin Kim, Yeonjun Hwang, Minju Gwak, et al. Web-shepherd: Advancing prms for reinforcing web agents. arXiv preprint arXiv:2505.15277, 2025. (Cited on page 16) Qualcomm AI Research is an initiative of Qualcomm T echnologies, Inc. 31

    arXiv 2025

  80. [80]

    Scaling autonomous agents via auto- matic reward modeling and planning

    Zhenfang Chen, Delin Chen, Rui Sun, Wenjun Liu, and Chuang Gan. Scaling autonomous agents via auto- matic reward modeling and planning. arXiv preprint arXiv:2502.12130, 2025. (Cited on page 16)

    Pith/arXiv arXiv 2025

Showing first 80 references.