CUSUM-Shaped Inference-Time Monitoring and Targeted Re-Decoding for Quantized Small Language Model Reasoning

Ayoub Belfatmi; El Hassane Ettifouri; Mahaman Sanoussi Yahaya Alassan; Walid Dahhane

REVIEW 2 major objections 6 minor 17 references

Inference-time monitoring and targeted rollback can recover some wrong answers in a quantized reasoning model, with the measured gain on fresh problems positive but uncertain.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 10:40 UTC pith:7WS6KQ6E

load-bearing objection A transparent, well-scoped empirical study of an inference-time monitor-and-repair loop; the headline accuracy gain is plausible but not statistically established, and the clean post-threshold audit rests on file timestamps rather than preregistration. the 2 major comments →

arxiv 2607.20129 v1 pith:7WS6KQ6E submitted 2026-07-22 cs.AI

CUSUM-Shaped Inference-Time Monitoring and Targeted Re-Decoding for Quantized Small Language Model Reasoning

El Hassane Ettifouri , Ayoub Belfatmi , Mahaman Sanoussi Yahaya Alassan , Walid Dahhane This is my paper

classification cs.AI

keywords MGT-BCUSUMinference-time monitoringtest-time backtrackingquantized reasoning modelsMATH-500chain-of-thoughtselective decoding

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that an external, weight-preserving controller, MGT-B, can watch a small quantized reasoning model as it generates and, when a trajectory looks unproductive, roll back to an earlier point and re-decode under constraints, improving final-answer accuracy. The key evidence is a chronology-aware audit: on 240 MATH-500 problems first seen only after the alarm threshold was chosen manually, accuracy goes from 82 to 88 correct (+2.50 points; 13 corrections vs 7 regressions), a direction that persists but is statistically uncertain. On a larger exploratory set of 467 pairs, the gain is +4.50 points with a small p-value, but it includes IDs exposed before threshold selection and is not independent. All no-alarm outputs are identical to vanilla, so the controller changes output only when it fires. The authors frame the contribution as evidence for a selective monitoring-and-repair mechanism in one specific setting, not a general or certified reasoning improvement.

Core claim

The central claim is that a monitoring-guided backtracking controller, MGT-B, can improve exact final-answer accuracy of a 4-bit 1.5B distilled reasoning model on MATH-500 by detecting unproductive trajectories during decoding and repairing them through rollback and constrained re-decoding. On the 240-pair chronology-audit set—constructed to include only problem IDs first observed after manual threshold selection—accuracy rises from 82/240 to 88/240 (+2.50 percentage points), with 13 corrections and 7 regressions. A broader 467-pair exploratory set shows 146/467 to 167/467 (+4.50 points), but is not treated as confirmatory because it includes IDs available before or during threshold choice.

What carries the argument

The central object is MGT-B, an external CUSUM-shaped controller. At each token it records pre-sampling entropy and chosen-token log-probability, forms six overlapping-window features (entropy, negative log-probability, repetition, confident repetition, positive and negative local entropy change), maps the score to a position-conditional empirical upper-tail probability, and accumulates mixture betting factors in a reset statistic S_j = max(0, S_{j-1}) + log e_j. An alarm fires when S_j crosses a threshold; the controller estimates a rollback point from reset history, restores token, key-value-cache, feature, and detector state, and re-decodes with lower temperature, a repetition penalty, an

Load-bearing premise

The load-bearing assumption is that the 240-pair chronology-audit genuinely separated problem IDs first observed after the manual threshold choice from IDs seen before, based on file timestamps and manifests; if unarchived tuning or timestamp gaps exist, those pairs are no longer post-threshold evidence and the primary audit loses its temporal claim.

What would settle it

Re-run the exact paired comparison with a threshold fixed in advance on separate development data and a disjoint set of held-out MATH-500 problems (or a second model); if the accuracy difference on first-observed post-threshold IDs is zero or negative, the central claim fails. Alternatively, inspect the archived file manifests and hashes: if any ID in the 240-pair audit appears in pre-threshold artifacts, or if threshold files were modified after evaluation, the chronology audit is invalidated.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

If the gain is real, inference-time monitoring can rescue some wrong answers without retraining or changing model weights.
No-alarm outputs are identical to vanilla, so a selective monitor can be applied safely: it only risks changing outputs when it fires.
On the exploratory comparison, monitoring-guided rollback outperforms random and periodic rollback under equal or slightly larger sampling budgets, implying the trigger schedule itself, not backtracking alone, carries value.
A confirmatory test would need a preregistered threshold chosen on separate development data and evaluated on a disjoint, untouched held-out problem set.
The controller adds inference-side cost through log-softmax/entropy computation and rollback re-decoding, but does not require extra model forward passes before an alarm.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If replicated, component ablations could separate whether the gain comes from the six-feature monitor, the CUSUM accumulation, the rollback point estimate, or the constrained re-decoding (temperature, repetition penalty, n-gram blocking).
A natural next experiment is to run the same controller on a full-precision or larger model: quantized reasoners are known to overproduce branch markers and loops, so the alarm may be especially useful under quantization, but the paper does not claim that.
A testable extension is comparing MGT-B against best-of-n sampling under the same total token budget; if selective re-decoding beats independent sampling, that would argue for adaptive compute allocation over fixed parallel paths.
The chronology audit could be stress-tested by reconstructing the file timeline from an independent archive of hashes and manifests; if any post-threshold first-observed ID appears in pre-threshold artifacts, the audit's main evidence weakens.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

A transparent, well-scoped empirical study of an inference-time monitor-and-repair loop; the headline accuracy gain is plausible but not statistically established, and the clean post-threshold audit rests on file timestamps rather than preregistration.

read the letter

The thing to know: this is an unusually honest paper about a plausible but unproven idea. The authors build a CUSUM-shaped controller that watches a 4-bit 1.5B model during MATH-500 decoding and, on alarm, rolls back and re-decodes with constraints. The primary 240-pair chronology audit shows 82/240 to 88/240 (+2.50 points, p=0.26), which is a consistent direction but not significance. The larger 467-pair set is +4.50 points with p=0.0007, but the paper itself says that set is contaminated by manual threshold selection. So the headline is weak, and they know it.

What is genuinely good: the negative result for centered token log-probability (Eq. 5) is clean and useful—a valid martingale that can be blind to confident repetition. The no-alarm identity property (all 316 no-alarm outputs are byte-identical to vanilla) is a real software-level guarantee and gives the intervention nice selectivity. The token accounting, separating emitted/sampled/deleted tokens, is the right way to frame cost. And the writing is candid: they explicitly disclaim anytime-valid guarantees, call the 467-pair p-value exploratory, and say the calibration is an in-sample fit.

Soft spots, in proportion. The central limitation is structural: the clean post-threshold audit depends on file timestamps and manifests, not a preregistered threshold. The stress-test note is right—if the chronology is wrong, only the contaminated 467-pair evidence remains. But this is not a hidden flaw; the paper says it in §5.2. Second, the controlled comparison against random/periodic rollback and self-correction is weaker than it looks: the control artifacts were lost to a storage reset, so those aggregate results cannot be regenerated from the released snapshot. That matters, though the stored numbers are internally consistent. Third, there are no component ablations, so we cannot tell if the gain comes from the features, the CUSUM reset, the rollback placement, or the re-decoding parameters. The paper acknowledges this too.

The bottom line: this is a serious, well-executed empirical pilot that does not overclaim. The direction of the effect is positive across two sets, but the clean evidence is not statistically significant and the post-threshold status is not airtight. It deserves referee time—mainly to push for a preregistered threshold, a preserved control pipeline, and ablations. I would take it to reading group because it is a good example of how to report inference-time control honestly, even if the headline is modest. I would not cite it for the accuracy gain yet; I would cite it for the careful methodology and the centered-log-probability negative result. Send it to peer review, but expect major revision toward stronger evidence rather than acceptance as-is.

Referee Report

2 major / 6 minor

Summary. The paper proposes MGT-B, an inference-time monitoring and rollback controller for a 4-bit quantized DeepSeek-R1-Distill-Qwen-1.5B model evaluated on MATH-500. The controller maps overlapping-window features over pre-sampling uncertainty and degeneration signals to position-conditional empirical tail probabilities, accumulates mixture betting factors with a CUSUM-shaped reset statistic, and on alarm estimates a rollback point, restores token/KV-cache/feature/detector state, and re-decodes with constrained decoding. The central reported result is a 240-pair chronology-audit set intended to contain only problem IDs first observed after the manual log threshold h=10 was written; accuracy changes from 82/240 to 88/240 (+2.50 percentage points; 13 corrections, 7 regressions; exact McNemar p=0.2632). A broader 467-pair historical-coverage set shows +4.50 percentage points (p=0.000753) but is explicitly labeled contaminated by threshold-selection exposure. The paper carefully states that neither analysis is confirmatory, that the empirical factors are not validated as an e-process or e-detector, and that the evidence supports only a selective monitoring-and-repair behavior for one model, one precision, and one task.

Significance. If the effect is real, MGT-B offers a training-free selective repair mechanism: no-alarm outputs remain byte-identical, alarmed trajectories sometimes recover a wrong vanilla answer, and the intervention is compared against budget-relaxed random/periodic rollback, restart, and self-correction controls. The paper's strengths are its transparency: it distinguishes the operational CUSUM-shaped statistic from a validated e-process, reports token accounting and paired uncertainty intervals, and explicitly discloses in-sample calibration, threshold-selection contamination, and the absence of component ablations. The empirical support is modest, however: the primary audit has p=0.2632 and a confidence interval including zero, its post-threshold status rests on unpreregistered file timestamps, and the control comparison's raw artifacts are unavailable. The contribution is best viewed as an exploratory methods paper with a clear evaluation protocol, not as established evidence of a robust accuracy improvement.

major comments (2)

[§5.2, §8] The central distinction between the 240-pair chronology-audit set and the contaminated 467-pair set is the claim that the 240 IDs were first observed after threshold_log10.json was created. The paper concedes that the chronology is reconstructed from file timestamps and manifests, and that this is neither a preregistration nor proof that no unarchived tuning occurred. This is load-bearing: the entire "new problems" inference depends on temporal separation. If the chronology is wrong, the 240-pair audit is just another threshold-exposed subset and the only remaining evidence is the 467-pair set, which the paper itself labels non-independent. No amount of honest caveat removes the structural dependence of the conclusion on an unverifiable temporal reconstruction. I would require either a prospectively fixed threshold (with a hashed threshold file created before any evaluation), or a re-fra
[§6.4 and Appendix E] The selectivity claim ("not explained by backtracking alone") rests on the random/periodic rollback, restart, and self-correction controls. Appendix E states that the implementations and raw artifacts for these controls are unavailable because the execution environment was lost after a storage reset; only aggregate results survive. Consequently, the control comparison cannot be independently regenerated or audited. This is a serious reproducibility gap for a load-bearing comparison. If the raw artifacts cannot be recovered, the paper should report the control comparison as non-reproducible exploratory evidence and state this limitation at the point of use in §6.4, not only in an appendix.

minor comments (6)

[§6.1] The phrase "the positive direction therefore persists" is stronger than p=0.2632 and the 95% CI [-1.25, +6.25] support; recommend "is directionally consistent" or "shows a positive but statistically uncertain direction."
[Table 4] The threshold ablation reports counts and accuracy but not a paired significance test. Since the rest of the paper uses exact McNemar tests, reporting the McNemar p for 59/200 vs 71/200 would make the table consistent and informative.
[§5.2] The auditability claim would be easier to verify if the paper included, in the main text or a provenance appendix, the artifact timestamps and SHA-256 hashes for threshold_log10.json and the main analysis outputs. Currently the reader must trust the manifests without seeing them.
[Table 1 caption] The caption says the analyses are retrospective, but it does not state that the paired bootstrap intervals omit threshold-selection uncertainty, seed variability, and potential unarchived adjustments. Adding this explicitly would prevent over-interpretation.
[§6.2] The byte-identical no-alarm property is reported for the 467-pair historical-coverage set but not for the 240-pair chronology-audit set, which is the primary set. If available, state the same property for the 240-pair set.
[Appendix E] The statement that the repository unit suite passes is useful, but specifying the exact test command and the Python/package environment would strengthen the reproducibility claim.

Circularity Check

0 steps flagged

No significant circularity: the primary chronology-audit result is an out-of-sample measurement, and the acknowledged in-sample calibration is not presented as a prediction.

full rationale

The paper's central empirical claim is the accuracy change on the 240-pair chronology-audit set, which is explicitly constructed to exclude the 260 IDs present in pre-threshold calibration/seed artifacts (§5.2, §6.1). The q_j calibration in Eq. (9) and the threshold selection in §4.5 use the same calibration trajectories, but the paper explicitly labels this an in-sample fit and does not treat the resulting alarm rate as a validated prediction. The accuracy gain on the disjoint audit set is not statistically forced: McNemar p=0.2632 and the bootstrap interval includes zero. The self-citation to the prior preprint (Anonymous, 2026) is not load-bearing because the negative result is re-derived in §4.1 and the paper states that no empirical result from the prior pilot is reused (§1, Appendix C). The chronology limitation (file timestamps vs. preregistration) is a reproducibility/validity threat, not a definitional circularity; no equation reduces to its own inputs, and no fitted parameter is renamed as a prediction.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 4 invented entities

The central claim rests on a hand-chosen threshold h=10, hand-set feature weights, and a calibration mapping q_j built from the same trajectories used to select the threshold; the paper explicitly flags these as in-sample. The empirical factors are not proven conditionally super-uniform, so no anytime-valid guarantee backs the reset statistic. The chronology audit assumes timestamps/manifests accurately separate pre/post-threshold IDs.

free parameters (5)

Manual log threshold h = 10 = 10 (log scale; original-scale equivalent 22026.466)
Chosen manually after observing automatic-threshold and development outcomes on 200 seed-1 IDs; the paper itself calls it an empirical operating point (§4.5, §7.1).
Feature score weights w = (0.15, 0.10, 0.20, 0.35, 0.18, 0.02)
Fixed configuration values rather than learned coefficients; hand-chosen, not justified by data and not ablated (§4.2, Appendix B).
Window length / stride = 64 / 32 tokens
Chosen; controls the feature granularity and statistical dependence of overlapping windows (§5.3, Appendix B).
Re-decode intervention parameters = temperature 0.6, repetition penalty 1.1, rollback margin 64 tokens, max rerolls 3
Hand-set intervention policy; part of composite repair that is not individually ablated (§5.3, Appendix B).
Beta exponent grid and probability clip = {0.1, 0.3, 0.5, 0.7}; clip 1e-6
Chosen betting construction parameters; no sensitivity analysis (§4.4, Appendix B).

axioms (4)

domain assumption A trajectory whose vanilla generation yields a correct, extractable final answer and does not exhaust its generation cap is 'healthy'.
Used to build the empirical reference distribution for q_j (§4.3); correct final answers can contain erroneous intermediate reasoning.
ad hoc to paper The empirical tail probability q_j can be treated as if it were approximately conditionally super-uniform under the null, so the reset statistic can serve as an operational alarm.
The paper explicitly states this premise is not established: overlapping dependent windows and finite reused pools break conditional validity; the statistic is not an e-process (§4.4, §8).
domain assumption Artifact timestamps and manifests reliably separate problem IDs first observed after the manual threshold choice from earlier exposures.
The chronology audit and its 240-pair claim rest on this; the paper cautions it is not a preregistration and cannot rule out unarchived adjustments (§5.2, §8).
ad hoc to paper The rollback point estimated from reset history identifies a recoverable prefix.
No temporal degradation labels or true changepoints exist to validate alarm localization; recovery after alarm only evidences the joint pipeline (§6.2, §8).

invented entities (4)

Position-conditional empirical upper-tail probability q_j no independent evidence
purpose: Maps the multi-feature window score to a position-bucket empirical tail probability used by the betting accumulation.
Defined by Eq. (9) over the same calibration trajectories that are used to select the threshold; no independent validation of conditional super-uniformity.
Mixture betting factor e_j no independent evidence
purpose: Incremental nonnegative factor accumulated in the reset statistic; structurally resembles an e-value but without the required conditional expectation bound.
Paper states 'We therefore call e_j a betting factor, not a validated incremental e-value' (§4.4).
CUSUM-shaped reset statistic S_j no independent evidence
purpose: Alarm trigger via S_j >= h; reduces influence of calm prefix.
Operational only; no Ville-type or average-run-length guarantee is claimed and the manual threshold is an empirical operating point (§4.4, §4.5).
Reset-history rollback point estimator no independent evidence
purpose: Chooses the token position to truncate to upon alarm.
Maps most recent nonpositive S_j to a window then extends by a fixed 64-token margin; no ground-truth changepoints to validate it (§4.6).

pith-pipeline@v1.3.0-alltime-deepseek · 14530 in / 15294 out tokens · 128197 ms · 2026-08-01T10:40:53.987089+00:00 · methodology

0 comments

read the original abstract

Quantized small autoregressive reasoning models can enter long, repetitive, or unproductive trajectories, yet inference-time compute is usually allocated without observing how a trajectory develops. Building on an earlier token-level e-CUSUM controller, we develop MGT-B (Monitoring-Guided Test-time Backtracking), a revised external controller that maps overlapping windows of pre-sampling uncertainty and degeneration features to position-conditional empirical tail probabilities, accumulates mixture betting factors with a CUSUM-shaped reset, and responds to an alarm by estimating a rollback point, restoring token and key-value-cache state, and performing constrained re-decoding. To audit whether the effect persists on problem identities first observed after the manual choice of log threshold h = 10, we retrospectively exclude 260 IDs present in pre-threshold artifacts and retain the chronologically first post-threshold pair for each remaining ID, yielding a 240-pair chronology-audit set. On this set, accuracy changes from 82/240 to 88/240 (+2.50 percentage points; 13 corrections, 7 regressions; exact McNemar p = 0.2632; paired bootstrap 95% interval [-1.25, +6.25]). A broader 467-pair historical-coverage set of seed-matched pairs changes accuracy from 146/467 to 167/467 (+4.50 points; McNemar p = 0.000753), but includes 200 seed-1 IDs available before or during threshold selection and is reported only as an exploratory estimate. All 316 no-alarm outputs in the 467-pair set are identical to vanilla, while the 151 alarmed trajectories contain 29 corrections and 8 regressions. Neither analysis is confirmatory, and the empirical factors are not established as a valid e-process or e-detector. The results support a selective monitoring-and-repair mechanism for the studied MATH-500 setting, rather than a general or theoretically certified reasoning improvement.

Figures

Figures reproduced from arXiv: 2607.20129 by Ayoub Belfatmi, El Hassane Ettifouri, Mahaman Sanoussi Yahaya Alassan, Walid Dahhane.

**Figure 2.** Figure 2: Representative reset-statistic traces. Window-update order is used because rollback makes [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗

discussion (0)

Reference graph

Works this paper leans on

17 extracted references · 1 canonical work pages

[1]

The unreasonable effectiveness of entropy minimization in LLM reasoning.arXiv preprint arXiv:2505.15134,

Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. The unreasonable effectiveness of entropy minimization in LLM reasoning.arXiv preprint arXiv:2505.15134,

Pith/arXiv arXiv
[4]

Fact-checking the output of large language models via token-level uncertainty quantification

Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, and Maxim Panov. Fact-checking the output of large language models via token-level uncertainty quantification. InFindings of the Association for Computational Linguistic...

2024
[6]

Mitigating overthinking in large reasoning language models via reasoning path deviation monitoring.arXiv preprint arXiv:2603.14251,

Weixin Guan, Liang Li, Jiapeng Liu, Bing Li, Peng Fu, Chengyang Fang, Xiaoshuai Hao, Can Ma, and Weiping Wang. Mitigating overthinking in large reasoning language models via reasoning path deviation monitoring.arXiv preprint arXiv:2603.14251,

arXiv
[8]

RuikangLiu, YuxuanSun, ManyiZhang, HaoliBai, XianzhiYu, TiezhengYu, ChunYuan, andLuHou

URLhttps://openreview.net/forum?id=v8L0pN6EOi. RuikangLiu, YuxuanSun, ManyiZhang, HaoliBai, XianzhiYu, TiezhengYu, ChunYuan, andLuHou. Quanti- zation hurts reasoning? an empirical study on quantized reasoning models.arXiv preprint arXiv:2504.04823,

Pith/arXiv arXiv
[9]

Quantized reasoning models think they need to think longer, but they do not.arXiv preprint arXiv:2606.00206,

Sanae Lotfi, Polina Kirichenko, Steven Li, and Zechun Liu. Quantized reasoning models think they need to think longer, but they do not.arXiv preprint arXiv:2606.00206,

Pith/arXiv arXiv
[10]

URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/ 91edff07232fb1b55a505a9e9f6c0ff3-Abstract-Conference.html. 14 E. S. Page. Continuous inspection schemes.Biometrika, 41(1/2):100–115,

2023
[13]

WATCH: Adaptive monitoring for AI deployments via weighted-conformal martingales.arXiv preprint arXiv:2505.04608,

Drew Prinster, Xing Han, Anqi Liu, and Suchi Saria. WATCH: Adaptive monitoring for AI deployments via weighted-conformal martingales.arXiv preprint arXiv:2505.04608,

Pith/arXiv arXiv
[14]

Glenn Shafer

doi: 10.1214/23-STS894. Glenn Shafer. Testing by betting: A strategy for statistical and scientific communication.Journal of the Royal Statistical Society: Series A, 184(2):407–431,

doi:10.1214/23-sts894
[16]

Ken Tsui

URL https://arxiv.org/abs/2408.03314. Ken Tsui. Self-correction bench: Uncovering and addressing the self-correction blind spot in large language models.arXiv preprint arXiv:2507.02778,

Pith/arXiv arXiv
[17]

Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu

URLhttps://openreview.net/forum?id=1PL1NIMMrw. Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Thoughts are all over the place: On the underthinking of o1-like LLMs.arXiv preprint arXiv:2501.18585,

Pith/arXiv arXiv
[18]

Step back to leap forward: Self-backtracking for boosting reasoning of language models.arXiv preprint arXiv:2502.04404,

Xiao-Wen Yang, Xuan-Yi Zhu, Wen-Da Wei, Ding-Chu Zhang, Jie-Jing Shao, Zhi Zhou, Lan-Zhe Guo, and Yu-Feng Li. Step back to leap forward: Self-backtracking for boosting reasoning of language models.arXiv preprint arXiv:2502.04404,

Pith/arXiv arXiv
[19]

Wenhong Zhu, Hongkun Hao, and Rui Wang

URL https://proceedings.neurips.cc/paper_files/paper/ 2023/hash/271db9922b8d1f4dd7aaef84ed5ac703-Abstract-Conference.html. Wenhong Zhu, Hongkun Hao, and Rui Wang. Penalty decoding: Well suppress the self-reinforcement effect in open-ended text generation. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,

2023
[100]

Adaptive conformal inference by betting.arXiv preprint arXiv:2412.19318,

Aleksandr Podkopaev, Darren Xu, and Kuang-Chih Lee. Adaptive conformal inference by betting.arXiv preprint arXiv:2412.19318,

Pith/arXiv arXiv
[1954]

doi: 10.1093/biomet/41.1-2

work page doi:10.1093/biomet/41.1-2
[2023]

Quantization meets reasoning: Exploring and mitigating degradation of low-bit LLMs in mathematical reasoning.arXiv preprint arXiv:2505.11574,

Zhen Li, Yupeng Su, Songmiao Wang, Runming Yang, Congkai Xie, Aofan Liu, Ming Li, Jiannong Cao, Yuan Xie, Ngai Wong, and Hongxia Yang. Quantization meets reasoning: Exploring and mitigating degradation of low-bit LLMs in mathematical reasoning.arXiv preprint arXiv:2505.11574,

arXiv
[2024]

URLhttps://aclanthology.org/2024.findings-acl.558/

doi: 10.18653/v1/2024.findings-acl.558. URLhttps://aclanthology.org/2024.findings-acl.558/. Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. Deep think with confidence.arXiv preprint arXiv:2508.15260,

Pith/arXiv arXiv 2024
[2025]

Zenghao Duan, Liang Pang, Zihao Wei, Wenbin Duan, Yuxin Tian, Shicheng Xu, Jingcheng Deng, Zhiyi Yin, and Xueqi Cheng

URLhttps://arxiv.org/abs/2501.12948. Zenghao Duan, Liang Pang, Zihao Wei, Wenbin Duan, Yuxin Tian, Shicheng Xu, Jingcheng Deng, Zhiyi Yin, and Xueqi Cheng. Circular reasoning: Understanding self-reinforcing loops in large reasoning models. arXiv preprint arXiv:2601.05693,

Pith/arXiv arXiv

[1] [1]

The unreasonable effectiveness of entropy minimization in LLM reasoning.arXiv preprint arXiv:2505.15134,

Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, and Hao Peng. The unreasonable effectiveness of entropy minimization in LLM reasoning.arXiv preprint arXiv:2505.15134,

Pith/arXiv arXiv

[2] [4]

Fact-checking the output of large language models via token-level uncertainty quantification

Ekaterina Fadeeva, Aleksandr Rubashevskii, Artem Shelmanov, Sergey Petrakov, Haonan Li, Hamdy Mubarak, Evgenii Tsymbalov, Gleb Kuzmin, Alexander Panchenko, Timothy Baldwin, Preslav Nakov, and Maxim Panov. Fact-checking the output of large language models via token-level uncertainty quantification. InFindings of the Association for Computational Linguistic...

2024

[3] [6]

Mitigating overthinking in large reasoning language models via reasoning path deviation monitoring.arXiv preprint arXiv:2603.14251,

Weixin Guan, Liang Li, Jiapeng Liu, Bing Li, Peng Fu, Chengyang Fang, Xiaoshuai Hao, Can Ma, and Weiping Wang. Mitigating overthinking in large reasoning language models via reasoning path deviation monitoring.arXiv preprint arXiv:2603.14251,

arXiv

[4] [8]

RuikangLiu, YuxuanSun, ManyiZhang, HaoliBai, XianzhiYu, TiezhengYu, ChunYuan, andLuHou

URLhttps://openreview.net/forum?id=v8L0pN6EOi. RuikangLiu, YuxuanSun, ManyiZhang, HaoliBai, XianzhiYu, TiezhengYu, ChunYuan, andLuHou. Quanti- zation hurts reasoning? an empirical study on quantized reasoning models.arXiv preprint arXiv:2504.04823,

Pith/arXiv arXiv

[5] [9]

Quantized reasoning models think they need to think longer, but they do not.arXiv preprint arXiv:2606.00206,

Sanae Lotfi, Polina Kirichenko, Steven Li, and Zechun Liu. Quantized reasoning models think they need to think longer, but they do not.arXiv preprint arXiv:2606.00206,

Pith/arXiv arXiv

[6] [10]

URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/ 91edff07232fb1b55a505a9e9f6c0ff3-Abstract-Conference.html. 14 E. S. Page. Continuous inspection schemes.Biometrika, 41(1/2):100–115,

2023

[7] [13]

WATCH: Adaptive monitoring for AI deployments via weighted-conformal martingales.arXiv preprint arXiv:2505.04608,

Drew Prinster, Xing Han, Anqi Liu, and Suchi Saria. WATCH: Adaptive monitoring for AI deployments via weighted-conformal martingales.arXiv preprint arXiv:2505.04608,

Pith/arXiv arXiv

[8] [14]

Glenn Shafer

doi: 10.1214/23-STS894. Glenn Shafer. Testing by betting: A strategy for statistical and scientific communication.Journal of the Royal Statistical Society: Series A, 184(2):407–431,

doi:10.1214/23-sts894

[9] [16]

Ken Tsui

URL https://arxiv.org/abs/2408.03314. Ken Tsui. Self-correction bench: Uncovering and addressing the self-correction blind spot in large language models.arXiv preprint arXiv:2507.02778,

Pith/arXiv arXiv

[10] [17]

Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu

URLhttps://openreview.net/forum?id=1PL1NIMMrw. Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Thoughts are all over the place: On the underthinking of o1-like LLMs.arXiv preprint arXiv:2501.18585,

Pith/arXiv arXiv

[11] [18]

Step back to leap forward: Self-backtracking for boosting reasoning of language models.arXiv preprint arXiv:2502.04404,

Xiao-Wen Yang, Xuan-Yi Zhu, Wen-Da Wei, Ding-Chu Zhang, Jie-Jing Shao, Zhi Zhou, Lan-Zhe Guo, and Yu-Feng Li. Step back to leap forward: Self-backtracking for boosting reasoning of language models.arXiv preprint arXiv:2502.04404,

Pith/arXiv arXiv

[12] [19]

Wenhong Zhu, Hongkun Hao, and Rui Wang

URL https://proceedings.neurips.cc/paper_files/paper/ 2023/hash/271db9922b8d1f4dd7aaef84ed5ac703-Abstract-Conference.html. Wenhong Zhu, Hongkun Hao, and Rui Wang. Penalty decoding: Well suppress the self-reinforcement effect in open-ended text generation. InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,

2023

[13] [100]

Adaptive conformal inference by betting.arXiv preprint arXiv:2412.19318,

Aleksandr Podkopaev, Darren Xu, and Kuang-Chih Lee. Adaptive conformal inference by betting.arXiv preprint arXiv:2412.19318,

Pith/arXiv arXiv

[14] [1954]

doi: 10.1093/biomet/41.1-2

work page doi:10.1093/biomet/41.1-2

[15] [2023]

Quantization meets reasoning: Exploring and mitigating degradation of low-bit LLMs in mathematical reasoning.arXiv preprint arXiv:2505.11574,

Zhen Li, Yupeng Su, Songmiao Wang, Runming Yang, Congkai Xie, Aofan Liu, Ming Li, Jiannong Cao, Yuan Xie, Ngai Wong, and Hongxia Yang. Quantization meets reasoning: Exploring and mitigating degradation of low-bit LLMs in mathematical reasoning.arXiv preprint arXiv:2505.11574,

arXiv

[16] [2024]

URLhttps://aclanthology.org/2024.findings-acl.558/

doi: 10.18653/v1/2024.findings-acl.558. URLhttps://aclanthology.org/2024.findings-acl.558/. Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. Deep think with confidence.arXiv preprint arXiv:2508.15260,

Pith/arXiv arXiv 2024

[17] [2025]

Zenghao Duan, Liang Pang, Zihao Wei, Wenbin Duan, Yuxin Tian, Shicheng Xu, Jingcheng Deng, Zhiyi Yin, and Xueqi Cheng

URLhttps://arxiv.org/abs/2501.12948. Zenghao Duan, Liang Pang, Zihao Wei, Wenbin Duan, Yuxin Tian, Shicheng Xu, Jingcheng Deng, Zhiyi Yin, and Xueqi Cheng. Circular reasoning: Understanding self-reinforcing loops in large reasoning models. arXiv preprint arXiv:2601.05693,

Pith/arXiv arXiv