Pith. sign in

REVIEW 3 major objections 5 minor 200 references

A language model can localize the exact token where its reasoning first goes wrong by comparing its own token probabilities under real versus neutral feedback, then reuse the valid prefix and re-generate only the faulty suffix—achieving bet

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 07:23 UTC pith:UT7A7Q2J

load-bearing objection TTEL is an empirical win with an unproven mechanism: the token-level error localization claim is plausible but not backed by a proper control. the 3 major comments →

arxiv 2607.21453 v2 pith:UT7A7Q2J submitted 2026-07-23 cs.LG

Test-Time Scaling via Error Localization

Rajiv Shailesh Chitale , Rahul Madhavan , Taneesh Gupta , Deepanway Ghosal , Aravindan Raghuveer This is my paper
classification cs.LG
keywords test-time scalingerror localizationtoken-level credit assignmentinference-time searchprefix branchingprobability contrastpass@kLLM reasoning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to show that inference-time compute can be spent far more efficiently if a model knows where its reasoning went wrong, not merely that it failed. TTEL re-scores the failed trajectory under two contexts—real feedback and a neutral instruction—and treats the largest surviving probability drop as the error token. It truncates there and branches a new generation from the retained prefix. The authors report strictly dominating Pareto frontiers over independent sampling, multi-turn refinement, and recursive self-aggregation on LiveCodeBench, AIME-2025, and HMMT-2025; for Qwen3-8B on LiveCodeBench, pass@64 rises to 71.0% at 360.4k average tokens versus 64.6% at 735.0k tokens for independent sampling. If correct, it means token-level credit assignment is available for free at inference time, without training a verifier or updating weights.

Core claim

TTEL's central claim is that the divergence between a model's original token probabilities and its token probabilities after seeing failure feedback contains a usable localization signal: a sharp, feedback-specific drop marks the step where the trajectory first becomes inconsistent with a successful solution. To separate this signal from generic context-appendage effects, the paper subtracts a baseline spike obtained with non-diagnostic feedback, yielding a filtered spike score g_t, and branches at its argmax. The paper further proves a conditional identity: branching from the retained prefix beats restarting from scratch by exactly (1−p_a)(s_a−s_¬a), the probability that a fresh attempt wou

What carries the argument

The filtered spike score g_t = p_t^(B)(f∅) − p_t^(T)(f), the difference between the token's probability under a null-context baseline and under the true feedback-conditioned teacher. The student probability p_t^(S) recorded during generation is compared against both re-scored versions; the baseline subtraction removes probability shifts caused merely by appending text. TTEL selects t* = argmax g_t over positions passing thresholds, truncates the trajectory at t*−1, and branches a new generation from the retained prefix, storing prefixes in a search tree. Theorem 1 supplies the formal advantage of this anchoring: the gain over restarting is (1−p_a)(s_a−s_¬a), so the benefit is positive exactl

Load-bearing premise

The filtered spike score g_t is assumed to point at the token where reasoning first becomes inconsistent with a successful solution, but the paper never validates localization against ground-truth error positions—it only shows that branching at argmax g_t improves pass@k, and Theorem 1 assumes the chosen anchor has higher success probability than not anchoring.

What would settle it

Take a set of failed trajectories whose true error positions are known (human annotations, process reward model labels, or mutation-based ground truth), compute g_t, and check whether the argmax lands inside the annotated erroneous step far more often than chance. If the hit rate is near chance while pass@k gains persist, the gains are not caused by error localization; equivalently, a control that branches at a random retained prefix at the same token budget would match TTEL.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If TTEL is right, best-of-K sampling is provably wasteful: the same or better pass@k is attainable while generating roughly half the tokens, since valid prefixes are reused instead of discarded.
  • Token-level error localization works with generic self-feedback in domains without an execution environment (AIME, HMMT), so the mechanism is not contingent on verifier-rich settings; richer environment feedback (LiveCodeBench variants) improves it further.
  • The method is a pure inference-time intervention: no gradient updates, no trained process reward model, and no per-domain programmatic chunking, so it can wrap any model that can re-score its own tokens.
  • The branching advantage theorem implies a restart-based strategy is dominated whenever the model can identify any prefix that makes success more likely than not; repeated branching from multiple localized errors should compound this advantage.
  • The null-baseline filter is not a minor detail but load-bearing: removing it increases Turn-1 spike detections from 19.3 to 486.0 and degrades downstream pass@k to 0.592.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next test the paper does not run: compare argmax g_t against ground-truth error annotations (e.g., human-marked or step-verifier positions) on a sample of failed trajectories. If the branch point rarely aligns with the true error, the pass@k gains would need a different explanation—perhaps branching anywhere in a low-quality region helps.
  • The same probability-contrast signal that drives TTEL is already used offline in self-distillation objectives; the paper's result suggests the train-time and test-time uses could be unified, so a single feedback-conditioned scoring function could both improve a policy and guide its search.
  • Because the signal requires only log-probabilities over the model's own tokens, it should transfer to agentic and tool-use settings where feedback arrives as environment errors; a cheap addition would be to use the spike score as an early-exit or restart criterion inside existing agent loops.
  • The worst-case budget behavior deserves scrutiny: if a trajectory contains no surviving spike, TTEL restarts from scratch, so on problems where feedback is uninformative its token savings could vanish; an adaptively tuned threshold or spike-validity estimator might preserve the Pareto gain.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes TTEL, an inference-time search algorithm that tries to localize the token position where a failed reasoning trajectory first goes wrong. After a trajectory is generated and environment feedback is obtained, the model re-scores the same trajectory under the feedback and under a null context; the filtered spike score g_t = p_t^{(B)}(f_empty) - p_t^{(T)}(f) (Eq. 7) is thresholded to define E_filtered, and the branch point is the argmax of g_t over E_filtered (Alg. 1, lines 11-15). TTEL then truncates at that point and regenerates the suffix, reusing the prefix. The paper reports pass@k vs. generated-token Pareto curves for Qwen3-8B and Qwen3-4B-Thinking-2507 on LiveCodeBench, AIME-2025, and HMMT-2025, with ablations on the full reasoning trace, environment feedback, and the null-baseline filter. A theorem is given claiming a branching advantage over unanchored restart.

Significance. If the localization claim is correct, the paper makes a useful contribution: it repurposes the SDPO-style teacher/student log-probability contrast as an online, training-free search heuristic, and shows large token-efficiency gains over independent sampling and multi-turn refinement. The empirical reporting is careful in several respects: seed-level standard deviations are given for all main tables, the threshold sensitivity is swept in Appendix E, and the ablations in Figures 7-8 demonstrate that both the full reasoning trace and the null-feedback filter materially affect the method. However, the paper's signature claim that g_t identifies the actual error token is not directly tested. The experiments compare TTEL against baselines that differ in multiple axes simultaneously (prefix reuse, feedback-conditioned re-scoring, tree structure, context content), so the observed gains could in principle come from the search structure rather than from token-level credit assignment. The theorem is a conditional identity and does not supply the missing link. The paper is therefore publishable in principle, but the central mechanism needs a dedicated control experiment and, ideally, direct lo

major comments (3)
  1. [§4.1, Eq. (7), Algorithm 1 lines 13-15] The core claim is that the filtered spike score g_t localizes the token where reasoning first becomes inconsistent with a successful solution, and that branching at argmax_{t in E_filtered} g_t drives the reported Pareto gains. No experiment varies only the branch-point choice while holding the rest of the search fixed. The baselines differ from TTEL in at least three ways: they do not retain prefixes, do not re-score under feedback, and do not use a tree. The observed token-efficiency could therefore be explained by prefix reuse and increased continuation diversity alone, without g_t carrying genuine credit-assignment information. Appendix E sweeps the threshold tau, but that keeps the argmax rule; it does not test whether the position information matters. I request a control that branches at a random token from E_filtered (or at a fixed position) under the same prefix-reuse, same feedb
  2. [§6, Theorem 1] Theorem 1 does not close the gap left by the missing branch-point control. The proof shows Pr(Branch) - Pr(Restart) = (1 - p_a)(s_a - s_not-a) under the assumption s_a > s_not-a, and g_t appears nowhere in the statement. The condition s_a > s_not-a is exactly what has to be established for the specific anchor a selected by Eq. (9). As written, the theorem is a conditional identity about branching from any anchor, not a justification of TTEL's selection rule. It should be either strengthened by deriving conditions under which argmax g_t selects an anchor with s_a > s_not-a, or explicitly reframed as a framework statement, with the selection rule left to the empirical control requested above.
  3. [§7-8 and Appendix B] The TTEL vs. Multi-Turn comparison is confounded by context content. In Appendix B, Multi-Turn Refinement is given answer-only non-thinking responses for all previous turns, whereas TTEL's student prompt includes the full reasoning trace of the last attempt and the teacher re-scores that trace. Figure 7 shows that the full trace materially changes TTEL's results (0.630 vs. 0.597 for TTEL-GenFB). The paper does not report a Multi-Turn variant that also receives the full trace, so part of TTEL's advantage over Multi-Turn could be due to the richer context alone rather than to localization and prefix-anchored branching. The authors should match the context content across baselines or explicitly argue why the asymmetric setup is the intended comparison.
minor comments (5)
  1. [§7 and Appendix B] The main text states the spike threshold tau = 0.06 but not the value of the null-baseline threshold tau_B, which appears in Eq. (8). The value tau_B = 0.06 is only given in Appendix B. Please state both hyperparameters where the method is first defined.
  2. [Algorithm 1] At the first iteration, line 6 calls GENERATE with f initialized to empty. This is presumably 'no feedback', but the pseudocode should say explicitly that an empty feedback string is allowed, or should condition generation on the problem prompt only at the root.
  3. [§4.1 and Appendix D] Notation is inconsistent between the main text (Delta_t(f), Delta_t(f_empty), g_t) and the qualitative example (Delta^(T), Delta^(B)). Please unify the notation so the figures and appendix can be read against Eq. (7).
  4. [Appendix F] The definitions of 'spike validity', 'correction rate', and 'retention rate' are compressed and somewhat circular ('a spike is invalid if the answer from that turn is correct'). Please define these quantities precisely, including which rollout they are measured on, so the reader can interpret the depth analysis.
  5. [General] No code or reproducibility statement is provided. Even if release is not possible, a statement about availability and exact versioning of the Qwen models and LiveCodeBench subset would help.

Circularity Check

0 steps flagged

No load-bearing circularity: empirical evaluation is benchmark-based; Theorem 1 is a conditional identity rather than a circular derivation.

full rationale

TTEL's core empirical claims are evaluated against external benchmarks (LiveCodeBench, AIME-25, HMMT-25) using standard baselines, and the branch-point score g_t is a defined probability contrast (Eq. 7), not a parameter fitted to target answers. The filtered spike set and argmax selection are algorithmically specified and the reported pass@k values come from held-out benchmark performance, so the main results are not equivalent to the method's inputs by construction. The theory section does contain a conditional identity — Theorem 1 derives the branching advantage from the assumption s_a > s_¬a via the law of total probability — but the paper does not claim this theorem proves that g_t satisfies that inequality; it is a limitation in the theoretical justification rather than a circular empirical step. Self-citations (Gupta et al. 2025a,b; Srivastava et al. 2025) appear in related-work positioning only and are not load-bearing. The absence of a random-branch-point control is an experimental confound, not a definitional circularity. Overall, no specific reduction of a prediction to an input was found, so the circularity burden is low.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The method's contribution is mostly empirical. It relies on hand-set thresholds (τ, τ_B), a hand-designed null prompt, and the unproven premise that feedback-conditioned probability drops mark actual reasoning errors. The formal theorem does not remove this burden because it assumes the anchor is beneficial.

free parameters (4)
  • Spike threshold τ = 0.06
    Chosen from a sensitivity sweep on the evaluation set (LCBv6, Qwen3-8B, k=16, Appendix E); controls which tokens enter E_filtered.
  • Null-baseline threshold τ_B = 0.06
    Same as τ; used to reject tokens whose probability drop is also large under non-diagnostic feedback.
  • Max checkpoint depth D_c = 16,384 tokens
    Set equal to the per-attempt generation cap; restricts branching to prefixes shallower than this. Not swept.
  • Null-feedback instruction f∅ = 'Repeat the previous attempt word by word, but skip portions of redundant thinking'
    Hand-designed prompt acting as the non-diagnostic baseline; the validity of the whole filter depends on this instruction being truly non-diagnostic.
axioms (5)
  • domain assumption Autoregressive prefix consistency: freely generating a sequence that begins with a is distributionally equivalent to explicitly anchoring generation at a (Theorem 1).
    Invoked in Theorem 1; approximately true for standard autoregressive decoding, but the implementation changes the prompt format when reusing a prefix, so it may not hold exactly.
  • domain assumption Filtered spike g_t localizes the true erroneous token.
    Core premise in Sections 4.1-5; never validated against ground-truth error positions; only one qualitative example in Appendix D.
  • domain assumption The null feedback f∅ captures generic context-induced probability shifts so that subtraction isolates feedback-specific disagreement.
    Relies on the hand-written f∅ being non-diagnostic and representative; no independent check is provided.
  • domain assumption For the chosen anchor, success probability is higher than without it (s_a > s_¬a).
    Theorem 1 only proves the identity; the inequality is assumed for the anchor selected by argmax g_t.
  • domain assumption Environment or generic 'unsuccessful' feedback is sufficiently informative for error localization in math and code.
    Used in experiments; for math benchmarks there are no execution traces, only terminal failure states.

pith-pipeline@v1.3.0-alltime-deepseek · 27177 in / 13837 out tokens · 129890 ms · 2026-08-01T07:23:09.972282+00:00 · methodology

0 comments
read the original abstract

Scaling inference-time computation has emerged as a reliable method to improve the performance of large language models on complex reasoning and programming tasks. However, standard approaches such as independent sampling and sequential multi-turn refinement operate without token-level credit assignment, resulting in computational inefficiency, since valid reasoning prefixes are frequently discarded. In this work, we introduce Test-Time Scaling via Error Localization (TTEL), an inference-time algorithm that utilizes fixed or environment feedback to perform token-level error localization. By comparing conditional probabilities under informed feedback against a null-context baseline, TTEL isolates the step at which an error occurred. The algorithm then truncates the trajectory and branches a new generation, maximally reusing the valid prefix. Extensive evaluations demonstrate that TTEL establishes strictly dominating Pareto frontiers across sequential reasoning domains, measured by pass-at-k vs. generated-token cost. With Qwen3-8B on LiveCodeBench, TTEL attains a pass@64 of 71.0% while generating approximately half as many tokens as independent sampling (360.4k vs. 735.0k). Generalizing to math benchmarks AIME-2025 and HMMT-2025, TTEL cleanly outperforms competing test-time baselines across both Qwen3-8B and Qwen3-4B-Thinking-2507.

Figures

Figures reproduced from arXiv: 2607.21453 by Aravindan Raghuveer, Deepanway Ghosal, Rahul Madhavan, Rajiv Shailesh Chitale, Taneesh Gupta.

Figure 1
Figure 1. Figure 1: Search Tree Typology Comparison. (A) Independent Sampling generates entirely separate full-length trajectories, leading to high computational waste from parallel explo￾ration of similar paths. (B) Sequential Refinement methods link full trajectories end-to-end, evaluating only at terminal steps. (C) TTEL structures search as a prefix-sharing tree, explicitly capitalizing on token-level error localization t… view at source ↗
Figure 2
Figure 2. Figure 2: Token-Level Error Localization. TTEL filters raw feedback-induced probability drops (blue) against a non-diagnostic context baseline (orange) to distinguish genuine errors from con￾text artifacts. The maximal filtered spike (green) is selected as the precise branch point t ∗ 1 . Given a prompt x, the model first generates a candidate trajectory y = (y1, . . . , yN) under standard autore￾gressive decoding. … view at source ↗
Figure 3
Figure 3. Figure 3: TTEL Search and Branching Dynam￾ics. The reasoning trajectory iteratively evolves to isolate errors. At Iteration 1, the model detects a filtered error spike t ∗ 1 (red) and branches a new generation from the retained prefix y<t ∗ 1 (green). If a trajectory fails without a discernible spike (e.g., Iteration 3 where Efiltered = ∅), the tree search trig￾gers a restart from the root prompt. Once the localized… view at source ↗
Figure 4
Figure 4. Figure 4: Pareto plot of pass@k vs. average tokens generated per question on LiveCodeBench V6. 8 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Pareto plot of pass@k vs. average tokens generated per question on AIME-25. 50k 100k 200k 500k 1000k Average Cumulative Tokens (Log Scale) 0.40 0.45 0.50 0.55 0.60 0.65 Pass@k T=0 T=3 k=4 k=16 k=64 k=4 k=16 k=64 k=4 k=16 k=64 Qwen3-8B 50k 100k 200k 500k 1000k Average Cumulative Tokens (Log Scale) 0.2 0.3 0.4 0.5 0.6 0.7 Pass@k T=0 T=3 k=4 k=16 k=64 k=4 k=16 k=64 k=4 k=16 k=64 Qwen3-4B-Thinking-2507 TTEL Mu… view at source ↗
Figure 6
Figure 6. Figure 6: Pareto plot of pass@k vs. average tokens generated per question on HMMT-25. Generalization to Math. We further evaluate scaling on the AIME-25 and HMMT-25 benchmarks, where the environment only provides terminal failure states without interme￾diate diagnostic feedback [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The impact of context ablation on LiveCodeBench V6 for Qwen3-8B ( [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Null-baseline removal leads to massive false-positive spike detections and lowers [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative analysis of spike detection and correction for Question 21 in LCBv6. [PITH_FULL_IMAGE:figures/full_fig_p025_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Total spike count and average checkpoint depth for successive turn numbers [PITH_FULL_IMAGE:figures/full_fig_p026_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Spike count, validity of spikes and success rates based on Checkpoint Depth [PITH_FULL_IMAGE:figures/full_fig_p027_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

200 extracted references · 105 linked inside Pith

  1. [1]

    arXiv preprint arXiv:2308.05374 , year=

    Trustworthy llms: a survey and guideline for evaluating large language models' alignment , author=. arXiv preprint arXiv:2308.05374 , year=

    Pith/arXiv arXiv

  2. [2]

    arXiv preprint arXiv:2408.03314 , year=

    Scaling llm test-time compute optimally can be more effective than scaling model parameters , author=. arXiv preprint arXiv:2408.03314 , year=

    Pith/arXiv arXiv

  3. [3]

    arXiv preprint arXiv:2509.26626 , year=

    Recursive self-aggregation unlocks deep thinking in large language models , author=. arXiv preprint arXiv:2509.26626 , year=

    arXiv

  4. [4]

    Advances in neural information processing systems , volume=

    Tree of thoughts: Deliberate problem solving with large language models , author=. Advances in neural information processing systems , volume=

  5. [5]

    Advances in neural information processing systems , volume=

    Self-refine: Iterative refinement with self-feedback , author=. Advances in neural information processing systems , volume=

  6. [6]

    arXiv preprint arXiv:2403.07974 , year=

    Livecodebench: Holistic and contamination free evaluation of large language models for code , author=. arXiv preprint arXiv:2403.07974 , year=

    Pith/arXiv arXiv

  7. [7]

    arXiv preprint arXiv:2601.20802 , year=

    Reinforcement Learning via Self-Distillation , author=. arXiv preprint arXiv:2601.20802 , year=

    Pith/arXiv arXiv

  8. [8]

    Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou

    H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models , author=. The Eleventh International Conference on Learning Representations , volume=

  9. [9]

    arXiv preprint arXiv:2405.14734 , year=

    Simpo: Simple preference optimization with a reference-free reward , author=. arXiv preprint arXiv:2405.14734 , year=

    Pith/arXiv arXiv

  10. [10]

    arXiv preprint arXiv:2212.08073 , year=

    Constitutional ai: Harmlessness from ai feedback , author=. arXiv preprint arXiv:2212.08073 , year=

    Pith/arXiv arXiv

  11. [11]

    Econometrica , volume=

    The bargaining problem , author=. Econometrica , volume=

  12. [12]

    arXiv preprint arXiv:2601.05242 , year=

    GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization , author=. arXiv preprint arXiv:2601.05242 , year=

    Pith/arXiv arXiv

  13. [13]

    arXiv preprint arXiv:1606.06565 , year=

    Concrete problems in AI safety , author=. arXiv preprint arXiv:1606.06565 , year=

    Pith/arXiv arXiv

  14. [14]

    arXiv preprint arXiv:2503.14476 , year=

    Dapo: An open-source llm reinforcement learning system at scale , author=. arXiv preprint arXiv:2503.14476 , year=

    Pith/arXiv arXiv

  15. [15]

    Reinforce++: Stabilizing critic-free policy optimization with global advantage normalization, 2025 , author=

    2025

  16. [16]

    arXiv preprint arXiv:2505.12843 , year=

    Bias Fitting to Mitigate Length Bias of Reward Model in RLHF , author=. arXiv preprint arXiv:2505.12843 , year=

    Pith/arXiv arXiv

  17. [17]

    arXiv preprint arXiv:2402.14740 , year=

    Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms , author=. arXiv preprint arXiv:2402.14740 , year=

    Pith/arXiv arXiv

  18. [18]

    IEEE/ACM Transactions on networking , volume=

    Fair end-to-end window-based congestion control , author=. IEEE/ACM Transactions on networking , volume=. 2002 , publisher=

    2002

  19. [19]

    Advances in neural information processing systems , volume=

    Policy gradient for coherent risk measures , author=. Advances in neural information processing systems , volume=

  20. [20]

    arXiv preprint arXiv:2103.09568 , year=

    A practical guide to multi-objective reinforcement learning and planning , author=. arXiv preprint arXiv:2103.09568 , year=

    Pith/arXiv arXiv

  21. [21]

    arXiv preprint arXiv:2310.10076 , year=

    Verbosity bias in preference labeling by large language models , author=. arXiv preprint arXiv:2310.10076 , year=

    Pith/arXiv arXiv

  22. [22]

    Advances in neural information processing systems , volume=

    Judging llm-as-a-judge with mt-bench and chatbot arena , author=. Advances in neural information processing systems , volume=

  23. [23]

    arXiv preprint arXiv:2409.13156 , year=

    Rrm: Robust reward model training mitigates reward hacking , author=. arXiv preprint arXiv:2409.13156 , year=

    Pith/arXiv arXiv

  24. [24]

    arXiv preprint arXiv:2407.01085 , year=

    Explaining length bias in llm-based preference evaluations , author=. arXiv preprint arXiv:2407.01085 , year=

    Pith/arXiv arXiv

  25. [25]

    arXiv preprint arXiv:2506.16507 , year=

    Robust Reward Modeling via Causal Rubrics , author=. arXiv preprint arXiv:2506.16507 , year=

    Pith/arXiv arXiv

  26. [26]

    arXiv preprint arXiv:2511.12573 , year=

    Mitigating Length Bias in RLHF through a Causal Lens , author=. arXiv preprint arXiv:2511.12573 , year=

    arXiv

  27. [27]

    Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    Loose lips sink ships: Mitigating length bias in reinforcement learning from human feedback , author=. Findings of the Association for Computational Linguistics: EMNLP 2023 , pages=

    2023

  28. [28]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Deep reinforcement learning that matters , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  29. [29]

    arXiv preprint arXiv:2504.12501 , year=

    Reinforcement learning from human feedback , author=. arXiv preprint arXiv:2504.12501 , year=

    Pith/arXiv arXiv

  30. [30]

    Multi-preference optimization: Generalizing dpo via set-level contrasts, 2025 , year=

    Gupta, Taneesh and Madhavan, Rahul and Zhang, Xuchao and Natarajan, Nagarajan and Bansal, Chetan and Rajmohan, Saravan , journal=. Multi-preference optimization: Generalizing dpo via set-level contrasts, 2025 , year=

    2025

  31. [31]

    arXiv preprint arXiv:2502.18293 , year=

    AMPO: Active Multi-Preference Optimization , author=. arXiv preprint arXiv:2502.18293 , year=

    Pith/arXiv arXiv

  32. [32]

    On the generalized distance in statistics , author=. Sankhy. 2018 , publisher=

    2018

  33. [33]

    Econometrica: Journal of the Econometric Society , pages=

    Other solutions to Nash's bargaining problem , author=. Econometrica: Journal of the Econometric Society , pages=. 1975 , publisher=

    1975

  34. [34]

    Resonance , volume=

    Mahalanobis distance , author=. Resonance , volume=

  35. [35]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    Pith/arXiv arXiv

  36. [36]

    2017 , publisher=

    Collective choice and social welfare: Expanded edition , author=. 2017 , publisher=

    2017

  37. [37]

    International conference on machine learning , pages=

    Trust region policy optimization , author=. International conference on machine learning , pages=. 2015 , organization=

    2015

  38. [38]

    Proceedings of the AAAI Conference on Artificial Intelligence , volume=

    Preference ranking optimization for human alignment , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

  39. [39]

    arXiv preprint arXiv:2403.19270 , year=

    sDPO: Don't Use Your Data All at Once , author=. arXiv preprint arXiv:2403.19270 , year=

    Pith/arXiv arXiv

  40. [40]

    arXiv preprint arXiv:2309.06657 , year=

    Statistical rejection sampling improves preference optimization , author=. arXiv preprint arXiv:2309.06657 , year=

    Pith/arXiv arXiv

  41. [41]

    Advances in Neural Information Processing Systems , volume=

    Fine-grained human feedback gives better rewards for language model training , author=. Advances in Neural Information Processing Systems , volume=

  42. [42]

    International Conference on Machine Learning , pages=

    Scaling laws for reward model overoptimization , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  43. [43]

    Advances in Neural Information Processing Systems , volume=

    Large language model as attributed training data generator: A tale of diversity and bias , author=. Advances in Neural Information Processing Systems , volume=

  44. [44]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    Pith/arXiv arXiv

  45. [45]

    arXiv preprint arXiv:2410.04350 , year=

    TIS-DPO: Token-level Importance Sampling for Direct Preference Optimization With Estimated Weights , author=. arXiv preprint arXiv:2410.04350 , year=

    Pith/arXiv arXiv

  46. [46]

    arXiv preprint arXiv:2211.04486 , year=

    Active example selection for in-context learning , author=. arXiv preprint arXiv:2211.04486 , year=

    Pith/arXiv arXiv

  47. [47]

    arXiv preprint arXiv:1708.00489 , year=

    Active learning for convolutional neural networks: A core-set approach , author=. arXiv preprint arXiv:1708.00489 , year=

    Pith/arXiv arXiv

  48. [48]

    nature , volume=

    Mastering the game of Go with deep neural networks and tree search , author=. nature , volume=. 2016 , publisher=

    2016

  49. [49]

    nature , volume=

    Mastering the game of go without human knowledge , author=. nature , volume=. 2017 , publisher=

    2017

  50. [50]

    2004 , publisher=

    Multidimensional knapsack problems , author=. 2004 , publisher=

    2004

  51. [51]

    Knapsack problems , pages=

    Introduction to NP-Completeness of knapsack problems , author=. Knapsack problems , pages=. 2004 , publisher=

    2004

  52. [52]

    Part II: Multiple, multidimensional, and quadratic knapsack problems , author=

    Knapsack problems—An overview of recent advances. Part II: Multiple, multidimensional, and quadratic knapsack problems , author=. Computers & Operations Research , volume=. 2022 , publisher=

    2022

  53. [53]

    Sampling techniques for supervised or unsupervised tasks , pages=

    Core-sets: Updated survey , author=. Sampling techniques for supervised or unsupervised tasks , pages=. 2020 , publisher=

    2020

  54. [54]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

    Deep metric learning via facility location , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

  55. [55]

    1983 , institution=

    The uncapicitated facility location problem , author=. 1983 , institution=

    1983

  56. [56]

    Proceedings of the statistical data analysis based on the L1 norm conference, neuchatel, switzerland , volume=

    Clustering by means of medoids , author=. Proceedings of the statistical data analysis based on the L1 norm conference, neuchatel, switzerland , volume=

  57. [57]

    2009 , publisher=

    Active learning literature survey , author=. 2009 , publisher=

    2009

  58. [58]

    Advances in neural information processing systems , volume=

    Big self-supervised models are strong semi-supervised learners , author=. Advances in neural information processing systems , volume=

  59. [59]

    arXiv preprint arXiv:2401.01335 , year=

    Self-play fine-tuning converts weak language models to strong language models , author=. arXiv preprint arXiv:2401.01335 , year=

    Pith/arXiv arXiv

  60. [60]

    Advances in neural information processing systems , volume=

    Coresets for clustering with fairness constraints , author=. Advances in neural information processing systems , volume=

  61. [61]

    The 5th Workshop on Mathematical Reasoning and AI at NeurIPS 2025 , year=

    Adaptive Control for Test-time Scaling , author=. The 5th Workshop on Mathematical Reasoning and AI at NeurIPS 2025 , year=

    2025

  62. [62]

    arXiv preprint arXiv:2412.21187 , year=

    Do not think that much for 2+ 3=? on the overthinking of o1-like llms , author=. arXiv preprint arXiv:2412.21187 , year=

    Pith/arXiv arXiv

  63. [63]

    arXiv preprint arXiv:2507.06261 , year=

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=

    Pith/arXiv arXiv

  64. [64]

    arXiv preprint arXiv:2406.06592 , year=

    Improve mathematical reasoning in language models by automated process supervision , author=. arXiv preprint arXiv:2406.06592 , year=

    Pith/arXiv arXiv

  65. [65]

    Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    CARMO: Dynamic Criteria Generation for Context Aware Reward Modelling , author=. Findings of the Association for Computational Linguistics: ACL 2025 , pages=

    2025

  66. [66]

    Advances in Neural Information Processing Systems , volume=

    Star: Bootstrapping reasoning with reasoning , author=. Advances in Neural Information Processing Systems , volume=

  67. [67]

    arXiv preprint arXiv:2402.06457 , year=

    V-star: Training verifiers for self-taught reasoners , author=. arXiv preprint arXiv:2402.06457 , year=

    Pith/arXiv arXiv

  68. [68]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Math-shepherd: Verify and reinforce llms step-by-step without human annotations , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  69. [69]

    arXiv preprint arXiv:2211.14275 , year=

    Solving math word problems with process-and outcome-based feedback , author=. arXiv preprint arXiv:2211.14275 , year=

    Pith/arXiv arXiv

  70. [70]

    arXiv preprint arXiv:2211.12588 , year=

    Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks , author=. arXiv preprint arXiv:2211.12588 , year=

    Pith/arXiv arXiv

  71. [71]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Active prompting with chain-of-thought for large language models , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  72. [72]

    International conference on machine learning , pages=

    Pal: Program-aided language models , author=. International conference on machine learning , pages=. 2023 , organization=

    2023

  73. [73]

    arXiv preprint arXiv:2210.00720 , year=

    Complexity-based prompting for multi-step reasoning , author=. arXiv preprint arXiv:2210.00720 , year=

    Pith/arXiv arXiv

  74. [74]

    arXiv preprint arXiv:2408.00724 , year=

    Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models , author=. arXiv preprint arXiv:2408.00724 , year=

    Pith/arXiv arXiv

  75. [75]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  76. [76]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    s1: Simple test-time scaling , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pages=

    2025

  77. [77]

    The twelfth international conference on learning representations , year=

    Let's verify step by step , author=. The twelfth international conference on learning representations , year=

  78. [78]

    arXiv preprint arXiv:2407.21787 , year=

    Large language monkeys: Scaling inference compute with repeated sampling , author=. arXiv preprint arXiv:2407.21787 , year=

    Pith/arXiv arXiv

  79. [79]

    arXiv preprint arXiv:2601.18734 , year=

    Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models , author=. arXiv preprint arXiv:2601.18734 , year=

    Pith/arXiv arXiv

  80. [80]

    arXiv preprint arXiv:2310.04406 , year=

    Language agent tree search unifies reasoning acting and planning in language models , author=. arXiv preprint arXiv:2310.04406 , year=

    Pith/arXiv arXiv

Showing first 80 references.