Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Self-Regulation and Requesting Interventions

T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that an offline pipeline of random help rollouts, a process reward model, and tabular dynamic programming can train an LLM helper policy that requests interventions almost as effectively as an always-intervene policy…

desk verdict A useful offline recipe for learning when to ask for help, but the 'optimal helper' claim rests on a Bellman derivation that does not hold up. read the letter →

arxiv 2502.04576 v1 pith:ZLWDXU2X submitted 2025-02-07 cs.LG cs.CL

So Yeon Min , Yue Wu , Jimin Sun , Max Kaufmann , Fahim Tajwar , Yonatan Bisk , Ruslan Salakhutdinov This is my paper · ORCID
classification cs.LGcs.CL
keywords self-regulationrequestinginterventionsprocessrewardmodelstabularreinforcementlearningbudgetedassistanceLLMagentsSituatedInstructionFollowingofflineRL
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to give LLM-based agents a metacognitive ability: knowing when they cannot proceed and should call on an external intervention—a stronger model, extra search, or a human—without exhausting a limited budget. It argues that the right intervention schedule can be learned offline by combining a process reward model (a learned scorer of a state's success probability) with tabular dynamic programming over transition statistics collected in a single pass of random help calls. The output is a helper policy that decides help versus no-help at each state, and the paper shows on Situated Instruction Following tasks that this policy can nearly match the success rate of an always-intervene policy while using only a fraction of its interventions. If the claim holds, expensive intervention calls can be planned once in training and reused across different budget constraints.

What carries the argument

The load-bearing object is the usage/policy iteration equation. The value function is split as $V_s(r) = S_s - r M_s(r)$, where $S_s$ is expected discounted success and $M_s(r)$ is expected discounted help count; then the policy takes help at $s$ iff $r < \Delta p_s / \Delta M_s$, with $\Delta p_s = p^{\mathrm{help}}(s) - p^{\mathrm{nohelp}}(s)$ and $\Delta M_s$ defined as a success-probability-weighted combination of the help and no-help usage values. The PRM supplies $p(s)$, the transition table $\hat{P}(s'|s,a)$ is estimated from random offline rollouts, and the tabular iteration re-solves for each candidate cost $r$ without retraining the policy. This is what allows a single data-collection pass to serve multiple intervention budgets.

What would settle it

Build a small tabular MDP with known transition probabilities and success probabilities, enumerate every deterministic help/no-help policy to find the true best policy for a given cost $r$, and compare it with the policy produced by the paper's usage/policy iteration; any mismatch for a nonzero $r$ would refute the optimality claim. A cheaper version is to rerun the algorithm with the appendix's unweighted usage difference $\Delta M_s = M^{\mathrm{help}}_s - M^{\mathrm{nohelp}}_s$ and check whether the policies differ.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that budgeted help-seeking reduces to a ratio test on each state: ask for help exactly when the gain in success probability per unit of expected extra usage exceeds the reward cost $r$, i.e. when $r < \Delta p_s / \Delta M_s$. The method alternates between computing the expected discounted usage $M_s(r)$ under the current policy and updating the policy by this threshold, which the paper argues is equivalent to Bellman value iteration and converges to a unique fixed point. The converged policy for a chosen $r$ labels the offline trajectories, and a small model is fine-tuned to emulate those labels. This is what the authors mean by 'optimal helper behavior': in the harder task split their helper reaches 62.5% success with 1.0 intervention per task on average, close to the 67.5% success of the always-intervene policy that uses 7.8 interventions.

Load-bearing premise

The central claim depends on the dynamic programming rule for when asking for help is optimal being exactly right; the paper's own appendix derives that rule with a slightly different formula than the main algorithm uses, and if that formula is wrong the helper policy is not the optimal one the paper says it is.

Editorial extensions

If this is right

  • With a mid-range cost parameter on the difficult S obj split, the helper attains 62.5% success using an average of 1.0 intervention per task, close to the 67.5% success of the always-intervene policy that uses 7.8 interventions.
  • The DP's expected usage from the start state, $E[U] = M_{s_0}(r)$, tracks the observed usage well at small budgets, so the budget parameter can be selected from training data alone.
  • The same offline transition data can be reused across different cost parameters and budgets, because only the fast tabular DP is re-run for each $r$.
  • The ratio test extends to multiple intervention types by giving each intervention its own cost and budget and selecting the action with the smallest combined cost.
  • Training the helper on all DP-labeled states, rather than only on states reached on optimal trajectories, reduces the gap between realized and expected usage on unseen states.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The main theorem's threshold uses $\Delta M_s = p^{\mathrm{help}}(s)M^{\mathrm{help}}_s - p^{\mathrm{nohelp}}(s)M^{\mathrm{nohelp}}_s$, but the appendix's direct derivation of the help-versus-nohelp comparison yields an unweighted usage difference $M^{\mathrm{help}}_s - M^{\mathrm{nohelp}}_s$; if the weighted form is not the true Bellman comparison, the tabular policy may solve a different optimiz
  • The same offline recipe—PRM plus tabular DP plus supervised fine-tuning of a policy head—should transfer to any environment with a finite set of text states and expensive external interventions, provided a transition model can be estimated offline.
  • A direct way to test the threshold logic in isolation would be to construct a small synthetic MDP, enumerate all deterministic policies by brute force, and compare the paper's usage/policy iteration output with the true optimum for each $r$.
  • The paper's 'toggling' diagnosis implies that difficulty-only helpers will keep failing in sequential settings; the dynamic-programming awareness of state transitions, not the PRM score itself, is the active ingredient.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes an offline framework for training an LLM-based helper policy that decides when to request an intervention (e.g., a stronger model or MCTS) during task execution. The method collects transition data with randomly triggered interventions, learns PRMs for success probabilities, runs a tabular 'usage/policy iteration' DP to find a reward parameter r that meets an intervention budget, labels training states with the resulting policy, and fine-tunes a helper LLM via SFT. The authors claim the DP is equivalent to value iteration and that the helper delivers 'optimal' behavior, achieving near always-intervene success with far fewer interventions on Situated Instruction Following tasks.

Significance. The problem is timely and the empirical setup is extensive: the method consistently outperforms random and PRM-threshold baselines across task types and intervention types (Tables 2 and 3), the E[U] estimates align reasonably with observed usage, and the seen/unseen state analysis in Table 4 addresses a real coverage concern. If the optimality claim were supported, the offline, budget-adaptive approach would be a valuable contribution. However, the theoretical derivation underpinning that claim contains a load-bearing error, so the significance as stated is not established.

major comments (3)
  1. [Section 5.2 and Appendix C] The threshold condition pi(s)=help iff r < Delta_p_s / Delta_M_s is not a consequence of the Bellman recursion. Substituting V_s = S_s - r M_s into the Bellman comparison gives the condition r < [gamma * sum_{s'} (P_help(s'|s) - P_nohelp(s'|s)) S_{s'}] / [M_help_s - M_nohelp_s], where M_help_s = 1 + gamma * sum_{s'} P_help(s'|s) M_{s'} and M_nohelp_s = gamma * sum_{s'} P_nohelp(s'|s) M_{s'}. The numerator is a discounted success-value difference, not p_help(s)-p_nohelp(s), and the denominator is a usage difference, not the p-weighted expression p_help(s) M_help_s - p_nohelp(s) M_nohelp_s. Appendix C simply asserts the p-weighted expression after 'isolating the cost component'; the p weights never arise from the comparison. Therefore the boxed equation is not equivalent to value iteration for the reward regime of Section 5.1, and the abstract's claim of 'optimal helper behavior' is unsupported.
  2. [Section 5.3, Phase 2] The algorithm updates only M_s and the policy, never S_s. In a Bellman-optimal solution, the decision at s depends on future success values S_{s'} under the optimal continuation policy, which the PRM-based p_help(s) and p_nohelp(s) do not provide. The fixed point of this M-only iteration is a fixed point of a different operator, so E[U] = M_{s0}(r) and the implied pi*(r) are not shown to be the optimal usage and policy for the stated MDP. This also invalidates the convergence argument in Appendix D, which relies on the assumed equivalence to value iteration.
  3. [Section 5.2, definitions of p_help(s) and p_nohelp(s)] The quantities p_help(s) and p_nohelp(s) are static success probabilities estimated by the PRM, presumably under the base actor's behavior after an intervention, not under the optimal helper policy. The paper itself demonstrates in Section 4.3 that PRM-based state difficulty alone fails in multi-step settings because it ignores transition dynamics and policy effects. The DP derivation must explain how these PRM probabilities connect to the S-values appearing in the correct Bellman condition; no such connection is given.
minor comments (6)
  1. [Abstract] The sentence 'A key challenge is enabling agents with a limited intervention budget C is to decide when to request assistance' contains a grammatical error; consider revising.
  2. [Section 5.2 and Appendix C] The heading 'Derivaion' in Section 5.2 and the word 'defintiion' in Appendix C are typos.
  3. [Section 5.2] The notation M_help_s and M_nohelp_s is used in the threshold formula before being defined as the two branches of the M_s(r) recursion; please make the definitions explicit at first use.
  4. [Appendix C] The derivation divides by r when isolating the help branch, assuming r > 0. The case r = 0 is not handled, although it is a legitimate value for which the threshold condition must be amended.
  5. [Table 3 and Appendix B] The description of the state-wise PRM thresholding baseline ('calibrate thresholds for 20% and 50% of states and trigger each intervention randomly half of the times') is ambiguous; specify how the thresholds are selected and what 'randomly half of the times' means.
  6. [Table 5] The row label 'I(s[0:5→T)' is missing a closing bracket; it should read 'I(s[0:5]→T)'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DP policy is model-based, PRM labels are outcome-derived, and final success/usage are out-of-sample; the Sec. 5.2 threshold gap is a correctness risk, not a circular reduction.

full rationale

The paper's derivation chain is not circular. The PRM is trained on binary success/failure outcomes from base-actor rollouts, independent of the final test success metric. The tabular usage/policy iteration in Sec. 5.2 is intended as a Bellman-derived solver over the estimated transition model, and the helper is trained by SFT on the resulting labels; the reported test success rates (Table 2) and observed usage U are measured in the environment, not constructed from the fitted r or E[U] values. E[U] is computed from the estimated model and used to select r, but test U is an out-of-sample observation, so the paper's budget-compliance claim is a genuine empirical check. The self-citation of Min et al. (2025) supplies the SIF benchmark, an external and falsifiable task suite, so it is not load-bearing. Two non-circular weaknesses are worth noting: Appendix C's derivation of the threshold r < Δp_s/ΔM_s with p-weighted ΔM_s is not actually shown ('isolating the cost component −r yields...' skips the algebra), and Appendix D proves convergence by asserting equivalence with standard Bellman recursion; if the assertion fails, the 'optimal' claim is unsupported. These are correctness/proof gaps, not reductions of the prediction to the inputs. Because no quoted equation makes the reported success or usage equal by construction to a fitted value, the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim relies on the estimated PRM being a faithful p(s), the transition counts representing true dynamics, and the DP threshold condition being the correct optimality criterion. The first two are domain assumptions; the last is questionable given the Appendix C algebra.

free parameters (5)
  • reward scale r = r_high, r_mid, r_low (values not specified)
    Search parameter that controls intervention frequency to meet budget C; chosen per task and intervention type in Sec. 5.3 and Tables 2-3.
  • PRM self-regulation threshold = not given
    Threshold on max(1-PRM score) calibrated on a held-out validation set to maximize accuracy in Sec. 4.2.
  • MCTS UCT exploration constant c = 0.25
    Chosen for UCT in Appendix A; affects the MCTS intervention quality.
  • discount factor gamma = not stated
    Used in the DP recurrences in Sec. 5.2 and Appendix C; the value is not reported, making the exact optimization objective ambiguous.
  • intervention trigger probabilities for data collection = 0.0, 0.1, 0.3, 0.5, 0.7, 0.9, 1.0
    Hand-selected schedule for collecting transition data in Phase 1 (Sec. 5.3); the choice affects coverage of the estimated transition model.
assumptions (5)
  • standard math Bellman equations and value iteration converge to the optimal policy for the estimated MDP.
    Used throughout Sec. 5.2 and Appendix D; the convergence argument relies on standard value iteration properties.
  • domain assumption The PRM accurately estimates p(s), the probability of eventual success from state s.
    The DP uses PRM scores as p(s); if the PRM is biased the computed policy is suboptimal in the true environment. Stated in Sec. 4.1.
  • domain assumption The transition model estimated from finite rollouts (counts) approximates the true environment dynamics.
    Phase 1 estimates P(s'|s,a) from a limited number of random-intervention rollouts; coverage and sample size are not quantified.
  • domain assumption The helper policy trained by SFT on DP labels generalizes to test states.
    Phase 3 assumes the LLM helper can imitate the tabular policy on unseen states; Sec. 6.2 shows degradation on unseen states for trajectory-only training, so this assumption is only partially validated.
  • domain assumption The state representation is Markovian (text state contains all relevant information).
    The tabular DP assumes transitions depend only on current state and action, ignoring the agent's history or memory.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Self-Regulation and Requesting Interventions." pith.science (2026). https://pith.science/paper/ZLWDXU2X

@misc{pith2026250204576,
  author       = {Pith},
  title        = {Pith review of: Self-Regulation and Requesting Interventions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZLWDXU2X}},
  note         = {Machine review of arXiv:2502.04576}
}
abstract

Human intelligence involves metacognitive abilities like self-regulation, recognizing limitations, and seeking assistance only when needed. While LLM Agents excel in many domains, they often lack this awareness. Overconfident agents risk catastrophic failures, while those that seek help excessively hinder efficiency. A key challenge is enabling agents with a limited intervention budget $C$ is to decide when to request assistance. In this paper, we propose an offline framework that trains a "helper" policy to request interventions, such as more powerful models or test-time compute, by combining LLM-based process reward models (PRMs) with tabular reinforcement learning. Using state transitions collected offline, we score optimal intervention timing with PRMs and train the helper model on these labeled trajectories. This offline approach significantly reduces costly intervention calls during training. Furthermore, the integration of PRMs with tabular RL enhances robustness to off-policy data while avoiding the inefficiencies of deep RL. We empirically find that our method delivers optimal helper behavior.

Figures

Figures reproduced from arXiv: 2502.04576 by the authors.

Figure 1
Figure 1. Unreliable agents and training challenges. (a) An unre￾liable agent neither completes the assigned task nor communicates its inability, causing surprise and catastrophe. (b) When there is a budget C on interventions requested during inference, a key challenge is determining a reward function that guides the agent to request help appropriately. (c) For both behavior cloning/reinforce￾ment learning, obtaining an optim… view at source ↗
Figure 2
Figure 2. (a) A SIF task requires the agent to locate objects, interact with humans, and perform household tasks in a sequence of discrete actions. Assuming perfect visual perception, the relevant segment is highlighted in orange; states are represented in text. (b) A brief overview of Self-Regulation and Requesting Intervention, in comparison to the base agent. 3. Task and Setup Task We use the Situated Instruction Following… view at source ↗
Figure 3
Figure 3. p(s) measured by the PRM across the task. Interventions on PRM-chosen states (red line and stars) cause repeated toggling that traps the agent in low-p(s) regions, resulting in worse out￾comes than random interventions (blue line and stars), which ends at step 10 with task success. 2. However, I(st) consistently outperforms I(st → T). Un￾der similar intervention budgets, intervening selectively at difficult states c… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Method Overview. (a) We combine tabular state dynamics with a process reward model (PRM), implemented as a large language model, to perform offline tabular reinforcement learning. Our method consists of iterative usage/policy computation. πs = help denoted as πs = 1 fo…
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Seen vs. unseen states in the training data of the helper. The orange region highlights all states collected in Phase I (Sec. 5.3), each labeled with π ∗ (Nohelp or Help action). The green arrow illustrates π ∗ rollout from the initial state s0. Qualitative example [P…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Knowledge Augmented Complex Problem Solving with Large Language Models: A Survey

    cs.LG 2025-05 unverdicted novelty 2.0 of 10

    A survey that organizes LLM-based complex problem solving into three components, multi-step reasoning, domain knowledge, and result verification, and reviews methods and challenges across four application domains.

Reference graph

Works this paper leans on

23 extracted references · 16 canonical work pages · cited by 1 Pith paper

  1. [1]

    PRM-Thresholded: All steps

    PRM scores reliably identify tasks needing assistance Looking at Table 5 underTask-wise Intervention, we see that PRM-Thresholded usage (rows titled “PRM-Thresholded: All steps” or “PRM-Thresholded: First five steps”) fre- quently achieves higher or comparable SR relative to Ran- dom usage with similar or lower Usage. For example, under the Oracle columns...

    work page

  2. [2]

    State-wise interventions typically outperform task- wise methods. Comparing the Task-wise Intervention block (rows 8–18 in the table) with theState-wise Interven- tion block (rows 20–25), even random state-wise triggers often achieve higher SR for the same or lower usage. For instance, under A More Powerful Model for GPT4o mini (columns 6–8): • Task-wise ...

    work page

  3. [3]

    toggling

    PRM-based selection can underperform random se- lection due to transition dynamics (the “toggling” issue). Despite PRM-based triggers being more principled for pick- ing out tough states, Table 5 shows examples wherePRM- Thresholded with a low percentage budget does (much) worse than Random. Focusing on the A More Powerful Model columns for GPT4o mini (co...

    work page

  4. [4]

    Update counters:Mi s ← Mi s(πs)

    work page

  5. [5]

    Lin, Z., Trivedi, S., and Sun, J

    URL https://openreview.net/forum? id=v8L0pN6EOi. Lin, Z., Trivedi, S., and Sun, J. Generating with confi- dence: Uncertainty quantification for black-box large language models. Transactions on Machine Learn- ing Research, 2024. ISSN 2835-8856. URL https: //openreview.net/forum?id=DWkJCSxKU5. Liu, I.-J., Yuan, X., C ˆot´e, M.-A., Oudeyer, P.-Y ., and Schwi...

    work page 2024

  6. [6]

    Mallen, A., Asai, A., Zhong, V ., Das, R., Khashabi, D., and Hajishirzi, H

    URL https://openreview.net/forum? id=S37hOerQLB. Mallen, A., Asai, A., Zhong, V ., Das, R., Khashabi, D., and Hajishirzi, H. When not to trust language mod- els: Investigating effectiveness of parametric and non- parametric memories. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meet- ing of the Association for Com...

    arXiv 2023

  7. [9]

    Ruan, Y ., Dong, H., Wang, A., Pitis, S., Zhou, Y ., Ba, J., Dubois, Y ., Maddison, C

    URL https://openreview.net/forum? id=4ZK8ODNyFXx. Ruan, Y ., Dong, H., Wang, A., Pitis, S., Zhou, Y ., Ba, J., Dubois, Y ., Maddison, C. J., and Hashimoto, T. Identify- ing the risks of LM agents with an LM-emulated sandbox. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview. net/forum?id=GEcwtMk1uA. Schick, ...

    arXiv 2024

  8. [10]

    org/CorpusID:247613322

    URL https://api.semanticscholar. org/CorpusID:247613322. Xie, A., Tajwar, F., Sharma, A., and Finn, C. When to ask for help: Proactive interventions in autonomous re- inforcement learning. In Oh, A. H., Agarwal, A., Bel- grave, D., and Cho, K. (eds.), Advances in Neural In- formation Processing Systems , 2022. URL https: //openreview.net/forum?id=L9EXtg7h...

    work page 2022

Show all 23 references
  1. [11]

    Yang, J., Jimenez, C

    URL https://openreview.net/forum? id=gjeQKFxFpZ. Yang, J., Jimenez, C. E., Wettig, A., Lieret, K., Yao, S., Narasimhan, K. R., and Press, O. SWE-agent: Agent- computer interfaces enable automated software engi- neering. In The Thirty-eighth Annual Conference on Neural Informat...

    2024 arXiv

  2. [12]

    Zhai, Y ., Bai, H., Lin, Z., Pan, J., Tong, S., Zhou, Y ., Suhr, A., Xie, S., LeCun, Y ., Ma, Y ., and Levine, S

    URL https://openreview.net/forum? id=_3ELRdg2sgI. Zhai, Y ., Bai, H., Lin, Z., Pan, J., Tong, S., Zhou, Y ., Suhr, A., Xie, S., LeCun, Y ., Ma, Y ., and Levine, S. Fine-tuning large vision-language models as decision-making agents via reinforcement learning. In The Thirty-eigh...

  3. [13]

    Random %

    URL https://openreview.net/forum? id=nBjmMF2IZU. Zhou, K., Jurafsky, D., and Hashimoto, T. Navigating the grey area: How expressions of uncertainty and overconfi- dence affect language models. In Bouamor, H., Pino, J., and Bali, K. (eds.), Proceedings of the 2023 Conference on...

    2023 doi

  4. [17]

    13 Self-Regulation and Requesting Interventions Table 5

    if nohelp ats Ss−rMs(r) = γ ∑ s′ Pnohelp(s′|s) ( Ss′−rMs′(r) ) . 13 Self-Regulation and Requesting Interventions Table 5. Task-wise and state-wise interventions performances. Oracle A More Powerful Model MCTS GPT4o mini SFT-ed Llama GPT4o mini SFT-ed Llama GPT4o mini SFT-ed Ll...

  5. [18]

    having achieved success

    The help branch. Ss−rMs(r) = −r +γ ∑ s′ Phelp(s′|s) ( Ss′−rMs′(r) ) . Again using the piecewise defintiion of eq. 5 that γ ∑ s′ Phelp(s′|s)Sπ s′, we can isolate the usage terms: −rMs(r) = −r − rγ ∑ s′ Phelp(s′|s)Ms′(r), ⇐⇒ r ( 1−γ ∑ s′ Phelp(s′|s)Ms′(r) ) = rMs(r). Dividing th...

  6. [19]

    Compute usage for each action: Mi s(helpj) = { 1 +γ∑ s′Phelpj(s′|s)Mi s′, ifi =j, γ∑ s′Phelpj(s′|s)Mi s′, ifi̸=j, Mi s(nohelp) =γ ∑ s′ Pnohelp(s′|s)Mi s′

  7. [20]

    Compute ∆pi s and ∆Mi s, then check the ratio test ratioi(s)>ri

  8. [21]

    Policy update: πs = arg min a∈{help1,...,helpK,nohelp} { K∑ i=1 riMi s(a) } subject to ratioi >r i

  9. [23]

    Finally, we output the stable usage counters{Mi s}K i=1 and the final policyπ∗

    Check convergence: if maxs,i ⏐⏐Mi s− oldi s ⏐⏐<ε , stop. Finally, we output the stable usage counters{Mi s}K i=1 and the final policyπ∗. Phase 3: Final Policy Representation (SFT or Other). • We store the final help/nohelp decisions in a table π∗(s). • For statess in the train...

  10. [195]

    naacl-long.195/

    URL https://aclanthology.org/2024. naacl-long.195/. Qu, Y ., Zhang, T., Garg, N., and Kumar, A. Recursive in- trospection: Teaching language model agents how to self- improve. In The Thirty-eighth Annual Conference on Neu- ral Information Processing Systems, 2024. URL https: /...

    2024

  11. [2019]

    org/CorpusID:208527038

    URL https://api.semanticscholar. org/CorpusID:208527038. Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from human preferences. Advances in neural information pro- cessing systems, 30, 2017. D´ıaz-Rodr´ıguez, N., Del S...

    2017 arXiv

  12. [2021]

    org/CorpusID:239016703

    URL https://api.semanticscholar. org/CorpusID:239016703. Qiao, S., Gui, H., Lv, C., Jia, Q., Chen, H., and Zhang, N. Making language models better tool learners with execution feedback. In Duh, K., Gomez, H., and Bethard, S. (eds.), Proceedings of the 2024 Conference of the No...

    2024

  13. [2022]

    org/CorpusID:250451161

    URL https://api.semanticscholar. org/CorpusID:250451161. Kuhn, L., Gal, Y ., and Farquhar, S. Semantic uncer- tainty: Linguistic invariances for uncertainty estima- tion in natural language generation. In The Eleventh International Conference on Learning Representations,

  14. [2023]

    Lightman, H., Kosaraju, V ., Burda, Y ., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K

    URL https://openreview.net/forum? id=VD-AYtP0dve. Lightman, H., Kosaraju, V ., Burda, Y ., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step. In The Twelfth International Conference on Learning Representations,

  15. [2024]

    Jiang, Z., Araki, J., Ding, H., and Neubig, G

    URL https://openreview.net/forum? id=CVpuVe1N22. Jiang, Z., Araki, J., Ding, H., and Neubig, G. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962– 977, 2...

    2021 arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.