Pith. sign in

REVIEW 2 major objections 5 minor 79 references

Personalizing Large Language Model Agents with Small Policy Models

T0 review · 2 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A frozen LLM agent can be personalized per user by learning only a residual preference vector over factorized execution choices, and the paper proves this learning achieves O~(d^{3/2}sqrt(n)) regret against the best feasible action.

desk verdict FABLE is a clean, honest paper with a genuine regret bound, but its central empirical claim—online adaptation personalizes a frozen agent—is not yet isolated from promotion and prespecified default/cost scoring, and all users are synthetic. read the letter →

arxiv 2608.00215 v1 pith:MIHXHUL4 submitted 2026-07-31 cs.AI

Dian Jin , Zhi Zhang , Huichao Li , Yihe Pan , Rundong Huang , Doudou Zhou This is my paper
classification cs.AI
keywords LLMagentspersonalizationcontextualbanditsThompsonsamplingfactorizedactionspaceregretboundfeasibilityconstraintsBayesianposterior
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes FABLE, a small policy layer that sits outside a frozen, potentially black-box LLM agent and learns, per user, how the agent should execute tasks: whether to consult memory, call tools, ask clarifying questions, and how to phrase answers. The key move is to treat each interaction as one bandit round with scalar feedback only for the executed action, and to model expected feedback as a fixed default-and-cost score plus a user-specific residual that is linear in a factorized feature vector. This lets a single feedback signal update all related action components, and lets stated preferences from onboarding enter as revisable prior evidence. The paper proves that with a calibrated Gaussian sampling scale, FABLE achieves high-probability regret O~(d^{3/2}sqrt(n)) against the best feasible action per round, and gives an anytime-valid rule for promoting learned preferences into persistent state.

What carries the argument

The central object is the factorized residual preference model: a fixed feature map phi(c,a) with main effects for memory, tool, and response modes plus pairwise interactions, combined with a known default-and-cost baseline b(c,a) - lambda cost(c,a). The learner maintains a Gaussian posterior over the residual parameter theta*, centered by revisable onboarding pseudo-observations in the same coordinates, and selects actions by Thompson sampling from theta_t ~ N(mu_t, nu_t^2 Sigma_t), then maximizing the total score within the feasible set A_t. This residualization means feedback is first centered against the frozen system's reference behavior, and the factorization means one scalar outcome u

What would settle it

Pick a user whose true reward is a non-linear function of the factorized features, e.g., reward = 1 if phi(a)^T theta* > 0 else -1, run FABLE for n rounds, and measure regret against the best feasible action: if average regret fails to decay at the predicted sqrt(n)-rate and instead stays linear, the linear residual-reward model that Theorem 1 relies on is falsified.

Watch

Extended reading notes

Core claim

FABLE learns a per-user execution policy over the joint action space A = M x T x S (memory mode, information-acquisition mode, response behavior) without modifying the host LLM. Expected feedback is modeled as E[y|c,a] = b(c,a) + phi(c,a)^T theta* - lambda cost(c,a), where b and cost are prespecified and only theta* is learned. Using Bayesian Thompson sampling on theta* with sampling scale nu_t = sqrt(9d log(t/delta)) and restricting sampling to an externally supplied feasible set A_t, the paper proves Theorem 1: with probability 1-delta, regret against the best feasible action is O~(d^{3/2}sqrt(n)), and expected regret is the same order when delta = n^{-2}. It further characterizes exactly

Load-bearing premise

The guarantees and the method's usefulness rest on the assumption that a user's expected feedback is exactly a prespecified default-and-cost score plus a linear function of the factorized action features; if real users' preferences are not linear in these features, the regret bound no longer describes actual behavior.

Editorial extensions

If this is right

  • Proprietary or black-box LLM agents can be personalized without fine-tuning, gradients, or hidden-state access; only the policy layer and interaction history change.
  • Sparse, selected-action feedback becomes usable: a single like/dislike signal can update memory, tool, and response-style preferences together instead of being wasted on one flat arm.
  • Hard constraints (permissions, tool availability, mandatory confirmation) are enforced before exploration, so learning never attempts a prohibited action; the regret comparator is the best feasible action, not an unconstrained oracle.
  • Onboarding preferences are treated as finite-precision prior evidence, so they can be corrected by later behavior rather than locked in.
  • Learned preferences can be written back to persistent agent state only when an anytime-valid confidence sequence excludes zero, giving a formal guarantee against false promotion of a preference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The identifiability result implies a practical boundary: for a user who persistently disables memory, the agent cannot learn that user's memory preferences from behavior alone; any claimed memory-personalization for such users must come from other signals.
  • The same factorized-residual recipe could be applied to other frozen-system decisions beyond agent execution, e.g., summarization length, notification frequency, or UI density, wherever scalar feedback is available and actions share components.
  • The regret bound depends on the prespecified default score being a good baseline; one testable extension is to let b and cost themselves be learned slowly at a meta-level across users, while keeping the per-user residual layer intact.
  • Because the theory and experiments are built on synthetic profiles and hand-tuned baselines, the most valuable next check is real-user deployment where the linear residual model can be tested directly against behavior; if real feedback is substantially non-linear in the features, the guarantee no longer applies.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes FABLE, a per-user policy layer for a frozen, potentially black-box LLM agent. It models each execution decision as a factorized action (memory mode, information-acquisition mode, response style), subtracts a prespecified default-and-cost score, and learns a user-specific residual preference via Bayesian contextual Thompson sampling, with exploration restricted to an externally supplied feasible set. The central theoretical result is a high-probability and expected regret bound of Õ(d^{3/2}√n) against the best feasible action under a linear residual-reward model (Theorem 1), together with identifiability characterizations for inactive feature directions and an anytime-valid false-promotion control rule. Experiments on tau2-bench, PAHF, and Math500 compare FABLE against rule-only and LinTS baselines and ablations. The paper is transparent about several limitations, including synthetic users, three-seed variation, and the lack of a statistically certified task-success improvement.

Significance. If the theoretical results are correct, FABLE provides a useful and auditable template for lightweight personalization of frozen agents. The regret proof is a careful adaptation of Agrawal–Goyal with a known offset and a general prior center; the identifiability propositions and the promotion confidence-sequence control are clean and self-contained. The experiments use cluster-bootstrap confidence intervals and report explicit negative results, which is good practice. However, the empirical support for the central online-adaptation mechanism is weaker than the abstract suggests: the one positive tau2 contrast bundles posterior updating with promotion, while the update-only no-promotion arm is numerically worse than frozen. The theory is sound relative to the stated model, but the benchmark reward is constructed using the same cost table the algorithm treats as a known offset, so the experiments do not test behavior under model misspecification or real-user heterogeneity.

major comments (2)
  1. [§5.1, Table 1; Appendix C.1] The main evidence for 'online adaptation' is the paired contrast FABLE (full) minus FABLE (frozen): +0.024 alignment with 95% CI [0.003, 0.045]. This contrast bundles posterior updating with promotion. The FABLE (no promotion) arm still performs online updates yet has alignment 0.686, below FABLE (frozen)'s 0.693, and full minus no-promotion is +0.031 [0.012, 0.048]. Thus the data do not support an update-only benefit; if anything, posterior updating alone is numerically harmful on this metric and the measured benefit is attributable to the promotion writeback. The abstract's claim that matched comparisons support 'online adaptation' and the main-text sentence 'Relative to FABLE (frozen), online adaptation improves alignment by 0.024' should be revised to acknowledge the bundled contrast and the contradictory no-promotion result.
  2. [§3.2, §5.1; Appendix F] The empirical evaluation is conducted entirely under the paper's own linear residual model. The tau2 evaluation reward y_t^eval is constructed with the same prespecified cost table c(a_t) that FABLE uses as a known offset in Eq. (1), and every user is a synthetic profile (two verbosity profiles on tau2, one rubric user on Math500, 20 personas on PAHF). Consequently, the experiments show that the algorithm works when Eq. (1) holds by construction, but do not probe robustness to misspecification or real-user heterogeneity. The paper's 'personalization' claim in the title and abstract is therefore broader than the evidence supports. This is a limitation rather than an internal inconsistency, and the paper does disclose it in Appendix C.4, but the main text should state it alongside the headline results.
minor comments (5)
  1. [§4.2 proof of Theorem 1] The step deriving Pr(at ∉ C_t | F_t) ≥ p0 − 1/(t+1)^2 is not fully explained. The subtracted term appears to guard against the Gaussian-tail event, but the lower-tail event f̃_t(a*_t) ≥ f_t(a*_t) already has probability at least p0 conditional on F_t. Clarify the role of the 1/(t+1)^2 term.
  2. [Table 2] The caption reports 'mean ± SD over three seed means.' With only three seeds, the SD reflects seed variation rather than per-episode noise; this is stated in C.4, but the main-text table could include a one-sentence reminder.
  3. [Appendix H] The 20-round controlled experiment is underpowered (exact McNemar p = 0.2188). The paper's cautious framing is appropriate, but the appendix could be shortened or moved to supplementary material.
  4. [Abstract / §3.5] The phrase 'calibrated variant' in the abstract is undefined until Section 4. Consider defining the sampling schedule in the abstract or using a less technical term.
  5. [Eq. (32)] The feature map includes both full one-hot e_M(m) and reduced vectors e_{M\{m0}}(m); the relation between these blocks (e.g., no-memory main effect vs. non-null memory contrasts) should be stated explicitly to avoid apparent duplication.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Theorem 1 is a genuine conditional regret bound, and the supplementary identifiability/promotion results are self-contained proofs.

full rationale

The claimed derivation chain is self-contained rather than circular. Theorem 1 is proved in Appendix A directly from Assumptions 1–3 by adapting standard external linear-bandit analyses (Abbasi-Yadkori, Pál, and Szepesvári 2011; Agrawal and Goyal 2013; Abeille and Lazaric 2017); the proof does not invoke any conclusion of this paper, and the known default–cost offset is absorbed as a round-dependent known score. The regret comparator is the best feasible action under the assumed linear residual model (Eq. 1), which is the standard oracle in bandit theory, not a prediction about real-user reward outside the model. Proposition 2 is a linear-algebra characterization of identifiability, and Proposition 5/Theorem 5 follow from self-normalized martingale inequalities plus the promotion rule's confidence-interval threshold; none of these reduce to their inputs by construction. The evaluation reward in tau2 does embed the same hand-tuned cost table c(a_t) that FABLE treats as a known offset, and the full-vs-frozen contrast bundles posterior updating with promotion—Appendix C.1 explicitly concedes: 'This contrast combines posterior updating with any promotion it triggers and does not identify an update-only effect: the no-promotion arm still updates but has alignment 0.686, below frozen FABLE's 0.693.' These are experimental-construction and causal-attribution risks, not circular derivations. The paper itself flags these limitations and the absence of an update-only causal benefit. No load-bearing self-citation, imported uniqueness theorem, or ansatz-smuggling step was found, so the appropriate finding is 'no significant circularity.'

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The ledger shows the paper is explicit about its model assumptions but carries many hand-set constants. The default–cost tables and the cost weight are the most influential: they are treated as 'known' outside the learning, yet they are authored by the same team and embedded in the evaluation reward, so part of what is being optimized is the authors' own construction. The theory itself depends only on the standard linear-bandit assumptions.

free parameters (9)
  • cost weight λ = 1 (PAHF, Math500); 0.2 inside tau2 reward
    Scales the operational-cost penalty in the training/evaluation objective; hand-chosen, not learned.
  • observation noise variance σ² = 1 (theory), 0.25 (Math500), 0.05 (PAHF)
    Gaussian working likelihood scale for posterior updates; hand-set per experiment.
  • base prior precision λ0 = 1.0 (Math500)
    Sets Σ_base = λ0^{-1} I; part of the cold-start prior.
  • onboarding precision κ_j = assigned by LLM encoder; 1.0 in Math500
    Precision of pseudo-observations on preference directions; hand-set.
  • default score tables (bmem, btool, bamb, brisk, btask) = coefficients such as 0.18, 0.04, 0.16, 0.20, 0.08 (Appendix F)
    Hand-constructed 'known' reference behavior; appears in both the model and the evaluation reward, so it materially affects results.
  • cost tables (c_M, c_T, c_S) = 0, 0.02–0.12 (Appendix F)
    Hand-constructed operational costs embedded in the tau2 reward signal.
  • clarification and post-correction penalties = 0.08, 0.15 (PAHF)
    Hand-set rewards in the PAHF feedback adapter.
  • promotion hyperparameters = α=0.05, n_min=5, prior radius 1.0 (Math500)
    Anytime-valid CI level and evidence threshold; chosen by the authors.
  • 10 calibration tasks per seed–domain shard (tau2) = not described
    Tasks excluded 'for calibration'; the procedure is unspecified, so it may be fitting constants to the benchmark.
assumptions (6)
  • domain assumption Linear residual reward: E[y_t | c, a] = b(c,a) + φ(c,a)^T θ* − λ cost(c,a), with the known offset b−λcost being correct
    Eq. (1); the whole theory and posterior recursion depend on this. In experiments this is enforced by the authors' own reward formula.
  • standard math Sub-Gaussian centered noise with bounded features and prior-centered residual radius (Assumptions 1–2)
    Needed for the AYPK self-normalized bound and the linear-TS regret analysis.
  • domain assumption Predictable nonempty feasible sets supplied before each decision (Assumption 3)
    Feasibility is assumed exogenous; the learner neither estimates nor expands the set.
  • domain assumption Gaussian working likelihood for the posterior update
    Section 3.4; exact conjugacy only if noise is Gaussian; otherwise it is an approximate filtering heuristic.
  • domain assumption Known, stable default and cost scores
    Section 3.2/Appendix F; the paper treats b and cost as prespecified, but in the evaluation they are author-chosen and appear in the reported reward.
  • domain assumption Scalar feedback in [-1,1] fully summarizes the user reaction
    Section 3.1; the feedback adapter loses everything not captured in the scalar.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Personalizing Large Language Model Agents with Small Policy Models." pith.science (2026). https://pith.science/paper/MIHXHUL4

@misc{pith2026260800215,
  author       = {Pith},
  title        = {Pith review of: Personalizing Large Language Model Agents with Small Policy Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MIHXHUL4}},
  note         = {Machine review of arXiv:2608.00215}
}
read the original abstract

Large language model (LLM) agents can retrieve memory, call tools, ask clarifying questions, and vary response style, yet adapting these execution decisions to an individual user remains difficult. Fine-tuning a separate LLM is costly or impossible for proprietary systems, while prompts and memory primarily expose user information to the agent rather than adapt its execution decisions from feedback. We formulate personalization of a frozen agent as online learning of a per-user execution policy from scalar feedback observed only for the executed action. We propose FABLE (Factorized Adaptive Bandit Layer for Execution), a lightweight policy layer outside a potentially black-box host agent. FABLE factorizes memory, information-acquisition, and response decisions so feedback updates related choices; filters actions through an externally specified feasible set before exploration; and learns user-specific residual preferences relative to a fixed default-and-cost score via Bayesian contextual Thompson sampling. Under a linear residual-reward model, a calibrated variant inherits an expected-regret bound against the best feasible action. We also characterize preferences unidentifiable under persistent feasibility constraints and provide anytime-valid false-promotion control. Across personalized-reasoning, controlled-feedback, and executable tool-use evaluations, FABLE improves several preference-sensitive behaviors relative to rule-only control while remaining competitive on end-to-end task performance.

Figures

Figures reproduced from arXiv: 2608.00215 by the authors.

Figure 1
Figure 1. FABLE around a frozen host: onboarding initializes [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Controlled two-direction experiment. Curves are [PITH_FULL_IMAGE:figures/full_fig_p023_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

79 extracted references · 5 linked inside Pith

  1. [1]

    2023 , note =

    Yao, Shunyu and Zhao, Jeffrey and Yu, Dian and Du, Nan and Shafran, Izhak and Narasimhan, Karthik and Cao, Yuan , booktitle =. 2023 , note =

    2023

  2. [2]

    arXiv preprint arXiv:2302.04761 , year =

    Schick, Timo and Dwivedi-Yu, Jane and Dess. arXiv preprint arXiv:2302.04761 , year =

    arXiv

  3. [3]

    Zhou, Shuyan and Xu, Frank F. and Zhu, Hao and Zhou, Xuhui and Lo, Robert and Sridhar, Abishek and Cheng, Xianyi and Ou, Tianyue and Bisk, Yonatan and Fried, Daniel and Alon, Uri and Neubig, Graham , booktitle =. 2024 , note =

    2024

  4. [4]

    Xie, Tianbao and Zhang, Danyang and Chen, Jixuan and Li, Xiaochuan and Zhao, Siheng and Cao, Ruisheng and Hua, Toh Jing and Cheng, Zhoujun and Shin, Dongchan and Lei, Fangyu and Liu, Yitao and Xu, Yiheng and Zhou, Shuyan and Savarese, Silvio and Xiong, Caiming and Zhong, Victor and Yu, Tao , journal =

  5. [5]

    and Song, Yufan and Li, Boxuan and Tang, Yuxuan and Jain, Kritanjali and Bao, Mengxue and Wang, Zora Z

    Xu, Frank F. and Song, Yufan and Li, Boxuan and Tang, Yuxuan and Jain, Kritanjali and Bao, Mengxue and Wang, Zora Z. and Zhou, Xuhui and Guo, Zhitong and Cao, Murong and others , journal =

  6. [6]

    Transactions on Machine Learning Research (TMLR) , year =

    Personalization of Large Language Models: A Survey , author =. Transactions on Machine Learning Research (TMLR) , year =

  7. [7]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    Democratizing Large Language Models via Personalized Parameter-Efficient Fine-Tuning , author =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    2024

  8. [8]

    Liu, Jiongnan and Zhu, Yutao and Wang, Shuting and Wei, Xiaochi and Min, Erxue and Lu, Yu and Wang, Shuaiqiang and Yin, Dawei and Dou, Zhicheng , booktitle =

Show all 79 references
  1. [9]

    Qu, Zikun and Zhang, Min and Kong, Mingze and Li, Xiang and Shang, Zhiwei and Wang, Zhiyong and Ban, Yikun and Qiu, Shuang and Shu, Yao and Dai, Zhongxiang , journal =

  2. [10]

    and Li, Xian , booktitle =

    Zhang, Weizhi and Zhang, Xinyang and Zhang, Chenwei and Yang, Liangwei and Shang, Jingbo and Wei, Zhepei and Zou, Henry Peng and Huang, Zijie and Wang, Zhengyang and Gao, Yifan and Pan, Xiaoman and Xiong, Lian and Liu, Jingguo and Yu, Philip S. and Li, Xian , booktitle =

  3. [11]

    Tao, Zhen and Lai, Riwei and Yu, Chenyun and Chen, Weixin and Chen, Li and Kong, Beibei and Cheng, Lei and Zhuo, Chengxiang and Li, Zang and Sun, Qingqiang , journal =

  4. [12]

    Memory as a Controlled Process: Learned Adaptive Memory Management for

    Jiang, Eric Hanchen and Zhang, Zhi and Wu, Yuchen and Li, Levina and Liu, Dong and Liang, Xiao and Sun, Rui and Li, Yubei and Sun, Edward and Luo, Haozheng and Kang, Zhaolu and Caliskan, Aylin and Chang, Kai-Wei and Wu, Ying Nian , journal =. Memory as a Controlled Process: Le...

  5. [13]

    User Preference Modeling for Conversational

    Hao, Yuren and Mehri, Shuhaib and Zhai, ChengXiang and Hakkani-T. User Preference Modeling for Conversational. arXiv preprint arXiv:2603.20939 , year =

  6. [14]

    Optimizing User Profiles via Contextual Bandits for Retrieval-Augmented

    Du, Linfeng and Yuan, Ye and Zhao, Zichen and Lyu, Fuyuan and Penaloza, Emiliano and Chen, Xiuying and Sun, Zipeng and Kang, Jikun and Charlin, Laurent and Liu, Xue and Wu, Haolun , journal =. Optimizing User Profiles via Contextual Bandits for Retrieval-Augmented

  7. [15]

    Er, Suleyman Armagan and Ribeiro, Danilo and Virkar, Yogesh and Lakew, Surafel and Kalyanpur, Adi and Gung, James and Delteil, Thomas and Gupta, Arshit , journal =

  8. [16]

    Training Proactive and Personalized

    Sun, Weiwei and Zhou, Xuhui and Du, Weihua and Wang, Xingyao and Welleck, Sean and Neubig, Graham and Sap, Maarten and Yang, Yiming , journal =. Training Proactive and Personalized

  9. [17]

    Proceedings of the ACM Web Conference 2025 , year =

    Large Language Models Empowered Personalized Web Agents , author =. Proceedings of the ACM Web Conference 2025 , year =

    2025

  10. [18]

    2025 , publisher =

    Xu, Qiancheng and Li, Yongqi and Xia, Heming and Liu, Fan and Yang, Min and Li, Wenjie , booktitle =. 2025 , publisher =

    2025

  11. [19]

    Findings of the Association for Computational Linguistics: ACL 2026 , pages =

    Towards Preference Following in Tool Calling Language Agents , author =. Findings of the Association for Computational Linguistics: ACL 2026 , pages =. 2026 , publisher =

    2026

  12. [20]

    Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =

    Unsupervised Human Preference Learning , author =. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing , pages =. 2024 , publisher =

    2024

  13. [21]

    Online Personalizing White-box

    Chen, Zekai and Daniel, Weeden and Chen, Po-yu and Buet-Golfouse, Francois , journal =. Online Personalizing White-box

  14. [22]

    Proceedings of the 41st International Conference on Machine Learning , pages =

    Coactive Learning for Large Language Models using Implicit User Feedback , author =. Proceedings of the 41st International Conference on Machine Learning , pages =. 2024 , volume =

    2024

  15. [23]

    Learning to Control

    Yi, Haiwen and Song, Xinyuan , journal =. Learning to Control

  16. [24]

    Yu, Sheldon and Wu, Junda and Li, Xintong and Kuang, Nikki Lijing and Zhou, Sizhe and Yu, Tong and Han, Jiawei and Shang, Jingbo and McAuley, Julian , journal =

  17. [25]

    2024 , note =

    Salemi, Alireza and Mysore, Sheshera and Bendersky, Michael and Zamani, Hamed , booktitle =. 2024 , note =

    2024

  18. [26]

    Two Tales of Persona in

    Tseng, Yu-Min and Huang, Yu-Chao and Hsiao, Teng-Yun and Chen, Wei-Lin and Huang, Chao-Wei and Meng, Yu and Chen, Yun-Nung , booktitle =. Two Tales of Persona in. 2024 , note =

    2024

  19. [27]

    Personalized Reasoning: Just-In-Time Personalization and Why

    Li, Shuyue Stella and Bose, Avinandan and Brahman, Faeze and Du, Simon Shaolei and Koh, Pang Wei and Fazel, Maryam and Tsvetkov, Yulia , journal =. Personalized Reasoning: Just-In-Time Personalization and Why

  20. [28]

    arXiv preprint arXiv:2602.16173 , year =

    Learning Personalized Agents from Human Feedback , author =. arXiv preprint arXiv:2602.16173 , year =

  21. [29]

    2025 , note =

    Patel, Maithili and Puig, Xavier and Desai, Ruta and Mottaghi, Roozbeh and Chernova, Sonia and Truong, Joanne and Rai, Akshara , booktitle =. 2025 , note =

    2025

  22. [30]

    Clarify When Necessary: Resolving Ambiguity Through Interaction with

    Zhang, Michael JQ and Choi, Eunsol , booktitle =. Clarify When Necessary: Resolving Ambiguity Through Interaction with. 2025 , publisher =

    2025

  23. [31]

    2025 , publisher =

    Cheng, Zihao and Wang, Hongru and Liu, Zeming and Guo, Yuhang and Guo, Yuanfang and Wang, Yunhong and Wang, Haifeng , booktitle =. 2025 , publisher =

    2025

  24. [32]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents , author =. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2025 , publisher =

    2025

  25. [33]

    Measuring Mathematical Problem Solving with the

    Hendrycks, Dan and Burns, Collin and Kadavath, Saurav and Arora, Akul and Basart, Steven and Tang, Eric and Song, Dawn and Steinhardt, Jacob , booktitle =. Measuring Mathematical Problem Solving with the. 2021 , note =

    2021

  26. [34]

    arXiv preprint arXiv:2305.20050 , year =

    Let's Verify Step by Step , author =. arXiv preprint arXiv:2305.20050 , year =

    arXiv

  27. [35]

    Barres, Victor and Dong, Honghua and Ray, Soham and Si, Xujie and Narasimhan, Karthik , journal =

  28. [36]

    arXiv preprint arXiv:2601.02702 , year =

    Learning User Preferences Through Interaction for Long-Term Collaboration , author =. arXiv preprint arXiv:2601.02702 , year =

  29. [37]

    arXiv preprint arXiv:2604.17886 , year =

    Latent Preference Modeling for Cross-Session Personalized Tool Calling , author =. arXiv preprint arXiv:2604.17886 , year =

    arXiv

  30. [38]

    and Stoica, Ion and Gonzalez, Joseph E

    Packer, Charles and Wooders, Sarah and Lin, Kevin and Fang, Vivian and Patil, Shishir G. and Stoica, Ion and Gonzalez, Joseph E. , journal =

  31. [39]

    Chhikara, Prateek and Khant, Dev and Aryan, Saket and Singh, Taranjeet and Yadav, Deshraj , journal =

  32. [40]

    2025 , note =

    Xu, Wujiang and Liang, Zujie and Mei, Kai and Gao, Hang and Tan, Juntao and Zhang, Yongfeng , booktitle =. 2025 , note =

    2025

  33. [41]

    Zhong, Wanjun and Guo, Lianghong and Gao, Qiqi and Ye, He and Wang, Yanlin , journal =

  34. [42]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Deep Reinforcement Learning from Human Preferences , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  35. [43]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Training Language Models to Follow Instructions with Human Feedback , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  36. [44]

    Advances in Neural Information Processing Systems (NeurIPS) , year =

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

  37. [45]

    arXiv preprint arXiv:2408.10075 , year =

    Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning , author =. arXiv preprint arXiv:2408.10075 , year =

    arXiv

  38. [46]

    Chen, Daiwei and Chen, Yi and Rege, Aniket and Vinayak, Ramya Korlakai , journal =

  39. [47]

    , journal =

    Park, Chanwoo and Liu, Mingyang and Kong, Dingwen and Zhang, Kaiqing and Ozdaglar, Asuman E. , journal =

  40. [48]

    2025 , note =

    Chen, Ruizhe and Zhang, Xiaotian and Luo, Meng and Chai, Wenhao and Liu, Zuozhu , booktitle =. 2025 , note =

    2025

  41. [49]

    Active Preference Optimization for Sample Efficient

    Das, Nirjhar and Chakraborty, Souradip and Pacchiano, Aldo and Chowdhury, Sayak Ray , booktitle =. Active Preference Optimization for Sample Efficient. 2025 , note =

    2025

  42. [50]

    arXiv preprint arXiv:2402.09401 , year =

    Reinforcement Learning from Human Feedback with Active Queries , author =. arXiv preprint arXiv:2402.09401 , year =

    arXiv

  43. [51]

    Saha, Aadirupa and Pacchiano, Aldo and Lee, Jonathan , booktitle =. Dueling. 2023 , volume =

    2023

  44. [52]

    Proceedings of the 19th International Conference on World Wide Web (WWW) , pages =

    A Contextual-Bandit Approach to Personalized News Article Recommendation , author =. Proceedings of the 19th International Conference on World Wide Web (WWW) , pages =. 2010 , note =

    2010

  45. [53]

    Proceedings of the 26th International Conference on Artificial Intelligence and Statistics , pages =

    Mixed-Effect Thompson Sampling , author =. Proceedings of the 26th International Conference on Artificial Intelligence and Statistics , pages =. 2023 , volume =

    2023

  46. [54]

    and Murphy, Susan A

    Tomkins, Sabina and Liao, Peng and Klasnja, Predrag V. and Murphy, Susan A. , journal =. 2021 , doi =

    2021

  47. [55]

    Proceedings of the 37th International Conference on Machine Learning , pages =

    Graphical Models Meet Bandits: A Variational Thompson Sampling Approach , author =. Proceedings of the 37th International Conference on Machine Learning , pages =. 2020 , volume =

    2020

  48. [56]

    Advances in Neural Information Processing Systems , pages =

    Contextual Bandits and Imitation Learning with Preference-Based Active Queries , author =. Advances in Neural Information Processing Systems , pages =

  49. [57]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Action Centered Contextual Bandits , author =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  50. [58]

    Proceedings of the 35th International Conference on Machine Learning (ICML) , pages =

    Semiparametric Contextual Bandits , author =. Proceedings of the 35th International Conference on Machine Learning (ICML) , pages =. 2018 , volume =

    2018

  51. [59]

    Proceedings of the 36th International Conference on Machine Learning (ICML) , pages =

    Warm-starting Contextual Bandits: Robustly Combining Supervised and Bandit Feedback , author =. Proceedings of the 36th International Conference on Machine Learning (ICML) , pages =. 2019 , volume =

    2019

  52. [60]

    Proceedings of the 39th International Conference on Machine Learning , pages =

    Contextual Bandits with Large Action Spaces: Made Practical , author =. Proceedings of the 39th International Conference on Machine Learning , pages =. 2022 , volume =

    2022

  53. [61]

    Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =

    Nonstochastic Contextual Combinatorial Bandits , author =. Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =. 2023 , volume =

    2023

  54. [62]

    Machine Learning , volume =

    Regret Bounds for Sleeping Experts and Bandits , author =. Machine Learning , volume =. 2010 , doi =

    2010

  55. [63]

    arXiv preprint arXiv:2410.05362 , year =

    Monea, Giovanni and Bosselut, Antoine and Brantley, Kiant. arXiv preprint arXiv:2410.05362 , year =

  56. [64]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Improved Algorithms for Linear Stochastic Bandits , author =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  57. [65]

    Proceedings of the 30th International Conference on Machine Learning (ICML) , pages =

    Thompson Sampling for Contextual Bandits with Linear Payoffs , author =. Proceedings of the 30th International Conference on Machine Learning (ICML) , pages =. 2013 , volume =

    2013

  58. [66]

    Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =

    Linear Thompson Sampling Revisited , author =. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) , pages =. 2017 , volume =

    2017

  59. [67]

    Journal of Computational and Graphical Statistics , volume =

    Sparse Principal Component Analysis , author =. Journal of Computational and Graphical Statistics , volume =

  60. [68]

    and Lanckriet, Gert R

    d'Aspremont, Alexandre and El Ghaoui, Laurent and Jordan, Michael I. and Lanckriet, Gert R. G. , journal =. A Direct Formulation for Sparse

  61. [69]

    Foundations and Trends in Machine Learning , volume =

    A Tutorial on Thompson Sampling , author =. Foundations and Trends in Machine Learning , volume =. 2018 , note =

    2018

  62. [70]

    Bandit Algorithms , author =

  63. [71]

    Pattern Recognition and Machine Learning , author =

  64. [72]

    Machine Learning: A Probabilistic Perspective , author =

  65. [73]

    Annals of Statistics , volume =

    Time-Uniform, Nonparametric, Nonasymptotic Confidence Sequences , author =. Annals of Statistics , volume =

  66. [74]

    Proceedings of the 37th International Conference on Machine Learning (ICML) , pages =

    Improved Optimistic Algorithms for Logistic Bandits , author =. Proceedings of the 37th International Conference on Machine Learning (ICML) , pages =. 2020 , volume =

    2020

  67. [75]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Linear Stochastic Bandits Under Safety Constraints , author =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  68. [76]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Conservative Contextual Linear Bandits , author =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  69. [77]

    IEEE Transactions on Signal Processing , volume =

    Safe Linear Thompson Sampling With Side Information , author =. IEEE Transactions on Signal Processing , volume =. 2021 , doi =

    2021

  70. [78]

    Advances in Neural Information Processing Systems (NeurIPS) , pages =

    Factored Bandits , author =. Advances in Neural Information Processing Systems (NeurIPS) , pages =

  71. [79]

    2025 , note =

    Zhang, Zhaowei and Bai, Fengshuo and Chen, Qizhi and Ma, Chengdong and Wang, Mingzhi and Sun, Haoran and Zheng, Zilong and Yang, Yaodong , booktitle =. 2025 , note =

    2025

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.