Pith. sign in

REVIEW 3 major objections 5 minor 41 references

Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learning

T0 review · 3 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a language model's candidate reward vectors give preference learning enough of a head start to infer a user's intended quadruped behavior from as few as four rankings, with lower error than language-only or…

desk verdict The LLM-prior idea is sound and the simulation backs it, but the 'four queries' result rides on an unvalidated sub-segment preference assumption that can contradict the paper's own preference model. read the letter →

arxiv 2502.03717 v2 pith:JXSFXQNW submitted 2025-02-06 cs.RO cs.AI

Jaden Clark , Joey Hejna , Dorsa Sadigh This is my paper · ORCID
classification cs.ROcs.AI
keywords quadrupedlocomotionpreferencelearninglargelanguagemodelsrewardparameterizationsampleefficiencyexpressivebehaviorhumanfeedbackfew-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the reason few-shot preference learning for robot behaviors is sample-hungry is not the learning algorithm but the low quality of the candidate behaviors being ranked. The proposed method, Language-Guided Preference Learning (LGPL), uses a large language model to generate candidate reward parameterizations for a quadruped — desired velocity, pitch, and gait primitive — and then asks a user to rank the resulting motions. Preference learning refines those candidates into the user's intended task vector. In simulation, LGPL reaches a mean-squared error of 0.223 against ground-truth tasks with four rankings, versus 0.455 for standard preference learning and 0.821 for a language-only baseline; in user studies, people preferred LGPL's behaviors about 76% of the time. The point of the result is that a coarse language prior can replace many preference queries, making human-aligned expressive locomotion practical in real time.

What carries the argument

The load-bearing object is a differentiable task parameterization $\omega \in \mathbb{R}^5$ (desired velocity, desired pitch, and indicators for trotting, pacing, and bounding) that defines the reward $r_\omega(s,a) = \sum_j \alpha_j \phi_j(s,a,\omega_j)$. The LLM is prompted to generate diverse candidate $\omega$ vectors, which are rolled out into trajectories. Preference learning then minimizes binary cross-entropy between user rankings and the Bradley-Terry preference probabilities induced by $r_\omega$, with the dataset expanded by slicing trajectories into sub-segments: each segment pair inherits the user's ranking of the whole trajectory. This sub-segment augmentation is what turns four ranked trajectories into enough comparisons to fit a five-dimensional reward vector.

What would settle it

A user study that collects both full-trajectory and segment-level preference judgments for the same trajectory pairs: if segment-level preferences conflict with full-trajectory preferences on a substantial fraction of pairs, the augmented comparisons are fabricated and LGPL's few-query advantage should disappear when the augmentation is removed.

Watch

Extended reading notes

Core claim

The central discovery is that LLM-generated reward vectors act as high-information priors that make preference learning effective in the few-query regime. Instead of sampling task parameterizations uniformly from the task space — which is almost certainly uninformative with four samples — LGPL prompts an LLM with expert examples and asks for diverse candidate vectors for the requested behavior. These candidates are rolled out through a fixed task-conditioned policy, the user ranks the trajectories, and the ranking labels are augmented by assuming preferences hold over sub-segments, yielding many training comparisons. The learned reward vector is optimized with a Bradley-Terry preference model, and at deployment the same task-conditioned policy is run with the learned $\omega$, requiring no retraining. The paper reports that this yields 53% lower L2 error than preference learning alone and 62% lower than LLM parameterization alone, and that users preferred LGPL behaviors 76% of the time in offline and online studies.

Load-bearing premise

The method assumes that a user's preference between two full trajectories also holds for every matching short segment of those trajectories, and this assumption generates most of the training comparisons.

Editorial extensions

If this is right

  • LGPL can personalize a quadruped's gait to a new user with four ranked trajectories, eliminating the need for hundreds or thousands of preference queries reported for standard preference learning.
  • At deployment no policy retraining is required: the existing task-conditioned policy is executed with the learned reward vector, so adaptation can happen in real time.
  • Language-only parameterization is the weakest of the three approaches in the paper's experiments, which suggests that a coarse language prior alone is not enough to capture emotive gaits, but is sufficient as a starting point for preference refinement.
  • The method's reward model must be differentiable in the task parameters, which is satisfied by current quadruped locomotion rewards; extending LGPL to nondifferentiable or high-dimensional task spaces would require additional machinery.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Implicit in the paper's framing: the LLM's role is not to answer correctly but to propose a diverse, plausible set around the true task; this reframes query selection in few-shot preference learning as a sampling-prior problem rather than an active-learning problem.
  • The four-query result likely depends on the task space being low-dimensional (five parameters) and on the LLM prior being close to the expert-defined ground truth; in higher-dimensional task spaces the paper's own limitations section predicts sample efficiency will degrade.
  • A natural extension the authors do not test: use the same LLM prior-plus-preference pipeline with human feedback given on sub-segments only, which would directly validate the load-bearing augmentation assumption.
  • The 76% preference rate is measured against baselines in the paper's user studies; a calibration study comparing LGPL to an ideal expert-tuned gait would tell whether the learned behaviors are not merely better than baselines but close to what a human actually wants.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The manuscript introduces Language-Guided Preference Learning (LGPL), a method that uses an LLM to propose candidate reward parameterizations for a quadruped locomotion policy and then refines the task vector from a small number of human rankings via preference learning with sub-segment data augmentation. The authors evaluate LGPL in Isaac Gym simulation with oracle preferences for five emotive tasks, and in offline and online user studies on a Pupper v3 quadruped, reporting lower MSE than both language-only and standard preference-learning baselines and higher user preference.

Significance. The central idea—using an LLM prior to initialize preference-learning queries—is timely and potentially valuable for real-time personalization of robot behavior. The paper has notable strengths: a controlled simulation study with five tasks and repeated seeds, oracle preference labels that directly test the learning objective, deployment on a real quadruped without retraining the policy, and both offline and interactive human evaluations. However, the headline 'as few as four queries' result currently rests on an unvalidated assumption about sub-segment preferences, and the human-evidence base is small. If the augmentation assumption is validated or removed, and the evaluation details are made reproducible, the method would be a solid practical contribution.

major comments (3)
  1. [Section III-C, Eq. (2)] Eq. (1) defines trajectory preferences through cumulative reward over the entire trajectory, but Section III-C assumes that the same preference 'also holds over sub-segments of the trajectories' and augments the dataset with (T-k)^2 * C(n,2) comparisons. This is not a harmless augmentation: for the dense velocity, pitch, and contact-pattern rewards used in Section IV-A, a preferred full trajectory can easily have lower cumulative reward than a dispreferred one on a particular length-k sub-segment, so the propagated labels can contradict the very model in Eq. (1) that is used to optimize omega. The concern is especially acute at n=4, where there are only 6 genuine comparisons and the optimization is dominated by the augmented labels. The paper does not report k or T, does not validate that full-trajectory rankings transfer to sub-segments, and does not compare LGPL against a version with augmentation disabled. I ask the authors to add a no-augmentation ablation, report k and T, and empirically check the consistency of propagated sub-segment labels against oracle reward sums; without this, the 'four queries' result does not establish that LGPL identifies user intent.
  2. [Section IV-D, Fig. 5, Table I] The human-evaluation evidence is thin: the offline study has 11 participants and the active study has 5 participants, and the active study asks each participant to describe two tasks, so the headline 76.67% win rate is based on roughly 10 task instances. No per-participant results, statistical tests, or confidence intervals are reported, and the reported +/- values are not specified as standard deviations or standard errors. Given that the human-alignment claim is a central part of the paper, the authors should report effect sizes and significance tests, or substantially temper the claim.
  3. [Section IV-A and Section IV-B] The exact LLM prompt is not included in the manuscript and is only referred to the project page, which is a reproducibility concern because the LLM-generated candidates are the core of the method. In addition, the baselines are inconsistently defined: the implementation list includes 'LLM ranked', but the figures and text use 'LPL' without defining it, and Figure 3 uses 'LLM' where the text and Figure 4 use L2R. Every compared method must be precisely specified for the reported comparisons to be interpretable.
minor comments (5)
  1. [Section III-C] The sub-segment notation is ambiguous: if each sub-segment tau~j_i has length k, the number of sub-segments should be T-k+1 rather than T-k, and the list tau~1_i, tau~2_i, ..., tau~k_i does not match the stated count.
  2. [Section IV-A] There is a typo in 'We train a train an RL policy'; the sentence should be corrected.
  3. [Figure 3] Figure 3 does not indicate what any shaded regions or error bars represent; with only 5 seeds, the authors should report the dispersion explicitly.
  4. [Abstract] The percentages in the abstract do not exactly match the reported MSE values: 0.223 versus 0.455 is a 51% reduction, and 0.223 versus 0.821 is a 73% reduction, not 53% and 62%; please reconcile the numbers.
  5. [Section IV-C] The sentence 'Each trajectory is split into 5 segments. Then we query users for preferences between trajectories' is ambiguous about whether preferences are over whole trajectories or over segments; this should be clarified.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: LGPL's LLM prior and preference signal are independent inputs; the synthetic oracle benchmark does not reduce the result to the method's assumptions.

full rationale

LGPL's derivation chain is not circular. The LLM prior (Sec. III-B) is an external knowledge source that proposes candidate parameterizations, while the ranking signal (Sec. III-C) comes from human or oracle comparisons generated from a hand-specified ground truth omega* (Sec. IV-B). The preference objective (Eq. 2) is the standard Bradley-Terry log-loss and is not defined in terms of the LLM candidates or the target MSE. The sub-segment augmentation assumption (Sec. III-C) is an unvalidated inductive assumption that could bias training, but it does not make the learned omega equal to its input by construction; it only changes the training labels. The simulation's oracle labels are produced from the same cumulative-reward model as Eq. (1), which is a self-consistent synthetic benchmark rather than a circular reduction: the result is evaluated by how well the learned parameter recovers the hidden ground truth, not by how well it reproduces the input. The paper's self-citations ([12], [18], [21], [23], [25], [27], [31]) appear only in related-work or background contexts and are not load-bearing premises for LGPL's claims. No fitted parameter is renamed as a prediction, and no uniqueness theorem from the authors is imported. The Limitations section explicitly acknowledges genuine assumptions (differentiability, query legibility, dimensionality) rather than re-deriving them away, which further supports that the claims are empirical rather than definitionally forced.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the Bradley-Terry preference model, the sub-segment preference transfer assumption, the optimality of the pretrained task-conditioned policy, the differentiability of the reward, and the informativeness of LLM in-context priors. No new physical or mathematical entities are introduced. Free parameters include the sub-segment length, reward weights, and LLM sampling configuration, which are not all specified in the paper.

free parameters (4)
  • Sub-segment length k = Not specified in paper
    The number of timesteps in each sub-segment used for preference augmentation is a hyperparameter chosen by the authors; it directly controls the size of the training dataset and affects the learned reward estimate.
  • Reward weights α_j = Not specified in paper
    The reward function rω is a weighted sum of differentiable terms with weights α_j. These weights are presumably set by hand or from prior work, and they influence the preference learning objective.
  • LLM sampling parameters (e.g., temperature) = Not specified in paper
    The diversity and quality of LLM-generated candidate ω depend on sampling parameters, which are not reported. The prompt text is also not included in the paper.
  • Number of candidate trajectories n = 4, 8, 12 in experiments
    The number of LLM-generated candidates and corresponding rankings is an experimental choice that directly affects the MSE results.
assumptions (5)
  • standard math Human preferences follow the Bradley-Terry model
    Equation (1) assumes pairwise preference probabilities are proportional to exponentials of cumulative rewards. This is a standard model from prior work, cited as [35], and is necessary for the preference learning objective.
  • domain assumption Preferences over full trajectories extend to sub-segments
    To augment the dataset, the paper assumes the user's ranking over full trajectories also applies to sub-segments of those trajectories (Section III-C). This is inherited from [37] and is not validated in this paper.
  • domain assumption The task-conditioned policy π* is near-optimal across all tasks in the training distribution
    The method assumes the RL-trained policy rolls out trajectories that adequately represent the reward parameterizations ω, so preference comparisons over these rollouts are informative. This is an unstated assumption about the fidelity of the policy.
  • domain assumption The reward function rω is differentiable with respect to ω
    This is stated as a requirement in Section III-C and is necessary for gradient-based preference learning. It restricts the expressiveness of possible tasks, as acknowledged in the limitations.
  • domain assumption LLM in-context examples are representative of desired behaviors
    The LLM is prompted with expert-provided example parameterizations and language descriptions (Section III-B). The method assumes these examples enable the LLM to generate diverse and accurate candidates for new user-specified behaviors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learning." pith.science (2026). https://pith.science/paper/JXSFXQNW

@misc{pith2026250203717,
  author       = {Pith},
  title        = {Pith review of: Efficiently Generating Expressive Quadruped Behaviors via Language-Guided Preference Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JXSFXQNW}},
  note         = {Machine review of arXiv:2502.03717}
}
read the original abstract

Expressive robotic behavior is essential for the widespread acceptance of robots in social environments. Recent advancements in learned legged locomotion controllers have enabled more dynamic and versatile robot behaviors. However, determining the optimal behavior for interactions with different users across varied scenarios remains a challenge. Current methods either rely on natural language input, which is efficient but low-resolution, or learn from human preferences, which, although high-resolution, is sample inefficient. This paper introduces a novel approach that leverages priors generated by pre-trained LLMs alongside the precision of preference learning. Our method, termed Language-Guided Preference Learning (LGPL), uses LLMs to generate initial behavior samples, which are then refined through preference-based feedback to learn behaviors that closely align with human expectations. Our core insight is that LLMs can guide the sampling process for preference learning, leading to a substantial improvement in sample efficiency. We demonstrate that LGPL can quickly learn accurate and expressive behaviors with as few as four queries, outperforming both purely language-parameterized models and traditional preference learning approaches. Website with videos: https://lgpl-gaits.github.io/

Figures

Figures reproduced from arXiv: 2502.03717 by the authors.

Figure 1
Figure 1. Our approach, LGPL, leverages LLMs to generate high quality candidate behaviors (1) for preference learning (2). This [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An overview of the LGPL method. A LLM is provided with in-context examples of task parameterizations [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Results of our simulation study to evaluate query efficiency. For each task, an expert-defined ground truth [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Results for the offline user study. % win rate signifies [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Results for the feedback-driven user study. % win rate [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 14 canonical work pages

  1. [1]

    Walk these ways: Tuning robot control for generalization with multiplicity of behavior,

    G. B. Margolis and P. Agrawal, “Walk these ways: Tuning robot control for generalization with multiplicity of behavior,” in Conference on Robot Learning. PMLR, 2023, pp. 22–31

    2023

  2. [2]

    Rma: Rapid motor adaptation for legged robots,

    A. Kumar, Z. Fu, D. Pathak, and J. Malik, “Rma: Rapid motor adaptation for legged robots,” arXiv preprint arXiv:2107.04034 , 2021

    arXiv 2021

  3. [3]

    Learning quadrupedal locomotion over challenging terrain,

    J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,” Science robotics , vol. 5, no. 47, p. eabc5986, 2020

    2020

  4. [4]

    Inverse reward design,

    D. Hadfield-Menell, S. Milli, P. Abbeel, S. J. Russell, and A. Dragan, “Inverse reward design,” Advances in neural information processing systems, vol. 30, 2017

    2017

  5. [5]

    Reward design with language models,

    M. Kwon, S. M. Xie, K. Bullard, and D. Sadigh, “Reward design with language models,” arXiv preprint arXiv:2303.00001 , 2023

    arXiv 2023

  6. [6]

    Language to rewards for robotic skill synthesis,

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.- T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik et al., “Language to rewards for robotic skill synthesis,” arXiv preprint arXiv:2306.08647 , 2023

    arXiv 2023

  7. [7]

    Saytap: Lan- guage to quadrupedal locomotion,

    Y . Tang, W. Yu, J. Tan, H. Zen, A. Faust, and T. Harada, “Saytap: Lan- guage to quadrupedal locomotion,” arXiv preprint arXiv:2306.07580 , 2023

    arXiv 2023

  8. [8]

    Eureka: Human- level reward design via coding large language models,

    Y . J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Ja- yaraman, Y . Zhu, L. Fan, and A. Anandkumar, “Eureka: Human- level reward design via coding large language models,” arXiv preprint arXiv:2310.12931, 2023

    arXiv 2023

Show all 41 references
  1. [9]

    Interactive learning from policy- dependent human feedback,

    J. MacGlashan, M. K. Ho, R. Loftin, B. Peng, G. Wang, D. L. Roberts, M. E. Taylor, and M. L. Littman, “Interactive learning from policy- dependent human feedback,” in International conference on machine learning. PMLR, 2017, pp. 2285–2294

    2017

  2. [10]

    Deep reinforcement learning from human preferences,

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems , vol. 30, 2017

    2017

  3. [11]

    Pebble: Feedback-efficient interactive reinforcement learning via relabeling experience and unsupervised pre- training,

    K. Lee, L. Smith, and P. Abbeel, “Pebble: Feedback-efficient interactive reinforcement learning via relabeling experience and unsupervised pre- training,” arXiv preprint arXiv:2106.05091 , 2021

    2021 arXiv

  4. [12]

    Few-shot preference learning for human- in-the-loop rl,

    D. J. Hejna III and D. Sadigh, “Few-shot preference learning for human- in-the-loop rl,” in Conference on Robot Learning . PMLR, 2023, pp. 2014–2025

    2023

  5. [13]

    Barkour: Bench- marking animal-level agility with quadruped robots,

    K. Caluwaerts, A. Iscen, J. C. Kew, W. Yu, T. Zhang, D. Freeman, K.-H. Lee, L. Lee, S. Saliceti, V . Zhuang et al. , “Barkour: Bench- marking animal-level agility with quadruped robots,” arXiv preprint arXiv:2305.14654, 2023

    2023 arXiv

  6. [14]

    Minimizing energy consump- tion leads to the emergence of gaits in legged robots,

    Z. Fu, A. Kumar, J. Malik, and D. Pathak, “Minimizing energy consump- tion leads to the emergence of gaits in legged robots,” arXiv preprint arXiv:2111.01674, 2021

    2021 arXiv

  7. [15]

    Fast and efficient locomotion via learned gait transitions,

    Y . Yang, T. Zhang, E. Coumans, J. Tan, and B. Boots, “Fast and efficient locomotion via learned gait transitions,” inConference on robot learning. PMLR, 2022, pp. 773–783

    2022

  8. [16]

    Code as policies: Language model programs for embodied control,

    J. Liang, W. Huang, F. Xia, P. Xu, K. Hausman, B. Ichter, P. Florence, and A. Zeng, “Code as policies: Language model programs for embodied control,” in 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023, pp. 9493–9500

    2023

  9. [17]

    Do as i can, not as i say: Grounding language in robotic affordances,

    A. Brohan, Y . Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian et al., “Do as i can, not as i say: Grounding language in robotic affordances,” in Conference on robot learning. PMLR, 2023, pp. 287–318

    2023

  10. [18]

    Generative expressive robot behaviors using large language models,

    K. Mahadevan, J. Chien, N. Brown, Z. Xu, C. Parada, F. Xia, A. Zeng, L. Takayama, and D. Sadigh, “Generative expressive robot behaviors using large language models,” in Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction , 2024, pp. 482– 491

    2024

  11. [19]

    Text2interaction: Establishing safe and preferable human-robot interaction,

    J. Thumm, C. Agia, M. Pavone, and M. Althoff, “Text2interaction: Establishing safe and preferable human-robot interaction,”arXiv preprint arXiv:2408.06105, 2024

    2024 arXiv

  12. [20]

    Learning to learn faster from human feedback with language model predictive control,

    J. Liang, F. Xia, W. Yu, A. Zeng, M. G. Arenas, M. Attarian, M. Bauza, M. Bennice, A. Bewley, A. Dostmohamed et al. , “Learning to learn faster from human feedback with language model predictive control,” arXiv preprint arXiv:2402.11450 , 2024

    2024 arXiv

  13. [21]

    Language instructed reinforcement learning for human-ai coordination,

    H. Hu and D. Sadigh, “Language instructed reinforcement learning for human-ai coordination,” in International Conference on Machine Learning. PMLR, 2023, pp. 13 584–13 598

    2023

  14. [22]

    Open problems and fundamental limitations of reinforcement learning from human feedback,

    S. Casper, X. Davies, C. Shi, T. K. Gilbert, J. Scheurer, J. Rando, R. Freedman, T. Korbak, D. Lindner, P. Freire et al. , “Open problems and fundamental limitations of reinforcement learning from human feedback,” arXiv preprint arXiv:2307.15217 , 2023

    2023 arXiv

  15. [23]

    Learning human objectives from sequences of physical corrections,

    M. Li, A. Canberk, D. P. Losey, and D. Sadigh, “Learning human objectives from sequences of physical corrections,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 2877–2883

    2021

  16. [24]

    Interactively shaping agents via human reinforcement: The tamer framework,

    W. B. Knox and P. Stone, “Interactively shaping agents via human reinforcement: The tamer framework,” in Proceedings of the fifth in- ternational conference on Knowledge capture , 2009, pp. 9–16

    2009

  17. [25]

    Learning multimodal rewards from rankings,

    V . Myers, E. Biyik, N. Anari, and D. Sadigh, “Learning multimodal rewards from rankings,” in Conference on robot learning . PMLR, 2022, pp. 342–352

    2022

  18. [26]

    Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations,

    D. Brown, W. Goo, P. Nagarajan, and S. Niekum, “Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations,” in International conference on machine learning . PMLR, 2019, pp. 783–792

    2019

  19. [27]

    Sadigh, A

    D. Sadigh, A. D. Dragan, S. Sastry, and S. A. Seshia, Active preference- based learning of reward functions , 2017

    2017

  20. [28]

    Fine-tuning language models from human preferences. arxiv 2019,

    D. M. Ziegler, N. Stiennon, J. Wu, T. B. Brown, A. Radford, D. Amodei, P. Christiano, and G. Irving, “Fine-tuning language models from human preferences. arxiv 2019,” arXiv preprint arXiv:1909.08593 , 1909

    2019 arXiv

  21. [29]

    Preference-conditioned language- guided abstraction,

    A. Peng, A. Bobu, B. Z. Li, T. R. Sumers, I. Sucholutsky, N. Kumar, T. L. Griffiths, and J. A. Shah, “Preference-conditioned language- guided abstraction,” in Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction , 2024, pp. 572–581

    2024

  22. [30]

    Maple: A framework for active preference learning guided by large language models,

    S. Mahmud, M. Nakamura, and S. Zilberstein, “Maple: A framework for active preference learning guided by large language models,” arXiv preprint arXiv:2412.07207, 2024

    2024 arXiv

  23. [31]

    Batch active preference-based learning of reward functions,

    E. Biyik and D. Sadigh, “Batch active preference-based learning of reward functions,” in Conference on robot learning . PMLR, 2018, pp. 519–528

    2018

  24. [32]

    Few-shot in-context preference learning using large language models,

    C. Yu, H. Lu, J. Gao, Q. Tan, X. Yang, Y . Wang, Y . Wu, and E. Vinitsky, “Few-shot in-context preference learning using large language models,” arXiv preprint arXiv:2410.17233 , 2024

    2024 arXiv

  25. [33]

    B-pref: Bench- marking preference-based reinforcement learning,

    K. Lee, L. Smith, A. Dragan, and P. Abbeel, “B-pref: Bench- marking preference-based reinforcement learning,” arXiv preprint arXiv:2111.03026, 2021

    2021 arXiv

  26. [34]

    A bayesian approach for policy learning from trajectory preference queries,

    A. Wilson, A. Fern, and P. Tadepalli, “A bayesian approach for policy learning from trajectory preference queries,” Advances in neural infor- mation processing systems , vol. 25, 2012

    2012

  27. [35]

    Rank analysis of incomplete block designs: I. the method of paired comparisons,

    R. A. Bradley and M. E. Terry, “Rank analysis of incomplete block designs: I. the method of paired comparisons,” Biometrika, vol. 39, no. 3/4, pp. 324–345, 1952

    1952

  28. [36]

    Safe imitation learning via fast bayesian reward inference from preferences,

    D. Brown, R. Coleman, R. Srinivasan, and S. Niekum, “Safe imitation learning via fast bayesian reward inference from preferences,” inInterna- tional Conference on Machine Learning. PMLR, 2020, pp. 1165–1177

    2020

  29. [37]

    Surf: Semi-supervised reward learning with data augmentation for feedback- efficient preference-based reinforcement learning,

    J. Park, Y . Seo, J. Shin, H. Lee, P. Abbeel, and K. Lee, “Surf: Semi-supervised reward learning with data augmentation for feedback- efficient preference-based reinforcement learning,” arXiv preprint arXiv:2203.10050, 2022

    2022 arXiv

  30. [38]

    Isaac gym: High performance gpu-based physics simulation for robot learning,

    V . Makoviychuk, L. Wawrzyniak, Y . Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa et al. , “Isaac gym: High performance gpu-based physics simulation for robot learning,” arXiv preprint arXiv:2108.10470, 2021

    2021 arXiv

  31. [39]

    Prox- imal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Prox- imal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  32. [40]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat et al. , “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  33. [41]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhou et al. , “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems , vol. 35, pp. 24 824–24 837, 2022

    2022

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.