Pith. sign in

REVIEW 4 major objections 4 minor 91 references

Contrastive Learning from Exploratory Actions: Leveraging Natural Interactions for Preference Elicitation

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Users' natural exploratory actions while customizing a robot — browsing, selecting, ignoring — are implicit preference labels, and contrastive features learned from them outperform self-supervised features on all four tested criteria.

desk verdict CLEA is a genuinely useful new data source for preference-aligned features; the main unresolved question is what 'ignored' behaviors actually mean when users never saw them. read the letter →

arxiv 2501.01367 v1 pith:CSSS7A4Z submitted 2025-01-02 cs.RO cs.AIcs.HCcs.LG

Nathaniel Dennler , Stefanos Nikolaidis , Maja Matarić This is my paper · ORCID
classification cs.ROcs.AIcs.HCcs.LG
keywords preferenceelicitationcontrastivelearningexploratorysearchrobotbehaviorrepresentationrewardhuman-robotinteractionmultimodalsignalsself-supervised
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks where robot-behavior features that capture what users actually care about can come from without forcing users to label data. It proposes that exploratory search — the selecting and ignoring users naturally do while customizing a robot — is itself a preference signal, and trains trajectory features with a contrastive triplet loss on those actions. Using data from 25 users designing signals for a Kuri robot, the authors learn features and then evaluate them with 42 new users ranking behaviors. They report that CLEA-based features outperform self-supervised features across visual, auditory, and kinetic modalities on completeness, simplicity, minimality, and explainability.

What carries the argument

The central object is the CLEA loss, a symmetric triplet loss. For each set of behaviors a user was shown, equation (1) splits it into explored ($\psi(\xi)=1$, selected or tested) and ignored ($\psi(\xi)=0$) behaviors; equations (2)–(4) sample two behaviors from one side and one from the other, then push the same-side features closer than the cross-side feature by a margin $\alpha$, summed over all users and datasets. This trains a feature map $\Phi: \Xi \to \mathbb{R}^d$ so that Euclidean distance in feature space reflects population-level preference agreement rather than physical reconstruction. Downstream, a Bradley-Terry model over rankings, equation (5), turns the features into a reward function $R_H$.

What would settle it

Train CLEA twice on the same logged exploratory actions, once with every unselected behavior as a negative and once with only behaviors the user demonstrably viewed (via gaze tracking or interaction logs) as negatives, then compare held-out ranking accuracy on the 42-participant task; if accuracy does not change, the ignored-as-negative assumption is not load-bearing, and if it falls, the assumption carries the result.

Watch

Extended reading notes

Core claim

The paper's claim is that exploratory actions are a legitimate and usable preference signal: users who scroll, filter, select, and ignore robot behaviors while customizing a robot are implicitly partitioning behaviors into categories that matter to them, and a symmetric triplet loss over that partition learns features that align with user preferences across modalities. The authors train on 25 users' exploratory actions in a signal-design task with a Kuri robot and evaluate on 42 new users who ranked behaviors. CLEA-based feature spaces outperformed self-supervised autoencoder and VAE baselines, as well as random and pretrained networks, on completeness, simplicity, minimality, and explainability across all three modalities.

Load-bearing premise

The load-bearing premise is that a behavior a user scrolls past without selecting is a true negative — actively deemed irrelevant — rather than simply unseen.

Editorial extensions

If this is right

  • Robot customization interfaces can generate training data as a by-product of the user's main task, so personalization no longer requires a separate labeling phase.
  • CLEA can be combined with existing feature learners such as autoencoders and variational autoencoders, and the paper argues the same loss can extend to trajectory similarity queries and labeled behaviors.
  • Because CLEA features transfer to a new population of users, a single round of exploratory-action data collection can serve many downstream users.
  • CLEA feature spaces support simple linear reward models in low dimensions, meaning preference elicitation can need fewer user queries and simpler models.
  • Explainability-by-example improves: top-ranked behaviors land near exemplars from the customization session in CLEA feature spaces.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the authors leave implicit: applying the same explored/ignored partition to other logged browsing data, such as recommender scrolls or video thumbnail selections, would show whether CLEA transfers to any interface where ignoring is observable.
  • The ignored-as-negative assumption could be probed directly by training on gaze-filtered negatives; if accuracy holds, the signal comes from true rejection rather than from interface exposure.
  • CLEA learns a population-level embedding; adapting the embedding to a single user after a few of their own exploratory actions is a natural next step the paper does not demonstrate.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes contrastive learning from exploratory actions (CLEA), a method for training trajectory-feature encoders from implicit labels generated during open-ended robot customization. In a signal-design study, 25 users browsed up to 100 visual, auditory, and kinetic behavior options, selecting some for physical-robot evaluation; the paper treats selected behaviors as positives and unselected behaviors as negatives in a symmetric triplet loss (Eqs. 2-4). The learned features are evaluated on a disjoint set of 42 users who rank behaviors, comparing CLEA, CLEA+AE, and CLEA+VAE with Random, Pretrained, AE, and VAE baselines on four criteria: completeness (test preference accuracy), simplicity and minimality (AUC of alignment of a linear reward model), and explainability (cosine similarity to nearest exemplar). The authors report that CLEA-based features outperform self-supervised features on most metrics in visual and auditory modalities, with the kinetic completeness comparison showing no significant differences.

Significance. The central idea is attractive: exploratory search is already required for customization, so using its byproduct as preference labels avoids proxy tasks entirely. The evaluation has real strengths: separate training and testing populations, no overlap between training objective and evaluation metrics, predefined hypotheses from Bobu et al.'s framework, Bonferroni-corrected post-hoc tests, and appendices with pseudocode and training details. If the label assumption is verified, CLEA would be a practical and scalable way to obtain user-aligned behavior representations. The reported gains are substantial in several metrics (e.g., visual completeness of CLEA+AE). The main risk is not circularity but label validity: the negative set may contain behaviors the user never considered, and at least one modality shows a null completeness result.

major comments (4)
  1. [III-B, Eq. (1); IV-A, Fig. 4] Equation (1) partitions each presented set Di into explored (psi=1) and ignored (psi=0), and the CLEA triplet loss (Eqs. 2-4) treats every ignored behavior as a negative. The semantic claim is that psi=0 means the user evaluated the behavior and found it irrelevant. In the RoSiD exploratory interface, however, participants could scroll through and keyword-filter up to 100 options, and the study did not log which options were actually visible or attended. A large fraction of Dig is therefore likely to be behaviors the user never considered, so the negative label may reflect interface exposure rather than stated preference. Because this partition is the only human supervision used to train CLEA, the load-bearing assumption is currently unverified. I ask the authors to (a) add viewport/scroll/filter logging and report results when negatives are restricted to options verifiably displayed but not selected, or (b) provide an explicit label-noise analysis showing that the learned features are not driven by exposure patterns.
  2. [V-D, 'Evaluating Completeness'; Appendix E] The abstract claims CLEA features outperform self-supervised features on completeness, but Appendix E reports that in the kinetic modality there were no significant differences between algorithms, with Random, CLEA, and CLEA+AE empirically highest (p = .008, eta^2 = .247). This directly undermines the completeness claim for one of the three modalities and the unqualified statement in Section VI that CLEA 'significantly increased performance in all modalities.' Please either qualify the completeness claims to the visual and auditory modalities, or provide additional evidence for the kinetic modality.
  3. [V-D, 'Evaluating Simplicity and Minimality'; Table I] The text states that 'a CLEA-based feature space has the highest AUC Alignment in 13 of the 15 experiments,' but Table I shows that in several dimension-modality cells the best-performing algorithm is not flagged as significant, and in some cells a non-CLEA baseline has the highest value (e.g., VAE at Visual-16, AE at Kinetic-8). Counting rank-order wins without requiring significance overstates the support for H2. Please report the number of cells in which a CLEA-based method significantly outperforms all baselines after correction, and keep the '13 of 15' claim only if it refers to significant wins or is rewritten accordingly.
  4. [V-D, 'Evaluating Simplicity and Minimality'; Appendix C-F] The evaluation of minimality compares AUC Alignment across feature spaces of dimensions 8, 16, 32, 64, and 128, but the paper does not specify how the lower-dimensional feature spaces are constructed from the 128-dimensional encoders. If these are truncations or linear projections of a 128-d embedding, the minimality comparison does not evaluate the ability of each algorithm to learn a low-dimensional representation; it evaluates the information retained by the first d coordinates. Please state the construction and, if appropriate, retrain or report results for encoders with output dimension d.
minor comments (4)
  1. [III-A, Eq. (5)] The same symbol RH is used for the user's true reward function and for the learned reward model; this makes Eq. (6) and the later discussion of omega_true and omega_est confusing. Please use distinct notation, e.g., R* and R-hat.
  2. [V-D, Minimality paragraph] 'CLAE+V AE' appears to be a typo for 'CLEA+V AE'; please correct it.
  3. [Appendix C] Hyperparameter selection reports only the chosen margin and KL weight; reporting performance across the sweep or describing the validation procedure would help readers gauge sensitivity.
  4. [Fig. 5] Error bars are described as 'mean standard error across participants'; since TPA is a per-participant proportion, consider also reporting per-participant distributions or confidence intervals.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CLEA's training signal, hyperparameter selection, and held-out ranking evaluation are distinct, so the central claim is empirically self-contained.

full rationale

The paper's load-bearing claim is that features trained with the CLEA triplet loss (Eqs. 2-4) on exploratory actions from one set of users outperform self-supervised features for preference elicitation on a separate set of users. I find no circular step. The training objective partitions behaviors into explored and ignored sets (Eq. 1) and optimizes a symmetric triplet loss; it does not incorporate any of the four evaluation metrics (test preference accuracy, AUC alignment, or cosine similarity to exemplars). Evaluation uses newly collected ranking data from 42 naive participants, split 70/30 into training and test sets, which prevents the reward models from being tested on their own training labels. Hyperparameter selection uses query-based data from the customization session (Appendix C), which is mild model selection rather than a fitted prediction being reported as a prediction. The assumption that 'ignored' behaviors were considered and rejected is a data-quality limitation, not a definitional equivalence; the paper's own result that Random ties CLEA in the kinetic modality indicates that the evaluation is not forced by the training objective. Self-citations, including the RoSiD interface [7] and prior preliminary CLEA results [76], support tooling and prior evidence, but the present study's central comparison is independently evaluated on new participants and is not justified solely by self-citation. Therefore the derivation chain does not reduce to its inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on one ad hoc assumption about ignored behaviors and several standard modeling assumptions from inverse reinforcement learning; no physical entities are introduced.

free parameters (3)
  • Contrastive margin alpha = 0.1 (visual), 0.1 (auditory), 2 (kinetic)
    Tuned via a sweep using query-based validation data from the customization session; controls triplet separation and directly affects learned feature geometry.
  • VAE KL weight beta = 1 (visual), 10 (auditory), 10 (kinetic)
    Tuned via a sweep on the same validation data; used in VAE and CLEA+VAE training.
  • Feature dimensionality = 128 for main evaluation; 8, 16, 32, 64, 128 for minimality/simplicity
    Hand-chosen based on prior work; the embedding size directly determines the minimality results.
assumptions (4)
  • domain assumption There exists a low-dimensional feature map Phi from behaviors to R^d such that the user reward can be written as R_H(Phi(xi)).
    Invoked in Section III-A as a standard assumption from inverse reinforcement learning to make reward learning tractable.
  • domain assumption All behaviors in Xi accomplish the task without errors, so user preferences are about style rather than competence.
    Stated in Section III-A; needed so that exploratory choices can be interpreted as preference signals.
  • ad hoc to paper Explored behaviors are preferred over ignored behaviors, so the ignored set can serve as negative examples in the triplet loss.
    This is the load-bearing assumption behind Equation (1) and the CLEA loss; it is plausible but not verified against actual user attention.
  • standard math Rankings can be decomposed into independent pairwise comparisons modeled by the Bradley-Terry model.
    Adopted in Section III-C from prior preference-learning work; standard and reasonable for this setting.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Contrastive Learning from Exploratory Actions: Leveraging Natural Interactions for Preference Elicitation." pith.science (2026). https://pith.science/paper/CSSS7A4Z

@misc{pith2026250101367,
  author       = {Pith},
  title        = {Pith review of: Contrastive Learning from Exploratory Actions: Leveraging Natural Interactions for Preference Elicitation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CSSS7A4Z}},
  note         = {Machine review of arXiv:2501.01367}
}
read the original abstract

People have a variety of preferences for how robots behave. To understand and reason about these preferences, robots aim to learn a reward function that describes how aligned robot behaviors are with a user's preferences. Good representations of a robot's behavior can significantly reduce the time and effort required for a user to teach the robot their preferences. Specifying these representations -- what "features" of the robot's behavior matter to users -- remains a difficult problem; Features learned from raw data lack semantic meaning and features learned from user data require users to engage in tedious labeling processes. Our key insight is that users tasked with customizing a robot are intrinsically motivated to produce labels through exploratory search; they explore behaviors that they find interesting and ignore behaviors that are irrelevant. To harness this novel data source of exploratory actions, we propose contrastive learning from exploratory actions (CLEA) to learn trajectory features that are aligned with features that users care about. We learned CLEA features from exploratory actions users performed in an open-ended signal design activity (N=25) with a Kuri robot, and evaluated CLEA features through a second user study with a different set of users (N=42). CLEA features outperformed self-supervised features when eliciting user preferences over four metrics: completeness, simplicity, minimality, and explainability.

Figures

Figures reproduced from arXiv: 2501.01367 by the authors.

Figure 1
Figure 1. Example exploratory search process. Users engaging in exploratory search test out different robot behaviors to learn what the robot is capable of and what they prefer the robot to do. the user’s preferences as modeling the user’s internal reward function, which can be addressed with inverse reinforcement learning [1, 2]. In this context, the reward function takes in numerical “features” of the robot’s behavior, e.g.… view at source ↗
Figure 2
Figure 2. CLEA: Contrastive Learning from Exploratory Actions. Users engage in exploratory search to select their preferred robot behaviors. We automatically generate data from their exploratory actions to learn features that facilitate future interactive learning processes. Our contributions are highlighted in pink, and the enabling work that CLEA supports in highlighted in green. thousands of possible robot behaviors. Parti… view at source ↗
Figure 3
Figure 3. Customization session setup. Participant designing signals for the modified Kuri robot using the query-based interface [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Exploratory search interface. The exploratory search interface the participants used to design robot signals. Participants could explore visual, auditory, and kinetic robot behaviors by scrolling through the behavior options and by typing in search terms to filter resu…
Figure 5
Figure 5. Figure 5: Completeness results. Across three modalities, feature spaces using CLEA are able to accurately predict user preferences. Error bars show mean standard error across participants. explainability tools. We adopt this framework for our analysis and present our additional …
Figure 6
Figure 6. Figure 6: Minimality results. Alignment of a linear reward model across numbers of pairwise comparisons for the smallest sized feature space. Shaded region indicates mean standard error. and Random were tied for the highest TPA, but had a higher TPA than the other methods (p < .…
Figure 7
Figure 7. Figure 7: Explainability Results. We examined the similarity between top￾ranked signals from the ranking user study and the nearest exemplar signal from the robot customization session. We found that CLEA-based feature spaces have higher similarities, facilitating explanation by…
Figure 8
Figure 8. Figure 8: Qualitative results. The leftmost image shows a reference image that users actually selected when designing signals for the robot. The other images [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Robustness results. Accuracy of a linear reward model across different levels of injected noise. Features using CLEA maintain higher performance [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Direct reward modeling results. We show the completeness across [PITH_FULL_IMAGE:figures/full_fig_p017_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

91 extracted references · 59 canonical work pages

  1. [1]

    Algorithms for inverse reinforcement learn- ing

    A. Y . Ng and S. Russell, “Algorithms for inverse reinforcement learn- ing.” in Icml, vol. 1, 2000, p. 2

    2000

  2. [2]

    Apprenticeship learning via inverse rein- forcement learning,

    P. Abbeel and A. Y . Ng, “Apprenticeship learning via inverse rein- forcement learning,” in Proceedings of the twenty-first international conference on Machine learning , 2004, p. 1

    2004

  3. [3]

    Aligning human and robot representations,

    A. Bobu, A. Peng, P. Agrawal, J. A. Shah, and A. D. Dragan, “Aligning human and robot representations,” in Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, pp. 42–54

    2024

  4. [4]

    Sirl: Similarity-based implicit representation learning,

    A. Bobu, Y . Liu, R. Shah, D. S. Brown, and A. D. Dragan, “Sirl: Similarity-based implicit representation learning,” in Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction, 2023, pp. 565–574

    2023

  5. [5]

    Pebble: Feedback-efficient interactive reinforcement learning via relabeling experience and unsupervised pre- training,

    K. Lee, L. Smith, and P. Abbeel, “Pebble: Feedback-efficient interactive reinforcement learning via relabeling experience and unsupervised pre- training,” arXiv preprint arXiv:2106.05091 , 2021

    arXiv 2021

  6. [6]

    Representation matters: Offline pretraining for sequential decision making,

    M. Yang and O. Nachum, “Representation matters: Offline pretraining for sequential decision making,” in International Conference on Machine Learning. PMLR, 2021, pp. 11 784–11 794

    2021

  7. [7]

    The rosid tool: Empowering users to design multimodal signals for human- robot collaboration,

    N. Dennler, D. Delgado, D. Zeng, S. Nikolaidis, and M. J. Matari ´c, “The rosid tool: Empowering users to design multimodal signals for human- robot collaboration,” in Experimental Robotics: The 18th International Symposium. Springer, 2023

    2023

  8. [8]

    Asking easy questions: A user-friendly approach to active reward learning,

    E. Bıyık, M. Palan, N. C. Landolfi, D. P. Losey, and D. Sadigh, “Asking easy questions: A user-friendly approach to active reward learning,” arXiv preprint arXiv:1910.04365 , 2019

    arXiv 1910

Show all 91 references
  1. [9]

    Active preference- based learning of reward functions,

    D. Sadigh, A. D. Dragan, S. Sastry, and S. A. Seshia, “Active preference- based learning of reward functions,” 2017

    2017

  2. [10]

    Preference-driven texture modeling through interactive generation and search,

    S. Lu, M. Zheng, M. C. Fontaine, S. Nikolaidis, and H. Culbertson, “Preference-driven texture modeling through interactive generation and search,” IEEE transactions on haptics, vol. 15, no. 3, pp. 508–520, 2022

    2022

  3. [11]

    Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations,

    D. Brown, W. Goo, P. Nagarajan, and S. Niekum, “Extrapolating beyond suboptimal demonstrations via inverse reinforcement learning from observations,” in International conference on machine learning . PMLR, 2019, pp. 783–792

    2019

  4. [12]

    Learning multimodal rewards from rankings,

    V . Myers, E. Biyik, N. Anari, and D. Sadigh, “Learning multimodal rewards from rankings,” in Conference on robot learning . PMLR, 2022, pp. 342–352

    2022

  5. [13]

    Learning from suboptimal demonstration via self-supervised reward regression,

    L. Chen, R. Paleja, and M. Gombolay, “Learning from suboptimal demonstration via self-supervised reward regression,” in Conference on robot learning. PMLR, 2021, pp. 1262–1277

    2021

  6. [14]

    An interactive framework for learning continuous actions policies based on corrective feedback,

    C. Celemin and J. Ruiz-del Solar, “An interactive framework for learning continuous actions policies based on corrective feedback,” Journal of Intelligent & Robotic Systems , vol. 95, pp. 77–97, 2019

    2019

  7. [15]

    Learning from physical human corrections, one feature at a time,

    A. Bajcsy, D. P. Losey, M. K. O’Malley, and A. D. Dragan, “Learning from physical human corrections, one feature at a time,” in Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, 2018, pp. 141–149

    2018

  8. [16]

    Learning human objectives from sequences of physical corrections,

    M. Li, A. Canberk, D. P. Losey, and D. Sadigh, “Learning human objectives from sequences of physical corrections,” in 2021 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2021, pp. 2877–2883

    2021

  9. [17]

    Active reward learning from critiques,

    Y . Cui and S. Niekum, “Active reward learning from critiques,” in 2018 IEEE international conference on robotics and automation (ICRA). IEEE, 2018, pp. 6907–6914

    2018

  10. [18]

    Including uncertainty when learning from human corrections,

    D. P. Losey and M. K. O’Malley, “Including uncertainty when learning from human corrections,” in Conference on Robot Learning . PMLR, 2018, pp. 123–132

    2018

  11. [19]

    Asking the right questions: Facilitating semantic constraint specification for robot skill learning and repair,

    A. Tabrez, J. Kawell, and B. Hayes, “Asking the right questions: Facilitating semantic constraint specification for robot skill learning and repair,” in 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2021, pp. 6217–6224

    2021

  12. [20]

    Planning with large language models via corrective re-prompting,

    S. S. Raman, V . Cohen, E. Rosen, I. Idrees, D. Paulius, and S. Tellex, “Planning with large language models via corrective re-prompting,” in NeurIPS 2022 Foundation Models for Decision Making Workshop, 2022

    2022

  13. [21]

    Grounding complex natural language commands for temporal tasks in unseen environments,

    J. X. Liu, Z. Yang, I. Idrees, S. Liang, B. Schornstein, S. Tellex, and A. Shah, “Grounding complex natural language commands for temporal tasks in unseen environments,” in Conference on Robot Learning . PMLR, 2023, pp. 1084–1110

    2023

  14. [22]

    Tidybot: Personalized robot assis- tance with large language models,

    J. Wu, R. Antonova, A. Kan, M. Lepert, A. Zeng, S. Song, J. Bohg, S. Rusinkiewicz, and T. Funkhouser, “Tidybot: Personalized robot assis- tance with large language models,” Autonomous Robots, vol. 47, no. 8, pp. 1087–1102, 2023

    2023

  15. [23]

    The empathic framework for task learning from implicit human feedback,

    Y . Cui, Q. Zhang, B. Knox, A. Allievi, P. Stone, and S. Niekum, “The empathic framework for task learning from implicit human feedback,” in Conference on Robot Learning . PMLR, 2021, pp. 604–626

    2021

  16. [24]

    On using social signals to enable flexible error-aware hri,

    M. Stiber, R. H. Taylor, and C.-M. Huang, “On using social signals to enable flexible error-aware hri,” in Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction , 2023, pp. 222– 230

    2023

  17. [25]

    Human-robot cross-training: computational formulation, modeling and evaluation of a human team training strat- egy,

    S. Nikolaidis and J. Shah, “Human-robot cross-training: computational formulation, modeling and evaluation of a human team training strat- egy,” in 2013 8th ACM/IEEE international conference on human-robot interaction (HRI). IEEE, 2013, pp. 33–40

    2013

  18. [26]

    Efficient model learning from joint-action demonstrations for human-robot collaborative tasks,

    S. Nikolaidis, R. Ramakrishnan, K. Gu, and J. Shah, “Efficient model learning from joint-action demonstrations for human-robot collaborative tasks,” in Proceedings of the tenth annual ACM/IEEE international conference on human-robot interaction , 2015, pp. 189–196

    2015

  19. [27]

    Maximum entropy inverse reinforcement learning

    B. D. Ziebart, A. L. Maas, J. A. Bagnell, and A. K. Dey, “Maximum entropy inverse reinforcement learning.” in Aaai, vol. 8. Chicago, IL, USA, 2008, pp. 1433–1438

    2008

  20. [28]

    Learning from imperfect demonstrations from agents with varying dynamics,

    Z. Cao and D. Sadigh, “Learning from imperfect demonstrations from agents with varying dynamics,” IEEE Robotics and Automation Letters , vol. 6, no. 3, pp. 5231–5238, 2021

    2021

  21. [29]

    Inferring non-stationary human preferences for human-agent teams,

    D. Hughes, A. Agarwal, Y . Guo, and K. Sycara, “Inferring non-stationary human preferences for human-agent teams,” in 2020 29th IEEE Inter- national Conference on Robot and Human Interactive Communication (RO-MAN). IEEE, 2020, pp. 1178–1185

    2020

  22. [30]

    Inquire: Interactive querying for user- aware informative reasoning,

    T. Fitzgerald, P. Koppol, P. Callaghan, R. Q. J. H. Wong, R. Simmons, O. Kroemer, and H. Admoni, “Inquire: Interactive querying for user- aware informative reasoning,” in 6th Annual Conference on Robot Learning, 2022

    2022

  23. [31]

    Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences,

    E. Bıyık, D. P. Losey, M. Palan, N. C. Landolfi, G. Shevchuk, and D. Sadigh, “Learning reward functions from diverse sources of human feedback: Optimally integrating demonstrations and preferences,” The International Journal of Robotics Research , vol. 41, no. 1, pp. 45–67, 2022

    2022

  24. [32]

    Understanding the relationship between interactions and outcomes in human-in-the-loop machine learning,

    Y . Cui, P. Koppol, H. Admoni, S. Niekum, R. Simmons, A. Steinfeld, and T. Fitzgerald, “Understanding the relationship between interactions and outcomes in human-in-the-loop machine learning,” in International Joint Conference on Artificial Intelligence , 2021

    2021

  25. [33]

    Inverse reward design,

    D. Hadfield-Menell, S. Milli, P. Abbeel, S. J. Russell, and A. Dragan, “Inverse reward design,” Advances in neural information processing systems, vol. 30, 2017

    2017

  26. [34]

    Modeling variation in human feedback with user inputs: An exploratory methodology,

    J. Huang, R. M. Aronson, and E. S. Short, “Modeling variation in human feedback with user inputs: An exploratory methodology,” in Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, pp. 303–312

    2024

  27. [35]

    Safe imitation learning via fast bayesian reward inference from preferences,

    D. Brown, R. Coleman, R. Srinivasan, and S. Niekum, “Safe imitation learning via fast bayesian reward inference from preferences,” inInterna- tional Conference on Machine Learning. PMLR, 2020, pp. 1165–1177

    2020

  28. [36]

    Curl: Contrastive unsupervised representations for reinforcement learning,

    M. Laskin, A. Srinivas, and P. Abbeel, “Curl: Contrastive unsupervised representations for reinforcement learning,” in International conference on machine learning . PMLR, 2020, pp. 5639–5650

    2020

  29. [37]

    Recurrent world models facilitate policy evolution,

    D. Ha and J. Schmidhuber, “Recurrent world models facilitate policy evolution,” Advances in neural information processing systems , vol. 31, 2018

    2018

  30. [38]

    Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,

    Y . Wang, Z. Sun, J. Zhang, Z. Xian, E. Biyik, D. Held, and Z. Erickson, “Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,” arXiv preprint arXiv:2402.03681 , 2024

    2024 arXiv

  31. [39]

    State representations in robotics: Identifying relevant factors of variation using weak supervision,

    C. Chamzas, M. Lippi, M. C. Welle, A. Varava, A. Marino, L. E. Kavraki, and D. Kragic, “State representations in robotics: Identifying relevant factors of variation using weak supervision,” Robot Learn. Wksp. at NeurIPS , 2020

    2020

  32. [40]

    Weakly super- vised causal representation learning,

    J. Brehmer, P. De Haan, P. Lippe, and T. S. Cohen, “Weakly super- vised causal representation learning,” Advances in Neural Information Processing Systems, vol. 35, pp. 38 319–38 331, 2022

    2022

  33. [41]

    Human-driven feature selection for a robotic agent learning classification tasks from demon- stration,

    K. Bullard, S. Chernova, and A. L. Thomaz, “Human-driven feature selection for a robotic agent learning classification tasks from demon- stration,” in 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2018, pp. 6923–6930

    2018

  34. [42]

    Designing robot learners that ask good questions,

    M. Cakmak and A. L. Thomaz, “Designing robot learners that ask good questions,” in Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction, 2012, pp. 17–24

    2012

  35. [43]

    Feature expan- sive reward learning: Rethinking human input,

    A. Bobu, M. Wiggert, C. Tomlin, and A. D. Dragan, “Feature expan- sive reward learning: Rethinking human input,” in Proceedings of the 2021 ACM/IEEE International Conference on Human-Robot Interaction, 2021, pp. 216–224

    2021

  36. [44]

    Fine-grained driving behavior prediction via context-aware multi-task inverse reinforcement learning,

    K. Nishi and M. Shimosaka, “Fine-grained driving behavior prediction via context-aware multi-task inverse reinforcement learning,” in 2020 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2020, pp. 2281–2287

    2020

  37. [45]

    Task-induced rep- resentation learning,

    J. Yamada, K. Pertsch, A. Gunjal, and J. J. Lim, “Task-induced rep- resentation learning,” in 10th International Conference on Learning Representations, ICLR 2022 . International Conference on Learning Representations, ICLR, 2022

    2022

  38. [46]

    Mind meld: Personalized meta-learning for robot-centric imitation learning,

    M. L. Schrum, E. Hedlund-Botti, N. Moorman, and M. C. Gombolay, “Mind meld: Personalized meta-learning for robot-centric imitation learning,” in 2022 17th ACM/IEEE International Conference on Human- Robot Interaction (HRI) . IEEE, 2022, pp. 157–165

    2022

  39. [47]

    Reciprocal mind meld: Improving learning from demonstration via personalized, recip- rocal teaching,

    M. L. Schrum, E. Hedlund-Botti, and M. Gombolay, “Reciprocal mind meld: Improving learning from demonstration via personalized, recip- rocal teaching,” in Conference on Robot Learning . PMLR, 2023, pp. 956–966

    2023

  40. [48]

    Extending the technology acceptance model with moti- vation and social factors,

    R. Thompson, “Extending the technology acceptance model with moti- vation and social factors,” AMCIS 1998 Proceedings , p. 254, 1998

    1998

  41. [49]

    Relating motivation to information and communication technology acceptance: Self-determination theory perspective,

    Y . Lee, J. Lee, and Y . Hwang, “Relating motivation to information and communication technology acceptance: Self-determination theory perspective,” Computers in Human Behavior , vol. 51, pp. 418–428, 2015

    2015

  42. [50]

    Exploratory search: from finding to understanding,

    G. Marchionini, “Exploratory search: from finding to understanding,” Communications of the ACM , vol. 49, no. 4, pp. 41–46, 2006

    2006

  43. [51]

    The history of information retrieval research,

    M. Sanderson and W. B. Croft, “The history of information retrieval research,” Proceedings of the IEEE , vol. 100, no. Special Centennial Issue, pp. 1444–1451, 2012

    2012

  44. [52]

    Modern information retrieval: A brief overview,

    A. Singhal, “Modern information retrieval: A brief overview,” IEEE Data Eng. Bull. , vol. 24, no. 4, pp. 35–43, 2001

    2001

  45. [53]

    Engage!: co-designing search engine result pages to foster interactions,

    G. Allen, B. L. Peterson, D. K. Ratakonda, M. N. Sakib, J. A. Fails, C. Kennington, K. L. Wright, and M. S. Pera, “Engage!: co-designing search engine result pages to foster interactions,” in Interaction Design and Children, 2021, pp. 583–587

    2021

  46. [54]

    Searchlens: Composing and capturing complex user interests for exploratory search,

    J. C. Chang, N. Hahn, A. Perer, and A. Kittur, “Searchlens: Composing and capturing complex user interests for exploratory search,” inProceed- ings of the 24th International Conference on Intelligent User Interfaces , 2019, pp. 498–509

    2019

  47. [55]

    Grapevine: A profile-based exploratory search and recommendation system for finding research advisors,

    B. Rahdari, P. Brusilovsky, D. Babichenko, E. B. Littleton, R. Patel, J. Fawcett, and Z. Blum, “Grapevine: A profile-based exploratory search and recommendation system for finding research advisors,” Proceedings of the Association for Information Science and Technology , vol. ...

    2020

  48. [56]

    Beyond actions: Exploring the discovery of tactics from user logs,

    J. He, P. Qvarfordt, M. Halvey, and G. Golovchinsky, “Beyond actions: Exploring the discovery of tactics from user logs,” Information Process- ing & Management , vol. 52, no. 6, pp. 1200–1226, 2016

    2016

  49. [57]

    Click-through rate prediction in online advertising: A literature review,

    Y . Yang and P. Zhai, “Click-through rate prediction in online advertising: A literature review,” Information Processing & Management , vol. 59, no. 2, p. 102853, 2022

    2022

  50. [58]

    Learning perceptual ker- nels for visualization design,

    C ¸ . Demiralp, M. S. Bernstein, and J. Heer, “Learning perceptual ker- nels for visualization design,” IEEE transactions on visualization and computer graphics, vol. 20, no. 12, pp. 1933–1942, 2014

    1933

  51. [59]

    Dimensionality reduction by learning an invariant mapping,

    R. Hadsell, S. Chopra, and Y . LeCun, “Dimensionality reduction by learning an invariant mapping,” in 2006 IEEE computer society con- ference on computer vision and pattern recognition (CVPR’06) , vol. 2. IEEE, 2006, pp. 1735–1742

    2006

  52. [60]

    Deep metric learning using triplet network,

    E. Hoffer and N. Ailon, “Deep metric learning using triplet network,” in Similarity-based pattern recognition: third international workshop, SIM- BAD 2015, Copenhagen, Denmark, October 12-14, 2015. Proceedings

    2015

  53. [61]

    Springer, 2015, pp. 84–92

    2015

  54. [62]

    Query chains: learning to rank from implicit feedback,

    F. Radlinski and T. Joachims, “Query chains: learning to rank from implicit feedback,” in Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining , 2005, pp. 239–248

    2005

  55. [63]

    Transfer learning of human preferences for proactive robot assistance in assembly tasks,

    H. Nemlekar, N. Dhanaraj, A. Guan, S. K. Gupta, and S. Nikolaidis, “Transfer learning of human preferences for proactive robot assistance in assembly tasks,” in Proceedings of the 2023 ACM/IEEE International Conference on Human-Robot Interaction , 2023, pp. 575–583

    2023

  56. [64]

    Approx- imating gradients for differentiable quality diversity in reinforcement learning,

    B. Tjanaka, M. C. Fontaine, J. Togelius, and S. Nikolaidis, “Approx- imating gradients for differentiable quality diversity in reinforcement learning,” in Proceedings of the Genetic and Evolutionary Computation Conference, 2022, pp. 1102–1111

    2022

  57. [65]

    Active model learning and diverse action sampling for task and motion plan- ning,

    Z. Wang, C. R. Garrett, L. P. Kaelbling, and T. Lozano-P ´erez, “Active model learning and diverse action sampling for task and motion plan- ning,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2018, pp. 4107–4114

    2018

  58. [66]

    Robotic vision for human-robot interaction and collaboration: A survey and systematic review,

    N. Robinson, B. Tidd, D. Campbell, D. Kuli ´c, and P. Corke, “Robotic vision for human-robot interaction and collaboration: A survey and systematic review,” ACM Transactions on Human-Robot Interaction , vol. 12, no. 1, pp. 1–66, 2023

    2023

  59. [67]

    Deep reinforcement learning: A brief survey,

    K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, “Deep reinforcement learning: A brief survey,” IEEE Signal Processing Magazine, vol. 34, no. 6, pp. 26–38, 2017

    2017

  60. [68]

    Dataset search: a survey,

    A. Chapman, E. Simperl, L. Koesten, G. Konstantinidis, L.-D. Ib ´a˜nez, E. Kacprzak, and P. Groth, “Dataset search: a survey,” The VLDB Journal, vol. 29, no. 1, pp. 251–272, 2020

    2020

  61. [69]

    Advances in collaborative filtering,

    Y . Koren, S. Rendle, and R. Bell, “Advances in collaborative filtering,” Recommender systems handbook , pp. 91–142, 2021

    2021

  62. [70]

    Faceted metadata for image search and browsing,

    K.-P. Yee, K. Swearingen, K. Li, and M. Hearst, “Faceted metadata for image search and browsing,” in Proceedings of the SIGCHI conference on Human factors in computing systems , 2003, pp. 401–408

    2003

  63. [71]

    Rank analysis of incomplete block designs: I. the method of paired comparisons,

    R. A. Bradley and M. E. Terry, “Rank analysis of incomplete block designs: I. the method of paired comparisons,” Biometrika, vol. 39, no. 3/4, pp. 324–345, 1952

    1952

  64. [72]

    Active preference-based gaussian process regression for reward learning and optimization,

    E. Bıyık, N. Huynh, M. J. Kochenderfer, and D. Sadigh, “Active preference-based gaussian process regression for reward learning and optimization,” The International Journal of Robotics Research , vol. 43, no. 5, pp. 665–684, 2024

    2024

  65. [73]

    Team, “Kuri,” Aug 2018

    R. Team, “Kuri,” Aug 2018. [Online]. Available: https://robotsguide. com/robots/kuri

    2018

  66. [74]

    Poe: Acoustic soft robotic proprioception for omnidirectional end-effectors,

    U. Yoo, Z. Lopez, J. Ichnowski, and J. Oh, “Poe: Acoustic soft robotic proprioception for omnidirectional end-effectors,” arXiv preprint arXiv:2401.09382, 2024

    2024 arXiv

  67. [75]

    Design metaphors for understanding user expectations of socially interactive robot embodiments,

    N. Dennler, C. Ruan, J. Hadiwijoyo, B. Chen, S. Nikolaidis, and M. Matari ´c, “Design metaphors for understanding user expectations of socially interactive robot embodiments,” ACM Transactions on Human- Robot Interaction, vol. 12, no. 2, pp. 1–41, 2023

    2023

  68. [76]

    Concerning trends in likert scale usage in human-robot interaction: Towards improving best practices,

    M. Schrum, M. Ghuy, E. Hedlund-Botti, M. Natarajan, M. Johnson, and M. Gombolay, “Concerning trends in likert scale usage in human-robot interaction: Towards improving best practices,” ACM Transactions on Human-Robot Interaction, vol. 12, no. 3, pp. 1–32, 2023

    2023

  69. [77]

    Using exploratory search to learn representations for human preferences,

    N. S. Dennler, S. Nikolaidis, and M. Mataric, “Using exploratory search to learn representations for human preferences,” in Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction, 2024, pp. 392–396

    2024

  70. [78]

    X-clip: End- to-end multi-grained contrastive learning for video-text retrieval,

    Y . Ma, G. Xu, X. Sun, M. Yan, J. Zhang, and R. Ji, “X-clip: End- to-end multi-grained contrastive learning for video-text retrieval,” in Proceedings of the 30th ACM International Conference on Multimedia , 2022, pp. 638–647

    2022

  71. [79]

    Ast: Audio spectrogram trans- former,

    Y . Gong, Y .-A. Chung, and J. Glass, “Ast: Audio spectrogram trans- former,” arXiv preprint arXiv:2104.01778 , 2021

    2021 arXiv

  72. [80]

    Learning elastic embeddings for customizing on-device recommenders,

    T. Chen, H. Yin, Y . Zheng, Z. Huang, Y . Wang, and M. Wang, “Learning elastic embeddings for customizing on-device recommenders,” in Pro- ceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining , 2021, pp. 138–147

    2021

  73. [81]

    Personalizing user engagement dynamics in a non-verbal communication game for cerebral palsy,

    N. Dennler, C. Yunis, J. Realmuto, T. Sanger, S. Nikolaidis, and M. Matari ´c, “Personalizing user engagement dynamics in a non-verbal communication game for cerebral palsy,” in 2021 30th IEEE Interna- tional Conference on Robot & Human Interactive Communication (RO- MAN). IEE...

    2021

  74. [82]

    Self-supervised pretraining improves self-supervised pretraining,

    C. J. Reed, X. Yue, A. Nrusimha, S. Ebrahimi, V . Vijaykumar, R. Mao, B. Li, S. Zhang, D. Guillory, S. Metzger, K. Keutzer, and T. Darrell, “Self-supervised pretraining improves self-supervised pretraining,” in Proceedings of the IEEE/CVF Winter Conference on Applications of C...

    2022

  75. [83]

    Case-based explanation of non-case-based learning methods

    R. Caruana, H. Kangarloo, J. D. Dionisio, U. Sinha, and D. Johnson, “Case-based explanation of non-case-based learning methods.” in Pro- ceedings of the AMIA Symposium . American Medical Informatics Association, 1999, p. 212

    1999

  76. [84]

    Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning,

    N. Papernot and P. McDaniel, “Deep k-nearest neighbors: Towards confident, interpretable and robust deep learning,” arXiv preprint arXiv:1803.04765, 2018

    2018 arXiv

  77. [85]

    The bayesian case model: a generative approach for case-based reasoning and prototype classification,

    B. Kim, C. Rudin, and J. Shah, “The bayesian case model: a generative approach for case-based reasoning and prototype classification,” in Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 , ser. NIPS’14. Cambridge, MA, USA: MI...

    2014

  78. [86]

    Generating visual explanations,

    L. A. Hendricks, Z. Akata, M. Rohrbach, J. Donahue, B. Schiele, and T. Darrell, “Generating visual explanations,” in Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14 . Springer, 2016, pp. 3–19

    2016

  79. [87]

    Distributed representations of words and phrases and their composi- tionality,

    T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their composi- tionality,” Advances in neural information processing systems , vol. 26, 2013

    2013

  80. [88]

    Learning language-conditioned robot behavior from offline data and crowd- sourced annotation,

    S. Nair, E. Mitchell, K. Chen, S. Savarese, and C. Finn, “Learning language-conditioned robot behavior from offline data and crowd- sourced annotation,” in Conference on Robot Learning . PMLR, 2022, pp. 1303–1315

    2022

  81. [89]

    Roboclip: One demonstration is enough to learn robot policies,

    S. Sontakke, J. Zhang, S. Arnold, K. Pertsch, E. Bıyık, D. Sadigh, C. Finn, and L. Itti, “Roboclip: One demonstration is enough to learn robot policies,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  82. [90]

    Person search with natural language description,

    S. Li, T. Xiao, H. Li, B. Zhou, D. Yue, and X. Wang, “Person search with natural language description,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 1970–1979

    2017

  83. [91]

    Fast, cheap, and good: Why animated gifs engage us,

    S. Bakhshi, D. A. Shamma, L. Kennedy, Y . Song, P. De Juan, and J. Kaye, “Fast, cheap, and good: Why animated gifs engage us,” in Proceedings of the 2016 chi conference on human factors in computing systems, 2016, pp. 575–586. APPENDIX ADDITIONAL DEMOGRAPHIC AND STUDY INFORMAT...

    2016

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.