Pith. sign in

REVIEW 3 major objections 5 minor 169 references

Steering Robots with Inference-Time Interactions

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Frozen robot policies can be steered by user interactions at inference time, with no retraining, if errors stem from task misspecification rather than missing skills.

desk verdict A thesis that compiles three solid papers and one genuinely interesting framework, but the flagship TLI theorem overclaims: the 'any LTL' guarantee only works for demonstrated transitions, not for arbitrary LTL plans. read the letter →

arxiv 2506.14287 v1 pith:AVP6CNTK submitted 2025-06-17 cs.RO cs.AIcs.HCcs.LG

Yanwei Wang This is my paper
classification cs.ROcs.AIcs.HCcs.LG
keywords inference-timesteeringimitationlearningdiffusionpolicytaskandmotiontemporallogicmodeabstractioncounterfactualperturbationshuman-robotinteraction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This thesis argues that pretrained imitation policies can be corrected at deployment by letting users steer behavior generation, instead of finetuning the policy. The key distinction is that many failures, like a pick-and-place system grasping the wrong object, come from task misspecification rather than from missing skills. If the right skill already exists in the policy, user interactions such as pointing, sketching, or physical nudges can switch the policy to that skill or adjust the continuous motion. The thesis proposes two frameworks: Inference-Time Policy Steering, which treats steering as conditional sampling from the policy's likelihood distribution, and Task and Motion Imitation, which constrains motion to stay within mode boundaries that encode task constraints. The payoff would be that a class of robot errors can be fixed instantly, without data collection or retraining, by the person using the robot.

What carries the argument

The central object is the mode, defined as a set of robot and environment configurations that share the same sensor reading, with mode boundaries encoding task constraints such as maintaining a grasp. The argument's carrying mechanism is a bisimulation between a discrete symbolic plan and a continuous policy: each mode's policy must be invariant (motion stays inside the mode) and reachable (motion eventually reaches the next mode). In the TLI instantiation, invariance is enforced by iteratively estimating convex mode boundaries with cutting planes and modulating the dynamical-system flow, while reachability comes from the global asymptotic stability of the learned dynamical systems. In the ITPS instantiation, the mechanism is stochastic sampling, an annealed MCMC procedure that samples from the product of the pretrained diffusion policy and an energy-based user-intent model rather than from their sum.

What would settle it

Run LTL-DS on a task whose mode is non-convex, such as a C-shaped reach-around region, and observe whether the cutting-plane-modulated dynamical system leaves the mode through the concave side; leaving would violate Assumption 1 and invalidate Theorem 2 in that setting. A complementary test is to steer a policy that demonstrably lacks a required skill, where success would falsify the claim that these methods only address task misspecification.

Watch

Extended reading notes

Core claim

The central claim is that a frozen, pretrained imitation policy can be made steerable so that user interactions at inference time correct misaligned predictions while keeping the policy weights unchanged. For multitask policies, the thesis shows that steering can be framed as sampling from the product of the policy's learned likelihood distribution and a user-intent objective, implemented as a stochastic sampling procedure for diffusion policies that balances alignment with constraint satisfaction. For single-task, multistep policies, the thesis introduces Temporal Logic Imitation, which recovers discrete mode boundaries from demonstrations and modulates stable dynamical-system policies so that any continuous rollout satisfying the mode sequence also satisfies the user-specified linear temporal logic task specification. The formally stated result is Theorem 2: under three assumptions, the continuous trace generated by LTL-DS satisfies any LTL specification, meaning inference-time perturbations can be corrected without retraining.

Load-bearing premise

The central claim rests on the assumption that the pretrained policy already contains every needed skill, so errors come from choosing or shaping the wrong one; if a required skill is entirely absent, steering cannot supply it.

Editorial extensions

If this is right

  • If the central claim holds, robots deployed with generalist policies can be corrected by end users through pointing, sketching, or physical nudges, without requiring the user to collect data or run training.
  • The alignment-constraint satisfaction tradeoff identified by the paper implies that steering methods that maximize user alignment will also tend to increase task failures unless the sampling procedure explicitly preserves likelihood constraints.
  • The TLI theorem provides a formal success guarantee for steering multistep policies under external perturbations, assuming the modes are convex, perturbations are finite, and all encountered modes were seen in demonstrations.
  • The GLiDE classifier-learning method shows that mode boundaries can be recovered from a few successful demonstrations supplemented by synthetically generated counterfactual failures, reducing the need for dense human annotation.
  • For policies that genuinely lack a needed skill, the paper's methods are not claimed to help; finetuning remains necessary in that case.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would apply stochastic sampling to non-diffusion generative policies, such as VAE-based or flow-matching models, to see whether the same product-of-distributions framing preserves constraints in those architectures as well.
  • The convex-mode assumption in TLI could be relaxed by replacing cutting-plane halfspaces with learned barrier functions or nonlinear mode boundaries, which would extend the formal guarantee to a wider class of manipulation tasks.
  • The mode classifiers learned by GLiDE could double as online monitors that flag invalid transitions during deployment, giving users an explanation of why a trajectory failed in addition to enabling replanning.
  • If interaction logs from steering are automatically collected, they could serve as training data for an interaction-conditioned policy $a = \pi(s, z)$, which the thesis identifies as a future direction but which would follow naturally from the same framework.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This PhD thesis compiles three published frameworks for adapting frozen imitation-learning policies to user preferences at inference time without finetuning. Chapter 2 presents ITPS, which encodes user interactions (pointing, sketching, physical corrections) as cost functions and steers diffusion policies through guided diffusion or a proposed stochastic-sampling MCMC procedure; experiments measure motion/task alignment and constraint satisfaction in Maze2D, block stacking, and a real kitchen. Chapter 3 introduces TLI, a mode-based imitation policy combining an LTL/GR(1) automaton with per-mode stable dynamical systems, cutting-plane mode-boundary estimation, and modulation to enforce mode invariance; it claims a formal guarantee (Theorem 2) that the continuous trace satisfies any LTL specification under three assumptions. Chapter 4 introduces GLiDE, which uses LLM-generated mode plans and feasibility matrices plus counterfactual perturbations to learn grounding classifiers that segment demonstrations into modes, with experiments in 2D navigation, Robosuite, and real-robot tasks. The conclusion argues for inference-time steering as a bridge between imperfect pretrained policies and deployment.

Significance. If the central claims held, the work would be significant: it offers training-free correction mechanisms, a rare formal link between discrete LTL plans and continuous learned policies, and a low-annotation pipeline for semantic grounding. The thesis has real strengths: each framework is evaluated in simulation and on a physical Franka robot; TLI includes a proof under stated assumptions and 30 human-perturbation trials (Appendix A.8); GLiDE includes ablations of each loss term and of LLM feature selection; limitation paragraphs are openly stated. The stochastic-sampling contribution in Chapter 2 is a clean algorithmic idea with a favorable empirical tradeoff. However, the headline formal claim is not established by the manuscript: Theorem 2's proof has a guard-reachability gap (Major Comment 1), and the abstract's claim of 'correcting misaligned policy predictions' is at odds with the alignment–constraint satisfaction tradeoff documented in Chapter 2 (Major Comment 3). The overall significance is therefore real but lower and narrower than claimed.

major comments (3)
  1. [Appendix A.1, Lemma 3; Section 3.4.2, Eq. (3.7); Appendix A.3] The stress-test concern lands. Lemma 3 asserts that after an unseen transition σi⇒σj, 'having learned f_j also validates the existence of x_j^*—and, thus, a continuous trajectory toward G_jk,' where k is the next mode required by the LTL automaton. But Appendix A.3 constructs x_j^* as the average of the last in-mode states before the exits that were actually demonstrated, so x_j^* lies on the guard to the mode that followed j in the demonstrations, not on the guard to an arbitrary LTL-required successor. Assumption 3 requires only that all modes appear in demonstrations, not that all transitions appear. The modulation in Eq. (3.7) only zeroes the radial velocity component at cuts; it never creates outward flow toward an undemonstrated guard. Thus the reachability leg of the bisimulation argument in Lemma 3 fails in general, and Theorem 2's 'any LTL specification' is unsupported. The soup-scooping and color-tracing examples avoid the gap only because their undemonstrated edges are backward edges into modes whose demonstrated successor lies on the original chain; this does not generalize. The theorem must be restated under a strengthened assumption (e.g., all LTL-required outgoing transitions, or their transition-specific policies as in Appendix A.7, are present in the learned skill library) or accompanied by a new mechanism for guard-directed flow.
  2. [Section 3.3.2 and Theorem 2] The theorem's quantification over 'any LTL specification' is also inconsistent with the method's GR(1) restriction. Section 3.3.2 defines the task logic as the GR(1) fragment (Eq. 3.3), and Appendix A.6 gives a GR(1) formula; the proof relies on the GR(1) property that formula clauses of the form (F G φ) are disallowed. If the claim is meant for all LTL, the paper must either prove it for the full logic or state Theorem 2 for GR(1) formulas only. This is a scope/correctness issue in a load-bearing theorem statement and should be fixed in the same revision as Major Comment 1.
  3. [Abstract; Section 2.3, Tables 2.2 and 2.3] The abstract's claim that the proposed frameworks 'correct misaligned policy predictions without requiring additional training' is not supported by Chapter 2's own data. In Table 2.3, the random-sampling baseline has CS=90% and TA=38%, while stochastic sampling reaches TA=71% at CS=73%, and output perturbation reaches TA=89% at CS=37%; Section 2.1 states the alignment-constraint satisfaction tradeoff explicitly, and Section 5.2 concedes that post hoc steering 'inevitably introduces some distribution shift.' The thesis should frame inference-time steering as a tunable tradeoff between alignment and constraint satisfaction rather than a correction mechanism that preserves policy validity.
minor comments (5)
  1. [Section 2.3.2, Table 2.2, Figure 2.7] The notation 'GD(β_i<50=0)' in Table 2.2 is undefined; clarify whether the guide ratio is zeroed for all i<50 and specify the threshold I used in the modified-GD experiment described near Figure 2.7.
  2. [Table 2.1] The table header lists ACT with dashes for GD and SS; state explicitly in the caption or text that diffusion-specific steering methods are inapplicable to the ACT policy class.
  3. [Section 3.5, Assumption 1] Convexity of real manipulation modes is asserted but never verified in the robot experiments (A.8–A.10); a sentence on whether the learned cutting-plane approximation is compatible with nonconvex modes, or on why convexity is a reasonable approximation for the tested tasks, would help.
  4. [Section 4.2.4, Eq. (4.3)] The pseudo-attractor distance threshold and the loss weights λ_s, λ_f, λ_i, λ_d are free parameters with no sensitivity analysis; report their values and, ideally, a small robustness study.
  5. [Appendix A.7] The extension to transition-conditioned policies is described only briefly and is not connected to the main Theorem 2 proof; given Major Comment 1, the relationship between per-transition DS skills and the per-mode DS used in the theorem should be made explicit.

Circularity Check

1 steps flagged · score 6.0 of 10

Theorem 2's 'any LTL specification' guarantee partially reduces to the demonstrated transitions: the attractor fitted in Appendix A.3 is used in Lemma 3 as if it lay on every LTL-required guard.

  1. fitted input called prediction [Appendix A.1, Lemma 3 and Theorem 2; Appendix A.3, attractor construction]
    "We use the average location of the last states (transition states to the next AP) in each AP as the attractor for the corresponding DS. ... Having learned f_j also validates the existence of x_j^*—and, thus, a continuous trajectory toward G_jk."

    By construction, x_j^* lies on the guard to the mode that actually followed σ_j in the demonstrations (call it l), because A.3 defines it as the average of the last in-mode states before demonstrated exits. Lemma 3 uses the existence of x_j^* to conclude a trajectory toward G_jk, the guard required by an arbitrary LTL plan. This silently identifies the LTL-required successor k with the demonstrated successor l. Assumption 3 only guarantees that every mode, not every transition, appears in demonstrations, so an undemonstrated transition may require a guard G_jk on which no attractor has been placed. Theorem 2 then claims 'satisfies any LTL specification' from this lemma.

full rationale

I walked the thesis's claimed derivation chain. The ITPS and GLiDE chapters are largely non-circular: user objectives enter as external cost functions, the sampling methods are adaptations of published MCMC/guidance procedures, and the empirical TA/CS numbers are measured against external environments (Maze2D, Robosuite, real kitchen, real robot), not against internally fitted quantities. GLiDE's success/failure losses are supervised by trajectory-level success labels and by LLM-generated feasibility matrices; the citation to the author's prior TLI paper [40] defines the forward model but is not used as a theorem load-bearing the evaluation. The circular step I find is in the TLI proof of Theorem 2: the attractor x_j^* is fitted to demonstrated exits, then Lemma 3 uses it to claim reachability to an arbitrary LTL-required guard G_jk. This makes the theorem's reachability guarantee equivalent to the demonstrated transition set by construction. The rest of TLI, including the invariance proof via cutting planes and the Lyapunov argument, is an independent construction and not circular. The practical experiments in Chapters 2 and 4 are self-contained against external benchmarks, which is why the score is 6 rather than higher: the circularity is partial and confined to the universal 'any LTL' claim in the theoretical chapter.

Assumptions & free parameters 4 free parameters · 9 assumptions · 0 invented entities

The central claims rely on a set of explicit domain assumptions rather than on fitted parameters. The most important are the convex-mode and perfect-sensor assumptions in the TLI guarantee, and the assumption that failures stem from task misspecification rather than skill deficiency. Free parameters are mostly standard hyperparameters (guide ratios, sampling steps, loss weights) that are tuned per experiment and affect the strength of steering but do not constitute fitted predictions of the target results.

free parameters (4)
  • ITPS guide ratio beta_i for guided diffusion and stochastic sampling = beta=20 (GD, Maze2D), beta=60 (SS, Maze2D), beta=100 (SS, kitchen), beta=5 (GD, kitchen)
    Chosen per experiment via sensitivity analysis (Figure 2.11) to balance alignment and constraint satisfaction; the value changes with method and task.
  • MCMC sampling steps M in stochastic sampling = M=4
    Fixed in the Maze2D experiments; controls how closely sampling approximates the product of policy and objective distributions.
  • GLiDE loss weights lambda_s, lambda_f, lambda_i, lambda_d = Not reported numerically in the text
    Hyperparameters balancing success, failure, initialization, and dynamics losses in Eq. 4.2 and 4.3; chosen by hand and not justified by a principled selection procedure.
  • Pseudo-attractor distance threshold in GLiDE mode-based motion generation = Not reported numerically
    Threshold above which the potential field toward the pseudo-attractor is applied; a design choice affecting recovery behavior under perturbations.
assumptions (9)
  • standard math Standard correctness of LTL-to-automaton conversion and GR(1) synthesis (Spot library, Kress-Gazit et al.)
    Used in Chapter 3 to convert the user-specified LTL formula into a task automaton and to argue that all discrete plans are correct by construction.
  • standard math Lyapunov stability conditions for learnable dynamical systems (Eq. 3.5) and global asymptotic stability of the learned DS
    Underpins the reachability property in Chapter 3; relies on the DS learning method of Khansari-Zadeh/Billard and Figueroa/Billard.
  • domain assumption Assumption 1: All modes are convex
    Stated in Chapter 3 (Section 3.5); guarantees the existence of cutting planes and convex inner approximations of mode boundaries. Not verified for real manipulation tasks.
  • domain assumption Assumption 2: There are finitely many external perturbations of arbitrary magnitude
    Used in the TLI proof of Theorem 2 to bound the number of required cuts. If perturbations are continuous or adversarial without bound, the guarantee fails.
  • domain assumption Assumption 3: Perturbations only cause transitions to modes already seen in demonstrations
    Requires demonstration coverage of all possible modes; if a novel sensor state appears, the method can fail because no policy exists for that mode.
  • domain assumption Perfect sensors detect mode transitions
    Acknowledged as a limitation in Chapter 3's conclusion; real sensors are noisy and thresholds are hand-tuned (e.g., distance thresholds in the soup-scooping task).
  • domain assumption The learned diffusion policy's likelihood p0 approximates the data manifold of valid trajectories
    ITPS frames steering as sampling from p0(τ)q(τ); the guarantee of constraint satisfaction depends on the policy having faithfully learned the valid trajectory distribution.
  • domain assumption An oracle provides task success labels for perturbed trajectories in GLiDE
    GLiDE requires binary success/failure labels for each perturbed rollout, obtained from simulation or human annotation; in real-robot experiments this requires reset mechanisms and significant data collection effort.
  • domain assumption LLMs provide a correct feasibility matrix and state features
    GLiDE's mode recovery is anchored to the LLM-generated plan and feasibility matrix; if the LLM is wrong about valid transitions or feature selection, the recovered modes can be misspecified (as shown in Figure 4.4d for a 4-mode instead of 5-mode matrix).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Steering Robots with Inference-Time Interactions." pith.science (2026). https://pith.science/paper/AVP6CNTK

@misc{pith2026250614287,
  author       = {Pith},
  title        = {Pith review of: Steering Robots with Inference-Time Interactions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AVP6CNTK}},
  note         = {Machine review of arXiv:2506.14287}
}
read the original abstract

Imitation learning has driven the development of generalist policies capable of autonomously solving multiple tasks. However, when a pretrained policy makes errors during deployment, there are limited mechanisms for users to correct its behavior. While collecting additional data for finetuning can address such issues, doing so for each downstream use case is inefficient at deployment. My research proposes an alternative: keeping pretrained policies frozen as a fixed skill repertoire while allowing user interactions to guide behavior generation toward user preferences at inference time. By making pretrained policies steerable, users can help correct policy errors when the model struggles to generalize-without needing to finetune the policy. Specifically, I propose (1) inference-time steering, which leverages user interactions to switch between discrete skills, and (2) task and motion imitation, which enables user interactions to edit continuous motions while satisfying task constraints defined by discrete symbolic plans. These frameworks correct misaligned policy predictions without requiring additional training, maximizing the utility of pretrained models while achieving inference-time user objectives.

Figures

Figures reproduced from arXiv: 2506.14287 by the authors.

Figure 1.1
Figure 1.1. Steering robots with pointing inputs, where multimodal motion predictions [PITH_FULL_IMAGE:figures/full_fig_p009_1_1.png] view at source ↗
Figure 1.2
Figure 1.2. Vision language action model (VLA) failure case in a new environment [ [PITH_FULL_IMAGE:figures/full_fig_p010_1_2.png] view at source ↗
Figure 1.3
Figure 1.3. (Left) Prior work finetunes pretrained policies for robustness or steerability. [PITH_FULL_IMAGE:figures/full_fig_p011_1_3.png] view at source ↗
Figures from the paper (27 more)
Figure 1.4
Figure 1.4. Figure 1.4: (Left) Inference-Time Policy Steering. For a multitask policy, interaction z steers sampling from one probable region to another within a multimodal distribution. (Right) Task and Motion Imitation. For a multistep policy, interaction z steers the continuous motion tr…
Figure 2.1
Figure 2.1. Figure 2.1: Inference-Time Policy Steering (ITPS). We present a novel framework to unify various forms of human interactions to steer a frozen genera￾tive policy. User interactions “prompt” pre-trained policies to synthesize aligned behaviors at inference time. While inference-t…
Figure 2.2
Figure 2.2. Figure 2.2: Policy Steering Methods. Given user input, methods (a-c) incorporate the alignment objective either before or after inference via (a) perturbation, (b) ranking, or (c) initialization, whereas methods (d,e) integrate the objective directly during inference. approach p…
Figure 2.3
Figure 2.3. Figure 2.3: Guided Diffusion vs. Stochastic Sampling. In a toy example aiming to sample likely data points from a pre-trained distribution while aligning with a target point, GD samples approximate the sum of two distributions, whereas SS samples approximate their product, as il…
Figure 2.4
Figure 2.4. Figure 2.4: Alignment vs. Collision in Maze2D. We compare various sampling methods with ACT and DP steered using sketch input. (1) Steering improves alignment at the cost of constraint satisfaction and increased collisions. Moreover, (2) Multimodal policies (DP) steered with PR …
Figure 2.5
Figure 2.5. Figure 2.5: Maze2D Qualitative Comparisons. We visualize trajectories (color-coded from blue to red over time) sampled with various steering methods from two policy classes (ACT and DP) given a sketch in gray. Trajectory thickness reflects similarity to the sketch after ranking,…
Figure 2.6
Figure 2.6. Figure 2.6: Robustness of ACT/DP in Maze2D [PITH_FULL_IMAGE:figures/full_fig_p024_2_6.png]
Figure 2.7
Figure 2.7. Figure 2.7: Block Stacking Qualitative Comparisons. (a) Unconditional sampling from a DP may miss intended plans, which (b) PR cannot recover, but (c) GD can. (d) Adjusting the number of diffusion steps with steering (set guide ratio βi = 0) balances similarity to the sketch ver…
Figure 2.8
Figure 2.8. Figure 2.8: Multimodal Skills. At inference time, users can steer execution towards a preferred, valid sequence by clicking a pixel in the scene camera view to specify the intended skill. The corresponding 3D location of the pixel is visualized with a red sphere that turns green…
Figure 2.9
Figure 2.9. Figure 2.9: Multimodal Valid Sequence for Kitchen Cleaning. Steering selects a preferred legal sequence of skills to be executed until the terminal state is reached. This task requires a minimum of six steps [PITH_FULL_IMAGE:figures/full_fig_p027_2_9.png]
Figure 2.10
Figure 2.10. Figure 2.10: Tradeoff Between Alignment and Distribution Shift. As the user steers the policy to align with their intent, inference-time interactions may exacerbate distribution shift and lead to execution failure [PITH_FULL_IMAGE:figures/full_fig_p027_2_10.png]
Figure 2.11
Figure 2.11. Figure 2.11: Sensitivity to Guide Ratio βi. When βi is small, steering (via point input in this case) is ineffective for both GD and SS. As ξi increases, GD begins to produce incoherent trajectories, while SS successfully identifies the intended skill. The same βi is applied for…
Figure 3.1
Figure 3.1. Figure 3.1: (a) A successful replay of the scooping task. The robot (b) is robust to motion￾level perturbations; (c) experiences an invariance failure (i.e., drops material) after a task-level perturbation; and (d) re-scoops after a task-level perturbation, avoiding failure afte…
Figure 3.2
Figure 3.2. Figure 3.2: Mode abstraction of a 2D soup-scooping task: [PITH_FULL_IMAGE:figures/full_fig_p031_3_2.png]
Figure 3.3
Figure 3.3. Figure 3.3: (a) Task automaton for a scooping task LTL. Mode a, b, c, d are reaching, scooping, transporting, and done mode respectively. Atomic proposition r, s, t denote sensing the spoon reaching the soup, soup on the spoon, and task success respectively. During successful de…
Figure 3.4
Figure 3.4. Figure 3.4: An illustration of iterative estimation of a mode boundary with cutting planes. A [PITH_FULL_IMAGE:figures/full_fig_p036_3_4.png]
Figure 3.5
Figure 3.5. Figure 3.5: Policy rollouts from different starting states for a randomly generated convex [PITH_FULL_IMAGE:figures/full_fig_p037_3_5.png]
Figure 3.6
Figure 3.6. Figure 3.6: The success rate (%) of a single-mode reaching task. As we begin to sample out of [PITH_FULL_IMAGE:figures/full_fig_p038_3_6.png]
Figure 3.7
Figure 3.7. Figure 3.7: Empirically, the invariance of a single mode requires only a finite number of cuts [PITH_FULL_IMAGE:figures/full_fig_p038_3_7.png]
Figure 3.8
Figure 3.8. Figure 3.8: Rollouts of a multi-step scooping task under perturbations. The first row shows [PITH_FULL_IMAGE:figures/full_fig_p039_3_8.png]
Figure 4.1
Figure 4.1. Figure 4.1: GLiDE framework. Given a common-sense LLM that understands (a) the appropriate state abstractions for a task and (b) how to solve the task via a sequence of manipulation modes in semantic space and (c) a few unsegmented human demonstrations that embody the transition…
Figure 4.2
Figure 4.2. Figure 4.2: (a-c) Example perturbations causing replays (red) to deviate from successful [PITH_FULL_IMAGE:figures/full_fig_p046_4_2.png]
Figure 4.3
Figure 4.3. Figure 4.3: (a) Example feasibility matrices. Specifically, [PITH_FULL_IMAGE:figures/full_fig_p048_4_3.png]
Figure 4.4
Figure 4.4. Figure 4.4: Grounding of 2D navigation task. (a) Given six demonstrations that start in [PITH_FULL_IMAGE:figures/full_fig_p050_4_4.png]
Figure 4.5
Figure 4.5. Figure 4.5: Illustration of the real robot 2D navigation task (a), where the end-effector [PITH_FULL_IMAGE:figures/full_fig_p053_4_5.png]
Figure 4.6
Figure 4.6. Figure 4.6: Illustration of the robot scooping task. Here the perturbations are human-initiated [PITH_FULL_IMAGE:figures/full_fig_p054_4_6.png]
Figure 5.1
Figure 5.1. Figure 5.1: (Left) VLA Performance vs. Product Stage. To enable end-users to achieve individualized task success with a generalist policy, we require robust pretraining to establish a strong base policy, finetuning to adapt the model to specific environments and tasks, and steer…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

169 extracted references · 25 canonical work pages

  1. [1]

    A survey of robot learning from demonstration

    B. D. Argall, S. Chernova, M. Veloso, and B. Browning. “A survey of robot learning from demonstration”. In:Robotics and autonomous systems57.5 (2009), pp. 469–483

    2009

  2. [2]

    Diffusionpolicy: Visuomotor policy learning via action diffusion

    C.Chi,S.Feng,Y.Du,Z.Xu,E.Cousineau,B.Burchfiel,andS.Song.“Diffusionpolicy: Visuomotor policy learning via action diffusion”. In:arXiv preprint arXiv:2303.04137 (2023)

    arXiv 2023

  3. [3]

    Learning fine-grained bimanual manipulation with low-cost hardware

    T. Z. Zhao, V. Kumar, S. Levine, and C. Finn. “Learning fine-grained bimanual manipulation with low-cost hardware”. In:arXiv preprint arXiv:2304.13705(2023)

    arXiv 2023

  4. [4]

    Bc-z: Zero-shot task generalization with robotic imitation learning

    E. Jang, A. Irpan, M. Khansari, D. Kappler, F. Ebert, C. Lynch, S. Levine, and C. Finn. “Bc-z: Zero-shot task generalization with robotic imitation learning”. In: Conference on Robot Learning. PMLR. 2022, pp. 991–1002

    2022

  5. [5]

    Open x-embodiment: Robotic learning datasets and rt-x models

    A. O’Neill, A. Rehman, A. Gupta, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, et al. “Open x-embodiment: Robotic learning datasets and rt-x models”. In:arXiv preprint arXiv:2310.08864(2023)

    arXiv 2023

  6. [6]

    On the opportunities and risks of foundation models

    R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al. “On the opportunities and risks of foundation models”. In:arXiv preprint arXiv:2108.07258(2021)

    arXiv 2021

  7. [7]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, X. Chen, K. Choromanski, T. Ding, D. Driess, A. Dubey, C. Finn, et al. “Rt-2: Vision-language-action models transfer web knowledge to robotic control”. In:arXiv preprint arXiv:2307.15818(2023)

    arXiv 2023

  8. [8]

    OpenVLA: An Open-Source Vision-Language-Action Model

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. “OpenVLA: An Open-Source Vision-Language-Action Model”. In:arXiv preprint arXiv:2406.09246(2024)

    arXiv 2024

Show all 169 references
  1. [9]

    Em- bodied Red Teaming for Auditing Robotic Foundation Models

    S. Karnik, Z.-W. Hong, N. Abhangi, Y.-C. Lin, T.-H. Wang, and P. Agrawal. “Em- bodied Red Teaming for Auditing Robotic Foundation Models”. In:arXiv preprint arXiv:2411.18676(2024)

    2024 arXiv

  2. [10]

    Interactive language: Talking to robots in real time

    C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, and P. Florence. “Interactive language: Talking to robots in real time”. In:IEEE Robotics and Automation Letters(2023)

    2023

  3. [11]

    Yell at your robot: Improving on-the-fly from language corrections

    L. X. Shi, Z. Hu, T. Z. Zhao, A. Sharma, K. Pertsch, J. Luo, S. Levine, and C. Finn. “Yell at your robot: Improving on-the-fly from language corrections”. In:arXiv preprint arXiv:2403.12910(2024)

    2024 arXiv

  4. [12]

    Formalizing assistive teleoperation

    A. D. Dragan and S. S. Srinivasa. “Formalizing assistive teleoperation”. In: (2013). 85

    2013

  5. [13]

    Human-in-the-loop optimization of shared autonomy in assistive robotics

    D. Gopinath, S. Jain, and B. D. Argall. “Human-in-the-loop optimization of shared autonomy in assistive robotics”. In:IEEE robotics and automation letters2.1 (2016), pp. 247–254

    2016

  6. [14]

    Preference learning in assistive robotics: Observational repeated inverse reinforcement learning

    B. Woodworth, F. Ferrari, T. E. Zosa, and L. D. Riek. “Preference learning in assistive robotics: Observational repeated inverse reinforcement learning”. In:Machine learning for healthcare conference. PMLR. 2018, pp. 420–439

    2018

  7. [15]

    Inference-time intervention: Eliciting truthful answers from a language model

    K. Li, O. Patel, F. Viégas, H. Pfister, and M. Wattenberg. “Inference-time intervention: Eliciting truthful answers from a language model”. In:Advances in Neural Information Processing Systems36 (2023), pp. 41451–41530

    2023

  8. [16]

    Diagnosis, feedback, adaptation: A human-in-the-loop framework for test-time policy adaptation

    A.Peng,A.Netanyahu,M.K.Ho,T.Shu,A.Bobu,J.Shah,andP.Agrawal.“Diagnosis, feedback, adaptation: A human-in-the-loop framework for test-time policy adaptation”. In:International Conference on Machine Learning. PMLR. 2023, pp. 27630–27641

    2023

  9. [17]

    Self-supervised policy adaptation during deployment

    N. Hansen, R. Jangir, Y. Sun, G. Alenyà, P. Abbeel, A. A. Efros, L. Pinto, and X. Wang. “Self-supervised policy adaptation during deployment”. In:arXiv preprint arXiv:2007.04309(2020)

    2020 arXiv

  10. [18]

    Lora: Low-rank adaptation of large language models. arXiv 2021

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. “Lora: Low-rank adaptation of large language models. arXiv 2021”. In:arXiv preprint arXiv:2106.09685(2021)

    2021 arXiv

  11. [19]

    Inference-time policy adapters (ipa): Tailoring extreme-scale lms without fine-tuning

    X. Lu, F. Brahman, P. West, J. Jang, K. Chandu, A. Ravichander, L. Qin, P. Am- manabrolu, L. Jiang, S. Ramnath, et al. “Inference-time policy adapters (ipa): Tailoring extreme-scale lms without fine-tuning”. In:arXiv preprint arXiv:2305.15065(2023)

    2023 arXiv

  12. [20]

    Efficiently guiding imitation learning agents with human gaze

    A. Saran, R. Zhang, E. S. Short, and S. Niekum. “Efficiently guiding imitation learning agents with human gaze”. In:arXiv preprint arXiv:2002.12500(2020)

    2020 arXiv

  13. [21]

    Leveraging Haptic Feedback to Improve Data Quality and Quantity for Deep Imitation Learning Models

    C. Cuan, A. Okamura, and M. Khansari. “Leveraging Haptic Feedback to Improve Data Quality and Quantity for Deep Imitation Learning Models”. In:IEEE Transactions on Haptics(2024)

    2024

  14. [22]

    A reduction of imitation learning and structured prediction to no-regret online learning

    S. Ross, G. Gordon, and D. Bagnell. “A reduction of imitation learning and structured prediction to no-regret online learning”. In:Proceedings of the fourteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings. 2011, ...

    2011

  15. [23]

    Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation

    Z. Fu, T. Z. Zhao, and C. Finn. “Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-body teleoperation”. In:arXiv preprint arXiv:2401.02117(2024)

    2024 arXiv

  16. [24]

    Multi-modal imitation learning from unstructured demonstrations using generative adversarial nets

    K. Hausman, Y. Chebotar, S. Schaal, G. Sukhatme, and J. Lim. “Multi-modal imitation learning from unstructured demonstrations using generative adversarial nets”. In:arXiv preprint arXiv:1705.10479(2017)

    2017 arXiv

  17. [25]

    Multi-modal motion planning in non-expansive spaces

    K. Hauser and J.-C. Latombe. “Multi-modal motion planning in non-expansive spaces”. In:The International Journal of Robotics Research29.7 (2010), pp. 897–915

    2010

  18. [26]

    Learning to switch between sensorimotor primitives using multimodal haptic signals

    Z. Su, O. Kroemer, G. E. Loeb, G. S. Sukhatme, and S. Schaal. “Learning to switch between sensorimotor primitives using multimodal haptic signals”. In:International Conference on Simulation of Adaptive Behavior. Springer. 2016, pp. 170–182. 86

    2016

  19. [27]

    Diffusion Models Beat GANs on Image Synthesis

    P. Dhariwal and A. Q. Nichol. “Diffusion Models Beat GANs on Image Synthesis”. In: Advances in Neural Information Processing Systems. 2021

    2021

  20. [28]

    Inference-Time Policy Steering through Human Interactions

    Y. Wang, L. Wang, Y. Du, B. Sundaralingam, X. Yang, Y.-W. Chao, C. Perez-D’Arpino, D. Fox, and J. Shah. “Inference-Time Policy Steering through Human Interactions”. In:arXiv preprint arXiv:2411.16627(2024)

    2024 arXiv

  21. [29]

    Is con- ditional generative modeling all you need for decision-making?

    A. Ajay, Y. Du, A. Gupta, J. Tenenbaum, T. Jaakkola, and P. Agrawal. “Is con- ditional generative modeling all you need for decision-making?” In:arXiv preprint arXiv:2211.15657(2022)

    2022 arXiv

  22. [30]

    Dynamical movement primitives: learning attractor models for motor behaviors

    A. J. Ijspeert, J. Nakanishi, H. Hoffmann, P. Pastor, and S. Schaal. “Dynamical movement primitives: learning attractor models for motor behaviors”. In:Neural computation25.2 (2013), pp. 328–373

    2013

  23. [31]

    Dart: Noise injection for robust imitation learning

    M. Laskey, J. Lee, R. Fox, A. Dragan, and K. Goldberg. “Dart: Noise injection for robust imitation learning”. In:Conference on robot learning. PMLR. 2017, pp. 143–156

    2017

  24. [32]

    Watch and Match: Supercharging Imitation with Regularized Optimal Transport

    S. Haldar, V. Mathur, D. Yarats, and L. Pinto. “Watch and Match: Supercharging Imitation with Regularized Optimal Transport”. In:CoRL(2022)

    2022

  25. [33]

    Learning stable nonlinear dynamical systems with gaussian mixture models

    S. M. Khansari-Zadeh and A. Billard. “Learning stable nonlinear dynamical systems with gaussian mixture models”. In:IEEE Transactions on Robotics27.5 (2011), pp. 943– 957

    2011

  26. [34]

    Learning robot objectives from physical human interaction

    A. Bajcsy, D. P. Losey, M. K. O’malley, and A. D. Dragan. “Learning robot objectives from physical human interaction”. In:Conference on Robot Learning. PMLR. 2017, pp. 217–226

    2017

  27. [35]

    APhysically-ConsistentBayesianNon-ParametricMixture Model for Dynamical System Learning

    N.FigueroaandA.Billard.“APhysically-ConsistentBayesianNon-ParametricMixture Model for Dynamical System Learning.” In:CoRL. 2018, pp. 927–946

    2018

  28. [36]

    Integrated Task and Motion Planning

    C. R. Garrett, R. Chitnis, R. Holladay, B. Kim, L. P. Kaelbling, and T. Lozano-Pérez. “Integrated Task and Motion Planning”. In:Ann. Rev. Control Robot. Auton. Syst.4 (2021), pp. 265–293

    2021

  29. [37]

    Synthesis for robots: Guarantees and feedback for robot behavior

    H. Kress-Gazit, M. Lahijanian, and V. Raman. “Synthesis for robots: Guarantees and feedback for robot behavior”. In:Annual Review of Control, Robotics, and Autonomous Systems1 (2018), pp. 211–236

    2018

  30. [38]

    Guided imitation of task and motion planning

    M. J. McDonald and D. Hadfield-Menell. “Guided imitation of task and motion planning”. In:Conference on Robot Learning. PMLR. 2022, pp. 630–640

    2022

  31. [39]

    Imi- tating task and motion planning with visuomotor transformers

    M. Dalal, A. Mandlekar, C. Garrett, A. Handa, R. Salakhutdinov, and D. Fox. “Imi- tating task and motion planning with visuomotor transformers”. In:arXiv preprint arXiv:2305.16309(2023)

    2023 arXiv

  32. [40]

    Temporal logic imitation: Learning plan-satisficing motion policies from demonstrations

    Y. Wang, N. Figueroa, S. Li, A. Shah, and J. Shah. “Temporal logic imitation: Learning plan-satisficing motion policies from demonstrations”. In:arXiv preprint arXiv:2206.04632(2022)

    2022 arXiv

  33. [41]

    Grounding predicates through actions

    T. Migimatsu and J. Bohg. “Grounding predicates through actions”. In:2022 Inter- national Conference on Robotics and Automation (ICRA). IEEE. 2022, pp. 3498– 3504. 87

    2022

  34. [42]

    Grounding Language Plans in Demonstrations Through Counterfactual Perturbations

    Y. Wang, T.-H. Wang, J. Mao, M. Hagenow, and J. Shah. “Grounding Language Plans in Demonstrations Through Counterfactual Perturbations”. In:arXiv preprint arXiv:2403.17124(2024)

    2024 arXiv

  35. [43]

    An algorithmic perspective on imitation learning

    T. Osa, J. Pajarinen, G. Neumann, J. A. Bagnell, P. Abbeel, J. Peters, et al. “An algorithmic perspective on imitation learning”. In:Foundations and Trends®in Robotics7.1-2 (2018), pp. 1–179

    2018

  36. [44]

    Octo: An open-source generalist robot policy

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xu, et al. “Octo: An open-source generalist robot policy”. In:arXiv preprint arXiv:2405.12213(2024)

    2024 arXiv

  37. [45]

    Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations

    J. Urain, A. Mandlekar, Y. Du, M. Shafiullah, D. Xu, K. Fragkiadaki, G. Chalvatzaki, and J. Peters. “Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations”. In:arXiv preprint arXiv:2408.04380(2024)

    2024 arXiv

  38. [46]

    Rt-trajectory: Robotic task generalization via hindsight trajectory sketches

    J. Gu, S. Kirmani, P. Wohlhart, Y. Lu, M. G. Arenas, K. Rao, W. Yu, C. Fu, K. Gopalakrishnan, Z. Xu, et al. “Rt-trajectory: Robotic task generalization via hindsight trajectory sketches”. In:arXiv preprint arXiv:2311.01977(2023)

    2023 arXiv

  39. [47]

    A point-and-click interface for the real world: laser designation of objects for mobile manipulation

    C. C. Kemp, C. D. Anderson, H. Nguyen, A. J. Trevor, and Z. Xu. “A point-and-click interface for the real world: laser designation of objects for mobile manipulation”. In: Proceedings of the 3rd ACM/IEEE international conference on human robot interaction. 2008, pp. 241–248

    2008

  40. [48]

    A review of intent detection, arbitration, and communication aspects of shared control for physical human–robot interaction

    D. P. Losey, C. G. McDonald, E. Battaglia, and M. K. O’Malley. “A review of intent detection, arbitration, and communication aspects of shared control for physical human–robot interaction”. In:Applied Mechanics Reviews70.1 (2018), p. 010804

    2018

  41. [49]

    Physical interaction as communication: Learning robot objectives online from human corrections

    D. P. Losey, A. Bajcsy, M. K. O’Malley, and A. D. Dragan. “Physical interaction as communication: Learning robot objectives online from human corrections”. In:The International Journal of Robotics Research41.1 (2022), pp. 20–44

    2022

  42. [50]

    Billard, S

    A. Billard, S. Mirrazavi, and N. Figueroa.Learning for adaptive and reactive robot control: a dynamical systems approach. Mit Press, 2022

    2022

  43. [51]

    Planning with diffusion for flexible behavior synthesis

    M. Janner, Y. Du, J. B. Tenenbaum, and S. Levine. “Planning with diffusion for flexible behavior synthesis”. In:arXiv preprint arXiv:2205.09991(2022)

    2022 arXiv

  44. [52]

    Efficient Trajectory Forecasting and Generation with Conditional Flow Matching

    S. Ye and M. Gombolay. “Efficient Trajectory Forecasting and Generation with Conditional Flow Matching”. In:arXiv preprint arXiv:2403.10809(2024)

    2024 arXiv

  45. [53]

    Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc

    Y. Du, C. Durkan, R. Strudel, J. B. Tenenbaum, S. Dieleman, R. Fergus, J. Sohl- Dickstein, A. Doucet, and W. S. Grathwohl. “Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and mcmc”. In:International conference on machine learning. PMLR. 202...

    2023

  46. [54]

    Dynamic time warping

    M. Müller. “Dynamic time warping”. In:Information retrieval for music and motion (2007), pp. 69–84

    2007

  47. [55]

    To the noise and back: Diffusion for shared autonomy

    T. Yoneda, L. Sun, B. Stadie, M. Walter, et al. “To the noise and back: Diffusion for shared autonomy”. In:arXiv preprint arXiv:2302.12244(2023). 88

    2023 arXiv

  48. [56]

    seaborn: statistical data visualization

    M. L. Waskom. “seaborn: statistical data visualization”. In:Journal of Open Source Software6.60 (2021), p. 3021.doi: 10.21105/joss.03021.url: https://doi.org/10. 21105/joss.03021

    2021 doi

  49. [57]

    D4rl: Datasets for deep data-driven reinforcement learning

    J. Fu, A. Kumar, O. Nachum, G. Tucker, and S. Levine. “D4rl: Datasets for deep data-driven reinforcement learning”. In:arXiv preprint arXiv:2004.07219(2020)

    2020 arXiv

  50. [58]

    Denoising diffusion implicit models

    J. Song, C. Meng, and S. Ermon. “Denoising diffusion implicit models”. In:arXiv preprint arXiv:2010.02502(2020)

    2020 arXiv

  51. [59]

    Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments

    M. Mittal et al. “Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments”. In:IEEE Robotics and Automation Letters8.6 (2023), pp. 3740–3747. doi: 10.1109/LRA.2023.3270034

    2023

  52. [60]

    Curobo: Parallelized collision-free robot motion generation

    B. Sundaralingam, S. K. S. Hari, A. Fishman, C. Garrett, K. Van Wyk, V. Blukis, A. Millane, H. Oleynikova, A. Handa, F. Ramos, et al. “Curobo: Parallelized collision-free robot motion generation”. In:2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE. 2...

    2023

  53. [61]

    Building a multimodal human-robot interface

    D. Perzanowski, A. C. Schultz, W. Adams, E. Marsh, and M. Bugajska. “Building a multimodal human-robot interface”. In:IEEE intelligent systems16.1 (2001), pp. 16– 21

    2001

  54. [62]

    Review of interfaces for industrial human-robot interaction

    J. Berg and S. Lu. “Review of interfaces for industrial human-robot interaction”. In: Current Robotics Reports1 (2020), pp. 27–34

    2020

  55. [63]

    Do as i can, not as i say: Grounding language in robotic affordances

    A. Brohan, Y. Chebotar, C. Finn, K. Hausman, A. Herzog, D. Ho, J. Ibarz, A. Irpan, E. Jang, R. Julian, et al. “Do as i can, not as i say: Grounding language in robotic affordances”. In:Conference on robot learning. PMLR. 2023, pp. 287–318

    2023

  56. [64]

    RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches

    P. Sundaresan, Q. Vuong, J. Gu, P. Xu, T. Xiao, S. Kirmani, T. Yu, M. Stark, A. Jain, K. Hausman, et al. “RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches”. In:arXiv preprint arXiv:2403.02709(2024)

    2024 arXiv

  57. [65]

    Goal-conditionedimitationlearning

    Y.Ding,C.Florensa,P.Abbeel,andM.Phielipp.“Goal-conditionedimitationlearning”. In:Advances in neural information processing systems32 (2019)

    2019

  58. [66]

    What matters in language conditioned robotic imitation learning over unstructured data

    O. Mees, L. Hermann, and W. Burgard. “What matters in language conditioned robotic imitation learning over unstructured data”. In:IEEE Robotics and Automation Letters 7.4 (2022), pp. 11205–11212

    2022

  59. [67]

    Diffusion co-policy for synergistic human-robot collaborative tasks

    E. Ng, Z. Liu, and M. Kennedy. “Diffusion co-policy for synergistic human-robot collaborative tasks”. In:IEEE Robotics and Automation Letters(2023)

    2023

  60. [68]

    Behavior generation with latent actions

    S. Lee, Y. Wang, H. Etukuru, H. J. Kim, N. M. M. Shafiullah, and L. Pinto. “Behavior generation with latent actions”. In:arXiv preprint arXiv:2403.03181(2024)

    2024 arXiv

  61. [69]

    Compositional visual generation with composable diffusion models

    N. Liu, S. Li, Y. Du, A. Torralba, and J. B. Tenenbaum. “Compositional visual generation with composable diffusion models”. In:European Conference on Computer Vision. Springer. 2022, pp. 423–439

    2022

  62. [70]

    Hierarchical policies for cluttered-scene grasping with latent plans

    L. Wang, X. Meng, Y. Xiang, and D. Fox. “Hierarchical policies for cluttered-scene grasping with latent plans”. In:IEEE Robotics and Automation Letters7.2 (2022), pp. 2883–2890. 89

    2022

  63. [71]

    Learning latent plans from play

    C. Lynch, M. Khansari, T. Xiao, V. Kumar, J. Tompson, S. Levine, and P. Sermanet. “Learning latent plans from play”. In:Conference on robot learning. PMLR. 2020, pp. 1113–1132

    2020

  64. [72]

    Model Based Planning with Energy Based Models

    Y. Du, T. Lin, and I. Mordatch. “Model Based Planning with Energy Based Models”. In:Conference on Robot Learning. 2019

    2019

  65. [73]

    Energy- based models as zero-shot planners for compositional scene rearrangement

    N. Gkanatsios, A. Jain, Z. Xian, Y. Zhang, C. Atkeson, and K. Fragkiadaki. “Energy- based models as zero-shot planners for compositional scene rearrangement”. In:arXiv preprint arXiv:2304.14391(2023)

    2023 arXiv

  66. [74]

    Compositional Diffusion-Based Continuous Constraint Solvers

    Z. Yang, J. Mao, Y. Du, J. Wu, J. B. Tenenbaum, T. Lozano-Pérez, and L. P. Kaelbling. “Compositional Diffusion-Based Continuous Constraint Solvers”. In:arXiv preprint arXiv:2309.00966(2023)

    2023 arXiv

  67. [75]

    Goal-conditioned imitation learning using score-based diffusion policies

    M. Reuss, M. Li, X. Jia, and R. Lioutikov. “Goal-conditioned imitation learning using score-based diffusion policies”. In:arXiv preprint arXiv:2304.02532(2023)

    2023 arXiv

  68. [76]

    Generative skill chaining: Long-horizon skill planning with diffusion models

    U. A. Mishra, S. Xue, Y. Chen, and D. Xu. “Generative skill chaining: Long-horizon skill planning with diffusion models”. In:Conference on Robot Learning. PMLR. 2023, pp. 2905–2925

    2023

  69. [77]

    Se (3)-diffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion

    J. Urain, N. Funk, J. Peters, and G. Chalvatzaki. “Se (3)-diffusionfields: Learning smooth cost functions for joint grasp and motion optimization through diffusion”. In: 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE. 2023, pp. 5923–5930

    2023

  70. [78]

    Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance

    M. Nakamoto, O. Mees, A. Kumar, and S. Levine. “Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance”. In:arXiv preprint arXiv:2410.13816(2024)

    2024 arXiv

  71. [79]

    Poco: Policy composition from and for heterogeneous robot learning

    L. Wang, J. Zhao, Y. Du, E. H. Adelson, and R. Tedrake. “Poco: Policy composition from and for heterogeneous robot learning”. In:arXiv preprint arXiv:2402.02511 (2024)

    2024 arXiv

  72. [80]

    Recent advances in robot learning from demonstration

    H. Ravichandar, A. S. Polydoros, S. Chernova, and A. Billard. “Recent advances in robot learning from demonstration”. In:Annual Review of Control, Robotics, and Autonomous Systems3 (2020), pp. 297–330

    2020

  73. [81]

    Robot learning from demonstration: a task-level planning approach

    S. Ekvall and D. Kragic. “Robot learning from demonstration: a task-level planning approach”. In:International Journal of Advanced Robotic Systems5.3 (2008), p. 33

    2008

  74. [82]

    Incremental learning of subtasks from unsegmented demonstration

    D. H. Grollman and O. C. Jenkins. “Incremental learning of subtasks from unsegmented demonstration”. In:2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE. 2010, pp. 261–266

    2010

  75. [83]

    Learning stable task sequences from demonstration with linear parameter varying systems and hidden Markov models

    J. R. Medina and A. Billard. “Learning stable task sequences from demonstration with linear parameter varying systems and hidden Markov models”. In:Conference on Robot Learning. PMLR. 2017, pp. 175–184

    2017

  76. [84]

    Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning

    A. Gupta, V. Kumar, C. Lynch, S. Levine, and K. Hausman. “Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning”. In:arXiv preprint arXiv:1910.11956(2019). 90

    2019 arXiv

  77. [85]

    Learning to generalize across long-horizon tasks from human demonstrations

    A. Mandlekar, D. Xu, R. Martín-Martín, S. Savarese, and L. Fei-Fei. “Learning to generalize across long-horizon tasks from human demonstrations”. In:arXiv preprint arXiv:2003.06085(2020)

    2020 arXiv

  78. [86]

    Modeling Long-horizon Tasks as Sequential Interaction Landscapes

    S. Pirk, K. Hausman, A. Toshev, and M. Khansari. “Modeling Long-horizon Tasks as Sequential Interaction Landscapes”. In:arXiv preprint arXiv:2006.04843(2020)

    2020 arXiv

  79. [87]

    Towards associative skill memories

    P. Pastor, M. Kalakrishnan, L. Righetti, and S. Schaal. “Towards associative skill memories”. In:2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012). IEEE. 2012, pp. 309–315

    2012

  80. [88]

    Towards learning hierarchical skills for multi-phase manipulation tasks

    O. Kroemer, C. Daniel, G. Neumann, H. Van Hoof, and J. Peters. “Towards learning hierarchical skills for multi-phase manipulation tasks”. In:2015 IEEE international conference on robotics and automation (ICRA). IEEE. 2015, pp. 1503–1510

    2015

  81. [89]

    Trajectories and keyframes for kinesthetic teaching: A human-robot interaction perspective

    B. Akgun, M. Cakmak, J. W. Yoo, and A. L. Thomaz. “Trajectories and keyframes for kinesthetic teaching: A human-robot interaction perspective”. In:Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction. 2012, pp. 391–398

    2012

  82. [90]

    C-learn: Learning geometric constraints from demonstrations for multi-step manipulation in shared autonomy

    C. Pérez-D’Arpino and J. A. Shah. “C-learn: Learning geometric constraints from demonstrations for multi-step manipulation in shared autonomy”. In:2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE. 2017, pp. 4058– 4065

    2017

  83. [91]

    Robot learning from demon- stration by constructing skill trees

    G. Konidaris, S. Kuindersma, R. Grupen, and A. Barto. “Robot learning from demon- stration by constructing skill trees”. In:The International Journal of Robotics Research 31.3 (2012), pp. 360–375

    2012

  84. [92]

    Incremental Seman- tically Grounded Learning from Demonstration

    S. Niekum, S. Chitta, A. G. Barto, B. Marthi, and S. Osentoski. “Incremental Seman- tically Grounded Learning from Demonstration.” In:Robotics: Science and Systems. Vol. 9. Berlin, Germany. 2013, pp. 10–15607

    2013

  85. [93]

    Multi-level discovery of deep options

    R. Fox, S. Krishnan, I. Stoica, and K. Goldberg. “Multi-level discovery of deep options”. In:arXiv preprint arXiv:1703.08294(2017)

    2017 arXiv

  86. [94]

    N. B. Figueroa Fernandez.From High-Level to Low-Level Robot Learning of Complex Tasks: Leveraging Priors, Metrics and Dynamical Systems. Tech. rep. EPFL, 2019

    2019

  87. [95]

    Dynamic Movement Primitives in Robotics: A Tutorial Survey

    M. Saveriano, F. J. Abu-Dakka, A. Kramberger, and L. Peternel. “Dynamic Movement Primitives in Robotics: A Tutorial Survey”. In:CoRRabs/2102.03861 (2021). arXiv: 2102.03861.url: https://arxiv.org/abs/2102.03861

    2021 arXiv

  88. [96]

    Billard, S

    A. Billard, S. S. Mirrazavi Salehian, and N. Figueroa.Learning for Adaptive and Reactive Robot Control: A Dynamical Systems Approach. Cambridge, USA: MIT Press, 2022

    2022

  89. [97]

    A. J. Van Der Schaft and J. M. Schumacher.An introduction to hybrid dynamical systems. Vol. 251. Springer London, 2000

    2000

  90. [98]

    Discrete abstractions of hybrid systems

    R. Alur, T. A. Henzinger, G. Lafferriere, and G. J. Pappas. “Discrete abstractions of hybrid systems”. In:Proceedings of the IEEE88.7 (2000), pp. 971–984. 91

    2000

  91. [99]

    Temporal logic motion planning for mobile robots

    G. E. Fainekos, H. Kress-Gazit, and G. J. Pappas. “Temporal logic motion planning for mobile robots”. In:Proceedings of the 2005 IEEE International Conference on Robotics and Automation. IEEE. 2005, pp. 2020–2025

    2005

  92. [100]

    Bayesian inference of temporal task specifications from demonstrations

    A. J. Shah, P. Kamath, S. Li, and J. A. Shah. “Bayesian inference of temporal task specifications from demonstrations”. In: (2018)

    2018

  93. [101]

    Synthesis of reactive (1) designs

    N. Piterman, A. Pnueli, and Y. Sa’ar. “Synthesis of reactive (1) designs”. In:In- ternational Workshop on Verification, Model Checking, and Abstract Interpretation. Springer. 2006, pp. 364–380

    2006

  94. [102]

    Temporal-logic-based reactive mission and motion planning

    H. Kress-Gazit, G. E. Fainekos, and G. J. Pappas. “Temporal-logic-based reactive mission and motion planning”. In:IEEE transactions on robotics25.6 (2009), pp. 1370– 1381

    2009

  95. [103]

    Spot 2.0 — a framework for LTL andω-automata manipulation

    A. Duret-Lutz, A. Lewkowicz, A. Fauchille, T. Michaud, E. Renault, and L. Xu. “Spot 2.0 — a framework for LTL andω-automata manipulation”. In:Proceedings of the 14th International Symposium on Automated Technology for Verification and Analysis (ATVA’16). Vol. 9938. Lecture Not...

    2016 doi

  96. [104]

    A dynamical system approach to realtime obstacle avoidance

    S. M. Khansari-Zadeh and A. Billard. “A dynamical system approach to realtime obstacle avoidance”. In:Autonomous Robots32.4 (2012), pp. 433–454

    2012

  97. [105]

    Avoidance of convex and concave obstacles with convergence ensured through contraction

    L. Huber, A. Billard, and J.-J. Slotine. “Avoidance of convex and concave obstacles with convergence ensured through contraction”. In:IEEE Robotics and Automation Letters4.2 (2019), pp. 1462–1469

    2019

  98. [106]

    S. Boyd, S. P. Boyd, and L. Vandenberghe.Convex optimization. Cambridge university press, 2004

    2004

  99. [107]

    Learningtemporallogicformulasfromsuboptimal demonstrations: theory and experiments

    G.Chou,N.Ozay,andD.Berenson.“Learningtemporallogicformulasfromsuboptimal demonstrations: theory and experiments”. In:Autonomous Robots(2021), pp. 1–26

    2021

  100. [108]

    Symbolic planning and control of robot motion [grand challenges of robotics]

    C. Belta, A. Bicchi, M. Egerstedt, E. Frazzoli, E. Klavins, and G. J. Pappas. “Symbolic planning and control of robot motion [grand challenges of robotics]”. In:IEEE Robotics & Automation Magazine14.1 (2007), pp. 61–70

    2007

  101. [109]

    Automaton-guided controller synthesis for nonlinear systems with temporal logic

    E. M. Wolff, U. Topcu, and R. M. Murray. “Automaton-guided controller synthesis for nonlinear systems with temporal logic”. In:2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE. 2013, pp. 4332–4339

    2013

  102. [110]

    Motion planning with temporal-logic specifications: Progress and challenges

    E. Plaku and S. Karaman. “Motion planning with temporal-logic specifications: Progress and challenges”. In:AI communications29.1 (2016), pp. 151–162

    2016

  103. [111]

    Synthesis of nonlinear continuous controllers for verifiably correct high-level, reactive behaviors

    J. A. DeCastro and H. Kress-Gazit. “Synthesis of nonlinear continuous controllers for verifiably correct high-level, reactive behaviors”. In:The International Journal of Robotics Research34.3 (2015), pp. 378–394

    2015

  104. [112]

    Temporal logic motion planning in unknown environments

    A. M. Ayala, S. B. Andersson, and C. Belta. “Temporal logic motion planning in unknown environments”. In:2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE. 2013, pp. 5279–5284. 92

    2013

  105. [113]

    Iterative temporal planning in uncertain environments with partial satisfaction guarantees

    M. Lahijanian, M. R. Maly, D. Fried, L. E. Kavraki, H. Kress-Gazit, and M. Y. Vardi. “Iterative temporal planning in uncertain environments with partial satisfaction guarantees”. In:IEEE Transactions on Robotics32.3 (2016), pp. 583–599

    2016

  106. [114]

    Elaborating on learned demonstrations with temporal logic specifications

    C. Innes and S. Ramamoorthy. “Elaborating on learned demonstrations with temporal logic specifications”. In:arXiv preprint arXiv:2002.00784(2020)

    2020 arXiv

  107. [115]

    Learning From Demonstrations Using Signal Temporal Logic in Stochastic and Continuous Domains

    A. G. Puranic, J. V. Deshmukh, and S. Nikolaidis. “Learning From Demonstrations Using Signal Temporal Logic in Stochastic and Continuous Domains”. In:IEEE Robotics and Automation Letters6.4 (2021), pp. 6250–6257

    2021

  108. [116]

    Reactive Task and Motion Planning under Temporal Logic Specifications

    S. Li, D. Park, Y. Sung, J. A. Shah, and N. Roy. “Reactive Task and Motion Planning under Temporal Logic Specifications”. In:arXiv preprint arXiv:2103.14464(2021)

    2021 arXiv

  109. [117]

    Neural learning of stable dynamical sys- tems based on data-driven lyapunov candidates

    K. Neumann, A. Lemme, and J. J. Steil. “Neural learning of stable dynamical sys- tems based on data-driven lyapunov candidates”. In:2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE. 2013, pp. 1216–1222

    2013

  110. [118]

    Safe nonlinear control using robust neural lyapunov-barrier functions

    C. Dawson, Z. Qin, S. Gao, and C. Fan. “Safe nonlinear control using robust neural lyapunov-barrier functions”. In:Conference on Robot Learning. PMLR. 2022, pp. 1724– 1735

    2022

  111. [119]

    Learning Partially Contracting Dynam- ical Systems from Demonstrations

    H. C. Ravichandar, I. Salehi, and A. P. Dani. “Learning Partially Contracting Dynam- ical Systems from Demonstrations.” In:CoRL. 2017, pp. 369–378

    2017

  112. [120]

    Learning reactive motion policies in multiple task spaces from human demonstrations

    M. A. Rana, A. Li, H. Ravichandar, M. Mukadam, S. Chernova, D. Fox, B. Boots, and N. Ratliff. “Learning reactive motion policies in multiple task spaces from human demonstrations”. In:Conference on Robot Learning. PMLR. 2020, pp. 1457–1468

    2020

  113. [121]

    Learning control barrier functions from expert demonstrations

    A. Robey, H. Hu, L. Lindemann, H. Zhang, D. V. Dimarogonas, S. Tu, and N. Matni. “Learning control barrier functions from expert demonstrations”. In:2020 59th IEEE Conference on Decision and Control (CDC). IEEE. 2020, pp. 3717–3724

    2020

  114. [122]

    Learning barrier functions for constrained motion planning with dynamical systems

    M. Saveriano and D. Lee. “Learning barrier functions for constrained motion planning with dynamical systems”. In:2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE. 2019, pp. 112–119

    2019

  115. [123]

    Safe Control with Learned Certificates: A Sur- vey of Neural Lyapunov, Barrier, and Contraction methods

    C. Dawson, S. Gao, and C. Fan. “Safe Control with Learned Certificates: A Sur- vey of Neural Lyapunov, Barrier, and Contraction methods”. In:arXiv preprint arXiv:2202.11762(2022)

    2022 arXiv

  116. [124]

    Demonstration-Guided motion planning

    G. Ye and R. Alterovitz. “Demonstration-Guided motion planning”. In:Robotics research. Springer, 2017, pp. 291–307

    2017

  117. [125]

    Closed-loop global motion planning for reactive execution of learned tasks

    C. Bowen and R. Alterovitz. “Closed-loop global motion planning for reactive execution of learned tasks”. In:2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE. 2014, pp. 1754–1760

    2014

  118. [126]

    Learning complex dexterous manipulation with deep reinforcement learning and demonstrations

    A. Rajeswaran, V. Kumar, A. Gupta, G. Vezzani, J. Schulman, E. Todorov, and S. Levine. “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations”. In:arXiv preprint arXiv:1709.10087(2017)

    2017 arXiv

  119. [127]

    LLM+ P: Empowering Large Language Models with Optimal Planning Proficiency

    B. Liu, Y. Jiang, X. Zhang, Q. Liu, S. Zhang, J. Biswas, and P. Stone. “LLM+ P: Empowering Large Language Models with Optimal Planning Proficiency”. In: arXiv:2304.11477(2023). 93

    2023 arXiv

  120. [128]

    Do as I Can, Not as I Say: Grounding Language in Robotic Affordances

    M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakrishnan, K. Hausman, et al. “Do as I Can, Not as I Say: Grounding Language in Robotic Affordances”. In:arXiv:2204.01691(2022)

    2022 arXiv

  121. [129]

    Voyager: An Open-Ended Embodied Agent with Large Language Models

    G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandku- mar. “Voyager: An Open-Ended Embodied Agent with Large Language Models”. In: arXiv:2305.16291(2023)

    2023 arXiv

  122. [130]

    Text2motion: From natural language instructions to feasible plans

    K. Lin, C. Agia, T. Migimatsu, M. Pavone, and J. Bohg. “Text2motion: From natural language instructions to feasible plans”. In:Autonomous Robots47.8 (2023), pp. 1345– 1365

    2023

  123. [131]

    M. T. Mason.Mechanics of robotic manipulation. MIT press, 2001

    2001

  124. [132]

    Explanation-based learning: An alternative view

    G. DeJong and R. Mooney. “Explanation-based learning: An alternative view”. In: Machine learning1 (1986), pp. 145–176

    1986

  125. [133]

    Explanation-based manipulator learning: Acquisition of planning ability through observation

    A. Segre and G. DeJong. “Explanation-based manipulator learning: Acquisition of planning ability through observation”. In:Proceedings. 1985 IEEE International Conference on Robotics and Automation. Vol. 2. IEEE. 1985, pp. 555–560

    1985

  126. [134]

    Low-cost exoskeletons for learning whole-arm manipulation in the wild

    H. Fang, H.-S. Fang, Y. Wang, J. Ren, J. Chen, R. Zhang, W. Wang, and C. Lu. “Low-cost exoskeletons for learning whole-arm manipulation in the wild”. In:arXiv preprint arXiv:2309.14975(2023)

    2023 arXiv

  127. [135]

    Gello:Ageneral,low-cost,andintuitive teleoperation framework for robot manipulators

    P.Wu,Y.Shentu,Z.Yi,X.Lin,andP.Abbeel.“Gello:Ageneral,low-cost,andintuitive teleoperation framework for robot manipulators”. In:arXiv preprint arXiv:2309.13037 (2023)

    2023 arXiv

  128. [136]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

    C. Chi, Z. Xu, C. Pan, E. Cousineau, B. Burchfiel, S. Feng, R. Tedrake, and S. Song. “Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots”. In:arXiv preprint arXiv:2402.10329(2024)

    2024 arXiv

  129. [137]

    Segment anything

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, et al. “Segment anything”. In:Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023, pp. 4015–4026

    2023

  130. [138]

    Voxposer: Composable 3d value maps for robotic manipulation with language models

    W. Huang, C. Wang, R. Zhang, Y. Li, J. Wu, and L. Fei-Fei. “Voxposer: Composable 3d value maps for robotic manipulation with language models”. In:arXiv preprint arXiv:2307.05973(2023)

    2023 arXiv

  131. [139]

    Tedrake.Underactuated Robotics

    R. Tedrake.Underactuated Robotics. Algorithms for Walking, Running, Swimming, Flying, and Manipulation. 2023.url: https://underactuated.csail.mit.edu

    2023

  132. [140]

    Rapidly-Exploring Random Trees: A New Tool for Path Planning

    S. LaValle. “Rapidly-Exploring Random Trees: A New Tool for Path Planning”. In: Research Report 9811(1998)

    1998

  133. [141]

    robosuite: A modular simulation framework and benchmark for robot learning

    Y. Zhu, J. Wong, A. Mandlekar, R. Martín-Martín, A. Joshi, S. Nasiriany, and Y. Zhu. “robosuite: A modular simulation framework and benchmark for robot learning”. In: arXiv preprint arXiv:2009.12293(2020)

    2020 arXiv

  134. [142]

    Modular Networks for Compositional Instruction Following

    R. Corona, D. Fried, C. Devin, D. Klein, and T. Darrell. “Modular Networks for Compositional Instruction Following”. In:NAACL-HLT. 2021. 94

    2021

  135. [143]

    Modular Multitask Reinforcement Learning with Policy Sketches

    J. Andreas, D. Klein, and S. Levine. “Modular Multitask Reinforcement Learning with Policy Sketches”. In:ICML. 2017

    2017

  136. [144]

    Alignment-Based Compositional Semantics for Instruction Following

    J. Andreas and D. Klein. “Alignment-Based Compositional Semantics for Instruction Following”. In:EMNLP. 2015

    2015

  137. [145]

    Language as an Abstraction for Hierarchical Deep Reinforcement Learning

    Y. Jiang, S. S. Gu, K. P. Murphy, and C. Finn. “Language as an Abstraction for Hierarchical Deep Reinforcement Learning”. In:NeurIPS. 2019

    2019

  138. [146]

    Skill Induction and Planning with Latent Language

    P. Sharma, A. Torralba, and J. Andreas. “Skill Induction and Planning with Latent Language”. In:ACL. 2022

    2022

  139. [147]

    Learning Rational Subgoals from Demonstrations and Instructions

    Z. Luo, J. Mao, J. Wu, T. Lozano-Pérez, J. B. Tenenbaum, and L. P. Kaelbling. “Learning Rational Subgoals from Demonstrations and Instructions”. In:AAAI. 2023

    2023

  140. [148]

    Program guided agent

    S.-H. Sun, T.-L. Wu, and J. J. Lim. “Program guided agent”. In:ICLR. 2020

    2020

  141. [149]

    Bradley, A

    C. Bradley, A. Pacheck, G. J. Stein, S. Castro, H. Kress-Gazit, and N. Roy.Learning and Planning for Temporally Extended Tasks in Unknown Environments. 2021. arXiv: 2104.10636[cs.RO]

    2021 arXiv

  142. [150]

    Teaching Multiple Tasks to an RL Agent Using LTL

    R. Toro Icarte, T. Q. Klassen, R. Valenzano, and S. A. McIlraith. “Teaching Multiple Tasks to an RL Agent Using LTL”. In:AAMAS. 2018

    2018

  143. [151]

    Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation

    S. Tellex, T. Kollar, S. Dickerson, M. Walter, A. Banerjee, S. Teller, and N. Roy. “Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation”. In:AAAI. 2011

    2011

  144. [152]

    Instance-based counterfactual explanations for time series classification

    E. Delaney, D. Greene, and M. T. Keane. “Instance-based counterfactual explanations for time series classification”. In:International Conference on Case-Based Reasoning. Springer. 2021, pp. 32–47

    2021

  145. [153]

    Visual pre-training for navigation: What can we learn from noise?

    Y. Wang, C.-Y. Ko, and P. Agrawal. “Visual pre-training for navigation: What can we learn from noise?” In:arXiv preprint arXiv:2207.00052(2022)

    2022 arXiv

  146. [154]

    Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents

    W. Huang, P. Abbeel, D. Pathak, and I. Mordatch. “Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents”. In:ICML. 2022

    2022

  147. [155]

    Pre-trained language models for interactive decision-making

    S. Li, X. Puig, C. Paxton, Y. Du, C. Wang, L. Fan, T. Chen, D.-A. Huang, E. Akyürek, A. Anandkumar, et al. “Pre-trained language models for interactive decision-making”. In:Advances in Neural Information Processing Systems35 (2022), pp. 31199–31212

    2022

  148. [156]

    Language conditioned imitation learning over unstructured data

    C. Lynch and P. Sermanet. “Language conditioned imitation learning over unstructured data”. In:arXiv preprint arXiv:2005.07648(2020)

    2020 arXiv

  149. [157]

    LISA: Learning interpretable skill abstractions from language

    D. Garg, S. Vaidyanath, K. Kim, J. Song, and S. Ermon. “LISA: Learning interpretable skill abstractions from language”. In:Advances in Neural Information Processing Systems35 (2022), pp. 21711–21724

    2022

  150. [158]

    Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning

    R. M. Byrne. “Counterfactuals in Explainable Artificial Intelligence (XAI): Evidence from Human Reasoning.” In:IJCAI. 2019, pp. 6276–6282

    2019

  151. [159]

    A survey of algorithmic recourse: definitions,formulations,solutions,andprospects

    A.-H. Karimi, G. Barthe, B. Schölkopf, and I. Valera. “A survey of algorithmic recourse: definitions,formulations,solutions,andprospects”.In:arXiv preprint arXiv:2010.04050 (2020). 95

    2020 arXiv

  152. [160]

    Versatile Demonstration Interface: Toward More Flexible Robot Demonstration Collection

    M. Hagenow, D. Kontogiorgos, Y. Wang, and J. Shah. “Versatile Demonstration Interface: Toward More Flexible Robot Demonstration Collection”. In:arXiv preprint arXiv:2410.19141(2024)

    2024 arXiv

  153. [161]

    Towards online adapta- tion for autonomous household assistants

    B. A. Newman, C. J. Paxton, K. Kitani, and H. Admoni. “Towards online adapta- tion for autonomous household assistants”. In:Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction. 2023, pp. 506–510

    2023

  154. [162]

    Towards online learning from corrective demonstrations

    R. A. Gutierrez, E. S. Short, S. Niekum, and A. L. Thomaz. “Towards online learning from corrective demonstrations”. In:arXiv preprint arXiv:1810.01036(2018)

    2018 arXiv

  155. [163]

    TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learning

    J.Wu,W.Chong,R.Holmberg,A.Prasad,Y.Gao,O.Khatib,S.Song,S.Rusinkiewicz, and J. Bohg. “TidyBot++: An Open-Source Holonomic Mobile Manipulator for Robot Learning”. In:arXiv preprint arXiv:2412.10447(2024)

    2024 arXiv

  156. [164]

    Languagemodelsarefew-shotlearners

    T.B.Brown.“Languagemodelsarefew-shotlearners”.In:arXiv preprint arXiv:2005.14165 (2020)

    2020 arXiv

  157. [165]

    Learning transferable visual models from natural language supervision

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. “Learning transferable visual models from natural language supervision”. In:International conference on machine learning. PmLR. 2021, pp. 8748–8763

    2021

  158. [166]

    The bitter lesson

    R. Sutton. “The bitter lesson”. In:Incomplete Ideas (blog)13.1 (2019), p. 38

    2019

  159. [167]

    π0: A Vision-Language-Action Flow Model for General Robot Control

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. “π0: A Vision-Language-Action Flow Model for General Robot Control”. In:arXiv preprint arXiv:2410.24164(2024)

    2024 arXiv

  160. [168]

    Gemini robotics: Bringing ai into the physical world

    G. R. Team, S. Abeyruwan, J. Ainslie, J.-B. Alayrac, M. G. Arenas, T. Armstrong, A. Balakrishna, R. Baruch, M. Bauza, M. Blokzijl, et al. “Gemini robotics: Bringing ai into the physical world”. In:arXiv preprint arXiv:2503.20020(2025)

    2025 arXiv

  161. [169]

    Interpretable apprenticeship learning with temporal logic specifications

    D. Kasenberg and M. Scheutz. “Interpretable apprenticeship learning with temporal logic specifications”. In:2017 IEEE 56th Annual Conference on Decision and Control (CDC). IEEE. 2017, pp. 4914–4921. 96

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.