Towards Human-level Dexterity via Robot Learning

Gagan Khandate

REVIEW 3 major objections 9 minor 177 references

Towards Human-level Dexterity via Robot Learning

T0 review · 3 major / 9 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A stability-checked planning tree supplies the resets and pre-training data that let reinforcement learning teach a robotic hand to finger-gait using touch alone.

desk verdict The reset-distribution core is real and the hardware results are a genuine step forward, but the main simulation evidence is confounded with reset-state evaluation and the headline claims outrun the evidence. read the letter →

arxiv 2507.09117 v1 pith:7RWVNTP2 submitted 2025-07-12 cs.RO cs.AI

Gagan Khandate This is my paper · ORCID

classification cs.ROcs.AI

keywords dexterousmanipulationfinger-gaitingreinforcementlearningsampling-basedplanningresetdistributionimitationpre-trainingtactilesensingin-hand

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The thesis argues that the main obstacle to learning dexterous in-hand manipulation with reinforcement learning is exploration: random action noise almost always drops the object before it discovers the narrow set of stable fingertip grasps that make finger-gaiting possible. To fix this, it proposes structured exploration, in which a non-holonomic Rapidly-exploring Random Tree (G-RRT) is grown in simulation and only states that pass a stability check are added, and the resulting tree is used twice—its nodes become the reset distribution for policy-gradient training, and its transitions supply action labels for imitation pre-training. The author claims this is the first demonstration that sampling-based-planning reset distributions enable more efficient RL for dexterous in-hand manipulation, and that the resulting policies finger-gait both convex and non-convex objects (sphere, cube, cuboid, L-shape) on real hardware using only tactile and proprioceptive sensing. If this is right, the hard exploration problem in dexterous manipulation can be largely bypassed by a generic planner plus a stability oracle, rather than by hand-designed rewards or millions of teleoperated demonstrations.

What carries the argument

The load-bearing object is G-RRT, a non-holonomic Rapidly-exploring Random Tree that explores by sampling random actions, simulating them with a physics model, and admitting a new node only if the object remains grasped and undropped after a settling interval—the only manipulation-specific component. From this tree the method extracts paths with the largest object rotation (for finger-gaiting) or paths from the root to displaced nodes (for the go-to-root task), uses the union of nodes as a uniform reset distribution for reinforcement learning, and converts consecutive states into action labels $a_k = \beta(q_{k+1} - q_k)$ for imitation pre-training of the actor, with the critic pre-trained on rollouts of that imitative policy. Around this core sit the hand-centric decomposition of arbitrary reorientation into rotations about cardinal axes, a reward based on angular velocity about the target axis when at least three fingertips are in contact, and an asymmetric actor-critic PPO that gives the critic object pose but the policy only proprioception and binary touch.

What would settle it

Train the R×R method on a hard L-shape with a deliberately weakened planner—for example, pruning all tree paths that break a specific finger contact—and check whether the policy still learns to gait. The paper's own ablations predict near-zero learning below roughly $10^4$ tree nodes, so an even sharper falsifier would be an object where a $10^5$-node tree of stable states gives no improvement over a random stable-grasp reset distribution, indicating that manifold coverage is not the active mechanism.

Watch

Extended reading notes

Core claim

The central claim is that sampling-based planning can chart the useful part of the state space for dexterous manipulation—the manifold of stable precision grasps—and that this map is exactly what reinforcement learning needs to explore efficiently. Concretely, G-RRT adds a node only if, after a random action is played out in simulation, the object is still held after a two-second settling interval; this single stability check, with no other manipulation-specific guidance, grows a tree along non-holonomic transitions such as breaking and re-making finger contacts. The paper shows that starting every training episode from a node on a high-rotation path through this tree, together with pre-training the actor on the scaled joint-angle differences between consecutive nodes, lets policy-gradient RL learn finger-gaiting for spheres, cubes, cuboids, and concave L- and U-shapes in tens of millions of steps, and that these policies transfer to a real 15-degree-of-freedom hand with only a curriculum of motor limits, latency, friction, and perturbation forces. The thesis positions this as the first example of sampling-based-planning reset distributions enabling efficient RL for dexterous in-hand manipulation, and the first finger-gaiting of non-convex shapes with purely intrinsic sensing.

Load-bearing premise

The load-bearing premise is that G-RRT, guided only by a stability check, grows a tree dense enough in the stable-manifold state space that its nodes and transitions transfer useful exploration to reinforcement learning; if the tree misses critical regions, the method degrades to random RL with resets.

Editorial extensions

If this is right

Training times drop from billions to tens of millions of environment steps on the hard manipulation tasks, because the reset distribution guarantees the agent repeatedly starts in states that random exploration would rarely reach.
Policies relying only on joint positions, setpoints, and binary touch transfer to real hardware without dense camera tracking, and keep working when lighting changes or the palm faces down.
Non-convex objects such as L-shapes, previously absent from in-hand manipulation demonstrations, become tractable for finger-gaiting in simulation and on the physical hand.
Imitation pre-training from quasi-static tree transitions speeds convergence, but only when coupled with the tree-based reset distribution; warm-starting from a fixed start state provides little benefit.
The same two-stage recipe—planner explores, reinforcement learning exploits—is proposed as a template for other hard-exploration motor tasks such as agile locomotion and bimanual manipulation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable corollary the thesis leaves implicit: the reset distribution carries most of the benefit, so replacing G-RRT with any planner that covers the stable manifold—for example a learned generative model of stable grasps—should reproduce much of the gain.
Because the stability check is a cheap 'object not dropped' oracle, the tree could in principle be grown on a real robot with automatic resets, turning the thesis's simulation-only assumption into a hardware bootstrapping procedure.
The thesis explicitly flags that simulated motor learning still faces sim-to-real and simulation-scaling costs (Section 6.5); a fair reading is that the exploration benefit must be large enough to justify that infrastructure expense.
The final chapter's visuo-tactile human demonstrations offer a complementary bootstrap: if human hand data can be mapped into a shared latent action space, combining that route with structured-exploration resets could further cut the number of robot demonstrations needed.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The reset-distribution idea is the real contribution here. Chapter 5's combination of G-RRT with RL, using tree states as resets and tree transitions for imitation pre-training, is a coherent and useful recipe, and the real-hardware finger-gaiting on cube, cuboid, and L-shape with only tactile and proprioceptive sensing is a genuine step forward. Credit is also due for the baselines and ablations: SGS, ER, FI, and GC are the right comparators, and the action-scale and tree-size ablations show the method is not magic. The reader's circularity burden is low, because the reset distribution is generated by a stability check and task-relevant path selection, not fitted to the final policy.

But the stress-test concern is real and central. In simulation, training and validation both use reset states sampled from the G-RRT tree. There is no natural-start evaluation reported for go-to-root or arbitrary reorientation, so the sample-efficiency gains may be an artifact of evaluating close to the tree's successful paths. The real-hardware trials do start from a natural grasp and support the finger-gaiting claim, but they cover only a subset of objects and report ten trials with a median, no error bars. For a thesis claiming "human-level dexterity" in the abstract, the evidence base is narrower than the language.

Other soft spots are minor by comparison. Chapter 6 (Diffusion Roadmaps) is explicitly preliminary, and Chapter 7 (visuo-tactile human demonstrations) shows attention maps but no downstream policy; both are honestly signposted as ongoing work, so they are not presented as finished results. The citation pattern is typical for a thesis: heavy reliance on the author's prior papers, which is defensible since those papers have independent support.

This is a thesis for readers working on exploration for contact-rich manipulation, and the core Chapter 5 deserves a serious referee if condensed into a paper. The authors should be pushed to report natural-start evaluation and proper error bars before the strong claims are accepted. I would not desk reject this; I would send it to review with a request for those additions.

Referee Report

3 major / 9 minor

Summary. This doctoral thesis develops learning methods for dexterous in-hand manipulation with a fully actuated 15-DoF five-fingered hand, using only intrinsic tactile and proprioceptive sensing. Chapter 3 introduces hand-centric decomposition of reorientation into cardinal-axis tasks and uses stable-grasp sampling (SGS) as an initial-state prior for PPO, achieving simulated finger-gaiting and finger-pivoting. Chapter 4 proposes value-guided exploration (VGE), interleaving simple sub-skill controllers with an off-policy learner to provide action priors. Chapter 5, the core contribution, uses a general non-holonomic RRT (G-RRT) with a stability check to grow a tree of stable fingertip states; task-relevant tree paths provide a reset distribution (R×R) and joint-position differences provide imitation pre-training action labels (R×R+IPT). The method learns finger-gaiting on easy, medium, and hard objects (including concave L- and U-shapes), as well as Go-to-root and Arbitrary Reorientation tasks in simulation, and finger-gaiting transfers to real hardware for cylinder, cube, cuboid, and L-shape objects. Chapters 6 and 7 present preliminary extensions: Diffusion Roadmaps with rapidly-exploring reinforcement learning, and a visuo-tactile human demonstration paradigm using a ViTacT encoder.

Significance. If the central claims hold, this is a meaningful contribution to dexterous manipulation. To my knowledge it is the first demonstration of learned finger-gaiting precision manipulation of concave objects with only intrinsic sensing, including on physical hardware. The core idea — converting sampling-based planning trees into RL reset distributions and pre-training data — is clean and well motivated, with appropriate grounding in reset-distribution analysis (ref [109]) and a strong baseline set (SGS, ER, FI, FI+IPT, GC). The thesis is commendably transparent: algorithmic pseudocode is given for G-RRT, R×R, and IPT; Chapter 5 training curves aggregate three seeds; ablations cover tree size, action scale α, Kmax, and feedback components; and Chapters 6–7 are explicitly labeled preliminary. The main caveat is that the simulation evaluation for all three Chapter 5 tasks is performed from the same reset-state distribution used for training, so the quantitative gains over baselines require a natural-start evaluation to be fully established; the hardware results partially offset this for finger-gaiting but not for the reorientation tasks.

major comments (3)

[Sec 5.6.1, Fig 5.14, Alg 5] The simulation evaluation for all three tasks is confounded with the training reset distribution. In Alg 5 (line 21), every training rollout starts from x0 drawn from D, where D is built from task-relevant G-RRT tree paths, and Sec 5.6.1 reports training curves only. Fig 5.14's caption is explicit: 'Validation performance is with reset states from paths extracted from the largest tree.' Thus both the reported training returns (Figs 5.3, 5.5, 5.6, 5.8) and the reported validation use the same state distribution on which the policy was trained, so the curves measure competence on reset states rather than on the tasks' natural initial distributions. The reset distribution itself is not circular (G-RRT is task-agnostic and path selection precedes RL), but the evaluation protocol is. For the paper's central claim that SBP-generated reset distributions enable efficient acquisition of finger-gaiting, Go-to-root, and Arbitrary Reorientation, a simulation evaluation from natural starts is needed: e.g., a fixed canonical grasp for finger-gaiting and a uniform or held-out initial pose distribution for the reorientation tasks. The real-hardware trials (Sec 5.6.2) do start from natural grasps and partially support the finger-gaiting claim, but they cover none of the reorientation tasks, leaving the sample-efficiency advantage over baselines potentially attributable in part to the evaluation distribution.
[Sec 5.6.2, Table 5.1] The hardware evidence underlying the headline claim is reported as a single median over ten consecutive trials per object, with no per-trial distribution, no inter-quartile range, and no statement of the trial protocol (maximum trial duration, definition of a drop, whether the initial grasp was standardized, and how the 'time needed to perform these rotations' was measured across trials of heterogeneous length). For cuboid and L-shape the median is 1.5 revolutions, and a median of 1.5 over ten trials is compatible with a wide range of per-trial outcomes. Please report the full per-trial results (e.g., each trial's revolution count or min/max) and the protocol details; this table is the primary independent support for the claim that finger-gaiting transfers to hardware for non-convex objects.
[Ch 1 contribution bullet 4; Sec 7.7] The fourth contribution bullet (Chapter 1) states that the visuo-tactile demonstration paradigm introduces a 'first-of-its-kind method' and 'sets a new standard for imitation learning in dexterous manipulation,' but Chapter 7's experimental content contains no policy learning or transfer results: the only evaluation is attention-map visualization on a pick task, and Sec 7.7.2 concludes with 'We hope to achieve this in future work.' As submitted, the experimental evidence does not substantiate the contribution as stated. Either temper the contribution claim to match the preliminary status of Chapter 7, or include at least preliminary fine-tuning/imitation results on the robot demonstrations; note that 20 robot and 500 human demonstrations are collected but no downstream result is reported.

minor comments (9)

[Sec 1.7] Section 1.7 states that 'Chapter 1 and Chapter 2 present two studies' on state and action priors, but these studies actually appear in Chapters 3 and 4; the thesis summary cross-references are misnumbered and should be corrected.
[Ch 2, opening paragraph] The opening paragraph of Chapter 2 contains empty in-text cross-references ('Section ,' 'Section ,') left over from the thesis compilation; these placeholders should be resolved or removed.
[Sec 6.3.2] Section 6.3.2 (Finger-gaiting In-hand Manipulation) appears as a heading with no accompanying text before Section 6.3.3; either add the intended content or delete the heading.
[Abstract] The abstract's phrase 'contrary to Moravec's paradox' appears to express the opposite of what is intended: Moravec's paradox already asserts that low-level sensorimotor skills are deceptively difficult, which is exactly the observation the thesis makes; the wording should be rephrased.
[Figs 5.3, 5.5, 5.6] The training curves are described as aggregated over three seeds but no variance information (error bars or shading) is shown; given that Chapter 4 reports training robustness across seeds, adding per-seed spread for the Chapter 5 headline comparisons would help assess reliability.
[Sec 5.6.2, final paragraph] The sentence 'These policies are identical to the policies used for sim-to-real transfer, as per visual comparison in simulation' is unclear: the preceding sentences say the transferred policies were trained with the version of G-RRT using the three-contact constraint, while the current method forgoes that constraint; clarify which tree version generated the deployed policies.
[Sec 5.4.2, Eq (5.1)] The imitation pre-training action scale β = 2 is a free parameter that is not ablated; given that the paper carefully ablates α, tree size, and Kmax, a small sensitivity study for β (or a justification of the chosen value) would make the IPT contribution self-contained.
[Sec 5.5.2, Sec 5.6.1] Several numeric values appear to have lost superscript formatting (e.g., '105 nodes,' '2 × 104 nodes,' '104 nodes'); please verify that these render as 10^5, 2 × 10^4, and 10^4 in the published version.
[Sec 5, contribution bullet 1] The phrase 'reset distributions generated via SBP with kinematic constraints' is imprecise for the current method: G-RRT (Alg 4) imposes a dynamic stability check (two seconds of no-drop simulation), and the three-contact kinematic constraint was explicitly discontinued in Sec 5.3; align the wording with the actual algorithm.

Circularity Check

1 steps flagged · score 6.0 of 10

Simulation validation for Go-to-root is performed on the same reset states used for training and imitation pre-training, so those reported gains are partly by construction; real-hardware finger-gaiting remains independent.

fitted input called prediction [Sec 5.5.1 (Go-to-root), Sec 5.6.1 (Fig. 5.14 caption), Alg. 5 line 21]
""Validation performance is with reset states from paths extracted from the largest tree." (Fig. 5.14 caption) ... "Collect rollout τ with πθ from initial state x0 where x0 ∼ D the buffer of best states from sampling-based planning." (Alg. 5, line 21)"

For Go-to-root, the tree root is the desired canonical orientation and the reset distribution is built by backtracking from nodes with large displacement to that root. Validation is then performed on reset states from the same tree, i.e. the exact distribution used for training and for imitation pre-training, whose action labels come from the same paths (Eq. 5.1: ak = β(qk+1 − qk)). Success from such states is partly forced: the policy is initialized and trained on transitions that connect these states toward the goal, so the reported success/sample-efficiency on these states does not demonstrate acquisition from a natural initial state.

full rationale

Most of the derivation chain is self-contained and not circular. The G-RRT planner (Alg. 4) uses only a stability check and random actions, and the reset distribution is built from the resulting tree rather than fitted to the final policy; the comparison against SGS and other reset baselines is meaningful. The main circularity is in the simulation evaluation protocol: the paper validates final performance using reset states drawn from the same tree paths that define the training reset distribution and the imitation pre-training data. For Go-to-root, where the tree root is the goal and paths are extracted by backtracking toward the root, evaluation on these states is partly a restatement of the training data. The paper explicitly writes that validation performance is measured on reset states from the largest tree, and no natural-start simulation evaluation is provided for Go-to-root or Arbitrary Reorientation. The real-hand finger-gaiting results (Table 5.1) start from natural grasps, so the central hardware claim is not circular. Self-citations to prior work [23, 93] describe methods that are re-derived in the thesis and are not load-bearing for the main result. Overall score reflects partial circularity in the simulation predictions, not a fully forced derivation.

Assumptions & free parameters 5 free parameters · 5 assumptions · 1 invented entities

The free parameters are hand-tuned hyperparameters of the planner, reward, and training (reward thresholds, action scale, imitation scaling, tree size, KL coefficients). The axioms are standard RL/planning assumptions plus domain assumptions about simulator fidelity and reward design. ViTacT is an invented architecture with no external validation beyond attention visualization.

free parameters (5)

r_max, phi_max (reward clipping and axis-alignment threshold) = 0.5, 0.5
Set by hand in Chapter 3 (Eq 3.2) to shape the reward and avoid local optima; the paper does not provide a sensitivity analysis.
G-RRT action scale alpha = 0.15
Tuned via ablation (Fig 5.13); the paper reports alpha = 0.15 as optimal. This is a free parameter of the planning algorithm.
Imitation pre-training action scaling beta = 2
Chosen in Sec 5.4.2 to convert joint-position differences between tree nodes into action labels (Eq 5.1).
Tree size and number of reset states = 1e5 nodes; 2e4 reset states
Determined empirically via ablation (Fig 5.14); the thesis reports larger trees improve performance but no principled selection method.
KL regularization coefficients beta1, beta2 = not specified
In Eq 7.3 (Chapter 7) these balance latent state/action distribution regularization against the prediction loss; values are not stated.

assumptions (5)

domain assumption MuJoCo and IsaacGym soft-contact simulations accurately model the real hand, fingertips, and object dynamics for training and sim-to-real transfer.
Used throughout Chapters 3 and 5; policies are trained in these simulators and transferred to hardware with domain randomization.
ad hoc to paper The stability check in G-RRT (2 seconds without the object dropping) is a valid proxy for states that can serve as resets for RL.
Sec 5.3, line 8; the check determines which nodes enter the tree and thus the reset distribution.
domain assumption The reward functions (angular velocity about an axis, contact count constraints, success heuristics) are appropriate and do not induce degenerate policies.
Eq 3.2, 5.4 and task definitions in Sec 5.5.1.
ad hoc to paper Action labels for imitation pre-training can be obtained by scaling joint-position differences between tree states by beta=2 (Eq 5.1).
Sec 5.4.2; assumes quasi-static transitions are approximately valid demonstrations.
domain assumption A shared latent action space can be learned across human and robot embodiments from observation-only human demonstrations plus a small set of robot demonstrations.
Chapter 7 formulates this as the core premise for cross-embodiment transfer; not validated beyond a pick task.

invented entities (1)

ViTacT (Visuo-tactile Transformer)
purpose: Encodes multi-view visual and tactile human demonstrations into a shared latent space for cross-embodiment imitation learning.
The thesis shows attention maps for a simple pick task (Sec 7.7.2) but no downstream dexterous policy, so the architecture has no falsifiable handle outside the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Human-level Dexterity via Robot Learning." pith.science (2026). https://pith.science/paper/7RWVNTP2

@misc{pith2026250709117,
  author       = {Pith},
  title        = {Pith review of: Towards Human-level Dexterity via Robot Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7RWVNTP2}},
  note         = {Machine review of arXiv:2507.09117}
}

read the original abstract

Dexterous intelligence -- the ability to perform complex interactions with multi-fingered hands -- is a pinnacle of human physical intelligence and emergent higher-order cognitive skills. However, contrary to Moravec's paradox, dexterous intelligence in humans appears simple only superficially. Many million years were spent co-evolving the human brain and hands including rich tactile sensing. Achieving human-level dexterity with robotic hands has long been a fundamental goal in robotics and represents a critical milestone toward general embodied intelligence. In this pursuit, computational sensorimotor learning has made significant progress, enabling feats such as arbitrary in-hand object reorientation. However, we observe that achieving higher levels of dexterity requires overcoming very fundamental limitations of computational sensorimotor learning. I develop robot learning methods for highly dexterous multi-fingered manipulation by directly addressing these limitations at their root cause. Chiefly, through key studies, this disseration progressively builds an effective framework for reinforcement learning of dexterous multi-fingered manipulation skills. These methods adopt structured exploration, effectively overcoming the limitations of random exploration in reinforcement learning. The insights gained culminate in a highly effective reinforcement learning that incorporates sampling-based planning for direct exploration. Additionally, this thesis explores a new paradigm of using visuo-tactile human demonstrations for dexterity, introducing corresponding imitation learning techniques.

Figures

Figures reproduced from arXiv: 2507.09117 by the authors.

**Figure 1.1.** Several representative robot learning skills are presented, categorized by the difficulty of collecting demonstrations and performing exploration. Learning multi-fingered dexterous manipulation is particularly challenging due to the extreme difficulty in both collecting demonstrations and achieving effective exploration, making it a complex problem for both imitation and reinforcement learning. 5 [PITH_FULL_IMAGE:f… view at source ↗

**Figure 3.1.** Hand-centric decomposition of in-hand re-orientation into re-orientation about cardinal axes. Our proposed method for large-angle arbitrary in-hand reorientation is thus to decompose the problem of achieving arbitrary angular velocity of the object into learning separate policies about the cardinal axes as shown in [PITH_FULL_IMAGE:figures/full_fig_p040_3_1.png] view at source ↗

**Figure 3.2.** Learning axis conditional continuous re-orientation kˆ. We use the component of angular velocity ω about kˆ as reward when the object is in a grasp with 3 or more fingertips, i.e nc ≥ 3. where m is the number of fingers. Our policies command set-point changes ∆qd. 3.3 Learning axis-specific re-orientation We now describe the procedure for learning in-hand reorientation policies for an arbitrary but fixed axis. Let k… view at source ↗

Figures from the paper (43 more)

**Figure 3.3.** Figure 3.3: (a) Sampling fingertips around the object. (b) Diverse relevant initial grasps sampled for efficient exploration. Algorithm 1 Stable Grasp Sampling (SGS) Input:ρobj , ρhand, ts, nc,min ▷ object pose distribution, hand pose distribution, simulation settling time, mini…

**Figure 3.4.** Figure 3.4: A learned finger-gaiting policy that can continuously re-orient the target object about the hand z-axis. The policy only uses sensing modalities intrinsic to the hand (such as touch and proprioception), and does not require explicit object pose information from exter…

**Figure 3.5.** Figure 3.5: Finger-gaiting and finger-pivoting our policies achieve to re-orient about z-axis and x-axis respectively. Key frames are shown for two objects, dodecahedron and cube. orientation via in-grasp manipulation and drop the object after maximum re-orientation achievable w…

**Figure 3.6.** Figure 3.6: Average returns for (a) z-axis re-orientation and (b) x-axis re-orientation. Learning with wide range of initial grasps sampled via SGS succeeds, while using a fixed initial state fails. PPO without the use of initial state distribution [PITH_FULL_IMAGE:figures/full…

**Figure 3.7.** Figure 3.7: Robustness of our policies with increasing sensor noise and perturbation forces on the object. 3.5.2 Generalization We study generalization properties of our policies by evaluating it on different objects in the object set. We consider the transfer score, which is th…

**Figure 3.8.** Figure 3.8: Cross transfer scores for policies with and without qd in feedback. 3.5.3 Observations on feedback While our work provides some insight w.r.t the important components of our feedback through our robustness and generalization results, many interesting questions remain…

**Figure 3.9.** Figure 3.9: Ablations holding out different components of feedback. For each experiment, color filled cells in the observation vector shown above the training curve indicate which of the components of the observation vector are provided to the policy. The key takeaway from these…

**Figure 4.1.** Figure 4.1: Our method of interweaving the policy and sub-skill controller (ex. contact switching controller) during training allows the policy to effectively learn dexterous finger-gaiting skills as shown. Videos can be found at project page: roamlab.github.io/vge In this work,…

**Figure 4.2.** Figure 4.2: Sub-skill controllers used for learning finger-gaiting: (left) In-grasp manipulation controller (right) Contact switching controller as described in Sec 4.1 . optimality and robustness is challenging for these controllers. Nevertheless, sub-optimal modelbased contro…

**Figure 4.3.** Figure 4.3: Training curves for using (a) In-grasp manipulation controller (b) Contact switching controller (c) Finger-gaiting controller over all the evaluation conditions listed in Sec 4.3.1. Our method that interleaves following controllers with policy (VGE, VGE + BC) learns …

**Figure 4.4.** Figure 4.4: (a) The 5-fingered, 15-dof dexterous hand grasping the cube we use in our experiments (b) Joint velocity profile of the joints of the finger making and breaking contact. It involves detach and attach phases. Note that the distal flexion joint is held fixed [PITH_FUL…

**Figure 4.5.** Figure 4.5: Training curve for multiple objects using finger-gaiting controller. t [PITH_FULL_IMAGE:figures/full_fig_p060_4_5.png]

**Figure 4.6.** Figure 4.6: Keyframes of the gaits achieved for multiple objects simultaneously with a single policy. • RL: Standard off-policy RL without the use of controllers and without any modification to the behavior policy or loss function is the first baseline. • RL + BC loss: We also t…

**Figure 4.7.** Figure 4.7: Average episode returns of a policy trained on the cube with increasing perturbation forces and noise evaluated separately as shown. The policy sustains 0.1rad of noise in joint positions q, up to 50% error in contact normal nm and a large error of 3mm in contact pos…

**Figure 5.1.** Figure 5.1: Our method illustrated with an abstract state-space consisting of narrow stable regions (beige) between large unstable regions (black). The proposed two-stage approach uses sampling-based planning to explore the challenging state-space and leverages the information w…

**Figure 5.2.** Figure 5.2: The object shapes for which we learn finger-gaiting. From left to right: the easy, medium and hard categories. we pre-train the critic network on rollouts of the imitation policy to avoid washing it out with a randomly initialized critic [173]. This method aligns wit…

**Figure 5.3.** Figure 5.3: Training performance of our methods and a number of baselines for the Finger-gaiting task on the object categories shown in [PITH_FULL_IMAGE:figures/full_fig_p081_5_3.png]

**Figure 5.4.** Figure 5.4: Key frames of the policies for the Finger-gaiting achieved with our method R×R for representative objects in simulation. 65 [PITH_FULL_IMAGE:figures/full_fig_p081_5_4.png]

**Figure 5.5.** Figure 5.5: Training performance of our methods and a number of baselines for the Finger-gaiting task on hard objects shown in [PITH_FULL_IMAGE:figures/full_fig_p082_5_5.png]

**Figure 5.6.** Figure 5.6: Training performance for the Go-to-root task with our methods and other best-performing baselines. 66 [PITH_FULL_IMAGE:figures/full_fig_p082_5_6.png]

**Figure 5.7.** Figure 5.7: Training performance for the Arbitrary Reorientation task comparing our methods (R×R, R×R + IPT) with Stable Grasp Sampler (SGS) baseline. 200 100 0 135° L 90° L avg episode reward 200 100 0 Long 90° L 0 Step 100M avg episode reward U 0 100M Step Ours, RxR Ours, RxR …

**Figure 5.8.** Figure 5.8: Training performance for the Go-to-root task with our methods and other best-performing baselines for objects in hard category. time Arbitrary Reorientation Go-to-root [PITH_FULL_IMAGE:figures/full_fig_p083_5_8.png]

**Figure 5.9.** Figure 5.9: Key frames of goal-reaching tasks, Go-to-root and Arbitrary Reorientation, in simulation. 67 [PITH_FULL_IMAGE:figures/full_fig_p083_5_9.png]

**Figure 5.10.** Figure 5.10: Key frames of the Finger-gaiting policies transferred to the real hand. time [PITH_FULL_IMAGE:figures/full_fig_p086_5_10.png]

**Figure 5.11.** Figure 5.11: Key frames of a Finger-gaiting policy transferred to the hand in the "palm-down" orientation. 70 [PITH_FULL_IMAGE:figures/full_fig_p086_5_11.png]

**Figure 5.12.** Figure 5.12: Tree expansion performance for G-RRT. We plot the number of attempted tree expansions (i.e. iterations through the main loop, on a log scale) against the maximum object z-axis rotation achieved by any tree node so far. We plot performance for different values of Kma…

**Figure 5.13.** Figure 5.13: G-RRT action-scale (α) ablation. α = 0.15 is optimal. from the tree contain increasingly more effective gaits, likely closer to the optimal policy. We see that we need a sufficiently large tree with at least 104 nodes to enable learning. However, training is most re…

**Figure 5.14.** Figure 5.14: The final performance of policies for Finger-gaiting (left) and Go-to-root (right) tasks using trees of increasing sizes. Validation performance is with reset states from paths extracted from the largest tree. Go-to-root (a) (b) Finger-gaiting actionscale actionscal…

**Figure 5.15.** Figure 5.15: Action scale ablation for the Finger-gaiting and Go-to-root tasks with L [PITH_FULL_IMAGE:figures/full_fig_p091_5_15.png]

**Figure 5.16.** Figure 5.16: Ablation of policy feedback components which highlights the importance of touch feedback. number of examples, are robust to variations in the encountered states. However, the standard RL exploration techniques (random perturbations in action space) are ineffective i…

**Figure 6.1.** Figure 6.1: Mazes used for 2D navigation where the task is to reach navigate between fixed start and randomly sampled goal positions. Mazes (a) through (c) are in increasing order of difficulty. L = L dif fusion + L Q−value = E k∼0...N−1 (xt,A0 t )∼W (ϵ k , ϵη(xt , A0 t + ϵ k , …

**Figure 6.2.** Figure 6.2: The rollouts achieved by our policy in mazes (a) through (c) are in increasing order of difficulty. 6.3.4 Results 2D Maze Navigation Our results show that RRL (DRM + DiffQL) outperforms all other baselines, achieving over a 90% success rate across all mazes, includin…

**Figure 6.3.** Figure 6.3: Angular rotation via Diffusion Roadmaps for cube for node merging thresholds ϵ. enable high degrees of reorientation. Training policies using Diffusion Roadmaps and RRL for finger-gaiting remains an exciting direction for future work. 6.4 Conclusion and ongoing work …

**Figure 7.1.** Figure 7.1: The key idea of our method is to learn from a large dataset consisting of human demonstrations alongside a small dataset of robot demonstrations by mapping them to a shared latent action space. and action representations from human demonstrations, specifically visuo-…

**Figure 7.2.** Figure 7.2: An illustration of the proposed method of collecting visuo-tactile human demonstrations. Multiple camera views patches and tactile sensor feedback can be encoded via visuo-tactile transformer (ViTacT). gies have been developed for tactile sensing, the ability to rend…

**Figure 7.3.** Figure 7.3: Latent actions achieved by learning forward dynamics in representation space achieved by minimizing the prediction error in the latent space - d(xt+1, ˆxt+1) where d is some similarity measure. We adopt model-based observation-only imitation learning and learn to ext…

**Figure 7.4.** Figure 7.4: Learning task conditioned latent policy ψ with supervision from inverse model h() where u denotes the task. In stage-2, ψ ′ (zt |ot , u) is trained to directly predict latent actions with supervision from inverse dynamics model h() obtained in the stage-1. Note that …

**Figure 7.5.** Figure 7.5: Learning robot action head g() with robot demonstrations DR In a final imitation stage, while freezing all other networks, the action head g() is trained to predict the true actions with imitation learning loss of choice. 7.4 Preliminaries: self-distillation for repr…

**Figure 7.6.** Figure 7.6: Learning forward dynamics via DINO style representation learning We minimize cross-entropy loss to compare the ground truth and future state embedding to learn the visuo-tactile transformer. d(xt+1, xˆt+1) = −softmax[ϕ(ot+1)] log softmax[ϕ(oˆt+1)] (7.2) The total los…

**Figure 7.7.** Figure 7.7: Pre-training with human demonstrations followed by fine-tuning and imitation with robot demonstrations 7.6 Pre-training and fine-tuning We learn the encoder ϕ, forward model f, inverse model g and imitation policy backbone ψ first by pre-training with human demonstra…

**Figure 7.8.** Figure 7.8: The robot setup shown above consists of 3 camera views - two side views and 1 wrist view as shown above is equipped with a two-fingered gripper. The tactile feedback is obtained via tactile sensor embedded in the tip of the gripper fingers index thumb side view (left…

**Figure 7.9.** Figure 7.9: Human data collection setup consists camera views similar to the that of robot. Two side cameras and one wrist camera view. The tactile data is collected finger cap device that embeds the singletact capacitive tactile sensor. 7.7.1 Task and demonstrations The task is…

**Figure 7.10.** Figure 7.10: The attention visualized during different phases of the demonstraions i.e reach, grasp, and lift phases. 100 epochs till convergence. The attention maps of ViTacT transformer. The attention heatmap shows greater attention towards the dynamic elements of the scene as…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

177 extracted references · 77 canonical work pages

[109]

Optimalityandapproximationwith policy gradient methods in markov decision processes,

A.Agarwal,S.M.Kakade,J.D.Lee,andG.Mahajan,“Optimalityandapproximationwith policy gradient methods in markov decision processes,” inProceedings of Thirty Third Conference on Learning Theory, J. Abernethy and S. Agarwal, Eds., ser. Proceedings of Machine Learning Research, vol. 125, PMLR, 2020, pp. 64–66. 111

work page 2020
[1]

Ritter and R

H. Ritter and R. Haschke,Hands, Dexterity, and the Brain. CRC Press/Taylor & Francis, 2015

2015
[2]

The coevolution of encephalization and manual dexterity in hominins and other primates,

J. Baker, R. A. Barton, and C. Venditti, “The coevolution of encephalization and manual dexterity in hominins and other primates,”bioRxiv, Sep. 2024

2024
[3]

Developmentofin-handmanipulationand relationship with activities,

R.Humphry,K.Jewell,andR.C.Rosenberger,“Developmentofin-handmanipulationand relationship with activities,”Am. J. Occup. Ther., vol. 49, no. 8, pp. 763–771, Sep. 1995

1995
[4]

How babies use their hands to learn about objects: Exploration, reach-to-grasp, manipulation, and tool use,

A. W. Needham and E. L. Nelson, “How babies use their hands to learn about objects: Exploration, reach-to-grasp, manipulation, and tool use,”Wiley Interdiscip. Rev. Cogn. Sci., vol. 14, no. 6, e1661, Nov. 2023

2023
[5]

Exploring hand dexterity in children with myelomeningocele,

S. Steinhart, E. Kornitzer, P. L. Weiss, and M. Katz-Leurer, “Exploring hand dexterity in children with myelomeningocele,”J. Pediatr. Rehabil. Med., vol. 14, no. 4, pp. 613–619, 2021

2021
[6]

The early motor milestones in infancy and later motor skills in toddlers: A structural equation model of motor development,

H. Viholanen, T. Ahonen, M. Cantell, A. Tolvanen, and H. Lyytinen, “The early motor milestones in infancy and later motor skills in toddlers: A structural equation model of motor development,”Phys. Occup. Ther. Pediatr., vol. 26, no. 1-2, pp. 91–113, 2006

2006
[7]

Role of uncertainty in sensorimotor control,

R. J. van Beers, P. Baraduc, and D. M. Wolpert, “Role of uncertainty in sensorimotor control,”Philos.Trans.R.Soc.Lond.BBiol.Sci. ,vol.357,no.1424,pp.1137–1145,Aug. 2002

2002

Show all 177 references

[8]

A literature review of the universal and atomic elements of complex cognition,

D. Cabrera, L. Cabrera, and E. Cabrera, “A literature review of the universal and atomic elements of complex cognition,”Journal of Systems Thinking, vol. 3, pp. 1–85, Oct. 2023

2023
[9]

DIGIT: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation,

M. Lambeta et al., “DIGIT: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation,”IEEE Robot. Autom. Lett., vol. 5, no. 3, pp. 3838–3845, Jul. 2020

2020
[10]

Data-driventactilesensingusingspatiallyoverlappingsignals,

P.Piacenza,“Data-driventactilesensingusingspatiallyoverlappingsignals,”Ph.D.disser- tation, Columbia University, 2020

2020
[11]

GelSight: High-resolution robot tactile sensors for estimating geometry and force,

W. Yuan, S. Dong, and E. H. Adelson, “GelSight: High-resolution robot tactile sensors for estimating geometry and force,”Sensors (Basel), vol. 17, no. 12, p. 2762, Nov. 2017

2017
[12]

Re-evaluation of the distribution of meissner’s corpuscles in human skin,

S. H. Kim and Y. H. Lee, “Re-evaluation of the distribution of meissner’s corpuscles in human skin,”Anat. Cell Biol., vol. 53, no. 3, pp. 325–329, Sep. 2020. 104

2020
[13]

Tactile dexterity: Manipulation primitives with tactile feedback,

F. R. Hogan, J. Ballester, S. Dong, and A. Rodriguez, “Tactile dexterity: Manipulation primitives with tactile feedback,”arXiv [cs.RO], Feb. 2020

2020
[14]

Surveyoflearning-basedapproaches for robotic in-hand manipulation,

A.I.Weinberg,A.Shirizly,O.Azulay,andA.Sintov,“Surveyoflearning-basedapproaches for robotic in-hand manipulation,”Front. Robot. AI, vol. 11, p. 1455431, Nov. 2024

2024
[15]

In-handmanipulationinyoungchildren: Rotation of an object in the fingers,

C.Pehoski,A.Henderson,andL.Tickle-Degnen,“In-handmanipulationinyoungchildren: Rotation of an object in the fingers,”The American Journal of Occupational Therapy, vol. 51, no. 7, pp. 544–552, 1997

1997
[16]

Solving rubik’s cube with a robot hand,

OpenAI et al., “Solving rubik’s cube with a robot hand,”arXiv [cs.LG], Oct. 2019

2019
[17]

Dextrous tactile in-hand manipulation using a modular reinforcement learning architecture,

J. Pitz, L. Röstel, L. Sievers, and B. Bäuml, “Dextrous tactile in-hand manipulation using a modular reinforcement learning architecture,”arXiv [cs.RO], Mar. 2023

2023
[18]

Learning purely tactile in-hand manipulation with a torque-controlled hand,

L. Sievers, J. Pitz, and B. Bäuml, “Learning purely tactile in-hand manipulation with a torque-controlled hand,” in2022 International Conference on Robotics and Automation (ICRA), May 2022, pp. 2745–2751

2022
[19]

AnyRotate: Gravity-invariant in-hand object rotation with sim-to-real touch,

M. Yang et al., “AnyRotate: Gravity-invariant in-hand object rotation with sim-to-real touch,”arXiv [cs.RO], May 2024

2024
[20]

Transferring dexterous manipulation from GPU simulation to a remote real-world TriFinger,

A. Allshire et al., “Transferring dexterous manipulation from GPU simulation to a remote real-world TriFinger,”arXiv [cs.RO], Aug. 2021

2021
[21]

Visual dexterity: In-hand dexterous manipulation from depth,

T. Chen, M. Tippur, S. Wu, V. Kumar, E. Adelson, and P. Agrawal, “Visual dexterity: In-hand dexterous manipulation from depth,”arXiv [cs.RO], Nov. 2022

2022
[22]

In-hand object rotation via rapid motor adaptation,

H. Qi, A. Kumar, R. Calandra, Y. Ma, and J. Malik, “In-hand object rotation via rapid motor adaptation,”arXiv [cs.RO], Oct. 2022

2022
[23]

On the feasibility of learning finger- gaiting in-hand manipulation with intrinsic sensing,

G. Khandate, M. Haas-Heger, and M. Ciocarlie, “On the feasibility of learning finger- gaiting in-hand manipulation with intrinsic sensing,” in2022 International Conference on Robotics and Automation (ICRA), May 2022, pp. 2752–2758

2022
[24]

Sampling- based exploration for reinforcement learning of dexterous manipulation,

G. Khandate, S. Shang, E. T. Chang, T. L. Saidi, J. Adams, and M. Ciocarlie, “Sampling- based exploration for reinforcement learning of dexterous manipulation,” inRobotics: Science and Systems XIX, vol. 19, Jul. 2023

2023
[25]

General in-hand object rotation with vision and touch,

H. Qi et al., “General in-hand object rotation with vision and touch,” Aug. 2023

2023
[26]

Rotating without seeing: Towards in-handdexteritythroughtouch,

Z.-H. Yin, B. Huang, Y. Qin, Q. Chen, and X. Wang, “Rotating without seeing: Towards in-handdexteritythroughtouch,”in Robotics:ScienceandSystemsXIX ,vol.19,Jul.2023. 105

2023
[27]

Dextroustactilein-handmanipulationusinga modular reinforcement learning architecture,

J.Pitz,L.Röstel,L.Sievers,andB.Bäuml,“Dextroustactilein-handmanipulationusinga modular reinforcement learning architecture,” in2023 IEEE International Conference on Robotics and Automation (ICRA), IEEE, May 2023

2023
[28]

Tactile sensing for dexterous in-hand manip- ulation in robotics—a review,

H. Yousef, M. Boukallel, and K. Althoefer, “Tactile sensing for dexterous in-hand manip- ulation in robotics—a review,”Sens. Actuators A Phys., vol. 167, no. 2, pp. 171–187, Jun. 2011

2011
[29]

Tactile sensors for friction estimation and incipient slip detection—toward dexterous robotic manipulation: A review,

W. Chen, H. Khamis, I. Birznieks, N. F. Lepora, and S. J. Redmond, “Tactile sensors for friction estimation and incipient slip detection—toward dexterous robotic manipulation: A review,”IEEE Sens. J., vol. 18, no. 22, pp. 9049–9064, Nov. 2018

2018
[30]

TEXterity – tactile extrinsic deXterity: Simultaneous tactile estimation and control for extrinsic dexterity,

S. Kim, A. Bronars, P. Patre, and A. Rodriguez, “TEXterity – tactile extrinsic deXterity: Simultaneous tactile estimation and control for extrinsic dexterity,”arXiv [cs.RO], Feb. 2024

2024
[31]

Generalizedexplorationinpolicysearch,

H.vanHoof,D.Tanneberg,andJ.Peters,“Generalizedexplorationinpolicysearch,” Mach. Learn., vol. 106, no. 9, pp. 1705–1724, Oct. 2017

2017
[32]

Towards learning to play piano with dexterous hands and touch,

H. Xu, Y. Luo, S. Wang, T. Darrell, and R. Calandra, “Towards learning to play piano with dexterous hands and touch,” in2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, Oct. 2022, pp. 10410–10416

2022
[33]

Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation,

S. Suresh et al., “Neural feels with neural fields: Visuo-tactile perception for in-hand manipulation,”arXiv [cs.RO], Dec. 2023

2023
[34]

Thefutureliesinapairoftactilehands,

N.F.Lepora,“Thefutureliesinapairoftactilehands,” Sci.Robot.,vol.9,no.91,eadq1501, Jun. 2024

2024
[35]

DexiTac: Soft dexterous tactile gripping,

C. Lu, K. Tang, M. Yang, T. Yue, and N. F. Lepora, “DexiTac: Soft dexterous tactile gripping,” arXiv [cs.RO], May 2024

2024
[36]

Lessons from learning to spin “pens

J. Wang et al., “Lessons from learning to spin “pens”,”arXiv [cs.RO], Jul. 2024

2024
[37]

Diffusion policy: Visuomotor policy learning via action diffusion,

C. Chi et al., “Diffusion policy: Visuomotor policy learning via action diffusion,”arXiv [cs.RO], Mar. 2023

2023
[38]

Learningfine-grainedbimanualmanipulation with low-cost hardware,

T.Z.Zhao,V.Kumar,S.Levine,andC.Finn,“Learningfine-grainedbimanualmanipulation with low-cost hardware,”arXiv [cs.RO], Apr. 2023

2023
[39]

Waypoint-based imitation learning for robotic manipulation,

A. Sharma, L. X. Shi, T. Z. Zhao, and C. Finn, “Waypoint-based imitation learning for robotic manipulation,” Aug. 2023

2023
[40]

Learningfine-grainedbimanualmanipulation with low-cost hardware,

T.Z.Zhao,V.Kumar,S.Levine,andC.Finn,“Learningfine-grainedbimanualmanipulation with low-cost hardware,” inRobotics: Science and Systems XIX, vol. 19, Jul. 2023. 106

2023
[41]

Diffusion policy: Visuomotor policy learning via action diffusion,

C. Chi et al., “Diffusion policy: Visuomotor policy learning via action diffusion,” in Robotics: Science and Systems XIX, vol. 19, Jul. 2023

2023
[42]

RoboAgent: Generalizationandefficiencyinrobotmanipulationviasemanticaugmentationsandaction chunking,

H. Bharadhwaj, J. Vakil, M. Sharma, A. Gupta, S. Tulsiani, and V. Kumar, “RoboAgent: Generalizationandefficiencyinrobotmanipulationviasemanticaugmentationsandaction chunking,”arXiv [cs.RO], Sep. 2023

2023
[43]

Learningtograsptheungraspablewithemergentextrinsicdexterity,

W.ZhouandD.Held,“Learningtograsptheungraspablewithemergentextrinsicdexterity,” arXiv [cs.RO], Nov. 2022

2022
[44]

Extremeparkourwithleggedrobots,

X.Cheng,K.Shi,A.Agarwal,andD.Pathak,“Extremeparkourwithleggedrobots,” arXiv [cs.RO], Sep. 2023

2023
[45]

Learning quadrupedal locomotion over challenging terrain,

J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter, “Learning quadrupedal locomotion over challenging terrain,”Sci Robot, vol. 5, no. 47, Oct. 2020

2020
[46]

Learning agile and dynamic motor skills for legged robots,

J. Hwangbo et al., “Learning agile and dynamic motor skills for legged robots,”Sci Robot, vol. 4, no. 26, Jan. 2019

2019
[47]

Robot parkour learning,

Z. Zhuang et al., “Robot parkour learning,”arXiv [cs.RO], Sep. 2023

2023
[48]

BC-Z:Zero-shottaskgeneralizationwithroboticimitationlearning,

E.Jang etal.,“BC-Z:Zero-shottaskgeneralizationwithroboticimitationlearning,” arXiv [cs.RO], Feb. 2022

2022
[49]

RT-1:Roboticstransformerforreal-worldcontrolatscale,

A.Brohan etal.,“RT-1:Roboticstransformerforreal-worldcontrolatscale,” arXiv[cs.RO], Dec. 2022

2022
[50]

RoboCat: A self-improving foundation agent for robotic manipula- tion,

K. Bousmalis et al., “RoboCat: A self-improving foundation agent for robotic manipula- tion,”arXiv [cs.RO], Jun. 2023

2023
[51]

RT-2:Vision-language-actionmodelstransferwebknowledgetorobotic control,

B.Zitkovich etal.,“RT-2:Vision-language-actionmodelstransferwebknowledgetorobotic control,” Aug. 2023

2023
[52]

Domainrandomization for transferring deep neural networks from simulation to the real world,

J.Tobin,R.Fong,A.Ray,J.Schneider,W.Zaremba,andP.Abbeel,“Domainrandomization for transferring deep neural networks from simulation to the real world,”arXiv [cs.RO], Mar. 2017

2017
[53]

Online vs. offline adaptive domain randomization benchmark,

G. Tiboni, K. Arndt, G. Averta, V. Kyrki, and T. Tommasi, “Online vs. offline adaptive domain randomization benchmark,”arXiv [cs.RO], Jun. 2022

2022
[54]

RMA: Rapid motor adaptation for legged robots,

A. Kumar, Z. Fu, D. Pathak, and J. Malik, “RMA: Rapid motor adaptation for legged robots,”arXiv [cs.LG], Jul. 2021

2021
[55]

Reconcilingrealitythroughsimulation:Areal-to-sim-to-realapproachfor robust manipulation,

M.Torne etal.,“Reconcilingrealitythroughsimulation:Areal-to-sim-to-realapproachfor robust manipulation,”arXiv [cs.RO], Mar. 2024. 107

2024
[56]

Anoverviewofdexterousmanipulation,

A.Okamura,N.Smaby,andM.Cutkosky,“Anoverviewofdexterousmanipulation,” Pro- ceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065), vol. 1, 255–262vol.1, Apr. 2000

2000
[57]

On dexterity and dexterous manipulation,

R. R. Ma and A. M. Dollar, “On dexterity and dexterous manipulation,” in2011 15th International Conference on Advanced Robotics (ICAR), Jun. 2011, pp. 1–7

2011
[58]

Reorientingobjectswitharobothandusinggraspgaits,

S.LeveroniandK.Salisbury,“Reorientingobjectswitharobothandusinggraspgaits,”in Robotics Research, Springer London, 1996, pp. 39–51

1996
[59]

Dextrous manipulation by rolling and finger gaiting,

L. Han and J. C. Trinkle, “Dextrous manipulation by rolling and finger gaiting,” in Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146), vol. 1, May 1998, 730–735 vol.1

1998
[60]

Manipulation gaits: Sequences of grasp control tasks,

R. Platt, A. H. Fagg, and R. A. Grupen, “Manipulation gaits: Sequences of grasp control tasks,”in IEEEInternationalConferenceonRoboticsandAutomation,2004.Proceedings. ICRA ’04. 2004, vol. 1, Apr. 2004, 801–806 Vol.1

2004
[61]

Dexterous manipulation planning using probabilistic roadmaps in continuous grasp subspaces,

J.-P. Saut, A. Sahbani, S. El-Khoury, and V. Perdereau, “Dexterous manipulation planning using probabilistic roadmaps in continuous grasp subspaces,” in2007 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems, Oct. 2007, pp. 2907–2912

2007
[62]

Regrasps by a multifingered hand based on primitives,

T. Omata and M. A. Farooqi, “Regrasps by a multifingered hand based on primitives,” in Proceedings of IEEE International Conference on Robotics and Automation, vol. 3, Apr. 1996, 2774–2780 vol.3

1996
[63]

Chapter 8 markov decision processes,

M. L. Puterman, “Chapter 8 markov decision processes,” inHandbooks in Operations ResearchandManagementScience ,ser.Handbooksinoperationsresearchandmanagement science, vol. 2, Elsevier, Jan. 1990, pp. 331–434

1990
[64]

MuJoCo: A physics engine for model-based control,

E. Todorov, T. Erez, and Y. Tassa, “MuJoCo: A physics engine for model-based control,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. 2012, pp. 5026–5033

2012
[65]

Synthesisandstabilizationofcomplexbehaviorsthrough online trajectory optimization,

Y.Tassa,T.Erez,andE.Todorov,“Synthesisandstabilizationofcomplexbehaviorsthrough online trajectory optimization,” in2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. 2012, pp. 4906–4913

2012
[66]

Discoveryofcomplexbehaviorsthroughcontact- invariant optimization,

I.Mordatch,E.Todorov,andZ.Popović,“Discoveryofcomplexbehaviorsthroughcontact- invariant optimization,”ACM Trans. Graph., vol. 31, no. 4, pp. 1–8, Jul. 2012

2012
[67]

Real-time robust finger gaits planning under object shape and dynamics uncertainties,

Y. Fan, T. Tang, H.-C. Lin, Y. Zhao, and M. Tomizuka, “Real-time robust finger gaits planning under object shape and dynamics uncertainties,”arXiv [cs.RO], Oct. 2017. 108

2017
[68]

Geometric in-hand regrasp planning: Alternating optimization of finger gaits and in-grasp manipulation,

B. Sundaralingam and T. Hermans, “Geometric in-hand regrasp planning: Alternating optimization of finger gaits and in-grasp manipulation,”arXiv [cs.RO], Apr. 2018

2018
[69]

Solvingchallengingdexterousmanipulationtaskswith trajectory optimisation and reinforcement learning,

H.CharlesworthandG.Montana,“Solvingchallengingdexterousmanipulationtaskswith trajectory optimisation and reinforcement learning,”arXiv [cs.RO], Sep. 2020

2020
[70]

Complementarity-free multi-contact modeling and optimization for dexterous manipulation,

W. Jin, “Complementarity-free multi-contact modeling and optimization for dexterous manipulation,”arXiv [cs.RO], Aug. 2024

2024
[71]

Randomized manipulation planning for a multi-fingered hand by switching contact modes,

M. Yashima, Y. Shiina, and H. Yamaguchi, “Randomized manipulation planning for a multi-fingered hand by switching contact modes,” in2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422), vol. 2, Sep. 2003, 2689–2694 vol.2

2003
[72]

Finger gaits planning for multifingered manipulation,

J. Xu, T. J. Koo, and Z. Li, “Finger gaits planning for multifingered manipulation,” in 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. 2007, pp. 2932–2937

2007
[73]

Learning dexterous in-hand manipulation,

OpenAI et al., “Learning dexterous in-hand manipulation,”arXiv [cs.LG], Aug. 2018

2018
[74]

Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,

A. Rajeswaran et al., “Learning complex dexterous manipulation with deep reinforcement learning and demonstrations,”arXiv preprint arXiv, Sep. 2017

2017
[75]

Dexterousmanipulationwithdeepreinforcement learning: Efficient, general, and low-cost,

H.Zhu,A.Gupta,A.Rajeswaran, etal.,“Dexterousmanipulationwithdeepreinforcement learning: Efficient, general, and low-cost,”on Robotics and ..., 2019

2019
[76]

State-only imitation learning for dexter- ous manipulation,

I. Radosavovic, X. Wang, L. Pinto, and J. Malik, “State-only imitation learning for dexter- ous manipulation,” in2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Sep. 2021, pp. 7865–7871

2021
[77]

Generalizationindexterousmanipulation via geometry-aware multi-task learning,

W.Huang,I.Mordatch,P.Abbeel,andD.Pathak,“Generalizationindexterousmanipulation via geometry-aware multi-task learning,”arXiv [cs.RO], Nov. 2021

2021
[78]

Isaac gym: High performance GPU-based physics simulation for robot learning,

V. Makoviychuk et al., “Isaac gym: High performance GPU-based physics simulation for robot learning,”arXiv [cs.RO], Aug. 2021

2021
[79]

DeXtreme: Transfer of agile in-hand manipulation from simulation to reality,

A. Handa et al., “DeXtreme: Transfer of agile in-hand manipulation from simulation to reality,”arXiv [cs.RO], Oct. 2022

2022
[80]

Precision object manipulation with a multifingered robot hand,

P. Michelman, “Precision object manipulation with a multifingered robot hand,”IEEE Trans. Rob. Autom., vol. 14, no. 1, pp. 105–113, Feb. 1998

1998
[81]

Learningrobotin-handmanipulation with tactile features,

H.vanHoof,T.Hermans,G.Neumann,andJ.Peters,“Learningrobotin-handmanipulation with tactile features,” in2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), ieeexplore.ieee.org, Nov. 2015, pp. 121–127. 109

2015
[82]

Melnik, L

A. Melnik, L. Lach, M. Plappert, T. Korthals, R. Haschke, and H. Ritter,Tactile sensing anddeepreinforcementlearningforin-handmanipulationtasks , https://www.ai.rug. nl/oel/papers/tactile_sensing_OEL.pdf, Accessed: 2021-7-22

2021
[83]

Using tactile sensing to improve the sample efficiency and performance of deep deterministic policy gradients for simulated in-hand manipulation tasks,

A. Melnik, L. Lach, M. Plappert, T. Korthals, R. Haschke, and H. Ritter, “Using tactile sensing to improve the sample efficiency and performance of deep deterministic policy gradients for simulated in-hand manipulation tasks,”Front Robot AI, vol. 8, p. 538773, Jun. 2021

2021
[84]

Deepdynamicsmodelsforlearningdexterous manipulation,

A.Nagabandi,K.Konolige,S.Levine, etal.,“Deepdynamicsmodelsforlearningdexterous manipulation,”Conference on Robot, 2020

2020
[85]

Modelpredictiveactor-critic:Acceleratingrobotskillacquisitionwithdeepreinforcement learning,

A. S. Morgan, D. Nandha, G. Chalvatzaki, C. D’Eramo, A. M. Dollar, and J. Peters, “Modelpredictiveactor-critic:Acceleratingrobotskillacquisitionwithdeepreinforcement learning,”arXiv [cs.RO], Mar. 2021

2021
[86]

Surprisinglyrobustin-handmanipulation: An empirical study,

A.Bhatt,A.Sieler,S.Puhlmann,andO.Brock,“Surprisinglyrobustin-handmanipulation: An empirical study,”arXiv [cs.RO], Jan. 2022

2022
[87]

Onthefeasibilityoflearningfinger-gaiting in-hand manipulation with intrinsic sensing,

G.Khandate,M.Haas-Heger,andM.Ciocarlie,“Onthefeasibilityoflearningfinger-gaiting in-hand manipulation with intrinsic sensing,”arXiv [cs.RO], Sep. 2021

2021
[88]

A system for general in-hand object re-orientation,

T. Chen, J. Xu, and P. Agrawal, “A system for general in-hand object re-orientation,” Nov. 2021

2021
[89]

Estimator-coupled reinforcement learning for robust purely tactile in-hand manipulation,

L. Röstel, J. Pitz, L. Sievers, and B. Bäuml, “Estimator-coupled reinforcement learning for robust purely tactile in-hand manipulation,”arXiv [cs.RO], Nov. 2023

2023
[90]

Rotating without seeing: Towards in-hand dexterity through touch,

Z.-H. Yin, B. Huang, Y. Qin, Q. Chen, and X. Wang, “Rotating without seeing: Towards in-hand dexterity through touch,”arXiv [cs.RO], Mar. 2023

2023
[91]

Robot synesthesia: In-hand manipulation with visuotactile sensing,

Y. Yuan et al., “Robot synesthesia: In-hand manipulation with visuotactile sensing,”arXiv [cs.RO], Dec. 2023

2023
[92]

Generalin-handobjectrotationwithvisionandtouch,

H.Qi etal.,“Generalin-handobjectrotationwithvisionandtouch,” CoRL,vol.abs/2309.09979, Sep. 2023

2023 arXiv
[93]

Sampling-based exploration for reinforcement learning of dexterous manipulation,

G. Khandate et al., “Sampling-based exploration for reinforcement learning of dexterous manipulation,”arXiv [cs.RO], Mar. 2023

2023
[94]

Hierarchical tactile-based control decomposition of dexterous in-hand manipulation tasks,

F. Veiga, R. Akrour, and J. Peters, “Hierarchical tactile-based control decomposition of dexterous in-hand manipulation tasks,”Front Robot AI, vol. 7, p. 521448, Nov. 2020

2020
[95]

Learninghierarchicalcontrol for robust in-hand manipulation,

T.Li,K.Srinivasan,M.Q.-H.Meng,W.Yuan,andJ.Bohg,“Learninghierarchicalcontrol for robust in-hand manipulation,”arXiv [cs.RO], Oct. 2019. 110

2019
[96]

AC-teach: A bayesianactor-criticmethodforpolicylearningwithanensembleofsuboptimalteachers,

A. Kurenkov, A. Mandlekar, R. Martin-Martin, S. Savarese, and A. Garg, “AC-teach: A bayesianactor-criticmethodforpolicylearningwithanensembleofsuboptimalteachers,” arXiv [cs.LG], Sep. 2019

2019
[97]

Learningdexterousmanipulationfromsuboptimalexperts,

R.Jeong etal.,“Learningdexterousmanipulationfromsuboptimalexperts,” arXiv[cs.RO], Oct. 2020

2020
[98]

Zhang, PEX: Policy expansion for bridging offline-to-online reinforcement learning (ICLR23)

H. Zhang, PEX: Policy expansion for bridging offline-to-online reinforcement learning (ICLR23)
[99]

R ×R:RapideXplorationforreinforcementlearningviasampling-based reset distributions and imitation pre-training,

G.Khandate etal.,“R ×R:RapideXplorationforreinforcementlearningviasampling-based reset distributions and imitation pre-training,”arXiv [cs.RO], Jan. 2024

2024
[100]

Curiosity-driven exploration by self- supervised prediction,

D. Pathak, P. Agrawal, A. A. Efros, and T. Darrell, “Curiosity-driven exploration by self- supervised prediction,”arXiv [cs.LG], May 2017

2017
[101]

Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,”arXiv [cs.LG], Jan. 2018

2018
[102]

A survey of exploration methods in reinforcement learning,

S. Amin, M. Gomrokchi, H. Satĳa, H. van Hoof, and D. Precup, “A survey of exploration methods in reinforcement learning,”arXiv:2109.00157 [cs], Sep. 2021

2021 arXiv
[103]

Parameter space noise for exploration,

M. Plappert et al., “Parameter space noise for exploration,”arXiv [cs.LG], Jun. 2017

2017
[104]

Overcoming explo- ration in reinforcement learning with demonstrations,

A. Nair, B. McGrew, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Overcoming explo- ration in reinforcement learning with demonstrations,”arXiv:1709.10089 [cs], Sep. 2017

2017 arXiv
[105]

Go-explore: A new approach for hard-exploration problems,

A. Ecoffet, J. Huizinga, J. Lehman, K. O. Stanley, and J. Clune, “Go-explore: A new approach for hard-exploration problems,” arXiv, Jan. 2019

2019
[106]

Firstreturn,thenexplore,

A.Ecoffet,J.Huizinga,J.Lehman,K.O.Stanley,andJ.Clune,“Firstreturn,thenexplore,” Nature, vol. 590, no. 7847, pp. 580–586, Feb. 2021

2021
[107]

Exploring restart distributions,

A. Tavakoli, V. Levdik, R. Islam, C. M. Smith, and P. Kormushev, “Exploring restart distributions,” arXiv, Nov. 2018

2018
[108]

Benchmarking deep rein- forcement learning for continuous control,

Y. Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel, “Benchmarking deep rein- forcement learning for continuous control,”arXiv [cs.LG], Apr. 2016

2016
[110]

Rapidly-exploring random trees : A new tool for path planning,

S. LaValle, “Rapidly-exploring random trees : A new tool for path planning,”The annual research report, 1998

1998
[111]

Optimal kinodynamic motion planning using incremental sampling-basedmethods,

S. Karaman and E. Frazzoli, “Optimal kinodynamic motion planning using incremental sampling-basedmethods,”in 49thIEEEConferenceonDecisionandControl(CDC) ,Dec. 2010, pp. 7681–7687

2010
[112]

Kinodynamic RRT*: Asymptotically optimal motion planning for robots with linear dynamics,

D. J. Webb and J. van den Berg, “Kinodynamic RRT*: Asymptotically optimal motion planning for robots with linear dynamics,” in2013 IEEE International Conference on Robotics and Automation, May 2013, pp. 5054–5061

2013
[113]

Probabilistic roadmaps for path planning in high-dimensional configuration spaces,

L. E. Kavraki, P. Svestka, J.-C. Latombe, and M. H. Overmars, “Probabilistic roadmaps for path planning in high-dimensional configuration spaces,”IEEE Trans. Rob. Autom., vol. 12, no. 4, pp. 566–580, Aug. 1996

1996
[114]

Analysisofprobabilisticroadmaps for path planning,

L.E.Kavraki,M.N.Kolountzakis,andJ.-C.Latombe,“Analysisofprobabilisticroadmaps for path planning,”IEEE Trans. Rob. Autom., vol. 14, no. 1, pp. 166–171, Feb. 1998

1998
[115]

MPC-MPNet: Model-predictive motion planning networks for fast, near-optimal planning under kinodynamic constraints,

L. Li, Y. Miao, A. H. Qureshi, and M. C. Yip, “MPC-MPNet: Model-predictive motion planning networks for fast, near-optimal planning under kinodynamic constraints,”IEEE Robotics and Automation Letters, vol. 6, no. 3, pp. 4496–4503, Jul. 2021

2021
[116]

RL-RRT: Kinodynamic mo- tion planning via learning reachability estimators from RL policies,

H.-T. L. Chiang, J. Hsu, M. Fiser, L. Tapia, and A. Faust, “RL-RRT: Kinodynamic mo- tion planning via learning reachability estimators from RL policies,”IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 4298–4305, Oct. 2019

2019
[117]

Long-rangeindoornavigationwithPRM-RL,

A.Francis etal.,“Long-rangeindoornavigationwithPRM-RL,” IEEETrans.Rob. ,vol.36, no. 4, pp. 1115–1134, Aug. 2020

2020
[118]

Learning-guided exploration for efficient sampling-based motion planning in high dimensions,

L. Schramm and A. Boularias, “Learning-guided exploration for efficient sampling-based motion planning in high dimensions,” in2022 International Conference on Robotics and Automation (ICRA), IEEE, May 2022

2022
[119]

Sample-efficientlearningofnonprehen- sile manipulation policies via physics-based informed state distributions,

L.Pinto,A.Mandalika,B.Hou,andS.Srinivasa,“Sample-efficientlearningofnonprehen- sile manipulation policies via physics-based informed state distributions,”arXiv [cs.RO], Oct. 2018

2018
[120]

Reinforcementlearningwithprobabilistically complete exploration,

P.Morere,G.Francis,T.Blau,andF.Ramos,“Reinforcementlearningwithprobabilistically complete exploration,”arXiv [cs.LG], Jan. 2020

2020
[121]

Harnessing reinforcement learning for neural motion plan- ning,

T. Jurgenson and A. Tamar, “Harnessing reinforcement learning for neural motion plan- ning,”arXiv [cs.RO], Jun. 2019

2019
[122]

Learning a decentralized multi-arm motion planner,

H. Ha, J. Xu, and S. Song, “Learning a decentralized multi-arm motion planner,”arXiv [cs.RO], Nov. 2020. 112

2020
[123]

Learning from demonstration,

S. Schaal, “Learning from demonstration,” Advances in Neural Information Processing Systems, vol. 9, 1996

1996
[124]

DexPilot: Vision based teleoperation of dexterous robotic hand-arm system,

A. Handa et al., “DexPilot: Vision based teleoperation of dexterous robotic hand-arm system,”arXiv [cs.CV], Oct. 2019

2019
[125]

AnyTeleop: A general vision-based dexterous robot arm-hand teleoperation system,

Y. Qin et al., “AnyTeleop: A general vision-based dexterous robot arm-hand teleoperation system,”arXiv [cs.RO], Jul. 2023

2023
[126]

Dexterous imitation made easy: A learning-basedframeworkforefficientdexterousmanipulation,

S. P. Arunachalam, S. Silwal, B. Evans, and L. Pinto, “Dexterous imitation made easy: A learning-basedframeworkforefficientdexterousmanipulation,” arXiv[cs.RO],Mar.2022

2022
[127]

Implicitbehavioralcloning,

P.Florence etal.,“Implicitbehavioralcloning,”in ConferenceonRobotLearning ,PMLR, Jan. 2022, pp. 158–168

2022
[128]

N.M.Shafiullah,Z.J.C.Altanzaya,andL.Pinto, Behaviortransformers:Cloningkmodes with one stone
[129]

Dexterity from touch: Self-supervised pre-training of tactile representations with robotic play,

I. Guzey, B. Evans, S. Chintala, and L. Pinto, “Dexterity from touch: Self-supervised pre-training of tactile representations with robotic play,”arXiv [cs.RO], Mar. 2023

2023
[130]

Offlinereinforcementlearning:Tutorial,review, and perspectives on open problems,

S.Levine,A.Kumar,G.Tucker,andJ.Fu,“Offlinereinforcementlearning:Tutorial,review, and perspectives on open problems,”arXiv [cs.LG], May 2020

2020
[131]

AWAC: Accelerating online reinforcement learning with offline datasets,

A. Nair, A. Gupta, M. Dalal, and S. Levine, “AWAC: Accelerating online reinforcement learning with offline datasets,”arXiv [cs.LG], Jun. 2020

2020
[132]

Conservative Q-learning for offline rein- forcementlearning,

A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative Q-learning for offline rein- forcementlearning,” AdvancesinNeuralInformationProcessingSystems ,vol.33,pp.1179– 1191, 2020

2020
[133]

ShouldIrunofflinereinforcementlearningor behavioralcloning?

A.Kumar,J.Hong,A.Singh,andS.Levine,“ShouldIrunofflinereinforcementlearningor behavioralcloning?”In InternationalConferenceonLearningRepresentations ,Oct.2021

2021
[134]

Concept2Robot: Learning manipulation concepts from instructions and human demonstrations,

L. Shao, T. Migimatsu, Q. Zhang, K. Yang, and J. Bohg, “Concept2Robot: Learning manipulation concepts from instructions and human demonstrations,”Int. J. Rob. Res., vol. 40, no. 12-14, pp. 1419–1434, Dec. 2021

2021
[135]

Learning generalizable robotic reward functions from “in-the-wild

A. S. Chen, S. Nair, and C. Finn, “Learning generalizable robotic reward functions from “in-the-wild” human videos,”arXiv [cs.RO], Mar. 2021

2021
[136]

Videopredictionmodelsasrewardsforreinforcementlearning,

A.Escontrela etal.,“Videopredictionmodelsasrewardsforreinforcementlearning,” arXiv [cs.LG], May 2023. 113

2023
[137]

Cross-domaintransferviasemanticskillimitation,

K.Pertsch etal.,“Cross-domaintransferviasemanticskillimitation,” arXiv[cs.LG],Dec. 2022

2022
[139]

XIRL: Cross- embodiment inverse reinforcement learning,

K. Zakka, A. Zeng, P. Florence, J. Tompson, J. Bohg, and D. Dwibedi, “XIRL: Cross- embodiment inverse reinforcement learning,”arXiv [cs.RO], Jun. 2021

2021
[140]

XSkill:Crossembodimentskilldiscovery,

M.Xu,Z.Xu,C.Chi,M.Veloso,andS.Song,“XSkill:Crossembodimentskilldiscovery,” arXiv [cs.RO], Jul. 2023

2023
[141]

The “something something

R. Goyal et al., “The “something something” video database for learning and evaluating visual common sense,”arXiv [cs.CV], Jun. 2017

2017
[142]

2, https://developer.qualcomm

Moving objects dataset: Something-something v. 2, https://developer.qualcomm. com/software/ai-datasets/something-something, Accessed: 2024-2-23

2024
[143]

Ego4D: Around the world in 3,000 hours of egocentric video,

K. Grauman et al., “Ego4D: Around the world in 3,000 hours of egocentric video,”arXiv [cs.CV], Oct. 2021

2021
[144]

Theunsurprisingeffectiveness of pre-trained vision models for control,

S.Parisi,A.Rajeswaran,S.Purushwalkam,andA.Gupta,“Theunsurprisingeffectiveness of pre-trained vision models for control,”arXiv [cs.CV], Mar. 2022

2022
[145]

R3M: A universal visual representation for robot manipulation,

S. Nair, A. Rajeswaran, V. Kumar, C. Finn, and A. Gupta, “R3M: A universal visual representation for robot manipulation,”arXiv [cs.RO], Mar. 2022

2022
[146]

Affordances from human videos as a versatile representation for robotics,

S. Bahl, R. Mendonca, L. Chen, U. Jain, and D. Pathak, “Affordances from human videos as a versatile representation for robotics,”arXiv [cs.RO], Apr. 2023

2023
[147]

Structured world models from human videos,

R. Mendonca, S. Bahl, and D. Pathak, “Structured world models from human videos,” arXiv [cs.RO], Aug. 2023

2023
[148]

Generalflowasfoundationaffordanceforscalable robot learning,

C.Yuan,C.Wen,T.Zhang,andY.Gao,“Generalflowasfoundationaffordanceforscalable robot learning,”arXiv [cs.RO], Jan. 2024

2024
[149]

From one hand to multiple hands: Imitation learning for dexterous manipulation from single-camera teleoperation,

Y. Qin, H. Su, and X. Wang, “From one hand to multiple hands: Imitation learning for dexterous manipulation from single-camera teleoperation,”arXiv [cs.RO], Apr. 2022

2022
[150]

VideoDex: Learning dexterity from internet videos,

K. Shaw, S. Bahl, and D. Pathak, “VideoDex: Learning dexterity from internet videos,” arXiv [cs.RO], Dec. 2022

2022
[151]

Neural dynamic policies for end-to-end sensorimotor learning,

S. Bahl, M. Mukadam, A. Gupta, and D. Pathak, “Neural dynamic policies for end-to-end sensorimotor learning,”arXiv [cs.LG], Dec. 2020. 114

2020
[152]

Teach a robot to FISH: Versatile imitation from one minute of demonstrations,

S. Haldar, J. Pari, A. Rai, and L. Pinto, “Teach a robot to FISH: Versatile imitation from one minute of demonstrations,”arXiv [cs.RO], Mar. 2023

2023
[153]

MimicPlay: Long-horizon imitation learning by watching human play,

C. Wang et al., “MimicPlay: Long-horizon imitation learning by watching human play,” arXiv [cs.RO], Feb. 2023

2023
[154]

Imitating latent policies from observation

A. D. Edwards, H. Sahni, Y. Schroecker, and C. L. Isbell, “Imitating latent policies from observation.”
[155]

Learning to act without actions,

D. Schmidt and M. Jiang, “Learning to act without actions,” Oct. 2023

2023
[156]

Pushing the limits of cross-embodiment learning for manipulation and navigation,

J. Yang et al., “Pushing the limits of cross-embodiment learning for manipulation and navigation,”arXiv [cs.RO], Feb. 2024

2024
[157]

Elastic tactile sensor glove for dexterous teaching by demonstra- tion,

P. Ruppel and J. Zhang, “Elastic tactile sensor glove for dexterous teaching by demonstra- tion,”Sensors (Basel), vol. 24, no. 6, p. 1912, Mar. 2024

1912
[158]

Sensory glove for dynamic hand proprioception and tactile sensing,

J. C. S. McCaw, M. C. Yuen, and R. Kramer-Bottiglio, “Sensory glove for dynamic hand proprioception and tactile sensing,” inVolume 2B: 44th Design Automation Conference, American Society of Mechanical Engineers, Aug. 2018, V02BT03A025

2018
[160]

Learning the signatures of the human grasp using a scalable tactile glove,

S. Sundaram, P. Kellnhofer, Y. Li, J.-Y. Zhu, A. Torralba, and W. Matusik, “Learning the signatures of the human grasp using a scalable tactile glove,”Nature, vol. 569, no. 7758, pp. 698–702, May 2019

2019
[161]

AwearableRObotichandforhand-over-handimitationlearning,

D.WeiandH.Xu,“AwearableRObotichandforhand-over-handimitationlearning,” arXiv [cs.RO], Sep. 2023

2023
[162]

Developmentandap- plicationsofhigh-densitytactilesensingglove,

T.Sagisaka,Y.Ohmura,A.Nagakubo,K.Ozaki,andY.Kuniyoshi,“Developmentandap- plicationsofhigh-densitytactilesensingglove,”in Haptics:Perception,Devices,Mobility, and Communication, ser. Lecture notes in computer science, Berlin, Heidelberg: Springer Berlin Heidelberg, 2012, pp. 445–456

2012
[163]

Tactile gloves for autonomousgraspingwiththeNASA/DARPArobonaut,

T. B. Martin, R. O. Ambrose, M. A. Diftler, R. Platt, and M. J. Butzer, “Tactile gloves for autonomousgraspingwiththeNASA/DARPArobonaut,”in IEEEInternationalConference onRoboticsandAutomation,2004.Proceedings.ICRA’04.2004 ,vol.2,IEEE,2004,1713– 1718 Vol.2

2004
[164]

High-density con- formable tactile sensing glove,

T. Sagisaka, Y. Ohmura, Y. Kuniyoshi, A. Nagakubo, and K. Ozaki, “High-density con- formable tactile sensing glove,” in2011 11th IEEE-RAS International Conference on Hu- manoid Robots, IEEE, Oct. 2011, pp. 537–542. 115

2011
[165]

Tactile sensorized glove for force and motion sensing,

J. C. Yeo, C. Lee, Z. Wang, and C. T. Lim, “Tactile sensorized glove for force and motion sensing,” in2016 IEEE SENSORS, IEEE, Oct. 2016, pp. 1–3

2016
[166]

A tactile glove design and authoring system for immersive multimedia,

Y. Kim, J. Cha, J. Ryu, and I. Oakley, “A tactile glove design and authoring system for immersive multimedia,”IEEE Multimed., vol. 17, no. 3, pp. 34–45, 2010

2010
[167]

On the sample complexity of reinforcement learning,

S. M. Kakade, “On the sample complexity of reinforcement learning,” Ph.D. dissertation, University of London, University College London (United Kingdom), Ann Arbor, United States, 2003

2003
[168]

Thelinearprogrammingapproachtoapproximatedynamic programming,

D.P.deFariasandB.VanRoy,“Thelinearprogrammingapproachtoapproximatedynamic programming,”Oper. Res., vol. 51, no. 6, pp. 850–865, Dec. 2003

2003
[169]

Proximal policy opti- mization algorithms,

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy opti- mization algorithms,”arXiv [cs.LG], Jul. 2017

2017
[170]

Addressing function approximation error in actor-critic methods,

S. Fujimoto, H. van Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,”arXiv [cs.AI], Feb. 2018

2018
[171]

Complexin-handmanipula- tionviacompliance-enabledfingergaitingandmulti-modalplanning,

A.S.Morgan,K.Hang,B.Wen,K.Bekris,andA.M.Dollar,“Complexin-handmanipula- tionviacompliance-enabledfingergaitingandmulti-modalplanning,” IEEERobot.Autom. Lett., vol. 7, no. 2, pp. 4821–4828, Apr. 2022

2022
[172]

Rearrangementplanningusingobject-centric and robot-centric action spaces,

J.E.King,M.Cognetti,andS.S.Srinivasa,“Rearrangementplanningusingobject-centric and robot-centric action spaces,” in2016 IEEE International Conference on Robotics and Automation (ICRA), IEEE, May 2016

2016
[173]

MoDem:Accelerating visualmodel-basedreinforcementlearningwithdemonstrations,

N.Hansen,Y.Lin,H.Su,X.Wang,V.Kumar,andA.Rajeswaran,“MoDem:Accelerating visualmodel-basedreinforcementlearningwithdemonstrations,” arXiv[cs.LG],Dec.2022

2022
[174]

Imitation bootstrapped reinforcement learning,

H. Hu, S. Mirchandani, and D. Sadigh, “Imitation bootstrapped reinforcement learning,” arXiv [cs.LG], Nov. 2023

2023
[175]

A sensorized multicurved robot finger with data-driven touch sensing via overlapping light signals,

P. Piacenza, K. Behrman, B. Schifferer, I. Kymissis, and M. Ciocarlie, “A sensorized multicurved robot finger with data-driven touch sensing via overlapping light signals,” arXiv [cs.RO], Apr. 2020

2020
[176]

Asymmetric actor critic for image-based robot learning,

L. Pinto, M. Andrychowicz, P. Welinder, W. Zaremba, and P. Abbeel, “Asymmetric actor critic for image-based robot learning,”arXiv [cs.RO], Oct. 2017

2017
[177]

Learning dexterous in-hand manipulation,

O. M. Andrychowiczet al., “Learning dexterous in-hand manipulation,”Int. J. Rob. Res., vol. 39, no. 1, pp. 3–20, Jan. 2020. 116

2020
[178]

Towards generalization and sim- plicity in continuous control,

A. Rajeswaran, K. Lowrey, E. Todorov, and S. Kakade, “Towards generalization and sim- plicity in continuous control,”Adv. Neural Inf. Process. Syst., vol. abs/1703.02660, Mar. 2017

2017 arXiv
[179]

Behavior generation with latent actions,

S. Lee, Y. Wang, H. Etukuru, H. Jin Kim, N. M. M. Shafiullah, and L. Pinto, “Behavior generation with latent actions,”arXiv [cs.LG], Mar. 2024. 117

2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [109]

Optimalityandapproximationwith policy gradient methods in markov decision processes,

A.Agarwal,S.M.Kakade,J.D.Lee,andG.Mahajan,“Optimalityandapproximationwith policy gradient methods in markov decision processes,” inProceedings of Thirty Third Conference on Learning Theory, J. Abernethy and S. Agarwal, Eds., ser. Proceedings of Machine Learning Research, vol. 125, PMLR, 2020, pp. 64–66. 111

work page 2020

[2] [1]

Ritter and R

H. Ritter and R. Haschke,Hands, Dexterity, and the Brain. CRC Press/Taylor & Francis, 2015

2015

[3] [2]

The coevolution of encephalization and manual dexterity in hominins and other primates,

J. Baker, R. A. Barton, and C. Venditti, “The coevolution of encephalization and manual dexterity in hominins and other primates,”bioRxiv, Sep. 2024

2024

[4] [3]

Developmentofin-handmanipulationand relationship with activities,

R.Humphry,K.Jewell,andR.C.Rosenberger,“Developmentofin-handmanipulationand relationship with activities,”Am. J. Occup. Ther., vol. 49, no. 8, pp. 763–771, Sep. 1995

1995

[5] [4]

How babies use their hands to learn about objects: Exploration, reach-to-grasp, manipulation, and tool use,

A. W. Needham and E. L. Nelson, “How babies use their hands to learn about objects: Exploration, reach-to-grasp, manipulation, and tool use,”Wiley Interdiscip. Rev. Cogn. Sci., vol. 14, no. 6, e1661, Nov. 2023

2023

[6] [5]

Exploring hand dexterity in children with myelomeningocele,

S. Steinhart, E. Kornitzer, P. L. Weiss, and M. Katz-Leurer, “Exploring hand dexterity in children with myelomeningocele,”J. Pediatr. Rehabil. Med., vol. 14, no. 4, pp. 613–619, 2021

2021

[7] [6]

The early motor milestones in infancy and later motor skills in toddlers: A structural equation model of motor development,

H. Viholanen, T. Ahonen, M. Cantell, A. Tolvanen, and H. Lyytinen, “The early motor milestones in infancy and later motor skills in toddlers: A structural equation model of motor development,”Phys. Occup. Ther. Pediatr., vol. 26, no. 1-2, pp. 91–113, 2006

2006

[8] [7]

Role of uncertainty in sensorimotor control,

R. J. van Beers, P. Baraduc, and D. M. Wolpert, “Role of uncertainty in sensorimotor control,”Philos.Trans.R.Soc.Lond.BBiol.Sci. ,vol.357,no.1424,pp.1137–1145,Aug. 2002

2002