SIME: Enhancing Policy Self-Improvement with Modal-level Exploration

Cewu Lu; Hongjie Fang; Jun Lv; Wenye Yu; Yang Jin; Yong-Lu Li

REVIEW 4 major objections 4 minor 38 references

SIME: Enhancing Policy Self-Improvement with Modal-level Exploration

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The blocker to robot self-improvement is data diversity, not volume; annealed latent noise creates that diversity, and one retraining round on the policy's own interactions raises success by about 16.1 percent with no new demonstrations.

desk verdict Solid, well-run self-improvement pipeline for imitation policies; the 'modal-level' mechanism is plausibly real but the paper's own evidence does not yet distinguish new modes from added stochasticity plus selection. read the letter →

arxiv 2505.01396 v1 pith:ION6RKJ7 submitted 2025-05-02 cs.RO cs.AIcs.LG

Yang Jin , Jun Lv , Wenye Yu , Hongjie Fang , Yong-Lu Li , Cewu Lu This is my paper · ORCID

classification cs.ROcs.AIcs.LG

keywords policyself-improvementmodal-levelexplorationdiffusionimitationlearningdataselectionrobotmanipulationlatent-spacenoiseannealed

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Robot policies trained from a handful of human demonstrations often repeat one behavior when left to practice alone, so the data they collect is redundant and fine-tuning on it does not help. The paper claims that the right place to explore is the policy's reasoning space, meaning the latent output of the observation encoder, rather than the action space. Adding annealed Gaussian noise there makes a diffusion policy try distinct solution modes at inference, and filtering the resulting interactions keeps successful portions of hard states and weights segments by predicted value gains. On five simulation tasks with 10-40 demonstrations, one round of this self-improvement raises average success by roughly 16.1 percent, and a real-world cup-stacking experiment shows a larger gain. If this is right, imitation-learning robots can get better from their own practice without additional human data or reward engineering.

What carries the argument

The central object is a modal modulation factor $\gamma(k)$ applied to the observation encoder's output during inference: the encoder output $\psi(o_t)$ is replaced by $\psi(o_t) + \gamma(k)n$, where $n$ is Gaussian noise and $\gamma(k)$ follows a linear annealing schedule, full noise early in denoising and zero at the end. The paper calls this modal-level exploration because it steers the conditional denoising process into different solution basins while preserving precision in the final steps. Two selection mechanisms then decide what the noise-generated data is worth learning from. Inter-demo selection keeps only the successful segments of trials in scenarios whose success rate is below a threshold, discarding easy scenarios and failed trials. Intra-demo selection weights each segment by $\exp((V_{t+H}-V_t)/\beta)$ using value estimates from Implicit Q-Learning, so the corrective portions of long trajectories are learned more strongly. Together they convert raw practice data into a focused training set that, combined with the original demonstrations, improves the policy.

What would settle it

Run the same pipeline with noise injected into a fixed random projection of the observation encoding, matched in variance and annealing; if the success-rate histogram and the self-improvement gain are indistinguishable from SIME's, the specific encoder-level locus is not the active ingredient. A complementary check is to count distinct successful trajectory modes per initial state under modal-level versus action-level noise; if the number of modes is unchanged and only failure variance grows, the diversity claim is refuted.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a robot's ability to improve from its own experience is limited less by the size of the interaction dataset than by its diversity, and diversity is best created at the level of the policy's internal representation. Concretely, a diffusion policy's conditional latent $z_t = \psi(o_t)$ is perturbed as $z_t = \psi(o_t) + \gamma(k)n$ with $n \sim \mathcal{N}(0,\sigma^2 I)$ and an annealed gain $\gamma(k)$ that fades as denoising proceeds. This produces distinct behavioral modes, such as grasping the same object from different sides, whereas action-level noise or maximum diffusion randomness mostly fails to change the behavior. When the resulting trajectories are filtered to keep successful portions of low-success-rate scenarios and reweighted by predicted value gains, the policy trained on the union of original demonstrations and selected self-collected data beats its initial version on every task reported. The headline number is an average success-rate increase of roughly 16.1 percent after one self-improvement round across state- and image-based variants.

Load-bearing premise

The load-bearing premise is that the injected noise makes the robot try genuinely new ways of completing the task, not merely fail more randomly; if most of the added variety is random failure, the gains would come from data selection rather than from modal-level exploration.

Editorial extensions

If this is right

After one round of self-improvement, policies trained on only 10 to 40 demonstrations beat their initial versions on every reported task, with an average relative success gain of about 16.1 percent.
Continuing the process for several rounds keeps improving the policy, while the same pipeline without modal-level exploration shows diminishing gains in later rounds.
Action-level noise and maximum diffusion randomness do not produce useful practice data; the locus of exploration, in the encoder latent rather than the action, is what makes the difference.
Keeping only successful segments from low-success-rate scenarios is crucial: replacing inter-demo selection with random selection at the same sample count nearly removes the gain, and value-weighted intra-demo selection adds a further increment.
Because the method changes only inference-time noise and data filtering, it can be applied as a plug-in to an already trained imitation policy without retraining the architecture.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural test of the mechanism is to measure the number of distinct successful trajectory modes per initial state, for example by clustering trajectories on waypoint geometry; such a count would distinguish genuine multi-modality from high-variance execution, which the paper's success-rate histogram does not do.
If modal-level exploration is the active ingredient, the same latent perturbation should transfer to other conditional generative policies, such as flow-matching or autoregressive action models, since it operates on a generic observation encoder rather than on diffusion-specific components.
The annealed noise profile could be made task-aware: tasks requiring precise contact at the end of the trajectory should keep the noise switched off longer, while tasks needing broad search could anneal earlier; this is a concrete hyperparameter extension the paper does not test.
One could also combine the selected self-collected data with a reward signal and run a value-based or policy-gradient update on top of it, which might extend the one-round gains beyond what weighted imitation alone achieves.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The paper is worth reading and probably worth refereeing. What is actually new: it takes condition-annealed latent perturbation from CADS, applies it to robot diffusion policies at inference, and couples it with two selection stages (keep successful trials in low-success states, weight segments by value increments). The simulation results are carefully reported for four seeds, the ablation table supports that exploration plus selection beats fine-tuning on self-collected data alone, and the multi-round curve is a nice touch. The real-world demonstration, though limited, is honest about being a single task with a single run.

Where I agree with the stress-test note: the central mechanism is not directly measured. Fig. 3 shows a wider per-state success-rate distribution, but that is exactly what you'd get from action-level stochasticity. Fig. 4 and Fig. 6 are qualitative. There is no count of distinct behavioral modes, no metric for 'mode diversity' as opposed to variance. So the paper's headline claim—that modal-level exploration creates genuinely new on-task strategies—rests on an inference rather than a measurement. That is the load-bearing soft spot. I would not call it fatal: the selection results are strong enough that the pipeline works even if the mechanism turns out to be 'stochastic exploration plus filtering.' But the title and abstract oversell the modal-level story.

Second soft spot, also real: the main SIME-vs-baseline comparison fixes training iterations but not selected-data volume. Table III shows selection alone contributes most of the gain (10.8 of 11.6 percentage points), and random selection at matched volume nearly kills the improvement. So the comparison in Tables I-II may partly reflect the exploration method simply generating more usable successful data, not better modes. That confound is not addressed.

Minor issues: real-world table has no variance, hyperparameters (sigma, kappa1, kappa2, beta, IQL details) are underspecified, and there is no empirical comparison to [27]'s novelty filtering, which is the closest prior method. All fixable.

I disagree with the reader on one point: the circularity burden is low, and the paper does not define a target quantity by its own equations, so I don't see a circularity problem. The selection threshold theta=0.5 is a heuristic, but it is evaluated in Table III, not hidden.

Who is this for: anyone working on autonomous data collection, self-improvement for imitation policies, or practical robot learning with limited demonstrations. It is a solid empirical contribution with a clear, reproducible setup (code and scripts promised), but the theoretical novelty is modest. A serious referee should engage with it, mainly to push the authors to either measure modes directly or soften the claim. I would accept it for peer review, and I'd ask for the missing measurements and ablations before publication.

Referee Report

4 major / 4 minor

Summary. The paper proposes SIME, a plug-and-play inference-time method for diffusion policies that injects annealed Gaussian noise into the observation encoder output (Eqs. 5-8) in order to induce what the authors call modal-level exploration. The resulting self-collected trajectories are filtered by inter-demo selection (keep successful trials from states with low success rate) and intra-demo selection (value-weighted regression, Eq. 9), then combined with the original demonstrations to fine-tune the policy. The method is evaluated on five RoboMimic tasks in both state and image space, with four seeds per experiment, and on a real-world cup-stacking task. The main empirical claim is that one round of SIME self-improvement yields on average about a 16.1% absolute success-rate gain, roughly twice the gain of the baseline self-improvement procedure, and that further rounds continue to improve performance.

Significance. If the central mechanism holds, SIME is a simple and attractive recipe for improving imitation-learning policies without additional human data: it does not modify the training objective, it is applicable to any diffusion policy with an encoder, and the accompanying code is released. The paper's strengths include multi-seed simulation results with reported standard deviations, ablation and selection tables that separately support the contribution of data selection, and a multi-round experiment showing continued gains. The main weakness is that the hypothesized modal-level exploration mechanism is only supported indirectly; the evidence does not yet distinguish genuinely new behavioral modes from a unimodal stochastic policy, and the headline comparison between SIME and the baseline does not control the amount of selected training data.

major comments (4)

[Sec. V-B, Fig. 3] The diversity analysis does not establish that latent perturbation creates distinct behavioral modes. A wider distribution of per-state success rates over repeated trials is exactly what a unimodal policy with added stochasticity would produce; the statement that 56% of scenarios show partial success 'due to the multi-modal attempts of the policy' is an unsupported causal attribution. The trajectory visualizations in Fig. 4 and Fig. 6 are qualitative and show only one or two examples. Please provide a quantitative mode-level measure, for example by clustering successful trajectories by observed behavior and reporting the number of distinct clusters with and without SIME, or explicitly soften the claim to 'stochastic exploration' if such a measure is not available.
[Sec. V-C, Tables I-II and Sec. V-A2] The comparison between the baseline and SIME does not control the number of selected demonstrations. The statement in Sec. V-A2 that 'we fix the number of training iterations to ensure a fair comparison' controls compute, not data volume; because SIME collects more diverse data and both methods apply inter-demo selection, the selected dataset sizes very likely differ, yet no demo counts are reported in Tables I-II. Since Table III shows that at a fixed size of 40 demos selection alone contributes +10.8 percentage points (Random vs. SR<0.5), the differences in Tables I-II (e.g., +8.6% in vision space) may in part reflect a data-volume confound. Please report the number of selected demonstrations for both methods in the main comparison, or subsample to equal counts as in Table III.
[Sec. IV-B, Eqs. (5)-(8)] The central exploration mechanism is under-specified: the noise scale sigma in Eq. (5) and the annealing thresholds kappa1 and kappa2 in Eq. (8) are never given numerical values or a sensitivity analysis, and the inter-demo threshold theta is fixed at 0.5 with no justification. Because the magnitude of the latent perturbation determines how strongly the policy deviates from its learned behavior, the method as described is not reproducible without these values. Please report the hyperparameters used in all experiments and include a sensitivity study (at least for sigma and theta).
[Sec. IV-C, Eq. (9) and Sec. V-F, Table V] The real-world component of the evaluation is a single run with no variance: 10 initial states and 5 attempts each result in 50 trials, and the reported 117.6% improvement is presented without uncertainty. In addition, the automated intra-demo selection via IQL is only sketched; Eq. (9) uses V_t and beta, but the paper does not define how V_t is computed, on which data the IQL value function is trained, or what beta value is used. Please either report multiple real-world runs with variance or clearly label this as a preliminary demonstration, and provide the missing details for the intra-demo selection procedure.

minor comments (4)

[Throughout] There are several typos and rendering artifacts, including 'S elf' and 'M odal-level' in the title, '10 trails' in Sec. V-B, and missing symbol labels in the legend of Fig. 3 ('DP( =0.0)' instead of 'DP(η=0.0)').
[Sec. IV-B] The phrase 'perceptive perturbation' is likely intended to be 'perceptual perturbation'; please rephrase for clarity.
[Sec. V-D, Table III] The table reports Demo Nums for different selection strategies, but it is not stated whether the four seeds are aggregated or whether the numbers correspond to one seed; please clarify.
[Sec. II-D] The related-work discussion would benefit from a clearer statement of how SIME differs from the state-novelty filtering of [27]: both methods use selection, but SIME adds an exploration mechanism; the current text says this but could be more explicit about the novelty claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: self-improvement gains are evaluated on held-out task success, not on the selection metric by construction.

full rationale

The central claim is not circular. The modal-level exploration mechanism in Eq. (6) injects annealed Gaussian noise into the observation encoder output and is defined independently of the success-rate quantities used for data selection. The inter-demo selection threshold (θ=0.5) and intra-demo weighted regression in Eq. (9) are heuristics applied to self-collected rollouts, but the reported improvement is measured as task success rate on RoboMimic evaluation trials and real-world cup stacking, not as the diversity or selection score itself. The annealing schedule is explicitly attributed to external prior work [30], and the key ablations compare against action-level noise, diffusion randomness, and random data selection, so the improvement is not forced by the method's own equations. The paper's definition of 'diversity' via per-state success-rate distributions may be a weaker proxy for genuinely distinct behavioral modes, and the modal-level mechanism may not be fully confirmed by the evidence, but that is a validity or measurement concern rather than a circularity. Self-citations in the reference list are limited to related-work examples and are not load-bearing for the core derivation.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The central pipeline depends on several hand-set hyperparameters and assumptions about the diffusion policy's latent space. None of these are derived; they are empirical choices. No new physical entities, forces, or conserved quantities are introduced; the only invented construct is the inference-time latent perturbation, which is a procedural modification, not an entity.

free parameters (5)

sigma (latent noise scale) = not reported
Added to the observation encoder output in Eq. 5; the magnitude controls exploration intensity and is not specified in the text.
kappa1, kappa2 (annealing schedule thresholds) = not reported
Defined in Eq. 8; they control how long exploration noise acts during denoising and no values or sensitivity analysis are given.
theta (inter-demo success threshold) = 0.5
Set in Section V-C; states with success rate below 0.5 are treated as challenging, and only successful portions are kept.
beta (value-reweighting temperature) = not reported
Appears in Eq. 9 as the temperature of exp((V_{t+H}-V_t)/beta); no value or tuning procedure is provided.
per-task demonstration counts = 10/20/20/20/40
Lift uses 10, Can/Square/Transport use 20, and ToolHang uses 40 (Section V-A); hand-chosen to create a few-shot setting, affecting the baseline and final results.

assumptions (6)

standard math The diffusion policy training objective and denoising process (Eqs. 1-3) describe the policy and remain valid after latent perturbation during inference.
Relies on the established diffusion policy formulation [6]; used throughout Sections III and IV.
ad hoc to paper The observation encoder output psi(o_t) is a safe space to inject noise, so perturbed latents still correspond to plausible action distributions.
Eq. 6 introduces z_t = psi(o_t) + gamma(k)n without a derivation; this is the core assumption of modal-level exploration.
domain assumption The linear annealing schedule gamma(k) (Eq. 8) preserves high-precision action generation near the end of denoising.
Motivated by CADS [30]; no proof is given, and kappa1 and kappa2 are not fixed in the text.
domain assumption Successful attempts in states with low success rate (SR < 0.5) carry the highest learning value.
Inter-demo selection heuristic in Section IV-C1; no independent evidence beyond the reported gains.
domain assumption IQL value estimates can identify corrective high-value segments for reweighted behavior cloning (Eq. 9).
Intra-demo selection relies on an IQL-trained value function and a temperature beta, neither fully specified.
domain assumption Multiple attempts from the same initial state are a valid proxy for exploration diversity and learning value.
Used in the diversity analysis and inter-demo selection (Section V-B); in the real-world experiment, states are only approximately reset.

how reviews work

0 comments

Cite this review

Pith. "Pith review of SIME: Enhancing Policy Self-Improvement with Modal-level Exploration." pith.science (2026). https://pith.science/paper/ION6RKJ7

@misc{pith2026250501396,
  author       = {Pith},
  title        = {Pith review of: SIME: Enhancing Policy Self-Improvement with Modal-level Exploration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ION6RKJ7}},
  note         = {Machine review of arXiv:2505.01396}
}

read the original abstract

Self-improvement requires robotic systems to initially learn from human-provided data and then gradually enhance their capabilities through interaction with the environment. This is similar to how humans improve their skills through continuous practice. However, achieving effective self-improvement is challenging, primarily because robots tend to repeat their existing abilities during interactions, often failing to generate new, valuable data for learning. In this paper, we identify the key to successful self-improvement: modal-level exploration and data selection. By incorporating a modal-level exploration mechanism during policy execution, the robot can produce more diverse and multi-modal interactions. At the same time, we select the most valuable trials and high-quality segments from these interactions for learning. We successfully demonstrate effective robot self-improvement on both simulation benchmarks and real-world experiments. The capability for self-improvement will enable us to develop more robust and high-success-rate robotic control strategies at a lower cost. Our code and experiment scripts are available at https://ericjin2002.github.io/SIME/

Figures

Figures reproduced from arXiv: 2505.01396 by the authors.

**Figure 1.** Overview of SIME. With modal-level exploration, the robot can generate more diverse and multi-modal interaction data. By learning from the most valuable trials and high-quality segments from these interactions, the robot can effectively refine its capabilities through self-improvement. goal is to reduce dependence on human-collected data while allowing policies to refine their skills beyond the limitations of initia… view at source ↗

**Figure 2.** Pipeline overview. The robot learns a policy from human demonstrations, explores multi-modal interaction behaviors in the reasoning space, collects and selects valuable trajectories and segments, and refines the policy. B. Modal-level Exploration To address this issue, we propose a simple yet effective approach, SIME, to enhance the modal-level exploration capabilities of the diffusion policy during inference. Our… view at source ↗

**Figure 4.** , before introducing modal-level exploration, the policy Without Ours With Ours [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Figures from the paper (3 more)

**Figure 3.** Figure 3: Diversity analysis. We tested 1,000 different scenarios, conducting 10 trials for each. After independently calculating the success rate for each scenario, we analyzed the distribution of success rates. As shown, modal-level exploration allows the policy to exhibit div…

**Figure 5.** Figure 5: 0 1 2 3 4 5 Round 0.3 0.4 0.5 0.6 0.7 0.8 Success Rate Can-20(Baseline) Can-20(Ours) Can-10(Baseline) Can-10(Ours) [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Keyframes of real-world manipulation. During two different trials, the policy tries to grab the cup in two different ways (from the left side and the right side). randomly initiated 10 different states and offered the policy 5 attempts to complete the task from each in…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 13 canonical work pages

[27]

So You Think You Can Scale Up Autonomous Robot Data Collection?

Suvir Mirchandani et al. “So You Think You Can Scale Up Autonomous Robot Data Collection?” In: Conference on Robot Learning . 2024

work page 2024
[1]

Gpt-4 technical report

Josh Achiam et al. “Gpt-4 technical report”. In: arXiv preprint arXiv:2303.08774 (2023)

arXiv 2023
[2]

From imitation to refinement–residual rl for precise visual assembly

Lars Lien Ankile et al. “From imitation to refinement–residual rl for precise visual assembly”. In: CoRL 2024 Workshop on Mastering Robot Manipulation in a World of Abundant Data . 2024

work page 2024
[3]

Robocat: A self-improving generalist agent for robotic manipulation

Konstantinos Bousmalis et al. “Robocat: A self-improving generalist agent for robotic manipulation”. In: arXiv preprint arXiv:2306.11706 (2023)

arXiv 2023
[4]

Rt-2: Vision-language-action models transfer web knowledge to robotic control

Anthony Brohan et al. “Rt-2: Vision-language-action models transfer web knowledge to robotic control”. In: arXiv preprint arXiv:2307.15818 (2023)

arXiv 2023
[5]

Towards Effective Utilization of Mixed-Quality Demonstrations in Robotic Manipulation via Segment-Level Selection and Optimization

Jingjing Chen et al. “Towards Effective Utilization of Mixed-Quality Demonstrations in Robotic Manipulation via Segment-Level Selection and Optimization”. In: arXiv preprint arXiv:2409.19917 (2024)

arXiv 2024
[6]

Diffusion Policy: Visuomotor Policy Learn- ing via Action Diffusion

Cheng Chi et al. “Diffusion Policy: Visuomotor Policy Learn- ing via Action Diffusion”. In: The International Journal of Robotics Research (2024)

2024
[7]

Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

Cheng Chi et al. “Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots”. In: arXiv preprint arXiv:2402.10329 (2024)

arXiv 2024

Show all 38 references

[8]

Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot

Hao-Shu Fang et al. “Rh20t: A comprehensive robotic dataset for learning diverse skills in one-shot”. In: 2024 IEEE Interna- tional Conference on Robotics and Automation (ICRA) . IEEE. 2024, pp. 653–660

2024
[9]

AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons

Hongjie Fang et al. “AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons”. In: arXiv preprint arXiv:2503.03081 (2025)

2025 arXiv
[10]

Implicit behavioral cloning

Pete Florence et al. “Implicit behavioral cloning”. In: Confer- ence on Robot Learning . PMLR. 2022, pp. 158–168

2022
[11]

Off-policy deep reinforcement learning without exploration

Scott Fujimoto, David Meger, and Doina Precup. “Off-policy deep reinforcement learning without exploration”. In: In- ternational conference on machine learning . PMLR. 2019, pp. 2052–2062

2019
[12]

Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

Tuomas Haarnoja et al. “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor”. In: International conference on machine learning . Pmlr. 2018, pp. 1861–1870

2018
[13]

Teach a robot to fish: Versatile imita- tion from one minute of demonstrations

Siddhant Haldar et al. “Teach a robot to fish: Versatile imita- tion from one minute of demonstrations”. In: arXiv preprint arXiv:2303.01497 (2023)

2023 arXiv
[14]

Deep residual learning for image recog- nition

Kaiming He et al. “Deep residual learning for image recog- nition”. In: Proceedings of the IEEE conference on computer vision and pattern recognition . 2016, pp. 770–778

2016
[15]

Transic: Sim-to-real policy transfer by learning from online correction

Yunfan Jiang et al. “Transic: Sim-to-real policy transfer by learning from online correction”. In: arXiv preprint arXiv:2405.10315 (2024)

2024 arXiv
[16]

DROID: A Large-Scale In-The- Wild Robot Manipulation Dataset

Alexander Khazatsky et al. “DROID: A Large-Scale In-The- Wild Robot Manipulation Dataset”. In: Robotics: Science and Systems. 2024

2024
[17]

Segment anything

Alexander Kirillov et al. “Segment anything”. In: Proceedings of the IEEE/CVF international conference on computer vision . 2023, pp. 4015–4026

2023
[18]

Offline reinforcement learning with implicit q-learning

Ilya Kostrikov, Ashvin Nair, and Sergey Levine. “Offline reinforcement learning with implicit q-learning”. In: arXiv preprint arXiv:2110.06169 (2021)

2021 arXiv
[19]

Conservative q-learning for offline re- inforcement learning

Aviral Kumar et al. “Conservative q-learning for offline re- inforcement learning”. In: Advances in neural information processing systems 33 (2020), pp. 1179–1191

2020
[20]

Continuous control with deep reinforcement learning

Timothy P Lillicrap et al. “Continuous control with deep reinforcement learning”. In: arXiv preprint arXiv:1509.02971 (2015)

2015 arXiv
[21]

Robot learning on the job: Human- in-the-loop autonomy and learning during deployment

Huihan Liu et al. “Robot learning on the job: Human- in-the-loop autonomy and learning during deployment”. In: The International Journal of Robotics Research (2022), p. 02783649241273901

2022
[22]

Precise and Dexterous Robotic Manipu- lation via Human-in-the-Loop Reinforcement Learning

Jianlan Luo et al. “Precise and Dexterous Robotic Manipu- lation via Human-in-the-Loop Reinforcement Learning”. In: arXiv preprint arXiv:2410.21845 (2024)

2024 arXiv
[23]

Serl: A software suite for sample-efficient robotic reinforcement learning

Jianlan Luo et al. “Serl: A software suite for sample-efficient robotic reinforcement learning”. In: 2024 IEEE International Conference on Robotics and Automation (ICRA) . IEEE. 2024, pp. 16961–16969

2024
[24]

Human-agent joint learning for effi- cient robot manipulation skill acquisition

Shengcheng Luo et al. “Human-agent joint learning for effi- cient robot manipulation skill acquisition”. In: arXiv preprint arXiv:2407.00299 (2024)

2024 arXiv
[25]

Sam-rl: Sensing-aware model-based reinforce- ment learning via differentiable physics-based simulation and rendering

Jun Lv et al. “Sam-rl: Sensing-aware model-based reinforce- ment learning via differentiable physics-based simulation and rendering”. In: The International Journal of Robotics Research (2023), p. 02783649241284653

2023
[26]

What matters in learning from offline human demonstrations for robot manipulation

Ajay Mandlekar et al. “What matters in learning from offline human demonstrations for robot manipulation”. In: arXiv preprint arXiv:2108.03298 (2021)

2021 arXiv
[28]

Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collabora- tion

Abby O’Neill et al. “Open X-Embodiment: Robotic Learning Datasets and RT-X Models : Open X-Embodiment Collabora- tion”. In: ICRA. IEEE, 2024, pp. 6892–6903

2024
[29]

Advantage-weighted regression: Simple and scalable off-policy reinforcement learning

Xue Bin Peng et al. “Advantage-weighted regression: Simple and scalable off-policy reinforcement learning”. In: arXiv preprint arXiv:1910.00177 (2019)

2019 arXiv
[30]

CADS: Unleashing the diversity of diffusion models through condition-annealed sampling

Seyedmorteza Sadat et al. “CADS: Unleashing the diversity of diffusion models through condition-annealed sampling”. In: arXiv preprint arXiv:2310.17347 (2023)

2023 arXiv
[31]

Proximal policy optimization algo- rithms

John Schulman et al. “Proximal policy optimization algo- rithms”. In: arXiv preprint arXiv:1707.06347 (2017)

2017 arXiv
[32]

Behavior transformers: Cloning k modes with one stone

Nur Muhammad Shafiullah et al. “Behavior transformers: Cloning k modes with one stone”. In: Advances in neural information processing systems 35 (2022), pp. 22955–22968

2022
[33]

Residual policy learning

Tom Silver et al. “Residual policy learning”. In: arXiv preprint arXiv:1812.06298 (2018)

2018 arXiv
[34]

De- noising diffusion implicit models

Jiaming Song, Chenlin Meng, and Stefano Ermon. “De- noising diffusion implicit models”. In: arXiv preprint arXiv:2010.02502 (2020)

2020 arXiv
[35]

RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effective

Chenxi Wang et al. “RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effective”. In: arXiv preprint arXiv:2404.12281 (2024)

2024 arXiv
[36]

Exponentially weighted imitation learning for batched historical data

Qing Wang et al. “Exponentially weighted imitation learning for batched historical data”. In: Advances in Neural Informa- tion Processing Systems 31 (2018)

2018
[37]

Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model

Xiu Yuan et al. “Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model”. In: arXiv preprint arXiv:2412.13630 (2024)

2024 arXiv
[38]

Learning Fine-Grained Bimanual Manip- ulation with Low-Cost Hardware

Tony Z. Zhao et al. “Learning Fine-Grained Bimanual Manip- ulation with Low-Cost Hardware”. In: Robotics: Science and Systems. 2023

2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [27]

So You Think You Can Scale Up Autonomous Robot Data Collection?

Suvir Mirchandani et al. “So You Think You Can Scale Up Autonomous Robot Data Collection?” In: Conference on Robot Learning . 2024

work page 2024

[2] [1]

Gpt-4 technical report

Josh Achiam et al. “Gpt-4 technical report”. In: arXiv preprint arXiv:2303.08774 (2023)

arXiv 2023

[3] [2]

From imitation to refinement–residual rl for precise visual assembly

Lars Lien Ankile et al. “From imitation to refinement–residual rl for precise visual assembly”. In: CoRL 2024 Workshop on Mastering Robot Manipulation in a World of Abundant Data . 2024

work page 2024

[4] [3]

Robocat: A self-improving generalist agent for robotic manipulation

Konstantinos Bousmalis et al. “Robocat: A self-improving generalist agent for robotic manipulation”. In: arXiv preprint arXiv:2306.11706 (2023)

arXiv 2023

[5] [4]

Rt-2: Vision-language-action models transfer web knowledge to robotic control

Anthony Brohan et al. “Rt-2: Vision-language-action models transfer web knowledge to robotic control”. In: arXiv preprint arXiv:2307.15818 (2023)

arXiv 2023

[6] [5]

Towards Effective Utilization of Mixed-Quality Demonstrations in Robotic Manipulation via Segment-Level Selection and Optimization

Jingjing Chen et al. “Towards Effective Utilization of Mixed-Quality Demonstrations in Robotic Manipulation via Segment-Level Selection and Optimization”. In: arXiv preprint arXiv:2409.19917 (2024)

arXiv 2024

[7] [6]

Diffusion Policy: Visuomotor Policy Learn- ing via Action Diffusion

Cheng Chi et al. “Diffusion Policy: Visuomotor Policy Learn- ing via Action Diffusion”. In: The International Journal of Robotics Research (2024)

2024

[8] [7]

Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

Cheng Chi et al. “Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots”. In: arXiv preprint arXiv:2402.10329 (2024)

arXiv 2024