Pith. sign in

REVIEW 3 major objections 4 minor 50 references

Flow-based Domain Randomization for Learning and Sequencing Robotic Skills

T0 review · 3 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GoFlow learns which random environments to train on, and reuses that distribution as an out-of-distribution detector for multi-step planning.

desk verdict GoFlow is a genuinely new way to learn domain randomization distributions with normalizing flows, and the simulation evidence is solid, but the missing statement of how the flow is restricted to the parameter box is a real hole the authors need to patch. read the letter →

arxiv 2502.01800 v2 pith:4375A3P2 submitted 2025-02-03 cs.RO cs.AIcs.LG

Aidan Curtis , Eric Li , Michael Noseworthy , Nishad Gothoskar , Sachin Chitta , Hui Li , Leslie Pack Kaelbling , Nicole Carey This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords domainrandomizationnormalizingflowssim-to-realtransferreinforcementlearningbelief-spaceplanningout-of-distributiondetectioncontact-richmanipulationgearinsertion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces GoFlow, a domain-randomization method that replaces hand-tuned environment sampling distributions with a normalizing flow trained alongside the policy. The flow is optimized to favor environments that are challenging for the current policy but still solvable, through an entropy-regularized reward objective with a self-paced KL term. Across six simulated domains the trained flow reaches higher coverage than fixed uniform, no randomization, and three learned baselines, and it transfers to a real gear-insertion task with 9/10 successes. The same learned distribution, combined with a privileged value function, is used as an out-of-distribution detector that tells a belief-space planner when to gather more information.

What carries the argument

The central object is the normalizing-flow sampling distribution $p_\phi$ over domain parameters, implemented as a neural spline flow, which is updated by importance sampling from a uniform distribution over the parameter box. The reward term $R = (|\Xi|/B) \sum_i p_\phi(\xi_i) J(\xi_i)$ and entropy term $\hat{H} = -|\Xi| \mathbb{E}_u[p_\phi \log p_\phi]$ are unbiased estimates of the flow's expected reward and differential entropy only when $p_\phi$ is a probability density on that box. A privileged critic $V_\psi(s, \xi)$ that sees the true environment parameters stabilizes policy learning, and the thresholded pair $(p_\phi > \epsilon, V_\psi > J_T)$ defines the belief-space precondition used by the planner.

What would settle it

Compute the integral of the trained flow's density over the randomization box at the end of training (e.g., by sampling uniformly and averaging $|\Xi| p_\phi(\xi)$): if it deviates from 1, or if density outside the box is non-negligible, the reward and entropy gradients in Algorithm 1 are biased. A second check is to retrain GoFlow with the flow density renormalized or clipped to the box and compare coverage curves; if coverage changes substantially, the unnormalized flow was the source of the reported training instability.

Watch

Extended reading notes

Core claim

GoFlow claims that a normalizing-flow sampling distribution $p_\phi(\xi)$, optimized by maximizing $\mathbb{E}_{\xi\sim p}[J_\xi(\pi)] + \alpha H(p) - \beta D_{\mathrm{KL}}(p_{\mathrm{old}}\|p)$, yields policies that succeed on a larger fraction of the environment parameter space than existing learned or fixed domain randomization. The flow's expressivity lets it capture multimodal, off-center, and correlated regions of the parameter space where the task is feasible, which the paper shows is where simpler parametric distributions fail. In real hardware gear insertion under in-hand pose uncertainty, the GoFlow-trained policy succeeded in 9 of 10 trials versus 6 of 10 for full uniform randomization and 3-5 for other baselines. The paper further argues that the learned distribution, thresholded together with a privileged critic, gives a belief-space precondition: if the belief lies inside the flow's high-density region and the expected value exceeds a threshold, the skill is likely to succeed, allowing a BFS planner to decide when to inspect objects more closely.

Load-bearing premise

The flow's density is treated as a probability distribution over the bounded randomization box, so the importance-sampled reward and entropy estimates are unbiased only if the flow integrates to one over that box and assigns no mass outside it; the paper never states how the flow is restricted to the box.

Editorial extensions

If this is right

  • If GoFlow's coverage gains hold beyond the tested domains, learned flows could replace hand tuning of randomization distributions in sim-to-real pipelines, removing a delicate manual step.
  • The learned distribution can be repurposed after training as a precondition for high-level planning, so the same artifacts (flow plus critic) support both skill learning and skill sequencing.
  • Because GoFlow degrades more gracefully than baselines as parameter ranges grow (Appendix A.6), it may enable training on much wider uncertainty intervals than current practice.
  • Real-world gear insertion results suggest contact-rich assembly is a feasible target for fully automated randomization learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The importance-sampling correction treats $p_\phi$ as a density on the bounded box; if the flow leaks mass outside the box or does not integrate to one, the reward and entropy gradients are biased, matching the training instability the paper reports in Section 7. A direct check would be to numerically integrate the learned flow's density over the box during training.
  • Coverage is measured against a uniform test distribution, so 'more coverage' is only as meaningful as the chosen box; whether the found regions correspond to physically plausible real-world variations is not established by the simulated metric.
  • The same flow-as-precondition trick could be applied to other uncertainty-aware planners, since it converts a generative model of training environments into a calibrated success-region estimate without extra real-world data.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces GoFlow, a domain randomization method that maintains a normalizing-flow sampling distribution p_φ(ξ) optimized through an entropy-regularized, self-paced objective (Eq. 5) jointly with a PPO-trained policy. The flow is updated using importance-sampling estimates of the expected return and differential entropy computed from uniform samples over the parameter box (Algorithm 1, Appendix A.2). The authors report coverage comparisons against no DR, uniform DR, ADR, LSDR, and DORAEMON across six simulated domains (Figure 3) and a real-world gear insertion task (Table 1), and they extend the learned distribution plus a privileged value function to belief-space preconditions for multi-step planning (Section 6).

Significance. The central idea—using normalizing flows rather than fixed parametric families for learned domain randomization—is well motivated and, if the optimization is correct, would be a meaningful step over Gaussian and beta-distribution baselines. The paper is thorough in its experimental surface: five seeds, six simulated domains, real-robot trials, sensitivity analyses for parameter range and success threshold, and a code release. The importance-sampling strategy to avoid collapse is a nice practical touch. However, the load-bearing estimation issue described below (normalization of the flow on the bounded parameter box) means the paper as written does not establish that GoFlow optimizes Eq. 5; this must be resolved before the coverage claims can be fully credited.

major comments (3)
  1. [Appendix A.2, Algorithm 1 (lines 7–9)] The derivations of R and H-hat are unbiased estimates of E_{ξ∼p_φ}[J_ξ(π)] and H(p_φ) only if p_φ is a probability density with respect to the uniform measure on Ξ, i.e., ∫_Ξ p_φ dξ = 1 and p_φ = 0 outside Ξ. Section 3.3 defines the flow as an invertible map on R^d with a standard normal base, so unless an explicit bounding procedure (e.g., a final squashing transformation, a bounded base distribution, or renormalized truncation) is applied and described, the estimates in lines 7–8 compute instead ∫_Ξ p_φ J dξ and −∫_Ξ p_φ log p_φ dξ, which are not equal to the stated objective terms. The missing normalization also propagates to the KL term in line 9 and to the threshold ϵ in Eq. (7). The authors should specify exactly how the flow is restricted to Ξ and re-derive the estimates in Appendix A.2 under that restriction; this is load-bearing for the paper's central claim that GoFlow optimizes Eq. 5.
  2. [Appendix A.9 and Tables 1–2] The statistical claims are not supported as reported. The text says pairwise t-tests were performed, but the caption of Table 2 says 'one-tailed z test, α=0.05'; with five seeds per method, a z-test is inappropriate, and a t-test with n=5 has very low power. For the real-world results, Table 1 reports 9/10 for GoFlow versus 6/10 for FullDR; with ten binary trials, a Fisher exact test gives p≈0.30, so the claim of statistical significance in the caption is not justified. Please use appropriate exact or permutation tests, report effect sizes, and temper the significance statements accordingly.
  3. [Appendix A.5] Hyperparameters for GoFlow, LSDR, DORAEMON, and ADR were selected by running a single seed for each setting and choosing the setting with the best coverage. Because coverage is the evaluation metric, this is selection on the test metric; the five-seed curves in Figure 3 are then generated with the selected hyperparameters, which can inflate the reported advantage of GoFlow and the baselines in different ways. Please either report the sweep-averaged performance, select hyperparameters on a separate validation split, or at least quantify how much the single-seed selection shifts the final coverage estimates.
minor comments (4)
  1. [Table 2 caption] The caption says 'Mean and standard error (SDE) of the final reward value' but the text in Section 5.2 defines coverage as the proportion of uniform samples above the threshold; the caption should say 'final coverage value'.
  2. [Algorithm 1, line 8] The entropy term is written as an expectation over u(ξ); since only B samples are drawn, please clarify that it is a Monte Carlo estimate with the same samples used for R.
  3. [Section 5.1] The toy-domain statement that the domain 'violates the feasibility of the center point of the range' is made clear by Figure 2, but the reward and observation model of the energy function should be stated in the text rather than left to the figure.
  4. [Appendix A.8] The phrase 'systematically perturb the end-effector pose by a random ±0.01 meter translational offset' is contradictory; it should read 'randomly perturb' to match the described random offset.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: GoFlow's objective, importance-sampled estimators, and uniform-coverage evaluation form an open-loop chain; same-group citations (Bayes3D, Curtis et al. 2024) are components rather than load-bearing justifications for the central claim.

full rationale

The paper's central derivation chain is self-contained and open-loop. The GoFlow objective (Eq. 5) maximizes E_{ξ~p}[J_ξ(π)] + αH(p) − βD_KL(p_old||p), and no term in it contains the coverage metric, the threshold J_T, or the uniform test distribution u(ξ) that defines the reported coverage, so the headline coverage numbers are not forced by the objective. The importance-sampling estimates in Algorithm 1 (Lines 7-8) are derived in Appendix A.2 as standard identities: E_p[J] = E_u[(p/u)J] and H(p) = −E_u[(p/u)log p], so no fitted quantity is being renamed as a prediction. The one genuine technical gap — the flow is defined on R^d (Section 3.3) while the identities require p_φ to be a probability density on the bounded box Ξ — is an omitted normalization detail that can bias gradients, not a circular reduction. The empirical claims are validated against external baselines (FullDR, NoDR, DORAEMON, LSDR, ADR) on six simulated domains and on a real robot (Table 1, 9/10 insertions), so the comparison does not reduce to the paper's own outputs. Same-group citations appear only as components: Bayes3D supplies the belief update in Section 6.3, and Curtis et al. (2024) supplies the belief-space precondition formalism in Section 6.1, but neither justifies the central coverage claim, which stands on the experiments. The Section 6.2 precondition is mildly self-referential — p_φ was trained on the same reward J whose threshold J_T defines success in Eq. 7, so {p_φ > ε} functions as a training-support mask for the value function — but the resulting multi-step plans were executed successfully on hardware, providing external validation that breaks the loop. The paper itself discloses its real limitations (flow training instability, environment-specific threshold tuning, Section 7), which is consistent with honest non-circular reporting. One statistical caveat that is not circularity: Appendix A.5 selects each method's hyperparameters by the best final coverage on the same environments where coverage is later reported, a selection-bias weakness that does not amount to a definitional reduction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several hand-tuned hyperparameters (alpha, beta, J_T), the assumption that the flow density is properly normalized on the bounded parameter box, and the untested calibration of the value function in the planner section. None of these are derived from first principles.

free parameters (3)
  • entropy coefficient alpha = per environment, chosen from search [0.1, 0.5, 1.0, 1.5, 2.0] (individual best values not reported)
    Controls the trade-off between reward and distribution spread in Eq. 5; selected by a single-seed hyperparameter sweep in A.5.1.
  • self-paced KL coefficient beta = per environment, chosen from search [0.0, 0.1, 0.5, 1.0, 2.0]
    Controls stability versus sample efficiency (A.5.1, Section 7); selected by a single-seed sweep.
  • success threshold J_T = Cartpole 50, Ant 700, Quadcopter 15, Quadruped 1.5, Humanoid 1000, Gears 50
    Hand-selected per environment below the no-randomization optimal performance (A.3); defines the coverage metric and is used to choose hyperparameters; sensitivity analysis in A.7.
assumptions (4)
  • domain assumption The environment parameter space Xi is a bounded box with a well-defined uniform distribution u(xi), and the uniform test distribution is the target for coverage.
    Used throughout Algorithm 1 and the coverage evaluation (A.3, Section 5.2); if the feasible region is not a box or the uniform measure is not the right reference, the reported coverage numbers are less meaningful.
  • ad hoc to paper The policy gradient and the alternating flow updates in Algorithm 1 converge to a useful joint solution.
    No convergence guarantee is given for the coupled max_{p,pi} objective in Eq. 5 with the self-paced KL term; stability is asserted empirically and the authors note flow training instability in Section 7.
  • ad hoc to paper The privileged value function V_psi(s, xi) is calibrated inside the high-density region of the learned flow p_phi(xi) > epsilon, justifying its use in the belief-space precondition (Eq. 7).
    Introduced without calibration evidence; the paper acknowledges in Section 7 that removing the epsilon threshold would require uncertainty quantification of the value function.
  • domain assumption Bayes3D's posterior over object pose is an accurate probabilistic belief for the planner.
    The multi-step planner (Section 6.4) relies on Bayes3D (Gothoskar et al., 2023) for belief updates; no quantitative validation of the planner's success rate is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Flow-based Domain Randomization for Learning and Sequencing Robotic Skills." pith.science (2026). https://pith.science/paper/4375A3P2

@misc{pith2026250201800,
  author       = {Pith},
  title        = {Pith review of: Flow-based Domain Randomization for Learning and Sequencing Robotic Skills},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4375A3P2}},
  note         = {Machine review of arXiv:2502.01800}
}
read the original abstract

Domain randomization in reinforcement learning is an established technique for increasing the robustness of control policies trained in simulation. By randomizing environment properties during training, the learned policy can become robust to uncertainties along the randomized dimensions. While the environment distribution is typically specified by hand, in this paper we investigate automatically discovering a sampling distribution via entropy-regularized reward maximization of a normalizing-flow-based neural sampling distribution. We show that this architecture is more flexible and provides greater robustness than existing approaches that learn simpler, parameterized sampling distributions, as demonstrated in six simulated and one real-world robotics domain. Lastly, we explore how these learned sampling distributions, combined with a privileged value function, can be used for out-of-distribution detection in an uncertainty-aware multi-step manipulation planner.

Figures

Figures reproduced from arXiv: 2502.01800 by the authors.

Figure 1
Figure 1. An architecture diagram for our actor-critic RL training setup using a normalizing flow to seed environment parameters across episodes. 4. Method In this section, we introduce GoFlow, a method for learned domain randomization that goes with the flow by adaptively adjusting the domain randomization process using normal￾izing flows. In traditional domain randomization setups, the distribution p(ξ) is predefined. Howev… view at source ↗
Figure 2
Figure 2. An illustrative domain showing the learned sampling functions over the space of unobserved parameters for the tested baselines. Compared to other learning methods, GoFlow correctly models the multimodality and inter-variable dependencies of the underlying reward function. This toy domain, along with other domains in our experiments, violates some of the assumptions made by prior works, such as the feasibility of the… view at source ↗
Figure 3
Figure 3. The coverage ratio over the target distribution across five random seeds for each of the environments. The bands around each curve indicate the standard error. lem is made difficult by the uncertainty the robot has about the precise location of the gear relative to the hand. The agent must learn to rely on signals of proprioception and force feedback to guide the gear into the gear shaft. The action space consists o… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: A multi-step manipulation plan using probabilistic pose estimation to estimate and update beliefs over time. The three rows show the robot state st, the observation ot, and the robot belief bt at each timestep. The red dotted line in the belief indicates the marginal e…
Figure 5
Figure 5. Figure 5: A visual example of the precondition computation described in Section 6.2 for the gear assembly plan shown in [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: GoFlow hyperparameter sweep results for α 0.0 0.5 1.0 1.5 Timesteps ×10 7 0.00 0.05 0.10 0.15 Coverage Cartpole 0.0 1.5 3.0 Timesteps ×10 6 0.000 0.025 0.050 0.075 Coverage Ant 0 3 6 9 Timesteps ×10 6 0.00 0.02 0.04 0.06 Coverage Quadcopter 0.0 2.5 5.0 7.5 Timesteps ×1…
Figure 7
Figure 7. Figure 7: GoFlow hyperparameter sweep results for β after fixing the best α for each environment 0.0 0.5 1.0 1.5 Timesteps ×10 7 0.00 0.02 0.04 0.06 Coverage Cartpole 0.0 1.5 3.0 Timesteps ×10 6 0.0000 0.0025 0.0050 0.0075 Coverage Ant 0 3 6 9 Timesteps ×10 6 0.05 0.00 0.05 Cove…
Figure 8
Figure 8. Figure 8: LSDR hyperparameter sweep results for αL 0.0 0.5 1.0 1.5 Timesteps ×10 7 0.00 0.02 0.04 Coverage Cartpole 0.0 1.5 3.0 Timesteps ×10 6 0.05 0.00 0.05 Coverage Ant 0 3 6 9 Timesteps ×10 6 0.05 0.00 0.05 Coverage Quadcopter 0.0 2.5 5.0 7.5 Timesteps ×10 6 0.00 0.05 0.10 C…
Figure 9
Figure 9. Figure 9: DORAEMON hyperparameter sweep results for ϵD Method FullDR NoDR DORAEMON LSDR ADR GoFlow Success Rate 6/10 3/10 5/10 5/10 5/10 9/10 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: DORAEMON hyperparameter sweep results for αD after fixing the best ϵD 0.0 0.5 1.0 1.5 Timesteps ×10 7 0.002 0.004 0.006 0.008 Coverage Cartpole 0.0 1.5 3.0 Timesteps ×10 6 0.05 0.00 0.05 Coverage Ant 0 3 6 9 Timesteps ×10 6 0.05 0.00 0.05 Coverage Quadcopter 0.0 2.5 5…
Figure 11
Figure 11. Figure 11: ADR hyperparameter sweep results for t − 1.5 3.0 Timesteps ×10 6 10 6 10 3 10 0 Coverage GOFLOW 1.5 3.0 Timesteps ×10 6 10 6 10 4 10 2 10 0 Coverage FullDR 1.5 3.0 Timesteps ×10 6 10 6 10 4 10 2 10 0 Coverage NoDR 1.5 3.0 Timesteps ×10 6 10 6 10 4 10 2 10 0 Coverage D…
Figure 12
Figure 12. Figure 12: Coverage vs range experiment results discussed in Section A.6 0.0 0.4 0.8 1.2 Multiplier 0.00 0.05 0.10 Coverage Cartpole 0.0 0.4 0.8 1.2 Multiplier 0.0 0.5 1.0 Coverage Ant 0.0 0.4 0.8 1.2 Multiplier 0.0 0.2 0.4 Coverage Quadcopter 0.0 0.4 0.8 1.2 Multiplier 0.0 0.2 …
Figure 13
Figure 13. Figure 13: Coverage vs threshold experiment results discussed in Section A.7 16 [PITH_FULL_IMAGE:figures/full_fig_p016_13.png]
Figure 14
Figure 14. Figure 14: A visualization of the beliefs over the object pose under the initial image (first three columns) and after closer inspection (last three columns) as generated from the posterior of the model described in Section 6.3. The colormap corresponds to the log probability of…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 20 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Distributionally Adaptive Meta Reinforcement Learning

    Ajay, A., Gupta, A., Ghosh, D., Levine, S., and Agrawal, P. Distributionally adaptive meta reinforcement learning, 2023. URL https://arxiv.org/abs/2210.03104

    work page Pith review arXiv 2023

  3. [3]

    Closing the sim-to-real loop: Adapting simulation randomization with real world experience

    Chebotar, Y., Handa, A., Makoviychuk, V., Macklin, M., Issac, J., Ratliff, N., and Fox, D. Closing the sim-to-real loop: Adapting simulation randomization with real world experience. In 2019 International Conference on Robotics and Automation (ICRA), pp.\ 8973--8979. IEEE, 2019

    work page 2019

  4. [4]

    Understanding domain randomization for sim-to-real transfer

    Chen, X., Hu, J., Jin, C., Li, L., and Wang, L. Understanding domain randomization for sim-to-real transfer. CoRR, abs/2110.03239, 2021. URL https://arxiv.org/abs/2110.03239

    arXiv 2021

  5. [5]

    Task-Directed Exploration in Continuous POMDPs for Robotic Manipulation of Articulated Objects

    Curtis, A., Kaelbling, L., and Jain, S. Task-directed exploration in continuous pomdps for robotic manipulation of articulated objects, 2022. URL https://arxiv.org/abs/2212.04554

    work page Pith review arXiv 2022

  6. [6]

    Curtis, A., Matheos, G., Gothoskar, N., Mansinghka, V., Tenenbaum, J., Lozano-Pérez, T., and Kaelbling, L. P. Partially observable task and motion planning with uncertainty and risk awareness, 2024. URL https://arxiv.org/abs/2403.10454

    arXiv 2024

  7. [7]

    Sequential monte carlo samplers

    Del Moral, P., Doucet, A., and Jasra, A. Sequential monte carlo samplers. Journal of the Royal Statistical Society Series B: Statistical Methodology, 68 0 (3): 0 411--436, 2006

    2006

  8. [8]

    Neural spline flows

    Durkan, C., Bekasov, A., Murray, I., and Papamakarios, G. Neural spline flows. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019

    work page 2019

Show all 50 references
  1. [9]

    T., Tenenbaum, J

    Gothoskar, N., Ghavami, M., Li, E., Curtis, A., Noseworthy, M., Chung, K., Patton, B., Freeman, W. T., Tenenbaum, J. B., Klukas, M., et al. Bayes3d: fast learning and inference in structured generative models of 3d objects and scenes. arXiv preprint arXiv:2312.08715, 2023

    2023 arXiv

  2. [10]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp.\ 1861--1870. PMLR, 2018

    2018

  3. [11]

    Vision-force-fused curriculum learning for robotic contact-rich assembly tasks

    Jin, P., Lin, Y., Song, Y., Li, T., and Yang, W. Vision-force-fused curriculum learning for robotic contact-rich assembly tasks. Frontiers in Neurorobotics, 17: 0 1280773, October 2023. doi:10.3389/fnbot.2023.1280773

    2023

  4. [12]

    L., Navarro-Guerrero, N., and Knoll, A

    Josifovski, J., Malmir, M., Klarmann, N., Z agar, B. L., Navarro-Guerrero, N., and Knoll, A. Analysis of randomization effects on sim2real transfer in reinforcement learning for robotic manipulation tasks. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Sys...

    2022

  5. [13]

    and Lozano-Perez, T

    Kaelbling, L. and Lozano-Perez, T. Integrated task and motion planning in belief space. The International Journal of Robotics Research, 32: 0 1194--1227, 08 2013. doi:10.1177/0278364913484072

    2013 doi

  6. [14]

    A probabilistic interpretation of self-paced learning with applications to reinforcement learning

    Klink, P., Abdulsamad, H., Belousov, B., D'Eramo, C., Peters, J., and Pajarinen, J. A probabilistic interpretation of self-paced learning with applications to reinforcement learning. CoRR, abs/2102.13176, 2021. URL https://arxiv.org/abs/2102.13176

    2021 arXiv

  7. [15]

    A., and Peters, J

    Kober, J., Bagnell, J. A., and Peters, J. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research, 32 0 (11): 0 1238--1274, 2013

    2013

  8. [16]

    RL for latent mdps: Regret guarantees and a lower bound

    Kwon, J., Efroni, Y., Caramanis, C., and Mannor, S. RL for latent mdps: Regret guarantees and a lower bound. CoRR, abs/2102.04939, 2021. URL https://arxiv.org/abs/2102.04939

    2021 arXiv

  9. [17]

    Megapose: 6d pose estimation of novel objects via render & compare, 2022

    Labbé, Y., Manuelli, L., Mousavian, A., Tyree, S., Birchfield, S., Tremblay, J., Carpentier, J., Aubry, M., Fox, D., and Sivic, J. Megapose: 6d pose estimation of novel objects via render & compare, 2022. URL https://arxiv.org/abs/2212.06870

    2022 arXiv

  10. [18]

    Learning active task-oriented exploration policies for bridging the sim-to-real gap

    Liang, J., Saxena, S., and Kroemer, O. Learning active task-oriented exploration policies for bridging the sim-to-real gap. CoRR, abs/2006.01952, 2020. URL https://arxiv.org/abs/2006.01952

    2006 arXiv

  11. [19]

    and Li, H

    Luo, J. and Li, H. A learning approach to robot-agnostic force-guided high precision assembly. In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 2151--2157. IEEE, 2021

    2021

  12. [20]

    J., and Paull, L

    Mehta, B., Diaz, M., Golemo, F., Pal, C. J., and Paull, L. Active domain randomization. In Conference on Robot Learning, pp.\ 1162--1176. PMLR, 2020

    2020

  13. [21]

    A., Xue, S., Chen, Y., and Xu, D

    Mishra, U. A., Xue, S., Chen, Y., and Xu, D. Generative skill chaining: Long-horizon skill planning with diffusion models, 2023. URL https://arxiv.org/abs/2401.03360

    2023 arXiv

  14. [22]

    L., Singh, R., Guo, Y., Mazhar, H., Mandlekar, A., Babich, B., State, G., Hutter, M., and Garg, A

    Mittal, M., Yu, C., Yu, Q., Liu, J., Rudin, N., Hoeller, D., Yuan, J. L., Singh, R., Guo, Y., Mazhar, H., Mandlekar, A., Babich, B., State, G., Hutter, M., and Garg, A. Orbit: A unified simulation framework for interactive robot learning environments. IEEE Robotics and Automat...

    2023

  15. [23]

    Mozifian, M., Higuera, J. C. G., Meger, D., and Dudek, G. Learning domain randomization distributions for transfer of locomotion policies. CoRR, abs/1906.00410, 2019. URL http://arxiv.org/abs/1906.00410

    1906 arXiv

  16. [24]

    Assessing transferability from simulation to reality for reinforcement learning

    Muratore, F., Gienger, M., and Peters, J. Assessing transferability from simulation to reality for reinforcement learning. IEEE transactions on pattern analysis and machine intelligence, 43 0 (4): 0 1172--1183, 2019

    2019

  17. [25]

    Bayesian domain randomization for sim-to-real transfer

    Muratore, F., Eilers, C., Gienger, M., and Peters, J. Bayesian domain randomization for sim-to-real transfer. CoRR, abs/2003.02471, 2020. URL https://arxiv.org/abs/2003.02471

    2003 arXiv

  18. [26]

    Neural posterior domain randomization

    Muratore, F., Gruner, T., Wiese, F., Belousov, B., Gienger, M., and Peters, J. Neural posterior domain randomization. In Faust, A., Hsu, D., and Neumann, G. (eds.), Proceedings of the 5th Conference on Robot Learning, volume 164 of Proceedings of Machine Learning Research, pp....

    2022

  19. [27]

    Augmenting reinforcement learning with behavior primitives for diverse manipulation tasks

    Nasiriany, S., Liu, H., and Zhu, Y. Augmenting reinforcement learning with behavior primitives for diverse manipulation tasks. CoRR, abs/2110.03655, 2021. URL https://arxiv.org/abs/2110.03655

    2021 arXiv

  20. [28]

    Forge: Force-guided exploration for robust contact-rich manipulation under uncertainty, 2024

    Noseworthy, M., Tang, B., Wen, B., Handa, A., Roy, N., Fox, D., Ramos, F., Narang, Y., and Akinola, I. Forge: Force-guided exploration for robust contact-rich manipulation under uncertainty, 2024. URL https://arxiv.org/abs/2408.04587

    2024 arXiv

  21. [29]

    Solving rubik's cube with a robot hand

    OpenAI, Akkaya, I., Andrychowicz, M., Chociej, M., Litwin, M., McGrew, B., Petron, A., Paino, A., Plappert, M., Powell, G., Ribas, R., Schneider, J., Tezak, N., Tworek, J., Welinder, P., Weng, L., Yuan, Q., Zaremba, W., and Zhang, L. Solving rubik's cube with a robot hand. CoR...

    1910 arXiv

  22. [30]

    a henb \

    Packer, C., Gao, K., Kos, J., Kr \" a henb \" u hl, P., Koltun, V., and Song, D. Assessing generalization in deep reinforcement learning. CoRR, abs/1810.12282, 2018. URL http://arxiv.org/abs/1810.12282

    2018 arXiv

  23. [31]

    B., Andrychowicz, M., Zaremba, W., and Abbeel, P

    Peng, X. B., Andrychowicz, M., Zaremba, W., and Abbeel, P. Sim-to-real transfer of robotic control with dynamics randomization. CoRR, abs/1710.06537, 2017. URL http://arxiv.org/abs/1710.06537

    2017 arXiv

  24. [32]

    Asymmetric actor critic for image-based robot learning

    Pinto, L., Andrychowicz, M., Welinder, P., Zaremba, W., and Abbeel, P. Asymmetric actor critic for image-based robot learning. CoRR, abs/1710.06542, 2017. URL http://arxiv.org/abs/1710.06542

    2017 arXiv

  25. [33]

    C., and Fox, D

    Ramos, F., Possas, R. C., and Fox, D. Bayessim: adaptive domain randomization via probabilistic inference for robotics simulators. CoRR, abs/1906.01728, 2019. URL http://arxiv.org/abs/1906.01728

    1906 arXiv

  26. [34]

    Z., Dai, H., Burchfiel, B., and Majumdar, A

    Ren, A. Z., Dai, H., Burchfiel, B., and Majumdar, A. Adaptsim: Task-driven simulation adaptation for sim-to-real transfer, 2023. URL https://arxiv.org/abs/2302.04903

    2023 arXiv

  27. [35]

    Rezende, D. J. and Mohamed, S. Variational inference with normalizing flows. In Proceedings of the 32nd International Conference on Machine Learning (ICML). PMLR, 2015

    2015

  28. [36]

    Rozet, F. et al. Zuko : Normalizing flows in pytorch, 2022. URL https://pypi.org/project/zuko

    2022

  29. [37]

    and Hino, H

    Sagawa, S. and Hino, H. Gradual domain adaptation via normalizing flows, 2024. URL https://arxiv.org/abs/2206.11492

    2024 arXiv

  30. [38]

    A., Solowjow, E., and Levine, S

    Schoettler, G., Nair, A., Luo, J., Bahl, S., Ojea, J. A., Solowjow, E., and Levine, S. Deep reinforcement learning for industrial insertion tasks with visual inputs and natural rewards. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 55...

    2020

  31. [39]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017. URL http://arxiv.org/abs/1707.06347

    2017 arXiv

  32. [40]

    Sutton, R. S. and Barto, A. G. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA, 2018. ISBN 0262039249

    2018

  33. [41]

    A., Akinola, I., Handa, A., Sukhatme, G

    Tang, B., Lin, M. A., Akinola, I., Handa, A., Sukhatme, G. S., Ramos, F., Fox, D., and Narang, Y. Industreal: Transferring contact-rich assembly tasks from simulation to reality, 2023 a . URL https://arxiv.org/abs/2305.17110

    2023 arXiv

  34. [42]

    A., Akinola, I., Handa, A., Sukhatme, G

    Tang, B., Lin, M. A., Akinola, I., Handa, A., Sukhatme, G. S., Ramos, F., Fox, D., and Narang, Y. Industreal: Transferring contact-rich assembly tasks from simulation to reality. In Robotics: Science and Systems, 2023 b

    2023

  35. [43]

    Domain randomization via entropy maximization, 2024

    Tiboni, G., Klink, P., Peters, J., Tommasi, T., D'Eramo, C., and Chalvatzaki, G. Domain randomization via entropy maximization, 2024. URL https://arxiv.org/abs/2311.01885

    2024 arXiv

  36. [44]

    Crossing the gap: A deep dive into zero-shot sim-to-real transfer for dynamics

    Valassakis, E., Ding, Z., and Johns, E. Crossing the gap: A deep dive into zero-shot sim-to-real transfer for dynamics. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.\ 5372--5379. IEEE, 2020

    2020

  37. [45]

    Robust fast adaptation from adversarially explicit task distribution generation, 2025

    Wang, C., Lv, Y., Mao, Y., Qu, Y., Xu, Y., and Ji, X. Robust fast adaptation from adversarially explicit task distribution generation, 2025. URL https://arxiv.org/abs/2407.19523

    2025 arXiv

  38. [46]

    Foundationpose: Unified 6d pose estimation and tracking of novel objects, 2024

    Wen, B., Yang, W., Kautz, J., and Birchfield, S. Foundationpose: Unified 6d pose estimation and tracking of novel objects, 2024. URL https://arxiv.org/abs/2312.08344

    2024 arXiv

  39. [47]

    K., and Turk, G

    Yu, W., Liu, C. K., and Turk, G. Policy transfer with strategy optimization. CoRR, abs/1810.05751, 2018. URL http://arxiv.org/abs/1810.05751

    2018 arXiv

  40. [48]

    A modular robotic arm control stack for research: Franka-interface and frankapy

    Zhang, K., Sharma, M., Liang, J., and Kroemer, O. A modular robotic arm control stack for research: Franka-interface and frankapy. arXiv preprint arXiv:2011.02398, 2020

    2011 arXiv

  41. [49]

    Bridging the sim-to-real gap with dynamic compliance tuning for industrial insertion, 2024

    Zhang, X., Tomizuka, M., and Li, H. Bridging the sim-to-real gap with dynamic compliance tuning for industrial insertion, 2024. URL https://arxiv.org/abs/2311.07499

    2024 arXiv

  42. [50]

    The ingredients of real-world robotic reinforcement learning

    Zhu, H., Yu, J., Gupta, A., Shah, D., Hartikainen, K., Singh, A., Kumar, V., and Levine, S. The ingredients of real-world robotic reinforcement learning. arXiv preprint arXiv:2004.12570, 2020

    2004 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.