pith. sign in

arxiv: 2510.17640 · v3 · submitted 2025-10-20 · 💻 cs.RO · cs.AI· cs.LG

RESample: A Robust Data Augmentation Framework via Exploratory Sampling for Robotic Manipulation

Yuquan Xue , Guanxing Lu , Zhenyu Wu , Chuanrui Zhang , Bofang Jia , Zhengyi Gu , Ziwei Wang This is my paper

Pith reviewed 2026-05-18 06:07 UTC · model grok-4.3

classification 💻 cs.RO cs.AIcs.LG
keywords data augmentationexploratory samplingVLA modelsrobotic manipulationimitation learningdistribution coverageOOD robustness
0
0 comments X

The pith

A coverage-guided sampling method adds targeted exploratory trajectories to robot datasets, improving VLA success rates by 12 percent with 10-20 percent extra data.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces RESample to automatically augment robotic manipulation datasets by identifying coverage gaps during policy rollouts and sampling new exploratory actions to fill them. This targets the problem of limited distribution in demonstration data that mostly includes successful trajectories, which hinders handling of out-of-distribution scenarios. A lightweight Coverage Function estimates state density to guide sampling efficiently toward underrepresented regions. If effective, this leads to more robust VLA models that can recover from errors using only a small fraction of additional data. Validation on benchmarks and real tasks shows 12% gains with 10-20% extra samples.

Core claim

RESample uses an exploratory sampling mechanism that detects low-coverage states in the training dataset during rollouts and samples actions to generate new trajectories, directed by a lightweight Coverage Function that quantifies coverage density, thereby extending the data distribution and enhancing model performance in robotic manipulation.

What carries the argument

Exploratory sampling mechanism guided by the lightweight Coverage Function to identify and target low-coverage state regions for data extension.

Load-bearing premise

The Coverage Function accurately captures the state distribution and directs sampling to useful rather than noisy trajectories.

What would settle it

Observing no performance improvement or worse results when using RESample-augmented data versus original data on OOD test tasks would falsify the effectiveness claim.

Figures

Figures reproduced from arXiv: 2510.17640 by Bofang Jia, Chuanrui Zhang, Guanxing Lu, Yuquan Xue, Zhengyi Gu, Zhenyu Wu, Ziwei Wang.

Figure 1
Figure 1. Figure 1: Illustration of OOD challenges in demo-driven imi [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An Overview of RESample Framework. Starting from expert demonstrations (Source Dataset), RESample introduces an exploratory sampling mechanism that leverages policy-critic disagreement to expose OOD states. These challenging states are integrated into an augmented replay buffer and assembled into recovery trajectories, forming an OOD Recovery Dataset. The process establishes a feedback loop: the policy is … view at source ↗
Figure 3
Figure 3. Figure 3: Real-World Experimental Setup. We conduct real￾world experiments on a Galaxea A1 robotic arm equipped with a parallel gripper, a wrist-mounted RealSense D435i camera, and an external RealSense L515 camera for a third-person perspective. IV. EXPERIMENTS To empirically validate the efficacy of our proposed RESam￾ple, a framework for mitigating OOD challenges of imitation learning policies, We conduct a compr… view at source ↗
Figure 4
Figure 4. Figure 4: Sample Comparison. Visualization of action sampled by policy before and after applying our RESample framework. The action generated by the augmented policy demonstrates more robustness, while the updated action-critic significantly expands the safety action boundary with higher Q-values overall. either with or without our framework on each task for 100 epochs with the same hyperparameters as in the simulat… view at source ↗
Figure 5
Figure 5. Figure 5: Cross-Task Augmentation. The augmented data generated from task 2, 4, 5, 8 can be effectively transferred to other tasks within the same category, leading to an additional performance boost of 5-10% on average. Even a marginal decreases in Object (-1.2%) occur due to slight perturbations from forced exploration. Nevertheless, the overall average performance still benefits significantly from the robustness … view at source ↗
Figure 6
Figure 6. Figure 6: Real-World Trajectory Flow. Comparison between the baseline and our RESample framework in real-world stacking cubes task. The policy with our framework has a higher Q-value overall and demonstrates more robust recovery behaviors during the task execution. C. Real-World Experimental Results In real-world experiments, we designed four manipulation tasks of varying complexity to evaluate the practical effec￾t… view at source ↗
Figure 7
Figure 7. Figure 7: Mixing Ratio of Augmented Data. We vary the proportion of augmented data in the training set to assess its impact on policy performance. The optimal performance is achieved with a mixing ratio of 40% augmented data, resulting in a success rate of 89.0%. that do not effectively expose the policy’s weaknesses. In contrast, our full RESample framework achieves the highest success rate of 76.5%, demonstrating … view at source ↗
read the original abstract

Vision-Language-Action (VLA) models have demonstrated remarkable performance on complex tasks through imitation learning in recent robotic manipulation works. Based on large-scale and high-quality demonstration datasets, existing imitation learning method arms VLA models acquired with strong capabilities. However, these datasets that predominantly consist of successful trajectories, are costly to collect and often limited in distribution, leading to capability bottlenecks when faced with out-of-distribution (OOD) scenarios during deployment while unable to recover. To address this issue, we propose an automated data augmentation framework named RESample that effectively improves the distribution coverage of VLA training datasets through the well-designed exploratory sampling mechanism. Specifically, the exploratory sampling mechanism identifies the potential coverage gaps during the policy rollout and actively samples exploratory actions to extend the coverage of training data with high sample efficiency. Furthermore, to effectively reflect the distribution of the training dataset, we propose a lightweight Coverage Function that indicates the coverage density of states in the training dataset, which further guides the exploratory sampling process to focus on low-coverage regions. To validate the effectiveness of our method, we conduct extensive experiments on the LIBERO benchmark as well as a series of real-world robotic tasks, demonstrating a significant performance gain of 12% of our proposed RESample over baselines, with only 10-20% additional samples compared to original training data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript proposes RESample, an automated data augmentation framework for Vision-Language-Action (VLA) models in robotic manipulation. It introduces an exploratory sampling mechanism that identifies coverage gaps during policy rollouts and samples exploratory actions to extend training data distribution. A lightweight Coverage Function is defined to estimate state coverage density from successful trajectories and direct sampling toward low-coverage regions. Experiments on the LIBERO benchmark and real-world tasks report a 12% performance gain over baselines using only 10-20% additional samples.

Significance. If the central claims hold after validation, the framework could improve OOD robustness for imitation-learned VLA policies with minimal extra data collection cost, addressing a practical bottleneck in robotic manipulation. The sample-efficiency focus and automated nature are potentially valuable strengths if the gains are shown to arise from targeted coverage rather than data volume alone.

major comments (2)
  1. Abstract and §4 (Experiments): the reported 12% gain and sample-efficiency claim supply no information on specific baselines, number of random seeds, statistical tests, or controls for total training compute and data volume; without these, it is impossible to determine whether improvements derive from the exploratory mechanism or simply from extra trajectories.
  2. §3.2 (Coverage Function): the function is presented as a lightweight density indicator that guides sampling to low-coverage states, yet no formulation, feature set, or validation against policy recovery metrics is supplied; if it is a generic nearest-neighbor or kernel estimator without action-conditioned or task-relevant features, it risks flagging irrelevant states (background, sensor noise) and the 12% gain would reduce to an effect of data volume.
minor comments (1)
  1. Notation for the Coverage Function and exploratory sampling steps should be introduced with explicit equations or pseudocode rather than prose descriptions alone.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We are grateful to the referee for the thoughtful and constructive review of our manuscript on RESample. The comments have helped us identify areas where additional details and clarifications will strengthen the presentation of our work. Below, we provide point-by-point responses to the major comments. We have revised the manuscript to incorporate the suggested improvements, including expanded experimental details and a more explicit description of the Coverage Function.

read point-by-point responses

  1. Referee: Abstract and §4 (Experiments): the reported 12% gain and sample-efficiency claim supply no information on specific baselines, number of random seeds, statistical tests, or controls for total training compute and data volume; without these, it is impossible to determine whether improvements derive from the exploratory mechanism or simply from extra trajectories.

    Authors: We thank the referee for this observation. Upon review, we recognize that the abstract and the summary in §4 could be more precise regarding the experimental protocol. In the revised manuscript, we have updated the abstract to note that results are averaged over multiple random seeds and include controls for data volume. In §4, we now explicitly list the baselines (including standard VLA fine-tuning and data augmentation via random sampling), report results with 5 random seeds including mean and standard deviation, perform statistical significance testing using t-tests, and provide an ablation study where we add the same volume of non-exploratory samples. This ablation shows that the performance gain is significantly higher with RESample's targeted approach (12% vs. 4% for random addition), confirming that the improvement is due to the exploratory sampling mechanism rather than increased data volume alone. We have also ensured that training compute is matched across comparisons. revision: yes

  2. Referee: §3.2 (Coverage Function): the function is presented as a lightweight density indicator that guides sampling to low-coverage states, yet no formulation, feature set, or validation against policy recovery metrics is supplied; if it is a generic nearest-neighbor or kernel estimator without action-conditioned or task-relevant features, it risks flagging irrelevant states (background, sensor noise) and the 12% gain would reduce to an effect of data volume.

    Authors: We appreciate the referee's concern regarding the Coverage Function in §3.2. The original presentation was indeed concise, and we agree that providing the explicit formulation is necessary to alleviate worries about it being overly generic. In the revised version, we have added the mathematical definition of the Coverage Function as a weighted kernel density estimate over state representations obtained from a frozen vision-language encoder, with features selected to emphasize task-relevant elements such as object positions and gripper states. We include a validation subsection demonstrating that low-coverage scores correlate with higher policy failure rates in recovery tasks. Furthermore, we present an ablation where a generic nearest-neighbor estimator is used instead, resulting in inferior performance, thus validating the importance of the task-relevant feature set. This addresses the risk of flagging irrelevant states like background noise. revision: yes

Circularity Check

0 steps flagged

No circularity: RESample introduces proposed Coverage Function and sampling mechanism without reducing claims to fitted inputs or self-citations by construction.

full rationale

The paper's core contribution is a proposed exploratory sampling framework guided by a newly introduced lightweight Coverage Function that estimates state coverage density from the training dataset. This function is presented as an original design choice rather than a fitted parameter or result derived from the performance gains it enables. No equations or steps in the abstract or described method equate the sampling guidance or reported 12% improvement to the inputs by construction; the gains are validated through external experiments on LIBERO and real-world tasks. The derivation chain remains self-contained with independent empirical support.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the unverified effectiveness of the proposed Coverage Function and the assumption that exploratory actions collected via the sampling mechanism improve rather than degrade policy performance.

axioms (1)
  • domain assumption The Coverage Function provides a faithful indicator of state coverage density that can be used to prioritize sampling.
    Introduced in the abstract to guide the exploratory sampling process toward low-coverage regions.
invented entities (1)
  • Coverage Function no independent evidence
    purpose: Lightweight indicator of coverage density of states in the training dataset to focus exploratory sampling.
    Proposed as a new component to reflect dataset distribution and direct data augmentation.

pith-pipeline@v0.9.0 · 5788 in / 1265 out tokens · 31253 ms · 2026-05-18T06:07:10.459346+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Escaping the Diversity Trap in Robotic Manipulation via Anchor-Centric Adaptation

    cs.RO 2026-05 unverdicted novelty 6.0

    Anchor-Centric Adaptation escapes the diversity trap by prioritizing repeated demonstrations at core anchors over broad coverage, yielding higher success rates under fixed data budgets in robotic manipulation.

Reference graph

Works this paper leans on

32 extracted references · 32 canonical work pages · cited by 1 Pith paper · 14 internal anchors

  1. [1]

    A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning

    S. Ross, G. J. Gordon, and J. A. Bagnell, “A reduction of imitation learning and structured prediction to no- regret online learning,” 2011. [Online]. Available: https: //arxiv.org/abs/1011.0686

    work page internal anchor Pith review Pith/arXiv arXiv 2011

  2. [2]

    Octo: An Open-Source Generalist Robot Policy

    O. M. Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, T. Kreiman, C. Xuet al., “Octo: An open-source generalist robot policy,”arXiv preprint arXiv:2405.12213, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  3. [3]

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0,

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jainet al., “Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0,” inProceedings of International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 6892–6903

    work page 2024

  4. [4]

    OpenVLA: An Open-Source Vision-Language-Action Model

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Bal- akrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. San- ketiet al., “Openvla: An open-source vision-language-action model,”arXiv preprint arXiv:2406.09246, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  5. [5]

    $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichteret al., “π 0: A vision- language-action flow model for general robot control,”arXiv preprint arXiv:2410.24164, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  6. [6]

    RT-1: Robotics Transformer for Real-World Control at Scale

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Hausman, A. Herzog, J. Hsu et al., “Rt-1: Robotics transformer for real-world control at scale,”arXiv preprint arXiv:2212.06817, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  7. [7]

    Causal confusion in imitation learning,

    P. De Haan, D. Jayaraman, and S. Levine, “Causal confusion in imitation learning,”Proceedings of Advances in Neural Information Processing Systems (NeurIPS), vol. 32, 2019

    work page 2019

  8. [8]

    Sim-to-real transfer of robotic control with dynamics ran- domization,

    X. B. Peng, M. Andrychowicz, W. Zaremba, and P. Abbeel, “Sim-to-real transfer of robotic control with dynamics ran- domization,” inProceedings of International Conference on Robotics and Automation (ICRA). IEEE, 2018, pp. 3803–3810

    work page 2018

  9. [9]

    Domain randomization for transferring deep neural networks from simulation to the real world,

    J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, “Domain randomization for transferring deep neural networks from simulation to the real world,” inProceedings of International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 23–30

    work page 2017

  10. [10]

    MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations

    A. Mandlekar, S. Nasiriany, B. Wen, I. Akinola, Y . Narang, L. Fan, Y . Zhu, and D. Fox, “Mimicgen: A data generation sys- tem for scalable robot learning using human demonstrations,” arXiv preprint arXiv:2310.17596, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  11. [11]

    Demogen: Synthetic demonstration genera- tion for data-efficient visuomotor policy learning.arXiv preprint arXiv:2502.16932, 2025

    Z. Xue, S. Deng, Z. Chen, Y . Wang, Z. Yuan, and H. Xu, “De- mogen: Synthetic demonstration generation for data-efficient visuomotor policy learning,”arXiv preprint arXiv:2502.16932, 2025

    work page arXiv 2025

  12. [12]

    Image augmentation is all you need: Regularizing deep reinforcement learning from pixels,

    I. Kostrikov, D. Yarats, and R. Fergus, “Image augmentation is all you need: Regularizing deep reinforcement learning from pixels,”arXiv preprint arXiv:2004.13649, 2020

    work page arXiv 2004

  13. [13]

    Solving Rubik's Cube with a Robot Hand

    I. Akkaya, M. Andrychowicz, M. Chociej, M. Litwin, B. Mc- Grew, A. Petron, A. Paino, M. Plappert, G. Powell, R. Ribas et al., “Solving rubik’s cube with a robot hand,”arXiv preprint arXiv:1910.07113, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1910

  14. [14]

    VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning

    G. Lu, W. Guo, C. Zhang, Y . Zhou, H. Jiang, Z. Gao, Y . Tang, and Z. Wang, “Vla-rl: Towards masterful and general robotic manipulation with scalable reinforcement learning,” arXiv preprint arXiv:2505.18719, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  15. [15]

    Reinbot: Amplifying robot visual-language manipulation with reinforcement learning.arXiv preprint arXiv:2505.07395, 2025

    H. Zhang, Z. Zhuang, H. Zhao, P. Ding, H. Lu, and D. Wang, “Reinbot: Amplifying robot visual-language manipulation with reinforcement learning,”arXiv preprint arXiv:2505.07395, 2025

    work page arXiv 2025

  16. [16]

    Libero: Benchmarking knowledge transfer for lifelong robot learning,

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone, “Libero: Benchmarking knowledge transfer for lifelong robot learning,”Proceedings of Advances in Neural Information Pro- cessing Systems (NeurIPS), vol. 36, pp. 44 776–44 791, 2023

    work page 2023

  17. [17]

    Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey

    R. Shao, W. Li, L. Zhang, R. Zhang, Z. Liu, R. Chen, and L. Nie, “Large vlm-based vision-language-action mod- els for robotic manipulation: A survey,”arXiv preprint arXiv:2508.13073, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  18. [18]

    Rt-2: Vision-language- action models transfer web knowledge to robotic control,

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahidet al., “Rt-2: Vision-language- action models transfer web knowledge to robotic control,” in Conference on Robot Learning (CoRL). PMLR, 2023, pp. 2165–2183

    work page 2023

  19. [19]

    Diffusion transformer policy.arXiv preprint arXiv:2410.15959,

    Z. Hou, T. Zhang, Y . Xiong, H. Pu, C. Zhao, R. Tong, Y . Qiao, J. Dai, and Y . Chen, “Diffusion transformer policy,”arXiv preprint arXiv:2410.15959, 2024

    work page arXiv 2024

  20. [20]

    RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, “Rdt-1b: a diffusion foundation model for bimanual manipulation,”arXiv preprint arXiv:2410.07864, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  21. [21]

    Diffusion policy: Visuomotor policy learning via action diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via action diffusion,”International Journal of Robotics Research (IJRR), p. 02783649241273668, 2023

    work page 2023

  22. [22]

    $\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, M. Y . Galliker, D. Ghosh, L. Groom, K. Hausman, B. Ichter, S. Jakubczak, T. Jones, L. Ke, D. LeBlanc, S. Levine, A. Li-Bell, M. Mothukuri, S. Nair, K. Pertsch, A. Z. Ren, L. X. Shi, L. Smith, J. T. Springenberg, K. Stachowicz, J. Tanner, Q. V...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  23. [23]

    Gemma: Open Models Based on Gemini Research and Technology

    G. Team, T. Mesnard, C. Hardin, R. Dadashi, S. Bhupatiraju, S. Pathak, L. Sifre, M. Rivi `ere, M. S. Kale, J. Loveet al., “Gemma: Open models based on gemini research and technol- ogy,”arXiv preprint arXiv:2403.08295, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  24. [24]

    Imitation is not enough: Robustifying imitation with reinforcement learning for challenging driving scenarios,

    Y . Lu, J. Fu, G. Tucker, X. Pan, E. Bronstein, R. Roelofs, B. Sapp, B. White, A. Faust, S. Whitesonet al., “Imitation is not enough: Robustifying imitation with reinforcement learning for challenging driving scenarios,” in2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 7553–7560

    work page 2023

  25. [25]

    Reinforcement learning with augmented data,

    M. Laskin, K. Lee, A. Stooke, L. Pinto, P. Abbeel, and A. Srini- vas, “Reinforcement learning with augmented data,”Advances in neural information processing systems, vol. 33, pp. 19 884– 19 895, 2020

    work page 2020

  26. [26]

    Sime: En- hancing policy self-improvement with modal-level exploration,

    Y . Jin, J. Lv, W. Yu, H. Fang, Y .-L. Li, and C. Lu, “Sime: En- hancing policy self-improvement with modal-level exploration,” arXiv preprint arXiv:2505.01396, 2025

    work page arXiv 2025

  27. [27]

    Grape: Generalizing robot policy via preference alignment.arXiv preprint arXiv:2411.19309, 2024

    Z. Zhang, K. Zheng, Z. Chen, J. Jang, Y . Li, S. Han, C. Wang, M. Ding, D. Fox, and H. Yao, “Grape: Generalizing robot policy via preference alignment,”arXiv preprint arXiv:2411.19309, 2024

    work page arXiv 2024

  28. [28]

    Challenges of Real-World Reinforcement Learning

    G. Dulac-Arnold, D. Mankowitz, and T. Hester, “Chal- lenges of real-world reinforcement learning,”arXiv preprint arXiv:1904.12901, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1904

  29. [29]

    Concrete Problems in AI Safety

    D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Man ´e, “Concrete problems in ai safety,”arXiv preprint arXiv:1606.06565, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  30. [30]

    Conservative q-learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q-learning for offline reinforcement learning,”Proceedings of Advances in Neural Information Processing Systems (NeurIPS), vol. 33, pp. 1179–1191, 2020

    work page 2020

  31. [31]

    Cal-ql: Calibrated offline rl pre-training for efficient online fine-tuning,

    M. Nakamoto, S. Zhai, A. Singh, M. Sobol Mark, Y . Ma, C. Finn, A. Kumar, and S. Levine, “Cal-ql: Calibrated offline rl pre-training for efficient online fine-tuning,”Advances in Neural Information Processing Systems, vol. 36, pp. 62 244–62 269, 2023

    work page 2023

  32. [32]

    Soft actor- critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor- critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” inInternational conference on machine learning. Pmlr, 2018, pp. 1861–1870

    work page 2018