Pith. sign in

REVIEW 4 major objections 7 minor 40 references

PRISM: Projection-based Reward Integration for Scene-Aware Real-to-Sim-to-Real Transfer with Few Demonstrations

T0 review · 4 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PRISM claims a robot can learn robust object-manipulation policies from five demonstrations by reconstructing the scene in simulation and letting a vision-language model, prompted with human-provided projection relationships, supply the…

desk verdict A genuinely novel VLM-reward mechanism with an unverified real-to-sim replay foundation and too-thin evaluation. read the letter →

arxiv 2504.20520 v1 pith:24HJ23SN submitted 2025-04-29 cs.RO cs.AI

Haowen Sun , Han Wang , Chengzhong Ma , Shaolong Zhang , Jiawei Ye , Xingyu Chen , Xuguang Lan This is my paper · ORCID
classification cs.ROcs.AI
keywords real-to-sim-to-realtransferroboticmanipulationreinforcementlearningvision-languagemodelrewardfew-shotimitationprojectionrelationshipsactionfeasibilitypredictor6Dobjectposeestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PRISM is an integrated real-to-sim-to-real pipeline for learning robotic manipulation from few demonstrations. It claims that this pipeline lets a robot reach an 82% average success rate across six tasks when both the robot's initial position and the object's pose are randomized, using only five expert demonstrations. The core idea is to replace handcrafted reward engineering with a learned reward model supervised by a vision-language model queried with human-guided projection relationships, then train a reinforcement-learning policy in a simulation scene reconstructed from real images and fine-tune it with the few real demonstrations. A sympathetic reader would care because, if true, this makes RL-based control practical from very little human data and without manual reward design.

What carries the argument

The load-bearing object is the projection-based reward model $r_\psi(\{o_j\}_{j=1}^{N_c}, a)$, a binary classifier over multi-view simulated images and actions, trained with cross-entropy on vision-language-model labels $y\in\{0,1\}$ that encode whether human-guided projection relationships hold across all views. It carries the argument by supplying dense task reward without manual reward engineering. Two supporting mechanisms do the rest of the work: the pose-refinement optimization of Eq. (2), which enforces collision-free, gravity-consistent, trajectory-consistent object poses so demonstrations replay with claimed 100% accuracy, and the action-feasibility predictor $r^\text{real}_\phi(o^\text{RGB-D}_t, a_t)$, a single-view RGB-D classifier derived from the reward model that blocks premature or misaligned gripper actions in the real world.

What would settle it

For one demonstration, compare the refined simulated trajectory output by Eq. (2) against the recorded real demonstration: compute the maximum gripper-object distance and the maximum object-table penetration depth at every timestep, and render simulated frames for visual comparison. If any timestep violates the alignment threshold or shows interpenetration, the 100% replay-accuracy premise is false and the policy is being trained on incorrect simulated dynamics.

Watch

Extended reading notes

Core claim

The paper's central claim is that human-guided object projection relationships, defined as the viewpoint-dependent ordering of object projections caused by occlusion, can be converted into a reward signal for reinforcement learning and that this reward signal transfers from simulation to reality. The paper constructs each simulation scene by segmenting the first real image, retrieving matching 3D models, and estimating and refining object poses so that expert demonstrations replay exactly; it then trains a binary projection-based reward model on vision-language-model labels obtained from multi-view simulated images, uses that reward to train a behavior-cloning-augmented soft actor-critic policy in simulation, and finally co-trains the policy on real demonstrations while deriving a single-view RGB-D action-feasibility predictor from the reward model. Experiments on six real manipulation tasks are reported as showing consistently high success under clean and randomized conditions, and ablations are presented attributing most of the gain to the projection-relationship prompts and to the feasibility predictor.

Load-bearing premise

The pose-refinement step is assumed to make simulated replays of expert demonstrations physically accurate and 100% consistent with reality, but the paper reports no quantitative check of that replay accuracy.

Editorial extensions

If this is right

  • Using five demonstrations, a 3D model library, and a vision-language model prompt template, the same pipeline can produce policies that tolerate roughly 10 cm object-pose perturbations and varied robot starts across six manipulation tasks.
  • Reward engineering for a new skill can be reduced to specifying a goal object and a projection relationship, since the prompting template is shared across skills in the same category.
  • The action-feasibility predictor is a transferable filter: attaching it to a generalist policy baseline is reported to improve that baseline's real-world success, suggesting the reward model encodes reusable execution judgments.
  • Increasing demonstration count from five to fifteen does not close the generalization gap for behavior cloning, which supports the paper's argument that RL exploration, not more imitation data, supplies robustness.
  • Four camera viewpoints in simulation are reported sufficient for the vision-language model to label rewards accurately enough to train a working policy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The necessity of projection relationships is demonstrated only for the tested vision-language model and six occlusion-relevant tasks; for transparent objects, non-rigid objects, or heavily occluded end-effectors, a projection-ordering prompt may not carry enough geometric information and a different supervision modality would be needed.
  • Because the vision-language model is the sole source of reward labels, any systematic VLM misjudgment that survives multi-view consistency voting will be baked into the policy; the paper does not isolate how labeling errors at the reported accuracy affect final task success.
  • The pose-refinement objective is built around gripper-object alignment at key states, so tasks whose success criteria are not expressed as gripper-object distances would require new constraint families, limiting the pipeline's generality beyond pick-place-insert-stack style skills.
  • A natural extension is to apply the same projection-prompt reward model to other RL algorithms or other policies as a plug-in filter; the reported improvement when the feasibility predictor is attached to a generalist baseline hints that this signal is not tied to PRISM's own policy.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes PRISM, a real-to-sim-to-real pipeline that builds simulation scenes from a single RGB-D image and a 3D model library, refines object poses from a few expert demonstrations, trains an RL policy with a learned projection-based reward supervised by VLM labels from multi-view observations, co-trains on real demonstrations, and transfers an action-feasibility predictor to the real world. The central empirical claim is that PRISM achieves roughly 82% average success under randomized initial robot and object poses across six manipulation tasks, substantially outperforming behavior cloning and VLA baselines, and that the projection-based reward and co-training are necessary for this performance. The paper also reports a VLM reward-labeling accuracy analysis as a function of the number of viewpoints.

Significance. If the reported results hold, the paper makes a useful contribution: it combines real-to-sim scene construction, VLM-supervised reward learning through human-guided projection relationships, and demonstration co-training in a way that can address the few-demonstration generalization problem. The multi-view consistency check for VLM labels is a sensible way to reduce noisy reward supervision, and the action-feasibility predictor is a practical mechanism for real-world deployment. The paper does not claim machine-checked proofs or release code, so its value rests on the empirical evaluation. The main ideas are plausible and the ablation on the projection reward supports their importance, but the evidence is currently weakened by missing verification of the real-to-sim replay foundation, limited statistical reporting, and several unexplained quantitative claims in the ablation and baseline comparisons.

major comments (4)
  1. [Section IV-A, Eq. (2)] The load-bearing assumption that each expert demonstration maps to a dedicated simulation environment with '100% replay accuracy' is asserted but not quantitatively verified. Equation (2) minimizes the cumulative gripper-to-object distance subject to environment and trajectory constraints, but C_traj only includes 'gripper alignment states and task final states,' not intermediate contact, lift, or placement phases, and the optimization has no dynamics-consistency term. The paper does not report whether replaying the real action sequence in the refined scene actually moves the objects as recorded, whether contacts succeed, or what the trajectory error is. Since all reward learning, RL exploration, and co-training use D_sim, an inaccurate D_sim means the reported Table I success rates cannot be attributed to the real-to-sim mechanism. Please provide a per-task quantitative replay check, such as final-object-pose error, contact-success rate, and trajectory tracking error, or explicitly report the failure cases.
  2. [Section V, Table I and Section V-A] The statistical basis for the central comparison is thin and the baseline setup is incompletely specified. Each cell reports only 10 rollouts, there are no error bars, confidence intervals, or multiple-seed statistics, and Section V states that 'all real-world evaluations use the best policy obtained for each method.' Selecting the best policy across training runs before reporting 10 rollouts can inflate the reported gaps. In addition, it is not stated whether OpenVLA and OCTO were fine-tuned on the demonstration data or used zero-shot with pretrained weights; if the latter, the comparison does not isolate the contribution of learning from few demonstrations. Please report results over at least three seeds with mean and variance, describe the policy-selection protocol, and clarify the VLA training status.
  3. [Section V-B and Section V-A] Several quantitative claims do not follow from the tables. The abstract and contributions state '68% higher average success rate than the baseline' without defining the baseline or the percentage; Section V-A states that co-training gives a '59% improvement,' but the randomized totals in Table I are 13/60 without co-training and 49/60 with co-training, which is a 277% relative improvement and a 36-percentage-point gap. Table II reports only three tasks, yet the text claims projection relationships improve success by 'nearly 70%,' that the action feasibility predictor improves success by 20%, and that adding the predictor to OCTO gives a 37% improvement; these percentages are not derivable from the 10-trial counts in Table II or from the Table I OCTO totals. Please report all tasks, define the comparison denominators, and recompute or correct these percentages.
  4. [Section IV-C and Section V-B] The action feasibility predictor r_real_phi is described as 'derived from the pre-trained projection-based reward model,' but it takes single-view RGB-D input and uses a U-Net architecture with separate RGB and depth encoders, whereas the reward model r_psi takes multi-view RGB images and uses a DINO transformer with an MLP head. The paper does not specify how the predictor is derived from the reward model, whether it is distilled, retrained with new labels, or merely initialized from it, nor does it report the predictor's own accuracy. Because the predictor is a major component of the sim-to-real transfer claim in Table II, please provide the training procedure, labeling source, and a quantitative evaluation of the predictor, including false-positive and false-negative rates.
minor comments (7)
  1. [Abstract and Section I] The '68% higher average success rate' claim is undefined and should be replaced with a precise comparison, such as absolute success rates and relative improvement with the baseline explicitly named.
  2. [Equation (2)] The notation in the optimization above Eq. (2) is unclear: 'Msim,T' in the subscript and the range T are not defined, and the constraint set notation should distinguish the per-demonstration trajectory length from the number of simulation objects.
  3. [Table I caption] The caption says 'clean and distractor initial conditions' while the text says 'randomized and non-randomized conditions'; please reconcile the terminology and define what the distractor or randomization condition includes.
  4. [Section V-A] The sentence 'incorporating co-training leads to a 59% improvement in task success rates' is inconsistent with the numbers in Table I and should be corrected or removed.
  5. [Section V-B] The percentages 'nearly 70%,' '20%,' and '37%' in the projection-relationship and action-feasibility-predictor analyses are not supported by the 10-trial counts in Table II; please report exact numerators and denominators.
  6. [Section V-C, Figure 6] Figure 6 reports labeling accuracy without numeric values on the y-axis or in the text; please provide the actual accuracy numbers per skill and per viewpoint count, and clarify how the ground-truth labels are defined relative to the handcrafted reward functions.
  7. [Section IV-A] The procedure for retrieving or scanning the 3D models in the library is described only as 'online search or scanning'; please provide details of the model sources, the number of models per task, and any manual alignment or scaling steps.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PRISM's derivation chain is self-contained, with the projection-based reward serving as a reward-specification input rather than a fitted prediction.

full rationale

PRISM's central derivation is self-contained. The real-to-sim stage in Section IV-A uses Eq. (2) to refine object poses from expert demonstrations, but the reported real-world success rates are evaluated by task-completion criteria that are not shown to be identical to the projection-based reward labels; no fitted parameter or learned reward model directly encodes the reported success count. The projection-based reward model is trained on VLM labels generated from human-guided prompts, and the policy is then optimized against that reward; this is reward specification, not circular prediction, because the final success criterion (e.g., object lifted, button pressed, object placed) is not, by construction, the same function as the reward label. The '100% replay accuracy' statement in Section IV-A is an unverified assumption about the pose-refinement optimization; Eq. (2) minimizes gripper-object distance subject to environment and trajectory constraints and does not by construction guarantee full dynamics-consistent replay, so it is a correctness risk rather than a circular step. No self-citations are load-bearing, no uniqueness theorem or ansatz is imported from the authors' prior work, and no known result is merely renamed. Accordingly, the paper's claims do not reduce to their inputs, and the circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on hand-chosen noise magnitudes, viewpoint counts, and thresholds, plus assumptions about VLM generalization, projection-relationship sufficiency, pose refinement quality, and simulator fidelity. No new physical entities are introduced.

free parameters (5)
  • epsilon_g (spatial alignment threshold in trajectory constraints)
    Hand-chosen threshold in Eq. (2) for gripper-object alignment in key states; not reported or swept.
  • Object pose perturbation range for RL noise injection = ±10 cm
    Hand-chosen randomization magnitude in simulation training (Section V).
  • End-effector pose noise range = Not specified
    Noise added to the end effector pose during rollouts (Section V).
  • Number of viewpoints Nc = 4
    Simulation uses scene, right, front, and bird views; choice not justified quantitatively.
  • VLM choice = QWen-72B or GPT-4 Vision
    The labeler is a design choice; no comparison between VLMs is reported.
assumptions (5)
  • domain assumption Pretrained VLMs generalize across tasks and environments
    Stated in Section IV-B: 'We assume that the VLMs have been pretrained on large-scale and diverse image-text corpora...'
  • ad hoc to paper Human-guided projection relationships capture task-specific spatial success criteria
    The method's reward definition relies on the human providing correct projection relationships; no analysis of how failures in these relationships affect learning is given.
  • domain assumption The pose refinement optimization in Eq. (2) recovers physically plausible, trajectory-consistent object poses
    Needed for the claimed 100% replay accuracy; no quantitative validation of refined poses is reported.
  • domain assumption Simulator dynamics and visual appearance are close enough to the real world for policy transfer
    Implicit in all real-to-sim-to-real work; the paper does not measure sim-to-real gap directly.
  • domain assumption Multi-view observations reduce VLM labeling errors
    Used as the rationale for multi-view reward labels; supported only by Figure 6's qualitative trend.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PRISM: Projection-based Reward Integration for Scene-Aware Real-to-Sim-to-Real Transfer with Few Demonstrations." pith.science (2026). https://pith.science/paper/24HJ23SN

@misc{pith2026250420520,
  author       = {Pith},
  title        = {Pith review of: PRISM: Projection-based Reward Integration for Scene-Aware Real-to-Sim-to-Real Transfer with Few Demonstrations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/24HJ23SN}},
  note         = {Machine review of arXiv:2504.20520}
}
read the original abstract

Learning from few demonstrations to develop policies robust to variations in robot initial positions and object poses is a problem of significant practical interest in robotics. Compared to imitation learning, which often struggles to generalize from limited samples, reinforcement learning (RL) can autonomously explore to obtain robust behaviors. Training RL agents through direct interaction with the real world is often impractical and unsafe, while building simulation environments requires extensive manual effort, such as designing scenes and crafting task-specific reward functions. To address these challenges, we propose an integrated real-to-sim-to-real pipeline that constructs simulation environments based on expert demonstrations by identifying scene objects from images and retrieving their corresponding 3D models from existing libraries. We introduce a projection-based reward model for RL policy training that is supervised by a vision-language model (VLM) using human-guided object projection relationships as prompts, with the policy further fine-tuned using expert demonstrations. In general, our work focuses on the construction of simulation environments and RL-based policy training, ultimately enabling the deployment of reliable robotic control policies in real-world scenarios.

Figures

Figures reproduced from arXiv: 2504.20520 by the authors.

Figure 1
Figure 1. Human-guided Object Projection Relationships. Multi-view simulated images are generated from scene view, and human-guided object projection relationships are used as prompts to query the VLM to evaluate whether each view satisfies the task-specific spatial requirement. or utilize object configuration files [13] to build controllable, task-specific environments for RL training. While these approaches help reduce doma… view at source ↗
Figure 2
Figure 2. PRISM System Overview. 1) Transfer the real-world scene to the simulator by estimating object poses in the environment and collecting simulation data (see Section IV-A). 2) Train the reward model using human-guided object projection relationships and apply it to RL. Injecting initialization noise in the simulation enhances the robustness of the control policy (see Section IV-B). 3) Fine-tune the learned policy from … view at source ↗
Figure 3
Figure 3. Two-stage VLM querying process for task-specific reward labeling. The pre-task prompt evaluates whether the current state satisfies the condition for executing the action, while the post-task prompt evaluates task completion. This template generalizes across all similar tasks. support—i.e., no object should penetrate the ground or tabletop at any time. These constraints are enforced by the simulator, which continuou… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative Results for Real-world Robot Experiments. The simulation environment is constructed based on RGB-D images and a 3D model library, with additional noise injected to facilitate the learning of a robust control policy. The policy is evaluated on six tasks in t…
Figure 5
Figure 5. Figure 5: Results for Action Feasibility Predictor. The first row shows positions that typically lead to task failure, while the second row displays positions suitable for successful execution. introduce novel states that the policy has not encountered during training, making it…
Figure 6
Figure 6. Figure 6: The accuracy of VLM reward labels. The x-axis represents the number of camera viewpoints in the simulation environment, while the y￾axis shows the percentage where the VLM preference labels are correct and incorrect. Labeling accuracy improves as the number of views in…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 8 canonical work pages

  1. [1]

    Open- vla: An open-source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. , “Open- vla: An open-source vision-language-action model,” arXiv preprint arXiv:2406.09246, 2024

    arXiv 2024

  2. [2]

    Octo: An open-source generalist robot policy,

    Octo Model Team, D. Ghosh, H. Walke, K. Pertsch, K. Black, O. Mees, S. Dasari, J. Hejna, C. Xu, J. Luo, T. Kreiman, Y . Tan, L. Y . Chen, P. Sanketi, Q. Vuong, T. Xiao, D. Sadigh, C. Finn, and S. Levine, “Octo: An open-source generalist robot policy,” in Proceedings of Robotics: Science and Systems , Delft, Netherlands, 2024

    2024

  3. [3]

    Cot-vla: Visual chain-of-thought reasoning for vision-language-action models,

    Q. Zhao, Y . Lu, M. J. Kim, Z. Fu, Z. Zhang, Y . Wu, Z. Li, Q. Ma, S. Han, C. Finn, et al. , “Cot-vla: Visual chain-of-thought reasoning for vision-language-action models,” arXiv preprint arXiv:2503.22020, 2025

    arXiv 2025

  4. [4]

    Rdt-1b: a diffusion foundation model for bimanual manipulation,

    S. Liu, L. Wu, B. Li, H. Tan, H. Chen, Z. Wang, K. Xu, H. Su, and J. Zhu, “Rdt-1b: a diffusion foundation model for bimanual manipulation,” 2024. [Online]. Available: https: //arxiv.org/abs/2410.07864

    arXiv 2024

  5. [5]

    Diffusion policy: Visuomotor policy learning via ac- tion diffusion,

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song, “Diffusion policy: Visuomotor policy learning via ac- tion diffusion,” The International Journal of Robotics Research , p. 02783649241273668, 2023

    2023

  6. [6]

    Cogact: A foundational vision- language-action model for synergizing cognition and action in robotic manipulation,

    Q. Li, Y . Liang, Z. Wang, L. Luo, X. Chen, M. Liao, F. Wei, Y . Deng, S. Xu, Y . Zhang, et al. , “Cogact: A foundational vision- language-action model for synergizing cognition and action in robotic manipulation,” arXiv preprint arXiv:2411.19650 , 2024

    arXiv 2024

  7. [7]

    Reconciling reality through simulation: A real- to-sim-to-real approach for robust manipulation,

    M. Torne, A. Simeonov, Z. Li, A. Chan, T. Chen, A. Gupta, and P. Agrawal, “Reconciling reality through simulation: A real- to-sim-to-real approach for robust manipulation,” arXiv preprint arXiv:2403.03949, 2024

    arXiv 2024

  8. [8]

    Real–sim–real transfer for real-world robot control policy learning with deep reinforcement learning,

    N. Liu, Y . Cai, T. Lu, R. Wang, and S. Wang, “Real–sim–real transfer for real-world robot control policy learning with deep reinforcement learning,” Applied Sciences, vol. 10, no. 5, p. 1555, 2020

    work page 2020

Show all 40 references
  1. [9]

    A real2sim2real method for robust object grasping with neural surface reconstruction,

    L. Wang, R. Guo, Q. Vuong, Y . Qin, H. Su, and H. Christensen, “A real2sim2real method for robust object grasping with neural surface reconstruction,” in 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE) . IEEE, 2023, pp. 1–8

    2023

  2. [10]

    3d gaussian splatting for real-time radiance field rendering

    B. Kerbl, G. Kopanas, T. Leimk ¨uhler, and G. Drettakis, “3d gaussian splatting for real-time radiance field rendering.” ACM Trans. Graph., vol. 42, no. 4, pp. 139–1, 2023

    2023

  3. [11]

    Splatsim: Zero-shot sim2real transfer of rgb manipulation policies using gaussian splatting,

    M. N. Qureshi, S. Garg, F. Yandun, D. Held, G. Kantor, and A. Silwal, “Splatsim: Zero-shot sim2real transfer of rgb manipulation policies using gaussian splatting,” 2024. [Online]. Available: https://arxiv.org/abs/2409.10161

    2024 arXiv

  4. [12]

    Robogsim: A real2sim2real robotic gaussian splatting simulator,

    X. Li, J. Li, Z. Zhang, R. Zhang, F. Jia, T. Wang, H. Fan, K.-K. Tseng, and R. Wang, “Robogsim: A real2sim2real robotic gaussian splatting simulator,” arXiv preprint arXiv:2411.11839 , 2024

    2024 arXiv

  5. [13]

    Urdformer: A pipeline for constructing articulated simulation environments from real-world images,

    Z. Chen, A. Walsman, M. Memmel, K. Mo, A. Fang, K. Vemuri, A. Wu, D. Fox, and A. Gupta, “Urdformer: A pipeline for constructing articulated simulation environments from real-world images,” arXiv preprint arXiv:2405.11656, 2024

    2024 arXiv

  6. [14]

    A system for general in-hand object re-orientation,

    T. Chen, J. Xu, and P. Agrawal, “A system for general in-hand object re-orientation,” in Conference on Robot Learning . PMLR, 2022, pp. 297–307

    2022

  7. [15]

    Visual dexterity: In-hand reorientation of novel and complex object shapes,

    T. Chen, M. Tippur, S. Wu, V . Kumar, E. Adelson, and P. Agrawal, “Visual dexterity: In-hand reorientation of novel and complex object shapes,” Science Robotics, vol. 8, no. 84, p. eadc9244, 2023

    2023

  8. [16]

    Learning dexterous in-hand manipulation,

    O. M. Andrychowicz, B. Baker, M. Chociej, R. Jozefowicz, B. Mc- Grew, J. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, et al., “Learning dexterous in-hand manipulation,” The International Journal of Robotics Research , vol. 39, no. 1, pp. 3–20, 2020

    2020

  9. [17]

    Dextreme: Transfer of agile in-hand manipulation from simu- lation to reality,

    A. Handa, A. Allshire, V . Makoviychuk, A. Petrenko, R. Singh, J. Liu, D. Makoviichuk, K. Van Wyk, A. Zhurkevich, B. Sundaralingam, et al., “Dextreme: Transfer of agile in-hand manipulation from simu- lation to reality,” in 2023 IEEE International Conference on Robotics and Au...

    2023

  10. [18]

    Real2sim2real: Self-supervised learning of physical single-step dynamic actions for planar robot casting,

    V . Lim, H. Huang, L. Y . Chen, J. Wang, J. Ichnowski, D. Seita, M. Laskey, and K. Goldberg, “Real2sim2real: Self-supervised learning of physical single-step dynamic actions for planar robot casting,” in 2022 International Conference on Robotics and Automation (ICRA) . IEEE, 2...

    2022

  11. [19]

    Rl- gsbridge: 3d gaussian splatting based real2sim2real method for robotic manipulation learning,

    Y . Wu, L. Pan, W. Wu, G. Wang, Y . Miao, F. Xu, and H. Wang, “Rl- gsbridge: 3d gaussian splatting based real2sim2real method for robotic manipulation learning,” arXiv preprint arXiv:2409.20291 , 2024

    2024 arXiv

  12. [20]

    Retinagan: An object-aware approach to sim-to-real transfer,

    D. Ho, K. Rao, Z. Xu, E. Jang, M. Khansari, and Y . Bai, “Retinagan: An object-aware approach to sim-to-real transfer,” in 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021, pp. 10 920–10 926

    2021

  13. [21]

    Genaug: Retargeting behaviors to unseen situations via generative augmentation,

    Z. Chen, S. Kiami, A. Gupta, and V . Kumar, “Genaug: Retargeting behaviors to unseen situations via generative augmentation,” arXiv preprint arXiv:2302.06671, 2023

    2023 arXiv

  14. [22]

    Cacti: A framework for scalable multi-task multi-scene visual imitation learning,

    Z. Mandi, H. Bharadhwaj, V . Moens, S. Song, A. Rajeswaran, and V . Kumar, “Cacti: A framework for scalable multi-task multi-scene visual imitation learning,” arXiv preprint arXiv:2212.05711 , 2022

    2022 arXiv

  15. [23]

    Language to rewards for robotic skill synthesis,

    W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. G. Arenas, H.- T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik, et al., “Language to rewards for robotic skill synthesis,” arXiv preprint arXiv:2306.08647, 2023

    2023 arXiv

  16. [24]

    Text2reward: Automated dense reward function generation for reinforcement learning,

    T. Xie, S. Zhao, C. H. Wu, Y . Liu, Q. Luo, V . Zhong, Y . Yang, and T. Yu, “Text2reward: Automated dense reward function generation for reinforcement learning,” in International Conference on Learning Rep- resentations (ICLR), 2024 (07/05/2024-11/05/2024, Vienna, Austria) , 2024

    2024

  17. [25]

    Eureka: Human- level reward design via coding large language models,

    Y . J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Ja- yaraman, Y . Zhu, L. Fan, and A. Anandkumar, “Eureka: Human- level reward design via coding large language models,” arXiv preprint arXiv:2310.12931, 2023

    2023 arXiv

  18. [26]

    Motif: Intrinsic motivation from artificial intelligence feedback,

    M. Klissarov, P. D’Oro, S. Sodhani, R. Raileanu, P.-L. Bacon, P. Vin- cent, A. Zhang, and M. Henaff, “Motif: Intrinsic motivation from artificial intelligence feedback,” arXiv preprint arXiv:2310.00166 , 2023

    2023 arXiv

  19. [27]

    Liv: Language-image representations and rewards for robotic control,

    Y . J. Ma, V . Kumar, A. Zhang, O. Bastani, and D. Jayaraman, “Liv: Language-image representations and rewards for robotic control,” in International Conference on Machine Learning . PMLR, 2023, pp. 23 301–23 320

    2023

  20. [28]

    Dreureka: Language model guided sim-to-real transfer,

    Y . J. Ma, W. Liang, H.-J. Wang, S. Wang, Y . Zhu, L. Fan, O. Bastani, and D. Jayaraman, “Dreureka: Language model guided sim-to-real transfer,” arXiv preprint arXiv:2406.01967 , 2024

    2024 arXiv

  21. [29]

    Vision-language models are zero-shot reward models for reinforce- ment learning,

    J. Rocamonde, V . Montesinos, E. Nava, E. Perez, and D. Lindner, “Vision-language models are zero-shot reward models for reinforce- ment learning,” arXiv preprint arXiv:2310.12921 , 2023

    2023 arXiv

  22. [30]

    Lift: Unsupervised reinforcement learning with foundation models as teachers,

    T. Nam, J. Lee, J. Zhang, S. J. Hwang, J. J. Lim, and K. Pertsch, “Lift: Unsupervised reinforcement learning with foundation models as teachers,” arXiv preprint arXiv:2312.08958 , 2023

    2023 arXiv

  23. [31]

    Con- stitutional ai: Harmlessness from ai feedback,

    Y . Bai, S. Kadavath, S. Kundu, A. Askell, J. Kernion, A. Jones, A. Chen, A. Goldie, A. Mirhoseini, C. McKinnon, et al. , “Con- stitutional ai: Harmlessness from ai feedback,” arXiv preprint arXiv:2212.08073, 2022

    2022 arXiv

  24. [32]

    Pebble: Feedback-efficient interac- tive reinforcement learning via relabeling experience and unsupervised pre-training,

    K. Lee, L. Smith, and P. Abbeel, “Pebble: Feedback-efficient interac- tive reinforcement learning via relabeling experience and unsupervised pre-training,” arXiv preprint arXiv:2106.05091 , 2021

    2021 arXiv

  25. [33]

    Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,

    Y . Wang, Z. Sun, J. Zhang, Z. Xian, E. Biyik, D. Held, and Z. Erickson, “Rl-vlm-f: Reinforcement learning from vision language foundation model feedback,” arXiv preprint arXiv:2402.03681 , 2024

    2024 arXiv

  26. [34]

    Deep reinforcement learning from human preferences,

    P. F. Christiano, J. Leike, T. Brown, M. Martic, S. Legg, and D. Amodei, “Deep reinforcement learning from human preferences,” Advances in neural information processing systems , vol. 30, 2017

    2017

  27. [35]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo, P. Dollar, and R. Girshick, “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , October 2023, pp. 4015–4026

    2023

  28. [36]

    FoundationPose: Unified 6d pose estimation and tracking of novel objects,

    B. Wen, W. Yang, J. Kautz, and S. Birchfield, “FoundationPose: Unified 6d pose estimation and tracking of novel objects,” in CVPR, 2024

    2024

  29. [37]

    Sam-6d: Segment anything model meets zero-shot 6d object pose estimation,

    J. Lin, L. Liu, D. Lu, and K. Jia, “Sam-6d: Segment anything model meets zero-shot 6d object pose estimation,” arXiv preprint arXiv:2311.15707, 2023

    2023 arXiv

  30. [38]

    Qwen technical report,

    J. Bai, S. Bai, Y . Chu, Z. Cui, K. Dang, X. Deng, Y . Fan, W. Ge, Y . Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang, S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. ...

    2023 arXiv

  31. [39]

    Openai. gpt-4v(ision) system card

    “Openai. gpt-4v(ision) system card.” 2023

    2023

  32. [40]

    Emerging properties in self-supervised vision trans- formers,

    M. Caron, H. Touvron, I. Misra, H. J ´egou, J. Mairal, P. Bojanowski, and A. Joulin, “Emerging properties in self-supervised vision trans- formers,” in Proceedings of the International Conference on Computer Vision (ICCV), 2021

    2021

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.