Visual-Based Forklift Learning System Enabling Zero-Shot Sim2Real Without Real-World Data

Hiroya Makino; Koshi Oishi; Seigo Ito; Teruki Kato

REVIEW 4 major objections 6 minor 34 references

Visual-Based Forklift Learning System Enabling Zero-Shot Sim2Real Without Real-World Data

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A forklift control policy trained entirely in a photorealistic CAD simulation transfers to a physical 1/14-scale forklift and completes pallet loading in 60% of real trials, with no real-world training data.

desk verdict A genuine zero-shot sim2real forklift demo with real hardware, but the headline 60% rests on only ten trials and the 'no heuristics' claim is softer than it looks. read the letter →

arxiv 2412.11503 v2 pith:RCRJNSS4 submitted 2024-12-16 cs.RO

Koshi Oishi , Teruki Kato , Hiroya Makino , Seigo Ito This is my paper · ORCID

classification cs.RO

keywords zero-shotsim2realdeepreinforcementlearningcounterbalanceforkliftpalletloadingdomainrandomizationphotorealisticsimulationvision-basedcontrolsyntheticdata

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a counterbalance forklift can learn pallet loading entirely in a photorealistic, CAD-built simulation and then be deployed on a physical 1/14-scale forklift with no real-world training data or heuristic retuning. The authors train a vision-based deep reinforcement learning policy for approaching the pallet and a supervised decision policy for deciding when to lift, both using only synthetic images, and report a 60% success rate in real pallet-loading trials, matching the average of three human operators, plus a 90% accuracy on the lift decision. If this transfers, it matters because real forklift training is dangerous and data-scarce, and the same recipe could be scaled to full-size machines after safety checks on the small platform. The method's key claim is that photorealism plus domain randomization is enough to cross the sim-to-real gap without collecting any real-world data.

What carries the argument

The load-bearing components are: a photorealistic digital environment built entirely from CAD data, with fluorescent-light reflection modeling and per-environment randomization of speeds, actions, floor and pallet colors, light intensity, and color temperature; an end-to-end vision-based approach policy using two 224 by 224 camera images encoded by a pretrained ResNet, concatenated with velocity, yaw rate, and two past actions, then mapped by a five-layer network to throttle and steering; a proximal policy optimization reward shaped around a clothoid reference trajectory plus penalties for pallet contact, excessive speed, erratic action changes, and freezing; and a supervised decision policy trained on synthetic success and failure image pairs. The role of this machinery is to make the learned image features and control actions robust enough to survive the gap between the CAD simulator and the real small-scale forklift.

What would settle it

Repaint the real pallet with a color outside the ±20% RGB range used in training, or run the task with the starting position outside the randomized green triangle shown in the paper's Fig. 4a; if the success rate collapses, the reported zero-shot transfer is tied to the specific simulation coverage rather than to a general sim2real capability.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that a standard end-to-end approach, using two side-mounted cameras feeding a pretrained ResNet, with throttle and steering actions learned by proximal policy optimization, can be trained in a photorealistic digital environment built from CAD data and transferred directly to a physical 1/14-scale counterbalance forklift with front-wheel drive and rear-wheel steering. The approach policy achieved a 60% success rate over 10 real trials in a 1.8 m enclosure, and the loading decision policy, trained on synthetic success and failure images, was correct in 9 of 10 trials. The authors interpret this as evidence of zero-shot sim2real without heuristic additions, positioning the 1/14-scale platform as a safe validation step toward full-scale counterbalance forklift automation.

Load-bearing premise

The transfer depends on the hand-tuned simulator dynamics and the chosen randomization ranges covering the real 1.8 m test environment; the paper provides no evidence these ranges generalize to other pallets, lighting, or room layouts.

Editorial extensions

If this is right

If the central claim holds, a full-scale counterbalance forklift could be trained with the same pipeline using only CAD data and then validated on the 1/14-scale platform before any full-size trial.
The same combination of photorealistic rendering and domain randomization could be applied to other warehouse vehicles whose real-world data is costly or unsafe to collect.
Because the decision policy was trained entirely on synthetic images, the approach implies that high-level loading decisions can also be learned without real-world failure data.
The demonstrated 60% success rate, equal to the human average in the paper's own comparison, sets a concrete baseline for future sim2real forklift controllers to beat.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

My inference: the fixed pallet location and the small 1.8 m enclosure mean the demonstrated zero-shot claim should be read as environment-specific; moving the pallet or changing the room size would be a stricter test that the paper does not run.
My inference: the reported 60% success equals the average of three untrained human operators, so the method clears a practical baseline, but the single human at 90% shows the task and the policy both have headroom.
A testable extension the paper does not pursue: systematically expanding the domain randomization ranges during training and measuring real success rate would show whether the current ranges are the reason transfer works or merely sufficient for this one room.
If the approach scales, the 1/14-scale platform itself becomes a reusable testbed for counterbalance forklift automation, since it shares the full-size configuration's front-wheel drive, rear-wheel steering, and hydraulic lift.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look: this is the first real-world validation, as far as I can tell, of a vision-based DRL controller for a counterbalance forklift pallet approach trained without any real images or real-world data. The simulator is built from CAD, the domain randomization is standard but not lazy, and the hardware (1/14-scale, front-drive/rear-steer) is a sensible safety proxy. The two-policy split – a PPO approach policy and a supervised loading-decision policy – is clean, and the human-operator comparison, while small, at least shows the task is non-trivial.

The main soft spot is exactly what the stress test says: the 60% success rate comes from 10 trials, so the 95% binomial CI is roughly 26–88%. The decision-policy 90% is also 9/10. For a paper whose contribution is 'zero-shot sim2real works,' that is not statistically established; it is a demonstration. The fixed pallet location, fixed 1.8m enclosure, and no variation in lighting or pallet pose in the real test mean even the point estimate tells you little about generalization. That should be stated plainly in the paper, and it is a legitimate reason for an editor to ask for more experiments or at least a confidence interval.

Two smaller things. First, the abstract's 'does not require heuristic additions' is overstated: the reward function is heavily shaped (eight alpha weights, clothoid reference, speed penalties), and the simulator drivetrain parameters were tuned to match the real forklift. That is fine, but it is heuristics by another name. Second, the lack of disclosed hyperparameters and reward weights makes reproduction harder, though this is normal for a venue paper.

Take it for what it is: a useful proof-of-concept, not a robust empirical claim. The system is plausible, the writing is clear, and the authors are honest about the limited scope. I would send it to peer review at a robotics venue, with the expectation that the authors add more trials, vary the real test conditions, and dial back the 'no heuristic additions' language. Not a desk reject, but not a strong accept either.

Bring it to reading group if you want a concrete example of the sim2real demo genre and a good discussion of what constitutes sufficient evidence for transfer.

Referee Report

4 major / 6 minor

Summary. The paper proposes a vision-based deep reinforcement learning system for pallet loading with a 1/14-scale counterbalance forklift. The approach policy is trained end-to-end in a photorealistic NVIDIA Isaac Sim environment built from CAD data, using domain randomization, and then transferred zero-shot to a physical 1/14-scale forklift without real-world policy fine-tuning. A separate supervised decision policy determines whether to lift the forks, also trained on synthetic data. The authors report a 60% success rate over 10 real trials for the approach policy, a 90% success rate for the loading decision policy, and compare performance with three human operators. The central claim is that zero-shot sim2real is achievable for this forklift task using a simple, purely simulation-trained method.

Significance. If the reported result holds, the paper demonstrates a potentially practical recipe for transferring vision-based DRL forklift control from a CAD-based photorealistic simulator to physical hardware without real-world training data. Strengths include the simulator built entirely from CAD, the construction of a real 1/14-scale forklift with front-wheel drive/rear-wheel steering and hydraulic lift, the use of domain randomization, the inclusion of a human-operator comparison, and the deployment of a synthetic-data-trained decision policy. However, the empirical evidence is currently thin: 10 real trials for the headline number, no statistical analysis, no variation of the real test configuration, and a potential ambiguity about whether real-world measurements were used to calibrate the simulator dynamics. As a proof-of-concept for a specific fixed environment, the paper is valuable; as a demonstration of general zero-shot sim2real robustness, it needs additional evidence.

major comments (4)

[V-B, Table II] The central claim of a 60% real-world success rate is based on 10 trials. With 6/10 successes, the Clopper-Pearson 95% confidence interval is approximately [0.26, 0.88], so the data are consistent with a true success rate as low as 26%. The paper provides no confidence intervals, no trial-by-trial outcomes, no specification of how the ten initial poses were sampled, and no analysis of why the four failures occurred. Please report per-trial details, compute a confidence interval, and substantially increase the number of trials across varied initial configurations to support the zero-shot sim2real claim.
[V-B] The reported 90% success rate for the loading decision policy lacks an explicit denominator and protocol. The text says 'A decision error occurred once when the forks were inserted only on one side,' implying 9/10, but it is not stated whether the decision policy was evaluated in all 10 approach trials, including those where the approach failed. Please specify how the decision-policy evaluations were defined, how correctness was judged when the forks were misaligned, and give the exact number of evaluations and correct decisions.
[IV-A and V-A] The real test environment is a single fixed configuration: a 1.8 m square with a pallet at a fixed location, a fixed starting triangle, and no variation of lighting, pallet offset, or floor appearance. The domain randomization ranges in Table I are applied only during simulation; the real experiments do not probe these ranges. Consequently, the demonstration establishes transfer to one specific configuration, not general zero-shot sim2real robustness. Please add real experiments in which the pallet position, lighting, and starting poses are varied within the training randomization ranges, or clearly scope the claim to the tested configuration.
[IV-C] The statement that the digital drivetrain controller parameters 'were configured to match the speed response of the real forklift to the command inputs closely' suggests that real-world measurements (e.g., of speed response) were used to build the simulator. This appears inconsistent with the title's 'Without Real-World Data' and the abstract's 'eliminating the need for real-world data.' Please clarify exactly which real-world data, if any, were used to calibrate the simulator, and qualify the zero-shot/no-real-data claim accordingly. If only manufacturer specifications or CAD data were used, state so explicitly.

minor comments (6)

[I] The final paragraph of the introduction states that concluding remarks and future directions are presented in Section V, but the conclusions actually appear in Section VI. Please correct the section reference.
[III-C-2, Eq. (7)] The pallet-contact penalty rp uses a threshold of 0.01 m/s for the pallet velocity. Please state whether this threshold is in real-world units or scaled units and briefly justify the value.
[V-A] The decision policy is triggered 3 s after the forklift comes to a stop. This fixed delay appears to be a manually chosen heuristic; please explain how this value was selected and whether the results are sensitive to it.
[V-B, Table II] The average human success rate of 60% is the mean of 50%, 40%, and 90% for the three operators. Given the large variance, please state the number of trials per operator and consider reporting per-operator outcomes rather than only the average.
[Figure 8] The trajectory plot shows all 10 trials together. It would be much more informative to distinguish successful from failed trials and to mark the locations of failures (e.g., collisions, misalignment, or premature stops).
[References] Reference [29] is cited as 'Rrl: Resnet as representation for reinforcement learning.' Please update the citation to the published version (if available) and ensure the method name matches the text (e.g., 'R3L' or 'Resnet as representation').

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the sim-trained policies are evaluated on independent real-world trials, with only benign system identification in the simulator.

full rationale

The central claim is that a DRL approach policy and a supervised decision policy, trained entirely in an Isaac Sim environment built from CAD data and domain randomization, transfer zero-shot to a 1/14-scale forklift without using real-world training data. The training pipeline does not take real-world data as an input: rewards are computed from simulator privileged states, observations are synthetic camera images, and the decision-policy dataset is generated inside Isaac Sim (Section IV-E). The only real-world information entering the system is the drivetrain controller matching in Section IV-C, where the parameters "were configured to match the speed response of the real forklift to the command inputs closely." This is system identification used to make the simulator faithful, not a fitted parameter that is later renamed as a prediction; it does not by construction determine the pallet-approach success rate. The real experiments in Section V are an external benchmark against which the sim-trained policies are tested. No load-bearing equation is defined in terms of the target result, and no self-citation chain is used to justify the sim2real transfer. Statistical concerns about the small number of trials (10 trials, wide binomial confidence interval) are matters of evidence and correctness risk, not circularity. Therefore the derivation chain is self-contained and no circular step is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper's central claim rests on a set of hand-chosen parameters (reward weights, domain randomization ranges, simulator calibration) and domain assumptions about the fidelity of the CAD-built simulation. It introduces no new physical entities or theoretical objects. The free parameters are numerous and mostly undisclosed, which weakens the 'no heuristic additions' claim and the reproducibility of the result.

free parameters (4)

Reward weights alpha_1 through alpha_8 = Not reported
Hand-tuned coefficients in Eqs. (6) and (7) shaping approach, clothoid deviation, orientation, contact, speed, and action penalties. These directly determine the policy and are not derived from data.
PPO hyperparameters (learning rate, clip epsilon, GAE lambda, value/entropy coefficients c1, c2, c3) = Not reported
Not specified in the paper; chosen by the authors and load-bearing for training stability and final policy performance.
Domain randomization ranges = Speed/action ±10%, colors ±20% RGB, light 100-100000 lm, 2000-7500 K
These ranges (Table I) are hand-chosen. The policy transfers only if the real environment falls inside these ranges; no sensitivity analysis is provided.
Simulator drivetrain controller parameters = Not reported
Section IV-C states these were 'configured to match the speed response of the real forklift to the command inputs closely.' This is a form of system identification using real-world measurements, and the values are not disclosed.

assumptions (4)

domain assumption The CAD-built Isaac Sim environment with the specified domain randomization approximates the real 1/14-scale environment closely enough for zero-shot transfer.
The entire approach rests on this assumption; Section IV-A and Table I define the randomization ranges, but no evidence shows they span real-world variation.
domain assumption ImageNet-pretrained ResNet features are informative for the forklift's two onboard camera views.
Invoked in Section III-B without adaptation. The transfer of generic image features to this specific industrial visual domain is assumed, not validated.
standard math PPO with generalized advantage estimation, as cited in [30], correctly optimizes the policy in this setting.
Standard RL background; the paper uses the algorithm as a black box. This is a standard assumption in DRL papers.
domain assumption The real 1.8m square enclosure, fixed pallet location, and triangular starting region match the training setup.
Section V-A describes matching the real setup to the training setup; generalization to other pallet positions or room layouts is not claimed or tested.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Visual-Based Forklift Learning System Enabling Zero-Shot Sim2Real Without Real-World Data." pith.science (2026). https://pith.science/paper/RCRJNSS4

@misc{pith2026241211503,
  author       = {Pith},
  title        = {Pith review of: Visual-Based Forklift Learning System Enabling Zero-Shot Sim2Real Without Real-World Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RCRJNSS4}},
  note         = {Machine review of arXiv:2412.11503}
}

read the original abstract

Forklifts are used extensively in various industrial settings and are in high demand for automation. In particular, counterbalance forklifts are highly versatile and employed in diverse scenarios. However, efforts to automate these processes are lacking, primarily owing to the absence of a safe and performance-verifiable development environment. This study proposes a learning system that combines a photorealistic digital learning environment with a 1/14-scale robotic forklift environment to address this challenge. Inspired by the training-based learning approach adopted by forklift operators, we employ an end-to-end vision-based deep reinforcement learning approach. The learning is conducted in a digitalized environment created from CAD data, making it safe and eliminating the need for real-world data. In addition, we safely validate the method in a physical setting utilizing a 1/14-scale robotic forklift with a configuration similar to that of a real forklift. We achieved a 60% success rate in pallet loading tasks in real experiments using a robotic forklift. Our approach demonstrates zero-shot sim2real with a simple method that does not require heuristic additions. This learning-based approach is considered a first step towards the automation of counterbalance forklifts.

Figures

Figures reproduced from arXiv: 2412.11503 by the authors.

**Figure 2.** Approach policy design. inputs obtained from the camera images and velocity data, respectively. The camera images, which are 352 × 288 RGB images, are first resized to 224 × 224. These resized images are then converted into feature vectors, Ll ∈ R 512 and Lr ∈ R 512, using a ResNet pretrained on ImageNet [27], [28]. Subsequently, these feature vectors are used as inputs for the policy. This method was proposed as Re… view at source ↗

**Figure 3.** State used for calculating R+. TABLE I: Domain randomization targets Item Range Observed speed ±10% of value Action ±10% of command Floor color ±20% RGB Pallet stand color ±20% RGB Pallet and load color ±20% RGB Light intensity 100–100000 lm Light temp. 2000–7500 K a negative reward is assigned if the pallet starts to move. rv and ra are penalties that are designed to bridge the gap between the digital and real envi… view at source ↗

Figures from the paper (5 more)

**Figure 6.** Figure 6: Two USB cameras were connected to the Raspberry [PITH_FULL_IMAGE:figures/full_fig_p004_6.png]

**Figure 4.** Figure 4: Digital and real environments. (a), (b), and (c) repr [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 6.** Figure 6: Real forklift control system. dataset. V. REAL DEMONSTRATION This section presents the results of deploying our method in a real environment. As our method employs zero-shot sim2real, the policies trained in the digital environment were transferred directly to the real…

**Figure 7.** Figure 7: Dataset for decision policy. TABLE II: Demonstration results Ours Man A Man B Man C Success rate (%) 60 50 40 90 Time (s) 6.5 7.7 17.2 24.5 by positioning the forklift within a triangular area, similar to that during the training in the digital environment, as shown in…

**Figure 9.** Figure 9: Demonstration of forklift task. the forks were inserted only on one side. Snapshots of a successful attempt are presented in [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 31 canonical work pages

[1]

Automation for your warehouse

Linde Material Handling, “Automation for your warehouse.” [Online]. Available: https://www.linde-mh.com/en/Solutions/Intralogistics-Automation

work page
[2]

Human-level control through deep reinforcement learnin g,

V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. V eness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ost rovski et al. , “Human-level control through deep reinforcement learnin g,” nature, vol. 518, no. 7540, pp. 529–533, 2015

work page 2015
[3]

End-to-en d training of deep visuomotor policies,

S. Levine, C. Finn, T. Darrell, and P . Abbeel, “End-to-en d training of deep visuomotor policies,” Journal of Machine Learning Research , vol. 17, no. 39, pp. 1–40, 2016

work page 2016
[4]

Champion-level drone racing using deep rei nforce- ment learning,

E. Kaufmann, L. Bauersfeld, A. Loquercio, M. M¨ uller, V . Koltun, and D. Scaramuzza, “Champion-level drone racing using deep rei nforce- ment learning,” Nature, vol. 620, no. 7976, pp. 982–987, 2023

work page 2023
[5]

Sim-to-real tr ansfer in deep reinforcement learning for robotics: a survey,

W. Zhao, J. Queralta, and T. Westerlund, “Sim-to-real tr ansfer in deep reinforcement learning for robotics: a survey,” in 2020 IEEE Symposium Series on Computational Intelligence (SSCI) , 2020, pp. 737–744

work page 2020
[6]

Learning agile soccer skills for a bipedal robot with deep r einforce- ment learning,

T. Haarnoja, B. Moran, G. Lever, S. Huang, D. Tirumala, J. Humplik, M. Wulfmeier, S. Tunyasuvunakool, N. Siegel, R. Hafner, M. B loesch, K. Hartikainen, A. Byravan, L. Hasenclever, Y . Tassa, F. Sad eghi, N. Batchelor, F. Casarini, S. Saliceti, C. Game, N. Sreendra , K. Patel, M. Gwira, A. Huber, N. Hurley, F. Nori, R. Hadsell, and N. Hees s, “Learning ag...

work page 2024
[7]

Learning robust perceptive locomotion for quadrupedal ro bots in the wild,

T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal ro bots in the wild,” Science Robotics , vol. 7, 2022

work page 2022
[8]

Domain randomization for transferring deep neural networ ks from simulation to the real world,

J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P . Abbeel, “Domain randomization for transferring deep neural networ ks from simulation to the real world,” in 2017 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS) , 2017, pp. 23–30

work page 2017

Show all 34 references

[9]

S im-to- real transfer of robotic control with dynamics randomizati on,

X. B. Peng, M. Andrychowicz, W. Zaremba, and P . Abbeel, “S im-to- real transfer of robotic control with dynamics randomizati on,” in 2018 IEEE International Conference on Robotics and Automation ( ICRA), 2018, pp. 3803–3810

2018
[10]

Orbit: A uniﬁed simulation framewor k for interactive robot learning environments,

M. Mittal, C. Y u, Q. Y u, J. Liu, N. Rudin, D. Hoeller, J. Y u an, R. Singh, Y . Guo, H. Mazhar, A. Mandlekar, B. Babich, G. State , M. Hutter, and A. Garg, “Orbit: A uniﬁed simulation framewor k for interactive robot learning environments,” IEEE Robotics and Automa- tion Let...

2023
[11]

Ros: an open-source robot operatin g system,

M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. L eibs, R. Wheeler, and A. Y . Ng, “Ros: an open-source robot operatin g system,” in ICRA workshop on open source software , vol. 3. Kobe, Japan, 2009, p. 5

2009
[12]

Clos ed-loop pallet manipulation in unstructured environments,

M. Walter, S. Karaman, E. Frazzoli, and S. Teller, “Clos ed-loop pallet manipulation in unstructured environments,” in 2010 IEEE/RSJ International Conference on Intelligent Robots and System s (IROS) , 2010

2010
[13]

A voice-commandable robo tic forklift working alongside humans in minimally-prepared o utdoor environments,

S. Teller, M. Walter, M. Antone, A. Correa, R. Davis, L. F letcher, E. Frazzoli, J. Glass, J. How, A. Huang, J. Jeon, S. Karaman, B. Luders, N. Roy, and T. Sainath, “A voice-commandable robo tic forklift working alongside humans in minimally-prepared o utdoor environments,” in...

2010
[14]

Autonomous load carrier app roaching based on deep reinforcement learning with compressed visua l informa- tion,

S. Hadwiger and T. Meisen, “Autonomous load carrier app roaching based on deep reinforcement learning with compressed visua l informa- tion,” in 2022 5TH International Conference on Artiﬁcial Intelligen ce for Industries, AI4I , 2022, pp. 48–53

2022
[15]

D4r l: Datasets for deep data-driven reinforcement learning,

J. Fu, A. Kumar, O. Nachum, G. Tucker, and S. Levine, “D4r l: Datasets for deep data-driven reinforcement learning,” arXiv preprint arXiv:2004.07219, 2020

2004 arXiv
[16]

Learning hierarchical interactive multi-object search f or mobile ma- nipulation,

F. Schmalstieg, D. Honerkamp, T. Welschehold, and A. V a lada, “Learning hierarchical interactive multi-object search f or mobile ma- nipulation,” IEEE Robotics and Automation Letters , vol. 8, pp. 8549– 8556, 2023

2023
[17]

Sim-to-real transfer of active suspens ion control using deep reinforcement learning,

V . Wiberg, E. Wallin, A. F¨ alldin, T. Semberg, M. Rossan der, E. Wad- bro, and M. Servin, “Sim-to-real transfer of active suspens ion control using deep reinforcement learning,” Robotics and Autonomous Sys- tems, vol. 179, 2024

2024
[18]

Sim2real manipulation on unknown objects with tac tile- based reinforcement learning,

E. Su, C. Jia, Y . Qin, W. Zhou, A. Macaluso, B. Huang, and X. Wang, “Sim2real manipulation on unknown objects with tac tile- based reinforcement learning,” in 2024 IEEE International Conference on Robotics and Automation (ICRA) , 2024, pp. 9234–9241

2024
[19]

Image augmenta tion is all you need: Regularizing deep reinforcement learning from pi xels,

D. Y arats, I. Kostrikov, and R. Fergus, “Image augmenta tion is all you need: Regularizing deep reinforcement learning from pi xels,” in International Conference on Learning Representations , 2021

2021
[20]

CAD 2 RL: Real single-image ﬂight without a single real image,

F. Sadeghi and S. Levine, “CAD 2 RL: Real single-image ﬂight without a single real image,” in Robotics: Science and Systems XIII , 2017

2017
[21]

Sim-To-R eal via Sim-To-Sim: Data-Efﬁcient Robotic Grasping via Random ized-To- Canonical Adaptation Networks ,

S. James, P . Wohlhart, M. Kalakrishnan, D. Kalashnikov , A. Irpan, J. Ibarz, S. Levine, R. Hadsell, and K. Bousmalis, “ Sim-To-R eal via Sim-To-Sim: Data-Efﬁcient Robotic Grasping via Random ized-To- Canonical Adaptation Networks ,” in 2019 IEEE/CVF Conference on Computer Vis...

2019
[22]

Learning robot soccer from egocentric vision with deep rei nforcement learning,

D. Tirumala, M. Wulfmeier, B. Moran, S. Huang, J. Humpli k, G. Lever, T. Haarnoja, L. Hasenclever, A. Byravan, N. Batche lor, N. sreendra, K. Patel, M. Gwira, F. Nori, M. Riedmiller, and N . Heess, “Learning robot soccer from egocentric vision with deep rei nforcement learning...

2024
[23]

Nerf: Representing scenes as neural radianc e ﬁelds for view synthesis,

B. Mildenhall, P . P . Srinivasan, M. Tancik, J. T. Barron , R. Ramamoor- thi, and R. Ng, “Nerf: Representing scenes as neural radianc e ﬁelds for view synthesis,” Communications of the ACM , vol. 65, no. 1, pp. 99–106, 2021

2021
[24]

Nerf2real: Sim2real transfer of vision-guided b ipedal motion skills using neural radiance ﬁelds,

A. Byravan, J. Humplik, L. Hasenclever, A. Brussee, F. N ori, T. Haarnoja, B. Moran, S. Bohez, F. Sadeghi, B. Vujatovic, an d N. Heess, “Nerf2real: Sim2real transfer of vision-guided b ipedal motion skills using neural radiance ﬁelds,” in 2023 IEEE International Conference on ...

2023
[25]

Isaac Sim - Robotics Simulation and Synthetic Data Gen- eration

NVIDIA, “Isaac Sim - Robotics Simulation and Synthetic Data Gen- eration.” [Online]. Available: https://developer.nvidi a.com/isaac-sim
[26]

Orbit-surgical: An open-simulation framework for learni ng surgical augmented dexterity,

Q. Y u, M. Moghani, K. Dharmarajan, V . Schorp, W. C. H. Pan itch, J. Liu, K. Hari, H. Huang, M. Mittal, K. Goldberg, and A. Garg, “Orbit-surgical: An open-simulation framework for learni ng surgical augmented dexterity,” in 2024 IEEE International Conference on Robotics and A...

2024
[27]

Deep residual learni ng for image recognition,

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learni ng for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2016, pp. 770–778

2016
[28]

Imagenet: A large-scale hierarchical image database,

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fe i, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVP R), 2009, pp. 248–255

2009
[29]

Rrl: Resnet as representation f or rein- forcement learning,

R. M. Shah and V . Kumar, “Rrl: Resnet as representation f or rein- forcement learning,” in the 38th International Conference on Machine Learning, vol. 139, 2021, pp. 9465–9476

2021
[30]

Proximal policy optimization algorithms,

J. Schulman, F. Wolski, P . Dhariwal, A. Radford, and O. K limov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

2017 arXiv
[31]

Indoor ob stacle discovery on reﬂective ground via monocular camera,

F. Xue, Y . Chang, T. Wang, Y . Zhou, and A. Ming, “Indoor ob stacle discovery on reﬂective ground via monocular camera,” International Journal of Computer Vision , vol. 132, no. 3, pp. 987–1007, 2024

2024
[32]

Lesu - manufacturer specialized in manufacturing simulation models

LESU Model Technology Co., Ltd., “Lesu - manufacturer specialized in manufacturing simulation models.” [Online ]. Available: https://www.rclesu.com/
[33]

rl-games: A high-p erformance framework for reinforcement learning,

D. Makoviichuk and V . Makoviychuk, “rl-games: A high-p erformance framework for reinforcement learning,” May 2021. [Online] . Available: https://github.com/Denys88/rl games

2021
[34]

Omniverse isaac gym reinforcement learning environments for isaac sim,

NVIDIA, “Omniverse isaac gym reinforcement learning environments for isaac sim,” 2022. [Online]. Available: https://github.com/isaac-sim/OmniIsaacGymEnvs

2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Automation for your warehouse

Linde Material Handling, “Automation for your warehouse.” [Online]. Available: https://www.linde-mh.com/en/Solutions/Intralogistics-Automation

work page

[2] [2]

Human-level control through deep reinforcement learnin g,

V . Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. V eness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ost rovski et al. , “Human-level control through deep reinforcement learnin g,” nature, vol. 518, no. 7540, pp. 529–533, 2015

work page 2015

[3] [3]

End-to-en d training of deep visuomotor policies,

S. Levine, C. Finn, T. Darrell, and P . Abbeel, “End-to-en d training of deep visuomotor policies,” Journal of Machine Learning Research , vol. 17, no. 39, pp. 1–40, 2016

work page 2016

[4] [4]

Champion-level drone racing using deep rei nforce- ment learning,

E. Kaufmann, L. Bauersfeld, A. Loquercio, M. M¨ uller, V . Koltun, and D. Scaramuzza, “Champion-level drone racing using deep rei nforce- ment learning,” Nature, vol. 620, no. 7976, pp. 982–987, 2023

work page 2023

[5] [5]

Sim-to-real tr ansfer in deep reinforcement learning for robotics: a survey,

W. Zhao, J. Queralta, and T. Westerlund, “Sim-to-real tr ansfer in deep reinforcement learning for robotics: a survey,” in 2020 IEEE Symposium Series on Computational Intelligence (SSCI) , 2020, pp. 737–744

work page 2020

[6] [6]

Learning agile soccer skills for a bipedal robot with deep r einforce- ment learning,

T. Haarnoja, B. Moran, G. Lever, S. Huang, D. Tirumala, J. Humplik, M. Wulfmeier, S. Tunyasuvunakool, N. Siegel, R. Hafner, M. B loesch, K. Hartikainen, A. Byravan, L. Hasenclever, Y . Tassa, F. Sad eghi, N. Batchelor, F. Casarini, S. Saliceti, C. Game, N. Sreendra , K. Patel, M. Gwira, A. Huber, N. Hurley, F. Nori, R. Hadsell, and N. Hees s, “Learning ag...

work page 2024

[7] [7]

Learning robust perceptive locomotion for quadrupedal ro bots in the wild,

T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V . Koltun, and M. Hutter, “Learning robust perceptive locomotion for quadrupedal ro bots in the wild,” Science Robotics , vol. 7, 2022

work page 2022

[8] [8]

Domain randomization for transferring deep neural networ ks from simulation to the real world,

J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P . Abbeel, “Domain randomization for transferring deep neural networ ks from simulation to the real world,” in 2017 IEEE/RSJ International Con- ference on Intelligent Robots and Systems (IROS) , 2017, pp. 23–30

work page 2017