{"id":"00846fde-42b5-40d1-b7bd-ad0085fccb63","arxiv_id":"2502.07380","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Wheeled Lab is an open-source ecosystem that trains three zero-shot RL policies on low-cost wheeled robots in Isaac Lab and deploys them in the real world.","lead":"Wheeled Lab pairs low-cost RC cars with the Isaac Lab simulator to train robot-driving policies entirely in simulation, then deploys them on real cars for drifts, ramps, and visual navigation. It is an open-source package aimed at making modern sim-to-real robot learning affordable for classrooms and small labs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'first zero-shot drift' claim is not established: the appendix admits iterative Sim2Real2Sim parameter narrowing and real friction measurement, so transfer required real-world tuning; quantitative evaluation is absent.","rationale":"I agree with the reader's weakest assumption: the zero-shot transfer claim depends on simulator fidelity, and the paper's own appendix reveals iterative real-world tuning that blurs the boundary between simulation and reality. The open-source ecosystem, cost reporting, and qualitative demonstrations are genuine contributions and should be credited; the issue is specifically the strength of the 'first zero-shot' novelty claim. The manuscript's Limitations section and Appendix B are in-scope evidence that the pipeline required iteration, so this is not an external standard being applied unfairly. The reader's CONDITIONAL verdict is appropriate: the paper should be accepted conditional on clarifying what 'zero-shot' means, disclosing the role of Sim2Real2Sim cycles, and providing quantitative success rates for drift and elevation. My proposed retraining-and-deployment check would settle whether the transfer is attributable to generalizable domain randomization or to tuned simulation parameters, and would either substantiate or revise the headline claim.","tokens_in":15158,"tokens_out":6876,"duration_ms":69130,"concrete_test":"Retrain πdrift from scratch using the released code, but freeze the simulator parameters to the broad, datasheet-only ranges initially reported (friction 0.2–0.8, no Sim2Real2Sim narrowing of actuator gains), and deploy on the same carpeted oval with VIO-only state estimation and no motion-capture correction. Report success rate over N=20 laps. If the policy cannot complete a full lap without falling back to additional simulator tuning, the 'zero-shot, no-online-fine-tuning' claim should be revised to 'transfer after offline simulator identification'; if it succeeds, the claim would be strengthened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption behind the headline contribution—'first direct Sim2Real transfer for drifting without online fine-tuning' (Section 1)—is that the sim-to-real gap was closed by domain randomization and massive parallelization, rather than by real-world feedback. The appendix shows otherwise. B.1.1 states 'a few Sim2Real2Sim ... cycles were spent narrowing the randomized gain ranges for the throttle actuator settings,' and B.1.2 describes a 'long thread of Sim2Sim cycles' plus a spring-scale friction measurement used to set the friction randomization midpoint. Section 6 openly says the released policies 'must be met with patience and an intent to iterate.' These are real-world parameter-identification and iterative-debugging steps; whether they count as 'zero-shot' depends entirely on an unstated, narrow definition of the term. The experimental evidence for drift (Section 4.1) is qualitative—no trial counts, success rates, or failure statistics—so the 'first' claim cannot be distinguished from a policy that was tuned into existence. The evaluation also uses a motion capture system for 1 Hz integrator-drift correction, while VIO-only runs are 'not used for data collection,' so the deployment is not demonstrated under the stated onboard state-estimation conditions. The central claim therefore rests on a definitional and evidentiary gap, not on the underlying open-source ecosystem, which is a real contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Wheeled Lab, an open-source ecosystem that integrates low-cost wheeled robots (HOUND and MuSHR) with Isaac Lab for reinforcement-learning-based Sim2Real research and education. The authors contribute modular environment and configuration abstractions, then demonstrate three learned policies trained with PPO in Isaac Lab and deployed on real hardware: aggressive drifting, elevation traversal, and visual navigation. The manuscript includes ablations over domain randomization, perturbations, parallelization, architecture, and image augmentation, and it makes a central novelty claim that this is the first direct Sim2Real transfer for drifting without online fine-tuning.","tokens_in":15443,"tokens_out":5365,"duration_ms":47761,"significance":"If the claims hold, Wheeled Lab would be a valuable community resource: it lowers the cost barrier to modern Sim2Real methods, provides reproducible baseline configurations for accessible platforms, and demonstrates end-to-end pipelines for three distinct tasks. The paper's concrete strengths are the breadth of the integrated hardware/software stack, the comparative survey of existing ecosystems, and the public release of code, configurations, and videos. However, the quantitative evaluation is currently too thin to support the strongest novelty claims, and the appendix documents real-world calibration steps that are downplayed in the main text. The infrastructure contribution is solid; the 'first zero-shot drifting' claim is the main risk.","major_comments":[{"comment":"The claim of 'first zero-shot transfer' for drifting is not supported as stated. Appendix B.1.1 describes 'a few Sim2Real2Sim cycles were spent narrowing the randomized gain ranges for the throttle actuator settings,' and Appendix B.1.2 describes a 'long thread of Sim2Sim cycles' plus a real spring-scale measurement used to set the friction randomization midpoint to about 0.4. These are real-world calibration and system-identification steps, and the Related Work paragraph explicitly says the authors distance their methods from 'gain tuning and extensive system identification.' The term 'zero-shot' therefore needs a precise definition, and the claim should be softened to something like 'without online policy fine-tuning after simulator calibration,' or the paper must show that no real-world information entered policy training. Section 6's admission that the released policies 'must be met with patience and an intent to iterate' further weakens the out-of-the-box zero-shot framing.","section":"Section 1, Section 5, Appendix B.1.1-B.1.2"},{"comment":"The drift experiments lack quantitative trial-level statistics. The text reports complete laps, a maximum controlled slip angle of 58 degrees, and an average speed of about 1.6 m/s, but it gives no number of trials, success rate, failure count, or a precise definition of 'complete laps.' The setup also says that motion capture is used for evaluation and occasional 1 Hz integrator-drift correction, while successful VIO-only runs are only shown on the website and are 'not used for data collection.' The reported deployment is therefore not demonstrated under the stated onboard state-estimation conditions, and the 'first' claim cannot be distinguished from a policy tuned into existence. Please report trial counts and success/failure rates, and either use onboard VIO for the reported runs or explicitly acknowledge the motion-capture assistance in the main-text results.","section":"Section 4.1"},{"comment":"The quantitative visual-policy comparison rests on five trials per setting. The conclusion that image augmentation is 'essential' is supported by 0/5 versus 3/5 and 0/5 versus 1/5, but the CNN-versus-MLP generalization conclusion (3/5 versus 1/5) is within the range of random variation; no confidence intervals, per-trial descriptions, or failure-mode analyses are given. Additionally, linear and angular velocities are obtained from the motion capture system, so the policy is not evaluated under fully onboard state estimation. More trials or a statistical treatment is needed before drawing architecture-level conclusions.","section":"Section 4.3, Table 3"},{"comment":"The elevation policy evaluation is qualitative only. The text states that the baseline 'primarily deviates from any elevation features' and that the trained policy 'can both traverse the ramp safely and navigate through subsequent obstacles,' but no trial counts, success rates, or failure statistics are reported. Since elevation traversal is one of the three headline demonstrations, quantitative evaluation should be added or the claims should be scaled back to match the evidence.","section":"Section 4.2, Figure 6"}],"minor_comments":[{"comment":"The 'Turn-Left-Go-Right' reward explicitly rewards counter-steering when the angular velocity and steering command are opposite, so the drifting behavior is partly shaped by the reward design; the main text should acknowledge this when describing the behavior as discovered rather than engineered.","section":"Section B.1.4"},{"comment":"The sentence 'Successful VIO-only runs not used for data collection' is missing punctuation and is easy to misread; please clarify whether the VIO-only runs were used for qualitative demonstration only.","section":"Section 4.1"},{"comment":"The header '# of Success / Trial' is ambiguous; it should read '# successes out of 5 trials' for each condition.","section":"Table 3"},{"comment":"Lines 18-22 use a non-standard 'Do ... do While' pseudo-code structure; a conventional repeat-until loop with an explicit condition would be clearer.","section":"Algorithm 1"},{"comment":"References [39]-[43] contain incomplete bibliographic entries, with missing publication dates and venues; please complete them.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The ecosystem and released artifacts are the strongest part of the paper, and they are likely to be a useful community resource. The main risk is the 'first zero-shot drifting' claim, which is overstated relative to the appendix's documented Sim2Real2Sim and real-world friction-calibration steps. I would ask the authors to foreground the ecosystem contribution, precisely define what they mean by zero-shot, and add trial-level quantitative evaluation for all three tasks. If such data cannot be provided, the novelty claims should be scaled back. These issues are fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Know this: the real value of Wheeled Lab is the open-source ecosystem that hooks Isaac Lab to cheap wheeled robots; the 'first zero-shot drift' claim is a stretch given what the appendix admits. The drift and elevation demos are thin on quantitative evidence.\n\nWhat's genuinely new here is the integration. Putting Isaac Lab's massive parallelization, domain randomization, and sensor simulation onto HOUND, MuSHR, and F1Tenth platforms is a practical contribution that could actually lower the floor for teaching and for labs that can't afford quadrupeds. The three deployed skills—drift, elevation traversal, visual navigation—show the pipeline works end to end. The visual policy gets a real quantitative comparison (5 trials per setting) and the augmentation ablation shows a clear effect. The code and hardware are open-source, and Section 6 is refreshingly honest about the need to iterate and about the lack of a usability study.\n\nThe soft spots are mostly around the headline claim. The abstract says 'direct Sim2Real transfer for drifting without online fine-tuning,' but Appendix B.1.1 says a few Sim2Real2Sim cycles were spent narrowing randomized gain ranges for the throttle actuator, and B.1.2 describes measuring friction with a spring scale and using that value as the midpoint of randomization after a long thread of Sim2Sim cycles. That is system identification and manual tuning. It may not be policy fine-tuning, but it's not zero-shot in the usual sense. The authors could have defined their terms and acknowledged this. The drift results have no trial counts, success rates, or failure statistics—just overlaid trajectories and a max slip angle. The elevation results are similarly qualitative. And the motion capture system is used for 1 Hz integrator-drift correction, with VIO-only runs not used for data collection, so the deployment conditions are more favorable than the stated onboard sensing.\n\nOverall, this is a solid ecosystem paper that overclaims its scientific novelty. A serious referee should engage with it, but the authors should be asked to either soften the zero-shot language or provide quantitative evidence and a precise definition of what zero-shot includes. I'd cite the project as a resource, and I'd bring it to reading group for a useful discussion about evidence standards in sim2real.","headline":"Worth reading for the open-source ecosystem, but the 'first zero-shot drift' claim overreaches the evidence.","tokens_in":15963,"tokens_out":3055,"would_cite":true,"duration_ms":25835,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A low-cost, open-source wheeled-robot ecosystem built on Isaac Lab achieves zero-shot sim-to-real transfer for drifting, elevation traversal, and visual navigation.","keywords":["sim-to-real transfer","reinforcement learning","domain randomization","autonomous drifting","elevation map navigation","visual navigation","low-cost robotics","open-source robotics"],"falsifier":"Take the released open-source drifting stack, set it up on a second unit of the same platform without any retraining, and run it on a different indoor surface whose friction is measured with the same spring-scale method to be below the 0.4 training midpoint (say 0.3). The paper's zero-shot claim predicts the policy should still complete laps reliably; if it consistently spins out and fails to finish a lap, the claim would be falsified.","tokens_in":14994,"feed_emoji":"🏎️","tokens_out":9460,"duration_ms":76043,"temperature":0.7,"pith_summary":"Wheeled Lab connects inexpensive, open-source wheeled robots to a modern, GPU-accelerated robot-learning simulator and claims that this combination is enough to train policies in simulation that transfer directly to real hardware with no real-world fine-tuning. The headline demonstration is the first zero-shot transfer of a drifting policy: a small RC car trained entirely in simulation completes drifting laps on indoor carpet, with a maximum controlled side-slip angle of about 58 degrees and an average speed near 1.6 m/s. The same pipeline also yields an elevation policy that uses a local 2.5 by 2.5 meter elevation map to distinguish walls from traversable ramps, and a visual policy that follows a white figure-eight path on a black floor from grayscale camera images. If the paper is right, the expensive, specialist tooling of modern Sim2Real—massive parallelization, domain randomization, sensor simulation—can fit on roughly $900 to $3,000 platforms and be placed in the hands of classrooms and hobbyists.","feed_headline":"First zero-shot drift transfer on low-cost wheeled robots","feed_subtitle":"Open-source Isaac Lab stack turns a $3,000 RC car into a drifter, hill climber, and visual navigator.","key_machinery":"The central object is the Wheeled Lab training stack, which extends Isaac Lab with modular run, agent, and environment configurations so that scene, observation, reward, and hyperparameter choices are bundled and reproducible. The policies are trained with PPO in massively parallel simulation (1,024 parallel agents for drift and elevation; over 2,000 for visual map generation) using domain randomization across friction, actuator gains, and visual appearance, plus perturbation simulation and observation corruption. Drifting also required physical changes to the platform—tape on the tires and a rear-wheel-drive conversion—and a friction estimate made with a cheap spring scale (about 0.4), used as the center of the randomization; several simulation-to-simulation cycles narrowed the actuator gain ranges. Hand-designed rewards carry much of the task: the drifting reward includes cross-track distance, speed, side-slip stability, progress, turn energy, and a \"turn-left-go-right\" term that encourages the policy to discover counter-steering as an alternate mode, while the visual policy uses velocity and traversability rewards plus image augmentation such as color jittering and Gaussian blur.","core_discovery":"The central claim is that a coherent, reproducible training stack—built on the Isaac Lab simulator and organized into run, agent, and environment abstractions—can close the sim-to-real gap for three distinct tasks on low-cost wheeled platforms. For drifting, the paper claims the first direct Sim2Real transfer that requires no online fine-tuning: the policy learns to cut throttle, steer sharply inward to throw out the rear wheels, then counter-steer while throttling through the turn, and it keeps full laps at about 1.6 m/s with controlled slip angles reaching 58 degrees. For elevation, the policy learns to use a local body-centric elevation map to approach and climb ramps while avoiding walls, something the baseline policy trained without parallelization, perturbation, and domain randomization cannot do reliably. For vision, the paper shows that a grayscale 40 by 60 pixel observation with aggressive image augmentation transfers to a real white-on-black figure-eight course, whereas policies without augmentation fail; in five trials the best configuration succeeds three times. These are stated as demonstrations of a pipeline, not as guarantees of performance on all surfaces or all hardware.","pith_inferences":["A natural testable extension is to run the same training recipe on other open-source wheeled platforms, such as an F1Tenth-style car, to see whether the framework's platform-agnostic abstractions generalize as the paper's design implies.","The spring-scale friction measurement and the several simulation-to-simulation gain-narrowing cycles suggest that this zero-shot pipeline is not fully parameter-free; a likely next step, which the paper names as robust adaptation, is online adjustment of friction and actuator parameters on more varied surfaces.","The grayscale simplification strategy suggests a testable design principle: reducing sensor complexity may beat increasing rendering fidelity for visual Sim2Real on structured tasks, and comparing the current grayscale pipeline to a photorealistic RGB pipeline on the same figure-eight course would settle this.","Because the paper's educational claim is not yet backed by a user study, a controlled classroom evaluation of how quickly novices can go from simulation to deployment would be the natural way to test the accessibility argument."],"forward_implications":["Agile driving research becomes accessible to budget-constrained labs, since a drifting policy can be trained in simulation and deployed on a roughly $3,000 RC car without real-world fine-tuning.","Elevation-based traversability reasoning, previously standard for legged robots, can be trained end-to-end on wheeled platforms using local elevation maps and transferred to real ramps and obstacles.","Camera-based reinforcement learning becomes approachable with simple grayscale observations and aggressive image augmentation, lowering the data and rendering demands of visual Sim2Real.","Classrooms and hobbyists can learn the full modern robotics loop—designing rewards, training in parallel simulation, deploying on hardware, and iterating—on open-source platforms rather than proprietary or expensive ones.","The strong baseline failures indicate that earlier low-fidelity wheeled-robot ecosystems omit the very techniques (massive parallelization, perturbation, domain randomization) that make modern Sim2Real work."],"supporting_citations":[{"why":"Supplies Isaac Lab, the open-source simulation and learning framework that the entire Wheeled Lab stack extends.","marker":"[4]"},{"why":"Provides the HOUND platform used for the drifting policy deployment.","marker":"[1]"},{"why":"Provides the MuSHR platform used for the elevation and visual policy deployments.","marker":"[2]"},{"why":"Supplies the Proximal Policy Optimization algorithm used to train every policy.","marker":"[48]"},{"why":"Supplies the RSL RL library and its massively parallel training implementation used for learning.","marker":"[49]"},{"why":"Earlier agile driving work that required real-world fine-tuning; the paper treats it as the baseline for its zero-shot drifting claim.","marker":"[12]"},{"why":"Prior simulation-aided RL for drifting that needed real-world fine-tuning, the key comparison for claiming the first zero-shot drift transfer.","marker":"[27]"},{"why":"Demonstrates elevation-map-based parkour training on legged robots, the methodology the elevation policy adapts to wheeled platforms.","marker":"[6]"}],"fun_headline_variants":["Zero-shot sim2real drift on an open-source $3k RC car","Open-source Isaac Lab stack turns a $3k RC car into a drifter","Budget bot learns to drift, climb, and see via zero-shot sim2real","Isaac Lab + RC car: zero-shot drift, climb, and navigation for $3k","First zero-shot sim2real for drifting on low-cost open-source robots"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim depends on the assumption that the Isaac Lab simulation, together with hand-designed rewards and coarse physical measurements (a spring-scale friction estimate of about 0.4 and actuator gains taken from datasheets and narrowed in simulation-to-simulation cycles), models the real HOUND and MuSHR platforms faithfully enough that policies transfer to real hardware with no real-world policy updates.","fun_headline_variants_meta":{"raw":{"variants":["Zero-shot sim2real drift on an open-source $3k RC car","Open-source Isaac Lab stack turns a $3k RC car into a drifter","Budget bot learns to drift, climb, and see via zero-shot sim2real","Isaac Lab + RC car: zero-shot drift, climb, and navigation for $3k","First zero-shot sim2real for drifting on low-cost open-source robots"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000708,"raw_usage":{"total_tokens":3198,"prompt_tokens":963,"completion_tokens":2235,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":579,"completion_tokens_details":{"reasoning_tokens":2129}},"tokens_in":579,"tokens_out":2235,"duration_ms":15336,"temperature":1.0,"reasoning_tokens":2129,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:56:06.290967+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the released open-source drifting stack, set it up on a second unit of the same platform without any retraining, and run it on a different indoor surface whose friction is measured with the same spring-scale method to be below the 0.4 training midpoint (say 0.3). The paper's zero-shot claim predicts the policy should still complete laps reliably; if it consistently spins out and fails to finish a lap, the claim would be falsified.","supporting_citations":[{"cited_title":"Demonstrating HOUND: A Low-cost Research Platform for High-speed Off-road Underactuated Nonholonomic Driving","cited_arxiv_id":"2311.11199","evidence_quote":"Provides the HOUND platform used for the drifting policy deployment."},{"cited_title":"Williams, N","cited_arxiv_id":null,"evidence_quote":"Earlier agile driving work that required real-world fine-tuning; the paper treats it as the baseline for its zero-shot drifting claim."},{"cited_title":"Cutler and J","cited_arxiv_id":null,"evidence_quote":"Prior simulation-aided RL for drifting that needed real-world fine-tuning, the key comparison for claiming the first zero-shot drift transfer."},{"cited_title":"Hoeller, N","cited_arxiv_id":null,"evidence_quote":"Demonstrates elevation-map-based parkour training on legged robots, the methodology the elevation policy adapts to wheeled platforms."}],"review_version":1}