{"id":"52ff0b9e-56cb-4927-be2c-253babc860ad","arxiv_id":"2506.14039","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A reinforcement learning policy trained in a randomized simulator with motor dynamics and observation delays transfers to hardware and lands a morphing quadrotor through mid-air transformation, beating an MPC baseline in impact velocity and simulated disturbance recovery.","lead":"An end-to-end reinforcement learning controller lands a morphing quadrotor drone by transforming mid-air on real hardware, but only when motor dynamics and observation delays are included in training. The learned controller compares favorably against model-predictive control in agility and simulated disturbance recovery.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Paper's own Sec IV attributes RL roll oscillations to inaccurate latency/motor time constant, so the simulator's two enabling ingredients are unvalidated; the transfer could be robustness rather than model fidelity.","rationale":"The reader's weakest assumption is that the simplified simulator is an accurate surrogate for the real ATMO hardware, and the paper itself provides direct evidence that this assumption is only approximately satisfied: Sec IV reports roll oscillations caused by inaccurate estimates of system latency and motor time constant, the very quantities the paper says are essential for sim-to-real transfer. I agree with the reader's identification of this as the load-bearing weakness. The hardware demonstration is real evidence and should be credited; the policy did transfer and complete the maneuver. The absence of repeated hardware trials and error bars is also relevant, but the more fundamental issue is that the model ingredients claimed to enable transfer are admitted to be imprecise, making the simulator-based disturbance and actuator-failure comparisons unvalidated predictions rather than demonstrated hardware behavior. The proposed system-identification test would settle whether the simulator mismatch explains the observed roll oscillations and whether the causal story holds. Since the reader already issued a CONDITIONAL verdict and my concern reinforces that conditionality rather than overturning it, the verdict should remain UNCHANGED.","tokens_in":11183,"tokens_out":7617,"duration_ms":85604,"concrete_test":"Perform system identification on the ATMO hardware: measure the end-to-end observation delay (state estimate to policy output to motor command) and the motor time constant via step response. Re-run the trained policy in Isaac Lab with these measured values instead of nominal Tm=0.15 s and 20 ms delay, and compare roll-oscillation amplitude and landing success against the Sec IV hardware run. If simulated roll oscillations match hardware, the simulator is adequate; if not, the claim that motor dynamics and delays enable transfer is unsupported and Secs V-VI results remain simulation-only predictions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that end-to-end RL transfers to ATMO hardware only because the simulator includes motor dynamics and a 20 ms observation delay (Secs III.D, IV). However, Sec IV states that the RL controller 'exhibits larger oscillations in the roll angle' and attributes this to 'inaccurate estimation of the system latency as well as the motor dynamics time constant.' This is an explicit self-reported limitation that directly undercuts the two model ingredients claimed to be necessary for transfer. If the actual latency and motor time constant differ from the simulated 20 ms and Tm = 0.15 s, then the policy's successful hardware landing may be due to robustness from domain randomization rather than to accurate modeling of these dynamics. The simulator-fidelity assumption is therefore not merely a detail: it is the load-bearing explanation for why the policy transfers. The same unvalidated surrogate underpins the quantitative RL-vs-MPC claims in Secs V and VI, which are simulation-only and which the paper itself defers to future hardware experiments (Sec VII). As a result, the headline comparison rests on a simulator whose most critical parameters are admitted to be inaccurate.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the morpho-transition maneuver for the ATMO quadrotor, comparing an end-to-end RL policy trained with PPO and domain randomization in Isaac Lab against the model-based MPC controller from the authors' prior work. The central claims are that the RL policy transfers to hardware, achieves a final tilt angle of 65 degrees with an impact velocity of 0.5 m/s (versus 60 degrees and 1.0 m/s for MPC), exhibits better disturbance rejection than MPC for small-to-moderate pushes in simulation, and recovers from partial actuator failures without explicit fault knowledge. The authors additionally claim that the RL policy transfers to hardware only because the simulator includes motor dynamics and a 20 ms observation delay. The paper contributes an open-source simulator extension, a benchmark protocol, and a hardware demonstration.","tokens_in":11367,"tokens_out":3453,"duration_ms":35415,"significance":"If the claims hold, this is a useful contribution: it is one of the first demonstrations of end-to-end RL for mid-air transformation, it provides a principled comparison with an MPC baseline, and the code release and simulator extension are valuable for the community. The strength of the paper is its explicit attention to the ingredients needed for sim-to-real transfer, namely motor dynamics, observation delays, and domain randomization. The central result, however, rests on a single hardware demonstration per controller and on simulation-only benchmarks whose predictive value for the real robot is not established, so the quantitative claims should be treated with caution until the evidentiary basis is strengthened.","major_comments":[{"comment":"The hardware comparison is based on one representative run per controller, with no trial counts, success criteria, or error statistics. The central quantitative claims (RL final tilt 65 deg vs. MPC 60 deg; impact velocity 0.5 m/s vs. 1.0 m/s) are therefore not statistically supported. Please report multiple trials per controller, define a success/failure criterion, and report mean and variance, or explicitly characterize the hardware section as a single demonstration.","section":"Sec. IV, Fig. 5"},{"comment":"The paper states that the RL controller 'exhibits larger oscillations in the roll angle' and attributes this to 'inaccurate estimation of the system latency as well as the motor dynamics time constant.' This is a self-reported limitation on exactly the two simulator ingredients that Sec. III.D and the abstract claim are necessary for transfer, namely the 20 ms observation delay and Tm = 0.15 s. The observation weakens the causal claim: the policy may transfer because of robustness from domain randomization rather than because those parameters are accurately modeled. Please provide hardware measurements of the actual delay and motor time constant, or re-frame the claim as 'the policy transfers when trained with these modeled effects' rather than 'only transfers because these effects are accurately captured.'","section":"Sec. IV, final paragraph"},{"comment":"All quantitative comparisons of disturbance recovery and actuator failure are performed in the same simplified simulator used for training, with a single rollout per condition and no variance reporting; the paper itself defers hardware characterization to future work (Sec. VII). The statement in Sec. VII that 'the RL method performs better at small disturbances' should therefore be labeled as a simulation-only result. Please report how many random seeds and initial conditions were used per push condition, and add error bars or confidence intervals.","section":"Secs. V and VI"},{"comment":"The closed-loop morphing linkage is modeled as a pure tilt-velocity integrator with a stiff PD controller (kp = 1e15, kd = 1e5), while the paper acknowledges that the real mechanism is difficult to simulate. Because the tilt dynamics and ground-contact behavior are central to the morpho-transition maneuver, this unvalidated simplification is a threat to the sim-to-real explanation. Please provide sensitivity tests of the policy to variations in the tilt-mechanism model or justify the simplification with hardware measurements.","section":"Sec. III.A"}],"minor_comments":[{"comment":"The symbol phi_d is used in Eq. (11) but is not defined before its use in Sec. III.A; please define the desired tilt angle and its relationship to ubody.","section":"Eq. (11)"},{"comment":"The text says the reward coefficients a1...a9 are given in the appendix, but Eq. (14) uses a0; the appendix correctly lists a0...a9. Please align the notation.","section":"Sec. III.E"},{"comment":"The figure reports an 'interpolated heat map' but no color scale or axis labels are visible; please add a colorbar and define the units for impact velocity and distance to goal.","section":"Fig. 6"},{"comment":"The statement that 'training without randomizing these parameters results in neither sim-to-sim nor sim-to-real transfer' is a strong empirical claim with no supporting figure or table; please substantiate it with data or soften the wording.","section":"Sec. III.B"},{"comment":"The abstract says the RL controller 'only transfers to hardware if motor dynamics and observation delays are taken into account,' while Sec. IV reports that the hardware roll oscillations are attributed to inaccurate estimation of exactly those quantities; please reconcile these statements in the final version.","section":"Abstract and Sec. IV"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong candidate for a conference venue, but the hardware conclusions are not yet supported by the evidence. The main gap is the lack of repeated hardware trials and the unvalidated simulator parameters that the authors themselves identify as inaccurate. This is fixable within the scope of the manuscript by adding trials, measurements, and by softening the causal claims about the role of delay and motor dynamics. I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nShort version: this is the first hardware demonstration of an end-to-end RL policy for mid-air morpho-transition on the ATMO platform, and it includes a clean comparison to the authors' own MPC baseline. The RL policy lands at 65° tilt versus MPC's 60°, with lower impact velocity, and the simulation studies show better small-disturbance recovery and tolerance to partial actuator failure. That is a real result and worth engagement.\n\nWhat's genuinely new: RL has been used for aggressive quadrotor flight and landing before, but not, as the related work correctly notes, for mid-air transformation with ground contact. The paper also makes a specific claim that sim-to-real transfer fails unless motor dynamics and a 20 ms observation delay are included in training. That is an instructive finding, even if the evidence is partly anecdotal.\n\nThe paper is honest about its biggest soft spot: Sec IV reports roll oscillations on hardware and attributes them to inaccurate estimates of the very two parameters—latency and motor time constant—that the abstract says are essential for transfer. The stress-test note is right to poke at that. If the real latency is not 20 ms and Tm is not 0.15 s, then the transfer might say more about robustness from domain randomization than about model fidelity. That does not kill the paper, but it does mean the headline claim \"only transfers if...\" is not fully supported.\n\nOther soft spots are proportional but real. The hardware section shows one representative run per controller, with no trial counts or statistics. The disturbance and actuator-failure benchmarks are entirely simulation-based, and the 64-point grid looks like one rollout per condition—no error bars, no repeated seeds. The abstract presents those simulation-only results without qualification. These are addressable and should be fixed before any archival journal version.\n\nThe MPC baseline is the authors' own prior work, tested in the same simulator that ignores the effects under study. That is not circular—it is a fair baseline—but an independent validation of the simulator against hardware for the MPC case would strengthen the comparison.\n\nBottom line: a solid contribution for the robotics and sim-to-real RL community, with a genuine hardware first. It deserves serious refereeing; the weaknesses are about evidence quality, not about the core result. I'd recommend asking for repeated hardware trials and proper statistics on the sim benchmarks before accepting an archival version.","headline":"First hardware demo of end-to-end RL for mid-air morpho-transition, with a fair but thin comparison against MPC; the core result stands, but the paper's own admission about latency/motor-dynamics mismatch undermines its strongest sim-to-real claim.","tokens_in":11911,"tokens_out":1845,"would_cite":true,"duration_ms":18440,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement-learning controller lands a morphing quadrotor at a 65-degree tilt, beating an MPC baseline that reaches only 60 degrees.","keywords":["quadrotor morpho-transition","reinforcement learning","sim-to-real transfer","model predictive control","domain randomization","actuator failure recovery","aerial-ground transition","disturbance rejection"],"falsifier":"Measure the actual step response of an ATMO rotor from commanded RPM to measured thrust, and the actual end-to-end observation delay; if the best-fit motor time constant falls outside the trained randomization range of 0.10–0.20 s, or the latency is not close to a single 20 ms lumped delay, the paper's explanation for successful transfer is falsified. A second direct test: run the trained policy on hardware with an extra 40 ms of artificial observation delay; if it still lands at the reported tilt and impact velocity, then observation-delay modeling is not the load-bearing element claimed.","tokens_in":10958,"feed_emoji":"🚁","tokens_out":9642,"duration_ms":84484,"temperature":0.7,"pith_summary":"The paper aims to establish that an end-to-end reinforcement-learning controller can perform quadrotor morpho-transition—transforming in mid-air from a flying quadrotor into a wheeled rover and landing on those wheels—and that it can outperform a model-predictive-control baseline on real hardware. The central claim is that the RL policy transfers from simulation to the physical ATMO robot only when the simulator models first-order motor lag with a 0.15 s time constant and a lumped 20 ms observation delay. With those in place, the policy lands at a 65-degree tilt with a 0.5 m/s impact velocity, while the MPC baseline reaches 60 degrees at 1.0 m/s. The paper also claims the learned policy rejects small and moderate pushes better than MPC and continues to recover from substantial pushes under partial rotor loss, while MPC recovers only from the very largest pushes. If these claims hold, mid-air transformation control no longer needs hand-engineered contact planning or manually tuned attitude loops.","feed_headline":"RL lands morphing quadrotor at 65-degree tilt, beating MPC's 60","feed_subtitle":"Teaching the simulator about motor lag and sensor delay is what lets the learned policy land at 65 degrees.","key_machinery":"The load-bearing mechanism is the simulator's hardware-relevant loop closure: the robot is modeled as a rigid body whose tilt angle obeys a pure integrator $\\dot{\\phi} = \\dot{\\phi}_{\\max} u_{\\mathrm{body}}$, whose four rotors obey a first-order motor filter $\\dot{\\Omega} = T_m^{-1}(u_{\\mathrm{aero}} - \\Omega)$ with nominal $T_m = 0.15$ s, and whose policy observations are delayed by a single 20 ms step. Around this core, the authors randomize the motor time constant over $[0.10, 0.20]$ s, the thrust and moment coefficients by ±20%, the tilt speed, and apply one random push per episode; the reward is shaped to land on the wheels near the goal while penalizing contact impulse and thrust during ground contact, and the critic receives privileged information about disturbance forces, push timing, and contact impulse. A full rotation-matrix observation, rather than a quaternion, is also required for transfer. The decisive claim is that removing either the motor filter or the observation delay breaks sim-to-real transfer.","core_discovery":"The central claim is that a single neural-network policy, trained from raw observations straight to motor-speed and tilt-velocity commands, can carry ATMO through a morpho-transition, and that the deciding factor for hardware transfer is not the algorithm but simulation fidelity in three specific places: the first-order response of the rotors, a one-step 20 ms delay on observations, and randomized motor, thrust, tilt, and disturbance parameters. The trained policy lands at 65 degrees, past the 60-degree tilt at which the thrusters saturate, with a lower impact velocity than the MPC baseline, and in simulation it absorbs push disturbances up to roughly five to six times an individual rotor's thrust—beyond what it was trained on—before degrading. Under a partial rotor failure that scales the four rotor coefficients by [0.8, 0.9, 0.85, 1.1], the RL policy still recovers from substantial pushes, whereas the MPC baseline fails nearly all such tests. The trade-off visible in the hardware data is that the RL landing has larger roll oscillations, which the authors attribute to imperfect knowledge of latency and the motor time constant.","pith_inferences":["If the roll oscillation comes from residual latency mismatch, moving policy inference onto the flight controller and cutting the communication hop should reduce it without retraining; the paper's own diagnosis makes this a testable prediction.","The complementary failure regions point to a natural hybrid: let the learned policy handle the precise final landing phase, and let a model-based planner take over when the estimated disturbance exceeds a threshold.","The same simulation design—first-order actuation, a fixed observation delay, and randomization, rather than more detailed physics—may transfer to other contact-rich aerial maneuvers such as perching or hopping.","An immediate extension would be to train under correlated rotor failures rather than only uniform coefficient randomization, to test whether the observed actuator-failure recovery is a general robustness property or an artifact of the training distribution."],"forward_implications":["Morpho-transition can be solved as one end-to-end learned task: no separate flight and ground phases, no contact-phase switching, and no hand-tuned attitude loops are needed for a successful wheel landing.","The same training recipe—include motor dynamics and a one-step observation delay, then randomize dynamics—should extend to other tilting-rotor or foldable quadrotors that change thrust orientation mid-air.","Because the learned policy operates beyond the 60-degree static saturation tilt, morpho-transition can reach deeper drive-like configurations, expanding the usable landing envelope for confined spaces.","The policy's recovery from partial rotor loss without explicit fault detection demonstrates a graceful-degradation mode for learned flight controllers, within a bounded disturbance region.","The measured split—good small-push recovery for RL, good large-push recovery for MPC—maps a concrete operating envelope for each controller rather than declaring a single winner."],"supporting_citations":[{"why":"Introduces the ATMO platform, its dynamics model, and the MPC controller that serves as the baseline and the comparison target.","marker":"[1]"},{"why":"Shows that including observed rotor speeds as policy/critic observations stabilizes trained flight control, a design choice reused here.","marker":"[11]"},{"why":"Supplies the asymmetric actor-critic scheme that gives the critic privileged disturbance and contact information.","marker":"[26]"},{"why":"Provides the PPO training algorithm used to train the end-to-end policy.","marker":"[27]"},{"why":"Demonstrates massively parallel GPU simulation for learning agile, contact-rich legged maneuvers, the training paradigm extended to morpho-transition.","marker":"[24]"},{"why":"Computes the analytic equations of motion for the morphing quadrotor used in the simulator.","marker":"[17]"},{"why":"Shows an MPC extension that can handle actuator faults, used as the point of comparison for the RL controller's failure recovery.","marker":"[29]"}],"fun_headline_variants":["RL lands at 65° only if sim includes motor lag","MPC transfers out-of-box; RL needs realistic delays","RL beats MPC tilt, but rolls more on landing","Sim fidelity, not algorithm, is key for morpho transfer","RL quadrotor survives 5x rotor-thrust pushes, MPC doesn't"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simplified simulator—stiff tilt integrator, first-order motors with a 0.15 s time constant, one 20 ms observation delay, and no higher-order aerodynamics—is faithful enough that a policy trained on it behaves the same on the real ATMO; if any of those approximations misrepresent the hardware, the claimed transfer fails or the landing degrades.","fun_headline_variants_meta":{"raw":{"variants":["RL lands at 65° only if sim includes motor lag","MPC transfers out-of-box; RL needs realistic delays","RL beats MPC tilt, but rolls more on landing","Sim fidelity, not algorithm, is key for morpho transfer","RL quadrotor survives 5x rotor-thrust pushes, MPC doesn't"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00131,"raw_usage":{"total_tokens":5340,"prompt_tokens":948,"completion_tokens":4392,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":4305}},"tokens_in":564,"tokens_out":4392,"duration_ms":28122,"temperature":1.0,"reasoning_tokens":4305,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:55:06.690880+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the actual step response of an ATMO rotor from commanded RPM to measured thrust, and the actual end-to-end observation delay; if the best-fit motor time constant falls outside the trained randomization range of 0.10–0.20 s, or the latency is not close to a single 20 ms lumped delay, the paper's explanation for successful transfer is falsified. A second direct test: run the trained policy on hardware with an extra 40 ms of artificial observation delay; if it still lands at the reported tilt and impact velocity, then observation-delay modeling is not the load-bearing element claimed.","supporting_citations":[{"cited_title":"ATMO: an aerially transforming morphobot for dynamic ground-aerial transition,","cited_arxiv_id":null,"evidence_quote":"Introduces the ATMO platform, its dynamics model, and the MPC controller that serves as the baseline and the comparison target."},{"cited_title":"Cat-like jumping and landing of legged robots in low gravity using deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Demonstrates massively parallel GPU simulation for learning agile, contact-rich legged maneuvers, the training paradigm extended to morpho-transition."},{"cited_title":"proNEu: Derivation of ana- lytical kinematics and dynamics,","cited_arxiv_id":null,"evidence_quote":"Computes the analytic equations of motion for the morphing quadrotor used in the simulator."},{"cited_title":"Nonlinear MPC for quadrotor fault-tolerant control,","cited_arxiv_id":null,"evidence_quote":"Shows an MPC extension that can handle actuator faults, used as the point of comparison for the RL controller's failure recovery."}],"review_version":1}