{"id":"beed5cc5-59a4-48c5-9b6d-e5dbd1c2e731","arxiv_id":"2412.11764","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"SimpleFlight, a PPO framework combining five design choices, cuts real-world quadrotor tracking error by over 50% and successfully tracks infeasible zigzag trajectories zero-shot.","lead":"This paper identifies five training-design factors that let a reinforcement-learning quadrotor policy transfer from simulation to a real drone without any fine-tuning. The resulting framework, SimpleFlight, more than halves tracking error on a Crazyflie drone compared with state-of-the-art RL baselines and handles sharp-turn trajectories that other methods fail on.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The zero-shot recipe is partly tuned on the same real-world benchmark used for evaluation (λ in Fig. 4, DR and batch size in §V-B3/4), so the >50% comparison may be an in-sample estimate rather than a transferable prediction.","rationale":"The reader's weakest assumption was simulator fidelity and low-level controller matching. That is a legitimate concern, but the paper's empirical success across two platforms and the open-sourced code/checkpoints partly mitigate it. The selection-on-evaluation issue is more direct and more load-bearing for the paper's scientific contribution: the paper claims a transferable recipe and identifies five critical factors, yet several factor choices (λ, DR ranges, batch-size recommendation) are explicitly made from real-world results on the same figure-eight benchmark used in the final comparison. This does not invalidate the hardware demonstration, but it undermines the generality of the 'what matters' claim and makes the quantitative >50% reduction an optimistic in-sample estimate. The proposed held-out test would settle whether the recipe transfers without benchmark-specific tuning. Since the reader already assigned a CONDITIONAL verdict, this concern does not change the verdict but sharpens the condition under which the central claim would be accepted.","tokens_in":13366,"tokens_out":5649,"duration_ms":56419,"concrete_test":"Hold out a set of real-world benchmark trajectories not used in any design choice. Re-run the factor selection with λ, DR ranges, and batch size chosen from simulation-only criteria or from a separate real-world validation set, then deploy the frozen policy on the held-out trajectories and recompute MED. If the >50% reduction over DATT/Fly shrinks materially, or if failures appear on infeasible trajectories, the headline claim was inflated by test-set tuning. A minimal version: freeze the architecture and all factors, vary only λ using simulated validation MED, deploy on the existing figure-eight suite, and compare the resulting MED against the λ=0.4 results in Table III.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SimpleFlight is a zero-shot recipe whose five factors transfer directly from simulation to real drones. That claim is load-bearing on the recipe not being tuned to the same benchmark trajectories used for evaluation. Section V-B2 selects the smoothness weight λ using real-world figure-eight results (Fig. 4) and sets λ=0.4 based on those real-world experiments. Section V-B3 selects DR behavior using real-world figure-eight performance (Table II), and Section V-B4 derives batch-size guidance from real-world figure-eight curves (Fig. 5). These figure-eight tasks are the same tasks used in the final comparison (Table III). Consequently, the reported >50% reduction and the 'only one capable' claim are at risk of being in-sample estimates rather than predictive evidence for the recipe. The paper itself acknowledges at the end of Section V-C2 that the baseline comparison 'may not be entirely fair,' which further cautions against treating the quantitative margin as exact, but the more serious confound is selection on the evaluation tasks. A policy deployed zero-shot can still have hyperparameters tuned on real-world test tasks, and the current write-up does not separate those two stages.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies five factors for learning zero-shot sim-to-real RL policies for quadrotor trajectory tracking: actor inputs including velocity and rotation matrix, critic-only time vector, action-difference smoothness regularization, system identification with selective domain randomization, and large batch sizes. These factors are combined into a PPO-based framework called SimpleFlight. The authors validate SimpleFlight on a Crazyflie 2.1 and a custom 'Air' quadrotor, reporting over 50% reduction in mean Euclidean distance tracking error against two RL baselines (Fly, DATT) and comparable or better performance than a fine-tuned MPC baseline (PAMPC). The paper also integrates SimpleFlight into the Omnidrones simulator and open-sources code and checkpoints.","tokens_in":13609,"tokens_out":3916,"duration_ms":39168,"significance":"If the central claim holds, this is a valuable empirical study that consolidates several known-but-scattered design choices into a concrete recipe and demonstrates real-world viability on low-thrust-to-weight quadrotors. The open-source release and Omnidrones integration are concrete assets for reproducibility. However, the headline quantitative claim is compromised by the fact that several recipe components were selected using real-world performance on the same figure-eight benchmark that appears in the final evaluation, and the paper itself acknowledges that the baseline comparison is not entirely fair. These issues make the reported 50% improvement an in-sample estimate rather than a validated zero-shot prediction, so the strongest claims in the abstract and Section V need revision or additional holdout experiments before they can be accepted as stated.","major_comments":[{"comment":"The smoothness reward coefficient λ is set to 0.4 based on real-world figure-eight tracking performance (Fig. 4), the domain-randomization ranges and conclusions are derived from real-world figure-eight experiments (Table II), and the batch-size recommendation is based on real-world figure-eight curves (Fig. 5). These same figure-eight trajectories (slow, normal, fast) are then reported in the final comparison of Table III. Consequently, the reported \"more than 50% reduction\" and the \"only one capable of successfully completing all benchmarking trajectories\" claims are in-sample estimates rather than predictive evidence for a zero-shot recipe. The paper should provide a holdout evaluation on trajectory types or parameter settings that were not used for any hyperparameter selection, or it should explicitly state that the final comparison shares the tuning tasks and correspondingly soften the zero-shot claim.","section":"§V-B2, V-B3, V-B4; Fig. 4; Table II; Fig. 5; Table III"},{"comment":"The paper states that the comparison in Table III \"may not be entirely fair\" because the policies use different simulators, modeling approaches, and input/output spaces. Since the headline claim of over 50% error reduction is relative to these baselines, this caveat should be stated in the abstract and in the main text where the claim is made, not only at the end of Section V-C2. Additionally, for DATT on zigzag trajectories, the footnote reports that 4 out of 10 attempts failed and only the successful trials are averaged; the failure rate itself is a primary metric and should be reported in the table or text before any claim that SimpleFlight is \"the only one\" capable of completing all benchmarks.","section":"§V-C2 and Table III"},{"comment":"The selective-DR conclusion is based on artificially induced +30% parameter offsets to simulate miscalibration. While this is a reasonable synthetic test, the real calibration errors of the Crazyflie are not quantified, so the practical recommendation that DR should be applied only to thrust coefficient kf is not directly supported by measured calibration uncertainty. A sensitivity analysis over the actual estimated parameter errors, or a statement of the calibration accuracy, would strengthen the factor-4 claim.","section":"§V-B3 and Table II"}],"minor_comments":[{"comment":"The word \"polynominal\" is a typo and should be \"polynomial.\"","section":"Abstract"},{"comment":"The text \"U A Vs\" in the introduction should be \"UAVs\" without spaces.","section":"Section I"},{"comment":"The text \"17-dimensinal\" is a typo and should be \"17-dimensional.\"","section":"Section III-B"},{"comment":"The observed \"approximately 63.6%\" performance degradation when replacing the rotation matrix with a quaternion is not derived from any displayed curve; please specify the exact quantities or figure from which this percentage is computed.","section":"Section V-B1"},{"comment":"The smoothness reward is presented in Eq. (4) as a generic rsmooth, but Table I evaluates components using raux = e^{-A}; please clarify the relationship between these two formulations, particularly how ||ut - ut-1||2 is inserted as A.","section":"Section V-B2 and Table I"},{"comment":"For trajectories where some trials fail, the MED is reported only over successful trials (e.g., DATT zigzag). Please add a column or annotation showing the number of successful trials per method and trajectory, since \"∞\" alone does not distinguish partial from total failure.","section":"Table III"},{"comment":"References [7] and [39] both cite Williams et al. on information-theoretic MPC; please merge or differentiate them.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is a strong empirical study with real-world validation and open-source artifacts, and the five-factor decomposition is a useful organizing principle. However, the central quantitative claim is currently undermined by selection on the evaluation benchmark and by the acknowledged unfairness of the baseline comparison. These issues are addressable within the manuscript's scope via a holdout evaluation or by reframing the claims as in-sample tuning results, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my read. The real contribution is a systematic ablation of five training factors for zero-shot sim-to-real quadrotor control, packaged as a concrete recipe (SimpleFlight) with code and checkpoints. Each factor is known in isolation, but the combined study and the cross-platform validation on Crazyflie and a custom quadrotor are genuinely useful. The standout capability is tracking infeasible zigzag/pentagram trajectories on a low-thrust-to-weight platform, which prior RL baselines fail. Real-world evidence is substantial: two platforms, several trajectory types, and a tuned MPC baseline on the custom platform.\n\nThe soft spots are real but proportionate. The stress-test concern lands: lambda, DR ranges, and batch size are selected using real-world figure-eight performance (Fig. 4, Table II, Fig. 5), and figure-eight appears in the final Table III. So the >50% reduction over baselines is partly an in-sample estimate for that task, not a purely predictive claim about the recipe. The paper's own remark that the comparison 'may not be entirely fair' is honest but doesn't remove the selection issue. Also, a single randomly-selected policy from three seeds is deployed; fine for robustness evidence, but limited for variance. Some training details (exact batch size, final DR ranges) are missing.\n\nThe central capability claim still holds: pentagram and zigzag are not used for hyperparameter selection, and the policy completes them. That is the strongest evidence. The quantitative margin on figure-eight should be framed as 'with tuning on this task.'\n\nWho this is for: robotics researchers working on sim-to-real RL for aerial vehicles, especially those wanting a concrete baseline. The paper deserves a serious referee; the empirical work is substantial and the code release makes it checkable. My recommendation: accept, but require the authors to separate tuning from evaluation—e.g., report a hold-out trajectory set for hyperparameter selection, and state all training details. The flaws are correctable.\n\nFor peer review: yes, send it out.","headline":"Useful empirical recipe for zero-shot quadrotor RL, but the headline margin is partly in-sample because key hyperparameters were tuned on the same real-world tasks used for evaluation.","tokens_in":14174,"tokens_out":2454,"would_cite":true,"duration_ms":23677,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SimpleFlight claims that five concrete training choices—velocity and rotation-matrix actor inputs, critic time vector, action-difference smoothness, selective domain randomization, and large batches—make a PPO quadrotor policy transfer…","keywords":["zero-shot sim-to-real","quadrotor control","reinforcement learning","trajectory tracking","domain randomization","system identification","smoothness regularization","proximal policy optimization"],"falsifier":"Run the paper's Offset+30% condition on a real quadrotor: if deliberately shifting the thrust coefficient $k_f$ by +30% does not raise tracking error well above the calibrated policy's error, the claim that $k_f$ is a sensitive parameter and that system identification is load-bearing is contradicted. A complementary test is to compare simulated and real motor step responses, since the low-level controller alignment premise must hold for any zero-shot transfer.","tokens_in":13145,"feed_emoji":"🚁","tokens_out":9586,"duration_ms":83852,"temperature":0.7,"pith_summary":"This paper tries to establish that zero-shot sim-to-real transfer for quadrotor control does not require a new algorithm or a bespoke network architecture: five concrete training choices are enough. The choices are feeding the actor velocity and a rotation matrix rather than a quaternion, giving the critic a time vector, penalizing the difference between successive actions, calibrating key dynamic parameters with domain randomization only where it helps, and using large batch sizes. The authors package these into a PPO-based framework called SimpleFlight and report that on a nano quadrotor it reduces trajectory tracking error by more than 50% relative to state-of-the-art RL baselines. Because the same policy, deployed without fine-tuning, also completes infeasible zigzag trajectories that baselines fail, the recipe claims to cover both smooth and aggressive flight.","feed_headline":"Five training choices cut quadrotor tracking error by half","feed_subtitle":"A PPO recipe with rotation-matrix inputs and smooth actions transfers zero-shot from simulation to real drones.","key_machinery":"The machinery is the SimpleFlight training pipeline: an asymmetric actor-critic PPO setup whose actions are CTBR commands (collective thrust and body rates). The actor observes relative positions to ten future reference points, linear velocity, and the rotation matrix; the critic observes the same plus a scalar time vector. The reward augments the tracking task with $e^{-\\|u_t-u_{t-1}\\|_2}$, which directly penalizes jumps between successive commands. Before training, four dynamic parameters (mass, inertia, thrust coefficient, and motor time constant) are calibrated, and domain randomization is applied selectively, principally to the thrust coefficient; the low-level controller inside the simulator is aligned to the firmware's response. Training runs in a GPU-parallel simulator with large batches, and the resulting policy is deployed without modification.","core_discovery":"SimpleFlight claims that a PPO-based policy trained with five specific choices—velocity and rotation matrix in the actor's observation, a time vector in the critic's observation, an action-difference smoothness reward $\\|u_t-u_{t-1}\\|_2$, calibrated dynamics with domain randomization applied only to the thrust coefficient, and large training batch sizes—can be deployed directly on real quadrotors without fine-tuning. In real-world tests on a nano quadrotor, it reports a reduction of more than 50% in mean Euclidean distance to the reference trajectory compared with state-of-the-art RL baselines, and it is the only tested policy that completes all benchmark trajectories, including fast pentagram and infeasible zigzag paths. The same policy class also transfers to a larger custom quadrotor, where it slightly outperforms a finely tuned model-predictive controller. The paper's claim is that the five factors, not any new algorithm or architecture, are what make zero-shot transfer work.","pith_inferences":["The paper's ablations imply that rotation-matrix inputs are a first-order factor for agile flight, so the same representation choice likely matters for other rotation-controlled robots, not just quadrotors.","Since the lookahead horizon is fixed at $N=10$ without ablation, a natural extension is to sweep $N$ on infeasible trajectories, where longer horizons should matter most.","The selective-domain-randomization result suggests a practical tuning rule: randomize only parameters that are hard to measure and performance-sensitive, and leave precisely measurable ones fixed.","The time-vector result hints that adding task-phase information to value networks could be a cheap general trick for non-stationary RL problems beyond quadrotor control."],"forward_implications":["Any future RL quadrotor controller that adopts these five factors should expect at least a halving of trajectory tracking error on smooth trajectories compared with current RL baselines, under zero-shot deployment.","Small thrust-to-weight quadrotors can track infeasible zigzag trajectories that previously required optimization-based methods, because the action-difference smoothness regularizer supports long-horizon planning without clipping agility.","Larger batch sizes are a free lunch: they improve real-world generalization with no change to architecture or reward.","Domain randomization should be applied selectively; randomizing well-calibrated mass or inertia can hurt real-world performance, while randomizing the thrust coefficient helps when calibration is imperfect.","The recipe is platform-transferable: the same policy class transfers to a second, larger custom quadrotor and remains competitive with a finely tuned MPC controller."],"supporting_citations":[{"why":"Supplies the deep adaptive trajectory tracking baseline whose reported real-world errors SimpleFlight is compared against.","marker":"[11]"},{"why":"Introduces the collective-thrust-and-body-rate action space that SimpleFlight adopts as its control interface.","marker":"[15]"},{"why":"Provides the high-parallel GPU simulator used to train SimpleFlight at large batch sizes.","marker":"[17]"},{"why":"Presents the prior RL racing policy whose inclusion of the previous action in the actor input SimpleFlight tests and rejects.","marker":"[31]"},{"why":"Supplies the direct-rotor-speed RL baseline that fails on infeasible trajectories in the comparison.","marker":"[33]"},{"why":"Provides the proximal policy optimization algorithm on which the SimpleFlight training loop is built.","marker":"[36]"},{"why":"Establishes that low-dimensional rotation representations are discontinuous, motivating the rotation matrix in the actor input.","marker":"[37]"},{"why":"Supplies the finely tuned model-predictive controller baseline used to stress-test SimpleFlight on the custom quadrotor.","marker":"[38]"}],"fun_headline_variants":["Five PPO tweaks realize zero-shot drone control, halving track error","Zero-shot sim-to-real RL for quadrotors: five key factors revealed","SimpleFlight halves tracking error without real-world tuning","PPO with five design choices flies real drones zero-shot","Quadrotor RL: five training choices enable zero-shot deployment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole zero-shot claim rests on the simulated low-level controller and the rigid-body dynamics matching the real quadrotor's firmware and airframe closely enough that a policy trained in simulation faces the same input-output mapping on hardware.","fun_headline_variants_meta":{"raw":{"variants":["Five PPO tweaks realize zero-shot drone control, halving track error","Zero-shot sim-to-real RL for quadrotors: five key factors revealed","SimpleFlight halves tracking error without real-world tuning","PPO with five design choices flies real drones zero-shot","Quadrotor RL: five training choices enable zero-shot deployment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000757,"raw_usage":{"total_tokens":3406,"prompt_tokens":1032,"completion_tokens":2374,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":648,"completion_tokens_details":{"reasoning_tokens":2286}},"tokens_in":648,"tokens_out":2374,"duration_ms":17546,"temperature":1.0,"reasoning_tokens":2286,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:36:27.821300+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the paper's Offset+30% condition on a real quadrotor: if deliberately shifting the thrust coefficient $k_f$ by +30% does not raise tracking error well above the calibrated policy's error, the claim that $k_f$ is a sensitive parameter and that system identification is load-bearing is contradicted. A complementary test is to compare simulated and real motor step responses, since the low-level controller alignment premise must hold for any zero-shot transfer.","supporting_citations":[{"cited_title":"Datt: Deep adaptive trajectory tracking for quadrotor control,","cited_arxiv_id":null,"evidence_quote":"Supplies the deep adaptive trajectory tracking baseline whose reported real-world errors SimpleFlight is compared against."},{"cited_title":"A benchmark comparison of learned control policies for agile quadrotor flight,","cited_arxiv_id":null,"evidence_quote":"Introduces the collective-thrust-and-body-rate action space that SimpleFlight adopts as its control interface."},{"cited_title":"Omnidrones: An efficient and flexible platform for reinforcement learning in drone control,","cited_arxiv_id":null,"evidence_quote":"Provides the high-parallel GPU simulator used to train SimpleFlight at large batch sizes."},{"cited_title":"Champion-level drone racing using deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Presents the prior RL racing policy whose inclusion of the previous action in the actor input SimpleFlight tests and rejects."},{"cited_title":"On the continuity of rotation representations in neural networks,","cited_arxiv_id":null,"evidence_quote":"Establishes that low-dimensional rotation representations are discontinuous, motivating the rotation matrix in the actor input."}],"review_version":1}