{"id":"375c5b31-2483-43a2-8ae0-f5e15eb6806c","arxiv_id":"2411.13079","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Neural internal model control adds a rigid-body predictive error, the gap between commanded and actual body motion, as a feedback signal to an RL policy, improving disturbance robustness on quadrotors and quadrupeds.","lead":"NeuralIMC adds a simple rigid-body prediction of the robot's next state to the input of a reinforcement-learned controller, using the difference between predicted and actual motion as a feedback signal. The method is tested on quadrotors and quadruped robots, and on a real quadrotor carrying rope-suspended payloads, where it improves tracking robustness.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Predictive model's command-to-acceleration conversion is underspecified and may be degenerate for quadrupeds, gating the central claim.","rationale":"The reader's conditional verdict is appropriate; I do not find a reason to reject. However, the most load-bearing assumption is not just that the mapping is approximate, but that the mapping is well-defined. The paper's own ablation on noise tolerance and model simplification presupposes a particular implementation, but the text leaves room for the predictive model to be driven by measurements rather than commands, especially in the quadruped case. A code inspection and targeted ablation would settle this without adversarial assumptions. This concern is aligned with, but more specific than, the reader's weakest_assumption, because it identifies a potential degeneracy where the predictive error carries no command information at all.","tokens_in":13358,"tokens_out":5003,"duration_ms":53291,"concrete_test":"Open the released repository and inspect the predictive-error computation for both morphologies. For the quadrotor, verify whether a_k is computed as R * [0,0,c]/m - g using the randomized mass and current orientation, and whether \\dot{\\omega}_k is set to zero or derived from finite differences of bodyrate commands; then re-run the zigzag evaluation with a deliberately wrong nominal mass (e.g., 1.5x) in the predictive model only, comparing PEF-on vs PEF-off. For the quadruped, test whether replacing the differentiated reference-velocity 'acceleration' with zero changes the predictive error and the results; if it does not, the predictive model is not contributing command information and the method reduces to feeding measured velocity increments.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Eqs. (1)-(6) define the predictive error from commanded linear acceleration a_k and angular acceleration \\dot{\\omega}_k, but the paper never states how these are obtained from the actual action spaces. For quadrotors (Sec. III-C1) the action is CTBR: collective thrust is a force/motor command, not an acceleration, and bodyrate is an angular velocity, not an angular acceleration; the mass used in the conversion and the treatment of \\dot{\\omega}_k (zero? finite difference of bodyrate commands?) are omitted. For quadrupeds (Sec. III-C2), reference velocity commands are differentiated to obtain accelerations, but the commands are piecewise constant in the evaluation protocol; their derivative is zero almost everywhere, so the 'predictive model' input carries no acceleration information and the predictive error degenerates to the negative of the measured velocity increment—a signal that is available without any internal model. If the released implementation actually uses measured acceleration or a different mapping, the method is not the IMC-style command-driven predictive error claimed in Fig. 2, and the robustness attribution in Tables II/III is ambiguous. This is load-bearing because the central claim is that explicit predictive error feedback from a simplified rigid-body model driven by policy commands is the cause of the gains.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Neural Internal Model Control (NeuralIMC), which combines a simplified Newton-Euler rigid-body predictive model with model-free RL policies by feeding the predictive error—the difference between the predicted next body state and the actual measured state—as an additional input to the policy. The method is implemented for quadrotors using a CTBR action space and for quadrupedal robots using reference velocity commands, and evaluated in simulation against several baselines (PPO, RMA, HIM, DATT, CAL, L1-based controllers) as well as in a real-world quadrotor experiment with suspended payloads. The authors claim consistent improvements in tracking accuracy and robustness across in-distribution and out-of-distribution dynamics, and release source code.","tokens_in":13620,"tokens_out":5893,"duration_ms":53138,"significance":"If the claims hold, the paper offers a simple, arguably general plug-in mechanism for improving robustness in RL-based control: replacing learned or handcrafted disturbance estimators with an explicit white-box rigid-body prediction and using the mismatch as policy feedback. The experimental effort is substantial—five seeds with 1024 parallel quadrotor trials and 4096 parallel environments for quadrupeds, OOD evaluation, ablations on history length, noise tolerance, and model complexity, plus a real-world deployment. The release of code and the explicit nature of the predictive model are clear strengths. However, the paper itself acknowledges limited hyperparameter tuning of baselines, and the precise definition of the predictive model inputs is under-specified, which currently leaves the central claim partially unsupported.","major_comments":[{"comment":"The predictive model requires acceleration inputs a_k and \\dot{\\omega}_k, but the paper never specifies how these are derived from the actual action spaces: for quadrotors, the CTBR action contains collective thrust and bodyrate (a force and an angular velocity, not accelerations), and for quadrupeds the reference velocity commands are piecewise constant so their derivative is zero almost everywhere, making the predictor's acceleration input zero and the 'predictive error' degenerate to the measured velocity increment. Please specify the exact command-to-acceleration mapping used in the implementation, including the nominal mass used for thrust conversion and the treatment of \\dot{\\omega}_k; if the quadruped predictor is indeed driven by zero acceleration, state how the resulting signal differs from a simple state-difference feedback and why the IMC interpretation remains valid.","section":"§III-C1–III-C2, Eqs. (1)–(6)"},{"comment":"The paper acknowledges that 'limited hyperparameter sweeps in our experiments may contribute to the baselines’ degraded performance.' Since the central comparative claim is that NeuralIMC 'surpasses recent state-of-the-art control methods,' this admission is a direct caveat on that claim; the privileged variants DATT(GT) and RMA(T) mitigate but do not eliminate the concern, because those privileged variants are not the actual deployed baselines. Please either report additional hyperparameter tuning for the baselines or rephrase the conclusions to reflect that the comparison is against a particular (possibly suboptimally tuned) set of baselines.","section":"§IV-B, Table II and the paragraph following it"},{"comment":"In the model-complexity ablation, the full-dynamics model actually achieves lower evaluation errors than the simplified model under both static (0.095 vs 0.101 m) and disturbed (0.105 vs 0.119 m) conditions on the zigzag task, so the claim that 'performance degradation is marginal' is misleading; the text should report these numbers accurately and should not extrapolate from this single-morphology experiment to the claim that simplification 'improves generalization across different morphologies.'","section":"§IV-D, Table V"}],"minor_comments":[{"comment":"The caption contains a typo: 'applirope' should likely be 'applicable to'.","section":"Fig. 2 caption"},{"comment":"There is a typo in the text: 'RL-baesd' should be 'RL-based'.","section":"Sec. II-B"},{"comment":"The sentence 'To integrate our predictive model into the control loop,, we use reference velocity commands as model inputs' contains a double comma after 'loop'.","section":"Sec. III-C2"},{"comment":"The text reads 'MA VROS' but the intended name of the ROS-to-MAVLink bridge is 'MAVROS'.","section":"Sec. IV-E"},{"comment":"The row labels 'Train' and 'Eval' are ambiguous; they appear to distinguish in-distribution from out-of-distribution dynamics evaluation, but the caption only explains the blue cells and percentages, not the row grouping.","section":"Table II"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of RA-L and the code release is commendable. The main technical concern is the underspecified command-to-acceleration mapping in the predictive model, which directly affects whether the claimed mechanism is what drives the improvements. The authors should provide the missing implementation details and, if the quadruped predictor indeed uses zero acceleration, re-frame the interpretation or add a version with a non-trivial command signal. The hyperparameter-tuning admission should also be addressed with additional experiments or softened claims before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nNeuralIMC does something genuinely simple: instead of learning or hand-tuning a disturbance estimator, it feeds the difference between a fixed rigid-body kinematic prediction and the actual body state into an RL policy. No training for the internal model, no adaptive law, no latent encoder. That is new relative to RMA, HIM, and DATT, and it makes the method look like a genuinely plug-in signal. The quadrotor and quadruped experiments are extensive—five seeds, large parallel trials, OOD evaluation, privileged baselines—and the real-world rope-payload demo is a meaningful sanity check. The paper also ships code. The ablations showing that history doesn't help and the noise-tolerance study are useful additions.\n\nNow the soft spots, in proportion. The main one is the command-to-acceleration conversion in the predictive model, and it is load-bearing. Equations (1)-(6) need linear acceleration a_k and angular acceleration \\dot{\\omega}_k, but the paper never states how these come from CTBR for quadrotors (collective thrust is a force, bodyrate is an angular velocity) or from reference velocity commands for quadrupeds. For quadrupeds the reference commands in evaluation are piecewise constant, so differentiating them gives zero almost everywhere; the \"predictive error\" then degenerates to a measured velocity increment that needs no internal model. If the released code actually uses measured acceleration or some other mapping, the method is not quite the command-driven IMC structure claimed in Fig. 2, and the robustness attribution gets muddy. This is not fatal for the quadrotor story—there the conversion, while unspecified, is more plausibly a correct acceleration proxy—but it does gate the generality claim across morphologies. The paper's own admission of limited hyperparameter sweeps for baselines is honest but means the \"surpasses SOTA\" wording is softer than it looks; the privileged baselines help, though. Main tables report averages without error bars, so we can't judge variance even with five seeds.\n\nI don't think the circularity charge lands; the predictive model has no fitted parameters and the error is computed from states and commands, so this is feedback, not circular reasoning.\n\nWho gains: anyone working on robust RL control for aerial or legged robots, especially people who want a cheap disturbance signal without learned estimators. It deserves a serious referee, but a refereed revision should pin down the exact action-to-acceleration mapping and show the quadruped signal is not empty. I'd send it to review, with a note that the mapping issue has to be resolved.","headline":"NeuralIMC offers a genuinely simple plug-in disturbance signal for RL control, with strong simulation work and a real drone demo, but the paper never specifies how policy commands become the predictor's acceleration inputs—and for quadrupeds that gap may empty out the signal.","tokens_in":14100,"tokens_out":1754,"would_cite":true,"duration_ms":17576,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes Neural Internal Model Control, which feeds a rigid-body predictive-error signal into an RL policy, and presents evidence that this signal improves trajectory tracking and robustness on quadrotors and quadrupedal robots.","keywords":["robust control","reinforcement learning","internal model control","predictive error feedback","quadrotor control","quadrupedal locomotion","rigid-body dynamics","sim-to-real transfer"],"falsifier":"Train a quadrotor policy with and without predictive error feedback while intentionally mis-specifying the mass used to convert collective thrust into acceleration and applying no external disturbance; if the error-feedback policy is not clearly better in that clean setting, the error signal is dominated by conversion error rather than by external disturbances, and the claimed mechanism is not the one producing robustness.","tokens_in":13157,"feed_emoji":"🤖","tokens_out":8734,"duration_ms":82323,"temperature":0.7,"pith_summary":"Neural Internal Model Control proposes that a robot controller becomes more robust when the policy receives one extra input: the difference between the body state predicted by a simplified rigid-body model and the body state actually observed. This predictive error grows when external disturbances or unmodeled dynamics push the robot away from the predicted motion, and the paper argues that a model-free reinforcement learning policy can learn to act on it without needing a full-dynamics model or a hand-tuned disturbance estimator. The framework is generic enough to be wrapped around existing learned controllers, and the paper presents simulation results on quadrotors and quadrupedal robots plus a real quadrotor carrying rope-suspended payloads as evidence that the extra signal improves tracking and robustness.","feed_headline":"Predictive error feedback steers robots through disturbances","feed_subtitle":"Feeding the gap between rigid-body prediction and observed state beats hand-tuned estimators on drones and quadrupeds.","key_machinery":"The load-bearing object is the predictive error feedback loop: an explicit predictor built from the Newton-Euler update equations converts the policy's command into a next-body-state prediction, and the gap between prediction and observation is encoded as a small error vector fed back into the policy at the next step. The deliberate simplification is to model only the robot body as a rigid transformation, so the predictor needs no actuator models, terrain models, or learned latent dynamics. In the Internal Model Control tradition, the difference between plant and model responses carries the disturbance information; here the RL policy learns to compensate for whatever the simplified predictor misses, which is why the authors describe the loop as closing model-based and model-free control.","core_discovery":"The central claim is that a white-box rigid-body predictor, using only Newton-Euler kinematics for the main body, provides a feedback signal that makes an RL-trained control policy robust to disturbances and out-of-distribution dynamics. The predictor integrates the policy's acceleration and angular-acceleration commands into a next body state, and the predictive error vector, composed of position, velocity, angular velocity, and orientation cosine distance between predicted and observed states, is appended to the policy input. The paper asserts that this error directly signals state drift and disturbance magnitude, and that a policy trained on it outperforms estimator-based and estimator-free baselines on quadrotor trajectory tracking and quadruped locomotion, including under randomized external forces and unseen dynamics. Real-world deployment on a quadrotor with rope-suspended payloads is offered as evidence that the benefit survives sim-to-real transfer.","pith_inferences":["The paper leaves open exactly how command-to-acceleration conversions are implemented; a direct test would vary that conversion across a range of nominal masses and observe when the predictive error stops improving control.","Because the predictor is morphology-agnostic, the same error vector could be applied to other robots with body-state estimation, such as humanoids or aerial manipulators, whenever a command-to-acceleration proxy can be defined.","The result that predictive error feedback matches or beats a privileged teacher receiving ground-truth disturbances suggests that explicit rigid-body error can replace learned latent disturbance encoders in other teacher-student pipelines; that replacement is testable by removing the encoder and keeping the error input."],"forward_implications":["Adding predictive error feedback to a PPO-trained policy reduces trajectory tracking error on quadrotors compared with L1-based controllers, CAL, RMA, and DATT baselines, with the largest gains under out-of-distribution dynamics.","On quadrupedal robots the same feedback structure improves linear velocity and yaw-rate tracking over PPO and HIM, and matches or exceeds a privileged RMA variant that receives ground-truth disturbances.","One-step predictive error feedback is sufficient: longer short-term or long-term input histories do not improve performance, according to the paper's ablations.","The simplified rigid-body predictor performs nearly as well as a full-dynamics model on quadrotor tracking, indicating that the simplification is not the main source of degradation.","Real quadrotor experiments with 200 g and 330 g rope-suspended payloads show the method keeps tracking stable, and removing the predictive error feedback degrades altitude control under the payload."],"supporting_citations":[{"why":"Defines the internal model principle that the framework adapts by comparing predicted and actual responses.","marker":"[6]"},{"why":"Supplies the classical internal model control design the method extends to learned policies.","marker":"[3]"},{"why":"Provides the Hybrid Internal Model baseline and the quadruped training setup the paper follows and improves upon.","marker":"[13]"},{"why":"Provides the DATT baseline with L1 adaptive disturbance estimation and the quadrotor trajectory and disturbance protocol used for evaluation.","marker":"[20]"},{"why":"Provides the RMA teacher-student baseline whose privileged variant serves as an upper-bound comparison.","marker":"[12]"},{"why":"Supplies the proximal policy optimization algorithm used to train all policies.","marker":"[27]"},{"why":"Supplies the massively parallel simulation environment used to train quadruped controllers.","marker":"[17]"},{"why":"Supplies the quadrotor dynamics model used in the customized simulator.","marker":"[40]"},{"why":"Supplies the exponential-map quaternion update formula used by the rigid-body predictor.","marker":"[37]"}],"fun_headline_variants":["Predictive error feedback makes robot control robust","White-box physics feedback sharpens RL control","Neural Internal Model Control beats estimators on drones and quadrupeds","Feed the prediction gap to get resilient robot policies","Robots learn to self-correct via predictive error"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's feedback signal only carries disturbance information if the policy's raw output can be treated as an acceleration or angular-acceleration command for the rigid-body model, and the paper does not specify or validate the conversion for collective thrust and reference velocity commands.","fun_headline_variants_meta":{"raw":{"variants":["Predictive error feedback makes robot control robust","White-box physics feedback sharpens RL control","Neural Internal Model Control beats estimators on drones and quadrupeds","Feed the prediction gap to get resilient robot policies","Robots learn to self-correct via predictive error"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000213,"raw_usage":{"total_tokens":1396,"prompt_tokens":893,"completion_tokens":503,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":429}},"tokens_in":509,"tokens_out":503,"duration_ms":5292,"temperature":1.0,"reasoning_tokens":429,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:51:32.459126+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a quadrotor policy with and without predictive error feedback while intentionally mis-specifying the mass used to convert collective thrust into acceleration and applying no external disturbance; if the error-feedback policy is not clearly better in that clean setting, the error signal is dominated by conversion error rather than by external disturbances, and the claimed mechanism is not the one producing robustness.","supporting_citations":[{"cited_title":"Internal model control: Pid controller design,","cited_arxiv_id":null,"evidence_quote":"Supplies the classical internal model control design the method extends to learned policies."},{"cited_title":"DATT: Deep adaptive trajectory tracking for quadrotor control,","cited_arxiv_id":null,"evidence_quote":"Provides the DATT baseline with L1 adaptive disturbance estimation and the quadrotor trajectory and disturbance protocol used for evaluation."},{"cited_title":"Thrust mixing, saturation, and body-rate control for accurate aggressive quadrotor flight,","cited_arxiv_id":null,"evidence_quote":"Supplies the quadrotor dynamics model used in the customized simulator."}],"review_version":1}