Pith. sign in

REVIEW 2 major objections 6 minor 24 references

All Eyes, no IMU: Learning Flight Attitude from Vision Alone

T0 review · 2 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A quadrotor can hold position in real flight using only a downward event camera and a small recurrent neural network, with no inertial sensors in the control loop.

desk verdict A real first: a quadrotor hovered with its attitude and rate loop driven entirely by an event camera and a learned network, but the 'generic environments' claim outstrips the closed-loop evidence. read the letter →

arxiv 2507.11302 v1 pith:GAEE4B3R submitted 2025-07-15 cs.RO cs.CV

Jesse J. Hagenaars , Stein Stroobants , Sander M. Bohte , Guido C.H.E. De Croon This is my paper · ORCID
classification cs.ROcs.CV
keywords event-basedvisionflightattitudeestimationquadrotorcontrolIMUreplacementrecurrentconvolutionalneuralnetworkeventcameravision-onlygeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a quadrotor can estimate its own attitude and rotation rate using only a downward-facing event camera, with a small recurrent convolutional neural network converting the event stream into control estimates at 200 Hz. The authors show that these estimates can stand in for an inertial measurement unit (IMU) and gyroscope in the inner attitude and rate control loop, and they demonstrate position-hold hover with the network in control for about ten minutes of real flight. They position this as the first vision-only approach to flight control that works in generic environments rather than structured scenes. A secondary finding is that a network with memory and a wide field of view performs best in absolute terms, but a network restricted to a center crop, forced to rely on motion cues rather than horizon-like visual lines, generalizes relatively better to unseen environments.

What carries the argument

The central mechanism is a small recurrent convolutional network, roughly 425k parameters, whose convolutional encoder downsamples 5 ms event-count frames into features that pass through a gated recurrent unit (GRU) memory block and are decoded into attitude and rotation-rate predictions. The recurrence is what accumulates information over time, which the experiments show is needed to estimate rotation rates; removing it raises rate RMSE from 10.65 to 18.65 degrees per second. The estimates enter an otherwise standard quadrotor control loop where a proportional angle controller and a PID rate controller would normally receive filtered IMU signals, allowing the learned network to act as a drop-in stand-in for the IMU.

What would settle it

Run the same trained network in an unfamiliar environment while recording attitude with an independent motion-capture system rather than the flight controller's IMU filter, and command fast pitch and roll maneuvers. If the network's estimates diverge under high rates or show a bias that matches the IMU filter's known bias, the claim that it has learned a general internal motion model is weakened.

Watch

Extended reading notes

Core claim

The authors' claim is that a compact recurrent convolutional network can estimate a quadrotor's attitude and rotation rate from event-camera frames alone, and that these estimates are accurate enough to replace an inertial measurement unit in the inner attitude and rate control loop during real flight. In their flight tests the network ran at 200 Hz, feeding a standard angle controller and PID rate controller, and the drone held position for roughly ten minutes while the network was in control. They also find that memory is necessary for rate estimation, that a wide field of view including horizon-like cues gives the best absolute accuracy, and that denying the network those cues through a center crop improves relative generalization and hints at an internally learned motion model that relates visual motion to attitude.

Load-bearing premise

The paper's training labels and evaluation ground truth are the flight controller's own IMU-based estimates, so if those estimates carry systematic bias, the learned network will inherit that bias and the apparently stable hover could hide it.

Editorial extensions

If this is right

  • Closed-loop hover with the network in control works for about ten minutes, with most attitude errors within plus or minus 3 degrees and rate errors within plus or minus 18 degrees per second.
  • Recurrent memory is required: a feedforward variant's rotation-rate RMSE more than doubles, so temporal integration is what enables rate estimation.
  • Halving event-camera resolution retains most estimation quality with lower compute, while quarter resolution adds a noticeable prediction delay.
  • A network restricted to a center crop has worse absolute accuracy but better relative generalization to unseen environments, and it outperforms the full-field-of-view network on a rotation-only sequence with no horizon-like cues.
  • The full events-to-attitude pipeline runs at about 200 Hz on an onboard embedded GPU, fast enough for stable control of the tested quadrotor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of this result is to train the same network with an explicit optical-flow or contrast-maximization objective; the authors suggest this as future work, but a direct comparison would show whether forcing motion-based inference improves cross-environment generalization further.
  • If the approach transfers to smaller platforms, the IMU, its filter, and part of the sensor wiring can be removed from insect-scale robots, changing the weight and power budget; the paper points toward this but does not build it.
  • Because the ground-truth labels come from the flight controller's IMU-based estimate, the reported error distributions are relative to that estimate; an independent motion-capture reference could reveal how much of the systematic underestimation is learned bias versus label bias.
  • The paper's own data show underestimation grows at high angles and fast rates, which suggests aggressive maneuvering, beyond the hover regime tested, is an open boundary for vision-only control.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The manuscript presents a vision-only attitude and rotation-rate estimator for a quadrotor, built from a compact recurrent convolutional network that consumes 5 ms event-camera frames. The network is trained with supervised labels taken from the flight controller's IMU-based state estimate, runs on-board at about 200 Hz, and is used to replace the IMU in the inner attitude and rate control loop. The authors report real position-hold flights totaling roughly ten minutes, with error histograms and a comparison of architectures, input modalities, resolutions, and generalization to four unseen sequences including ECD poster rotation. The central claim is that this is the first vision-only approach to flight control for generic environments, capable of replacing the inertial measurement unit in a traditional flight control loop.

Significance. If the claims hold, this is an important systems demonstration: it shows stable closed-loop attitude and rate control without inertial sensors, using an event camera and a small neural network (425k parameters) entirely on-board. The paper's strengths are the real flight tests, the architectural ablations, the explicit study of memory and field of view, and the commitment to release data, code, and hardware setup instructions. The main gap is that the closed-loop evidence is limited to the full-field-of-view network, which the paper itself shows exploits horizon-like static cues, while the cue-free center-crop network is only evaluated offline. In addition, the quantitative error claims are computed against the same IMU-based estimator that provided the training labels, so the reported numbers are not absolute errors. These issues do not invalidate the hover demonstration, but they mean the advertised 'generic environments' claim and the specific error magnitudes are broader than what the evidence currently supports.

major comments (2)
  1. [Results, 'Generalization and internal motion model'; Fig. 5; abstract] The in-the-loop flight tests (Fig. 2) use the full-FoV Vision network, and the paper's own generalization analysis shows that this network exploits horizon-like static cues at the edges of the field of view (white arrows in Fig. 5). The center-crop network, which is forced to work without such cues, is evaluated only offline on the ECD poster-rotation sequence and is never placed in the control loop. The abstract's claim of 'flight control for use in generic environments' is therefore broader than the evidence: the closed-loop demonstration supports environments with horizon-like cues, and the observability discussion around Eqs. (1)-(3) indicates that the cue-free regime is the harder and less certain case. I would ask the authors to either add closed-loop tests with a cue-free (center-crop) network or revise the abstract and introduction to state the demonstrated scope precisely.
  2. [Methods, 'Training and network details'; Fig. 2; Results, 'In-the-loop flight tests'] Training labels and evaluation ground truth both come from the flight controller's IMU-based state estimate. The paper acknowledges (Results, 'In-the-loop flight tests') that these are 'not absolute ground truth' and can have bias, and Fig. 2 subtracts the biases of both estimates before computing the histograms. Consequently, the reported error magnitudes (e.g., most errors within ±3 deg and ±18 deg/s) are relative to the IMU reference, not absolute attitude and rate errors, and any systematic error in the reference can be learned by the network. This does not undermine the successful hover demonstration, but it does mean the quantitative error claims need an independent check. Since a motion-capture system is already used for plotting trajectories, I recommend reporting attitude and rate errors against motion capture for at least a validation sequence, with bias included.
minor comments (6)
  1. [Methods, 'Robot setup'] The text contains a typo: 'paramters' should be 'parameters'.
  2. [Fig. 6 caption] 'Auxilary' should be 'Auxiliary'.
  3. [Methods, 'Training and network details'] Equation (5) defines MASD using x_{i+1} and x_i without specifying whether these are predictions or targets; please clarify and use notation consistent with Eq. (4).
  4. [Fig. 2 caption] Please state explicitly in the caption that the histogram errors have the biases of both estimates subtracted, since the main text only mentions this in the body.
  5. [Table 1] No error bars or significance tests are reported; given the small differences between Vision and VisionMotor, indicate variability across runs or at least the amount of test data.
  6. [Abstract and Introduction] The 'first' claim should be qualified ('to our knowledge') and supported with a comparison to the closest prior work, since the literature on visual attitude estimation for drones is extensive.

Circularity Check

0 steps flagged · score 0.0 of 10

No construction-level circularity found; IMU-label evaluation is a stated accuracy limitation, not a circular derivation.

full rationale

The paper is an empirical supervised-learning and closed-loop control study, not an analytical derivation chain. The only input/output overlap is that the network is trained on attitude and rate labels from the flight controller's IMU-based estimator and later evaluated against the same estimator (Sec. 2: 'We consider the flight controller estimator as ground truth'; Methods: 'ground-truth attitude and rate coming from the flight controller's state estimation'). The authors explicitly flag that this is not absolute ground truth, and they subtract biases in the reported histograms. This limits the absolute accuracy of the error numbers but is not a construction-level circularity: evaluation is on held-out, in-the-loop data, and the central feasibility claim is independently grounded by the stable hover flight in which the network's own estimates, not the IMU labels, drive the inner attitude/rate control loop. The theoretical support from de Croon et al. [4] is used as background motivation for observability and is not load-bearing for the empirical result; no fitted parameter is renamed as a prediction, and no uniqueness or ansatz claim is imported from a self-citation chain. The gap between the abstract's 'generic environments' wording and the actual in-loop demonstration with a full-FoV network that exploits horizon-like cues is a generalization concern, not circularity.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper is empirical machine learning, not a derivation. The central demonstration rests on supervised labels from an IMU-based state estimator, on the visual texture and motion content of the environment, and on the cited observability result from de Croon et al. No new physical entities are introduced. The listed free parameters are training and system hyperparameters chosen by hand, not scientific constants.

free parameters (3)
  • attitude loss weight = 10
    Hand-tuned in the Methods to balance attitude and rotation-rate loss magnitudes; affects the learned policy but is not a scientific constant.
  • event frame accumulation time = 5 ms
    Chosen in the Methods as a latency and bandwidth trade-off for 200 Hz control; performance depends on this choice.
  • pixel subsampling factor = every other pixel (320x240 stream)
    Chosen to avoid bandwidth saturation; Figure 4 shows that resolution affects error and prediction delay.
assumptions (3)
  • domain assumption Attitude is partially observable from optical flow combined with a motion model, as established by de Croon et al. [4].
    Used in the Methods to motivate that visual motion can reveal attitude; the paper does not re-derive this result and the learned network may exploit other cues instead.
  • domain assumption The flight controller's IMU-based attitude and rate estimate is a reliable training label.
    Training and error evaluation rely on this estimate; the paper acknowledges that it can have bias.
  • domain assumption The environment provides sufficient visual texture and motion cues for the event camera at 5 ms frames to carry the attitude signal.
    Needed for supervision to contain usable information; generalization to textureless environments is not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of All Eyes, no IMU: Learning Flight Attitude from Vision Alone." pith.science (2026). https://pith.science/paper/GAEE4B3R

@misc{pith2026250711302,
  author       = {Pith},
  title        = {Pith review of: All Eyes, no IMU: Learning Flight Attitude from Vision Alone},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GAEE4B3R}},
  note         = {Machine review of arXiv:2507.11302}
}
read the original abstract

Vision is an essential part of attitude control for many flying animals, some of which have no dedicated sense of gravity. Flying robots, on the other hand, typically depend heavily on accelerometers and gyroscopes for attitude stabilization. In this work, we present the first vision-only approach to flight control for use in generic environments. We show that a quadrotor drone equipped with a downward-facing event camera can estimate its attitude and rotation rate from just the event stream, enabling flight control without inertial sensors. Our approach uses a small recurrent convolutional neural network trained through supervised learning. Real-world flight tests demonstrate that our combination of event camera and low-latency neural network is capable of replacing the inertial measurement unit in a traditional flight control loop. Furthermore, we investigate the network's generalization across different environments, and the impact of memory and different fields of view. While networks with memory and access to horizon-like visual cues achieve best performance, variants with a narrower field of view achieve better relative generalization. Our work showcases vision-only flight control as a promising candidate for enabling autonomous, insect-scale flying robots.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 14 canonical work pages

  1. [1]

    IEEE Transactions on Automatic Control53, 1203–1218 (2008) https://doi.org/10.1109/ TAC.2008.923738

    Mahony, R., Hamel, T., Pflimlin, J.-M.: Nonlinear Complementary Filters on the Special Orthogonal Group. IEEE Transactions on Automatic Control53, 1203–1218 (2008) https://doi.org/10.1109/ TAC.2008.923738

    work page arXiv 2008

  2. [2]

    Insect Mechanics and Control, vol

    Taylor, G.K., Krapp, H.G.: Sensory Systems and Flight Stability: What do Insects Measure and Why? In: Advances in Insect Physiology. Insect Mechanics and Control, vol. 34, pp. 231–316. Academic Press, London (2007). https://doi.org/10.1016/S0065-2806(07)34005-8

    work page doi:10.1016/s0065-2806(07)34005-8 2007

  3. [3]

    In: Frontiers in Sensing, pp

    Srinivasan, M.V., Moore, R.J.D., Thurrowgood, S., Soccol, D., Bland, D.: From biology to engineer- ing: Insect vision and applications to robotics. In: Frontiers in Sensing, pp. 19–39. Springer, Vienna (2012). https://doi.org/10.1007/978-3-211-99749-9 2

    work page doi:10.1007/978-3-211-99749-9 2012

  4. [4]

    Nature610, 485–490 (2022) https://doi.org/10.1038/s41586-022-05182-2

    de Croon, G.C.H.E., Dupeyroux, J.J.G., De Wagter, C., Chatterjee, A., Olejnik, D.A., Ruffier, F.: Accommodating unobservability to control flight attitude with optic flow. Nature610, 485–490 (2022) https://doi.org/10.1038/s41586-022-05182-2

    work page doi:10.1038/s41586-022-05182-2 2022

  5. [5]

    arXiv (2025)

    Yu, Z., Tran, J., Li, C., Weber, A., Talwekar, Y.P., Fuller, S.: TinySense: A Lighter Weight and More Power-efficient Avionics System for Flying Insect-scale Robots. arXiv (2025). https://doi.org/ 10.48550/arXiv.2501.03416

    arXiv doi:10.48550/arxiv.2501.03416 2025

  6. [6]

    Michael C., , I

    Ettinger, S.M., , N. Michael C., , I. Peter G., and Waszak, M.: Vision-guided flight stability and control for micro air vehicles. Advanced Robotics17, 617–640 (2003) https://doi.org/10.1163/ 156855303769156983

    work page 2003

  7. [7]

    Autonomous Robots29, 17–34 (2010) https://doi.org/10.1007/s10514-010-9183-2

    Mondrag´ on, I.F., Olivares-M´ endez, M.A., Campoy, P., Mart ´ ınez, C., Mejias, L.: Unmanned aerial vehicles UA Vs attitude, height, motion estimation and control using visual systems. Autonomous Robots29, 17–34 (2010) https://doi.org/10.1007/s10514-010-9183-2

    work page doi:10.1007/s10514-010-9183-2 2010

  8. [8]

    In: 2008 IEEE International Conference on Robotics and Automation, pp

    Bazin, J.-C., Kweon, I., Demonceaux, C., Vasseur, P.: UA V Attitude estimation by vanishing points in catadioptric images. In: 2008 IEEE International Conference on Robotics and Automation, pp. 2743–2749 (2008). https://doi.org/10.1109/ROBOT.2008.4543626

    work page arXiv 2008

Show all 24 references
  1. [9]

    Journal of Intelligent & Robotic Systems65, 295–308 (2012) https://doi.org/ 10.1007/s10846-011-9588-y

    Shabayek, A.E.R., Demonceaux, C., Morel, O., Fofi, D.: Vision Based UA V Attitude Estimation: Progress and Insights. Journal of Intelligent & Robotic Systems65, 295–308 (2012) https://doi.org/ 10.1007/s10846-011-9588-y

    2012 doi

  2. [10]

    IEEE Transactions on Pattern Analysis and Machine Intelligence, 1–1 (2020) https://doi.org/10.1109/TPAMI.2020

    Gallego, G., Delbruck, T., Orchard, G., Bartolozzi, C., Taba, B., Censi, A., Leutenegger, S., Davison, A., Conradt, J., Daniilidis, K., Scaramuzza, D.: Event-based Vision: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1–1 (2020) https://doi.org/10.1...

    2020 arXiv

  3. [11]

    IEEE Robotics and Automation Letters2, 632–639 (2017) https://doi.org/10.1109/LRA.2016.2647639

    Gallego, G., Scaramuzza, D.: Accurate Angular Velocity Estimation With an Event Camera. IEEE Robotics and Automation Letters2, 632–639 (2017) https://doi.org/10.1109/LRA.2016.2647639

    2017

  4. [12]

    In: 2020 IEEE International Conference on Robotics and Automation, pp

    Gehrig, M., Shrestha, S.B., Mouritzen, D., Scaramuzza, D.: Event-Based Angular Velocity Regression with Spiking Networks. In: 2020 IEEE International Conference on Robotics and Automation, pp. 4195–4202 (2020). https://doi.org/10.1109/ICRA40945.2020.9197133

    2020

  5. [13]

    arXiv (2025)

    Greatorex, H., Mastella, M., Cotteret, M., Richter, O., Chicca, E.: Event-Based Vision for Egomotion Estimation Using Precise Event Timing. arXiv (2025). https://doi.org/10.48550/arXiv.2501.11554 12

    2025 doi arXiv

  6. [14]

    IEEE Robotics and Automation Letters, 1–8 (2025) https://doi.org/10.1109/ LRA.2025.3527311

    Da Costa, D.R., Vasseur, P., Morbidi, F.: Gyrevento: Event-based Omnidirectional Visual Gyroscope in a Manhattan World. IEEE Robotics and Automation Letters, 1–8 (2025) https://doi.org/10.1109/ LRA.2025.3527311

    2025

  7. [15]

    In: Robotics: Science and Systems XX, vol

    Geles, I., Bauersfeld, L., Romero, A., Xing, J., Scaramuzza, D.: Demonstrating Agile Flight from Pixels without State Estimation. In: Robotics: Science and Systems XX, vol. 20 (2024)

    2024

  8. [16]

    In: 8th Annual Conference on Robot Learning (2024)

    Xing, J., Romero, A., Bauersfeld, L., Scaramuzza, D.: Bootstrapping Reinforcement Learning with Imitation for Vision-Based Agile Flight. In: 8th Annual Conference on Robot Learning (2024)

    2024

  9. [17]

    arXiv (2025)

    Romero, A., Shenai, A., Geles, I., Aljalbout, E., Scaramuzza, D.: Dream to Fly: Model-Based Rein- forcement Learning for Vision-Based Drone Flight. arXiv (2025). https://doi.org/10.48550/arXiv. 2501.14377

    2025 doi arXiv

  10. [18]

    Science Robotics9, 0591 (2024) https://doi.org/10.1126/scirobotics.adi0591

    Paredes-Vall´ es, F., Hagenaars, J.J., Dupeyroux, J., Stroobants, S., Xu, Y., de Croon, G.C.H.E.: Fully neuromorphic vision and control for autonomous drone flight. Science Robotics9, 0591 (2024) https://doi.org/10.1126/scirobotics.adi0591

    2024 doi

  11. [19]

    IEEE Robotics and Automation Letters10, 4858–4865 (2025) https://doi.org/10.1109/LRA.2025

    Stroobants, S., De Wagter, C., de Croon, G.C.H.E.: Neuromorphic Attitude Estimation and Control. IEEE Robotics and Automation Letters10, 4858–4865 (2025) https://doi.org/10.1109/LRA.2025. 3553418

    2025 doi

  12. [20]

    The International Journal of Robotics Research36, 142–149 (2017) https://doi.org/10.1177/0278364917691115

    Mueggler, E., Rebecq, H., Gallego, G., Delbruck, T., Scaramuzza, D.: The event-camera dataset and simulator: Event-based data for pose estimation, visual odometry, and SLAM. The International Journal of Robotics Research36, 142–149 (2017) https://doi.org/10.1177/0278364917691115

    2017 doi

  13. [21]

    In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp

    Gallego, G., Rebecq, H., Scaramuzza, D.: A Unifying Contrast Maximization Framework for Event Cameras, With Applications to Motion, Depth, and Optical Flow Estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3867–3876 (2018)

    2018

  14. [22]

    https: //doi.org/10.1038/s41467-024-47811-6

    Richter, O., Xing, Y., De Marchi, M., Nielsen, C., Katsimpris, M., Cattaneo, R., Ren, Y., Hu, Y., Liu, Q., Sheik, S., Demirci, T., Qiao, N.: Speck: A Smart Event-Based Vision Sensor with a Low Latency 327K Neuron Convolutional Neuronal Network Processing Pipeline (2024). https...

    2024 doi

  15. [23]

    In: International Conference on Learning Representations (2016)

    Clevert, D.-A., Unterthiner, T., Hochreiter, S.: Fast and Accurate Deep Network Learning by Expo- nential Linear Units (ELUs). In: International Conference on Learning Representations (2016). https://doi.org/10.48550/arXiv.1511.07289

    2016 doi arXiv

  16. [24]

    Science Robotics7, 6074 (2022) https://doi.org/10.1126/ scirobotics.abm6074 13

    Macenski, S., Foote, T., Gerkey, B., Lalancette, C., Woodall, W.: Robot Operating System 2: Design, architecture, and uses in the wild. Science Robotics7, 6074 (2022) https://doi.org/10.1126/ scirobotics.abm6074 13

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.