Image-Domain Tilt Constrained Distributed Fusion for Maneuvering UAV Tracking with Multi-Camera Electro-Optical Observations
Pith reviewed 2026-07-02 04:07 UTC · model grok-4.3
The pith
Image-domain tilt from rotorcraft images supplies acceleration constraints that reduce short-horizon UAV prediction error in multi-camera fusion.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper establishes that image-domain roll and pitch, extracted from the target image, can be introduced as acceleration-related pseudo-observations in a distributed state estimator. When these pseudo-observations are added to a model containing position, velocity, and acceleration, and when the filter is run across asynchronous multi-camera data with augmented attitude error states, short-horizon prediction error decreases.
What carries the argument
Image-domain tilt, defined as the apparent roll and pitch of the UAV in the image, used as acceleration-related pseudo-observations in the distributed filter.
If this is right
- Simulation prediction RMSE falls from 1.991 m to 0.821 m when roll and pitch observations are added.
- Cumulative prediction error drops 60.75 percent in simulation.
- Real distributed experiments show an 18.10 percent reduction in cumulative prediction error.
- The filter remains robust to intermittent detections through Mahalanobis gating and covariance widening.
Where Pith is reading between the lines
- The same tilt-to-acceleration mapping could be tested on other image-detectable targets if their orientation features can be labeled similarly.
- The auto-labeling pipeline that uses IMU synchronization to generate tilt labels may lower the cost of preparing training data for related image-based maneuver estimators.
Load-bearing premise
The apparent roll and pitch visible in the image of the rotorcraft reliably correspond to its acceleration states.
What would settle it
A side-by-side comparison of prediction error with and without the tilt pseudo-observations on the same set of aggressive maneuvers, where error does not decrease when tilt is added.
Figures
read the original abstract
Short-horizon prediction is essential for electro-optical UAV tracking, especially when the target is small, maneuvering, or intermittently observed. Image center, line-of-sight, and range measurements provide direct constraints on target position, but their constraints on acceleration are weak. As a result, prediction can lag during aggressive maneuvers. This paper proposes an image-domain tilt constrained distributed fusion method for maneuvering UAV tracking. The method uses the apparent roll and pitch of a rotorcraft target in the image as low-level maneuver cues. A weak-prior auto-labeling pipeline first generates oriented bounding box and image-domain tilt labels from synchronized video, gimbal IMU, and UAV IMU data. A YOLO-OBB detector is then trained to provide online target position and tilt measurements. The front-end Python implementation is publicly available at github.com/ShineMinxing/PythonYOLO. In the fusion stage, the UAV state is modeled by position, velocity, and acceleration. Image-domain roll and pitch are introduced as acceleration-related pseudo-observations. For distributed tracking, one mobile gimbal camera and two fixed ground cameras are fused asynchronously. Camera attitude error states are augmented into the filter to absorb extrinsic drift and cross-camera systematic inconsistency. A Mahalanobis gate with time-since-last-valid covariance widening is used to reject false detections and handle dropouts. In simulation, adding roll/pitch observations reduces the prediction RMSE from 1.991 m to 0.821 m and decreases the cumulative prediction error by 60.75\%. In real distributed experiments, a self-consistency evaluation shows an 18.10\% reduction in cumulative prediction error. The results show that image-domain tilt can provide useful acceleration constraints for robust short-horizon UAV prediction.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an image-domain tilt constrained distributed fusion method for short-horizon maneuvering UAV tracking with multi-camera electro-optical observations. It extracts apparent roll and pitch via a YOLO-OBB detector trained on auto-labeled data from synchronized video/IMU sources, introduces these as acceleration pseudo-observations in a position-velocity-acceleration state model, augments camera attitude errors for distributed asynchronous fusion across one mobile gimbal and two fixed cameras, applies Mahalanobis gating with covariance widening, and reports simulation RMSE reduction from 1.991 m to 0.821 m (60.75% cumulative error drop) plus 18.10% real-experiment cumulative error reduction.
Significance. If the tilt-to-acceleration mapping holds, the approach could strengthen acceleration constraints for prediction when direct position measurements are weak or intermittent. The public front-end Python implementation at github.com/ShineMinxing/PythonYOLO supports reproducibility.
major comments (1)
- [Fusion stage description] Fusion stage (abstract and methods): image-domain roll/pitch are introduced as acceleration-related pseudo-observations, but no derivation, validation, or error-propagation analysis of the tilt-to-acceleration mapping is supplied, nor is the setting of the pseudo-observation covariance described; this mapping is load-bearing for the reported prediction improvements.
minor comments (2)
- [Abstract] Abstract: quantitative results are stated without reference to the specific filter equations, baseline trackers, or simulation parameters used for the RMSE and cumulative-error metrics.
- [Methods] The weak-prior auto-labeling pipeline for generating tilt labels is mentioned but lacks sufficient detail on synchronization or label quality metrics.
Simulated Author's Rebuttal
We thank the referee for the constructive feedback. We address the single major comment below and will revise the manuscript to strengthen the presentation of the fusion stage.
read point-by-point responses
-
Referee: [Fusion stage description] Fusion stage (abstract and methods): image-domain roll/pitch are introduced as acceleration-related pseudo-observations, but no derivation, validation, or error-propagation analysis of the tilt-to-acceleration mapping is supplied, nor is the setting of the pseudo-observation covariance described; this mapping is load-bearing for the reported prediction improvements.
Authors: We agree that the current manuscript lacks sufficient detail on this mapping. In the revised version we will insert a new subsection under Methods that (i) derives the tilt-to-acceleration pseudo-observation relation from the rotorcraft thrust-vector geometry and small-angle approximations, (ii) validates the mapping against the synchronized IMU ground truth used in the auto-labeling pipeline, (iii) presents the first-order error-propagation analysis from image tilt uncertainty to acceleration pseudo-observation, and (iv) explicitly states how the pseudo-observation covariance is initialized from the YOLO-OBB detector covariance and then widened by a fixed factor determined in simulation. These additions will make the load-bearing assumptions transparent. revision: yes
Circularity Check
No significant circularity detected
full rationale
The paper models UAV state with position/velocity/acceleration and augments the filter with image-derived roll/pitch as acceleration pseudo-observations. Performance gains are shown by direct comparison (with vs. without tilt inputs) in simulation (RMSE drop from 1.991 m to 0.821 m) and real experiments (18.10% error reduction). No equations, self-citations, or fitted parameters reduce the claimed acceleration constraints to the inputs by construction. The derivation is self-contained against external benchmarks and uses standard filtering with added visual measurements.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Image-domain roll and pitch of the rotorcraft provide valid acceleration-related pseudo-observations
Reference graph
Works this paper leans on
-
[1]
Drone detection in airport environments: A literature review,
S. O. de Macedo, M. Caetano, and R. M. da Costa, “Drone detection in airport environments: A literature review,”Array, vol. 28, p. 100511, Dec. 2025. 22
work page 2025
-
[2]
Deep Convolutional Autoencoder for Estimation of Nonstationary Noise in Images,
S. G. Bahncmiri, M. Ponomarenko, and K. Egiazarian, “Deep Convolutional Autoencoder for Estimation of Nonstationary Noise in Images,” in2019 8th European Workshop on Visual Information Processing (EUVIP), Oct. 2019, pp. 238–243
work page 2019
-
[3]
B. Qi, S. Zhang, W. Chen, Y . Fu, and B. Ren, “Adaptive Kalman Filters Based on Elliptically Contoured Distributions for Heavy-Tailed and Nonstationary Measurement Noises,”IEEE Transactions on Instrumentation and Measurement, vol. 74, pp. 1–15, 2025
work page 2025
-
[4]
Y . Chen, Y . Cai, Y . Deng, and J. Liu, “Stochastic Event-Triggered Robust Tracking Algorithm Under Nonstationary Heavy- Tailed Noise and Packet Dropouts,”IEEE Transactions on Automation Science and Engineering, vol. 23, pp. 4428–4441, 2026
work page 2026
-
[5]
Deep learning based image classification for embedded devices: A systematic review,
L. F. R. Moreira, R. Moreira, B. A. N. Travencolo, and A. R. Backes, “Deep learning based image classification for embedded devices: A systematic review,”Neurocomputing, vol. 623, p. 129402, Mar. 2025
work page 2025
-
[6]
Review of Event Camera-Based Target Detection and Tracking Algorithms,
J. Qiu, Y . Zhang, Y . Fang, P. Li, and K. Zheng, “Review of Event Camera-Based Target Detection and Tracking Algorithms,” Laser & Optoelectronics Progress, vol. 62, no. 4, p. 0400004, Feb. 2025
work page 2025
-
[7]
Optimizing Real-Time Image Processing in Augmented Reality with Low-Latency Edge AI,
Gokila Deepa G, Gomathi S, Aadhitya S, Sujitha R, Sundarrajan M, and M. D. Choudhry, “Optimizing Real-Time Image Processing in Augmented Reality with Low-Latency Edge AI,” in2024 2nd International Conference on Signal Processing, Communication, Power and Embedded System (SCOPES), Dec. 2024, pp. 1–6
work page 2024
-
[8]
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,
S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137–1149, Jun. 2017
work page 2017
-
[9]
Detection and Recognition of Drones using Deep Convolution Neural Networks,
V . S. Karthikeya Nalam, V . S. Amar Koushik Tanniru, A. Posani, and M. Suneetha, “Detection and Recognition of Drones using Deep Convolution Neural Networks,” in2022 IEEE 6th Conference on Information and Communication Technology (CICT), Nov. 2022, pp. 1–5
work page 2022
-
[10]
Performance Comparison of YOLO Algorithms in Drone Detection,
R. Jain, S. Shrivastav, S. Kakde, and R. Raut, “Performance Comparison of YOLO Algorithms in Drone Detection,” in 2024 IEEE International Conference on Smart Power Control and Renewable Energy (ICSPCRE), Jul. 2024, pp. 1–6
work page 2024
-
[11]
Pre-trained Deep Learning Networks for Advanced Visible Imagery Drone Detection and Recognition,
H. J. Al dawasari, M. Bilal, M. Moinuddin, K. Arshad, and K. Assaleh, “Pre-trained Deep Learning Networks for Advanced Visible Imagery Drone Detection and Recognition,” in2023 IEEE 15th International Conference on Computational Intelligence and Communication Networks (CICN), Dec. 2023, pp. 316–320
work page 2023
-
[12]
Small Object Detection and Classification Using YOLO on Multi-Dataset Drone Imagery,
V . Taware, M. Bhandare, S. Bhirud, S. Sawant, and A. Joshi, “Small Object Detection and Classification Using YOLO on Multi-Dataset Drone Imagery,” in2025 IEEE Pune Section International Conference (PuneCon), Dec. 2025, pp. 1–6
work page 2025
-
[13]
Light-YOLOv5: A Lightweight Drone Detector for Resource-Constrained Cameras,
J. Han, R. Cao, A. Brighente, and M. Conti, “Light-YOLOv5: A Lightweight Drone Detector for Resource-Constrained Cameras,”IEEE Internet of Things Journal, vol. 11, no. 6, pp. 11 046–11 057, Mar. 2024
work page 2024
-
[14]
Experimental Analysis of Fine-Tuned Drone Detection YOLO Models,
S. A. Dogan, J. Walatkiewicz, S. Tout, O. Darwish, and O. Spantidi, “Experimental Analysis of Fine-Tuned Drone Detection YOLO Models,” in2025 1st International Conference on Secure IoT, Assured and Trusted Computing (SATC), Feb. 2025, pp. 1–5
work page 2025
-
[15]
An Ensemble-Based IoT-Enabled Drones Detection Scheme for a Safe Community,
J. Singh, K. Sharma, M. Wazid, A. K. Das, and A. V . Vasilakos, “An Ensemble-Based IoT-Enabled Drones Detection Scheme for a Safe Community,”IEEE Open Journal of the Communications Society, vol. 4, pp. 1946–1956, 2023
work page 1946
-
[16]
Real-Time UA V Detection Using an Enhanced YOLO v8 Model,
A. F. Serageldin, H. A. Elsayed, and L. Abdel-Hamid, “Real-Time UA V Detection Using an Enhanced YOLO v8 Model,” in2025 42nd National Radio Science Conference (NRSC), vol. 1, May 2025, pp. 193–201
work page 2025
-
[17]
The Discovery of Ceres: How Gauss Became Famous,
D. Teets and K. Whitehead, “The Discovery of Ceres: How Gauss Became Famous,”Mathematics Magazine, vol. 72, no. 2, pp. 83–93, Apr. 1999
work page 1999
-
[18]
The Linear Predictor for a Single Time Series,
N. Wiener, “The Linear Predictor for a Single Time Series,” inExtrapolation, Interpolation, and Smoothing of Stationary Time Series: With Engineering Applications. MIT Press, 1964, pp. 56–80. 23
work page 1964
-
[19]
A new approach to linear filtering and prediction problems,
R. E. Kalman, “A new approach to linear filtering and prediction problems,”Journal of Basic Engineering, vol. 82, no. 1, pp. 35–45, Mar. 1960
work page 1960
-
[20]
Y . Bar-Shalom, X. R. Li, and T. Kirubarajan,Estimation with Applications to Tracking and Navigation: Theory Algorithms and Software. John Wiley & Sons, 2001
work page 2001
-
[21]
A new extension of the kalman filter to nonlinear systems,
S. J. Julier and J. K. Uhlmann, “A new extension of the kalman filter to nonlinear systems,” inSignal Processing, Sensor Fusion, and Target Recognition Vi, vol. 3068. Bellingham: Spie - Int Soc Optical Engineering, 1997, pp. 182–193
work page 1997
-
[22]
The unscented kalman filter for nonlinear estimation,
E. Wan and R. Van Der Merwe, “The unscented kalman filter for nonlinear estimation,” inProceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373), Oct. 2000, pp. 153–158
work page 2000
-
[23]
The Unscented Particle Filter,
R. van der Merwe, A. Doucet, N. de Freitas, and E. Wan, “The Unscented Particle Filter,” inAdvances in Neural Information Processing Systems, vol. 13. MIT Press, 2000
work page 2000
-
[24]
Unscented filtering and nonlinear estimation,
S. Julier and J. Uhlmann, “Unscented filtering and nonlinear estimation,”Proceedings of the IEEE, vol. 92, no. 3, pp. 401–422, Mar. 2004
work page 2004
-
[25]
I. Arasaratnam and S. Haykin, “Cubature kalman filters,”IEEE Transactions on Automatic Control, vol. 54, no. 6, pp. 1254–1269, Jun. 2009
work page 2009
-
[26]
Novel-Approach to Nonlinear Non-Gaussian Bayesian State Estimation,
N. Gordon, D. Salmond, and A. Smith, “Novel-Approach to Nonlinear Non-Gaussian Bayesian State Estimation,”IEE PROCEEDINGS-F RADAR AND SIGNAL PROCESSING, vol. 140, no. 2, pp. 107–113, Apr. 1993
work page 1993
-
[27]
Particle Filters for Mobile Robot Localization,
D. Fox, S. Thrun, W. Burgard, and F. Dellaert, “Particle Filters for Mobile Robot Localization,” inSequential Monte Carlo Methods in Practice, ser. Statistics for Engineering and Information Science, A. Doucet, N. de Freitas, and N. Gordon, Eds. New York, NY: Springer, 2001, pp. 401–428
work page 2001
-
[28]
The Interacting Multiple Model Algorithm for Systems with Markovian Switching Coefficients,
H. Blom and Y . Barshalom, “The Interacting Multiple Model Algorithm for Systems with Markovian Switching Coefficients,”IEEE TRANSACTIONS ON AUTOMATIC CONTROL, vol. 33, no. 8, pp. 780–783, Aug. 1988
work page 1988
-
[29]
New interacting multiple model algorithms for the tracking of the manoeuvring target,
X. Fu, Y . Jia, J. Du, and F. Yu, “New interacting multiple model algorithms for the tracking of the manoeuvring target,” IET CONTROL THEORY AND APPLICATIONS, vol. 4, no. 10, pp. 2184–2194, Oct. 2010
work page 2010
-
[30]
Exploration of adaptive filters for target tracking in the presence of model uncertainty,
T. Q. Truong, “Exploration of adaptive filters for target tracking in the presence of model uncertainty,” in2010 Sixth International Conference on Intelligent Sensors, Sensor Networks and Information Processing. IEEE, Dec. 2010, pp. 1–6
work page 2010
-
[31]
Robust Estimation of a Location Parameter,
P. J. Huber, “Robust Estimation of a Location Parameter,” inBreakthroughs in Statistics, S. Kotz and N. L. Johnson, Eds. New York, NY: Springer New York, 1992, pp. 492–518
work page 1992
-
[32]
G. Zames, “Feedback and optimal sensitivity: Model reference transformations, multiplicative seminorms, and approximate inverses,”IEEE Transactions on Automatic Control, vol. 26, no. 2, pp. 301–320, Apr. 1981
work page 1981
-
[33]
A framework for state-space estimation with uncertain models,
A. H. Sayed, “A framework for state-space estimation with uncertain models,”IEEE TRANSACTIONS ON AUTOMATIC CONTROL, vol. 46, no. 7, pp. 998–1013, Jul. 2001
work page 2001
-
[34]
S. S. Ge and J. Wang, “Robust adaptive tracking for time-varying uncertain nonlinear systems with unknown control coefficients,”IEEE TRANSACTIONS ON AUTOMATIC CONTROL, vol. 48, no. 8, pp. 1463–1469, Aug. 2003
work page 2003
-
[35]
Robust Gaussian-sum ensemble Kalman filter and its application in bearings-only tracking,
H.-n. Jiang and Y .-l. Cai, “Robust Gaussian-sum ensemble Kalman filter and its application in bearings-only tracking,” Kongzhi Lilun yu Yingyong = Control Theory & Applications, vol. 35, no. 2, Feb. 2018
work page 2018
-
[36]
Robust cubature Kalman filter target tracking algorithm based on genernalized M-estiamtion,
H. Wu, S.-X. Chen, B.-F. Yang, and K. Chen, “Robust cubature Kalman filter target tracking algorithm based on genernalized M-estiamtion,”ACTA PHYSICA SINICA, vol. 64, no. 21, p. 218401, Nov. 2015
work page 2015
-
[37]
Combination of IMM Algorithm and ASTRWCKF for Maneuvering Target Tracking,
J. Ma and X. Guo, “Combination of IMM Algorithm and ASTRWCKF for Maneuvering Target Tracking,”IEEE Access, vol. 8, pp. 143 095–143 103, 2020. 24
work page 2020
-
[38]
Adaptive Weighted Ridge Regression Estimator for Time-Varying Sensitivity Identification,
Z. Tang, Y . Liu, T. Liu, X. Xu, and J. Liu, “Adaptive Weighted Ridge Regression Estimator for Time-Varying Sensitivity Identification,”IEEE Transactions on Power Systems, vol. 39, no. 1, pp. 2377–2380, Jan. 2024
work page 2024
-
[39]
Research on the anti-UA V distributed system for airports : YOLOv5-based auto- targeting device,
R. Liu, Y . Xiao, Z. Li, and H. Cao, “Research on the anti-UA V distributed system for airports : YOLOv5-based auto- targeting device,” in2022 3rd International Conference on Computer Vision, Image and Deep Learning & International Conference on Computer Engineering and Applications (CVIDL & ICCEA), May 2022, pp. 864–867
work page 2022
-
[40]
MIT Cheetah 3: Design and Control of a Robust, Dynamic Quadruped Robot,
G. Bledt, M. J. Powell, B. Katz, J. Di Carlo, P. M. Wensing, and S. Kim, “MIT Cheetah 3: Design and Control of a Robust, Dynamic Quadruped Robot,” in2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Oct. 2018, pp. 2245–2252
work page 2018
-
[41]
MPC-based Controller with Terrain Insight for Dynamic Legged Locomotion,
O. Villarreal, V . Barasuol, P. M. Wensing, D. G. Caldwell, and C. Semini, “MPC-based Controller with Terrain Insight for Dynamic Legged Locomotion,” in2020 IEEE International Conference on Robotics and Automation (ICRA), May 2020, pp. 2436–2442
work page 2020
-
[42]
Perceptive Locomotion through Nonlinear Model Predictive Control,
R. Grandia, F. Jenelten, S. Yang, F. Farshidian, and M. Hutter, “Perceptive Locomotion through Nonlinear Model Predictive Control,” Aug. 2022
work page 2022
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.