Terminal Constraint Model Predictive Control for Image-Based Visual Servoing of UAVs with Kalman Filter-Based Moment Loss Compensation
Pith reviewed 2026-05-22 05:41 UTC · model grok-4.3
The pith
Terminal constraints and Kalman prediction stabilize image-based visual servoing for UAVs under constraints and feature loss.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The TC-MPC explicitly incorporates terminal-state constraints and a terminal cost into the IBVS error dynamics, ensuring recursive feasibility, improved convergence behavior, and closed-loop stability under control and state constraints. In parallel, the Kalman filter predicts the temporal evolution of image moments during short-term visual degradation, enabling the controller to preserve control continuity when moment measurements are partially unavailable.
What carries the argument
Terminal-constraint model predictive control (TC-MPC) applied to IBVS error dynamics, combined with Kalman filter-based prediction of image moments during loss.
If this is right
- Recursive feasibility of the MPC optimization is maintained at every step.
- Closed-loop stability is achieved despite input and state constraints.
- Convergence of image errors improves under the terminal cost.
- Control actions remain continuous even when visual moments are temporarily unavailable.
Where Pith is reading between the lines
- This framework could be adapted to ground robots or manipulators facing similar visual servoing challenges with constraints.
- Extending the prediction horizon or filter design might handle longer visual outages in more complex environments.
- Real-world deployment could benefit from integrating additional sensor fusion to reduce reliance on the KF assumption.
Load-bearing premise
The Kalman filter accurately models and predicts the short-term changes in image moments even when direct measurements are unavailable due to degradation.
What would settle it
A flight test in which image moments are artificially blocked for several seconds and the UAV's tracking error either stays bounded or grows beyond acceptable limits would confirm or refute the stability and continuity claims.
Figures
read the original abstract
Image-Based Visual Servoing (IBVS) provides an efficient vision-guided control paradigm for unmanned aerial vehicles (UAVs) by directly regulating image-space errors. However, conventional IBVS controllers are vulnerable to two critical issues: loss of closed-loop stability near the target due to input and state constraints, and control failure caused by intermittent loss of moment-based visual features under aggressive motion. To address these challenges, this paper proposes a terminal-constraint model predictive control (TC-MPC) framework for IBVS, integrated with a Kalman filter (KF)-based state-prediction mechanism. The TC-MPC explicitly incorporates terminal-state constraints and a terminal cost into the IBVS error dynamics, ensuring recursive feasibility, improved convergence behavior, and closed-loop stability under control and state constraints. In parallel, the Kalman filter predicts the temporal evolution of image moments during short-term visual degradation, enabling the controller to preserve control continuity when moment measurements are partially unavailable. The proposed approach is validated through real-time UAV visual servoing experiments.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a terminal-constraint model predictive control (TC-MPC) framework for image-based visual servoing (IBVS) of UAVs. Terminal-state constraints and a terminal cost are incorporated into the IBVS error dynamics to ensure recursive feasibility, improved convergence, and closed-loop stability under control and state constraints. A Kalman filter predicts the evolution of image moments during short-term visual degradation to maintain control continuity. The approach is validated through real-time UAV visual servoing experiments.
Significance. If the invariance and stability arguments hold, the work would offer a practical method for maintaining stable IBVS performance under input/state constraints and intermittent visual features, which is relevant for UAV applications in real-world conditions. The real-time experimental validation provides concrete evidence of feasibility in hardware.
major comments (2)
- [TC-MPC framework description] The central claim that terminal-state constraints plus terminal cost ensure recursive feasibility and closed-loop stability (as stated in the abstract) requires positive invariance of the terminal set under the closed-loop map. The manuscript does not show that this invariance holds for the nonlinear IBVS image-moment dynamics when the state is replaced by Kalman-filter predictions that contain error; without this step the standard MPC recursive-feasibility argument does not go through.
- [Kalman filter integration] The weakest assumption—that the Kalman filter can accurately predict image-moment evolution during visual degradation—is used to claim control continuity, yet no error bounds, covariance analysis, or sensitivity study of the closed-loop system to KF prediction error is supplied. This directly affects the stability guarantee when measurements are unavailable.
minor comments (1)
- [Notation and problem formulation] The abstract and method description would benefit from explicit definitions of the IBVS error vector and the precise form of the terminal cost to improve readability.
Simulated Author's Rebuttal
We thank the referee for the constructive and insightful comments on our manuscript. We have addressed each major point below with clarifications and revisions to strengthen the theoretical foundations and practical analysis.
read point-by-point responses
-
Referee: The central claim that terminal-state constraints plus terminal cost ensure recursive feasibility and closed-loop stability (as stated in the abstract) requires positive invariance of the terminal set under the closed-loop map. The manuscript does not show that this invariance holds for the nonlinear IBVS image-moment dynamics when the state is replaced by Kalman-filter predictions that contain error; without this step the standard MPC recursive-feasibility argument does not go through.
Authors: We appreciate this observation on the invariance requirement. In the revised version, we have added a new subsection (Section IV-C) and Appendix B that explicitly proves positive invariance of the terminal set for the nonlinear IBVS dynamics. The proof accounts for bounded Kalman filter prediction errors by showing that the terminal set is robustly invariant under a perturbed closed-loop map when the KF error remains within the filter's covariance bounds (derived from the Riccati equation). This restores the standard recursive feasibility argument while preserving the stability guarantees. revision: yes
-
Referee: The weakest assumption—that the Kalman filter can accurately predict image-moment evolution during visual degradation—is used to claim control continuity, yet no error bounds, covariance analysis, or sensitivity study of the closed-loop system to KF prediction error is supplied. This directly affects the stability guarantee when measurements are unavailable.
Authors: We agree that explicit treatment of KF prediction errors is essential for the stability claims. The revised manuscript now includes a dedicated analysis in Section V-B: we derive L2 error bounds on the predicted image moments using the KF covariance propagation, and we present a sensitivity study (both in simulation and with experimental data) showing how closed-loop stability margins degrade gracefully with increasing prediction error. These additions quantify the conditions under which control continuity is maintained during short visual losses. revision: yes
Circularity Check
No circularity: standard MPC terminal constraints and KF prediction applied to IBVS without reduction to inputs by construction
full rationale
The paper's central derivation applies established terminal-constraint MPC stability arguments (recursive feasibility via terminal set invariance and terminal cost) and Kalman filter state prediction to the IBVS error dynamics. No step reduces a claimed prediction or stability result to a fitted parameter, self-defined quantity, or unverified self-citation chain. The abstract and described framework treat the IBVS dynamics as the plant to which known MPC and KF methods are applied, with validation via experiments rather than internal redefinition. This is the common case of an honest application paper whose claims remain externally falsifiable against standard MPC theory.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption The IBVS error dynamics admit a predictive model suitable for terminal-constraint MPC.
Lean theorems connected to this paper
-
IndisputableMonolith/Foundation/RealityFromDistinction.leanreality_from_one_distinction unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
The TC-MPC explicitly incorporates terminal-state constraints and a terminal cost into the IBVS error dynamics, ensuring recursive feasibility...
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
e(k+1)=A e(k)+B u(k) with A=I4, B=Ts L(t); cost J with Q,R,P terminal
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Collision avoidance of multi unmanned aerial vehicles: A re- view,
S. Huang, R.S. H. Teo, and K. K. Tan, “Collision avoidance of multi unmanned aerial vehicles: A re- view,”
-
[2]
GO- Flock: Goal-Oriented Flocking in 3D Unknown En- vironments with Depth Maps,
Y . R. Tan, W. Liu, W. L. Leong, J. G. Z. Tan, W. W. H. Yong, S. Foong, F. Shi, and R. S. H. Teo, “GO- Flock: Goal-Oriented Flocking in 3D Unknown En- vironments with Depth Maps,” inProceedings of the IEEE/RSJ International Conference on Intelli- gent Robots and Systems (IROS), 2025, pp. 2598– 2605
work page 2025
-
[3]
EGO-Planner: An ESDF-Free Gradient-Based Lo- cal Planner for Quadrotors,
X. Zhou, Z. Wang, H. Ye, C. Xu, and F. Gao, “EGO-Planner: An ESDF-Free Gradient-Based Lo- cal Planner for Quadrotors,”IEEE Robotics and Au- tomation Letters, vol. 6, no. 2, pp. 478–485, 2020
work page 2020
-
[4]
STAR: Swarm Technology for Aerial Robotics Re- search,
J. Chiun, Y . R. Tan, Y . Cao, J. Tan, and G. Sartoretti, “STAR: Swarm Technology for Aerial Robotics Re- search,” inProceedings of the 24th International Conference on Control, Automation and Systems (ICCAS), 2024, pp. 141–146
work page 2024
-
[5]
Optical-Flow Based Strategies for Landing VTOL UA Vs in Cluttered Environments,
L. Rosa, T. Hamel, R. Mahony, and C. Samson, “Optical-Flow Based Strategies for Landing VTOL UA Vs in Cluttered Environments,”IF AC Proceed- ings V olumes, vol. 47, no. 3, pp. 3176–3183, 2014
work page 2014
-
[6]
Shape Visual Servoing of a Cable Suspended Between Two Drones,
L. Smolentsev, A. Krupa, and F. Chaumette, “Shape Visual Servoing of a Cable Suspended Between Two Drones,”IEEE Robotics and Automation Let- ters, vol. 9, no. 12, pp. 11473–11480, 2024
work page 2024
-
[7]
Vision-guided servoing with feature-based trajectory generation (for robots),
J. T. Feddema and O. R. Mitchell, “Vision-guided servoing with feature-based trajectory generation (for robots),” IEEE Transactions on Robotics and Automation, vol. 5, no. 5, pp. 691–700, 1989
work page 1989
-
[8]
Dynamic sensor-based control of robots with visual feed- back,
L. Weiss, A. Sanderson, and C. Neuman, “Dynamic sensor-based control of robots with visual feed- back,” IEEE Journal on Robotics and Automation, vol. 3, no. 5, pp. 404–417, 1987
work page 1987
-
[9]
Fixed-time fractional- order sliding mode control for image-based visual servoing of hexarotor,
A. K. Kamath, N. T. Chan, and M. Feroskhan, “Fixed-time fractional- order sliding mode control for image-based visual servoing of hexarotor,” in 2024 International Conference on Unmanned Air- craft Systems(ICUAS). IEEE, 2024, pp. 512–521
work page 2024
-
[10]
A new parti- tioned approach to image-based visual servo con- trol,
P. I. Corke and S. A. Hutchinson, “A new parti- tioned approach to image-based visual servo con- trol,” IEEE Transactions on Robotics and Automa- tion, vol. 17, no. 4, pp. 507–515, 2001
work page 2001
-
[11]
Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach,
T. Hamel and R. Mahony, “Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach,” IEEE Trans. Robot. Au- tom., vol. 18, no. 2, pp. 187–198, 2002
work page 2002
-
[12]
Perception-aware image-based visual servoing of aggressive quadrotor uavs,
C. Qin, Q. Yu, H. H. Go, and H. H.-T. Liu, “Perception-aware image-based visual servoing of aggressive quadrotor uavs,” IEEE/ASME Trans- actions on Mechatronics, vol. 28, no. 4, pp. 2020–2028, 2023
work page 2020
-
[13]
Point-based and region-based image moments for visual servoing of planar objects,
O. Tahri and F. Chaumette, “Point-based and region-based image moments for visual servoing of planar objects,” IEEE Transactions on Robotics, vol. 21, no. 6, pp. 1116–1127, 2005
work page 2005
-
[14]
Ibvs based on adaptive sliding mode con- trol for a quadrotor target tracking under perturba- tions,
A. Miranda-Moya, H. Castaneda, J. Gordillo, and H. Wang, “Ibvs based on adaptive sliding mode con- trol for a quadrotor target tracking under perturba- tions,” Mechatronics, vol. 88, p. 102909, 2022
work page 2022
-
[15]
Experimental validation of dynamic visual servo- ing for a quadrotor using a virtual camera,
G. Fink, H. Xie, A. F. Lynch, and M. Jagersand, “Experimental validation of dynamic visual servo- ing for a quadrotor using a virtual camera,” in 2015 International conference on unmanned aircraft sys- tems (ICUAS). IEEE, 2015, pp. 1231–1240
work page 2015
-
[16]
GPS denied IBVS-based navigation and collision avoidance of UA V using a low-cost RGB camera,
X. Wang, Y . R. Tan, W. Leong, S. Huang, R. Teo, and C. Xiang, “GPS denied IBVS-based navigation and collision avoidance of UA V using a low-cost RGB camera,” arXiv preprint arXiv:2509.17435, 2025
-
[17]
Sampling-based path planning and model predictive image-based visual servoing for quadrotor uavs,
M. Li, H.-N. Wu, and Z.-Y . Liu, “Sampling-based path planning and model predictive image-based visual servoing for quadrotor uavs,” in 2017 Chi- nese Automation Congress (CAC). IEEE, 2017, pp. 6237– 6242
work page 2017
-
[18]
Quadrotor Going Through a Window and Landing: An Image-Based Visual Servo Con- trol Approach,
Z. Tang, R. Cunha, D. Cabecinhas, T. Hamel, and C. Silvestre, “Quadrotor Going Through a Window and Landing: An Image-Based Visual Servo Con- trol Approach,”Control Engineering Practice, vol. 112, p. 104827, 2021
work page 2021
-
[19]
Gurobi Optimizer Reference Manual,
Gurobi Optimization, LLC, “Gurobi Optimizer Reference Manual,” 2025. [Online]. Available: https://www.gurobi.com
work page 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.