pith. sign in

arxiv: 2605.22443 · v1 · pith:FNVFV5GOnew · submitted 2026-05-21 · 💻 cs.RO

Terminal Constraint Model Predictive Control for Image-Based Visual Servoing of UAVs with Kalman Filter-Based Moment Loss Compensation

Pith reviewed 2026-05-22 05:41 UTC · model grok-4.3

classification 💻 cs.RO
keywords image-based visual servoingmodel predictive controlterminal constraintsKalman filterUAV controlvisual feature loss compensationstability guarantees
0
0 comments X

The pith

Terminal constraints and Kalman prediction stabilize image-based visual servoing for UAVs under constraints and feature loss.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper develops a terminal-constraint model predictive control approach tailored for image-based visual servoing in unmanned aerial vehicles. By embedding terminal state constraints and a terminal cost directly into the visual error dynamics, the method aims to secure recursive feasibility and closed-loop stability even when actuator and state limits are active. A Kalman filter runs alongside to forecast the behavior of image moments during brief periods when visual data drops out, allowing the controller to continue without interruption. The result is a system that avoids the instability common in standard IBVS setups near targets or during fast maneuvers. If the approach holds, UAVs gain reliable vision-guided flight in real conditions where visual contact is not continuous.

Core claim

The TC-MPC explicitly incorporates terminal-state constraints and a terminal cost into the IBVS error dynamics, ensuring recursive feasibility, improved convergence behavior, and closed-loop stability under control and state constraints. In parallel, the Kalman filter predicts the temporal evolution of image moments during short-term visual degradation, enabling the controller to preserve control continuity when moment measurements are partially unavailable.

What carries the argument

Terminal-constraint model predictive control (TC-MPC) applied to IBVS error dynamics, combined with Kalman filter-based prediction of image moments during loss.

If this is right

  • Recursive feasibility of the MPC optimization is maintained at every step.
  • Closed-loop stability is achieved despite input and state constraints.
  • Convergence of image errors improves under the terminal cost.
  • Control actions remain continuous even when visual moments are temporarily unavailable.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This framework could be adapted to ground robots or manipulators facing similar visual servoing challenges with constraints.
  • Extending the prediction horizon or filter design might handle longer visual outages in more complex environments.
  • Real-world deployment could benefit from integrating additional sensor fusion to reduce reliance on the KF assumption.

Load-bearing premise

The Kalman filter accurately models and predicts the short-term changes in image moments even when direct measurements are unavailable due to degradation.

What would settle it

A flight test in which image moments are artificially blocked for several seconds and the UAV's tracking error either stays bounded or grows beyond acceptable limits would confirm or refute the stability and continuity claims.

Figures

Figures reproduced from arXiv: 2605.22443 by C. Xiang, S. H. R. Teo, S. Huang, W. L. W. Leong, X. Wang, Y. Cao, Y. R. Tan.

Figure 1
Figure 1. Figure 1: A UAV workflow using MPC-based control to track AprilTags and navigate through gates. The drone is high￾lighted by the red box. The inset images in (b), (c), and (d) show the onboard camera view, where the drone detects and tracks the AprilTag to reach the desired pose. (a) Pure IBVS (b) MPC without constraints (c) MPC with input constraint (d) MPC with state and input constraints [PITH_FULL_IMAGE:figures… view at source ↗
Figure 2
Figure 2. Figure 2: Comparison of UAV velocity tracking performance under different MPC configurations. The figures show current velocities and velocity commands plotted against time. Figure (a) shows an example of oscillatory motion encountered during tracking as marked out by the black box. for quadrotor UAV control. The proposed IBVS formu￾lation is based on image moment features, which alle￾viates the singularity issues c… view at source ↗
Figure 3
Figure 3. Figure 3: IBVS control using MPC with the proposed Kalman filter, demonstrating reduced oscillatory behav￾ior during visual tracking. performance compared with [17]. Future work will focus on extending the framework to real gate detection without the use of AprilTags, as well as investigating high-speed and agile UAV maneuvers un￾der the proposed visual servoing framework. REFERENCES [1] S. Huang, R.S. H. Teo, and K… view at source ↗
read the original abstract

Image-Based Visual Servoing (IBVS) provides an efficient vision-guided control paradigm for unmanned aerial vehicles (UAVs) by directly regulating image-space errors. However, conventional IBVS controllers are vulnerable to two critical issues: loss of closed-loop stability near the target due to input and state constraints, and control failure caused by intermittent loss of moment-based visual features under aggressive motion. To address these challenges, this paper proposes a terminal-constraint model predictive control (TC-MPC) framework for IBVS, integrated with a Kalman filter (KF)-based state-prediction mechanism. The TC-MPC explicitly incorporates terminal-state constraints and a terminal cost into the IBVS error dynamics, ensuring recursive feasibility, improved convergence behavior, and closed-loop stability under control and state constraints. In parallel, the Kalman filter predicts the temporal evolution of image moments during short-term visual degradation, enabling the controller to preserve control continuity when moment measurements are partially unavailable. The proposed approach is validated through real-time UAV visual servoing experiments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper proposes a terminal-constraint model predictive control (TC-MPC) framework for image-based visual servoing (IBVS) of UAVs. Terminal-state constraints and a terminal cost are incorporated into the IBVS error dynamics to ensure recursive feasibility, improved convergence, and closed-loop stability under control and state constraints. A Kalman filter predicts the evolution of image moments during short-term visual degradation to maintain control continuity. The approach is validated through real-time UAV visual servoing experiments.

Significance. If the invariance and stability arguments hold, the work would offer a practical method for maintaining stable IBVS performance under input/state constraints and intermittent visual features, which is relevant for UAV applications in real-world conditions. The real-time experimental validation provides concrete evidence of feasibility in hardware.

major comments (2)
  1. [TC-MPC framework description] The central claim that terminal-state constraints plus terminal cost ensure recursive feasibility and closed-loop stability (as stated in the abstract) requires positive invariance of the terminal set under the closed-loop map. The manuscript does not show that this invariance holds for the nonlinear IBVS image-moment dynamics when the state is replaced by Kalman-filter predictions that contain error; without this step the standard MPC recursive-feasibility argument does not go through.
  2. [Kalman filter integration] The weakest assumption—that the Kalman filter can accurately predict image-moment evolution during visual degradation—is used to claim control continuity, yet no error bounds, covariance analysis, or sensitivity study of the closed-loop system to KF prediction error is supplied. This directly affects the stability guarantee when measurements are unavailable.
minor comments (1)
  1. [Notation and problem formulation] The abstract and method description would benefit from explicit definitions of the IBVS error vector and the precise form of the terminal cost to improve readability.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and insightful comments on our manuscript. We have addressed each major point below with clarifications and revisions to strengthen the theoretical foundations and practical analysis.

read point-by-point responses
  1. Referee: The central claim that terminal-state constraints plus terminal cost ensure recursive feasibility and closed-loop stability (as stated in the abstract) requires positive invariance of the terminal set under the closed-loop map. The manuscript does not show that this invariance holds for the nonlinear IBVS image-moment dynamics when the state is replaced by Kalman-filter predictions that contain error; without this step the standard MPC recursive-feasibility argument does not go through.

    Authors: We appreciate this observation on the invariance requirement. In the revised version, we have added a new subsection (Section IV-C) and Appendix B that explicitly proves positive invariance of the terminal set for the nonlinear IBVS dynamics. The proof accounts for bounded Kalman filter prediction errors by showing that the terminal set is robustly invariant under a perturbed closed-loop map when the KF error remains within the filter's covariance bounds (derived from the Riccati equation). This restores the standard recursive feasibility argument while preserving the stability guarantees. revision: yes

  2. Referee: The weakest assumption—that the Kalman filter can accurately predict image-moment evolution during visual degradation—is used to claim control continuity, yet no error bounds, covariance analysis, or sensitivity study of the closed-loop system to KF prediction error is supplied. This directly affects the stability guarantee when measurements are unavailable.

    Authors: We agree that explicit treatment of KF prediction errors is essential for the stability claims. The revised manuscript now includes a dedicated analysis in Section V-B: we derive L2 error bounds on the predicted image moments using the KF covariance propagation, and we present a sensitivity study (both in simulation and with experimental data) showing how closed-loop stability margins degrade gracefully with increasing prediction error. These additions quantify the conditions under which control continuity is maintained during short visual losses. revision: yes

Circularity Check

0 steps flagged

No circularity: standard MPC terminal constraints and KF prediction applied to IBVS without reduction to inputs by construction

full rationale

The paper's central derivation applies established terminal-constraint MPC stability arguments (recursive feasibility via terminal set invariance and terminal cost) and Kalman filter state prediction to the IBVS error dynamics. No step reduces a claimed prediction or stability result to a fitted parameter, self-defined quantity, or unverified self-citation chain. The abstract and described framework treat the IBVS dynamics as the plant to which known MPC and KF methods are applied, with validation via experiments rather than internal redefinition. This is the common case of an honest application paper whose claims remain externally falsifiable against standard MPC theory.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claims rest on standard control-theory modeling assumptions and the predictive accuracy of the Kalman filter for image moments; no new entities are postulated and no free parameters are explicitly fitted in the abstract.

axioms (1)
  • domain assumption The IBVS error dynamics admit a predictive model suitable for terminal-constraint MPC.
    Invoked when terminal-state constraints and terminal cost are added to guarantee recursive feasibility and stability.

pith-pipeline@v0.9.0 · 5740 in / 1301 out tokens · 65141 ms · 2026-05-22T05:41:09.335909+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages

  1. [1]

    Collision avoidance of multi unmanned aerial vehicles: A re- view,

    S. Huang, R.S. H. Teo, and K. K. Tan, “Collision avoidance of multi unmanned aerial vehicles: A re- view,”

  2. [2]

    GO- Flock: Goal-Oriented Flocking in 3D Unknown En- vironments with Depth Maps,

    Y . R. Tan, W. Liu, W. L. Leong, J. G. Z. Tan, W. W. H. Yong, S. Foong, F. Shi, and R. S. H. Teo, “GO- Flock: Goal-Oriented Flocking in 3D Unknown En- vironments with Depth Maps,” inProceedings of the IEEE/RSJ International Conference on Intelli- gent Robots and Systems (IROS), 2025, pp. 2598– 2605

  3. [3]

    EGO-Planner: An ESDF-Free Gradient-Based Lo- cal Planner for Quadrotors,

    X. Zhou, Z. Wang, H. Ye, C. Xu, and F. Gao, “EGO-Planner: An ESDF-Free Gradient-Based Lo- cal Planner for Quadrotors,”IEEE Robotics and Au- tomation Letters, vol. 6, no. 2, pp. 478–485, 2020

  4. [4]

    STAR: Swarm Technology for Aerial Robotics Re- search,

    J. Chiun, Y . R. Tan, Y . Cao, J. Tan, and G. Sartoretti, “STAR: Swarm Technology for Aerial Robotics Re- search,” inProceedings of the 24th International Conference on Control, Automation and Systems (ICCAS), 2024, pp. 141–146

  5. [5]

    Optical-Flow Based Strategies for Landing VTOL UA Vs in Cluttered Environments,

    L. Rosa, T. Hamel, R. Mahony, and C. Samson, “Optical-Flow Based Strategies for Landing VTOL UA Vs in Cluttered Environments,”IF AC Proceed- ings V olumes, vol. 47, no. 3, pp. 3176–3183, 2014

  6. [6]

    Shape Visual Servoing of a Cable Suspended Between Two Drones,

    L. Smolentsev, A. Krupa, and F. Chaumette, “Shape Visual Servoing of a Cable Suspended Between Two Drones,”IEEE Robotics and Automation Let- ters, vol. 9, no. 12, pp. 11473–11480, 2024

  7. [7]

    Vision-guided servoing with feature-based trajectory generation (for robots),

    J. T. Feddema and O. R. Mitchell, “Vision-guided servoing with feature-based trajectory generation (for robots),” IEEE Transactions on Robotics and Automation, vol. 5, no. 5, pp. 691–700, 1989

  8. [8]

    Dynamic sensor-based control of robots with visual feed- back,

    L. Weiss, A. Sanderson, and C. Neuman, “Dynamic sensor-based control of robots with visual feed- back,” IEEE Journal on Robotics and Automation, vol. 3, no. 5, pp. 404–417, 1987

  9. [9]

    Fixed-time fractional- order sliding mode control for image-based visual servoing of hexarotor,

    A. K. Kamath, N. T. Chan, and M. Feroskhan, “Fixed-time fractional- order sliding mode control for image-based visual servoing of hexarotor,” in 2024 International Conference on Unmanned Air- craft Systems(ICUAS). IEEE, 2024, pp. 512–521

  10. [10]

    A new parti- tioned approach to image-based visual servo con- trol,

    P. I. Corke and S. A. Hutchinson, “A new parti- tioned approach to image-based visual servo con- trol,” IEEE Transactions on Robotics and Automa- tion, vol. 17, no. 4, pp. 507–515, 2001

  11. [11]

    Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach,

    T. Hamel and R. Mahony, “Visual servoing of an under-actuated dynamic rigid-body system: An image-based approach,” IEEE Trans. Robot. Au- tom., vol. 18, no. 2, pp. 187–198, 2002

  12. [12]

    Perception-aware image-based visual servoing of aggressive quadrotor uavs,

    C. Qin, Q. Yu, H. H. Go, and H. H.-T. Liu, “Perception-aware image-based visual servoing of aggressive quadrotor uavs,” IEEE/ASME Trans- actions on Mechatronics, vol. 28, no. 4, pp. 2020–2028, 2023

  13. [13]

    Point-based and region-based image moments for visual servoing of planar objects,

    O. Tahri and F. Chaumette, “Point-based and region-based image moments for visual servoing of planar objects,” IEEE Transactions on Robotics, vol. 21, no. 6, pp. 1116–1127, 2005

  14. [14]

    Ibvs based on adaptive sliding mode con- trol for a quadrotor target tracking under perturba- tions,

    A. Miranda-Moya, H. Castaneda, J. Gordillo, and H. Wang, “Ibvs based on adaptive sliding mode con- trol for a quadrotor target tracking under perturba- tions,” Mechatronics, vol. 88, p. 102909, 2022

  15. [15]

    Experimental validation of dynamic visual servo- ing for a quadrotor using a virtual camera,

    G. Fink, H. Xie, A. F. Lynch, and M. Jagersand, “Experimental validation of dynamic visual servo- ing for a quadrotor using a virtual camera,” in 2015 International conference on unmanned aircraft sys- tems (ICUAS). IEEE, 2015, pp. 1231–1240

  16. [16]

    GPS denied IBVS-based navigation and collision avoidance of UA V using a low-cost RGB camera,

    X. Wang, Y . R. Tan, W. Leong, S. Huang, R. Teo, and C. Xiang, “GPS denied IBVS-based navigation and collision avoidance of UA V using a low-cost RGB camera,” arXiv preprint arXiv:2509.17435, 2025

  17. [17]

    Sampling-based path planning and model predictive image-based visual servoing for quadrotor uavs,

    M. Li, H.-N. Wu, and Z.-Y . Liu, “Sampling-based path planning and model predictive image-based visual servoing for quadrotor uavs,” in 2017 Chi- nese Automation Congress (CAC). IEEE, 2017, pp. 6237– 6242

  18. [18]

    Quadrotor Going Through a Window and Landing: An Image-Based Visual Servo Con- trol Approach,

    Z. Tang, R. Cunha, D. Cabecinhas, T. Hamel, and C. Silvestre, “Quadrotor Going Through a Window and Landing: An Image-Based Visual Servo Con- trol Approach,”Control Engineering Practice, vol. 112, p. 104827, 2021

  19. [19]

    Gurobi Optimizer Reference Manual,

    Gurobi Optimization, LLC, “Gurobi Optimizer Reference Manual,” 2025. [Online]. Available: https://www.gurobi.com