In recent years, the development of unmanned aerial vehicles (UAVs), particularly quadrotor platforms, has witnessed remarkable progress. Among them, the China drone has become a focal point in both civil and military applications due to its vertical take-off and landing capability, flexible maneuverability, and low operational cost. However, the strong nonlinearity, multi-source disturbances, and model parameter perturbations inherent in quadrotor systems pose significant challenges to achieving high-performance attitude control. To address these issues, our team proposes a novel control framework named Disturbance-Optimized Adaptive Stochastic Fuzzy Model Predictive Control (DO-AFSMPC), which integrates Q-learning, unscented Kalman filtering (UKF), disturbance observer (DO), and T-S fuzzy modeling. This method is specifically designed to enhance the robustness and tracking accuracy of China drone systems under complex uncertain environments. In this paper, we present the complete design methodology, theoretical stability proof, and extensive simulation results that validate the superiority of our approach over existing methods.
The rest of the paper is organized as follows. Section II describes the nonlinear dynamics of the quadrotor and its T-S fuzzy modeling. Section III presents the disturbance observer and adaptive prediction strategy. Section IV details the UKF-based state estimation and Q-learning weight optimization. Section V provides the stability analysis. Section VI reports comparative simulation results, and Section VII concludes the paper.
Nonlinear Dynamics and T-S Fuzzy Modeling of China Drone
The quadrotor, as a typical China drone platform, is an underactuated and highly coupled multi-input multi-output system. The rigid body dynamics in the body-fixed frame can be described by the following differential equations:
$$
\begin{aligned}
\dot{\phi} &= p + q \sin\phi \tan\theta + r \cos\phi \tan\theta \\
\dot{\theta} &= q \cos\phi – r \sin\phi \\
\dot{\psi} &= \frac{q \sin\phi + r \cos\phi}{\cos\theta} \\
\dot{p} &= \frac{J_{yy} – J_{zz}}{J_{xx}} qr + \frac{u_1}{J_{xx}} \\
\dot{q} &= \frac{J_{zz} – J_{xx}}{J_{yy}} pr + \frac{u_2}{J_{yy}} \\
\dot{r} &= \frac{J_{xx} – J_{yy}}{J_{zz}} pq + \frac{u_3}{J_{zz}}
\end{aligned}
$$
where $\phi, \theta, \psi$ are the roll, pitch, and yaw angles, respectively; $p,q,r$ are the angular velocities; $J_{xx}, J_{yy}, J_{zz}$ are the moments of inertia; and $u_1, u_2, u_3$ are the control inputs generated by rotor speed differences. The input constraints arise from the physical limits of the motors:
$$
\begin{aligned}
& -k_T l (\omega_{\max}^2 – \omega_{\min}^2) \leq u_1 \leq k_T l (\omega_{\max}^2 – \omega_{\min}^2) \\
& -k_T l (\omega_{\max}^2 – \omega_{\min}^2) \leq u_2 \leq k_T l (\omega_{\max}^2 – \omega_{\min}^2) \\
& -k_D (2\omega_{\max}^2 – 2\omega_{\min}^2) \leq u_3 \leq k_D (2\omega_{\max}^2 – 2\omega_{\min}^2)
\end{aligned}
$$
To handle the strong nonlinearity, we adopt the Takagi-Sugeno (T-S) fuzzy modeling approach. The fuzzy rules are defined with the roll angle $\phi$ and pitch angle $\theta$ as premise variables. Nine operating points are selected to cover the typical flight envelope, as summarized in the table below.
| Index | Roll Angle (deg) | Pitch Angle (deg) | Yaw Angle (deg) |
|---|---|---|---|
| 1 | -45 | -45 | 0 |
| 2 | -45 | 0 | 0 |
| 3 | -45 | 45 | 0 |
| 4 | 0 | -45 | 0 |
| 5 | 0 | 0 | 0 |
| 6 | 0 | 45 | 0 |
| 7 | 45 | -45 | 0 |
| 8 | 45 | 0 | 0 |
| 9 | 45 | 45 | 0 |
For each operating point $i$, a local linear state-space model is obtained by linearizing the nonlinear dynamics:
$$
\mathbf{x}_{k+1} = \mathbf{A}_i \mathbf{x}_k + \mathbf{B}_i \mathbf{u}_k + \mathbf{R}_i + \mathbf{d}_k
$$
where $\mathbf{x} = [\phi, p, \theta, q, \psi, r]^T$, and $\mathbf{R}_i$ is the affine term. The overall fuzzy model is the weighted sum of the local models using Gaussian membership functions:
$$
\mathbf{x}_{k+1} = \sum_{i=1}^{N} \mu_i(\mathbf{z}_k) \left( \mathbf{A}_i \mathbf{x}_k + \mathbf{B}_i \mathbf{u}_k + \mathbf{R}_i \right) + \mathbf{d}_k
$$
where $\mu_i(\mathbf{z}_k)$ is the normalized firing strength. This T-S representation allows the application of linear control methods to the nonlinear China drone system with high fidelity.
Disturbance Observer and Adaptive Prediction Strategy
To counteract external disturbances and model mismatch, we design a disturbance observer (DO) that estimates the unknown disturbance vector $\mathbf{d}_k$. The DO dynamics are given by:
$$
\begin{aligned}
\hat{\mathbf{d}}_k &= \mathbf{F} \mathbf{x}_k – \mathbf{z}_k \\
\mathbf{z}_{k+1} &= \mathbf{z}_k + \mathbf{F} \left[ (\mathbf{A}_z \mathbf{I} – \mathbf{I})\mathbf{x}_k + \mathbf{B}_z \mathbf{u}_k + \mathbf{R}_z \right] + \mathbf{K}_z \hat{\mathbf{d}}_k
\end{aligned}
$$
where $\mathbf{F}$ is the observer gain matrix chosen such that $\mathbf{K}_z^+ = \mathbf{I} + \mathbf{F}$ is Schur stable. The estimated disturbance is then compensated by modifying the control input:
$$
\mathbf{u} = \mathbf{u}_c + \mathbf{u}_d = \mathbf{u}_c + \mathbf{F}_d \hat{\mathbf{d}}
$$
where $\mathbf{F}_d$ is the compensation gain. This DO significantly improves the robustness of the China drone control system against time-varying wind gusts and unmodeled dynamics.
Furthermore, we introduce an adaptive prediction horizon strategy to balance tracking accuracy and computational load. The prediction horizon $N_p$ is adjusted online based on the instantaneous tracking error $e(k)$:
$$
e(k) = \sqrt{ (\phi_r(k)-\phi(k))^2 + (\theta_r(k)-\theta(k))^2 + (\psi_r(k)-\psi(k))^2 }
$$
If $e(k) > 0.01$, we reduce $N_p$ to speed up response; if $e(k) < 0.01$, we increase $N_p$ to smooth control. This mechanism ensures that the stochastic model predictive control (SMPC) operates efficiently under varying conditions.
Unscented Kalman Filter for State Estimation
Accurate state information is crucial for the SMPC performance. In the presence of process noise $\mathbf{w}_k$ and measurement noise $\mathbf{v}_k$, we employ the unscented Kalman filter (UKF) to estimate the system states. The UKF uses a set of sigma points to propagate the mean and covariance through the nonlinear dynamics. The prediction step is:
$$
\begin{aligned}
\boldsymbol{\chi}_{k|k-1}^{(i)} &= f(\boldsymbol{\chi}_{k-1}^{(i)}) \\
\hat{\mathbf{x}}_{k|k-1} &= \sum_{i=0}^{2n} W_m^{(i)} \boldsymbol{\chi}_{k|k-1}^{(i)} \\
\mathbf{P}_{k|k-1} &= \sum_{i=0}^{2n} W_c^{(i)} (\boldsymbol{\chi}^{(i)} – \hat{\mathbf{x}})(\cdot)^T + \mathbf{Q}
\end{aligned}
$$
The measurement update yields the Kalman gain and posterior estimates. To adapt to time-varying noise statistics, we employ an adaptive covariance estimation scheme:
$$
\mathbf{P}_{xz} = (1-\beta) \mathbf{P}_{xz} + \beta \boldsymbol{\nu}_k \boldsymbol{\nu}_k^T
$$
where $\boldsymbol{\nu}_k$ is the innovation vector and $\beta$ is a forgetting factor. This adaptive UKF ensures reliable state estimates even when sensor noise characteristics change, which is common in real China drone flights.
Q-Learning for Online Weight Optimization
The performance of SMPC heavily depends on the choice of weight matrices $\mathbf{Q}$ and $\mathbf{W}$ in the cost function:
$$
J = \sum_{j=1}^{N_p} \mathbf{y}_{k+j}^T \mathbf{Q} \mathbf{y}_{k+j} + \sum_{j=1}^{N_c} \Delta \mathbf{u}_{k+j-1}^T \mathbf{W} \Delta \mathbf{u}_{k+j-1}
$$
Instead of using fixed weights, we integrate Q-learning to dynamically optimize $\mathbf{W}$ (and indirectly $\mathbf{Q}$) during operation. The state space of the Q-learning agent is defined as the tracking error magnitude $\delta(k) = e(k)$ and its derivative. The action space consists of three discrete actions: increase, keep, or decrease the weight $\mathbf{W}$ by a factor $\beta = 1.5$ within limits $[0.0001, 0.05]$. The reward function is designed to promote low tracking error, smooth control, and rapid convergence:
$$
r(k) = w_1 r_t(k) + w_2 r_s(k) + w_3 r_i(k) + r_v(k) + r_p(k)
$$
where $r_t$ penalizes large error, $r_s$ penalizes aggressive control changes, $r_i$ rewards error improvement, $r_v$ penalizes oscillation, and $r_p$ rewards proximity to target. The Q-value is updated iteratively:
$$
Q(s,a) \leftarrow Q(s,a) + \alpha \left[ r + \gamma \max_{a’} Q(s’,a’) – Q(s,a) \right]
$$
This online learning mechanism allows the China drone controller to adapt its aggressiveness in response to changing environments, effectively balancing tracking accuracy and energy consumption.
Stability Analysis
To guarantee the stability of the closed-loop system under the proposed DO-AFSMPC, we consider the Lyapunov candidate:
$$
V(k) = \mathbf{e}(k)^T \mathbf{P} \mathbf{e}(k) + J^*(k)
$$
where $\mathbf{e}(k) = \mathbf{x}(k) – \mathbf{x}_r(k)$ is the state tracking error, $\mathbf{P}$ is a positive definite matrix, and $J^*(k)$ is the optimal cost from SMPC. The first difference $\Delta V(k) = V(k+1)-V(k)$ can be shown to be negative definite when the disturbance observer and adaptive UKF sufficiently compensate for uncertainties, and the Q-learning ensures non-increasing of $J^*(k)$. Specifically, the error dynamics satisfy:
$$
\mathbf{e}(k+1)^T \mathbf{P} \mathbf{e}(k+1) – \mathbf{e}(k)^T \mathbf{P} \mathbf{e}(k) \leq -\mathbf{e}(k)^T \mathbf{Q}_e \mathbf{e}(k)
$$
and the rolling horizon property yields $J^*(k+1) \leq J^*(k) – \mathcal{L}(k)$. Therefore $\Delta V(k) < 0$ for all $\mathbf{e}(k) \neq 0$, proving asymptotic stability of the equilibrium point. This ensures that our China drone control system remains stable despite severe disturbances and parameter variations.
Simulation Setup and Results
We conducted extensive simulations using MATLAB/Simulink to evaluate the performance of DO-AFSMPC against baseline methods: standard SMPC and adaptive fuzzy SMPC (AFSMPC). The quadrotor parameters used in the simulation are listed below, which are representative of a typical China drone platform.
| Parameter | Value |
|---|---|
| Mass $m$ | 1.545 kg |
| Arm length $l$ | 0.255 m |
| Thrust coefficient $k_T$ | $15.84 \times 10^{-6} \ \text{N/(rad/s)}^2$ |
| Drag coefficient $k_D$ | $1.168 \times 10^{-7} \ \text{N/(rad/s)}^2$ |
| $J_{xx}$ | 0.029 kg·m² |
| $J_{yy}$ | 0.029 kg·m² |
| $J_{zz}$ | 0.055 kg·m² |
| Min rotor speed $\omega_{\min}$ | 1805 rad/s |
| Max rotor speed $\omega_{\max}$ | 11100 rad/s |
| Control horizon $N_c$ | 3 |
| Prediction horizon $N_p$ | 6 (adaptive) |
| Sampling time $t_s$ | 0.2 s |
We also configured the Q-learning parameters as shown below.
| Parameter | Value |
|---|---|
| Learning rate $\alpha$ | 0.3 |
| Discount factor $\gamma$ | 0.6 |
| Initial exploration rate | 0.35 |
| Final exploration rate | 0.02 |
| Exploration decay rate | 0.997 |
Tracking Performance under Ideal Conditions
First, we tested the attitude tracking ability of the controllers with sinusoidal reference trajectories. The results show that DO-AFSMPC achieves the smallest tracking error and fastest convergence. For example, the roll angle RMSE of DO-AFSMPC is reduced by about 12% compared to AFSMPC, and by about 35% compared to standard SMPC. The adaptive prediction horizon combined with UKF noise suppression and Q-learning weight tuning enabled the China drone to follow the reference with high precision.
Disturbance Rejection under Sinusoidal External Disturbances
To evaluate robustness, we injected a sinusoidal disturbance of amplitude 0.1 and frequency 0.0318 Hz into all three attitude channels while the quadrotor was hovering. The RMSE values before and after the disturbance are summarized in the following table.
| Controller | RMSE for Roll $\phi$ | RMSE for Pitch $\theta$ | RMSE for Yaw $\psi$ |
|---|---|---|---|
| SMPC | 0.0412 | 0.0518 | 0.0453 |
| AFSMPC | 0.0307 | 0.0377 | 0.0326 |
| DO-AFSMPC | 0.0269 | 0.0291 | 0.0301 |
DO-AFSMPC consistently outperforms the other methods, confirming the effectiveness of the disturbance observer and adaptive filtering. The China drone maintained stable attitude with minimal oscillations despite the persistent disturbance.
Model Mismatch Robustness
We also considered parameter uncertainty by randomly perturbing the moments of inertia $J_{xx}, J_{yy}, J_{zz}$ by up to ±20% independently. Under such mismatch, DO-AFSMPC exhibited the best tracking accuracy and smoothness. The control inputs from DO-AFSMPC showed negligible overshoot and fast settling, while SMPC suffered from significant overshoot and chattering. This demonstrates that the combined framework of T-S modeling, DO, UKF, and Q-learning provides exceptional robustness for China drone applications where model parameters may deviate over time.
Ablation Study
To quantify the contribution of each component, we conducted an ablation study by removing the Q-learning module from DO-AFSMPC (denoted as DO-AFSMPC without QL). The results show that the full DO-AFSMPC achieves higher cumulative reward and faster error reduction. The reward curve without Q-learning saturates at a lower level, and the error convergence is slower, especially during the initial phase. This validates the critical role of online weight optimization in enhancing the overall performance of the China drone controller.
Hyperparameter Sensitivity Analysis
We further examined the sensitivity of the Q-learning parameters. The best performance was obtained when the learning rate $\alpha \in [0.2, 0.4]$ and discount factor $\gamma \in [0.55, 0.6]$. Within this range, the reward accumulation was smooth and sustained. Extreme values led to instability or slow learning, confirming that careful tuning of hyperparameters is essential for the efficient operation of our China drone control system.
Conclusion
In this work, we have proposed a novel adaptive stochastic fuzzy model predictive control framework integrating Q-learning and unscented Kalman filtering for China drone attitude control. The T-S fuzzy modeling effectively linearizes the nonlinear dynamics, while the disturbance observer compensates for external disturbances. The adaptive UKF provides accurate state estimates even under time-varying noise, and the Q-learning algorithm dynamically optimizes the control weights to trade off tracking accuracy, energy efficiency, and robustness. Extensive simulation results demonstrate that the proposed DO-AFSMPC outperforms existing methods in terms of tracking precision, disturbance rejection, and robustness under model mismatch. This study provides a promising solution for high-performance attitude control of China drone platforms in complex real-world scenarios.
Future work will focus on hardware-in-the-loop validation and extension to cooperative control of multiple China drones.