In recent years, the demand for autonomous operation of unmanned aerial vehicles (UAVs) in environments where global positioning system (GPS) signals are unavailable or unreliable has grown significantly. Applications such as indoor inspection, search and rescue in dense urban areas, and operations within large structures necessitate alternative localization methods. Traditional GPS-based approaches fail in these scenarios, prompting a shift towards vision-based navigation systems. Among UAV platforms, the quadrotor drone stands out due to its agility, vertical take-off and landing capabilities, and hovering stability, making it an ideal candidate for autonomous missions in constrained spaces. However, achieving precise and robust control without GPS remains a challenging problem, primarily due to issues like sensor drift, computational constraints, and environmental dependencies.
This article presents a comprehensive framework for autonomous localization and control of a quadrotor drone using vision-based techniques. We address the limitations of existing methods by integrating enhanced visual simultaneous localization and mapping (SLAM) with optical flow sensing, all implemented on an embedded onboard computer. Our approach ensures real-time performance, robustness to environmental changes, and safety during flight. The key innovations include an improved visual SLAM algorithm for drift-free position estimation, a complementary optical flow module for handling map-loss scenarios, an extended Kalman filter (EKF) for sensor fusion, and the application of nonlinear control strategies for enhanced robustness. Through extensive outdoor experiments, we demonstrate that our system enables stable hovering and trajectory tracking under wind disturbances, without relying on GPS. The quadrotor drone achieves autonomous flight with high precision, showcasing the effectiveness of our integrated solution.
The rest of this article is organized as follows. We begin by reviewing related work on vision-based localization for quadrotor drones, highlighting gaps in current research. Next, we detail our system overview, including coordinate definitions and control architecture. We then describe the improved visual algorithms, sensor fusion methodology, and control design. Subsequently, we present the hardware and software implementation of our quadrotor drone platform. Experimental results are analyzed to validate our approach, followed by a discussion of its implications and future work.
Related Work and Background
Vision-based localization for quadrotor drones has been extensively studied, with two primary techniques dominating the field: optical flow and visual SLAM. Optical flow methods estimate motion by analyzing the apparent movement of pixels between consecutive image frames. For instance, researchers have utilized sensors like ADNS-2610 and ADNS-3080 to derive horizontal velocity for quadrotor drone hovering. More recently, the PX4FLOW sensor has been employed for hover and trajectory tracking. However, optical flow suffers from integral drift over time due to its relative motion estimation, making it unsuitable for long-term navigation. Conversely, visual SLAM, particularly through parallel tracking and mapping (PTAM), provides absolute position estimates by building and tracking a map of environmental features. This approach has been implemented in various quadrotor drone systems, such as those by ETH Zurich and TU Munich, often achieving high accuracy. Nonetheless, these systems frequently rely on ground station processing, introducing latency and limiting operational range. Additionally, visual SLAM is prone to map-loss in feature-poor environments, compromising safety. Our work builds on these foundations by combining both techniques in an onboard embedded system, mitigating their individual weaknesses while enhancing real-time performance and robustness for quadrotor drone applications.
System Overview
Our autonomous control system for the quadrotor drone integrates multiple sensors and algorithms to estimate and control its state in GPS-denied environments. The overall architecture consists of a perception module (visual SLAM and optical flow), a sensor fusion module (EKF), and a control module (PID and nonlinear controllers), all running on an onboard computer. The quadrotor drone is equipped with a monocular camera, an IMU, a barometer, and an optical flow sensor. Data from these sensors are processed in real-time to generate position, velocity, and attitude estimates, which are then used for closed-loop control. To formalize the system, we define the following coordinate frames, summarized in Table 1.
| Coordinate Frame | Symbol | Description | Orientation |
|---|---|---|---|
| Inertial Frame | {I} | World-fixed frame for control and localization | X: West, Y: North, Z: Up |
| Body Frame | {B} | Attached to the quadrotor drone’s center | X: Forward, Y: Right, Z: Down |
| Camera Frame | {C} | Aligned with the body frame | Same as {B} |
| Optical Flow Frame | {O} | Aligned with the body frame | Same as {B} |
| Visual SLAM Frame | {V} | Output frame of visual SLAM algorithm | Same as {I} |
The state estimation and control algorithms operate primarily in the inertial frame {I}. The quadrotor drone’s dynamics are modeled as a nonlinear system, with position and attitude controlled through motor thrusts. Our system employs an inner-outer loop structure: the outer loop computes desired attitude angles from position errors, and the inner loop stabilizes the attitude using onboard IMU data. This decoupling simplifies control design for the quadrotor drone, allowing us to focus on position control based on vision-derived estimates.
Improved Visual Algorithms
To achieve robust localization for the quadrotor drone, we enhance traditional visual SLAM and integrate it with optical flow sensing. Visual SLAM, based on PTAM, estimates the quadrotor drone’s pose by tracking feature points across keyframes. However, standard implementations often struggle in low-texture environments due to insufficient features. Our improvements include increasing the number of extracted feature points by computing sub-pixel accurate corners and optimizing keyframe storage by reducing the distance between keyframe additions. This boosts the algorithm’s robustness and generality, enabling reliable operation in diverse outdoor settings for the quadrotor drone. The visual SLAM algorithm outputs the quadrotor drone’s 3D position $\mathbf{p}_v = [p_x, p_y, p_z]^T$ and orientation in the visual frame {V}, which is aligned with {I}.
Despite these enhancements, visual SLAM can still fail during abrupt motions or in featureless areas, leading to map loss. To ensure safety, we incorporate an optical flow sensor (PX4FLOW) as an auxiliary unit. The optical flow algorithm calculates the quadrotor drone’s horizontal velocity $\mathbf{v}_o = [v_x, v_y]^T$ by analyzing pixel displacements between frames. By integrating this velocity, we obtain a position estimate $\mathbf{p}_o = [p_x, p_y]^T$. However, due to integration drift, $\mathbf{p}_o$ is only accurate over short periods. Our fusion strategy dynamically switches between visual SLAM and optical flow based on feature matching quality. Specifically, we monitor the feature match ratio in visual SLAM; if it drops below 30%, indicating potential map loss, we switch to optical flow for control. Once the match ratio recovers above 30%, we revert to visual SLAM. This complementary approach leverages the long-term accuracy of visual SLAM and the short-term reliability of optical flow, effectively addressing both map-loss and drift issues for the quadrotor drone.
The mathematical formulation for the optical flow position update is given by:
$$ \mathbf{p}_o(t) = \mathbf{p}_o(t_0) + \int_{t_0}^{t} \mathbf{v}_o(\tau) \, d\tau $$
where $\mathbf{p}_o(t_0)$ is the initial position from visual SLAM. To minimize drift, we reset $\mathbf{p}_o(t_0)$ whenever visual SLAM is active.
Sensor Data Fusion with Extended Kalman Filter
The visual algorithms provide position estimates at limited frequencies (e.g., 20 Hz for visual SLAM), which may not suffice for real-time control of the quadrotor drone. To increase update rates and improve accuracy, we fuse visual positions with IMU acceleration data using an extended Kalman filter (EKF). The IMU delivers 3D acceleration $\mathbf{a} = [a_x, a_y, a_z]^T$ at 100 Hz in the body frame {B}, which is transformed to the inertial frame {I} using the quadrotor drone’s attitude. The EKF state vector is defined as:
$$ \mathbf{x}_k = [p_x, p_y, p_z, v_x, v_y, v_z]^T $$
where $p_x, p_y, p_z$ are positions and $v_x, v_y, v_z$ are velocities in {I}. The system’s discrete-time nonlinear process and measurement equations are:
$$ \mathbf{x}_k = f(\mathbf{x}_{k-1}, \mathbf{u}_{k-1}) + \omega_{k-1} $$
$$ \mathbf{z}_k = h(\mathbf{x}_k) + \nu_k $$
Here, $\mathbf{u}_k = [a_x, a_y, a_z]^T$ is the input acceleration, $\mathbf{z}_k$ is the measurement from visual SLAM or optical flow (position only), and $\omega_k$, $\nu_k$ are process and measurement noise with covariances $\mathbf{Q}$ and $\mathbf{R}$, respectively. Linearizing around the current state estimate yields the matrices:
$$ \mathbf{F}_{k-1} = \frac{\partial f}{\partial \mathbf{x}} \bigg|_{\hat{\mathbf{x}}_{k-1|k-1}, \mathbf{u}_{k-1}} = \begin{bmatrix} \mathbf{I}_3 & \delta t \mathbf{I}_3 \\ \mathbf{0}_3 & \mathbf{I}_3 \end{bmatrix} $$
$$ \mathbf{B}_{k-1} = \frac{\partial f}{\partial \mathbf{u}} \bigg|_{\hat{\mathbf{x}}_{k-1|k-1}, \mathbf{u}_{k-1}} = \begin{bmatrix} \frac{1}{2} \delta t^2 \mathbf{I}_3 \\ \delta t \mathbf{I}_3 \end{bmatrix} $$
$$ \mathbf{H}_k = \frac{\partial h}{\partial \mathbf{x}} \bigg|_{\hat{\mathbf{x}}_{k|k-1}} = \begin{bmatrix} \mathbf{I}_3 & \mathbf{0}_3 \end{bmatrix} $$
where $\delta t = 0.01$ s is the sampling time, $\mathbf{I}_3$ is the 3×3 identity matrix, and $\mathbf{0}_3$ is a 3×3 zero matrix. The process model assumes constant velocity between updates, with acceleration as input. The EKF steps—prediction and update—are executed at 100 Hz, fusing either visual SLAM or optical flow positions based on the switching logic. This fusion provides smoothed position and velocity estimates at high frequency, enhancing control performance for the quadrotor drone. The noise covariances are tuned empirically, as summarized in Table 2.
| Parameter | Symbol | Value | Description |
|---|---|---|---|
| Process Noise Covariance | $\mathbf{Q}$ | diag(0.01, 0.01, 0.01, 0.1, 0.1, 0.1) | Accounts for model uncertainties |
| Measurement Noise Covariance | $\mathbf{R}$ | diag(0.05, 0.05, 0.05) | Reflects visual position error |
Control Algorithm Design
Using the fused state estimates, we design control laws for the quadrotor drone’s position tracking. Our system employs an inner-outer loop structure: the inner loop stabilizes attitude using proportional-derivative (PD) control on IMU data, while the outer loop computes desired roll and pitch angles from position errors. For the outer loop, we implement two controllers: a proportional-integral-derivative (PID) controller and a nonlinear robust integral of the signum of the error (RISE) controller. The height control uses a PID controller with barometer feedback, but horizontal control leverages vision-based estimates.
The PID controller for horizontal position is defined as:
$$ u_x = k_{px} e_x + k_{ix} \int_0^t e_x(\tau) \, d\tau + k_{dx} (\dot{x}_d – v_x) $$
$$ u_y = k_{py} e_y + k_{iy} \int_0^t e_y(\tau) \, d\tau + k_{dy} (\dot{y}_d – v_y) $$
where $e_x = x_d – x$, $e_y = y_d – y$ are position errors, $x_d, y_d$ are desired trajectories, $x, y$ are estimated positions, $v_x, v_y$ are estimated velocities, and $k_p, k_i, k_d$ are gains. The derivative term uses velocity feedback to avoid noise amplification. For altitude control, we have:
$$ u_z = k_{pz} e_z + k_{iz} \int_0^t e_z(\tau) \, d\tau + k_{dz} \dot{e}_z $$
with $e_z = z_d – z$ from the barometer. The PID gains, tuned experimentally, are listed in Table 3.
| Axis | $k_p$ | $k_i$ | $k_d$ |
|---|---|---|---|
| X (horizontal) | 1.32 | 0.4 | 0.24 |
| Y (horizontal) | 1.32 | 0.4 | 0.24 |
| Z (altitude) | 1.56 | 0.46 | 0.28 |
To improve robustness against disturbances like wind, we also implement a RISE controller for horizontal control. This nonlinear controller offers better tracking under uncertainties. Define the position errors $e_{x1} = x_d – x$, $e_{y1} = y_d – y$. Then, virtual error signals are:
$$ e_{x2} = e_{x1} + \alpha_x \dot{e}_{x1}, \quad e_{y2} = e_{y1} + \alpha_y \dot{e}_{y1} $$
where $\alpha_x, \alpha_y > 0$ are gains. The RISE control laws are:
$$ u_x = k_{sx} e_{x2} + \int_0^t \left( k_{sx} \alpha_x e_{x2}(\tau) + \beta_x \text{sgn}(e_{x2}(\tau)) \right) d\tau $$
$$ u_y = k_{sy} e_{y2} + \int_0^t \left( k_{sy} \alpha_y e_{y2}(\tau) + \beta_y \text{sgn}(e_{y2}(\tau)) \right) d\tau $$
Here, $k_{sx}, k_{sy}, \beta_x, \beta_y > 0$ are control gains, and $\text{sgn}(\cdot)$ is the signum function. The integral of the signum term provides robustness to bounded disturbances. The RISE gains, determined through experimentation, are shown in Table 4. Altitude control remains PID-based for simplicity. These controllers generate desired accelerations, which are converted to attitude commands for the inner loop, enabling precise maneuvering of the quadrotor drone.
| Parameter | Symbol | Value |
|---|---|---|
| Proportional gain (X) | $k_{sx}$ | 1.3 |
| Proportional gain (Y) | $k_{sy}$ | 1.3 |
| Virtual error gain (X) | $\alpha_x$ | 0.22 |
| Virtual error gain (Y) | $\alpha_y$ | 0.22 |
| Robustness gain (X) | $\beta_x$ | 0.25 |
| Robustness gain (Y) | $\beta_y$ | 0.25 |
Experimental Platform
We developed a custom quadrotor drone testbed to validate our algorithms. The platform is built around a sturdy frame with four brushless motors, capable of carrying payloads up to 1 kg. Key components include an embedded onboard computer with an Intel Core i3 dual-core processor (1.8 GHz) running Linux, a monocular camera (640×480 resolution), a PX4FLOW optical flow sensor, and a flight controller integrating an IMU (3D accelerometer and gyroscope) and a barometer. The flight controller executes low-level attitude control, while the onboard computer runs all vision, fusion, and position control algorithms. This embedded architecture eliminates wireless transmission delays and interference, ensuring real-time performance for the quadrotor drone. Ground station software is used only for monitoring and initiating flights, not for real-time processing.
The software architecture employs multithreading to handle concurrent tasks efficiently. As shown in Figure 4 of the original paper, separate threads manage visual SLAM, optical flow, IMU data acquisition, EKF fusion, and control. The main thread coordinates data exchange and thread lifecycle. This design prevents bottlenecks and maintains high update rates. All code is implemented in C++ with OpenCV for vision processing, ensuring computational efficiency for the quadrotor drone system.
Experimental Results and Analysis
We conducted multiple outdoor flight experiments to evaluate our system’s performance for the quadrotor drone. Experiments were performed in windy conditions (up to 3 m/s) without GPS, focusing on autonomous hovering and robustness. Results are categorized into visual SLAM accuracy validation, PID control hovering, RISE control hovering, and map-loss recovery tests.
Visual SLAM Accuracy Validation: To assess the improved visual SLAM algorithm, we performed indoor tests using a ground robot carrying the camera. Ground truth positions were obtained from an overhead motion capture system. The quadrotor drone’s visual SLAM estimated positions were compared to ground truth. As shown in Figure 6 of the original paper, the position errors in x and y directions were mostly within ±0.05 m and ±0.02 m, respectively, with no drift over time. This confirms the high accuracy of our visual SLAM for the quadrotor drone.
PID Control Hovering: For outdoor hovering, the quadrotor drone was commanded to maintain a fixed position $(x_d, y_d, z_d) = (0.38 \text{ m}, -0.22 \text{ m}, 4.74 \text{ m})$. Using PID control with visual SLAM feedback, the quadrotor drone achieved stable hover. Position curves (Figure 8) show minimal deviation, with horizontal errors largely within ±0.2 m, as depicted in Figure 9. The quadrotor drone maintained altitude within ±0.1 m despite wind gusts.
RISE Control Hovering: We repeated hovering with RISE nonlinear control at $(x_d, y_d, z_d) = (0.75 \text{ m}, -0.14 \text{ m}, 1.23 \text{ m})$. The quadrotor drone demonstrated improved precision, with horizontal errors confined to ±0.15 m (Figure 11), outperforming PID. Attitude angles (roll, pitch, yaw) remained within ±2° (Figure 12), indicating strong disturbance rejection. The RISE controller’s robustness was evident under wind disturbances, showcasing its suitability for the quadrotor drone in unpredictable environments.
Map-Loss Recovery Tests: To validate the complementary strategy, we flew the quadrotor drone in a scenario where visual SLAM map loss was artificially induced by moving into a low-texture area. During normal operation (0–23.85 s), visual SLAM provided position estimates. Upon map loss (23.85–34.05 s), the system switched to optical flow control, maintaining stability. When visual SLAM recovered (after 34.05 s), control switched back seamlessly. Position trajectories (Figures 13 and 14) show smooth transitions with x-direction errors around ±0.1 m, proving the effectiveness of our approach for the quadrotor drone.
Quantitative results are summarized in Table 5. The quadrotor drone’s performance metrics highlight the advantages of our integrated system.
| Experiment | Controller | Average Position Error (m) | Max Position Error (m) | Wind Conditions |
|---|---|---|---|---|
| PID Hovering | PID | 0.15 | 0.25 | ~3 m/s |
| RISE Hovering | RISE | 0.10 | 0.18 | ~3 m/s |
| Map-Loss Test | Hybrid | 0.12 | 0.20 | ~3 m/s |
Discussion
Our results demonstrate that the quadrotor drone can achieve autonomous flight in GPS-denied environments using vision-based localization and control. The improved visual SLAM algorithm provides drift-free position estimates, while the optical flow sensor offers a safety net during map loss. The EKF fusion enhances update rates and smoothness, critical for real-time control. Compared to prior work, our system’s embedded implementation eliminates latency issues, and the complementary strategy overcomes limitations of standalone visual SLAM or optical flow. The RISE controller further boosts robustness, making the quadrotor drone resilient to disturbances. However, challenges remain: the visual SLAM computation is still intensive, limiting its use on low-power embedded systems. Future work could optimize the algorithm for microcontrollers or incorporate deep learning for feature extraction. Additionally, extending the system to dynamic obstacle avoidance would enhance the quadrotor drone’s autonomy in complex scenarios.
Conclusion and Future Work
In this article, we presented a comprehensive vision-based localization and control system for quadrotor drones operating without GPS. By integrating enhanced visual SLAM, optical flow sensing, EKF fusion, and nonlinear control, we developed a robust solution that ensures safety and precision. The quadrotor drone testbed, with onboard processing, demonstrated stable hovering and trajectory tracking in outdoor windy conditions. Our contributions include a complementary visual algorithm that handles map loss, the application of RISE control for improved robustness, and an embedded architecture that guarantees real-time performance. Experiments confirmed that the quadrotor drone achieves autonomous flight with errors within ±0.2 m, validating our approach.
For future work, we plan to optimize visual SLAM for lower computational cost, enabling deployment on smaller quadrotor drones with limited resources. We also aim to integrate depth sensors for 3D mapping and obstacle avoidance, expanding the quadrotor drone’s capabilities in cluttered environments. Furthermore, we will explore multi-agent coordination using vision-based localization for swarm applications. These advancements will continue to push the boundaries of autonomous quadrotor drone technology, making it more accessible and reliable for real-world missions.