In recent years, autonomous tracking of moving targets using unmanned aerial vehicles (UAVs), particularly quadrotor drones, has become a critical research focus for applications in aerial cinematography, surveillance, and inspection. Traditional manual piloting for tracking is highly demanding, requiring simultaneous control of flight trajectory and camera aiming, often leading to suboptimal results. Therefore, the development of robust autonomous tracking systems is paramount.
A common limitation in many existing systems is the use of a fixed, body-mounted camera. The camera’s field of view is intrinsically linked to the aircraft’s attitude. During aggressive maneuvers or when the target moves rapidly, this can cause the target to leave the frame, resulting in tracking failure. To overcome this significant drawback, we propose a comprehensive tracking control system for a quadrotor drone equipped with a Pan-Tilt-Zoom (PTZ) gimbal. This system decouples the visual sensing axis from the aircraft’s body axis, allowing the camera to independently track the target, thereby enlarging the effective tracking envelope and significantly reducing the risk of target loss. This paper details the system’s design, encompassing dynamic modeling, visual perception algorithms, relative position estimation, and the design of tracking controllers for both the gimbal and the quadrotor drone.
The core of our approach involves using the PTZ camera to acquire images, detecting and tracking a ground target based on its color and shape features, calculating the relative position between the quadrotor drone and the target, and finally using this estimated state to drive discrete cascade controllers for stable tracking flight.
1. Dynamic Modeling of the Quadrotor Drone
A quadrotor drone is an underactuated system with six degrees of freedom (three translational and three rotational) but only four independent control inputs (the thrusts of the four rotors). Its dynamic model is derived from rigid-body mechanics. The rotational motion is governed by:
$$ T = J \dot{\omega} + \omega \times J\omega $$
Where \( T \) is the torque vector, \( J \) is the inertia matrix, and \( \omega \) is the angular velocity vector. For a standard “+” configuration quadrotor drone and by leveraging the small-angle assumption and neglecting Coriolis terms for controller design, the simplified rotational dynamics about the body-fixed frame axes (roll \(\phi\), pitch \(\theta\), yaw \(\psi\)) can be expressed as:
$$
\begin{aligned}
\ddot{\phi} &= \frac{l}{\sqrt{2}J_x}(U_2) \\
\ddot{\theta} &= \frac{l}{\sqrt{2}J_y}(U_3) \\
\ddot{\psi} &= \frac{1}{J_z}(U_4)
\end{aligned}
$$
Here, \( l \) is the arm length from the center of mass to a rotor, \( J_x, J_y, J_z \) are the moments of inertia, and \( U_2, U_3, U_4 \) are the control inputs related to torques. The translational motion in the inertial frame, derived from Newton’s second law and coordinate transformation, is:
$$
\begin{aligned}
\ddot{x} &= \frac{U_1}{m} (\cos\phi \sin\theta \cos\psi + \sin\phi \sin\psi) \\
\ddot{y} &= \frac{U_1}{m} (\cos\phi \sin\theta \sin\psi – \sin\phi \cos\psi) \\
\ddot{z} &= \frac{U_1}{m} (\cos\phi \cos\theta) – g
\end{aligned}
$$
Where \( m \) is the total mass, \( g \) is gravitational acceleration, and \( U_1 \) is the total thrust input. For the purpose of horizontal displacement control and by assuming the yaw angle \( \psi \) is regulated to zero, the equations simplify to:
$$
\begin{aligned}
\ddot{x} &\approx \frac{U_1}{m} \theta \\
\ddot{y} &\approx -\frac{U_1}{m} \phi \\
\ddot{z} &= \frac{U_1}{m} – g
\end{aligned}
$$
This simplification reveals the fundamental coupling: horizontal acceleration is primarily achieved by controlling the pitch \( \theta \) and roll \( \phi \) angles.
2. Visual Perception and Relative Position Estimation
The visual perception pipeline is responsible for detecting the target in the image stream and estimating its relative 3D position with respect to the quadrotor drone.
2.1 Target Detection and Visual Tracking
The initial target acquisition is based on feature detection. Each captured frame is first white-balanced using the Gray World Algorithm to mitigate illumination effects. The image is then converted to the HSV color space where a target is identified using predefined thresholds for hue, saturation, and value. This process creates a binary mask. Contours are extracted from this mask, and shape recognition is performed using Hu moments to distinguish the target from other objects of similar color. Once identified, a bounding box is initialized.
For robust, frame-to-frame tracking, we employ the Kernelized Correlation Filter (KCF) tracker. The KCF algorithm models the target appearance with a kernelized classifier trained on circulantly shifted samples around the target location, allowing for extremely efficient detection in the subsequent frame. This provides a reliable pixel coordinate \( (u_t, v_t) \) of the target’s center in the image plane at time \( t \).
2.2 Relative Position Calculation via Pinhole Model
The key step is translating the 2D pixel coordinates into a 3D relative position vector \( \mathbf{P}_{target}^{c} = [X_c, Y_c, Z_c]^T \) in the camera frame. We use the pinhole camera model. The standard projection for a camera looking straight down (pitch=0) is straightforward, deriving from similar triangles:
$$
\frac{x}{f} = \frac{X_c}{Z_c}, \quad \frac{y}{f} = \frac{Y_c}{Z_c}
$$
Where \( (x, y) \) are coordinates in the normalized image plane (centered), \( f \) is the focal length, and \( Z_c \) is the altitude \( h \). However, this fails when the target moves towards the image edge. Our system uses a PTZ gimbal, allowing the camera to pitch and track the target. The geometry with a non-zero pitch angle \( \beta \) is more complex and is derived below.
Let the camera have a pitch angle \( \beta \) (defined positive when tilting up from the downward vertical). The camera’s gimbal ensures its roll and yaw relative to the body are controlled, but for this derivation, we consider the camera frame. The goal is to find the ground target’s position \( (X_c, Y_c) \) given its image coordinates \( (u, v) \), camera altitude \( h \), focal length \( f \), principal point \( (u_0, v_0) \), and pitch \( \beta \).
First, compute normalized image coordinates:
$$
x = \frac{u – u_0}{f}, \quad y = \frac{v – v_0}{f}
$$
The angle \( \alpha \) between the camera’s optical axis and the ray to the target in the camera’s X-Z plane (responsible for the \( Y_c \) coordinate) is given by \( \alpha = \arctan(y) \). From the side-view geometry, in the plane containing the optical axis and the target ray, we have a right triangle formed by the altitude \( h \), the ground range \( Y_c \), and the angle \( \alpha + \beta \). This yields:
$$
Y_c = h \cdot \tan(\alpha + \beta)
$$
For the \( X_c \) coordinate, we consider the plane perpendicular to the previous one. The physical distance \( l \) from the camera origin to the target is \( l = h / \cos(\alpha + \beta) \). The projection of this ray onto the plane defining the \( X_c \) direction creates a similar triangle relationship with the normalized coordinate \( x \). The distance from the camera to an imaginary plane perpendicular to the optical axis at the target’s range is \( z = l \cos(\alpha) \). The similar triangles in this perpendicular plane give:
$$
\frac{x}{f} = \frac{X_c}{z} = \frac{X_c}{l \cos(\alpha)}
$$
Substituting \( l = h / \cos(\alpha + \beta) \), we get the full solution:
$$
\boxed{
\begin{aligned}
X_c &= \frac{h \cdot \cos(\alpha)}{f \cdot \cos(\alpha + \beta)} \cdot x \\
Y_c &= h \cdot \tan(\alpha + \beta) \\
\text{where} \quad \alpha &= \arctan\left( \frac{v – v_0}{f} \right), \quad x = \frac{u – u_0}{f}
\end{aligned}}
$$
This set of equations allows our quadrotor drone tracking system to accurately estimate the relative position of a target even when the gimbal is actively tilting to keep it in view, which is a significant advancement over fixed-downward camera systems.
3. Control System Design
The control architecture is decomposed into two main subsystems: the gimbal attitude controller and the quadrotor drone flight controller.
3.1 Gimbal Tracking Control
The objective of the gimbal controller is to keep the target centered in the image. We use the target’s pixel coordinates \( (u_t, v_t) \) to generate error signals. A cascade Proportional-Derivative (PD) control structure is employed. The outer loop controls the angular position (pitch and yaw of the gimbal) based on the pixel error. The inner loop controls the angular rate. The derivative action helps dampen the motion and improve stability. Integral action is deliberately omitted because the target setpoint is not constant; an integrator would wind up due to the constantly moving target, introducing instability rather than eliminating steady-state error.
The discrete-time PD control law for, e.g., the pitch axis is:
$$
\omega_{cmd}(k) = K_{p} \cdot e_{\theta}(k) + K_{d} \cdot \frac{e_{\theta}(k) – e_{\theta}(k-1)}{T_s}
$$
Where \( e_{\theta}(k) \) is the error between the desired and current pitch angle derived from the image \( v \)-coordinate, \( \omega_{cmd} \) is the commanded angular rate to the inner loop, \( T_s \) is the sample time, and \( K_p, K_d \) are tunable gains.
3.2 Quadrotor Drone Tracking Control
The flight controller for the quadrotor drone uses the estimated relative position \( (X_c, Y_c) \) and altitude \( Z_c = h \) (from a separate sensor like a barometer or ultrasonic rangefinder) as feedback. We design a hierarchical, discrete-time cascade PID control structure.
First, we stabilize the inner loops: altitude, roll, pitch, and yaw. Each of these loops typically uses a PID controller. For instance, the altitude hold controller:
$$
U_1(k) = K_{p,z} \cdot e_z(k) + K_{i,z} \cdot \sum_{j=0}^{k} e_z(j)T_s + K_{d,z} \cdot \frac{e_z(k)-e_z(k-1)}{T_s} + mg
$$
Where \( e_z = Z_{desired} – Z_{measured} \), and the \( mg \) term provides a feedforward compensation for gravity.
The key to tracking is the outer position loop. The desired relative position \( (X_{des}, Y_{des}) \) is typically set to zero (i.e., maintain the target directly below a specific point relative to the drone). The position errors \( e_x = X_{des} – X_c \) and \( e_y = Y_{des} – Y_c \) are fed into two separate PID controllers. The outputs of these controllers are not direct motor commands but are interpreted as desired lean angles for the quadrotor drone. Specifically, the position controller generates setpoints for the inner attitude loops:
$$
\begin{aligned}
\theta_{desired}(k) &= PID_{x}( e_x(k) ) \\
\phi_{desired}(k) &= PID_{y}( -e_y(k) ) \quad \text{(Sign depends on coordinate frame)}
\end{aligned}
$$
The inner attitude controllers then work to achieve these desired roll and pitch angles, thereby generating the horizontal accelerations needed to follow the target. This nested control strategy is standard for quadrotor drone positioning.
4. Experimental Implementation and Results
The proposed system was implemented and tested on a commercial quadrotor drone platform (DJI M100) equipped with a custom 2-axis PTZ gimbal and an onboard computer (NVIDIA Jetson TX2) for image processing. The system parameters are summarized below.
| Parameter | Symbol | Value | Unit |
|---|---|---|---|
| Total Mass | \( m \) | 3.286 | kg |
| Arm Length | \( l \) | 0.325 | m |
| Moment of Inertia (X) | \( J_x \) | 0.05503 | kg·m² |
| Moment of Inertia (Y) | \( J_y \) | 0.05503 | kg·m² |
| Moment of Inertia (Z) | \( J_z \) | 0.097175 | kg·m² |
| Max Rotor Thrust | \( F_{max} \) | 21 | N |
| Camera Focal Length | \( f \) | 500 | pixels |
A series of experiments were conducted to validate each module and the integrated system.
Visual Perception Test: The drone was held stationary with the camera facing down. The target (a distinctively colored object) was moved within its field of view. The KCF tracker successfully followed the target, outputting a continuous stream of pixel coordinates without loss of track under moderate motion.
Gimbal Tracking Test: With the quadrotor drone stationary, the target was moved. The gimbal PD controller successfully adjusted the camera’s pitch and yaw angles to keep the target near the image center. Data logs confirmed smooth and responsive gimbal motion proportional to target displacement.
Integrated Flight Test: The full autonomous tracking system was engaged. The quadrotor drone took off and maintained a constant altitude. A ground target was moved along a predetermined path. The system demonstrated stable tracking performance. The following data illustrates the core result: the estimated target trajectory on the ground \( (X_{c,est}, Y_{c,est}) \) and the corresponding quadrotor drone‘s horizontal position \( (X_{drone}, Y_{drone}) \) were recorded. The controller successfully minimized the relative position error, keeping the drone above the target. The incorporation of the PTZ gimbal was crucial; during sharp target turns, the gimbal actively panned, preventing the target from leaving the frame—a scenario that would cause a failure in a fixed-camera system.
5. Conclusion
This paper presented a complete tracking control system for a quadrotor drone utilizing an actively controlled PTZ camera. The system addresses a key weakness of fixed-perspective tracking by allowing the camera to independently follow the target, greatly enhancing robustness. We derived the dynamic model of the quadrotor drone, developed a visual perception pipeline combining color/shape detection with KCF tracking, and most importantly, provided a detailed geometrical derivation for estimating the relative 3D position of a ground target when the camera is at an arbitrary pitch angle. This estimation serves as the critical feedback for the flight control system. Discrete-time cascade PID/PD controllers were designed for both gimbal steering and quadrotor drone displacement control. Experimental results on a physical platform confirm the effectiveness of the proposed methods. The quadrotor drone was able to autonomously and stably track a moving ground target, validating the overall system design. Future work may involve integrating more advanced trackers, improving robustness to occlusion, and enabling tracking in fully 3D environments.