In recent years, the rapid advancement of unmanned aerial vehicles (UAVs) has transformed both military and civilian applications, spanning logistics delivery, disaster response, urban traffic management, and battlefield reconnaissance. The employment of UAV formations, rather than single platforms, significantly enlarges the coverage area and provides more comprehensive situational awareness. However, two fundamental challenges remain: efficient three-dimensional trajectory planning in dynamic cluttered environments, and reliable visual information transmission under severely constrained wireless bandwidth. This study addresses these challenges through a threefold approach: a novel deep reinforcement learning algorithm for formation trajectory planning, a dynamic region-of-interest selection and fusion strategy for bandwidth reduction, and a practical edge-computing object detection software system. Throughout this work, I focus on the unique constraints of unmanned aerial vehicles operating in low-altitude dense environments.
The remainder of this article is organized as follows. First, I introduce the mathematical formulation for UAV formation trajectory planning, including environmental, kinematic, and formation models. Second, I propose the SPER-TD3 algorithm, which integrates a SumTree-based prioritized experience replay with an improved Twin Delayed Deep Deterministic Policy Gradient framework. Third, I present a bandwidth-efficient region-of-interest selection and fusion method that leverages YOLOv10 and TensorRT on the Jetson AGX Orin edge platform. Fourth, I describe the design and implementation of a target detection software system for UAV edge computing scenarios. Finally, I summarize the key findings and outline future research directions.
1. Problem Formulation for UAV Formation Trajectory Planning
1.1 Environment Model
I consider a three-dimensional task area in which a UAV formation must navigate from a starting point to a target point, maintaining a safe formation and avoiding obstacles. The environment is modeled in a Cartesian coordinate system, with obstacles represented as canonical geometric primitives: spheres, cylinders, and cones. For a UAV located at position \(q = (x,y,z)\), the collision condition can be expressed using an obstacle potential function \(\Phi(q)\). For a spherical obstacle centered at \((x_{sp}, y_{sp}, z_{sp})\) with radius \(r\), the function is given by
$$
\Phi_{sphere}(q) = \frac{(x-x_{sp})^2}{r^2} + \frac{(y-y_{sp})^2}{r^2} + \frac{(z-z_{sp})^2}{r^2}.
$$
Similarly, for a cylindrical obstacle with base center \((x_{cy}, y_{cy}, z_{cy})\), radius \(r\), and height \(H\), the condition is
$$
\Phi_{cylinder}(q) = \frac{(x-x_{cy})^2}{r^2} + \frac{(y-y_{cy})^2}{r^2},
$$
provided \(z \le H\). For a conical obstacle, I use the combined radial and vertical criterion to determine whether the UAV lies inside the cone. The UAV is considered to collide with an obstacle when \(\Phi(q) < 1\).
1.2 Kinematic Model
The UAV is assumed to be a rotary-wing vehicle with motion described by its three-dimensional position, heading angle \(\psi\), and flight path angle \(\gamma\). At time step \(t\), the position increment is computed from the previous position and the control inputs. The kinematic equations are
$$
\dot{x} = v \cos\psi \cos\gamma,\quad
\dot{y} = v \sin\psi \cos\gamma,\quad
\dot{z} = v \sin\gamma,
$$
where \(v\) is the speed. After applying motion constraints, the next position is obtained by
$$
q_{t+1} = q_t + \begin{bmatrix} \Delta x_{res} \\ \Delta y_{res} \\ \Delta z_{res} \end{bmatrix},
$$
where the subscript \(res\) denotes the constrained increments. The heading and path angles are bounded by maximum values \(\psi_{\max}\) and \(\gamma_{\max}\), respectively.
1.3 Formation Model using a Virtual Leader
I employ an angle-based virtual leader method to control the formation. The leader UAV’s forward direction vector \(\mathbf{F}\) is defined by consecutive positions:
$$
\mathbf{F} = \frac{q_{t+1} – q_t}{\lVert q_{t+1} – q_t \rVert}.
$$
To generate a virtual leader position for each follower, I rotate a vector about an axis \(\mathbf{V} = \mathbf{F} \times \mathbf{K}\), where \(\mathbf{K}\) is a non-parallel unit vector. Using Rodrigues’ rotation formula, the rotated vector \(\mathbf{V}_{rot}\) is
$$
\mathbf{V}_{rot} = \mathbf{V}\cos\theta + (\mathbf{V}’ \times \mathbf{V})\sin\theta + \mathbf{V}'(\mathbf{V}’ \cdot \mathbf{V})(1-\cos\theta),
$$
where \(\mathbf{V}’\) is the normalized rotation axis and \(\theta\) is the rotation angle. The virtual leader position is then
$$
q_{vl} = q_{leader} + D\,\mathbf{V}_{rot},
$$
where \(D\) is the desired distance between the leader and the virtual leader. Followers track their respective virtual leader positions while maintaining a specified relative offset.
1.4 Objective and Constraints
The trajectory planning problem aims to minimize a weighted sum of trajectory length, collision probability, and formation deviation. The total trajectory length is computed as
$$
L = \sum_{i=1}^{N-1} \lVert q_{i+1} – q_i \rVert,
$$
where \(N\) is the number of waypoints. The collision probability is estimated using the minimum distance to obstacles:
$$
P_{collision} = \exp\left(-\lambda \min_{o \in \mathcal{O}} \lVert q – o \rVert \right),
$$
where \(\lambda\) is a decay coefficient. The formation deviation term measures the sum of distances between followers and their virtual target positions. Thus, the overall objective is
$$
\min_{q_1,\dots,q_N} \left[ \omega_1 L + \omega_2 P_{collision} + \omega_3 \sum_{i=2}^{n} \lVert q_{i} – q_{i,follower}^{target} \rVert \right],
$$
where \(\omega_1,\omega_2,\omega_3\) are positive weights. This multi-objective optimization is solved using a deep reinforcement learning approach.
2. The SPER-TD3 Algorithm
2.1 Markov Decision Process Formulation
I formulate the trajectory planning problem as a Markov Decision Process (MDP), defined by the tuple \((\mathcal{S}, \mathcal{A}, P, R)\). The state space \(\mathcal{S}\) includes the UAV’s relative position to each obstacle and to the goal. For obstacle \(i\), the state is
$$
s_i = [\Delta x_i, \Delta y_i, \Delta z_i, \Delta x_{goal}, \Delta y_{goal}, \Delta z_{goal}],
$$
where \(\Delta x_i,\Delta y_i,\Delta z_i\) are the components of the vector from the UAV to the obstacle center, and \(\Delta x_{goal},\Delta y_{goal},\Delta z_{goal}\) are the components of the vector from the UAV to the goal.
The action space consists of the steering angles generated for each obstacle by the policy network:
$$
a_i = \eta_i,
$$
where \(\eta_i\) is the steering angle produced by the network for obstacle \(i\). The reward function is composed of three components: collision avoidance \(R_{col}\), goal reaching \(R_{len}\), and angle smoothness \(R_{ang}\). The total reward is
$$
R = R_{col} + R_{len} + R_{ang}.
$$
The collision reward is designed as
$$
R_{col} = \begin{cases}
-1, & \text{if } d(q, obs) \le R_{obs}, \\
-k\, d(q, obs) / (d_{safe} – R_{obs}), & \text{if } R_{obs} < d(q, obs) < d_{safe}, \\
0.1, & \text{otherwise},
\end{cases}
$$
where \(d(q,obs)\) is the distance to the obstacle center, \(R_{obs}\) is the obstacle radius, and \(d_{safe}\) is the safety distance. The goal-reaching reward is
$$
R_{len} = \begin{cases}
+3, & \text{if } d(q, q_{goal}) < d_{threshold}, \\
– d(q, q_{goal}) / d_{start-goal}, & \text{otherwise},
\end{cases}
$$
and the smoothness reward penalizes large changes in heading and path angles:
$$
R_{ang} = -0.1 \left( \frac{|\Delta \psi|}{\psi_{\max}} + \frac{|\Delta \gamma|}{\gamma_{\max}} \right).
$$
2.2 SumTree Prioritized Experience Replay
Conventional experience replay samples uniformly, which can lead to inefficient learning when important transitions are rare. To address this, I incorporate a prioritized experience replay mechanism based on a SumTree data structure. The priority of each transition is defined by the temporal difference (TD) error:
$$
p = |\delta|^{\alpha} + \epsilon,
$$
where \(\alpha\) controls the degree of prioritization and \(\epsilon\) is a small positive constant. Transitions are stored in a SumTree, which enables efficient sampling with probability proportional to priority. During network updates, importance sampling weights are calculated as
$$
w_j = \frac{(N \cdot p_j)^{-\beta}}{\max_i (N \cdot p_i)^{-\beta}},
$$
where \(N\) is the buffer size and \(\beta\) is a hyperparameter that anneals over time. This mechanism improves convergence speed and training stability.
2.3 Network Architecture
I design modified Actor and Critic networks to enhance feature extraction and temporal reasoning. The Actor network takes the state \(s\) as input and passes it through a Long Short-Term Memory (LSTM) layer with 128 units, followed by a multi-layer perceptron (MLP) with two hidden layers of 256 neurons and ReLU activations. The output layer uses a tanh activation to produce the action. The Critic network concatenates the state and action, processes the state through a self-attention layer with 6 attention heads, then through an LSTM layer and an MLP with the same structure as the Actor, ultimately outputting the Q-value. The self-attention mechanism can be expressed as
$$
\text{Attention}(Q,K,V) = \text{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V,
$$
where \(Q\), \(K\), and \(V\) are the query, key, and value matrices, and \(d_k\) is the dimension of the key. This architecture allows the network to selectively focus on important parts of the state history, improving decision quality in complex environments.
2.4 Training Procedure
The SPER-TD3 algorithm follows the standard TD3 training loop with several modifications. The pseudocode is summarized below:
| Step | Operation |
|---|---|
| 1 | Initialize Actor networks \(\theta\) and Critic networks \(\phi_1,\phi_2\), and target networks \(\theta’\leftarrow\theta\), \(\phi’\leftarrow\phi\) |
| 2 | Initialize SumTree buffer with priority exponent \(\alpha\) and importance sampling exponent \(\beta\) |
| 3 | For each training episode, observe state \(s\) |
| 4 | Select action \(a = \pi(s;\theta) + \epsilon\), where \(\epsilon \sim \mathcal{N}(0,\sigma^2)\) |
| 5 | Execute action, observe reward \(r\) and next state \(s’\) |
| 6 | Compute TD error \(\delta\) using the target networks |
| 7 | Store transition \((s,a,r,s’,|\delta|^\alpha+\epsilon)\) in SumTree |
| 8 | Sample a batch according to priorities and compute importance weights |
| 9 | Update Critic networks by minimizing the weighted mean squared error |
| 10 | Every \(d\) steps, update Actor network using the deterministic policy gradient |
| 11 | Soft-update target networks with coefficient \(\tau\) |
| 12 | Update priorities in SumTree with new TD errors |
2.5 Trajectory Smoothing
To reduce sharp turns and improve path smoothness, I implement a curvature-based dynamic smoothing algorithm. The curvature metric between the averaged past direction \(\mathbf{F}_{ave}\) and the normalized total force vector \(\mathbf{F}’\) is
$$
\theta_{curv} = \arccos\left(\frac{\mathbf{F}_{ave} \cdot \mathbf{F}’}{\lVert \mathbf{F}_{ave} \rVert \lVert \mathbf{F}’ \rVert}\right).
$$
The smoothing factor \(\alpha\) is adaptively adjusted:
$$
\alpha = \begin{cases}
0.5, & \text{if } \theta_{curv} > \pi/4, \\
1, & \text{otherwise}.
\end{cases}
$$
The smoothed direction is computed as
$$
\mathbf{F}_{smooth} = \alpha \mathbf{F}_{ave} + (1-\alpha)\mathbf{F}’.
$$
Finally, the follower position is updated using \(q_{t+1}^{follower} = q_t^{follower} + v\, \mathbf{F}_{smooth}\), followed by kinematic constraint checks. This approach significantly reduces the steering angles and enhances flight safety and energy efficiency.
3. Experimental Validation of Trajectory Planning
3.1 Simulation Setup and Baselines
I evaluate the proposed SPER-TD3 algorithm in two three-dimensional environments: a large-obstacle scenario spanning \(10 \times 10 \times 6\) km with five obstacles of various shapes, and a small-obstacle scenario spanning \(15 \times 15 \times 3\) km with nine obstacles. I compare SPER-TD3 against eight baseline deep reinforcement learning algorithms: TD3, DDPG, SAC, and PPO, each using either fully centralized or fully decentralized training frameworks, as well as MADDPG. The simulation parameters are listed in the table below.
| Parameter | Value |
|---|---|
| Discount factor \(\gamma\) | 0.99 |
| Actor learning rate | 0.001 |
| Critic learning rate | 0.001 |
| Soft update coefficient \(\tau\) | 0.005 |
| Priority exponent \(\alpha\) | 0.6 |
| Importance sampling exponent \(\beta\) | 0.4 |
| Replay buffer capacity | \(10^6\) |
| Batch size | 512 |
| Maximum steps per episode | 500 |
| Maximum episodes | 500 |
3.2 Evaluation Metrics
I evaluate the algorithms using four primary metrics: trajectory length, reward magnitude, success rate, and trajectory safety coverage. The safety coverage is defined as the percentage of trajectory points that are located outside a specified danger radius around any obstacle. To combine these metrics, I introduce an importance comprehensive evaluation value \(I\):
$$
I = \frac{1}{4}\left( L’ + R’ + S’ + C’ \right),
$$
where \(L’\), \(R’\), \(S’\), and \(C’\) are normalized versions of the four metrics, each scaled to the interval \([0,1]\).
3.3 Simulation Results
The trajectory smoothing algorithm is first validated by comparing the turning angles of followers before and after smoothing. Using the artificial potential field method as the baseline, the average turning angle for follower 1 decreases from 89.61° to 51.67°, and for follower 2 from 114.03° to 54.68°, confirming a substantial improvement in smoothness.
| UAV | Metric | Before Smoothing | After Smoothing |
|---|---|---|---|
| Follower 1 | Average turning angle (deg) | 89.61 | 51.67 |
| Follower 2 | Average turning angle (deg) | 114.03 | 54.68 |
Next, the performance of SPER-TD3 is compared against all baselines. Table 3 summarizes the results for the large-obstacle environment.
| Algorithm | Trajectory Length | Reward Magnitude | Success Rate (%) | Safety Coverage (%) | Importance Value |
|---|---|---|---|---|---|
| SPER-TD3 | 13.26 | 29.96 | 98.80 | 33.33 | 0.650 |
| MADDPG | 13.17 | 29.92 | 99.00 | 19.40 | 0.596 |
| Decentralized TD3 | 13.33 | 30.92 | 98.40 | 24.84 | 0.585 |
| Decentralized DDPG | 12.84 | 28.54 | 97.40 | 22.72 | 0.599 |
| Decentralized SAC | 12.74 | 29.86 | 92.20 | 30.43 | 0.478 |
| Centralized SAC | 13.77 | 31.47 | 90.60 | 26.57 | 0.498 |
| Centralized TD3 | 13.60 | 30.70 | 98.00 | 31.43 | 0.642 |
| Centralized DDPG | 13.78 | 32.01 | 94.00 | 31.22 | 0.477 |
| Centralized PPO | 13.92 | 31.91 | 72.20 | 26.57 | 0.376 |
The results demonstrate that SPER-TD3 achieves the highest importance comprehensive evaluation value, surpassing the second-best algorithm (centralized TD3) by approximately 8.5% and the worst algorithm (centralized PPO) by 72.9%. Furthermore, SPER-TD3 exhibits superior training stability and convergence speed, as evidenced by the average episode reward curves. In both large- and small-obstacle environments, the proposed algorithm converges faster than all baseline methods and maintains a higher final reward, indicating better sample efficiency and robustness. The key advantage stems from the combination of prioritized experience replay and the improved Actor-Critic architecture, which enables the UAV to learn more effectively from critical experiences while avoiding local optima.
4. Bandwidth-Optimized Region-of-Interest Selection and Fusion
4.1 Overview of the Method
In UAV reconnaissance missions, the wireless transmission bandwidth is often severely limited. Transmitting full-resolution video at all times can lead to congestion and packet loss. To mitigate this, I propose a region-of-interest (ROI) selection and fusion method that transmits only the critical regions within the video while maintaining a static background frame. The overall processing flow consists of the following steps:
- Target classification: The ground control station specifies the types of targets to be detected (e.g., vehicles, persons) and transmits these specifications to the UAV.
- Video acquisition: The UAV captures video using an onboard camera.
- Bandwidth measurement: The UAV periodically measures the available wireless link bandwidth.
- Onboard detection: The edge computing module runs a deep learning model to detect targets in the video.
- ROI generation: When a target of interest is detected, the UAV generates an ROI around the target and adjusts the camera gimbal to focus on it.
- ROI transmission: The ROI video is compressed and transmitted to the ground station, along with a low-frequency static background frame.
- Video fusion: The ground station fuses the received ROI with the stored background frame to reconstruct a full view.
4.2 Dynamic ROI Generation
I implement a dynamic ROI algorithm that continuously tracks the detected target. The central idea is to identify whether the target is stationary or moving slowly. If the target remains within a pre-defined static ROI for more than 2 seconds, the frame is captured as the static background. Subsequently, only the ROI region is updated and transmitted at the full frame rate. The background is refreshed every 2 seconds to accommodate slow changes in lighting or background. The algorithm is summarized in Table 4.
| Step | Description |
|---|---|
| 1 | Run YOLOv10 detection on the current frame, keeping only detections with confidence \(>0.5\) and in the desired classes |
| 2 | Compute the center of the largest detection box |
| 3 | If a static ROI exists and the current center lies within it, accumulate the stationary time; otherwise, update the dynamic ROI and reset the stationary time |
| 4 | If the stationary time exceeds 2 s, extract the static ROI and save the current frame as the background |
| 5 | Fuse the dynamic ROI with the background frame to produce the output video |
4.3 TensorRT Acceleration on Edge Devices
To achieve real-time inference on the UAV’s edge platform, I deploy YOLOv10 using NVIDIA TensorRT on a Jetson AGX Orin module. TensorRT applies several optimization techniques, including layer fusion, precision calibration (FP16/INT8), dynamic tensor memory management, kernel auto-tuning, and dynamic slicing. These optimizations reduce computational overhead and memory usage without sacrificing significant accuracy. The optimized pipeline is as follows:
- Convert the trained YOLOv10 model to ONNX format, then to a TensorRT engine.
- Apply FP16 precision to speed up inference while maintaining acceptable mean average precision.
- Perform input preprocessing (resize, normalization) and bind GPU memory.
- Run inference with the TensorRT engine and decode the output tensors to bounding boxes.
4.4 Performance Evaluation
I evaluated the system on two real-world video scenes: a sparse crowd scene and a dense crowd scene. The benchmark platform was the Jetson AGX Orin, and the test set consisted of 300 consecutive frames from each scene. Table 5 shows the average inference time and frame rate for CPU, GPU, and TensorRT implementations.
| Computing Environment | Inference Time (ms) Sparse | Inference Time (ms) Dense | Frame Rate (FPS) Sparse | Frame Rate (FPS) Dense |
|---|---|---|---|---|
| CPU | 224.23 | 224.37 | 4.48 | 4.48 |
| GPU | 88.82 | 87.89 | 14.48 | 14.51 |
| TensorRT | 28.48 | 27.86 | 45.71 | 46.15 |
The TensorRT-optimized implementation reduces the average inference time by approximately 87.3% compared to CPU and by 68.1% compared to GPU. The frame rate exceeds 45 FPS, satisfying real-time requirements. The ROI-based transmission scheme further reduces the required wireless link bandwidth by up to 50% at peak, with a mean reduction of 35.2% compared to the full video stream. In the dense crowd scenario, the system successfully transmits the ROI containing the crowd while updating the static background frame every 2 seconds, maintaining high visual quality and a miss rate below 2%.
5. Edge Computing Target Detection Software Design and Verification
5.1 System Architecture
I design and implement a target detection software system tailored to UAV edge computing scenarios. The system comprises two main parts: the onboard (airborne) component and the ground control component. The onboard component runs on the Jetson AGX Orin and includes the TensorRT-accelerated YOLOv10 detector, the ROI generation and fusion module, an RTSP streaming server, and a bandwidth measurement module. The ground component is a PyQt5-based graphical user interface that provides video display, bandwidth monitoring, remote command execution, and interactive parameter configuration.
5.2 Software Workflow
The workflow can be summarized as follows. First, the UAV formation executes the planned trajectories using the SPER-TD3 algorithm. During flight, each UAV captures video through its camera and streams the raw data to the edge computing module. The TensorRT accelerated detector performs real-time object detection. When a target of interest is detected, the ROI selection module extracts the region surrounding the target. The ROI video is then compressed and transmitted through an RTSP stream over the wireless link to the ground station. Meanwhile, the ground station receives the ROI stream and fuses it with the previously stored static background to reconstruct the full scene. The software also provides a bandwidth test interface for evaluating link quality.
5.3 Interface Design
The main target detection interface consists of four panels:
- ROI Settings: Allows the user to select target classes (e.g., pedestrian, vehicle), set ROI size thresholds, and load the detection model.
- ROI Display: Shows the detected ROI region overlaid on the video.
- Video Stream Settings: Configures the camera IP address, RTSP URL, protocol, resolution, and frame rate.
- Bandwidth Comparison: Displays real-time bandwidth consumption curves for the original video versus the ROI video.
The bandwidth test interface includes server settings, client settings, test results, and dynamic graphs of bandwidth and latency. It supports TCP/UDP protocols, adjustable test duration, bitrate, and packet size. The software has been validated in a real field experiment using a DJI M350 RTK UAV equipped with a Zenmuse H30T gimbal and a Ubinode SDR self-organizing network module. The ground station used a laptop with an i7-12700H CPU and GTX 3060 GPU.
5.4 Field Test Results
The bandwidth test showed that the network latency fluctuated between 0.015 and 0.04 ms, and the average throughput reached 505.2 Mbit/s with a very low standard deviation. These results confirm that the wireless link is stable enough for high-quality video transmission. In the field experiment, the software successfully detected and transmitted the ROI of a dense audience area near a track. The ROI video was displayed smoothly without noticeable delay, and the bandwidth comparison chart showed a significant reduction in bandwidth consumption after applying the ROI algorithm. This validates the system’s practicality for real-world military reconnaissance and traffic monitoring scenarios.
6. Conclusions and Future Work
In this article, I have presented a comprehensive study on UAV formation three-dimensional trajectory planning and object detection. The main contributions are threefold:
- I proposed the SPER-TD3 algorithm, which integrates SumTree-based prioritized experience replay with an improved Actor-Critic architecture. Extensive simulations demonstrate that SPER-TD3 outperforms eight state-of-the-art deep reinforcement learning algorithms in terms of trajectory safety coverage, smoothing, success rate, and overall importance value. The algorithm also exhibits faster convergence and better training stability.
- I developed a bandwidth-efficient ROI selection and fusion method that combines YOLOv10 detection with TensorRT acceleration. The system significantly reduces wireless link bandwidth requirements while maintaining high-quality video output. Field tests on the Jetson AGX Orin platform confirmed real-time performance and robustness.
- I designed and implemented a complete edge computing target detection software with a user-friendly QT-based interface. The software integrates trajectory planning, object detection, ROI processing, and bandwidth analysis, providing a practical solution for deploying unmanned aerial vehicles in low-airspace applications.
Future research directions include extending the SPER-TD3 algorithm to heterogeneous UAV formations (e.g., mixing rotary-wing and fixed-wing platforms), incorporating multi-modal sensors (thermal infrared, radar) for robust ROI extraction under adverse weather conditions, and developing more lightweight neural network architectures for energy-constrained edge devices. Additionally, I plan to investigate transformer-based detectors to further improve small-object detection accuracy in high-altitude reconnaissance.
In summary, this research demonstrates the synergy between advanced trajectory planning and intelligent video processing for unmanned aerial vehicles. By addressing the core challenges of dynamic obstacle avoidance, cooperative formation control, and bandwidth-limited communication, the proposed methods contribute to the safe and efficient deployment of UAV swarms in complex operational environments.