The rapid advancement of unmanned aerial vehicle (UAV) technology has catalyzed the proliferation of complex multi-agent applications, with formation drone light shows standing out as a visually spectacular and technically demanding domain. Executing precise, synchronized, and safe aerial choreography in three-dimensional spaces laden with static and dynamic obstacles presents a fundamental path planning challenge. Traditional methods often struggle with balancing global optimality, local obstacle avoidance, and the smooth, cohesive movement required for an aesthetically pleasing formation drone light show.
This work addresses the core path planning problem for a fleet of UAVs operating in a cluttered 3D environment. The objective is to guide the formation from a designated start point to a target destination while avoiding collisions with obstacles and maintaining safe inter-agent distances. We model static threats as cylindrical obstacles. For the \(i\)-th obstacle with base center \((x_{a_i}, y_{a_i}, z_{a_i})\), radius \(r_{a_i}\), and height \(h_{a_i}\), its threat zone is defined as all points \((x, y, z)\) satisfying:
$$(x – x_{a_i})^2 + (y – y_{a_i})^2 \leq r_{a_i}^2, \quad z_{a_i} \leq z \leq z_{a_i} + h_{a_i}$$
The distance between the \(j\)-th UAV at position \(\mathbf{P}_j = (x_j, y_j, z_j)\) and this obstacle is:
$$d^{obs}_{i,j} = \begin{cases}
\| \mathbf{P}_j – (x_{a_i}, y_{a_i}, z_{a_i}) \|_2 – r_{a_i}, & \text{if } z_{a_i} \leq z_j \leq z_{a_i} + h_{a_i} \\
\infty, & \text{otherwise}
\end{cases}$$
The distance to the target \(\mathbf{P}_g = (x_g, y_g, z_g)\) is \(d^g_j = \|\mathbf{P}_j – \mathbf{P}_g\|_2\), and the distance between UAV \(j\) and UAV \(m\) is \(d^u_{j,m} = \|\mathbf{P}_j – \mathbf{P}_m\|_2\).
Path quality is evaluated using two primary metrics: Path Length and Path Smoothness. For a path consisting of \(M\) waypoints \(\mathbf{p}_1, \mathbf{p}_2, …, \mathbf{p}_M\), the total length \(L\) is:
$$L = \sum_{v=1}^{M-1} \| \mathbf{p}_{v+1} – \mathbf{p}_{v} \|_2$$
Smoothness \(S\), measured as the average angular change per meter, should ideally reside within \((0, 10)\, ^\circ/m\) for stable flight suitable for a formation drone light show. It is calculated as:
$$S = \frac{1}{M-2} \sum_{v=1}^{M-2} \arccos\left(\frac{(\mathbf{p}_{v} – \mathbf{p}_{v-1}) \cdot (\mathbf{p}_{v+1} – \mathbf{p}_{v})}{\| \mathbf{p}_{v} – \mathbf{p}_{v-1} \|_2 \cdot \| \mathbf{p}_{v+1} – \mathbf{p}_{v} \|_2}\right)$$
The core constraints are obstacle avoidance (\(d^{obs}_{i,j} > d^{obs}_o\), where \(d^{obs}_o\) is a safety threshold) and inter-agent collision avoidance (\(d^u_{j,m} > d^u_o\)).
To overcome the limitations of traditional Artificial Potential Field (APF) methods—namely local minima, oscillatory paths, and non-optimal global routes—we propose a hybrid strategy. This strategy decomposes the problem: a leader UAV determines a globally optimal path using a Deep Reinforcement Learning (DRL) model, and follower UAVs track this virtual leader using a significantly enhanced APF method for local, smooth trajectory adjustment and collision avoidance.
1. Leader Path Planning via Double Deep Q-Network (DDQN)
The leader’s task is to find a near-optimal, collision-free path in an unknown 3D environment. We frame this as a Markov Decision Process (MDP) solved by a DDQN agent with Prioritized Experience Replay (PER). The state space \(S\) is a discrete 3D grid, encapsulating the agent’s position, target position, and obstacle information. The action space \(A\) includes 26 discrete movement directions to adjacent grid cells. The agent follows an \(\epsilon\)-greedy policy \(\pi(s)\):
$$\pi(s) = \begin{cases}
\text{random action from } A, & \text{with probability } \epsilon \\
\arg\max_{a \in A} Q(s, a; \theta), & \text{with probability } 1-\epsilon
\end{cases}$$
where \(Q(s, a; \theta)\) is the action-value function approximated by the primary network with parameters \(\theta\).
The reward function \(r_t\) is designed to encourage goal-directed and efficient behavior:
$$r_t = \kappa (G_t – G_{t+1}) + \delta + r_{env}$$
where \(G_t = \|\mathbf{P}_g – \mathbf{P}_t\|_2\) is the current distance to goal, \(\kappa\) scales the distance reward, \(\delta\) is a small negative step penalty, and \(r_{env}\) is a large terminal reward (+200 for reaching goal, -50 for collision or boundary violation).
DDQN improves upon DQN by decoupling action selection and value estimation to reduce overestimation bias. The target value \(y_t\) for updating the primary network is:
$$y_t = r_t + \gamma Q(s_{t+1}, \arg\max_{a’} Q(s_{t+1}, a’; \theta); \theta^-)$$
Here, \(\gamma\) is the discount factor, and \(\theta^-\) are the parameters of a slowly updated target network. PER samples transitions \((s_t, a_t, r_t, s_{t+1})\) with probability proportional to their temporal-difference error (TD-error) \(\delta_t = |y_t – Q(s_t, a_t; \theta)|\), prioritizing learning from surprising experiences. The core algorithm is summarized below.
| Hyperparameter | Value |
|---|---|
| Batch Size (\(k\)) | 256 |
| Target Update Frequency (\(L\)) | 50 steps |
| Discount Factor (\(\gamma\)) | 0.99 |
| Soft Update Parameter (\(\tau\)) | 0.001 |
| Experience Replay Size | 100,000 |
| Initial/Final \(\epsilon\) | 0.8 / 0.03 |
| Learning Rate (\(\alpha\)) | \(1.5\times10^{-5} \rightarrow 6\times10^{-6}\) |
Simulation in a bounded environment with cylindrical obstacles shows the DDQN leader path is significantly superior to a traditional APF path, being 21.6% shorter and 91.2% smoother, making it an ideal virtual trajectory for a coordinated formation drone light show.
2. Enhanced Artificial Potential Field for Follower Trajectory Control
Followers must track the virtual leader’s path while dynamically avoiding obstacles and each other. The traditional APF for the \(j\)-th follower sums attractive force from its assigned target point (derived from the leader’s path) and repulsive forces from obstacles and other UAVs:
$$\mathbf{F}^{af}_j = \mathbf{F}^{att}_j + \sum_{i=1}^{N} \mathbf{F}^{rep-obs}_{i,j} + \sum_{\substack{m=1 \\ m \neq j}}^{n} \mathbf{F}^{rep}_{j,m}$$
The classical forms are:
$$\mathbf{F}^{att}_j = -k_{atto} \cdot d^g_j \cdot \frac{\partial d^g_j}{\partial \mathbf{P}_j}$$
$$\mathbf{F}^{rep-obs}_{i,j} = \begin{cases}
k_{repo} \left(\frac{1}{d^{obs}_{i,j}} – \frac{1}{d^{obs}_o}\right) \frac{1}{(d^{obs}_{i,j})^2} \frac{\partial d^{obs}_{i,j}}{\partial \mathbf{P}_j}, & \text{if } d^{obs}_{i,j} \leq d^{obs}_o \\
0, & \text{otherwise}
\end{cases}$$
To suppress oscillations and improve convergence for a stable formation drone light show, we introduce adaptive coefficients for the attractive gain, repulsive gain, and motion step size.
Adaptive Attractive Gain: The gain decreases as the UAV approaches its sub-target to prevent overshoot and oscillation.
$$k_{att}(j) = k_{atto} \left( 1 + \rho \frac{d^g_j}{d^g_j + \eta} \right)$$
Here, \(\rho\) controls the strength of distance-based adjustment, and \(\eta\) is a smoothing factor.
Adaptive Repulsive Gain: The repulsive strength increases non-linearly as the UAV gets closer to an obstacle or neighbor, ensuring stronger evasion in critical situations.
$$k_{rep}^{obs}(i,j) = k_{repo} \left( \frac{d^{obs}_o}{d^{obs}_{i,j} + \tau} \right)^{\xi}$$
where \(\xi\) is a power coefficient controlling the non-linearity, and \(\tau\) is a small constant for numerical stability.
Adaptive Step Size: The movement step \(l_u(j)\) is adjusted based on the environment. It increases in open space for faster convergence and decreases near obstacles for finer, smoother control.
$$l_u(j) = l_0 \frac{d^g_j}{d^g_j + \mu \cdot d^{obs}_{min}(j)}$$
where \(l_0\) is the base step, \(\mu\) is an obstacle influence weight, and \(d^{obs}_{min}(j)\) is the distance to the nearest obstacle.
The enhanced forces for the follower’s APF are then:
$$\mathbf{\tilde{F}}^{att}_j = -k_{att}(j) \cdot d^g_j \cdot \frac{\partial d^g_j}{\partial \mathbf{P}_j}$$
$$\mathbf{\tilde{F}}^{rep-obs}_{i,j} = \begin{cases}
k_{rep}^{obs}(i,j) \left(\frac{1}{d^{obs}_{i,j}} – \frac{1}{d^{obs}_o}\right) \frac{1}{(d^{obs}_{i,j})^2} \frac{\partial d^{obs}_{i,j}}{\partial \mathbf{P}_j}, & \text{if } d^{obs}_{i,j} \leq d^{obs}_o \\
0, & \text{otherwise}
\end{cases}$$
An ablation study was conducted to determine optimal hyperparameters for a follower UAV, measuring the impact on path length \(L\) and smoothness \(S\). The results guided the final parameter selection, summarized below.
| Hyperparameter | Tested Range | Optimal Value | Primary Influence |
|---|---|---|---|
| Obstacle Weight (\(\mu\)) | [4, 44] | 28 | Dominant effect on smoothing (78% improvement) |
| Smoothing Factor (\(\eta\)) | [1, 5] | 3 | Significant smoothing (18.3% improvement) |
| Repulsion Power (\(\xi\)) | [1, 3] | 1.5 | Moderate smoothing (2.1% improvement) |
| Distance Coefficient (\(\rho\)) | [0.5, 5] | 4.5 | Minor refinement |
3. Integrated Formation Control: Virtual Leader-Follower Framework
The complete system employs a single virtual leader-multiple follower (\(L_v – F_\phi\)) architecture. The DDQN-generated path provides the virtual leader’s trajectory \(\mathbf{P}^L = [\mathbf{P}^L_1, \mathbf{P}^L_2, …]\). Each follower \(j\), starting with a formation offset \(\mathbf{d}_j\) relative to the leader, uses the enhanced APF to compute its control force \(\mathbf{\tilde{F}}^{af}_j = [F^x_j, F^y_j, F^z_j]\). Its position update at step \(k\), given the leader’s position \(\mathbf{P}^L_k\), is:
$$
\begin{bmatrix}
x^{F_j}_j \\
y^{F_j}_j \\
z^{F_j}_j
\end{bmatrix}_k =
\begin{bmatrix}
x^L_k + d^x_j + l_u(j) \cos\alpha_j \cos\beta_j \\
y^L_k + d^y_j + l_u(j) \sin\alpha_j \cos\beta_j \\
z^L_k + d^z_j + l_u(j) \sin\beta_j
\end{bmatrix}
$$
where the direction angles are derived from the APF force vector: \(\alpha_j = \arctan2(F^y_j, F^x_j)\) and \(\beta_j = \arcsin(F^z_j / \|\mathbf{\tilde{F}}^{af}_j\|_2)\). This ensures followers dynamically adhere to the formation geometry while reacting to the environment, a crucial capability for a resilient formation drone light show.
4. Simulation Results and Performance Analysis
The proposed hybrid method was tested in a 3D environment with multiple cylindrical obstacles. A formation of one leader and five followers in a triangular pattern was tasked with navigating from start to goal. The performance was compared against a baseline where both leader and followers used traditional APF. The key APF parameters used were: \(l_0=0.8\), \(d^{obs}_o=5\), \(k_{atto}=2\), \(k_{repo}=20\).
The results demonstrate the effectiveness of our approach for cohesive formation drone light show path planning. The DDQN-generated leader path provided a globally efficient guide. More importantly, followers using the enhanced APF exhibited markedly superior performance compared to those using traditional APF.
| UAV | Method | Path Length \(L\) (m) | Path Smoothness \(S\) (°/m) | Improvement in \(L\) | Improvement in \(S\) |
|---|---|---|---|---|---|
| Follower 1 | Traditional APF | 132.7 | 14.8 | ~21% (avg) | ~84% (avg) |
| Follower 1 | Enhanced APF (Ours) | 114.2 | 2.5 | ||
| Follower 2 | Traditional APF | 141.5 | 25.1 | ||
| Follower 2 | Enhanced APF (Ours) | 114.6 | 2.3 | ||
| Follower 3 | Traditional APF | 129.8 | 13.2 | ||
| Follower 3 | Enhanced APF (Ours) | 114.1 | 2.3 |
The enhanced APF successfully mitigated the path oscillations common in the traditional method. The adaptive parameters allowed followers to converge efficiently along the virtual leader’s track, maintaining tight formation keeping (\(d^u_{j,m} > d^u_o\) at all times) while seamlessly avoiding all obstacles. The average path length for the formation was 114m, and the average smoothness was 2.3°/m, meeting the stringent requirements for a fluid and visually appealing formation drone light show.
5. Conclusion
This paper presents a robust hybrid framework for 3D path planning in multi-UAV formations, with direct applicability to complex formation drone light show choreography. By integrating a DDQN-based global planner for the virtual leader and a novel enhanced APF with adaptive parameters for followers, the method effectively addresses the limitations of traditional approaches. Key contributions include:
- The use of DDQN with PER to generate a globally near-optimal, smooth guide path, resolving the target non-reachability and local minima issues of APF.
- The design of adaptive attractive/repulsive gains and step-size mechanisms within the APF framework, which significantly suppress path oscillations and improve convergence speed and smoothness for follower trajectories.
- A virtual leader-follower control scheme that leverages the above components to achieve safe, efficient, and cohesive formation navigation in obstacle-dense environments.
Simulation results confirm that the framework enables a drone formation to reach its target while perfectly avoiding obstacles and inter-agent collisions. The resulting paths are both short and exceptionally smooth, critical for the energy efficiency and visual quality of a large-scale formation drone light show. Future work will focus on extending the framework to dynamic environments and integrating multi-agent reinforcement learning for fully decentralized, scalable coordination in even more ambitious aerial displays.