In recent years, the application of unmanned aerial vehicles (UAVs) has expanded significantly, particularly in entertainment domains such as formation drone light shows. These shows involve multiple drones coordinating to create dynamic aerial patterns, offering mesmerizing visual experiences. However, controlling large-scale drone swarms for precise formation drone light show performances poses substantial challenges, including collision avoidance, formation maintenance, and adaptability to dynamic environments. Traditional methods, like leader-follower or virtual structure approaches, often struggle with scalability and real-time adjustments. As a researcher in this field, I have explored advanced reinforcement learning techniques to address these issues. In this article, I present a novel approach based on the MIX-MAPPO algorithm, which leverages the Menger sponge fractal structure and attention mechanisms to enhance formation control for drone light shows. This method not only simplifies the modeling process but also improves convergence speed and performance, making it ideal for complex formation drone light show scenarios.
The core of our work revolves around designing a formation model inspired by the Menger sponge fractal. This structure exhibits self-similarity, allowing for scalable and efficient formation patterns essential for large-scale formation drone light show performances. By abstracting drones as intelligent agents, we integrate multi-agent reinforcement learning to enable autonomous coordination. The MIX-MAPPO algorithm, an extension of the Multi-Agent Proximal Policy Optimization (MAPPO), incorporates grouping strategies and attention mechanisms to optimize learning. In the following sections, I will detail the formation model, algorithmic framework, experimental results, and applications in formation drone light show contexts. Throughout, I emphasize the importance of robust control systems for creating stunning and reliable drone displays.
To begin, let’s consider the motion model for drones in a formation drone light show. Each drone is treated as a point mass in a Cartesian coordinate system, with its dynamics described by simplified equations. The position and velocity of drone \(i\) are represented as \(\mathbf{p}_i = [x_i, y_i, z_i]^T\) and \(\mathbf{v}_i = [v_{x_i}, v_{y_i}, v_{z_i}]^T\), respectively. Assuming level flight at a constant altitude \(z_i = \zeta\), the motion can be modeled as:
$$ \dot{\mathbf{p}}_i = \mathbf{v}_i, \quad \dot{\mathbf{v}}_i = \mathbf{u}_i $$
where \(\mathbf{u}_i = [u_{x_i}, u_{y_i}]^T\) is the control input in the X and Y directions. This simplification allows us to focus on formation control without delving into individual drone aerodynamics, which is crucial for managing hundreds of drones in a formation drone light show.
The formation structure is based on the Menger sponge fractal, known for its self-similar properties. This enables hierarchical clustering of drones, starting from a basic unit of five drones: one leader and four followers arranged in a square-like pattern. This unit forms the foundation for larger formations, mirroring the fractal’s iterative nature. For a formation drone light show, such structures facilitate easy expansion and pattern diversity. The first-order formation, as shown in the image below, consists of a central leader drone surrounded by four followers at a distance \(d_{ij}\), with a safety radius \(d_{safe}\). This configuration protects the leader and ensures stability during performances.
In a formation drone light show, visual appeal relies on precise geometric patterns. The Menger sponge structure allows us to scale up by treating each first-order formation as a single entity. The second-order formation comprises four first-order units arranged similarly, maintaining communication links within and between leaders. This hierarchical approach supports both centralized control within units and distributed coordination across the swarm, essential for synchronized movements in large-scale formation drone light show displays. The Laplacian matrix for the first-order formation, representing communication topology, is given by:
$$ \mathbf{L} = \mathbf{D} – \mathbf{A} = \begin{bmatrix}
4 & -1 & -1 & -1 & -1 \\
-1 & 1 & 0 & 0 & 0 \\
-1 & 0 & 1 & 0 & 0 \\
-1 & 0 & 0 & 1 & 0 \\
-1 & 0 & 0 & 0 & 1
\end{bmatrix} $$
This matrix indicates connectivity among the five drones, with a rank of 4, enabling efficient information exchange. For higher-order formations, we iterate this process, leveraging self-similarity to manage complexity. In a formation drone light show, this means we can easily design intricate patterns like spirals or cascading waves by grouping drones into fractal-based clusters.
To control these formations, we employ the MIX-MAPPO algorithm, which combines MAPPO with attention mechanisms and grouping strategies. Traditional MAPPO suffers from slow convergence and limited adaptability as the number of agents increases—a critical issue for formation drone light shows involving dozens or hundreds of drones. Our MIX-MAPPO addresses this by grouping drones based on the Menger sponge structure and using attention to weight inputs. Specifically, follower drones utilize a critic network with attention, focusing on states and actions within their sub-formation, while leader drones use a simpler PPO-based critic network. This reduces input dimensionality and accelerates learning.
The critic network for followers incorporates an attention layer that assigns weights to the aggregated states and actions of drones in the same group. If we denote the group for drone \(i\) as \(G_{X_T(i)}\), where \(X_T\) represents heterogeneous information types (e.g., search, transport, operation), the value function becomes:
$$ Q^{\mu_i}(\mathbf{x}_i, G_{X_T(i)}) $$
Here, \(\mathbf{x}_i\) is the global state, and the group-based aggregation simplifies the critic’s input. The loss function for updating parameters \(\theta_i\) is:
$$ L(\theta_i) = \frac{1}{S} \sum_{i=1}^{S} \left[ \left( Q^{\mu_i}(\mathbf{x}_i, G_{X_T(i)}) – y \right)^2 \right] $$
with target \(y\) defined as:
$$ y = r_i + \gamma Q^{\mu’_i}(\mathbf{x}_i, G_{X_T(i)}) \big|_{\mathbf{a}’_j = \mu’_j(\mathbf{o}_j)} $$
where \(r_i\) is the reward, \(\gamma\) is the discount factor, and \(\mathbf{a}’_j\) are actions from target policies. The policy gradient update is:
$$ \nabla_{\theta_i} J(\mu_i) = \frac{1}{S} \sum_{i=1}^{S} \nabla_{\theta_i} \mu_i(\mathbf{a}_i | \mathbf{o}_i) \nabla_{\mathbf{a}_i} Q^{\mu_i}(\mathbf{x}_i, G_{X_T(i)}) $$
This grouping mechanism, inspired by the fractal structure, allows drones to prioritize relevant information, enhancing efficiency in formation drone light show tasks. For leaders, the critic network uses only its own state and action, reducing complexity. The attention mechanism in followers computes weights based on the relevance of other drones’ states, using a softmax function over query-key pairs. If we represent the attention score for follower \(i\) regarding drone \(j\) as \(\alpha_{ij}\), it is computed as:
$$ \alpha_{ij} = \frac{\exp(\mathbf{q}_i \cdot \mathbf{k}_j / \sqrt{d})}{\sum_{k=1}^{n} \exp(\mathbf{q}_i \cdot \mathbf{k}_k / \sqrt{d})} $$
where \(\mathbf{q}_i\) and \(\mathbf{k}_j\) are query and key vectors derived from states, and \(d\) is the dimensionality. This enables drones to focus on critical neighbors, improving formation coherence in dynamic formation drone light show environments.
Reward design is pivotal for training drones to achieve desired formations. In a formation drone light show, drones must reach target positions quickly, maintain formation, and avoid collisions. We define separate rewards for leaders and followers, along with collision avoidance and environmental rewards. For leader drone \(l\), the reward is based on distance to its target position \(\mathbf{p}_{tar}\):
$$ r_{lea} = -\min \| \mathbf{p}_{lea} – \mathbf{p}_{tar} \|^2 $$
This encourages leaders to move efficiently toward their goals, which is essential for coordinating the overall formation drone light show pattern. For follower drone \(f\), the reward depends on its distance to the leader \(\mathbf{p}_l\):
$$ r_f = -\min \| \mathbf{p}_f – \mathbf{p}_l \|^2 $$
This ensures followers stay close to their leaders, maintaining sub-formation integrity. To avoid collisions, we introduce a safety distance \(d_{safe}\). The collision avoidance reward \(r_{avoid}\) is:
$$ r_{avoid} = \begin{cases}
0 & \text{if } d_{ij} \ge d_{safe} \\
-(d_{safe} – d_{ij}) & \text{if } d_{ij} < d_{safe} \\
-\alpha_{collision} & \text{if collision occurs}
\end{cases} $$
where \(d_{ij}\) is the distance between drones \(i\) and \(j\), and \(\alpha_{collision}\) is a penalty constant. This is crucial for formation drone light shows, where close proximity can lead to accidents. Additionally, we simulate environmental influences, such as wind or magnetic fields, with a reward that attracts drones to the origin:
$$ r_{env} = -\min \left( 0.1 \times \| \mathbf{p}_{lea} – \mathbf{p}_0 \|^2 \right) $$
where \(\mathbf{p}_0 = [0, 0, \zeta]^T\) is the origin. This tests the algorithm’s robustness in adverse conditions, ensuring reliable performance during outdoor formation drone light show events.
To validate our approach, we conducted extensive experiments comparing MIX-MAPPO with baseline algorithms like DDPG, PPO, MADDPG, and MAPPO. The training platform was based on OpenAI’s Multi-Agent Particle Environment, with hardware including a Xenon E5-2620 CPU and NVIDIA 2070 Super GPU. We focused on metrics such as reward convergence, training time, and formation completion rate, all relevant to optimizing formation drone light show performances. The key hyperparameters are summarized in Table 1.
| Parameter | Description | Value |
|---|---|---|
| \(\gamma\) | Discount factor for cumulative rewards | 0.95 |
| \(lr_C\) | Learning rate for critic network | 0.01 |
| \(lr_A\) | Learning rate for actor network | 0.001 |
| batch-size | Number of samples per batch | 1024 |
| num-units | Neurons in hidden layers | 128 |
| num-layer | Layers in fully connected neural network | 4 |
| max-episode-size | Maximum steps per episode | 100 |
| num-episodes | Total training episodes | 8000 |
| \(M\) | Experience replay buffer size | \(1 \times 10^6\) |
The reward curves over episodes, as shown in our experiments, indicate that MIX-MAPPO achieves higher rewards and faster convergence compared to other algorithms. For formation drone light show applications, this means quicker learning of complex patterns and better stability. Specifically, the average reward per episode for MIX-MAPPO reached around -50 after 2000 episodes, while MAPPO and MADDPG required over 3000 episodes to achieve similar levels. DDPG and PPO, being single-agent methods, performed poorly in multi-agent settings, with rewards stagnating at -200 or lower. This highlights the superiority of multi-agent approaches for formation drone light show control.
Training time is another critical factor, especially for deploying algorithms in real-time formation drone light show systems. Table 2 compares the time taken by each algorithm to complete 8000 episodes.
| Algorithm | Training Time (hours) |
|---|---|
| DDPG | 4.21 |
| PPO | 3.87 |
| MADDPG | 4.93 |
| MAPPO | 4.88 |
| MIX-MAPPO | 3.59 |
MIX-MAPPO required the least time due to its grouping and attention mechanisms, which reduce computational overhead. This efficiency is vital for rapid prototyping and updates in formation drone light show productions, where schedules are tight. In contrast, MADDPG and MAPPO took longer because of their higher complexity in handling multi-agent interactions.
Formation completion rate (CR) measures the percentage of drones reaching their target positions in a formation drone light show scenario. We define CR as:
$$ CR = \frac{m}{n} \times 100\% $$
where \(m\) is the number of drones at desired positions, and \(n\) is the total number. Over 100 experimental runs, MIX-MAPPO achieved an average CR of 97%, outperforming other algorithms as summarized in Table 3.
| Algorithm | Completion Rate (CR) (%) |
|---|---|
| DDPG | 37 |
| PPO | 41 |
| MADDPG | 79 |
| MAPPO | 91 |
| MIX-MAPPO | 97 |
This high completion rate ensures that formation drone light show patterns are executed accurately, with minimal drones out of position. The improvement stems from MIX-MAPPO’s ability to handle heterogeneous information and focus on relevant interactions, which is essential when drones have different roles (e.g., leaders vs. followers) in a formation drone light show.
We also conducted motion capture experiments to validate the practical feasibility of MIX-MAPPO for formation drone light show performances. Using five physical drones, we deployed the trained policy to control takeoff, formation assembly, movement, and landing. The drones successfully formed and maintained the Menger sponge-based pattern, with follower distances stabilizing around the leader. This demonstrates that MIX-MAPPO can be directly applied to real-world formation drone light show systems, ensuring robust and synchronized displays. The attention mechanism allowed drones to adapt to minor disturbances, such as wind gusts, by adjusting weights in real-time—a key advantage for outdoor formation drone light show events.
In terms of algorithmic details, the MIX-MAPPO algorithm integrates the Menger sponge fractal into the training process. Drones are grouped according to the fractal hierarchy, with each group sharing a critic network. The attention mechanism computes importance scores for neighboring drones, enabling selective information processing. For a formation drone light show, this means drones can prioritize cues from nearby drones in the same sub-formation, reducing noise and improving coordination. The overall algorithm flow is as follows: initialize the environment and group drones based on fractal structure; for each episode, drones select actions using their policy networks; store experiences in a replay buffer; update networks using sampled batches with attention-weighted inputs; and repeat until convergence. This process is efficient for large-scale formation drone light show training, as it scales linearly with the number of groups rather than individual drones.
To further illustrate the mathematical formulation, consider the state representation for a drone in a formation drone light show. The state \(\mathbf{o}_i\) includes its position \(\mathbf{p}_i\), velocity \(\mathbf{v}_i\), and relative positions to neighbors in its group. With attention, the aggregated state for follower \(i\) is:
$$ \mathbf{h}_i = \sum_{j \in G_{X_T(i)}} \alpha_{ij} \cdot \mathbf{o}_j $$
where \(\alpha_{ij}\) is the attention weight. This aggregation is fed into the critic network to estimate the value function. The actor network, on the other hand, uses the local state \(\mathbf{o}_i\) to output action \(\mathbf{a}_i\). This decoupling simplifies learning and enhances performance in dynamic formation drone light show environments.
In addition to simulation, we explored scalability for massive formation drone light show displays. By extending the Menger sponge to third-order formations (20 drones) and beyond, MIX-MAPPO maintained stable control with only a slight increase in training time. This scalability is achieved through the self-similar property: each higher-order formation reuses learned policies from lower levels. For a formation drone light show involving hundreds of drones, we can train on smaller groups and then combine them, significantly reducing computational resources. This modular approach is ideal for commercial formation drone light show companies that need to design new patterns frequently.
The reward functions were fine-tuned to balance formation accuracy and energy efficiency—a concern for battery-powered drones in formation drone light show performances. We added a small penalty for control effort:
$$ r_{effort} = -0.01 \cdot \| \mathbf{u}_i \|^2 $$
This encourages smooth movements, conserving battery life and reducing wear on drones. Combined with the collision avoidance reward, it ensures safe and sustainable formation drone light show operations over multiple shows.
Comparative analysis with other state-of-the-art methods confirms MIX-MAPPO’s advantages. For instance, in a benchmark formation drone light show task where drones must form a rotating star pattern, MIX-MAPPO achieved a success rate of 95% within 500 episodes, while MAPPO reached 88% and MADDPG 75%. The success rate is defined as the percentage of episodes where all drones complete the pattern within a time limit. This demonstrates MIX-MAPPO’s robustness for complex formation drone light show choreographies. The key factors contributing to this performance are the fractal-based grouping, which reduces state space dimensionality, and the attention mechanism, which enhances information relevance. These innovations make MIX-MAPPO a top choice for next-generation formation drone light show control systems.
Looking ahead, there are several directions for improving formation drone light show control. First, extending the Menger sponge structure to three dimensions could enable volumetric formations, adding depth to aerial displays. Second, integrating real-time trajectory planning with reinforcement learning could allow adaptive formation drone light show patterns in response to audience movements or environmental changes. Third, exploring federated learning approaches could enhance privacy and efficiency when training across multiple formation drone light show fleets. These advancements will further solidify the role of AI-driven methods in entertainment and beyond.
In conclusion, the MIX-MAPPO algorithm, grounded in fractal geometry and attention mechanisms, offers a powerful solution for formation drone light show control. It addresses scalability, convergence speed, and adaptability challenges, enabling stunning and reliable aerial performances. Through extensive experiments and real-world tests, we have shown its superiority over existing methods. As formation drone light shows continue to evolve, such intelligent control systems will be essential for pushing the boundaries of creativity and technology. I believe that our work paves the way for more immersive and dynamic formation drone light show experiences, captivating audiences worldwide.
To summarize the key equations and parameters, we present a consolidated table below. This serves as a quick reference for implementing MIX-MAPPO in formation drone light show applications.
| Component | Equation/Value | Description |
|---|---|---|
| Motion Model | \(\dot{\mathbf{p}}_i = \mathbf{v}_i, \dot{\mathbf{v}}_i = \mathbf{u}_i\) | Drone dynamics simplified for formation control |
| Leader Reward | \(r_{lea} = -\min \| \mathbf{p}_{lea} – \mathbf{p}_{tar} \|^2\) | Encourages leaders to reach target positions |
| Follower Reward | \(r_f = -\min \| \mathbf{p}_f – \mathbf{p}_l \|^2\) | Ensures followers maintain proximity to leaders |
| Collision Avoidance | \(r_{avoid} = \begin{cases} 0 & \text{if } d_{ij} \ge d_{safe} \\ -(d_{safe} – d_{ij}) & \text{if } d_{ij} < d_{safe} \\ -\alpha_{collision} & \text{if collision} \end{cases}\) | Penalizes unsafe distances to prevent crashes |
| Attention Weight | \(\alpha_{ij} = \frac{\exp(\mathbf{q}_i \cdot \mathbf{k}_j / \sqrt{d})}{\sum_{k} \exp(\mathbf{q}_i \cdot \mathbf{k}_k / \sqrt{d})}\) | Computes relevance of neighbor \(j\) to drone \(i\) |
| Group Value Function | \(Q^{\mu_i}(\mathbf{x}_i, G_{X_T(i)})\) | Critic network output for drone \(i\) in group \(G\) |
| Discount Factor | \(\gamma = 0.95\) | Balances immediate and future rewards |
| Safety Distance | \(d_{safe} = 2.0 \, \text{m}\) | Minimum allowed distance between drones |
This comprehensive framework underscores the potential of MIX-MAPPO for revolutionizing formation drone light show technology. By continuously refining these methods, we can unlock new possibilities for artistic expression and technical innovation in aerial displays.