In recent years, the proliferation of unmanned aerial vehicles (UAVs) has posed significant challenges in both military and civilian domains. As a researcher in the field of defense systems, I have observed that traditional anti-aircraft weapons are often inefficient and costly for countering low-altitude, small, and slow UAVs. To address this issue, I propose a novel compound anti-UAV approach that integrates multiple types of interception equipment. This method leverages deep reinforcement learning and evolutionary algorithms to dynamically allocate tasks, ensuring effective suppression of UAV threats. In this article, I will detail the system modeling, algorithm design, and simulation validation of this approach, emphasizing the keyword “anti-UAV” throughout the discussion. The core innovation lies in combining deep Q-network (DQN) with optimization techniques to solve dynamic task assignment problems in anti-UAV systems.
The increasing use of UAVs in applications such as aerial photography, agriculture, logistics, and surveillance has heightened the need for robust anti-UAV defenses. However, existing anti-UAV systems often rely on single-type interception equipment, which lacks adaptability in diverse scenarios. My work focuses on developing a composite anti-UAV system that utilizes laser weapons, net-based interceptors, and radio frequency jammers. By integrating these equipment types, the system can achieve comprehensive coverage and enhanced interception capabilities. The task assignment problem in such a system is complex due to dynamic UAV movements and resource constraints. Traditional multi-objective optimization algorithms fail to handle these dynamics effectively, prompting the use of deep reinforcement learning for initial decision-making.
I begin by modeling the anti-UAV system as a Markov decision process (MDP), where the state represents UAV positions, actions correspond to interception decisions, and rewards guide learning. The MDP framework allows the system to adapt to changing environments in real-time. Unlike static optimization models, this approach considers sequential decision-making, which is crucial for anti-UAV operations. The state space S is defined by the coordinates of UAVs in a grid-based defense area, such that $S = \{(x_1, y_1), (x_2, y_2), \dots, (x_n, y_n)\}$, where n is the number of UAVs. Actions A are binary, indicating whether an interception equipment engages a UAV (1) or remains idle (0). The transition probability P and reward R are designed to maximize interception success while minimizing resource wastage. The reward function is defined as:
$$R(s, a) =
\begin{cases}
1 & \text{if UAV is within range and equipment engages} \\
-1 & \text{if UAV is within range and equipment does not engage} \\
-1 & \text{if UAV is out of range and equipment engages} \\
1 & \text{if UAV is out of range and equipment does not engage}
\end{cases}$$
This reward structure encourages precise engagements, which is essential for efficient anti-UAV defense. To solve the MDP, I employ an improved DQN algorithm that eliminates overestimation of Q-values by using the current state for prediction rather than the next state. The Q-value update formula is modified as follows:
$$Q_e(s_t, a_t) = Q_e(s_t, a_t) + \alpha \left[ r + \gamma \max_{a_{t+1}} Q_t(s_t, a_t) – Q_e(s_t, a_t) \right]$$
Here, $Q_e$ is the evaluation network, $Q_t$ is the target network, $\alpha$ is the learning rate, and $\gamma$ is the discount factor. This improvement accelerates convergence and enhances learning efficiency, making it suitable for anti-UAV applications where rapid decision-making is critical. The DQN module outputs an initial task assignment matrix X, where $X_{i,j} = 1$ if equipment j is assigned to UAV i. This matrix serves as a suboptimal solution for further optimization.
Next, I apply an evolutionary algorithm to refine the initial assignments. The optimization model aims to maximize the overall interception probability while adhering to constraints such as equipment range and engagement limits. The fitness function for the evolutionary algorithm is based on the cumulative reward from the DQN module, ensuring alignment with the anti-UAV objectives. The optimization process involves selection, crossover, and mutation operators, which iteratively improve the assignment strategies. The final output is a set of multiple interception schemes, providing flexibility for commanders in anti-UAV operations. This hybrid approach combines the adaptability of deep reinforcement learning with the precision of evolutionary optimization, addressing the limitations of traditional methods in dynamic anti-UAV scenarios.
To validate the proposed method, I developed a simulation environment based on a realistic anti-UAV defense scenario. The protected area is an airport runway surrounded by open terrain, with a grid size of 300 meters to account for UAV speeds and equipment reaction times. Nine interception equipment units are deployed, including three lasers, three net-based interceptors, and three radio frequency jammers. Their parameters are summarized in Table 1, which highlights the diversity in range, success rate, and operational characteristics essential for compound anti-UAV systems.
| Parameter | Laser Weapon | Net Interceptor | RF Jammer |
|---|---|---|---|
| Range Radius (m) | 300–1500 | 20–300 | 0–1500 |
| Interception Probability | 0.95 | 0.8 | 0.8 |
| Horizontal Angle (°) | [0, 360] | [-140, 140] | [-180, 180] |
| Vertical Angle (°) | [0, 80] | [5, 60] | [-10, 60] |
| Reaction Time (s) | 6 | 10 | 3 |
| Re-engagement Time (s) | 6 | 2 | 0 |
The UAVs are modeled with a speed of 25 m/s, representing typical “low, slow, and small” threats in anti-UAV contexts. The simulation considers two attack scenarios: simultaneous UAV incursions and staggered incursions with time intervals. The DQN algorithm is trained with a neural network architecture comprising two hidden layers (32 and 64 neurons) and ReLU activation functions. Hyperparameters such as learning rate and discount factor are tuned to optimize performance for anti-UAV task assignment. The training process involves experience replay with a ternary memory buffer (state, action, reward), which enhances stability and convergence.
In the first scenario, six UAVs enter the defense area from different nodes. The initial assignments generated by the DQN module are compared with those from standard DQN and Double DQN algorithms. The results, shown in Table 2, demonstrate that the improved DQN achieves precise engagements without wasting resources or missing targets, which is vital for effective anti-UAV defense. For instance, the improved DQN correctly assigns equipment to UAVs within range, while other methods either over-engage or under-engage, reducing anti-UAV efficiency.
| Method | Equipment | Target T1 | Target T2 | Target T3 | Target T4 | Target T5 | Target T6 |
|---|---|---|---|---|---|---|---|
| Improved DQN | F1 | 0 | 1 | 0 | 0 | 0 | 0 |
| F2 | 0 | 0 | 0 | 0 | 1 | 0 | |
| F3 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Standard DQN | F1 | 0 | 1 | 0 | 0 | 1 | 1 |
| F2 | 0 | 0 | 0 | 0 | 1 | 0 | |
| F3 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Double DQN | F1 | 0 | 0 | 0 | 0 | 0 | 0 |
| F2 | 0 | 0 | 0 | 0 | 1 | 0 | |
| F3 | 0 | 0 | 0 | 0 | 0 | 0 |
The evolutionary algorithm then optimizes these assignments, producing multiple schemes, as detailed in Table 3. Scheme A and Scheme B offer different engagement options, allowing commanders to choose based on real-time anti-UAV strategies. This flexibility is a key advantage of the compound anti-UAV system, as it adapts to varying threat levels and operational constraints.
| Scheme | Equipment | Target T1 | Target T2 | Target T3 | Target T4 | Target T5 | Target T6 |
|---|---|---|---|---|---|---|---|
| A | F1 | 0 | 0 | 0 | 0 | 0 | 0 |
| F2 | 0 | 0 | 0 | 0 | 1 | 0 | |
| F7 | 0 | 0 | 0 | 1 | 0 | 0 | |
| B | F1 | 0 | 0 | 0 | 0 | 0 | 0 |
| F2 | 0 | 0 | 0 | 0 | 1 | 0 | |
| F8 | 0 | 0 | 0 | 1 | 0 | 0 |
In the second scenario, UAVs incursions are staggered with 5-second intervals. The improved DQN algorithm generates assignments that maximize cumulative interception probability, achieving a value of 1.6 compared to 0.96 for naive strategies. This demonstrates the efficacy of dynamic task assignment in anti-UAV systems, where timing and resource allocation are critical. The algorithm’s ability to handle temporal variations underscores its suitability for real-world anti-UAV applications.
To further analyze the algorithm’s performance, I evaluated convergence speed and solution quality using loss and reward metrics. Figure 1 illustrates the loss values over iterations for the improved DQN, standard DQN, and Double DQN. The improved DQN converges faster and more stably, with loss decreasing to near zero within 1000 epochs, while other methods exhibit fluctuations or divergence. This robustness is essential for reliable anti-UAV operations, where system stability directly impacts defense effectiveness.
The reward comparison, shown in Figure 2, reveals that the improved DQN achieves higher average rewards, indicating better task assignment decisions. The reward function is defined as the sum of immediate rewards per episode, and the improved DQN consistently outperforms others after 1500 epochs. This translates to more efficient anti-UAV engagements, reducing missed targets and resource wastage. The evolutionary optimization further enhances these rewards by refining assignments, as quantified by the fitness function $F(X) = \sum_{i=1}^{n} \sum_{j=1}^{m} X_{i,j} \cdot P_{i,j}$, where $P_{i,j}$ is the interception probability for equipment j against UAV i. This function guides the search for optimal anti-UAV strategies.
The compound anti-UAV system’s deployment model is based on a 3 km × 9 km grid, with equipment positioned to cover the airport runway. The interception ranges are modeled as circles, with lasers having the longest range (up to 1500 m), followed by RF jammers (up to 1500 m but with different engagement characteristics), and net interceptors (up to 300 m). The UAVs move according to a random walk model with five actions: stay, east, south, west, and north. The state update is governed by the equation $s_{t+1} = s_t + v \cdot \Delta t \cdot a_t$, where $v$ is UAV speed, $\Delta t$ is time step, and $a_t$ is the action vector. This simulation environment mimics real-world anti-UAV scenarios, allowing for rigorous testing.
The DQN algorithm’s training involves 2000 epochs per agent, with nine agents corresponding to the interception equipment. The experience replay buffer stores 10000 tuples, and the target network updates every 100 steps. The learning rate $\alpha$ is set to 0.001, and the discount factor $\gamma$ is 0.99. These parameters are optimized through grid search to ensure peak anti-UAV performance. The neural network’s weights are initialized using Xavier initialization, and the optimizer is Adam, which handles sparse gradients well in anti-UAV task assignment.
Moreover, the evolutionary algorithm uses a population size of 50, with crossover probability 0.8 and mutation probability 0.2. The selection operator is tournament selection, which maintains diversity in anti-UAV strategies. The algorithm runs for 100 generations, sufficient for convergence in most anti-UAV cases. The output includes Pareto-optimal solutions, providing a trade-off between interception probability and resource usage—a crucial consideration in anti-UAV system design.
In terms of computational efficiency, the improved DQN reduces training time by 50% compared to standard DQN, thanks to the modified Q-value update. This efficiency is vital for deploying anti-UAV systems in field conditions where computational resources may be limited. The hybrid approach also scales well with increasing numbers of UAVs and equipment, as the DQN module handles state-space complexity through function approximation, while the evolutionary algorithm focuses on local optimization.
The anti-UAV system’s effectiveness is further quantified by the overall interception success rate, defined as $S = \frac{\sum_{i=1}^{k} I_i}{k}$, where $k$ is the number of UAV engagements and $I_i$ is an indicator of successful interception. In simulations, the compound system achieves an average success rate of 0.92, surpassing single-equipment systems (e.g., laser-only at 0.85, net-only at 0.70, RF-only at 0.88). This highlights the synergy in compound anti-UAV approaches, where diverse equipment complements each other’s weaknesses.
To address potential challenges, such as UAV swarms or electronic countermeasures, the system incorporates adaptive reward shaping. For instance, if a UAV employs evasive maneuvers, the reward function penalizes missed engagements more heavily, encouraging the DQN to learn robust anti-UAV policies. Additionally, the evolutionary algorithm can incorporate new constraints, such as equipment damage or fuel limits, ensuring the anti-UAV system remains operational under adverse conditions.
In conclusion, the proposed method offers a significant advancement in anti-UAV technology by integrating multiple interception types and dynamic task assignment. The improved DQN algorithm provides fast, accurate decisions, while evolutionary optimization refines these decisions for maximum effectiveness. Simulation results confirm that the compound anti-UAV system outperforms traditional methods in terms of convergence, reward, and success rate. This work paves the way for more resilient and adaptable anti-UAV defenses, capable of countering evolving UAV threats in open-area scenarios. Future research will explore deep reinforcement learning with multi-agent coordination and real-time sensor integration for enhanced anti-UAV capabilities.
The mathematical foundation of the anti-UAV task assignment can be summarized with key equations. The Bellman equation for the Q-value in the improved DQN is:
$$Q^*(s, a) = \mathbb{E} \left[ r + \gamma \max_{a’} Q^*(s’, a’) \mid s, a \right]$$
However, in our modified version, we approximate this using the current state:
$$\hat{Q}(s, a) = r + \gamma \max_{a’} Q(s, a’)$$
This reduces overestimation errors common in anti-UAV applications. The loss function for training is mean squared error:
$$L(\theta) = \mathbb{E}_{(s,a,r) \sim D} \left[ \left( r + \gamma \max_{a’} Q_t(s, a’; \theta^-) – Q_e(s, a; \theta) \right)^2 \right]$$
where $\theta$ are the network parameters, and $D$ is the experience buffer. For the evolutionary algorithm, the fitness function incorporates anti-UAV-specific metrics:
$$F(X) = \sum_{i=1}^{n} \sum_{j=1}^{m} X_{i,j} \cdot P_{i,j} – \lambda \sum_{j=1}^{m} C_j$$
Here, $C_j$ is the cost of using equipment j, and $\lambda$ is a regularization parameter to balance interception success and resource consumption in anti-UAV operations. This holistic approach ensures that the system not only neutralizes threats but also does so efficiently, a cornerstone of modern anti-UAV strategy.
Ultimately, the compound anti-UAV system represents a paradigm shift from static defenses to dynamic, intelligent systems. By leveraging deep learning and optimization, it addresses the limitations of current anti-UAV technologies, offering a scalable solution for protecting critical infrastructure. As UAV threats continue to evolve, such innovative anti-UAV methods will be indispensable for maintaining security and safety in both civilian and military domains.