In this paper, we address the challenge of agile and low-complexity task scheduling for China UAV swarms in integrated reconnaissance–strike–battle damage assessment (BDA) missions. The problem involves heterogeneous unmanned aerial vehicles (UAVs) collaborating to execute a chain of operations under tight constraints including task precedence, resource compatibility, time windows, range limits, and risk. Traditional heuristic and deep reinforcement learning methods suffer from high computational complexity or insufficient real-time performance when faced with coupled constraints. We propose a novel algorithm that fuses Heterogeneous Graph Neural Networks (HGNN) with Proximal Policy Optimization (PPO). The scheduling state is represented as a Heterogeneous Disjunctive Graph (HDG), which encodes multiple entity types and multi-semantic constraints into fixed-dimensional graph embeddings. The decision process is formulated as a Markov Decision Process (MDP) under a feasible candidate set of operation–UAV assignment edges. An end-to-end policy is trained using PPO with an adaptive learning rate schedule, dynamic clipping, and entropy regularization to enhance stability and convergence. Extensive simulations demonstrate that our method significantly improves scheduling efficiency and overall mission utility while maintaining low computational overhead. Compared to genetic algorithms, PPO without graph structure, and GAT-based greedy heuristics, our approach achieves superior performance in terms of cumulative reward, flight distance cost, and risk mitigation across different problem scales. The trained policy can be directly applied to online rolling-horizon scheduling, making it well-suited for real-time China UAV swarm operations.
Modern warfare increasingly demands agile and coordinated operations from China UAV swarms. A typical scenario involves reconnaissance, strike, and BDA tasks performed against multiple targets by heterogeneous UAVs. The scheduling problem is a combinatorial optimization that must satisfy strict temporal dependencies, resource compatibility, and safety constraints. Traditional methods such as genetic algorithms (GA) can find feasible solutions but suffer from high computational cost and slow convergence, especially for large-scale swarms. Deep reinforcement learning (DRL) offers end-to-end decision-making, yet direct application of DRL often fails to capture the structural relationships among tasks and resources. Graph neural networks (GNNs) provide a natural way to encode relational information, and heterogeneous GNNs can model different entity types and interactions. In this work, we integrate HGNN with PPO to build a scheduling policy that is both efficient and robust.
We consider a scenario where a China UAV swarm is deployed to perform missions over a set of detected targets. Each target requires three sequential operations: reconnaissance (R), strike (S), and BDA (B). The swarm consists of UAVs with different capabilities (e.g., reconnaissance, strike, BDA) launched from multiple bases. Each operation has a time window, duration, and a required UAV type. UAVs have limited range and can execute only one operation at a time. The objective is to maximize the cumulative value of completed target chains while minimizing total flight distance and exposure risk. The optimization model is given by:
$$
\max \, \omega_S J_S – \omega_D J_D – \omega_R J_R
$$
subject to constraints:
- Each operation must be assigned to exactly one UAV with matching capability.
- Precedence: reconnaissance → strike → BDA for each target.
- Time windows: start time of each operation must lie within its allowed interval.
- Range: total flight distance of each UAV cannot exceed its maximum range.
- Non-overlap: a UAV cannot execute more than one operation simultaneously.
The reward terms are defined as:
$$
J_S = \sum_{i} C_{M_i} g_{M_i}, \quad J_D = \sum_{k} \frac{D_k}{L_k}, \quad J_R = \sum_{i} \sum_{k} C_{M_i} \rho_{i,k}
$$
where \(C_{M_i}\) is the target value, \(g_{M_i}\) indicates completion of the full chain, \(D_k\) is the total flight distance of UAV \(k\), \(L_k\) its maximum range, and \(\rho_{i,k}\) is the damage probability derived from target threat and exposure time.
We model the scheduling state as a Heterogeneous Disjunctive Graph (HDG). The graph contains three types of nodes: operation nodes (including dummy start and end), UAV nodes, and target-related dependency edges. Operation–UAV edges represent candidate assignments. At each decision step, we filter the edge set to only include feasible edges that satisfy all hard constraints (precedence, resource, time windows, range). This reduces the action space and ensures feasibility. The HGNN architecture consists of three components: a graph attention encoder, a graph representation learning module, and a dual-head output network. The encoder uses multi-head attention to aggregate information from neighbor nodes. Node features are transformed into a common latent space via type-specific linear mappings. The attention coefficient between node \(i\) and node \(j\) is:
$$
e_{ij} = \text{LeakyReLU}\left( \mathbf{a}^T [ \mathbf{W} \mathbf{x}_i \parallel \mathbf{W} \mathbf{x}_j ] \right)
$$
Then normalized using softmax over neighbors. Multi-head outputs are concatenated to form hidden representations. After L layers, we apply a fusion layer with dropout and ReLU, followed by global attention pooling to obtain a graph-level embedding. The policy head computes scores for each candidate edge using the concatenation of operation node embedding, UAV node embedding, and edge features. A masked softmax over the feasible action set yields the action probability distribution. The value head estimates the state value from the graph embedding via a feedforward network.
Table 1 summarizes the key hyperparameters of our HGNN encoder.
| Parameter | Value |
|---|---|
| Number of attention heads | 4 |
| Attention layer dimension | 64 |
| Hidden layer dimension | 256 |
| Dropout probability | 0.1 |
| Activation function | LeakyReLU |
| Number of GAT layers | 3 |
The training algorithm employs PPO with clipped surrogate objective, generalized advantage estimation (GAE), and adaptive learning rate scheduling. Table 2 gives the PPO parameters used.
| Parameter | Value |
|---|---|
| Discount factor \(\gamma\) | 0.99 |
| GAE parameter \(\lambda\) | 0.95 |
| Clip range \(\epsilon\) | 0.2 |
| Batch size | 64 |
| Max gradient norm | 0.5 |
| Policy loss weight | 1 |
| Value loss weight | 0.5 |
| Entropy loss weight | 0.01 |
| Learning rate initial | 0.0003 |
| Learning rate decay | halve after 50 episodes without improvement |
| Exploration rate (during training) | 0.1 (random action) |
We conducted experiments using a China UAV swarm simulation environment with scenarios of varying scale: small (12 UAVs, 2 targets), medium (30 UAVs, 5 targets), and large (200 UAVs, 30 targets). For each scenario, we compared our method (HGNN-PPO) against GA, PPO without graph structure (PPO-NOGNN), and GAT with greedy selection (GAT+greedy). Results are normalized per scale to facilitate multi-indicator comparison. Table 3 presents the normalized performance scores.
| Method | Scale | Reward | Distance cost | Risk penalty |
|---|---|---|---|---|
| HGNN-PPO | 12 | 0.85 | 1.00 | 0.90 |
| 30 | 1.00 | 1.00 | 1.00 | |
| 200 | 1.00 | 1.00 | 1.00 | |
| GA | 12 | 1.00 | 0.73 | 0.00 |
| 30 | 0.62 | 0.80 | 0.00 | |
| 200 | 0.00 | 0.71 | 0.00 | |
| PPO-NOGNN | 12 | 0.20 | 0.00 | 0.40 |
| 30 | 0.00 | 0.00 | 0.29 | |
| 200 | 0.29 | 0.00 | 0.58 | |
| GAT+greedy | 12 | 0.00 | 0.45 | 1.00 |
| 30 | 0.25 | 0.50 | 0.71 | |
| 200 | 0.43 | 0.43 | 0.74 |
The average reward curves during training for different scales (illustrated in the training loss curves) show that our method converges stably and achieves higher final reward. For the large-scale scenario, the total loss variance decreases significantly after 150 episodes, confirming training stability. The learning rate schedule is shown typically: linear warmup followed by adaptive decay.
We also recorded the computational time (in seconds) for each method. Table 4 compares the solution times.
| Scale | HGNN-PPO | GA | PPO-NOGNN | GAT+greedy |
|---|---|---|---|---|
| 12 | 0.62 | 1.85 | 0.71 | 0.48 |
| 30 | 1.74 | 6.90 | 2.10 | 1.36 |
| 200 | 1.89 | 16.50 | 2.64 | 1.47 |
In the online application experiment, we used a small scenario with two bases, 12 UAVs, and two targets. The scheduling Gantt chart (not shown by figure number) illustrates how the algorithm assigns operations over time, respecting precedence and resource constraints. The system triggers a rolling decision whenever a target’s reconnaissance time window opens. Our method generates real-time feasible assignments that achieve high mission utility while controlling distance and risk. For instance, the total reward reached 0.85 (normalized) with low distance cost and risk.
Another important aspect is the ability to handle dynamic changes. Although not explicitly tested in this paper, the event-triggered rolling framework allows re-scheduling when new targets appear or UAV status changes. Future work will incorporate systematic perturbation tests.
In conclusion, we have developed an agile low-complexity task scheduling algorithm for China UAV swarms by combining Heterogeneous Graph Neural Networks with Proximal Policy Optimization. The key contributions include: (1) a unified scheduling framework based on heterogeneous disjunctive graph representation; (2) a state-action aligned modeling where actions are operation–UAV edges and candidate feasibility is enforced by constraint pre-screening; (3) a multi-head attention-based HGNN that learns expressive node embeddings and produces action probabilities through masked softmax; (4) an adaptive PPO training mechanism with learning rate scheduling, dynamic clipping, and entropy regularization to ensure stable and efficient learning. Comprehensive simulations demonstrate that our method outperforms GA, PPO-NOGNN, and GAT+greedy in terms of reward, distance cost, and risk across multiple scales, with solution times well below those of pure optimization methods. The trained policy can be deployed directly for online rolling scheduling, making it suitable for real-time China UAV swarm missions. Future work may explore lightweight online adaptation to handle strong disturbances such as UAV failures or task changes, further enhancing robustness.