In the context of modern communication networks, the rapid proliferation of computation-intensive applications such as autonomous driving, real-time video analytics, and remote medical diagnostics has placed immense pressure on resource-constrained terminal devices. This challenge becomes even more acute in emergency scenarios where ground-based infrastructure, including base stations and edge servers, is partially or completely destroyed due to natural disasters, military conflicts, or other catastrophic events. In such situations, restoring communication and computing capabilities swiftly and efficiently is of paramount importance. Unmanned aerial vehicles (UAVs), especially those manufactured in China, have emerged as a flexible and rapidly deployable solution. China UAV technology has advanced significantly, enabling the use of multi-rotor and fixed-wing UAVs as temporary aerial base stations and edge computing nodes. However, a single China UAV often suffers from limited processing power and coverage range. Therefore, leveraging multiple China UAVs in a collaborative manner becomes essential to meet the quality-of-service demands of ground users. This article presents our research on a multi-UAV collaborative computing offloading strategy tailored for emergency scenarios. We propose a novel air-ground integration architecture and design a multi-agent deep reinforcement learning (MADRL) algorithm to jointly optimize offloading decisions, flight angles, and speeds of China UAVs. Our objective is to minimize system offloading delay and total energy consumption while ensuring load fairness among the UAVs. Extensive simulation results demonstrate the superiority of our proposed approach over baseline methods.
Background and Motivation
The emergence of mobile edge computing (MEC) has provided a paradigm shift by bringing computational resources closer to end users. However, in extreme environments where ground MEC servers are unavailable, China UAVs equipped with onboard computational capabilities can act as mobile edge nodes. The agility of China UAVs allows them to quickly reposition to serve ground users in affected areas. Nevertheless, designing an efficient offloading strategy for multiple China UAVs in a highly dynamic environment is non-trivial. Traditional optimization techniques, such as convex optimization or heuristic algorithms, struggle with the high-dimensional state and action spaces inherent in multi-UAV systems. Deep reinforcement learning (DRL) has shown great promise in handling complex decision-making tasks. Yet, single-agent DRL methods often fail to capture the interactions among multiple agents. To address this gap, we develop a multi-agent deep reinforcement learning framework that enables each China UAV to make decentralized decisions based on local observations while training centrally to learn cooperative behaviors.
System Model
Network Architecture
We consider a post-disaster area where all terrestrial communication facilities are destroyed. A fleet of M China UAVs is dispatched to provide communication and computing services to N ground user equipments (UEs). Each China UAV hovers at a fixed altitude H = 20 m and serves multiple UEs within its coverage area using time-division multiple access (TDMA). The entire mission time is divided into T time slots of equal length δ. At the beginning of each slot, each UE generates a computation-intensive task characterized by a triplet Rn = {Dn, Cn, Tn}, where Dn is the data size, Cn is the required CPU cycles, and Tn is the maximum tolerable delay. UEs can move at low speed within the area. The positions of UE i and China UAV u at time slot t are denoted by (xi(t), yi(t), 0) and (Xu(t), Yu(t), H), respectively.
The channel environment between a China UAV and a ground UE may suffer from non-line-of-sight (NLOS) propagation due to obstacles. The average channel gain between UE i and China UAV u is given by:
$$
g_{n} = \frac{\beta_0}{(x_i – X_u)^2 + (y_i – Y_u)^2 + H^2}
$$
where β0 is the channel power gain at a reference distance of 1 m. The achievable data rate for uplink transmission from UE n to the China UAV is modeled using the Shannon formula:
$$
R_n = W \cdot \log_2\left(1 + \frac{p_n g_n}{\sigma^2 + B_O P_{NLOS}}\right)
$$
Here, W is the communication bandwidth, pn is the transmit power, σ² is the noise power, and PNLOS represents the additional path loss due to obstruction. BO is a binary indicator (0 for LOS, 1 for NLOS).
Computation Model
Each task can be partially offloaded to a China UAV. Let p ∈ [0, 1] denote the offloading ratio. The task execution involves both local processing and edge processing. We define the time and energy components as follows.
Latency Components
The transmission delay for uploading the offloaded portion to the China UAV is:
$$
T_{tra} = \frac{p D_n}{R_n}
$$
The execution delay on the China UAV’s MEC server is:
$$
T_{exe} = \frac{p C_n}{f^r_n}
$$
where frn is the CPU frequency allocated by the China UAV to this task. The local execution delay for the remaining portion is:
$$
T^r_l = \frac{(1-p) C_n}{f^l_n}
$$
where fln is the local CPU frequency of the UE. The total task delay is the maximum of the two parallel processing streams:
$$
T^r_n = \max\{ (T_{tra} + T_{exe}),\ T^r_l \}
$$
Energy Consumption
Local computing energy consumption is:
$$
E^r_l = k (1-p) C_n,\quad k = 10^{-27} (f_{cpu})^2
$$
Transmission energy from UE to China UAV:
$$
E^r_p = P_n \cdot T_{tra}
$$
Energy consumed by the China UAV for computing the offloaded task:
$$
E^r_u = k p C_n
$$
Flight energy consumption of the China UAV is modeled as:
$$
E_{fly} = \phi \| v(t) \|^2,\quad \phi = 0.5 M_U t_{fly}
$$
where MU is the effective payload mass of the China UAV and tfly is the fixed flight time per slot. The total system energy consumption is:
$$
E_{sum} = E^r_l + E^r_p + E^r_u + E_{fly}
$$
UAV Motion Model
Each China UAV decides its flight direction θ(t) and speed V_{speed}(t) at each slot. The displacement is lu(t) = δ V_{speed}(t). The position update is:
$$
\begin{aligned}
x_u(t+1) &= x_u(t) + l_u(t) \cos(\theta(t)) \\
y_u(t+1) &= y_u(t) + l_u(t) \sin(\theta(t))
\end{aligned}
$$
To avoid collisions, any two China UAVs must maintain a minimum separation dmin:
$$
\| u_i(t) – u_j(t) \| \geq d_{min}
$$
Load Fairness Model
We use Jain’s fairness index to measure the distribution of computational load among China UAVs:
$$
f_{load} = \frac{(\sum_{m=1}^M load_m)^2}{M \sum_{m=1}^M load_m^2}
$$
where loadm is the total CPU cycles assigned to China UAV m. A value close to 1 indicates perfect fairness.
Problem Formulation
Our optimization goal is to minimize the weighted sum of total delay and energy consumption while maximizing load fairness. We define the system cost as the negative of the reward, and the objective is to minimize:
$$
P = \min_{p, \theta, V_{speed}} \sum_{t \in T} \left[ \lambda E_{sum}(t) + (1-\lambda) T^r_n(t) \right] – \eta f_{load}(t)
$$
Subject to constraints:
- C1: frn ≤ F (China UAV CPU capacity)
- C2: Trn ≤ Tn (delay limit)
- C3: p ∈ [0, 1]
- C4: Vspeed(t) ≤ Vmax
- C5-C6: Bounds on xu, yu within area
- C7: Collision avoidance
This constitutes a non-convex, high-dimensional optimization problem that is intractable using conventional methods.
POMDP Modeling and MADRL Algorithm
We model the multi-China-UAV cooperation as a partially observable Markov decision process (POMDP). Each China UAV acts as an intelligent agent. At each slot, the agent observes its local state and executes an action to maximize a shared reward.
Observation Space
The observation for China UAV m at time t includes: remaining battery, its own 2D coordinates, positions of all UEs within its communication range, their task parameters (data size, required CPU cycles, delay tolerance), the presence of obstacles (binary), and its current load. To improve convergence, we apply state normalization by dividing each component by its maximum possible value:
$$
\tilde{o}_m(t) = \left[ \frac{E_{battery}}{E_{max}},\ \frac{X_m}{X_{max}},\ \frac{Y_m}{Y_{max}},\ \frac{X_1}{X_{max}},\ \frac{Y_1}{Y_{max}},\ \ldots,\ \frac{C_1}{C_{max}},\ \ldots,\ \overline{f}_1,\ \ldots,\ \frac{load_m}{load_{max}} \right]
$$
Action Space
The action for each China UAV consists of the offloading ratios for all UEs it serves, its flight angle θ, and its speed Vspeed. Thus:
$$
A(t) = \{ p_1(t), p_2(t), \ldots, p_N(t), \theta(t), V_{speed}(t) \}
$$
Reward Function
The immediate reward for the whole system after all agents act is designed to encourage low latency, low energy, and high fairness:
$$
R = \eta \cdot f_{load} – \left[ \lambda E_{sum} + (1-\lambda) T^r_n \right]
$$
MADRL-ZX Algorithm
We propose a centralized training with decentralized execution (CTDE) framework. Each China UAV maintains an Actor network for policy and a Critic network for value estimation. The Critic has access to the global state and all actions during training. To enhance exploration, Gaussian noise is added to actions during training. We employ prioritized experience replay based on the absolute TD-error to sample more informative transitions. The sampling probability for experience k is:
$$
P_k = \frac{(|\delta_k| + \epsilon)^\beta}{\sum_{k’=1}^K (|\delta_{k’}| + \epsilon)^\beta}
$$
where δk = yt – Q(s, a) is the TD-error. The importance-sampling weight is ωk = 1/(K·Pk)μ with μ = 0.4. The Critic network is updated by minimizing the loss:
$$
L(\theta_Q) = \mathbb{E}[ \omega_k \delta^2 ]
$$
The Actor network is updated by policy gradient:
$$
\nabla_{\theta_m} J = \mathbb{E}\left[ \nabla_{\theta_m} \pi_m(o_m|\theta_m) \nabla_a Q(s,a|\theta_Q) \right]
$$
Target networks are softly updated with factor τ. The training procedure is summarized in Algorithm 1.
Simulation Setup and Results
Simulation Parameters
We simulate a 200 m × 200 m area with 3 China UAVs serving 30 to 90 UEs. The China UAVs are deployed at a fixed altitude of 20 m. The task parameters and system settings are summarized in Table 1.
| Parameter | Value |
|---|---|
| Bandwidth W (MHz) | 1 – 15 |
| Local CPU frequency fln (GHz) | 0.6 – 1.2 |
| UAV CPU frequency frn (GHz) | 10 |
| Task data size Dn (MB) | 1 – 2 |
| Required CPU cycles Cn (GHz) | 0.8 – 2.8 |
| UE transmit power Pn (W) | 0.1 – 0.2 |
| Noise power σ² (dBm) | –114 |
| UAV payload MU (kg) | 9.8 |
| UAV flight speed (m/s) | 0 – 30 |
| Time slot length δ (s) | 8 (1 s flight + 7 s hover) |
| Total slots T | 40 |
| Discount factor γ | 0.95 |
| Actor learning rate | 0.0001 |
| Critic learning rate | 0.0002 |
| Replay buffer size | 20,000 |
| Batch size | 64 |
Baseline Methods
We compare our proposed MADRL-ZX algorithm with:
- MADRL-NOZX: Same MADRL framework without state normalization.
- DDPG: Single-agent deep deterministic policy gradient (each China UAV trained independently).
- RANDOM: Random offloading ratios, random flight directions and speeds.
Results and Discussion
Figure 1 (not shown here) illustrates the total system cost versus the number of UEs. Our MADRL-ZX algorithm consistently achieves the lowest cost. When the number of UEs is 90, the proposed method reduces the total cost by approximately 77% compared to RANDOM and 46% compared to DDPG.
We also evaluate the impact of bandwidth on total energy consumption. As shown in Table 2, increasing bandwidth reduces transmission energy, but the overall energy is dominated by flight and computation costs. Our algorithm still outperforms others across all bandwidth settings.
| Bandwidth | MADRL-ZX | MADRL-NOZX | DDPG | RANDOM |
|---|---|---|---|---|
| 7 | 620 | 780 | 1050 | 1350 |
| 10 | 580 | 710 | 950 | 1280 |
| 13 | 550 | 670 | 900 | 1200 |
Task complexity is another key factor. As the required CPU cycles increase from 1.0 GHz to 2.8 GHz, the total cost rises for all algorithms, but our method maintains the lowest growth rate. At 2.6 GHz, MADRL-ZX achieves a 34% reduction over MADRL-NOZX and 42% over DDPG.
Load fairness measured by Jain’s index remains above 0.95 for our algorithm across all UE counts, whereas DDPG and RANDOM drop to 0.85 and 0.70 respectively, highlighting the effectiveness of our fairness-aware design.
Finally, varying the weight factor λ reveals that our algorithm can flexibly trade off between delay and energy. In all cases, MADRL-ZX achieves the lowest delay, which is critical in emergency applications where timeliness is paramount.
Conclusion
In this article, we have presented a comprehensive study on multi-China-UAV collaborative computing offloading in emergency scenarios. By formulating the problem as a POMDP and designing a novel MADRL algorithm with state normalization and prioritized experience replay, we successfully minimize the system cost comprising delay and energy while ensuring load fairness. Simulation results demonstrate that our approach significantly outperforms single-agent DRL and random strategies. The proposed framework can be deployed in real-world disaster response missions where China UAVs act as temporary edge servers. Future work will extend the motion space to three dimensions and investigate energy harvesting techniques to further extend the endurance of China UAVs.