In this study, I investigate the critical problem of multi-drone collaborative computing offloading in emergency scenarios, such as sudden network infrastructure failures caused by natural disasters or accidents. The core challenge is to rapidly deploy unmanned aerial vehicles (UAVs) as temporary base stations and edge computing nodes to restore communication and computational services for ground users. Single drone technology faces significant limitations in processing capacity and coverage range, making multi-drone collaboration essential. I propose an air-ground integrated architecture and design a novel multi-agent deep reinforcement learning (MADRL) algorithm to jointly optimize offloading decisions, flight angles, and speeds of drones. The goal is to minimize system offloading delay and total energy consumption while ensuring load fairness among drones. Simulation results demonstrate that my approach reduces total system cost by approximately 77% compared to random processing and 46% compared to the Deep Deterministic Policy Gradient (DDPG) algorithm. This work highlights the superiority of drone technology in extreme edge computing environments.
Introduction
With the advent of 5G, computation-intensive applications such as autonomous driving and real-time identity recognition have significantly improved daily life. However, the exponential growth of data generated by terminal devices poses severe challenges to resource-constrained devices. Mobile Edge Computing (MEC) has emerged as a paradigm that addresses the shortcomings of both local processing and cloud computing by offloading tasks to nearby servers. However, in remote mountainous areas or during emergencies where ground communication infrastructure is destroyed, reliable MEC services become difficult to obtain. Drone technology offers a solution due to its flexibility, rapid deployment, and wide coverage. Multiple drones working collaboratively can handle more complex tasks, but their coordination in dynamic and uncertain environments remains a challenge. In this paper, I propose a novel offloading strategy that leverages drone technology to achieve efficient computation in emergency scenarios.
Related Work
Previous research on UAV-assisted MEC has primarily focused on optimizing latency, energy consumption, throughput, and quality of service using traditional optimization methods or heuristic algorithms. For example, some studies applied successive convex approximation to minimize UAV energy consumption while meeting service requirements. Others proposed alternating optimization algorithms to jointly schedule computational resources, bandwidth, and trajectories. However, these methods struggle with high-dimensional, dynamic, and complex state spaces. Deep reinforcement learning (DRL) has emerged as a powerful tool for such problems, enabling agents to learn optimal policies through interaction with the environment. Some works used DRL to maximize energy efficiency or minimize user energy consumption by optimizing drone trajectories and offloading decisions. Nevertheless, several limitations exist: (1) most studies assumed line-of-sight channels, ignoring obstacles; (2) ground user mobility was often neglected; (3) load fairness among drones was rarely considered; (4) flight energy consumption of drones was not fully modeled. My work addresses these gaps by incorporating non-line-of-sight channels, user mobility, fairness constraints, and comprehensive energy modeling, all while leveraging advanced drone technology.
System Model
Network Model
I consider an emergency scenario where ground communication facilities have been destroyed. A fleet of M drones (indexed by {1,2,…,M}) is deployed to provide communication and computation services to N ground user equipment (UEs) (indexed by {1,2,…,N}). Time is divided into n time slots {t₁, t₂, …, tₙ} of equal duration δ. Each UE generates a computation-intensive task at each time slot. The area is divided into non-overlapping sub-regions, with each drone hovering over a sub-region and serving multiple UEs using time-division multiple access (TDMA). UEs can move at low speeds. Each task is represented by a triplet Rₙ = {Dₙ, Cₙ, Tₙ}, where Dₙ is data size, Cₙ is required CPU cycles, and Tₙ is the maximum tolerable delay.
The position of UE i at a given slot is (xᵢ, yᵢ, 0), while drone u is at (Xᵤ, Yᵤ, H). The average channel gain gₙ between UE and drone is given by:
$$ g_{n} = \frac{\beta_0}{d_k^2} = \frac{\beta_0}{(x_i – X_u)^2 + (y_i – Y_u)^2 + H^2} $$
where β₀ is the channel power gain at 1 m reference distance, and dₖ is the Euclidean distance. Due to obstacles, the communication rate between UE and drone is:
$$ R_n = W \cdot \log_2\left(1 + \frac{p_n g_n}{\sigma^2 + B_O P_{\text{NLOS}}}\right) $$
Here, W is the bandwidth, pₙ is transmission power, σ² is noise power, P_{\text{NLOS}} is the non-line-of-sight loss, and B_O is a binary indicator for blockage (0: clear, 1: blocked).
Computing Model
I adopt a partial offloading approach. The fraction p (0 ≤ p ≤ 1) of the task is offloaded to a drone, while the remainder (1-p) is processed locally. The transmission delay for offloading is:
$$ T_{tra} = \frac{p D_n}{R_n} $$
The execution delay on the drone:
$$ T_{exe} = \frac{p C_n}{f_n^r} $$
where fₙʳ is the drone’s MEC computational capability. The local execution time:
$$ T_n^{rl} = \frac{(1-p) C_n}{f_n^l} $$
where fₙˡ is the UE’s CPU frequency. Since the downloaded result is typically small, the total delay is:
$$ T_n^r = \max\left\{ (T_{tra} + T_{exe}), T_n^{rl} \right\} $$
Energy Consumption Model
Energy consumption includes local computation, transmission, drone computation, and drone flight. Local computation energy:
$$ E_n^{rl} = k (1-p) C_n, \quad k = 10^{-27} (f_{cpu})^2 $$
Transmission energy from UE to drone:
$$ E_n^{rp} = P_n T_{tra} $$
Drone computation energy:
$$ E_n^{ru} = k p C_n $$
Drone flight energy, following [14]:
$$ E_{fly} = \phi \| \nu(i) \|^2, \quad \phi = 0.5 M_U t_{fly} $$
where M_U is the drone payload and t_{fly} is fixed flight time. The total system energy is:
$$ E_{sum} = E_n^{rl} + E_n^{rp} + E_n^{ru} + E_{fly} $$
Drone Motion Model
Drone u at time slot t has horizontal coordinates (Xᵤ(t), Yᵤ(t)). With flight angle θ(t) and speed V_speed(t), the distance traveled is lᵤ(t) = δ V_speed(t). The next coordinates:
$$ x_u(t+1) = x_u(t) + l_u(t) \cos(\theta(t)) $$
$$ y_u(t+1) = y_u(t) + l_u(t) \sin(\theta(t)) $$
To avoid collisions, the distance between any two drones must satisfy:
$$ \| u_i(t) – u_j(t) \| \ge d_{min} $$
Fairness Load Model
I use Jain’s fairness index to measure load balance among drones:
$$ f_{load} = \frac{(\sum_{m=1}^M \text{load}_m)^2}{M (\sum_{m=1}^M \text{load}_m^2)} $$
where load_m is the computational load on drone m. A value close to 1 indicates perfect fairness.
Problem Formulation
My objective is to minimize the total delay and total energy consumption while maximizing load fairness. The optimization problem is formulated as:
$$ P = \max \sum_{t \in T} \frac{f_{load}(t)}{\lambda E_{sum}(t) + (1-\lambda) T_n^r(t)} $$
subject to constraints:
- C1: $f_n^r \le F$ (assigned CPU frequency not exceeding max)
- C2: $T_n^r \le T_n$ (task completion within deadline)
- C3: $p \in [0,1]$ (offloading fraction)
- C4: $V_{speed}(t) \le V_{max}$ (speed limit)
- C5: $X_u(t) \le X_{max}$
- C6: $Y_u(t) \le Y_{max}$ (area bounds)
- C7: $\| u_i(t) – u_j(t) \| \ge d_{min}$ (collision avoidance)
where λ is a weighting factor. This is a non-convex, NP-hard problem due to the coupling of continuous and discrete variables. I adopt a multi-agent deep reinforcement learning approach to solve it.
Reinforcement Learning Algorithm Design
Partially Observable Markov Decision Process (POMDP)
I model the problem as a POMDP where each drone is an agent. The observation space for agent m at time t includes:
$$ o_m(t) = \{ E_{battery}(t), X_m(t), Y_m(t), X_1(t), Y_1(t), \dots, X_n(t), Y_n(t), C_1(t), \dots, C_n(t), f_1(t), \dots, f_n(t), \text{load}_n \} $$
Here, fₙ(t) indicates whether UE n is blocked. The global state S(t) = {o₁(t), …, oₘ(t)}. The action space for each agent:
$$ A(t) = \{ p_1(t), \dots, p_N(t), \theta(t), V_{speed}(t) \} $$
The reward function is designed to encourage fairness and minimize cost:
$$ R = \eta \cdot f_{load} – [\lambda E_{sum}(t) + (1-\lambda) T_n^r(t)] $$
where η is a scaling factor. The objective is to maximize cumulative discounted reward.
MADRL-ZX Algorithm
I propose a multi-agent deep reinforcement learning algorithm named MADRL-ZX, which employs a centralized training with decentralized execution (CTDE) architecture. Each agent has an Actor network and a Critic network, each with online and target networks. The Actor outputs actions based on local observations, while the centralized Critic evaluates actions using global information. To improve exploration, Gaussian noise is added to actions. Experience replay with prioritized sampling is used to accelerate convergence. The priority of experience k is:
$$ P_k = \frac{(|\delta_k| + \epsilon)^\beta}{\sum_{k’=1}^K (|\delta_{k’}| + \epsilon)^\beta} $$
where δₖ is the temporal-difference error. Importance sampling weights:
$$ \omega_k = \frac{1}{(K \cdot P_k)^\mu} $$
The Critic loss function:
$$ L(\theta^Q) = E[\omega_k (\delta)^2] $$
The Actor policy gradient:
$$ \Delta_{\theta_m} J = E\left[ \nabla_{\theta_m} \pi_m(o_m | \theta_m) \nabla_a Q(s,a | \theta^Q) \right] $$
Soft updates are applied to target networks:
$$ \theta_m’ = \tau \theta_m + (1-\tau) \theta_m’ $$
$$ \theta_Q’ = \tau \theta_Q + (1-\tau) \theta_Q’ $$
I also introduce state normalization to stabilize training. Each observation component is divided by its maximum value to bring all features into [0,1]. The normalized observation:
$$ \bar{o}(t) = \left\{ \frac{E_{battery}}{E_{max}}, \frac{X_m}{X_{max}}, \frac{Y_m}{Y_{max}}, \frac{X_1}{X_{max}}, \dots, \frac{C_1}{C_{max}}, \dots, \overline{f}_1, \dots, \overline{\text{load}} \right\} $$
The algorithm pseudocode is summarized below.
| Step | Description |
|---|---|
| 1 | Initialize replay buffer, Actor/Critic online and target networks |
| 2 | For each iteration (1 to K): |
| 3 | Initialize and normalize state |
| 4 | For each time slot t (1 to T): |
| 5 | For each drone (1 to M): |
| 6 | Select action a_t with noise; execute; observe reward |
| 7 | Store transition (s_t, a_t, r_t, s_{t+1}) in buffer |
| 8 | If t % learn_step == 0: |
| 9 | For each drone: |
| 10 | Sample mini-batch using priority P_k |
| 11 | Update Critic via minimizing L(θ^Q) |
| 12 | Update Actor via policy gradient |
| 13 | Soft update target networks |
Simulation and Results Analysis
Simulation Setup
Simulations are conducted using Python 3.8 and PyTorch. UEs are distributed in a 200 m × 200 m area. Three drones (M=3) collaborate to serve ground UEs. Each drone has a payload of 9.8 kg and flies at fixed height 20 m. Initial positions: (20,100), (50,100), (100,100). Three hotspot regions: (50,160), (160,160), (170,80). UEs move randomly at low speed. Total flight cycle h = 320 s, with 1 s movement and 7 s hover per cycle. System cost = -R. Results are averaged over multiple runs. Key parameters are listed in Table 1.
| Parameter | Value |
|---|---|
| Channel bandwidth W | 1-15 MHz |
| UE CPU frequency fₙˡ | 0.6-1.2 GHz |
| Drone MEC frequency fₙʳ | 10 GHz |
| Task data size Dₙ | 1-2 MB |
| Drone height H | 20 m |
| Required CPU cycles Cₙ | 0.8-2.8 GHz |
| UE transmit power Pₙ | 0.1-0.2 W |
| Noise power σ² | -114 dBm |
| Optimizer | Adam |
| Replay buffer size | 20,000 |
| Actor learning rate α₁ | 0.0001 |
| Critic learning rate α₂ | 0.0002 |
| Discount factor γ | 0.95 |
| Batch size | 64 |
| Drone speed | 0-30 m/s |
| Drone payload M_U | 9.8 kg |
Comparison Benchmarks
- MADRL-NOZX: MADRL-ZX without state normalization.
- DDPG: Single-agent Deep Deterministic Policy Gradient.
- RANDOM: Random flight direction, speed, and offloading ratio.
Results
Impact of number of UEs (Fig. 4 in original): As the number of UEs increases, total system cost rises for all algorithms. MADRL-ZX consistently achieves the lowest cost. At 90 UEs, my algorithm reduces cost by about 77% compared to RANDOM and 46% compared to DDPG. This demonstrates the scalability and efficiency of drone technology in dense user environments.
Impact of bandwidth (Fig. 5): Wider bandwidth reduces transmission energy, resulting in lower total energy consumption. However, since flight and computation energy dominate, the reduction is limited. MADRL-ZX outperforms others across all bandwidth levels.
Impact of task complexity (Fig. 6): As required CPU cycles increase from 1 GHz to 2.6 GHz, total cost rises. At maximum complexity, MADRL-ZX reduces cost by 34% over MADRL-NOZX and 42% over DDPG.
Fairness index (Fig. 7): MADRL-ZX maintains a Jain index above 0.95 across different UE numbers, indicating excellent load fairness. DDPG and RANDOM show lower fairness due to unbalanced task distribution.
Impact of weighting factor λ (Fig. 8): Increasing λ emphasizes energy minimization over delay, but MADRL-ZX still achieves the lowest delay among all algorithms, making it suitable for latency-sensitive emergency scenarios.
Conclusion
In this paper, I addressed the problem of multi-drone collaborative computing offloading in emergency scenarios. By proposing an air-ground integrated architecture and a novel MADRL algorithm, I effectively reduced system offloading delay and total energy consumption while ensuring load fairness. Extensive simulations validated the superiority of my approach over baseline methods. The results underscore the potential of drone technology in providing rapid and reliable edge computing services during infrastructure outages. Future work will extend the drone motion to three-dimensional space to further improve flexibility and coverage.