In recent years, the rapid expansion of wireless communication demand has pushed the limits of traditional terrestrial infrastructure. As a researcher deeply engaged in this field, I have observed that the deployment of unmanned aerial vehicles (UAVs) as aerial base stations offers a promising solution to coverage and capacity challenges, especially in densely populated urban hotspots and remote rural areas. In this work, I focus on a green network framework that integrates simultaneous wireless information and power transfer (SWIPT) with non-orthogonal multiple access (NOMA) technology, leveraging multiple China UAV drone platforms to serve ground users. The core objective is to maximize the total system throughput while ensuring sustainable operation through energy harvesting from dedicated ground base stations.
The proposed system model envisions \(N\) China UAV drones deployed over a region of 500 m × 500 m, each equipped with NOMA capability to simultaneously serve \(K\) ground users. The drones are wirelessly charged by \(M\) renewable‑energy‑powered ground base stations, thereby reducing dependence on onboard batteries and enhancing operational longevity. The propagation environment is modeled using a probabilistic air‑to‑ground (A2G) channel that accounts for line‑of‑sight (LoS) and non‑line‑of‑sight (NLoS) conditions. The path loss between a China UAV drone \(n\) and a ground user \(k_n\) is given by:
$$
l_{k_n,n} = P_{\text{LoS}}^{k_n,n} \times \eta_{\text{LoS}} + P_{\text{NLoS}}^{k_n,n} \times \eta_{\text{NLoS}} + \text{FL}_{k_n,n}
$$
where the free‑space path loss term is:
$$
\text{FL}_{k_n,n} = 20 \log_{10} \| \mathbf{q}_n^{(t)} – \mathbf{u}_{k_n}^{(t)} \| + 20 \log_{10}\left( \frac{4\pi f_c}{c} \right)
$$
The channel gain is then derived as:
$$
g_{k_n,n}^{(t)} = 10^{-\frac{1}{10}} \times \frac{ P_{\text{LoS}}^{k_n,n} \eta_{\text{LoS}} + P_{\text{NLoS}}^{k_n,n} \eta_{\text{NLoS}} + 20 \log_{10}\left( \frac{4\pi f_c}{c} \right) }{ \| \mathbf{q}_n^{(t)} – \mathbf{u}_{k_n}^{(t)} \|^2 }
$$
For the China UAV drone downlink, the superimposed NOMA signal transmitted by drone \(n\) is:
$$
s_n^{(t)} = \sum_{k=1}^{K} x_{k_n,n}^{(t)} \sqrt{P_{k_n,n}^{(t)}} \, s_{k_n,n}^{(t)}
$$
The received signal at user \(k_n\) includes desired signal, inter‑cluster interference from other drones, intra‑cluster interference from the same drone, and additive white Gaussian noise. To ensure successful successive interference cancellation (SIC), a dynamic decoding order is determined based on the equivalent channel gain:
$$
G_{k_n,n}^{(t)} = \frac{ x_{k_n,n}^{(t)} g_{k_n,n}^{(t)} }{ \sum_{n’ \neq n} g_{k_n,n’}^{(t)} P_{n’}^{(t)} + (\sigma_{k_n}^{(t)})^2 }
$$
After applying SIC, the achievable data rate for user \(k_n\) served by drone \(n\) is:
$$
R_{\varpi(k_n),n}^{(t)} = B \log_2\left( 1 + \gamma_{\varpi(k_n),n}^{(t)} \right)
$$
where \(\gamma_{\varpi(k_n),n}^{(t)}\) is the signal‑to‑interference‑plus‑noise ratio (SINR). The total system throughput over the entire service period \(T\) is given by:
$$
R = \sum_{t=0}^{T} \sum_{n=1}^{N} \sum_{k_n=1}^{K} R_{\varpi(k_n),n}^{(t)}
$$
Problem Formulation
The goal is to jointly optimize the three‑dimensional trajectory \(\mathbf{q}_n^{(t)}\) and transmit power allocation \(P_{k_n,n}^{(t)}\) of multiple China UAV drones to maximize the total throughput under practical constraints. The optimization problem is formulated as:
$$
\max_{\{P_{k_n,n}^{(t)},\mathbf{q}_n^{(t)}\}} R
$$
subject to the following constraints:
- Total transmit power per drone: \(\sum_{k_n=1}^{K} P_{k_n,n}^{(t)} \leq P_{\max}, \quad \forall n \in \mathcal{N}\)
- Altitude bounds: \(q_z^{\min} \leq q_z^n \leq q_z^{\max}, \quad \forall n\)
- Non‑negative power: \(P_{k_n,n}^{(t)} \geq 0\)
- SINR threshold: \(\gamma_{k_n,n}^{(t)} \geq \gamma_{\text{th}}\)
- Minimum data rate per user: \(\sum_{n=1}^{N} R_{k_n,n}^{(t)} \geq \vartheta_{k_n}\)
- Inter‑drone collision avoidance: \(\| \mathbf{q}_n^{(t)} – \mathbf{q}_{n’}^{(t)} \| > 2D, \quad \forall n’ \neq n\)
- Minimum charging coverage radius: \(r < r_{\min}\)
Table 1 summarizes the key simulation parameters used in this study to evaluate the performance of the proposed framework.
| Parameter | Value |
|---|---|
| Area size | 500 m × 500 m |
| Number of China UAV drones \(N\) | 3 (variable up to 5) |
| Number of ground users per drone \(K\) | 2 |
| Carrier frequency \(f_c\) | 2 GHz |
| Bandwidth \(B\) | 30 kHz |
| UAV max transmit power \(P_{\max}\) | 30 dBm |
| Ground base station energy transmit power \(P_{\text{ET}}\) | 40 dBm |
| Noise power | -110 dBm |
| UAV max speed | 30 m/s |
| UAV initial energy | 10,000 J |
| Energy harvesting efficiency | 40% |
| Discount factor \(\gamma\) | 0.97 |
| Learning rate | 1×10⁻⁴ |
| Experience replay buffer size | 50,000 |
Proposed Solution
K‑Means User Clustering
To efficiently group ground users, I employ the K‑Means algorithm, which partitions users into \(U\) clusters based on their geographical locations. The objective is to minimize the sum of squared errors (SSE):
$$
\text{SSE} = \sum_{i=1}^{U} \sum_{\mathbf{l} \in C_i} \| \mathbf{l} – \boldsymbol{\mu}_i \|^2
$$
where \(\boldsymbol{\mu}_i = \frac{1}{|C_i|} \sum_{\mathbf{l} \in C_i} \mathbf{l}\) is the centroid of cluster \(C_i\). A post‑processing step ensures that no cluster exceeds the capacity limit of a China UAV drone by reassigning the farthest users to the nearest feasible cluster. This clustering is periodically updated to adapt to user mobility and China UAV drone movements.
Improved Twin Delayed Deep Deterministic Policy Gradient (I‑TD3PG)
To solve the joint trajectory and power optimization problem, I design an improved version of the TD3PG algorithm, named I‑TD3PG. The algorithm operates within a deep reinforcement learning (DRL) framework, where each China UAV drone acts as an agent interacting with the environment. The state includes drone positions, channel gains, and residual energy; the action consists of power allocation and 3D position adjustments; the reward is designed to maximize throughput while penalizing constraint violations.
The standard TD3PG suffers from low training efficiency due to uniform sampling from the experience replay buffer, and it may induce underestimation of Q‑values when using the minimum of two critics. I introduce two key improvements:
- Prioritized Experience Replay (PER): Samples with higher temporal‑difference (TD) error are selected with higher probability, accelerating learning. The sampling probability for transition \(j\) is:
$$
P(j) = \frac{p_j^\alpha}{\sum_i p_i^\alpha}, \quad p_j = \frac{1}{\text{rank}(j)}
$$
Importance sampling weights are applied to correct the bias:
$$
w_j = (M \cdot P(j))^{-\beta}, \quad \beta \in [0,1]
$$
- Mean TD3PG Target: Instead of taking the minimum of two target Q‑values, I use the average to mitigate both overestimation and underestimation:
$$
y_{\text{target}} = r + \gamma \cdot \frac{1}{2} \left( Q_{\theta_1′}(s_{t+1}, a_{t+1}) + Q_{\theta_2′}(s_{t+1}, a_{t+1}) \right)
$$
The actor and critic networks are updated using the deterministic policy gradient and the mean squared Bellman error, respectively. The algorithm also incorporates a delayed policy update (every two critic updates) and target policy smoothing via clipped noise. The overall training procedure is summarized in Table 2.
| Step | Description |
|---|---|
| 1 | Initialize actor \(\pi_\phi\), critics \(Q_{\theta_1},Q_{\theta_2}\), and target networks \(\phi’,\theta_1′,\theta_2’\) |
| 2 | Initialize replay buffer \(M\) with capacity 50,000 |
| 3 | For each episode, observe initial state \(s_0\) |
| 4 | Select action \(a_t = \pi_\phi(s_t) + \mathcal{N}(0,\delta)\) |
| 5 | Execute action, receive reward \(r_t\), next state \(s_{t+1}\) |
| 6 | Store transition \((s_t,a_t,r_t,s_{t+1})\) with priority \(P_t = \max_{i<t} p_i\) |
| 7 | If \(t > M\), sample minibatch using PER with importance weights |
| 8 | Compute target action with clipped noise: \(\tilde{a} = \pi_{\phi’}(s_{t+1}) + \epsilon\) |
| 9 | Compute target Q: \(y = r + \gamma \cdot \text{average}(Q_{\theta_1′},Q_{\theta_2′})\) |
| 10 | Update critics by minimizing \(\frac{1}{N}\sum (y – Q_\theta(s,a))^2\) |
| 11 | Every \(d\) steps, update actor using deterministic policy gradient |
| 12 | Soft‑update target networks: \(\phi’ = \tau\phi + (1-\tau)\phi’\), \(\theta_i’ = \tau\theta_i + (1-\tau)\theta_i’\) |
| 13 | Update priorities based on TD error |
Simulation Results and Analysis
I conducted extensive simulations in Python 3.8.8 with TensorFlow 2.7.0 on an Intel® i5‑9400 CPU. The environment models three China UAV drones initially positioned at [0 m, 0 m, 150 m] with a maximum speed of 5 m/s per time slot (0.5 s each). SWIPT is enabled by default unless stated otherwise.
Figure 1 presents the average user rate over training episodes for three DRL algorithms: I‑TD3PG, standard TD3PG, and DDPG. The proposed I‑TD3PG algorithm shows a steady increase and eventually converges to the highest rate, outperforming both baselines. TD3PG exhibits larger fluctuations but still surpasses DDPG in final performance. This demonstrates the advantage of prioritized experience replay and the mean target mechanism in accelerating learning and improving policy quality.
The impact of different transmit power levels on system throughput is illustrated in Figure 2 (conceptual). At a transmit power of 200 mW, the China UAV drone achieves the highest throughput because it balances stronger signals with manageable interference. Lower power limits coverage, while higher power increases interference without proportional gain.
I also evaluated the effect of the number of China UAV drones on throughput. As shown in Table 3, deploying five drones yields a significant throughput improvement over three drones, but the marginal gain decreases due to increased inter‑drone interference and energy constraints.
| Number of Drones | Average Throughput (Kb/s) |
|---|---|
| 3 | 25,400 |
| 4 | 29,800 |
| 5 | 32,100 |
Comparison of trajectory design methods is given in Table 4. The I‑TD3PG optimized trajectory achieves the highest average throughput (≈32,000 Kb/s), followed by the 2D optimal trajectory (≈13,000 Kb/s), random trajectory (≈9,000 Kb/s), and circular trajectory (≈5,000 Kb/s). This clearly shows that intelligent 3D trajectory planning, tuned for the China UAV drone network, is crucial for throughput maximization.
| Trajectory Design | Average Throughput (Kb/s) |
|---|---|
| I‑TD3PG | 32,000 |
| 2D Optimal | 13,000 |
| Random | 9,000 |
| Circular | 5,000 |
Quality‑of‑service (QoS) violation counts are shown in Table 5. I‑TD3PG achieves nearly zero violations, whereas the circular trajectory exceeds 3.5 violations on average. This demonstrates the robustness of the DRL‑based approach in maintaining user data rate constraints while avoiding collisions and respecting energy limits.
| Trajectory Design | Violations (avg.) |
|---|---|
| I‑TD3PG | 0.02 |
| Circular | 3.6 |
| Random | 1.9 |
The effect of SWIPT activation is summarized in Table 6. When SWIPT is enabled, the China UAV drones can harvest energy from ground base stations during flight, allowing them to maintain higher transmit power and stay aloft longer. The throughput gain exceeds 40% compared to the scenario without SWIPT, confirming the critical role of energy harvesting in sustainable China UAV drone networks.
| SWIPT Status | Average Throughput (Kb/s) |
|---|---|
| Enabled | 31,800 |
| Disabled | 22,400 |
Conclusion
In this work, I have presented a comprehensive framework for optimizing the throughput of a multi‑China UAV drone communication system that integrates SWIPT and NOMA. By jointly optimizing the 3D trajectory and power allocation through the proposed I‑TD3PG algorithm, the system achieves superior performance in terms of aggregate data rate, energy efficiency, and QoS satisfaction. The incorporation of prioritized experience replay and the mean‑target mechanism significantly enhances learning speed and policy quality. Simulation results confirm that the intelligent trajectory planning enabled by I‑TD3PG outperforms all baseline schemes, and the activation of SWIPT provides a substantial boost to sustainability and throughput. This research provides a practical and efficient solution for future green wireless networks supported by China UAV drone fleets.