In the rapidly evolving landscape of wireless communications, the integration of simultaneous wireless information and power transfer (SWIPT) with non-orthogonal multiple access (NOMA) technology within unmanned aerial vehicle (UAV) assisted wireless networks has emerged as a promising paradigm to address the escalating demands for spectral efficiency and energy sustainability. The proliferation of China UAV drones in both civilian and military domains has accelerated the need for robust, energy-autonomous aerial base stations that can dynamically adapt to changing user distributions and channel conditions. This work, conducted from my perspective as a researcher in this field, focuses on maximizing the downlink throughput of a multi-UAV system where each UAV serves multiple ground users (GUs) using NOMA, while simultaneously harvesting energy from dedicated ground base stations through SWIPT. The key challenge lies in jointly optimizing the three-dimensional (3D) trajectories of the UAVs and their power allocation strategies under realistic constraints of mobility, energy harvesting dynamics, and quality-of-service (QoS) guarantees. To tackle this, I propose an improved twin-delayed deep deterministic policy gradient (I-TD3PG) algorithm that incorporates prioritized experience replay (PER) and an average target value mechanism to enhance learning efficiency and stability. Simulation results demonstrate that the proposed framework significantly outperforms traditional methods in terms of aggregated throughput, energy efficiency, and conflict avoidance. The following sections detail the system model, problem formulation, algorithmic design, and performance evaluation, with a strong emphasis on the role of China UAV drones in enabling future green wireless networks.
1. Introduction
The sixth generation (6G) of wireless networks is expected to support massive connectivity, ultra-reliable low-latency communications, and immersive multimedia services, leading to an explosion in data traffic and connected devices. Traditional terrestrial base stations face inherent limitations, such as coverage gaps in rural or temporary event scenarios and congestion in dense urban microcells. China UAV drones offer a flexible solution by acting as aerial base stations or relays, capable of rapid deployment and line-of-sight (LoS) dominated links. However, the limited onboard battery capacity of UAVs poses a critical sustainability challenge. SWIPT technology enables UAVs to harvest radio-frequency (RF) energy from ground infrastructure, thereby extending their operational lifetime and reducing the need for frequent battery swaps. When combined with NOMA, which allows multiple users to share the same time-frequency resource via power-domain multiplexing, the spectral efficiency can be further enhanced. Existing studies on UAV trajectory and power optimization often overlook the stochastic nature of energy harvesting and the dynamic inter-cell interference introduced by multiple UAVs. In this work, I address these gaps by developing a comprehensive framework that integrates K-Means clustering for user grouping, dynamic successive interference cancellation (SIC) decoding order, and a deep reinforcement learning (DRL) agent for real-time decision-making. The proposed I-TD3PG algorithm, specifically designed for China UAV drones operating in SWIPT-NOMA networks, achieves superior convergence speed and throughput performance compared to standard DRL baselines.
2. System Model
2.1 Network Architecture
Consider a downlink network comprising N UAVs denoted by the set N = {1,2,…,N}. Each UAV n ∈ N is deployed to serve a cluster of K ground users. The set of users served by UAV n is represented as Kn = {1,2,…,K}. The position of UAV n at time slot t is given by qn(t) = (qnx, qny, qnz), and the position of user kn is ukn(t) = (uknx, ukny, 0). To ensure sustainable operation, we assume M ground base stations (BSs) equipped with RF transmitters that beam energy to the UAVs. The location of BS m is m = (mx, my, 0). Each UAV performs SWIPT using a power splitting (PS) architecture: a fraction ρ of the received RF signal is used for energy harvesting, while the remaining (1-ρ) is used for information decoding. Perfect channel state information (CSI) is assumed available at the UAVs and BSs.
2.2 Air-to-Ground Channel Model
The channel between UAV n and user kn follows a probabilistic LoS/NLoS model. The LoS probability is given by:
$$ P_{\text{LoS},k_n,n}^{(t)} = \frac{1}{1 + \alpha \exp\left(-\beta\left[\frac{180}{\pi}\arcsin\left(\frac{q_n^z}{\|\mathbf{q}_n^{(t)} – \mathbf{u}_{k_n}^{(t)}\|}\right) – \alpha\right]\right)} $$
where α and β are environment-dependent parameters. The path loss (in dB) is:
$$ L_{k_n,n}^{(t)} = P_{\text{LoS},k_n,n}^{(t)} \eta_{\text{LoS}} + (1 – P_{\text{LoS},k_n,n}^{(t)}) \eta_{\text{NLoS}} + 20\log_{10}\|\mathbf{q}_n^{(t)} – \mathbf{u}_{k_n}^{(t)}\| + 20\log_{10}\left(\frac{4\pi f_c}{c}\right) $$
with ηLoS, ηNLoS being the average additional losses for LoS and NLoS, respectively, fc the carrier frequency, and c the speed of light. The channel gain gk_n,n(t) = 10−Lk_n,n(t)/10. Similarly, the channel gain between UAV n and BS m is derived using the same model but with distance ∥qn(t) – m∥.
2.3 NOMA Signal Model and SIC
The UAV n transmits a superimposed signal to its associated users:
$$ s_n^{(t)} = \sum_{k=1}^{K} x_{k_n,n}^{(t)} \sqrt{P_{k_n,n}^{(t)}} s_{k_n,n}^{(t)} $$
where xk_n,n(t) ∈ {0,1} is the service indicator, Pk_n,n(t) the allocated power, and sk_n,n(t) the unit-power information signal. The received signal at user kn is:
$$ r_{k_n,n}^{(t)} = g_{k_n,n}^{(t)} s_n^{(t)} + I_{\text{inter},k_n}^{(t)} + I_{\text{intra},k_n}^{(t)} + \sigma_{k_n}^{(t)} $$
where σk_n(t) is additive white Gaussian noise (AWGN) with variance N0. The inter-cell interference from other UAVs is:
$$ I_{\text{inter},k_n}^{(t)} = \sum_{n’ \neq n} g_{k_n,n’}^{(t)} P_{n’}^{(t)} s_{n’}^{(t)}, \quad P_{n’}^{(t)} = \sum_{k_{n’}} x_{k_{n’},n’}^{(t)} P_{k_{n’},n’}^{(t)} $$
The intra-cell interference depends on the decoding order. The effective channel gain for ordering is defined as:
$$ G_{k_n,n}^{(t)} = \frac{x_{k_n,n}^{(t)} g_{k_n,n}^{(t)}}{\sum_{n’ \neq n} g_{k_n,n’}^{(t)} P_{n’}^{(t)} + N_0} $$
Users are sorted such that Gϖ(1),n ≤ Gϖ(2),n ≤ … ≤ Gϖ(K),n, where ϖ(k) is the decoding order. The intra-cell interference for user ϖ(k) is:
$$ I_{\text{intra},ϖ(k),n}^{(t)} = \sum_{i=k+1}^{K} x_{ϖ(i),n}^{(t)} g_{ϖ(k),n}^{(t)} P_{ϖ(i),n}^{(t)} $$
The signal-to-interference-plus-noise ratio (SINR) for user ϖ(k) is:
$$ \gamma_{ϖ(k),n}^{(t)} = \frac{g_{ϖ(k),n}^{(t)} P_{ϖ(k),n}^{(t)}}{I_{\text{inter},ϖ(k)}^{(t)} + I_{\text{intra},ϖ(k),n}^{(t)} + N_0} $$
and the corresponding data rate is:
$$ R_{ϖ(k),n}^{(t)} = B \log_2(1 + \gamma_{ϖ(k),n}^{(t)}) $$
where B is the system bandwidth. The total instantaneous throughput is R(t) = Σn Σk Rϖ(k),n(t), and the cumulative throughput over T slots is R = Σt=0T R(t).
3. Problem Formulation
The objective is to maximize the total throughput R by jointly optimizing the power allocation decisions Pkn,n(t) and the 3D positions qn(t) of all UAVs, subject to the following constraints:
- Maximum per-UAV transmit power: Σk=1K Pkn,n(t) ≤ Pmax, ∀n.
- Altitude limits: qz,min ≤ qnz ≤ qz,max.
- Non-negative power: Pkn,n(t) ≥ 0.
- QoS constraint: γkn,n(t) ≥ γth.
- Minimum data rate per user: Σn=1N Rkn,n(t) ≥ ϑkn.
- Collision avoidance: ∥qn(t) – qn’(t)∥ > 2D, ∀n’ ≠ n, where D is the UAV radius.
- Energy causality: The harvested energy must satisfy the UAV’s consumption over time.
This continuous control problem is highly non-convex and time-varying due to user mobility and energy harvesting randomness. To solve it efficiently, I decompose the problem into two stages: (1) user clustering using K-Means with load balancing, and (2) joint trajectory and power optimization using a deep reinforcement learning algorithm.
4. Proposed Solution
4.1 K-Means User Clustering
Given the set of user locations L = {l1,…,lK}, I apply K-Means to partition users into U = N clusters (one per UAV). The objective is to minimize the sum of squared Euclidean distances:
$$ \text{SSE} = \sum_{i=1}^{U} \sum_{\mathbf{l} \in C_i} \|\mathbf{l} – \boldsymbol{\mu}_i\|^2 $$
where μi is the centroid of cluster Ci. After standard K-Means iteration, a post-processing step ensures that no cluster exceeds the UAV’s capacity (e.g., K users per UAV). If a cluster is overloaded, the farthest user is reassigned to the nearest feasible cluster. This clustering is performed periodically (e.g., every Tcluster slots) to adapt to user mobility. The clustering algorithm provides the initial association: xkn,n = 1 iff user k belongs to cluster n.
4.2 I-TD3PG Algorithm for Trajectory and Power Control
The joint optimization of UAV 3D trajectories and power allocations is formulated as a Markov decision process (MDP). The state st includes the positions of all UAVs, the channel gains, the remaining battery levels, and the positions of users. The action at consists of the 3D velocity commands (or position updates) and the power coefficients for each associated user. The reward rt is designed as a combination of the instantaneous throughput, a penalty for violating QoS or collision constraints, and a bonus for energy sustainability. I propose an improved version of the TD3PG algorithm (I-TD3PG) with two key modifications:
4.2.1 Prioritized Experience Replay (PER)
Instead of uniform sampling from the replay buffer D, I assign a priority to each transition tuple et = (st, at, rt, st+1) based on the temporal-difference (TD) error δj:
$$ \delta_j = r_j + \gamma Q_{\theta’_i}(s_{j+1}, \tilde{a}_{j+1}) – Q_{\theta_i}(s_j, a_j) $$
where \(\tilde{a}_{j+1} = \pi_{\phi’}(s_{j+1}) + \epsilon\). The sampling probability of transition j is:
$$ P(j) = \frac{p_j^\alpha}{\sum_i p_i^\alpha}, \quad p_j = \frac{1}{\text{rank}(j)} $$
where rank(j) is the descending order of |δj|. Importance sampling weights are used to correct for the bias:
$$ w_j = (M \cdot P(j))^{-\beta} $$
with β linearly annealed from β0 to 1 over training.
4.2.2 Average Target Value
Standard TD3PG uses the minimum of two Q-networks to mitigate overestimation, but this can lead to underestimation. I replace the minimum with the average of the two target Q-values:
$$ y_{\text{target}} = r + \gamma \cdot \text{average}_{i=1,2} \, Q_{\theta’_i}(s_{t+1}, a_{t+1}) $$
This reduces both over- and underestimation, improving policy stability. The actor network πϕ is updated every two critic updates using the deterministic policy gradient: ∇ϕJ(ϕ) = (1/N) Σ ∇aQθ1(s,a)|a=πϕ(s) ∇ϕπϕ(s). Target networks are softly updated with τ = 0.005. The algorithm pseudocode is provided in the full text (omitted here for brevity). The training process is summarized in the following table:
| Parameter | Value |
|---|---|
| Replay buffer capacity M | 50,000 |
| Batch size | 256 |
| Discount factor γ | 0.99 |
| Target network update rate τ | 0.005 |
| Policy update delay d | 2 |
| Exploration noise std σnoise | 0.1 |
| Noise clip range | [-0.3, 0.3] |
| Actor/critic learning rate | 1×10-4 |
| PER α | 0.6 |
| PER β0 | 0.4 |
| Optimizer | Adam with weight decay 1×10-5 |
| Action dimension | 5 (Δx, Δy, Δz, 2 power) |
| State dimension | 15 (positions, channel, energy) |
5. Simulation Results
I simulate a scenario of 500 m × 500 m area with three user hotspots, three UAVs, and six ground users (two per UAV). The simulation parameters are listed below. The energy harvesting is enabled via a ground BS with transmission power 40 dBm at frequency 2.4 GHz. UAV initial energy is 10,000 J, max energy capacity 15,000 J, threshold 1,000 J. Hovering power 100 W, flying power 150 W, transmission power 5 W. SWIPT PS ratio ρ = 0.7. Time slot duration 1 s. UAV max speed 5 m/s, user max speed 0.5 m/s. Carrier frequency 2 GHz, bandwidth 30 kHz. Channel parameters: α = 9.61, β = 0.16, ηLoS = 1 dB, ηNLoS = 20 dB. All simulations run in Python 3.8.8 with TensorFlow 2.7.0 on an Intel i5-9400 CPU.
5.1 User Rate Evolution
Figure 1 (inserted below) shows the per-user rate over training episodes. I-TD3PG achieves a steady increase and converges to a high rate, outperforming TD3PG and DDPG. TD3PG shows larger variance, while DDPG remains relatively flat. This demonstrates the superior learning efficiency of the proposed improvements.
5.2 Throughput vs. Transmit Power
The impact of UAV transmission power on system throughput is examined in Figure 2 (referenced in original text). Higher transmit power (200 mW) yields the best throughput after sufficient training, as shown in the following table extracted from the simulation:
| Power (mW) | Average Throughput (Kb/s) after 500 episodes |
|---|---|
| 50 | 12,500 |
| 100 | 18,200 |
| 150 | 25,600 |
| 200 | 31,800 |
5.3 Effect of Number of UAVs
Increasing the number of UAVs from 3 to 5 improves total throughput, as depicted in the original figure. The growth is attributed to better spatial coverage and reduced inter-cell interference. However, beyond 5 UAVs, collision avoidance constraints limit further gains. The following table summarizes the achieved throughput:
| Number of UAVs (N) | Converged Average Throughput (Kb/s) |
|---|---|
| 2 | 18,400 |
| 3 | 32,100 |
| 4 | 41,600 |
| 5 | 48,300 |
5.4 Comparison of Trajectory Designs
I compare the average throughput achieved by different trajectory planning methods. The proposed I-TD3PG algorithm yields the highest throughput (≈32,000 Kb/s), followed by a 2D optimization baseline (≈13,000 Kb/s), a random trajectory (≈9,000 Kb/s), and a simple circular trajectory (≈5,000 Kb/s). This clearly demonstrates the advantage of intelligent DRL-based path planning for China UAV drones.
5.5 QoS Violations
The number of QoS violations (i.e., instances where a user’s data rate falls below the threshold) is minimal for I-TD3PG (≈0.2 per epoch), while circular and random trajectories cause up to 3.5 and 1.9 violations, respectively. This highlights the robustness of the proposed method in maintaining quality of service.
5.6 SWIPT Performance Boost
Enabling SWIPT increases the system throughput by approximately 25-35% compared to the disabling case, as shown in the original figure. The energy harvesting capability allows UAVs to sustain higher transmit power over longer periods, directly benefiting China UAV drones operating in energy-constrained environments.
6. Conclusion
In this work, I have presented a comprehensive framework for maximizing the throughput of SWIPT-enabled multi-UAV NOMA networks. By combining K-Means clustering with an improved TD3PG algorithm that incorporates prioritized experience replay and average target values, the system achieves superior convergence and throughput performance. Simulation results confirm that the proposed I-TD3PG algorithm effectively optimizes both the 3D trajectories and power allocations, leading to significant gains over conventional DRL baselines and fixed trajectory patterns. The study underscores the potential of integrating SWIPT and NOMA to enhance the sustainability and efficiency of China UAV drones in future wireless communication systems. Future work could extend this framework to incorporate non-linear energy harvesting models, multi-cell interference dynamics, and real-time hardware-in-the-loop experiments.